ADR 0100 after re-review: the bus and registry stay reachable for enrolment; the mesh guards the store in a table that only refuses; a machine in use defined; the flip refuses while a found container is held; held containers and returning to adopted spelled out

This commit is contained in:
2026-09-22 16:32:37 +02:00
parent 02c40bcab4
commit f3152d827f
7 changed files with 112 additions and 74 deletions
@@ -116,12 +116,15 @@ intention, and each thing the mesh would otherwise take must say what it does in
recorded, until the operator **takes** the module on that node — the cutover, done when the recorded, until the operator **takes** the module on that node — the cutover, done when the
module's data has moved. Assigning prepares; taking migrates. Without the distinction the rule module's data has moved. Assigning prepares; taking migrates. Without the distinction the rule
never fires: the host only ever sees what assigned modules declare. never fires: the host only ever sees what assigned modules declare.
- **The firewall found on the machine stays in force.** The mesh does not load its own table on an - **The firewall found on the machine stays in force.** The mesh loads no table on an adopted node
adopted node. What it needs open it declares as openings the host converges *through the found that drops by default or accepts. What it needs open it declares as openings the host converges
firewall*, on the incoming and the forwarded path, marked as the mesh's and re-checked on every *through the found firewall*, on the incoming and the forwarded path, marked as the mesh's and
reconcile so a reload or reboot does not lose them. An accept in a table of its own would not re-checked on every reconcile so a reload or reboot does not lose them. An accept in a table of
help: the found firewall's drop would still be final. And the mesh protects its own ports itself, its own would not help: the found firewall's drop would still be final. What a table of its own
on the forwarded path, since the found firewall may not. *can* do is refuse, and a refusal is final too — so the mesh guards the store and the broker's
management port from outside the private network in a table that only refuses, which the found
firewall may not do and cannot undo. The bus, the registry and the hub's port stay open to
anywhere: a node enrols before it has a private-network address.
- **The foundation's ports are the node's to give** — set at genesis, checked free, and kept as - **The foundation's ports are the node's to give** — set at genesis, checked free, and kept as
that node's settings, read everywhere they are used, so adopting the foundation as modules does that node's settings, read everywhere they are used, so adopting the foundation as modules does
not move them back. not move them back.
@@ -76,9 +76,10 @@ authoritative, and every declaration it sends says whether the node is adopted a
have been taken on it. A node stays adopted until the operator converges it. An adopted node is have been taken on it. A node stays adopted until the operator converges it. An adopted node is
said to be adopted wherever the mesh reports a node's state. said to be adopted wherever the mesh reports a node's state.
**A converged genesis refuses a machine in use.** Raised without saying adopted on a machine with **A converged genesis refuses a machine in use.** A machine is in use when a container is running
an active firewall or services listening that are not the mesh's, genesis refuses and names what on it or a port is listening that is neither ssh nor one of the operating system's own services.
it found — a forgotten flag must not close a working machine. Raised without saying adopted on such a machine, genesis refuses and names what it found — a
forgotten flag must not close a working machine.
**Before a node is adopted, its predecessor's control is stopped by the operator** — the daemons **Before a node is adopted, its predecessor's control is stopped by the operator** — the daemons
that write its configuration. Its services keep running on what they have. that write its configuration. Its services keep running on what they have.
@@ -92,23 +93,34 @@ file and a found container as they are, records the file's original content befo
happens to it, and reports each as held. Assigning a module on an adopted node prepares it: what happens to it, and reports each as held. Assigning a module on an adopted node prepares it: what
the module declares that is not found is created; what is found is held. **Taking a module** on a the module declares that is not found is created; what is found is held. **Taking a module** on a
node is its cutover — the operator's act, done when that module's data has moved — and from then node is its cutover — the operator's act, done when that module's data has moved — and from then
on the module's resources converge on that node like any other. A held file that changes while on the module's resources converge on that node like any other. What is held is never removed,
held is reported as changed by something else, not reverted: that is how a predecessor still even when its module is unassigned, and a held file or container that changes while held — a file
writing is caught. A held file whose module is unassigned is left where it is, never removed. rewritten, a container stopped or replaced — is reported as changed by something else, not reverted
or restarted: that is how a predecessor still writing is caught.
**The firewall found on the machine stays in force.** The mesh loads no table of its own on an **The firewall found on the machine stays in force.** The mesh loads no table on an adopted node
adopted node — neither genesis's base ruleset nor the filter module's derived one. What the mesh that drops by default or that accepts — neither genesis's base ruleset nor the filter module's
needs is declared as **openings**: a resource that says a port is reachable, from where, on the derived one. What the mesh needs reachable is declared as **openings**: a resource that says a port
incoming path or the forwarded path — a published container port is forwarded. The host converges is reachable, from where, on the incoming path or the forwarded path — a published container port is
an opening through the found firewall in that firewall's own terms, marks it as the mesh's, and forwarded. The controller derives them from the same inputs as the filter: the `listens` of the
removes only what it marked; it re-checks each opening on every reconcile, so a reload or a reboot modules assigned there, the private network's hub port, and the foundation's ports. The host
of the found firewall does not lose it. An opening is a state, not a command, which is what lets it converges an opening through the found firewall in that firewall's own terms, marks it as the
travel over the link. The host reports which firewall it found, and a machine with a kind no host mesh's, and removes only what it marked; it re-checks each opening on every reconcile, so a reload
speaks is refused adoption rather than adopted with nothing protecting the mesh's ports. or a reboot of the found firewall does not lose it for longer than one reconcile. An opening is a
state, not a command, which is what lets it travel over the link. The host reports which firewall
it found. A machine with no firewall needs no openings; a machine with a kind no host speaks is
refused adoption.
**The mesh protects its own ports itself.** On an adopted node its foundation's ports get openings **The mesh guards its own ports itself, in a table of its own that only refuses.** It accepts by
from the private network and marked refusals from anywhere else, on the forwarded path — so the default and holds nothing but refusals, so it cannot close anything the machine serves — a drop in
store is unreachable from outside whether or not the found firewall filters forwarded traffic. any chain is final and an accept in it would change nothing — and it is the mesh's, so the found
firewall reloading does not touch it. It refuses the store's port and the broker's management port
from anywhere but the private network, matched on the port the packet was sent to, before the
container runtime redirects it. The bus and the registry stay reachable from anywhere, as a node
enrols over the bus and pulls from the registry before it has a private-network address
([ADR 0088](0088-the-foundation-filters-before-anything-listens.md)); so does the private network's
hub port. The store is unreachable from outside whatever the found firewall does, and on a machine
with none.
**The foundation's ports are the node's.** Every port the foundation binds is an input to genesis, **The foundation's ports are the node's.** Every port the foundation binds is an input to genesis,
checked free before anything is raised, refused with the name of what holds it. The ports given checked free before anything is raised, refused with the name of what holds it. The ports given
@@ -118,13 +130,16 @@ filter, the base ruleset, the private network's endpoint, and the addresses cons
The private network's address range must not overlap a tunnel the predecessor still runs; genesis The private network's address range must not overlap a tunnel the predecessor still runs; genesis
checks that too. checks that too.
**Converging a node is one act, previewed.** The preview lists what is reachable on the machine now **Converging a node is one act, previewed.** It refuses while an assigned module still holds a
— every listening socket and every published container port — and for each whether an assigned found container: each service is taken on its own, when its data has moved, never by the flip. The
module declares it or it will close. The flip then takes every module not yet taken, loads the preview lists what is reachable on the machine now — every listening socket and every published
mesh's derived filter, and retires the found firewall by disabling it, never by flushing: the container port — and for each whether an assigned module declares it or it will close, and every
container runtime's rules and the found firewall's own configuration stay on disk. Returning a module the flip will take, with what each will replace. The flip then takes those modules, loads the
converged node to adopted re-enables the firewall it retired. A node converges when its migration mesh's derived filter in place of its refusal-only table, and retires the found firewall by
is done; the mesh is migrated when every node has converged. disabling it, never by flushing: the container runtime's rules and the found firewall's own
configuration stay on disk. Returning a converged node to adopted unloads the derived filter,
restores the refusal-only table and enables the found firewall again; what was taken stays taken. A
node converges when its migration is done; the mesh is migrated when every node has converged.
**The order is the operator's:** the control-node first, adopted, its modules assigned and taken **The order is the operator's:** the control-node first, adopted, its modules assigned and taken
one at a time; then each other machine, adopted, migrated, converged in turn. The predecessor's one at a time; then each other machine, adopted, migrated, converged in turn. The predecessor's
@@ -144,7 +159,8 @@ open).
What got harder: What got harder:
- **The mesh must speak a firewall it did not install**, on both the incoming and the forwarded - **The mesh must speak a firewall it did not install**, on both the incoming and the forwarded
path. One kind is found on the machines measured; another is refused until a host speaks it. path. One kind is found on the machines measured; another is refused until a host speaks it. The
mesh's own guard does not depend on it: that table is the mesh's.
- **The host gains a guard it did not have** — keep what you found — and its report must say - **The host gains a guard it did not have** — keep what you found — and its report must say
which files and containers it holds, or an adopted node reads as converged. which files and containers it holds, or an adopted node reads as converged.
- **The declaration gains a node's mode and its taken modules**, and a resource, the opening. - **The declaration gains a node's mode and its taken modules**, and a resource, the opening.
@@ -154,7 +170,7 @@ What got harder:
nodes are adopted, so one left behind is visible. nodes are adopted, so one left behind is visible.
- **This narrows [ADR 0088](0088-the-foundation-filters-before-anything-listens.md) for adopted - **This narrows [ADR 0088](0088-the-foundation-filters-before-anything-listens.md) for adopted
nodes**: the base ruleset is not loaded on a node raised adopted, and its duty — the store never nodes**: the base ruleset is not loaded on a node raised adopted, and its duty — the store never
reachable from outside — passes to the mesh's own openings and refusals on the forwarded path. reachable from outside — passes to a table of the mesh's that only refuses.
## How it is checked ## How it is checked
@@ -163,29 +179,33 @@ port and denying the rest, a service container listening on that port under a na
module also uses, a file at a path that module declares, a stand-in for the predecessor's control module also uses, a file at a path that module declares, a stand-in for the predecessor's control
that would rewrite that file, and a container holding the registry's port. Then: that would rewrite that file, and a container holding the registry's port. Then:
- **Genesis converged** on it refuses and names the firewall and the listener. - **Genesis converged** on it refuses and names the running container and the listener.
- **Genesis adopted, with the registry's port held**, refuses and names the holder; with another - **Genesis adopted, with the registry's port held**, refuses and names the holder; with another
port given, the foundation comes up — and adopting the foundation as modules leaves it on that port given, the foundation comes up — and adopting the foundation as modules leaves it on that
port. port.
- **Nothing that serves changed**: the service is reachable from a second machine, the file is byte - **Nothing that serves changed**: the service is reachable from a second machine, the file is byte
for byte what it was, and the found firewall's rules differ only by rules marked as the mesh's. for byte what it was, and the found firewall's rules differ only by rules marked as the mesh's.
- **The store is unreachable from outside** — probed from a machine off the private network — and - **The store is unreachable from outside** — probed from a machine off the private network, and
reachable over it. again after the found firewall is reloaded — and reachable over it; the bus is reachable from a
machine that has not yet enrolled.
- **The mesh works through the found firewall, and keeps working after it is reloaded and after the - **The mesh works through the found firewall, and keeps working after it is reloaded and after the
machine reboots**: the second machine enrols, and the openings are there again. machine reboots**: the second machine enrols, and the openings are there again.
- **A predecessor still writing is caught**: with the stand-in left running, the held file's change - **A predecessor still writing is caught**: with the stand-in left running, the held file's change
is reported and not reverted. is reported and not reverted.
- **Assigning prepares, taking cuts over**: the module assigned holds the found container and file; - **Assigning prepares, taking cuts over**: the module assigned holds the found container and file;
taken, it replaces them and its port is opened. taken, it replaces them and its port is opened.
- **Converging previews, then changes**: the preview names the service's port and a published port - **Converging previews, then changes**: it refuses while the service's module holds its found
no firewall rule mentions; after the flip the mesh's derived filter is loaded, the found firewall container; once that module is taken, the preview names the service's port and a published port no
is disabled with its configuration still on disk, the declared port is open and the undeclared one firewall rule mentions, and the modules it will take; after the flip the mesh's derived filter is
closed. Returned to adopted, the found firewall is enabled again. loaded, the found firewall is disabled with its configuration still on disk, the declared port is
open and the undeclared one closed. Returned to adopted, the found firewall is enabled again and
the derived filter is gone.
Unit tests hold the host to keeping a found file and container on an adopted node, converging them Unit tests hold the host to keeping a found file and container on an adopted node, converging them
once taken, never removing a held file, and reporting a held file that changed; genesis to refusing once taken, never removing what it holds, and reporting a held file or container that changed;
a held port and a converged raise on a machine in use; the controller to carrying the mode and the genesis to refusing a held port and a converged raise on a machine in use; the controller to
taken modules in every declaration. carrying the mode and the taken modules in every declaration, deriving the openings, and refusing
a flip while a found container is held.
## References ## References
+8 -2
View File
@@ -139,8 +139,9 @@ name, that the host's store has no record of writing. On an adopted node the hos
found for any module not yet taken: it records a found file's original content before anything found for any module not yet taken: it records a found file's original content before anything
else, and it reports the file or container as held — a report that says what it holds, so an else, and it reports the file or container as held — a report that says what it holds, so an
adopted node never reads as converged. Once the module is taken, its resources converge like any adopted node never reads as converged. Once the module is taken, its resources converge like any
other. A held file that changes while held is reported as changed by something else, not other. What is held is never removed, even when its module is unassigned, and a held file or
reverted; a held file is never removed, even when its module is unassigned. The host also container that changes while held is reported as changed by something else, not reverted or
restarted. The host also
converges a new resource, the **opening** — a port made reachable through the firewall it found converges a new resource, the **opening** — a port made reachable through the firewall it found
([08-connectivity](08-connectivity.md)) — and reports which firewall it found. This is the ([08-connectivity](08-connectivity.md)) — and reports which firewall it found. This is the
companion the host's ownership rule needed: *never touch what you did not create, unless adoption companion the host's ownership rule needed: *never touch what you did not create, unless adoption
@@ -212,6 +213,11 @@ ready; the current bundle simply does not. **All of them are built:**
| `container` | **built** | pinned by digest ([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)); identified by a label carrying a digest of the declaration that made it, because a runtime normalises what it is given and that is indistinguishable from drift | | `container` | **built** | pinned by digest ([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)); identified by a label carrying a digest of the declaration that made it, because a runtime normalises what it is given and that is indistinguishable from drift |
| `action` | **built** | bundle-only ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)); verify is mandatory and is the idempotency check as well as the read-back | | `action` | **built** | bundle-only ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)); verify is mandatory and is the idempotency check as well as the read-back |
One more shape is decided and not yet built: **`opening`**, on adopted nodes only
([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)) — a port
made reachable, from where, on the incoming or forwarded path, through the firewall found on the
machine, marked as the mesh's and re-checked on every reconcile.
**A service says what it must reflect, and that is declared state rather than a command.** **A service says what it must reflect, and that is declared state rather than a command.**
`restart-on` names files whose change means the unit must be restarted — because a running service `restart-on` names files whose change means the unit must be restarted — because a running service
does not re-read its configuration, and replacing a file, finding the service already running and does not re-read its configuration, and replacing a file, finding the service already running and
+4 -2
View File
@@ -178,8 +178,10 @@ port never answers, the bus's does.
machine already serving under a predecessor has a firewall of its own, and a second, stricter machine already serving under a predecessor has a firewall of its own, and a second, stricter
table would close everything it serves. There the base ruleset is not loaded and the filter module table would close everything it serves. There the base ruleset is not loaded and the filter module
is not assigned until the node converges; the foundation's ports are opened through the found is not assigned until the node converges; the foundation's ports are opened through the found
firewall from the private network and refused from anywhere else on the forwarded path, which firewall, and a table of the mesh's that only refuses keeps the store's port and the broker's
keeps the same promise — the store's port never answers from outside — by other means. The management port from anyone off the private network — the same promise, the store's port never
answering from outside, kept by other means. The bus and the registry stay reachable from anywhere,
as they are here, because a node enrols and pulls before it has a private-network address. The
foundation's ports themselves are the node's, given at genesis and kept as its settings. **Checked** foundation's ports themselves are the node's, given at genesis and kept as its settings. **Checked**
by the adoption bed, which makes the same outside probe. by the adoption bed, which makes the same outside probe.
+25 -17
View File
@@ -537,24 +537,32 @@ is why the check reads packets.*
### On an adopted node ### On an adopted node
*2026-09-22, [ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md).* **The firewall found on the machine stays in force**, and the mesh *2026-09-22, [ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md).* **The firewall found on the machine stays in force**, and the mesh
loads no table of its own there — neither genesis's base ruleset nor the derived one. Every base loads no table there that drops by default or that accepts — neither genesis's base ruleset nor
chain at a hook runs and a drop in any is final, whatever the other firewall is written in, so a the derived one. Every base chain at a hook runs and a drop in any is final, whatever the other
second, stricter table would close every port the machine serves. What the mesh needs reachable firewall is written in, so a second, stricter table would close every port the machine serves, and
it declares as **openings**: a port, from where, on the incoming path or the forwarded path — a an accept in one would open nothing the found firewall drops. What the mesh needs reachable it
declares as **openings**: a port, from where, on the incoming path or the forwarded path — a
published container port is forwarded, and a firewall that filters only incoming traffic never published container port is forwarded, and a firewall that filters only incoming traffic never
sees it. The host converges each opening through the found firewall in that firewall's own terms, sees it. The controller derives them from what the filter would be derived from: the assigned
marks it as the mesh's, removes only what it marked, and re-checks every opening on each reconcile modules' `listens`, the hub's port, the foundation's ports. The host converges each opening through
so a reload or a reboot does not lose it. An opening is state, not a command, so it travels over the found firewall in that firewall's own terms, marks it as the mesh's, removes only what it
the link like any other resource. **The mesh protects its own ports itself**: its foundation's ports marked, and re-checks every opening on each reconcile so a reload or a reboot does not lose it for
are opened from the private network and refused from anywhere else on the forwarded path, so the longer than one reconcile. An opening is state, not a command, so it travels over the link like any
store stays unreachable from outside whether or not the found firewall filters forwarded traffic. other resource. **The mesh guards its own ports in a table of its own that only refuses** —
One kind of found firewall is spoken; a machine with another is refused adoption rather than accepting by default and holding nothing but refusals, so it cannot close what the machine serves,
adopted unprotected. Converging the node previews what is reachable now — listening sockets and and the found firewall's reload does not touch it. It refuses the store's port and the broker's
published ports — and what will close, then loads the derived filter and disables the found management port from outside the private network, matched on the port the packet was sent to; the
firewall without flushing it. *How it is checked:* the adoption bed asserts the found firewall's bus, the registry and the hub's port stay reachable from anywhere, as a node enrols and pulls
rules differ only by the mesh's marked rules, that the store is unreachable from off the private before it has a private-network address. One kind of found firewall is spoken; a machine with none
network, that a second machine enrols through the openings before and after a reload and a reboot, needs no openings, and a machine with another kind is refused adoption. Converging the node refuses
and that after the flip the declared port is open and the undeclared one closed. while a found container is still held; otherwise it previews what is reachable now — listening
sockets and published ports — what will close and which modules it will take, then loads the
derived filter in place of the refusal-only table and disables the found firewall without flushing
it. *How it is checked:* the adoption bed asserts the found firewall's rules differ only by the
mesh's marked rules, that the store is unreachable from off the private network before and after
the found firewall reloads, that a machine not yet enrolled reaches the bus, that a second machine
enrols through the openings before and after a reload and a reboot, and that after the flip the
declared port is open and the undeclared one closed.
## 5 — Certificates ## 5 — Certificates
+4 -5
View File
@@ -391,12 +391,11 @@ should be and sends it; the host applies the difference and removes what is no l
The host removes what it *made* and leaves what it merely *configured*. Uninstalling a container The host removes what it *made* and leaves what it merely *configured*. Uninstalling a container
runtime because a declaration changed would stop every container on the node. runtime because a declaration changed would stop every container on the node.
---
*On an adopted node* ([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)), *On an adopted node* ([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)),
what the host is holding was found, not made, so nothing held is ever removed: a held file whose what the host is holding was found, not made, so nothing held is ever removed: a held file or
module is unassigned stays where it is. container whose module is unassigned stays where it is.
---
## enrolled ⇄ disconnected ## enrolled ⇄ disconnected
+5 -5
View File
@@ -110,11 +110,11 @@ ruleset, the private network's endpoint, the addresses consumers are given — r
the control-node measured, the store's and the bus's usual ports were free and the registry's, the the control-node measured, the store's and the bus's usual ports were free and the registry's, the
broker's management port and, as the private network's hub, its port were held. Genesis also broker's management port and, as the private network's hub, its port were held. Genesis also
checks the private network's range does not overlap a tunnel the predecessor runs. Genesis adopted checks the private network's range does not overlap a tunnel the predecessor runs. Genesis adopted
**does not load the base ruleset**: the machine's own firewall already filters, and the mesh opens **does not load the base ruleset**: the machine's own firewall already filters; the mesh opens its
its foundation's ports through it and refuses them from outside itself foundation's ports through it and keeps the store from outside with a table of its own that only
([08-connectivity](08-connectivity.md)). **A converged genesis refuses a machine in use** — an refuses ([08-connectivity](08-connectivity.md)). **A converged genesis refuses a machine in use** —
active firewall or listeners that are not the mesh's — so a forgotten flag cannot close a working a container running, or a port listening that is neither ssh nor the operating system's own — so a
machine. *How it is checked:* the adoption bed raises genesis converged on a machine in use and forgotten flag cannot close a working machine. *How it is checked:* the adoption bed raises genesis converged on a machine in use and
asserts the refusal, then adopted with the registry's port held and asserts the refusal names its asserts the refusal, then adopted with the registry's port held and asserts the refusal names its
holder, then with another port given asserts the foundation comes up, stays on that port once holder, then with another port given asserts the foundation comes up, stays on that port once
adopted as modules, and the machine's service is still reachable. adopted as modules, and the machine's service is still reachable.