novox/hq
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Merge pull request 'ADR 0100 (proposed): a node in use is adopted before it is converged' (#74) from feat/adoption-mode into main

Browse Source
This commit was merged in pull request #74.
This commit is contained in:
jschoubben
2026-09-22 16:35:32 +02:00
parent 21baf397a8 3d4ab23830
commit dfadfd23c0
9 changed files with 532 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
01-RESEARCH/012-the-minimum-viable-node/00-overview.md
+14
View File
@@ -7,6 +7,9 @@ touches:
- 03-DESIGN/01-to-be/05-the-node-host.md
- 03-DESIGN/00-as-is/05-runtime-and-installation.md
- 01-RESEARCH/011-the-module-graph/00-overview.md
- 02-DECISIONS/0078-the-store-and-broker-are-modules.md
- 02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md
- 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
---
# 012 — The minimum viable node, and adopting what is already there
@@ -180,6 +183,16 @@ same makes a node where something the mesh needed never happened indistinguishab
where a log level differed. Whether a failed line still lets adoption complete is therefore
reopened by adding severity, and is not decided here.
## Adoption as a mode, for the migration
*2026-09-22.* The migration from the predecessor mesh gave the middle state a length. A machine
running the predecessor is **adopted** when the mesh comes up on it — the predecessor's control
stopped, the machine's firewall and files kept in force, the mesh opening what it needs through
them — and stays adopted while its modules migrate one at a time, until the operator **converges**
it. Measured on the control-node, and weighed against the alternatives, in
[*migrating a node that is in use*](migrating-a-node-in-use.md); decided in
[ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md).
## Open questions
| Question | Why it is open |
@@ -191,6 +204,7 @@ reopened by adding severity, and is not decided here.
| Can a module say which of its settings are load-bearing? | The question that dissolves the conflict rule rather than choosing a side. A setting the module *requires* cannot be kept from the machine without producing something installed and broken; a setting it merely *prefers* should always yield. Until a module can say which is which, adoption is defaulting in the dark. Belongs with the graph. |
| Does a `failed` line still let adoption complete? | *Flags inform, they do not block* was decided about conflicts, where the mesh chose and the machine works. A failure is *we could not*, which is different in kind — and treating them alike hides the worse one behind the commoner one. |
| How is a flagged conflict reconciled, and by whom? | The briefing hands it to a session. What that session is empowered to change, and whether the resolution is recorded so the next adoption does not re-raise it, is undecided. |
| ~~How long is a machine adopted?~~ | **Decided** ([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)) — for as long as it is being migrated: a mode per node, recorded, ended by an explicit and previewed flip. |
| Where does the kept original live, and for how long? | Whether it is recorded in the node's state so adoption is visibly reversible, and whether it is returned when the mesh stops managing the thing. |
| What shape is a briefing? | Structured enough to be acted on, prose enough to be read. It is the first thing a session on a new node sees, which makes it an interface rather than a log. |
| Does owning a package mean owning its version? | Owning configuration and owning the package are different scopes. The second means the mesh decides which version is installed, and that decision then has to survive the machine's own package manager updating it. |
01-RESEARCH/012-the-minimum-viable-node/migrating-a-node-in-use.md
+154
View File
@@ -0,0 +1,154 @@
# Migrating a node that is in use — adoption as a mode, not a moment
*2026-09-22. Measured on the machine that will be the control-node, which is running the
predecessor mesh today. Nothing below names it; the counts are its own.*
## The question
The mesh replaces a predecessor mesh that is running, on the same machines, with the services
people use. The control-node is decided: it is the machine that already carries the predecessor's
broker and build pipeline. So the question is not *where* the mesh starts but **how a machine
running the predecessor becomes a node of the mesh without its services noticing** — and then how
each of the other machines follows.
## The operator's proposal
Proposed by the operator, 2026-09-22, and the shape this document tests:
1. **Stop the predecessor's control on a machine** — its daemons that write configuration: the
network and firewall configuration above all, which decide what is reachable and what is
blocked. Its services keep running; only the control over their configuration stops.
2. **Bring the mesh up on that machine in adoption mode.** It takes custody of those files and
keeps what it finds in force.
3. **Migrate the modules one at a time**, data preserved, per the cutover procedure.
4. **Move to the next machine and repeat** — adopted first, keeping its local configuration, then
migrated.
5. **When every machine is migrated, flip adoption mode**, and the mesh takes full control of the
configuration it has been holding.
This is the research above made concrete. It keeps the conflict rule already decided here — *on
conflict, what is on the machine stays* — and gives the middle state, *adopted*, a length: not a
one-time import before generating starts, but a mode that lasts for as long as the machine is
being migrated, ended by an explicit act.
## What the machine actually looks like
Measured, read-only:
| | |
|---|---|
| Containers running | 60, all the predecessor's services and their stores |
| The predecessor's control | user-level daemons, separate from the services; none of the 20 running system services is the predecessor's control |
| Firewall | the predecessor's, active: 54 incoming rules and 52 forwarding rules, each served port allowed explicitly — how a default-deny firewall reads |
| Files the predecessor's configuration sync writes | 12, of which 3 are system files (an ssh server drop-in, the package manager's configuration, one service's configuration); the rest are the operator's shell and agent files |
| Per-service configuration | environment and composition files per service, written by the predecessor's service tooling |
**Stopping the predecessor's control stops nothing that serves.** The services are containers and
system units that run without it; what stops is the rewriting of their configuration. Nothing
changes on the machine until something else writes.
## What collides, measured rather than assumed
An earlier note assumed the mesh's foundation could not stand beside the predecessor because both
want the store's and the broker's standard ports. **The measurement says otherwise.** The
predecessor publishes its own store on a non-standard port and its broker on another; the standard
ports the foundation binds for its store and its bus are free.
What does collide:
| The mesh wants | Held by | When |
|---|---|---|
| the registry's port | the predecessor's registry | at genesis — the foundation raises a registry |
| the broker's management port, on loopback | the predecessor's broker | at genesis |
| the private network's port | the predecessor's own tunnel | at genesis on a control-node that is the private network's hub; otherwise when the node is placed on it |
| the web ports | the predecessor's reverse proxy | when the route proxy is assigned |
| the resolver's port | a resolver the predecessor runs | when a resolver module is assigned |
On the control-node, which is the private network's hub, three of these fall at genesis and cannot
be deferred; two only when a particular module is taken, the moment its predecessor stops anyway.
Today the foundation's ports are **fixed**: written in the installer's bundle and in the
catalogue's manifests, so a collision is found when a container fails to bind, not before — and a
port changed at genesis would be changed back when the foundation is adopted as modules, since the
applier recreates a container whose declared spec differs. The private network's address range
must also stay clear of the range the predecessor's tunnel uses; on the machines measured they are
distinct.
## What would break if the mesh came up as it is today
**The firewall.** Genesis loads a base ruleset — drop anything undeclared — in its own table
([ADR 0088](../../02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md)). The
predecessor's firewall is a different table. The kernel runs every base chain registered at the
same hook, in priority order: an accept ends only its own chain and the packet goes on to the next,
and a drop in any is final — whether the other firewall's chains are nftables or legacy iptables.
So the
mesh's base ruleset would drop everything the predecessor's firewall allows and the mesh has not
declared — every web, mail and database port in the table above — the moment genesis ran.
**The files.** The host writes a declared file whatever it finds at the path, reporting it as
updated. A file the predecessor left — the ssh server drop-in, the resolver's configuration — is
replaced the first time a module declaring that path is assigned, before that module's service
has moved.
**The container names.** The applier keys a container on its name. A catalogue module whose
container carries the same name as the predecessor's service it replaces takes that container over
the moment it is assigned: today, *assigning a module is migrating it*, never a preparation.
**The published ports.** The foundation publishes its ports on every interface, and a published
container port reaches the container through the forwarded path, not the incoming one. A firewall
that filters only incoming traffic never sees it. The base ruleset is what keeps the store
unreachable from outside today — and it is the thing that cannot be loaded on this machine.
## The pipeline freezes while the control-node migrates
The predecessor's build pipeline and its coordinator run on the control-node. Stopping its control
there stops the predecessor's updates for **every** machine it manages, until the migration is
done. Their services keep running; they receive nothing new. That is the price of the proposal and
it is worth stating, not a reason against it: the migration is the period in which the predecessor
is being replaced, and it does not need to keep changing.
## What adoption mode has to mean
For the proposal to hold, *adopted* must be a **state the mesh records per node**, not an
intention, and each thing the mesh would otherwise take must say what it does in that state:
- **What is found is kept until its module is taken.** *Found* is precise: present at a declared
path or name with no record in the host's store. A found file or container is held, its original
recorded, until the operator **takes** the module on that node — the cutover, done when the
module's data has moved. Assigning prepares; taking migrates. Without the distinction the rule
never fires: the host only ever sees what assigned modules declare.
- **The firewall found on the machine stays in force.** The mesh loads no table on an adopted node
that drops by default or holds an accept. What it needs open it declares as openings the host converges
*through the found firewall*, on the incoming and the forwarded path, marked as the mesh's and
re-checked on every reconcile so a reload or reboot does not lose them. An accept in a table of
its own would not help: the found firewall's drop would still be final. What a table of its own
*can* do is refuse, and a refusal is final too — so the mesh guards the store and the broker's
management port from everyone but the private network and the machine itself, in a table that
only refuses, ahead of the container runtime's redirect — which the found firewall does not do
and cannot undo. The bus, the registry and the hub's port stay open to
anywhere: a node enrols before it has a private-network address.
- **The foundation's ports are the node's to give** — set at genesis, checked free, and kept as
that node's settings, read everywhere they are used, so adopting the foundation as modules does
not move them back.
- **The flip is per node and previewed** from what is actually reachable — listening sockets and
published ports, not the found firewall's allow list, which does not see what a container runtime
forwards.
## Options weighed
| Option | Verdict |
|---|---|
| Cut the machine over in one go: stop the predecessor's store, broker, registry and proxy, raise the foundation in their place | Rejected. Every predecessor service goes down until it has migrated, and the predecessor's other machines lose their broker. The rollback is restarting the predecessor, which is a recovery, not a step. |
| A separate machine as the control-node | Rejected. Contradicts the decision that the control-node is the machine that already carries the predecessor's broker and pipeline. |
| Converge on joining, as the mesh does today | Rejected. The base ruleset closes every predecessor port at genesis, and found files are replaced before their services move. |
| Make only the foundation's ports configurable and otherwise converge | Rejected as insufficient. It solves the bind collisions and none of the firewall or file ones. |
| Treat assigning a module as migrating it | Rejected on review. The rule that keeps found files would never fire — the host only sees what assigned modules declare — and an assignment on an adopted node would be an outage rather than a preparation. |
| **Adoption as a mode, per node, ended by an explicit flip** | The proposal. It is the conflict rule already decided here, given a duration. |
## What this leaves open
Answered by the decision record this feeds — [ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)
— only for the migration's needs. The general questions above stay open: whether a module can say
which settings are load-bearing, what shape a briefing takes, how a flagged conflict is reconciled.
One is newly sharp: the mesh opening ports through a firewall it did not install needs to speak that
firewall. There is one kind on the machines measured; a machine with another is not covered until
someone writes for it.
02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
+221
View File
@@ -0,0 +1,221 @@
---
topic: the mesh
status: accepted
date: 2026-09-22
deciders: jochen
reconstructed: false
extends: 02-DECISIONS/0078-the-store-and-broker-are-modules.md
---
# 100. A node in use is adopted before it is converged
## Context
The mesh replaces a predecessor mesh that is running, on the same machines, with the services
people use. The control-node is the machine that already carries the predecessor's broker and
build pipeline. The operator proposed the migration's shape: stop the predecessor's control on a
machine, bring the mesh up there in an adoption mode that keeps the machine's configuration in
force, migrate its modules one at a time, move to the next machine, and flip adoption off when every
machine is done.
Measured on the control-node ([research 012,
*migrating a node that is in use*](../01-RESEARCH/012-the-minimum-viable-node/migrating-a-node-in-use.md)):
60 predecessor containers; the predecessor's control in user-level daemons separate from its
services; its firewall active, with 54 incoming and 52 forwarding rules allowing each served port
explicitly; 12 files its configuration sync writes, 3 of them system files.
Four things in the mesh as it stands break that shape:
1. **The base ruleset closes the machine.** Genesis loads a drop-by-default table
([ADR 0088](0088-the-foundation-filters-before-anything-listens.md)). Every base chain at a hook
runs in priority order; an accept ends only its own chain and a drop in any is final — whether
the other firewall's chains are nftables or legacy iptables. So the predecessor's allowed ports
would close at genesis.
2. **The host replaces what it finds.** A declared file is written whatever is at its path, except
a file declared create-once. A declared container replaces a running one of the same name. So
assigning a module the predecessor also runs replaces the predecessor's service and its files at
once.
3. **Some foundation ports are held.** The foundation's store and bus ports are free — the
predecessor publishes its own elsewhere — but the registry's port and the broker's management
port are held, and on a control-node that is the private network's hub, so is the private
network's port. The foundation's ports are fixed in the installer's bundle and the catalogue's
manifests, so a collision surfaces as a container that fails to bind, and a port changed at
genesis would be changed back when the foundation is adopted as modules.
4. **Published container ports are forwarded, not received.** The foundation publishes its ports
on every interface; a predecessor firewall that filters only incoming traffic never sees them.
The base ruleset is what keeps the store unreachable from outside, and it is the thing that
cannot be loaded.
The mesh already adopts in one place: the foundation's store and broker are taken over in place as
modules, keyed on the container that is already running
([ADR 0078](0078-the-store-and-broker-are-modules.md)). The research this record draws on already
settled the conflict rule for adoption: *on conflict, what is on the machine stays*.
## Considered Options
1. **Cut each machine over in one go** — stop the predecessor's store, broker, registry and proxy,
raise the foundation in their place. Rejected: every predecessor service on the machine is down
until it has migrated, the predecessor's other machines lose their broker, and the rollback is
restarting the predecessor — a recovery, not a step.
2. **Put the control-node on a separate machine.** Rejected: the control-node is decided.
3. **Converge on joining, as today.** Rejected: the base ruleset closes the machine at genesis, and
the predecessor's services and files are replaced as soon as any module naming them is assigned.
4. **Only make the foundation's ports configurable.** Rejected as insufficient: it answers the bind
collisions and neither the firewall nor the files.
5. **Treat assigning a module as migrating it.** Considered and rejected on review: it makes the
rule that keeps found files never fire — the host only ever sees files of assigned modules — and
it makes an assignment on an adopted node an outage rather than a preparation.
6. **Adoption as a mode per node; each module taken explicitly; the node converged by an explicit
flip.** Adopted.
## Decision
**A node is adopted or converged, and the controller records which.** The operator says so: at
genesis for the control-node, and in the enrolment token for the others. The controller is
authoritative, and every declaration it sends says whether the node is adopted and which modules
have been taken on it. A node stays adopted until the operator converges it. An adopted node is
said to be adopted wherever the mesh reports a node's state.
**A converged genesis refuses a machine in use.** A machine is in use when a container is running
on it, or a port is listening on an address other than loopback that is not ssh's. Raised without
saying adopted on such a machine, genesis refuses and names every container and listener it
counted — a forgotten flag must not close a working machine.
**Before a node is adopted, its predecessor's control is stopped by the operator** — the daemons
that write its configuration. Its services keep running on what they have.
**Found means present with no record.** A file at a declared path, or a container at a declared
name, that the host's store has no record of writing is *found*. A file the host wrote in an
earlier life of the node is not found; its record says so.
**On an adopted node, what is found is kept until its module is taken.** The host keeps a found
file and a found container as they are, records the file's original content before anything else
happens to it, and reports each as held. Assigning a module on an adopted node prepares it: what
the module declares that is not found is created; what is found is held. **Taking a module** on a
node is its cutover — the operator's act, done when that module's data has moved — and from then
on the module's resources converge on that node like any other. What is held is never removed,
even when its module is unassigned, and a held file or container that changes while held — a file
rewritten, a container stopped or replaced — is reported as changed by something else, not reverted
or restarted: that is how a predecessor still writing is caught.
**The firewall found on the machine stays in force.** The mesh loads no table on an adopted node
that drops by default or holds an accept — neither genesis's base ruleset nor the filter module's
derived one. What the mesh needs reachable is declared as **openings**: a resource that says a port
is reachable, from where, on the incoming path or the forwarded path — a published container port is
forwarded. The controller derives them from the same inputs as the filter, each from where the
filter would admit it: the `listens` of the modules assigned there, the private network's hub port
and the bus and the registry from anywhere, the store's port and the broker's management port from
the private network. The host
converges an opening through the found firewall in that firewall's own terms, marks it as the
mesh's, and removes only what it marked; it re-checks each opening on every reconcile, so a reload
or a reboot of the found firewall does not lose it for longer than one reconcile. An opening is a
state, not a command, which is what lets it travel over the link. The host reports which firewall
it found. A machine with no firewall needs no openings; a machine with a kind no host speaks is
refused adoption.
**The mesh guards its own ports itself, in a table of its own that only refuses.** It passes
everything by default and holds nothing but refusals, and the two ports it refuses are the
foundation's own, checked free at genesis, so it cannot close anything the machine serves; it is
the mesh's, so the found firewall reloading does not touch it. It refuses the store's port and the
broker's management port except from the private network and from the machine itself — its
loopback and the container runtime's own networks, known by the interface a packet arrives on and
never by its source address alone — at the prerouting hook, ahead of the runtime's
destination translation, so it matches the port the packet was sent to, for both address
families. The bus and the registry stay reachable from anywhere, as a node
enrols over the bus and pulls from the registry before it has a private-network address
([ADR 0088](0088-the-foundation-filters-before-anything-listens.md)); so does the private network's
hub port. The store is unreachable from outside whatever the found firewall does, and on a machine
with none.
**The foundation's ports are the node's.** Every port the foundation binds is an input to genesis,
checked free before anything is raised, refused with the name of what holds it. The ports given
become that node's settings for the foundation's modules — the catalogue's numbers are only their
defaults — and every place that uses them reads them from there: the modules' containers, the
filter, the base ruleset, the private network's endpoint, and the addresses consumers are given.
The private network's address range must not overlap a tunnel the predecessor still runs; genesis
checks that too.
**Converging a node is one act, previewed.** It refuses while an assigned module still holds a
found container: each service is taken on its own, when its data has moved, never by the flip. The
preview lists what is reachable on the machine now — every listening socket and every published
container port — and for each whether an assigned module declares it or it will close, and every
module the flip will take, with the held files each will replace. The flip then takes those modules, loads the
mesh's derived filter in place of its refusal-only table, and retires the found firewall by
disabling it, never by flushing: the container runtime's rules and the found firewall's own
configuration stay on disk. Returning a converged node to adopted unloads the derived filter,
restores the refusal-only table, enables the found firewall again and converges the openings
through it once more; what was taken stays taken. A
node converges when its migration is done; the mesh is migrated when every node has converged.
**The order is the operator's:** the control-node first, adopted, its modules assigned and taken
one at a time; then each other machine, adopted, migrated, converged in turn. The predecessor's
pipeline runs on the control-node, so its updates stop for every machine while the migration runs;
that is accepted.
## Consequences
Each step says what it changes before it changes it. Adopting a node changes nothing that serves;
assigning a module adds what is not there; taking a module replaces one service; the flip replaces
the firewall, after naming every port it will close. Two steps are not undone by the mesh: taking a
module replaces the predecessor's container, and the kept original of a file is recorded but not
yet restored by any act of the mesh
([research 012](../01-RESEARCH/012-the-minimum-viable-node/00-overview.md) leaves where it lives
open).
What got harder:
- **The mesh must speak a firewall it did not install**, on both the incoming and the forwarded
path. One kind is found on the machines measured; another is refused until a host speaks it. The
mesh's own guard does not depend on it: that table is the mesh's.
- **The host gains a guard it did not have** — keep what you found — and its report must say
which files and containers it holds, or an adopted node reads as converged.
- **The declaration gains a node's mode and its taken modules**, and a resource, the opening.
- **Genesis grows inputs, and they outlive genesis.** The foundation's ports stop being constants;
every reader of them reads the node's settings.
- **A node can sit adopted indefinitely.** Nothing forces the flip; the mesh's status says which
nodes are adopted, so one left behind is visible.
- **This narrows [ADR 0088](0088-the-foundation-filters-before-anything-listens.md) for adopted
nodes**: the base ruleset is not loaded on a node raised adopted, and its duty — the store never
reachable from outside — passes to a table of the mesh's that only refuses.
## How it is checked
A lab bed prepares a machine the way the predecessor leaves one: its firewall allowing a served
port and denying the rest, a service container listening on that port under a name a catalogue
module also uses, a file at a path that module declares, a stand-in for the predecessor's control
that would rewrite that file, and a container holding the registry's port. Then:
- **Genesis converged** on it refuses and names every container and listener it counted.
- **Genesis adopted, with the registry's port held**, refuses and names the holder; with another
port given, the foundation comes up — and adopting the foundation as modules leaves it on that
port.
- **Nothing that serves changed**: the service is reachable from a second machine, the file is byte
for byte what it was, and the found firewall's rules differ only by rules marked as the mesh's.
- **The store is unreachable from outside** — probed from a machine off the private network, and
again after the found firewall is reloaded — and reachable over it and from a container on the
node itself; the bus is reachable from a machine that has not yet enrolled.
- **The mesh works through the found firewall, and keeps working after it is reloaded and after the
machine reboots**: the second machine enrols, and the openings are there again.
- **A predecessor still writing is caught**: with the stand-in left running, the held file's change
is reported and not reverted.
- **Assigning prepares, taking cuts over**: the module assigned holds the found container and file;
taken, it replaces them and its port is opened.
- **Converging previews, then changes**: it refuses while the service's module holds its found
container; once that module is taken, the preview names the service's port and a published port no
firewall rule mentions, and the modules it will take; after the flip the mesh's derived filter is
loaded, the found firewall is disabled with its configuration still on disk, the declared port is
open and the undeclared one closed. Returned to adopted, the found firewall is enabled again and
the derived filter is gone.
Unit tests hold the host to keeping a found file and container on an adopted node, converging them
once taken, never removing what it holds, and reporting a held file or container that changed;
genesis to refusing a held port and a converged raise on a machine in use; the controller to
carrying the mode and the taken modules in every declaration, deriving the openings, and refusing
a flip while a found container is held.
## References
- [research 012 — the minimum viable node, and adopting what is already there](../01-RESEARCH/012-the-minimum-viable-node/00-overview.md),
and its document [*migrating a node that is in use*](../01-RESEARCH/012-the-minimum-viable-node/migrating-a-node-in-use.md)
- [ADR 0005](0005-the-node-host.md), [ADR 0011](0011-managed-files-are-generated-never-edited.md),
[ADR 0078](0078-the-store-and-broker-are-modules.md), [ADR 0088](0088-the-foundation-filters-before-anything-listens.md)
02-DECISIONS/README.md
+1
View File
@@ -88,6 +88,7 @@ python3 00-META/checks/index.py fail if stale
- **0083** — [One push leaves the mesh consistent](0083-one-push-leaves-the-mesh-consistent.md)
- **0088** — [The foundation filters before anything listens](0088-the-foundation-filters-before-anything-listens.md)
- **0090** — [A failure that repeats is said to be stuck](0090-a-failure-that-repeats-is-said-to-be-stuck.md)
- **0100** — [A node in use is adopted before it is converged](0100-a-node-in-use-is-adopted-before-it-is-converged.md)
### Its tiers, from the bottom up
03-DESIGN/01-to-be/05-the-node-host.md
+26 -1
View File
@@ -2,8 +2,9 @@
layer: to-be
status: in-progress
code: [mesh-host]
updated: 2026-09-21
updated: 2026-09-22
decisions:
- 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
- 02-DECISIONS/0019-how-this-repository-works.md
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
- 02-DECISIONS/0005-the-node-host.md
@@ -130,6 +131,25 @@ the link; never asked downward.
[`09-the-node-lifecycle.md`](09-the-node-lifecycle.md)**, in full and in one place. This document
is the component; that one is what happens to it.
## What it finds, on an adopted node
*2026-09-22, [ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md).* A declaration says whether the node is adopted, and which of its
modules have been **taken**. *Found* is a file at a declared path, or a container at a declared
name, that the host's store has no record of writing. On an adopted node the host keeps what it
found for any module not yet taken: it records a found file's original content before anything
else, and it reports the file or container as held — a report that says what it holds, so an
adopted node never reads as converged. Once the module is taken, its resources converge like any
other. What is held is never removed, even when its module is unassigned, and a held file or
container that changes while held is reported as changed by something else, not reverted or
restarted. The host also
converges a new resource, the **opening** — a port made reachable through the firewall it found
([08-connectivity](08-connectivity.md)) — and reports which firewall it found. This is the
companion the host's ownership rule needed: *never touch what you did not create, unless adoption
made it yours — and while the node is adopted, not until its module is taken.* *How it is
checked:* unit tests hold the host to keeping a found file and container, converging them once
taken, never removing a held file and reporting one that changed; the adoption bed asserts a found
file byte for byte unchanged until its module is taken.
## Where a declaration comes from
One behaviour, two sources
@@ -193,6 +213,11 @@ ready; the current bundle simply does not. **All of them are built:**
| `container` | **built** | pinned by digest ([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)); identified by a label carrying a digest of the declaration that made it, because a runtime normalises what it is given and that is indistinguishable from drift |
| `action` | **built** | bundle-only ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)); verify is mandatory and is the idempotency check as well as the read-back |
One more shape is decided and not yet built: **`opening`**, on adopted nodes only
([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)) — a port
made reachable, from where, on the incoming or forwarded path, through the firewall found on the
machine, marked as the mesh's and re-checked on every reconcile.
**A service says what it must reflect, and that is declared state rather than a command.**
`restart-on` names files whose change means the unit must be restarted — because a running service
does not re-read its configuration, and replacing a file, finding the service already running and
03-DESIGN/01-to-be/07-the-foundation.md
+14 -1
View File
@@ -11,8 +11,9 @@ code:
- mesh-catalog modules/postgres
- mesh-catalog modules/lavinmq
- mesh-lab test/integration/mesh.test.ts (a bare machine becomes a mesh)
updated: 2026-09-21
updated: 2026-09-22
decisions:
- 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
- 02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
- 02-DECISIONS/0078-the-store-and-broker-are-modules.md
@@ -172,6 +173,18 @@ ruleset the machine loaded. **Checked** by the installer's bundle test (order an
the genesis bed, which probes the machine from outside for the length of the install: the store's
port never answers, the bus's does.
**Except on a machine raised adopted**
([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)). A
machine already serving under a predecessor has a firewall of its own, and a second, stricter
table would close everything it serves. There the base ruleset is not loaded and the filter module
is not assigned until the node converges; the foundation's ports are opened through the found
firewall, and a table of the mesh's that only refuses keeps the store's port and the broker's
management port from anyone but the private network and the machine itself — the same promise, the store's port never
answering from outside, kept by other means. The bus and the registry stay reachable from anywhere,
as they are here, because a node enrols and pulls before it has a private-network address. The
foundation's ports themselves are the node's, given at genesis and kept as its settings. **Checked**
by the adoption bed, which makes the same outside probe.
## Raising it
The order, from [research 011](../../01-RESEARCH/011-the-module-graph/worked-provider.md):
03-DESIGN/01-to-be/08-connectivity.md
+39 -2
View File
@@ -7,8 +7,9 @@ code:
- mesh-controller internal/identity/authority.go
- mesh-host internal/identity/serving.go
- mesh-host internal/apply (the service that reflects a rule set)
updated: 2026-09-21
updated: 2026-09-22
decisions:
- 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
- 02-DECISIONS/0099-a-step-that-runs-once-names-what-it-reads.md
- 02-DECISIONS/0098-a-fact-a-provider-makes-at-first-start-is-fetched-from-it.md
- 02-DECISIONS/0005-the-node-host.md
@@ -441,7 +442,8 @@ stopgap until the manifest layer can carry a label and a domain separately
**Derived from what is assigned here, and from the overlay's shape** — a node's open ports are a
consequence of what runs on it and who must reach it, not an independent declaration to keep in
step by hand.
step by hand. That is true of a converged node; an adopted one keeps the firewall it was found
with until it converges (below).
**A rule names its source** ([ADR 0007](../../02-DECISIONS/0007-connectivity.md)).
A rule with no source is open, and must say so rather than appear to restrict something. `scope:`
@@ -532,6 +534,41 @@ undeclared one does not — then the module is removed and the port closes with
rule. *A rule set that is written but never loaded passes every check that reads the file, which
is why the check reads packets.*
### On an adopted node
*2026-09-22, [ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md).* **The firewall found on the machine stays in force**, and the mesh
loads no table there that drops by default or holds an accept — neither genesis's base ruleset nor
the derived one. Every base chain at a hook runs and a drop in any is final, whatever the other
firewall is written in, so a second, stricter table would close every port the machine serves, and
an accept in one would open nothing the found firewall drops. What the mesh needs reachable it
declares as **openings**: a port, from where, on the incoming path or the forwarded path — a
published container port is forwarded, and a firewall that filters only incoming traffic never
sees it. The controller derives them from what the filter would be derived from, each from where
the filter would admit it: the assigned modules' `listens`, the hub's port, the bus and the registry
from anywhere; the store's port and the broker's management port from the private network. The host converges each opening through
the found firewall in that firewall's own terms, marks it as the mesh's, removes only what it
marked, and re-checks every opening on each reconcile so a reload or a reboot does not lose it for
longer than one reconcile. An opening is state, not a command, so it travels over the link like any
other resource. **The mesh guards its own ports in a table of its own that only refuses** —
passing everything by default and holding nothing but refusals of the foundation's own two ports,
checked free at genesis, so it cannot close what the machine serves, and the found firewall's
reload does not touch it. It refuses the store's port and the broker's management port except from
the private network and the machine itself — loopback and the container runtime's networks, known by the interface a packet arrives on, never by
its source address alone — at
the prerouting hook, before the container runtime redirects the packet, so it matches the port the
packet was sent to, for both address families; the
bus, the registry and the hub's port stay reachable from anywhere, as a node enrols and pulls
before it has a private-network address. One kind of found firewall is spoken; a machine with none
needs no openings, and a machine with another kind is refused adoption. Converging the node refuses
while a found container is still held; otherwise it previews what is reachable now — listening
sockets and published ports — what will close and which modules it will take, then loads the
derived filter in place of the refusal-only table and disables the found firewall without flushing
it. *How it is checked:* the adoption bed asserts the found firewall's rules differ only by the
mesh's marked rules, that the store is unreachable from off the private network before and after
the found firewall reloads and reachable from a container on the node, that a machine not yet enrolled reaches the bus, that a second machine
enrols through the openings before and after a reload and a reboot, and that after the flip the
declared port is open and the undeclared one closed.
## 5 — Certificates
**Two authorities, kept separate on purpose.**
03-DESIGN/01-to-be/09-the-node-lifecycle.md
+36 -5
View File
@@ -8,8 +8,9 @@ code:
- mesh-host packaging/nox-mesh-host-network.sh
- mesh-controller internal/token
- mesh-controller internal/inventory/nodes.go
updated: 2026-08-31
updated: 2026-09-22
decisions:
- 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
- 02-DECISIONS/0005-the-node-host.md
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
@@ -282,8 +283,29 @@ runs. Building the start mechanism before deciding that would be building it for
## Adoption: what happens to what is already there
Adoption is not a state. It is what the **first apply** does when it is told to own something a
machine already has ([research 012](../../01-RESEARCH/012-the-minimum-viable-node/00-overview.md)).
**An enrolled node is adopted or converged, and the mesh records which** ([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)).
This was once said the other way — adoption as only what the first apply does — and a machine
being migrated from a mesh already running on it made the middle state last: while its services
move one at a time, what it already has must stay in force. So *adopted* is a second axis of an
enrolled node, not a step on the path above: a machine in use is enrolled adopted, and stays so
until its migration is done and the operator converges it. A machine that was empty is enrolled
converged, as before
([research 012](../../01-RESEARCH/012-the-minimum-viable-node/00-overview.md)).
The operator says a node is adopted — at genesis for the control-node, in the enrolment token for
the others — and the controller records it and says so in every declaration, with the modules
**taken** on that node. On an adopted node what is found is held until its module is taken
([05-the-node-host](05-the-node-host.md)): assigning a module prepares it, taking it is its
cutover. The firewall found there stays in force and the mesh opens what it needs through it
([08-connectivity](08-connectivity.md)). **Converging is one act per node, previewed**: it refuses
while an assigned module still holds a found container; otherwise it lists what is reachable on the
machine now — listening sockets and published ports — whether an assigned module declares each or
it will close, and which modules it will take, then takes them, loads the mesh's own filter in place
of its refusal-only table and disables the found firewall without flushing it. Returning a
converged node to adopted unloads the derived filter, restores the refusal-only table, enables the
found firewall again and converges the openings through it; what was taken stays taken. *How it is checked:* a lab bed prepares a machine the way
a predecessor leaves one and asserts nothing that serves changes until a module is taken or the
node is converged, and that the flip closes exactly what the preview said.
A candidate machine is not empty. It has a package manager, probably a container runtime,
configuration somebody chose. [ADR 0005](../../02-DECISIONS/0005-the-node-host.md)
@@ -300,11 +322,16 @@ an installation. This is a *never* rule, and it earns that from the worst loss i
a tool acting on a path it did not own.
**On conflict, the machine's configuration wins.** Adoption always completes; the conflict is
flagged and reconciled afterwards. A machine in use keeps working exactly as it did.
flagged and reconciled afterwards. A machine in use keeps working exactly as it did. Two things are
not conflicts in this sense ([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)):
a module the operator has *taken* replaces what it found, because that is its cutover; and genesis
refuses a port the foundation needs that something else holds, because a foundation that cannot
bind is not adopted but broken.
**Adoption produces a briefing**, not just a result: what it found, what it took over, and what
it could not resolve — with each line marked `ok`, `kept`, `unknown` or `failed`, and the overall
outcome **derived** from the worst line rather than stated alongside it.
outcome **derived** from the worst line rather than stated alongside it. A file or container the
host is holding on an adopted node is a `kept` line for as long as it is held.
---
@@ -366,6 +393,10 @@ should be and sends it; the host applies the difference and removes what is no l
The host removes what it *made* and leaves what it merely *configured*. Uninstalling a container
runtime because a declaration changed would stop every container on the node.
*On an adopted node* ([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)),
what the host is holding was found, not made, so nothing held is ever removed: a held file or
container whose module is unassigned stays where it is.
---
## enrolled ⇄ disconnected
03-DESIGN/01-to-be/17-raising-a-mesh.md
+27 -2
View File
@@ -5,8 +5,9 @@ code:
- mesh-host cmd/mesh-bootstrap
- mesh-host internal/bootstrap
- mesh-lab test/integration/whole-mesh-full.test.ts
updated: 2026-09-21
updated: 2026-09-22
decisions:
- 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
- 02-DECISIONS/0067-genesis-is-a-pivot.md
- 02-DECISIONS/0069-a-module-is-a-repository-and-a-path.md
- 02-DECISIONS/0006-the-substrate-and-the-control-plane.md
@@ -97,6 +98,27 @@ the temporary one.** The pivot is complete: what raised the mesh is gone, and wh
like any other. From here the mesh can build and roll out its own upgrades, including to the thing
that runs it.
### Genesis on a machine in use
*2026-09-22, [ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md).* A control-node that is already running a predecessor mesh is raised
**adopted**, said so by the operator. The operator stops the predecessor's control on it first —
the daemons that write its configuration — and its services keep running. **Every port the
foundation binds is an input to genesis**, checked free before anything is raised, refused with the
name of what holds it; the ports given become the node's settings for the foundation's modules, so
adopting the foundation as modules keeps them, and every reader of them — the filter, the base
ruleset, the private network's endpoint, the addresses consumers are given — reads them there. On
the control-node measured, the store's and the bus's usual ports were free and the registry's, the
broker's management port and, as the private network's hub, its port were held. Genesis also
checks the private network's range does not overlap a tunnel the predecessor runs. Genesis adopted
**does not load the base ruleset**: the machine's own firewall already filters; the mesh opens its
foundation's ports through it and keeps the store from outside with a table of its own that only
refuses ([08-connectivity](08-connectivity.md)). **A converged genesis refuses a machine in use** —
a container running, or a port listening on an address other than loopback that is not ssh's —
and names every one it counted, so a forgotten flag cannot close a working machine. *How it is checked:* the adoption bed raises genesis converged on a machine in use and
asserts the refusal, then adopted with the registry's port held and asserts the refusal names its
holder, then with another port given asserts the foundation comes up, stays on that port once
adopted as modules, and the machine's service is still reachable.
## After the pivot, and still part of installing
Genesis ends with a mesh of one that runs, and that is not the same as a mesh that works. What it
@@ -142,7 +164,10 @@ What remains after *that* belongs to somebody else: adding machines, and decidin
A machine joins with the host binary and a token. It does not raise a foundation, does not install a
registry, and is never enrolled twice. The mesh already knows how to tell a machine what to be;
joining is the point at which a machine starts listening.
joining is the point at which a machine starts listening. A machine in use joins **adopted**: the
token says so, the operator has stopped the predecessor's control on it first, and from then on it
keeps what it has until each module is taken
([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)).
## Where the line falls