novox/hq
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Order the records the way the system is learned

Browse Source 
Jochen asked whether the order made sense. It did not -- it followed when
things happened to be decided, which after consolidation is fictional anyway
since record 5 alone folds decisions taken across a week.

Concretely wrong before: the domain statement sat at 8, after five engineering
rules; the constitution was scattered across 5, 12 and 17; the tiers landed at
15, 16, 21 and 22 with process records in between.

Now it walks: what the mesh is (1-3), its tiers from the bottom up (4-8), what
runs on them and how it gets there (9-10), how it is built (11-16), how it is
checked (17-18), how we work (19-23).

Two things made this safe rather than free. It is a permutation, not a
compaction, so the renames go through temporary names -- otherwise two files
want one slot and one is lost. And the reference rewrite is a single
simultaneous pass, because almost every number moved into a slot another number
was vacating; replacing one at a time would have cascaded and pointed things at
the wrong record while still resolving.

Verified: 284 [ADR NNNN](path) links across the repository, all with matching
text and target.

The ordering principle is now stated in 19 rather than left implicit -- the
repository already said "the numbering is the flow" about its folders, and
there was no reason for the records to be the exception.
This commit is contained in:
jschoubben
2026-08-28 23:30:42 +02:00
parent e1febe8e0f
commit 333356cff3
85 changed files with 471 additions and 465 deletions
Download Patch File Download Diff File Expand all files Collapse all files
03-DESIGN/00-as-is/00-overview.md
+12 -12
View File
@@ -4,9 +4,9 @@ status: implemented
code: [hal]
updated: 2026-08-23
decisions:
- 02-DECISIONS/0001-nodes-communicate-over-a-broker.md
- 02-DECISIONS/0019-modules-and-the-graph.md
- 02-DECISIONS/0021-the-substrate-and-the-control-plane.md
- 02-DECISIONS/0002-nodes-communicate-over-a-broker.md
- 02-DECISIONS/0009-modules-and-the-graph.md
- 02-DECISIONS/0006-the-substrate-and-the-control-plane.md
---
# The mesh as it stands
@@ -28,11 +28,11 @@ onto it and can be regenerated.
containerised service is a module. A set of capabilities with no service behind them is a
module. A bare marker whose whole content is that a node has it is a module. The mesh's own
components are modules on exactly the same terms as everything else it carries
([ADR 0019](../../02-DECISIONS/0019-modules-and-the-graph.md)).
([ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md)).
**An agent** is a participant. Some agents are human. What differs is modality — how the agent
acts — and not category: both hold identity, both act, both accumulate memory
([ADR 0007](../../02-DECISIONS/0007-agents-are-persistent-employees.md)).
([ADR 0003](../../02-DECISIONS/0003-agents-are-persistent-employees.md)).
## Where truth lives
@@ -40,10 +40,10 @@ The repository defines **what exists**: the modules, what each declares, how eac
The mesh database defines **what runs where**: which node is assigned which module, at which
selection, with which overrides, plus the settings every node reads. No node-to-module mapping
is ever committed ([ADR 0021](../../02-DECISIONS/0021-the-substrate-and-the-control-plane.md)).
is ever committed ([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)).
Everything on a node's disk is **derived** from those two, and is regenerated rather than
edited ([ADR 0002](../../02-DECISIONS/0002-managed-files-are-generated-never-edited.md)). A node that
edited ([ADR 0011](../../02-DECISIONS/0011-managed-files-are-generated-never-edited.md)). A node that
loses its database keeps running from a local cache, which is deliberate and has the obvious
cost: the cache carries no indication of its own age.
@@ -51,7 +51,7 @@ cost: the cache carries no indication of its own age.
Nothing dials a node. Every node dials the broker outbound, owns an exchange named for itself,
and consumes from its own request queue
([ADR 0001](../../02-DECISIONS/0001-nodes-communicate-over-a-broker.md)). Three message shapes carry
([ADR 0002](../../02-DECISIONS/0002-nodes-communicate-over-a-broker.md)). Three message shapes carry
everything: requests expecting a reply, commands instructing that a stage of work be done, and
events stating that something happened.
@@ -65,9 +65,9 @@ goes to where the capability is.
A push to the forge is the only trigger. What follows is three silos with deliberately
different cardinality: compile once, package and upload once, then install-configure-start-
verify **on every assigned node**
([ADR 0023](../../02-DECISIONS/0023-delivery.md)). What travels between
([ADR 0010](../../02-DECISIONS/0010-delivery.md)). What travels between
build and node is a self-contained build output, so a deploy is extract-and-run and touches no
network ([ADR 0023](../../02-DECISIONS/0023-delivery.md)).
network ([ADR 0010](../../02-DECISIONS/0010-delivery.md)).
Modules are resolved into dependency levels and a level completes before the next begins, so a
module always builds against its dependencies as they were just published.
@@ -78,7 +78,7 @@ A module declares what it **provides** and what it **requires**. The mesh satisf
requirement: it creates the resource, generates the credential, records the grant, and writes
the values where the module will read them. The module never learns which node its database
lives on, and nobody ever writes a credential by hand
([ADR 0019](../../02-DECISIONS/0019-modules-and-the-graph.md)).
([ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md)).
This is the property the mesh's whole shape rests on, and it is why provisioning is treated as
a core concern rather than as plumbing.
@@ -92,7 +92,7 @@ named for a feature the module does not declare, a stage that reported it had di
message rather than that the effect happened, a package that 404ed from every mirror while the
job went green.
[ADR 0023](../../02-DECISIONS/0023-delivery.md) is the response, and it is applied
[ADR 0010](../../02-DECISIONS/0010-delivery.md) is the response, and it is applied
instance by instance rather than enforced by a mechanism. New instances are still being found.
That is an as-is fact, not a criticism: it is the single most useful thing to know about this
system before changing it.
03-DESIGN/00-as-is/01-mesh-and-transport.md
+2 -2
View File
@@ -4,8 +4,8 @@ status: implemented
code: [hal]
updated: 2026-08-23
decisions:
- 02-DECISIONS/0001-nodes-communicate-over-a-broker.md
- 02-DECISIONS/0021-the-substrate-and-the-control-plane.md
- 02-DECISIONS/0002-nodes-communicate-over-a-broker.md
- 02-DECISIONS/0006-the-substrate-and-the-control-plane.md
---
# The mesh and its transport
03-DESIGN/00-as-is/02-modules-and-manifests.md
+5 -5
View File
@@ -4,9 +4,9 @@ status: implemented
code: [hal]
updated: 2026-08-23
decisions:
- 02-DECISIONS/0019-modules-and-the-graph.md
- 02-DECISIONS/0003-schema-changes-are-numbered-migrations.md
- 02-DECISIONS/0004-no-npm-workspace.md
- 02-DECISIONS/0009-modules-and-the-graph.md
- 02-DECISIONS/0013-schema-changes-are-numbered-migrations.md
- 02-DECISIONS/0014-no-npm-workspace.md
---
# Modules, manifests and features
@@ -81,7 +81,7 @@ recorded in the knowledge base; both presented as "the change did not apply" wit
## Dependencies between modules
Modules depend on each other, above all on the shared library they all build against. There is
**no workspace** ([ADR 0004](../../02-DECISIONS/0004-no-npm-workspace.md)): each module is a standalone
**no workspace** ([ADR 0014](../../02-DECISIONS/0014-no-npm-workspace.md)): each module is a standalone
package consuming published dependencies, including the mesh's own.
The pipeline resolves modules into dependency **levels** and completes a level before starting
@@ -96,7 +96,7 @@ since (see
A module that owns state owns its migrations: numbered, written in the module's own language,
compiled with it, frozen once they have run anywhere, and idempotent so that re-running is safe
([ADR 0003](../../02-DECISIONS/0003-schema-changes-are-numbered-migrations.md)).
([ADR 0013](../../02-DECISIONS/0013-schema-changes-are-numbered-migrations.md)).
Two kinds exist and the distinction matters: migrations against the module's **own** local
state, and migrations against a **provisioned** resource, which run on the node that consumes
03-DESIGN/00-as-is/03-provisioning.md
+2 -2
View File
@@ -4,8 +4,8 @@ status: implemented
code: [hal]
updated: 2026-08-23
decisions:
- 02-DECISIONS/0019-modules-and-the-graph.md
- 02-DECISIONS/0002-managed-files-are-generated-never-edited.md
- 02-DECISIONS/0009-modules-and-the-graph.md
- 02-DECISIONS/0011-managed-files-are-generated-never-edited.md
---
# Provisioning
03-DESIGN/00-as-is/04-delivery.md
+5 -5
View File
@@ -4,9 +4,9 @@ status: implemented
code: [hal]
updated: 2026-08-23
decisions:
- 02-DECISIONS/0023-delivery.md
- 02-DECISIONS/0023-delivery.md
- 02-DECISIONS/0023-delivery.md
- 02-DECISIONS/0010-delivery.md
- 02-DECISIONS/0010-delivery.md
- 02-DECISIONS/0010-delivery.md
---
# Delivery — from a push to a running node
@@ -31,7 +31,7 @@ merge that created no pipeline, and nothing said so**.
## Three silos
Cardinality is the whole point, and the three differ
([ADR 0023](../../02-DECISIONS/0023-delivery.md)):
([ADR 0010](../../02-DECISIONS/0010-delivery.md)):
| Silo | Runs | Where | Does |
|---|---|---|---|
@@ -50,7 +50,7 @@ later stage runs.
## The artifact
The artifact is **build output** — compiled and bundled with its dependency graph inlined —
never a filtered copy of source ([ADR 0023](../../02-DECISIONS/0023-delivery.md)).
never a filtered copy of source ([ADR 0010](../../02-DECISIONS/0010-delivery.md)).
A deploy is extract-and-run and touches no network.
The consequence is the whole cost of the decision: **anything not in the build output does not
03-DESIGN/00-as-is/05-runtime-and-installation.md
+5 -5
View File
@@ -4,8 +4,8 @@ status: implemented
code: [hal]
updated: 2026-08-23
decisions:
- 02-DECISIONS/0019-modules-and-the-graph.md
- 02-DECISIONS/0010-the-mesh-creates-no-symlinks.md
- 02-DECISIONS/0009-modules-and-the-graph.md
- 02-DECISIONS/0012-the-mesh-creates-no-symlinks.md
---
# The node runtime, and how a node comes into being
@@ -38,7 +38,7 @@ suggestive word in the system names the node runtime, and the component whose ma
"mesh messaging" is documented elsewhere as the interactive runtime. Anatomy makes attractive
names and poor boundaries.
[ADR 0008](../../02-DECISIONS/0008-mesh-brokers-nodes-host-agents-think.md) replaces this with names
[ADR 0001](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md) replaces this with names
taken from what each part owns. Until then, this is the vocabulary in the code.
## Starting a module
@@ -57,14 +57,14 @@ outstanding local migrations, create data directories with the right ownership,
service under supervision.
**The installer is the only thing that creates a link** ([ADR
0011](../../02-DECISIONS/0010-the-mesh-creates-no-symlinks.md)). It reconciles rather than assumes: a
0011](../../02-DECISIONS/0012-the-mesh-creates-no-symlinks.md)). It reconciles rather than assumes: a
missing link is created, a stale one repointed, and a real file found where a link belongs is
adopted into the node's override area and replaced. Nothing else — not a hook, not a fix, not a
person debugging — creates one.
That is the as-is. The intent is to remove linking altogether and derive a real file instead,
which the reconciliation machinery already makes possible
([ADR 0010](../../02-DECISIONS/0010-the-mesh-creates-no-symlinks.md), proposed). What is described above
([ADR 0012](../../02-DECISIONS/0012-the-mesh-creates-no-symlinks.md), proposed). What is described above
is what runs today.
## Supervision
03-DESIGN/00-as-is/06-configuration-and-secrets.md
+4 -4
View File
@@ -4,8 +4,8 @@ status: implemented
code: [hal]
updated: 2026-08-23
decisions:
- 02-DECISIONS/0002-managed-files-are-generated-never-edited.md
- 02-DECISIONS/0019-modules-and-the-graph.md
- 02-DECISIONS/0011-managed-files-are-generated-never-edited.md
- 02-DECISIONS/0009-modules-and-the-graph.md
---
# Configuration and secrets
@@ -17,7 +17,7 @@ files is **generated**.
A managed file is derived from the mesh database. A synchroniser rewrites it when the values
behind it change. The write path is the mesh operation that owns the value; the file is an
output ([ADR 0002](../../02-DECISIONS/0002-managed-files-are-generated-never-edited.md)).
output ([ADR 0011](../../02-DECISIONS/0011-managed-files-are-generated-never-edited.md)).
An edit to a managed file survives until the next synchronisation and is then overwritten
silently, taking whatever it was fixing with it — bringing back the bug the edit had removed,
@@ -61,7 +61,7 @@ are both left behind. Configuration is additive in practice, whatever the manife
Generated secrets are produced by the mesh, never authored. Provisioned credentials arrive as
database overrides written by the provisioner and are marked as such, so they can be
distinguished from a deliberate override and cleaned up when the grant is removed
([ADR 0019](../../02-DECISIONS/0019-modules-and-the-graph.md)).
([ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md)).
Nothing in the repository contains a credential. The repository has no per-node content at all,
which is what makes that guarantee structural rather than a matter of care.
03-DESIGN/00-as-is/07-knowledge.md
+1 -1
View File
@@ -40,7 +40,7 @@ owning approval and promotion at the boundary. Proposals to edit are reviewed ra
applied.
This is where the mesh's **governed** documents live, including the constitution injected into
design sessions ([ADR 0005](../../02-DECISIONS/0005-the-mesh-is-governed-by-a-constitution.md)).
design sessions ([ADR 0020](../../02-DECISIONS/0020-the-mesh-is-governed-by-a-constitution.md)).
## Why both
03-DESIGN/00-as-is/08-agents-and-work.md
+6 -6
View File
@@ -4,8 +4,8 @@ status: implemented
code: [hal]
updated: 2026-08-23
decisions:
- 02-DECISIONS/0007-agents-are-persistent-employees.md
- 02-DECISIONS/0005-the-mesh-is-governed-by-a-constitution.md
- 02-DECISIONS/0003-agents-are-persistent-employees.md
- 02-DECISIONS/0020-the-mesh-is-governed-by-a-constitution.md
---
# Agents and work
@@ -17,7 +17,7 @@ model they run under is the employee model, not a worker pool.
An agent is a singular named identity with a home node, a workspace on that node, accumulating
memory, and an explicit lifecycle
([ADR 0007](../../02-DECISIONS/0007-agents-are-persistent-employees.md)).
([ADR 0003](../../02-DECISIONS/0003-agents-are-persistent-employees.md)).
| Property | Meaning |
|---|---|
@@ -45,7 +45,7 @@ Both hold identity, both act, both accumulate memory.
The mesh does not currently record modality completely. Which user, on which node, a human
agent acts as is **required by the model and not stored** — an open question carried over from
[ADR 0008](../../02-DECISIONS/0008-mesh-brokers-nodes-host-agents-think.md).
[ADR 0001](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md).
## Work
@@ -70,7 +70,7 @@ template that names the phases.
This is where governance meets execution. The constitution is injected into every eligible
meeting turn — agents do not fetch it, it arrives — and a check phase verifies the meeting's
output against it before the meeting may proceed
([ADR 0005](../../02-DECISIONS/0005-the-mesh-is-governed-by-a-constitution.md)). A named violation
([ADR 0020](../../02-DECISIONS/0020-the-mesh-is-governed-by-a-constitution.md)). A named violation
blocks progress.
Meeting turns run on the orchestrator's node regardless of where the participating agents are
@@ -84,5 +84,5 @@ integrate through the record, never through a shared schema* — being violated
own largest component, and it is the reason work that belongs to one domain keeps having to be
implemented in another.
[ADR 0008](../../02-DECISIONS/0008-mesh-brokers-nodes-host-agents-think.md) dissolves that arrangement.
[ADR 0001](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md) dissolves that arrangement.
Until it does, this is the shape.
03-DESIGN/00-as-is/09-interfaces-and-observability.md
+2 -2
View File
@@ -4,8 +4,8 @@ status: implemented
code: [hal]
updated: 2026-08-23
decisions:
- 02-DECISIONS/0001-nodes-communicate-over-a-broker.md
- 02-DECISIONS/0023-delivery.md
- 02-DECISIONS/0002-nodes-communicate-over-a-broker.md
- 02-DECISIONS/0010-delivery.md
---
# Interfaces and observability
03-DESIGN/00-as-is/10-module-catalogue.md
+8 -8
View File
@@ -4,16 +4,16 @@ status: implemented
code: [hal]
updated: 2026-08-23
decisions:
- 02-DECISIONS/0019-modules-and-the-graph.md
- 02-DECISIONS/0006-applications-live-in-their-own-repository.md
- 02-DECISIONS/0019-modules-and-the-graph.md
- 02-DECISIONS/0009-modules-and-the-graph.md
- 02-DECISIONS/0015-applications-live-in-their-own-repository.md
- 02-DECISIONS/0009-modules-and-the-graph.md
---
# The catalogue, and what its shape says
The catalogue holds **124 modules**. Thirty-three belong to the mesh's own domain; the other
ninety-one run *on* the mesh rather than being *of* it
([ADR 0008](../../02-DECISIONS/0008-mesh-brokers-nodes-host-agents-think.md)).
([ADR 0001](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md)).
The count is not the finding. The **shape** is.
@@ -55,11 +55,11 @@ connectivity is made four times.
the unit of one piece of software, because that is the only granularity the module system
offers.
This is the same failure [ADR 0008](../../02-DECISIONS/0008-mesh-brokers-nodes-host-agents-think.md)
This is the same failure [ADR 0001](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md)
names for the platform core — *boundaries drawn by deployment accident rather than by domain* —
appearing outside it, at four times the scale. The core is being recomposed; the flat level is
addressed in principle by
[ADR 0019](../../02-DECISIONS/0019-modules-and-the-graph.md), which
[ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md), which
deliberately does not yet settle the domain list.
## Where the shape came from
@@ -90,7 +90,7 @@ never stated as assumptions — they were just how the thing already worked.
**This is the most useful single fact for anyone changing the catalogue**, and it is why the
linking principle in particular reads as a deliberate architectural choice when it is an
inheritance. See [ADR 0010](../../02-DECISIONS/0010-the-mesh-creates-no-symlinks.md), whose case
inheritance. See [ADR 0012](../../02-DECISIONS/0012-the-mesh-creates-no-symlinks.md), whose case
this strengthens: the argument for links was never made *for a mesh*.
It also explains the measurement in
@@ -108,7 +108,7 @@ which is what makes dogfooding structural rather than a discipline, and what mak
module out of the repository safe.
**Placement is already decided.** A standalone application belongs in its own repository
([ADR 0006](../../02-DECISIONS/0006-applications-live-in-their-own-repository.md)), and reviewers reject
([ADR 0015](../../02-DECISIONS/0015-applications-live-in-their-own-repository.md)), and reviewers reject
it in the monorepo. The catalogue's flat level is not a dumping ground by policy; it is one by
history.
03-DESIGN/00-as-is/11-the-lab.md
+7 -7
View File
@@ -4,11 +4,11 @@ status: implemented
code: [mesh-lab]
updated: 2026-08-25
decisions:
- 02-DECISIONS/0009-the-lab.md
- 02-DECISIONS/0009-the-lab.md
- 02-DECISIONS/0009-the-lab.md
- 02-DECISIONS/0009-the-lab.md
- 02-DECISIONS/0009-the-lab.md
- 02-DECISIONS/0016-the-lab.md
- 02-DECISIONS/0016-the-lab.md
- 02-DECISIONS/0016-the-lab.md
- 02-DECISIONS/0016-the-lab.md
- 02-DECISIONS/0016-the-lab.md
---
# The lab, as it stands
@@ -63,7 +63,7 @@ is built.
**The drawing was never designed.** `diagram` renders a scenario as draw.io, from the
declaration or from the running instance, and it exists because it was asked for during the
build. It has tests and a decision record ([ADR 0014](../../02-DECISIONS/0014-a-picture-is-read-from-what-runs.md),
build. It has tests and a decision record ([ADR 0018](../../02-DECISIONS/0018-a-picture-is-read-from-what-runs.md),
proposed) but no document in the to-be layer. It is recorded here because it runs, not because
it was planned.
@@ -117,7 +117,7 @@ and snapshots roughly 76× slower, which does not make the lab slow, it makes it
`npm run check` — typecheck over source *and* tests, then the offline suite, then integration
against a real hypervisor. Mocking the hypervisor is forbidden
([ADR 0013](../../02-DECISIONS/0013-a-test-defends-a-decision.md), proposed): a test that fakes
([ADR 0017](../../02-DECISIONS/0017-a-test-defends-a-decision.md), proposed): a test that fakes
the system under integration asserts that the fake behaves as expected.
Integration tests **skip with a reason** on a machine that cannot raise scenarios, rather than
03-DESIGN/01-to-be/00-work-breakdown.md
+2 -2
View File
@@ -3,12 +3,12 @@ layer: to-be
status: designed
code: [hal]
updated: 2026-08-23
decisions: [02-DECISIONS/0008-mesh-brokers-nodes-host-agents-think.md]
decisions: [02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md]
---
# Work breakdown — the decomposition
How [ADR 0008](../../02-DECISIONS/0008-mesh-brokers-nodes-host-agents-think.md) gets built, in what order, and where a human must look.
How [ADR 0001](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md) gets built, in what order, and where a human must look.
Ordering is not preference. Each phase removes a constraint the next one needs gone.
03-DESIGN/01-to-be/01-end-to-end-testing.md
+5 -5
View File
@@ -4,9 +4,9 @@ status: in-progress
code: [mesh-lab]
updated: 2026-08-23
decisions:
- 02-DECISIONS/0009-the-lab.md
- 02-DECISIONS/0009-the-lab.md
- 02-DECISIONS/0011-how-this-repository-works.md
- 02-DECISIONS/0016-the-lab.md
- 02-DECISIONS/0016-the-lab.md
- 02-DECISIONS/0019-how-this-repository-works.md
---
# End-to-end testing
@@ -37,7 +37,7 @@ today, that is a gap in the vocabulary rather than a reason to privilege that sh
The design below describes a scenario as a complete mesh — forge (Gitea), coordinator,
delivery cascade — because what it tests is a module. **That is the larger of two classes, and
not the first one built** ([ADR 0009](../../02-DECISIONS/0009-the-lab.md)).
not the first one built** ([ADR 0016](../../02-DECISIONS/0016-the-lab.md)).
| | **Bootstrap scenario** | **Full scenario** |
|---|---|---|
@@ -127,7 +127,7 @@ drifts.
a mesh named by the request instead.
- **Scenarios must be concurrent and cheap.** Several agents working means several scenarios
at once, each needing its own network and nodes. A lab node is a virtual machine
([ADR 0009](../../02-DECISIONS/0009-the-lab.md)), and snapshots are
([ADR 0016](../../02-DECISIONS/0016-the-lab.md)), and snapshots are
what make repetition cheap — restoring a scenario costs far less than building one. The
earlier argument here, that only system containers made this affordable, was superseded: the
scale it assumed was invented rather than required.
03-DESIGN/01-to-be/02-scenario-declaration.md
+11 -11
View File
@@ -4,9 +4,9 @@ status: in-progress
code: [mesh-lab]
updated: 2026-08-25
decisions:
- 02-DECISIONS/0009-the-lab.md
- 02-DECISIONS/0009-the-lab.md
- 02-DECISIONS/0009-the-lab.md
- 02-DECISIONS/0016-the-lab.md
- 02-DECISIONS/0016-the-lab.md
- 02-DECISIONS/0016-the-lab.md
---
# The scenario declaration
@@ -15,7 +15,7 @@ A scenario is a **declaration of an underlay**, plus what to put on it. It is th
everything in the lab hangs off, so it is worth getting small.
It states what a hosting provider and a home router would provide, and nothing the mesh is
responsible for ([ADR 0009](../../02-DECISIONS/0009-the-lab.md)).
responsible for ([ADR 0016](../../02-DECISIONS/0016-the-lab.md)).
## Public networks are unrelated, and routed rather than bridged
@@ -87,7 +87,7 @@ Four consequences follow, and every one of them shapes this design:
address stop corresponding.
This is why the mesh dials outward and never inward
([ADR 0001](../../02-DECISIONS/0001-nodes-communicate-over-a-broker.md)), why a hub exists at
([ADR 0002](../../02-DECISIONS/0002-nodes-communicate-over-a-broker.md)), why a hub exists at
all, and why a node's endpoint is something a peer **learns** from arriving packets rather than
something anyone configures.
@@ -258,7 +258,7 @@ otherwise explicit declaration, and it exists because NAT has to run somewhere.
It is a **container, not a virtual machine** — a router is scenery rather than something under
test, so the fidelity argument that makes a node a virtual machine does not reach it
([ADR 0009](../../02-DECISIONS/0009-the-lab.md)). What a router must
([ADR 0016](../../02-DECISIONS/0016-the-lab.md)). What a router must
reproduce is kernel behaviour, and a container has the same kernel.
**`machines[].at`** — segment and addresses, or a **list** of them for a machine on several
@@ -299,7 +299,7 @@ belongs to a router it does not control, and asleep.
Whether the overlay survives that, re-forms, and is noticed to have changed endpoint is
**observed**, never arranged
([ADR 0009](../../02-DECISIONS/0009-the-lab.md)).
([ADR 0016](../../02-DECISIONS/0016-the-lab.md)).
## Why the addresses are load-bearing
@@ -320,13 +320,13 @@ it must be.
The format should make getting this wrong hard rather than merely documented: a segment without
a `gateway:` is a public segment, and an address in it — including a gateway's `address:` — that
is not documentation space is a declaration error, refused before anything is raised. That is
[ADR 0023](../../02-DECISIONS/0023-delivery.md) applied to a configuration
[ADR 0010](../../02-DECISIONS/0010-delivery.md) applied to a configuration
file: the failure it prevents is silent, so the check has to be loud.
## The same declaration serves both classes
The bootstrap and full scenarios differ **only in `place:`**
([ADR 0009](../../02-DECISIONS/0009-the-lab.md)). Everything
([ADR 0016](../../02-DECISIONS/0016-the-lab.md)). Everything
about the underlay is identical, which is what makes one a strict subset of the other rather
than a fork.
@@ -354,7 +354,7 @@ not first.
## What a scenario deliberately cannot say
- **Overlay addresses, the hub, peer configuration.** Outcomes, not inputs
([ADR 0009](../../02-DECISIONS/0009-the-lab.md)).
([ADR 0016](../../02-DECISIONS/0016-the-lab.md)).
- **What a machine is in mesh terms** — server or workstation, its site, its names. Mesh
configuration, established by the mesh.
- **A host's capability profile.** Detected, never declared.
@@ -543,7 +543,7 @@ cannot yet express.
Nothing here mentions overlay addresses, which node is the hub, who peers with whom, any name,
or any certificate. Research 004 recorded all of those for this topology, and **a scenario must
not state them** ([ADR 0009](../../02-DECISIONS/0009-the-lab.md)): they
not state them** ([ADR 0016](../../02-DECISIONS/0016-the-lab.md)): they
are what the mesh does, and a scenario that supplied them would be certifying its own work.
The absence is the point. Given the declaration above, whether a hub is elected, whether the
03-DESIGN/01-to-be/03-scenario-lifecycle.md
+7 -7
View File
@@ -4,16 +4,16 @@ status: in-progress
code: [mesh-lab]
updated: 2026-08-25
decisions:
- 02-DECISIONS/0009-the-lab.md
- 02-DECISIONS/0009-the-lab.md
- 02-DECISIONS/0009-the-lab.md
- 02-DECISIONS/0016-the-lab.md
- 02-DECISIONS/0016-the-lab.md
- 02-DECISIONS/0016-the-lab.md
---
# Scenario lifecycle
The first thing the lab must do, and the only thing it must do before anything else can be
written: **materialise a mesh, return it to a known state, and destroy it**
([ADR 0009](../../02-DECISIONS/0009-the-lab.md)).
([ADR 0016](../../02-DECISIONS/0016-the-lab.md)).
A [declaration](02-scenario-declaration.md) describes a scenario. This describes what happens
to one.
@@ -39,7 +39,7 @@ The order is not arbitrary — each step needs the one before it to exist:
1. **Segments.** Isolated links, one per declared segment, belonging to this instance and
joined to nothing outside it
([ADR 0009](../../02-DECISIONS/0009-the-lab.md)).
([ADR 0016](../../02-DECISIONS/0016-the-lab.md)).
2. **Gateways.** Derived, never declared as machines: a gateway is materialised for each
distinct `gateway:` declaration, sitting on both its segment and its parent, carrying the
translation, forwarding and mapping-expiry the declaration asked for.
@@ -60,7 +60,7 @@ habit.
## A failed raise leaves the wreckage
A step that fails stops the raise
([ADR 0023](../../02-DECISIONS/0023-delivery.md)) — and **does not tear
([ADR 0010](../../02-DECISIONS/0010-delivery.md)) — and **does not tear
down**.
Tearing down on failure destroys the only evidence of what went wrong, which is precisely
@@ -100,7 +100,7 @@ made after it, and returning undoes it like any other change.
## Reaching in
Everything the lab does to a machine goes through the virtualisation layer, never over IP
([ADR 0009](../../02-DECISIONS/0009-the-lab.md)). `exec` runs a
([ADR 0016](../../02-DECISIONS/0016-the-lab.md)). `exec` runs a
command on a machine and returns its output.
This has one consequence worth stating plainly: **a reachability question is asked from inside**.
03-DESIGN/01-to-be/04-lab-installation.md
+4 -4
View File
@@ -4,15 +4,15 @@ status: designed
code: [mesh-lab]
updated: 2026-08-24
decisions:
- 02-DECISIONS/0009-the-lab.md
- 02-DECISIONS/0023-delivery.md
- 02-DECISIONS/0016-the-lab.md
- 02-DECISIONS/0010-delivery.md
---
# Installing the lab on a clean machine
The lab has prerequisites — a virtualisation daemon, copy-on-write storage, a pool, an identity
permitted to talk to it — and it cannot get them from the mesh, because it is where the mesh is
built ([ADR 0009](../../02-DECISIONS/0009-the-lab.md)).
built ([ADR 0016](../../02-DECISIONS/0016-the-lab.md)).
So the lab needs an install path of its own. This describes it, and the shape it has to take is
determined by two failures observed while measuring
@@ -55,7 +55,7 @@ and unbounded at worst.
**The lab refuses to run degraded.** It does not warn and continue: a warning about a slow inner
loop is read once and ignored forever, and the loop stays slow. This is
[ADR 0023](../../02-DECISIONS/0023-delivery.md) applied where the failure is
[ADR 0010](../../02-DECISIONS/0010-delivery.md) applied where the failure is
performance rather than an error.
## Two ways the prerequisites arrive
03-DESIGN/01-to-be/05-the-node-host.md
+27 -27
View File
@@ -4,17 +4,17 @@ status: in-progress
code: [mesh-host]
updated: 2026-08-27
decisions:
- 02-DECISIONS/0011-how-this-repository-works.md
- 02-DECISIONS/0015-a-node-and-how-it-joins.md
- 02-DECISIONS/0016-the-node-host.md
- 02-DECISIONS/0015-a-node-and-how-it-joins.md
- 02-DECISIONS/0015-a-node-and-how-it-joins.md
- 02-DECISIONS/0023-delivery.md
- 02-DECISIONS/0016-the-node-host.md
- 02-DECISIONS/0016-the-node-host.md
- 02-DECISIONS/0016-the-node-host.md
- 02-DECISIONS/0016-the-node-host.md
- 02-DECISIONS/0016-the-node-host.md
- 02-DECISIONS/0019-how-this-repository-works.md
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
- 02-DECISIONS/0005-the-node-host.md
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
- 02-DECISIONS/0010-delivery.md
- 02-DECISIONS/0005-the-node-host.md
- 02-DECISIONS/0005-the-node-host.md
- 02-DECISIONS/0005-the-node-host.md
- 02-DECISIONS/0005-the-node-host.md
- 02-DECISIONS/0005-the-node-host.md
---
# The node host
@@ -25,11 +25,11 @@ Tier 0. The one thing ever installed by hand, and the only thing that changes a
A **statically linked binary that requires nothing to be present** — copy it onto a machine and
run it, and that is the whole installation
([ADR 0016](../../02-DECISIONS/0016-the-node-host.md)). Written in Go, because the
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)). Written in Go, because the
job is system-level and because the host shares no code with any other tier.
A single binary with one job: **apply declared state on this machine**
([ADR 0016](../../02-DECISIONS/0016-the-node-host.md)). Overlay
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)). Overlay
membership, packet filtering, packages, services, containers and filesystems are not six
concerns it carries; they are six instances of the one.
@@ -61,7 +61,7 @@ returns it.
Three properties, each following a recorded decision:
**A failed step fails the apply.** Not "logs and continues"
([ADR 0023](../../02-DECISIONS/0023-delivery.md)). A partial apply that
([ADR 0010](../../02-DECISIONS/0010-delivery.md)). A partial apply that
reports success is the mesh's most expensive shape.
**Each applier reads back.** Setting a value is not evidence the value took. The firewall is
@@ -69,7 +69,7 @@ asked whether the rule loaded; conntrack is asked what timeout it holds. This is
§5 as a component requirement rather than a review habit.
**What was applied is recorded after it works, never before**
([ADR 0014](../../02-DECISIONS/0014-a-picture-is-read-from-what-runs.md)). A failed apply leaves
([ADR 0018](../../02-DECISIONS/0018-a-picture-is-read-from-what-runs.md)). A failed apply leaves
the machine in whatever state it reached, and nothing must claim otherwise.
### store
@@ -78,17 +78,17 @@ Local, and **authoritative while disconnected**. Not a cache of the control plan
of what this node has applied and what it currently holds.
This is structural rather than convenient: if disconnection is an ordinary situation rather
than an exception ([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)), the
than an exception ([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)), the
store is what makes it ordinary. A laptop shut for a week comes back and reconciles; it does
not come back and ask what it is.
### link
The node's one connection to the control plane, and its security boundary
([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)).
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)).
It is the broker connection that already exists
([ADR 0001](../../02-DECISIONS/0001-nodes-communicate-over-a-broker.md)) — outbound,
([ADR 0002](../../02-DECISIONS/0002-nodes-communicate-over-a-broker.md)) — outbound,
node-initiated, per-node addressed — carrying **per-node identity instead of a shared
credential**. The node owns no password. It owns an identity, and that identity is what it
presents.
@@ -108,7 +108,7 @@ architecture, a network position.
capability is real when it is present, running and working, and the difference is the whole
point of detecting it.
The profile is what makes [ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)
The profile is what makes [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)
work: a node is a node, and what varies between them is here rather than in the definition.
### inventory
@@ -133,7 +133,7 @@ is the component; that one is what happens to it.
## Where a declaration comes from
One behaviour, two sources
([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)):
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)):
| Situation | Source |
|---|---|
@@ -150,7 +150,7 @@ the mesh, and the full peer set arrives derived.
## What a declaration is
Settled by [ADR 0016](../../02-DECISIONS/0016-the-node-host.md).
Settled by [ADR 0005](../../02-DECISIONS/0005-the-node-host.md).
**JSON**, because the host has no dependencies to spend and the standard library carries no
YAML. **An ordered list of typed resources**, each with a stable identity — the order is stated
@@ -187,8 +187,8 @@ Raising the substrate needs six shapes in the host's vocabulary, and **all six a
| `directory`, `file` | **built** | no machine dependency at all |
| `service` | **built** | running/stopped **and** enabled/disabled at boot — a unit started but not enabled stops being true at the next reboot |
| `package` | **built** | present, never upgraded, and **never uninstalled** — the host cannot know what else needs it, so dropping one is *forgotten*, not *removed* |
| `container` | **built** | pinned by digest ([ADR 0021](../../02-DECISIONS/0021-the-substrate-and-the-control-plane.md)); identified by a label carrying a digest of the declaration that made it, because a runtime normalises what it is given and that is indistinguishable from drift |
| `action` | **built** | bundle-only ([ADR 0016](../../02-DECISIONS/0016-the-node-host.md)); verify is mandatory and is the idempotency check as well as the read-back |
| `container` | **built** | pinned by digest ([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)); identified by a label carrying a digest of the declaration that made it, because a runtime normalises what it is given and that is indistinguishable from drift |
| `action` | **built** | bundle-only ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)); verify is mandatory and is the idempotency check as well as the read-back |
**The parser enforces the boundary rather than the caller remembering it.** `Parse` refuses an
action and is what the link uses; `ParseTrusted` permits one and is what the bundle uses. The
@@ -205,7 +205,7 @@ until it is done the substrate bootstrap has no end-to-end test.
**3 — link and store.** The node connects, receives declarations, and holds what it applied.
**4 — enrolment.** The one genuinely new mechanism in
[ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md); everything else there
[ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md); everything else there
is configuration of what already runs.
Stage 1 is deliberately the smallest useful thing. The lab currently raises **empty machines**
@@ -216,7 +216,7 @@ that, and every later stage is tested by a lab that already works.
**The lab is the harness.** A scenario places a host on a machine and asserts what it did —
against a real hypervisor, with the boundary never mocked
([ADR 0013](../../02-DECISIONS/0013-a-test-defends-a-decision.md)).
([ADR 0017](../../02-DECISIONS/0017-a-test-defends-a-decision.md)).
Each decision above owes a test:
@@ -239,7 +239,7 @@ Each decision above owes a test:
package, a unit, a container and a dataset *are*. Nothing has measured that surface, and it is
the residue of the question [`host-size.md`](../../01-RESEARCH/006-mesh-from-scratch/host-size.md)
answered.
- **Rescue.** [ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md) suggests it is
- **Rescue.** [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md) suggests it is
a node whose local state is discarded so the mesh re-derives it, and does not decide it.
- **What may expire.** An identity needing refresh to stay valid would make a laptop fail for
being a laptop ([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)).
being a laptop ([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)).
03-DESIGN/01-to-be/06-the-control-plane.md
+25 -25
View File
@@ -4,11 +4,11 @@ status: designed
code: []
updated: 2026-08-27
decisions:
- 02-DECISIONS/0016-the-node-host.md
- 02-DECISIONS/0021-the-substrate-and-the-control-plane.md
- 02-DECISIONS/0011-how-this-repository-works.md
- 02-DECISIONS/0008-mesh-brokers-nodes-host-agents-think.md
- 02-DECISIONS/0021-the-substrate-and-the-control-plane.md
- 02-DECISIONS/0005-the-node-host.md
- 02-DECISIONS/0006-the-substrate-and-the-control-plane.md
- 02-DECISIONS/0019-how-this-repository-works.md
- 02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md
- 02-DECISIONS/0006-the-substrate-and-the-control-plane.md
---
# The control plane
@@ -24,7 +24,7 @@ This document defines it. It does **not** design the contexts inside it; those a
> **The control plane is everything that needs to know about more than one node.**
That is the whole test, and it is not arbitrary — it follows from
[ADR 0016](../../02-DECISIONS/0016-the-node-host.md). The host applies and
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md). The host applies and
does not decide *because deciding needs knowledge the machine does not have*. So the line falls
exactly there:
@@ -44,7 +44,7 @@ catch it because the dependency direction is still correct.
## What is inside it
**Seven contexts and one interface**
([ADR 0021](../../02-DECISIONS/0021-the-substrate-and-the-control-plane.md)) —
([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)) —
each one earning its place by the test above rather than by being ours:
| | | needs to know about more than one node because |
@@ -64,7 +64,7 @@ anything else does. A task does not need to know a node exists, and *being ours
something infrastructure*. `ai` is folded into `config`: a provider licence is an ordinary grant.
**`record` is an open question rather than an eighth entry.** Contexts integrate through it
([ADR 0020](../../02-DECISIONS/0020-a-context-owns-its-store.md)), which makes it load-bearing,
([ADR 0008](../../02-DECISIONS/0008-a-context-owns-its-store.md)), which makes it load-bearing,
and [research 006](../../01-RESEARCH/006-mesh-from-scratch/skeleton.md) leaves *where it lives*
unresolved — putting it in the substrate risks recreating the circularity the tier design just
removed. Listing it here would settle by naming what has not been settled by arguing.
@@ -88,10 +88,10 @@ because each one alone reads like a detail:
| | |
|---|---|
| [ADR 0016](../../02-DECISIONS/0016-the-node-host.md) | the host never queries the mesh database |
| [ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md) | a node holds its own identity **and nothing else** — the shared database credential every node carries today is the exposure this exists to remove |
| [ADR 0020](../../02-DECISIONS/0020-a-context-owns-its-store.md) | a context is granted only what it **exclusively** owns: no shared writes, no read-only roles |
| [ADR 0021](../../02-DECISIONS/0021-the-substrate-and-the-control-plane.md) | there is no single mesh database, and nothing reads one |
| [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) | the host never queries the mesh database |
| [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md) | a node holds its own identity **and nothing else** — the shared database credential every node carries today is the exposure this exists to remove |
| [ADR 0008](../../02-DECISIONS/0008-a-context-owns-its-store.md) | a context is granted only what it **exclusively** owns: no shared writes, no read-only roles |
| [ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md) | there is no single mesh database, and nothing reads one |
### So how does anything get in
@@ -123,17 +123,17 @@ node ──► broker ──► the control plane, consuming
Seven contexts, **one deployable** — they are not separate services, so this is one process
consuming and dispatching internally, not seven consumers racing. Each context then writes only
the store it exclusively owns
([ADR 0020](../../02-DECISIONS/0020-a-context-owns-its-store.md)).
([ADR 0008](../../02-DECISIONS/0008-a-context-owns-its-store.md)).
**One consumer is a property worth having**, not just a consequence of
[ADR 0021](../../02-DECISIONS/0021-the-substrate-and-the-control-plane.md). The as-is records that
[ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md). The as-is records that
*two consumers accidentally sharing one queue silently split the traffic between them, each
receiving half of what it expects* — which has happened, between a module's daemon and its
capability server. With one consumer that class of fault cannot arise.
**And the broker is the buffer while the control plane is down.** Nodes go on publishing;
messages queue; the control plane drains them when it returns. That is what makes
[ADR 0021](../../02-DECISIONS/0021-the-substrate-and-the-control-plane.md)'s single control plane
[ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)'s single control plane
tolerable — an outage delays the mesh's *knowledge* rather than losing it.
**With one consequence that must be bounded before it is discovered:** a queue with no limit
@@ -149,12 +149,12 @@ before designing for throughput.** The registry is `inventory`'s store: nodes, m
assignments, versions. Those change when somebody changes something.
**Logs, metrics and health checks belong to `observability`**, which owns a different store
([ADR 0020](../../02-DECISIONS/0020-a-context-owns-its-store.md)). Sending them to the registry
([ADR 0008](../../02-DECISIONS/0008-a-context-owns-its-store.md)). Sending them to the registry
would be exactly the shared-schema mistake 0045 exists to stop, arriving through the back door
marked *performance*.
That leaves one genuine funnel: every context's writes go through the process that owns it, and
[ADR 0021](../../02-DECISIONS/0021-the-substrate-and-the-control-plane.md) says there is one of it.
[ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md) says there is one of it.
For a mesh of a handful of machines this is not a scaling problem, and **it should not be solved
by giving nodes database credentials** — that trades a bounded problem for an unbounded one. If
it ever binds, the answers are at the consumer: batch, apply backpressure, or move the highest
@@ -170,10 +170,10 @@ volume genuinely argues against a relational store.
- **Not a surface.** Tier 3 is how people and agents reach it. It has one interface; the
surfaces are what speak to that interface.
- **Not the substrate.** It *runs on* tier 1 — PostgreSQL, LavinMQ, MinIO, an OCI registry
([ADR 0021](../../02-DECISIONS/0021-the-substrate-and-the-control-plane.md)) — and cannot start without
([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)) — and cannot start without
them, which is what makes them a lower tier.
- **Not privileged on a node.** It has no more access to a machine than the declaration
vocabulary allows ([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)).
vocabulary allows ([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)).
## It is also a consumer
@@ -184,7 +184,7 @@ module needs, granted the same way.
That is the circularity the tiers exist to resolve rather than hide: the control plane cannot
provision its own database, because it is not running yet. So its **store** is raised from the
bundle the host carries, before there is a control plane to ask
([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md),
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md),
[research 011](../../01-RESEARCH/011-the-module-graph/worked-provider.md)).
Its virtual host and its bucket are **not** in the bundle — by the time they are wanted there is
@@ -198,15 +198,15 @@ it.
hosts, assigned to nodes by the same mechanism as everything else.
**One node runs it, and nothing takes over**
([ADR 0021](../../02-DECISIONS/0021-the-substrate-and-the-control-plane.md)). The node is assigned,
([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)). The node is assigned,
never elected — no promotion, no quorum, no split brain.
That is sound rather than merely cheap, because the design already tolerates the control plane
being absent by construction: a node reconciles from **its own** store
([ADR 0016](../../02-DECISIONS/0016-the-node-host.md)) and
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)) and
never needed to ask anybody to hold the state it was last given. So the control plane being down
is not a new failure mode — it is
[ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)'s ordinary disconnected
[ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)'s ordinary disconnected
situation, happening to every node at once. **What is lost is change, not operation.**
The honest half: this node is a single point of failure, recovery is restore rather than
@@ -216,14 +216,14 @@ every public name.
## Open
- ~~**The contexts themselves.**~~ **Decided** — seven, by
[ADR 0021](../../02-DECISIONS/0021-the-substrate-and-the-control-plane.md).
[ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md).
What remains open is narrower and named there: **where the record lives**, which research 006
leaves unresolved because the substrate is the one place it must not go.
- **How far it may be split.** One deployable today. Splitting a context out costs the single
interface a surface depends on
([research 011](../../01-RESEARCH/011-the-module-graph/00-overview.md)).
- ~~**How many run, and what a node does without one.**~~ **Resolved** by
[ADR 0021](../../02-DECISIONS/0021-the-substrate-and-the-control-plane.md). What remains is
[ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md). What remains is
measurement: nothing reports how long the control plane has been unreachable, or how close a
certificate is to expiry — both needed for restore-not-failover to be a plan rather than a
hope.
03-DESIGN/01-to-be/07-the-substrate.md
+19 -19
View File
@@ -4,14 +4,14 @@ status: designed
code: []
updated: 2026-08-27
decisions:
- 02-DECISIONS/0011-how-this-repository-works.md
- 02-DECISIONS/0016-the-node-host.md
- 02-DECISIONS/0015-a-node-and-how-it-joins.md
- 02-DECISIONS/0021-the-substrate-and-the-control-plane.md
- 02-DECISIONS/0016-the-node-host.md
- 02-DECISIONS/0021-the-substrate-and-the-control-plane.md
- 02-DECISIONS/0022-connectivity.md
- 02-DECISIONS/0016-the-node-host.md
- 02-DECISIONS/0019-how-this-repository-works.md
- 02-DECISIONS/0005-the-node-host.md
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
- 02-DECISIONS/0006-the-substrate-and-the-control-plane.md
- 02-DECISIONS/0005-the-node-host.md
- 02-DECISIONS/0006-the-substrate-and-the-control-plane.md
- 02-DECISIONS/0007-connectivity.md
- 02-DECISIONS/0005-the-node-host.md
---
# The substrate
@@ -27,22 +27,22 @@ Every module that needs a database asks the control plane's provisioning for one
plane needs a database too — and it cannot ask itself, because it is not running yet. That
circularity is not an awkwardness to work around; it *is* the definition. Anything on the wrong
side of it must be raised some other way, and the other way is the bundle the host carries
([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)).
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)).
The test, applied:
| | control plane needs it | can it grant itself one? | |
|---|---|---|---|
| a relational store — **PostgreSQL** | its own state lives there | no — provisioning needs the store | **substrate** |
| a message bus — **LavinMQ** | it reaches nodes over it ([ADR 0001](../../02-DECISIONS/0001-nodes-communicate-over-a-broker.md)) | no — it cannot grant itself a virtual host | **substrate** |
| a message bus — **LavinMQ** | it reaches nodes over it ([ADR 0002](../../02-DECISIONS/0002-nodes-communicate-over-a-broker.md)) | no — it cannot grant itself a virtual host | **substrate** |
| an object store — **MinIO** | artifacts and blobs it delivers | no — it needs a bucket to hold them | **substrate** |
| an image registry — **the OCI registry** | images it delivers to nodes | no — it needs a repository | **substrate** |
| an identity provider | only if it delegates authentication | — | **conditional, below** |
| ingress — **Traefik** | not to start; only to be reached by name | — it grants itself one afterwards | **not substrate** ([ADR 0022](../../02-DECISIONS/0022-connectivity.md)) |
| ingress — **Traefik** | not to start; only to be reached by name | — it grants itself one afterwards | **not substrate** ([ADR 0007](../../02-DECISIONS/0007-connectivity.md)) |
| anything else the mesh hosts | no | — | not substrate |
**The role and the product are both written**, here and everywhere
([ADR 0021](../../02-DECISIONS/0021-the-substrate-and-the-control-plane.md)). The role is what the argument
([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)). The role is what the argument
turns on — the test above works on roles, and would give the same answers for a different store.
The product is what actually gets installed and pinned, and a design that names only the role
does not record that the choice was ever made.
@@ -96,19 +96,19 @@ Being substrate and being in the bundle are two different questions:
| PostgreSQL | yes — the control plane's own state lives in it | **yes** — there is nowhere to put that state otherwise |
| LavinMQ | yes — it cannot grant itself a virtual host | **not established** — see below |
| MinIO | yes — it cannot grant itself a bucket | no — nothing is delivered before the mesh exists |
| the OCI registry | yes — it cannot grant itself a repository | no — the first node fetches upstream ([ADR 0021](../../02-DECISIONS/0021-the-substrate-and-the-control-plane.md)) |
| the OCI registry | yes — it cannot grant itself a repository | no — the first node fetches upstream ([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)) |
The three on the bottom rows are **substrate by role and ordinary by delivery**: by the time
they are wanted there is a control plane, and it provisions them the way it provisions anything.
That keeps the bundle to roughly one image rather than four, which is what makes it small enough
for the review [ADR 0016](../../02-DECISIONS/0016-the-node-host.md)
for the review [ADR 0005](../../02-DECISIONS/0005-the-node-host.md)
requires.
**Why pinned:** the bundle is applied when no mesh exists, so nothing can resolve a version, ask
a registry, or check a constraint. What the host carries must already be exact.
**Why references and not payload:** the bundle names images by **digest** and the host fetches
them ([ADR 0021](../../02-DECISIONS/0021-the-substrate-and-the-control-plane.md)). A first node is
them ([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)). A first node is
a real machine with a network; the sealed case is the lab, and the lab places images itself.
Reproducibility comes from pinning the identity of a thing rather than carrying its bytes, which
@@ -136,7 +136,7 @@ container, so a container runtime must be working before anything else happens
is a *package*, not a container.
**Which runtime is detected, not chosen**
([ADR 0016](../../02-DECISIONS/0016-the-node-host.md)): a machine that
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)): a machine that
already has one keeps it. On a machine with none, the control plane names the package, because
what it is called differs per system. It is:
@@ -145,7 +145,7 @@ what it is called differs per system. It is:
- **adopted rather than installed** when the machine already has one with configuration somebody
chose ([research 012](../../01-RESEARCH/012-the-minimum-viable-node/00-overview.md));
- a package, which needs the machine's own package manager and a network — both permitted by
[ADR 0021](../../02-DECISIONS/0021-the-substrate-and-the-control-plane.md).
[ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md).
So the host's bootstrap vocabulary is six shapes: **package**, **container**, **file**,
**directory**, **service**, and **action**. **All six are built**
@@ -155,7 +155,7 @@ blocked on the host any longer.
**Steps 2 and 3 happen before there is a mesh to do them**, which is why provisioning is part of
the bootstrap rather than a service consumers use later. They are **actions** the bundle
declares and the host runs
([ADR 0016](../../02-DECISIONS/0016-the-node-host.md)) — so the
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)) — so the
host's vocabulary grows by one shape rather than by one resource type per substrate service.
## Open
@@ -170,7 +170,7 @@ host's vocabulary grows by one shape rather than by one resource type per substr
question about the control plane's internal shape, not about the substrate**, which is why it is
not answered here.
- ~~**Whether the host can do step 2.**~~ **Resolved** by
[ADR 0016](../../02-DECISIONS/0016-the-node-host.md). A service
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md). A service
running on this machine is part of this machine, so the scope was never in question — the real
question was whether the host must learn what a database is, and it must not. The bundle
declares an **action**; the host runs it and verifies it, and what a database means stays with
03-DESIGN/01-to-be/08-connectivity.md
+24 -24
View File
@@ -4,13 +4,13 @@ status: designed
code: []
updated: 2026-08-27
decisions:
- 02-DECISIONS/0016-the-node-host.md
- 02-DECISIONS/0015-a-node-and-how-it-joins.md
- 02-DECISIONS/0022-connectivity.md
- 02-DECISIONS/0022-connectivity.md
- 02-DECISIONS/0015-a-node-and-how-it-joins.md
- 02-DECISIONS/0022-connectivity.md
- 02-DECISIONS/0021-the-substrate-and-the-control-plane.md
- 02-DECISIONS/0005-the-node-host.md
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
- 02-DECISIONS/0007-connectivity.md
- 02-DECISIONS/0007-connectivity.md
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
- 02-DECISIONS/0007-connectivity.md
- 02-DECISIONS/0006-the-substrate-and-the-control-plane.md
---
# Connectivity
@@ -31,7 +31,7 @@ node* — to each responsibility:
|---|---|---|
| **overlay** — who peers with whom, at what address | **every node**, and which of them can be dialled | control plane |
| **resolution** — which name is which node | **every node** | control plane |
| **exposure** — which public name reaches which container | **which node is publicly reachable** ([ADR 0022](../../02-DECISIONS/0022-connectivity.md)) | control plane |
| **exposure** — which public name reaches which container | **which node is publicly reachable** ([ADR 0007](../../02-DECISIONS/0007-connectivity.md)) | control plane |
| **filtering** — which port is open, to whom | what is assigned here, and the overlay's shape | control plane decides, host applies |
| **certificates** — who may present which name | which name belongs to which node | control plane |
@@ -53,7 +53,7 @@ It is also what removes the last two upward dependencies.
[Research 006](../../01-RESEARCH/006-mesh-from-scratch/host-size.md) counted exactly two modules
opening a direct connection to the control plane's database — `wireguard` and `traefik` — and
they are the reason every node permanently holds a credential to it
([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)). Both are connectivity
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)). Both are connectivity
modules. **Closing this context closes that set.**
## The order it comes up in
@@ -63,7 +63,7 @@ The one thing to get right, because everything else depends on it:
```
0 the node has an underlay address the machine's own — DHCP, or a provider gave it one
1 the node dials the mesh OVER THE UNDERLAY, at the address in its token
2 it proves itself, and is proved to the link exists (ADR 0015, ADR 0015)
2 it proves itself, and is proved to the link exists (ADR 0004, ADR 0004)
3 the mesh grants it an identity and an overlay address
4 the overlay comes up peer graph delivered as files
5 names resolve resolver config delivered as files
@@ -77,7 +77,7 @@ never be established on a new node. The link stays on the underlay permanently
outbound-only and carries its own identity, so it needs nothing the overlay provides.
**Nothing before step 3 can resolve a mesh name**, which is why the token carries an *address*
([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)). Today this is
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)). Today this is
patched with an `/etc/hosts` floor written underneath the resolver; under this design there is
nothing to patch.
@@ -89,7 +89,7 @@ which of those it may dial, and which must dial it.
**Inputs, all declared:**
- **reachability** — an endpoint, or none
([ADR 0022](../../02-DECISIONS/0022-connectivity.md)). Not
([ADR 0007](../../02-DECISIONS/0007-connectivity.md)). Not
inferred from the address shape, which is wrong for carrier-grade NAT, wrong for IPv6, and
wrong for a routable address behind a closed firewall.
- **site** — where the machine physically is, or nothing if it roams.
@@ -98,7 +98,7 @@ which of those it may dial, and which must dial it.
**Keys.** Each node generates its own keypair. **The private key never leaves the machine**; the
public key is published to the mesh. This is already true and it is already right — it is
[ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)'s *a node holds its own
[ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)'s *a node holds its own
identity* applied to the overlay, and it means the control plane computes a graph it cannot
itself impersonate.
@@ -141,17 +141,17 @@ expensively enough to be worth restating:
name and overlay address.
**What goes away:** the `/etc/hosts` floor. It exists because a node had to reach the mesh
database before its own DNS existed; with [ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)
database before its own DNS existed; with [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)
nothing needs a name before the link, and a fallback nothing needs is a path nothing tests.
## 3 — Exposure
Settled by [ADR 0022](../../02-DECISIONS/0022-connectivity.md); summarised here because
Settled by [ADR 0007](../../02-DECISIONS/0007-connectivity.md); summarised here because
this is where it belongs.
**A route is a grant.** A module that must be reachable declares it needs one; the proxy provides
it and hands back the public name. Ordinary
[ADR 0019](../../02-DECISIONS/0019-modules-and-the-graph.md)
[ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md)
vocabulary — the mirror of a database grant, where the consumer supplies a target and receives a
name rather than supplying nothing and receiving credentials.
@@ -164,14 +164,14 @@ the case is a mesh-level fact, which is the fourth reason exposure is control-pl
consequence of what runs on it and who must reach it, not an independent declaration to keep in
step by hand.
**A rule names its source** ([ADR 0022](../../02-DECISIONS/0022-connectivity.md)).
**A rule names its source** ([ADR 0007](../../02-DECISIONS/0007-connectivity.md)).
A rule with no source is open, and must say so rather than appear to restrict something. `scope:`
is removed rather than implemented: five manifests carry it today, it is referenced by no code,
and it is the clearest instance in the repository of *an unenforced rule is indistinguishable
from a wrong one, and costs more, because people believe it.*
**Unknown keys are refused** — the discipline the host's declaration parser already has
([ADR 0016](../../02-DECISIONS/0016-the-node-host.md)), and
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)), and
the one manifests lack. `scope:` survived because nothing rejected it.
## 5 — Certificates
@@ -197,7 +197,7 @@ worse than the lab problem that found it — every certificate experiment on a r
production issuance quota, and a retry loop can exhaust it for a week.
**The mesh CA is not a bootstrap concern.** A joining node verifies the control plane against the
fingerprint in its token ([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)),
fingerprint in its token ([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)),
so nothing needs the CA before membership. It certifies internal names afterwards, and that is
all it does.
@@ -208,7 +208,7 @@ The list is worth having in one place, because it is most of the argument:
- **The last two direct database connections from nodes** — `wireguard` and `traefik`, the only
two, both connectivity.
- **Therefore the database credential on every node**, and the object-store credential beside it.
[ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)'s central claim becomes
[ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)'s central claim becomes
true rather than aspirational.
- **The `/etc/hosts` floor**, and the bootstrap circularity it patched.
- **Hub election by address prefix**, and the silent no-hub failure when nobody knew the
@@ -219,7 +219,7 @@ The list is worth having in one place, because it is most of the argument:
## Open
- ~~**What happens when the hub is down.**~~ **Resolved** by
[ADR 0021](../../02-DECISIONS/0021-the-substrate-and-the-control-plane.md), together with `06`'s
[ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md), together with `06`'s
matching question — they were one question. Nothing takes over. WireGuard has no failover, the
hub is declared rather than elected, and non-co-located paths stop while co-located direct peers
and every already-assigned workload keep running. The recovery path is restore, and its deadline
@@ -227,9 +227,9 @@ The list is worth having in one place, because it is most of the argument:
- **Renumbering the overlay.** Made *possible* by declaring the hub rather than inferring it from
an address, but no procedure exists, and a graph delivered node by node has an ordering problem
while it is half-applied.
- **Revoking a route** when a module is unassigned ([ADR 0022](../../02-DECISIONS/0022-connectivity.md)).
- **Revoking a route** when a module is unassigned ([ADR 0007](../../02-DECISIONS/0007-connectivity.md)).
A stale public name pointing at nothing fails more visibly than a stale grant.
- **IPv6.** [ADR 0022](../../02-DECISIONS/0022-connectivity.md) makes
- **IPv6.** [ADR 0007](../../02-DECISIONS/0007-connectivity.md) makes
it expressible; nothing here says the overlay or the resolver handle it.
- **Reporting declared-versus-observed.** ADR 0022 makes the disagreement detectable and does not
- **Reporting declared-versus-observed.** ADR 0007 makes the disagreement detectable and does not
say who looks or what they are told.
03-DESIGN/01-to-be/09-the-node-lifecycle.md
+44 -44
View File
@@ -4,17 +4,17 @@ status: designed
code: []
updated: 2026-08-27
decisions:
- 02-DECISIONS/0015-a-node-and-how-it-joins.md
- 02-DECISIONS/0016-the-node-host.md
- 02-DECISIONS/0015-a-node-and-how-it-joins.md
- 02-DECISIONS/0015-a-node-and-how-it-joins.md
- 02-DECISIONS/0016-the-node-host.md
- 02-DECISIONS/0015-a-node-and-how-it-joins.md
- 02-DECISIONS/0016-the-node-host.md
- 02-DECISIONS/0023-delivery.md
- 02-DECISIONS/0016-the-node-host.md
- 02-DECISIONS/0016-the-node-host.md
- 02-DECISIONS/0016-the-node-host.md
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
- 02-DECISIONS/0005-the-node-host.md
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
- 02-DECISIONS/0005-the-node-host.md
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
- 02-DECISIONS/0005-the-node-host.md
- 02-DECISIONS/0010-delivery.md
- 02-DECISIONS/0005-the-node-host.md
- 02-DECISIONS/0005-the-node-host.md
- 02-DECISIONS/0005-the-node-host.md
---
# The node lifecycle
@@ -42,11 +42,11 @@ questions that were not being asked live.
**Only `enrolled` and `disconnected` are nodes**, and they are the same node in two situations
rather than two kinds of thing
([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)). **`hosted` is not a
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)). **`hosted` is not a
node** — it is a machine with a program on it that has not been told which mesh it belongs to.
There is no state for *the first node*. That is the point of
[ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md): the first node walks the
[ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md): the first node walks the
same path, in an unusual order.
---
@@ -54,7 +54,7 @@ same path, in an unusual order.
## unmanaged → hosted: installing
In the machine's own idiom, because the package manager and the init file are the system's
([ADR 0016](../../02-DECISIONS/0016-the-node-host.md)):
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)):
```
# Alpine — the intended first node
@@ -67,7 +67,7 @@ systemctl enable --now nox-mesh-host
```
Two lines each, and the init file behind them is four
([ADR 0016](../../02-DECISIONS/0016-the-node-host.md)) — it says
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)) — it says
*run the launcher at boot* and nothing else, so a third system is transcription rather than a
port.
@@ -103,7 +103,7 @@ WantedBy=multi-user.target
```
**Two lines of policy, and that is deliberate**
([ADR 0016](../../02-DECISIONS/0016-the-node-host.md)). The init is
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)). The init is
asked to *start this at boot* and *start it again if it exits*, and nothing else. Both are
expressible in OpenRC, runit, s6 and an Android `init.rc`, so porting this file is transcription
rather than design.
@@ -135,7 +135,7 @@ nox-mesh-host enrol --token <one-time token>
```
The token carries three things and is carried by a person
([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)): the broker's
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)): the broker's
address, the fingerprint to expect, and the right to join once.
What happens, in order:
@@ -153,7 +153,7 @@ a container runtime, an architecture. The profile is not a diagnostic; it is the
**The node computes nothing about the mesh.** It needs one peer to reach; the whole overlay is
derived centrally and pushed down
([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md),
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md),
[`08-connectivity.md`](08-connectivity.md)).
### The first declaration is the overlay, and nothing else
@@ -172,7 +172,7 @@ Three reasons, and the third is the one that matters when something goes wrong:
address and peer set are *assigned* — it generates a keypair, publishes the public half, and
receives the rest ([`08-connectivity.md`](08-connectivity.md)). So the overlay is the first
thing the mesh can give it, and it should be.
- **It is what [ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md) already
- **It is what [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md) already
says:** *a joining node does the minimum to be reachable, and nothing else.*
- **It is the way back in.** Once the overlay is up, the node is reachable over it — by SSH, by
anything. If a later declaration breaks the machine, there is a route to it that does not
@@ -187,10 +187,10 @@ Worth stating plainly, because the two rules read as a contradiction and are not
| | |
|---|---|
| **every node reaches every other node** | over the overlay — SSH, services, ordinary traffic. This is the point of having one |
| **every node consumes from the broker** | its own queue, over its own outbound connection ([ADR 0001](../../02-DECISIONS/0001-nodes-communicate-over-a-broker.md)) |
| **nothing dials a node to control it** | the host has no inbound control surface ([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)) |
| **every node consumes from the broker** | its own queue, over its own outbound connection ([ADR 0002](../../02-DECISIONS/0002-nodes-communicate-over-a-broker.md)) |
| **nothing dials a node to control it** | the host has no inbound control surface ([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)) |
**[ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md) is about the control
**[ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md) is about the control
channel, not about network reachability.** What it forbids is a listening thing that accepts
instructions and changes the machine. A node being reachable on the overlay — the whole purpose
of the overlay — is untouched by it, and so is a person opening a shell on it.
@@ -232,7 +232,7 @@ used months later on node two.
## Two kinds of host
Everything above assumes a machine with an init that runs the host at boot. Not every machine
has one ([ADR 0016](../../02-DECISIONS/0016-the-node-host.md)).
has one ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)).
| | **resident** | **episodic** |
|---|---|---|
@@ -245,7 +245,7 @@ has one ([ADR 0016](../../02-DECISIONS/0016-the-node-host.md)).
| can be the first node | yes | **no** |
**An episodic host being killed is disconnection, not failure.** That is
[ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md) doing the work it was written
[ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md) doing the work it was written
for: reachability is state, not class. Everything the design already does for a laptop that
closes — an authoritative local store, reconcile on start, *last heard from* reported without an
alarm — is what an episodic host needs, at a shorter period.
@@ -258,7 +258,7 @@ empty placeholder waiting to be filled in.
**Two things this changes for anything reading the mesh.** *Last heard from* is a much weaker
signal on an episodic host — a healthy phone looks like a dead server — so a reader has to know
which kind it is looking at. And a declaration may take a long time to land, which makes
[ADR 0023](../../02-DECISIONS/0023-delivery.md)'s separation of
[ADR 0010](../../02-DECISIONS/0010-delivery.md)'s separation of
*outstanding* from *failed* load-bearing rather than tidy.
**Still open:** how an episodic host is started in practice — an APK with a foreground service,
@@ -272,7 +272,7 @@ Adoption is not a state. It is what the **first apply** does when it is told to
machine already has ([research 012](../../01-RESEARCH/012-the-minimum-viable-node/00-overview.md)).
A candidate machine is not empty. It has a package manager, probably a container runtime,
configuration somebody chose. [ADR 0016](../../02-DECISIONS/0016-the-node-host.md)
configuration somebody chose. [ADR 0005](../../02-DECISIONS/0005-the-node-host.md)
says the host never touches what it did not create — adoption is the deliberate act of taking
ownership of exactly that, so it is a companion to that rule rather than an exception:
@@ -298,8 +298,8 @@ outcome **derived** from the worst line rather than stated alongside it.
**Changes are pushed, not polled.** A declaration arrives as a message on the link and the host
applies it then. The link is already open and outbound
([ADR 0001](../../02-DECISIONS/0001-nodes-communicate-over-a-broker.md),
[ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)) — asking it
([ADR 0002](../../02-DECISIONS/0002-nodes-communicate-over-a-broker.md),
[ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)) — asking it
repeatedly whether anything has changed would be slower to land *and* constant traffic to learn
nothing.
@@ -326,11 +326,11 @@ So the two periodic things do different jobs and should not be conflated:
without a heartbeat that is indistinguishable from a node that stopped. With one, *last heard
from* is a fact beside every node — which is what
[how long disconnected](#how-long-disconnected-and-who-is-told) reports and what
[ADR 0016](../../02-DECISIONS/0016-the-node-host.md) exists
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md) exists
because a stuck node cannot send.
**Rebooting mid-apply is safe by construction.** The store records each resource *after* it
worked ([ADR 0014](../../02-DECISIONS/0014-a-picture-is-read-from-what-runs.md)), so a host that
worked ([ADR 0018](../../02-DECISIONS/0018-a-picture-is-read-from-what-runs.md)), so a host that
dies half way through comes back, finds the completed ones already matching, and applies the
rest. The rule that exists to stop the host lying about what it did also makes it crash-safe.
@@ -357,7 +357,7 @@ runtime because a declaration changed would stop every container on the node.
## enrolled ⇄ disconnected
Not a failure. Not degraded. A situation
([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)).
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)).
A disconnected node **keeps reconciling against its own store**, so it goes on holding its
machine in the last state it was told to hold. A laptop shut for a week comes back and
@@ -365,7 +365,7 @@ reconciles; it does not come back and ask what it is.
What it cannot do: receive new declarations, be granted anything new, or have its certificates
renewed — which is the clock on the whole arrangement
([ADR 0021](../../02-DECISIONS/0021-the-substrate-and-the-control-plane.md)).
([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)).
**How long it has been disconnected is a fact the mesh must hold**, and nothing holds it today.
Without it, a node running last month's assignments looks exactly like one that is current.
@@ -384,7 +384,7 @@ nox-mesh-host profile # what can this machine actually do?
`apply FILE` accepts actions, because someone who can write that file and run this binary as
root can already do anything it can. The bound in
[ADR 0016](../../02-DECISIONS/0016-the-node-host.md) is on what a
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md) is on what a
**remote** party may push, not on what a person at the machine may do.
This replaces the three hand-run scripts that exist today — first node, joining, rescue — with
@@ -401,14 +401,14 @@ what it owns by the table above, reports, and drops its identity. The machine ke
installed and is back to `hosted`. Nothing is left behind that anybody has to remember.
**The node is gone.** Stolen, dead, or simply unreachable. The mesh cannot tell it anything, and
by [ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md) it will go on reconciling
by [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md) it will go on reconciling
its last declaration **forever**.
That is the honest consequence of making disconnection ordinary, and the answer is not to make
the host expire. It is that **the node holds nothing that outlives revocation**: its identity is
its own, and every grant it holds is a per-node credential at the provider
([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md),
[ADR 0020](../../02-DECISIONS/0020-a-context-owns-its-store.md)). Revoking is done at the
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md),
[ADR 0008](../../02-DECISIONS/0008-a-context-owns-its-store.md)). Revoking is done at the
database, the broker, the object store — not on the machine.
So a lost node keeps *running* and stops being able to *reach* anything. That is the best
@@ -438,7 +438,7 @@ remains locally authoritative for *operating*; the copy exists only for this.
## Upgrading the host
The host is delivered like anything else
([ADR 0023](../../02-DECISIONS/0023-delivery.md)), and this is worth
([ADR 0010](../../02-DECISIONS/0010-delivery.md)), and this is worth
walking through because tier 0 looks like it should be special and is not.
```
@@ -471,7 +471,7 @@ the test of whether this is really uniform.
3 it finishes the apply and reports never mid-way
4 it exits 0 having finished, not having been stopped
5 the launcher starts it again on the new binary — it supervises the host
rather than exec'ing it (ADR 0016), so this
rather than exec'ing it (ADR 0005), so this
needs nothing from the init
6 the new host reconciles on start trigger 1, confirming the machine still matches
```
@@ -489,7 +489,7 @@ own apply completes. A node must therefore report the version it is **running**,
installed — otherwise the mesh believes an upgrade landed at step 1.
**A version that crashes on start rolls itself back**
([ADR 0016](../../02-DECISIONS/0016-the-node-host.md)).
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)).
What the init starts is not the host but a **launcher**, and the launcher is where the policy
lives:
@@ -551,7 +551,7 @@ credentials still valid — the case
**The host reports what it owns, and the mesh keeps the last report.**
The store stays locally authoritative — a node operates from its own copy and needs nothing to
do so ([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)). What changes is that
do so ([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)). What changes is that
the mesh holds a **copy for recovery**, refreshed on every apply report.
So a node that loses its state file re-enrols, receives both the declaration *and* the record of
@@ -616,12 +616,12 @@ keeps cataloguing.
### Where the enrolment token comes from
**`mesh-control token issue` prints it once**, to the person running it. Single-use, and it
expires whether used or not ([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)).
expires whether used or not ([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)).
It is carried by hand — read off a screen, pasted into a terminal. That is the design rather than
a gap in it: its authenticity comes from the channel it travelled, which is what
lets a node verify a mesh it has never spoken to
([ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md)). A token emailed,
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)). A token emailed,
committed, or dropped in shared storage has lost the only property that makes it worth carrying.
**On the first node it comes from the control plane that was raised two commands ago**, which is
@@ -632,10 +632,10 @@ the same command against a mesh that is one machine old.
## Still open
- ~~**Automatic rollback of a bad host version.**~~ **Resolved** by
[ADR 0016](../../02-DECISIONS/0016-the-node-host.md): a launcher
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md): a launcher
counts failed starts and rolls back — shipped by the package, not the host binary, because a
binary that will not start cannot recover itself. It rolls back once; a second failure means
the machine is the problem, not the binary.
- **How a previous declaration is retained and chosen**, which is what rollback of anything else
would use ([ADR 0023](../../02-DECISIONS/0023-delivery.md)).
would use ([ADR 0010](../../02-DECISIONS/0010-delivery.md)).
- **A node returning after months** applies a very large jump in one go. Correct, and untested.
03-DESIGN/01-to-be/10-delivery.md
+7 -7
View File
@@ -4,13 +4,13 @@ status: designed
code: []
updated: 2026-08-28
decisions:
- 02-DECISIONS/0023-delivery.md
- 02-DECISIONS/0019-modules-and-the-graph.md
- 02-DECISIONS/0019-modules-and-the-graph.md
- 02-DECISIONS/0023-delivery.md
- 02-DECISIONS/0023-delivery.md
- 02-DECISIONS/0019-modules-and-the-graph.md
- 02-DECISIONS/0019-modules-and-the-graph.md
- 02-DECISIONS/0010-delivery.md
- 02-DECISIONS/0009-modules-and-the-graph.md
- 02-DECISIONS/0009-modules-and-the-graph.md
- 02-DECISIONS/0010-delivery.md
- 02-DECISIONS/0010-delivery.md
- 02-DECISIONS/0009-modules-and-the-graph.md
- 02-DECISIONS/0009-modules-and-the-graph.md
---
# Modules and delivery
03-DESIGN/01-to-be/README.md
+14 -14
View File
@@ -9,27 +9,27 @@ document is written and this one's status becomes `implemented`.
| Document | Covers | Rests on |
|---|---|---|
| [`00-work-breakdown.md`](00-work-breakdown.md) | How the decomposition gets built, in what order, and where a human must look | [ADR 0008](../../02-DECISIONS/0008-mesh-brokers-nodes-host-agents-think.md) |
| [`01-end-to-end-testing.md`](01-end-to-end-testing.md) | The lab: a real mesh a change can be run against before it reaches nodes | [ADR 0009](../../02-DECISIONS/0009-the-lab.md), [0029](../../02-DECISIONS/0009-the-lab.md) |
| [`02-scenario-declaration.md`](02-scenario-declaration.md) | What a scenario declares — the underlay, and what to place on it | [ADR 0009](../../02-DECISIONS/0009-the-lab.md) |
| [`03-scenario-lifecycle.md`](03-scenario-lifecycle.md) | What happens to a scenario — raise, snapshot, restore, move, destroy | [ADR 0009](../../02-DECISIONS/0009-the-lab.md) |
| [`04-lab-installation.md`](04-lab-installation.md) | Getting the lab onto a clean machine, and why it verifies capability rather than installation | [ADR 0023](../../02-DECISIONS/0023-delivery.md) |
| [`05-the-node-host.md`](05-the-node-host.md) | Tier 0 — the one thing installed by hand, and the only thing that changes a machine | [ADR 0016](../../02-DECISIONS/0016-the-node-host.md) |
| [`06-the-control-plane.md`](06-the-control-plane.md) | Tier 2 — what the term means, and the test for what belongs in it | [ADR 0016](../../02-DECISIONS/0016-the-node-host.md) |
| [`07-the-substrate.md`](07-the-substrate.md) | Tier 1 — what the control plane consumes and cannot grant itself | [ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md), [0048](../../02-DECISIONS/0021-the-substrate-and-the-control-plane.md) |
| [`08-connectivity.md`](08-connectivity.md) | One context in full — overlay, resolution, exposure, filtering, certificates | [ADR 0022](../../02-DECISIONS/0022-connectivity.md), [0050](../../02-DECISIONS/0022-connectivity.md), [0051](../../02-DECISIONS/0015-a-node-and-how-it-joins.md), [0055](../../02-DECISIONS/0021-the-substrate-and-the-control-plane.md) |
| [`09-the-node-lifecycle.md`](09-the-node-lifecycle.md) | How a machine becomes a node, stays one, and stops being one | [ADR 0015](../../02-DECISIONS/0015-a-node-and-how-it-joins.md), [0051](../../02-DECISIONS/0015-a-node-and-how-it-joins.md) |
| [`10-delivery.md`](10-delivery.md) | Modules, the three edges, and how a change becomes a running thing | [ADR 0023](../../02-DECISIONS/0023-delivery.md), [0064](../../02-DECISIONS/0019-modules-and-the-graph.md), [0065](../../02-DECISIONS/0019-modules-and-the-graph.md) |
| [`00-work-breakdown.md`](00-work-breakdown.md) | How the decomposition gets built, in what order, and where a human must look | [ADR 0001](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md) |
| [`01-end-to-end-testing.md`](01-end-to-end-testing.md) | The lab: a real mesh a change can be run against before it reaches nodes | [ADR 0016](../../02-DECISIONS/0016-the-lab.md), [0029](../../02-DECISIONS/0016-the-lab.md) |
| [`02-scenario-declaration.md`](02-scenario-declaration.md) | What a scenario declares — the underlay, and what to place on it | [ADR 0016](../../02-DECISIONS/0016-the-lab.md) |
| [`03-scenario-lifecycle.md`](03-scenario-lifecycle.md) | What happens to a scenario — raise, snapshot, restore, move, destroy | [ADR 0016](../../02-DECISIONS/0016-the-lab.md) |
| [`04-lab-installation.md`](04-lab-installation.md) | Getting the lab onto a clean machine, and why it verifies capability rather than installation | [ADR 0010](../../02-DECISIONS/0010-delivery.md) |
| [`05-the-node-host.md`](05-the-node-host.md) | Tier 0 — the one thing installed by hand, and the only thing that changes a machine | [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) |
| [`06-the-control-plane.md`](06-the-control-plane.md) | Tier 2 — what the term means, and the test for what belongs in it | [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) |
| [`07-the-substrate.md`](07-the-substrate.md) | Tier 1 — what the control plane consumes and cannot grant itself | [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md), [0048](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md) |
| [`08-connectivity.md`](08-connectivity.md) | One context in full — overlay, resolution, exposure, filtering, certificates | [ADR 0007](../../02-DECISIONS/0007-connectivity.md), [0050](../../02-DECISIONS/0007-connectivity.md), [0051](../../02-DECISIONS/0004-a-node-and-how-it-joins.md), [0055](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md) |
| [`09-the-node-lifecycle.md`](09-the-node-lifecycle.md) | How a machine becomes a node, stays one, and stops being one | [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md), [0051](../../02-DECISIONS/0004-a-node-and-how-it-joins.md) |
| [`10-delivery.md`](10-delivery.md) | Modules, the three edges, and how a change becomes a running thing | [ADR 0010](../../02-DECISIONS/0010-delivery.md), [0064](../../02-DECISIONS/0009-modules-and-the-graph.md), [0065](../../02-DECISIONS/0009-modules-and-the-graph.md) |
## Not yet written
- **The remaining six contexts.**
[ADR 0021](../../02-DECISIONS/0021-the-substrate-and-the-control-plane.md)
[ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)
settles the list at seven; `connectivity` is the first written in full
([`08`](08-connectivity.md)) and the other six do not exist yet. The work breakdown says in
what order they are needed.
- ~~**Domain grouping outside the core.**~~ **Not needed.**
[ADR 0019](../../02-DECISIONS/0019-modules-and-the-graph.md) is
superseded by [ADR 0019](../../02-DECISIONS/0019-modules-and-the-graph.md):
[ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md) is
superseded by [ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md):
there is no domain module to group into, so there is no domain list to settle. Relationships
are edges, and grouping is a tag and a query.