novox/hq
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Consolidate the design layer: one place per topic

Browse Source 
Jochen: a jungle of specs that slightly contradict or patch each other, and
what matters is a working state rather than history. Both are fair and both are
mine.

Measured rather than assumed. 05-the-node-host and 09-the-node-lifecycle both
covered enrolment, the install commands, the unit file, the launcher and
reconcile -- I wrote 09 without taking anything out of 05, so the same things
were said twice and could drift apart.

Split by what each document IS. 05 is the component: what the host is, its
parts, the declaration vocabulary, the build order, how it is verified. 09 is
what happens to it: install, enrol, run, upgrade, retire. The whole "The
process" section left 05, and the unit file moved to 09 where installing is
described. 05 goes from 338 lines to 245 and now points at 09 rather than
restating it.

09 also carried a 105-line "Resolved" section -- six mechanisms framed as
"these were open and here is the answer". The content is needed; the framing is
history, and history is what makes a document read as a changelog rather than a
description. Renamed to what it actually is and the was-open phrasing removed.

Also added 10-delivery.md, which did not exist: four accepted decisions --
0054, 0063, 0064, 0065 -- had no design document at all, which is the specific
reason the delivery picture felt scattered. It is now one document covering
modules, the three edges, the core library, and how a change becomes a running
thing, with a table of what each property is designed against and what must
exist before it can be built.
This commit is contained in:
jschoubben
2026-08-28 18:40:46 +02:00
parent 9d091c81e0
commit 10365f2eae
7 changed files with 243 additions and 117 deletions
Download Patch File Download Diff File Expand all files Collapse all files
03-DESIGN/01-to-be/05-the-node-host.md
+11 -104
View File
@@ -116,112 +116,19 @@ work: a node is a node, and what varies between them is here rather than in the
What this machine *is* — its identity, what it holds, what it has applied. Reported upward over
the link; never asked downward.
## The process
## What it is not
[ADR 0057](../../02-DECISIONS/0057-the-host-is-a-root-service-installed-as-a-package.md).
- It does not decide anything that needs another node.
- It never queries the mesh database.
- It has no listening surface.
- **It does not manage its own unit.** It manages `service` resources and its own unit is one —
the temptation is obvious and it ends with a host stopping itself half way through an apply,
leaving a machine with nothing running to fix it. The installation owns the host; the host owns
everything else.
**One `root` service on every node, plus command-line entry points for a person.** A machine
without one is not a node ([ADR 0036](../../02-DECISIONS/0036-a-node-is-a-managed-machine.md)) —
the host is what makes a machine managed, so there is no agentless node and no partial mode.
Root because there is no useful unprivileged subset: it writes under `/etc`, installs packages,
manages units and runs containers.
### Installing it
Two steps, and there is nothing else:
```
# 1 — put the host on the machine, in that machine's own idiom
apk add nox-mesh-host && rc-update add nox-mesh-host && rc-service nox-mesh-host start
pacman -S nox-mesh-host && systemctl enable --now nox-mesh-host
# 2 — hand it the mesh. The same on every machine.
nox-mesh-host enrol --token <one-time token>
```
**Step 1 differs per system and step 2 never does**, which is the shape of
[ADR 0060](../../02-DECISIONS/0060-the-host-is-built-per-operating-system.md): the package
manager and the init file are the system's, and everything after them is the mesh's.
The token carries the broker's address, the fingerprint to expect, and the right to join
([ADR 0051](../../02-DECISIONS/0051-the-enrolment-token-carries-the-mesh.md)). After that the
node is in the mesh and takes declarations like every other one.
**On a machine with no package repository, or no mesh to serve one:**
```
curl -fsSL https://<release>/mesh-host-<version>-x86_64.tar.gz | tar -xz -C /usr/local/bin
```
That path must never acquire a dependency. **The mesh's package repository is hosted on the
mesh**, so any installation route that needs the mesh is a circle — a first node cannot use it,
and neither can anyone repairing a mesh that is down.
### The unit
```ini
[Unit]
Description=Novox Mesh node host
After=network-online.target
Wants=network-online.target
[Service]
ExecStart=/usr/lib/nox-mesh-host/launch
Restart=always
RestartSec=5s
StateDirectory=mesh-host
[Install]
WantedBy=multi-user.target
```
**Two lines of policy, and that is deliberate**
([ADR 0061](../../02-DECISIONS/0061-the-host-asks-an-init-for-start-and-restart.md)). The init is
asked to *start this at boot* and *start it again if it exits*, and nothing else. Both are
expressible in OpenRC, runit, s6 and an Android `init.rc`, so porting this file is transcription
rather than design.
**`Restart=always` and not `on-failure`**: the host restarts onto a new binary by exiting
*cleanly*, so a supervisor that only restarts on failure would leave every upgraded node stopped,
having successfully upgraded.
**What the init does not do is decide when to give up.** Counting failed starts and rolling back
lives in the launcher, where it can be tested — `OnFailure=` in a unit file can only be read and
hoped for, and it is the one thing that has to work on a machine where nothing else does.
**The package owns this file. The host never does.** It manages `service` resources, and its own
unit is a service — the temptation is obvious and it ends with a host stopping itself half way
through an apply, leaving a machine with nothing running to fix it. A declaration naming the
host's own unit is **refused**, and that refusal is a test rather than a convention.
The line to hold: **the installation owns the host; the host owns everything else.**
### What it does while running
| | |
|---|---|
| holds the **link** | one outbound connection, node-initiated, nothing listening ([ADR 0039](../../02-DECISIONS/0039-the-link-is-the-security-boundary.md)) |
| **applies** what arrives | declarations of known shape, in the order given |
| **reads back and reports** | what it did, and what it now owns |
| **reconciles** | on start, on a declaration, on a timer, and on reconnect |
**The timer is the one that is easy to leave out.** Without it, a machine that drifted — a
person edited a file, a package upgrade replaced a config — stays drifted until somebody happens
to change a declaration. `owned` would then report what the host *applied* rather than what is
*there*, which is [ADR 0035](../../02-DECISIONS/0035-a-picture-is-read-from-what-runs.md)
violated by omission. **Ten minutes, configurable.**
**Disconnected is a working state, not a degraded one.** The host keeps reconciling against its
own store, so a laptop shut for a week comes back and reconciles rather than coming back and
asking what it is. That is
[ADR 0036](../../02-DECISIONS/0036-a-node-is-a-managed-machine.md) made operational.
### Upgrading it
**By the package manager, not by the mesh.** A running process that replaces its own binary and
restarts part-way through an apply is the self-management problem again. Recorded as a limit:
a fleet-wide host upgrade is not currently a mesh operation.
**How it is installed, enrolled, run, upgraded and retired is
[`09-the-node-lifecycle.md`](09-the-node-lifecycle.md)**, in full and in one place. This document
is the component; that one is what happens to it.
## Where a declaration comes from
03-DESIGN/01-to-be/09-the-node-lifecycle.md
+48 -10
View File
@@ -84,6 +84,45 @@ hosted on the mesh. Any route that needs the mesh in order to install the thing
mesh is a circle — unusable on a first node, and unusable by whoever is repairing a mesh that is
down, which is exactly when it is wanted.
### The unit it installs
```ini
[Unit]
Description=Novox Mesh node host
After=network-online.target
Wants=network-online.target
[Service]
ExecStart=/usr/lib/nox-mesh-host/launch
Restart=always
RestartSec=5s
StateDirectory=mesh-host
[Install]
WantedBy=multi-user.target
```
**Two lines of policy, and that is deliberate**
([ADR 0061](../../02-DECISIONS/0061-the-host-asks-an-init-for-start-and-restart.md)). The init is
asked to *start this at boot* and *start it again if it exits*, and nothing else. Both are
expressible in OpenRC, runit, s6 and an Android `init.rc`, so porting this file is transcription
rather than design.
**`Restart=always` and not `on-failure`**: the host restarts onto a new binary by exiting
*cleanly*, so a supervisor that only restarts on failure would leave every upgraded node stopped,
having successfully upgraded.
**What the init does not do is decide when to give up.** Counting failed starts and rolling back
lives in the launcher, where it can be tested — `OnFailure=` in a unit file can only be read and
hoped for, and it is the one thing that has to work on a machine where nothing else does.
**The package owns this file. The host never does.** It manages `service` resources, and its own
unit is a service — the temptation is obvious and it ends with a host stopping itself half way
through an apply, leaving a machine with nothing running to fix it. A declaration naming the
host's own unit is **refused**, and that refusal is a test rather than a convention.
The line to hold: **the installation owns the host; the host owns everything else.**
At this point the host is running and **doing nothing**. It has no identity, so there is nobody
to link to and nothing to apply. It answers `profile`, `inventory` and `version`, and waits.
@@ -485,13 +524,14 @@ every node, each one goes quiet, and the mesh reports a fleet of sleeping laptop
---
## Resolved
## Details that are easy to get wrong
The items this document opened, with the reasoning, because each was open for a reason.
Each of these has a wrong answer that looks reasonable, which is why they are written down
rather than left to be worked out.
### Re-enrolling as the same node
**A token is issued *for* a node record, and that is where the question is answered.**
**A token is issued *for* a node record**, and that is where a re-enrolment is decided.
```
mesh-control token issue --node workstation # this machine is that node again
@@ -539,11 +579,9 @@ whoever is looking, not a constant in the design.
**Adoption always completes. A node with a `failed` line is a node, and it is not eligible for
assignment until the failure is resolved.**
This keeps *flags inform, they do not block*
([research 012](../../01-RESEARCH/012-the-minimum-viable-node/00-overview.md)) exactly where it
was decided — for **conflicts**, where the mesh chose deliberately and the machine works — and
gives **failures** the different treatment they need, because a failure is not *we chose* but
*we could not*.
*Flags inform, they do not block* holds for **conflicts** — where the mesh chose deliberately and
the machine still works. A **failure** is different in kind: not *we chose* but *we could not*,
and it gets different treatment for that reason.
The distinction is between **joining** and **being given work**. Refusing to join makes a
machine in use unadoptable, which is the outcome that rule exists to prevent. Placing work on a
@@ -580,8 +618,8 @@ keeps cataloguing.
**`mesh-control token issue` prints it once**, to the person running it. Single-use, and it
expires whether used or not ([ADR 0039](../../02-DECISIONS/0039-the-link-is-the-security-boundary.md)).
It is carried by hand — read off a screen, pasted into a terminal. That is not a gap in the
design, it is the design: its authenticity comes from the channel it travelled, which is what
It is carried by hand — read off a screen, pasted into a terminal. That is the design rather than
a gap in it: its authenticity comes from the channel it travelled, which is what
lets a node verify a mesh it has never spoken to
([ADR 0051](../../02-DECISIONS/0051-the-enrolment-token-carries-the-mesh.md)). A token emailed,
committed, or dropped in shared storage has lost the only property that makes it worth carrying.
03-DESIGN/01-to-be/10-delivery.md
+180
View File
@@ -0,0 +1,180 @@
---
layer: to-be
status: designed
code: []
updated: 2026-08-28
decisions:
- 02-DECISIONS/0014-build-publish-and-deploy-are-three-silos.md
- 02-DECISIONS/0044-a-module-declares-presence-instantiation-and-exclusion.md
- 02-DECISIONS/0054-things-that-change-together-share-an-authority.md
- 02-DECISIONS/0058-delivery-ends-in-a-declaration.md
- 02-DECISIONS/0063-delivery-is-reconciliation-not-a-pipeline.md
- 02-DECISIONS/0064-a-build-edge-is-a-third-kind.md
- 02-DECISIONS/0065-the-core-library-is-the-meshs-domain.md
---
# Modules and delivery
How a change somebody makes becomes a thing running on machines.
This is the whole of it, current, in one place. Where a decision record is cited it is for the
reasoning behind a choice, not because the answer is somewhere else.
## A module
The unit of delivery: assignable to a node, versionable, replaceable on its own.
**Not a grouping.** There is no `networking` module containing four things — there are four
modules, named individually, with edges between them. Folders assert relationships; edges record
them, and only edges can be queried or kept true automatically.
**When several modules always change together**, that means they share an *authority* — one place
that decides for all of them. It does not mean they should be one artifact. Connectivity is the
worked example: one context decides the overlay, names, routes, filtering and certificates, and
`wireguard`, the resolver, the proxy and the firewall remain four modules, because they are
deployed to different sets of nodes.
> **Coherence is a context. Delivery is a module.**
## The three edges
A module's relationships to other modules. Two are declared; one is read from the code.
| edge | means | declared? | satisfied |
|---|---|---|---|
| **presence** | that thing must exist and be reachable here | yes, in the manifest | at provisioning |
| **instantiation** | that thing makes something for me and hands back credentials — a database, a bucket, a route | yes, in the manifest | at provisioning, and again whenever it must be |
| **build** | I was compiled against that artifact | **no — derived from imports** | **at build, once** |
**Why the build edge is derived and the others are not.** A runtime edge is an *intention*
somebody has about how the mesh should be wired, and only a person can state it. A build edge is
a *fact about code that already exists* — and a declared list of dependencies drifts from the
imports it describes, so the imports are what is read.
**Why the build edge is a different kind rather than a variant.** It is fixed inside an artifact
rather than negotiated when something runs, and its only remedy is a rebuild. Nothing can
re-provision it.
## The core library
One module everything is allowed to depend on, holding **the mesh's own domain**: a module, a
node, an assignment. Those three are what every context talks about and none of them owns.
The test for whether something belongs: *would this still mean the same thing in a context that
had never heard of the one it came from?* A node would. A pipeline stage would not — that is
delivery's. A grant would not — that is provisioning's.
**Types ship with the module that owns them**, not here. A consumer needing `inventory`'s types
depends on `inventory` — one narrow, visible edge — rather than everything depending on a hub
where the relationship cannot be seen. A library everything depends on is expensive to change
whether it holds types or code; what makes it expensive is the fan-in.
**This stays small on its own**, which is the point of choosing a domain rather than a drawer. A
domain model changes when what the mesh *is* changes, which is rare. *Shared code* changes
whenever anybody writes something reusable, which is constantly.
## Delivery is a comparison, not a pipeline
The control plane holds two facts and builds the difference:
```
what source exists ─┐
├─► differ? ─► build ─► judge ─► declare ─► nodes converge
what has been built from it ─┘
```
**A change becomes a build because source is ahead of artifacts.** Not because a message arrived.
An event makes it fast; nothing makes it necessary — so a missed webhook costs latency and cannot
cost correctness.
That is the same shape the host uses on a machine, one layer up:
| | reconciles | against |
|---|---|---|
| the control plane | artifacts | source |
| the host | machine state | declarations |
**There is no pipeline as a state machine.** No stage list something can be omitted from, and no
run to lose.
### An artifact is current, or it is not
> An artifact is out of date when **its source moved, or anything it was built against moved**.
So what is recorded against an artifact is a commit **and the identity of every artifact it was
built against** — its input closure. That is what makes *is this current?* answerable without
building anything, and what makes the rebuild set computable: take the changed module, follow
inbound build edges transitively, and that is what is stale. In order, because the edges are
directed.
**A shared change is a cascade, and that is inherent.** One change to the core library
invalidates nearly everything. The ordering comes from the graph, not from a hand-written list of
levels.
### The verdict
An artifact may not be declared until something has judged it fit. Two tiers, because one gate
would be both slow and unreliable:
| | judged by | when |
|---|---|---|
| **the module's own tests** | the build | **always** — this is most of it |
| **the lab** | a raised scenario | when an assertion genuinely needs a mesh |
**A run that failed for environmental reasons is not a verdict.** A machine that would not boot
says nothing about the artifact, and recording it as *unfit* is the same untruth as recording a
dispatch as a deploy. *Outstanding* and *failed* are different results.
### Declaring, and converging
Deploy is **one write**: the affected nodes' declarations now name the new artifact. It is not
once per node, and nothing is pushed to a machine.
Each host applies what it is told, reads back, and reports. A node that is switched off does it
when it wakes.
**What a delivery result means:**
```
meshboard source X · built from X · fit · declared on 5 · applied on 3, 2 outstanding
```
Not *the job went green*. **Outstanding is not failure** — a node that has not applied yet is a
fact with a timestamp, and it resolves itself when the node comes back.
## What this is designed against
Every property above answers something that has actually gone wrong, recorded in
[`00-as-is/04`](../00-as-is/04-delivery.md):
| what happened | what prevents it |
|---|---|
| a merge created no pipeline, and nothing said so | a change is found by comparison, not by an event |
| a package install 404'd from every mirror while the job went green | the applier is the reporter, and it reads back |
| a verify stage was built and never scheduled | verification is not a stage that can be left off a list |
| a service was reported started when the command merely returned | *green proves transport, not effect* — so nothing reports transport |
| the build node parked forever while every other node deployed | there is no fan-out to be asymmetric about |
## What must exist before this can be built
Not aspirations — things without which the above does not work:
1. **The module graph, with build edges.** No graph, no rebuild set and no ordering.
2. **A recorded input closure per artifact**, so currency is answerable without building.
3. **Something that notices a reconciler is not converging.** Below.
## Open
- **Does a fit artifact declare itself?** Nothing above says who moves the declaration. If it is
automatic, merging to main deploys to production — which may be wanted, and is far too large a
property to acquire by omission.
- **A reconciler that cannot reach its target retries forever.** A failed job stops and names its
step; a loop is silent. Without something that notices *this has been trying for an hour*, this
design reintroduces the fault it removes. **The largest open risk here.**
- **Reproducible builds.** If rebuilding unchanged source against unchanged inputs produced the
same digest, a cascade would stop at the first module whose output did not move. Without them,
one core-library commit redeploys the fleet with no behavioural change.
- **How a module publishes its own types**, which differs per language.
- **How the control plane upgrades itself.** It declares its own new version and the host applies
it — but if the new one is broken, the thing that would fix it is the thing that is broken. The
host has a launcher for exactly this; the control plane has nothing.
03-DESIGN/01-to-be/README.md
+1
View File
@@ -19,6 +19,7 @@ document is written and this one's status becomes `implemented`.
| [`07-the-substrate.md`](07-the-substrate.md) | Tier 1 — what the control plane consumes and cannot grant itself | [ADR 0038](../../02-DECISIONS/0038-a-node-joins-by-linking-first.md), [0048](../../02-DECISIONS/0048-the-substrate-is-named.md) |
| [`08-connectivity.md`](08-connectivity.md) | One context in full — overlay, resolution, exposure, filtering, certificates | [ADR 0049](../../02-DECISIONS/0049-a-route-is-a-grant.md), [0050](../../02-DECISIONS/0050-reachability-is-a-property-of-the-address.md), [0051](../../02-DECISIONS/0051-the-enrolment-token-carries-the-mesh.md), [0055](../../02-DECISIONS/0055-the-control-plane-is-the-node-coordinating-contexts.md) |
| [`09-the-node-lifecycle.md`](09-the-node-lifecycle.md) | How a machine becomes a node, stays one, and stops being one | [ADR 0038](../../02-DECISIONS/0038-a-node-joins-by-linking-first.md), [0051](../../02-DECISIONS/0051-the-enrolment-token-carries-the-mesh.md) |
| [`10-delivery.md`](10-delivery.md) | Modules, the three edges, and how a change becomes a running thing | [ADR 0063](../../02-DECISIONS/0063-delivery-is-reconciliation-not-a-pipeline.md), [0064](../../02-DECISIONS/0064-a-build-edge-is-a-third-kind.md), [0065](../../02-DECISIONS/0065-the-core-library-is-the-meshs-domain.md) |
## Not yet written