diff --git a/02-DECISIONS/0063-delivery-is-reconciliation-not-a-pipeline.md b/02-DECISIONS/0063-delivery-is-reconciliation-not-a-pipeline.md index 86b8654..114969d 100644 --- a/02-DECISIONS/0063-delivery-is-reconciliation-not-a-pipeline.md +++ b/02-DECISIONS/0063-delivery-is-reconciliation-not-a-pipeline.md @@ -1,5 +1,5 @@ --- -status: proposed +status: accepted date: 2026-08-28 deciders: jochen reconstructed: false diff --git a/02-DECISIONS/0064-a-build-edge-is-a-third-kind.md b/02-DECISIONS/0064-a-build-edge-is-a-third-kind.md index 844cd9b..ca28f0c 100644 --- a/02-DECISIONS/0064-a-build-edge-is-a-third-kind.md +++ b/02-DECISIONS/0064-a-build-edge-is-a-third-kind.md @@ -1,5 +1,5 @@ --- -status: proposed +status: accepted date: 2026-08-28 deciders: jochen reconstructed: false diff --git a/02-DECISIONS/0065-the-core-library-is-the-meshs-domain.md b/02-DECISIONS/0065-the-core-library-is-the-meshs-domain.md index 13a98d4..2b06ccc 100644 --- a/02-DECISIONS/0065-the-core-library-is-the-meshs-domain.md +++ b/02-DECISIONS/0065-the-core-library-is-the-meshs-domain.md @@ -1,5 +1,5 @@ --- -status: proposed +status: accepted date: 2026-08-28 deciders: jochen reconstructed: false diff --git a/03-DESIGN/01-to-be/05-the-node-host.md b/03-DESIGN/01-to-be/05-the-node-host.md index d4612bc..c0d8a9e 100644 --- a/03-DESIGN/01-to-be/05-the-node-host.md +++ b/03-DESIGN/01-to-be/05-the-node-host.md @@ -116,112 +116,19 @@ work: a node is a node, and what varies between them is here rather than in the What this machine *is* — its identity, what it holds, what it has applied. Reported upward over the link; never asked downward. -## The process +## What it is not -[ADR 0057](../../02-DECISIONS/0057-the-host-is-a-root-service-installed-as-a-package.md). +- It does not decide anything that needs another node. +- It never queries the mesh database. +- It has no listening surface. +- **It does not manage its own unit.** It manages `service` resources and its own unit is one — + the temptation is obvious and it ends with a host stopping itself half way through an apply, + leaving a machine with nothing running to fix it. The installation owns the host; the host owns + everything else. -**One `root` service on every node, plus command-line entry points for a person.** A machine -without one is not a node ([ADR 0036](../../02-DECISIONS/0036-a-node-is-a-managed-machine.md)) — -the host is what makes a machine managed, so there is no agentless node and no partial mode. - -Root because there is no useful unprivileged subset: it writes under `/etc`, installs packages, -manages units and runs containers. - -### Installing it - -Two steps, and there is nothing else: - -``` -# 1 — put the host on the machine, in that machine's own idiom -apk add nox-mesh-host && rc-update add nox-mesh-host && rc-service nox-mesh-host start -pacman -S nox-mesh-host && systemctl enable --now nox-mesh-host - -# 2 — hand it the mesh. The same on every machine. -nox-mesh-host enrol --token -``` - -**Step 1 differs per system and step 2 never does**, which is the shape of -[ADR 0060](../../02-DECISIONS/0060-the-host-is-built-per-operating-system.md): the package -manager and the init file are the system's, and everything after them is the mesh's. - -The token carries the broker's address, the fingerprint to expect, and the right to join -([ADR 0051](../../02-DECISIONS/0051-the-enrolment-token-carries-the-mesh.md)). After that the -node is in the mesh and takes declarations like every other one. - -**On a machine with no package repository, or no mesh to serve one:** - -``` -curl -fsSL https:///mesh-host--x86_64.tar.gz | tar -xz -C /usr/local/bin -``` - -That path must never acquire a dependency. **The mesh's package repository is hosted on the -mesh**, so any installation route that needs the mesh is a circle — a first node cannot use it, -and neither can anyone repairing a mesh that is down. - -### The unit - -```ini -[Unit] -Description=Novox Mesh node host -After=network-online.target -Wants=network-online.target - -[Service] -ExecStart=/usr/lib/nox-mesh-host/launch -Restart=always -RestartSec=5s -StateDirectory=mesh-host - -[Install] -WantedBy=multi-user.target -``` - -**Two lines of policy, and that is deliberate** -([ADR 0061](../../02-DECISIONS/0061-the-host-asks-an-init-for-start-and-restart.md)). The init is -asked to *start this at boot* and *start it again if it exits*, and nothing else. Both are -expressible in OpenRC, runit, s6 and an Android `init.rc`, so porting this file is transcription -rather than design. - -**`Restart=always` and not `on-failure`**: the host restarts onto a new binary by exiting -*cleanly*, so a supervisor that only restarts on failure would leave every upgraded node stopped, -having successfully upgraded. - -**What the init does not do is decide when to give up.** Counting failed starts and rolling back -lives in the launcher, where it can be tested — `OnFailure=` in a unit file can only be read and -hoped for, and it is the one thing that has to work on a machine where nothing else does. - -**The package owns this file. The host never does.** It manages `service` resources, and its own -unit is a service — the temptation is obvious and it ends with a host stopping itself half way -through an apply, leaving a machine with nothing running to fix it. A declaration naming the -host's own unit is **refused**, and that refusal is a test rather than a convention. - -The line to hold: **the installation owns the host; the host owns everything else.** - -### What it does while running - -| | | -|---|---| -| holds the **link** | one outbound connection, node-initiated, nothing listening ([ADR 0039](../../02-DECISIONS/0039-the-link-is-the-security-boundary.md)) | -| **applies** what arrives | declarations of known shape, in the order given | -| **reads back and reports** | what it did, and what it now owns | -| **reconciles** | on start, on a declaration, on a timer, and on reconnect | - -**The timer is the one that is easy to leave out.** Without it, a machine that drifted — a -person edited a file, a package upgrade replaced a config — stays drifted until somebody happens -to change a declaration. `owned` would then report what the host *applied* rather than what is -*there*, which is [ADR 0035](../../02-DECISIONS/0035-a-picture-is-read-from-what-runs.md) -violated by omission. **Ten minutes, configurable.** - -**Disconnected is a working state, not a degraded one.** The host keeps reconciling against its -own store, so a laptop shut for a week comes back and reconciles rather than coming back and -asking what it is. That is -[ADR 0036](../../02-DECISIONS/0036-a-node-is-a-managed-machine.md) made operational. - -### Upgrading it - -**By the package manager, not by the mesh.** A running process that replaces its own binary and -restarts part-way through an apply is the self-management problem again. Recorded as a limit: -a fleet-wide host upgrade is not currently a mesh operation. +**How it is installed, enrolled, run, upgraded and retired is +[`09-the-node-lifecycle.md`](09-the-node-lifecycle.md)**, in full and in one place. This document +is the component; that one is what happens to it. ## Where a declaration comes from diff --git a/03-DESIGN/01-to-be/09-the-node-lifecycle.md b/03-DESIGN/01-to-be/09-the-node-lifecycle.md index 30cf821..aa71a50 100644 --- a/03-DESIGN/01-to-be/09-the-node-lifecycle.md +++ b/03-DESIGN/01-to-be/09-the-node-lifecycle.md @@ -84,6 +84,45 @@ hosted on the mesh. Any route that needs the mesh in order to install the thing mesh is a circle — unusable on a first node, and unusable by whoever is repairing a mesh that is down, which is exactly when it is wanted. +### The unit it installs + +```ini +[Unit] +Description=Novox Mesh node host +After=network-online.target +Wants=network-online.target + +[Service] +ExecStart=/usr/lib/nox-mesh-host/launch +Restart=always +RestartSec=5s +StateDirectory=mesh-host + +[Install] +WantedBy=multi-user.target +``` + +**Two lines of policy, and that is deliberate** +([ADR 0061](../../02-DECISIONS/0061-the-host-asks-an-init-for-start-and-restart.md)). The init is +asked to *start this at boot* and *start it again if it exits*, and nothing else. Both are +expressible in OpenRC, runit, s6 and an Android `init.rc`, so porting this file is transcription +rather than design. + +**`Restart=always` and not `on-failure`**: the host restarts onto a new binary by exiting +*cleanly*, so a supervisor that only restarts on failure would leave every upgraded node stopped, +having successfully upgraded. + +**What the init does not do is decide when to give up.** Counting failed starts and rolling back +lives in the launcher, where it can be tested — `OnFailure=` in a unit file can only be read and +hoped for, and it is the one thing that has to work on a machine where nothing else does. + +**The package owns this file. The host never does.** It manages `service` resources, and its own +unit is a service — the temptation is obvious and it ends with a host stopping itself half way +through an apply, leaving a machine with nothing running to fix it. A declaration naming the +host's own unit is **refused**, and that refusal is a test rather than a convention. + +The line to hold: **the installation owns the host; the host owns everything else.** + At this point the host is running and **doing nothing**. It has no identity, so there is nobody to link to and nothing to apply. It answers `profile`, `inventory` and `version`, and waits. @@ -485,13 +524,14 @@ every node, each one goes quiet, and the mesh reports a fleet of sleeping laptop --- -## Resolved +## Details that are easy to get wrong -The items this document opened, with the reasoning, because each was open for a reason. +Each of these has a wrong answer that looks reasonable, which is why they are written down +rather than left to be worked out. ### Re-enrolling as the same node -**A token is issued *for* a node record, and that is where the question is answered.** +**A token is issued *for* a node record**, and that is where a re-enrolment is decided. ``` mesh-control token issue --node workstation # this machine is that node again @@ -539,11 +579,9 @@ whoever is looking, not a constant in the design. **Adoption always completes. A node with a `failed` line is a node, and it is not eligible for assignment until the failure is resolved.** -This keeps *flags inform, they do not block* -([research 012](../../01-RESEARCH/012-the-minimum-viable-node/00-overview.md)) exactly where it -was decided — for **conflicts**, where the mesh chose deliberately and the machine works — and -gives **failures** the different treatment they need, because a failure is not *we chose* but -*we could not*. +*Flags inform, they do not block* holds for **conflicts** — where the mesh chose deliberately and +the machine still works. A **failure** is different in kind: not *we chose* but *we could not*, +and it gets different treatment for that reason. The distinction is between **joining** and **being given work**. Refusing to join makes a machine in use unadoptable, which is the outcome that rule exists to prevent. Placing work on a @@ -580,8 +618,8 @@ keeps cataloguing. **`mesh-control token issue` prints it once**, to the person running it. Single-use, and it expires whether used or not ([ADR 0039](../../02-DECISIONS/0039-the-link-is-the-security-boundary.md)). -It is carried by hand — read off a screen, pasted into a terminal. That is not a gap in the -design, it is the design: its authenticity comes from the channel it travelled, which is what +It is carried by hand — read off a screen, pasted into a terminal. That is the design rather than +a gap in it: its authenticity comes from the channel it travelled, which is what lets a node verify a mesh it has never spoken to ([ADR 0051](../../02-DECISIONS/0051-the-enrolment-token-carries-the-mesh.md)). A token emailed, committed, or dropped in shared storage has lost the only property that makes it worth carrying. diff --git a/03-DESIGN/01-to-be/10-delivery.md b/03-DESIGN/01-to-be/10-delivery.md new file mode 100644 index 0000000..19c3bf8 --- /dev/null +++ b/03-DESIGN/01-to-be/10-delivery.md @@ -0,0 +1,180 @@ +--- +layer: to-be +status: designed +code: [] +updated: 2026-08-28 +decisions: + - 02-DECISIONS/0014-build-publish-and-deploy-are-three-silos.md + - 02-DECISIONS/0044-a-module-declares-presence-instantiation-and-exclusion.md + - 02-DECISIONS/0054-things-that-change-together-share-an-authority.md + - 02-DECISIONS/0058-delivery-ends-in-a-declaration.md + - 02-DECISIONS/0063-delivery-is-reconciliation-not-a-pipeline.md + - 02-DECISIONS/0064-a-build-edge-is-a-third-kind.md + - 02-DECISIONS/0065-the-core-library-is-the-meshs-domain.md +--- + +# Modules and delivery + +How a change somebody makes becomes a thing running on machines. + +This is the whole of it, current, in one place. Where a decision record is cited it is for the +reasoning behind a choice, not because the answer is somewhere else. + +## A module + +The unit of delivery: assignable to a node, versionable, replaceable on its own. + +**Not a grouping.** There is no `networking` module containing four things — there are four +modules, named individually, with edges between them. Folders assert relationships; edges record +them, and only edges can be queried or kept true automatically. + +**When several modules always change together**, that means they share an *authority* — one place +that decides for all of them. It does not mean they should be one artifact. Connectivity is the +worked example: one context decides the overlay, names, routes, filtering and certificates, and +`wireguard`, the resolver, the proxy and the firewall remain four modules, because they are +deployed to different sets of nodes. + +> **Coherence is a context. Delivery is a module.** + +## The three edges + +A module's relationships to other modules. Two are declared; one is read from the code. + +| edge | means | declared? | satisfied | +|---|---|---|---| +| **presence** | that thing must exist and be reachable here | yes, in the manifest | at provisioning | +| **instantiation** | that thing makes something for me and hands back credentials — a database, a bucket, a route | yes, in the manifest | at provisioning, and again whenever it must be | +| **build** | I was compiled against that artifact | **no — derived from imports** | **at build, once** | + +**Why the build edge is derived and the others are not.** A runtime edge is an *intention* +somebody has about how the mesh should be wired, and only a person can state it. A build edge is +a *fact about code that already exists* — and a declared list of dependencies drifts from the +imports it describes, so the imports are what is read. + +**Why the build edge is a different kind rather than a variant.** It is fixed inside an artifact +rather than negotiated when something runs, and its only remedy is a rebuild. Nothing can +re-provision it. + +## The core library + +One module everything is allowed to depend on, holding **the mesh's own domain**: a module, a +node, an assignment. Those three are what every context talks about and none of them owns. + +The test for whether something belongs: *would this still mean the same thing in a context that +had never heard of the one it came from?* A node would. A pipeline stage would not — that is +delivery's. A grant would not — that is provisioning's. + +**Types ship with the module that owns them**, not here. A consumer needing `inventory`'s types +depends on `inventory` — one narrow, visible edge — rather than everything depending on a hub +where the relationship cannot be seen. A library everything depends on is expensive to change +whether it holds types or code; what makes it expensive is the fan-in. + +**This stays small on its own**, which is the point of choosing a domain rather than a drawer. A +domain model changes when what the mesh *is* changes, which is rare. *Shared code* changes +whenever anybody writes something reusable, which is constantly. + +## Delivery is a comparison, not a pipeline + +The control plane holds two facts and builds the difference: + +``` +what source exists ─┐ + ├─► differ? ─► build ─► judge ─► declare ─► nodes converge +what has been built from it ─┘ +``` + +**A change becomes a build because source is ahead of artifacts.** Not because a message arrived. +An event makes it fast; nothing makes it necessary — so a missed webhook costs latency and cannot +cost correctness. + +That is the same shape the host uses on a machine, one layer up: + +| | reconciles | against | +|---|---|---| +| the control plane | artifacts | source | +| the host | machine state | declarations | + +**There is no pipeline as a state machine.** No stage list something can be omitted from, and no +run to lose. + +### An artifact is current, or it is not + +> An artifact is out of date when **its source moved, or anything it was built against moved**. + +So what is recorded against an artifact is a commit **and the identity of every artifact it was +built against** — its input closure. That is what makes *is this current?* answerable without +building anything, and what makes the rebuild set computable: take the changed module, follow +inbound build edges transitively, and that is what is stale. In order, because the edges are +directed. + +**A shared change is a cascade, and that is inherent.** One change to the core library +invalidates nearly everything. The ordering comes from the graph, not from a hand-written list of +levels. + +### The verdict + +An artifact may not be declared until something has judged it fit. Two tiers, because one gate +would be both slow and unreliable: + +| | judged by | when | +|---|---|---| +| **the module's own tests** | the build | **always** — this is most of it | +| **the lab** | a raised scenario | when an assertion genuinely needs a mesh | + +**A run that failed for environmental reasons is not a verdict.** A machine that would not boot +says nothing about the artifact, and recording it as *unfit* is the same untruth as recording a +dispatch as a deploy. *Outstanding* and *failed* are different results. + +### Declaring, and converging + +Deploy is **one write**: the affected nodes' declarations now name the new artifact. It is not +once per node, and nothing is pushed to a machine. + +Each host applies what it is told, reads back, and reports. A node that is switched off does it +when it wakes. + +**What a delivery result means:** + +``` +meshboard source X · built from X · fit · declared on 5 · applied on 3, 2 outstanding +``` + +Not *the job went green*. **Outstanding is not failure** — a node that has not applied yet is a +fact with a timestamp, and it resolves itself when the node comes back. + +## What this is designed against + +Every property above answers something that has actually gone wrong, recorded in +[`00-as-is/04`](../00-as-is/04-delivery.md): + +| what happened | what prevents it | +|---|---| +| a merge created no pipeline, and nothing said so | a change is found by comparison, not by an event | +| a package install 404'd from every mirror while the job went green | the applier is the reporter, and it reads back | +| a verify stage was built and never scheduled | verification is not a stage that can be left off a list | +| a service was reported started when the command merely returned | *green proves transport, not effect* — so nothing reports transport | +| the build node parked forever while every other node deployed | there is no fan-out to be asymmetric about | + +## What must exist before this can be built + +Not aspirations — things without which the above does not work: + +1. **The module graph, with build edges.** No graph, no rebuild set and no ordering. +2. **A recorded input closure per artifact**, so currency is answerable without building. +3. **Something that notices a reconciler is not converging.** Below. + +## Open + +- **Does a fit artifact declare itself?** Nothing above says who moves the declaration. If it is + automatic, merging to main deploys to production — which may be wanted, and is far too large a + property to acquire by omission. +- **A reconciler that cannot reach its target retries forever.** A failed job stops and names its + step; a loop is silent. Without something that notices *this has been trying for an hour*, this + design reintroduces the fault it removes. **The largest open risk here.** +- **Reproducible builds.** If rebuilding unchanged source against unchanged inputs produced the + same digest, a cascade would stop at the first module whose output did not move. Without them, + one core-library commit redeploys the fleet with no behavioural change. +- **How a module publishes its own types**, which differs per language. +- **How the control plane upgrades itself.** It declares its own new version and the host applies + it — but if the new one is broken, the thing that would fix it is the thing that is broken. The + host has a launcher for exactly this; the control plane has nothing. diff --git a/03-DESIGN/01-to-be/README.md b/03-DESIGN/01-to-be/README.md index 2bac554..6da1325 100644 --- a/03-DESIGN/01-to-be/README.md +++ b/03-DESIGN/01-to-be/README.md @@ -19,6 +19,7 @@ document is written and this one's status becomes `implemented`. | [`07-the-substrate.md`](07-the-substrate.md) | Tier 1 — what the control plane consumes and cannot grant itself | [ADR 0038](../../02-DECISIONS/0038-a-node-joins-by-linking-first.md), [0048](../../02-DECISIONS/0048-the-substrate-is-named.md) | | [`08-connectivity.md`](08-connectivity.md) | One context in full — overlay, resolution, exposure, filtering, certificates | [ADR 0049](../../02-DECISIONS/0049-a-route-is-a-grant.md), [0050](../../02-DECISIONS/0050-reachability-is-a-property-of-the-address.md), [0051](../../02-DECISIONS/0051-the-enrolment-token-carries-the-mesh.md), [0055](../../02-DECISIONS/0055-the-control-plane-is-the-node-coordinating-contexts.md) | | [`09-the-node-lifecycle.md`](09-the-node-lifecycle.md) | How a machine becomes a node, stays one, and stops being one | [ADR 0038](../../02-DECISIONS/0038-a-node-joins-by-linking-first.md), [0051](../../02-DECISIONS/0051-the-enrolment-token-carries-the-mesh.md) | +| [`10-delivery.md`](10-delivery.md) | Modules, the three edges, and how a change becomes a running thing | [ADR 0063](../../02-DECISIONS/0063-delivery-is-reconciliation-not-a-pipeline.md), [0064](../../02-DECISIONS/0064-a-build-edge-is-a-third-kind.md), [0065](../../02-DECISIONS/0065-the-core-library-is-the-meshs-domain.md) | ## Not yet written