Consolidate: 65 decision records to 23
Every remaining cluster merged. Each was one design that had been split across
several records because it was worked out over days rather than at once.
the node host 8 -> 1 applies not decides, depends on nothing,
per operating system, root service, the
launcher, episodic, what a declaration is,
actions from the bundle only
a node and how it joins 4 -> 1 what a node is, joining, the link as
security boundary, the enrolment token
modules and the graph 7 -> 1 everything is a module, no domain modules,
three edges, provisioning, the core library
substrate and control 6 -> 1 the test, seven contexts, one control plane,
plane the authority is not a database, the named
products, the pinned bundle
connectivity 3 -> 1 a route is a grant, reachability declared,
filter rules
delivery 5 -> 1 reconciliation not a pipeline, artifacts,
the three silos, a failed step, the verdict
the lab 5 -> 1 (earlier)
how this repository 10 -> 1 (earlier)
works
Nothing was dropped. Each consolidated record carries the reasoning of the ones
it absorbs -- the measurements, the incidents, the alternatives rejected --
because that reasoning is the only reason to keep a record at all. What is gone
is the fragmentation: eight files to read to understand tier 0, when tier 0 is
one component.
The four superseded records went too. They existed to point at their
successors, and the successors now contain what they said.
The checker made this safe. Each merge left dangling links -- 38 files after
the host merge alone -- and it named every one. Nothing was found by reading,
and a manual pass would certainly have missed some, including references inside
AGENTS.md which every session loads.
This commit is contained in:
@@ -1,73 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-03-14
|
||||
deciders: jochen
|
||||
reconstructed: true
|
||||
---
|
||||
|
||||
# 2. Everything is a module, and one manifest describes all of them
|
||||
|
||||
> Reconstructed after the fact from the evidence cited below.
|
||||
|
||||
## Context
|
||||
|
||||
The mesh carries several kinds of thing: containerised services with data and ports, pure
|
||||
capability providers with no service at all, and bare markers whose only content is that a
|
||||
node has them. Before this decision these were separate concepts with separate handling —
|
||||
the earlier vocabulary was *capabilities*, and services were installed by a different path
|
||||
than tools.
|
||||
|
||||
Every distinct kind of thing needs its own install path, its own change detection, its own
|
||||
place in the delivery pipeline, and its own documentation. Three kinds means three of each,
|
||||
and every new feature has to be built three times or, more commonly, once — leaving two kinds
|
||||
quietly unsupported.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **Separate concepts per kind** — a service registry, a tool registry, a node feature flag
|
||||
list. Rejected: it is what existed, and the cost was paid in every cross-cutting change.
|
||||
2. **One manifest, kind inferred from directory contents.** Chosen.
|
||||
3. **One manifest with an explicit `type:` field on every module.** Partly adopted — a service
|
||||
still declares itself — but the general rule became inference, because a declared list and
|
||||
the directory it describes drift, and the directory is the one that is true.
|
||||
|
||||
## Decision
|
||||
|
||||
Everything the mesh installs is a **module**: a directory with a manifest. The manifest
|
||||
declares identity, environment variables, what the module provides, what it requires, and how
|
||||
it is exposed. What kind of module it is follows from what the directory contains:
|
||||
|
||||
| Contains | Is |
|
||||
|---|---|
|
||||
| a compose definition | a service |
|
||||
| a tools directory | a capability provider |
|
||||
| a daemon or unit directory | a long-running process |
|
||||
| a configs directory | a source of managed files |
|
||||
| nothing but a manifest | a flag — presence is the whole content |
|
||||
|
||||
A module may be several of these at once. Each is a **feature**, and the delivery pipeline
|
||||
addresses features, not modules.
|
||||
|
||||
The mesh's own components are modules on exactly these terms. They get no privileged install
|
||||
path, no separate registry, and no exemption from the pipeline.
|
||||
|
||||
## Consequences
|
||||
|
||||
- One mechanism to learn, one to document, one to fix. A pipeline improvement reaches
|
||||
everything the mesh carries.
|
||||
- Dogfooding stops being a discipline and becomes structural: if the mesh's own components
|
||||
need an exception, the machinery is unfinished, and that is visible immediately.
|
||||
- Feature detection from directory contents means a directory rename silently changes what a
|
||||
module *is*. This has bitten repeatedly — a hook named for a feature the module does not
|
||||
have is skipped without complaint.
|
||||
- The manifest becomes load-bearing and grows. It is now the largest single point of
|
||||
coupling in the mesh.
|
||||
|
||||
## References
|
||||
|
||||
- `Rename capabilities → modules across the entire codebase`, 2026-03-14.
|
||||
- `Merge fail2ban, ufw, firewall apps into modules`, 2026-03-15 — the first modules to arrive
|
||||
by conversion rather than by creation.
|
||||
- Knowledge base: `modules`, `modules/manifest-reference`, `conventions/modules`.
|
||||
- The rename-breaks-detection shape: `troubleshooting/hooks-named-for-missing-feature`,
|
||||
`troubleshooting/health-check-tools-index-false-positive`.
|
||||
@@ -1,66 +0,0 @@
|
||||
---
|
||||
status: superseded
|
||||
superseded-by: 02-DECISIONS/0056-the-authority-is-the-control-plane-not-a-database.md
|
||||
date: 2026-04-02
|
||||
deciders: jochen
|
||||
reconstructed: true
|
||||
---
|
||||
|
||||
# 3. The mesh database is the source of truth; the repository is node-agnostic
|
||||
|
||||
> Reconstructed after the fact from the evidence cited below.
|
||||
|
||||
## Context
|
||||
|
||||
Two things must be known to run the mesh: **what exists** — which modules there are, what each
|
||||
declares, how each is built — and **what runs where** — which node hosts which module, with
|
||||
which settings, at which version.
|
||||
|
||||
The repository is the natural home of the first. It was initially also the home of the second:
|
||||
per-node directories held that node's configuration, and adopting a machine meant committing
|
||||
its files. That has three costs. A node cannot be changed without a commit, so runtime state
|
||||
and source share a review cadence they do not share a rhythm with. Two nodes cannot be
|
||||
reconciled, because nothing holds both. And the repository becomes an inventory of the
|
||||
installation, which is exactly the content that cannot be made public.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **Per-node directories in the repository.** Rejected — it is what existed. Every binding
|
||||
change is a commit and a deploy, and the repository accumulates an inventory of one
|
||||
particular mesh.
|
||||
2. **Configuration files distributed to nodes and edited there.** Rejected. There is then no
|
||||
authority: two nodes disagreeing have no arbiter, and drift is invisible until something
|
||||
breaks.
|
||||
3. **A mesh database as the single authority, cached locally for resilience.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
A single database holds every binding: which node hosts which module, at which selection, with
|
||||
which environment overrides, plus mesh-level settings that all nodes read. The runtime loads
|
||||
its configuration from that database at startup and falls back to a local cache when the
|
||||
database is unreachable.
|
||||
|
||||
**The repository defines what exists. The database defines what runs where.** No node-to-module
|
||||
mapping is ever committed.
|
||||
|
||||
A node is therefore not described anywhere in source. Bringing one into the mesh is a database
|
||||
operation.
|
||||
|
||||
## Consequences
|
||||
|
||||
- The repository becomes node-agnostic, and can be published without disclosing an
|
||||
installation. This repository's public stance rests on that property.
|
||||
- A binding changes without a commit, a build, or a deploy.
|
||||
- The local cache means a node survives losing the database, but a node running from cache is
|
||||
running from a snapshot with no indication of its age. Divergence is silent by construction.
|
||||
- The database is the hardest dependency in the mesh. It is also a module, provisioned like
|
||||
any other, which makes its bootstrap circular — resolved by the first-node initialisation
|
||||
script, and the reason such a script exists.
|
||||
- Nothing on a node is authoritative. That is what makes the next decision necessary.
|
||||
|
||||
## References
|
||||
|
||||
- `Phase 3: rename core modules to hal/ namespace`, 2026-04-02, and the mesh configuration
|
||||
tables that landed with it.
|
||||
- Knowledge base: `mesh` — "The repo is node-agnostic. It contains no per-node assignments."
|
||||
- The stale-cache shape: `troubleshooting/installed-version-and-deployments-are-stale`.
|
||||
@@ -11,7 +11,7 @@ reconstructed: true
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0003](0003-the-mesh-database-is-the-source-of-truth.md) put every binding in the mesh
|
||||
[ADR 0048](0048-the-substrate-and-the-control-plane.md) put every binding in the mesh
|
||||
database. But the things that consume those bindings — environment files, service
|
||||
definitions, daemon configuration, firewall rules — are files on a node's disk, because that
|
||||
is what the software reading them requires.
|
||||
|
||||
@@ -1,67 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-04-06
|
||||
deciders: jochen
|
||||
reconstructed: true
|
||||
---
|
||||
|
||||
# 5. Capabilities are provisioned on declaration, not configured by hand
|
||||
|
||||
> Reconstructed after the fact from the evidence cited below.
|
||||
|
||||
## Context
|
||||
|
||||
Most modules need something another module holds — a database, a cache, a bucket, a message
|
||||
vhost, an identity client. Wiring that by hand means creating the resource, creating a user,
|
||||
generating a credential, putting it in the consumer's configuration, and repeating all of it
|
||||
on every node the consumer runs on.
|
||||
|
||||
Every step is a place to make a mistake that surfaces much later, and the credential ends up
|
||||
written somewhere it can be read.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **Manual setup, documented.** Rejected. Documentation of a manual procedure is a
|
||||
description of the mistakes people will make.
|
||||
2. **A shared credential per resource type**, distributed to all consumers. Rejected: no
|
||||
isolation, and rotation becomes a mesh-wide outage.
|
||||
3. **Declared requirements, satisfied by the provider module.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
A module declares what it **provides** and what it **requires**. A requirement names the
|
||||
provider, the resource type, optionally a name and a target node, and a mapping from the
|
||||
resource's connection fields to the consumer's environment variables.
|
||||
|
||||
The mesh satisfies it: a provisioner belonging to the provider creates the resource and its
|
||||
credential, records the grant, and writes the mapped values as database overrides. The
|
||||
synchroniser from [ADR 0004](0004-managed-files-are-generated-never-edited.md) then
|
||||
materialises them. Neither the credential nor the topology is ever written by hand.
|
||||
|
||||
A requirement may name a provider on another node. The grant records consumer and provider
|
||||
nodes separately, so cross-node wiring is the same declaration.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **Provisioning becomes a core concern of the mesh, not plumbing.** A module asks for a
|
||||
capability; where it lives is the mesh's problem. This is the property
|
||||
[ADR 0015](0015-mesh-brokers-nodes-host-agents-think.md) later builds the whole domain model
|
||||
around.
|
||||
- Credentials are never authored, so they are never authored badly, and they are never in the
|
||||
repository.
|
||||
- Each consumer gets its own credential, so revocation is per-consumer.
|
||||
- Rotation is where this bites. A shared secret rotated for a new consumer invalidates the
|
||||
peers holding the old one, and this has taken the mesh down. The declaration model makes
|
||||
granting easy and says nothing about fan-out.
|
||||
- A module with no requirements skips the stage entirely, which is correct and also means the
|
||||
absence of provisioning is indistinguishable from provisioning that did not run.
|
||||
|
||||
## References
|
||||
|
||||
- `Remove shell/ helper library; split brain into independent workspaces`, 2026-04-06 — the
|
||||
provisioner daemon becomes its own component.
|
||||
- `Coordinator refactor: centralize pipeline orchestration`, 2026-04-04 — the provision-then-
|
||||
environment-then-start sequence becomes the coordinator's.
|
||||
- Knowledge base: `provisioning`, `provisioning/requires`.
|
||||
- The rotation failure: `troubleshooting/provision-rotation-invalidates-peers`,
|
||||
`troubleshooting/provision-adoption-rotates-live-credential`.
|
||||
@@ -1,71 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-06-05
|
||||
deciders: jochen
|
||||
reconstructed: true
|
||||
---
|
||||
|
||||
# 8. A step that fails must fail the job
|
||||
|
||||
> Reconstructed after the fact from the evidence cited below.
|
||||
|
||||
## Context
|
||||
|
||||
The mesh's expensive faults are not crashes. They are the operations that reported success and
|
||||
did nothing: an artifact that partially downloaded and was extracted anyway, a package that
|
||||
404ed from every mirror while the job went green, a hook that never ran because it was named
|
||||
for a feature the module does not declare, a deploy that reported the transport succeeded
|
||||
rather than that the effect happened.
|
||||
|
||||
Each of these was found long after it happened, by someone investigating an unrelated symptom.
|
||||
The cost is not the failure; it is the interval between the failure and anyone learning of it,
|
||||
during which decisions are made on the assumption that the thing worked.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **Continue on error and report at the end.** Rejected — it is largely what existed. A
|
||||
summary nobody reads is not a report, and later steps run against the state the failed step
|
||||
should have produced.
|
||||
2. **Continue on error, and let health checks catch the divergence.** Rejected. It converts a
|
||||
precise, located failure into a vague one discovered elsewhere, and requires a health check
|
||||
for every possible partial state.
|
||||
3. **Fail the step, fail the job, say which step.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
A step that fails stops the sequence it is part of, and the failure is surfaced where the work
|
||||
was requested — not only in a log.
|
||||
|
||||
Concretely, and these are the forms it takes:
|
||||
|
||||
- A scripted sequence gates each step on the previous one. A directory change that fails must
|
||||
stop the commands that assumed it.
|
||||
- An artifact that does not fully download is not extracted.
|
||||
- A stage reports the **effect** it achieved, not that it dispatched a message. "Started" must
|
||||
mean the thing is running, not that a command returned.
|
||||
- A template that cannot resolve a variable is not written half-rendered.
|
||||
|
||||
**Prefer failing to lying.** A green result that is not true costs more than a red one.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Failures are noisier and land earlier, on the person who caused them.
|
||||
- Some jobs that used to complete now stop. In every case examined so far, that job was
|
||||
producing a partial result that something downstream trusted.
|
||||
- This is a rule the mesh has adopted repeatedly rather than once, because each instance is
|
||||
written in a different place — a shell hook, a download path, a deploy stage. It is not
|
||||
enforced by a mechanism, and cannot currently be checked in general. New instances are still
|
||||
being found; the package-install case remains open as
|
||||
[`04-ISSUES/001`](../04-ISSUES/001-failed-package-install-reports-success/00-report.md).
|
||||
|
||||
## References
|
||||
|
||||
- `fix(installer): fail loudly when feature artifact download fails` (#244), 2026-06-05.
|
||||
- `A flavor template with an unresolved variable is written to disk instead of failing`
|
||||
(#710), 2026-08-08.
|
||||
- Knowledge base: `troubleshooting/deploy-reports-transport-not-effect`,
|
||||
`troubleshooting/service-started-is-not-ready`,
|
||||
`troubleshooting/green-pipeline-means-transport-not-effect`,
|
||||
`troubleshooting/silent-failures-and-stale-state`.
|
||||
- The core value it became: [`00-META/mission.md`](../00-META/mission.md), "Failure must
|
||||
be loud."
|
||||
@@ -1,60 +0,0 @@
|
||||
---
|
||||
status: superseded
|
||||
superseded-by: 02-DECISIONS/0018-the-mesh-creates-no-symlinks.md
|
||||
date: 2026-07-10
|
||||
deciders: jochen
|
||||
reconstructed: true
|
||||
---
|
||||
|
||||
# 11. The installer owns linking; nothing else creates a symlink
|
||||
|
||||
> Reconstructed after the fact from the evidence cited below. The incident that earned the rule
|
||||
> predates the record, and its date is not established here.
|
||||
|
||||
## Context
|
||||
|
||||
A service's definition lives in the module catalogue; its runtime directory and persistent data
|
||||
live outside it. The mesh connects the two by linking the definition into the runtime location
|
||||
— deliberately, so that runtime state and source stay separate while the running service reads
|
||||
a current definition.
|
||||
|
||||
A link is also the easiest thing in the world to create by hand while fixing something, and a
|
||||
container engine resolves a bind mount through it. A hand-made link pointed a volume somewhere
|
||||
it should not have, and **production data was lost**.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **Copy instead of linking.** Rejected. A copy goes stale silently, which trades data loss
|
||||
for a service running a definition nobody can find.
|
||||
2. **Allow links, document the hazard.** Rejected. The hazard is not knowable at the moment of
|
||||
the mistake — the link looks right and the resolution happens inside the container engine.
|
||||
3. **One component owns linking; everyone else is forbidden.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
The installer creates and repairs every link the mesh needs. It reconciles them: a missing
|
||||
link is created, a stale one is repointed, and a real file found where a link belongs is
|
||||
adopted into the node's override location and replaced.
|
||||
|
||||
**Nothing else creates a symlink** — not a hook, not a fix, not an agent, not a person
|
||||
debugging. The prohibition is absolute because the judgement required to make a safe exception
|
||||
is exactly the judgement that was not available at the moment it mattered.
|
||||
|
||||
## Consequences
|
||||
|
||||
- The class of failure is closed, at the cost of a rule that reads as arbitrary to anyone who
|
||||
has not seen the incident. That is why it is recorded here rather than only asserted.
|
||||
- Links become reconcilable state rather than incidental filesystem facts.
|
||||
- The rule is stated for humans and agents and is enforced by convention, not mechanism. A
|
||||
check does not exist.
|
||||
- The rule as written governs the mechanism rather than removing it. A link made by the
|
||||
installer resolves the same way as one made by hand, so the hazard is narrowed and not
|
||||
closed. [ADR 0018](0018-the-mesh-creates-no-symlinks.md) proposes widening this to "nothing
|
||||
links, the installer included"; until that is accepted, this record governs.
|
||||
|
||||
## References
|
||||
|
||||
- Recorded as a non-negotiable in the governed constitution page, §2: *"Symlinks to repos or
|
||||
service directories have caused production data loss via Docker volume path resolution. The
|
||||
installer handles all linking. Never create symlinks manually."*
|
||||
- Knowledge base: `services` — the reconciliation behaviour, including adoption of real files.
|
||||
@@ -1,63 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-04
|
||||
deciders: jochen
|
||||
reconstructed: true
|
||||
---
|
||||
|
||||
# 13. An artifact is build output, never a source tree
|
||||
|
||||
> Reconstructed after the fact from the evidence cited below.
|
||||
|
||||
## Context
|
||||
|
||||
A module is built once and deployed to every node assigned to it. What travels between those
|
||||
two events is the artifact.
|
||||
|
||||
For a long time the artifact was a filtered copy of the module's source directory. Deploying it
|
||||
therefore meant resolving and installing its dependencies **on the target node** — which
|
||||
requires the target to reach a package registry, at deploy time, for every node, every deploy.
|
||||
A node with no route to the registry could not deploy code that had already been built
|
||||
successfully.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **Ship source, install dependencies on the target.** Rejected — it is what existed. Deploy
|
||||
becomes a network operation with a failure mode per node, and the code that runs is
|
||||
assembled independently on each one.
|
||||
2. **Ship source plus its resolved dependency tree.** Rejected: large, slow, and it ships the
|
||||
dependency resolution's platform assumptions along with it.
|
||||
3. **Ship a self-contained build output; a failed bundle fails the build.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
The artifact is the module's **build output directory** — compiled and bundled, with its
|
||||
dependency graph inlined. Deploy is extract-and-run and touches no network.
|
||||
|
||||
A build that cannot produce a self-contained output **fails**. It does not fall back to
|
||||
shipping a dependency tree, because a fallback that works is a fallback that is never fixed —
|
||||
an application of [ADR 0008](0008-a-failed-step-fails-the-job.md).
|
||||
|
||||
## Consequences
|
||||
|
||||
- A node can deploy without reaching a registry. What was built is what runs, identically, on
|
||||
every node.
|
||||
- Deploys are faster and their failure modes are local.
|
||||
- **Everything not in the build output does not ship.** This is the decision's whole cost, and
|
||||
it was paid several times before it was understood: migrations that read the source layout,
|
||||
provisioning scripts that read the source layout, selection files never packaged at all. Each
|
||||
worked in development, where the source is present, and silently did nothing after deploy.
|
||||
- Any file a module needs at runtime must be deliberately placed into the build output. The
|
||||
rule "the artifact is `dist/`" has to be applied to every file kind, not just compiled code,
|
||||
and that generalisation was the expensive part.
|
||||
- Bundling has its own failure modes that a compiler will not catch — a bundler can exit
|
||||
successfully and produce output that cannot load.
|
||||
|
||||
## References
|
||||
|
||||
- `build: bundle artifacts so a deploy is extract-and-run` (#673), 2026-08-04.
|
||||
- The consequences, in order: `Provision migrations and seeds read the source layout, not the
|
||||
artifact` (#699), `Local migrations read the source layout too` (#700), both 2026-08-07.
|
||||
- Knowledge base: `pipeline/artifacts-are-build-output`, `pipeline/bundling`,
|
||||
`troubleshooting/shell-migrations-never-packaged`, `troubleshooting/flavors-never-packaged`,
|
||||
`troubleshooting/esbuild-silent-tla-breakage`.
|
||||
@@ -1,78 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-04
|
||||
deciders: jochen
|
||||
reconstructed: true
|
||||
---
|
||||
|
||||
# 14. Build, publish and deploy are three silos with different cardinality
|
||||
|
||||
> Reconstructed after the fact from the evidence cited below.
|
||||
|
||||
## Context
|
||||
|
||||
Delivery had been treated as one pipeline that a module passes through. It is not: its stages
|
||||
run a different number of times.
|
||||
|
||||
- Compiling happens **once per module feature**, on the build node.
|
||||
- Packaging and uploading happens **once per module feature**, on the build node.
|
||||
- Installing, configuring, starting and verifying happens **once per module feature per node**.
|
||||
|
||||
Conflating them is what made earlier versions slow and hard to reason about. Work that should
|
||||
happen once was being repeated per node, and the fan-out point was implicit rather than a
|
||||
boundary anything could observe.
|
||||
|
||||
The split had been declared before it was real. Packaging still happened inside the build,
|
||||
which meant the boundary existed in the documentation and not in the code.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **One pipeline, stages that know their own cardinality.** Rejected — it is what existed.
|
||||
Cardinality is then a property of each stage's implementation, and nothing can reason about
|
||||
the pipeline as a whole.
|
||||
2. **Two silos: build-and-publish, then deploy.** Rejected. It leaves packaging inside build,
|
||||
so build must know every module, every feature, and how each composes its artifact —
|
||||
exactly the coupling the split exists to remove. A failed upload then retries by re-sending
|
||||
a stale package instead of re-packaging.
|
||||
3. **Three silos, with an explicit handover between each.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
Delivery is three silos, and the boundaries are real:
|
||||
|
||||
| Silo | Runs | Where |
|
||||
|---|---|---|
|
||||
| **build** | once per module feature | the build node |
|
||||
| **publish** | once per module feature | the build node |
|
||||
| **deploy** | once per module feature **per node** | every assigned node |
|
||||
|
||||
Commands and events are addressed **per feature**, not per module.
|
||||
|
||||
Build compiles and hands over a **staged tree** — not a package. Publish applies the module's
|
||||
packaging rules, packages that tree, and uploads it. Publishing to a package registry *is*
|
||||
publishing, so a module whose artifact is a package publishes in the publish silo, not the
|
||||
build one.
|
||||
|
||||
Modules are resolved into dependency **levels**, and a level completes before the next begins,
|
||||
so a module always builds against its dependencies' freshly published versions.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Work that should happen once happens once. The fan-out point is explicit and observable.
|
||||
- A failed upload retries by re-packaging, because packaging belongs to the stage that
|
||||
uploads.
|
||||
- The handover is a staged tree in a known location rather than the build's working directory,
|
||||
which is reference-counted and cannot be assumed to still exist when a later stage runs.
|
||||
- The build node is now the only node that has already passed through two silos when the
|
||||
fan-out happens. Anything tracking a node's stage must account for **both** pre-fan-out
|
||||
stages; code that knew only about the first parked the build node forever while every other
|
||||
node deployed cleanly.
|
||||
- A recovery mechanism that knows a subset of the stages it guards is worse than none — it
|
||||
reports success over a stall it cannot see.
|
||||
|
||||
## References
|
||||
|
||||
- `publish owns packaging — the silos were not actually split` (#677), 2026-08-04.
|
||||
- Knowledge base: `pipeline/three-silos` — including the note that the older architecture
|
||||
documents claimed otherwise and were stale until 2026-08-06.
|
||||
- The build-node stage-tracking failure was observed on pipeline #5557.
|
||||
@@ -1,98 +0,0 @@
|
||||
---
|
||||
status: superseded
|
||||
superseded-by: 02-DECISIONS/0044-a-module-declares-presence-instantiation-and-exclusion.md
|
||||
date: 2026-08-23
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0015-mesh-brokers-nodes-host-agents-think.md
|
||||
---
|
||||
|
||||
# 17. Modules outside the platform core are grouped by domain, not by single function
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0015](0015-mesh-brokers-nodes-host-agents-think.md) recomposes the platform's own modules
|
||||
into bounded contexts named after their aggregates, and sends the rest out of the monorepo on
|
||||
the grounds that they run *on* the mesh rather than being *of* it.
|
||||
|
||||
That leaves the larger half unaddressed. Around three quarters of the catalogue are modules
|
||||
that are neither part of the mesh's domain nor standalone applications: a firewall, a VPN, an
|
||||
SSH daemon and a resolver; a file manager, a media player and a system monitor; a set of
|
||||
media-library services. Today each is its own module, because one module is the unit of *one
|
||||
piece of software*, and no other grouping exists.
|
||||
|
||||
The result is that the catalogue's shape records what was installed, not what anything is for.
|
||||
Four modules that together constitute "how a node is reachable" have no relationship the mesh
|
||||
can see: they cannot be assigned, versioned, reasoned about or replaced as one thing, and a
|
||||
change to how the mesh handles connectivity has to be made four times.
|
||||
|
||||
This is the same failure ADR 0015 names for the core — *boundaries drawn by deployment accident
|
||||
rather than by domain* — appearing outside it.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **Leave them as they are.** Rejected. The core gets domain boundaries and everything else
|
||||
keeps accident boundaries, so the catalogue becomes harder to read after the refactor than
|
||||
before it.
|
||||
2. **One module per piece of software, with a tag or category field.** Rejected. A label is not
|
||||
a boundary: it does not change what can be assigned, versioned or replaced as a unit, and it
|
||||
drifts from the thing it labels.
|
||||
3. **Group them into domain modules, each owning the software that serves one purpose.**
|
||||
Proposed here.
|
||||
4. **Extend ADR 0015's contexts to cover everything.** Rejected. Those contexts are named for
|
||||
the mesh's own aggregates; a media library is not an aggregate of the mesh, and forcing it
|
||||
into that model repeats the metaphor-naming mistake ADR 0015 exists to correct.
|
||||
|
||||
## Decision
|
||||
|
||||
*Proposed — the principle is settled; the domain list is not. See "Open" below.*
|
||||
|
||||
Modules that are not part of the platform core are grouped into **domain modules**. A domain
|
||||
is named for the concern it serves, and owns the software that serves it. The unit stops being
|
||||
one piece of software and becomes one purpose.
|
||||
|
||||
This extends ADR 0015 rather than replacing it. The eight bounded contexts for the mesh's own
|
||||
domain stand unchanged. This decision covers what ADR 0015 leaves outside them.
|
||||
|
||||
Naming follows the same rule as the core: **name the domain for what it does, not for what it
|
||||
is made of**. Connectivity, not a VPN implementation.
|
||||
|
||||
## Consequences
|
||||
|
||||
- A domain becomes assignable, versionable and replaceable as one thing. Changing how nodes
|
||||
reach each other is a change to one module.
|
||||
- The catalogue's shape starts describing purpose. A reader can tell what a mesh is *for* from
|
||||
its module list.
|
||||
- Swapping an implementation stops being a module replacement, with the data-volume and
|
||||
provisioning consequences that carries, and becomes a change inside a domain.
|
||||
- The count drops sharply, which is a symptom of the improvement rather than the point of it.
|
||||
- **Grouping conceals.** A domain module hides which implementation is in use, and every
|
||||
operational question — which port, which unit, which credential — gains an indirection.
|
||||
- The migration is not free and has no obvious increments: a domain is only useful once
|
||||
everything belonging to it has moved.
|
||||
- Some modules genuinely serve one purpose and are already correctly sized. Grouping for its
|
||||
own sake would be the same error in the other direction.
|
||||
|
||||
## Open
|
||||
|
||||
**The domain list is not settled and this record does not invent one.** What is decided is the
|
||||
principle; what is not decided is the set. Candidate groupings are visible in the catalogue —
|
||||
connectivity and reachability, node presentation and desktop, media libraries, observation and
|
||||
metrics, storage and data services — but naming them here would be reconstructing a decision
|
||||
that has not been taken.
|
||||
|
||||
Settling the list is a research effort, not an act of this record. Until it concludes, this
|
||||
ADR stays `proposed`. That effort is
|
||||
[`01-RESEARCH/005-domain-grouping`](../01-RESEARCH/005-domain-grouping/00-overview.md), and its
|
||||
first measurement already narrows this record's scope: co-change analysis supports grouping for
|
||||
reachability, argues against it for the provisioned infrastructure providers, and finds no
|
||||
signal either way for the fifty modules that never change alongside anything.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0015](0015-mesh-brokers-nodes-host-agents-think.md) — the core decomposition this
|
||||
extends, and its rule about naming a context after its aggregate.
|
||||
- [ADR 0010](0010-applications-live-in-their-own-repository.md) — standalone applications are
|
||||
already out of scope here; they are not domains and do not group.
|
||||
- [`03-DESIGN/00-as-is/10-module-catalogue.md`](../03-DESIGN/00-as-is/10-module-catalogue.md)
|
||||
— the catalogue's current shape, which is the evidence for the problem.
|
||||
@@ -3,15 +3,13 @@ status: accepted
|
||||
date: 2026-08-23
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
supersedes: 0011-the-installer-owns-linking.md
|
||||
extends: 0011-the-installer-owns-linking.md
|
||||
---
|
||||
|
||||
# 18. The mesh creates no symlinks — a derived file is a copy
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0011](0011-the-installer-owns-linking.md) responded to production data loss — a hand-made
|
||||
[ADR 0018](0018-the-mesh-creates-no-symlinks.md) responded to production data loss — a hand-made
|
||||
link, resolved through a container engine's volume handling, pointing a mount somewhere it
|
||||
should not have — by centralising linking in the installer and forbidding it everywhere else.
|
||||
|
||||
@@ -35,7 +33,7 @@ operation as reconciling a pointer, plus a comparison.
|
||||
So the mesh already has the machinery that makes a copy safe, and is using a link to solve a
|
||||
problem that machinery solves better. Worse, a link is conceptually the wrong shape: it makes
|
||||
the node's runtime state a *pointer into source*, which is the one thing
|
||||
[ADR 0003](0003-the-mesh-database-is-the-source-of-truth.md) and ADR 0004 exist to prevent.
|
||||
[ADR 0048](0048-the-substrate-and-the-control-plane.md) and ADR 0004 exist to prevent.
|
||||
State is derived onto nodes; it does not reach back.
|
||||
|
||||
## Considered options
|
||||
@@ -74,7 +72,7 @@ the rule exists at all, and the incident behind it is the reason anyone believes
|
||||
cost, and it is the whole cost: today a link cannot be stale, and a copy can. The answer has
|
||||
to be detection — the installer comparing what is on disk against what the mesh says should
|
||||
be — and it must be loud, because a silently stale definition is exactly the failure shape
|
||||
this mesh keeps producing ([ADR 0008](0008-a-failed-step-fails-the-job.md)).
|
||||
this mesh keeps producing ([ADR 0058](0058-delivery.md)).
|
||||
- Reconciliation gets more expensive: comparing content rather than checking a pointer's
|
||||
target, on every module, on every node.
|
||||
- Disk usage rises, trivially, and is not a consideration.
|
||||
@@ -95,7 +93,7 @@ Until those are answered this record stays `proposed`, and ADR 0011 remains the
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0011](0011-the-installer-owns-linking.md) — the incident, and the rule this widens.
|
||||
- [ADR 0018](0018-the-mesh-creates-no-symlinks.md) — the incident, and the rule this widens.
|
||||
- [ADR 0004](0004-managed-files-are-generated-never-edited.md) — the machinery that makes a
|
||||
copy safe.
|
||||
- [`03-DESIGN/00-as-is/05-runtime-and-installation.md`](../03-DESIGN/00-as-is/05-runtime-and-installation.md)
|
||||
|
||||
@@ -37,7 +37,7 @@ company-scoped one does not. So this is `hq` and the mesh's are `mesh-*`.
|
||||
| `novox/mesh-substrate` | 1 | the pinned tier-1 services, as declarations |
|
||||
| `novox/mesh-control` | 2 | the control plane and its contexts |
|
||||
| `novox/mesh-surfaces` | 3 | tools, web, cli — thin, no logic |
|
||||
| `novox/mesh-sdk` | — | the mesh's own domain ([ADR 0065](0065-the-core-library-is-the-meshs-domain.md)) |
|
||||
| `novox/mesh-sdk` | — | the mesh's own domain ([ADR 0044](0044-modules-and-the-graph.md)) |
|
||||
| `novox/mesh-lab` | — | the lab: scenario lifecycle, networking, placement |
|
||||
| `novox/hq` | — | this one |
|
||||
|
||||
|
||||
@@ -1,69 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-22
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
supersedes: none
|
||||
---
|
||||
|
||||
# 19. HQ is its own repository, and it is public
|
||||
|
||||
## Context
|
||||
|
||||
The mesh's reasoning — mission, research, design, decisions — began inside the code
|
||||
repository, under a folder there. The objection to moving it out was specific and good: the
|
||||
mesh already has an operational memory and a structured archive, and adding a third store
|
||||
repeats the mistake that consolidation was meant to fix.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **Keep it in the code repository.** Rejected, but the objection it rests on is correct and
|
||||
is answered rather than dismissed — see Consequences.
|
||||
2. **Put it in the structured archive**, alongside the governed documents. Rejected: the
|
||||
archive is not reviewable as a diff, and a design argument is exactly the thing that needs
|
||||
line-by-line review and a branch.
|
||||
3. **Its own repository.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
HQ is its own repository, and it is **public** — written for a reader who is not its author
|
||||
and has no access to the mesh it describes.
|
||||
|
||||
Three reasons it is separate:
|
||||
|
||||
- **The cadence differs.** A decision changes when thinking changes, not when code changes.
|
||||
Tying documents to a code branch merges them on the code's schedule.
|
||||
- **The reviewers differ.** A design argument is not reviewed the way an implementation is,
|
||||
and should not queue behind a build.
|
||||
- **The scope is wider than one repository.**
|
||||
[ADR 0015](0015-mesh-brokers-nodes-host-agents-think.md) sends most modules out of the
|
||||
monorepo; documentation governing several repositories cannot live inside one of them.
|
||||
|
||||
Being public is not incidental. It is enforceable only because
|
||||
[ADR 0003](0003-the-mesh-database-is-the-source-of-truth.md) made the code repository
|
||||
node-agnostic: there is no per-node content to leak. Nothing here may carry routable
|
||||
addresses, real domain names, hosting providers, node names, absolute paths, usernames,
|
||||
credentials, or operational detail useful only to an attacker.
|
||||
|
||||
The test is whether a paragraph would still teach a stranger running an entirely different
|
||||
mesh.
|
||||
|
||||
## Consequences
|
||||
|
||||
- A document and the code it describes can no longer land in one commit. Keeping them honest
|
||||
is a discipline rather than a mechanism — which is why decisions are recorded as they are
|
||||
taken, and why a document stating a rule must say how the rule is checked.
|
||||
- Research must state evidence without identifying the mesh it observed. The shape of a
|
||||
finding survives anonymisation; the instance does not travel.
|
||||
- **The objection is answered by indexing, not by location** — the claim being that these
|
||||
documents remain searchable beside everything else, one source with many surfaces.
|
||||
**That indexing does not exist.** Checked 2026-08-23, it returns nothing. Until it does, the
|
||||
objection stands unanswered and this repository is the third knowledge store it was argued
|
||||
not to be. Recorded as
|
||||
[`04-ISSUES/006`](../04-ISSUES/006-hq-is-not-indexed-into-the-knowledge-base/00-report.md).
|
||||
|
||||
## References
|
||||
|
||||
- Supersedes the earlier position that documentation lives inside the code repository under a
|
||||
folder there. That position was never recorded separately and has no record of its own.
|
||||
- [`README.md`](../README.md) — the public-repository rule in full.
|
||||
@@ -39,7 +39,7 @@ directly.
|
||||
Publishing is a playbook step, not a manual act, and it ends with **reading the page back and
|
||||
verifying the change is present**. A publish that reported success and did nothing is exactly
|
||||
the failure class this mesh keeps producing
|
||||
([ADR 0008](0008-a-failed-step-fails-the-job.md)).
|
||||
([ADR 0058](0058-delivery.md)).
|
||||
|
||||
Section numbering is stable, because the orchestrator and the review fragments cite sections by
|
||||
number.
|
||||
@@ -47,7 +47,7 @@ number.
|
||||
## Consequences
|
||||
|
||||
- One source, many surfaces — the same argument HQ's separation already rests on
|
||||
([ADR 0019](0019-hq-is-its-own-repository.md)), applied to the rules themselves.
|
||||
([ADR 0019](0019-how-this-repository-works.md)), applied to the rules themselves.
|
||||
- Each rule keeps the incident that earned it, in a place that is reviewed as a diff.
|
||||
- An edit to the derived page survives until the next sync and then vanishes. The playbook says
|
||||
so, and nothing mechanically prevents it.
|
||||
|
||||
@@ -0,0 +1,124 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-28
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
consolidates: [0038, 0039, 0051]
|
||||
---
|
||||
|
||||
# 36. A node, and how it joins
|
||||
|
||||
*Consolidated 2026-08-28 from four records.*
|
||||
|
||||
## What a node is
|
||||
|
||||
**A managed machine inside the mesh.** Not a device that is merely known about, not an
|
||||
unprivileged something. If the mesh does not manage it, it is not a node — it is a client, a
|
||||
peer, or a thing on the network, and those want their own names rather than a weakened version of
|
||||
this one.
|
||||
|
||||
**A disconnected node is still a node, in a different situation.** Reachability is **state, not
|
||||
class**. A machine switched off, roaming, or behind a connection that has dropped has not become
|
||||
a lesser kind of thing; it has a last-known state and a pending set of declarations.
|
||||
|
||||
The distinction people reach for is real, but it is **capability** — what this machine can be
|
||||
asked to do — and that belongs in the host's profile rather than in the definition of a node.
|
||||
|
||||
**This is the rule that does the most work elsewhere.** A single control plane is tolerable
|
||||
because its absence is every node in the ordinary disconnected situation at once. An episodic
|
||||
host on a phone is that situation more often. Neither needed a new mechanism.
|
||||
|
||||
## How it joins
|
||||
|
||||
**The host has one behaviour and two sources of declaration.** What differs between the first
|
||||
node and the fiftieth is not what the host does but where the declaration comes from — and, as
|
||||
above, that is a situation rather than a class.
|
||||
|
||||
| | declaration comes from |
|
||||
|---|---|
|
||||
| no mesh reachable | the pinned bundle the host carries |
|
||||
| mesh reachable | the control plane, over the link |
|
||||
|
||||
**The first node is not a different kind of node.** It is a node whose mesh is not up yet. It
|
||||
applies the bundle it carries, the control plane comes up on top of it, and from that moment it
|
||||
takes declarations like everything else. **Its specialness is temporary and self-erasing**, which
|
||||
is what the hand-run bootstrap scripts never were.
|
||||
|
||||
**A joining node does the minimum to be reachable and nothing else** — an identity, an address,
|
||||
and one peer to reach. It does **not** compute the overlay: the whole peer set is derived
|
||||
centrally and pushed down.
|
||||
|
||||
That is also why the migration is smaller than it looked. The hard part of the overlay — every
|
||||
node's key, address, site and reachability — is only needed to compute the *whole* mesh, and a
|
||||
joining node needs one peer.
|
||||
|
||||
## The link is the security boundary
|
||||
|
||||
**Everything reaching a node arrives one way**, and four properties make that a boundary rather
|
||||
than a pipe.
|
||||
|
||||
**It is outbound and node-initiated.** The node dials the control plane; nothing dials a node. Not
|
||||
only defensive — most nodes sit behind a household connection with no forwarded port, so an
|
||||
inbound control channel would work for one node and not the rest, and the difference would be
|
||||
invisible until it mattered. **A node has no listening control surface at all.**
|
||||
|
||||
**A node holds its own identity and nothing else.** No shared secret, no credential to anything it
|
||||
does not own. **Compromise of a node is compromise of that node** — which the current arrangement
|
||||
does not have, because every node permanently holds the same database and object-store
|
||||
credentials, and there is no mechanism that rotates one and informs everything holding it.
|
||||
|
||||
**Authority is mutual.** The node proves it may join, and the control plane proves it is the
|
||||
mesh. One-way is not enough: the host applies whatever the link delivers, so a node that cannot
|
||||
tell the mesh from something impersonating it will apply that something's declarations.
|
||||
|
||||
**What may be pushed is bounded by form, not by trust.** Declarations of known shape, never a
|
||||
command to run. Stated honestly, **this bounds form and not impact**: a compromised control plane
|
||||
can declare harmful state and the host will apply it faithfully, because that is what it is for.
|
||||
What the property buys is that the blast radius is *describable* — exactly what the declaration
|
||||
language can express, which can be reviewed. An arbitrary command channel has no such bound.
|
||||
|
||||
## The enrolment token carries the mesh
|
||||
|
||||
Mutual authority needs the node to verify something before it trusts anything, and that is a
|
||||
circle: verifying the mesh needs the mesh's certificate authority, and obtaining one means
|
||||
trusting whoever hands it over. There is a second circle beside it — a node must reach the mesh
|
||||
before the mesh has configured it, so it can resolve no mesh name.
|
||||
|
||||
**Both are the same shape: a node needs a fact about the mesh before it has any trustworthy way
|
||||
to obtain one.** So that fact arrives by a path other than the mesh.
|
||||
|
||||
**The token carries four things**, and it is the only thing a joining node needs:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **where** | the broker's **address**, not a name — there is no resolution yet, and this is why none is needed |
|
||||
| **what it is connecting to** | the fingerprint of the broker's certificate |
|
||||
| **who it will believe** | the control plane's signing identity |
|
||||
| **the right to join** | a one-time secret, useless once used and useless after it expires |
|
||||
|
||||
**Carried out of band**, by the person adopting the machine. That is what breaks both circles:
|
||||
its authenticity comes from the channel it travelled, not from anything the node can check
|
||||
afterwards. **Trust on first use, with the first use moved out of band** — the difference between
|
||||
a pin and a guess.
|
||||
|
||||
**The endpoint and the authority are two identities.** A node connects to the broker and takes
|
||||
instruction from the control plane behind it. Pinning only the broker would make the control
|
||||
plane's authority *transitive*, and a compromised broker could then forge declarations — which,
|
||||
since the host applies whatever the link delivers, is the whole machine. So the transport is
|
||||
verified once at connect, and **each declaration is verified by its signature, every time**.
|
||||
|
||||
**What this settles:** the mesh's certificate authority is not a bootstrap concern — it certifies
|
||||
internal names once a node is a member. Nothing needs name resolution before the link. And
|
||||
nothing is placed on disk beforehand except the token, which is the first moment *a node holds
|
||||
only its own identity* becomes true rather than aspirational.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **Declarations must be signed**, and the host must tell *this is not from the mesh I joined*
|
||||
apart from *this is malformed*. Rotating the signing identity is a fleet-wide operation with an
|
||||
overlapping rollover, and that is the cost of not trusting the broker.
|
||||
- **The token becomes security-critical**, because it carries the pin. Tampering substitutes the
|
||||
mesh — which is strictly better than the alternative, where there is nothing to tamper with and
|
||||
the node trusts the first answer unconditionally.
|
||||
- **A rejoining node is ordinary.** There is no long-lived secret to recover, so a node that lost
|
||||
its identity gets a new token.
|
||||
@@ -1,75 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-25
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
---
|
||||
|
||||
# 36. A node is a managed machine, and disconnection is a situation
|
||||
|
||||
## Context
|
||||
|
||||
[Research 006](../01-RESEARCH/006-mesh-from-scratch/00-overview.md) left open: *"does an
|
||||
unprivileged node earn a place in the inventory, or only a presence? Decides whether 'node'
|
||||
means one thing or two."*
|
||||
|
||||
The question came from requirement 6 — *Arch Linux only for now; ideally any device, including
|
||||
phones, on lighter terms* — and from the observation that some machines cannot be fully
|
||||
managed. A phone will not run the host. A laptop is absent for days.
|
||||
|
||||
The question assumed the answer was a **class**: full nodes and lesser ones, with the
|
||||
inventory recording the first and merely acknowledging the second.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **Two classes — nodes and presences.** An unprivileged device gets a lighter record and a
|
||||
reduced contract. Rejected: it makes "node" mean two things, so every context that reasons
|
||||
about nodes acquires a branch, and the branch is invisible in the type. The mesh already has
|
||||
one instance of this shape and it is the one this repository keeps writing issues about —
|
||||
a declared thing that is only sometimes honoured.
|
||||
2. **One class, membership by capability.** Everything is a node; what it can do is a property.
|
||||
Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
**A node is a managed machine inside the mesh.** Not a device that is merely known about, not
|
||||
an unprivileged something. If the mesh does not manage it, it is not a node — it is a client, a
|
||||
peer, or a thing on the network, and those want their own names rather than a weakened version
|
||||
of this one.
|
||||
|
||||
**A disconnected node is still a node, in a different situation.** Reachability is state, not
|
||||
class. A node that is switched off, roaming, or behind a connection that has dropped has not
|
||||
become a lesser kind of thing; it has a last-known state and a pending set of declarations.
|
||||
|
||||
The distinction the original question reached for is real, but it is **capability**, not kind —
|
||||
what this machine can be asked to do — and that belongs in the host's profile, not in the
|
||||
definition of a node.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **The inventory has one shape.** No branch, no second record type, no context that must ask
|
||||
which kind it is holding.
|
||||
- **Local state is structural, not a convenience.** If disconnection is an ordinary situation
|
||||
rather than an exception, the host's store is authoritative while disconnected by design —
|
||||
it is what makes the situation ordinary. This promotes `store/` from a component to a
|
||||
requirement.
|
||||
- **Absence is not failure.** A node that has not been seen is in a state, and the mesh must be
|
||||
able to say which. Anything that treats unreachable as broken will be wrong most of the time
|
||||
about a laptop.
|
||||
- **Devices that cannot be managed do not become nodes by being lenient about the word.** A
|
||||
phone that cannot run the host is not a node under this record. Whether the mesh should reach
|
||||
such devices at all, and as what, is not decided here and needs its own record if it is
|
||||
wanted.
|
||||
- **The reduced-contract idea is not lost, it is relocated.** What a given node can be asked to
|
||||
do is its profile — the host's capability detection — and varies per machine without varying
|
||||
what a node is.
|
||||
|
||||
## References
|
||||
|
||||
- [Research 006](../01-RESEARCH/006-mesh-from-scratch/00-overview.md) — the open question, and
|
||||
requirement 6 that raised it.
|
||||
- [ADR 0015](0015-mesh-brokers-nodes-host-agents-think.md) — *nodes host*; this says what a
|
||||
node is.
|
||||
- [Issue 007](../04-ISSUES/007-an-installed-package-is-not-a-capability/00-report.md) —
|
||||
capability as something detected rather than assumed, which is where the reduced contract
|
||||
now lives.
|
||||
@@ -1,97 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-25
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
---
|
||||
|
||||
# 37. The host applies; it does not decide
|
||||
|
||||
## Context
|
||||
|
||||
The skeleton absorbs overlay membership, packet filtering, package management, service
|
||||
supervision, the container runtime and filesystem management into tier 0, and
|
||||
[research 006](../01-RESEARCH/006-mesh-from-scratch/00-overview.md) called this *"the
|
||||
skeleton's biggest unproven claim. A binary whose whole argument is that it has no dependencies
|
||||
now carries six concerns."*
|
||||
|
||||
That claim has now been measured against the monorepo's `main`:
|
||||
[`host-size.md`](../01-RESEARCH/006-mesh-from-scratch/host-size.md).
|
||||
|
||||
The measurement says the question asked about the wrong axis.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **Absorb the six concerns as they are.** What the skeleton literally proposes. Rejected on
|
||||
evidence: two of the ten modules implementing them open a direct connection to the control
|
||||
plane's database and compute their own configuration. Absorbing those unchanged puts a
|
||||
Postgres client and knowledge of the mesh schema inside tier 0 — an upward dependency, and
|
||||
the tier rule is the whole of the bootstrap argument.
|
||||
2. **Leave them as modules.** Keeps the tier rule trivially, and keeps the fault that prompted
|
||||
the skeleton: four modules constituting *how a node is reachable* with no relationship the
|
||||
mesh can see, so one intent is expressed four times
|
||||
([research 005](../01-RESEARCH/005-domain-grouping/analysis.md) finding 4 measures this and
|
||||
finds it is the only place in the catalogue where the shape genuinely occurs).
|
||||
3. **Split each concern: decide centrally, apply locally.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
**The host has one concern: apply declared state on this machine.** The six are not six
|
||||
concerns it carries; they are instances of the one.
|
||||
|
||||
Each divides:
|
||||
|
||||
- **Deciding** — what this node's overlay, names, exposure, filtering, packages and services
|
||||
*should be*. This needs every other node, and belongs to the control plane.
|
||||
- **Applying** — putting that on the machine. This needs root and locality, and belongs to the
|
||||
host.
|
||||
|
||||
**The host never queries the mesh database.** A host that reads the control plane's schema is
|
||||
tier 0 depending on tier 2, and the tiers stop being a bootstrap answer the moment that is
|
||||
permitted once.
|
||||
|
||||
## Why the evidence supports it
|
||||
|
||||
**Size was the wrong worry.** The ten modules total 2 755 lines. The machinery that already
|
||||
applies state on a node — `meshware`, `env-sync`, `config-sync` — is 3 059. Everything being
|
||||
absorbed is smaller than what already exists to apply it. The host is not a new large thing; it
|
||||
already exists, spread across three core modules and unnamed.
|
||||
|
||||
**Eight of the ten are already pure appliers.** They receive derived state and put it on the
|
||||
machine. Absorbing them moves code that has no dependency to move.
|
||||
|
||||
**The split has already been happening, unnamed.** `dnsmasq-app` needs the same mesh-wide data
|
||||
as `wireguard` and does not query for it. Its own comments record why: the values were
|
||||
*"duplicated by hand on all four nodes"* until someone derived them centrally, after a rename
|
||||
meant editing four override rows nobody knew about. That is this decision, reached once by
|
||||
fixing a bug.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **Two modules must be split before they can be absorbed**, and they are the two hardest.
|
||||
`wireguard` needs every node's key, address, site and endpoint reachability; `traefik` needs
|
||||
certificates and every node's exposed names. The measurement says the design is right; it
|
||||
does not say the migration is cheap, and this record does not claim it is.
|
||||
- **The overlay and firewall modules stop existing** as the skeleton says — but the reason is
|
||||
now sharper than "they are host concerns". The host holds membership and applies filtering;
|
||||
the control plane decides policy; swappable backends stay modules.
|
||||
- **Six vocabularies remain.** Zero dependencies, but the host must still know what a WireGuard
|
||||
peer, an nftables rule, a package, a unit, a container and a dataset *are*. That surface is
|
||||
the residue of the original worry and is not measured by anything here.
|
||||
- **A dependency-direction lint is now load-bearing**, not a nicety. This record is a rule
|
||||
about direction, and per this repository's own standard a rule states how it is checked: an
|
||||
upward import fails the build. A tier rule enforced by intention is the same as no tier rule.
|
||||
- **What the host carries versus what it finds is still open.**
|
||||
[Issue 007](../04-ISSUES/007-an-installed-package-is-not-a-capability/00-report.md) — the
|
||||
host manages `wg`, `nft`, `pacman`, `docker`; it does not contain them, and *installed* is
|
||||
not the same as *usable*.
|
||||
|
||||
## References
|
||||
|
||||
- [`host-size.md`](../01-RESEARCH/006-mesh-from-scratch/host-size.md) — the measurement.
|
||||
- [Research 005](../01-RESEARCH/005-domain-grouping/analysis.md) — reachability as the only
|
||||
measured co-change cluster in the catalogue.
|
||||
- [ADR 0003](0003-the-mesh-database-is-the-source-of-truth.md) — what the control plane decides
|
||||
from.
|
||||
- [ADR 0019](0019-how-this-repository-works.md) — `mesh-host` as tier 0.
|
||||
- [ADR 0008](0008-a-failed-step-fails-the-job.md) — the standard the direction lint is held to.
|
||||
@@ -0,0 +1,155 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-28
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
consolidates: [0041, 0043, 0047, 0057, 0060, 0061, 0062]
|
||||
---
|
||||
|
||||
# 37. The node host
|
||||
|
||||
*Consolidated 2026-08-28 from eight records. Tier 0 is one component and was decided over a
|
||||
week; the reasoning is kept, the fragmentation is not.*
|
||||
|
||||
## It applies; it does not decide
|
||||
|
||||
**The host makes a machine match what it was told, and never works out what that should be.**
|
||||
|
||||
This is the line the whole tier rests on, and it is not about privilege — it is about what a
|
||||
single machine can *know*. Deciding needs knowledge the machine does not have: which nodes should
|
||||
run a store, which peers belong in an overlay, whether a node has been unreachable for a week.
|
||||
Anything needing a second node is the control plane's.
|
||||
|
||||
The practical form: **the host never queries the mesh's database and holds no credential to it.**
|
||||
Two modules in the current mesh do, and they are the reason every node permanently carries a
|
||||
database credential.
|
||||
|
||||
## It depends on nothing that must be installed first
|
||||
|
||||
**A statically linked binary. Copy it onto a machine and run it — that is the whole
|
||||
installation.** Written in Go, because the job is system-level and because a runtime that must be
|
||||
installed first would make the host depend on the thing it exists to install.
|
||||
|
||||
**What it needs from the machine is not a dependency in this sense.** An init is not installed;
|
||||
it is what the machine already is. A package manager is the distribution. Those are what a
|
||||
machine *is*, not what must be put on it before the host works.
|
||||
|
||||
## It is built per operating system
|
||||
|
||||
**`systemd` and `pacman` are the Arch host's implementation, not abstractions the mesh grows.**
|
||||
They are not independent choices: a machine has pacman *because* it is Arch, and the package
|
||||
manager, service manager and packaging format arrive together as one decision somebody made at
|
||||
install time.
|
||||
|
||||
```
|
||||
mesh-host-arch pacman · systemctl · a container runtime
|
||||
mesh-host-alpine apk · rc-service
|
||||
mesh-host-android neither — a partial host
|
||||
```
|
||||
|
||||
**Abstracting them was rejected on correctness, not effort.** The service applier reads systemd's
|
||||
`LoadState` to tell *not installed* apart from *stopped* — which is what stops it reporting
|
||||
absence as success — and OpenRC has no equivalent. An interface spanning both must drop it, and
|
||||
the lowest common denominator is exactly where that fault lives.
|
||||
|
||||
**Almost all of it is shared.** The declaration vocabulary, the store, the apply loop, the
|
||||
read-back discipline, the refusal model and the link are portable. Two appliers differ.
|
||||
|
||||
**A host that cannot implement a shape refuses it.** Android has no package manager it may drive
|
||||
and no init it may register with, so it implements `file`, `directory` and `action` and refuses
|
||||
the rest — the same refusal an unknown type gets, with a different reason. Those three are the
|
||||
portable floor, and they are what makes a partial host a real thing rather than a broken one.
|
||||
|
||||
## It is a root service, and it never manages its own unit
|
||||
|
||||
**Root**, because no useful part of the job is unprivileged: it writes under `/etc`, installs
|
||||
packages, manages units and runs containers.
|
||||
|
||||
**It cannot run in a container**, and the reason is decisive rather than stylistic: installing the
|
||||
container runtime is a step of the bootstrap, so a host inside a container would need the thing
|
||||
it exists to install. Everything above tier 0 is a container; the host is not. That split is the
|
||||
tier boundary made concrete.
|
||||
|
||||
**The installation owns the host; the host owns everything else.** It manages `service` resources
|
||||
and its own unit is one — the temptation is obvious and it ends with a host stopping itself half
|
||||
way through an apply, leaving a machine with nothing running to fix it.
|
||||
|
||||
## An init is asked for one thing
|
||||
|
||||
**Start this at boot.** That is all, and every init can express it — systemd, OpenRC, runit, s6.
|
||||
|
||||
**Everything else is a launcher the host ships**, which supervises it: restart it when it exits,
|
||||
count consecutive failures, roll back after too many, halt after that. Policy in a unit file can
|
||||
only be read and hoped for; a script with a counter can be tested, and this is the one piece that
|
||||
must work on a machine where the host does not.
|
||||
|
||||
**The launcher does not exec the host, it supervises it** — so restarting is ours rather than the
|
||||
init's. The cost is signals: a supervisor that exits while its child runs leaves the host to be
|
||||
*killed* rather than to *stop*, and an apply interrupted that way is the half-configured machine
|
||||
this design is about. So it traps the shutdown signal, passes it down, and waits.
|
||||
|
||||
**A clean exit is the upgrade path**, and it is the easiest thing to get wrong — twice now. The
|
||||
host stands aside for a new binary by exiting zero, so anything supervising must restart on a
|
||||
zero exit and must not count it as a failure.
|
||||
|
||||
**Recovery is local, and detection is the mesh's.** Nothing dials a node and a host that cannot
|
||||
start cannot report, so the node must recover itself. But a local supervisor sees one process
|
||||
failing and cannot tell a broken machine from a broken release — only something watching every
|
||||
node can, which is why a host rollout is staged and stops when nodes go quiet.
|
||||
|
||||
## A host may be episodic
|
||||
|
||||
**Resident or episodic, and both are hosts.** A phone has no init to register with and nothing
|
||||
worth supervising, because a supervisor would be killed alongside what it supervises. So it runs
|
||||
when the platform allows and is killed when the platform wants the memory — **and that is
|
||||
disconnection**, which is already an ordinary situation.
|
||||
|
||||
It needs no keep-alive and no new mechanism: the store is already authoritative while
|
||||
disconnected, reconcile already happens on start, and *last heard from* is already reported
|
||||
rather than alarmed on. An episodic host cannot be the first node, because every bootstrap step
|
||||
is a shape it refuses.
|
||||
|
||||
## What a declaration is
|
||||
|
||||
**An ordered list of resources the host owns.** JSON, because Go's standard library carries a
|
||||
JSON parser and no YAML, and the one binary whose argument is that it needs nothing must not
|
||||
gain a parser to buy authoring comfort in a machine-written document.
|
||||
|
||||
**Ordered, because ordering is a decision.** The host does not sort and does not resolve
|
||||
dependencies — that would be deciding, and deciding the thing most likely to differ between what
|
||||
the control plane intended and what the machine does.
|
||||
|
||||
**Every resource has a stable identity** — a name the control plane keeps across declarations, not
|
||||
a position and not a hash of content. It is what lets the store say *this is the same resource I
|
||||
applied last time*, which is what makes removal possible at all.
|
||||
|
||||
**Unknown is refused, whole.** A field the host does not know is something the control plane
|
||||
believes it asked for. A declaration naming one is rejected entirely, naming every problem at
|
||||
once — a host that applied the parts it understood would leave a machine that looks configured
|
||||
and is not.
|
||||
|
||||
**Six shapes:** `file`, `directory`, `service`, `package`, `container`, `action`. Every addition
|
||||
widens what a compromised control plane can express, so the list is a security artefact and grows
|
||||
deliberately.
|
||||
|
||||
### The bundle may carry actions; the link may not
|
||||
|
||||
An `action` runs a command, and the host never learns what it means. It is needed because the
|
||||
bootstrap creates a database before there is any mesh to ask for one, and the host must not learn
|
||||
what a database is.
|
||||
|
||||
**Permitted from the bundle, refused from the link**, and the asymmetry is the whole point: a
|
||||
bundle arrives *with* the binary, so anyone able to put a hostile action there could have put it
|
||||
in the host itself — refusing it buys nothing and costs the bootstrap. The link is a separate
|
||||
party, reachable separately, and an action there is an unbounded blast radius.
|
||||
|
||||
**An action must carry its own verification**, which is also its idempotency check. The host does
|
||||
not know what a database is, so *is it already there* is a question only the declaration can ask.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **The migration is smaller than it looks.** A joining node never needs mesh-wide state — it
|
||||
needs an identity, an address and one peer, and the rest arrives as declarations.
|
||||
- **What is applied is recorded after it works, never before.** A failed apply leaves the machine
|
||||
in whatever state it reached, and nothing must claim otherwise.
|
||||
- **A second operating system is additive**: two appliers and a four-line init file.
|
||||
@@ -1,106 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-25
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0037-the-host-applies-it-does-not-decide.md
|
||||
---
|
||||
|
||||
# 38. A node joins by linking first, and the mesh finishes the job
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0037](0037-the-host-applies-it-does-not-decide.md) settles that the host applies and the
|
||||
control plane decides. That leaves the case where there is no control plane to decide: the
|
||||
first node, which must raise a mesh from nothing, and the second, which must join one.
|
||||
|
||||
Raised by the operator: *"shouldn't the host have two modes — one for the initial node, setting
|
||||
up the mesh, so we know the full state; then when adopting a second node, we enter the mesh
|
||||
early and let our first node take over the mesh-related work? The host should only set up the
|
||||
bare minimum for the other nodes in the mesh to complete adoption."*
|
||||
|
||||
The instinct is right and it is the resolution of the gap ADR 0037 leaves open. The framing
|
||||
needs one correction, and the correction comes from what the mesh already does.
|
||||
|
||||
**Today there are three hand-run shell paths**: `install.d/adopt.sh` (183 lines),
|
||||
a separate first-node bootstrap, and `install.d/rescue.sh`. The skeleton names the cause —
|
||||
*"the first node is raised by a special script that exists only because of the circularity"* —
|
||||
and Move 1 exists to remove it. Three paths that do nearly the same thing, maintained
|
||||
separately, run by hand, outside anything that checks them.
|
||||
|
||||
**That is the two-mode problem, already at its worst.** A decision that gives the host two
|
||||
modes risks rebuilding `adopt.sh` and `bootstrap.sh` inside the binary, where they will drift
|
||||
in exactly the same way and be harder to see.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **Two modes — genesis and join.** What was proposed. Rejected as a *structure* while adopted
|
||||
as an *intent*: two modes is two code paths, the first is exercised once per mesh and the
|
||||
second constantly, so the rarely-run one rots. The current three scripts are the evidence.
|
||||
2. **One path, and the first node is special-cased inside it.** The conditional moves rather
|
||||
than disappearing, and now it is scattered instead of named.
|
||||
3. **One behaviour, two sources of declaration.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
**The host has one behaviour: apply the declaration it has.** What differs between the first
|
||||
node and the fiftieth is not what the host *does* but **where the declaration comes from** —
|
||||
and, exactly as in [ADR 0036](0036-a-node-is-a-managed-machine.md), that is a situation rather
|
||||
than a class.
|
||||
|
||||
| Situation | Declaration comes from |
|
||||
|---|---|
|
||||
| no mesh reachable | the pinned bundle the host carries (`substrate.lock`) |
|
||||
| mesh reachable | the control plane, over the link |
|
||||
|
||||
**The first node is not a different kind of node.** It is a node whose mesh is not up *yet*. It
|
||||
applies the bundle it carries, the control plane comes up on top of it, and from that moment it
|
||||
takes declarations like everything else. Its specialness is temporary and self-erasing, which
|
||||
is the property `adopt.sh` and the bootstrap script do not have.
|
||||
|
||||
**A joining node does the minimum to be reachable, and nothing else.** It establishes identity
|
||||
and a route to the control plane — the `link` — and then stops deciding. Everything after that
|
||||
arrives as declarations.
|
||||
|
||||
**The minimum is deliberately small:** an identity, an address, and one peer to reach. A
|
||||
joining node does **not** compute the overlay. It needs a single peer to reach the mesh; the
|
||||
full peer set is derived centrally and pushed down, like everything else.
|
||||
|
||||
## Why this resolves what 0037 left open
|
||||
|
||||
ADR 0037 records that `wireguard` and `traefik` are the two modules that must be split before
|
||||
they can be absorbed, and that they are the hardest because they need mesh-wide state.
|
||||
|
||||
**A joining node never needs that state.** The hard part of the overlay — every node's key,
|
||||
address, site and endpoint reachability — is only needed to compute the *whole* mesh, which is
|
||||
the control plane's job. The node needs one peer. The rest arrives.
|
||||
|
||||
So the migration ADR 0037 calls expensive is smaller than it looked, and this record is what
|
||||
makes it smaller.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **Adoption stops being a script.** The three hand-run paths collapse into the host: joining
|
||||
is establishing a link, and rescue is a node whose local state is discarded so the mesh can
|
||||
re-derive it. Whether rescue is fully covered by this is not decided here.
|
||||
- **The bundle is a fallback, not a mode.** It is what a host applies when nothing better is
|
||||
available, which also covers a node that has been disconnected for a long time — ADR 0036's
|
||||
ordinary situation.
|
||||
- **The rarely-run path is now the common one.** The first node exercises the same code every
|
||||
other node exercises constantly. That is the whole reason for choosing this over two modes.
|
||||
- **The link becomes the security boundary.** Everything a node applies arrives through it, so
|
||||
what may be pushed, and how a joining node proves it is entitled to join, is its own
|
||||
question — taken up by [ADR 0039](0039-the-link-is-the-security-boundary.md).
|
||||
- **The bundle must be able to raise the substrate alone.** Whether one host can bring up the
|
||||
four pinned services with no mesh present is Move 1 of the skeleton and remains unproven.
|
||||
This record depends on it and does not establish it.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0037](0037-the-host-applies-it-does-not-decide.md) — the split this completes.
|
||||
- [ADR 0036](0036-a-node-is-a-managed-machine.md) — situation rather than class, applied here
|
||||
to the first node.
|
||||
- [Research 006, Move 1](../01-RESEARCH/006-mesh-from-scratch/skeleton.md) — the pinned bundle,
|
||||
and the special script it exists to remove.
|
||||
- [`00-as-is/05-runtime-and-installation.md`](../03-DESIGN/00-as-is/05-runtime-and-installation.md)
|
||||
— how a node comes into being today.
|
||||
@@ -1,190 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-25
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0038-a-node-joins-by-linking-first.md
|
||||
---
|
||||
|
||||
# 39. The link is the security boundary
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0038](0038-a-node-joins-by-linking-first.md) makes the link the one channel a node takes
|
||||
declarations from, and names the gap it leaves: *"everything a node applies arrives through it,
|
||||
so what may be pushed, and how a joining node proves it is entitled to join, is now a question
|
||||
worth its own record."*
|
||||
|
||||
This is that record. It is a design decision about a boundary that does not exist yet — but
|
||||
what it replaces is measured, and that is the argument.
|
||||
|
||||
Settled as: **a node owns no password. It owns an identity, and that identity is what it
|
||||
presents to the broker.**
|
||||
|
||||
### What adoption does today
|
||||
|
||||
`install.d/adopt.sh` asks the operator to paste credentials in by hand:
|
||||
|
||||
```
|
||||
The meshware module needs registry database and minio credentials.
|
||||
REGISTRY_DB_PASSWORD=<postgres password from novox>
|
||||
REGISTRY_MINIO_PASSWORD=<minio password from novox>
|
||||
```
|
||||
|
||||
plus an `NPM_TOKEN` for the private registry. These are not adoption-time credentials that are
|
||||
then discarded: `wireguard` and `traefik` open a `pg` connection on every reconcile
|
||||
([ADR 0037](0037-the-host-applies-it-does-not-decide.md)).
|
||||
|
||||
**So every node permanently holds a credential to the control plane's database, and to the
|
||||
object store.** They are the same credentials on every node. There is no rotation —
|
||||
[`00-as-is/06`](../03-DESIGN/00-as-is/06-configuration-and-secrets.md) records that *"there is
|
||||
no mechanism that rotates one and informs everything holding it. Where a rotation has been
|
||||
done, it has been done by hand, and doing it wrong has taken services down."*
|
||||
|
||||
Compromise of any node is therefore compromise of the mesh's database, and there is no
|
||||
mechanism to recover from it.
|
||||
|
||||
### The link is not new
|
||||
|
||||
Written first as though the link were a thing to build. It is not.
|
||||
[ADR 0001](0001-nodes-communicate-over-a-broker.md) already has it: *every node connects
|
||||
outbound to a single broker; nothing ever connects to a node*, each node declaring an exchange
|
||||
named for itself and consuming from its own queue
|
||||
([`00-as-is/01`](../03-DESIGN/00-as-is/01-mesh-and-transport.md)).
|
||||
|
||||
That is already outbound-only, already per-node addressed, and already the one channel
|
||||
everything arrives through. **This record is not proposing a channel. It is proposing that the
|
||||
channel carry per-node identity instead of one shared credential.**
|
||||
|
||||
The same as-is records the fault, for the broker rather than the database: *"the broker is a
|
||||
single point of failure and a single point of trust. Its credential is mesh-wide, so rotating
|
||||
it is a mesh-wide operation, and doing it wrong has taken the broker down."*
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **Keep shared credentials, scope them per node.** Least change: give each node its own
|
||||
database role. Rejected — it makes the blast radius smaller without changing its shape, and
|
||||
it keeps tier 0 speaking the control plane's schema, which ADR 0037 forbids for reasons that
|
||||
are not about security at all.
|
||||
2. **Accept the exposure as the cost of simplicity.** A shared credential is one thing to
|
||||
understand and nothing to build, and the objection to replacing it is real: mutual
|
||||
authentication fails opaquely, and a node that cannot link is harder to debug than a node
|
||||
with a wrong password. Rejected on the ground that the simplicity is what makes it
|
||||
unrotatable — the credential cannot be changed *because* everything holds the same one, so
|
||||
the arrangement's convenience and its unfixability are the same property.
|
||||
3. **Mutual authority on a node-initiated link, with the node holding nothing but its own
|
||||
identity.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
**The link is the only way anything reaches a node**, and four properties make it a boundary
|
||||
rather than a pipe.
|
||||
|
||||
### It is outbound and node-initiated
|
||||
|
||||
The node dials the control plane. Nothing dials a node. This is not only defensive — it is what
|
||||
the topology already requires: most nodes sit behind a household connection with no forwarded
|
||||
port ([research 004](../01-RESEARCH/004-lab-network/00-overview.md)), so an inbound control
|
||||
channel would work for the hosted node and not for the rest, and the difference would be
|
||||
invisible until it mattered.
|
||||
|
||||
A node therefore has **no listening control surface at all**.
|
||||
|
||||
### A node holds its own identity and nothing else
|
||||
|
||||
No shared secret, no credential to anything it does not own. A node's identity authenticates it
|
||||
to the control plane and grants access to nothing else.
|
||||
|
||||
**Compromise of a node is compromise of that node.** That is the property today's arrangement
|
||||
does not have, and it is the main reason for this record.
|
||||
|
||||
### Authority is mutual
|
||||
|
||||
The node proves it may join, and **the control plane proves it is the mesh**. One-way is not
|
||||
enough here: the host applies whatever the link delivers, so a node that cannot tell the mesh
|
||||
from something impersonating it will apply that something's declarations. Given ADR 0038, an
|
||||
attacker who can answer a joining node's first call owns the machine.
|
||||
|
||||
### What may be pushed is bounded by form, not by trust
|
||||
|
||||
The control plane may push **declarations of known shape** and nothing else. It may not push a
|
||||
command to run. The host's vocabulary is finite, versioned and auditable, and anything outside
|
||||
it is refused rather than best-effort interpreted.
|
||||
|
||||
**Stated honestly: this bounds form, not impact.** A compromised control plane can declare
|
||||
harmful state — a malicious package, an open firewall — and the host will apply it faithfully,
|
||||
because that is what it is for. What the property buys is that the blast radius is describable:
|
||||
it is exactly what the declaration language can express, which can be reviewed. An arbitrary
|
||||
command channel has no such bound. This is a real limit and not a defence-in-depth story.
|
||||
|
||||
### Joining is a deliberate, bounded act
|
||||
|
||||
A joining node presents a **one-time, short-lived enrolment token** issued by the mesh for that
|
||||
purpose, and exchanges it for its own durable identity. The token grants exactly one thing:
|
||||
the right to become a node. It is not a credential to any service, it does not persist after
|
||||
exchange, and it expires whether used or not.
|
||||
|
||||
This replaces hand-carried shared secrets with a thing that is useless once used and useless
|
||||
after a while.
|
||||
|
||||
## What this actually costs
|
||||
|
||||
The objection to weigh is overhead, and it is smaller than it looks because most of it is
|
||||
already running.
|
||||
|
||||
| Property | Where it comes from |
|
||||
|---|---|
|
||||
| outbound, node-initiated | already true — ADR 0001 |
|
||||
| per-node addressing | already true — per-node exchange and queue |
|
||||
| per-node credential | a broker user per node; the broker already has users, virtual hosts and per-queue permissions |
|
||||
| mutual authority | transport-level certificates on a connection that already exists |
|
||||
| bounded by form | already true — three message shapes and only three |
|
||||
| **enrolment** | **the one genuinely new mechanism** |
|
||||
|
||||
And ADR 0037 subtracts rather than adds: under it a node holds **no** database credential at
|
||||
all, so this record replaces three hand-carried shared secrets with one per-node identity that
|
||||
grants only identity.
|
||||
|
||||
**It must fail legibly.** A boundary that refuses a node without saying why is worse than the
|
||||
credential it replaced, because a wrong password at least announces itself. A node that cannot
|
||||
link must report which side rejected it and on what grounds, in terms someone can act on. This
|
||||
is `how-we-build` §5 applied to a security mechanism: a refusal that proves only that something
|
||||
went wrong is transport reported as effect.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **ADR 0037 removes a standing exposure as a side effect.** Its rule — the host never queries
|
||||
the mesh database — was chosen for tier discipline. It also removes the reason every node
|
||||
holds the database password. Worth recording because the two arguments are independent and
|
||||
both hold.
|
||||
- **Rotation becomes possible and is still not designed.** Per-node identities can be revoked
|
||||
individually, which is what makes rotation tractable at all. The mechanism —
|
||||
what rotates, on what trigger, and how holders learn — is **not decided here** and remains
|
||||
the open weakness `00-as-is/06` records.
|
||||
- **The enrolment token has to come from somewhere.** Issuing it is a control-plane operation
|
||||
and the first node has no control plane, so the first node's identity is self-issued and
|
||||
becomes the root of trust when the mesh comes up. **That is a real asymmetry** — the one
|
||||
place ADR 0038's "no special first node" does not fully hold — and it is named here rather
|
||||
than hidden.
|
||||
- **A declaration vocabulary is now a security artefact, not only a design one.** Every
|
||||
addition widens what a compromised control plane can express. That is a reason to keep it
|
||||
small and a reason for additions to be reviewed as such.
|
||||
- **Offline nodes need identities that survive disconnection.** Per
|
||||
[ADR 0036](0036-a-node-is-a-managed-machine.md) disconnection is ordinary, so an identity
|
||||
that must be refreshed to remain valid would make a laptop fail for being a laptop. What
|
||||
expires and what does not is **not decided here**.
|
||||
- **This is a boundary that does not exist yet.** Nothing in the current mesh implements any of
|
||||
it, and the migration from shared credentials to per-node identity touches every node and the
|
||||
substrate. No estimate is offered.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0038](0038-a-node-joins-by-linking-first.md) — the link, and the gap this fills.
|
||||
- [ADR 0037](0037-the-host-applies-it-does-not-decide.md) — why the host stops holding database
|
||||
credentials at all.
|
||||
- [ADR 0036](0036-a-node-is-a-managed-machine.md) — disconnection as ordinary, which constrains
|
||||
what may expire.
|
||||
- [`00-as-is/06-configuration-and-secrets.md`](../03-DESIGN/00-as-is/06-configuration-and-secrets.md)
|
||||
— secrets today, and the absence of rotation.
|
||||
- [Research 004](../01-RESEARCH/004-lab-network/00-overview.md) — why most nodes cannot accept
|
||||
an inbound connection.
|
||||
@@ -1,78 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-26
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0037-the-host-applies-it-does-not-decide.md
|
||||
---
|
||||
|
||||
# 41. The host depends on nothing that must be installed first
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0019](0019-how-this-repository-works.md) calls tier 0 *"the one binary installed by hand"*,
|
||||
and [research 006](../01-RESEARCH/006-mesh-from-scratch/00-overview.md) states the property the
|
||||
whole tier rests on: *"a binary whose whole argument is that it has no dependencies"*.
|
||||
|
||||
Building it forced the question that phrase had been carrying unexamined. Everything else in
|
||||
the mesh is TypeScript, and `how-we-build` §8 says so. A TypeScript host needs a runtime present
|
||||
before it can run — so the thing installed by hand becomes **two** things, and the second must
|
||||
be installed by the means the host exists to replace.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **TypeScript, with a runtime installed first.** Simplest, and matches every other
|
||||
repository. Rejected: it breaks the property the tier is built on. A host that cannot run
|
||||
until something else has been installed by hand is not the bottom of the stack.
|
||||
2. **TypeScript, bundled as a single executable.** Preserves the language and produces one
|
||||
file. Rejected on two grounds: it carries roughly ninety megabytes of runtime to preserve a
|
||||
language choice, and single-executable bundling is a young feature — tier 0 is the worst
|
||||
place in the system to discover its edges.
|
||||
3. **A statically linked binary in a language built for it.** Chosen; Go.
|
||||
|
||||
## Decision
|
||||
|
||||
**The host is a single statically linked binary that requires nothing to be present.** Copy it
|
||||
onto a machine and run it. That is the whole installation.
|
||||
|
||||
**It is written in Go.** The job is system-level — run commands, write files, speak to the
|
||||
firewall, the overlay, the service manager and the package manager — which is what Go's
|
||||
ecosystem is for, and it cross-compiles to every architecture the mesh might reach, including
|
||||
the lighter devices requirement 6 anticipates.
|
||||
|
||||
**The second language costs less here than anywhere else it could appear**, and the reason is
|
||||
architectural rather than convenient. [ADR 0037](0037-the-host-applies-it-does-not-decide.md)
|
||||
means the host never queries the mesh database.
|
||||
[ADR 0039](0039-the-link-is-the-security-boundary.md) means it only ever receives declarations.
|
||||
So the host shares **no code** with any other tier — not a client, not a schema, not the SDK.
|
||||
It is joined to the mesh by a message contract and nothing else.
|
||||
|
||||
The language boundary therefore falls exactly on an architectural boundary that already exists.
|
||||
A second language usually costs duplicated logic; here there is none to duplicate.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **`how-we-build` §8 needs a scope.** It reads *"TypeScript throughout"*, which was true when
|
||||
everything was a service or a surface. It is now scoped to those, with tier 0 named as the
|
||||
exception and this record as the reason. That is a constitution change, and the sync it owes
|
||||
is part of it.
|
||||
- **Agents must write Go to work on the host.** A real cost, and the one genuine argument
|
||||
against this. It is bounded by the host being the only thing in tier 0 — nothing else in the
|
||||
mesh acquires a second language because of this.
|
||||
- **The dependency-direction lint the design calls for gets easier, not harder.** A Go module
|
||||
cannot accidentally import a TypeScript control-plane client; the boundary is enforced by
|
||||
there being no path across it.
|
||||
- **Cross-compilation replaces per-node builds.** The host is built once per architecture and
|
||||
copied, rather than built on the machine it runs on — which is what makes *"copy it and run
|
||||
it"* true rather than nearly true.
|
||||
- **Two toolchains in the lab.** Scenarios that place a host need a Go build available, and the
|
||||
lab is TypeScript. The binary is built before the scenario runs, not inside it.
|
||||
- **This is reversible at a cost that will only grow.** It is being taken at the moment the
|
||||
first line is written, which is the cheapest point it will ever be taken.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0019](0019-how-this-repository-works.md) — *the one binary installed by hand*.
|
||||
- [ADR 0037](0037-the-host-applies-it-does-not-decide.md) — why the host shares no code.
|
||||
- [ADR 0039](0039-the-link-is-the-security-boundary.md) — why it receives declarations only.
|
||||
- [`05-the-node-host.md`](../03-DESIGN/01-to-be/05-the-node-host.md) — the design this serves.
|
||||
@@ -73,5 +73,5 @@ the previous sync reported success and changed nothing.
|
||||
## References
|
||||
|
||||
- [`how-we-build.md`](../00-META/how-we-build.md) §2 — the rule, now carrying this.
|
||||
- [ADR 0008](0008-a-failed-step-fails-the-job.md) — the standard a checkpoint is held to: a
|
||||
- [ADR 0058](0058-delivery.md) — the standard a checkpoint is held to: a
|
||||
step that reports success without doing anything is the fault, not the shortcut.
|
||||
|
||||
@@ -1,140 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-26
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0037-the-host-applies-it-does-not-decide.md
|
||||
---
|
||||
|
||||
# 43. A declaration is an ordered list of resources the host owns
|
||||
|
||||
## Context
|
||||
|
||||
[`05-the-node-host.md`](../03-DESIGN/01-to-be/05-the-node-host.md) leaves *what a declaration
|
||||
is* open and calls it the first thing to settle in build. Stage 2 — applying with no mesh
|
||||
present — cannot start without it.
|
||||
|
||||
Three constraints already bind it, and between them they decide most of the shape:
|
||||
|
||||
- **Data, not instructions**, with a finite, versioned vocabulary and anything outside it
|
||||
refused rather than interpreted ([ADR 0039](0039-the-link-is-the-security-boundary.md)).
|
||||
- **The host applies; it does not decide** ([ADR 0037](0037-the-host-applies-it-does-not-decide.md)).
|
||||
- **The host depends on nothing** ([ADR 0041](0041-the-host-depends-on-nothing.md)).
|
||||
|
||||
## Decision
|
||||
|
||||
### JSON, because the host has no dependencies to spend
|
||||
|
||||
Go's standard library carries `encoding/json` and no YAML. A YAML declaration would put a
|
||||
third-party parser inside the one binary whose entire argument is that it needs nothing — to
|
||||
gain authoring comfort in a document that is, in the ordinary case, generated by a machine and
|
||||
read by a machine.
|
||||
|
||||
The mesh's *authoring* formats stay YAML. What crosses the link is JSON.
|
||||
|
||||
### An ordered list, because ordering is a decision
|
||||
|
||||
A declaration states the order its resources are applied in. The host does not sort, does not
|
||||
resolve dependencies, and does not decide what must come before what.
|
||||
|
||||
This follows from [ADR 0037](0037-the-host-applies-it-does-not-decide.md) more strictly than it
|
||||
first appears. A host that derived ordering from declared dependencies would be **deciding**,
|
||||
and it would be deciding the thing most likely to differ between what the control plane
|
||||
intended and what the machine does. The control plane knows what depends on what; it says so by
|
||||
saying when.
|
||||
|
||||
Consequence accepted: the control plane must order correctly, and a mis-ordered declaration
|
||||
fails at the step that needed something not yet there — which is at least the *right* failure,
|
||||
naming the resource rather than a mystery.
|
||||
|
||||
### Every resource has a stable identity
|
||||
|
||||
Not a position, not a hash of its content: a name the control plane keeps stable across
|
||||
declarations. It is what lets the store say *this is the same resource I applied last time*,
|
||||
which is what makes convergence possible at all.
|
||||
|
||||
### Unknown is refused, never skipped
|
||||
|
||||
An unknown declaration version, an unknown resource type, or an unknown field is a **refusal of
|
||||
the whole declaration**. Not a warning, not a skip, not best-effort.
|
||||
|
||||
A host that skipped what it did not understand would apply most of a declaration and report
|
||||
success — a node that looks configured and is not, which is
|
||||
[04-ISSUES/003](../04-ISSUES/003-firewall-scope-is-read-by-no-code/00-report.md) with the
|
||||
declaration on the other side of the wire. Refusing whole also means an older host cannot be
|
||||
handed a newer vocabulary and quietly do half of it.
|
||||
|
||||
### Complete for what the host owns, and only that
|
||||
|
||||
*Desired state* invites the question of removal, and the honest answer needs a boundary.
|
||||
|
||||
**The host removes what it previously applied and is no longer declared.** It knows what it
|
||||
applied because it recorded it (`store`), so this is a fact it holds rather than an inference.
|
||||
|
||||
**The host never removes anything it did not create.** A machine has things on it that the mesh
|
||||
did not put there, and a converger that treats *not declared* as *must not exist* deletes them.
|
||||
The rule that prevents production data loss elsewhere in this repository is the same one:
|
||||
[ADR 0018](0018-the-mesh-creates-no-symlinks.md) exists because a tool did something to a path
|
||||
it did not own.
|
||||
|
||||
So: authoritative over its own footprint, inert everywhere else.
|
||||
|
||||
### Addressed, and checked when it can be
|
||||
|
||||
A declaration names who it is for. A host that has an identity refuses one addressed elsewhere.
|
||||
A host that has no identity yet — the first node, applying the bundle it carries — has nothing
|
||||
to check against and applies it.
|
||||
|
||||
## Where the list comes from
|
||||
|
||||
This record specifies what the host **accepts**. What produces a declaration is deliberately
|
||||
not settled here, and the reason is worth stating rather than leaving as an omission.
|
||||
|
||||
**Today, and at stage 2: by hand.** `substrate.lock` is authored and pinned — a person writes
|
||||
the resources and writes the order. That is the first node's path, where there is no control
|
||||
plane to derive anything from.
|
||||
|
||||
**Afterwards: the control plane derives it**, from three things it already holds — which
|
||||
modules are assigned to this node, what those modules' configuration resolves to, and what each
|
||||
module declares it needs.
|
||||
|
||||
**And the order comes from the graph.** Each module expands to resources; the modules are
|
||||
ordered by their declared dependencies on one another. That is
|
||||
[research 011](../01-RESEARCH/011-the-module-graph/00-overview.md) — `requires`, `provides`,
|
||||
`excludes` — and a declaration is the graph's output, flattened for one node.
|
||||
|
||||
So this record is complete on the consumer side and silent on the producer side, because the
|
||||
producer does not exist and its shape is what 011 is investigating. The consumer can be settled
|
||||
first because the host must refuse what it does not understand whoever wrote it.
|
||||
|
||||
**What this means for ordering.** [ADR 0037](0037-the-host-applies-it-does-not-decide.md) puts
|
||||
the ordering decision in the control plane; 011 decides how the control plane makes it. If the
|
||||
graph turns out not to determine a total order, that is 011's problem to solve and not the
|
||||
host's — the host will still be handed a list, and will still apply it as given.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **Ordering is now a control-plane responsibility**, and getting it wrong is a class of bug
|
||||
that will appear. It is the correct place for it: the alternative puts a dependency solver in
|
||||
tier 0 and a decision in the wrong tier.
|
||||
- **The vocabulary is a security artefact.** Every type added widens what a compromised control
|
||||
plane can express, so additions are reviewed as such rather than as features.
|
||||
- **Removal is bounded but not free.** A resource dropped from a declaration is deleted on the
|
||||
next apply, so removing a line is an act with an effect — which is the point, and is worth
|
||||
saying out loud because it does not look like one.
|
||||
- **The store becomes load-bearing at stage 2**, earlier than the build order suggests. Nothing
|
||||
can be removed without knowing what was applied, so the record of applied resources arrives
|
||||
with the first apply rather than with the link.
|
||||
- **A closed address space bounds what the first types can be.** A scenario has no route to a
|
||||
package repository, so a declaration whose resources must be fetched cannot be applied in the
|
||||
lab at all. The first vocabulary is therefore what needs no network — files, directories,
|
||||
service state — and packages and containers wait on *where `place:` gets its artifacts from*,
|
||||
which is open in
|
||||
[`02-scenario-declaration.md`](../03-DESIGN/01-to-be/02-scenario-declaration.md).
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0037](0037-the-host-applies-it-does-not-decide.md) — why ordering is not the host's.
|
||||
- [ADR 0039](0039-the-link-is-the-security-boundary.md) — bounded by form.
|
||||
- [ADR 0041](0041-the-host-depends-on-nothing.md) — why JSON.
|
||||
- [ADR 0008](0008-a-failed-step-fails-the-job.md) — why a refusal is whole.
|
||||
@@ -1,126 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-26
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
supersedes: 0017-modules-outside-the-core-are-grouped-by-domain.md
|
||||
---
|
||||
|
||||
# 44. A module declares presence, instantiation and exclusion
|
||||
|
||||
## Context
|
||||
|
||||
[Research 011](../01-RESEARCH/011-the-module-graph/00-overview.md) set out to find the
|
||||
catalogue's missing structure. It opened with *what does a graph delete?* and the answer for the
|
||||
existing system was **nothing — it is already there**: 126 manifests, 103 edges, no cycles,
|
||||
nothing dangling, and a resolver already used for build, load and install order.
|
||||
|
||||
So the work became a design question rather than a discovery one, worked through twenty cases
|
||||
and one provider in full.
|
||||
|
||||
## Decision
|
||||
|
||||
### Two kinds of edge, one graph
|
||||
|
||||
**Presence** — the thing must exist and be reachable. Nothing is created, nothing flows back.
|
||||
*An editor requires a terminal. A dashboard requires a container runtime.*
|
||||
|
||||
**Instantiation** — a provider is asked to make something *for this consumer*, and hands back
|
||||
what the consumer needs to use it. *A game requires a database from the store, and receives one,
|
||||
with credentials.*
|
||||
|
||||
Instantiation implies presence; presence does not imply instantiation. They differ in whether
|
||||
something is created, whether a payload returns, whether it can be revoked, and whether the
|
||||
provider holds state about it — which is too much to collapse into one relation for tidiness.
|
||||
|
||||
### Names are concrete unless providers are genuinely substitutable
|
||||
|
||||
A requirement names either a **concrete** thing — that module and no other — or an **abstract**
|
||||
name satisfied by whatever provides it.
|
||||
|
||||
**An abstract name is legitimate only where a consumer can be switched between providers without
|
||||
changing.** `terminal` passes. `database` fails: a consumer speaking one store's protocol does
|
||||
not speak another's, so the name would promise what no provider delivers and the resolver would
|
||||
report a requirement satisfied that is not.
|
||||
|
||||
**The adapter is what creates an interface.** A name has several providers *and a contract they
|
||||
all satisfy*, or it is not an interface. Nothing declares itself to be one.
|
||||
|
||||
**Where there is no contract there is a tag.** *Database* remains a useful word for finding
|
||||
things and grouping them in a catalogue. Tags describe; edges bind; keeping them apart is what
|
||||
stops a second relationship appearing that looks like a dependency and is not.
|
||||
|
||||
### Exclusion is the third relation, and it is not derivable
|
||||
|
||||
`excludes` names what cannot coexist with this. Two modules providing one name look
|
||||
interchangeable, and two things wanting one port look independent, right up until installing the
|
||||
second breaks the first.
|
||||
|
||||
### A node provides names too
|
||||
|
||||
A node's profile is a set of provided names — a display server, a container runtime, an
|
||||
architecture. A module requiring one is satisfied by **the node**, exactly as one requiring a
|
||||
store is satisfied by another module.
|
||||
|
||||
**So capability checking is not a separate mechanism.** There is one question — *is this name
|
||||
provided by anything available here?* — and a graphical application cannot land on a node
|
||||
without a display server for the same reason, through the same code, that it cannot land
|
||||
without its dependencies.
|
||||
|
||||
This makes the host's capability detection an **input to resolution** rather than a report for a
|
||||
person. And what a node provides is partly *derived*: installing a container runtime makes the
|
||||
node provide `container-runtime` thereafter.
|
||||
|
||||
### Constraints, never placement
|
||||
|
||||
A module says what must be true of a node and never which node. Which node runs what is an
|
||||
inventory decision, and today's catalogue decides it in manifests — a module pinning its store
|
||||
to a named node, so a second node cannot provide it without editing the consumer.
|
||||
|
||||
### Scope decides which provider, and the binding is written down
|
||||
|
||||
A consumer of an instantiation edge declares the **scope of its own need**: one instance shared
|
||||
across every instance of itself, or one each. That decides without naming a node — a shared need
|
||||
cannot be met by something each node runs separately.
|
||||
|
||||
The mesh then binds, and **the binding is recorded and sticky**. Not recomputed: a resolver that
|
||||
re-derives which store serves a consumer will one day derive a different answer and relocate a
|
||||
database. **Where it is recorded follows the scope** — a shared grant belongs to the module, a
|
||||
per-instance grant to the assignment.
|
||||
|
||||
### A module, a node, and an assignment
|
||||
|
||||
Three entities. The assignment carries what belongs to neither end: where state lives, how it is
|
||||
reached, configuration derived from the hardware, and which provider instance serves this
|
||||
consumer.
|
||||
|
||||
Recorded because the alternative was tried: modules were once node-agnostic, and it did not
|
||||
survive — there was nowhere for these to live.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **[ADR 0017](0017-modules-outside-the-core-are-grouped-by-domain.md) is superseded.** Folders
|
||||
assert relationships; edges record them. What grouping was for — finding things, seeing what
|
||||
belongs together — is a tag and a query over the graph, neither of which anybody keeps true by
|
||||
hand. The domain module goes with it: there is no `networking` thing to install, there are
|
||||
concrete modules named individually.
|
||||
- **Provider stops being a category**, as service and application already had. Any hosted thing
|
||||
can be a factory — an identity provider grants clients, a mail server grants mailboxes. It is
|
||||
a facet, not a kind.
|
||||
- **`excludes` and node capabilities do not exist in any manifest today**, so this adds
|
||||
declarations rather than removing them. What it removes is listed in
|
||||
[ADR 0045](0045-a-context-owns-its-store.md), which is the other half of this design.
|
||||
- **A resolver is still needed**, with version constraints and conflicts. What it delegates to
|
||||
the platform's package manager rather than reimplementing is **not decided here**.
|
||||
- **Two axes remain undeclarable**: how many instances a module should have — not derivable, and
|
||||
opposite for a broker and a store — and what a provider hands back, which differs between
|
||||
credentials and a command.
|
||||
|
||||
## References
|
||||
|
||||
- [Research 011](../01-RESEARCH/011-the-module-graph/00-overview.md) — the measurement, the
|
||||
cases, the worked provider, and what happens to `feature`.
|
||||
- [ADR 0045](0045-a-context-owns-its-store.md) — the ownership half.
|
||||
- [ADR 0043](0043-a-declaration-is-an-ordered-list-of-owned-resources.md) — what the graph
|
||||
produces for a node.
|
||||
- [ADR 0002](0002-everything-is-a-module.md) — survives; this says what a module declares.
|
||||
@@ -0,0 +1,111 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-28
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
consolidates: [0002, 0005, 0017, 0054, 0064, 0065]
|
||||
---
|
||||
|
||||
# 44. Modules and the graph
|
||||
|
||||
*Consolidated 2026-08-28 from six records.*
|
||||
|
||||
## Everything is a module
|
||||
|
||||
One kind of thing, one manifest describing all of them. A database, a web application, a window
|
||||
manager and a firewall rule set are all modules — not because they are alike, but because
|
||||
**anything else means a second kind of thing with its own rules, and then a third.**
|
||||
|
||||
**A module is the unit of delivery**: assignable to a node, versionable, replaceable on its own.
|
||||
|
||||
## There are no domain modules
|
||||
|
||||
An earlier decision grouped modules by domain — four things constituting *how a node is
|
||||
reachable* becoming one `networking` module. **That was wrong, and the correction is worth
|
||||
keeping** because the observation behind it was right.
|
||||
|
||||
The measurement holds: reachability is the **only** place in the catalogue where modules
|
||||
genuinely change together under one intent. What did not hold is the conclusion. Tight coupling
|
||||
means they share an **authority** — one place that decides for all of them — and not that they
|
||||
should be one artifact. `wireguard` and the proxy are deployed to different sets of nodes, so a
|
||||
module containing both would be assigned where half of it is unwanted.
|
||||
|
||||
> **Coherence is a context. Delivery is a module.**
|
||||
|
||||
**Folders assert relationships; edges record them.** What grouping was for — finding things,
|
||||
seeing what belongs together — is a tag and a query, neither of which anybody has to keep true by
|
||||
hand.
|
||||
|
||||
## Three edges
|
||||
|
||||
| edge | means | declared? | satisfied |
|
||||
|---|---|---|---|
|
||||
| **presence** | that thing must exist and be reachable here | yes | at provisioning |
|
||||
| **instantiation** | that thing makes something for me and hands back credentials — a database, a bucket, a route | yes | at provisioning, and again whenever it must be |
|
||||
| **build** | I was compiled against that artifact | **no — read from imports** | **at build, once** |
|
||||
|
||||
**Instantiation implies presence; presence does not imply instantiation.**
|
||||
|
||||
**A route is an instantiation edge**, and it is worth noticing because the direction is the mirror
|
||||
of a database: the consumer supplies a target and receives a *name*, rather than supplying nothing
|
||||
and receiving credentials. Same edge.
|
||||
|
||||
**Provider stops being a category.** Any hosted thing can be a factory — an identity provider
|
||||
grants clients, a mail server grants mailboxes. It is a facet, not a kind.
|
||||
|
||||
**A module may also declare exclusion**, because some things cannot coexist on one machine and
|
||||
that is a fact about the module rather than about a particular node.
|
||||
|
||||
### Why the build edge is a different kind
|
||||
|
||||
It is fixed inside an artifact rather than negotiated when something runs, and **its only remedy
|
||||
is a rebuild** — nothing can re-provision it.
|
||||
|
||||
It is also **derived rather than declared**, and the asymmetry is deliberate: a runtime edge is an
|
||||
*intention* somebody has about how the mesh should be wired, and only a person can state it. A
|
||||
build edge is a *fact about code that already exists*, and a declared list of dependencies drifts
|
||||
from the imports it describes.
|
||||
|
||||
**An artifact is out of date when its source moved, or when anything it was built against moved.**
|
||||
So what is recorded is a commit *and the identity of every artifact it was built against*, which
|
||||
is what makes the rebuild set computable and *is this current?* answerable without building.
|
||||
|
||||
**The graph measures design quality, not just build order.** A module with many inbound build
|
||||
edges is one whose every change is expensive — and that is readable before anything is built. The
|
||||
current shared library is exactly that, and nobody could see it because nothing drew the edges.
|
||||
|
||||
## Provisioning is declared, never configured by hand
|
||||
|
||||
A module declares what it **provides** and what it **requires**. The mesh satisfies it: a
|
||||
provisioner belonging to the provider creates the resource and its credential, records the grant,
|
||||
and the values are derived onto the consumer. **Neither the credential nor the topology is ever
|
||||
written by hand.** A requirement may name a provider on another node, so cross-node wiring is the
|
||||
same declaration.
|
||||
|
||||
## The core library is the mesh's domain
|
||||
|
||||
One module everything may depend on. It holds **what is true of the mesh regardless of which
|
||||
context you are in**: a module, a node, an assignment.
|
||||
|
||||
The test: *would this still mean the same thing in a context that had never heard of the one it
|
||||
came from?* A node would. A pipeline stage would not — that is delivery's.
|
||||
|
||||
**Types ship with the module that owns them**, not here. A consumer needing `inventory`'s types
|
||||
depends on `inventory` — one narrow, visible edge — rather than everything depending on a hub
|
||||
where the relationship cannot be seen. **A library everything depends on is expensive to change
|
||||
whether it holds types or code; the fan-in is what makes it expensive**, which is why *types, not
|
||||
behaviour* was the wrong guard.
|
||||
|
||||
**It stays small on its own.** A domain model changes when what the mesh *is* changes, which is
|
||||
rare. A drawer labelled *shared* changes whenever anybody writes something reusable, which is
|
||||
constantly — and *who else might want this* always answers yes, which is how the current one grew.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **Fewer things will be shared, and some code will be written twice.** That is the trade: the
|
||||
current library exists because sharing felt free. Two similar functions in two modules is often
|
||||
the better answer.
|
||||
- **The check is a measurement rather than a prohibition.** Inbound build edges say when something
|
||||
is becoming a hub, while it is happening rather than after.
|
||||
- **Reading build edges needs a language-aware tool per language**, which is the real cost and the
|
||||
reason declaring them looks tempting. It is still wrong.
|
||||
@@ -3,14 +3,14 @@ status: accepted
|
||||
date: 2026-08-26
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0044-a-module-declares-presence-instantiation-and-exclusion.md
|
||||
extends: 0044-modules-and-the-graph.md
|
||||
---
|
||||
|
||||
# 45. A context owns its store, exclusively
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0044](0044-a-module-declares-presence-instantiation-and-exclusion.md) settles what a module
|
||||
[ADR 0044](0044-modules-and-the-graph.md) settles what a module
|
||||
declares. This settles what a grant may be, and it is the half that **removes** things.
|
||||
|
||||
`how-we-build` §4 already says *contexts integrate through the record, never through a shared
|
||||
@@ -45,7 +45,7 @@ modules is the mesh showing its own data, not a boundary crossing. What is forbi
|
||||
|
||||
### Asking or subscribing is derived, not chosen
|
||||
|
||||
[ADR 0036](0036-a-node-is-a-managed-machine.md) makes disconnection an ordinary situation. So:
|
||||
[ADR 0036](0036-a-node-and-how-it-joins.md) makes disconnection an ordinary situation. So:
|
||||
|
||||
- **Anything that must keep working while disconnected cannot ask** — there is nobody to ask. It
|
||||
keeps a local copy, which means subscribing.
|
||||
@@ -79,7 +79,7 @@ The first clear list of what the design deletes rather than adds:
|
||||
- **Three contexts must move out of the registry database**, taking thirteen tables with them.
|
||||
Their dependency on the registry then shrinks to almost nothing — one of them needs a single
|
||||
table.
|
||||
- **The node appliers were already handled.** [ADR 0037](0037-the-host-applies-it-does-not-decide.md)
|
||||
- **The node appliers were already handled.** [ADR 0037](0037-the-node-host.md)
|
||||
stopped the host querying the mesh database for tier reasons unrelated to this, and it removes
|
||||
most of the remaining direct readers as a side effect.
|
||||
- **What a consumer does about events missed while disconnected is not decided** — replay from a
|
||||
@@ -90,4 +90,4 @@ The first clear list of what the design deletes rather than adds:
|
||||
- [`how-we-build.md`](../00-META/how-we-build.md) §4 — the rule this makes enforceable.
|
||||
- [Research 011](../01-RESEARCH/011-the-module-graph/worked-provider.md) — the count, the worked
|
||||
provider, and the dashboard case.
|
||||
- [ADR 0036](0036-a-node-is-a-managed-machine.md) — why asking or subscribing is derived.
|
||||
- [ADR 0036](0036-a-node-and-how-it-joins.md) — why asking or subscribing is derived.
|
||||
|
||||
@@ -1,83 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-26
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0038-a-node-joins-by-linking-first.md
|
||||
---
|
||||
|
||||
# 46. The installer fetches what it pins
|
||||
|
||||
## Context
|
||||
|
||||
Stage 2 of [the node host](../03-DESIGN/01-to-be/05-the-node-host.md) needs to raise the
|
||||
substrate, and a substrate service is a container, and a container needs an image. Where the
|
||||
image comes from had been blocking implementation.
|
||||
|
||||
The blocking version of the question assumed the machine might have no network, which produced a
|
||||
bad trilemma: carry every image inside the artifact, fetch at apply time, or have something else
|
||||
place them first. Carrying them makes a three-megabyte binary into a multi-hundred-megabyte one
|
||||
and strains [ADR 0041](0041-the-host-depends-on-nothing.md).
|
||||
|
||||
**The assumption was wrong, and it came from the lab.** A scenario is a closed address space by
|
||||
design — that is what lets two scenarios hold the same addresses without meeting. Production is
|
||||
not: a machine being adopted has a network, and one that does not is a machine where very little
|
||||
works anyway.
|
||||
|
||||
## Decision
|
||||
|
||||
**The installer fetches what the bundle pins.**
|
||||
|
||||
`substrate.lock` carries **references, not payload** — an image name and a digest. At apply time
|
||||
the host fetches them.
|
||||
|
||||
| Situation | Fetched from |
|
||||
|---|---|
|
||||
| a first node, no mesh yet | upstream, wherever the image ordinarily lives |
|
||||
| every node after that | the mesh's own registry |
|
||||
| **the lab** | **nowhere — the lab places them first** |
|
||||
|
||||
**Pinned by digest, not by tag.** A tag moves; a digest does not. Reproducibility comes from
|
||||
pinning the identity of the thing, not from carrying its bytes — which is what makes fetching
|
||||
acceptable rather than a compromise.
|
||||
|
||||
**The lab is the exception, and it is the lab's problem.** A sealed scenario cannot reach a
|
||||
registry, so the lab places images into a machine the same way it already places the host
|
||||
binary. That is a property of a test environment, and letting it dictate the production design
|
||||
would be the tail wagging the dog.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **[ADR 0041](0041-the-host-depends-on-nothing.md) survives untouched.** *Copy it onto a
|
||||
machine and run it* remains literally true: one binary, a few megabytes, which then fetches
|
||||
what it was told to fetch. The alternative would have quietly redefined the property that
|
||||
decision rests on.
|
||||
- **The bundle stays small and reviewable.** A list of pinned references is something a person
|
||||
can read and check. A bundle containing images is not.
|
||||
- **An apply can fail because something is unreachable**, which a self-contained artifact could
|
||||
not. That is the cost, it is accepted, and it must fail *legibly* — naming what it could not
|
||||
fetch and from where, not "install failed".
|
||||
- **The lab needs a way to place images**, and the machine it places them into needs a container
|
||||
runtime, which a sealed scenario cannot install either. Both are lab-installation concerns and
|
||||
neither is solved here.
|
||||
**The runtime half is now done** — the lab builds a base image on a machine with a network and
|
||||
raises sealed machines from it. **The image half turned out to collide with this record**: an
|
||||
image placed from an archive cannot keep its digest, and this record has the host refuse
|
||||
anything unpinned, so the lab can satisfy neither form. See
|
||||
[04-ISSUES/009](../04-ISSUES/009-a-digest-pinned-image-cannot-be-placed-in-the-lab/00-report.md);
|
||||
the resolution is a registry inside the scenario, which is what a real node pulls from anyway.
|
||||
- **The build-time-versus-apply-time reframing in
|
||||
[research 012](../01-RESEARCH/012-the-minimum-viable-node/00-overview.md) narrows.** It still
|
||||
holds for what a *tailored installer* contains — the missing pieces for a given machine — but
|
||||
it does not have to hold for images, because fetching them is available and cheap. Recorded
|
||||
because the reframing was general and is now qualified.
|
||||
- **Nothing here says what happens when a fetch is impossible on a real node.** An air-gapped
|
||||
machine is not a case the mesh has, and if one appears this decision is what it revisits.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0041](0041-the-host-depends-on-nothing.md) — the property this preserves.
|
||||
- [ADR 0038](0038-a-node-joins-by-linking-first.md) — the bundle this fills in.
|
||||
- [`07-the-substrate.md`](../03-DESIGN/01-to-be/07-the-substrate.md) — what the bundle pins.
|
||||
- [Research 012](../01-RESEARCH/012-the-minimum-viable-node/00-overview.md) — the reframing this
|
||||
qualifies.
|
||||
@@ -1,99 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-26
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0043-a-declaration-is-an-ordered-list-of-owned-resources.md
|
||||
---
|
||||
|
||||
# 47. The bundle may carry actions the link may not
|
||||
|
||||
## Context
|
||||
|
||||
[The substrate](../03-DESIGN/01-to-be/07-the-substrate.md) is raised in five steps, and steps two
|
||||
and three happen before a mesh exists:
|
||||
|
||||
```
|
||||
1 the host applies the bundle the store runs; no mesh exists
|
||||
2 a database is created in it before there is anything to ask
|
||||
3 the control plane's schema applied a migration against that database
|
||||
4 the control plane starts
|
||||
```
|
||||
|
||||
That was recorded as *the sharpest unresolved thing in the bootstrap path*, on the grounds that
|
||||
creating a database inside a running store is not state on a machine.
|
||||
|
||||
**That framing was wrong, and it was reading
|
||||
[ADR 0043](0043-a-declaration-is-an-ordered-list-of-owned-resources.md) too narrowly.** *State
|
||||
on this machine* does not mean the filesystem and the service manager. A service running on this
|
||||
machine is part of this machine. Writing a file and creating a database in a local store differ
|
||||
in mechanism, not in scope.
|
||||
|
||||
The real question was underneath: **must the host learn what a database is?**
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **Give the host a `database` resource type.** Then tier 0 knows Postgres — and next a bucket,
|
||||
a virtual host, a repository. The host acquires the substrate's vocabulary one service at a
|
||||
time, which is what [ADR 0037](0037-the-host-applies-it-does-not-decide.md) exists to stop.
|
||||
Rejected.
|
||||
2. **Have the control plane do all provisioning, including the bootstrap.** Rejected because it
|
||||
is not there yet: at step 2 the thing that would do the provisioning does not exist.
|
||||
3. **The bundle declares an action; the host runs it and reads back.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
**The host runs actions the bundle declares, and never learns what they mean.**
|
||||
|
||||
A bundle step says *run this, against this, and here is how to tell whether it worked*. The host
|
||||
executes it and verifies. What a database is stays knowledge of the module that provides one;
|
||||
the host knows only how to run a declared action against something local and check the result.
|
||||
|
||||
**Actions are permitted in the bundle and forbidden over the link.**
|
||||
|
||||
[ADR 0039](0039-the-link-is-the-security-boundary.md) says what arrives over the link is
|
||||
*declarations of known shape, never a command to run*. That stands, unchanged. The bundle is a
|
||||
different trust path and the asymmetry is deliberate:
|
||||
|
||||
| | the bundle | the link |
|
||||
|---|---|---|
|
||||
| where it comes from | the installer somebody built and copied onto the machine | a remote party |
|
||||
| when it is fixed | at build time, pinned and reviewable | at any moment |
|
||||
| what compromise means | whoever built the installer, who also built the binary | a control plane, which may be compromised separately |
|
||||
| may contain an action | **yes** | **no** |
|
||||
|
||||
The reasoning is that a bundle arrives *with* the binary. Anyone able to put a hostile action in
|
||||
it could equally have put it in the host itself, so refusing actions there buys nothing while
|
||||
costing the bootstrap. The link has no such property: it is a separate party, reachable
|
||||
separately, and an action there is the unbounded blast radius ADR 0039 refuses.
|
||||
|
||||
**And ongoing provisioning is not the host's at all.** Once a mesh exists, a consumer on one
|
||||
node granted a database on another is provisioned by the control plane. The host never performs
|
||||
a provisioning action from the link, so the asymmetry costs it nothing.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **There are two provisioning paths, and that is the answer rather than a problem.** The host
|
||||
runs bootstrap actions locally from the bundle; the control plane provisions across the mesh
|
||||
afterwards. The earlier worry — *one mechanism with a tier boundary inside it* — dissolves,
|
||||
because they are two mechanisms with different actors, scopes and trust models.
|
||||
- **The host's vocabulary grows by one shape, not one per service.** `action` joins `directory`,
|
||||
`file` and `service`. Adding a substrate service does not add a host resource type.
|
||||
- **An action must say how to verify itself.** A step that runs and reports success without a
|
||||
read-back is the fault this project is about, and an action is the easiest place to reintroduce
|
||||
it — so the verification is part of the declaration rather than left to the applier.
|
||||
- **This is the escape hatch [research 011](../01-RESEARCH/011-the-module-graph/features.md)
|
||||
warned about.** Arbitrary code, in the one place it is hardest to remove later. It is bounded
|
||||
by being bundle-only and by requiring its own verification, and that boundary is the whole
|
||||
defence — it should be watched rather than trusted.
|
||||
- **The bundle becomes something a person must be able to read.** If it can run commands, the
|
||||
reason to keep it small and pinned stops being convenience and becomes review.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0043](0043-a-declaration-is-an-ordered-list-of-owned-resources.md) — what a declaration is.
|
||||
- [ADR 0039](0039-the-link-is-the-security-boundary.md) — bounded by form, over the link.
|
||||
- [ADR 0037](0037-the-host-applies-it-does-not-decide.md) — why the host must not learn a
|
||||
service's vocabulary.
|
||||
- [`07-the-substrate.md`](../03-DESIGN/01-to-be/07-the-substrate.md) — the bootstrap order this
|
||||
unblocks.
|
||||
@@ -0,0 +1,130 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-28
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
consolidates: [0003, 0046, 0053, 0055, 0056]
|
||||
---
|
||||
|
||||
# 48. The substrate and the control plane
|
||||
|
||||
*Consolidated 2026-08-28 from six records.*
|
||||
|
||||
## The control plane is what needs to know about more than one node
|
||||
|
||||
That is the whole test, and it follows from the host applying rather than deciding: **deciding
|
||||
needs knowledge a single machine does not have.**
|
||||
|
||||
| question | whose |
|
||||
|---|---|
|
||||
| write this file, with this content, with this mode | the **host** |
|
||||
| which nodes should run the store | the **control plane** |
|
||||
| is this unit running | the **host** |
|
||||
| which peers belong in this node's overlay | the **control plane** |
|
||||
| has this node been unreachable for a week | the **control plane** — nobody else is watching |
|
||||
|
||||
**Anything a single machine could answer alone is not the control plane's.**
|
||||
|
||||
### Seven contexts and one interface
|
||||
|
||||
**inventory, config, connectivity, provisioning, delivery, observability, identity** — plus
|
||||
`api`, the one interface every surface speaks to. Each earns its place by the test above rather
|
||||
than by being ours.
|
||||
|
||||
**`work`, `knowledge` and `stream` are mesh-hosted applications, not control plane.** A task does
|
||||
not need to know a node exists. *Being ours does not make something infrastructure.*
|
||||
|
||||
**Where the record lives is deliberately open.** Contexts integrate through it, which makes it
|
||||
load-bearing, and putting it in the substrate risks recreating the circularity the tiers just
|
||||
removed. Listing it as an eighth context would settle by naming what has not been settled by
|
||||
arguing.
|
||||
|
||||
### One node runs it, and nothing takes over
|
||||
|
||||
**Declared, never elected.** No promotion, no quorum, no fencing, no split brain — none of it
|
||||
built, so none of it can be subtly wrong.
|
||||
|
||||
**That is sound rather than merely cheap**, because the design already tolerates its absence by
|
||||
construction: a node reconciles from its own store and never needed to ask anybody to hold the
|
||||
state it was last given. **The control plane being down is not a new failure mode — it is every
|
||||
node in the ordinary disconnected situation at once.** What is lost is *change*, not *operation*.
|
||||
|
||||
The honest half: this node is a single point of failure, recovery is **restore rather than
|
||||
failover** — which makes backup the availability mechanism rather than hygiene — and
|
||||
**certificate renewal is the clock.** An outage outlasting a renewal window expires every public
|
||||
name, which turns an inconvenience into an outage on a timer. Nothing measures that today.
|
||||
|
||||
## The authority is the control plane, not a database
|
||||
|
||||
**There is no single mesh database.** Each context owns its store exclusively, and *the mesh
|
||||
database* names a thing that will not exist.
|
||||
|
||||
**No node reads any of them** — not for writes, not for reads. A node is *told* what to own, over
|
||||
the link, in a bounded vocabulary; it **states** what it applied, and the owning context writes.
|
||||
The difference is the security boundary: something that can write cannot be prevented from
|
||||
writing anything.
|
||||
|
||||
**A node runs from its own store always, not as a fallback.** The current arrangement's nastiest
|
||||
property is that *a node running from cache looks identical to a node running from the database*,
|
||||
with no age on the cache and nothing reporting divergence. Under this there is no second mode to
|
||||
be mistaken for the first.
|
||||
|
||||
**What survives from the original decision:** the repository defines what exists, the mesh defines
|
||||
what runs where, and no node-to-module mapping is ever committed. That is what makes the
|
||||
repositories node-agnostic and why anything about the mesh can be published at all.
|
||||
|
||||
**The error underneath was a category error**: *source of truth* named a storage location when it
|
||||
meant an **authority**. Once the store is the answer, *which database* becomes the question, and
|
||||
shared schemas follow.
|
||||
|
||||
## The substrate is what the control plane consumes and cannot grant itself
|
||||
|
||||
Every module needing a database asks provisioning for one. The control plane needs a database too
|
||||
and cannot ask itself, because it is not running yet. **That circularity is the definition**, and
|
||||
anything on the wrong side of it is raised from the bundle the host carries.
|
||||
|
||||
| role | product | |
|
||||
|---|---|---|
|
||||
| relational store | **PostgreSQL** | its own state lives there |
|
||||
| message bus | **LavinMQ** | it cannot grant itself a virtual host |
|
||||
| object store | **MinIO** | it cannot grant itself a bucket |
|
||||
| image registry | **an OCI registry** | it cannot grant itself a repository |
|
||||
| identity provider | — | **conditional**: substrate only if the control plane delegates authentication, which is undecided |
|
||||
|
||||
**The role and the product are both written.** The role is what the argument turns on; the product
|
||||
is what gets installed and pinned, and a design that names only the role does not record that the
|
||||
choice was made. **The dependency is on the protocol** — AMQP, S3, OCI — which is what keeps
|
||||
naming them safe. The store is the exception: the provisioning model uses databases, roles and
|
||||
schemas as PostgreSQL means them.
|
||||
|
||||
**A container runtime is detected, not chosen** — docker or podman, because a machine that
|
||||
already has one keeps it. Only the version probe differs between them; the behavioural difference
|
||||
(podman has no daemon, so containers do not return after a reboot unless a unit is enabled)
|
||||
belongs in the declaration rather than the host.
|
||||
|
||||
**Being substrate and being in the bundle are different questions.** Only PostgreSQL must precede
|
||||
the control plane; the rest are substrate by role and ordinary by delivery, provisioned once
|
||||
there is a control plane to do it.
|
||||
|
||||
## The installer fetches what it pins
|
||||
|
||||
`substrate.lock` carries **references, not payload** — an image name and a **digest**, fetched at
|
||||
apply time. A tag moves; a digest does not, and reproducibility comes from pinning the identity of
|
||||
a thing rather than carrying its bytes.
|
||||
|
||||
The assumption that a machine might have no network came from the lab and was wrong: a machine
|
||||
being adopted has one, and the sealed case is the lab, which places images itself.
|
||||
|
||||
**Its contents are per operating system** even though its mechanism is not — package names, unit
|
||||
names and service names all differ, so an Arch host embeds an Arch bundle.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **The bundle stays small and reviewable.** A list of pinned references is something a person can
|
||||
read; a bundle containing images is not.
|
||||
- **An apply can fail because something is unreachable**, which a self-contained artifact could
|
||||
not. That must fail *legibly*, naming what could not be fetched and from where.
|
||||
- **Cross-context reporting is harder, and that is the point.** Anything wanting to see across
|
||||
contexts consumes their events or calls their interfaces.
|
||||
- **A queue with no limit grows until the broker's disk is full**, and the broker is what every
|
||||
node depends on. The bound is per queue and is not decided.
|
||||
@@ -1,106 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-27
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0019-how-this-repository-works.md
|
||||
---
|
||||
|
||||
# 48. The substrate is named
|
||||
|
||||
## Context
|
||||
|
||||
[The substrate](../03-DESIGN/01-to-be/07-the-substrate.md) is defined by a test — *what the
|
||||
control plane consumes and cannot grant itself* — and the design layer describes its members
|
||||
entirely by role: a relational store, a message bus, an object store, an image registry.
|
||||
|
||||
**No design document names a product.** Postgres appears in zero of them. The names occur only
|
||||
in the as-is layer and in research, describing what already runs.
|
||||
|
||||
That is a gap rather than a discipline. The rule it came from —
|
||||
[`01-RESEARCH`](../01-RESEARCH/README.md)'s *research never identifies the mesh it observed* —
|
||||
is about node names and domains, not about software. Nothing is protected by declining to write
|
||||
*Postgres* in a public repository, and something is lost: a design that never names a product
|
||||
does not record that the choice was made.
|
||||
|
||||
Two costs, both already accrued:
|
||||
|
||||
- **`substrate.lock` cannot be written from the design.** It pins images by digest, and a digest
|
||||
belongs to a named image.
|
||||
- **A reader cannot tell a settled choice from an unexamined one.** "A relational store" reads
|
||||
the same whether the store was chosen deliberately or never considered.
|
||||
|
||||
## Decision
|
||||
|
||||
**The substrate is named, and the names are these:**
|
||||
|
||||
| Role | Product | Why |
|
||||
|---|---|---|
|
||||
| relational store | **PostgreSQL** | In use, understood, and the provisioning model already assumes its notions of database, role and schema. |
|
||||
| message bus | **LavinMQ** | In use, speaks AMQP, which is what [ADR 0001](0001-nodes-communicate-over-a-broker.md) assumes. Interchangeable with other AMQP brokers at the protocol level, which is what makes it a safe choice rather than a locked-in one. |
|
||||
| object store | **MinIO** | In use, speaks the S3 protocol, which is the closest thing to a portable object-store interface. |
|
||||
| image registry | **the OCI distribution registry** | In use, and the format is the standard rather than a vendor's. |
|
||||
| container runtime | **Docker or Podman** — *detected, not chosen* | See below. The other four rows name one product; this one names two, and the difference is the point. |
|
||||
|
||||
### Outside the substrate
|
||||
|
||||
The gap is not only the substrate's. The design layer names roles for these too, and the same
|
||||
correction applies — a role is a legitimate abstraction, but the product belongs beside it:
|
||||
|
||||
| Role, as the design says it | Product | Tier |
|
||||
|---|---|---|
|
||||
| **the forge** | **Gitea** | a hosted workload — the mesh builds from it but does not need it to run |
|
||||
| **the coordinator** | the mesh's own pipeline | tier 2 — part of the control plane, not a product |
|
||||
| **ingress** — *exposure*, *certificates* | **Traefik** | not substrate — [ADR 0049](0049-a-route-is-a-grant.md) |
|
||||
|
||||
**Ingress was a real gap rather than a naming one**, and it is closed by
|
||||
[ADR 0049](0049-a-route-is-a-grant.md): applying the same test shows it is **not** substrate, and
|
||||
a route is an ordinary grant. Recorded here because finding it was the point — naming the
|
||||
products is what made the unnamed role visible.
|
||||
|
||||
**The container runtime is the one row that is not a choice at all**, and it stopped being one
|
||||
after this record was written ([ADR 0060](0060-the-host-is-built-per-operating-system.md)). The
|
||||
host detects what the machine has and uses it, because adoption keeps a machine's existing
|
||||
configuration rather than replacing it — so naming a single runtime here contradicted a rule
|
||||
already decided. Both are supported, checked against a real podman: only the version probe
|
||||
differs, and one behavioural difference (podman has no daemon, so containers do not return after
|
||||
a reboot unless `podman-restart.service` is enabled) belongs in the declaration rather than the
|
||||
host.
|
||||
|
||||
**Identity is deliberately absent.** Whether an identity provider is substrate at all depends on
|
||||
whether the control plane delegates authentication, which is undecided
|
||||
([`07-the-substrate.md`](../03-DESIGN/01-to-be/07-the-substrate.md)). Naming a product before
|
||||
deciding whether the role exists would be the mistake this record is correcting, in reverse.
|
||||
|
||||
**The role and the product are both written.** A design says *the relational store (PostgreSQL)*
|
||||
rather than one or the other. The role is what the argument turns on; the product is what gets
|
||||
installed, and a reader needs both.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **`substrate.lock` becomes writable.** It pins named images by digest, which was impossible
|
||||
while the design refused to say which images.
|
||||
- **Continuity is the argument, and it is a real one.** Every choice here is what already runs.
|
||||
Nothing was re-litigated, because nothing about the new shape gives a reason to — and changing
|
||||
a substrate service is a migration of the mesh's own state, which is not a cost to pay for
|
||||
novelty.
|
||||
- **Protocols, not products, are what the design depends on.** The bus is reached over AMQP, the
|
||||
object store over S3, the registry over the OCI protocol. Replacing a product is then a
|
||||
substrate migration rather than a redesign — which is the property that makes naming them safe
|
||||
rather than a commitment that cannot be revisited.
|
||||
- **The relational store is the exception**, and it should be said. The provisioning model uses
|
||||
databases, roles and schemas as Postgres means them, and
|
||||
[ADR 0044](0044-a-module-declares-presence-instantiation-and-exclusion.md) already records that
|
||||
two stores from different vendors are not substitutable for a consumer. Replacing it is not a
|
||||
swap.
|
||||
- **The rule that caused this is narrowed, not repealed.** Research still does not identify the
|
||||
mesh it observed — node names, domains, addresses. Product names were never in scope, and the
|
||||
over-application cost the design layer its concreteness.
|
||||
|
||||
## References
|
||||
|
||||
- [`07-the-substrate.md`](../03-DESIGN/01-to-be/07-the-substrate.md) — the test these satisfy.
|
||||
- [ADR 0001](0001-nodes-communicate-over-a-broker.md) — why the bus speaks AMQP.
|
||||
- [ADR 0046](0046-the-installer-fetches-what-it-pins.md) — pinning by digest, which needs a name.
|
||||
- [ADR 0044](0044-a-module-declares-presence-instantiation-and-exclusion.md) — why the store is
|
||||
the one that cannot simply be swapped.
|
||||
@@ -1,153 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-27
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0044-a-module-declares-presence-instantiation-and-exclusion.md
|
||||
---
|
||||
|
||||
# 49. A route is a grant
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0048](0048-the-substrate-is-named.md) named the substrate's products and, in doing so,
|
||||
found a hole rather than a naming problem: the `connectivity` context lists *exposure* and
|
||||
*certificates* among its responsibilities, and **no document says what terminates TLS, how a
|
||||
public name reaches a container, or which tier owns any of it.**
|
||||
|
||||
Traefik is what does it today, and how it does it is the problem.
|
||||
[Research 006](../01-RESEARCH/006-mesh-from-scratch/host-size.md) counted it as one of only two
|
||||
modules that **open a direct Postgres connection to the control plane's database** — reading
|
||||
`nodes` and `mesh_ca` and computing its own configuration from them.
|
||||
|
||||
That is three violations in one module:
|
||||
|
||||
- **[ADR 0037](0037-the-host-applies-it-does-not-decide.md)** — it decides, on the node, from
|
||||
mesh-wide knowledge.
|
||||
- **[ADR 0045](0045-a-context-owns-its-store.md)** — it reads another context's tables directly.
|
||||
- **[ADR 0039](0039-the-link-is-the-security-boundary.md)** — it is the reason every node
|
||||
permanently holds a credential to the control plane's database.
|
||||
|
||||
So exposure was never designed; it was accreted, and it is one of the two things standing
|
||||
between the current arrangement and the security boundary ADR 0039 describes.
|
||||
|
||||
## Decision
|
||||
|
||||
### Ingress is not substrate
|
||||
|
||||
The test from [`07-the-substrate.md`](../03-DESIGN/01-to-be/07-the-substrate.md), applied
|
||||
honestly:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Does the control plane need a route to **start**? | **No.** It listens locally. |
|
||||
| Does a **node** need one to reach it? | **No** — the node dials out over AMQP, and has no listening control surface at all ([ADR 0039](0039-the-link-is-the-security-boundary.md)). |
|
||||
| Does anything need one **before the control plane runs**? | **No.** |
|
||||
|
||||
**So ingress is an ordinary provider module**, provisioned like anything else once a mesh exists.
|
||||
|
||||
The strongest objection deserves stating rather than dodging: the control plane's `api` is the
|
||||
one interface every surface speaks to, so eventually it *does* want a public name. But *wanting
|
||||
one later* is not *needing one to start* — that distinction is the entire substrate test, and
|
||||
ingress is on the ordinary side of it. At bootstrap the first node's surface is reached locally,
|
||||
and the mesh grants itself a route afterwards, the same way it grants itself a bucket.
|
||||
|
||||
### A route is an instantiation edge
|
||||
|
||||
A module that must be reachable declares it needs a **route**, and the proxy module provides
|
||||
one. This is [ADR 0044](0044-a-module-declares-presence-instantiation-and-exclusion.md)'s
|
||||
instantiation edge with no extension: a provider makes something for a consumer and hands back
|
||||
an identity.
|
||||
|
||||
The direction is worth noticing, because it is the mirror of a database and could be mistaken
|
||||
for a different kind of thing:
|
||||
|
||||
| | a database grant | a route grant |
|
||||
|---|---|---|
|
||||
| consumer supplies | nothing | where to send traffic |
|
||||
| consumer receives | credentials | **the public name it is reachable at** |
|
||||
|
||||
Both are still *the provider made something and told the consumer how to use it*, which is what
|
||||
the edge means. Nothing new is required.
|
||||
|
||||
### Exposure is three facts at two scopes, and that is why it is a context
|
||||
|
||||
The reason this cannot simply live on the node:
|
||||
|
||||
| The fact | Scope | Whose |
|
||||
|---|---|---|
|
||||
| the public name resolves to an address | **mesh** — which node is publicly reachable | the **control plane**, `connectivity` |
|
||||
| a certificate valid for that name exists | **mesh** — issued once, for a name, used on one node | the **control plane**, `connectivity` |
|
||||
| the proxy maps that name to that container | **node** | the **host**, applying a declaration |
|
||||
|
||||
Two of the three need to know about more than one node, which is exactly
|
||||
[the control plane's definition](../03-DESIGN/01-to-be/06-the-control-plane.md). The third is a
|
||||
single machine's business. The line falls where the tier rule already puts it, and the current
|
||||
arrangement is wrong precisely because Traefik does all three on the node.
|
||||
|
||||
### The proxy reads files; it does not read the mesh
|
||||
|
||||
The connectivity context computes the proxy's configuration and the certificate, and they arrive
|
||||
over the link as **`file` resources**.
|
||||
|
||||
**This costs zero new host vocabulary.** `file` already exists — it was one of the first three
|
||||
shapes built. The proxy becomes a `container` with `file` configuration, which the host already
|
||||
knows how to apply and read back.
|
||||
|
||||
And it removes a database credential from every node, which is half of what
|
||||
[ADR 0039](0039-the-link-is-the-security-boundary.md) is for.
|
||||
[ADR 0037](0037-the-host-applies-it-does-not-decide.md) decided this in principle; **neither
|
||||
offender has actually been changed**, and this applies it to one of the two.
|
||||
|
||||
### A node without a public address is routed through one that has
|
||||
|
||||
Most nodes sit behind a connection with no forwarded port
|
||||
([research 004](../01-RESEARCH/004-lab-network/00-overview.md)), so exposure cannot assume the
|
||||
workload's node is reachable. Two cases, and the mesh must handle both because the difference is
|
||||
invisible until it matters:
|
||||
|
||||
- **the node is publicly reachable** — the proxy runs there and the route is direct;
|
||||
- **it is not** — a publicly reachable node proxies to it across the overlay.
|
||||
|
||||
Which case applies is a mesh-level fact, which is the fourth reason exposure is control-plane
|
||||
work. Research 004 already records the hard variant — *a node that is publicly named but sits
|
||||
behind NAT* — as the case the lab exists to get right.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **Traefik's upward dependency is removed, and the module mostly disappears.** It computed its
|
||||
own configuration; now it is an image plus files somebody else derived. Of the 426 lines
|
||||
research 006 counted, what survives is a declaration.
|
||||
- **One of the two direct database connections goes.** `wireguard` is the other and is *not*
|
||||
yet handled — [ADR 0050](0050-reachability-is-a-property-of-the-address.md) is what handles it,
|
||||
and only both together close the set ADR 0039 identified. Until then a node still holds the
|
||||
credential, so this record alone changes the design and not the exposure.
|
||||
- **Certificate issuance becomes a control-plane responsibility with a real constraint**: an
|
||||
ACME challenge can only be answered at a publicly reachable address, so issuance happens
|
||||
through a public node regardless of where the workload runs. The lab already runs its own
|
||||
issuer, so this is testable ([research 004](../01-RESEARCH/004-lab-network/00-overview.md)).
|
||||
- **The proxy is a presence edge for anything exposed.** A module with a route needs the proxy on
|
||||
its node — ordinary ADR 0044 vocabulary, no special case.
|
||||
- **This does not settle the mesh's internal CA.** `mesh_ca` is the *other* thing Traefik reads,
|
||||
and it belongs to a different question: ADR 0039 requires the control plane to prove it is the
|
||||
mesh, which needs something a joining node can verify before it trusts anything. **Internal
|
||||
identity and public exposure are two certificate stories and this record only closes the
|
||||
second.** Conflating them is what made the gap hard to see.
|
||||
**Since closed** by [ADR 0051](0051-the-enrolment-token-carries-the-mesh.md): the joining node
|
||||
verifies the control plane against a fingerprint carried in its enrolment token, so nothing
|
||||
needs the CA before membership.
|
||||
- **Nothing says how a route is revoked** when a module is unassigned. The grant model implies
|
||||
it — removing a consumer drops what it was granted
|
||||
([ADR 0045](0045-a-context-owns-its-store.md)) — but a stale public name pointing at nothing is
|
||||
a more visible failure than a stale database, and it is not designed.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0044](0044-a-module-declares-presence-instantiation-and-exclusion.md) — the edge a route is.
|
||||
- [ADR 0037](0037-the-host-applies-it-does-not-decide.md) — why the proxy may not decide.
|
||||
- [ADR 0039](0039-the-link-is-the-security-boundary.md) — the credential this removes.
|
||||
- [ADR 0048](0048-the-substrate-is-named.md) — which named this gap and left it open.
|
||||
- [Research 006](../01-RESEARCH/006-mesh-from-scratch/host-size.md) — the count and the two
|
||||
offenders.
|
||||
- [Research 004](../01-RESEARCH/004-lab-network/00-overview.md) — the topology and the lab's
|
||||
own issuer.
|
||||
@@ -0,0 +1,124 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-28
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
consolidates: [0050, 0052]
|
||||
---
|
||||
|
||||
# 49. Connectivity
|
||||
|
||||
*Consolidated 2026-08-28 from three records. Overlay, resolution, exposure, filtering and
|
||||
certificates are one design.*
|
||||
|
||||
## Why it is control-plane work
|
||||
|
||||
Apply the test — *everything that needs to know about more than one node* — and not one of the
|
||||
five can be answered by a machine on its own:
|
||||
|
||||
| | needs to know |
|
||||
|---|---|
|
||||
| **overlay** — who peers with whom | every node, and which can be dialled |
|
||||
| **resolution** — which name is which node | every node |
|
||||
| **exposure** — which public name reaches which container | which node is publicly reachable |
|
||||
| **filtering** — which port is open, to whom | what is assigned here, and the overlay's shape |
|
||||
| **certificates** — who may present which name | which name belongs to which node |
|
||||
|
||||
That is exactly what the current arrangement gets wrong, by computing all five on the node from a
|
||||
direct database connection. Two modules do this, and they are the only two left holding a
|
||||
credential to the control plane's database.
|
||||
|
||||
**The shape of the fix, once for all five:** the connectivity context computes the configuration;
|
||||
it arrives over the link as `file` resources; the service reads files and knows nothing about the
|
||||
mesh. **This costs no new host vocabulary.**
|
||||
|
||||
## A route is a grant
|
||||
|
||||
**Ingress is not substrate.** The control plane does not need a route to start — it listens
|
||||
locally — and no node needs one to reach it, because the node dials out and has no listening
|
||||
control surface. It grants itself a route afterwards, the way it grants itself a bucket.
|
||||
|
||||
The strongest objection deserves stating: the `api` is the one interface every surface speaks to,
|
||||
so eventually it *does* want a public name. But **wanting one later is not needing one to
|
||||
start**, and that distinction is the entire substrate test.
|
||||
|
||||
**A module that must be reachable declares it needs a route; the proxy provides one.** Ordinary
|
||||
instantiation, with the direction mirrored — the consumer supplies a target and receives a name.
|
||||
|
||||
**Exposure is three facts at two scopes**, which is why it cannot live on the node:
|
||||
|
||||
| the fact | scope |
|
||||
|---|---|
|
||||
| the public name resolves to an address | **mesh** — which node is publicly reachable |
|
||||
| a certificate valid for that name exists | **mesh** — issued once, used on one node |
|
||||
| the proxy maps that name to that container | **node** |
|
||||
|
||||
**A node without a public address is proxied by one that has**, across the overlay. Most nodes sit
|
||||
behind a connection with no forwarded port, so exposure cannot assume the workload's node is
|
||||
reachable.
|
||||
|
||||
## Reachability is declared, not inferred
|
||||
|
||||
The overlay's peer graph is computed from whether a node can be dialled, and that was inferred
|
||||
from a regular expression over the address. **The address is evidence of reachability; it is not
|
||||
the fact**, and the gap has already cost:
|
||||
|
||||
| address | the regex says | actually |
|
||||
|---|---|---|
|
||||
| `100.64.0.0/10` — carrier-grade NAT | **public** | **not reachable.** An endpoint is written to an address nothing can reach |
|
||||
| any IPv6 address | public | the test is v4 shapes only |
|
||||
| a routable address behind a closed firewall | public | not reachable |
|
||||
| a documentation range standing in for a public segment | private | reachable — this is the lab bug |
|
||||
|
||||
**A test environment having to choose its addresses to satisfy a regex is the regex telling us it
|
||||
is not a fact.**
|
||||
|
||||
So: **an endpoint, or none** — declared. And **the hub is declared, never derived from an address
|
||||
prefix**, because an election decided by the first four characters of an address fails silently,
|
||||
cannot be queried, and makes a renumbering an outage.
|
||||
|
||||
**The address remains evidence and stops being the fact.** Where an observed endpoint disagrees
|
||||
with a declared one, the disagreement is a **reportable condition**, not a silent correction.
|
||||
|
||||
**What does not change** is the lesson underneath: role does not imply reachability — a
|
||||
home-hosted node is a server that cannot be dialled. This keeps that and stops encoding it as a
|
||||
pattern match.
|
||||
|
||||
## A filter rule names its source
|
||||
|
||||
`scope: public` is declared in five manifests, is part of no rule type, and is **referenced by no
|
||||
code**. So five manifests appear to restrict a port and restrict nothing — on the modules most
|
||||
worth restricting.
|
||||
|
||||
**A rule names its source. `from:` is the only way to scope one, and a rule without one is open**
|
||||
— which it must say plainly rather than appear to deny.
|
||||
|
||||
**`scope:` is removed rather than implemented**, because giving it meaning would leave two ways to
|
||||
express one thing. And the general fix is that **an unknown key is refused**: the host's
|
||||
declaration parser already works this way, and manifests are the layer where that discipline is
|
||||
missing. `scope:` survived because nothing rejected it, and it spread by copying to five
|
||||
manifests.
|
||||
|
||||
## Order, and what it costs
|
||||
|
||||
**The link runs on the underlay and never on the overlay.** The overlay is configured by the mesh,
|
||||
so a link requiring it could never be established on a new node.
|
||||
|
||||
**The first declaration is the overlay and nothing else** — because a node's address and peers are
|
||||
*assigned* so it cannot come earlier, and because it is the way back in. A node reachable over the
|
||||
overlay can be fixed by hand if a later declaration breaks the machine; **a large first
|
||||
declaration risks a node that is broken and unreachable at once.**
|
||||
|
||||
**Reachable is not the same as having a control surface.** Every node reaches every other over the
|
||||
overlay — SSH, services, ordinary traffic — and every node consumes from the broker. What is
|
||||
forbidden is a listening thing that accepts instructions and changes the machine.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **The last two direct database connections leave the nodes**, and with them the database
|
||||
credential every node carries.
|
||||
- **The `/etc/hosts` floor goes**, along with the bootstrap circularity it patched.
|
||||
- **Two certificate authorities stay separate on purpose**: a public one for public names, the
|
||||
mesh's own for internal ones. A single-CA lab would hide any bug living in the split.
|
||||
- **What happens when the hub is down**: nothing takes over. Non-co-located paths stop; co-located
|
||||
peers and every assigned workload keep running.
|
||||
@@ -1,119 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-27
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0016-the-lab.md
|
||||
---
|
||||
|
||||
# 50. Reachability is declared, not inferred from an address
|
||||
|
||||
## Context
|
||||
|
||||
The overlay's peer graph is computed on each node by the `wireguard` module, which decides —
|
||||
per pair — whether to write an `Endpoint` for a peer. The rule
|
||||
([research 004](../01-RESEARCH/004-lab-network/analysis.md)) is a regular expression:
|
||||
|
||||
```js
|
||||
const isPrivate = (a) => /^(10\.|127\.|192\.168\.|172\.(1[6-9]|2\d|3[01])\.)/.test(a);
|
||||
```
|
||||
|
||||
Not private ⇒ assumed reachable ⇒ an `Endpoint` is written. Private ⇒ no endpoint, and the peer
|
||||
must initiate.
|
||||
|
||||
The module's own comments record what this cost to arrive at: testing `profile === "server"`
|
||||
was tried and was wrong, because a home-hosted node **is** a server and is not publicly
|
||||
reachable — *"role does not imply reachability; the address does."*
|
||||
|
||||
**That lesson is right and the implementation of it is not.** The address is evidence of
|
||||
reachability; it is not the fact itself, and the gap between the two has already caused failures
|
||||
and will cause more:
|
||||
|
||||
| Address | The regex says | Actually |
|
||||
|---|---|---|
|
||||
| `100.64.0.0/10` — carrier-grade NAT ([RFC 6598](https://www.rfc-editor.org/rfc/rfc6598)) | **public** | **not reachable.** An endpoint is written to an address nothing can reach. |
|
||||
| any IPv6 address, including `fd00::/8` unique-local | **public** | unique-local is not reachable; the regex tests v4 shapes only |
|
||||
| a routable address behind a closed firewall | public | not reachable |
|
||||
| `10.200.0.0/24` standing in for a public segment | private | reachable — this is the lab bug |
|
||||
|
||||
The CGNAT row is the serious one. A node on a carrier-grade NAT address presents exactly the
|
||||
failure already recorded for the hairpin case: *1.77 MiB sent, 0 B received, no handshake.* The
|
||||
mesh silently never forms, and it presents as a WireGuard fault rather than an addressing one.
|
||||
|
||||
The lab row is the same bug seen from the other side. [Research 004](../01-RESEARCH/004-lab-network/00-overview.md)
|
||||
calls the required substitution *"the single most important fact in this document"* — a
|
||||
simulated public segment must use TEST-NET-3, or nothing can ever initiate. **A test environment
|
||||
having to choose its addresses to satisfy a regex is the regex telling us it is not a fact.**
|
||||
|
||||
There is a second inference in the same code, and it is worse because nothing records it:
|
||||
|
||||
> **Hub election is by convention.** The hub is the node whose `profile='server'` *and* whose
|
||||
> overlay address begins `10.10.0.1`. A lab must assign that address to the node it intends as
|
||||
> hub **or there will be no hub** — and nothing says so.
|
||||
|
||||
An election decided by the first four characters of an address is not an election. It fails
|
||||
silently, it cannot be queried, and it makes a renumbering into an outage.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **Fix the regex.** Add CGNAT, add IPv6, add the ranges as they are discovered. Rejected: the
|
||||
list is unbounded, and each addition is written after the outage that revealed it. The
|
||||
firewall case cannot be fixed at all — no address shape encodes it.
|
||||
2. **Probe for reachability and cache the answer.** Attractive, and wrong as the *primary*
|
||||
source: at the moment the graph is computed a node may be legitimately down, and a probe
|
||||
cannot distinguish *unreachable* from *asleep* ([ADR 0036](0036-a-node-is-a-managed-machine.md)).
|
||||
Deriving topology from a liveness check makes the overlay flap with the network.
|
||||
3. **Declare it, and let observation contradict it.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
**A node's reachability is a declared fact on its record, not an inference from its address.**
|
||||
|
||||
Two facts, and the mesh stores both:
|
||||
|
||||
- **an endpoint, or none** — where peers may reach this node, if anywhere. Absent means *this
|
||||
node initiates and is never dialled*, which is the safe default and the common case.
|
||||
- **its role in the overlay** — whether it is a hub. **Declared, never derived from an address.**
|
||||
|
||||
**The address remains evidence and stops being the fact.** When a node's observed endpoint
|
||||
disagrees with its declared one, that is a **reportable condition**, not a silent correction —
|
||||
the same discipline as [ADR 0035](0035-a-picture-is-read-from-what-runs.md): a picture is read
|
||||
from the system, and where the reading disagrees with the intent, the disagreement is the
|
||||
finding.
|
||||
|
||||
**The peer graph is computed by the control plane**, from these declared facts, and delivered to
|
||||
each node as configuration. It is not computed on the node, which is
|
||||
[ADR 0037](0037-the-host-applies-it-does-not-decide.md) and is what removes `wireguard`'s direct
|
||||
database connection ([ADR 0049](0049-a-route-is-a-grant.md) does the same for the proxy).
|
||||
|
||||
**What does not change** is the rule the comment was defending. Role still does not imply
|
||||
reachability — a home-hosted node is still a server that cannot be dialled. This record keeps
|
||||
that lesson and stops encoding it as a pattern match.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **CGNAT and IPv6 nodes become expressible**, which today they are not. Neither needs a code
|
||||
change to support; they need a field that says what is true.
|
||||
- **The lab stops needing its substitution.** TEST-NET-3 remains the right choice for a
|
||||
documentation range, but the scenario now says *this segment is reachable* rather than relying
|
||||
on an address shape to imply it. The constraint research 004 calls its most important fact
|
||||
becomes an ordinary declaration, and the validator's enforcement of it becomes unnecessary
|
||||
rather than load-bearing.
|
||||
- **Hub election becomes queryable and renumbering becomes safe.** Both follow from the same
|
||||
change and neither is possible today.
|
||||
- **Two facts can now disagree, and something must say so.** Declared-versus-observed is a new
|
||||
reportable condition and a new way to be wrong — a node declared reachable that is not will
|
||||
fail exactly as it does today until somebody looks. The gain is that *looking* is now possible;
|
||||
the regex offered nothing to compare against.
|
||||
- **Somebody must set these when a node joins.** [ADR 0038](0038-a-node-joins-by-linking-first.md)
|
||||
has the mesh finish the job after the link, so this is one more thing it finishes — and the
|
||||
honest default (no endpoint, not a hub) is correct for every node except the ones somebody
|
||||
deliberately publishes.
|
||||
|
||||
## References
|
||||
|
||||
- [Research 004](../01-RESEARCH/004-lab-network/analysis.md) — the regex, the hub convention, and
|
||||
the hairpin failure.
|
||||
- [ADR 0037](0037-the-host-applies-it-does-not-decide.md) — why the graph is computed centrally.
|
||||
- [ADR 0035](0035-a-picture-is-read-from-what-runs.md) — declared versus observed.
|
||||
- [`08-connectivity.md`](../03-DESIGN/01-to-be/08-connectivity.md) — the design this serves.
|
||||
@@ -1,145 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-27
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0039-the-link-is-the-security-boundary.md
|
||||
---
|
||||
|
||||
# 51. The enrolment token carries where the mesh is and how to recognise it
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0039](0039-the-link-is-the-security-boundary.md) requires **mutual** authority: the node
|
||||
proves it may join, and **the control plane proves it is the mesh**. It states why one-way is not
|
||||
enough — the host applies whatever the link delivers, so *an attacker who can answer a joining
|
||||
node's first call owns the machine.*
|
||||
|
||||
It does not say **how** the control plane proves itself, and
|
||||
[ADR 0049](0049-a-route-is-a-grant.md) deferred the question again, noting only that internal
|
||||
identity and public exposure are two different certificate stories.
|
||||
|
||||
Left unanswered it produces a circle. Verifying the mesh needs the mesh's CA. Obtaining the CA
|
||||
means trusting whatever hands it over — which is the thing being verified.
|
||||
|
||||
There is a second circle in the same place, and today they are solved by the same unfortunate
|
||||
mechanism. A node must reach the mesh before the mesh has configured it, so it cannot yet
|
||||
resolve any mesh name. [Research 004](../01-RESEARCH/004-lab-network/analysis.md) records the
|
||||
workaround:
|
||||
|
||||
> `dnsmasq-app` generates `.internal` names on each node and writes an `/etc/hosts` block **as a
|
||||
> floor underneath, because a node must reach the mesh DB before its own DNS exists.**
|
||||
|
||||
and, separately, that a joining node must have *"registry database and object-store host and
|
||||
credentials, plus an npm token"* placed on disk beforehand — the credentials
|
||||
[ADR 0039](0039-the-link-is-the-security-boundary.md) exists to remove.
|
||||
|
||||
**Both circles are the same shape: a node needs some fact about the mesh before it has any
|
||||
trustworthy way to obtain one.** Whatever supplies that fact must arrive by a path other than the
|
||||
mesh.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **Ship the CA with the host binary.** Then the binary is mesh-specific, which
|
||||
[ADR 0041](0041-the-host-depends-on-nothing.md) and
|
||||
[ADR 0046](0046-the-installer-fetches-what-it-pins.md) both work to avoid, and rotating the
|
||||
CA means rebuilding and redistributing the host everywhere. Rejected.
|
||||
2. **Trust on first use, plainly.** Accept whatever answers the first call and pin it. Rejected:
|
||||
it is exactly the attack ADR 0039 names, and the first call is the one moment the node has no
|
||||
way to tell.
|
||||
3. **A public certificate authority for the control plane's own endpoint.** Workable, and it
|
||||
makes joining depend on public DNS and public issuance for a link that is otherwise entirely
|
||||
the mesh's business. Rejected as a dependency, not as a technique — a mesh whose nodes cannot
|
||||
join because an unrelated public authority is having a bad day has bought nothing.
|
||||
4. **The token carries it.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
**The enrolment token carries four things**, and it is the only thing a joining node needs:
|
||||
|
||||
| | | |
|
||||
|---|---|---|
|
||||
| **where** | the **broker's address**, not a name | a node dials the broker ([ADR 0001](0001-nodes-communicate-over-a-broker.md)); there is no resolution yet, and this is why none is needed |
|
||||
| **what it is connecting to** | the fingerprint of the **broker's** certificate | so the node reaches the mesh's bus and not something answering in its place |
|
||||
| **who it will believe** | the **control plane's** signing identity | what makes the mesh provable rather than assumed |
|
||||
| **the right to join** | the one-time secret ADR 0039 already specifies | useless once used, useless after it expires |
|
||||
|
||||
### The endpoint and the authority are two identities, not one
|
||||
|
||||
This is worth separating because collapsing it is the easy mistake, and the collapsed version
|
||||
silently fails to deliver what [ADR 0039](0039-the-link-is-the-security-boundary.md) asks for.
|
||||
|
||||
A node connects to the **broker** and takes instruction from the **control plane**, which sits
|
||||
behind it. Pinning only the broker would make the control plane's authority *transitive* — the
|
||||
node would believe a declaration because of where it arrived from. **A compromised broker could
|
||||
then forge declarations**, and since the host applies whatever the link delivers, that is the
|
||||
whole machine.
|
||||
|
||||
So the node verifies **the transport** and **each declaration** separately:
|
||||
|
||||
- the broker, by its certificate, at connect time;
|
||||
- the control plane, by a **signature on the declaration itself**, every time.
|
||||
|
||||
Then 0039's *the control plane proves it is the mesh* holds against a hostile broker rather than
|
||||
assuming a friendly one — which matters because the broker is the one component every node must
|
||||
reach and the one most exposed.
|
||||
|
||||
The token is issued by the mesh for one enrolment and **carried out of band** — by the person
|
||||
adopting the machine. That is what breaks both circles: its authenticity comes from the channel
|
||||
it travelled, not from anything the node can check afterwards.
|
||||
|
||||
**This is trust-on-first-use with the first use moved out of band**, which is the difference
|
||||
between a pin and a guess. The node does not accept whatever answers; it accepts the one thing
|
||||
it was told to expect, before it spoke to anything.
|
||||
|
||||
### What this settles
|
||||
|
||||
**The mesh CA is not a bootstrap concern.** It is how `.internal` names are certified once a node
|
||||
is a member, and nothing needs it earlier. The open item
|
||||
[ADR 0049](0049-a-route-is-a-grant.md) left — *what the control plane presents to a node that
|
||||
trusts nothing yet* — is closed: it presents the identity whose fingerprint the token named.
|
||||
|
||||
**Nothing needs name resolution before the link exists**, because the token carries an address.
|
||||
The `/etc/hosts` floor exists to solve a problem that stops existing, and it should go rather than
|
||||
be carried forward — a fallback nothing needs is a path nothing tests.
|
||||
|
||||
**Nothing is placed on disk beforehand except the token.** No database credential, no object-store
|
||||
credential, no registry token. That is ADR 0039's central claim finally made true at the one
|
||||
moment it was still false, and it is the difference between *a node holds only its own identity*
|
||||
being a design statement and being a fact.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **The token becomes security-critical in a way it was not**, because it now carries the pin.
|
||||
Tampering with it in transit substitutes the mesh. That is a real exposure and it is strictly
|
||||
better than the alternative: without a pin there is nothing to tamper *with*, and the node
|
||||
trusts the first answer unconditionally. The exposure moves to a channel a person controls and
|
||||
can verify, from one nobody could.
|
||||
- **Token delivery is now a designed step, not an incidental one.** It is short-lived and
|
||||
single-use, so interception is bounded — but how it reaches a machine is part of adoption and
|
||||
needs saying. [Research 012](../01-RESEARCH/012-the-minimum-viable-node/00-overview.md) is
|
||||
where that belongs.
|
||||
- **Rotating the control plane's identity invalidates outstanding tokens**, which is correct and
|
||||
needs to fail legibly. A node presenting a token with a stale fingerprint must be told that,
|
||||
not left to time out.
|
||||
- **The address in the token can go stale.** If the control plane moves, unissued tokens point
|
||||
somewhere wrong. Tokens are short-lived, which bounds it; moving the control plane is
|
||||
[`06`](../03-DESIGN/01-to-be/06-the-control-plane.md)'s undesigned territory regardless.
|
||||
- **Declarations must be signed, and that is a real requirement rather than a note.** The host
|
||||
verifies a signature before applying anything, which adds a key to what it must carry and a
|
||||
failure mode it must report legibly — *this declaration is not from the mesh I joined* is a
|
||||
different condition from *this declaration is malformed*, and they must not read alike.
|
||||
- **Rotating the control plane's signing identity is a fleet-wide operation**, because every node
|
||||
holds the previous one. That is the cost of not trusting the broker, it is accepted, and it
|
||||
needs a rollover that overlaps rather than a flag day.
|
||||
- **A rejoining node is an ordinary case, not a special one.** A node that has lost its identity
|
||||
gets a new token. There is no recovery path to design because there is no long-lived secret to
|
||||
recover.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0039](0039-the-link-is-the-security-boundary.md) — the mutual authority this implements.
|
||||
- [ADR 0038](0038-a-node-joins-by-linking-first.md) — the join this is the first step of.
|
||||
- [ADR 0049](0049-a-route-is-a-grant.md) — the open item this closes.
|
||||
- [Research 004](../01-RESEARCH/004-lab-network/analysis.md) — the `/etc/hosts` floor and the
|
||||
credentials placed beforehand.
|
||||
@@ -1,71 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-27
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0043-a-declaration-is-an-ordered-list-of-owned-resources.md
|
||||
---
|
||||
|
||||
# 52. A filter rule names its source, or it is not a rule
|
||||
|
||||
## Context
|
||||
|
||||
[Research 004](../01-RESEARCH/004-lab-network/analysis.md) found this while looking for something
|
||||
else:
|
||||
|
||||
> Several manifests declare `scope: public` on firewall rules — `wireguard`, `traefik`, `gitea`,
|
||||
> `mailu`, `qbittorrent`. It is **not part of the rule type** and is **referenced by no code** in
|
||||
> the firewall path. Real scoping is done with `from:`.
|
||||
|
||||
**So five manifests appear to restrict a port and restrict nothing.** Anyone reading them —
|
||||
including whoever wrote the next one by copying — sees an access control that does not exist.
|
||||
|
||||
Research 004 already names the shape: it is `how-we-build`'s *an unenforced rule is
|
||||
indistinguishable from a wrong one, and costs more, because people believe it.* This is that
|
||||
rule, in the firewall, on the modules most worth restricting.
|
||||
|
||||
The mechanism that let it happen is worth more than the instance. `scope:` was accepted because
|
||||
**unknown keys were ignored**. Nothing rejected it, nothing warned, and it spread by copying for
|
||||
long enough to reach five manifests.
|
||||
|
||||
## Decision
|
||||
|
||||
**A rule names its source.** `from:` is the only way to scope a rule, and a rule without one is
|
||||
open — which it must therefore say plainly rather than imply otherwise.
|
||||
|
||||
**`scope:` is removed, not implemented.** Giving it meaning would leave two ways to express one
|
||||
thing, and a manifest carrying both would need a precedence rule nobody would remember. The five
|
||||
manifests are corrected to `from:` where they meant to restrict something, and left open where
|
||||
they did not — and finding out which is which is part of the work, not a formality.
|
||||
|
||||
**An unknown key is refused.** This is the general fix and the reason to bother:
|
||||
|
||||
> A manifest carrying a key the schema does not define is **rejected**, naming the key.
|
||||
|
||||
The host already works this way — its declaration parser sets `DisallowUnknownFields` and
|
||||
collects every problem into one refusal
|
||||
([ADR 0043](0043-a-declaration-is-an-ordered-list-of-owned-resources.md)). **Manifests are the
|
||||
layer where that discipline is missing**, and `scope:` is what missing looks like: not a wrong
|
||||
value, an invented one, silently accepted for months.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **This class of fiction stops at validation** rather than at an audit. A misspelled `form:`,
|
||||
an invented `scope:`, a key from a different schema — each fails on the manifest that
|
||||
introduces it, once, instead of spreading.
|
||||
- **Existing manifests will fail validation**, and some of those failures will be keys somebody
|
||||
believed were doing something. That is the finding, not the cost — but it means the refusal
|
||||
cannot be switched on without reading every manifest first.
|
||||
- **The firewall becomes reviewable.** Today a rule's real effect is only visible by knowing
|
||||
which keys are fictional. Afterwards the manifest says what happens.
|
||||
- **Five manifests need a decision each**, and `wireguard` and `traefik` genuinely are open to
|
||||
the world — they must be, which the manifest should state rather than appear to deny.
|
||||
- **It does not make the rules correct**, only honest. A rule that says `from:` anywhere is
|
||||
legitimately open; this record ensures it says so.
|
||||
|
||||
## References
|
||||
|
||||
- [Research 004](../01-RESEARCH/004-lab-network/analysis.md) §7 — the finding.
|
||||
- [`how-we-build.md`](../00-META/how-we-build.md) — the rule about unenforced rules.
|
||||
- [ADR 0043](0043-a-declaration-is-an-ordered-list-of-owned-resources.md) — unknown-is-refused,
|
||||
where it already holds.
|
||||
@@ -1,123 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-27
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0036-a-node-is-a-managed-machine.md
|
||||
---
|
||||
|
||||
# 53. One node runs the control plane, and nothing takes over
|
||||
|
||||
## Context
|
||||
|
||||
Two design documents left the same question open from opposite ends:
|
||||
|
||||
- [`06-the-control-plane.md`](../03-DESIGN/01-to-be/06-the-control-plane.md) — *how many nodes
|
||||
run it, and what happens when the one running it is down.*
|
||||
- [`08-connectivity.md`](../03-DESIGN/01-to-be/08-connectivity.md) — *what happens when the hub
|
||||
is down*, given that WireGuard has no failover and every non-co-located pair routes through it.
|
||||
|
||||
Both were drifting toward the same answer by default: redundancy. A second hub, a standby control
|
||||
plane, an election to decide which is live. That direction is expensive in a specific way — it is
|
||||
not one feature but a property that every layer must then honour, and each layer gets it wrong
|
||||
independently.
|
||||
|
||||
**It is also not wanted.** This is a mesh of a handful of machines with one node hosting the
|
||||
registry, not a distributed system, and building for a failure mode nobody asked to survive would
|
||||
buy nothing while complicating everything.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **A standby control plane with promotion.** Needs a replicated store, an election, a fencing
|
||||
mechanism so two promoted planes cannot both write, and a rehearsed promotion procedure —
|
||||
which is only trustworthy if it is *practised*, and an unpractised failover is reliably worse
|
||||
than none. Rejected.
|
||||
2. **Multiple control planes, each authoritative for part of the mesh.** Trades availability for
|
||||
a partition problem and a merge problem, and contradicts
|
||||
[ADR 0045](0045-a-context-owns-its-store.md)'s exclusive ownership at the worst possible layer.
|
||||
Rejected.
|
||||
3. **One, declared, with no failover.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
**One node runs the control plane and hosts its store. It is declared, never elected, and nothing
|
||||
takes over when it is down.**
|
||||
|
||||
**No node holds a contended role.** A node runs the control plane because it was *assigned* to,
|
||||
by the same mechanism that assigns anything else
|
||||
([ADR 0002](0002-everything-is-a-module.md), [`06`](../03-DESIGN/01-to-be/06-the-control-plane.md)).
|
||||
There is no promotion, no election, no quorum, no consensus, and therefore no split brain. The
|
||||
overlay hub is declared the same way and for the same reason
|
||||
([ADR 0050](0050-reachability-is-a-property-of-the-address.md)).
|
||||
|
||||
### Why this is sound, and not merely cheap
|
||||
|
||||
**The design already tolerates the control plane being absent, and it tolerates it by
|
||||
construction rather than by luck.**
|
||||
|
||||
[ADR 0036](0036-a-node-is-a-managed-machine.md) settles that *reachability is state, not class* —
|
||||
a disconnected node has a last-known state and a pending set of declarations. The host reconciles
|
||||
from **its own** store ([ADR 0043](0043-a-declaration-is-an-ordered-list-of-owned-resources.md)),
|
||||
not from the mesh, and [ADR 0037](0037-the-host-applies-it-does-not-decide.md) means it never
|
||||
needed to ask anybody in order to keep a machine in the state it was last told to hold.
|
||||
|
||||
So:
|
||||
|
||||
> **The control plane being down is not a new failure mode. It is every node in the ordinary
|
||||
> disconnected situation, at the same time.**
|
||||
|
||||
That is the argument. A property the design already has for one node does not stop being true
|
||||
because it applies to all of them at once. **What is lost is change, not operation** — every node
|
||||
goes on running exactly what it was last told to run.
|
||||
|
||||
### What is actually lost while it is down
|
||||
|
||||
Worth listing, because "nodes keep working" is true and is not the whole picture:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| new assignments, new modules, new versions | **stop** |
|
||||
| new grants and provisioning | **stop** |
|
||||
| a new node joining | **stops** — the token is issued by the mesh |
|
||||
| collection of health, logs and reports | **stops** |
|
||||
| non-co-located overlay paths | **stop** — the hub is the route |
|
||||
| co-located direct peers | keep working |
|
||||
| everything already assigned, on every node | **keeps running** |
|
||||
|
||||
## Consequences
|
||||
|
||||
- **A large amount of machinery is never built**, and this is the point: leader election, quorum,
|
||||
fencing, a replicated store, split-brain reconciliation, promotion runbooks, and the question
|
||||
*which node is authoritative* recurring at every layer. None of it exists, so none of it can be
|
||||
subtly wrong.
|
||||
- **The control-plane node is a single point of failure, and the design says so plainly.** That is
|
||||
a deliberate position, not an oversight, and stating it is what keeps it deliberate — an
|
||||
undocumented single point of failure is discovered during the outage.
|
||||
- **Recovery is restore, not failover — so backup becomes the availability story.** It stops being
|
||||
hygiene and becomes the mechanism the whole arrangement rests on. An unverified backup here is
|
||||
not a risk to the backup; it is the mesh having no recovery path at all.
|
||||
[Research 012](../01-RESEARCH/012-the-minimum-viable-node/00-overview.md) is where that lives,
|
||||
and it is now load-bearing rather than prudent.
|
||||
- **Certificate renewal is the clock, and it is the sharpest consequence.** Nodes keep running
|
||||
indefinitely, but the control plane owns issuance
|
||||
([ADR 0049](0049-a-route-is-a-grant.md)), so an outage lasting longer than a renewal window
|
||||
expires public certificates and takes down every public name. **That converts an inconvenience
|
||||
into an outage on a timer**, and it is the real bound on how long recovery may take — not
|
||||
patience, not the number of stopped features.
|
||||
- **The bound is not measured anywhere.** Nothing today reports how close a certificate is to
|
||||
expiry or how long the control plane has been unreachable, and both are needed for the above to
|
||||
be a plan rather than a hope.
|
||||
- **It can be revisited without unpicking anything.** Nothing here assumes singularity in a way
|
||||
that would have to be undone — the store is exclusively owned, the host is independent, and the
|
||||
hub is declared. Adding redundancy later means adding it, not reversing this.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0036](0036-a-node-is-a-managed-machine.md) — disconnection as an ordinary situation, which
|
||||
is the whole argument.
|
||||
- [ADR 0037](0037-the-host-applies-it-does-not-decide.md) and
|
||||
[ADR 0043](0043-a-declaration-is-an-ordered-list-of-owned-resources.md) — why a node keeps
|
||||
running without anybody to ask.
|
||||
- [ADR 0049](0049-a-route-is-a-grant.md) — certificate issuance, which sets the recovery clock.
|
||||
- [`06`](../03-DESIGN/01-to-be/06-the-control-plane.md), [`08`](../03-DESIGN/01-to-be/08-connectivity.md) —
|
||||
the two open questions this closes.
|
||||
@@ -1,117 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-27
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0044-a-module-declares-presence-instantiation-and-exclusion.md
|
||||
---
|
||||
|
||||
# 54. Things that change together share an authority, not a package
|
||||
|
||||
## Context
|
||||
|
||||
[`how-we-build.md`](../00-META/how-we-build.md) — the constitution, injected wherever work is
|
||||
decided ([ADR 0009](0009-the-mesh-is-governed-by-a-constitution.md),
|
||||
[ADR 0025](0025-hq-is-the-source-of-the-constitution.md)) — carries this rule:
|
||||
|
||||
> ### Group by domain, not by single function
|
||||
>
|
||||
> A module is a purpose, not a piece of software. Four modules that together constitute "how a
|
||||
> node is reachable" and cannot be assigned, versioned or replaced as one thing are four
|
||||
> accidents, not four boundaries. — ADR 0017
|
||||
|
||||
**[ADR 0017](0017-modules-outside-the-core-are-grouped-by-domain.md) is superseded**, and
|
||||
[ADR 0044](0044-a-module-declares-presence-instantiation-and-exclusion.md) says the opposite in
|
||||
as many words:
|
||||
|
||||
> The domain module goes with it: **there is no `networking` thing to install, there are
|
||||
> concrete modules named individually.**
|
||||
|
||||
So the governing document instructs agents to do the thing the decision record forbids. This is
|
||||
not a stale citation in a design note — it is
|
||||
[ADR 0040](0040-the-constitution-absorbs-what-is-enforced.md)'s concern running backwards, in the
|
||||
one document whose entire purpose is to be followed.
|
||||
|
||||
**The example makes it concrete.** *"How a node is reachable"* is exactly the case
|
||||
[`08-connectivity.md`](../03-DESIGN/01-to-be/08-connectivity.md) has now designed — and the
|
||||
design resolves it the other way: five responsibilities under one **context**, delivered by
|
||||
individual modules with edges between them. Anyone following the constitution would build the
|
||||
merged `networking` module that 0044 removed and 08 does not have.
|
||||
|
||||
## What was right about the old rule
|
||||
|
||||
The rule is not simply wrong, and replacing it badly would lose something measured.
|
||||
|
||||
[Research 005](../01-RESEARCH/005-domain-grouping/analysis.md) surveyed the whole catalogue and
|
||||
found that reachability is **the only** place where modules genuinely change together under one
|
||||
intent — the proxy with the resolver, the firewall with the overlay, repeatedly. That is a real
|
||||
observation about coupling, and the smell it identifies is real: four things that always change
|
||||
together and cannot be reasoned about separately are not four boundaries.
|
||||
|
||||
**The observation was right and the conclusion was wrong.** Coupling that tight means they share
|
||||
an *authority* — one place that decides for all of them. It does not mean they should be one
|
||||
installable artifact, and merging them into one is how the observation gets acted on badly:
|
||||
`wireguard` and `traefik` are deployed on different sets of nodes, so a module containing both
|
||||
would be assigned where half of it is unwanted.
|
||||
|
||||
## Decision
|
||||
|
||||
**Things that change together share an authority, not a package.**
|
||||
|
||||
Two units, deliberately separate, and conflating them is what ADR 0017 did:
|
||||
|
||||
| | is | example |
|
||||
|---|---|---|
|
||||
| a **context** | the unit of **coherence** — one authority, one store, one set of decisions | `connectivity` decides the overlay, names, routes, filtering and certificates |
|
||||
| a **module** | the unit of **delivery** — assignable, versionable, replaceable on its own | `wireguard`, the resolver, the proxy, the firewall — four, named individually |
|
||||
|
||||
**When several modules always change together, the answer is to name the context that decides for
|
||||
them** — not to merge them. Connectivity is the worked example and the proof: one authority, five
|
||||
responsibilities, four or more separately assigned modules, and no `networking` module anywhere.
|
||||
|
||||
**Relationships are edges, not folders**
|
||||
([ADR 0044](0044-a-module-declares-presence-instantiation-and-exclusion.md)). What grouping was
|
||||
for — finding things, seeing what belongs together — is a tag and a query, neither of which
|
||||
anybody has to keep true by hand.
|
||||
|
||||
### The constitution is amended
|
||||
|
||||
The section *Group by domain, not by single function* is **replaced**, not repaired, and the
|
||||
derived page republished ([ADR 0025](0025-hq-is-the-source-of-the-constitution.md)). The
|
||||
replacement keeps the observation and changes the instruction:
|
||||
|
||||
> ### Things that change together share an authority, not a package
|
||||
>
|
||||
> When several modules always change together under one intent, name the context that decides
|
||||
> for them. Do not merge them: they are delivered to different nodes, and a module that must be
|
||||
> assigned where half of it is unwanted is not a boundary either. Coherence is a context;
|
||||
> delivery is a module. — ADR 0054
|
||||
|
||||
## Consequences
|
||||
|
||||
- **The instruction now matches the design.** An agent reading the constitution and an agent
|
||||
reading 0044 reach the same answer, which they currently do not.
|
||||
- **Synced 2026-08-27**, playbook [05](../00-META/process/05-constitution-sync.md). The derived
|
||||
page carries the new rule and §4 keeps its number. **Verified by reading back**, not by the
|
||||
publish reporting success: the replacement text is present, and the old section's body — *scope
|
||||
under measurement: grouping is evidenced for reachability* — returns nothing.
|
||||
- **The sync found a drift the playbook exists to catch.** The published §4 and the source did not
|
||||
say the same thing: the source spoke of *four accidents, not four boundaries*, the published
|
||||
page of *one intent expressed four times*, and only the published page carried the scope
|
||||
caveat. Same rule, two texts, already diverging — which is the exact failure
|
||||
[ADR 0025](0025-hq-is-the-source-of-the-constitution.md) predicted and the reason the playbook
|
||||
re-publishes the whole page rather than patching a section.
|
||||
- **`context` becomes a word the constitution uses**, which raises the obvious next question —
|
||||
*which contexts are there* — and that is
|
||||
[ADR 0055](0055-the-control-plane-is-the-node-coordinating-contexts.md), not this record.
|
||||
- **This is the second time a superseded record was found still steering work.** The first was
|
||||
the to-be README citing 0017 for work still to do. Both were found by a review rather than by
|
||||
anything automatic, and nothing stops the third — **a superseded record has no mechanism that
|
||||
finds its live citations.** Worth an issue in its own right.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0044](0044-a-module-declares-presence-instantiation-and-exclusion.md) — what superseded 0017.
|
||||
- [ADR 0025](0025-hq-is-the-source-of-the-constitution.md) — why amending the source is not enough.
|
||||
- [Research 005](../01-RESEARCH/005-domain-grouping/analysis.md) — the measurement the old rule rested on.
|
||||
- [`08-connectivity.md`](../03-DESIGN/01-to-be/08-connectivity.md) — the worked example.
|
||||
@@ -1,131 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-27
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0015-mesh-brokers-nodes-host-agents-think.md
|
||||
---
|
||||
|
||||
# 55. The control plane is the node-coordinating contexts, and the rest are hosted
|
||||
|
||||
## Context
|
||||
|
||||
Three different context lists are in circulation and none of them was decided:
|
||||
|
||||
| Where | Count | Named |
|
||||
|---|---|---|
|
||||
| [ADR 0015](0015-mesh-brokers-nodes-host-agents-think.md), **accepted** | **nine** | mesh, agents, work, stream, delivery, knowledge, ai, observability, config |
|
||||
| [`06-the-control-plane.md`](../03-DESIGN/01-to-be/06-the-control-plane.md) | **ten** + `api` | record, inventory, config, connectivity, provisioning, delivery, observability, identity, work, knowledge |
|
||||
| `01-to-be/README.md` (until today) | **eight** | — |
|
||||
|
||||
`06` cites ADR 0015 in its own frontmatter while presenting a list that is not 0015's. And
|
||||
[research 006](../01-RESEARCH/006-mesh-from-scratch/skeleton.md), which produced the ten, said
|
||||
plainly what should happen next:
|
||||
|
||||
> This is an addition to an accepted record, so it is a **decision, not a drafting choice**. It
|
||||
> belongs in a new record that extends ADR 0015 — **not written here.**
|
||||
|
||||
**That record was never written, and the design used the list anyway.** This is exactly what
|
||||
`01-to-be`'s own rule forbids — *every statement here traces to a record; nothing arrives by
|
||||
drafting* — and it is why the count could drift three ways without anybody noticing.
|
||||
|
||||
### What changed silently
|
||||
|
||||
Reconciling the two lists, the differences are not cosmetic:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **added, with reasoning** | `connectivity` (research 006 Move 3; now designed in [`08`](../03-DESIGN/01-to-be/08-connectivity.md) and settled by [ADR 0049](0049-a-route-is-a-grant.md)–[0052](0052-a-filter-rule-names-its-source.md)) |
|
||||
| **added, argued but open** | `record` — research 006 explicitly leaves *where the record lives* unresolved |
|
||||
| **split** | 0015's `mesh` became `inventory` + `provisioning` |
|
||||
| **renamed** | 0015's `agents` became `identity` |
|
||||
| **dropped with no reasoning at all** | **`stream`** — threads, mentions, messages, meetings, notifications; **`ai`** — provider grants and rotation |
|
||||
|
||||
The last row is the finding. Two contexts holding real behaviour vanished between an accepted
|
||||
record and a design document, and nothing anywhere says they were removed or where their content
|
||||
went.
|
||||
|
||||
## Decision
|
||||
|
||||
**Apply `06`'s own test honestly, and it sorts the list for us.**
|
||||
|
||||
The test is *everything that needs to know about more than one node.* Run it:
|
||||
|
||||
| Context | Needs to know about more than one node? | |
|
||||
|---|---|---|
|
||||
| **inventory** | which nodes exist, what is assigned where | **control plane** |
|
||||
| **config** | derives settings and secrets **onto nodes** | **control plane** |
|
||||
| **connectivity** | who peers with whom, which node is reachable | **control plane** |
|
||||
| **provisioning** | grants between modules on different nodes | **control plane** |
|
||||
| **delivery** | source to artifact **to node** | **control plane** |
|
||||
| **observability** | health of nodes, including *unreachable for a week* | **control plane** |
|
||||
| **identity** | credentials delivered per agent's node bindings and modality ([ADR 0015](0015-mesh-brokers-nodes-host-agents-think.md)) | **control plane** |
|
||||
| **work** | a task does not need to know a node exists | **hosted** |
|
||||
| **knowledge** | a document does not either | **hosted** |
|
||||
| **stream** | nor does a message | **hosted** |
|
||||
| **ai** | a provider licence is a **grant** ([ADR 0005](0005-capabilities-are-provisioned-on-declaration.md)) and its delivery is `config`'s | **folded** |
|
||||
|
||||
**So: seven contexts and one interface.**
|
||||
|
||||
> **inventory, config, connectivity, provisioning, delivery, observability, identity** — plus
|
||||
> **`api`**, the one interface every surface speaks to.
|
||||
|
||||
**`work`, `knowledge` and `stream` are mesh-hosted applications, not control plane.** They are
|
||||
first-party, they ship with everything else, and they run on the mesh exactly the way anything
|
||||
else does. Being ours does not make them infrastructure.
|
||||
|
||||
**`ai` is not a context.** A provider licence is a grant like a database or a bucket, and
|
||||
delivering it is `config`'s existing job. 0015 already did the hard part here by removing the node
|
||||
licence — *a node holds no licence; an agent holds credentials* — and what remains needs no
|
||||
authority of its own.
|
||||
|
||||
**`record` is deferred, deliberately.** [ADR 0045](0045-a-context-owns-its-store.md) makes it
|
||||
load-bearing — contexts integrate through it — and research 006 leaves *where it lives* open, on
|
||||
the grounds that putting it in the substrate risks recreating the circularity the tier design just
|
||||
removed. Naming it a context here would settle by listing what has not been settled by arguing.
|
||||
**Seven is the decided count; the record is an eighth question, not an eighth entry.**
|
||||
|
||||
## The cost, stated plainly
|
||||
|
||||
**A surface composing across this boundary now reads more than one interface.** The board shows
|
||||
nodes *and* tasks *and* documents; under this decision that is the control plane's `api` plus
|
||||
work's and knowledge's.
|
||||
|
||||
This was raised as an objection before — *that only moves the problem up a layer* — and it
|
||||
deserves an honest answer rather than a reassurance. The answer is that
|
||||
[ADR 0045](0045-a-context-owns-its-store.md) already requires it: a surface reads **interfaces,
|
||||
never stores**, so a board was always going to compose rather than join. What this decision
|
||||
changes is the *number* of interfaces, not the kind of work. And `06`'s constraint — a single
|
||||
surface can compose contexts only while one interface sits in front of them — was already written
|
||||
about the control plane's contexts, and still holds for the seven.
|
||||
|
||||
**The alternative is available and should be named:** keep all ten under tier 2 and accept that
|
||||
*control plane* means *everything first-party*, not *everything node-coordinating*. Rejected
|
||||
because the node-coordinating test is doing real work elsewhere — it is what justified
|
||||
`connectivity`, and it is the same test that defines the substrate. A definition that sorts
|
||||
cleanly in one place and is waved through in another is not a definition.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **Three lists become one, and it is a decision rather than a draft.** `06` and the to-be README
|
||||
are corrected to seven, and `06`'s frontmatter stops citing a record it contradicts.
|
||||
- **`stream` is reinstated, as a hosted application.** It was dropped by accident; this puts it
|
||||
somewhere on purpose. Its content — threads, notifications, meetings — is real and has to live
|
||||
somewhere nameable.
|
||||
- **Tier 4 gains its first named residents.** Until now the tier existed with nothing in it, which
|
||||
is part of why contexts drifted upward into tier 2 unopposed.
|
||||
- **The eight-nine-ten drift had no mechanism that would have caught it**, and neither does the
|
||||
next one. A design document cites decisions in its frontmatter and nothing checks that what it
|
||||
says matches what they say.
|
||||
- **Splitting `mesh` into `inventory` and `provisioning` is inherited without fresh argument.**
|
||||
It is right under [ADR 0045](0045-a-context-owns-its-store.md) — they would own separate
|
||||
stores — but this record adopts it from research 006 rather than re-deriving it, and that is
|
||||
worth saying rather than implying it was examined.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0015](0015-mesh-brokers-nodes-host-agents-think.md) — the nine this extends.
|
||||
- [Research 006](../01-RESEARCH/006-mesh-from-scratch/skeleton.md) — the ten, and the instruction
|
||||
to write this record.
|
||||
- [ADR 0045](0045-a-context-owns-its-store.md) — why the boundary costs what it costs.
|
||||
- [`06-the-control-plane.md`](../03-DESIGN/01-to-be/06-the-control-plane.md) — the document this corrects.
|
||||
@@ -1,101 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-27
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
supersedes: 0003-the-mesh-database-is-the-source-of-truth.md
|
||||
extends: 0045-a-context-owns-its-store.md
|
||||
---
|
||||
|
||||
# 56. The authority is the control plane, not a database
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0003](0003-the-mesh-database-is-the-source-of-truth.md) is still `accepted` and still cited
|
||||
as live by two as-is documents. Its decision says:
|
||||
|
||||
> A **single database** holds every binding... The runtime **loads its configuration from that
|
||||
> database at startup** and **falls back to a local cache** when the database is unreachable.
|
||||
|
||||
Every clause has since been decided against, in four separate records, none of which marked it
|
||||
superseded:
|
||||
|
||||
| 0003 says | contradicted by |
|
||||
|---|---|
|
||||
| a **single** database holds every binding | [ADR 0045](0045-a-context-owns-its-store.md) — a context owns its store exclusively; three contexts must move out of the registry database, taking thirteen tables |
|
||||
| the runtime **loads from that database** | [ADR 0037](0037-the-host-applies-it-does-not-decide.md) — the host does not query the mesh database |
|
||||
| — | [ADR 0039](0039-the-link-is-the-security-boundary.md) — a node holds **no credential** to it |
|
||||
| it **falls back to a local cache** | [ADR 0043](0043-a-declaration-is-an-ordered-list-of-owned-resources.md) — the host has **its own store**, which is not a cache of anything |
|
||||
|
||||
The two modules that still made 0003 literally true — `wireguard` and `traefik`, the only direct
|
||||
database connections left — are removed by
|
||||
[ADR 0049](0049-a-route-is-a-grant.md) and [ADR 0050](0050-reachability-is-a-property-of-the-address.md).
|
||||
When those land, **nothing on any node reads the mesh database at all**, and 0003 will describe a
|
||||
mechanism with no remaining implementation.
|
||||
|
||||
**The error underneath is a category error, and it is the same one
|
||||
[ADR 0054](0054-things-that-change-together-share-an-authority.md) corrects elsewhere:** *source
|
||||
of truth* named a **storage location** when what it meant was an **authority**. Once the store is
|
||||
the answer, "which database" becomes the question, and shared schemas follow — which is precisely
|
||||
the thirteen-table tangle ADR 0045 exists to undo.
|
||||
|
||||
## Decision
|
||||
|
||||
> **The control plane is the authority for what runs where. A database is where one context keeps
|
||||
> its state.**
|
||||
|
||||
Three consequences of that sentence, replacing 0003's three clauses:
|
||||
|
||||
- **There is no single mesh database.** Each context owns its store exclusively
|
||||
([ADR 0045](0045-a-context-owns-its-store.md)). "The mesh database" is not a thing that exists;
|
||||
the registry database is `inventory`'s store, and other contexts have their own.
|
||||
- **No node reads any of them.** A node is told what to own, over the link, in a bounded
|
||||
vocabulary ([ADR 0037](0037-the-host-applies-it-does-not-decide.md),
|
||||
[ADR 0039](0039-the-link-is-the-security-boundary.md)). Reading the authority's storage is not
|
||||
how anything learns anything.
|
||||
- **A node runs from its own store, always — not as a fallback.** The host records what it owns
|
||||
and reconciles against it
|
||||
([ADR 0043](0043-a-declaration-is-an-ordered-list-of-owned-resources.md)). That store is the
|
||||
node's own record of what it applied, not a copy of somebody else's state.
|
||||
|
||||
### What survives from 0003, unchanged
|
||||
|
||||
The half that was right, and it is the half everything else cites:
|
||||
|
||||
> **The repository defines what exists. The mesh defines what runs where.** No node-to-module
|
||||
> mapping is ever committed, and a node is described nowhere in source.
|
||||
|
||||
That is what makes the repositories node-agnostic and it is why anything about the mesh can be
|
||||
published at all ([ADR 0019](0019-hq-is-its-own-repository.md)). Nothing here weakens it — this
|
||||
record changes *where the authority lives and how it is reached*, not whether bindings are
|
||||
committed.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **The cache mode disappears, and with it the fault it created.** The as-is records the sharp
|
||||
edge: *"a node running from cache looks identical to a node running from the database. There is
|
||||
no age on the cache and nothing reports divergence, so a node can be running yesterday's
|
||||
assignment set indefinitely without any signal that it is."* Under this record there is no
|
||||
second mode to be mistaken for the first — **a node always runs from its own store**, and
|
||||
whether it has heard from the mesh recently is a separate, reportable fact rather than an
|
||||
invisible one.
|
||||
- **"The mesh database" should stop being said**, including in conversation. It names a thing that
|
||||
will not exist, and it is the phrase that makes a shared schema sound reasonable.
|
||||
- **This closes the set of records that made nodes hold database credentials.** 0037 decided it,
|
||||
0039 named the exposure, 0049 and 0050 remove the two offenders, and this one removes the
|
||||
*record* that still authorised the arrangement — which was the last thing anybody could have
|
||||
cited in its defence.
|
||||
- **Two as-is documents cite 0003 and describe today's behaviour accurately.** They are not
|
||||
wrong and should not be changed: [ADR 0019](0019-how-this-repository-works.md) keeps the
|
||||
layers separate, and *as-is* describing a superseded decision is exactly what as-is is for. What
|
||||
changes is the citation's status, not its content.
|
||||
- **It does not say how today's mesh gets there.** Thirteen tables move, two modules are rewritten,
|
||||
and nothing here costs that. Research 006 already lists the migration as unaddressed, and this
|
||||
record adds to what must migrate rather than explaining it.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0003](0003-the-mesh-database-is-the-source-of-truth.md) — superseded by this.
|
||||
- [ADR 0045](0045-a-context-owns-its-store.md) — the ownership rule this generalises.
|
||||
- [ADR 0037](0037-the-host-applies-it-does-not-decide.md), [ADR 0039](0039-the-link-is-the-security-boundary.md) — why no node reads it.
|
||||
- [ADR 0054](0054-things-that-change-together-share-an-authority.md) — the same category error, corrected elsewhere.
|
||||
@@ -1,217 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-27
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0041-the-host-depends-on-nothing.md
|
||||
---
|
||||
|
||||
# 57. The host is a root service, installed as a package, that never manages itself
|
||||
|
||||
## Context
|
||||
|
||||
[`05-the-node-host.md`](../03-DESIGN/01-to-be/05-the-node-host.md) describes what the host *does*
|
||||
and never says what it *is* at runtime. Searched: the words *daemon*, *long-running*, *interval*,
|
||||
*poll* and *heartbeat* appear in none of it, nor in
|
||||
[ADR 0038](0038-a-node-joins-by-linking-first.md) or
|
||||
[ADR 0039](0039-the-link-is-the-security-boundary.md).
|
||||
|
||||
What exists today is a command that runs and exits — `mesh-host apply FILE`. What the design
|
||||
requires is a process holding an outbound link to the control plane. Nobody wrote down that
|
||||
these are different things, so several questions have no answer: does it reconcile on a timer,
|
||||
what happens when the link drops, and who installs the unit that starts it — given that the host
|
||||
is the thing that installs units.
|
||||
|
||||
## Decision
|
||||
|
||||
### It runs on every node, and that is what a node is
|
||||
|
||||
[ADR 0036](0036-a-node-is-a-managed-machine.md) defines a node as a managed machine. **The host
|
||||
is what makes it managed**, so a machine without one is not a node with a missing component; it
|
||||
is not a node. There is no partial mode, no agentless node, and no second way in.
|
||||
|
||||
### It is a root service
|
||||
|
||||
**Root**, because there is no useful subset of its job that is not privileged: it writes under
|
||||
`/etc`, installs packages, manages units, and runs containers. A host that dropped privilege
|
||||
could apply almost nothing, and the almost is where the confusion would live.
|
||||
|
||||
**A service rather than a command**, because it holds the link, and something must survive a
|
||||
reboot to hold it. The command-line entry points remain — they are how a person inspects and
|
||||
rescues a machine — but the ordinary case is a unit that is always up.
|
||||
|
||||
### It cannot run in a container, and the reason is the bootstrap
|
||||
|
||||
Worth stating because everything else the mesh runs *is* a container, which makes the host look
|
||||
like an exception somebody forgot to fix.
|
||||
|
||||
**Step 0 of the substrate bootstrap is installing the container runtime**
|
||||
([`07-the-substrate.md`](../03-DESIGN/01-to-be/07-the-substrate.md)). A host that ran inside a
|
||||
container could not perform it — it would need the thing it is there to install. On a machine
|
||||
with no runtime, nothing would ever start.
|
||||
|
||||
That is not the only reason, but it is the sufficient one:
|
||||
|
||||
- **It would break [ADR 0041](0041-the-host-depends-on-nothing.md).** *Copy it onto a machine and
|
||||
run it* stops being true when the machine must already have a container runtime.
|
||||
- **The isolation would be fiction.** To write `/etc`, install packages, manage units and run
|
||||
containers, it would need the host's mount, PID and network namespaces plus the runtime's own
|
||||
socket. A container with all of those is a process with extra steps.
|
||||
|
||||
**So the host is a plain process on the machine, and everything above tier 0 is a container.**
|
||||
That split is the tier boundary made concrete rather than an inconsistency.
|
||||
|
||||
### What it needs from an init, and why that is not a dependency
|
||||
|
||||
> **Overtaken by [ADR 0061](0061-the-host-asks-an-init-for-start-and-restart.md) and
|
||||
> [ADR 0060](0060-the-host-is-built-per-operating-system.md).** All three claims below were
|
||||
> true when written and are not now. Kept rather than rewritten, because what changed and why
|
||||
> is the useful part.
|
||||
|
||||
~~The host needs **four** things from whatever supervises it: start at boot, restart when it
|
||||
exits, give up after repeated failures, and run something else when it gives up.~~ **One**: run
|
||||
this at boot. The other three moved into a launcher the host ships, where they can be tested —
|
||||
a unit file's restart policy can only be read and hoped for
|
||||
([ADR 0061](0061-the-host-asks-an-init-for-start-and-restart.md)).
|
||||
|
||||
~~**Every machine the mesh targets already has systemd.**~~ **Alpine does not**, and it is the
|
||||
intended first node. It runs OpenRC.
|
||||
|
||||
~~**Abstracting over init systems is not done, because there is no second one to abstract
|
||||
over.**~~ There is now, and the answer is still not an abstraction: the host is built per
|
||||
operating system ([ADR 0060](0060-the-host-is-built-per-operating-system.md)), so each ships its
|
||||
own four-line init file. That the file is the *only* system-specific artefact is what survives,
|
||||
and it is what makes a second one transcription rather than a port.
|
||||
|
||||
The part that stands unchanged: **an init is not a dependency in
|
||||
[ADR 0041](0041-the-host-depends-on-nothing.md)'s sense.** 0041 is about what must be *installed
|
||||
before the host works*, and an init is not installed — it is what the machine already is.
|
||||
|
||||
### It never manages its own unit
|
||||
|
||||
**The host's own service file is not a resource the host applies.** The temptation is obvious —
|
||||
it manages units, and its own unit is a unit — and it ends with a host stopping itself half way
|
||||
through an apply, leaving a machine in a state nothing is running to fix.
|
||||
|
||||
So the boundary is: **the installation owns the host; the host owns everything else.** A
|
||||
declaration that names the host's own unit is refused rather than obeyed.
|
||||
|
||||
### It is installed as a package, and a tarball is the floor
|
||||
|
||||
Two mechanisms, and the second is not a fallback for the first failing — it is what makes the
|
||||
first possible.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **package** — `pacman -S nox-mesh-host` | the ordinary path. Carries the binary, the unit file, the state directory, and an upgrade path |
|
||||
| **tarball** — `curl … \| tar -xz` | the floor. One static binary, no repository, no distribution assumed |
|
||||
|
||||
**Why a package rather than only a binary.** [ADR 0041](0041-the-host-depends-on-nothing.md) says
|
||||
copying the binary onto a machine is the whole installation, and that remains true of the
|
||||
*binary*. But a unit file, a state directory and an upgrade path are real, and something has to
|
||||
own them. A package that installs one statically linked binary plus a unit file adds no runtime
|
||||
dependency — 0041 is about what must already be present for the host to work, not about how the
|
||||
bytes arrived.
|
||||
|
||||
**Why the tarball must keep working.** The package lives in a repository, and the mesh's own
|
||||
repository is hosted on the mesh. A first node cannot fetch from a mesh that does not exist yet,
|
||||
and neither can a node whose mesh is down — which is exactly when somebody is trying to fix it.
|
||||
**Any path that requires the mesh to install the thing that joins the mesh is a circle**, so the
|
||||
tarball is the path that is never allowed to acquire a dependency.
|
||||
|
||||
### The host may replace its own binary; it may not stop its own unit
|
||||
|
||||
The first draft of this record said the mesh must not upgrade the host at all. That was too
|
||||
broad, and it conflated two different acts.
|
||||
|
||||
**Replacing the binary is safe.** Unix keeps the running executable's inode open, so a package
|
||||
upgrade writes a new file and the running process continues on the old one, undisturbed.
|
||||
**Stopping the unit is what is unsafe** — that is the host killing itself part-way through an
|
||||
apply, leaving a machine with nothing running to finish or fix it.
|
||||
|
||||
So the host may apply a `package` naming itself. What it must never do is ask the service
|
||||
manager to restart it.
|
||||
|
||||
**The restart happens by exiting, not by asking.** When the host notices its own executable has
|
||||
been replaced, it finishes the apply it is in, reports what it did, and **exits cleanly**. The
|
||||
supervisor's `Restart=always` starts it again, on the new binary. Nothing stops the host; the
|
||||
host stops, having finished.
|
||||
|
||||
Three conditions, and they are the whole safety argument:
|
||||
|
||||
- **after** the apply completes and its outcomes are recorded — never mid-way;
|
||||
- **only** when the executable actually changed, which Linux reports plainly: a replaced
|
||||
`/proc/self/exe` reads as the old path marked deleted;
|
||||
- **exit zero**, so a restart is what a supervisor does next rather than a failure it backs off
|
||||
from.
|
||||
|
||||
This makes a fleet-wide host upgrade an ordinary declaration, which the first draft gave up.
|
||||
|
||||
### Changes are pushed. The timer is for drift, and only for drift
|
||||
|
||||
**The host does not poll for work.** A new declaration arrives as a message on the link, and the
|
||||
host applies it then ([ADR 0001](0001-nodes-communicate-over-a-broker.md)). Polling for updates
|
||||
over a connection that already exists would be strictly worse in both directions: slower to
|
||||
land, and constant traffic to learn nothing.
|
||||
|
||||
Four triggers, and only one of them is a clock:
|
||||
|
||||
| Trigger | Kind | Why |
|
||||
|---|---|---|
|
||||
| **a declaration arrives** | **pushed** | the ordinary path — this is how changes land |
|
||||
| **start** | event | the machine may have changed while nothing was running |
|
||||
| **reconnect** | event | declarations may have been missed while disconnected |
|
||||
| **a timer** | periodic | **drift**, and nothing else |
|
||||
|
||||
**The timer cannot be replaced by an event, and the reason is definitional.** Drift is change the
|
||||
*mesh did not make* — somebody edited a managed file, a distribution upgrade replaced a config,
|
||||
a container was stopped by hand. **Nothing will ever send a message about it**, because the thing
|
||||
that did it is not part of the mesh. A local periodic check is the only way to see it at all.
|
||||
|
||||
Without it, `owned` reports what the host *applied* rather than what is *there*, which is
|
||||
[ADR 0035](0035-a-picture-is-read-from-what-runs.md) violated by omission.
|
||||
|
||||
**Ten minutes**, configurable. The check is cheap: it asks the package database, the service
|
||||
manager and the container runtime about resources the host already knows it owns.
|
||||
|
||||
### It reports upward on a heartbeat
|
||||
|
||||
Separate from reconciling, and easy to conflate with it: the node tells the mesh what it is —
|
||||
its running version, what it holds, what it last applied — on link, after every apply, and
|
||||
periodically while idle.
|
||||
|
||||
**The heartbeat is what makes silence mean something.** Without it, the mesh cannot distinguish
|
||||
a node that is fine and has had nothing to do from one that stopped. With it, *last heard from*
|
||||
is a fact per node, and [ADR 0059](0059-a-host-that-cannot-start-rolls-itself-back.md) is what
|
||||
handles the case where the node cannot report at all.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **Adoption becomes two concrete steps**, which is the point of writing this down:
|
||||
install the package, then hand it a token. Nothing else.
|
||||
- **The host gains a mode it does not have**, and it is the larger half of stage 3. Today every
|
||||
entry point runs and exits.
|
||||
- **Refusing to manage its own unit needs enforcing, not just stating.** A declaration naming
|
||||
the host's unit must be refused by name, and that refusal is a test.
|
||||
- **A host that cannot reach the mesh keeps reconciling from its store**, which is
|
||||
[ADR 0036](0036-a-node-is-a-managed-machine.md) made operational rather than aspirational: a
|
||||
disconnected node is not merely tolerated, it is actively holding its machine in the last
|
||||
state it was told to hold.
|
||||
- **The timer makes drift visible and also makes it loud.** A resource the host cannot apply
|
||||
will now fail every ten minutes rather than once. That is correct and it needs somewhere to go
|
||||
other than a log nobody reads — which is `observability`'s, and it does not exist yet.
|
||||
- **A host upgrade is an ordinary declaration**, which is worth the care it needs: the exit
|
||||
path is the only place the host deliberately stops, and a bug there is a node that restarts in
|
||||
a loop or never comes back. It wants a test that the host does **not** exit when its binary is
|
||||
unchanged, as much as one that it does when it changed.
|
||||
- **A version-skewed fleet is now normal and needs saying.** Nodes restart onto the new binary
|
||||
at whatever moment their apply finishes, so "the fleet is upgraded" is a range rather than an
|
||||
instant. What a node reports as its version must be the **running** one, not the installed
|
||||
one, or the mesh will believe an upgrade landed before it took effect.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0041](0041-the-host-depends-on-nothing.md) — the property the package must not break.
|
||||
- [ADR 0036](0036-a-node-is-a-managed-machine.md) — what a node is, which this makes operational.
|
||||
- [ADR 0039](0039-the-link-is-the-security-boundary.md) — the link the process exists to hold.
|
||||
- [ADR 0051](0051-the-enrolment-token-carries-the-mesh.md) — the second of the two adoption steps.
|
||||
@@ -1,135 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-27
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0014-build-publish-and-deploy-are-three-silos.md
|
||||
---
|
||||
|
||||
# 58. Delivery ends in a declaration, not in a push to a node
|
||||
|
||||
## Context
|
||||
|
||||
Today a push produces a pipeline with three silos, and the third — **deploy** — runs *once per
|
||||
module per node*, sending a command to every assigned node telling it to install, configure,
|
||||
start and verify ([`00-as-is/04`](../03-DESIGN/00-as-is/04-delivery.md)).
|
||||
|
||||
That third silo is where the as-is document records the most damage:
|
||||
|
||||
- *"A green pipeline proves transport, not effect."* The stages report a message was dispatched
|
||||
and accepted, not that anything is running.
|
||||
- A service reported started when the container command merely returned. An image pull failure
|
||||
that did not fail the deploy. **A package install that 404ed from every mirror while the job
|
||||
went green** ([04-ISSUES/001](../04-ISSUES/001-failed-package-install-reports-success/00-report.md)).
|
||||
- A node left on old code after a failed download, with a version marker that had already
|
||||
advanced.
|
||||
- A verify stage built to close the gap, and *never scheduled*, because the coordinator's stage
|
||||
list did not include it.
|
||||
|
||||
Meanwhile [ADR 0037](0037-the-host-applies-it-does-not-decide.md) has given every node a
|
||||
component that does exactly what deploy does — applies state, reads back, reports — and does it
|
||||
continuously rather than once per pipeline. **Two mechanisms now change a node**, and only one
|
||||
of them checks its work.
|
||||
|
||||
## Decision
|
||||
|
||||
**A pipeline ends when the declaration is updated. The node applies it.**
|
||||
|
||||
The three silos become:
|
||||
|
||||
| Silo | Runs | Ends with |
|
||||
|---|---|---|
|
||||
| **build** | once per module | a self-contained artifact |
|
||||
| **publish** | once per module | that artifact addressable — an image by digest, a package in the mesh's repository |
|
||||
| **deploy** | **once, not once per node** | the affected nodes' **declarations updated** to name the new version |
|
||||
|
||||
**Deploy stops sending commands to nodes.** It changes what the control plane says each node
|
||||
should be, which is one write. What happens on the machines is the host's ordinary reconcile,
|
||||
on whatever schedule each node is on.
|
||||
|
||||
### Why this fixes the failure class rather than patching it
|
||||
|
||||
The as-is faults share one shape: **the thing that reported success was not the thing that did
|
||||
the work.** A coordinator dispatching a command can only report on dispatch.
|
||||
|
||||
Under this decision the reporter *is* the applier. The host already refuses to record a resource
|
||||
until it read it back ([ADR 0035](0035-a-picture-is-read-from-what-runs.md)), and already fails
|
||||
the whole apply on one failed step ([ADR 0008](0008-a-failed-step-fails-the-job.md)). A package
|
||||
that 404s cannot go green, because nothing between the package manager and the report has an
|
||||
opportunity to be optimistic.
|
||||
|
||||
**The verify stage disappears as a stage**, which is the strongest evidence for this shape:
|
||||
verification stops being a step that can be omitted from a list, and becomes a property of
|
||||
applying at all.
|
||||
|
||||
### What a pipeline result now means
|
||||
|
||||
The honest answer, and it is different from today's:
|
||||
|
||||
> **The declaration is updated, and here is which nodes have applied it.**
|
||||
|
||||
A pipeline **does not wait for every node**, because a node may be legitimately switched off for
|
||||
a week ([ADR 0036](0036-a-node-is-a-managed-machine.md)) and a delivery mechanism that blocks on
|
||||
a sleeping laptop is one nobody will use. It reports what landed and what has not landed *yet*:
|
||||
|
||||
```
|
||||
delivered declaration updated for 5 nodes
|
||||
applied 3 of 5
|
||||
outstanding 2 — last seen 4 days ago, 20 minutes ago
|
||||
```
|
||||
|
||||
**Outstanding is not failure**, and conflating them is how the old system got a stall with no
|
||||
error anywhere. A node that has not applied yet is a fact with a timestamp, and it resolves
|
||||
itself when the node comes back.
|
||||
|
||||
### The host is delivered the same way as everything else
|
||||
|
||||
The host is tier 0, which makes it tempting to treat as special. It is not:
|
||||
|
||||
1. a push to `mesh-host` builds a binary;
|
||||
2. publish packages it and puts it in **the mesh's own package repository** — which is a
|
||||
directory of files behind the object store and the proxy, so it needs no new machinery;
|
||||
3. deploy updates each node's declaration to name the new version;
|
||||
4. **the new declaration is pushed to each node**, and the host applies
|
||||
`package: nox-mesh-host` on arrival, exactly as it applies any other package.
|
||||
|
||||
Step 4 is a push and not a poll. The link is already open
|
||||
([ADR 0001](0001-nodes-communicate-over-a-broker.md)), so a node learns of a change when it is
|
||||
made — a node that is offline learns on reconnect, which is what makes *outstanding* a real
|
||||
category rather than a euphemism for lost.
|
||||
|
||||
**No new resource type is needed**, which is the test of whether this is uniform or a special
|
||||
case wearing a uniform. The repository is reachable because a `file` resource put its address in
|
||||
the package manager's configuration — an ordinary declaration, applied by the same host.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **The most consequential boundary in the mesh gets smaller.** The as-is calls the fan-out
|
||||
*"the most consequential boundary in the mesh"* and documents a defect class from the build
|
||||
node having passed through two silos while others had not. **There is no fan-out**: deploy is
|
||||
one write, and the asymmetry it created cannot arise.
|
||||
- **Detection stays the fragile input, and this does not fix it.** *A merge that created no
|
||||
pipeline, and nothing said so* is upstream of everything here and is untouched.
|
||||
- **Rollback becomes a declaration change**, which is a real gain — the previous version is
|
||||
still named in the previous declaration — but nothing here designs how a previous declaration
|
||||
is retained or chosen.
|
||||
- **"Deployed" needs redefining wherever it is used**, because it now means *told*, and the
|
||||
useful fact is *applied on node X at time T*. Anything reporting deployment state has to move
|
||||
to the second, or it will report success for work that has not happened — the exact fault this
|
||||
record is closing, reintroduced at the reporting layer.
|
||||
- **A node offline for a long time applies a large jump at once**, having missed intermediate
|
||||
versions. That is correct — the declaration is a desired state, not a queue of changes — but a
|
||||
machine returning after months applies a very different declaration than it left with, and
|
||||
nothing tests that path.
|
||||
- **The pipeline stops being able to lie and starts being able to be incomplete.** That is a
|
||||
better failure mode and it is still a failure mode: a result that is honest about two
|
||||
outstanding nodes is only useful if somebody looks at it.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0014](0014-build-publish-and-deploy-are-three-silos.md) — the silos this keeps and
|
||||
redefines the third of.
|
||||
- [ADR 0037](0037-the-host-applies-it-does-not-decide.md) — the applier that makes this possible.
|
||||
- [ADR 0008](0008-a-failed-step-fails-the-job.md), [ADR 0035](0035-a-picture-is-read-from-what-runs.md) —
|
||||
why the host cannot report success it did not verify.
|
||||
- [`00-as-is/04`](../03-DESIGN/00-as-is/04-delivery.md) — the faults this addresses.
|
||||
@@ -0,0 +1,145 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-28
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
consolidates: [0008, 0013, 0014, 0063]
|
||||
---
|
||||
|
||||
# 58. Delivery
|
||||
|
||||
*Consolidated 2026-08-28 from five records.*
|
||||
|
||||
## Delivery is a comparison, not a pipeline
|
||||
|
||||
**The control plane holds what source exists and what has been built from it, and builds the
|
||||
difference.** A change becomes a build because source is **ahead of artifacts** — answerable at
|
||||
any moment — rather than because a message arrived.
|
||||
|
||||
**An event makes it fast. Nothing makes it necessary.** A missed notification costs latency and
|
||||
cannot cost correctness.
|
||||
|
||||
That is the same shape the host uses on a machine, one layer up:
|
||||
|
||||
| | reconciles | against |
|
||||
|---|---|---|
|
||||
| the control plane | artifacts | source |
|
||||
| the host | machine state | declarations |
|
||||
|
||||
**This is not the current coordinator repaired.** That is a state machine over stages; the value
|
||||
of it here is as a catalogue of the ways this fails, and it has been used for exactly that.
|
||||
|
||||
**What disappears is the pipeline as a state machine** — no stage list something can be omitted
|
||||
from, which is how a verify stage was built and never scheduled, and no run to lose.
|
||||
|
||||
### Currency is the whole input closure
|
||||
|
||||
**An artifact is out of date when its source moved, or anything it was built against moved.** So a
|
||||
shared library changing invalidates everything with a transitive build edge to it, in dependency
|
||||
order, because a module cannot be built against a new library until it exists.
|
||||
|
||||
**The module graph is a prerequisite of this, not an enabler of it.** Without it there is no
|
||||
rebuild set and no ordering, and this cannot be implemented.
|
||||
|
||||
## An artifact is build output, never a source tree
|
||||
|
||||
Compiled and bundled with its dependency graph inlined. **A deploy is extract-and-run and touches
|
||||
no network.**
|
||||
|
||||
The consequence is the whole cost of the decision: **anything not in the build output does not
|
||||
ship.** Every file kind had to be brought into that rule separately, and each was discovered by
|
||||
something silently not happening after a deploy — migrations reading a source layout,
|
||||
provisioning scripts reading a source layout, selection files never packaged at all.
|
||||
|
||||
## Three silos, and the third is not a stage
|
||||
|
||||
The cardinality observation holds and is what the split is for:
|
||||
|
||||
| silo | runs | ends with |
|
||||
|---|---|---|
|
||||
| **build** | once per module | a self-contained artifact |
|
||||
| **publish** | once per module | that artifact addressable — an image by digest, a package in the mesh's repository |
|
||||
| **deploy** | **once, not once per node** | the affected nodes' **declarations updated** |
|
||||
|
||||
**Deploy stops sending commands to nodes.** It changes what the control plane says each node
|
||||
should be, which is one write. What happens on the machines is the host's ordinary reconcile.
|
||||
|
||||
**Why this fixes the failure class rather than patching it.** Every recorded fault shares one
|
||||
shape: *the thing that reported success was not the thing that did the work.* A coordinator
|
||||
dispatching a command can only report on dispatch. Under this the reporter **is** the applier —
|
||||
which already refuses to record a resource until it read it back, and already fails the whole
|
||||
apply on one failed step.
|
||||
|
||||
**The verify stage disappears as a stage**, which is the strongest evidence for the shape:
|
||||
verification stops being a step that can be omitted from a list and becomes a property of applying
|
||||
at all.
|
||||
|
||||
**There is no fan-out**, so the defect class that came from the build node having passed through
|
||||
two silos while others had not cannot arise.
|
||||
|
||||
## A step that fails must fail the job
|
||||
|
||||
A step that fails and lets the job continue **reports success for work that did not happen**.
|
||||
Absence of an error is not evidence of an effect.
|
||||
|
||||
This is the mesh's most consistent failure shape, and it is not incidental — it is what stage
|
||||
reporting measured. Documented instances: a service reported started when the container command
|
||||
merely returned; an image pull failure that did not fail the deploy; a package install that 404'd
|
||||
from every mirror while the job went green; a node left on old code after a failed download with
|
||||
a version marker that had already advanced.
|
||||
|
||||
## The verdict is tiered
|
||||
|
||||
An artifact may not be declared until something has judged it fit. **Two tiers, because one gate
|
||||
would be both slow and unreliable:**
|
||||
|
||||
| | judged by | when |
|
||||
|---|---|---|
|
||||
| the module's own tests | the build | **always** — this is most of it |
|
||||
| the lab | a raised scenario | when an assertion genuinely needs a mesh |
|
||||
|
||||
A lab scenario takes tens of seconds and can fail for reasons that have nothing to do with the
|
||||
artifact, and a shared-library change produces a cascade of dozens. One expensive
|
||||
non-deterministic gate fails in both directions: a flaky run marks a good artifact unfit, a lucky
|
||||
one marks a bad artifact fit, and **neither failure looks like itself.**
|
||||
|
||||
**A run that failed environmentally is not a verdict.** A machine that would not boot says nothing
|
||||
about the artifact, and recording it as *unfit* is the same untruth as recording a dispatch as a
|
||||
deploy.
|
||||
|
||||
## What a result means
|
||||
|
||||
> **The declaration is updated, and here is which nodes have applied it.**
|
||||
|
||||
A pipeline does not wait for every node — one may be legitimately switched off for a week, and a
|
||||
delivery mechanism that blocks on a sleeping laptop is one nobody will use.
|
||||
|
||||
```
|
||||
delivered declaration updated for 5 nodes
|
||||
applied 3 of 5
|
||||
outstanding 2 — last seen 4 days ago, 20 minutes ago
|
||||
```
|
||||
|
||||
**Outstanding is not failure**, and conflating them is how the old system produced a stall with no
|
||||
error anywhere.
|
||||
|
||||
## What must exist first
|
||||
|
||||
1. **The module graph, with build edges.** No graph, no rebuild set and no ordering.
|
||||
2. **A recorded input closure per artifact**, so currency is answerable without building.
|
||||
3. **Something that notices a reconciler is not converging.** Below.
|
||||
|
||||
## Open, and the first is the real risk
|
||||
|
||||
- **A loop that will not converge is harder to debug than a job that failed.** A failed job stops
|
||||
and names its step; a reconciler retries forever. Without something that notices *this has been
|
||||
trying for an hour*, the failure is **silence** — the fault this removes, reintroduced in a new
|
||||
place.
|
||||
- **The run identity people use is lost.** *Did my change go out?* is answerable today by opening
|
||||
a pipeline. Something must replace that or this is worse to live with, whatever its properties.
|
||||
- **Does a fit artifact declare itself?** If it does, merging to main deploys to production —
|
||||
which may be wanted and is far too large a property to acquire by omission.
|
||||
- **Rebuild storms are mostly behaviourally empty.** Reproducible builds would stop a cascade at
|
||||
the first module whose output did not move; without them one commit redeploys the fleet for no
|
||||
change in behaviour.
|
||||
- **Detection stays the fragile input for latency**, though no longer for correctness.
|
||||
@@ -1,189 +0,0 @@
|
||||
---
|
||||
status: superseded
|
||||
superseded-by: 02-DECISIONS/0061-the-host-asks-an-init-for-start-and-restart.md
|
||||
date: 2026-08-27
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0057-the-host-is-a-root-service-installed-as-a-package.md
|
||||
---
|
||||
|
||||
# 59. Two watchdogs: the mesh stages the rollout, the supervisor recovers the node
|
||||
|
||||
## Context
|
||||
|
||||
[`09-the-node-lifecycle.md`](../03-DESIGN/01-to-be/09-the-node-lifecycle.md) left one thing
|
||||
unsolved: a host upgraded to a version that crashes on start. The supervisor restarts it, backs
|
||||
off, and the node is stuck on a binary that will not run.
|
||||
|
||||
It was left because automatic recovery looked like *the host judging its own health*, which is
|
||||
the self-reference the rest of that document avoids.
|
||||
|
||||
**That objection does not survive being asked properly.** A keepalive is not the host judging
|
||||
itself — it is something else judging the host.
|
||||
|
||||
### There are two watchdogs, and they cannot do each other's job
|
||||
|
||||
A first draft of this record concluded the watchdog must be local, and stopped there. That was
|
||||
half an answer: it is true that recovery must be local, and false that the mesh has no part.
|
||||
|
||||
| | can see | can act |
|
||||
|---|---|---|
|
||||
| **the service manager**, on the node | that *this* process keeps dying | **yes** — restart it, replace it |
|
||||
| **the mesh** | that *eleven of twelve nodes* went quiet after one declaration | **no** — nothing dials a node |
|
||||
|
||||
**Recovery must be local**, and that half stands:
|
||||
|
||||
- **Nothing dials a node.** [ADR 0039](0039-the-link-is-the-security-boundary.md) makes the link
|
||||
outbound and node-initiated, with no listening control surface. The mesh has no way to reach
|
||||
in and act.
|
||||
- **The failure removes the reporting path.** A host that cannot start cannot link, so the mesh
|
||||
learns nothing *from that node* to act on.
|
||||
|
||||
**But detection is the mesh's**, and it is the half a local watchdog structurally cannot do. A
|
||||
node's supervisor sees one process failing and has no idea whether that is a broken machine or a
|
||||
broken release. **Only something watching every node can tell those apart** — and telling them
|
||||
apart is what decides whether the right response is *fix this machine* or *stop shipping this
|
||||
version immediately*.
|
||||
|
||||
### The failure this actually prevents
|
||||
|
||||
Sharper than "the node is down", and it is the reason to bother:
|
||||
|
||||
> **A host that will not start looks exactly like a machine that was switched off.**
|
||||
|
||||
[ADR 0036](0036-a-node-is-a-managed-machine.md) makes disconnection ordinary, and
|
||||
[`09`](../03-DESIGN/01-to-be/09-the-node-lifecycle.md) deliberately puts no alarm on it — a
|
||||
laptop shut for three weeks is doing nothing wrong. **A bad upgrade is therefore invisible**: it
|
||||
presents as the one condition the design has decided not to be alarmed by.
|
||||
|
||||
Worse, it arrives fleet-wide. A declaration naming a bad version reaches every node, and each
|
||||
one applies it, restarts, and stops talking. The mesh would report a fleet of quiet nodes and
|
||||
nothing else.
|
||||
|
||||
## Decision
|
||||
|
||||
**The mesh stages the rollout and stops when nodes go quiet. The service manager recovers the
|
||||
node it is on.** Prevention and recovery, and neither substitutes for the other.
|
||||
|
||||
### The mesh stages a host rollout
|
||||
|
||||
A host version does not reach every node at once. The delivery context updates a few nodes'
|
||||
declarations, **waits for those nodes to heartbeat on the new version**, and only then continues.
|
||||
|
||||
```
|
||||
update 2 nodes ─► heard from both, running the new version ─► continue
|
||||
└► silence past the window ─► STOP. Report.
|
||||
```
|
||||
|
||||
**Silence is the signal, and it is available because of the heartbeat**
|
||||
([ADR 0057](0057-the-host-is-a-root-service-installed-as-a-package.md)). A node that upgraded
|
||||
and cannot start stops reporting; that is indistinguishable from a switched-off machine *for one
|
||||
node*, and completely distinguishable across a batch that was all told the same thing at the
|
||||
same time.
|
||||
|
||||
**This is what keeps a bad release from becoming a fleet outage.** Local rollback repairs a node
|
||||
after the fact; staging means most nodes never receive the bad version at all. A stopped rollout
|
||||
is two broken nodes and a report, rather than every node quiet at once.
|
||||
|
||||
**It does not replace local recovery**, for two reasons. The canary nodes still break, and
|
||||
somebody has to be able to fix them. And a node that was offline during the staged rollout gets
|
||||
the declaration when it reconnects, with no batch around it and nothing watching — so it must be
|
||||
able to recover alone.
|
||||
|
||||
### On the node: the service manager rolls back
|
||||
|
||||
Four parts, and each one is chosen so it works when the host does not:
|
||||
|
||||
### 1 — A version is confirmed by starting, not by seeming well
|
||||
|
||||
On start, the host completes one full reconcile. If it does, it writes the running version to a
|
||||
plain file — `/var/lib/mesh-host/known-good` — and that is the whole of "confirmed".
|
||||
|
||||
**Deliberately not health.** Not *the link is up*, because a disconnected node is ordinary and a
|
||||
laptop on a train would roll itself back. Not *everything applied cleanly*, because a resource
|
||||
that fails is the machine's problem and not the binary's. The claim is narrow and checkable: **it
|
||||
started, and it got through a reconcile.**
|
||||
|
||||
### 2 — The rollback is not the host binary
|
||||
|
||||
The obvious mistake, and it would make the whole mechanism a no-op: `nox-mesh-host rollback`
|
||||
cannot be the recovery path for a `nox-mesh-host` that does not run.
|
||||
|
||||
Rollback is a **small script shipped by the package**, which reads the `known-good` file and
|
||||
asks the package manager to install that version. It shares no code with the host and does not
|
||||
import it.
|
||||
|
||||
### 3 — The supervisor triggers it, after giving up
|
||||
|
||||
```ini
|
||||
[Service]
|
||||
Restart=always
|
||||
StartLimitBurst=3
|
||||
StartLimitIntervalSec=120
|
||||
|
||||
[Unit]
|
||||
OnFailure=nox-mesh-host-rollback.service
|
||||
```
|
||||
|
||||
**`Restart=always`, not `on-failure`**, and the difference is load-bearing rather than a
|
||||
preference. [ADR 0057](0057-the-host-is-a-root-service-installed-as-a-package.md) has the host
|
||||
restart onto a new binary by **exiting cleanly** — and `on-failure` does not restart a process
|
||||
that exited zero. An earlier draft of this record specified `on-failure` and would have left
|
||||
every upgraded node stopped, having successfully upgraded. Caught by reading the two records
|
||||
against each other rather than by either alone.
|
||||
|
||||
Three failures in two minutes is a binary that does not work, not a transient. The supervisor
|
||||
stops trying, the unit enters a failed state, and `OnFailure` runs the rollback unit — which
|
||||
downgrades and starts the host again.
|
||||
|
||||
### 4 — With nothing to roll back to, it does not try
|
||||
|
||||
A machine whose host has *never* completed a reconcile has no `known-good`. The rollback unit
|
||||
finds nothing, does nothing, and says so.
|
||||
|
||||
That is the right outcome: there is no previous version, so the node was never working, and the
|
||||
failure belongs to the installation rather than to an upgrade. Attempting a rollback here would
|
||||
mean guessing at a version, which is how a recovery mechanism becomes a second fault.
|
||||
|
||||
### 5 — It rolls back once
|
||||
|
||||
The rollback unit records that it fired. If the rolled-back version *also* fails to start, it
|
||||
does **not** fire again — the node stops, loudly, in a failed state.
|
||||
|
||||
**Because a second rollback is a different diagnosis.** Once the previously-working binary also
|
||||
fails, the binary is not the problem: the machine is. Rolling back further would flap between
|
||||
two versions forever and bury the actual cause under a loop.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **The node keeps its own recovery**, which is the property the whole tier-0 design rests on:
|
||||
the host depends on nothing, and now its recovery depends on nothing either.
|
||||
- **The package cache must retain the previous version**, and that is a real requirement rather
|
||||
than an assumption — a package manager configured to clean its cache would delete the thing
|
||||
rollback needs. The package must pin that, and it is the sort of dependency that is discovered
|
||||
by the rollback failing.
|
||||
- **A fleet-wide bad upgrade becomes self-limiting.** Each node fails, rolls back, and comes
|
||||
back on the previous version — so the blast radius of a bad host release is one restart cycle
|
||||
per node rather than the entire fleet stopping.
|
||||
- **It catches "will not start" and nothing else.** A version that starts and is subtly wrong
|
||||
will not roll back, and should not: that is a bad release, which is delivery's problem
|
||||
([ADR 0058](0058-delivery-ends-in-a-declaration.md)), not a supervision problem.
|
||||
- **It adds a unit and a script the host does not own**, which is
|
||||
[ADR 0057](0057-the-host-is-a-root-service-installed-as-a-package.md)'s line holding: the
|
||||
installation owns the host, and now it owns the host's recovery too. Consistent, and it means
|
||||
both arrive and are versioned together.
|
||||
- **A node in permanent failure is silent, and that is now the last gap.** After a second
|
||||
failure the node is stopped and cannot report it. What notices is the mesh seeing a node that
|
||||
has not been heard from — which `09` records as a fact with no threshold, and which this makes
|
||||
more important to look at than it was.
|
||||
- **The rollback path is exercised only when it is needed**, which is when nobody can afford it
|
||||
to be wrong. It wants a test that boots a lab node onto a deliberately broken host and asserts
|
||||
the previous version comes back.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0057](0057-the-host-is-a-root-service-installed-as-a-package.md) — the restart-by-exiting
|
||||
this protects.
|
||||
- [ADR 0039](0039-the-link-is-the-security-boundary.md) — why the watchdog cannot be remote.
|
||||
- [ADR 0036](0036-a-node-is-a-managed-machine.md) — why the failure is invisible without this.
|
||||
- [`09-the-node-lifecycle.md`](../03-DESIGN/01-to-be/09-the-node-lifecycle.md) — the gap this closes.
|
||||
@@ -1,144 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-28
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0057-the-host-is-a-root-service-installed-as-a-package.md
|
||||
---
|
||||
|
||||
# 60. The host is built per operating system
|
||||
|
||||
## Context
|
||||
|
||||
Three of the host's six shapes need something from the machine: `service` needs a service
|
||||
manager, `package` needs a package manager, `container` needs a container runtime. The other
|
||||
three — `file`, `directory`, `action` — need only a filesystem and the ability to run something.
|
||||
|
||||
The host names those capabilities generically and implements them specifically. The detector
|
||||
reports `container-runtime`, `package-manager`, `service-manager`; the appliers call `docker`,
|
||||
`pacman` and `systemctl`. **So the design says capability and the code says Arch**, and nothing
|
||||
records which of those is the intent.
|
||||
|
||||
The question that surfaced it: what happens on a machine that has podman, or one that does not
|
||||
run systemd? And underneath it, a live contradiction —
|
||||
[research 012](../01-RESEARCH/012-the-minimum-viable-node/00-overview.md) decides that on
|
||||
conflict during adoption *the machine's configuration is kept*, so a machine with podman keeps
|
||||
podman, and then the container applier calls `docker` and fails.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **Abstract each capability behind an interface.** One host, adapters per service manager and
|
||||
package manager. Rejected, and the reason is correctness rather than effort: the service
|
||||
applier reads `LoadState` to tell *not installed* apart from *stopped*, which is what stops it
|
||||
reporting absence as success. An interface spanning systemd and OpenRC degrades to what both
|
||||
can express, and **the lowest common denominator is exactly where that fault lives**.
|
||||
2. **Support one operating system and say so.** Honest, and it makes every other machine
|
||||
permanently out of scope rather than not-yet.
|
||||
3. **A host per operating system.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
**The host is built for an operating system family, and `systemd` and `pacman` are the Arch
|
||||
host's implementation rather than abstractions the mesh has to grow.**
|
||||
|
||||
```
|
||||
mesh-host-arch-x86_64 pacman · systemctl · docker
|
||||
mesh-host-debian-x86_64 apt · systemctl · docker (when there is a machine)
|
||||
mesh-host-alpine-x86_64 apk · rc-service · podman (when there is a machine)
|
||||
```
|
||||
|
||||
**These are not independent choices and treating them as such was the error.** A machine has
|
||||
pacman *because* it is Arch. The package manager, the service manager and the packaging format
|
||||
arrive together, as one decision somebody made when they installed the operating system.
|
||||
|
||||
### Almost all of it is shared
|
||||
|
||||
Not a rewrite per operating system. The declaration vocabulary, the store, the apply loop, the
|
||||
read-back discipline, the refusal model and the link are all portable. **What differs is two
|
||||
appliers**, and the rest is compiled around them.
|
||||
|
||||
**The bundle is the exception, and an earlier version of this record wrongly listed it as
|
||||
portable.** Its *mechanism* is — one embedded declaration, applied with no mesh present. Its
|
||||
*contents* are not: package names, unit names and service names all differ, so an Arch host
|
||||
embeds an Arch bundle and an Alpine host an Alpine one. That is the same thing this record says
|
||||
about package names one section down, and missing it here is what made the distinction hard to
|
||||
see.
|
||||
|
||||
### The control plane names the package, because the host does not decide
|
||||
|
||||
A container runtime is `docker` on Arch and `docker.io` on Debian. Mapping *this node needs a
|
||||
container runtime* to a package name is **deciding**, which
|
||||
[ADR 0037](0037-the-host-applies-it-does-not-decide.md) puts outside the host.
|
||||
|
||||
It needs no new mechanism: the profile already reports what the machine is, so the declaration a
|
||||
node receives is already tailored to that node. The host receives a package name and installs it.
|
||||
|
||||
### A host that cannot implement a shape refuses it
|
||||
|
||||
The interesting case is not Debian, it is **Android** — no service manager it will lend us, no
|
||||
package installation, usually no root. Such a host implements `file`, `directory` and `action`,
|
||||
and nothing else.
|
||||
|
||||
That needs no new mechanism either. A host already refuses a type it does not know; *this host
|
||||
does not implement `package`* is the same refusal with a different reason, and the profile
|
||||
reports which shapes it implements so the control plane never sends one it cannot do.
|
||||
|
||||
**`file`, `directory` and `action` are the portable floor.** They work anywhere there is a
|
||||
filesystem and a way to run something, which makes a partial host a real thing rather than a
|
||||
broken one.
|
||||
|
||||
**A partial host can join a mesh and cannot be the first node.** Every step of raising a
|
||||
substrate is a `package`, a `container`, or an `action` against one, so the shapes it refuses
|
||||
are exactly the ones a bootstrap needs. Its bundle says so rather than being an empty
|
||||
placeholder.
|
||||
|
||||
**How such a host is started was left open here and is closed by
|
||||
[ADR 0062](0062-a-host-may-be-episodic.md)** — by narrowing what is required rather than
|
||||
building something. A host may be *episodic* rather than resident, and being killed by the
|
||||
platform is disconnection, which is already ordinary.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **Each implementation stays as sharp as its operating system allows.** The `LoadState`
|
||||
distinction survives because the Arch host knows it is systemd. Nothing is degraded to fit an
|
||||
interface spanning systems we do not run.
|
||||
- **Delivery already worked this way**, which is the strongest sign this is the right seam: the
|
||||
host ships as a package from the mesh's own repository
|
||||
([ADR 0058](0058-delivery-ends-in-a-declaration.md)), and a `.pkg.tar.zst` is an Arch artifact.
|
||||
A per-OS binary is consistent with what was already decided rather than an addition to it.
|
||||
- **The container runtime is a choice within a host, not an OS split** — Arch runs docker or
|
||||
podman — and it is **detected, not declared**, because adoption keeps what the machine already
|
||||
has. Two are supported.
|
||||
|
||||
This is the opposite answer to the one above, for a reason rather than by preference.
|
||||
Abstracting service managers is *lossy*: systemd and OpenRC are different models, and
|
||||
`LoadState` has no equivalent. Container runtimes deliberately converged on one CLI, so almost
|
||||
nothing is lost — checked against podman 6.1.0, `run`, `rm -f` and docker's own template
|
||||
syntax for reading state and labels all work unchanged. **Only the probe differs**
|
||||
(`{{.ServerVersion}}` against `{{.Version.Version}}`), which makes it a two-entry lookup
|
||||
rather than an interface.
|
||||
|
||||
**One difference is not in the CLI and would have shipped silently.** Podman accepts
|
||||
`--restart unless-stopped` and records it, and has no daemon to act on it: containers do not
|
||||
come back after a reboot unless `podman-restart.service` is enabled, which it is not by
|
||||
default. Every command reports success and the effect does not happen. That belongs in the
|
||||
**declaration** — a node using podman is told to enable that unit — rather than in the host,
|
||||
which keeps the host dumb and puts the difference where a person can read it.
|
||||
- **A second operating system is now additive rather than a redesign** — write two appliers, ship
|
||||
a package. And it will be designed against a real machine rather than a guess, which is the
|
||||
point of not building the abstraction now.
|
||||
- **The profile has to report the operating system**, and today it reports capabilities without
|
||||
saying which system they belong to. Small, and needed before the control plane can tailor a
|
||||
package name.
|
||||
- **Nothing states the machine requirements yet.** A machine missing a capability fails at apply
|
||||
time rather than being refused up front, even though the host already detects it. That is a
|
||||
gap this record makes visible and does not close.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0037](0037-the-host-applies-it-does-not-decide.md) — why the host is handed a package name.
|
||||
- [ADR 0041](0041-the-host-depends-on-nothing.md) — one static binary, now per system as well as
|
||||
per architecture.
|
||||
- [ADR 0058](0058-delivery-ends-in-a-declaration.md) — delivery, which was already per-OS.
|
||||
- [Research 012](../01-RESEARCH/012-the-minimum-viable-node/00-overview.md) — adoption keeping
|
||||
the machine's configuration, which hardcoding a runtime contradicts.
|
||||
@@ -1,130 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-28
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
supersedes: 0059-a-host-that-cannot-start-rolls-itself-back.md
|
||||
extends: 0060-the-host-is-built-per-operating-system.md
|
||||
---
|
||||
|
||||
# 61. The host asks an init for start and restart, and nothing else
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0059](0059-a-host-that-cannot-start-rolls-itself-back.md) built the node's recovery out of
|
||||
systemd's own features: `StartLimitBurst` to decide a binary is broken, `OnFailure` to run a
|
||||
rollback unit. It works, and it makes recovery — the thing that matters most when a node is
|
||||
stuck — the most systemd-specific part of the whole host.
|
||||
|
||||
[ADR 0060](0060-the-host-is-built-per-operating-system.md) makes the host per operating system,
|
||||
which raises the obvious question: how much of an init does the host actually need?
|
||||
|
||||
Counted honestly, three things, and only one of them is special:
|
||||
|
||||
| | any init? |
|
||||
|---|---|
|
||||
| start at boot | **yes** |
|
||||
| restart it when it exits | **yes** |
|
||||
| give up after N failures and run something else | **no** — that is systemd's `StartLimitBurst` and `OnFailure` |
|
||||
|
||||
So the recovery mechanism is the only reason the host needs *this* init rather than *an* init.
|
||||
And it is the part that must work on a machine where the host does not, which makes "it is
|
||||
expressed in unit-file syntax" a poor place for it: unit syntax is not something we can test, and
|
||||
the one time it runs is the one time nobody can afford it to be wrong.
|
||||
|
||||
**The substrate does not need systemd either**, which is what makes this worth doing rather than
|
||||
merely tidy. The bootstrap is package → container → action → container, and every mesh workload
|
||||
is a container the runtime restarts. Nothing in it declares a `service`.
|
||||
|
||||
## Decision
|
||||
|
||||
**An init is asked for one thing: start this at boot.**
|
||||
|
||||
An earlier version of this record asked for two — start, and restart on exit — and left the
|
||||
restart in the unit file while moving the give-up logic out. That was half a change: it kept the
|
||||
init deciding *when the host comes back*, which is the thing being removed. **The launcher does
|
||||
not exec the host; it supervises it**, so restarting is ours as well.
|
||||
|
||||
The cost of not exec'ing is signals, and it is the reason people reach for a service manager in
|
||||
the first place. A supervisor that exits while its child is still running leaves the host to be
|
||||
*killed* rather than to *stop*, and an apply interrupted that way is the half-configured machine
|
||||
this project is about. So the launcher traps the shutdown signal, passes it to the host, and
|
||||
waits.
|
||||
|
||||
**Everything else moves into a launcher**, which is what the init actually starts:
|
||||
|
||||
```
|
||||
init ──► nox-mesh-host-launch ──► nox-mesh-host (a child, not an exec)
|
||||
│
|
||||
└─ loop:
|
||||
halted? say so and stop; a person has to look
|
||||
too many failures? roll back once, then halt
|
||||
start the host, and wait
|
||||
exited 0 it upgraded itself — start the new binary,
|
||||
and do NOT count it
|
||||
crashed count it, back off, loop
|
||||
shutting down pass the signal down, wait, exit
|
||||
|
||||
host, on a completed reconcile ──► clears the counter, records known-good
|
||||
```
|
||||
|
||||
**The counter counts consecutive failures, not starts**, and the difference is not cosmetic.
|
||||
Counting starts meant a host that upgraded itself three times rolled itself back, having worked
|
||||
perfectly every time — the clean exit *is* the upgrade path
|
||||
([ADR 0057](0057-the-host-is-a-root-service-installed-as-a-package.md)).
|
||||
|
||||
**This keeps everything ADR 0059 decided and changes only where it lives.** Two watchdogs still,
|
||||
and neither substitutes for the other: the mesh stages a host rollout and stops when nodes go
|
||||
quiet; the node recovers itself. Recovery is still local, because nothing dials a node and a host
|
||||
that cannot start cannot report. It rolls back once, because a second failure of a
|
||||
previously-working binary is a different diagnosis. The rollback still shares no code with the
|
||||
host, because a binary that will not start cannot be its own recovery.
|
||||
|
||||
### What is gained by moving it
|
||||
|
||||
- **It becomes testable.** A shell script with a counter can be run against a stub package
|
||||
manager and asserted, which is how the rollback script is already tested. `OnFailure=` can be
|
||||
read and hoped for.
|
||||
- **The host becomes runnable under any init**, which is what makes an Alpine or Android host
|
||||
possible later rather than blocked on porting the recovery.
|
||||
- **The give-up policy stops being configuration and becomes code we own.** Three attempts is a
|
||||
decision, and it should live where decisions are read and tested.
|
||||
|
||||
### What it costs
|
||||
|
||||
- **One more process in the chain**, and it runs before the host on every start.
|
||||
- **The counter is the whole mechanism, and it is the part to get right.** Never cleared, and
|
||||
the node rolls back on a healthy boot; cleared too eagerly, and it never rolls back at all.
|
||||
It is cleared by the host on a **completed reconcile** — the same event that records
|
||||
known-good, for the same reason.
|
||||
- **The launcher is a thing that can itself be broken**, and nothing recovers it. That is one
|
||||
turtle down from where we were, not zero: the alternative was unit syntax, which also cannot
|
||||
recover itself and additionally cannot be tested.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **A clean exit is the upgrade path, and it is the easiest thing to get wrong.** Twice now:
|
||||
ADR 0059 specified `on-failure`, which would have left every upgraded node stopped; and the
|
||||
first supervising loop counted a clean exit as a failure, which would have rolled back a host
|
||||
that upgraded itself three times. Anything touching restart has to ask what a zero exit means
|
||||
here.
|
||||
- **`Restart=` in the unit file becomes a backstop, not the mechanism.** It brings the launcher
|
||||
back if the launcher itself is killed. It no longer decides anything about the host.
|
||||
- **The unit file becomes trivial**, which is the point: start, restart, a state directory.
|
||||
Nothing in it encodes policy, so porting it is transcription rather than design.
|
||||
- **A halted node is silent**, unchanged from 0059 and still the last gap. What notices is the
|
||||
mesh seeing a node it has not heard from.
|
||||
- **The rollback script grows into a launcher** rather than being replaced. Its tested behaviour —
|
||||
roll back once, refuse to guess with no known-good, fail loudly when the package is not
|
||||
cached — carries over and is where the new counter logic joins it.
|
||||
- **`service` survives as a shape**, and this record does not remove it. Almost nothing in the
|
||||
design declares one, but adoption takes over machines already in use whose units somebody
|
||||
chose, and saying the mesh may never manage those is a larger decision than this one.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0059](0059-a-host-that-cannot-start-rolls-itself-back.md) — superseded; its reasoning
|
||||
about two watchdogs, rolling back once, and recovery being local is kept in full.
|
||||
- [ADR 0060](0060-the-host-is-built-per-operating-system.md) — why this question was asked.
|
||||
- [ADR 0057](0057-the-host-is-a-root-service-installed-as-a-package.md) — restart-by-exiting,
|
||||
which constrains what the init must do.
|
||||
@@ -1,101 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-28
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0060-the-host-is-built-per-operating-system.md
|
||||
---
|
||||
|
||||
# 62. A host may be episodic, and being killed is ordinary
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0060](0060-the-host-is-built-per-operating-system.md) makes a partial host a real thing —
|
||||
Android implements `file`, `directory` and `action` and refuses the rest — and leaves one gap
|
||||
open, named but not closed:
|
||||
|
||||
> **Being STARTED on Android is not solved by this file, and it is the real gap.** Everywhere
|
||||
> else an init runs the launcher at boot. Here the equivalent is the app framework — a
|
||||
> foreground service, or something under Termux — both of which the system may kill when it
|
||||
> wants memory.
|
||||
|
||||
There is no way to keep a process running on an ordinary Android device. Registering with init
|
||||
needs root and an unlocked bootloader. A foreground service is the sanctioned alternative and is
|
||||
still subject to the system reclaiming memory, to Doze, and to whatever the manufacturer added
|
||||
on top. **The correct model is not a daemon that occasionally dies; it is something that runs
|
||||
when it is allowed to.**
|
||||
|
||||
[ADR 0061](0061-the-host-asks-an-init-for-start-and-restart.md) asks an init for *start at boot*
|
||||
and has a launcher supervise the host. Android grants neither half: nothing to ask, and nothing
|
||||
worth supervising, because the supervisor would be killed alongside what it supervises.
|
||||
|
||||
## Decision
|
||||
|
||||
**A host is either resident or episodic, and both are hosts.**
|
||||
|
||||
| | resident | episodic |
|
||||
|---|---|---|
|
||||
| started by | an init, at boot | whatever the platform allows — an app's foreground service, a scheduled wake |
|
||||
| supervised by | the launcher ([ADR 0061](0061-the-host-asks-an-init-for-start-and-restart.md)) | nothing; the platform decides when it runs |
|
||||
| the link | held open | opened while it runs |
|
||||
| stopping | shutdown, or a failure | **ordinary, and needs no explanation** |
|
||||
|
||||
**Being killed is not a failure to detect. It is disconnection**, which
|
||||
[ADR 0036](0036-a-node-is-a-managed-machine.md) already made an ordinary situation rather than
|
||||
an exception — *a node that is switched off, roaming, or behind a connection that has dropped
|
||||
has not become a lesser kind of thing.* An episodic host is that, more often.
|
||||
|
||||
This needs no new mechanism, and that is the argument for it. The store is already authoritative
|
||||
while disconnected. Reconcile already happens on start. The mesh already reports *last heard
|
||||
from* rather than alarming on silence
|
||||
([`09-the-node-lifecycle.md`](../03-DESIGN/01-to-be/09-the-node-lifecycle.md)). Every one of
|
||||
those was decided for laptops that close, and an episodic host is the same case with a shorter
|
||||
period.
|
||||
|
||||
### What an episodic host does not have
|
||||
|
||||
- **No launcher.** There is nothing to supervise it and nothing for it to supervise. The
|
||||
platform starts it; the platform stops it.
|
||||
- **No rollback.** [ADR 0061](0061-the-host-asks-an-init-for-start-and-restart.md)'s recovery
|
||||
reinstalls a previous package, and an episodic host has no package manager to reinstall from.
|
||||
A bad version is replaced the way the platform replaces applications.
|
||||
- **No bundle, and therefore no bootstrap.** Every step of raising a substrate is a shape a
|
||||
partial host refuses, so **a partial host can join a mesh and cannot be the first node.** The
|
||||
android bundle says exactly that instead of being an empty placeholder.
|
||||
|
||||
### What it still is
|
||||
|
||||
A node. It has an identity, it holds a store, it applies declarations, it reads back and
|
||||
reports. It is reachable in the inventory, it can be assigned work of the kinds it supports, and
|
||||
it is not a second class of thing in the model —
|
||||
[ADR 0036](0036-a-node-is-a-managed-machine.md) is explicit that reachability is state rather
|
||||
than class, and this is that rule doing the work it was written for.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **The gap 0060 left is closed by narrowing what is required, not by building something.** No
|
||||
Android daemon, no keep-alive service, no fighting the platform's process management — which
|
||||
would be a losing fight and a permanent source of bugs.
|
||||
- **The heartbeat matters more and means less.** An episodic host reports when it runs, so *last
|
||||
heard from* on a phone is a much weaker signal than on a server. Anything reading that fact
|
||||
has to know which kind of host it is looking at, or a healthy phone reads as a dead node.
|
||||
- **A declaration may take a long time to land**, because the node applies it only when the
|
||||
platform next runs it. *Outstanding* was already separated from *failed*
|
||||
([ADR 0058](0058-delivery-ends-in-a-declaration.md)) and this makes that separation
|
||||
load-bearing rather than tidy.
|
||||
- **How an episodic host is actually started is still platform work and is not designed here.**
|
||||
An APK with a foreground service, or Termux with its boot addon — both are real, both have
|
||||
costs, and choosing between them wants an actual device and an actual purpose for it.
|
||||
- **What an Android node is FOR remains unanswered**, and it should be answered before the
|
||||
platform work is done. A device that can write files and run commands is not a workload host;
|
||||
it is a presence, or somewhere an agent runs. Building the start mechanism before deciding
|
||||
that would be building it for nobody.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0036](0036-a-node-is-a-managed-machine.md) — disconnection as an ordinary situation,
|
||||
which this is an instance of rather than an extension to.
|
||||
- [ADR 0060](0060-the-host-is-built-per-operating-system.md) — partial hosts, and the gap this
|
||||
closes.
|
||||
- [ADR 0061](0061-the-host-asks-an-init-for-start-and-restart.md) — what a resident host has
|
||||
that this one does not.
|
||||
@@ -1,134 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-28
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0058-delivery-ends-in-a-declaration.md
|
||||
---
|
||||
|
||||
# 63. Delivery is reconciliation, not a pipeline
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0058](0058-delivery-ends-in-a-declaration.md) stopped *deploy* being a stage that pushes to
|
||||
nodes: the control plane says what should be true, the host reconciles, and the thing that
|
||||
reports is the thing that did the work. It fixed the third silo and left the first two as they
|
||||
were — **and it said plainly what it did not fix:**
|
||||
|
||||
> **Detection stays the fragile input, and this does not fix it.** *A merge that created no
|
||||
> pipeline, and nothing said so* is upstream of everything here and is untouched.
|
||||
|
||||
That is not a defect in the detector. It is what happens when a system's correctness depends on
|
||||
an **event arriving**. The as-is records the ways it has failed — a webhook truncating its commit
|
||||
list on a large merge, a forge address whose port broke module-path matching — and the shape is
|
||||
always the same: *nothing happened, and nothing said so.*
|
||||
|
||||
**The same move that fixed deploy fixes this, applied one level up.** This record is not a
|
||||
better detector. It is the removal of detection as a load-bearing mechanism.
|
||||
|
||||
**And this is not the current coordinator repaired.** The existing pipeline is a state machine
|
||||
over stages; what follows is not that with better inputs. The old system's value here is as a
|
||||
catalogue of the ways this can fail, and it has been used for exactly that.
|
||||
|
||||
## Decision
|
||||
|
||||
> **The control plane holds what source exists and what has been built from it, and builds the
|
||||
> difference.**
|
||||
|
||||
A change becomes a build because **source is ahead of artifacts** — a comparison, answerable at
|
||||
any moment — rather than because a message arrived.
|
||||
|
||||
**An event makes it fast. Nothing makes it necessary.** A push notification is an optimisation
|
||||
that lowers latency; a missed one costs latency and cannot cost correctness. That is the same
|
||||
property the host's drift timer has, and it is the whole point of both.
|
||||
|
||||
So the mesh is one idea at two layers:
|
||||
|
||||
| | reconciles | against |
|
||||
|---|---|---|
|
||||
| **the control plane** | artifacts | source |
|
||||
| **the host** | machine state | declarations |
|
||||
|
||||
**What disappears:** the pipeline as a state machine. There is no stage list something can be
|
||||
omitted from — which is how a verify stage was built and never scheduled — and no run to lose.
|
||||
|
||||
## What this answers
|
||||
|
||||
[Research 008](../01-RESEARCH/008-delivery-coordinator/00-overview.md) asked six questions. Two
|
||||
were answered by [ADR 0058](0058-delivery-ends-in-a-declaration.md); this answers the rest.
|
||||
|
||||
**What is a deployed state?** Not an event — **two comparisons**, both answerable on demand: does
|
||||
every node's reported state match what is declared, and is what is declared built from current
|
||||
source? A milestone can be claimed by something that did not check. A comparison cannot.
|
||||
|
||||
**What produces a verdict, and what is it about?** **An artifact, and it gates eligibility.** The
|
||||
mesh must not converge onto something broken, so an artifact may be declared only once something
|
||||
has judged it fit. The lab is what judges. This is sharper than the question expected: a verdict
|
||||
is not a report about a run, it is a property an artifact does or does not have.
|
||||
|
||||
**How does delivery work before self-hosting?** It mostly stops being a question. A reconciler
|
||||
needs to read source and write artifacts; where those live is a **binding**, external at first
|
||||
and internal later. A pipeline has stages that name their targets, which is why the transition
|
||||
looked hard.
|
||||
|
||||
## What survives from ADR 0014
|
||||
|
||||
[ADR 0014](0014-build-publish-and-deploy-are-three-silos.md) is a decision about **cardinality**,
|
||||
and the cardinality observation is right and unchanged: building is per module, publishing is per
|
||||
module, and what happens on nodes is per node. What changes is that those are no longer three
|
||||
**silos of a job**. They are steps of reconciling one artifact, and the third is not a step at all
|
||||
any more ([ADR 0058](0058-delivery-ends-in-a-declaration.md)).
|
||||
|
||||
## What this costs
|
||||
|
||||
Named because each is a way this can go wrong, and a decision that lists none has not been
|
||||
examined.
|
||||
|
||||
- **"Is this artifact current?" must be answerable without building it.** A commit recorded
|
||||
against each artifact does it, and that record becomes load-bearing: wrong, and the mesh either
|
||||
rebuilds forever or never rebuilds at all.
|
||||
- **Rebuild storms are real and mostly behaviourally empty.** One shared-library commit
|
||||
invalidates nearly everything, and most of those rebuilds produce artifacts that do the same
|
||||
thing they did before — so **the fleet is redeployed for no change in behaviour.** Reproducible
|
||||
builds would stop the cascade at the first module whose output did not move; without them, the
|
||||
storm is in the declarations rather than the builds
|
||||
([ADR 0064](0064-a-build-edge-is-a-third-kind.md)).
|
||||
- **The run identity people actually use is lost.** *Did my change go out?* is answerable today
|
||||
by opening a pipeline. With convergence there is no run to open, and **something has to replace
|
||||
that** — a query over the two comparisons above — or this will be worse to live with than what
|
||||
it replaces, whatever its properties.
|
||||
- **A loop that will not converge is harder to debug than a job that failed.** A failed job stops
|
||||
and names its step. A reconciler that cannot reach its target retries forever, and without
|
||||
something that notices *this has been trying for an hour*, the failure is silence — which is
|
||||
the fault this record is removing, reintroduced in a new place. **This is the real risk and it
|
||||
is not solved here.**
|
||||
|
||||
## What must exist first
|
||||
|
||||
Stated as a list because this record cannot be implemented without them, and saying so is better
|
||||
than discovering it:
|
||||
|
||||
1. **The module graph, including build edges** ([ADR 0064](0064-a-build-edge-is-a-third-kind.md)).
|
||||
Designed, not built. Without it there is no rebuild set and no ordering.
|
||||
2. **A recorded input closure per artifact** — its commit and the identity of everything it was
|
||||
built against — so *is this current?* is answerable without building.
|
||||
3. **Something that notices a reconciler is not converging.** Below, and the one that is a risk
|
||||
rather than a cost.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **Detection stops being correctness and becomes latency.** The specific faults the as-is
|
||||
records — a truncated commit list, a broken path match — become slow rather than silent.
|
||||
- **The coordinator is not ported.** What replaces it is a comparison and a build, and the
|
||||
existing implementation informs it only as a list of things that went wrong.
|
||||
- **Two things must be cheap that are not yet designed**: reading what source exists, and reading
|
||||
what has been built. Both are queries against providers the mesh will host, and both are on the
|
||||
path of everything above.
|
||||
- **Research 008 can close**, which it could not before this.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0058](0058-delivery-ends-in-a-declaration.md) — the same move, one level down.
|
||||
- [ADR 0014](0014-build-publish-and-deploy-are-three-silos.md) — the cardinality that survives.
|
||||
- [ADR 0035](0035-a-picture-is-read-from-what-runs.md) — why a comparison beats a claim.
|
||||
- [`00-as-is/04`](../03-DESIGN/00-as-is/04-delivery.md) — the pitfalls this is designed against.
|
||||
@@ -1,91 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-28
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0044-a-module-declares-presence-instantiation-and-exclusion.md
|
||||
---
|
||||
|
||||
# 64. A build edge is a third kind, and it is the one delivery runs on
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0044](0044-a-module-declares-presence-instantiation-and-exclusion.md) settled what a module
|
||||
declares, and [research 011](../01-RESEARCH/011-the-module-graph/00-overview.md) established the
|
||||
edges: **presence** — the thing exists and is reachable — and **instantiation** — the provider is
|
||||
asked to make something for this consumer and hands back credentials.
|
||||
|
||||
Both are **runtime** edges. They answer *the board needs a database from PostgreSQL.*
|
||||
|
||||
[ADR 0063](0063-delivery-is-reconciliation-not-a-pipeline.md) needs a different question
|
||||
answered: *what has to be rebuilt when this changes?* And that is not something either edge can
|
||||
say. The board being compiled against a shared library is not presence and not instantiation —
|
||||
nothing is provisioned, nothing hands back a credential, and the relationship is fixed inside the
|
||||
artifact rather than negotiated when it runs.
|
||||
|
||||
**So the graph as designed cannot drive delivery**, and finding that out is what stopped ADR 0063
|
||||
being approved as written.
|
||||
|
||||
## Decision
|
||||
|
||||
**A build edge is a third kind: this artifact was compiled against that one.**
|
||||
|
||||
It differs from the runtime edges in the way that matters, which is why it cannot be folded into
|
||||
them:
|
||||
|
||||
| | runtime edges | **build edge** |
|
||||
|---|---|---|
|
||||
| when it is satisfied | at provisioning, and again whenever it must be | **at build, once** |
|
||||
| what it binds | a consumer to a provider that is running | **an artifact to another artifact** |
|
||||
| how it is repaired | re-provision, re-grant | **rebuild — there is no other remedy** |
|
||||
| what changes it | the mesh's decisions | **somebody's commit** |
|
||||
|
||||
### An artifact's currency is its whole input closure
|
||||
|
||||
The correction this forces on [ADR 0063](0063-delivery-is-reconciliation-not-a-pipeline.md),
|
||||
which said an artifact is stale when its source moved. That is half of it.
|
||||
|
||||
> **An artifact is out of date when its source moved, or when anything it was built against
|
||||
> moved.**
|
||||
|
||||
So what is recorded against an artifact is not a commit. It is a commit **and the identity of
|
||||
every artifact it was built against** — which is what makes *is this current?* answerable without
|
||||
building, and what makes the cascade computable.
|
||||
|
||||
### It is derived, not declared
|
||||
|
||||
**Nobody writes a build edge in a manifest.** It is read from what the module actually imports,
|
||||
the way a package manager reads a lock file — because a declared list and the imports it
|
||||
describes drift, and the imports are the ones that are true. That is
|
||||
`how-we-build`'s *features are detected, not declared*, applied to dependencies.
|
||||
|
||||
**The runtime edges stay declared**, and the asymmetry is not an inconsistency: a runtime edge is
|
||||
an intention somebody has about how the mesh should be wired, and nothing but a person can state
|
||||
it. A build edge is a fact about code that already exists.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **The rebuild set becomes computable**, which is what
|
||||
[ADR 0063](0063-delivery-is-reconciliation-not-a-pipeline.md) assumed and did not have: change
|
||||
a module, take its transitive inbound build edges, and that is what is stale. In order, because
|
||||
the edges are directed.
|
||||
- **The graph measures design quality, not just build order.** A module with many inbound build
|
||||
edges is one whose every change is expensive — and *that is a fact about the design, readable
|
||||
before anything is built.* The current shared library is exactly this and nobody could see it,
|
||||
because nothing drew the edges.
|
||||
- **Fan-in becomes a thing that can be watched.** A module acquiring inbound build edges over
|
||||
time is one turning into a hub, and it is visible while it is happening rather than after.
|
||||
- **Reproducible builds would be worth much more than they look.** If rebuilding unchanged source
|
||||
against unchanged inputs produced the same digest, a cascade would stop at the first module
|
||||
whose output did not change. Without that, one shared-library commit redeploys everything
|
||||
behaviourally unchanged. **Not solved here**, and it is the difference between a cascade and a
|
||||
storm.
|
||||
- **Reading the edges needs a language-aware tool per language**, which is a real cost and the
|
||||
reason declaring them looks tempting. It is still wrong for the reason above.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0044](0044-a-module-declares-presence-instantiation-and-exclusion.md) — the two runtime
|
||||
edges this joins.
|
||||
- [ADR 0063](0063-delivery-is-reconciliation-not-a-pipeline.md) — what needs this to work.
|
||||
- [Research 011](../01-RESEARCH/011-the-module-graph/00-overview.md) — the graph this extends.
|
||||
@@ -1,87 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
date: 2026-08-28
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0064-a-build-edge-is-a-third-kind.md
|
||||
---
|
||||
|
||||
# 65. The core library is the mesh's domain, and types ship with their modules
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0019](0019-how-this-repository-works.md) describes the shared library as *"contracts shared
|
||||
across tiers: types, not behaviour"* — a guard against what the current one became, which is a
|
||||
package holding too much code and, with it, everybody's dependencies.
|
||||
|
||||
**The guard is aimed at the wrong thing.** *Types, not behaviour* sounds safe and does not
|
||||
address the fault: a library everything depends on is a hub whether it holds types or code. The
|
||||
fan-in is what makes a change expensive, and *types not behaviour* leaves the fan-in exactly
|
||||
where it was.
|
||||
|
||||
[ADR 0064](0064-a-build-edge-is-a-third-kind.md) makes this measurable rather than a matter of
|
||||
taste: a module's cost is its **inbound build edges**, and a type hub has as many as a code hub.
|
||||
|
||||
## Decision
|
||||
|
||||
### Types ship with the module they belong to
|
||||
|
||||
A type is part of a module's contract, so it travels with the module. A consumer needing
|
||||
`inventory`'s types depends on **`inventory`** — a real edge, narrow and visible — instead of both
|
||||
depending on a hub where the relationship cannot be seen.
|
||||
|
||||
**This trades one wide edge for several narrow ones, and that is the improvement.** Under
|
||||
[ADR 0064](0064-a-build-edge-is-a-third-kind.md) a change to one module's types now invalidates
|
||||
its actual consumers, rather than everything that touched the hub.
|
||||
|
||||
### The core library holds the mesh's own domain, and that is the test
|
||||
|
||||
Not *shared code*. A drawer labelled shared is a drawer everything goes in, which is how the
|
||||
current one grew.
|
||||
|
||||
> **The core library holds what is true of the mesh regardless of which context you are in.**
|
||||
|
||||
[Research 011](../01-RESEARCH/011-the-module-graph/00-overview.md) already found what that is:
|
||||
**a module, a node, and an assignment.** Those three are the mesh's domain — every context speaks
|
||||
about them, and none of them belongs to one context.
|
||||
|
||||
The test, applied to a candidate: *would this still mean the same thing in a context that had
|
||||
never heard of the one it came from?* A node does. A pipeline stage does not — that is
|
||||
delivery's. A grant does not — that is provisioning's.
|
||||
|
||||
**Domain-driven is the point rather than the label.** The current library is what happens when
|
||||
the organising idea is *who else might want this*: the answer is always yes, so everything is
|
||||
admitted. *Is this the mesh's domain* has a defensible no.
|
||||
|
||||
### Why this stays small on its own
|
||||
|
||||
A domain model changes when what the mesh **is** changes, which is rare. A shared-code drawer
|
||||
changes whenever anybody writes something reusable, which is constantly. So the core library
|
||||
inherits the property the graph is supposed to reveal — **few changes, many dependants** — rather
|
||||
than fighting for it.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **[ADR 0019](0019-how-this-repository-works.md)'s description of the shared library is
|
||||
superseded.** *Types, not behaviour* is replaced by *the mesh's domain*, and its types move to
|
||||
the modules that own them. The rest of 0030 — the naming rule and the repository list — stands.
|
||||
- **A module now publishes its own contract**, which it does not do today. That is real work and
|
||||
it is the same work as making a module a self-contained artifact, so it is not additional.
|
||||
- **The check is not the one that was proposed and is better.** *The build output contains no
|
||||
runtime code* would have enforced *types, not behaviour* — a rule now withdrawn. What replaces
|
||||
it is **inbound build edges**, which is a measurement rather than a prohibition: the core
|
||||
library should have many, and anything else acquiring many is turning into a hub. That is
|
||||
visible while it happens rather than after.
|
||||
- **Fewer things will be shared, and some code will be written twice.** That is the trade and it
|
||||
should be said plainly: the current library exists because sharing felt free. Under this it has
|
||||
a name, an owner and a visible edge, and two similar functions in two modules is often the
|
||||
better answer — `how-we-build` §8 already says three similar lines beat a premature
|
||||
abstraction.
|
||||
- **Nothing here says how a module publishes its types**, and the answer differs per language.
|
||||
That belongs with delivery.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0019](0019-how-this-repository-works.md) — the description this replaces.
|
||||
- [ADR 0064](0064-a-build-edge-is-a-third-kind.md) — what makes fan-in measurable.
|
||||
- [Research 011](../01-RESEARCH/011-the-module-graph/00-overview.md) — module, node, assignment.
|
||||
Reference in New Issue
Block a user