jschoubben
e1febe8e0f
The consolidation left a sparse sequence -- 1, 4, 6, 7, 9, 10, 12, 15, 16, 18, 19, 25, 34, 35, 36, 37, 40, 42, 44, 45, 48, 49, 58 -- where the gaps were only the archaeology of what used to be there. Renumbered contiguously. Renames run in ascending order, so every target number is already free and no two files ever collide. The reference rewrite is one simultaneous pass rather than a sequence of replacements. Numbers moved into slots other numbers were vacating -- the node host went 37 to 16 while the lab went 16 to 9 -- so replacing one at a time would have cascaded and silently pointed things at the wrong record. Seven plain-text references survived the merges as prose rather than links, naming records that no longer existed: the enrolment token, the link boundary, what a declaration is, reachability, the repository structure. Each mapped to the consolidated record that now holds it. Verified rather than assumed: every [ADR NNNN](path) link now has matching text and target, checked across the whole repository, and the checker passes. Frontmatter `consolidates:` lists dropped -- they named records that are gone, and each consolidated record already says in prose what it absorbed.
130 lines
6.7 KiB
Markdown
130 lines
6.7 KiB
Markdown
---
|
|
status: accepted
|
|
date: 2026-08-28
|
|
deciders: jochen
|
|
reconstructed: false
|
|
---
|
|
|
|
# 21. The substrate and the control plane
|
|
|
|
*Consolidated 2026-08-28 from six records.*
|
|
|
|
## The control plane is what needs to know about more than one node
|
|
|
|
That is the whole test, and it follows from the host applying rather than deciding: **deciding
|
|
needs knowledge a single machine does not have.**
|
|
|
|
| question | whose |
|
|
|---|---|
|
|
| write this file, with this content, with this mode | the **host** |
|
|
| which nodes should run the store | the **control plane** |
|
|
| is this unit running | the **host** |
|
|
| which peers belong in this node's overlay | the **control plane** |
|
|
| has this node been unreachable for a week | the **control plane** — nobody else is watching |
|
|
|
|
**Anything a single machine could answer alone is not the control plane's.**
|
|
|
|
### Seven contexts and one interface
|
|
|
|
**inventory, config, connectivity, provisioning, delivery, observability, identity** — plus
|
|
`api`, the one interface every surface speaks to. Each earns its place by the test above rather
|
|
than by being ours.
|
|
|
|
**`work`, `knowledge` and `stream` are mesh-hosted applications, not control plane.** A task does
|
|
not need to know a node exists. *Being ours does not make something infrastructure.*
|
|
|
|
**Where the record lives is deliberately open.** Contexts integrate through it, which makes it
|
|
load-bearing, and putting it in the substrate risks recreating the circularity the tiers just
|
|
removed. Listing it as an eighth context would settle by naming what has not been settled by
|
|
arguing.
|
|
|
|
### One node runs it, and nothing takes over
|
|
|
|
**Declared, never elected.** No promotion, no quorum, no fencing, no split brain — none of it
|
|
built, so none of it can be subtly wrong.
|
|
|
|
**That is sound rather than merely cheap**, because the design already tolerates its absence by
|
|
construction: a node reconciles from its own store and never needed to ask anybody to hold the
|
|
state it was last given. **The control plane being down is not a new failure mode — it is every
|
|
node in the ordinary disconnected situation at once.** What is lost is *change*, not *operation*.
|
|
|
|
The honest half: this node is a single point of failure, recovery is **restore rather than
|
|
failover** — which makes backup the availability mechanism rather than hygiene — and
|
|
**certificate renewal is the clock.** An outage outlasting a renewal window expires every public
|
|
name, which turns an inconvenience into an outage on a timer. Nothing measures that today.
|
|
|
|
## The authority is the control plane, not a database
|
|
|
|
**There is no single mesh database.** Each context owns its store exclusively, and *the mesh
|
|
database* names a thing that will not exist.
|
|
|
|
**No node reads any of them** — not for writes, not for reads. A node is *told* what to own, over
|
|
the link, in a bounded vocabulary; it **states** what it applied, and the owning context writes.
|
|
The difference is the security boundary: something that can write cannot be prevented from
|
|
writing anything.
|
|
|
|
**A node runs from its own store always, not as a fallback.** The current arrangement's nastiest
|
|
property is that *a node running from cache looks identical to a node running from the database*,
|
|
with no age on the cache and nothing reporting divergence. Under this there is no second mode to
|
|
be mistaken for the first.
|
|
|
|
**What survives from the original decision:** the repository defines what exists, the mesh defines
|
|
what runs where, and no node-to-module mapping is ever committed. That is what makes the
|
|
repositories node-agnostic and why anything about the mesh can be published at all.
|
|
|
|
**The error underneath was a category error**: *source of truth* named a storage location when it
|
|
meant an **authority**. Once the store is the answer, *which database* becomes the question, and
|
|
shared schemas follow.
|
|
|
|
## The substrate is what the control plane consumes and cannot grant itself
|
|
|
|
Every module needing a database asks provisioning for one. The control plane needs a database too
|
|
and cannot ask itself, because it is not running yet. **That circularity is the definition**, and
|
|
anything on the wrong side of it is raised from the bundle the host carries.
|
|
|
|
| role | product | |
|
|
|---|---|---|
|
|
| relational store | **PostgreSQL** | its own state lives there |
|
|
| message bus | **LavinMQ** | it cannot grant itself a virtual host |
|
|
| object store | **MinIO** | it cannot grant itself a bucket |
|
|
| image registry | **an OCI registry** | it cannot grant itself a repository |
|
|
| identity provider | — | **conditional**: substrate only if the control plane delegates authentication, which is undecided |
|
|
|
|
**The role and the product are both written.** The role is what the argument turns on; the product
|
|
is what gets installed and pinned, and a design that names only the role does not record that the
|
|
choice was made. **The dependency is on the protocol** — AMQP, S3, OCI — which is what keeps
|
|
naming them safe. The store is the exception: the provisioning model uses databases, roles and
|
|
schemas as PostgreSQL means them.
|
|
|
|
**A container runtime is detected, not chosen** — docker or podman, because a machine that
|
|
already has one keeps it. Only the version probe differs between them; the behavioural difference
|
|
(podman has no daemon, so containers do not return after a reboot unless a unit is enabled)
|
|
belongs in the declaration rather than the host.
|
|
|
|
**Being substrate and being in the bundle are different questions.** Only PostgreSQL must precede
|
|
the control plane; the rest are substrate by role and ordinary by delivery, provisioned once
|
|
there is a control plane to do it.
|
|
|
|
## The installer fetches what it pins
|
|
|
|
`substrate.lock` carries **references, not payload** — an image name and a **digest**, fetched at
|
|
apply time. A tag moves; a digest does not, and reproducibility comes from pinning the identity of
|
|
a thing rather than carrying its bytes.
|
|
|
|
The assumption that a machine might have no network came from the lab and was wrong: a machine
|
|
being adopted has one, and the sealed case is the lab, which places images itself.
|
|
|
|
**Its contents are per operating system** even though its mechanism is not — package names, unit
|
|
names and service names all differ, so an Arch host embeds an Arch bundle.
|
|
|
|
## Consequences
|
|
|
|
- **The bundle stays small and reviewable.** A list of pinned references is something a person can
|
|
read; a bundle containing images is not.
|
|
- **An apply can fail because something is unreachable**, which a self-contained artifact could
|
|
not. That must fail *legibly*, naming what could not be fetched and from where.
|
|
- **Cross-context reporting is harder, and that is the point.** Anything wanting to see across
|
|
contexts consumes their events or calls their interfaces.
|
|
- **A queue with no limit grows until the broker's disk is full**, and the broker is what every
|
|
node depends on. The bound is per queue and is not decided.
|