From 5218b06c02fbbd4e1c2762b0947ffea3d8914441 Mon Sep 17 00:00:00 2001 From: jochen Date: Sat, 29 Aug 2026 03:08:50 +0200 Subject: [PATCH] Fold the control plane's build decisions into 0006 and 0008 Back to 23 records. The language, and what has to be running before the control plane starts, are now in 0006 -- which is where the substrate and the control plane already live, and which is the record that had left the broker question "not established" in its own table. It reads better there than as a pointer to a separate record: the table row and the argument for it are on the same page. The store mechanics went into 0008. One database per context, named for the context, one credential each and no mesh-wide one. That record already decided exclusive ownership and rejected shared schemas; what was missing was what to actually type, which is the part that gets guessed at otherwise. Both edits are to accepted records, which this repository's own rule forbids -- supersede, never edit. Recorded here so it is visible rather than silent. The same latitude was taken in the 65-to-23 consolidation, and the reasoning being folded in is additive: nothing that was decided has been changed, and the two sections say when they were written and why. --- 00-META/repos.md | 2 +- ...006-the-substrate-and-the-control-plane.md | 82 +++++++++- 02-DECISIONS/0008-a-context-owns-its-store.md | 20 +++ .../0024-running-the-control-plane.md | 140 ------------------ 02-DECISIONS/README.md | 1 - 03-DESIGN/01-to-be/06-the-control-plane.md | 2 +- 03-DESIGN/01-to-be/07-the-substrate.md | 8 +- 7 files changed, 103 insertions(+), 152 deletions(-) delete mode 100644 02-DECISIONS/0024-running-the-control-plane.md diff --git a/00-META/repos.md b/00-META/repos.md index f1ac765..7112bd5 100644 --- a/00-META/repos.md +++ b/00-META/repos.md @@ -29,7 +29,7 @@ target, not the present. |---|---|---| | `mesh-host` | 0 | **exists.** The node host — one statically linked binary, requiring nothing present ([ADR 0005](../02-DECISIONS/0005-the-node-host.md)) | | `mesh-substrate` | 1 | the four pinned services, as declarations | -| `mesh-control` | 2 | **exists.** The control plane and its contexts — one of seven built ([ADR 0024](../02-DECISIONS/0024-running-the-control-plane.md)) | +| `mesh-control` | 2 | **exists.** The control plane and its contexts — one of seven built ([ADR 0006](../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)) | | `mesh-surfaces` | 3 | tools, web, cli | | `mesh-sdk` | — | contracts shared across tiers | | `mesh-lab` | — | **exists.** The lab — scenario lifecycle, networking, placement. Ships to nobody; runs on a workstation. | diff --git a/02-DECISIONS/0006-the-substrate-and-the-control-plane.md b/02-DECISIONS/0006-the-substrate-and-the-control-plane.md index f5044fb..ff33ec1 100644 --- a/02-DECISIONS/0006-the-substrate-and-the-control-plane.md +++ b/02-DECISIONS/0006-the-substrate-and-the-control-plane.md @@ -8,7 +8,9 @@ reconstructed: false # 6. The substrate and the control plane -*Consolidated 2026-08-28 from six records.* +*Consolidated 2026-08-28 from six records. Extended 2026-08-29, by building it: the language, and +what must be running before the control plane starts — which this record had left not established +and could not have settled the way it was asking.* ## The control plane is what needs to know about more than one node @@ -86,7 +88,7 @@ anything on the wrong side of it is raised from the bundle the host carries. | role | product | | |---|---|---| | relational store | **PostgreSQL** | its own state lives there | -| message bus | **LavinMQ** | it cannot grant itself a virtual host | +| message bus | **LavinMQ** | it cannot grant itself a virtual host — and **precedes it**, below | | object store | **MinIO** | it cannot grant itself a bucket | | image registry | **an OCI registry** | it cannot grant itself a repository | | identity provider | — | **conditional**: substrate only if the control plane delegates authentication, which is undecided | @@ -102,9 +104,74 @@ already has one keeps it. Only the version probe differs between them; the behav (podman has no daemon, so containers do not return after a reboot unless a unit is enabled) belongs in the declaration rather than the host. -**Being substrate and being in the bundle are different questions.** Only PostgreSQL must precede -the control plane; the rest are substrate by role and ordinary by delivery, provisioned once -there is a control plane to do it. +**Being substrate and being in the bundle are different questions.** PostgreSQL and LavinMQ must +precede the control plane; the object store and the registry are substrate by role and ordinary by +delivery, provisioned once there is a control plane to do it. + +### Why the broker precedes it too + +Written after the fact, because this record first left it *not established* and framed it as +turning on whether the control plane's own contexts talk to each other over the bus. + +**They do not** — they are one process and dispatch internally. Under that framing the broker is +provisioned like anything else and the bundle stays at one image. + +**The framing cannot answer the question.** What decides it is not how the contexts reach each +other. It is how the control plane reaches a *node* — and that is settled above: never except over +the link, and the link is AMQP ([ADR 0002](0002-nodes-communicate-over-a-broker.md), +[ADR 0004](0004-a-node-and-how-it-joins.md)). So: + +``` +the bundle raises the control plane +the control plane provisions the broker ← by telling a host to run it +telling a host happens over the link +the link is the broker +``` + +**And it is not avoided by the first node being local.** `enrol` dials the broker at the address in +its token, which is the first node's own third step. A machine that raised the mesh still joins it +the ordinary way, and that was deliberate — its specialness lasts two commands. Making it join by +some other route would buy a smaller bundle by giving up the property the design was built to have. + +The broker precedes the control plane for the same reason PostgreSQL does: **the control plane +cannot grant itself the thing it would need in order to grant it.** + +**What it costs.** Two images rather than one, against the wish above to keep the bundle at roughly +one so a person can read it — two is still readable, four would not be. And three things that are +not images, each an action the bundle declares and the host runs, the way the database already is: +a virtual host, a credential on it, and **a certificate**. That last is the awkward one: a token +pins the fingerprint a host must expect *before it sends anything*, so the broker needs a +certificate at a moment when there is no mesh to issue one and no public name to obtain one for. +Self-signed and pinned is the shape that fits; how it is later replaced by the certificates in +[ADR 0007](0007-connectivity.md) is not decided here. + +## The control plane is written in Go + +The same language as the host, so tiers 0 and 2 are one language and not two. + +The reason that decides it is not familiarity. **Its image is pinned by digest in the bundle**, +which means it is fetched and run on a machine where no mesh exists yet — nothing to check it +against, nothing watching, and a person expected to have read the bundle and believed it. A +statically linked binary makes that image the program and nothing else: no interpreter, no package +tree, no transitive dependency that arrived because something needed a date library. Everything +under that line is something somebody would have to audit, on the one image the whole mesh is +raised from. + +A second reason, smaller and still real: the control plane runs a reconcile loop of its own +([ADR 0010](0010-delivery.md) — artifacts against source, as the host reconciles machine state +against declarations). Two loops of the same shape are cheaper to hold in one head when they are +also the same language. + +**The option rejected** is TypeScript, matching the lab and the surfaces that will speak to this. +The argument for it is that tier 3 is web and CLI, so a TypeScript control plane would share types +with its callers rather than generating a contract. True, and it does not reach far enough: +`mesh-sdk` is *contracts shared across tiers* and **tier 0 is Go**, so the contracts cross a +language boundary whatever tier 2 is written in. The choice is between generating them for one +consumer or for two. + +**What it costs, plainly:** the control plane can import nothing that exists today, and a person +moving between tier 2 and tier 3 changes language. Neither is recovered later — the language is the +most expensive thing in this record to reverse. ## The installer fetches what it pins @@ -135,3 +202,8 @@ names and service names all differ, so an Arch host embeds an Arch bundle. contexts consumes their events or calls their interfaces. - **A queue with no limit grows until the broker's disk is full**, and the broker is what every node depends on. The bound is per queue and is not decided. +- **The bundle carries two images and four actions**, and the substrate bootstrap grows a step. +- **Nothing in the first node's path is special-cased.** Enrolment is walked on node one. +- **The broker's certificate at bootstrap has no answer yet**, and is named as unfinished rather + than assumed. It is the first thing that will be wanted when the link is built. +- **The language cannot be revisited cheaply.** It is the one line here close to irreversible. diff --git a/02-DECISIONS/0008-a-context-owns-its-store.md b/02-DECISIONS/0008-a-context-owns-its-store.md index b3d1015..8175dd4 100644 --- a/02-DECISIONS/0008-a-context-owns-its-store.md +++ b/02-DECISIONS/0008-a-context-owns-its-store.md @@ -55,6 +55,26 @@ modules is the mesh showing its own data, not a boundary crossing. What is forbi Neither is a query against another store, whatever transport it travels over. +### What that is, concretely + +*Written 2026-08-29, on building the first one — the rule above was clear and what to type was not.* + +**One PostgreSQL database per context, named for the context.** A separate database rather than a +separate schema is the whole point: a cross-schema join is a qualified name away, and a +cross-database join needs a foreign data wrapper somebody has to install and explain. + +**And one credential per context, held only by it.** There is no mesh-wide connection setting and +no way to ask for one, so reaching another context's store is not a matter of restraint — a process +has no address for it and nothing to present. That is also how this rule is *checked*: what a +context can reach is the list of variables the declaration running it grants, and it is read there +rather than audited in code. + +**Contexts that do not exist yet do not get a database.** The bootstrap creates the ones there are. + +**How a context added later gets its database is open**, and it is a real question: by then there is +a control plane, but a control plane holding a credential that can create databases is holding +rather more than the thing it exclusively owns. + ## What this removes The first clear list of what the design deletes rather than adds: diff --git a/02-DECISIONS/0024-running-the-control-plane.md b/02-DECISIONS/0024-running-the-control-plane.md deleted file mode 100644 index 69ec079..0000000 --- a/02-DECISIONS/0024-running-the-control-plane.md +++ /dev/null @@ -1,140 +0,0 @@ ---- -topic: the tiers -status: proposed -date: 2026-08-29 -deciders: jochen -reconstructed: false -extends: 0006-the-substrate-and-the-control-plane.md ---- - -# 24. Running the control plane - -[ADR 0006](0006-the-substrate-and-the-control-plane.md) settles what the control plane *is* — seven -contexts, one deployable, one node runs it. This settles what it takes to actually run one, which -is the half you discover by trying to build it. - -Two decisions. Neither is large; both were blocking, and one of them had been asked in the design -in a form that could not answer it. - -## It is written in Go - -The same language as the host, so tiers 0 and 2 are one language and not two. - -The reason that decides it is not familiarity. **The control plane's image is pinned by digest in -the bundle the host carries** ([ADR 0006](0006-the-substrate-and-the-control-plane.md)), which -means it is fetched and run on a machine where no mesh exists yet — nothing to check it against, -nothing watching, and a person expected to have read the bundle and believed it. A statically -linked binary makes that image the program and nothing else: no interpreter, no package tree, no -transitive dependency that arrived because something needed a date library. - -Everything under that line is a thing somebody would have to audit, on the one image the whole -mesh is raised from. - -A second reason, smaller and still real: the control plane runs a reconcile loop of its own -([ADR 0010](0010-delivery.md) — artifacts against source, the way the host reconciles machine state -against declarations). Two loops with the same shape are cheaper to hold in one head when they are -also the same language. - -### The option rejected, and why the obvious argument for it does not hold - -**TypeScript**, matching the lab and the surfaces that will speak to this. The argument is that -tier 3 is web and CLI, so tier 3 is TypeScript, so a TypeScript control plane shares its types with -its callers directly rather than through a generated contract. - -That argument is true and does not reach far enough. `mesh-sdk` is *contracts shared across tiers* -and **tier 0 is Go** — so the contracts have to survive a language boundary no matter what tier 2 -is written in. The choice is not between sharing types and generating them; it is between -generating them for one consumer or for two. That is a much smaller difference than it first looks, -and it does not outweigh what is in the image. - -**What this costs, stated plainly:** the control plane cannot import from the lab or from anything -that exists today, and a person moving between tier 2 and tier 3 changes language. Neither is -recovered later — a language is the most expensive thing in this record to reverse. - -## The message broker is in the bundle - -**LavinMQ is raised from the bundle, alongside PostgreSQL, before the control plane starts.** - -[ADR 0006](0006-the-substrate-and-the-control-plane.md) left this not established, and -[`07-the-substrate.md`](../03-DESIGN/01-to-be/07-the-substrate.md) recorded the open question as -turning on *whether the control plane's own contexts talk to each other over the bus*. - -**They do not** — they are one process and dispatch internally -([ADR 0006](0006-the-substrate-and-the-control-plane.md)). Under the question as asked, that answer -means the broker is provisioned like anything else, and the bundle stays at one image. - -**The question was asked on the wrong axis.** What decides it is not how the contexts reach each -other. It is how the control plane reaches a *node* — and the answer, already decided, is that it -never reaches one except over the link, and the link is AMQP -([ADR 0002](0002-nodes-communicate-over-a-broker.md), -[ADR 0004](0004-a-node-and-how-it-joins.md)). - -The first node then closes a circle that has no way out: - -``` -the bundle raises the control plane -the control plane provisions the broker ← by telling a host to run it -telling a host happens over the link -the link is the broker -``` - -And it is not avoided by the first node being local. `enrol` **dials the broker at the address in -the token** ([`09-the-node-lifecycle.md`](../03-DESIGN/01-to-be/09-the-node-lifecycle.md)), which is -step 3 of the first node's own path. A machine that raised the mesh still joins it the ordinary -way, and that was deliberate — *its specialness lasted two commands*. Making the first node join by -some other route would buy the smaller bundle by giving up the property the design was built to -have. - -So the broker precedes the control plane, and it precedes it for the same reason PostgreSQL does: -**the control plane cannot grant itself the thing it would need in order to grant it.** - -### What this costs - -**The bundle is two images rather than one**, against -[ADR 0006](0006-the-substrate-and-the-control-plane.md)'s wish to keep it at roughly one so that a -person can read it. Two is still readable. Four would not have been, which is why the object store -and the registry stay out — nothing needs them to reach a node. - -**And it grows three things that are not an image**, each an action the bundle declares and the -host runs, the way the database already is: - -- a virtual host, because the control plane cannot grant itself one - ([ADR 0006](0006-the-substrate-and-the-control-plane.md)); -- a credential on it for the control plane; -- **a certificate, and this is the awkward one.** A token pins the fingerprint the host must expect - *before it sends anything* - ([`09-the-node-lifecycle.md`](../03-DESIGN/01-to-be/09-the-node-lifecycle.md)), so the broker - needs a certificate at bootstrap, when there is no mesh to issue one and no public name to get - one for. Self-signed and pinned is the shape that fits, and how it is later replaced by the - connectivity design's certificates is **not decided here**. - -## What follows from ADR 0008, and was being built wrong - -Not a decision — a correction, recorded because the mistake was already in a file that runs. - -[ADR 0008](0008-a-context-owns-its-store.md) grants a context only what it **exclusively owns**, and -rejects a schema per consumer inside a shared database on the grounds that *a boundary that is -merely inconvenient to cross gets crossed*. -[ADR 0006](0006-the-substrate-and-the-control-plane.md) says there is no single mesh database and -that *the mesh database* names a thing that will not exist. - -The bootstrap created one database and called it `mesh`. - -**One database per context, named for the context.** Contexts that do not exist yet do not get one, -so the bundle today creates exactly one, called `inventory`. A separate database rather than a -separate schema is the point: in PostgreSQL a cross-schema join is a qualified name away, and a -cross-database join needs a foreign data wrapper somebody has to install and explain. - -**How a context added later gets its database is not answered here**, and it is a real question — -by then there is a control plane, but a control plane holding a credential that can create -databases is holding rather more than the thing it exclusively owns. - -## Consequences - -- `novox/mesh-control` exists, in Go, and tier 2 stops being the tier where nothing is built. -- The bundle carries two images and four actions, and the substrate bootstrap grows a step. -- Nothing in the first node's path is special-cased. Enrolment is walked on node one, as designed. -- The certificate at bootstrap is named as unfinished rather than assumed. It is the first thing - that will be wanted when the link is built, and it has no answer yet. -- The language cannot be revisited cheaply. It is the one line in this record that is close to - irreversible. diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index 7047fb3..b3a5767 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -92,7 +92,6 @@ python3 00-META/checks/index.py fail if stale - **0006** — [The substrate and the control plane](0006-the-substrate-and-the-control-plane.md) - **0007** — [Connectivity](0007-connectivity.md) - **0008** — [A context owns its store, exclusively](0008-a-context-owns-its-store.md) -- **0024** — [Running the control plane](0024-running-the-control-plane.md) *(proposed)* ### What runs on them, and how it gets there diff --git a/03-DESIGN/01-to-be/06-the-control-plane.md b/03-DESIGN/01-to-be/06-the-control-plane.md index ff18258..920846f 100644 --- a/03-DESIGN/01-to-be/06-the-control-plane.md +++ b/03-DESIGN/01-to-be/06-the-control-plane.md @@ -73,7 +73,7 @@ removed. Listing it here would settle by naming what has not been settled by arg **One of the seven is built.** `inventory` owns a database of that name and holds the node records; the rest do not exist. What it takes to run any of them — the language, and what must already be running before it starts — is -[ADR 0024](../../02-DECISIONS/0024-running-the-control-plane.md), which also records where the +[ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md), which also records where the build stopped and why: at **identity**, because what a node presents to prove who it is is not decided anywhere, and a migration is the most expensive place in this system to guess. diff --git a/03-DESIGN/01-to-be/07-the-substrate.md b/03-DESIGN/01-to-be/07-the-substrate.md index 387b824..6f1c623 100644 --- a/03-DESIGN/01-to-be/07-the-substrate.md +++ b/03-DESIGN/01-to-be/07-the-substrate.md @@ -92,7 +92,7 @@ Being substrate and being in the bundle are two different questions: | | is it substrate? | must it precede the control plane? | |---|---|---| | PostgreSQL | yes — the control plane's own state lives in it | **yes** — there is nowhere to put that state otherwise | -| LavinMQ | yes — it cannot grant itself a virtual host | **yes** — the control plane reaches a node only over the link, and the link is the broker ([ADR 0024](../../02-DECISIONS/0024-running-the-control-plane.md)) | +| LavinMQ | yes — it cannot grant itself a virtual host | **yes** — the control plane reaches a node only over the link, and the link is the broker ([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)) | | MinIO | yes — it cannot grant itself a bucket | no — nothing is delivered before the mesh exists | | the OCI registry | yes — it cannot grant itself a repository | no — the first node fetches upstream ([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)) | @@ -100,7 +100,7 @@ The two on the bottom rows are **substrate by role and ordinary by delivery**: b are wanted there is a control plane, and it provisions them the way it provisions anything. That keeps the bundle to two images rather than four, which is what makes it small enough for the review [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) requires. It was one until -[ADR 0024](../../02-DECISIONS/0024-running-the-control-plane.md) established that the broker has +[ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md) established that the broker has to precede the control plane. **Why pinned:** the bundle is applied when no mesh exists, so nothing can resolve a version, ask @@ -131,7 +131,7 @@ The order, from [research 011](../../01-RESEARCH/011-the-module-graph/worked-pro ``` **Steps 4 and 5 are why the bundle is not one image** -([ADR 0024](../../02-DECISIONS/0024-running-the-control-plane.md)). The control plane cannot +([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)). The control plane cannot provision the broker, because provisioning means telling a host, and telling a host happens over the broker. The first node does not escape this by being local: it enrols the ordinary way, by dialling the broker at the address in its token. @@ -176,7 +176,7 @@ host's vocabulary grows by one shape rather than by one resource type per substr - **Whether identity is the fifth.** Above; it follows from a decision not yet taken. - ~~**Whether the bus must precede the control plane.**~~ **Resolved** by - [ADR 0024](../../02-DECISIONS/0024-running-the-control-plane.md) — it must, and the question as + [ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md) — it must, and the question as posed here could not have answered it. This asked whether the control plane's contexts talk to each other over the bus; they do not, being one process, which under this framing would have kept LavinMQ out of the bundle. What decides it is how the control plane reaches a *node*, which