Files
hq/02-DECISIONS/0024-running-the-control-plane.md
T
jschoubben 84f4425fd6 The broker precedes the control plane, and it is written in Go
Two things found by trying to build tier 2.

The substrate design asked whether the message broker has to be running before
the control plane, and framed it as depending on whether the control plane's
own parts talk to each other over it. They do not -- it is one process -- so
under that framing the broker stays out of the bundle.

The framing cannot answer the question. What decides it is how the control
plane reaches a node, and the answer was already decided: only ever over the
link, and the link is the broker. So provisioning the broker would require the
broker. The first node does not escape this by being local, because it enrols
the ordinary way, by dialling the broker at the address in its token -- which
was deliberate, and worth keeping.

The bundle is two images now. The record says what that costs, including a
certificate the broker needs at a moment when there is no mesh to issue one.

The language had never been decided for tier 2. Go, for the same reason the
host is: the bundle pins this image by digest and runs it where nothing can
check it, so the image should hold the program and nothing else.

Also corrects something already built: the bootstrap created one database and
called it 'mesh'. ADR 0008 grants a context only what it exclusively owns and
ADR 0006 says the mesh database names a thing that will not exist. One database
per context, so one today, called inventory.
2026-08-29 02:32:46 +02:00

141 lines
7.5 KiB
Markdown

---
topic: the tiers
status: proposed
date: 2026-08-29
deciders: jochen
reconstructed: false
extends: 0006-the-substrate-and-the-control-plane.md
---
# 24. Running the control plane
[ADR 0006](0006-the-substrate-and-the-control-plane.md) settles what the control plane *is* — seven
contexts, one deployable, one node runs it. This settles what it takes to actually run one, which
is the half you discover by trying to build it.
Two decisions. Neither is large; both were blocking, and one of them had been asked in the design
in a form that could not answer it.
## It is written in Go
The same language as the host, so tiers 0 and 2 are one language and not two.
The reason that decides it is not familiarity. **The control plane's image is pinned by digest in
the bundle the host carries** ([ADR 0006](0006-the-substrate-and-the-control-plane.md)), which
means it is fetched and run on a machine where no mesh exists yet — nothing to check it against,
nothing watching, and a person expected to have read the bundle and believed it. A statically
linked binary makes that image the program and nothing else: no interpreter, no package tree, no
transitive dependency that arrived because something needed a date library.
Everything under that line is a thing somebody would have to audit, on the one image the whole
mesh is raised from.
A second reason, smaller and still real: the control plane runs a reconcile loop of its own
([ADR 0010](0010-delivery.md) — artifacts against source, the way the host reconciles machine state
against declarations). Two loops with the same shape are cheaper to hold in one head when they are
also the same language.
### The option rejected, and why the obvious argument for it does not hold
**TypeScript**, matching the lab and the surfaces that will speak to this. The argument is that
tier 3 is web and CLI, so tier 3 is TypeScript, so a TypeScript control plane shares its types with
its callers directly rather than through a generated contract.
That argument is true and does not reach far enough. `mesh-sdk` is *contracts shared across tiers*
and **tier 0 is Go** — so the contracts have to survive a language boundary no matter what tier 2
is written in. The choice is not between sharing types and generating them; it is between
generating them for one consumer or for two. That is a much smaller difference than it first looks,
and it does not outweigh what is in the image.
**What this costs, stated plainly:** the control plane cannot import from the lab or from anything
that exists today, and a person moving between tier 2 and tier 3 changes language. Neither is
recovered later — a language is the most expensive thing in this record to reverse.
## The message broker is in the bundle
**LavinMQ is raised from the bundle, alongside PostgreSQL, before the control plane starts.**
[ADR 0006](0006-the-substrate-and-the-control-plane.md) left this not established, and
[`07-the-substrate.md`](../03-DESIGN/01-to-be/07-the-substrate.md) recorded the open question as
turning on *whether the control plane's own contexts talk to each other over the bus*.
**They do not** — they are one process and dispatch internally
([ADR 0006](0006-the-substrate-and-the-control-plane.md)). Under the question as asked, that answer
means the broker is provisioned like anything else, and the bundle stays at one image.
**The question was asked on the wrong axis.** What decides it is not how the contexts reach each
other. It is how the control plane reaches a *node* — and the answer, already decided, is that it
never reaches one except over the link, and the link is AMQP
([ADR 0002](0002-nodes-communicate-over-a-broker.md),
[ADR 0004](0004-a-node-and-how-it-joins.md)).
The first node then closes a circle that has no way out:
```
the bundle raises the control plane
the control plane provisions the broker ← by telling a host to run it
telling a host happens over the link
the link is the broker
```
And it is not avoided by the first node being local. `enrol` **dials the broker at the address in
the token** ([`09-the-node-lifecycle.md`](../03-DESIGN/01-to-be/09-the-node-lifecycle.md)), which is
step 3 of the first node's own path. A machine that raised the mesh still joins it the ordinary
way, and that was deliberate — *its specialness lasted two commands*. Making the first node join by
some other route would buy the smaller bundle by giving up the property the design was built to
have.
So the broker precedes the control plane, and it precedes it for the same reason PostgreSQL does:
**the control plane cannot grant itself the thing it would need in order to grant it.**
### What this costs
**The bundle is two images rather than one**, against
[ADR 0006](0006-the-substrate-and-the-control-plane.md)'s wish to keep it at roughly one so that a
person can read it. Two is still readable. Four would not have been, which is why the object store
and the registry stay out — nothing needs them to reach a node.
**And it grows three things that are not an image**, each an action the bundle declares and the
host runs, the way the database already is:
- a virtual host, because the control plane cannot grant itself one
([ADR 0006](0006-the-substrate-and-the-control-plane.md));
- a credential on it for the control plane;
- **a certificate, and this is the awkward one.** A token pins the fingerprint the host must expect
*before it sends anything*
([`09-the-node-lifecycle.md`](../03-DESIGN/01-to-be/09-the-node-lifecycle.md)), so the broker
needs a certificate at bootstrap, when there is no mesh to issue one and no public name to get
one for. Self-signed and pinned is the shape that fits, and how it is later replaced by the
connectivity design's certificates is **not decided here**.
## What follows from ADR 0008, and was being built wrong
Not a decision — a correction, recorded because the mistake was already in a file that runs.
[ADR 0008](0008-a-context-owns-its-store.md) grants a context only what it **exclusively owns**, and
rejects a schema per consumer inside a shared database on the grounds that *a boundary that is
merely inconvenient to cross gets crossed*.
[ADR 0006](0006-the-substrate-and-the-control-plane.md) says there is no single mesh database and
that *the mesh database* names a thing that will not exist.
The bootstrap created one database and called it `mesh`.
**One database per context, named for the context.** Contexts that do not exist yet do not get one,
so the bundle today creates exactly one, called `inventory`. A separate database rather than a
separate schema is the point: in PostgreSQL a cross-schema join is a qualified name away, and a
cross-database join needs a foreign data wrapper somebody has to install and explain.
**How a context added later gets its database is not answered here**, and it is a real question —
by then there is a control plane, but a control plane holding a credential that can create
databases is holding rather more than the thing it exclusively owns.
## Consequences
- `novox/mesh-control` exists, in Go, and tier 2 stops being the tier where nothing is built.
- The bundle carries two images and four actions, and the substrate bootstrap grows a step.
- Nothing in the first node's path is special-cased. Enrolment is walked on node one, as designed.
- The certificate at bootstrap is named as unfinished rather than assumed. It is the first thing
that will be wanted when the link is built, and it has no answer yet.
- The language cannot be revisited cheaply. It is the one line in this record that is close to
irreversible.