Glossary, the mesh-controller/foundation vocabulary, ADR 0076, Phase 3 closed #43
+1
-1
@@ -30,7 +30,7 @@ named, and nothing should be designed around a particular one existing.
|
|||||||
- **A hosted model provider** supplies the thinking for non-human agents, drawn from a
|
- **A hosted model provider** supplies the thinking for non-human agents, drawn from a
|
||||||
shared pool of subscriptions — which is why budget pacing is a first-class concern.
|
shared pool of subscriptions — which is why budget pacing is a first-class concern.
|
||||||
- **Long-lived user services** rather than an orchestrator. No cluster scheduler, no cloud
|
- **Long-lived user services** rather than an orchestrator. No cluster scheduler, no cloud
|
||||||
control plane.
|
controller.
|
||||||
|
|
||||||
Defaults, not mandates. A second model provider is anticipated by design; nothing in the
|
Defaults, not mandates. A second model provider is anticipated by design; nothing in the
|
||||||
domain may assume one vendor's credential lifecycle.
|
domain may assume one vendor's credential lifecycle.
|
||||||
|
|||||||
@@ -0,0 +1,60 @@
|
|||||||
|
# Glossary — the words this repository uses, and the ones it stopped using
|
||||||
|
|
||||||
|
One name per thing. This page is the authority; where an older record says something else, that
|
||||||
|
record is being superseded, not this page. It exists because the terms kept drifting in
|
||||||
|
conversation — control plane / controller / master / hub for one thing, substrate / foundation for
|
||||||
|
another — and a mesh you cannot name precisely is a mesh two people describe differently.
|
||||||
|
|
||||||
|
## The mesh and its machines
|
||||||
|
|
||||||
|
- **node** — a machine in the mesh. There are 0..n of them, and each runs the host agent. A node is
|
||||||
|
just a machine that has joined; being one implies nothing about what it runs.
|
||||||
|
- **control-node** — the one node that also holds the `the-controller` seat. There is exactly one
|
||||||
|
per mesh. "control-node" is not a separate kind of machine — it is a node that additionally runs
|
||||||
|
the controller (and, today, the foundation). Lose it and the other nodes keep running what they
|
||||||
|
were last told; they simply cannot be told anything new.
|
||||||
|
- ~~master / slave~~, ~~hub / peer~~ — not used. The relationship is *controller and nodes*, and no
|
||||||
|
node is subordinate: a node applies declarations on its own and survives the control-node dying.
|
||||||
|
|
||||||
|
## What runs the mesh
|
||||||
|
|
||||||
|
- **controller** — the component that decides what each node should be, holds the mesh's records,
|
||||||
|
and tells nodes over the broker. Replaces **"control plane"** (borrowed from networking's
|
||||||
|
control-plane/data-plane, and opaque here).
|
||||||
|
- **mesh-controller** — the module that runs the controller. It **claims** the `the-controller`
|
||||||
|
seat at mesh scope, which is what makes it singular. Replaces the module name **`mesh-control`**.
|
||||||
|
(The git repository is still named `mesh-control` until it is renamed on the forge; the module,
|
||||||
|
container and image it produces are `mesh-controller`.)
|
||||||
|
- **foundation** — the store and the broker, raised at genesis before any module system exists.
|
||||||
|
Replaces **"substrate"** (a biology metaphor that landed for no one). The foundation is not a
|
||||||
|
third thing beside the store and broker — it *is* those two, named together.
|
||||||
|
- **store** — the one postgres server. It holds the controller's own context databases
|
||||||
|
(`inventory`, `identity`, `licences` — a context owns its store, [ADR 0008](../02-DECISIONS/0008-a-context-owns-its-store.md))
|
||||||
|
and every module's own database. One server, many databases — never one shared "mesh database".
|
||||||
|
- **broker** — the one lavinmq message bus. It carries the mesh bus on the `/` vhost and a vhost per
|
||||||
|
consumer that requires `amqp`.
|
||||||
|
|
||||||
|
## What the mesh stores and serves
|
||||||
|
|
||||||
|
- **package** — what code resolves when it is **compiled**: an npm/cargo/pypi dependency, by
|
||||||
|
**version**. Served by the **package-registry** (gitea). Only a builder talks to it.
|
||||||
|
- **artifact** — what the mesh delivers to a machine to **install and run**: an OCI image, by
|
||||||
|
**digest**. Served by the **artifact-store** (distribution). Every node pulls from it.
|
||||||
|
- These are two protocols, not one store being weak — see [ADR 0075](../02-DECISIONS/0075-two-stores-and-which-provides-what.md).
|
||||||
|
|
||||||
|
## How modules relate to the mesh
|
||||||
|
|
||||||
|
- **seat** — a named position at a scope (node / site / mesh) with a **capacity**. A capacity-1 seat
|
||||||
|
is exclusive (one holder); a higher-capacity seat is a **bench** (several holders coexist).
|
||||||
|
- **claim** — a module taking a spot on a seat. `claims: [{name, scope}]` in a manifest. A
|
||||||
|
mesh-scoped exclusive claim is how the mesh says "there is one of me" — e.g. `mesh-controller`
|
||||||
|
claims `the-controller`.
|
||||||
|
- **provision** — a service one module `provides` and others `require`; the mesh resolves a provider
|
||||||
|
and wires the two with an endpoint and a credential. This is separate from seats: a provision is
|
||||||
|
a service you offer, a seat is a slot you occupy.
|
||||||
|
|
||||||
|
## How this page is kept
|
||||||
|
|
||||||
|
A new name for an existing thing lands here first, in the same change that introduces it in code. A
|
||||||
|
record under `02-DECISIONS/` keeps whatever word it was written with — those are immutable — so a
|
||||||
|
term retired here may still appear there, and the mapping above is how to read it.
|
||||||
+2
-2
@@ -28,8 +28,8 @@ target, not the present.
|
|||||||
| Repository | Tier | Holds |
|
| Repository | Tier | Holds |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `mesh-host` | 0 | **exists.** The node host — one statically linked binary, requiring nothing present ([ADR 0005](../02-DECISIONS/0005-the-node-host.md)) |
|
| `mesh-host` | 0 | **exists.** The node host — one statically linked binary, requiring nothing present ([ADR 0005](../02-DECISIONS/0005-the-node-host.md)) |
|
||||||
| `mesh-substrate` | 1 | the four pinned services, as declarations |
|
| `mesh-foundation` | 1 | the four pinned services, as declarations |
|
||||||
| `mesh-control` | 2 | **exists.** The control plane and its contexts — one of seven built ([ADR 0006](../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)) |
|
| `mesh-control` | 2 | **exists.** The controller and its contexts — one of seven built ([ADR 0006](../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)) |
|
||||||
| `mesh-surfaces` | 3 | tools, web, cli |
|
| `mesh-surfaces` | 3 | tools, web, cli |
|
||||||
| `mesh-sdk` | — | the stable spine modules build against — the tool-serving harness, the messaging/event framework, the contracts and core primitives. Holds nothing per-module and nothing volatile ([ADR 0039](../02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md)). |
|
| `mesh-sdk` | — | the stable spine modules build against — the tool-serving harness, the messaging/event framework, the contracts and core primitives. Holds nothing per-module and nothing volatile ([ADR 0039](../02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md)). |
|
||||||
| `mesh-lab` | — | **exists.** The lab — scenario lifecycle, networking, placement. Ships to nobody; runs on a workstation. |
|
| `mesh-lab` | — | **exists.** The lab — scenario lifecycle, networking, placement. Ships to nobody; runs on a workstation. |
|
||||||
|
|||||||
@@ -0,0 +1,172 @@
|
|||||||
|
---
|
||||||
|
topic: the tiers
|
||||||
|
status: accepted
|
||||||
|
date: 2026-09-15
|
||||||
|
deciders: jochen
|
||||||
|
reconstructed: false
|
||||||
|
extends: 0039-what-the-sdk-holds-and-refuses.md
|
||||||
|
---
|
||||||
|
|
||||||
|
# 74. The mesh defines a module protocol; an SDK is an implementation of it
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
[ADR 0039](0039-what-the-sdk-holds-and-refuses.md) says what the SDK holds: the tool-serving
|
||||||
|
harness, the messaging and event framework, the contracts, and core primitives. It settles what
|
||||||
|
belongs in *an* SDK. It does not say what happens when there is more than one.
|
||||||
|
|
||||||
|
There is already more than one. **The contracts are expressed twice** — as Go types in the control
|
||||||
|
plane and the host, and as TypeScript types in the SDK — and nobody has felt it because both live
|
||||||
|
in one repository and one head.
|
||||||
|
|
||||||
|
**A correction, made after inspecting the wire rather than the types** (2026-09-16). This record
|
||||||
|
first claimed the two implementations already disagreed — `resource` vs `Provision`, `consumer`
|
||||||
|
meaning the module in one and the node in the other, headers declared on one side and emitted by
|
||||||
|
neither. **On inspection the live wire agrees**, and the claim was wrong:
|
||||||
|
|
||||||
|
- The grant types that disagreed (`Grant`, `Interface`, `Credential` in the SDK's `contracts`)
|
||||||
|
were **dead** — exported and imported by nothing. The live provisioning wire is the contributions
|
||||||
|
file, whose shape (`as`, `secret`, `node`, `at`, `values`) is the same on both sides. Those dead
|
||||||
|
types have been removed.
|
||||||
|
- The envelope agrees too: Go emits all five required headers, and `x-causation-id`/`x-schema` are
|
||||||
|
**optional** — the SDK sets them when a handler has a causation or a schema, and a bare event
|
||||||
|
carrying neither is correct, not a drift.
|
||||||
|
|
||||||
|
So the danger was never live disagreement. It was **dead types that contradicted the live wire**,
|
||||||
|
which read as the contract and were not — and are exactly what led this record to assert a drift
|
||||||
|
that inspection did not find. That is a sharper reason for the decision below, not a weaker one: a
|
||||||
|
type is only as good as its being the wire, and the way to guarantee that is to specify the wire and
|
||||||
|
check implementations against it, rather than to trust a hand-kept type to still describe it.
|
||||||
|
|
||||||
|
A failure of this kind does not announce itself. Two implementations that disagree about an
|
||||||
|
envelope do not fail to compile — they ignore each other's messages, and a mesh where a module
|
||||||
|
stops reacting looks exactly like a mesh where nothing happened.
|
||||||
|
|
||||||
|
## The question this settles
|
||||||
|
|
||||||
|
A module may be written in any language the mesh can build
|
||||||
|
([`18-building-a-module`](../03-DESIGN/01-to-be/18-building-a-module.md)). Every language needs an
|
||||||
|
SDK. What is an SDK *of*?
|
||||||
|
|
||||||
|
Two answers were available, and the obvious one is wrong.
|
||||||
|
|
||||||
|
**Shared types, generated.** Write the shapes once — a schema, an IDL — and generate Go, TypeScript,
|
||||||
|
Rust. It is the familiar answer and it solves the smaller half of the problem. The shapes are not
|
||||||
|
where the difficulty is.
|
||||||
|
|
||||||
|
**A specified wire, with a conformance suite.** The shapes are a consequence; what an SDK must get
|
||||||
|
right is *behaviour*.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
**The mesh defines a module protocol. An SDK is an implementation of that protocol in one
|
||||||
|
language, and nothing more.**
|
||||||
|
|
||||||
|
That is the whole of what an SDK is. Not a library a language happens to have, not a convenience
|
||||||
|
layer, not a place for helpers to accumulate — an implementation of a specified protocol, finished
|
||||||
|
when it implements it and correct when it agrees with every other implementation.
|
||||||
|
|
||||||
|
### The protocol is split per capability
|
||||||
|
|
||||||
|
**A module does not use all of it, so an SDK need not implement all of it.** A module that only
|
||||||
|
consumes events uses the event capability. One that serves tools uses the tool capability. A
|
||||||
|
provider uses provisioning. Nothing about consuming an event requires knowing how a grant is
|
||||||
|
answered.
|
||||||
|
|
||||||
|
So the protocol is a floor plus capabilities:
|
||||||
|
|
||||||
|
| part | what it covers | who needs it |
|
||||||
|
|---|---|---|
|
||||||
|
| **connection** — the floor | reading the sealed credential, pinning the certificate fingerprint, taking identity from the credential rather than the environment | everything |
|
||||||
|
| **events** | the envelope and its headers, the durable per-consumer queue, binding, at-least-once with dedup on `x-event-id` | a module that emits or consumes |
|
||||||
|
| **tools** | registration, the shared durable `serve.<key>` queue, request and reply | a module with a surface |
|
||||||
|
| **provisioning** | a grant in, a credential out, and what each carries | a module that provides something |
|
||||||
|
|
||||||
|
**This is the same shape the host already has.** A host declares which resource kinds it can apply,
|
||||||
|
and a partial host — one that can write files and run things but not manage users or containers —
|
||||||
|
is a real thing rather than a broken one ([ADR 0005](0005-the-node-host.md)). An SDK that implements
|
||||||
|
the floor and events is exactly as legitimate, and a module written against it is a module that
|
||||||
|
does events.
|
||||||
|
|
||||||
|
**So a language arrives in pieces rather than all at once.** A Rust SDK implementing connection and
|
||||||
|
events is useful the day it exists; tools and provisioning follow when something needs them. The
|
||||||
|
alternative — a language is unsupported until it is entirely supported — is what makes adding one a
|
||||||
|
project rather than a contribution.
|
||||||
|
|
||||||
|
**And what a language can be used for is then a fact the mesh can state**, rather than something an
|
||||||
|
author discovers by writing a module that cannot be built: the toolchain list says which languages
|
||||||
|
exist, and the conformance results say what each can do.
|
||||||
|
|
||||||
|
### What the specification covers
|
||||||
|
|
||||||
|
Per capability, what two implementations can disagree about:
|
||||||
|
|
||||||
|
- **the exchanges and queues** — which exchanges exist, that a consumer's queue is durable and
|
||||||
|
named `<node>.<module>.events`, that a tool is served from a shared durable `serve.<key>`
|
||||||
|
- **the envelope** — every header, which are required, what an unknown `x-` header means, and that
|
||||||
|
ignoring one is correct rather than lax
|
||||||
|
- **identity** — that a module's node and module name come from its sealed credential and not from
|
||||||
|
its environment, so what it emits matches what the mesh authorised
|
||||||
|
- **delivery** — at-least-once, and that dedup is on `x-event-id`, which only the emitter can make
|
||||||
|
- **the credential** — the sealed document's fields, and that a connection pins a certificate
|
||||||
|
fingerprint rather than trusting an authority
|
||||||
|
- **provisioning** — a grant in, a credential out, and what each carries
|
||||||
|
- **the vocabulary** — that `consumer` is one thing, named once
|
||||||
|
|
||||||
|
### Conformance is per capability
|
||||||
|
|
||||||
|
**A suite per part, and an SDK claims the parts it passes.** A monolithic pass/fail would make a
|
||||||
|
partial implementation indistinguishable from a broken one, which is the distinction this is built
|
||||||
|
on.
|
||||||
|
|
||||||
|
**And the suite is executable, not prose.** A specification nobody can run is a document two
|
||||||
|
implementations drift from while both believe they conform. Conformance is a set of fixtures — an
|
||||||
|
emitted event, a served tool call, a grant and its answer — that every SDK must produce and consume
|
||||||
|
byte-for-byte.
|
||||||
|
|
||||||
|
**The existing two implementations are the first two to be made to pass it.** Not a future language:
|
||||||
|
the drift above is present, and a suite that only new SDKs must satisfy would leave the disagreement
|
||||||
|
that already exists in place while certifying everything added afterwards against it.
|
||||||
|
|
||||||
|
## Why not generated types
|
||||||
|
|
||||||
|
Generation makes the shapes agree and leaves everything that matters unspecified. Two SDKs
|
||||||
|
generated from one schema can still name their queues differently, take identity from the
|
||||||
|
environment, dedup on the wrong field, or omit a header the other requires — and every one of those
|
||||||
|
is a mesh that runs and quietly does not work.
|
||||||
|
|
||||||
|
It also makes the contract into whatever the generator supports, which is a decision nobody made
|
||||||
|
about a boundary everything else depends on.
|
||||||
|
|
||||||
|
**The shapes are worth generating once the wire is specified.** That is a convenience, and it comes
|
||||||
|
second.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
**A language is a commitment, and now a divisible one.** Adding one means implementing the protocol
|
||||||
|
and passing the suites for the parts it claims. That is more work than transliterating types, and
|
||||||
|
it is the work that was always there — the difference is that it can be finished, and finished in
|
||||||
|
pieces, rather than believed.
|
||||||
|
|
||||||
|
**Versioning becomes possible.** `x-schema` exists for it and is never written. A specified envelope
|
||||||
|
with a version on the body is what lets a mesh hold a module built against an older SDK, which is
|
||||||
|
the ordinary state of any mesh that has been running for a while.
|
||||||
|
|
||||||
|
**The two current implementations agree on the live wire** — inspection showed it. What was wrong
|
||||||
|
was a set of dead types beside the wire, now removed. The suite's job here is therefore prevention:
|
||||||
|
to keep that agreement true as the wire changes, and to hold a new language's SDK to it, rather than
|
||||||
|
to repair a break that exists today.
|
||||||
|
|
||||||
|
**This does not make the mesh polyglot by itself**, and should not be reported as though it does. It
|
||||||
|
makes polyglot possible to do correctly. A Rust SDK is still a Rust SDK.
|
||||||
|
|
||||||
|
## How this is checked
|
||||||
|
|
||||||
|
| Rule | Checked by |
|
||||||
|
|---|---|
|
||||||
|
| One vocabulary | A word means one thing across implementations, checked by the fixtures using it. |
|
||||||
|
| The wire is what is specified | Both existing SDKs run the conformance suite in their own test suites, and a change to one that breaks a fixture fails there rather than in a mesh. |
|
||||||
|
| A new SDK is a passing SDK | A language is not listed as buildable for a capability until its SDK passes that capability's suite; the toolchain list and the conformance results name the same set. |
|
||||||
|
| A partial SDK is a real thing | An SDK implementing the floor and one capability passes, is listed for that capability, and a module using another is refused with the reason — rather than failing at runtime in a language nobody said was finished. |
|
||||||
|
| An unknown header is ignored | A fixture carries one, and every implementation accepts it. |
|
||||||
|
| Identity comes from the credential | A fixture sets an environment that disagrees with the credential, and the emitted event carries the credential's. |
|
||||||
@@ -0,0 +1,129 @@
|
|||||||
|
---
|
||||||
|
topic: the tiers
|
||||||
|
status: proposed
|
||||||
|
date: 2026-09-15
|
||||||
|
deciders: jochen
|
||||||
|
reconstructed: false
|
||||||
|
extends: 0014-no-npm-workspace.md
|
||||||
|
---
|
||||||
|
|
||||||
|
# 75. An artifact store is a provision; a package registry is a different one
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
Two questions have been circling, and they turn out to be one question asked twice.
|
||||||
|
|
||||||
|
**"Should gitea be the mesh's registry?"** It serves OCI images and a dozen package ecosystems, it
|
||||||
|
is already needed — genesis clones from one — and the mesh's own registry has neither
|
||||||
|
authentication ([issue 042](../04-ISSUES/042-nothing-gives-a-node-an-account-for-a-registry/00-report.md))
|
||||||
|
nor a transport a runtime will accept over a network
|
||||||
|
([issue 048](../04-ISSUES/048-nothing-makes-a-machine-trust-the-mesh-registry/00-report.md)). Gitea
|
||||||
|
has both.
|
||||||
|
|
||||||
|
**"Where does the SDK come from?"** [ADR 0014](0014-no-npm-workspace.md) already answers it — each
|
||||||
|
module consumes its dependencies, the mesh's own shared library included, *from the private
|
||||||
|
registry* — and nothing installs one, so today it comes from a git URL, which is
|
||||||
|
[issue 053](../04-ISSUES/053-the-sdk-is-pinned-twice-and-the-two-disagree/00-report.md).
|
||||||
|
|
||||||
|
**The framing that dissolves both:** `artifact-store` is already a provision, and `registry`
|
||||||
|
already provides it. So "should gitea be the registry" is not a question about replacing a
|
||||||
|
component. It is a question about **a second provider of an existing provision** — which this mesh
|
||||||
|
has a mechanism for, and uses for certificate authorities and VPNs already.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
**Two provisions, because they are two jobs.**
|
||||||
|
|
||||||
|
| provision | is | for |
|
||||||
|
|---|---|---|
|
||||||
|
| `artifact-store` | content-addressed blobs, pinned by digest, no versions, no ranges | what the **mesh** delivers to **machines** |
|
||||||
|
| `package-registry` | an ecosystem's own registry — npm, cargo, PyPI, Go | what **code** resolves when it is compiled |
|
||||||
|
|
||||||
|
They are not the same store with different clients. One is addressed by digest and immutable by
|
||||||
|
construction; the other is addressed by name and version, and resolves ranges. Conflating them is
|
||||||
|
how a mesh that pins everything ends up rebuilding one commit into two different things.
|
||||||
|
|
||||||
|
**`registry` remains the provider genesis installs.** Not because it is better, but because of what
|
||||||
|
it is: a directory and one container, no database, no control plane, installable at step 8 of an
|
||||||
|
install where neither exists yet. Gitea needs a store and provisioning, which means a control plane,
|
||||||
|
which means the pivot has already happened — and the pivot needs somewhere to publish to.
|
||||||
|
|
||||||
|
**Gitea also provides `artifact-store`, and a mesh may choose it.** Two providers of one provision
|
||||||
|
is a thing the mesh understands: it refuses, names both, and choosing is assigning the one you want.
|
||||||
|
|
||||||
|
**And it does not claim `the-artifact-store`.** That claim is node-scoped, so a module holding it
|
||||||
|
cannot share a machine with another that does — and a machine running gitea for git and packages
|
||||||
|
*alongside* a registry serving artifacts is an ordinary arrangement, not a conflict. They are
|
||||||
|
different ports doing different jobs.
|
||||||
|
|
||||||
|
The exclusivity that matters is mesh-wide and is already expressed: `provides` at mesh scope means
|
||||||
|
two providers are two answers, and the resolver refuses until one is assigned. Forbidding
|
||||||
|
co-residence adds nothing to that and forbids something reasonable. **Whether `registry` should
|
||||||
|
still hold that claim is left open here** — it may be protecting something about the port or the
|
||||||
|
data directory that is not written down, and removing a claim is not a thing to do from the outside
|
||||||
|
of a manifest.
|
||||||
|
A mesh that assigns gitea gets authentication and TLS for its artifacts — which is to say, **issues
|
||||||
|
042 and 048 are answered by choosing a provider that already solved them**, rather than by
|
||||||
|
reimplementing accounts and certificates in a registry that has none.
|
||||||
|
|
||||||
|
**Gitea provides `package-registry`.** That is ADR 0014's private registry, and it is one service
|
||||||
|
rather than one per ecosystem. `verdaccio` may provide it too, for npm alone, and is then a choice
|
||||||
|
somebody makes rather than the answer.
|
||||||
|
|
||||||
|
**The registry is not removed at the end of installing.** A mesh that never runs gitea still has an
|
||||||
|
artifact store. Retiring it is a migration a mesh performs, not a step an installation ends with.
|
||||||
|
|
||||||
|
## Why not simply gitea, from the start
|
||||||
|
|
||||||
|
Because genesis would need a control plane before the thing that stores the control plane's image,
|
||||||
|
and that is circular rather than merely awkward. It would also make one of the three things the
|
||||||
|
build loop cannot produce for itself into a stateful application with a database — the pivot is the
|
||||||
|
hardest part of this design already.
|
||||||
|
|
||||||
|
And it puts every artifact in the service that is also the trust anchor for everything the mesh will
|
||||||
|
ever run ([ADR 0071](0071-where-genesis-gets-its-source.md)), which records that forge serving a
|
||||||
|
cryptominer with tampered git operations. Two blast radii are better than one.
|
||||||
|
|
||||||
|
## Moving from one provider to the other is a designed act
|
||||||
|
|
||||||
|
**Not a removal.** Every image a machine runs is pinned to a digest at a named store, the control
|
||||||
|
plane's own included. Changing the provider means:
|
||||||
|
|
||||||
|
1. gitea installed, reachable, and holding an account the builder may publish with
|
||||||
|
2. every artifact mirrored
|
||||||
|
3. every declaration re-pinned, the control plane's **last**, because it is what performs the others
|
||||||
|
4. **every machine verified to have converged and to be able to pull from the new store**
|
||||||
|
5. only then the old provider unassigned, and its volume kept ([ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md))
|
||||||
|
|
||||||
|
Step 4 is the one that is easy to skip and the only thing between this and a mesh that cannot
|
||||||
|
restart its own control plane. A machine that reboots mid-migration pulls from a store that no
|
||||||
|
longer exists, and a local image cache hides that until exactly the moment it matters.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
**The bootstrap is unchanged**, which is the point of keeping the small provider.
|
||||||
|
|
||||||
|
**042 and 048 gain a second answer.** They can be fixed in the registry, or dissolved by choosing a
|
||||||
|
provider that already has accounts and TLS. The second is less work and more service.
|
||||||
|
|
||||||
|
**ADR 0014 becomes satisfiable.** There is a provision for the private registry, something that
|
||||||
|
provides it, and a module may depend on it — so the SDK can be published and consumed rather than
|
||||||
|
cloned, and issue 053 has somewhere to go.
|
||||||
|
|
||||||
|
**A mesh can be minimal or complete, and both are legitimate.** One with the small registry and no
|
||||||
|
gitea builds and runs modules and cannot serve packages. That is a real configuration, not a broken
|
||||||
|
one — the same way a partial host is real.
|
||||||
|
|
||||||
|
**And the bootstrap still has no package registry.** The first build of the shared base happens
|
||||||
|
before anything has installed one. That is the same pivot as everything else and it is **not solved
|
||||||
|
here**: it is named, so the next person does not discover it.
|
||||||
|
|
||||||
|
## How this is checked
|
||||||
|
|
||||||
|
| Rule | Checked by |
|
||||||
|
|---|---|
|
||||||
|
| Genesis needs no database | The installer raises a mesh of one on a machine with nothing, and the artifact store it installs has no store of its own. |
|
||||||
|
| Two providers are a choice, not a conflict | A mesh holding both is asked to resolve `artifact-store` and refuses, naming both, until one is assigned. |
|
||||||
|
| Providers may share a machine | A node is assigned both gitea and a registry, and both run — only one of them answers `artifact-store`. |
|
||||||
|
| The two stores are not interchangeable | A module depending on `package-registry` is not satisfied by `artifact-store`, and the refusal says why. |
|
||||||
|
| A migration is verified before it is finished | The old provider cannot be unassigned while any machine's declaration still names it. |
|
||||||
@@ -0,0 +1,87 @@
|
|||||||
|
---
|
||||||
|
topic: building it
|
||||||
|
status: proposed
|
||||||
|
date: 2026-09-16
|
||||||
|
deciders: jochen
|
||||||
|
reconstructed: false
|
||||||
|
extends: 0075-two-stores-and-which-provides-what.md
|
||||||
|
---
|
||||||
|
|
||||||
|
# 76. The SDK is a published package, and the toolchain resolves it by version
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
[ADR 0014](0014-no-npm-workspace.md) decided a module consumes its dependencies — the mesh's own
|
||||||
|
shared library included — from the private registry. [ADR 0075](0075-two-stores-and-which-provides-what.md)
|
||||||
|
decided the private registry is a `package-registry` provision, and that gitea provides it. What
|
||||||
|
neither settled, and what [issue 053](../04-ISSUES/053-the-sdk-is-pinned-twice-and-the-two-disagree/00-report.md)
|
||||||
|
left open, is the one build where the rule cannot simply be obeyed: **the first one.**
|
||||||
|
|
||||||
|
The TypeScript toolchain image is built *from* the SDK — it carries the SDK so that every module
|
||||||
|
compiled inside it resolves the shared library without each build fetching it. So the thing that
|
||||||
|
compiles TypeScript and the thing that contains the SDK were the same object, and that object
|
||||||
|
cannot be what builds the SDK. Stated as a question — "how does the SDK reach the registry before
|
||||||
|
the toolchain exists, when the toolchain is what builds it?" — it reads as a paradox.
|
||||||
|
|
||||||
|
It is not one. The paradox exists only because the toolchain *bakes a git-cloned copy* of the SDK.
|
||||||
|
The SDK itself is plain TypeScript: it needs `node` and `tsc` and nothing the mesh makes. A public
|
||||||
|
base image can build it. The circularity is a property of the workaround, not of the SDK.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
**The SDK is an ordinary published package in the mesh's `package-registry`, consumed by version.**
|
||||||
|
The git URL in the toolchain's manifest and the sibling-path lock beside it — the two halves of
|
||||||
|
issue 053 — are both removed. A build resolves the SDK the way it resolves any dependency, with a
|
||||||
|
lock that agrees with its manifest, so `npm ci` is the command and reproducibility is by
|
||||||
|
construction rather than by the machine the build ran on.
|
||||||
|
|
||||||
|
**The SDK is built with a public base image, not with the mesh's toolchain.** It is *not* one of
|
||||||
|
the components the loop cannot build — the control plane, the registry, the builder, the catalogue
|
||||||
|
([`12-a-module-repository`](../03-DESIGN/01-to-be/12-a-module-repository.md)), which arrive by
|
||||||
|
carrying an init builder because they are the loop's own machinery. The SDK is machinery for
|
||||||
|
nothing; it is an ordinary dependency the loop builds and publishes like any other. The only
|
||||||
|
constraint is narrow: it cannot be compiled *in the mesh toolchain*, because that toolchain is built
|
||||||
|
from it. So it is compiled on a public base image instead — which needs nothing the mesh makes — and
|
||||||
|
published before the toolchain that consumes it. It is not carried, because building it does not
|
||||||
|
wait on a mesh existing first.
|
||||||
|
|
||||||
|
**The toolchain base stays, thinned.** mesh-tools remains the image bundles are compiled in and the
|
||||||
|
one place the SDK is resolved — but it `npm ci`s the SDK by version from the registry instead of
|
||||||
|
baking a copy cloned from a git URL. Bundles keep borrowing its resolved dependencies; what changes
|
||||||
|
is that the version they borrow is named and honest. This was the shape chosen over dropping the
|
||||||
|
shared base entirely and having every bundle resolve the SDK itself: one resolution point, one
|
||||||
|
place to be right about the version.
|
||||||
|
|
||||||
|
**Genesis orders the publish before the first compile.** The package-registry provider is a public
|
||||||
|
image (gitea), so it comes up needing no toolchain; the SDK is published into it; only then is the
|
||||||
|
toolchain built, so the first `npm ci` has a registry to read from. Nothing in that chain is
|
||||||
|
circular, because the only thing that needed the toolchain — baking the SDK — is gone.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
Each language's toolchain repeats the shape: its own SDK, built from that language's public base
|
||||||
|
image, published to the same registry, resolved by version with that ecosystem's lockfile-honest
|
||||||
|
install (`npm ci`, `cargo` against a vendored or registry source, `pip` against a pinned set). The
|
||||||
|
warning in issue 053 — that whatever the TypeScript repository does the others will copy — is
|
||||||
|
answered by making the copied thing the correct one.
|
||||||
|
|
||||||
|
A change to the SDK is publish-then-consume, exactly as [ADR 0014](0014-no-npm-workspace.md) already
|
||||||
|
priced it: publish the new SDK version, then bump the toolchain (and any module pinning it directly)
|
||||||
|
to consume it. There is no shortcut that resolves an unpublished SDK, which is the property that was
|
||||||
|
missing.
|
||||||
|
|
||||||
|
mesh-tools is no longer an SDK carrier in the sense that mattered — it does not contain a copy
|
||||||
|
whose provenance is a branch head somebody force-pushes. It contains a version.
|
||||||
|
|
||||||
|
A mesh with no package-registry cannot build TypeScript. This is accepted and is not new: it is the
|
||||||
|
same shape as a mesh that cannot reach a forge being unable to be raised
|
||||||
|
([ADR 0071](0071-where-genesis-gets-its-source.md)). Installing brings the registry up first.
|
||||||
|
|
||||||
|
## How this is checked
|
||||||
|
|
||||||
|
| Rule | Checked by |
|
||||||
|
|---|---|
|
||||||
|
| The SDK a build compiles against is named, not cloned from a branch | The toolchain manifest pins `@novox/mesh-sdk` to a version, and the build runs `npm ci`, which refuses a lock that disagrees with the manifest. Issue 053's two checks become this one. |
|
||||||
|
| The SDK builds without the mesh's own toolchain | The SDK's build recipe names a public base image. A recipe that named the mesh toolchain would reintroduce the cycle and is refused in review. |
|
||||||
|
| The registry is up before the first compile | The genesis bed asserts the package-registry answers, and the SDK is published, before the base build runs. A base build that ran first would fail its `npm ci` with no registry, which is the positive control. |
|
||||||
|
| A second language repeats the shape, not a new one | When a second SDK is added, its recipe is compared to this one: public base, publish by version, lockfile-honest install. |
|
||||||
@@ -106,6 +106,9 @@ python3 00-META/checks/index.py fail if stale
|
|||||||
- **0071** — [Genesis clones from a mesh, and checks what it got](0071-where-genesis-gets-its-source.md)
|
- **0071** — [Genesis clones from a mesh, and checks what it got](0071-where-genesis-gets-its-source.md)
|
||||||
- **0072** — [Two graphs, and a build chain that orders itself](0072-two-graphs-and-the-build-chain.md)
|
- **0072** — [Two graphs, and a build chain that orders itself](0072-two-graphs-and-the-build-chain.md)
|
||||||
- **0073** — [The installer carries a builder, and the registry stays where it is](0073-the-installer-carries-a-builder.md)
|
- **0073** — [The installer carries a builder, and the registry stays where it is](0073-the-installer-carries-a-builder.md)
|
||||||
|
- **0074** — [The mesh defines a module protocol; an SDK is an implementation of it](0074-the-wire-is-specified-not-the-types.md)
|
||||||
|
- **0075** — [An artifact store is a provision; a package registry is a different one](0075-two-stores-and-which-provides-what.md) *(proposed)*
|
||||||
|
- **0076** — [The SDK is a published package, and the toolchain resolves it by version](0076-the-sdk-is-a-published-package.md) *(proposed)*
|
||||||
|
|
||||||
### What runs on them, and how it gets there
|
### What runs on them, and how it gets there
|
||||||
|
|
||||||
|
|||||||
@@ -92,7 +92,7 @@ What needs something *usable* retries, which is what both provisioners do and is
|
|||||||
anyway, because a dependency can restart long after everything was applied.
|
anyway, because a dependency can restart long after everything was applied.
|
||||||
|
|
||||||
**The network was a real gap, and the first thing in Phase 1 that needed a decision.** Adding a
|
**The network was a real gap, and the first thing in Phase 1 that needed a decision.** Adding a
|
||||||
shape widens what a compromised control plane can express, so
|
shape widens what a compromised controller can express, so
|
||||||
[ADR 0029](../../02-DECISIONS/0029-a-network-is-a-shape-because-an-action-cannot-be-undone.md)
|
[ADR 0029](../../02-DECISIONS/0029-a-network-is-a-shape-because-an-action-cannot-be-undone.md)
|
||||||
records why this one is worth it: an `action` could create a network and **nothing could ever
|
records why this one is worth it: an `action` could create a network and **nothing could ever
|
||||||
remove it**, because an action leaves no footprint the host can undo. The vocabulary is nine.
|
remove it**, because an action leaves no footprint the host can undo. The vocabulary is nine.
|
||||||
@@ -106,7 +106,7 @@ not after.
|
|||||||
|
|
||||||
*Done 2026-08-31. Worth recording because the task was not the one written down.*
|
*Done 2026-08-31. Worth recording because the task was not the one written down.*
|
||||||
|
|
||||||
**The control plane special-cases nothing.** `provides`, `requires`, `contributes` and `grants`
|
**The controller special-cases nothing.** `provides`, `requires`, `contributes` and `grants`
|
||||||
are name-agnostic — asking for a bucket needed no change to the mesh at all. What was missing was
|
are name-agnostic — asking for a bucket needed no change to the mesh at all. What was missing was
|
||||||
a provider, and the last step where something on the machine turns a delivered secret into a key
|
a provider, and the last step where something on the machine turns a delivered secret into a key
|
||||||
that works. So "add an object-store provision" was never mesh work.
|
that works. So "add an object-store provision" was never mesh work.
|
||||||
@@ -200,7 +200,7 @@ losing something.
|
|||||||
|
|
||||||
*2026-08-31.* **The old system's brain is switched off; its services keep running.**
|
*2026-08-31.* **The old system's brain is switched off; its services keep running.**
|
||||||
|
|
||||||
Not a migration and not a period of dual control. The old control plane — provisioning, the
|
Not a migration and not a period of dual control. The old controller — provisioning, the
|
||||||
coordinator, the pipeline, the things that *decide* and *write* — is stopped. Every workload it
|
coordinator, the pipeline, the things that *decide* and *write* — is stopped. Every workload it
|
||||||
was managing goes on running exactly as it is, because nothing is managing it. Then the new mesh
|
was managing goes on running exactly as it is, because nothing is managing it. Then the new mesh
|
||||||
takes ownership of them one at a time.
|
takes ownership of them one at a time.
|
||||||
@@ -217,7 +217,7 @@ stop having opinions.
|
|||||||
|
|
||||||
**Disabled, not merely stopped**, and this is the part that is easy to get wrong: those units are
|
**Disabled, not merely stopped**, and this is the part that is easy to get wrong: those units are
|
||||||
enabled, so stopping them lasts until the machine reboots. A reboot mid-conversion would bring the
|
enabled, so stopping them lasts until the machine reboots. A reboot mid-conversion would bring the
|
||||||
old control plane back and it would resume regenerating managed files underneath the new one —
|
old controller back and it would resume regenerating managed files underneath the new one —
|
||||||
which is the one situation where two systems really would be fighting over the same machine.
|
which is the one situation where two systems really would be fighting over the same machine.
|
||||||
|
|
||||||
**A service left running with nothing managing it is the safe state.** It has its data, its
|
**A service left running with nothing managing it is the safe state.** It has its data, its
|
||||||
|
|||||||
@@ -40,13 +40,13 @@ not the first one built** ([ADR 0016](../../02-DECISIONS/0016-the-lab.md)).
|
|||||||
|
|
||||||
| | **Bootstrap scenario** | **Full scenario** |
|
| | **Bootstrap scenario** | **Full scenario** |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| Contains | virtual machines, the host binary, a pinned substrate bundle | a complete mesh: forge, coordinator, delivery, modules |
|
| Contains | virtual machines, the host binary, a pinned foundation bundle | a complete mesh: forge, coordinator, delivery, modules |
|
||||||
| Verdict from | what the host reports about the state it reconciled | a pipeline result ending in verify |
|
| Verdict from | what the host reports about the state it reconciled | a pipeline result ending in verify |
|
||||||
| Exercises | the node host and the substrate | the control plane and everything above it |
|
| Exercises | the node host and the foundation | the controller and everything above it |
|
||||||
| Exists to | **develop the mesh** | **test what runs on it** |
|
| Exists to | **develop the mesh** | **test what runs on it** |
|
||||||
|
|
||||||
The bootstrap scenario is a **strict subset**: same virtualisation, same networking, same
|
The bootstrap scenario is a **strict subset**: same virtualisation, same networking, same
|
||||||
lifecycle — it simply stops before a control plane exists. Everything from *"Where this sits in
|
lifecycle — it simply stops before a controller exists. Everything from *"Where this sits in
|
||||||
the way work happens"* onward describes the full scenario, and applies once there is a
|
the way work happens"* onward describes the full scenario, and applies once there is a
|
||||||
coordinator to describe.
|
coordinator to describe.
|
||||||
|
|
||||||
@@ -343,7 +343,7 @@ from the existing system has run against any of it yet.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## The substrate
|
## The foundation
|
||||||
|
|
||||||
### A node is a system container
|
### A node is a system container
|
||||||
|
|
||||||
|
|||||||
@@ -205,7 +205,7 @@ machines:
|
|||||||
|
|
||||||
place:
|
place:
|
||||||
all: [host]
|
all: [host]
|
||||||
anchor: [substrate]
|
anchor: [foundation]
|
||||||
|
|
||||||
snapshot: raised
|
snapshot: raised
|
||||||
```
|
```
|
||||||
@@ -334,12 +334,12 @@ than a fork.
|
|||||||
# bootstrap — tiers 0 and 1
|
# bootstrap — tiers 0 and 1
|
||||||
place:
|
place:
|
||||||
all: [host]
|
all: [host]
|
||||||
anchor: [substrate]
|
anchor: [foundation]
|
||||||
|
|
||||||
# full — adds a control plane, a forge, and a module under test
|
# full — adds a controller, a forge, and a module under test
|
||||||
place:
|
place:
|
||||||
all: [host]
|
all: [host]
|
||||||
anchor: [substrate, control, forge]
|
anchor: [foundation, control, forge]
|
||||||
module: a-web-service
|
module: a-web-service
|
||||||
assert:
|
assert:
|
||||||
- the service answers on its published name
|
- the service answers on its published name
|
||||||
@@ -524,7 +524,7 @@ machines:
|
|||||||
|
|
||||||
place:
|
place:
|
||||||
all: [host]
|
all: [host]
|
||||||
anchor: [substrate]
|
anchor: [foundation]
|
||||||
|
|
||||||
snapshot: raised
|
snapshot: raised
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -47,10 +47,10 @@ mesh database, and it has no listening surface.
|
|||||||
|---|---|
|
|---|---|
|
||||||
| `apply` | reconciling declared state on this machine |
|
| `apply` | reconciling declared state on this machine |
|
||||||
| `store` | local state, authoritative while disconnected |
|
| `store` | local state, authoritative while disconnected |
|
||||||
| `link` | the single outbound connection to the control plane |
|
| `link` | the single outbound connection to the controller |
|
||||||
| `profile` | what this machine can be asked to do |
|
| `profile` | what this machine can be asked to do |
|
||||||
| `inventory` | what this machine is and has |
|
| `inventory` | what this machine is and has |
|
||||||
| `substrate.lock` | the pinned tier-1 descriptor, appliable with no mesh present |
|
| `foundation.lock` | the pinned tier-1 descriptor, appliable with no mesh present |
|
||||||
|
|
||||||
### apply
|
### apply
|
||||||
|
|
||||||
@@ -74,7 +74,7 @@ the machine in whatever state it reached, and nothing must claim otherwise.
|
|||||||
|
|
||||||
### store
|
### store
|
||||||
|
|
||||||
Local, and **authoritative while disconnected**. Not a cache of the control plane — the record
|
Local, and **authoritative while disconnected**. Not a cache of the controller — the record
|
||||||
of what this node has applied and what it currently holds.
|
of what this node has applied and what it currently holds.
|
||||||
|
|
||||||
This is structural rather than convenient: if disconnection is an ordinary situation rather
|
This is structural rather than convenient: if disconnection is an ordinary situation rather
|
||||||
@@ -84,7 +84,7 @@ not come back and ask what it is.
|
|||||||
|
|
||||||
### link
|
### link
|
||||||
|
|
||||||
The node's one connection to the control plane, and its security boundary
|
The node's one connection to the controller, and its security boundary
|
||||||
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)).
|
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)).
|
||||||
|
|
||||||
It is the broker connection that already exists
|
It is the broker connection that already exists
|
||||||
@@ -137,11 +137,11 @@ One behaviour, two sources
|
|||||||
|
|
||||||
| Situation | Source |
|
| Situation | Source |
|
||||||
|---|---|
|
|---|---|
|
||||||
| no mesh reachable | `substrate.lock` — the pinned bundle the host carries |
|
| no mesh reachable | `foundation.lock` — the pinned bundle the host carries |
|
||||||
| mesh reachable | the control plane, over the link |
|
| mesh reachable | the controller, over the link |
|
||||||
|
|
||||||
**The first node is not a different kind of node.** It is a node whose mesh is not up yet. It
|
**The first node is not a different kind of node.** It is a node whose mesh is not up yet. It
|
||||||
applies the bundle it carries, the control plane comes up on top of it, and from that moment it
|
applies the bundle it carries, the controller comes up on top of it, and from that moment it
|
||||||
takes declarations like every other node. Its specialness is temporary and self-erasing.
|
takes declarations like every other node. Its specialness is temporary and self-erasing.
|
||||||
|
|
||||||
**A joining node does the minimum to be reachable and nothing else** — an identity, an address,
|
**A joining node does the minimum to be reachable and nothing else** — an identity, an address,
|
||||||
@@ -155,7 +155,7 @@ Settled by [ADR 0005](../../02-DECISIONS/0005-the-node-host.md).
|
|||||||
**JSON**, because the host has no dependencies to spend and the standard library carries no
|
**JSON**, because the host has no dependencies to spend and the standard library carries no
|
||||||
YAML. **An ordered list of typed resources**, each with a stable identity — the order is stated
|
YAML. **An ordered list of typed resources**, each with a stable identity — the order is stated
|
||||||
rather than derived, because deriving it would be the host deciding the thing most likely to
|
rather than derived, because deriving it would be the host deciding the thing most likely to
|
||||||
differ from what the control plane intended.
|
differ from what the controller intended.
|
||||||
|
|
||||||
**Unknown is refused, never skipped.** An unknown version, type or field refuses the whole
|
**Unknown is refused, never skipped.** An unknown version, type or field refuses the whole
|
||||||
declaration. A host that skipped what it did not understand would apply most of it and report
|
declaration. A host that skipped what it did not understand would apply most of it and report
|
||||||
@@ -173,16 +173,16 @@ without one, applying the bundle it carries, has nothing to check against.
|
|||||||
Staged so each stage is verifiable in the lab before the next exists.
|
Staged so each stage is verifiable in the lab before the next exists.
|
||||||
|
|
||||||
**1 — profile and inventory.** The host runs on a machine, detects what it can do, and reports
|
**1 — profile and inventory.** The host runs on a machine, detects what it can do, and reports
|
||||||
what it is. No control plane, no declarations, no network. Verifiable immediately: the lab's
|
what it is. No controller, no declarations, no network. Verifiable immediately: the lab's
|
||||||
`place:` gains its first implementation, and a raised scenario finally contains something.
|
`place:` gains its first implementation, and a raised scenario finally contains something.
|
||||||
|
|
||||||
**2 — apply, from the bundle.** The host applies `substrate.lock` with no mesh present. This is
|
**2 — apply, from the bundle.** The host applies `foundation.lock` with no mesh present. This is
|
||||||
the first node's path, and it is the claim the skeleton's Move 1 rests on and has never proved:
|
the first node's path, and it is the claim the skeleton's Move 1 rests on and has never proved:
|
||||||
that one host can raise the substrate alone.
|
that one host can raise the foundation alone.
|
||||||
|
|
||||||
Raising the substrate uses **four** shapes — `package`, `container`, `service`, `action` —
|
Raising the foundation uses **four** shapes — `package`, `container`, `service`, `action` —
|
||||||
counted from the bundle that exists rather than reasoned about. `file` and `directory` are listed
|
counted from the bundle that exists rather than reasoned about. `file` and `directory` are listed
|
||||||
below because they are the cheapest to be sure of and a substrate that needed them would find them
|
below because they are the cheapest to be sure of and a foundation that needed them would find them
|
||||||
ready; the current bundle simply does not. **All of them are built:**
|
ready; the current bundle simply does not. **All of them are built:**
|
||||||
|
|
||||||
| | | |
|
| | | |
|
||||||
@@ -225,7 +225,7 @@ container runtime. All three were instead verified against a real machine — a
|
|||||||
labelled, replaced when its declaration changed, exec'd into and removed; an action that exits
|
labelled, replaced when its declaration changed, exec'd into and removed; an action that exits
|
||||||
zero and satisfies nothing failing the apply. That is lab-installation work
|
zero and satisfies nothing failing the apply. That is lab-installation work
|
||||||
([`04-lab-installation.md`](04-lab-installation.md)) rather than a constraint on the design, but
|
([`04-lab-installation.md`](04-lab-installation.md)) rather than a constraint on the design, but
|
||||||
until it is done the substrate bootstrap has no end-to-end test.
|
until it is done the foundation bootstrap has no end-to-end test.
|
||||||
|
|
||||||
**3 — link and store.** The node connects, receives declarations, and holds what it applied.
|
**3 — link and store.** The node connects, receives declarations, and holds what it applied.
|
||||||
|
|
||||||
@@ -313,7 +313,7 @@ reported to be distinguishable from one that reported an empty list.*
|
|||||||
|
|
||||||
## Open
|
## Open
|
||||||
|
|
||||||
- **Whether one host can raise the substrate alone.** Move 1 assumes it. Stage 2 tests it, and
|
- **Whether one host can raise the foundation alone.** Move 1 assumes it. Stage 2 tests it, and
|
||||||
if it is false the tier boundary moves.
|
if it is false the tier boundary moves.
|
||||||
- **What the host carries versus what it finds.** It manages `wg`, `nft`, `pacman`, `docker`;
|
- **What the host carries versus what it finds.** It manages `wg`, `nft`, `pacman`, `docker`;
|
||||||
it does not contain them, and how it obtains one it lacks is undecided —
|
it does not contain them, and how it obtains one it lacks is undecided —
|
||||||
@@ -329,15 +329,15 @@ reported to be distinguishable from one that reported an empty list.*
|
|||||||
|
|
||||||
## What was added to the vocabulary, and why each cost was worth paying
|
## What was added to the vocabulary, and why each cost was worth paying
|
||||||
|
|
||||||
*Written 2026-08-30. Every addition widens what a compromised control plane can express, so the
|
*Written 2026-08-30. Every addition widens what a compromised controller can express, so the
|
||||||
count is asserted by a test and a change to it is a decision rather than a convenience.*
|
count is asserted by a test and a change to it is a decision rather than a convenience.*
|
||||||
|
|
||||||
Four shapes raise the substrate. Five more exist because most of what a person installs is not a
|
Four shapes raise the foundation. Five more exist because most of what a person installs is not a
|
||||||
service:
|
service:
|
||||||
|
|
||||||
| | why |
|
| | why |
|
||||||
|---|---|
|
|---|---|
|
||||||
| **file**, **directory** | the substrate needs neither, and almost everything else does |
|
| **file**, **directory** | the foundation needs neither, and almost everything else does |
|
||||||
| **user** | a shell, a terminal, a chat client, a desktop are a package plus configuration **in somebody's home**. A mesh with no user owns `/etc` and nothing anybody looks at |
|
| **user** | a shell, a terminal, a chat client, a desktop are a package plus configuration **in somebody's home**. A mesh with no user owns `/etc` and nothing anybody looks at |
|
||||||
| **archive** | a theme is hundreds of files. Inlining them makes every declaration enormous and rewrites all of them when one changes |
|
| **archive** | a theme is hundreds of files. Inlining them makes every declaration enormous and rewrites all of them when one changes |
|
||||||
| **network** | a module of several containers has to let them reach each other by name, and doing it with an action would create something nothing could ever remove ([ADR 0029](../../02-DECISIONS/0029-a-network-is-a-shape-because-an-action-cannot-be-undone.md)) |
|
| **network** | a module of several containers has to let them reach each other by name, and doing it with an action would create something nothing could ever remove ([ADR 0029](../../02-DECISIONS/0029-a-network-is-a-shape-because-an-action-cannot-be-undone.md)) |
|
||||||
|
|||||||
+23
-23
@@ -12,7 +12,7 @@ decisions:
|
|||||||
- 02-DECISIONS/0019-how-this-repository-works.md
|
- 02-DECISIONS/0019-how-this-repository-works.md
|
||||||
---
|
---
|
||||||
|
|
||||||
# The control plane
|
# The controller
|
||||||
|
|
||||||
Tier 2. The term appears seventy-nine times across this repository and was defined nowhere,
|
Tier 2. The term appears seventy-nine times across this repository and was defined nowhere,
|
||||||
which is `how-we-build` §5 failing on this repository's own vocabulary.
|
which is `how-we-build` §5 failing on this repository's own vocabulary.
|
||||||
@@ -22,7 +22,7 @@ This document defines it. It does **not** design the contexts inside it; those a
|
|||||||
|
|
||||||
## The definition
|
## The definition
|
||||||
|
|
||||||
> **The control plane is everything that needs to know about more than one node.**
|
> **The controller is everything that needs to know about more than one node.**
|
||||||
|
|
||||||
That is the whole test, and it is not arbitrary — it follows from
|
That is the whole test, and it is not arbitrary — it follows from
|
||||||
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md). The host applies and
|
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md). The host applies and
|
||||||
@@ -32,11 +32,11 @@ exactly there:
|
|||||||
| Question | Whose |
|
| Question | Whose |
|
||||||
|---|---|
|
|---|---|
|
||||||
| write this file, with this content, with this mode | the **host** — one machine |
|
| write this file, with this content, with this mode | the **host** — one machine |
|
||||||
| which nodes should run the store | the **control plane** — needs every node |
|
| which nodes should run the store | the **controller** — needs every node |
|
||||||
| is this unit running | the **host** — one machine |
|
| is this unit running | the **host** — one machine |
|
||||||
| which peers belong in this node's overlay | the **control plane** — needs every node |
|
| which peers belong in this node's overlay | the **controller** — needs every node |
|
||||||
| what does this machine have installed | the **host** reports; the control plane **records** |
|
| what does this machine have installed | the **host** reports; the controller **records** |
|
||||||
| has this node been unreachable for a week | the **control plane** — nobody else is watching |
|
| has this node been unreachable for a week | the **controller** — nobody else is watching |
|
||||||
|
|
||||||
A useful consequence: **anything a single machine could answer alone is not the control
|
A useful consequence: **anything a single machine could answer alone is not the control
|
||||||
plane's.** If it needs no second node, putting it here is a mistake, and the tier rule will not
|
plane's.** If it needs no second node, putting it here is a mistake, and the tier rule will not
|
||||||
@@ -67,7 +67,7 @@ something infrastructure*. `ai` is folded into `config`: a provider licence is a
|
|||||||
**`record` is an open question rather than an eighth entry.** Contexts integrate through it
|
**`record` is an open question rather than an eighth entry.** Contexts integrate through it
|
||||||
([ADR 0008](../../02-DECISIONS/0008-a-context-owns-its-store.md)), which makes it load-bearing,
|
([ADR 0008](../../02-DECISIONS/0008-a-context-owns-its-store.md)), which makes it load-bearing,
|
||||||
and [research 006](../../01-RESEARCH/006-mesh-from-scratch/skeleton.md) leaves *where it lives*
|
and [research 006](../../01-RESEARCH/006-mesh-from-scratch/skeleton.md) leaves *where it lives*
|
||||||
unresolved — putting it in the substrate risks recreating the circularity the tier design just
|
unresolved — putting it in the foundation risks recreating the circularity the tier design just
|
||||||
removed. Listing it here would settle by naming what has not been settled by arguing.
|
removed. Listing it here would settle by naming what has not been settled by arguing.
|
||||||
|
|
||||||
**One of the seven is built.** `inventory` owns a database of that name and holds the node records;
|
**One of the seven is built.** `inventory` owns a database of that name and holds the node records;
|
||||||
@@ -89,7 +89,7 @@ in front of them.
|
|||||||
The question this answers: **can a node write to the registry database?** No — and not "only
|
The question this answers: **can a node write to the registry database?** No — and not "only
|
||||||
through one node", which is the weaker arrangement it might be mistaken for.
|
through one node", which is the weaker arrangement it might be mistaken for.
|
||||||
|
|
||||||
> **No node holds a credential to any control-plane store, for writing or for reading.**
|
> **No node holds a credential to any controller store, for writing or for reading.**
|
||||||
|
|
||||||
That is not a new rule here; it is four already taken, and it is worth seeing them together
|
That is not a new rule here; it is four already taken, and it is worth seeing them together
|
||||||
because each one alone reads like a detail:
|
because each one alone reads like a detail:
|
||||||
@@ -118,11 +118,11 @@ Reads work the same way in reverse — a node is *told*, in declarations. It nev
|
|||||||
|
|
||||||
### Who actually consumes, and who writes
|
### Who actually consumes, and who writes
|
||||||
|
|
||||||
**The control plane is the consumer. There is one of it, and the context that owns the data does
|
**The controller is the consumer. There is one of it, and the context that owns the data does
|
||||||
the write.**
|
the write.**
|
||||||
|
|
||||||
```
|
```
|
||||||
node ──► broker ──► the control plane, consuming
|
node ──► broker ──► the controller, consuming
|
||||||
├─ a node reported what it applied ─► inventory writes the registry
|
├─ a node reported what it applied ─► inventory writes the registry
|
||||||
├─ a node reported health ─► observability writes its own store
|
├─ a node reported health ─► observability writes its own store
|
||||||
└─ a grant was requested ─► provisioning writes its own store
|
└─ a grant was requested ─► provisioning writes its own store
|
||||||
@@ -139,9 +139,9 @@ the store it exclusively owns
|
|||||||
receiving half of what it expects* — which has happened, between a module's daemon and its
|
receiving half of what it expects* — which has happened, between a module's daemon and its
|
||||||
capability server. With one consumer that class of fault cannot arise.
|
capability server. With one consumer that class of fault cannot arise.
|
||||||
|
|
||||||
**And the broker is the buffer while the control plane is down.** Nodes go on publishing;
|
**And the broker is the buffer while the controller is down.** Nodes go on publishing;
|
||||||
messages queue; the control plane drains them when it returns. That is what makes
|
messages queue; the controller drains them when it returns. That is what makes
|
||||||
[ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)'s single control plane
|
[ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)'s single controller
|
||||||
tolerable — an outage delays the mesh's *knowledge* rather than losing it.
|
tolerable — an outage delays the mesh's *knowledge* rather than losing it.
|
||||||
|
|
||||||
**With one consequence that must be bounded before it is discovered:** a queue with no limit
|
**With one consequence that must be bounded before it is discovered:** a queue with no limit
|
||||||
@@ -177,7 +177,7 @@ volume genuinely argues against a relational store.
|
|||||||
node except through the host.
|
node except through the host.
|
||||||
- **Not a surface.** Tier 3 is how people and agents reach it. It has one interface; the
|
- **Not a surface.** Tier 3 is how people and agents reach it. It has one interface; the
|
||||||
surfaces are what speak to that interface.
|
surfaces are what speak to that interface.
|
||||||
- **Not the substrate.** It *runs on* tier 1 — PostgreSQL, LavinMQ, an OCI registry
|
- **Not the foundation.** It *runs on* tier 1 — PostgreSQL, LavinMQ, an OCI registry
|
||||||
([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)) — and cannot start without
|
([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)) — and cannot start without
|
||||||
them, which is what makes them a lower tier.
|
them, which is what makes them a lower tier.
|
||||||
- **Not privileged on a node.** It has no more access to a machine than the declaration
|
- **Not privileged on a node.** It has no more access to a machine than the declaration
|
||||||
@@ -185,19 +185,19 @@ volume genuinely argues against a relational store.
|
|||||||
|
|
||||||
## It is also a consumer
|
## It is also a consumer
|
||||||
|
|
||||||
The property that makes tier 2 unlike the others: **the control plane has requirements of its
|
The property that makes tier 2 unlike the others: **the controller has requirements of its
|
||||||
own.** It needs a PostgreSQL database, an AMQP virtual host, and a bucket — the same things any
|
own.** It needs a PostgreSQL database, an AMQP virtual host, and a bucket — the same things any
|
||||||
module needs, granted the same way.
|
module needs, granted the same way.
|
||||||
|
|
||||||
That is the circularity the tiers exist to resolve rather than hide: the control plane cannot
|
That is the circularity the tiers exist to resolve rather than hide: the controller cannot
|
||||||
provision its own database, because it is not running yet. So its **store** is raised from the
|
provision its own database, because it is not running yet. So its **store** is raised from the
|
||||||
bundle the host carries, before there is a control plane to ask
|
bundle the host carries, before there is a controller to ask
|
||||||
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md),
|
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md),
|
||||||
[research 011](../../01-RESEARCH/011-the-module-graph/worked-provider.md)).
|
[research 011](../../01-RESEARCH/011-the-module-graph/worked-provider.md)).
|
||||||
|
|
||||||
Its virtual host and its bucket are **not** in the bundle — by the time they are wanted there is
|
Its virtual host and its bucket are **not** in the bundle — by the time they are wanted there is
|
||||||
a control plane to grant them. Whether the bus must come first is
|
a controller to grant them. Whether the bus must come first is
|
||||||
[open](07-the-substrate.md#open), and it turns on whether these contexts talk to each other over
|
[open](07-the-foundation.md#open), and it turns on whether these contexts talk to each other over
|
||||||
it.
|
it.
|
||||||
|
|
||||||
## Where it runs
|
## Where it runs
|
||||||
@@ -209,10 +209,10 @@ hosts, assigned to nodes by the same mechanism as everything else.
|
|||||||
([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)). The node is assigned,
|
([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)). The node is assigned,
|
||||||
never elected — no promotion, no quorum, no split brain.
|
never elected — no promotion, no quorum, no split brain.
|
||||||
|
|
||||||
That is sound rather than merely cheap, because the design already tolerates the control plane
|
That is sound rather than merely cheap, because the design already tolerates the controller
|
||||||
being absent by construction: a node reconciles from **its own** store
|
being absent by construction: a node reconciles from **its own** store
|
||||||
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)) and
|
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)) and
|
||||||
never needed to ask anybody to hold the state it was last given. So the control plane being down
|
never needed to ask anybody to hold the state it was last given. So the controller being down
|
||||||
is not a new failure mode — it is
|
is not a new failure mode — it is
|
||||||
[ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)'s ordinary disconnected
|
[ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)'s ordinary disconnected
|
||||||
situation, happening to every node at once. **What is lost is change, not operation.**
|
situation, happening to every node at once. **What is lost is change, not operation.**
|
||||||
@@ -226,13 +226,13 @@ every public name.
|
|||||||
- ~~**The contexts themselves.**~~ **Decided** — seven, by
|
- ~~**The contexts themselves.**~~ **Decided** — seven, by
|
||||||
[ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md).
|
[ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md).
|
||||||
What remains open is narrower and named there: **where the record lives**, which research 006
|
What remains open is narrower and named there: **where the record lives**, which research 006
|
||||||
leaves unresolved because the substrate is the one place it must not go.
|
leaves unresolved because the foundation is the one place it must not go.
|
||||||
- **How far it may be split.** One deployable today. Splitting a context out costs the single
|
- **How far it may be split.** One deployable today. Splitting a context out costs the single
|
||||||
interface a surface depends on
|
interface a surface depends on
|
||||||
([research 011](../../01-RESEARCH/011-the-module-graph/00-overview.md)).
|
([research 011](../../01-RESEARCH/011-the-module-graph/00-overview.md)).
|
||||||
- ~~**How many run, and what a node does without one.**~~ **Resolved** by
|
- ~~**How many run, and what a node does without one.**~~ **Resolved** by
|
||||||
[ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md). What remains is
|
[ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md). What remains is
|
||||||
measurement: nothing reports how long the control plane has been unreachable, or how close a
|
measurement: nothing reports how long the controller has been unreachable, or how close a
|
||||||
certificate is to expiry — both needed for restore-not-failover to be a plan rather than a
|
certificate is to expiry — both needed for restore-not-failover to be a plan rather than a
|
||||||
hope.
|
hope.
|
||||||
- **What the interface is.** One interface is stated; its shape, and whether it is request,
|
- **What the interface is.** One interface is stated; its shape, and whether it is request,
|
||||||
@@ -2,7 +2,7 @@
|
|||||||
layer: to-be
|
layer: to-be
|
||||||
status: in-progress
|
status: in-progress
|
||||||
code:
|
code:
|
||||||
- mesh-host examples/substrate-first-node.lock
|
- mesh-host examples/foundation-first-node.lock
|
||||||
- mesh-host internal/apply
|
- mesh-host internal/apply
|
||||||
- mesh-lab test/integration/mesh.test.ts (a bare machine becomes a mesh)
|
- mesh-lab test/integration/mesh.test.ts (a bare machine becomes a mesh)
|
||||||
updated: 2026-08-31
|
updated: 2026-08-31
|
||||||
@@ -15,16 +15,16 @@ decisions:
|
|||||||
- 02-DECISIONS/0019-how-this-repository-works.md
|
- 02-DECISIONS/0019-how-this-repository-works.md
|
||||||
---
|
---
|
||||||
|
|
||||||
# The substrate
|
# The foundation
|
||||||
|
|
||||||
Tier 1. Defined the same way [the control plane](06-the-control-plane.md) is, because the same
|
Tier 1. Defined the same way [the controller](06-the-controller.md) is, because the same
|
||||||
gap applied: the word was load-bearing and unpinned.
|
gap applied: the word was load-bearing and unpinned.
|
||||||
|
|
||||||
## The definition
|
## The definition
|
||||||
|
|
||||||
> **The substrate is what the control plane consumes and cannot grant itself.**
|
> **The foundation is what the controller consumes and cannot grant itself.**
|
||||||
|
|
||||||
Every module that needs a database asks the control plane's provisioning for one. The control
|
Every module that needs a database asks the controller's provisioning for one. The control
|
||||||
plane needs a database too — and it cannot ask itself, because it is not running yet. That
|
plane needs a database too — and it cannot ask itself, because it is not running yet. That
|
||||||
circularity is not an awkwardness to work around; it *is* the definition. Anything on the wrong
|
circularity is not an awkwardness to work around; it *is* the definition. Anything on the wrong
|
||||||
side of it must be raised some other way, and the other way is the bundle the host carries
|
side of it must be raised some other way, and the other way is the bundle the host carries
|
||||||
@@ -32,19 +32,19 @@ side of it must be raised some other way, and the other way is the bundle the ho
|
|||||||
|
|
||||||
The test, applied:
|
The test, applied:
|
||||||
|
|
||||||
| | control plane needs it | can it grant itself one? | |
|
| | controller needs it | can it grant itself one? | |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| a relational store — **PostgreSQL** | its own state lives there | no — provisioning needs the store | **substrate** |
|
| a relational store — **PostgreSQL** | its own state lives there | no — provisioning needs the store | **foundation** |
|
||||||
| a message bus — **LavinMQ** | it reaches nodes over it ([ADR 0002](../../02-DECISIONS/0002-nodes-communicate-over-a-broker.md)) | no — it cannot grant itself a virtual host | **substrate** |
|
| a message bus — **LavinMQ** | it reaches nodes over it ([ADR 0002](../../02-DECISIONS/0002-nodes-communicate-over-a-broker.md)) | no — it cannot grant itself a virtual host | **foundation** |
|
||||||
| ~~an object store~~ | ~~artifacts and blobs it delivers~~ | — | **not substrate** — [ADR 0028](../../02-DECISIONS/0028-the-substrate-supplies-the-control-plane-and-nothing-else.md) |
|
| ~~an object store~~ | ~~artifacts and blobs it delivers~~ | — | **not foundation** — [ADR 0028](../../02-DECISIONS/0028-the-substrate-supplies-the-control-plane-and-nothing-else.md) |
|
||||||
| ~~an image registry~~ | ~~images it delivers to nodes~~ | — | **not substrate** — needed to operate, not to start ([ADR 0033](../../02-DECISIONS/0033-the-substrate-is-a-store-and-a-broker.md)) |
|
| ~~an image registry~~ | ~~images it delivers to nodes~~ | — | **not foundation** — needed to operate, not to start ([ADR 0033](../../02-DECISIONS/0033-the-substrate-is-a-store-and-a-broker.md)) |
|
||||||
| ~~an identity provider~~ | ~~only if it delegates authentication~~ | — | **not substrate** — it delegates to nothing ([ADR 0031](../../02-DECISIONS/0031-the-control-plane-authenticates-nobody.md)) |
|
| ~~an identity provider~~ | ~~only if it delegates authentication~~ | — | **not foundation** — it delegates to nothing ([ADR 0031](../../02-DECISIONS/0031-the-control-plane-authenticates-nobody.md)) |
|
||||||
| ingress — **Traefik** | not to start; only to be reached by name | — it grants itself one afterwards | **not substrate** ([ADR 0007](../../02-DECISIONS/0007-connectivity.md)) |
|
| ingress — **Traefik** | not to start; only to be reached by name | — it grants itself one afterwards | **not foundation** ([ADR 0007](../../02-DECISIONS/0007-connectivity.md)) |
|
||||||
| anything else the mesh hosts | no | — | not substrate |
|
| anything else the mesh hosts | no | — | not foundation |
|
||||||
|
|
||||||
*The object-store row was wrong, and how it was wrong is worth keeping.* It answered *can it
|
*The object-store row was wrong, and how it was wrong is worth keeping.* It answered *can it
|
||||||
grant itself one* — no, it cannot grant itself a bucket — while assuming the first column. **The
|
grant itself one* — no, it cannot grant itself a bucket — while assuming the first column. **The
|
||||||
control plane does not need an object store**: it has no S3 client and never has, and artifacts
|
controller does not need an object store**: it has no S3 client and never has, and artifacts
|
||||||
reach nodes as content-addressed blobs in the registry. The row was inherited from the system being
|
reach nodes as content-addressed blobs in the registry. The row was inherited from the system being
|
||||||
replaced, where an object store distributed module tarballs, and was never re-tested against the
|
replaced, where an object store distributed module tarballs, and was never re-tested against the
|
||||||
definition above it. *Both columns must be answered, and the second is true of almost any service.*
|
definition above it. *Both columns must be answered, and the second is true of almost any service.*
|
||||||
@@ -60,76 +60,76 @@ does not record that the choice was ever made.
|
|||||||
|
|
||||||
The dependency is on the **protocol**, not the product: AMQP for the bus, S3 for the object
|
The dependency is on the **protocol**, not the product: AMQP for the bus, S3 for the object
|
||||||
store, the OCI protocol for the registry. That is what keeps the naming safe rather than a
|
store, the OCI protocol for the registry. That is what keeps the naming safe rather than a
|
||||||
commitment that cannot be revisited — replacing one is a substrate migration, not a redesign.
|
commitment that cannot be revisited — replacing one is a foundation migration, not a redesign.
|
||||||
The store is the exception, and the exception matters: the provisioning model uses databases,
|
The store is the exception, and the exception matters: the provisioning model uses databases,
|
||||||
roles and schemas as PostgreSQL means them, so it is the one member that is not a swap.
|
roles and schemas as PostgreSQL means them, so it is the one member that is not a swap.
|
||||||
|
|
||||||
## What that resolves
|
## What that resolves
|
||||||
|
|
||||||
**Four or five?** [Research 006](../../01-RESEARCH/006-mesh-from-scratch/00-overview.md) asks
|
**Four or five?** [Research 006](../../01-RESEARCH/006-mesh-from-scratch/00-overview.md) asks
|
||||||
whether the identity provider is a substrate service, and the test answers it *conditionally* —
|
whether the identity provider is a foundation service, and the test answers it *conditionally* —
|
||||||
which is the honest answer rather than a number.
|
which is the honest answer rather than a number.
|
||||||
|
|
||||||
- If the control plane **delegates** authentication, it cannot serve anybody before the provider
|
- If the controller **delegates** authentication, it cannot serve anybody before the provider
|
||||||
exists, and it cannot grant itself a client. **Substrate.**
|
exists, and it cannot grant itself a client. **Foundation.**
|
||||||
- If it **authenticates natively**, the provider is an ordinary hosted service like any other.
|
- If it **authenticates natively**, the provider is an ordinary hosted service like any other.
|
||||||
**Not substrate.**
|
**Not foundation.**
|
||||||
|
|
||||||
So the count follows from a design decision that has not been taken, and the record should say
|
So the count follows from a design decision that has not been taken, and the record should say
|
||||||
that rather than assert four.
|
that rather than assert four.
|
||||||
|
|
||||||
**Why not "important infrastructure".** An identity provider, a mail server and an analytics
|
**Why not "important infrastructure".** An identity provider, a mail server and an analytics
|
||||||
service are all infrastructure by any ordinary reading, and none of them are substrate — the
|
service are all infrastructure by any ordinary reading, and none of them are foundation — the
|
||||||
control plane starts and runs without them. *Important* is not the test; *the control plane
|
controller starts and runs without them. *Important* is not the test; *the controller
|
||||||
cannot obtain it* is.
|
cannot obtain it* is.
|
||||||
|
|
||||||
## What the substrate is not
|
## What the foundation is not
|
||||||
|
|
||||||
- **Not tier 0.** The host raises the substrate; it is not part of it. The host carries the
|
- **Not tier 0.** The host raises the foundation; it is not part of it. The host carries the
|
||||||
declaration that brings the substrate up, and depends on nothing.
|
declaration that brings the foundation up, and depends on nothing.
|
||||||
- **Not the control plane.** These are services with no knowledge of the mesh. A store does not
|
- **Not the controller.** These are services with no knowledge of the mesh. A store does not
|
||||||
know what a node is.
|
know what a node is.
|
||||||
- **Not a place for logic.** The skeleton is explicit: tier 1 is *declarations only, no logic of
|
- **Not a place for logic.** The skeleton is explicit: tier 1 is *declarations only, no logic of
|
||||||
its own.* A substrate service is an upstream image, pinned, with configuration.
|
its own.* A foundation service is an upstream image, pinned, with configuration.
|
||||||
- **Not privileged.** The substrate is provisioned *from* by the control plane and grants
|
- **Not privileged.** The foundation is provisioned *from* by the controller and grants
|
||||||
nothing on its own initiative.
|
nothing on its own initiative.
|
||||||
- **Not the mesh's supply of anything**
|
- **Not the mesh's supply of anything**
|
||||||
([ADR 0028](../../02-DECISIONS/0028-the-substrate-supplies-the-control-plane-and-nothing-else.md)).
|
([ADR 0028](../../02-DECISIONS/0028-the-substrate-supplies-the-control-plane-and-nothing-else.md)).
|
||||||
A substrate service and a module of the same product are **different instances**. The mesh's own
|
A foundation service and a module of the same product are **different instances**. The mesh's own
|
||||||
PostgreSQL and a PostgreSQL a workload was given are two servers, and a node hosting both runs
|
PostgreSQL and a PostgreSQL a workload was given are two servers, and a node hosting both runs
|
||||||
two containers — expected, not duplication to be tidied away.
|
two containers — expected, not duplication to be tidied away.
|
||||||
|
|
||||||
The substrate is raised from the bundle before any mesh exists, so **it is not in the module
|
The foundation is raised from the bundle before any mesh exists, so **it is not in the module
|
||||||
graph**: a workload depending on it would depend on something the graph cannot see, cannot rotate
|
graph**: a workload depending on it would depend on something the graph cannot see, cannot rotate
|
||||||
a credential for, and cannot move. It would also put workload data in the store the control plane
|
a credential for, and cannot move. It would also put workload data in the store the controller
|
||||||
keeps its own state in, where a workload that fills a disk takes down the one thing needed to fix
|
keeps its own state in, where a workload that fills a disk takes down the one thing needed to fix
|
||||||
it.
|
it.
|
||||||
|
|
||||||
## The pinned bundle
|
## The pinned bundle
|
||||||
|
|
||||||
`substrate.lock` holds **what must exist before the control plane runs** — which is a smaller
|
`foundation.lock` holds **what must exist before the controller runs** — which is a smaller
|
||||||
set than the substrate, and the difference is easy to miss. It is the only place in the mesh
|
set than the foundation, and the difference is easy to miss. It is the only place in the mesh
|
||||||
where versions are pinned by hand rather than resolved.
|
where versions are pinned by hand rather than resolved.
|
||||||
|
|
||||||
Being substrate and being in the bundle are two different questions:
|
Being foundation and being in the bundle are two different questions:
|
||||||
|
|
||||||
| | is it substrate? | must it precede the control plane? |
|
| | is it foundation? | must it precede the controller? |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| PostgreSQL | yes — the control plane's own state lives in it | **yes** — there is nowhere to put that state otherwise |
|
| PostgreSQL | yes — the controller's own state lives in it | **yes** — there is nowhere to put that state otherwise |
|
||||||
| LavinMQ | yes — it cannot grant itself a virtual host | **yes** — the control plane reaches a node only over the link, and the link is the broker ([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)) |
|
| LavinMQ | yes — it cannot grant itself a virtual host | **yes** — the controller reaches a node only over the link, and the link is the broker ([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)) |
|
||||||
| the OCI registry | yes — it cannot grant itself a repository | no — the first node fetches upstream ([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)) |
|
| the OCI registry | yes — it cannot grant itself a repository | no — the first node fetches upstream ([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)) |
|
||||||
|
|
||||||
The registry is **substrate by role and ordinary by delivery**: by the time it is wanted there is
|
The registry is **foundation by role and ordinary by delivery**: by the time it is wanted there is
|
||||||
a control plane, and it provisions it the way it provisions anything. That keeps the bundle small
|
a controller, and it provisions it the way it provisions anything. That keeps the bundle small
|
||||||
enough for the review [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) requires — one
|
enough for the review [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) requires — one
|
||||||
substrate image until
|
foundation image until
|
||||||
[ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md) established that the
|
[ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md) established that the
|
||||||
broker has to precede the control plane, and two since.
|
broker has to precede the controller, and two since.
|
||||||
|
|
||||||
*Corrected 2026-08-31, from counting what the bundle holds rather than reasoning about it.* **It
|
*Corrected 2026-08-31, from counting what the bundle holds rather than reasoning about it.* **It
|
||||||
carries three images, not two** — PostgreSQL, LavinMQ, and the control plane itself, which the
|
carries three images, not two** — PostgreSQL, LavinMQ, and the controller itself, which the
|
||||||
sentence above had overlooked by counting only substrate services. The control plane is what the
|
sentence above had overlooked by counting only foundation services. The controller is what the
|
||||||
substrate exists to start, and it is in the bundle for the same reason they are: there is nothing
|
foundation exists to start, and it is in the bundle for the same reason they are: there is nothing
|
||||||
to fetch it with yet. It also carries seven actions, a package and a service.
|
to fetch it with yet. It also carries seven actions, a package and a service.
|
||||||
|
|
||||||
**Why pinned:** the bundle is applied when no mesh exists, so nothing can resolve a version, ask
|
**Why pinned:** the bundle is applied when no mesh exists, so nothing can resolve a version, ask
|
||||||
@@ -154,13 +154,13 @@ The order, from [research 011](../../01-RESEARCH/011-the-module-graph/worked-pro
|
|||||||
4 LavinMQ runs pulled by digest, from the bundle
|
4 LavinMQ runs pulled by digest, from the bundle
|
||||||
5 a virtual host, a credential, and actions, run locally
|
5 a virtual host, a credential, and actions, run locally
|
||||||
a self-signed certificate
|
a self-signed certificate
|
||||||
6 the control plane starts and only now is there a mesh
|
6 the controller starts and only now is there a mesh
|
||||||
7 the registry, and everything else the ordinary path
|
7 the registry, and everything else the ordinary path
|
||||||
are provisioned
|
are provisioned
|
||||||
```
|
```
|
||||||
|
|
||||||
**Steps 4 and 5 are why the bundle is not one image**
|
**Steps 4 and 5 are why the bundle is not one image**
|
||||||
([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)). The control plane cannot
|
([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)). The controller cannot
|
||||||
provision the broker, because provisioning means telling a host, and telling a host happens over
|
provision the broker, because provisioning means telling a host, and telling a host happens over
|
||||||
the broker. The first node does not escape this by being local: it enrols the ordinary way, by
|
the broker. The first node does not escape this by being local: it enrols the ordinary way, by
|
||||||
dialling the broker at the address in its token.
|
dialling the broker at the address in its token.
|
||||||
@@ -172,15 +172,15 @@ database* names a thing that will not exist
|
|||||||
database is a boundary a cross-context join cannot casually cross where a separate schema is not.
|
database is a boundary a cross-context join cannot casually cross where a separate schema is not.
|
||||||
|
|
||||||
Only PostgreSQL is raised from the bundle, for the reason in *The pinned bundle* above — the
|
Only PostgreSQL is raised from the bundle, for the reason in *The pinned bundle* above — the
|
||||||
rest of the substrate is wanted only once there is a control plane to provision it.
|
rest of the foundation is wanted only once there is a controller to provision it.
|
||||||
|
|
||||||
**Step 0 is easy to leave out and it is where several things meet.** A substrate service is a
|
**Step 0 is easy to leave out and it is where several things meet.** A foundation service is a
|
||||||
container, so a container runtime must be working before anything else happens — and a runtime
|
container, so a container runtime must be working before anything else happens — and a runtime
|
||||||
is a *package*, not a container.
|
is a *package*, not a container.
|
||||||
|
|
||||||
**Which runtime is detected, not chosen**
|
**Which runtime is detected, not chosen**
|
||||||
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)): a machine that
|
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)): a machine that
|
||||||
already has one keeps it. On a machine with none, the control plane names the package, because
|
already has one keeps it. On a machine with none, the controller names the package, because
|
||||||
what it is called differs per system. It is:
|
what it is called differs per system. It is:
|
||||||
|
|
||||||
- what the host's capability detection already reports, and the first use of that report by
|
- what the host's capability detection already reports, and the first use of that report by
|
||||||
@@ -191,7 +191,7 @@ what it is called differs per system. It is:
|
|||||||
[ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md).
|
[ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md).
|
||||||
|
|
||||||
So the bootstrap uses four shapes: **package**, **container**, **service** and **action** —
|
So the bootstrap uses four shapes: **package**, **container**, **service** and **action** —
|
||||||
*counted from `substrate-first-node.lock`, which is the only bundle there is*. It had said six,
|
*counted from `foundation-first-node.lock`, which is the only bundle there is*. It had said six,
|
||||||
adding `file` and `directory`, which this bootstrap never asks for.
|
adding `file` and `directory`, which this bootstrap never asks for.
|
||||||
|
|
||||||
All four are built, as are the host's other five
|
All four are built, as are the host's other five
|
||||||
@@ -202,7 +202,7 @@ on the host any longer — which is the claim that mattered, and it was true eit
|
|||||||
the bootstrap rather than a service consumers use later. They are **actions** the bundle
|
the bootstrap rather than a service consumers use later. They are **actions** the bundle
|
||||||
declares and the host runs
|
declares and the host runs
|
||||||
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)) — so the
|
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)) — so the
|
||||||
host's vocabulary grows by one shape rather than by one resource type per substrate service.
|
host's vocabulary grows by one shape rather than by one resource type per foundation service.
|
||||||
|
|
||||||
## Open
|
## Open
|
||||||
|
|
||||||
@@ -210,12 +210,12 @@ host's vocabulary grows by one shape rather than by one resource type per substr
|
|||||||
[ADR 0031](../../02-DECISIONS/0031-the-control-plane-authenticates-nobody.md): the control
|
[ADR 0031](../../02-DECISIONS/0031-the-control-plane-authenticates-nobody.md): the control
|
||||||
plane delegates authentication to nothing, so identity is an ordinary module. With the object
|
plane delegates authentication to nothing, so identity is an ordinary module. With the object
|
||||||
store gone ([ADR 0028](../../02-DECISIONS/0028-the-substrate-supplies-the-control-plane-and-nothing-else.md))
|
store gone ([ADR 0028](../../02-DECISIONS/0028-the-substrate-supplies-the-control-plane-and-nothing-else.md))
|
||||||
the substrate is three, and no member is conditional.
|
the foundation is three, and no member is conditional.
|
||||||
- ~~**Whether the bus must precede the control plane.**~~ **Resolved** by
|
- ~~**Whether the bus must precede the controller.**~~ **Resolved** by
|
||||||
[ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md) — it must, and the question as
|
[ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md) — it must, and the question as
|
||||||
posed here could not have answered it. This asked whether the control plane's contexts talk to
|
posed here could not have answered it. This asked whether the controller's contexts talk to
|
||||||
each other over the bus; they do not, being one process, which under this framing would have
|
each other over the bus; they do not, being one process, which under this framing would have
|
||||||
kept LavinMQ out of the bundle. What decides it is how the control plane reaches a *node*, which
|
kept LavinMQ out of the bundle. What decides it is how the controller reaches a *node*, which
|
||||||
is only ever over the link.
|
is only ever over the link.
|
||||||
- **What issues the broker's certificate at bootstrap.** New, and created by the row above. A
|
- **What issues the broker's certificate at bootstrap.** New, and created by the row above. A
|
||||||
token pins the fingerprint a host must expect before it sends anything
|
token pins the fingerprint a host must expect before it sends anything
|
||||||
@@ -223,7 +223,7 @@ host's vocabulary grows by one shape rather than by one resource type per substr
|
|||||||
moment when there is no mesh to issue one and no public name to obtain one for. Self-signed and
|
moment when there is no mesh to issue one and no public name to obtain one for. Self-signed and
|
||||||
pinned is the shape that fits; how it is later replaced by the certificates in
|
pinned is the shape that fits; how it is later replaced by the certificates in
|
||||||
[`08-connectivity.md`](08-connectivity.md) is not decided.
|
[`08-connectivity.md`](08-connectivity.md) is not decided.
|
||||||
- **How a context added later gets its database.** By then there is a control plane — but one
|
- **How a context added later gets its database.** By then there is a controller — but one
|
||||||
holding a credential that can create databases holds more than what it exclusively owns
|
holding a credential that can create databases holds more than what it exclusively owns
|
||||||
([ADR 0008](../../02-DECISIONS/0008-a-context-owns-its-store.md)).
|
([ADR 0008](../../02-DECISIONS/0008-a-context-owns-its-store.md)).
|
||||||
- ~~**Whether the host can do step 2.**~~ **Resolved** by
|
- ~~**Whether the host can do step 2.**~~ **Resolved** by
|
||||||
@@ -234,8 +234,8 @@ host's vocabulary grows by one shape rather than by one resource type per substr
|
|||||||
the module that provides one.
|
the module that provides one.
|
||||||
- **Whether one host can raise all three.** The claim under stage 2 of
|
- **Whether one host can raise all three.** The claim under stage 2 of
|
||||||
[the node host](05-the-node-host.md), never proved. If it is false, the tier boundary moves.
|
[the node host](05-the-node-host.md), never proved. If it is false, the tier boundary moves.
|
||||||
- **How the substrate is updated once a mesh exists.** Pinned by hand at bootstrap; afterwards
|
- **How the foundation is updated once a mesh exists.** Pinned by hand at bootstrap; afterwards
|
||||||
the control plane could deliver it like anything else, and nothing says whether it does.
|
the controller could deliver it like anything else, and nothing says whether it does.
|
||||||
|
|
||||||
## Raised, and observed
|
## Raised, and observed
|
||||||
|
|
||||||
@@ -243,7 +243,7 @@ host's vocabulary grows by one shape rather than by one resource type per substr
|
|||||||
|
|
||||||
**It works, and what that means precisely:** a machine with a container runtime and nothing else
|
**It works, and what that means precisely:** a machine with a container runtime and nothing else
|
||||||
applied the bundle its host carries and ended with a store, a database per context, those
|
applied the bundle its host carries and ended with a store, a database per context, those
|
||||||
contexts' schemas, a broker holding a certificate it generated itself, and the control plane
|
contexts' schemas, a broker holding a certificate it generated itself, and the controller
|
||||||
serving on top of them. Eleven resources, one command, no mesh to ask anything of.
|
serving on top of them. Eleven resources, one command, no mesh to ask anything of.
|
||||||
|
|
||||||
**Then it joined itself.** The same machine took a token, checked the broker against the
|
**Then it joined itself.** The same machine took a token, checked the broker against the
|
||||||
@@ -259,7 +259,7 @@ of a database and pushed to over the broker. What arrived and what did not is th
|
|||||||
|---|---|
|
|---|---|
|
||||||
| the password, in plain text | **on the machine only**, one file, mode 0600 |
|
| the password, in plain text | **on the machine only**, one file, mode 0600 |
|
||||||
| in the declaration that crossed the broker | absent |
|
| in the declaration that crossed the broker | absent |
|
||||||
| in the control plane's database | absent |
|
| in the controller's database | absent |
|
||||||
| in what the node reported back | absent |
|
| in what the node reported back | absent |
|
||||||
|
|
||||||
**One fault, and it was in the joining.** The token did not say what the mesh calls the machine,
|
**One fault, and it was in the joining.** The token did not say what the mesh calls the machine,
|
||||||
@@ -21,25 +21,25 @@ decisions:
|
|||||||
|
|
||||||
# Connectivity
|
# Connectivity
|
||||||
|
|
||||||
One of [the control plane's](06-the-control-plane.md) ten contexts, and the one with the most
|
One of [the controller's](06-the-controller.md) ten contexts, and the one with the most
|
||||||
moving parts: **overlay, resolution, exposure, filtering, certificates.**
|
moving parts: **overlay, resolution, exposure, filtering, certificates.**
|
||||||
|
|
||||||
It is written as a whole because the five are one design. They share inputs, they must agree, and
|
It is written as a whole because the five are one design. They share inputs, they must agree, and
|
||||||
every one of them today is computed in a different place by a different module from a different
|
every one of them today is computed in a different place by a different module from a different
|
||||||
copy of the same facts.
|
copy of the same facts.
|
||||||
|
|
||||||
## Why it is control-plane work
|
## Why it is controller work
|
||||||
|
|
||||||
Apply [the test](06-the-control-plane.md) — *everything that needs to know about more than one
|
Apply [the test](06-the-controller.md) — *everything that needs to know about more than one
|
||||||
node* — to each responsibility:
|
node* — to each responsibility:
|
||||||
|
|
||||||
| | needs to know | whose |
|
| | needs to know | whose |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| **overlay** — who peers with whom, at what address | **every node**, and which of them can be dialled | control plane |
|
| **overlay** — who peers with whom, at what address | **every node**, and which of them can be dialled | controller |
|
||||||
| **resolution** — which name is which node | **every node** | control plane |
|
| **resolution** — which name is which node | **every node** | controller |
|
||||||
| **exposure** — which public name reaches which container | **which node is publicly reachable** ([ADR 0007](../../02-DECISIONS/0007-connectivity.md)) | control plane |
|
| **exposure** — which public name reaches which container | **which node is publicly reachable** ([ADR 0007](../../02-DECISIONS/0007-connectivity.md)) | controller |
|
||||||
| **filtering** — which port is open, to whom | what is assigned here, and the overlay's shape | control plane decides, host applies |
|
| **filtering** — which port is open, to whom | what is assigned here, and the overlay's shape | controller decides, host applies |
|
||||||
| **certificates** — who may present which name | which name belongs to which node | control plane |
|
| **certificates** — who may present which name | which name belongs to which node | controller |
|
||||||
|
|
||||||
**Not one of the five can be answered by a machine on its own.** That is the whole reason this is
|
**Not one of the five can be answered by a machine on its own.** That is the whole reason this is
|
||||||
a context rather than a set of node-local modules — and it is exactly what the current
|
a context rather than a set of node-local modules — and it is exactly what the current
|
||||||
@@ -57,7 +57,7 @@ exist; WireGuard, the resolver and the proxy are all *a container or a package,
|
|||||||
|
|
||||||
It is also what removes the last two upward dependencies.
|
It is also what removes the last two upward dependencies.
|
||||||
[Research 006](../../01-RESEARCH/006-mesh-from-scratch/host-size.md) counted exactly two modules
|
[Research 006](../../01-RESEARCH/006-mesh-from-scratch/host-size.md) counted exactly two modules
|
||||||
opening a direct connection to the control plane's database — `wireguard` and `traefik` — and
|
opening a direct connection to the controller's database — `wireguard` and `traefik` — and
|
||||||
they are the reason every node permanently holds a credential to it
|
they are the reason every node permanently holds a credential to it
|
||||||
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)). Both are connectivity
|
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)). Both are connectivity
|
||||||
modules. **Closing this context closes that set.**
|
modules. **Closing this context closes that set.**
|
||||||
@@ -74,7 +74,7 @@ wanted the exception.
|
|||||||
|
|
||||||
**What made it look unavoidable:** a peer list cannot be written in a manifest. It is derived from
|
**What made it look unavoidable:** a peer list cannot be written in a manifest. It is derived from
|
||||||
every other machine, so it differs on each one and changes when any of them changes. So the
|
every other machine, so it differs on each one and changes when any of them changes. So the
|
||||||
manifest says its resources are **computed** — it names something in the control plane that works
|
manifest says its resources are **computed** — it names something in the controller that works
|
||||||
them out per node — and it is a module in every other respect: assigned, resolved, configured by
|
them out per node — and it is a module in every other respect: assigned, resolved, configured by
|
||||||
settings, and absent from a machine nobody gave it to.
|
settings, and absent from a machine nobody gave it to.
|
||||||
|
|
||||||
@@ -108,7 +108,7 @@ claim, and the collision is refused by name.
|
|||||||
**And the proxy's half, which was the other module reaching into the database.** A web application
|
**And the proxy's half, which was the other module reaching into the database.** A web application
|
||||||
requiring a reverse proxy has to say *which name, which port*, and there was nowhere to put it —
|
requiring a reverse proxy has to say *which name, which port*, and there was nowhere to put it —
|
||||||
`requires` says a thing must exist and never said what to do with it. A module now contributes to
|
`requires` says a thing must exist and never said what to do with it. A module now contributes to
|
||||||
a requirement, the control plane collects every contribution on a node, and the provider is given
|
a requirement, the controller collects every contribution on a node, and the provider is given
|
||||||
them as a file at a path it named. It reloads when that file changes, by the same `restart-on` the
|
them as a file at a path it named. It reloads when that file changes, by the same `restart-on` the
|
||||||
private network needed when a peer list changed under a running interface.
|
private network needed when a peer list changed under a running interface.
|
||||||
|
|
||||||
@@ -177,7 +177,7 @@ which of those it may dial, and which must dial it.
|
|||||||
**Keys.** Each node generates its own keypair. **The private key never leaves the machine**; the
|
**Keys.** Each node generates its own keypair. **The private key never leaves the machine**; the
|
||||||
public key is published to the mesh. This is already true and it is already right — it is
|
public key is published to the mesh. This is already true and it is already right — it is
|
||||||
[ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)'s *a node holds its own
|
[ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)'s *a node holds its own
|
||||||
identity* applied to the overlay, and it means the control plane computes a graph it cannot
|
identity* applied to the overlay, and it means the controller computes a graph it cannot
|
||||||
itself impersonate.
|
itself impersonate.
|
||||||
|
|
||||||
**Shape: a hub, with direct peering between co-located nodes.**
|
**Shape: a hub, with direct peering between co-located nodes.**
|
||||||
@@ -217,7 +217,7 @@ files were right, the services were up, and every node reported success.
|
|||||||
document's own warning, arriving in its implementation: *a more specific route to a dead
|
document's own warning, arriving in its implementation: *a more specific route to a dead
|
||||||
endpoint blackholes; it does not fall back to the general one.*
|
endpoint blackholes; it does not fall back to the general one.*
|
||||||
- **The container runtime closes the door the overlay needs.** Docker sets the FORWARD policy to
|
- **The container runtime closes the door the overlay needs.** Docker sets the FORWARD policy to
|
||||||
DROP, so a hub with `ip_forward` enabled still carries nothing between its spokes. The substrate
|
DROP, so a hub with `ip_forward` enabled still carries nothing between its spokes. The foundation
|
||||||
at tier 1 silently breaks the network at tier 2, and nothing in either tier's state says so. The
|
at tier 1 silently breaks the network at tier 2, and nothing in either tier's state says so. The
|
||||||
hub inserts its own rule above those chains and removes it on the way down.
|
hub inserts its own rule above those chains and removes it on the way down.
|
||||||
|
|
||||||
@@ -287,7 +287,7 @@ node. What routes it once it arrives is a proxy's, and stays separate.
|
|||||||
**The mesh writes the data and runs no daemon.** One wildcard per machine, from the same set that
|
**The mesh writes the data and runs no daemon.** One wildcard per machine, from the same set that
|
||||||
writes the hosts file. A resolver is third-party software and runs *on* the mesh rather than being
|
writes the hosts file. A resolver is third-party software and runs *on* the mesh rather than being
|
||||||
*of* it: the mesh has no business shipping one, choosing which one, or knowing its configuration
|
*of* it: the mesh has no business shipping one, choosing which one, or knowing its configuration
|
||||||
language. Swapping dnsmasq for unbound changes that module and nothing in the control plane.
|
language. Swapping dnsmasq for unbound changes that module and nothing in the controller.
|
||||||
|
|
||||||
**Two roles, two claims, because they are different things.** systemd-resolved cannot answer a
|
**Two roles, two claims, because they are different things.** systemd-resolved cannot answer a
|
||||||
wildcard at all — it routes the mesh's suffix to something that can. Treating serving and asking
|
wildcard at all — it routes the mesh's suffix to something that can. Treating serving and asking
|
||||||
@@ -377,7 +377,7 @@ vocabulary — the mirror of a database grant, where the consumer supplies a tar
|
|||||||
name rather than supplying nothing and receiving credentials.
|
name rather than supplying nothing and receiving credentials.
|
||||||
|
|
||||||
**A workload on an unreachable node is proxied by a reachable one, across the overlay.** Which is
|
**A workload on an unreachable node is proxied by a reachable one, across the overlay.** Which is
|
||||||
the case is a mesh-level fact, which is the fourth reason exposure is control-plane work.
|
the case is a mesh-level fact, which is the fourth reason exposure is controller work.
|
||||||
|
|
||||||
### What was built
|
### What was built
|
||||||
|
|
||||||
@@ -482,7 +482,7 @@ something:
|
|||||||
|
|
||||||
| | why not |
|
| | why not |
|
||||||
|---|---|
|
|---|---|
|
||||||
| decide what the machine **forwards** | the container runtime writes its own forwarding rules and a second policy is consulted alongside them, so a drop here stops every container on the node — the control plane included. Nothing in a manifest says what a machine routes, so there is nothing to derive it from either |
|
| decide what the machine **forwards** | the container runtime writes its own forwarding rules and a second policy is consulted alongside them, so a drop here stops every container on the node — the controller included. Nothing in a manifest says what a machine routes, so there is nothing to derive it from either |
|
||||||
| **flush the ruleset** when loading | that empties every table on the machine, the runtime's among them. Only the mesh's own table is replaced, and it is declared empty first so the replacement works on a machine loading one for the first time |
|
| **flush the ruleset** when loading | that empties every table on the machine, the runtime's among them. Only the mesh's own table is replaced, and it is declared empty first so the replacement works on a machine loading one for the first time |
|
||||||
| carry a **command to load itself** | the link may not carry an action ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)). A service is declared to reflect the file instead, so replacing it restarts what loads it — the shape that rule leaves, used here for the first time for its real purpose |
|
| carry a **command to load itself** | the link may not carry an action ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)). A service is declared to reflect the file instead, so replacing it restarts what loads it — the shape that rule leaves, used here for the first time for its real purpose |
|
||||||
|
|
||||||
@@ -549,7 +549,7 @@ defaults to the public authority's *production* endpoint. Two consequences, and
|
|||||||
worse than the lab problem that found it — every certificate experiment on a real node consumes
|
worse than the lab problem that found it — every certificate experiment on a real node consumes
|
||||||
production issuance quota, and a retry loop can exhaust it for a week.
|
production issuance quota, and a retry loop can exhaust it for a week.
|
||||||
|
|
||||||
**The mesh CA is not a bootstrap concern.** A joining node verifies the control plane against the
|
**The mesh CA is not a bootstrap concern.** A joining node verifies the controller against the
|
||||||
fingerprint in its token ([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)),
|
fingerprint in its token ([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)),
|
||||||
so nothing needs the CA before membership. It certifies internal names afterwards, and that is
|
so nothing needs the CA before membership. It certifies internal names afterwards, and that is
|
||||||
all it does.
|
all it does.
|
||||||
|
|||||||
@@ -142,12 +142,12 @@ nox-mesh-host enrol --token <one-time token>
|
|||||||
|
|
||||||
The token carries **four** things and is carried by a person
|
The token carries **four** things and is carried by a person
|
||||||
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)): the broker's
|
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)): the broker's
|
||||||
address, the fingerprint to expect, **the control plane's signing identity**, and the right to
|
address, the fingerprint to expect, **the controller's signing identity**, and the right to
|
||||||
join once.
|
join once.
|
||||||
|
|
||||||
**The fourth is the one this document listed three of.** A node connects to the broker and takes
|
**The fourth is the one this document listed three of.** A node connects to the broker and takes
|
||||||
instruction from the control plane behind it, and those are two different identities. Pinning only
|
instruction from the controller behind it, and those are two different identities. Pinning only
|
||||||
the broker would make the control plane's authority *transitive* — a compromised broker could then
|
the broker would make the controller's authority *transitive* — a compromised broker could then
|
||||||
forge declarations, which, since the host applies whatever the link delivers, is the whole machine.
|
forge declarations, which, since the host applies whatever the link delivers, is the whole machine.
|
||||||
So the transport is verified once at connect, and **each declaration is verified by its signature,
|
So the transport is verified once at connect, and **each declaration is verified by its signature,
|
||||||
every time**.
|
every time**.
|
||||||
@@ -158,10 +158,10 @@ What happens, in order:
|
|||||||
2. it checks the broker's certificate against the pinned fingerprint — *before* sending anything;
|
2. it checks the broker's certificate against the pinned fingerprint — *before* sending anything;
|
||||||
3. it presents the one-time secret **and its own public key**, which the mesh records;
|
3. it presents the one-time secret **and its own public key**, which the mesh records;
|
||||||
4. it reports its `profile` and `inventory` upward;
|
4. it reports its `profile` and `inventory` upward;
|
||||||
5. the control plane decides what this machine should be, and sends a declaration;
|
5. the controller decides what this machine should be, and sends a declaration;
|
||||||
6. the host applies it, reads back, and reports.
|
6. the host applies it, reads back, and reports.
|
||||||
|
|
||||||
**Step 4 is the one that is easy to miss and is what makes step 5 possible.** The control plane
|
**Step 4 is the one that is easy to miss and is what makes step 5 possible.** The controller
|
||||||
cannot decide what a machine should run without knowing what it *can* run — a graphical session,
|
cannot decide what a machine should run without knowing what it *can* run — a graphical session,
|
||||||
a container runtime, an architecture. The profile is not a diagnostic; it is the input.
|
a container runtime, an architecture. The profile is not a diagnostic; it is the input.
|
||||||
|
|
||||||
@@ -209,7 +209,7 @@ channel, not about network reachability.** What it forbids is a listening thing
|
|||||||
instructions and changes the machine. A node being reachable on the overlay — the whole purpose
|
instructions and changes the machine. A node being reachable on the overlay — the whole purpose
|
||||||
of the overlay — is untouched by it, and so is a person opening a shell on it.
|
of the overlay — is untouched by it, and so is a person opening a shell on it.
|
||||||
|
|
||||||
The distinction is *who can tell this machine what to be*: only the control plane, only over the
|
The distinction is *who can tell this machine what to be*: only the controller, only over the
|
||||||
link the node opened, only in declarations of known shape.
|
link the node opened, only in declarations of known shape.
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -219,18 +219,18 @@ link the node opened, only in declarations of known shape.
|
|||||||
The same path, with the mesh built in the middle of it.
|
The same path, with the mesh built in the middle of it.
|
||||||
|
|
||||||
```
|
```
|
||||||
# 1 — raise the substrate and the control plane from the carried bundle
|
# 1 — raise the foundation and the controller from the carried bundle
|
||||||
nox-mesh-host reconcile
|
nox-mesh-host reconcile
|
||||||
|
|
||||||
# 2 — the control plane now exists, and issues the first token
|
# 2 — the controller now exists, and issues the first token
|
||||||
mesh-control token issue
|
mesh-control token issue
|
||||||
|
|
||||||
# 3 — the machine joins the mesh it just raised
|
# 3 — the machine joins the mesh it just raised
|
||||||
nox-mesh-host enrol --token <token>
|
nox-mesh-host enrol --token <token>
|
||||||
```
|
```
|
||||||
|
|
||||||
Step 1 is the bootstrap from [`07-the-substrate.md`](07-the-substrate.md): a container runtime,
|
Step 1 is the bootstrap from [`07-the-foundation.md`](07-the-foundation.md): a container runtime,
|
||||||
then PostgreSQL, then the database, then the schema, then the control plane. It needs no identity
|
then PostgreSQL, then the database, then the schema, then the controller. It needs no identity
|
||||||
because nothing is being asked of anyone — the host is applying a declaration it already
|
because nothing is being asked of anyone — the host is applying a declaration it already
|
||||||
carries, to the machine it is already on.
|
carries, to the machine it is already on.
|
||||||
|
|
||||||
@@ -238,7 +238,7 @@ carries, to the machine it is already on.
|
|||||||
the bootstrap script never had. Its specialness lasted two commands.
|
the bootstrap script never had. Its specialness lasted two commands.
|
||||||
|
|
||||||
**And enrolment is exercised on node one.** The path every other node depends on is walked
|
**And enrolment is exercised on node one.** The path every other node depends on is walked
|
||||||
immediately, against a control plane on the same machine, rather than being written and first
|
immediately, against a controller on the same machine, rather than being written and first
|
||||||
used months later on node two.
|
used months later on node two.
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -265,7 +265,7 @@ closes — an authoritative local store, reconcile on start, *last heard from* r
|
|||||||
alarm — is what an episodic host needs, at a shorter period.
|
alarm — is what an episodic host needs, at a shorter period.
|
||||||
|
|
||||||
**It cannot be the first node**, and that is not a limitation to work around. Every step of
|
**It cannot be the first node**, and that is not a limitation to work around. Every step of
|
||||||
raising a substrate is a `package`, a `container` or an `action` against one, and a partial host
|
raising a foundation is a `package`, a `container` or an `action` against one, and a partial host
|
||||||
refuses the first two. So `mesh-host-android bundle` returns a file that says so rather than an
|
refuses the first two. So `mesh-host-android bundle` returns a file that says so rather than an
|
||||||
empty placeholder waiting to be filled in.
|
empty placeholder waiting to be filled in.
|
||||||
|
|
||||||
@@ -350,7 +350,7 @@ rest. The rule that exists to stop the host lying about what it did also makes i
|
|||||||
|
|
||||||
## Updating what the node holds
|
## Updating what the node holds
|
||||||
|
|
||||||
An ordinary declaration. Someone assigns a module; the control plane recomputes what that node
|
An ordinary declaration. Someone assigns a module; the controller recomputes what that node
|
||||||
should be and sends it; the host applies the difference and removes what is no longer declared.
|
should be and sends it; the host applies the difference and removes what is no longer declared.
|
||||||
|
|
||||||
**Removal is not symmetric, and the asymmetry is the design:**
|
**Removal is not symmetric, and the asymmetry is the design:**
|
||||||
@@ -410,7 +410,7 @@ one binary that has always been the same binary.
|
|||||||
|
|
||||||
Two cases, and they are genuinely different.
|
Two cases, and they are genuinely different.
|
||||||
|
|
||||||
**Graceful.** The control plane sends a final declaration that names nothing. The host removes
|
**Graceful.** The controller sends a final declaration that names nothing. The host removes
|
||||||
what it owns by the table above, reports, and drops its identity. The machine keeps the host
|
what it owns by the table above, reports, and drops its identity. The machine keeps the host
|
||||||
installed and is back to `hosted`. Nothing is left behind that anybody has to remember.
|
installed and is back to `hosted`. Nothing is left behind that anybody has to remember.
|
||||||
|
|
||||||
@@ -638,7 +638,7 @@ lets a node verify a mesh it has never spoken to
|
|||||||
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)). A token emailed,
|
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)). A token emailed,
|
||||||
committed, or dropped in shared storage has lost the only property that makes it worth carrying.
|
committed, or dropped in shared storage has lost the only property that makes it worth carrying.
|
||||||
|
|
||||||
**On the first node it comes from the control plane that was raised two commands ago**, which is
|
**On the first node it comes from the controller that was raised two commands ago**, which is
|
||||||
the same command against a mesh that is one machine old.
|
the same command against a mesh that is one machine old.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -78,7 +78,7 @@ whenever anybody writes something reusable, which is constantly.
|
|||||||
|
|
||||||
## Delivery is a comparison, not a pipeline
|
## Delivery is a comparison, not a pipeline
|
||||||
|
|
||||||
The control plane holds two facts and builds the difference:
|
The controller holds two facts and builds the difference:
|
||||||
|
|
||||||
```
|
```
|
||||||
what source exists ─┐
|
what source exists ─┐
|
||||||
@@ -94,7 +94,7 @@ That is the same shape the host uses on a machine, one layer up:
|
|||||||
|
|
||||||
| | reconciles | against |
|
| | reconciles | against |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| the control plane | artifacts | source |
|
| the controller | artifacts | source |
|
||||||
| the host | machine state | declarations |
|
| the host | machine state | declarations |
|
||||||
|
|
||||||
**There is no pipeline as a state machine.** No stage list something can be omitted from, and no
|
**There is no pipeline as a state machine.** No stage list something can be omitted from, and no
|
||||||
@@ -178,9 +178,9 @@ Not aspirations — things without which the above does not work:
|
|||||||
same digest, a cascade would stop at the first module whose output did not move. Without them,
|
same digest, a cascade would stop at the first module whose output did not move. Without them,
|
||||||
one core-library commit redeploys the fleet with no behavioural change.
|
one core-library commit redeploys the fleet with no behavioural change.
|
||||||
- **How a module publishes its own types**, which differs per language.
|
- **How a module publishes its own types**, which differs per language.
|
||||||
- **How the control plane upgrades itself.** It declares its own new version and the host applies
|
- **How the controller upgrades itself.** It declares its own new version and the host applies
|
||||||
it — but if the new one is broken, the thing that would fix it is the thing that is broken. The
|
it — but if the new one is broken, the thing that would fix it is the thing that is broken. The
|
||||||
host has a launcher for exactly this; the control plane has nothing.
|
host has a launcher for exactly this; the controller has nothing.
|
||||||
|
|
||||||
## What "behind" means, and what it used to mean
|
## What "behind" means, and what it used to mean
|
||||||
|
|
||||||
|
|||||||
@@ -41,9 +41,9 @@ it is enough to freeze it.** A board that reads the provisioning tables directly
|
|||||||
breaks when provisioning changes its tables, and the change then gets weighed against the board.
|
breaks when provisioning changes its tables, and the change then gets weighed against the board.
|
||||||
|
|
||||||
**So a board reads through interfaces and holds nothing.** Everything on the mesh page above is
|
**So a board reads through interfaces and holds nothing.** Everything on the mesh page above is
|
||||||
already answerable by asking the control plane — what nodes exist, what each resolves to, what it
|
already answerable by asking the controller — what nodes exist, what each resolves to, what it
|
||||||
takes from elsewhere, which module came from which commit. A board that asks those questions is a
|
takes from elsewhere, which module came from which commit. A board that asks those questions is a
|
||||||
client. A board that queries `inventory` is a second control plane with a worse contract.
|
client. A board that queries `inventory` is a second controller with a worse contract.
|
||||||
|
|
||||||
**It stores nothing of its own.** No cache that can disagree, no table of "what the mesh looked
|
**It stores nothing of its own.** No cache that can disagree, no table of "what the mesh looked
|
||||||
like last time". If a question is slow to answer, the answer belongs in the context that owns it,
|
like last time". If a question is slow to answer, the answer belongs in the context that owns it,
|
||||||
@@ -63,7 +63,7 @@ calls the security boundary, and a login there would guard a room whose door is
|
|||||||
building. This one faces everybody.
|
building. This one faces everybody.
|
||||||
|
|
||||||
**Which makes the identity provider the mesh's outermost gate.** The board is a presentation layer
|
**Which makes the identity provider the mesh's outermost gate.** The board is a presentation layer
|
||||||
over the control plane and the control plane's networked surfaces can change the mesh
|
over the controller and the controller's networked surfaces can change the mesh
|
||||||
([ADR 0035](../../02-DECISIONS/0035-one-implementation-several-surfaces.md)), so **whoever that
|
([ADR 0035](../../02-DECISIONS/0035-one-implementation-several-surfaces.md)), so **whoever that
|
||||||
provider admits can assign modules, from anywhere.** Said flatly because it is easy to arrive at
|
provider admits can assign modules, from anywhere.** Said flatly because it is easy to arrive at
|
||||||
one reasonable step at a time and then be surprised by.
|
one reasonable step at a time and then be surprised by.
|
||||||
|
|||||||
@@ -85,10 +85,10 @@ the worst possible moment.
|
|||||||
|
|
||||||
## The builder runs on a node
|
## The builder runs on a node
|
||||||
|
|
||||||
**Not in the control plane, and this is the same boundary as everywhere else.** Building needs a
|
**Not in the controller, and this is the same boundary as everywhere else.** Building needs a
|
||||||
container runtime and a working tree; what the control plane may send a machine is bounded by the
|
container runtime and a working tree; what the controller may send a machine is bounded by the
|
||||||
declaration language ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)), and *run this build*
|
declaration language ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)), and *run this build*
|
||||||
is not in it. The alternative — the control plane holding a container socket — would make it the
|
is not in it. The alternative — the controller holding a container socket — would make it the
|
||||||
one component that can do anything on any machine, which is the property the whole design is
|
one component that can do anything on any machine, which is the property the whole design is
|
||||||
arranged to avoid.
|
arranged to avoid.
|
||||||
|
|
||||||
@@ -96,7 +96,7 @@ So the builder is a program a machine runs, given work over the broker like anyt
|
|||||||
its own credential and nothing more.
|
its own credential and nothing more.
|
||||||
|
|
||||||
**A build is work, not state**, and that is why it does not travel as a declaration. Everything
|
**A build is work, not state**, and that is why it does not travel as a declaration. Everything
|
||||||
else the control plane sends a node is *what you should be*, reconciled forever. A build happens
|
else the controller sends a node is *what you should be*, reconciled forever. A build happens
|
||||||
once and is finished; as a declaration it would either rebuild on every reconcile or carry "and I
|
once and is finished; as a declaration it would either rebuild on every reconcile or carry "and I
|
||||||
already did this" — state about an event rather than about a machine.
|
already did this" — state about an event rather than about a machine.
|
||||||
|
|
||||||
@@ -203,7 +203,7 @@ at something. A build that failed before it knew what it was building keeps the
|
|||||||
is what a person goes and looks at.
|
is what a person goes and looks at.
|
||||||
|
|
||||||
Recording is idempotent on the correlation, because a result arrives twice — once as the answer to
|
Recording is idempotent on the correlation, because a result arrives twice — once as the answer to
|
||||||
whoever asked and once on the exchange, where the control plane is also listening. Two rows would
|
whoever asked and once on the exchange, where the controller is also listening. Two rows would
|
||||||
show one build as two, and which is real is not answerable afterwards.
|
show one build as two, and which is real is not answerable afterwards.
|
||||||
|
|
||||||
That is what a builds view reads, and until it existed there was nothing to read: a result was
|
That is what a builds view reads, and until it existed there was nothing to read: a result was
|
||||||
@@ -225,9 +225,16 @@ answered to the asker and kept nowhere.
|
|||||||
| kind | is |
|
| kind | is |
|
||||||
|---|---|
|
|---|---|
|
||||||
| **image** | built from a Dockerfile in this repository |
|
| **image** | built from a Dockerfile in this repository |
|
||||||
|
| **bundle** | this module's own code, compiled by the toolchain its language implies, then packed |
|
||||||
| **archive** | a directory in this repository, packed |
|
| **archive** | a directory in this repository, packed |
|
||||||
| **upstream** | an image somebody else built, mirrored into the mesh's own registry |
|
| **upstream** | an image somebody else built, mirrored into the mesh's own registry |
|
||||||
|
|
||||||
|
**The second was added later and is why most modules now need no Dockerfile.** An archive packs a
|
||||||
|
directory as it stands, so shipping compiled output meant compiling somewhere first — and that
|
||||||
|
meant every module repeating a recipe that is easy to get wrong in ways that fail elsewhere. The
|
||||||
|
whole surface a module has, and a module that exercises all of it, are in
|
||||||
|
[`18-building-a-module`](18-building-a-module.md).
|
||||||
|
|
||||||
**The third exists because a module usually runs software it did not write.** A database module
|
**The third exists because a module usually runs software it did not write.** A database module
|
||||||
ships configuration and a provisioner and does not build a database. Naming the upstream reference
|
ships configuration and a provisioner and does not build a database. Naming the upstream reference
|
||||||
directly would need every machine to reach a public registry, and would pin to a tag its owner can
|
directly would need every machine to reach a public registry, and would pin to a tag its owner can
|
||||||
@@ -374,13 +381,13 @@ have a route and one does not:
|
|||||||
|
|
||||||
| What | Why it cannot come through the loop | How it arrives |
|
| What | Why it cannot come through the loop | How it arrives |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| The control plane | It is what installs modules. Nothing can install it before it runs. | **Carried inside the installer** and published once there is a registry ([ADR 0067](../../02-DECISIONS/0067-genesis-is-a-pivot.md)) |
|
| The controller | It is what installs modules. Nothing can install it before it runs. | **Carried inside the installer** and published once there is a registry ([ADR 0067](../../02-DECISIONS/0067-genesis-is-a-pivot.md)) |
|
||||||
| The registry | It *is* where artifacts are delivered from. A store cannot be delivered through itself. | **Pulled from the public internet** — its image is an ordinary public one, never built ([`04-ISSUES/029`](../../04-ISSUES/029-the-artifact-store-cannot-be-delivered-by-the-artifact-store/00-report.md)) |
|
| The registry | It *is* where artifacts are delivered from. A store cannot be delivered through itself. | **Pulled from the public internet** — its image is an ordinary public one, never built ([`04-ISSUES/029`](../../04-ISSUES/029-the-artifact-store-cannot-be-delivered-by-the-artifact-store/00-report.md)) |
|
||||||
| The builder | It is what builds. Nothing builds it before it runs. | Built at genesis by the init builder ([ADR 0070](../../02-DECISIONS/0070-the-catalogue-owns-the-module-graph.md)) |
|
| The builder | It is what builds. Nothing builds it before it runs. | Built at genesis by the init builder ([ADR 0070](../../02-DECISIONS/0070-the-catalogue-owns-the-module-graph.md)) |
|
||||||
| The catalogue | It owns the module graph, and nothing can be resolved or installed without it. | Built at genesis by the init builder ([ADR 0070](../../02-DECISIONS/0070-the-catalogue-owns-the-module-graph.md)) |
|
| The catalogue | It owns the module graph, and nothing can be resolved or installed without it. | Built at genesis by the init builder ([ADR 0070](../../02-DECISIONS/0070-the-catalogue-owns-the-module-graph.md)) |
|
||||||
|
|
||||||
**How they arrive is settled and not yet built.** The installer carries an init builder, which
|
**How they arrive is settled and not yet built.** The installer carries an init builder, which
|
||||||
clones the source and builds the control plane, the catalogue and the builder before a mesh exists
|
clones the source and builds the controller, the catalogue and the builder before a mesh exists
|
||||||
to install anything. Two things about that are open and named in
|
to install anything. Two things about that are open and named in
|
||||||
[ADR 0070](../../02-DECISIONS/0070-the-catalogue-owns-the-module-graph.md): where the init builder
|
[ADR 0070](../../02-DECISIONS/0070-the-catalogue-owns-the-module-graph.md): where the init builder
|
||||||
clones from, given the forge normally runs on the mesh it would be rebuilding, and what it
|
clones from, given the forge normally runs on the mesh it would be rebuilding, and what it
|
||||||
@@ -393,7 +400,7 @@ three. This section previously said the list was closed at three, which was writ
|
|||||||
catalogue had an owner and is corrected here rather than left to be reasoned from.
|
catalogue had an owner and is corrected here rather than left to be reasoned from.
|
||||||
|
|
||||||
**And the answer for all four is now one mechanism, not four special cases.** Genesis carries an
|
**And the answer for all four is now one mechanism, not four special cases.** Genesis carries an
|
||||||
*init builder* and builds the core modules on the machine — control plane, catalogue and builder —
|
*init builder* and builds the core modules on the machine — controller, catalogue and builder —
|
||||||
rather than carrying a finished image of any of them. So the question is no longer "how does this
|
rather than carrying a finished image of any of them. So the question is no longer "how does this
|
||||||
one get here first?" asked once per component; it is answered once, by the thing that is carried
|
one get here first?" asked once per component; it is answered once, by the thing that is carried
|
||||||
being a builder rather than a result.
|
being a builder rather than a result.
|
||||||
@@ -401,12 +408,21 @@ being a builder rather than a result.
|
|||||||
Everything outside those four is either upstream — a third-party image pulled by digest — or built
|
Everything outside those four is either upstream — a third-party image pulled by digest — or built
|
||||||
by the builder from a repository and a path, and published to the registry.
|
by the builder from a repository and a path, and published to the registry.
|
||||||
|
|
||||||
|
**One of those built things has an ordering constraint worth naming, because it looks like a fifth
|
||||||
|
member of the list and is not.** The SDK the toolchain compiles against is built by the ordinary
|
||||||
|
builder and published like anything else — but it cannot be compiled *in the mesh toolchain*, since
|
||||||
|
that toolchain is built from it, and it is published to the *package* registry rather than the
|
||||||
|
artifact store. So it is compiled on a public base image and published before the toolchain that
|
||||||
|
consumes it ([ADR 0076](../../02-DECISIONS/0076-the-sdk-is-a-published-package.md)). It is not
|
||||||
|
carried and it is not machinery; it is a dependency with a sequence, which is why it belongs here as
|
||||||
|
a footnote to the rule rather than a row in the table.
|
||||||
|
|
||||||
**A carried artifact is not a differently-pinned artifact.** Once published it is named by a digest
|
**A carried artifact is not a differently-pinned artifact.** Once published it is named by a digest
|
||||||
the mesh's registry assigned, exactly like everything the builder produces. A reader cannot tell
|
the mesh's registry assigned, exactly like everything the builder produces. A reader cannot tell
|
||||||
from a running mesh which of its images were carried, and that is the point: carrying is how the
|
from a running mesh which of its images were carried, and that is the point: carrying is how the
|
||||||
first copy arrives, not what it permanently is.
|
first copy arrives, not what it permanently is.
|
||||||
|
|
||||||
*Checked by the thing already checked at genesis: after installing, the running control plane is
|
*Checked by the thing already checked at genesis: after installing, the running controller is
|
||||||
pinned to a digest the mesh's own registry assigned, and not to the id of the image the installer
|
pinned to a digest the mesh's own registry assigned, and not to the id of the image the installer
|
||||||
carried. The same check applies to the builder and to the registry, and it is the same check —
|
carried. The same check applies to the builder and to the registry, and it is the same check —
|
||||||
an image id where a registry digest belongs means the pivot did not finish.*
|
an image id where a registry digest belongs means the pivot did not finish.*
|
||||||
|
|||||||
@@ -70,7 +70,7 @@ The mesh generated the password, sealed it to the machine that must accept it, a
|
|||||||
plaintext** — so it cannot tell a database to start accepting it. Something on that machine reads
|
plaintext** — so it cannot tell a database to start accepting it. Something on that machine reads
|
||||||
what the host wrote and makes it true.
|
what the host wrote and makes it true.
|
||||||
|
|
||||||
That something is part of the module, not part of the control plane. **The control plane decides
|
That something is part of the module, not part of the controller. **The controller decides
|
||||||
and never touches a machine; a provisioner runs on the machine and touches it.** Two files, because
|
and never touches a machine; a provisioner runs on the machine and touches it.** Two files, because
|
||||||
the mesh could not compose a document containing a value it does not have:
|
the mesh could not compose a document containing a value it does not have:
|
||||||
|
|
||||||
|
|||||||
@@ -59,7 +59,7 @@ names neither the licence nor the mesh.
|
|||||||
|
|
||||||
**A key is read from a file or standard input, never an argument.** A key on a command line is a
|
**A key is read from a file or standard input, never an argument.** A key on a command line is a
|
||||||
key in shell history and in every process listing taken while it ran. It is never echoed back:
|
key in shell history and in every process listing taken while it ran. It is never echoed back:
|
||||||
what is stored is unreadable by whoever holds it, the control plane included, and printing it
|
what is stored is unreadable by whoever holds it, the controller included, and printing it
|
||||||
would put the one copy that matters on a terminal.
|
would put the one copy that matters on a terminal.
|
||||||
|
|
||||||
## Refusing is felt, and that is the design working
|
## Refusing is felt, and that is the design working
|
||||||
@@ -83,7 +83,7 @@ per machine, which is a step toward it and is not it.
|
|||||||
|
|
||||||
*2026-08-31: this gap now has named consumers rather than hypothetical ones.*
|
*2026-08-31: this gap now has named consumers rather than hypothetical ones.*
|
||||||
[ADR 0026](../../02-DECISIONS/0026-the-mesh-has-a-session-of-its-own.md) puts two sessions on the
|
[ADR 0026](../../02-DECISIONS/0026-the-mesh-has-a-session-of-its-own.md) puts two sessions on the
|
||||||
control-plane node — the node's own and the mesh's — each bound in its own right. See
|
controller node — the node's own and the mesh's — each bound in its own right. See
|
||||||
[`15-the-agent-session.md`](15-the-agent-session.md).
|
[`15-the-agent-session.md`](15-the-agent-session.md).
|
||||||
|
|
||||||
**And for sessions the gap is already closed, which was not obvious.** A binding is per module per
|
**And for sessions the gap is already closed, which was not obvious.** A binding is per module per
|
||||||
@@ -172,12 +172,12 @@ it; the metric is vendor-defined, so no false common unit is forced. Anthropic b
|
|||||||
In the lab, on real machines, in the order a person would meet it: a consumer is refused with both
|
In the lab, on real machines, in the order a person would meet it: a consumer is refused with both
|
||||||
candidates named; put on one and still refused because no key exists; the key is given on standard
|
candidates named; put on one and still refused because no key exists; the key is given on standard
|
||||||
input and not echoed; the public half arrives saying it came from a record rather than a machine;
|
input and not echoed; the public half arrives saying it came from a record rather than a machine;
|
||||||
the key arrives readable only by that machine — and it is **nowhere in the control plane's own
|
the key arrives readable only by that machine — and it is **nowhere in the controller's own
|
||||||
database**, nor in anything that crossed the broker.
|
database**, nor in anything that crossed the broker.
|
||||||
|
|
||||||
For the generalisation ([ADR 0050](../../02-DECISIONS/0050-model-access-is-vendor-agnostic.md)): a
|
For the generalisation ([ADR 0050](../../02-DECISIONS/0050-model-access-is-vendor-agnostic.md)): a
|
||||||
second vendor — `anthropic-api-key`, static-key — is bound to a consumer and exercises the whole path
|
second vendor — `anthropic-api-key`, static-key — is bound to a consumer and exercises the whole path
|
||||||
with the carve-out switched off, its key sealed per node and absent from the control plane's database.
|
with the carve-out switched off, its key sealed per node and absent from the controller's database.
|
||||||
For a `refreshable-grant` licence, the refresh token is asserted to exist (encrypted) **only on the
|
For a `refreshable-grant` licence, the refresh token is asserted to exist (encrypted) **only on the
|
||||||
manager node**, to be **absent from every holder's delivery**, and the delivered credential to be
|
manager node**, to be **absent from every holder's delivery**, and the delivered credential to be
|
||||||
access-token-only; a `static-key` licence stores no refresh token anywhere. A scenario with two
|
access-token-only; a `static-key` licence stores no refresh token anywhere. A scenario with two
|
||||||
|
|||||||
@@ -22,7 +22,7 @@ so, and the differences are few enough to list here:
|
|||||||
| **context root** | the node's | the mesh's |
|
| **context root** | the node's | the mesh's |
|
||||||
| **engram** | that node's | the mesh's |
|
| **engram** | that node's | the mesh's |
|
||||||
| **licence** | bound in its own right | bound in its own right |
|
| **licence** | bound in its own right | bound in its own right |
|
||||||
| **runs on** | that node | the node holding the control plane |
|
| **runs on** | that node | the node holding the controller |
|
||||||
| **how many** | one per node | one |
|
| **how many** | one per node | one |
|
||||||
|
|
||||||
Everything below applies to both unless it says otherwise.
|
Everything below applies to both unless it says otherwise.
|
||||||
@@ -64,9 +64,9 @@ as it reports anything else.
|
|||||||
**A node's session runs on that node**, and cannot be moved. Moved, one machine is answering as
|
**A node's session runs on that node**, and cannot be moved. Moved, one machine is answering as
|
||||||
another (ADR 0004).
|
another (ADR 0004).
|
||||||
|
|
||||||
**The mesh's session runs on the node holding the control plane.** The reasoning is in ADR 0026
|
**The mesh's session runs on the node holding the controller.** The reasoning is in ADR 0026
|
||||||
and is worth carrying here because it is easy to get backwards: this is not *the important agent
|
and is worth carrying here because it is easy to get backwards: this is not *the important agent
|
||||||
goes on the important machine*. It is that the control-plane node is already the one place
|
goes on the important machine*. It is that the controller node is already the one place
|
||||||
excepted from *compromise of a node is compromise of that node*, and an agent able to reach
|
excepted from *compromise of a node is compromise of that node*, and an agent able to reach
|
||||||
everything, placed anywhere else, would create a second such place.
|
everything, placed anywhere else, would create a second such place.
|
||||||
|
|
||||||
@@ -102,7 +102,7 @@ machine*.
|
|||||||
|
|
||||||
That is not sufficient here, and the shortfall is concrete rather than theoretical:
|
That is not sufficient here, and the shortfall is concrete rather than theoretical:
|
||||||
|
|
||||||
- the control-plane node hosts **two** sessions, which must be able to hold **different**
|
- the controller node hosts **two** sessions, which must be able to hold **different**
|
||||||
licences — a per-machine binding cannot express it at all;
|
licences — a per-machine binding cannot express it at all;
|
||||||
- *this node's session uses the personal licence, the mesh's uses the company one* is the ordinary
|
- *this node's session uses the personal licence, the mesh's uses the company one* is the ordinary
|
||||||
case, not an exotic one.
|
case, not an exotic one.
|
||||||
@@ -125,7 +125,7 @@ session a different agent from another, and memory is part of what makes it *tha
|
|||||||
and it is not a view over theirs. What the mesh has been asked, and what it worked out, is held
|
and it is not a view over theirs. What the mesh has been asked, and what it worked out, is held
|
||||||
in the mesh's root — not in the root of the node that happens to host it.
|
in the mesh's root — not in the root of the node that happens to host it.
|
||||||
|
|
||||||
**That distinction is the point of putting it there.** The control-plane node runs two sessions
|
**That distinction is the point of putting it there.** The controller node runs two sessions
|
||||||
on one machine. If memory belonged to the machine rather than to the root, they would share it,
|
on one machine. If memory belonged to the machine rather than to the root, they would share it,
|
||||||
and the mesh's recollection of a fortnight of questions would be indistinguishable from that
|
and the mesh's recollection of a fortnight of questions would be indistinguishable from that
|
||||||
node's own — which is the collision ADR 0026 exists to avoid, arriving through the back door.
|
node's own — which is the collision ADR 0026 exists to avoid, arriving through the back door.
|
||||||
@@ -188,7 +188,7 @@ here so the shape is not rediscovered.
|
|||||||
|
|
||||||
## Consequences
|
## Consequences
|
||||||
|
|
||||||
**Two sessions on one node, and no ambiguity.** The control-plane node hosts its own node session
|
**Two sessions on one node, and no ambiguity.** The controller node hosts its own node session
|
||||||
and the mesh session. ADR 0004's *one per node* forbids ambiguity about who answers when a **node**
|
and the mesh session. ADR 0004's *one per node* forbids ambiguity about who answers when a **node**
|
||||||
is addressed; these answer to different addresses.
|
is addressed; these answer to different addresses.
|
||||||
|
|
||||||
@@ -207,7 +207,7 @@ and these run in the lab on real machines:
|
|||||||
| Check | Defends |
|
| Check | Defends |
|
||||||
|---|---|
|
|---|---|
|
||||||
| a node is asked something and its session answers | ADR 0004 |
|
| a node is asked something and its session answers | ADR 0004 |
|
||||||
| the mesh is asked something and the mesh session answers, on the control-plane node | ADR 0026 |
|
| the mesh is asked something and the mesh session answers, on the controller node | ADR 0026 |
|
||||||
| both sessions on that node answer, to their own addresses, without ambiguity | ADR 0026 |
|
| both sessions on that node answer, to their own addresses, without ambiguity | ADR 0026 |
|
||||||
| a session switched off replies saying so, rather than timing out | ADR 0004 |
|
| a session switched off replies saying so, rather than timing out | ADR 0004 |
|
||||||
| a session whose engram was changed reports having applied it, like any declared file | ADR 0005 |
|
| a session whose engram was changed reports having applied it, like any declared file | ADR 0005 |
|
||||||
|
|||||||
@@ -24,7 +24,7 @@ checklist: what is already sayable, what is deliberately not, and what is missin
|
|||||||
| **depends on another module** | 65 | `requires` naming a module. A requirement naming a module means *that module*, not anything providing the name |
|
| **depends on another module** | 65 | `requires` naming a module. A requirement naming a module means *that module*, not anything providing the name |
|
||||||
| **system packages** | 28 | the `package` shape |
|
| **system packages** | 28 | the `package` shape |
|
||||||
| **a container** | 48 | the `container` shape, pinned by digest |
|
| **a container** | 48 | the `container` shape, pinned by digest |
|
||||||
| **systemd units** | 17 | a `file` for the unit, a `service` for the state it should be in |
|
| **systemd units** | 17 | a `process`, which the mesh writes the unit for. *Was: a `file` for the unit and a `service` for its state — which made every module author write unit syntax, and is why `process` exists ([`18`](18-building-a-module.md))* |
|
||||||
| **how to reach it** | 17 | `serves`, with the mesh adding which machine and where |
|
| **how to reach it** | 17 | `serves`, with the mesh adding which machine and where |
|
||||||
| **a public name** | 11 | requiring `route` and contributing the name |
|
| **a public name** | 11 | requiring `route` and contributing the name |
|
||||||
| **ports it opens** | 11 | `listens`, from which filtering is computed |
|
| **ports it opens** | 11 | `listens`, from which filtering is computed |
|
||||||
@@ -33,7 +33,7 @@ checklist: what is already sayable, what is deliberately not, and what is missin
|
|||||||
| **restart when something changes** | 11 | `restart-on` |
|
| **restart when something changes** | 11 | `restart-on` |
|
||||||
| **a generated credential** | 20 | `own-secrets`, sealed to the machine |
|
| **a generated credential** | 20 | `own-secrets`, sealed to the machine |
|
||||||
| **a value that differs per node** | 43 | settings, and `computed` for what only the mesh knows |
|
| **a value that differs per node** | 43 | settings, and `computed` for what only the mesh knows |
|
||||||
| **images built from source** | 2 | `build.artifacts` |
|
| **images built from source** | 2 | `build.artifacts` — an `image` from a Dockerfile, or a `bundle`, which names a language and lets the mesh choose the toolchain ([`18`](18-building-a-module.md)) |
|
||||||
|
|
||||||
## Deliberately not sayable
|
## Deliberately not sayable
|
||||||
|
|
||||||
@@ -163,14 +163,14 @@ same module. There is one derivation here, and there should stay one.
|
|||||||
sealing is worth its inconvenience.
|
sealing is worth its inconvenience.
|
||||||
|
|
||||||
**An image store is a module, and was written up here as something the mesh does.** It was
|
**An image store is a module, and was written up here as something the mesh does.** It was
|
||||||
considered for the substrate and removed, because the test is not *can it grant itself one* —
|
considered for the foundation and removed, because the test is not *can it grant itself one* —
|
||||||
nearly anything passes that — but whether the control plane needs it before it can give its first
|
nearly anything passes that — but whether the controller needs it before it can give its first
|
||||||
instruction. It does not. So a registry somebody runs for their own images is the same module as
|
instruction. It does not. So a registry somebody runs for their own images is the same module as
|
||||||
the one the mesh runs for its own: it offers a place to push, and claims that role once per
|
the one the mesh runs for its own: it offers a place to push, and claims that role once per
|
||||||
machine.
|
machine.
|
||||||
|
|
||||||
**A rule was enforced only at the far end.** A module may not declare an action, and the host
|
**A rule was enforced only at the far end.** A module may not declare an action, and the host
|
||||||
refused one correctly — but the control plane accepted it into the catalogue, resolved it and
|
refused one correctly — but the controller accepted it into the catalogue, resolved it and
|
||||||
pushed it, so the refusal arrived on a machine with nothing tying it back to the manifest. The
|
pushed it, so the refusal arrived on a machine with nothing tying it back to the manifest. The
|
||||||
rule held; it was just unusable, which is the same shape as the network shape that cost five
|
rule held; it was just unusable, which is the same shape as the network shape that cost five
|
||||||
failing tests before anyone read the host's log. It is now refused where it is written.
|
failing tests before anyone read the host's log. It is now refused where it is written.
|
||||||
|
|||||||
@@ -30,7 +30,7 @@ rules cannot all hold at once, and it is resolved by a pivot
|
|||||||
|
|
||||||
**Joining** happens on every machine after the first, and it is ordinary. A mesh exists, so it can
|
**Joining** happens on every machine after the first, and it is ordinary. A mesh exists, so it can
|
||||||
be asked for a token and told what the machine should be. Joining installs the host and nothing
|
be asked for a token and told what the machine should be. Joining installs the host and nothing
|
||||||
else: no temporary anything, no substrate raised by hand, no registry.
|
else: no temporary anything, no foundation raised by hand, no registry.
|
||||||
|
|
||||||
Confusing the two is what produced a procedure that only ever worked in a fixture. A bed that
|
Confusing the two is what produced a procedure that only ever worked in a fixture. A bed that
|
||||||
raises four machines the same way has not tested genesis at all — it has tested joining, four
|
raises four machines the same way has not tested genesis at all — it has tested joining, four
|
||||||
@@ -38,13 +38,13 @@ times, with the first one hand-fed.
|
|||||||
|
|
||||||
## What changed, and what did not
|
## What changed, and what did not
|
||||||
|
|
||||||
*2026-09-13.* The installer carries a builder now, and builds the control plane it raises. Three
|
*2026-09-13.* The installer carries a builder now, and builds the controller it raises. Three
|
||||||
records settle it: [ADR 0070](../../02-DECISIONS/0070-the-catalogue-owns-the-module-graph.md) that
|
records settle it: [ADR 0070](../../02-DECISIONS/0070-the-catalogue-owns-the-module-graph.md) that
|
||||||
genesis builds rather than carries, [ADR 0071](../../02-DECISIONS/0071-where-genesis-gets-its-source.md)
|
genesis builds rather than carries, [ADR 0071](../../02-DECISIONS/0071-where-genesis-gets-its-source.md)
|
||||||
where it clones from, and [ADR 0073](../../02-DECISIONS/0073-the-installer-carries-a-builder.md) how
|
where it clones from, and [ADR 0073](../../02-DECISIONS/0073-the-installer-carries-a-builder.md) how
|
||||||
the builder arrives — which also records an argument that failed. It was put that a produced image
|
the builder arrives — which also records an argument that failed. It was put that a produced image
|
||||||
must be published before anything can fetch it, so the registry would have to come up before the
|
must be published before anything can fetch it, so the registry would have to come up before the
|
||||||
control plane. It does not: the machine that builds the image is the machine that runs it, and a
|
controller. It does not: the machine that builds the image is the machine that runs it, and a
|
||||||
local image is named by the digest of its own configuration exactly as a carried one is. **Building
|
local image is named by the digest of its own configuration exactly as a carried one is. **Building
|
||||||
changes where the bytes came from, not where they are.**
|
changes where the bytes came from, not where they are.**
|
||||||
|
|
||||||
@@ -53,7 +53,7 @@ written. What follows describes the program that exists.
|
|||||||
|
|
||||||
## Genesis
|
## Genesis
|
||||||
|
|
||||||
The installer is a single program carrying **the builder** inside it — not the control plane
|
The installer is a single program carrying **the builder** inside it — not the controller
|
||||||
([ADR 0073](../../02-DECISIONS/0073-the-installer-carries-a-builder.md)). What cannot be fetched is
|
([ADR 0073](../../02-DECISIONS/0073-the-installer-carries-a-builder.md)). What cannot be fetched is
|
||||||
the thing that does the fetching, so that is what is carried; everything else is made here.
|
the thing that does the fetching, so that is what is carried; everything else is made here.
|
||||||
|
|
||||||
@@ -62,12 +62,12 @@ It proceeds in one direction, and every step is safe to run again.
|
|||||||
**First it refuses to start if the machine is not ready.** A container runtime, the ability to
|
**First it refuses to start if the machine is not ready.** A container runtime, the ability to
|
||||||
write where it must write, the host binary where it expects it — and a repository and a commit to
|
write where it must write, the host binary where it expects it — and a repository and a commit to
|
||||||
build from, because an installer told nothing would raise a store and a broker and then have
|
build from, because an installer told nothing would raise a store and a broker and then have
|
||||||
nothing to raise a control plane from. A machine that is not ready is told what is missing rather
|
nothing to raise a controller from. A machine that is not ready is told what is missing rather
|
||||||
than half-changed.
|
than half-changed.
|
||||||
|
|
||||||
**Then it loads the carried builder and builds the control plane with it**, from a repository on a
|
**Then it loads the carried builder and builds the controller with it**, from a repository on a
|
||||||
mesh that already exists and a named commit ([ADR 0071](../../02-DECISIONS/0071-where-genesis-gets-its-source.md)).
|
mesh that already exists and a named commit ([ADR 0071](../../02-DECISIONS/0071-where-genesis-gets-its-source.md)).
|
||||||
This is the same repository and path every later rebuild of the control plane will use, so what
|
This is the same repository and path every later rebuild of the controller will use, so what
|
||||||
raises the mesh is the same thing that will maintain it.
|
raises the mesh is the same thing that will maintain it.
|
||||||
|
|
||||||
**Then it describes what the machine will become.** The image it just made is named by the digest
|
**Then it describes what the machine will become.** The image it just made is named by the digest
|
||||||
@@ -75,7 +75,7 @@ of its own configuration — content-addressed and unforgeable, and requiring no
|
|||||||
it. That is legal precisely where nothing could have served one, and it is why building here needs
|
it. That is legal precisely where nothing could have served one, and it is why building here needs
|
||||||
no registry: the machine that made the image is the machine that will run it.
|
no registry: the machine that made the image is the machine that will run it.
|
||||||
|
|
||||||
**Then it raises the substrate and a temporary control plane, and waits for that control plane to
|
**Then it raises the foundation and a temporary controller, and waits for that controller to
|
||||||
answer.** At this point the machine is a mesh of one node with nothing joined to it.
|
answer.** At this point the machine is a mesh of one node with nothing joined to it.
|
||||||
|
|
||||||
**Then the machine joins the mesh it is itself running.** It enrols, and an agent runs on it. Being
|
**Then the machine joins the mesh it is itself running.** It enrols, and an agent runs on it. Being
|
||||||
@@ -84,13 +84,13 @@ enrolling is itself the thing that makes a mesh hear from a machine.
|
|||||||
|
|
||||||
**Then it installs a registry**, so the mesh has somewhere to keep its own images.
|
**Then it installs a registry**, so the mesh has somewhere to keep its own images.
|
||||||
|
|
||||||
**Then it publishes the control plane's image to that registry**, which is the moment the image
|
**Then it publishes the controller's image to that registry**, which is the moment the image
|
||||||
first receives a digest assigned by something other than itself. This is the carrying step, and it
|
first receives a digest assigned by something other than itself. This is the carrying step, and it
|
||||||
is the same step for all three things the build loop cannot produce for itself — the control plane,
|
is the same step for all three things the build loop cannot produce for itself — the controller,
|
||||||
the registry, and the builder. The rule and its closed list are in
|
the registry, and the builder. The rule and its closed list are in
|
||||||
[`12-a-module-repository`](12-a-module-repository.md#the-three-the-loop-cannot-build-and-there-are-only-three).
|
[`12-a-module-repository`](12-a-module-repository.md#the-three-the-loop-cannot-build-and-there-are-only-three).
|
||||||
|
|
||||||
**Then it installs the control plane again, as an ordinary module pinned to that digest, and drops
|
**Then it installs the controller again, as an ordinary module pinned to that digest, and drops
|
||||||
the temporary one.** The pivot is complete: what raised the mesh is gone, and what runs is a module
|
the temporary one.** The pivot is complete: what raised the mesh is gone, and what runs is a module
|
||||||
like any other. From here the mesh can build and roll out its own upgrades, including to the thing
|
like any other. From here the mesh can build and roll out its own upgrades, including to the thing
|
||||||
that runs it.
|
that runs it.
|
||||||
@@ -98,7 +98,7 @@ that runs it.
|
|||||||
## After the pivot, and still part of installing
|
## After the pivot, and still part of installing
|
||||||
|
|
||||||
Genesis ends with a mesh of one that runs, and that is not the same as a mesh that works. What it
|
Genesis ends with a mesh of one that runs, and that is not the same as a mesh that works. What it
|
||||||
has is a control plane, a store, a queue and a registry. What it cannot yet do is **produce
|
has is a controller, a store, a queue and a registry. What it cannot yet do is **produce
|
||||||
anything** — and almost every module in the catalogue is waiting to be produced, because a manifest
|
anything** — and almost every module in the catalogue is waiting to be produced, because a manifest
|
||||||
names what its artifacts are and nothing has made them.
|
names what its artifacts are and nothing has made them.
|
||||||
|
|
||||||
@@ -107,9 +107,9 @@ So installing continues:
|
|||||||
**The builder arrives, and installing is what brings it.** It is a module like any other and is
|
**The builder arrives, and installing is what brings it.** It is a module like any other and is
|
||||||
assigned to a machine like any other, but it cannot be built by the thing it is — see
|
assigned to a machine like any other, but it cannot be built by the thing it is — see
|
||||||
[`12-a-module-repository`](12-a-module-repository.md#the-three-the-loop-cannot-build-and-there-are-only-three).
|
[`12-a-module-repository`](12-a-module-repository.md#the-three-the-loop-cannot-build-and-there-are-only-three).
|
||||||
So it is carried, and it is already here: it is what built the control plane. The last step of
|
So it is carried, and it is already here: it is what built the controller. The last step of
|
||||||
installing publishes it into the mesh's own registry, installs it as an ordinary module pinned to
|
installing publishes it into the mesh's own registry, installs it as an ordinary module pinned to
|
||||||
that digest, and issues it a broker account — the same two acts the control plane went through,
|
that digest, and issues it a broker account — the same two acts the controller went through,
|
||||||
plus the one thing only a builder needs. The account is issued before the machine is sent anything,
|
plus the one thing only a builder needs. The account is issued before the machine is sent anything,
|
||||||
because a builder that arrives without its credential starts, finds nothing it may read, and waits,
|
because a builder that arrives without its credential starts, finds nothing it may read, and waits,
|
||||||
which looks exactly like a builder with no work.
|
which looks exactly like a builder with no work.
|
||||||
@@ -121,12 +121,12 @@ publishes each artifact into the mesh's own registry, and hands back the module
|
|||||||
pinned and the commit recorded. The mesh records that, and from then on the module is described by
|
pinned and the commit recorded. The mesh records that, and from then on the module is described by
|
||||||
something it made rather than by a placeholder.
|
something it made rather than by a placeholder.
|
||||||
|
|
||||||
**The control plane is built like the rest.** It was carried in and published once, which got the
|
**The controller is built like the rest.** It was carried in and published once, which got the
|
||||||
mesh running; building it from its own repository and path is what makes it upgradeable. The first
|
mesh running; building it from its own repository and path is what makes it upgradeable. The first
|
||||||
time that happens is the moment the mesh stops depending on the installer for anything.
|
time that happens is the moment the mesh stops depending on the installer for anything.
|
||||||
|
|
||||||
**And then the catalogue.** Every module with source of its own is built the same way. Until this
|
**And then the catalogue.** Every module with source of its own is built the same way. Until this
|
||||||
has happened a mesh can install only what is public or carried, which is the substrate and little
|
has happened a mesh can install only what is public or carried, which is the foundation and little
|
||||||
else.
|
else.
|
||||||
|
|
||||||
Only after all of that is the ordinary loop available: change a module's source, the mesh notices
|
Only after all of that is the ordinary loop available: change a module's source, the mesh notices
|
||||||
@@ -138,7 +138,7 @@ What remains after *that* belongs to somebody else: adding machines, and decidin
|
|||||||
|
|
||||||
## Joining
|
## Joining
|
||||||
|
|
||||||
A machine joins with the host binary and a token. It does not raise a substrate, does not install a
|
A machine joins with the host binary and a token. It does not raise a foundation, does not install a
|
||||||
registry, and is never enrolled twice. The mesh already knows how to tell a machine what to be;
|
registry, and is never enrolled twice. The mesh already knows how to tell a machine what to be;
|
||||||
joining is the point at which a machine starts listening.
|
joining is the point at which a machine starts listening.
|
||||||
|
|
||||||
@@ -169,8 +169,8 @@ paragraphs above describing the catalogue being built are a thing somebody now t
|
|||||||
thing that cannot happen.
|
thing that cannot happen.
|
||||||
|
|
||||||
**A module's declaration still has to be copied onto the machine by hand.** The installer reads the
|
**A module's declaration still has to be copied onto the machine by hand.** The installer reads the
|
||||||
registry's and the control plane's manifests from a checkout somebody put there. The control plane's
|
registry's and the controller's manifests from a checkout somebody put there. The controller's
|
||||||
now lives in the control plane's own repository, which the installer clones anyway, so this is a
|
now lives in the controller's own repository, which the installer clones anyway, so this is a
|
||||||
thing that can be removed rather than a thing that must be designed.
|
thing that can be removed rather than a thing that must be designed.
|
||||||
|
|
||||||
**A machine has no account for a registry that asks for one.** The mesh grants a consumer a
|
**A machine has no account for a registry that asks for one.** The mesh grants a consumer a
|
||||||
@@ -178,6 +178,16 @@ credential for a database; it does not yet do so for the store its own images li
|
|||||||
avoids the question by carrying the image it needs, which makes this a joining problem and a
|
avoids the question by carrying the image it needs, which makes this a joining problem and a
|
||||||
pulling problem, not a genesis one.
|
pulling problem, not a genesis one.
|
||||||
|
|
||||||
|
**The SDK still comes from a git URL, and the ordering that fixes it is decided but not built.**
|
||||||
|
[ADR 0076](../../02-DECISIONS/0076-the-sdk-is-a-published-package.md) settles that the package
|
||||||
|
registry (gitea) comes up and the SDK is published into it *before* the base toolchain is built, so
|
||||||
|
the toolchain resolves the SDK by version rather than cloning it — closing
|
||||||
|
[issue 053](../../04-ISSUES/053-the-sdk-is-pinned-twice-and-the-two-disagree/00-report.md). The
|
||||||
|
builder already knows how to be handed a package-registry credential and inject it into a build; what
|
||||||
|
is not yet wired is the genesis step that raises gitea and publishes the SDK ahead of the base, and
|
||||||
|
the toolchain's own manifest still names the SDK by a git URL. Until both land, the base build clones
|
||||||
|
the SDK inside `docker build`, which is slow and names a branch head rather than a version.
|
||||||
|
|
||||||
## How these rules are checked
|
## How these rules are checked
|
||||||
|
|
||||||
| Rule | Checked by |
|
| Rule | Checked by |
|
||||||
@@ -188,6 +198,6 @@ pulling problem, not a genesis one.
|
|||||||
| An image is named exactly | A machine refuses a bundle naming an image by tag. The refusal is exercised, not assumed. |
|
| An image is named exactly | A machine refuses a bundle naming an image by tag. The refusal is exercised, not assumed. |
|
||||||
| The installer is what installed this | **Nothing.** See above. |
|
| The installer is what installed this | **Nothing.** See above. |
|
||||||
| The builder can arrive on a fresh mesh | The installer carries it and installs it as its last step, and the genesis bed raises a machine by running the installer. A bed that raises one any other way fails its own acceptance check. |
|
| The builder can arrive on a fresh mesh | The installer carries it and installs it as its last step, and the genesis bed raises a machine by running the installer. A bed that raises one any other way fails its own acceptance check. |
|
||||||
| The control plane a mesh runs is one it built | The genesis bed asserts the running control plane is pinned to a digest this mesh's own registry serves, for an image built from a named repository and commit — not one the installer carried. |
|
| The controller a mesh runs is one it built | The genesis bed asserts the running controller is pinned to a digest this mesh's own registry serves, for an image built from a named repository and commit — not one the installer carried. |
|
||||||
| A core module is built rather than only carried | The control plane is rebuilt from its own repository and path, and the running mesh is upgraded to the result — the same path any module takes. |
|
| A core module is built rather than only carried | The controller is rebuilt from its own repository and path, and the running mesh is upgraded to the result — the same path any module takes. |
|
||||||
| Installing produced a mesh that can produce | A module with source of its own is asked for and comes back pinned to a digest this mesh's registry assigned, not to a placeholder. **Done by hand on a raised machine, not yet by a bed** — it built the shared base and then a module naming that base. Nothing automated asserts it, which makes this the weakest check on this page. |
|
| Installing produced a mesh that can produce | A module with source of its own is asked for and comes back pinned to a digest this mesh's registry assigned, not to a placeholder. **Done by hand on a raised machine, not yet by a bed** — it built the shared base and then a module naming that base. Nothing automated asserts it, which makes this the weakest check on this page. |
|
||||||
|
|||||||
@@ -0,0 +1,222 @@
|
|||||||
|
---
|
||||||
|
layer: to-be
|
||||||
|
status: proposed
|
||||||
|
code:
|
||||||
|
- mesh-control cmd/mesh-builder
|
||||||
|
- mesh-control internal/builder
|
||||||
|
- mesh-catalog modules/builder
|
||||||
|
updated: 2026-09-15
|
||||||
|
decisions:
|
||||||
|
- 02-DECISIONS/0040-what-a-module-is.md
|
||||||
|
- 02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md
|
||||||
|
- 02-DECISIONS/0069-a-module-is-a-repository-and-a-path.md
|
||||||
|
- 02-DECISIONS/0009-modules-and-the-graph.md
|
||||||
|
---
|
||||||
|
|
||||||
|
# Building a module
|
||||||
|
|
||||||
|
[`12-a-module-repository`](12-a-module-repository.md) says what a module may build — an image, an
|
||||||
|
archive, an upstream mirror — and where the result goes. This says **how a build is modelled**, and
|
||||||
|
why the current model does not fit what a module is.
|
||||||
|
|
||||||
|
## The domain, in one sentence
|
||||||
|
|
||||||
|
Turning a module's source into artifacts the mesh can pin, publish and deliver — and saying
|
||||||
|
truthfully what was produced and what it was produced against.
|
||||||
|
|
||||||
|
Everything else is somebody else's: *what* to build is the controller's, *what a build means* is
|
||||||
|
the catalogue's ([ADR 0072](../../02-DECISIONS/0072-two-graphs-and-the-build-chain.md)), *where an artifact runs* is the
|
||||||
|
controller's again. The builder's whole responsibility is the middle.
|
||||||
|
|
||||||
|
## The language
|
||||||
|
|
||||||
|
| term | is |
|
||||||
|
|---|---|
|
||||||
|
| **source** | a repository, a path within it, and a ref — resolved to one commit ([ADR 0069](../../02-DECISIONS/0069-a-module-is-a-repository-and-a-path.md)) |
|
||||||
|
| **recipe** | how *one* artifact is produced from that source |
|
||||||
|
| **toolchain** | what a recipe runs inside — a compiler, a runtime, the SDK |
|
||||||
|
| **artifact** | what a recipe produced, named by the digest of its content |
|
||||||
|
| **publication** | putting an artifact where machines can fetch it, by that digest |
|
||||||
|
| **announcement** | telling the mesh what was built, and every artifact it stood on |
|
||||||
|
| **build** | one request and its outcome, correlated, recorded whether it worked or not |
|
||||||
|
|
||||||
|
## What the model is today, and where it does not fit
|
||||||
|
|
||||||
|
**A toolchain is not modelled at all** — it arrives as two build arguments the module's own
|
||||||
|
Dockerfile declares and the mesh fills in. **A language is not a concept.** And a recipe is
|
||||||
|
effectively singular: producing anything *compiled* means writing a Dockerfile.
|
||||||
|
|
||||||
|
**Archives already work, and that is the corrected half of this.** An earlier draft of this
|
||||||
|
document said the builder refused them. It does not: an archive is packed deterministically,
|
||||||
|
hashed, published by digest, fetched by the machine and unpacked. Only the *local* builder used at
|
||||||
|
genesis refuses one, deliberately — an archive is bytes that mean nothing until something serves
|
||||||
|
them, and there is no registry yet.
|
||||||
|
|
||||||
|
**What an archive cannot do is compile.** `from` names a directory and the directory is packed as
|
||||||
|
it stands, so shipping compiled output means compiling somewhere first — which means a Dockerfile,
|
||||||
|
which is the burden this is about. The gap is not the artifact kind. It is that **no recipe both
|
||||||
|
builds and packs**.
|
||||||
|
|
||||||
|
The cost is not theoretical. To add a module that carries its own code today, an author writes a
|
||||||
|
Dockerfile that: declares two `ARG` bases with no defaults; compiles under a specific working
|
||||||
|
directory so the SDK resolves upward; invokes the compiler *by absolute path*, because the usual
|
||||||
|
`node_modules/.bin` entry is a symlink that the base image's own assembly resolves away; copies the
|
||||||
|
output into a second stage; and sets an environment variable naming the compiled entrypoints. Every
|
||||||
|
module repeats it. Miss any step and the failure arrives somewhere else — as a crash loop, a
|
||||||
|
placeholder digest, a module that builds and does nothing.
|
||||||
|
|
||||||
|
The evidence that this is too hard is in the catalogue: **most modules are not converted**, and the
|
||||||
|
two converted during one session were each wrong twice before they were right, against a working
|
||||||
|
example sitting open in the next window.
|
||||||
|
|
||||||
|
**A module is not a container** ([ADR 0040](../../02-DECISIONS/0040-what-a-module-is.md)) — it is
|
||||||
|
one piece of software and everything that makes it real: its provisioner, its tools, its hooks, its
|
||||||
|
scheduled steps, its event consumers. A build model whose only output is a container image is
|
||||||
|
modelling one of ten resource kinds and calling it the module.
|
||||||
|
|
||||||
|
## The model
|
||||||
|
|
||||||
|
**Recipe becomes explicit, and there is more than one kind.**
|
||||||
|
|
||||||
|
| recipe | produces | from |
|
||||||
|
|---|---|---|
|
||||||
|
| `image` | an image | a Dockerfile, when the software genuinely needs one |
|
||||||
|
| `archive` | a bundle, fetched by digest and unpacked | a directory, packed as it stands — **today** |
|
||||||
|
| `bundle` | the same, fetched and unpacked | this module's source, *compiled* by a toolchain and then packed — **the missing one** |
|
||||||
|
| `upstream` | a mirror | somebody else's pinned reference |
|
||||||
|
|
||||||
|
**Toolchain becomes explicit, and is derived rather than written.** A module says what it is written
|
||||||
|
in; the builder knows what that implies. The two base images stop being something an author names
|
||||||
|
and become something a toolchain *is*.
|
||||||
|
|
||||||
|
```
|
||||||
|
module says: language: typescript
|
||||||
|
builder knows: compile in the typescript toolchain, bundle, produce an archive
|
||||||
|
```
|
||||||
|
|
||||||
|
A Dockerfile remains available and stops being compulsory. It is the right answer for software that
|
||||||
|
needs a particular base, and the wrong answer for "compile my module's code", which is the same
|
||||||
|
operation every time.
|
||||||
|
|
||||||
|
**Why an archive and not always an image.** An archive is a content-addressed blob fetched over
|
||||||
|
plain HTTP and verified by its own digest, so it needs no registry account and no trusted transport
|
||||||
|
— it verifies itself. A container image is refused by a runtime over plain HTTP as *policy*, which
|
||||||
|
is why [issue 048](../../04-ISSUES/048-nothing-makes-a-machine-trust-the-mesh-registry/00-report.md)
|
||||||
|
exists. Modules that are the mesh's own code do not need a container's isolation from the mesh; they
|
||||||
|
need to run. Third-party software still arrives as an image, because that is how its author ships it.
|
||||||
|
|
||||||
|
## The invariants
|
||||||
|
|
||||||
|
1. **An artifact is named by the digest of its content.** Not by a tag, not by a path, never by a
|
||||||
|
placeholder. A recipe that cannot produce a digest has produced nothing.
|
||||||
|
2. **A build announces exactly what it made and everything it stood on.** The second half is what
|
||||||
|
makes build edges derived rather than declared ([ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md)).
|
||||||
|
3. **A failed build produces nothing and announces nothing.** A partial artifact under a digest the
|
||||||
|
mesh will later trust is worse than no artifact.
|
||||||
|
4. **A recipe with nowhere to publish refuses before it builds, not after.** An archive is bytes that
|
||||||
|
mean nothing until something serves them; producing one with no destination has produced nothing
|
||||||
|
usable, and saying so beats returning a path no other machine can read.
|
||||||
|
5. **A build is recorded whether or not it worked**, with everything it was told — the resolved
|
||||||
|
manifest, the path, what it stood on. A record that keeps only the outcome cannot be replayed to
|
||||||
|
anything that missed it ([issue 050](../../04-ISSUES/050-the-catalogue-knows-nothing-built-before-it/00-report.md)).
|
||||||
|
|
||||||
|
## What this costs, stated before it is chosen
|
||||||
|
|
||||||
|
**Every language is permanent.** It needs an SDK — broker client, sealed-credential reading, the
|
||||||
|
event envelope, tool serving — a toolchain image, and a bundler the builder understands. And the
|
||||||
|
spine changes rarely but cascades when it does
|
||||||
|
([ADR 0039](../../02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md)): with one language a contract
|
||||||
|
change is one edit; with four it is four that must land together, and a mesh whose SDKs disagree
|
||||||
|
about the envelope fails by ignoring messages rather than by failing to compile.
|
||||||
|
|
||||||
|
**So the contracts have to stop being expressed twice before they are expressed four times.** They
|
||||||
|
are already: the manifest, declaration and link shapes exist as Go structs in the controller and
|
||||||
|
as TypeScript types in the SDK, kept in step by hand. Nobody has felt it because both live in one
|
||||||
|
repository. A second *language* makes that drift; a specified envelope and schema that every SDK
|
||||||
|
implements makes a second language an implementation rather than a translation.
|
||||||
|
|
||||||
|
**Order matters, then.** Language-neutral contracts, then a second language. The other way round
|
||||||
|
makes the drift worse while hiding it.
|
||||||
|
|
||||||
|
## How these rules are checked
|
||||||
|
|
||||||
|
| rule | checked by |
|
||||||
|
|---|---|
|
||||||
|
| A module with its own code needs no Dockerfile | A module declaring only a language builds, and its artifact is pinned to a digest the mesh's registry assigned. |
|
||||||
|
| An archive is produced and delivered | A module declaring an archive is built, and the machine assigned it has the unpacked files — verified on the machine, not in the build's own output. |
|
||||||
|
| A Dockerfile still works | A module that declares one builds exactly as before, because software that needs a particular base has not stopped existing. |
|
||||||
|
| Nothing is announced that was not made | A build made to fail announces nothing, and the catalogue's graph is unchanged afterwards. |
|
||||||
|
| A build keeps what it was told | A catalogue started after a build asks for what it missed and receives the manifest and the artifacts stood on, not a summary. |
|
||||||
|
| The toolchain is the mesh's, not the author's | Changing the toolchain makes every module built against it stale, and each names what moved. |
|
||||||
|
|
||||||
|
## The whole surface, in one place
|
||||||
|
|
||||||
|
**A reference, and it is kept true by a test rather than by care.** A table like this goes stale the
|
||||||
|
day somebody adds a field, so `mesh-catalog/modules/showcase` is a module that uses all of it, and
|
||||||
|
`TestTheShowcaseModuleIsAValidManifest` fails when it stops doing so. Read the module when this
|
||||||
|
disagrees with it.
|
||||||
|
|
||||||
|
### What a module declares
|
||||||
|
|
||||||
|
| field | is |
|
||||||
|
|---|---|
|
||||||
|
| `module`, `version`, `slug` | its identity; the slug is the short name generated names are built from |
|
||||||
|
| `capabilities` | what a machine must have for this to run there |
|
||||||
|
| `provides` | **the shared seat** — several modules may fill one capability and coexist |
|
||||||
|
| `claims` | **the exclusive seat** — two modules claiming one thing in a scope cannot both be assigned there |
|
||||||
|
| `serves` | what a consumer must know to connect. The mesh fills in the assigned port ([ADR 0038](../../02-DECISIONS/0038-the-mesh-assigns-the-port.md)) |
|
||||||
|
| `requires` | what must be provided by something on the same node |
|
||||||
|
| `binds` | where the mesh writes what a requirement resolved to |
|
||||||
|
| `secrets` | where the mesh seals the credential for a requirement |
|
||||||
|
| `own-secrets` | secrets that are the module's own — a superuser, a broker account |
|
||||||
|
| `emits` / `consumes` | the event graph: 1:many, broadcast, no credential |
|
||||||
|
| `listens` | the port **its own software** uses, and from where. Filtering is computed from these |
|
||||||
|
| `contributes` / `receives` | values one module adds to another's configuration, and the other half |
|
||||||
|
| `accesses` | operator-owned paths it may use and must not own ([ADR 0051](../../02-DECISIONS/0051-shared-data-is-the-operators.md)) |
|
||||||
|
| `certificate` | a certificate for a name it serves |
|
||||||
|
| `grants` | credentials it must create for its consumers |
|
||||||
|
| `filtering` | rules beyond its own ports |
|
||||||
|
| `computed` | marks a module the controller generates rather than an author writing |
|
||||||
|
| `build.artifacts` | what it produces |
|
||||||
|
|
||||||
|
### What it builds
|
||||||
|
|
||||||
|
| kind | is |
|
||||||
|
|---|---|
|
||||||
|
| `bundle` | its own code, compiled by the toolchain its language implies, then packed |
|
||||||
|
| `archive` | a directory, packed as it stands |
|
||||||
|
| `image` | built from a Dockerfile — for software that needs a particular base |
|
||||||
|
| `upstream` | somebody else's image, mirrored and pinned by a digest this mesh assigned |
|
||||||
|
|
||||||
|
### What it puts on a machine
|
||||||
|
|
||||||
|
| resource | is | a module may |
|
||||||
|
|---|---|---|
|
||||||
|
| `directory` | a directory with a mode and an owner | ✅ |
|
||||||
|
| `file` | literal content, with `${bound:…}` and `${secret:…}` filled in | ✅ |
|
||||||
|
| `user` | a login | ✅ |
|
||||||
|
| `access` | a pre-existing path it may use and must not own | ✅ |
|
||||||
|
| `archive` | files fetched by digest and unpacked | ✅ |
|
||||||
|
| `package` | a package that must be present | ✅ |
|
||||||
|
| `network` | a named container network | ✅ |
|
||||||
|
| `container` | an image, in three modes | ✅ |
|
||||||
|
| `process` | **its own code**, in three modes | ✅ |
|
||||||
|
| `service` | an **existing** unit put into a state | for software shipping its own unit |
|
||||||
|
| `action` | a command to run | ❌ **refused** — [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) |
|
||||||
|
|
||||||
|
**The two at the bottom are the interesting rows.** `action` is refused outright: the link may not
|
||||||
|
carry a command, so a module needing something done ships a program that reconciles — which is what
|
||||||
|
a run-once `process` is. `service` installs no unit by design, which is right for software that
|
||||||
|
ships one and wrong for code the mesh built, which has no unit until the mesh writes it.
|
||||||
|
|
||||||
|
### How its code runs, and what that code can be
|
||||||
|
|
||||||
|
| mode | is | | shape | loaded by |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| *(default)* | a unit restarted when it exits | | tools | a tool host, over the broker |
|
||||||
|
| `run-once` | run to completion; what follows is gated on it | | event consumer | the same host, reacting |
|
||||||
|
| `schedule` | a timer; a missed fire happens when the machine returns | | provisioner | invoked when a consumer is granted |
|
||||||
|
| | | | a process | the machine's supervisor |
|
||||||
|
|
||||||
|
**Tools, hooks and consumers are not further modes**, which is the test of whether three is the
|
||||||
|
right number: they are loaded by a tool host, and a tool host is a process that stays up.
|
||||||
@@ -0,0 +1,186 @@
|
|||||||
|
---
|
||||||
|
layer: to-be
|
||||||
|
status: proposed
|
||||||
|
code:
|
||||||
|
- mesh-sdk src
|
||||||
|
- mesh-tools src/broker-amqp.ts
|
||||||
|
- mesh-control internal/link
|
||||||
|
updated: 2026-09-15
|
||||||
|
decisions:
|
||||||
|
- 02-DECISIONS/0074-the-wire-is-specified-not-the-types.md
|
||||||
|
- 02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md
|
||||||
|
- 02-DECISIONS/0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md
|
||||||
|
- 02-DECISIONS/0042-the-shape-of-an-event-on-the-wire.md
|
||||||
|
---
|
||||||
|
|
||||||
|
# The module protocol
|
||||||
|
|
||||||
|
**What a module's code and the mesh say to each other.** An SDK is an implementation of this in one
|
||||||
|
language and nothing more ([ADR 0074](../../02-DECISIONS/0074-the-wire-is-specified-not-the-types.md)).
|
||||||
|
|
||||||
|
This is a specification, so it says what is required rather than how anything is arranged. Where it
|
||||||
|
describes current behaviour that is *not yet* specified-and-conformed, it says so.
|
||||||
|
|
||||||
|
## The shape of it
|
||||||
|
|
||||||
|
A **floor** every implementation needs, and three **capabilities** that are independent of each
|
||||||
|
other. An SDK implements the floor plus whatever capabilities it claims; a module is refused at
|
||||||
|
build time if it uses a capability its language's SDK does not implement.
|
||||||
|
|
||||||
|
| part | a module uses it to |
|
||||||
|
|---|---|
|
||||||
|
| **connection** | reach the broker as itself |
|
||||||
|
| **events** | emit, and react to what others emit |
|
||||||
|
| **tools** | answer questions asked of it |
|
||||||
|
| **provisioning** | give a consumer an instance of what it provides |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## The floor: connection
|
||||||
|
|
||||||
|
### The credential
|
||||||
|
|
||||||
|
A module is given a **sealed credential** as a file, and told where by its declaration. The
|
||||||
|
document:
|
||||||
|
|
||||||
|
| field | is | required |
|
||||||
|
|---|---|---|
|
||||||
|
| `url` | an `amqps://` URL carrying the account's user and password | yes |
|
||||||
|
| `fingerprint` | sha256 of the certificate the broker must present | yes for a scoped account |
|
||||||
|
| `node` | the machine this account was issued for | yes for a scoped account |
|
||||||
|
| `module` | the module this account was issued for | yes for a scoped account |
|
||||||
|
|
||||||
|
A plain string rather than a document is a **bootstrap URL** — unscoped, for the moment before a
|
||||||
|
mesh can issue anything. An implementation accepts both and must not treat the second as ordinary.
|
||||||
|
|
||||||
|
### Connecting
|
||||||
|
|
||||||
|
- The connection **pins the fingerprint**. It does not trust a certificate authority, and it does
|
||||||
|
not skip verification. A broker presenting a different certificate is refused, whatever else is
|
||||||
|
true of it.
|
||||||
|
- A scoped account **does not declare exchanges**. The foundation owns them; an account that may
|
||||||
|
declare one is an account that may create a parallel mesh by typo.
|
||||||
|
- An implementation **declares its own queue** and nothing else.
|
||||||
|
|
||||||
|
### Identity
|
||||||
|
|
||||||
|
**A module's node and module name come from its credential, never from its environment.**
|
||||||
|
|
||||||
|
This is not a convenience. It is what makes what a module emits match what the mesh authorised: an
|
||||||
|
environment variable can be set by anything on the machine, and a module that took its identity
|
||||||
|
from one could emit events attributing them to another module. Where an environment variable and
|
||||||
|
the credential disagree, the credential wins and the variable is overwritten.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Capability: events
|
||||||
|
|
||||||
|
### The exchanges
|
||||||
|
|
||||||
|
| exchange | carries |
|
||||||
|
|---|---|
|
||||||
|
| `mesh.events` | every event |
|
||||||
|
| `mesh.events.dead` | what could not be handled |
|
||||||
|
|
||||||
|
### The queue
|
||||||
|
|
||||||
|
One **durable** queue per consumer, named `<node>.<module>.events`, with as many bindings as the
|
||||||
|
module has patterns. Durable because an event emitted while a module is restarting is exactly the
|
||||||
|
one that must not be lost.
|
||||||
|
|
||||||
|
**A message matching two bindings is delivered once**, so an implementation must match the routing
|
||||||
|
key against its own patterns locally to decide which handlers run. An implementation that ran every
|
||||||
|
handler whose exchange binding matched would run the wrong one.
|
||||||
|
|
||||||
|
### The envelope
|
||||||
|
|
||||||
|
Headers ride as AMQP headers. The body is JSON.
|
||||||
|
|
||||||
|
| header | is | required |
|
||||||
|
|---|---|---|
|
||||||
|
| `x-event-id` | a unique id, made by the emitter | yes |
|
||||||
|
| `x-source` | the module, context or node that emitted it | yes |
|
||||||
|
| `x-node` | the machine it was emitted from | yes |
|
||||||
|
| `x-time` | emit time, RFC-3339 | yes |
|
||||||
|
| `content-type` | always `application/json` | yes |
|
||||||
|
| `x-causation-id` | the event or command that caused this one | no |
|
||||||
|
| `x-schema` | a version of the body's shape | no |
|
||||||
|
|
||||||
|
**An unknown `x-` header is ignored, never refused.** An event is observed by parties that need not
|
||||||
|
all understand every header, and an implementation that refused one would make adding a header a
|
||||||
|
breaking change for everybody.
|
||||||
|
|
||||||
|
### Delivery
|
||||||
|
|
||||||
|
At-least-once. **Deduplication is on `x-event-id`**, which only the emitter can produce — a
|
||||||
|
consumer cannot tell a redelivery from a second event any other way.
|
||||||
|
|
||||||
|
### What is true, checked (2026-09-16)
|
||||||
|
|
||||||
|
Go emits all five required headers; the SDK requires exactly those. `x-causation-id` and `x-schema`
|
||||||
|
are **optional** — the SDK sets them when a handler has a causation or a schema, and reads them
|
||||||
|
back; a bare event carrying neither is correct. So the envelope agrees across the two
|
||||||
|
implementations. `x-schema` is available for versioning a body's shape and is set by whoever has a
|
||||||
|
version to declare.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Capability: tools
|
||||||
|
|
||||||
|
A module's tools are its operator-facing surface.
|
||||||
|
|
||||||
|
- A tool is served from a **shared durable queue**, `serve.<key>`. Shared, so several runtimes
|
||||||
|
serving one tool compete for a call rather than each answering it.
|
||||||
|
- A call is request and reply. The reply returns through the RPC exchange `mesh.rpc`, keyed by the
|
||||||
|
caller's own reply queue — **not** through the default exchange, which would let a caller publish
|
||||||
|
into any queue on the broker.
|
||||||
|
- A caller needs a **reply queue**, and that is what a module's scoped account may not declare
|
||||||
|
([issue 049](../../04-ISSUES/049-a-module-can-serve-tools-and-nothing-can-call-them/00-report.md)).
|
||||||
|
So a module may serve tools and may not call them, and nothing today issues an account to anything
|
||||||
|
that wants to ask.
|
||||||
|
|
||||||
|
### Not yet true
|
||||||
|
|
||||||
|
The caller's half has no account. Until that is settled, the only thing that can ask a module a
|
||||||
|
question is the foundation's bootstrap admin, which is not a protocol so much as a way in.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Capability: provisioning
|
||||||
|
|
||||||
|
A provider ships the provisioner that creates instances of what it offers
|
||||||
|
([ADR 0040](../../02-DECISIONS/0040-what-a-module-is.md)).
|
||||||
|
|
||||||
|
| direction | carries |
|
||||||
|
|---|---|
|
||||||
|
| **grant, in** | the provision, who is asking — **a module on a machine**, not a machine — and what the consumer contributed |
|
||||||
|
| **credential, out** | the fields the provision promises a consumer |
|
||||||
|
|
||||||
|
**Who is asking is one thing with two parts.** A machine routinely runs several modules wanting the
|
||||||
|
same provision, so a grant addressed to a node alone does not name a consumer, and withdrawing one
|
||||||
|
would take another's away.
|
||||||
|
|
||||||
|
### Checked, and it agrees (2026-09-16)
|
||||||
|
|
||||||
|
This looked like the sharpest disagreement and was not one. The live wire is the contributions file
|
||||||
|
— `as`, `secret`, `node`, `at`, `values` — and it is the same on both sides. The types that
|
||||||
|
disagreed (`Grant`, `Interface` in the SDK's `contracts`) were dead: exported, imported by nothing,
|
||||||
|
describing fields the wire does not carry. They have been removed. The lesson kept: a type beside
|
||||||
|
the wire that has drifted from it is worse than none, which is why the wire is specified and
|
||||||
|
implementations are checked against it rather than trusted to still match a hand-kept shape.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## How an implementation is checked
|
||||||
|
|
||||||
|
Per capability, against fixtures rather than prose — a specification nobody can run is a document
|
||||||
|
two implementations drift from while both believe they conform.
|
||||||
|
|
||||||
|
| Rule | Checked by |
|
||||||
|
|---|---|
|
||||||
|
| The floor is the floor | Every implementation reads the same credential fixture, and refuses one whose fingerprint does not match what the broker presents. |
|
||||||
|
| Identity comes from the credential | A fixture sets an environment that disagrees with the credential; the emitted event carries the credential's. |
|
||||||
|
| The envelope is the envelope | An emitted event is compared header by header against a fixture; a missing required header fails, an unknown `x-` header is accepted. |
|
||||||
|
| Delivery is at-least-once | A fixture delivered twice is handled once. |
|
||||||
|
| A grant names a consumer | A grant fixture is read by every implementation and yields the same module and the same node. |
|
||||||
|
| A partial SDK is legitimate | An implementation claiming the floor and events passes those suites and is listed for them; a module using tools in that language is refused at build time with the reason. |
|
||||||
@@ -0,0 +1,179 @@
|
|||||||
|
---
|
||||||
|
layer: to-be
|
||||||
|
status: proposed
|
||||||
|
code:
|
||||||
|
- mesh-catalog modules/showcase
|
||||||
|
- mesh-control internal/builder
|
||||||
|
- mesh-sdk src
|
||||||
|
updated: 2026-09-15
|
||||||
|
decisions:
|
||||||
|
- 02-DECISIONS/0074-the-wire-is-specified-not-the-types.md
|
||||||
|
- 02-DECISIONS/0040-what-a-module-is.md
|
||||||
|
- 02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md
|
||||||
|
---
|
||||||
|
|
||||||
|
# Writing a module
|
||||||
|
|
||||||
|
A worked guide. One module, four capabilities, four languages, and the packages it publishes.
|
||||||
|
|
||||||
|
The reference for *what can be said* is [`18-building-a-module`](18-building-a-module.md); the
|
||||||
|
reference for *what the code and the mesh say to each other* is
|
||||||
|
[`19-the-module-protocol`](19-the-module-protocol.md). This is how you actually write one.
|
||||||
|
|
||||||
|
## First: what is in the SDK, exactly
|
||||||
|
|
||||||
|
**The protocol, and nothing else** ([ADR 0074](../../02-DECISIONS/0074-the-wire-is-specified-not-the-types.md)).
|
||||||
|
An SDK is an implementation of the module protocol in one language. If something is not in the
|
||||||
|
protocol it does not belong in an SDK, and that rule is what stops it becoming the 34,000-line
|
||||||
|
shared library this design exists to avoid ([ADR 0039](../../02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md)).
|
||||||
|
|
||||||
|
Split the way the protocol is split:
|
||||||
|
|
||||||
|
| the SDK gives you | so that you can |
|
||||||
|
|---|---|
|
||||||
|
| **connection** — reads the sealed credential, pins the certificate, takes your identity from it | reach the broker as *this module on this machine*, and not be able to claim otherwise |
|
||||||
|
| **events** — emit, subscribe, the envelope, dedup | react to what happens in the mesh |
|
||||||
|
| **tools** — register and serve | be asked questions |
|
||||||
|
| **provisioning** — receive a grant, return a credential | give a consumer an instance of what you provide |
|
||||||
|
|
||||||
|
**What it does not give you, deliberately:**
|
||||||
|
|
||||||
|
- **No configuration loader.** Configuration arrives as files the mesh wrote and environment the
|
||||||
|
mesh set. Reading a file is not a thing an SDK needs to teach.
|
||||||
|
- **No API clients.** A Plex client belongs in the Plex module. It changes when Plex changes,
|
||||||
|
which has nothing to do with any other module — *frequent **and** cascading is the disease.*
|
||||||
|
- **No storage, no HTTP framework, no logging library.** Use the language's.
|
||||||
|
|
||||||
|
If you find yourself wanting to add something to the SDK, the test is ADR 0039's: **does editing it
|
||||||
|
recompile unrelated modules, and does it change often?** Both, and it does not belong.
|
||||||
|
|
||||||
|
## A module with four capabilities, in four languages
|
||||||
|
|
||||||
|
A module is **one piece of software** ([ADR 0040](../../02-DECISIONS/0040-what-a-module-is.md)) and
|
||||||
|
may still be written in several languages — each artifact names its own, compiles alone and is
|
||||||
|
packed alone.
|
||||||
|
|
||||||
|
```
|
||||||
|
showcase/
|
||||||
|
module.json
|
||||||
|
events/ ← TypeScript: reacts to what the mesh does
|
||||||
|
tools/ ← Go: answers questions
|
||||||
|
provisioner/ ← Rust: grants instances of what it provides
|
||||||
|
ingest/ ← Python: a scheduled job
|
||||||
|
```
|
||||||
|
|
||||||
|
### The manifest
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"module": "showcase",
|
||||||
|
"build": { "artifacts": [
|
||||||
|
{ "name": "events", "kind": "bundle", "language": "typescript",
|
||||||
|
"entrypoints": ["index.js"] },
|
||||||
|
{ "name": "tools", "kind": "bundle", "language": "go",
|
||||||
|
"entrypoints": ["tools"] },
|
||||||
|
{ "name": "provisioner", "kind": "bundle", "language": "rust",
|
||||||
|
"entrypoints": ["provisioner"] },
|
||||||
|
{ "name": "ingest", "kind": "bundle", "language": "python",
|
||||||
|
"entrypoints": ["ingest.py"] }
|
||||||
|
]},
|
||||||
|
"resources": [
|
||||||
|
{ "id": "events", "type": "process", "name": "showcase-events",
|
||||||
|
"artifact": "events", "run": ["node", "index.js"] },
|
||||||
|
{ "id": "tools", "type": "process", "name": "showcase-tools",
|
||||||
|
"artifact": "tools", "run": ["./tools"] },
|
||||||
|
{ "id": "grants", "type": "process", "name": "showcase-grants",
|
||||||
|
"artifact": "provisioner", "run": ["./provisioner"] },
|
||||||
|
{ "id": "ingest", "type": "process", "name": "showcase-ingest",
|
||||||
|
"artifact": "ingest", "run": ["python", "ingest.py"],
|
||||||
|
"schedule": "0 3 * * *" }
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Four artifacts, four toolchains, four processes, one module.** Nothing here says how any of them
|
||||||
|
is hosted — that is the mesh's, and it is why these are `process` rather than four containers.
|
||||||
|
|
||||||
|
### Step 1 — say what it is, in each language's own terms
|
||||||
|
|
||||||
|
Each part is an ordinary project in its language, depending on the mesh SDK for that language the
|
||||||
|
way it would depend on anything:
|
||||||
|
|
||||||
|
```
|
||||||
|
events/package.json "@novox/mesh-sdk": "^1.2.0"
|
||||||
|
tools/go.mod require novox.example/mesh-sdk v1.2.0
|
||||||
|
provisioner/Cargo.toml mesh-sdk = "1.2"
|
||||||
|
ingest/pyproject.toml dependencies = ["mesh-sdk~=1.2"]
|
||||||
|
```
|
||||||
|
|
||||||
|
**Resolved from the mesh's own package registries**, which are the ordinary registries for each
|
||||||
|
ecosystem, hosted by the mesh. An author does nothing unusual: `npm install`, `go mod tidy`,
|
||||||
|
`cargo build`, `pip install` all work, and a developer's laptop resolves exactly what a build does.
|
||||||
|
|
||||||
|
### Step 2 — write each part against its capability
|
||||||
|
|
||||||
|
Each uses only the part of the protocol it needs. The event consumer never learns what a grant is.
|
||||||
|
|
||||||
|
```
|
||||||
|
events/index.ts on("module.builder.built", …) → the events capability
|
||||||
|
tools/main.go tool("showcase_state", …) → the tools capability
|
||||||
|
provisioner/main.rs grant → credential → the provisioning capability
|
||||||
|
ingest/ingest.py emit("module.showcase.ingested", …) → events, emitting only
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 3 — the mesh does the rest
|
||||||
|
|
||||||
|
You do not write a Dockerfile, a unit file, or a port number. The builder picks the toolchain from
|
||||||
|
the language, compiles each artifact alone, and publishes it. The controller assigns the machine
|
||||||
|
and the ports; the host writes the units.
|
||||||
|
|
||||||
|
### What it costs you to use four languages
|
||||||
|
|
||||||
|
**Honestly: four sets of dependencies to keep current, and four SDKs that must agree.** The mesh
|
||||||
|
makes it possible, not free. A module in one language is simpler, and the reason to use four is
|
||||||
|
that one of them genuinely suits a part better — not that you can.
|
||||||
|
|
||||||
|
## Publishing a package is a capability
|
||||||
|
|
||||||
|
A module may publish libraries as well as run code. **The SDK is not special; it is simply the first
|
||||||
|
module that did this**, and its own consumers are the mesh's modules.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"module": "plex",
|
||||||
|
"build": { "artifacts": [
|
||||||
|
{ "name": "server", "kind": "upstream", "from": "plexinc/pms-docker@sha256:…" },
|
||||||
|
{ "name": "client-ts", "kind": "package", "language": "typescript", "from": "clients/typescript" },
|
||||||
|
{ "name": "client-rust", "kind": "package", "language": "rust", "from": "clients/rust" },
|
||||||
|
{ "name": "client-py", "kind": "package", "language": "python", "from": "clients/python" }
|
||||||
|
]}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
A `package` artifact is built and **published to the mesh's registry for that ecosystem**, under the
|
||||||
|
version its own project file declares. Another module then depends on it the ordinary way:
|
||||||
|
|
||||||
|
```
|
||||||
|
"@novox/plex-client": "^2.0.0"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Why this belongs to modules rather than being a separate thing.** A client for a piece of software
|
||||||
|
changes when that software changes, and the module that owns the software is the only thing that
|
||||||
|
knows. Putting the Plex client anywhere else is the shared-library disease with extra steps.
|
||||||
|
|
||||||
|
### What this leaves open
|
||||||
|
|
||||||
|
- **Which private registry.** That there *is* one is settled —
|
||||||
|
[ADR 0014](../../02-DECISIONS/0014-no-npm-workspace.md) says each module consumes its dependencies
|
||||||
|
from the private registry, the mesh's own shared library included. Which software serves it is
|
||||||
|
not: the catalogue holds `verdaccio`, and a git host usually serves package registries too.
|
||||||
|
- **And the bootstrap does not have one.** ADR 0014 assumes a registry exists; on a fresh mesh
|
||||||
|
nothing has installed one when the first SDK is built. That is the same pivot as everything else
|
||||||
|
and it has not been designed.
|
||||||
|
- **Who may publish.** A builder pushing a package needs an account on that registry, which is a
|
||||||
|
credential in the bootstrap path and does not exist yet.
|
||||||
|
- **Versions and ranges.** Everything else the mesh delivers is pinned by digest, and a range is
|
||||||
|
resolved at build time from whatever the registry holds. A lock file records what was chosen, and
|
||||||
|
the builder records that as the build edge — but *a mesh that can rebuild a commit and get a
|
||||||
|
different library* is a real change from how everything else here works, and it should be a
|
||||||
|
decision rather than a consequence.
|
||||||
@@ -0,0 +1,178 @@
|
|||||||
|
---
|
||||||
|
layer: to-be
|
||||||
|
status: proposed
|
||||||
|
code:
|
||||||
|
- mesh-host internal/bootstrap
|
||||||
|
- mesh-host cmd/mesh-bootstrap
|
||||||
|
- mesh-lab test/integration/one-node-mesh.test.ts
|
||||||
|
updated: 2026-09-15
|
||||||
|
decisions:
|
||||||
|
- 02-DECISIONS/0067-genesis-is-a-pivot.md
|
||||||
|
- 02-DECISIONS/0073-the-installer-carries-a-builder.md
|
||||||
|
- 02-DECISIONS/0071-where-genesis-gets-its-source.md
|
||||||
|
- 02-DECISIONS/0014-no-npm-workspace.md
|
||||||
|
---
|
||||||
|
|
||||||
|
# The installation, in full
|
||||||
|
|
||||||
|
Every step from a machine with nothing to a mesh that maintains itself. Written out explicitly
|
||||||
|
because it is the procedure everything else depends on, and because the parts that do not exist
|
||||||
|
yet are easier to see beside the parts that do.
|
||||||
|
|
||||||
|
[`17-raising-a-mesh`](17-raising-a-mesh.md) argues *why* it is shaped this way. This says *what
|
||||||
|
happens*, in order, with each step's name as the installer prints it.
|
||||||
|
|
||||||
|
## What must be true before anything starts
|
||||||
|
|
||||||
|
| | why |
|
||||||
|
|---|---|
|
||||||
|
| a container runtime | the foundation is containers, and the installer refuses without one |
|
||||||
|
| the host binary, where the installer expects it | it is what the machine becomes |
|
||||||
|
| a repository and a commit to build from | the installer carries a builder, not a controller, so it must be told what to make ([ADR 0073](../../02-DECISIONS/0073-the-installer-carries-a-builder.md)) |
|
||||||
|
| a way out to the internet | the store, the broker and the registry are pulled from it |
|
||||||
|
| a reachable git host, and a commit it serves | what is cloned is the trust anchor for everything this mesh will ever run ([ADR 0071](../../02-DECISIONS/0071-where-genesis-gets-its-source.md)) |
|
||||||
|
| the address other machines will reach this one on | a token carries it verbatim; the installer refuses to guess |
|
||||||
|
|
||||||
|
## Phase one — a machine becomes a mesh of one
|
||||||
|
|
||||||
|
Twelve steps, run by one program, each safe to run again.
|
||||||
|
|
||||||
|
| # | step | what happens | true afterwards |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 1 | `preflight` | everything above is checked | the machine is not half-changed by a missing prerequisite |
|
||||||
|
| 2 | `load` | the carried builder image is loaded | the mesh has the one thing it cannot fetch — the thing that does the fetching |
|
||||||
|
| 3 | `build` | the builder clones the named repository at the named commit and **builds the controller** | what will run is something this mesh made and can make again |
|
||||||
|
| 4 | `bundle` | the foundation template is written out, with the built controller's id in place of the placeholder | the machine has a description of what it will become |
|
||||||
|
| 5 | `apply` | store, broker, schemas, and a **temporary** controller are raised | a mesh of one exists and answers |
|
||||||
|
| 6 | `verify` | the controller is asked, rather than assumed | it replies, and says it has no machines |
|
||||||
|
| 7 | `enrol` | the machine joins the mesh it is itself running | the mesh has one node, and an agent runs on it |
|
||||||
|
| 8 | `registry` | the mesh's own artifact store is installed | there is somewhere to keep what this mesh makes |
|
||||||
|
| 9 | `publish` | the controller's image is pushed into it | the image has a digest something other than itself assigned |
|
||||||
|
| 10 | `controller` | it is installed again, as an ordinary module pinned to that digest | what runs is a module like any other |
|
||||||
|
| 11 | `retire` | the temporary controller is dropped | **the pivot is complete** — what raised the mesh is gone |
|
||||||
|
| 12 | `builder` | the carried builder is published, installed as a module, and issued a broker account | the mesh can produce |
|
||||||
|
|
||||||
|
**Steps 8 to 11 are the pivot** ([ADR 0067](../../02-DECISIONS/0067-genesis-is-a-pivot.md)). Before
|
||||||
|
them the controller is something the installer put there; after them it is something the mesh
|
||||||
|
holds a record of and can upgrade. The account in step 12 is issued *before* the machine is sent
|
||||||
|
anything, because a builder that arrives without its credential starts, finds nothing it may read,
|
||||||
|
and waits — which looks exactly like a builder with no work.
|
||||||
|
|
||||||
|
## Phase two — a mesh of one becomes a mesh that works
|
||||||
|
|
||||||
|
**Genesis ends with a mesh that runs, which is not the same as a mesh that works.** It has a control
|
||||||
|
plane, a store, a broker, a registry and a builder. It holds no module graph, has no private
|
||||||
|
network, no packet filter, and cannot resolve a name. Calling that "installed" is what let the
|
||||||
|
catalogue be missing from a test for weeks without anything complaining.
|
||||||
|
|
||||||
|
| # | step | what happens | why it is here |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 13 | the shared base is built | the toolchain and runtime every module with code of its own stands on | nothing else with code can be built until it exists |
|
||||||
|
| 14 | a store module is built and run | a database **provider**, which the foundation's store is not | the foundation's store is the controller's own memory, and offers nothing to anything |
|
||||||
|
| 15 | the catalogue is built and run | the module graph | without it the mesh cannot say what it holds, what a change reaches, or what must be rebuilt |
|
||||||
|
| 16 | the catalogue asks for what it missed | the builds made before it existed are replayed | on a fresh mesh those are always the base, the store and the catalogue itself ([issue 050](../../04-ISSUES/050-the-catalogue-knows-nothing-built-before-it/00-report.md)) |
|
||||||
|
| 17 | the controller is rebuilt from its own repository | and rolled out through the module path | the moment the mesh stops depending on the installer for anything |
|
||||||
|
| 18 | `networking` is assigned **and the node placed** | a private network, and names | assigning installs the module; placing says where this machine is on it. Both, or the names file is written empty |
|
||||||
|
| 19 | the packet filter is assigned | rules generated from what modules declared | until this, every rule the mesh computes has never been applied to anything |
|
||||||
|
|
||||||
|
## Phase three — machines arrive
|
||||||
|
|
||||||
|
Only now. A machine joining a mesh that cannot build anything proves that enrolment works, which
|
||||||
|
was never the doubtful part.
|
||||||
|
|
||||||
|
| # | step | what happens |
|
||||||
|
|---|---|---|
|
||||||
|
| 20 | a node record is made and a token issued | one-time, carrying the broker's address and the fingerprint to pin |
|
||||||
|
| 21 | the machine enrols and runs its agent | being heard from once is not an agent running; both are checked |
|
||||||
|
| 22 | it is placed on the private network | or nothing can bind a consumer on it to a provider elsewhere |
|
||||||
|
| 23 | modules are assigned to it | and it pulls what it needs from the mesh's registry |
|
||||||
|
|
||||||
|
## What is not yet true
|
||||||
|
|
||||||
|
Stated plainly, because a procedure that implies otherwise is worse than none.
|
||||||
|
|
||||||
|
**Steps 13 to 19 are not the installer's.** They are things somebody types. The installer ends at
|
||||||
|
step 12, and everything that turns a running mesh into a working one is manual — which is why a
|
||||||
|
test had to be written to find out they were missing.
|
||||||
|
|
||||||
|
**Step 23 does not work for a second machine.** It has no account on the registry
|
||||||
|
([issue 042](../../04-ISSUES/042-nothing-gives-a-node-an-account-for-a-registry/00-report.md)) and
|
||||||
|
no reason to trust a registry serving plain HTTP over the network
|
||||||
|
([issue 048](../../04-ISSUES/048-nothing-makes-a-machine-trust-the-mesh-registry/00-report.md)).
|
||||||
|
Both are invisible on a mesh of one, where the registry is loopback.
|
||||||
|
|
||||||
|
**There is no private package registry**, and [ADR 0014](../../02-DECISIONS/0014-no-npm-workspace.md)
|
||||||
|
assumes one: a module consumes its dependencies, the mesh's own shared library included, from it.
|
||||||
|
At step 13 nothing has installed one, so the first build of the shared base resolves the SDK some
|
||||||
|
other way — today by a git URL, which is
|
||||||
|
[issue 053](../../04-ISSUES/053-the-sdk-is-pinned-twice-and-the-two-disagree/00-report.md).
|
||||||
|
|
||||||
|
**The installer does not install the host's unit, though one exists.** `mesh-host/packaging/` ships
|
||||||
|
`nox-mesh-host.service` and two companions; the installer declines to place them because a unit file
|
||||||
|
is a packaging decision. So an install that does nothing further leaves a machine whose containers
|
||||||
|
come back after a reboot and whose agent does not — it runs the right things and can no longer be
|
||||||
|
told anything.
|
||||||
|
|
||||||
|
**This is a gap in packaging, not in the mesh**, and the distinction matters: the lab's
|
||||||
|
`--host-in-background` says in its own help that it does not survive a reboot, so a lab run failing
|
||||||
|
this is the lab being honest rather than the mesh being broken. What is missing is the step that
|
||||||
|
puts the shipped unit on the machine.
|
||||||
|
|
||||||
|
**Nothing asserts a mesh was installed this way.** A claim that a machine was brought up by this
|
||||||
|
procedure cannot be contradicted by anything afterwards.
|
||||||
|
|
||||||
|
## What a finished mesh holds
|
||||||
|
|
||||||
|
**Twelve, and after the pivot none of them is a specialty.** Every row is a module the mesh built,
|
||||||
|
holds a version of, and can upgrade — which is the whole claim, and is not true today for the first
|
||||||
|
two.
|
||||||
|
|
||||||
|
| # | module | provides | note |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 1 | `postgres` | `postgres-database` | **the controller's own records and every module's.** One server, not two |
|
||||||
|
| 2 | `lavinmq` | `amqp` | the broker every node dials, and what modules are granted vhosts on |
|
||||||
|
| 3 | `mesh-control` | *claims* `the-controller` | decides what runs where |
|
||||||
|
| 4 | `distribution` | `artifact-store` | what the mesh built, pinned by digest — the module is the software (Distribution), the provision is the job |
|
||||||
|
| 5 | `builder` | — | turns source into artifacts |
|
||||||
|
| 6 | `mesh-tools` | build inputs | the base everything with code compiles against. **Runs nowhere** |
|
||||||
|
| 7 | `mesh-catalog` | the module graph | what is held, what a change reaches, what must be rebuilt |
|
||||||
|
| 8 | `networking` | `private-network`, naming | requirements only — assigning it brings `mesh-wireguard` and `mesh-names` |
|
||||||
|
| 9 | `dnsmasq` + one of `resolved-split-dns` / `resolv-conf` | `wildcard-resolution` | names that actually resolve, on top of `mesh-resolver`'s data |
|
||||||
|
| 10 | `step-ca` | `acme-ca` | certificates for `.internal` |
|
||||||
|
| 11 | `firewall` | *claims* `the-packet-filter` | rules generated from what modules declared |
|
||||||
|
| 12 | `gitea` | `package-registry` | where a module's dependencies come from ([ADR 0014](../../02-DECISIONS/0014-no-npm-workspace.md)), and the git host |
|
||||||
|
|
||||||
|
### Why it is twelve and not thirteen
|
||||||
|
|
||||||
|
**The foundation's store and the `postgres` module are the same module.** They were two rows while the
|
||||||
|
foundation was a different *kind* of thing: a store raised from a bundle cannot provide
|
||||||
|
`postgres-database`, so anything wanting a database needed a second server. That is visible on any
|
||||||
|
mesh built today — `mesh-store` and `postgres`, two containers, **the same image**.
|
||||||
|
|
||||||
|
The naming rule settles which name survives
|
||||||
|
([ADR 0040](../../02-DECISIONS/0040-what-a-module-is.md)):
|
||||||
|
|
||||||
|
> Where the consumer **speaks a protocol** — a database's wire protocol and query dialect — the
|
||||||
|
> interface *is* the protocol. **"database" is not a capability; the protocol is.** Never false
|
||||||
|
> genericity: a name must not promise a swap the contract cannot deliver.
|
||||||
|
|
||||||
|
So there is no `store` module. The controller is coupled to postgres — its own queries use
|
||||||
|
`distinct on` and `on conflict`, which are not portable — and calling it `store` would advertise a
|
||||||
|
swap that would fail the first time somebody tried it.
|
||||||
|
|
||||||
|
**The same applies to the broker**, with a different outcome: `amqp` *is* a protocol and more than
|
||||||
|
one implementation speaks it, so `amqp` is a legitimate provision and `lavinmq` is one provider of
|
||||||
|
it. The foundation's broker and the `lavinmq` module collapse the same way.
|
||||||
|
|
||||||
|
### What this costs, and it is the last specialty
|
||||||
|
|
||||||
|
Adopting the store and the broker as modules is [issue 051](../../04-ISSUES/051-the-mesh-cannot-update-what-it-depends-on/00-report.md),
|
||||||
|
and it is the only part of this that has not been designed. The two hard parts:
|
||||||
|
|
||||||
|
- **upgrading a store the controller is reading from** — a rollout where the thing being replaced
|
||||||
|
is the thing holding the record of the rollout
|
||||||
|
- **upgrading a broker over the broker** — the instruction arrives on what it replaces, so the
|
||||||
|
machine must finish without being able to report progress
|
||||||
|
|
||||||
|
Both are operations with windows, and a machine rebooting inside one is an operator's problem during
|
||||||
|
an operation rather than a reason not to do it.
|
||||||
@@ -0,0 +1,118 @@
|
|||||||
|
---
|
||||||
|
layer: to-be
|
||||||
|
status: in-progress
|
||||||
|
updated: 2026-09-15
|
||||||
|
decisions:
|
||||||
|
- 02-DECISIONS/0067-genesis-is-a-pivot.md
|
||||||
|
- 02-DECISIONS/0074-the-wire-is-specified-not-the-types.md
|
||||||
|
- 02-DECISIONS/0075-two-stores-and-which-provides-what.md
|
||||||
|
- 02-DECISIONS/0014-no-npm-workspace.md
|
||||||
|
---
|
||||||
|
|
||||||
|
# The work ahead
|
||||||
|
|
||||||
|
Everything decided this cycle and not yet built, in the order its dependencies allow. Each phase
|
||||||
|
ends at something provable on a running mesh, because a phase that ends at a claim is a phase that
|
||||||
|
went missing without anything complaining.
|
||||||
|
|
||||||
|
## How this is built, and when it is run
|
||||||
|
|
||||||
|
**Implemented as code with unit tests, committed per change, and run in the lab ONCE the pieces that
|
||||||
|
would change the outcome are in place.** The mistake this corrects: repeatedly running a 20-minute
|
||||||
|
lab against a mesh mid-transformation, debugging paths the next phase deletes. The base-build hang,
|
||||||
|
for instance, is almost certainly the SDK resolving from a git URL inside a docker build (issue
|
||||||
|
053) — which Phase 2 removes. Debugging it on the current shape is debugging deprecated code.
|
||||||
|
|
||||||
|
So the lab run is the acceptance test at the end of the assembled work, not the tool for finding
|
||||||
|
each bug. Where a fault can be reasoned out of the code path, it is — reading, not running.
|
||||||
|
|
||||||
|
## Phase 0 — folded in
|
||||||
|
|
||||||
|
The installer's own regressions (stale carried builder, a diagnostic on the parsed stdout) are
|
||||||
|
fixed and committed. Whether it reaches a full green run is answered by the final lab run below,
|
||||||
|
after the phases that change its build path are in — not before.
|
||||||
|
|
||||||
|
## Phase 1 — the protocol is one thing, and correct *(mostly done: the drift was dead types)*
|
||||||
|
|
||||||
|
**Why here.** The Go controller and the TypeScript SDK disagree about what a grant carries
|
||||||
|
(`consumer` is the module in one, the node in the other). That is exercised by the installer's own
|
||||||
|
provisioning — the catalogue's database — so it belongs before more is built on it.
|
||||||
|
|
||||||
|
- [x] 1.1 the drift was not live — inspection showed the wire agrees (contributions file; envelope
|
||||||
|
required headers). The dead types that disagreed are removed, ADR 0074 and doc 19 corrected
|
||||||
|
- [ ] 1.2 a conformance fixture for the two live cross-language contracts — the event envelope and
|
||||||
|
the contributions file — checked in both suites, as **prevention** rather than repair
|
||||||
|
- [ ] 1.3 (deferred) a full per-capability suite when a third language is actually added; not
|
||||||
|
needed to keep two honest
|
||||||
|
|
||||||
|
**Done when.** A fixture pins the envelope and the contributions file, and a change to either side
|
||||||
|
that breaks agreement fails a test rather than a mesh.
|
||||||
|
|
||||||
|
## Phase 2 — the private package registry, and the SDK in it
|
||||||
|
|
||||||
|
**Why here.** ADR 0014 says a module consumes its dependencies, the SDK included, from the private
|
||||||
|
registry. Nothing installs one, so the SDK comes from a git URL (issue 053). Needs the installer
|
||||||
|
(Phase 0) and gitea.
|
||||||
|
|
||||||
|
- [ ] 2.1 gitea provides `package-registry` in full — the endpoints, an account the builder may
|
||||||
|
publish with
|
||||||
|
- [ ] 2.2 the builder publishes the SDK there on build, by version
|
||||||
|
- [ ] 2.3 modules consume it by version; the git URL and the sibling-path lock are gone
|
||||||
|
- [ ] 2.4 a second language's SDK published the same way, proving the path is not TypeScript-only
|
||||||
|
|
||||||
|
**Done when.** A module builds against the SDK resolved from the mesh's own registry, and issue 053
|
||||||
|
closes.
|
||||||
|
|
||||||
|
**Where it stands (2026-09-16).** The decision the bootstrap turned on is settled and recorded
|
||||||
|
([ADR 0076](../../02-DECISIONS/0076-the-sdk-is-a-published-package.md)): the SDK is a published
|
||||||
|
package, built on a public base image and published before the toolchain that consumes it; mesh-tools
|
||||||
|
stays the thin toolchain base but `npm ci`s the SDK by version. On the code, the builder now resolves
|
||||||
|
a package-registry credential — from a binding the mesh writes or from the environment for a hand-run
|
||||||
|
or bootstrap build — and injects it into an image build as a buildkit secret, never a layer, so a
|
||||||
|
token is not baked into the toolchain image. Unit-tested. Still ahead: gitea serving the registry in
|
||||||
|
full with a provisioner that mints tokens (2.1), the SDK built and published by the mesh (2.2), the
|
||||||
|
mesh-tools manifest flipped off the git URL to `npm ci` by version (2.3), and the genesis step that
|
||||||
|
raises gitea and publishes the SDK before the base build. Those close together in one lab run.
|
||||||
|
|
||||||
|
## Phase 3 — nothing is special after installation
|
||||||
|
|
||||||
|
**Why last.** The hardest and riskiest, and it needs everything above: an installer that completes,
|
||||||
|
a protocol that agrees, a registry to publish to. Issue 051.
|
||||||
|
|
||||||
|
- [x] 3.1 the store is adopted — raised at genesis, then held as the `postgres` module; one server,
|
||||||
|
the controller's records and the database of each module that asks for one. *(Adopted in place: the module's
|
||||||
|
server names the same container and the same pinned upstream image the foundation runs, so the
|
||||||
|
applier reconciles it rather than raising a second postgres. Proven 22/22 in the one-node lab —
|
||||||
|
the catalogue, a `postgres-database` consumer, gets its database from it. Follow-up: the store
|
||||||
|
binds `0.0.0.0` from genesis so a mesh consumer can reach it, but the packet filter is
|
||||||
|
installed later — a brief pre-filter window where `mesh-store` is open before `from: mesh`
|
||||||
|
clamps it; bring the filter up earlier or bind narrower at genesis.)*
|
||||||
|
- [x] 3.2 the broker is adopted — one `lavinmq`, the `/` vhost for the mesh bus, a vhost per
|
||||||
|
consumer that requires `amqp`; the second server gone. *(Adopted in place like the store; the
|
||||||
|
module names mesh-broker with the foundation's TLS spec, the provisioner runs host-networked
|
||||||
|
as lavinmq's default guest, amqp-ping reaches it. Proven 22/22.)*
|
||||||
|
- [x] 3.3 an upgrade of each, proven: a store the controller reads from, a broker over the
|
||||||
|
broker, each with a stated window. *(Lab steps S1/S2: move each module's source, build, roll
|
||||||
|
out, then a full push applies the server change and recreates the container. The store's
|
||||||
|
window is a pool reconnect; the broker's is longer — recreating the bus the push travels over,
|
||||||
|
so the mesh reconnects to the one that returns. Data survives on the named volumes.)*
|
||||||
|
- [x] 3.4 `status` can say the foundation is behind its source, which today it cannot form.
|
||||||
|
*(Given by the adoption: postgres/lavinmq are ordinary modules with a source now, so
|
||||||
|
`module list`/`status` reports them behind or current like any other — the question could
|
||||||
|
not form when they were bundle containers.)*
|
||||||
|
|
||||||
|
**Done when.** A mesh built from bare metal runs one postgres and one lavinmq, and can upgrade
|
||||||
|
either — so the twelve-module floor has no specialty left in it.
|
||||||
|
|
||||||
|
## The through-line
|
||||||
|
|
||||||
|
Each phase leaves the mesh more able to describe and rebuild itself, and each ends at a run rather
|
||||||
|
than a paragraph. The order is dependency, not preference: the installer carries everything, the
|
||||||
|
protocol is what everything speaks, the registry is what everything is built from, and adoption is
|
||||||
|
what makes the last two things ordinary.
|
||||||
|
|
||||||
|
## The one lab run
|
||||||
|
|
||||||
|
After Phases 1–3 are implemented and unit-tested and committed: rebuild the images, **verify each
|
||||||
|
carries its change**, and run the one-node installer once. All 22 checks green, twice, is the
|
||||||
|
acceptance test for the whole cycle — not a debugging loop, a proof that the assembled thing works.
|
||||||
@@ -15,8 +15,8 @@ document is written and this one's status becomes `implemented`.
|
|||||||
| [`03-scenario-lifecycle.md`](03-scenario-lifecycle.md) | What happens to a scenario — raise, snapshot, restore, move, destroy | [ADR 0016](../../02-DECISIONS/0016-the-lab.md) |
|
| [`03-scenario-lifecycle.md`](03-scenario-lifecycle.md) | What happens to a scenario — raise, snapshot, restore, move, destroy | [ADR 0016](../../02-DECISIONS/0016-the-lab.md) |
|
||||||
| [`04-lab-installation.md`](04-lab-installation.md) | Getting the lab onto a clean machine, and why it verifies capability rather than installation | [ADR 0010](../../02-DECISIONS/0010-delivery.md) |
|
| [`04-lab-installation.md`](04-lab-installation.md) | Getting the lab onto a clean machine, and why it verifies capability rather than installation | [ADR 0010](../../02-DECISIONS/0010-delivery.md) |
|
||||||
| [`05-the-node-host.md`](05-the-node-host.md) | Tier 0 — the one thing installed by hand, and the only thing that changes a machine | [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) |
|
| [`05-the-node-host.md`](05-the-node-host.md) | Tier 0 — the one thing installed by hand, and the only thing that changes a machine | [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) |
|
||||||
| [`06-the-control-plane.md`](06-the-control-plane.md) | Tier 2 — what the term means, and the test for what belongs in it | [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) |
|
| [`06-the-controller.md`](06-the-controller.md) | Tier 2 — what the term means, and the test for what belongs in it | [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) |
|
||||||
| [`07-the-substrate.md`](07-the-substrate.md) | Tier 1 — what the control plane consumes and cannot grant itself | [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md), [0048](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md) |
|
| [`07-the-foundation.md`](07-the-foundation.md) | Tier 1 — what the controller consumes and cannot grant itself | [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md), [0048](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md) |
|
||||||
| [`08-connectivity.md`](08-connectivity.md) | One context in full — overlay, resolution, exposure, filtering, certificates | [ADR 0007](../../02-DECISIONS/0007-connectivity.md), [0050](../../02-DECISIONS/0007-connectivity.md), [0051](../../02-DECISIONS/0004-a-node-and-how-it-joins.md), [0055](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md) |
|
| [`08-connectivity.md`](08-connectivity.md) | One context in full — overlay, resolution, exposure, filtering, certificates | [ADR 0007](../../02-DECISIONS/0007-connectivity.md), [0050](../../02-DECISIONS/0007-connectivity.md), [0051](../../02-DECISIONS/0004-a-node-and-how-it-joins.md), [0055](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md) |
|
||||||
| [`09-the-node-lifecycle.md`](09-the-node-lifecycle.md) | How a machine becomes a node, stays one, and stops being one | [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md), [0051](../../02-DECISIONS/0004-a-node-and-how-it-joins.md) |
|
| [`09-the-node-lifecycle.md`](09-the-node-lifecycle.md) | How a machine becomes a node, stays one, and stops being one | [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md), [0051](../../02-DECISIONS/0004-a-node-and-how-it-joins.md) |
|
||||||
| [`10-delivery.md`](10-delivery.md) | Modules, the three edges, and how a change becomes a running thing | [ADR 0010](../../02-DECISIONS/0010-delivery.md), [0064](../../02-DECISIONS/0009-modules-and-the-graph.md), [0065](../../02-DECISIONS/0009-modules-and-the-graph.md) |
|
| [`10-delivery.md`](10-delivery.md) | Modules, the three edges, and how a change becomes a running thing | [ADR 0010](../../02-DECISIONS/0010-delivery.md), [0064](../../02-DECISIONS/0009-modules-and-the-graph.md), [0065](../../02-DECISIONS/0009-modules-and-the-graph.md) |
|
||||||
@@ -27,6 +27,11 @@ document is written and this one's status becomes `implemented`.
|
|||||||
| [`15-the-agent-session.md`](15-the-agent-session.md) | One mechanism started twice — a node's session and the mesh's | [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md), [ADR 0026](../../02-DECISIONS/0026-the-mesh-has-a-session-of-its-own.md) |
|
| [`15-the-agent-session.md`](15-the-agent-session.md) | One mechanism started twice — a node's session and the mesh's | [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md), [ADR 0026](../../02-DECISIONS/0026-the-mesh-has-a-session-of-its-own.md) |
|
||||||
| [`16-module-coverage.md`](16-module-coverage.md) | What a module must be able to say, measured against 127 that exist | [ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md), [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) |
|
| [`16-module-coverage.md`](16-module-coverage.md) | What a module must be able to say, measured against 127 that exist | [ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md), [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) |
|
||||||
| [`17-raising-a-mesh.md`](17-raising-a-mesh.md) | How a mesh comes into existence, and how a machine joins one that exists | [ADR 0067](../../02-DECISIONS/0067-genesis-is-a-pivot.md), [ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md), [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) |
|
| [`17-raising-a-mesh.md`](17-raising-a-mesh.md) | How a mesh comes into existence, and how a machine joins one that exists | [ADR 0067](../../02-DECISIONS/0067-genesis-is-a-pivot.md), [ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md), [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) |
|
||||||
|
| [`18-building-a-module.md`](18-building-a-module.md) | How a build is modelled, and why a recipe that is always a Dockerfile does not fit what a module is | [ADR 0040](../../02-DECISIONS/0040-what-a-module-is.md), [ADR 0039](../../02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md), [ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md) |
|
||||||
|
| [`19-the-module-protocol.md`](19-the-module-protocol.md) | What a module's code and the mesh say to each other; an SDK is an implementation of it | [ADR 0074](../../02-DECISIONS/0074-the-wire-is-specified-not-the-types.md), [ADR 0042](../../02-DECISIONS/0042-the-shape-of-an-event-on-the-wire.md), [ADR 0043](../../02-DECISIONS/0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md) |
|
||||||
|
| [`20-writing-a-module.md`](20-writing-a-module.md) | A worked guide: one module, four capabilities, four languages, and the packages it publishes | [ADR 0074](../../02-DECISIONS/0074-the-wire-is-specified-not-the-types.md), [ADR 0040](../../02-DECISIONS/0040-what-a-module-is.md), [ADR 0039](../../02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md) |
|
||||||
|
| [`21-the-installation-in-full.md`](21-the-installation-in-full.md) | Every step from a bare machine to a mesh that maintains itself, and what is not yet true | [ADR 0067](../../02-DECISIONS/0067-genesis-is-a-pivot.md), [ADR 0073](../../02-DECISIONS/0073-the-installer-carries-a-builder.md), [ADR 0014](../../02-DECISIONS/0014-no-npm-workspace.md) |
|
||||||
|
| [`22-the-work-ahead.md`](22-the-work-ahead.md) | Everything decided and not yet built, in dependency order, each phase ending at a run | [ADR 0074](../../02-DECISIONS/0074-the-wire-is-specified-not-the-types.md), [ADR 0075](../../02-DECISIONS/0075-two-stores-and-which-provides-what.md), [ADR 0014](../../02-DECISIONS/0014-no-npm-workspace.md) |
|
||||||
|
|
||||||
## Not yet written
|
## Not yet written
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,80 @@
|
|||||||
|
---
|
||||||
|
status: open
|
||||||
|
opened: 2026-09-14
|
||||||
|
located-in: []
|
||||||
|
fixed-by:
|
||||||
|
amended-design:
|
||||||
|
---
|
||||||
|
|
||||||
|
# 048 — Nothing makes a machine trust the mesh's own registry
|
||||||
|
|
||||||
|
## Symptom
|
||||||
|
|
||||||
|
The registry module says, in its own manifest, that its port is open to the mesh because *every
|
||||||
|
machine pulls images and artifacts from here*. A machine that tries to do so is refused by its
|
||||||
|
container runtime before the mesh is involved at all: the registry serves plain HTTP, and a
|
||||||
|
container runtime treats any registry that is not loopback as HTTPS.
|
||||||
|
|
||||||
|
Nothing in the mesh arranges otherwise. There is no resource that configures a runtime's view of
|
||||||
|
the registry, no field in a manifest for it, and no step in joining that establishes it.
|
||||||
|
|
||||||
|
**It has never failed, and the reason it has never failed is the finding.** Every proof that a
|
||||||
|
machine can fetch a mesh-built artifact has been a proof about the machine that built it, where the
|
||||||
|
reference was loopback and loopback is trusted by default. The one bed that pushes to a routable
|
||||||
|
address gets away with it because *the lab* writes the runtime's configuration before the mesh is
|
||||||
|
raised, naming the documentation ranges its scenarios use. That file is the lab's, not the mesh's,
|
||||||
|
and no machine outside a bed has one.
|
||||||
|
|
||||||
|
Observed while fixing a neighbouring fault: the builder recorded artifacts under a loopback address
|
||||||
|
(fixed — the binding states where the provider is, and it now uses it). Correcting the address is
|
||||||
|
what makes this reachable, and therefore what makes this visible.
|
||||||
|
|
||||||
|
## Why this matters
|
||||||
|
|
||||||
|
**A stated rule is enforced by nothing.** The registry declares that the whole mesh pulls from it
|
||||||
|
and opens a port to the mesh accordingly. That is the design. Whether any machine can act on it
|
||||||
|
depends on a file the mesh does not write, does not read, and has no opinion about.
|
||||||
|
|
||||||
|
**It is the other half of [042](../042-nothing-gives-a-node-an-account-for-a-registry/00-report.md),
|
||||||
|
and the two are easy to confuse.** 042 is about a machine not being *allowed* to pull — no
|
||||||
|
credential. This is about a machine not being *able* to, regardless of credential, because the
|
||||||
|
transport is refused. A fix for 042 alone would leave a machine holding a valid account for a
|
||||||
|
registry its runtime will not talk to, which fails with an error about certificates and reads as a
|
||||||
|
credential problem.
|
||||||
|
|
||||||
|
**It decides something about the private network that has not been decided.** Serving artifacts
|
||||||
|
over plain HTTP is defensible if the private network is the boundary, and indefensible if it is
|
||||||
|
not — and either way it is a position, not an omission. Today it is an omission that happens to
|
||||||
|
work in one place.
|
||||||
|
|
||||||
|
**It is a joining problem before it is anything else.** The first machine never meets it. Every
|
||||||
|
machine after it does, on the first module the mesh built rather than fetched — which is most of
|
||||||
|
them.
|
||||||
|
|
||||||
|
## Evidence
|
||||||
|
|
||||||
|
A machine raised by the installer, asked what its runtime trusts:
|
||||||
|
|
||||||
|
```
|
||||||
|
Insecure Registries:
|
||||||
|
127.0.0.0/8
|
||||||
|
<the documentation ranges this lab uses>
|
||||||
|
::1/128
|
||||||
|
```
|
||||||
|
|
||||||
|
Everything but the loopback entry was written by the harness. The mesh contributed none of it.
|
||||||
|
|
||||||
|
## Open questions
|
||||||
|
|
||||||
|
- **Should the mesh's registry serve TLS?** It would make the transport a property of the registry
|
||||||
|
rather than of every machine that talks to it, and the mesh already issues certificates. What
|
||||||
|
signs it, and what a machine checks it against, are the real questions.
|
||||||
|
- **Or should a machine's runtime be configured by the mesh** — a resource that states what this
|
||||||
|
machine trusts, applied like any other? That puts a node-wide setting in a module's hands, which
|
||||||
|
may be the wrong ownership.
|
||||||
|
- **What is the boundary the answer assumes?** If plain HTTP inside the private network is
|
||||||
|
acceptable, that assumption should be recorded with what it takes for granted about who is on
|
||||||
|
that network — and checked, rather than inherited from a harness.
|
||||||
|
- **How does this interact with genesis?** The installer publishes before the mesh can grant or
|
||||||
|
configure anything, and gets away with loopback. Whatever is decided has to leave that moment
|
||||||
|
workable.
|
||||||
@@ -0,0 +1,72 @@
|
|||||||
|
---
|
||||||
|
status: open
|
||||||
|
opened: 2026-09-14
|
||||||
|
located-in: []
|
||||||
|
fixed-by:
|
||||||
|
amended-design:
|
||||||
|
---
|
||||||
|
|
||||||
|
# 049 — A module can serve tools, and nothing is allowed to call them
|
||||||
|
|
||||||
|
## Symptom
|
||||||
|
|
||||||
|
A module serves tools over the broker. Asking it one, from inside that same module's own container,
|
||||||
|
using its own credential, is refused:
|
||||||
|
|
||||||
|
```
|
||||||
|
Error: Channel closed by server: 403 (ACCESS-REFUSED) with message
|
||||||
|
"ACCESS_REFUSED - User '<node>-<module>' doesn't have permissions to queue 'amq.gen-...'"
|
||||||
|
```
|
||||||
|
|
||||||
|
A module's broker account is scoped to what it declares it emits and consumes. A tool call is a
|
||||||
|
request and a reply, and the reply arrives on a temporary queue the caller creates — which that
|
||||||
|
scope does not cover, and should not: a module that only publishes events has no business declaring
|
||||||
|
queues.
|
||||||
|
|
||||||
|
So the account is right and the request is reasonable, and there is no account that can make it.
|
||||||
|
|
||||||
|
## Why this matters
|
||||||
|
|
||||||
|
**A module's tools are its operator-facing surface, and nothing can reach it.** The catalogue serves
|
||||||
|
five: what this mesh holds, what a module is made of, what provides a given provision, what depends
|
||||||
|
on a module, and what is stale. Those are the questions the catalogue exists to answer, and today
|
||||||
|
the answer to "how does anyone ask one" is that they run a container joined to the broker's network
|
||||||
|
namespace and authenticate as the substrate's bootstrap admin.
|
||||||
|
|
||||||
|
**It is why a running module gets mistaken for a working one.** A test that cannot ask a module
|
||||||
|
anything checks that its container is up, and a container being up is not a claim about function.
|
||||||
|
That substitution has now hidden two separate faults in one week — a crash-looping runtime beside a
|
||||||
|
healthy container of a similar name, and a catalogue that was never installed at all.
|
||||||
|
|
||||||
|
**The mesh already has the shape for this.** An account scoped to a purpose, minted by the mesh and
|
||||||
|
sealed to a holder, is exactly what provisioning does. What is missing is the recognition that
|
||||||
|
*asking* is a use of a module, the way pulling is a use of the artifact store
|
||||||
|
([042](../042-nothing-gives-a-node-an-account-for-a-registry/00-report.md)) rather than something
|
||||||
|
that happens beneath it.
|
||||||
|
|
||||||
|
## Evidence
|
||||||
|
|
||||||
|
The refusal above, from a module invoking its own tool with the credential the mesh gave it. The
|
||||||
|
working alternative, which is the measure of the gap:
|
||||||
|
|
||||||
|
```
|
||||||
|
docker run --rm --network container:<the broker> \
|
||||||
|
-e MESH_BROKER_URL=amqp://<bootstrap admin>@127.0.0.1:<port>/ \
|
||||||
|
<the module's image> invoke <module> <tool> '{}'
|
||||||
|
```
|
||||||
|
|
||||||
|
That is the bootstrap credential, host-local, with no scope at all. It works, and nothing about it
|
||||||
|
should be how a mesh is asked a question.
|
||||||
|
|
||||||
|
## Open questions
|
||||||
|
|
||||||
|
- **Who is the caller?** An operator at a terminal, an agent acting for one, and another module are
|
||||||
|
three different holders with three different scopes, and only the last resembles anything the
|
||||||
|
mesh mints today.
|
||||||
|
- **Should calling be a grant like any other** — a module declares it may be asked, and a consumer
|
||||||
|
is issued an account that may create a reply queue and publish to that module's request queue?
|
||||||
|
- **Should the control plane be the way in?** It holds an admin connection already, and a
|
||||||
|
`mesh-control` subcommand for asking a module a question would need no new account — at the cost
|
||||||
|
of routing every question through one process and its privileges.
|
||||||
|
- **What does a module declare about being asked?** Serving a tool is already declared. Whether it
|
||||||
|
may be asked by anyone who can reach the broker, or only by named holders, is not.
|
||||||
@@ -0,0 +1,66 @@
|
|||||||
|
---
|
||||||
|
status: open
|
||||||
|
opened: 2026-09-14
|
||||||
|
located-in: []
|
||||||
|
fixed-by:
|
||||||
|
amended-design:
|
||||||
|
---
|
||||||
|
|
||||||
|
# 050 — The catalogue knows nothing that was built before it
|
||||||
|
|
||||||
|
## Symptom
|
||||||
|
|
||||||
|
A mesh raised from bare metal built six modules in order: the shared base, a store, the catalogue,
|
||||||
|
the control plane, a module, and a broker. Asked afterwards what it holds, the catalogue answered
|
||||||
|
with three — exactly the three built *after* it started running:
|
||||||
|
|
||||||
|
```
|
||||||
|
amqp-ping built from <the catalogue repository>
|
||||||
|
lavinmq built from <the catalogue repository>
|
||||||
|
mesh-control built from <the control plane repository>
|
||||||
|
```
|
||||||
|
|
||||||
|
Missing: the shared base, the store, and the catalogue itself.
|
||||||
|
|
||||||
|
The catalogue learns what exists by consuming the builder's announcement over the broker. Anything
|
||||||
|
built before it was running was announced to nobody, and nothing goes back for it.
|
||||||
|
|
||||||
|
## Why this matters
|
||||||
|
|
||||||
|
**The modules it cannot see are the ones a mesh is made of.** On a fresh mesh the things built
|
||||||
|
before the catalogue are, necessarily, the things the catalogue needed in order to exist: the base
|
||||||
|
it was compiled against and the store it keeps its records in. So the hole is not random — it is
|
||||||
|
always the foundation, on every mesh, at exactly the moment the graph is first populated.
|
||||||
|
|
||||||
|
**It breaks the question the catalogue exists to answer.** Build edges are derived facts between
|
||||||
|
versions, and the base is the node nearly every other module hangs off. A catalogue with no record
|
||||||
|
of the base cannot know that moving it makes everything standing on it stale, so `catalog_stale`
|
||||||
|
answers confidently and wrongly — the worst shape for a question about what to rebuild.
|
||||||
|
|
||||||
|
**Nothing reports the gap.** The catalogue does not know what it was not told, so it does not say
|
||||||
|
"three of six" — it says three, and a reader with no independent count believes it. This was found
|
||||||
|
by asking it a question and comparing the answer against what the mesh had just been watched doing.
|
||||||
|
|
||||||
|
**The record is not lost, only the catalogue's copy.** The control plane holds every build it
|
||||||
|
ordered, with commit, repository, path and resolved artifacts. So this is a gap between two stores
|
||||||
|
that should agree, rather than information nobody has.
|
||||||
|
|
||||||
|
## Open questions
|
||||||
|
|
||||||
|
- **Should the catalogue backfill on start** — ask the control plane for the builds it already
|
||||||
|
recorded and take them in? That fixes the fresh-mesh case and every case where the catalogue was
|
||||||
|
down while something was built, which is the same fault with a different cause.
|
||||||
|
- **Or should announcements be durable**, so a build announced while nothing was listening is
|
||||||
|
delivered when something is? That treats the catalogue as one consumer among several, which it
|
||||||
|
already is.
|
||||||
|
- **Or should the control plane be the record and the catalogue a projection of it?** Two stores
|
||||||
|
that must agree, kept in step by events, is the arrangement that produced this.
|
||||||
|
- **How would anyone notice next time?** The catalogue cannot compare itself to a source of truth
|
||||||
|
it does not have. Whatever is chosen, something has to be able to say "these disagree".
|
||||||
|
|
||||||
|
## How this would be checked
|
||||||
|
|
||||||
|
| Rule | Checked by |
|
||||||
|
|---|---|
|
||||||
|
| The catalogue holds every module the mesh built | A mesh is raised from bare metal and the catalogue is asked; its list is compared against the control plane's build records, and a module in one and not the other is a failure. |
|
||||||
|
| A change to the base makes what stands on it stale | The base is rebuilt on a fresh mesh and the catalogue names the modules that must follow — which requires it to know the base exists. |
|
||||||
@@ -0,0 +1,133 @@
|
|||||||
|
---
|
||||||
|
status: fixed
|
||||||
|
opened: 2026-09-14
|
||||||
|
located-in: [mesh-host, mesh-catalog]
|
||||||
|
fixed-by: mesh-host 56124c3; mesh-catalog 5e4dc37; mesh-lab 440e265
|
||||||
|
amended-design:
|
||||||
|
---
|
||||||
|
|
||||||
|
# 051 — The mesh can update everything except what it depends on
|
||||||
|
|
||||||
|
## Symptom
|
||||||
|
|
||||||
|
A mesh raised from bare metal runs its store and its broker from a bundle the installer wrote once,
|
||||||
|
at install time, with the images pinned in it. Nothing can change them afterwards. There is no
|
||||||
|
build for them, no version for them to be behind, no `upgrade … roll-out`, and no way for the mesh
|
||||||
|
to report that either is out of date — because the mesh holds no record of them as modules at all.
|
||||||
|
|
||||||
|
Every other thing on that machine has all of it.
|
||||||
|
|
||||||
|
Observed on a one-node mesh, where the duplication makes it plain — the same image, twice:
|
||||||
|
|
||||||
|
```
|
||||||
|
mesh-store <the postgres image> the control plane's own records
|
||||||
|
postgres <the same digest> the postgres module's server
|
||||||
|
mesh-postgres <mesh-built runtime> that module's provisioner and tools
|
||||||
|
```
|
||||||
|
|
||||||
|
One of those two servers can be rebuilt from source and rolled out. The other cannot, and it is the
|
||||||
|
one holding the mesh's inventory, identity and licences.
|
||||||
|
|
||||||
|
## Why this matters
|
||||||
|
|
||||||
|
**It is exactly backwards.** The store and the broker are the components every other thing depends
|
||||||
|
on, so they are the ones whose security updates matter most, and they are the only ones the mesh
|
||||||
|
has no mechanism to deliver. A release that reaches every module's database does not reach the one
|
||||||
|
the control plane keeps its own records in.
|
||||||
|
|
||||||
|
**It is invisible rather than reported.** `status` says what is behind its source. The substrate
|
||||||
|
cannot be behind anything, because the mesh does not know it exists as something with a source. So
|
||||||
|
a mesh with a year-old broker reports itself entirely current, which is worse than reporting a
|
||||||
|
problem.
|
||||||
|
|
||||||
|
**The pattern that would fix it already exists and is proven.** The control plane is carried in,
|
||||||
|
raised, and then adopted as an ordinary module pinned to the image that is running — the pivot in
|
||||||
|
[ADR 0067](../../02-DECISIONS/0067-genesis-is-a-pivot.md). Nothing about that pattern is specific
|
||||||
|
to the control plane. Applied to the store and the broker it would make the substrate a *moment*
|
||||||
|
rather than a *kind*: how a mesh starts, not what it permanently is.
|
||||||
|
|
||||||
|
**It is also why the same image runs twice.** A store that cannot be a module cannot provide
|
||||||
|
`postgres-database`, so a module wanting a database needs a second server. Adoption removes the
|
||||||
|
duplication as a side effect, and the control plane already reaches its three contexts through
|
||||||
|
three separate credentials — which is the shape of a consumer, not an owner.
|
||||||
|
|
||||||
|
**And the module it becomes is `postgres`, not `store`.** The naming rule settles it
|
||||||
|
([ADR 0040](../../02-DECISIONS/0040-what-a-module-is.md)): where a consumer speaks a protocol, the
|
||||||
|
interface *is* the protocol, and *"database" is not a capability*. The control plane's own queries
|
||||||
|
use `distinct on` and `on conflict`, so the coupling is to postgres and a `store` module would
|
||||||
|
promise a swap that fails the first time anybody tries it. The broker collapses the same way with a
|
||||||
|
different outcome — `amqp` **is** a protocol that several implementations speak, so `amqp` is a
|
||||||
|
legitimate provision and `lavinmq` is one provider of it.
|
||||||
|
|
||||||
|
So adoption is not only an upgrade path. It is two rows of a mesh's module list becoming one,
|
||||||
|
twice.
|
||||||
|
|
||||||
|
## And it is one SERVER each, not two
|
||||||
|
|
||||||
|
**The broker duplication is running today, not just latent.** The lavinmq module raises its own
|
||||||
|
`server` container and points its provisioner at `http://lavinmq:15672` — a second LavinMQ,
|
||||||
|
separate from the substrate's `mesh-broker`. A mesh with the module assigned runs both.
|
||||||
|
|
||||||
|
This is not how LavinMQ is meant to be used, and the module's own provisioner says so: a consumer
|
||||||
|
is given a *vhost* named for its login, isolated from every other consumer's by the vhost boundary
|
||||||
|
— *"the exact analog of postgres's database-per-login"*. One server hosts the mesh's own control
|
||||||
|
traffic on the `/` vhost and every consumer's broker as a vhost beside it. Two servers is the same
|
||||||
|
mistake as two postgres containers, wearing AMQP.
|
||||||
|
|
||||||
|
So adoption means the lavinmq module does not run a `server` of its own. Its server IS the
|
||||||
|
substrate broker, adopted; the module contributes the provisioner, the tools, the event consumer
|
||||||
|
and the run-once bootstrap that configures it — all against the one broker. Same for postgres: one
|
||||||
|
server, the control plane's records in their databases and every module's database beside them.
|
||||||
|
|
||||||
|
**The provisioner already assumes this** — it creates a vhost, not a broker — so the change is
|
||||||
|
removing the second server, not building a new isolation model. What has to be designed is only the
|
||||||
|
adoption itself: raising the one broker at genesis because nothing else can, then holding it as the
|
||||||
|
module, over the broker it is.
|
||||||
|
|
||||||
|
## What makes this harder than it looks
|
||||||
|
|
||||||
|
**The recursion is real, not incidental.** The control plane learns what modules exist by reading
|
||||||
|
its store. A store that is a module is a record inside the thing it is holding up. Adoption is what
|
||||||
|
resolves it — the store is raised by the installer because nothing else can, and only afterwards
|
||||||
|
becomes something the mesh has a record of — but the order of operations during an upgrade needs
|
||||||
|
stating, not assuming: a machine being sent a new store while the control plane is reading from it
|
||||||
|
is not a rollout, it is an outage.
|
||||||
|
|
||||||
|
**And the broker carries the rollout itself.** A declaration reaches a machine over the broker. A
|
||||||
|
broker upgrade is the mesh asking the machine to replace the thing the request arrived on.
|
||||||
|
|
||||||
|
## Open questions
|
||||||
|
|
||||||
|
- **Is adoption the right mechanism**, extending ADR 0067 to the substrate, or should the substrate
|
||||||
|
stay outside the module system and gain its own narrower update path?
|
||||||
|
- **What does a store upgrade look like** when the control plane is mid-read? Drain, quiesce, or a
|
||||||
|
window where the mesh accepts that it cannot be asked anything.
|
||||||
|
- **What does a broker upgrade look like** when the instruction travels over it? A machine that is
|
||||||
|
told to replace its broker has to complete the work without being able to report progress.
|
||||||
|
- **Does adoption also mean one server instead of two** by default, with a separate one for the
|
||||||
|
control plane as a choice for those who want the isolation — which is assigning a different
|
||||||
|
provider, a mechanism that already exists?
|
||||||
|
- **What reports it today?** Whatever is decided, `status` should be able to say the substrate is
|
||||||
|
behind. Today it cannot form the sentence.
|
||||||
|
|
||||||
|
## How this would be checked
|
||||||
|
|
||||||
|
| Rule | Checked by |
|
||||||
|
|---|---|
|
||||||
|
| The mesh can say its substrate is out of date | The store's source moves and `status` reports it behind, the way it does for any module. |
|
||||||
|
| The mesh can deliver a substrate update | A store or broker is upgraded on a running mesh and the control plane is answering afterwards, with its records intact. |
|
||||||
|
| Nothing runs twice without a reason | A mesh with one machine runs one postgres unless somebody asked for two. |
|
||||||
|
|
||||||
|
## Resolution
|
||||||
|
|
||||||
|
The foundation's store and broker are adopted in place as the ordinary `postgres` and `lavinmq`
|
||||||
|
modules (WBS Phase 3). Each module declares the container the foundation raised — the same name,
|
||||||
|
image, ports, volumes and args — so the applier reconciles it rather than raising a second server;
|
||||||
|
`mesh-host`'s `InstallStore` (phase3.go) carries in the genesis credentials the mesh cannot invent.
|
||||||
|
The servers bind mesh-wide so consumers reach them, and their provisioners run host-networked. An
|
||||||
|
upgrade of each is proven through its stated window — the store's a pool reconnect, the broker's the
|
||||||
|
harder case of recreating the bus the push travels over — and `status` now reports both as ordinary
|
||||||
|
modules that can be behind their source. A mesh built from bare metal runs one postgres and one
|
||||||
|
lavinmq, proven 22/22 in the one-node lab. Two follow-ups are tracked: [054](../054-the-adopted-store-and-broker-are-open-before-the-filter/00-report.md)
|
||||||
|
(the pre-filter exposure window) and [055](../055-the-adopted-store-and-broker-may-be-reachable-on-one-node-only/00-report.md)
|
||||||
|
(multi-node reachability).
|
||||||
@@ -0,0 +1,71 @@
|
|||||||
|
---
|
||||||
|
status: open
|
||||||
|
opened: 2026-09-14
|
||||||
|
located-in: []
|
||||||
|
fixed-by:
|
||||||
|
amended-design:
|
||||||
|
---
|
||||||
|
|
||||||
|
# 052 — The firewall closes the port the mesh runs on
|
||||||
|
|
||||||
|
## Symptom
|
||||||
|
|
||||||
|
A one-node mesh with the packet filter assigned generates this input chain:
|
||||||
|
|
||||||
|
```
|
||||||
|
policy drop
|
||||||
|
ct state established,related accept
|
||||||
|
iif "lo" accept
|
||||||
|
icmp / icmpv6 accept
|
||||||
|
<mesh address> tcp dport 22 accept ssh
|
||||||
|
<mesh address> tcp dport 5000 accept the registry
|
||||||
|
<mesh address> tcp dport 20000 accept
|
||||||
|
udp dport 51820 accept the overlay
|
||||||
|
```
|
||||||
|
|
||||||
|
**The substrate broker's port is not there.** Every machine dials it to enrol and to receive every
|
||||||
|
declaration it is ever sent. Nothing opens it.
|
||||||
|
|
||||||
|
The rules are generated from what modules declare they listen on. The broker is not a module — it
|
||||||
|
is raised by the installer from the bundle — so it declares nothing, and the generator has nothing
|
||||||
|
to generate from.
|
||||||
|
|
||||||
|
## Why this matters
|
||||||
|
|
||||||
|
**It is invisible on the mesh where it is first assembled and fatal on the next one.** A mesh of one
|
||||||
|
never dials its own broker across the network, so the missing rule changes nothing and the firewall
|
||||||
|
looks correct. The first machine that tries to join is refused at the packet filter, during
|
||||||
|
enrolment, before the mesh can report anything about it — and the cause is several steps from the
|
||||||
|
symptom.
|
||||||
|
|
||||||
|
**It would have been met during the migration, not before it.** The intended order is to raise the
|
||||||
|
anchor, assign its modules, then join the other machines. Assigning the firewall before the second
|
||||||
|
machine enrols is both the natural order and the one that breaks.
|
||||||
|
|
||||||
|
**It is [051](../051-the-mesh-cannot-update-what-it-depends-on/00-report.md) in a second place.** That
|
||||||
|
issue records that the substrate cannot be updated because the mesh holds no record of it. The same
|
||||||
|
absence means the firewall cannot know the substrate exists. Anything else generated from what
|
||||||
|
modules declare has the same hole: the store, the broker, and their ports are outside every such
|
||||||
|
computation.
|
||||||
|
|
||||||
|
**The store is the same shape and has not been checked.** It binds on the machine and is reached by
|
||||||
|
the control plane; whether that survives a default-drop policy has not been established here.
|
||||||
|
|
||||||
|
## Open questions
|
||||||
|
|
||||||
|
- **Should the substrate declare its listens** — which means the substrate being something the mesh
|
||||||
|
holds a record of, i.e. 051's adoption?
|
||||||
|
- **Or should the firewall have a floor** that is not derived from modules at all, the way ssh
|
||||||
|
already is? Ssh is in the rules for exactly this reason: a machine nobody can reach is a machine
|
||||||
|
nobody can repair, and that is not a thing any module declares. The broker is arguably the same
|
||||||
|
class of fact — the mesh cannot manage a machine it cannot talk to.
|
||||||
|
- **What else is in that class?** Whatever the answer, the question "which ports must be open for
|
||||||
|
the mesh itself to work" should be answerable from one place rather than assembled from modules
|
||||||
|
that happen to exist.
|
||||||
|
|
||||||
|
## How this would be checked
|
||||||
|
|
||||||
|
| Rule | Checked by |
|
||||||
|
|---|---|
|
||||||
|
| A machine with the firewall assigned can still be joined | The packet filter is assigned to an anchor, and a second machine enrols afterwards. |
|
||||||
|
| The mesh's own ports are open | The generated ruleset is compared against the substrate's own bindings, not only against module declarations. |
|
||||||
@@ -0,0 +1,76 @@
|
|||||||
|
---
|
||||||
|
status: open
|
||||||
|
opened: 2026-09-15
|
||||||
|
located-in: []
|
||||||
|
fixed-by:
|
||||||
|
amended-design:
|
||||||
|
---
|
||||||
|
|
||||||
|
# 053 — The SDK is pinned twice, and the two disagree
|
||||||
|
|
||||||
|
## Symptom
|
||||||
|
|
||||||
|
The module that carries the tool runtime names the SDK two ways, and they are not the same thing:
|
||||||
|
|
||||||
|
```
|
||||||
|
package.json @novox/mesh-sdk -> git+https://<the forge>/mesh-sdk.git#<a commit>
|
||||||
|
package-lock.json @novox/mesh-sdk -> ../mesh-sdk
|
||||||
|
```
|
||||||
|
|
||||||
|
The manifest names a commit in a repository any machine can reach. The lock names a **sibling
|
||||||
|
directory**, which exists on the workstation the lock was generated on and nowhere else.
|
||||||
|
|
||||||
|
It builds anyway, because the recipe runs `npm install` — which tolerates a lock that disagrees
|
||||||
|
with its manifest and re-resolves from the manifest. It is the one command that hides this.
|
||||||
|
|
||||||
|
## Why this matters
|
||||||
|
|
||||||
|
**A lock file exists to make a build reproducible, and this one describes one machine.** `npm ci` —
|
||||||
|
the command for exactly the case a lock is for — fails here, or worse, succeeds against whatever
|
||||||
|
happens to be at that path.
|
||||||
|
|
||||||
|
**It is the first thing a fresh mesh builds.** The toolchain image carries the SDK, and everything
|
||||||
|
with code of its own is compiled inside it. A dependency resolved differently on the build machine
|
||||||
|
than on a workstation is a difference in every module the mesh will ever build, arriving as a
|
||||||
|
compile error or a runtime mismatch far from here.
|
||||||
|
|
||||||
|
**And it is about to be copied.** Each language's toolchain will carry that language's SDK the same
|
||||||
|
way ([ADR 0074](../../02-DECISIONS/0074-the-wire-is-specified-not-the-types.md)). Whatever this
|
||||||
|
repository does, the Rust and Python ones will do, so the shape is worth getting right before there
|
||||||
|
are four of them.
|
||||||
|
|
||||||
|
## It is a violation, not an open question
|
||||||
|
|
||||||
|
[ADR 0014](../../02-DECISIONS/0014-no-npm-workspace.md) already decided this, and was accepted:
|
||||||
|
|
||||||
|
> Each module is an independent package that declares its dependencies and **consumes them from
|
||||||
|
> the private registry, including the mesh's own shared library.** A cross-package change is
|
||||||
|
> therefore two steps: publish the producer, then consume it.
|
||||||
|
|
||||||
|
So a git dependency at a pinned commit is not an alternative mechanism under consideration. It is
|
||||||
|
the mesh's own shared library being consumed by a means the record rules out, and the lock naming
|
||||||
|
a sibling directory is what that looks like when nobody publishes.
|
||||||
|
|
||||||
|
**Which makes the fix a direction rather than a discussion**: publish the SDK to the private
|
||||||
|
registry, consume it by version, and the lock stops being able to name a path that exists on one
|
||||||
|
machine.
|
||||||
|
|
||||||
|
## Open questions
|
||||||
|
|
||||||
|
- **Which private registry, and does the bootstrap have one?** The catalogue holds `verdaccio`;
|
||||||
|
a git host typically serves package registries too. ADR 0014 says *the* private registry as
|
||||||
|
though there is one, and today a fresh mesh has neither until something installs it — so the
|
||||||
|
first SDK build happens before the registry the record assumes.
|
||||||
|
- **Or should the SDK be a published package?** The catalogue holds a private registry module, and a
|
||||||
|
published package is how the rest of the world does this — at the cost of a mesh needing that
|
||||||
|
registry up before it can build anything, which is a bootstrap problem where there is currently
|
||||||
|
none.
|
||||||
|
- **What generates the lock, and on what?** A lock produced on a workstation with sibling checkouts
|
||||||
|
will keep saying this. A lock produced the way the image builds would not.
|
||||||
|
|
||||||
|
## How this would be checked
|
||||||
|
|
||||||
|
| Rule | Checked by |
|
||||||
|
|---|---|
|
||||||
|
| A build does not depend on the machine it runs on | The toolchain image builds with `npm ci` rather than `npm install`, which refuses a lock that disagrees with its manifest. |
|
||||||
|
| The SDK a module compiles against is the one named | The commit baked into the toolchain image is compared with the one the manifest pins. |
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
---
|
||||||
|
status: open
|
||||||
|
opened: 2026-09-16
|
||||||
|
located-in: []
|
||||||
|
fixed-by:
|
||||||
|
amended-design:
|
||||||
|
---
|
||||||
|
|
||||||
|
# 054 — The adopted store and broker are open before the packet filter exists
|
||||||
|
|
||||||
|
## Symptom
|
||||||
|
|
||||||
|
Adopting the store and broker as ordinary modules (issue 051) needs them reachable by
|
||||||
|
consumers across the mesh, so genesis now raises `mesh-store` and `mesh-broker` bound to
|
||||||
|
`0.0.0.0` rather than to loopback as the foundation used to. The packet filter is a module,
|
||||||
|
installed several steps after the store, the broker and the catalogue are already running.
|
||||||
|
|
||||||
|
Between each server coming up and the filter being applied, both listen on every interface
|
||||||
|
of the machine with only the genesis bootstrap credentials, and nothing drops traffic to
|
||||||
|
them. On a control-node that faces the network while it is being adopted, that is postgres
|
||||||
|
(bootstrap superuser) and a message broker open to anyone who can reach the machine, for the
|
||||||
|
length of the install.
|
||||||
|
|
||||||
|
## Why it matters
|
||||||
|
|
||||||
|
The design's rule is that what a port is reachable from is decided by the firewall, computed
|
||||||
|
from each module's `listens.from` — `mesh` for both of these. A rule enforced by nothing is
|
||||||
|
indistinguishable from a wrong one, and for the duration of this window that rule is enforced
|
||||||
|
by nothing: the thing that would apply it does not exist yet. It is the same window the ssh
|
||||||
|
rule already reasons about ("reached over the network, before the private network exists"),
|
||||||
|
but ssh is one guarded port and this is the mesh's whole store.
|
||||||
|
|
||||||
|
The foundation used to sidestep this by binding the store to loopback — only the co-located
|
||||||
|
control plane reached it — and adoption trades that away, because a module that adopts the
|
||||||
|
container in place must declare the same bind, and the module has to serve consumers.
|
||||||
|
|
||||||
|
## Open questions
|
||||||
|
|
||||||
|
- Can a default-deny base filter (drop everything but loopback, established, and ssh) be
|
||||||
|
applied at genesis, before the store and broker come up, and the mesh-scoped rules refined
|
||||||
|
once the private network has addresses to name? The consumers that must reach the store
|
||||||
|
(the catalogue) come up before the network step, so the `from: mesh` rule would have to be
|
||||||
|
in place by then.
|
||||||
|
- Or should the servers bind narrowly at genesis (loopback plus the container bridge) and
|
||||||
|
widen only once the filter that protects them exists — accepting that a bind change is a
|
||||||
|
recreate, so this would mean the store is recreated once during install?
|
||||||
|
- Is the window acceptable as-is, given the machine is mid-bootstrap and the exposure matches
|
||||||
|
what the pre-adoption `postgres`/`lavinmq` modules already had in steady state?
|
||||||
+40
@@ -0,0 +1,40 @@
|
|||||||
|
---
|
||||||
|
status: open
|
||||||
|
opened: 2026-09-16
|
||||||
|
located-in: []
|
||||||
|
fixed-by:
|
||||||
|
amended-design:
|
||||||
|
---
|
||||||
|
|
||||||
|
# 055 — The adopted store and broker may be reachable on the control-node only
|
||||||
|
|
||||||
|
## Symptom
|
||||||
|
|
||||||
|
The adopted `mesh-store` and `mesh-broker` bind `0.0.0.0` on the control-node. The
|
||||||
|
co-located provisioner and control plane reach them over loopback, and a co-located consumer
|
||||||
|
reaches them through the container bridge — which is what the one-node bed proves. A consumer
|
||||||
|
on ANOTHER machine reaches a provider by its `.internal` name over the private network, and
|
||||||
|
whether that path resolves to the control-node's bind is unproven: the one-node bed cannot
|
||||||
|
exercise it, and no multi-node bed installs the adopted store or broker.
|
||||||
|
|
||||||
|
## Why it matters
|
||||||
|
|
||||||
|
Phase 3's stated goal is a mesh — of any size — that runs one postgres and one lavinmq. The
|
||||||
|
whole point of a shared store and broker is that a module on any machine that is granted a
|
||||||
|
database or a vhost can open it. If the adopted servers are reachable only on the machine
|
||||||
|
they run on, a grant to a module placed elsewhere names an endpoint that machine cannot dial,
|
||||||
|
and the failure surfaces far from here as a consumer that cannot connect.
|
||||||
|
|
||||||
|
Before adoption this was a non-question: the store served the control plane alone and the
|
||||||
|
`postgres` module raised a second server that published mesh-wide. Collapsing to one server
|
||||||
|
means the one server has to be the mesh-wide one, reachable across the overlay.
|
||||||
|
|
||||||
|
## Open questions
|
||||||
|
|
||||||
|
- Does the mesh deliver the store and broker endpoint to a remote consumer as an address that
|
||||||
|
consumer can dial — the provider's overlay address — rather than a loopback or bridge
|
||||||
|
address meaningful only on the control-node?
|
||||||
|
- Is a `0.0.0.0` bind on the control-node reachable over the WireGuard overlay from a joined
|
||||||
|
machine, and is the packet filter's `from: mesh` rule enough to let it through?
|
||||||
|
- What is the smallest multi-node bed that would prove a database granted to a module on a
|
||||||
|
joined machine can be opened from there?
|
||||||
@@ -12,6 +12,12 @@ through them. Thin skills in `.claude/skills/` wrap these playbooks for invocati
|
|||||||
`hq-handoff`, `hq-sync-constitution`, `hq-status`); each defers to its playbook as
|
`hq-handoff`, `hq-sync-constitution`, `hq-status`); each defers to its playbook as
|
||||||
authoritative and adds only the mechanical scaffolding.
|
authoritative and adds only the mechanical scaffolding.
|
||||||
|
|
||||||
|
## Words
|
||||||
|
|
||||||
|
One name per thing. [`00-META/glossary.md`](00-META/glossary.md) is the authority on
|
||||||
|
vocabulary — *controller* (not "control plane"), *foundation* (not "substrate"), *node* and
|
||||||
|
*control-node*, *seat* / *bench* / *claim*, *package* vs *artifact*. Use those words.
|
||||||
|
|
||||||
## Ground rules
|
## Ground rules
|
||||||
|
|
||||||
- **Markdown only.** No new top-level folders without explicit confirmation.
|
- **Markdown only.** No new top-level folders without explicit confirmation.
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
# Novox HQ
|
# Novox HQ
|
||||||
|
|
||||||
The single source of truth for what Novox builds — what it **is**, what it is **becoming**,
|
The single source of truth for what Novox builds — what it **is**, what it is **becoming**,
|
||||||
and why. Today that is almost entirely **Novox Mesh**, the substrate everything else runs on. Implementation lives in `modules/`; the reasoning behind it lives here.
|
and why. Today that is almost entirely **Novox Mesh**, the foundation everything else runs on. Implementation lives in `modules/`; the reasoning behind it lives here.
|
||||||
|
|
||||||
## Structure
|
## Structure
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user