novox/hq
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

self-hosting, provisioning and delivery efforts, and the dotfiles origin

Browse Source 
The identity provider is settled as not-substrate: the mesh does not
require one, tier 2 authenticates natively, and it is a hosted service
like any other. Four substrate services, not five. The tier test's second
step gains the verb that matters — can the control plane START without it,
not function fully without it.

That verb answers the forge and the registries. They are not substrate and
they are not duplicated: the control plane starts and manages nodes
without a forge, it just cannot change itself. One gitea module, tier 4,
and the mesh's own instance is distinguished by what it is bound to rather
than by being a different module — the same answer as postgres, from the
same test. It also buys a property worth having: if the forge dies the
mesh keeps running.

Delivery needing them is not an upward dependency, resolved the way the
constitution already says to: tier 2 declares requirements, tier 4
provides implementations, the binding is data. The mechanism is
provisioning, and the new idea is that the control plane is itself a
consumer.

Self-hosting therefore becomes a state the mesh REACHES, not a
precondition. A first node comes up from pinned external artifacts and
re-binds to internal providers once they exist. Today's mesh assumes the
second state from the first moment, which is why the first-node path needs
a script that papers over an impossibility and is the least-exercised code
in the system. Made explicit, the transition is also reversible.

Research 007 and 008 opened for the two areas flagged as important and
complex, scoped from the weaknesses the as-is layer already documents
rather than started blank.

And the origin: this began as a dotfiles repository. The first two days
adopt dotfiles, add per-node overrides, and introduce service symlinking
with an ignore file. The flat one-directory-per-tool catalogue, linking
over copying, adoption of already-configured machines, per-node overrides
and the desktop modules are all inherited rather than chosen for a mesh.
That is the single most useful fact for anyone changing the catalogue, it
strengthens ADR 0018 — the case for links was never made for a mesh — and
it explains research 005's silent fifty: dotfiles-era entries for one tool
never shared a domain because they never had one.
This commit is contained in:
jschoubben
2026-08-23 20:55:14 +02:00
parent 7a20358113
commit daf3e17c32
5 changed files with 249 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
01-RESEARCH/006-mesh-from-scratch/00-overview.md
+11
View File
@@ -40,6 +40,17 @@ look unsolvable from inside that frame:
Designing from nothing is a way to find out which parts of the current shape are requirements
and which are residue.
**And there is more residue than expected, from a knowable source.** This began as a
**dotfiles repository** — the first two days of history adopt dotfiles, add per-node dotfile
overrides, and introduce service symlinking with an ignore file. The flat one-directory-per-tool
catalogue, linking rather than copying, adoption of already-configured machines, per-node
overrides, and the desktop modules are all inherited from that, not chosen for a mesh. Recorded
in [`03-DESIGN/00-as-is/10-module-catalogue.md`](../../03-DESIGN/00-as-is/10-module-catalogue.md).
That makes this effort's question sharper than "what would we do differently": much of what
looks like design is a generalisation of *place files on my machines*, never revisited because
it was never stated as an assumption.
## The requirements this is designed against
Stated by the operator, recorded here so the skeleton can be checked against them rather than
01-RESEARCH/006-mesh-from-scratch/code-skeleton.md
+98 -8
View File
@@ -15,7 +15,9 @@ A decision procedure, so placement is answerable rather than argued. Ask in orde
wins:
1. **Does it apply state on a machine?** → tier 0, inside the host.
2. **Can the control plane exist without it?** If *no* → tier 1, substrate.
2. **Can the control plane *start* without it?** If *no* → tier 1, substrate. Note the verb:
*start*, not *function fully*. A capability the control plane loses without something is not
the same as a thing it cannot come up without — see "becoming self-hosting" below.
3. **Does it decide what should be true across nodes?** → tier 2, a control-plane context.
4. **Is it a way to talk to tier 2, holding no logic of its own?** → tier 3, a surface.
5. **Otherwise** → tier 4, a workload.
@@ -50,13 +52,102 @@ inverse — substrate reaching into the catalogue for a module — would not be,
the naive "twice" answer would have created.
The same reasoning places the rest of the substrate: the bus, the object store, the image
registry. Each fails step 2, each lives once, each is pinned.
registry. Each fails step 2, each lives once, each is pinned. That is the whole substrate: four
services, and the deliberate absence of a fifth.
**One genuine boundary case, flagged rather than decided.** The identity provider passes step 2
only if the control plane delegates authentication rather than doing it natively. If tier 2
authenticates callers itself, the identity provider drops to tier 4 and the substrate has four
services instead of five. The test does not answer this; it turns it into a question with a
clear shape, which is what a test is for.
**The identity provider was raised as a boundary case and is settled: it is not substrate.**
The mesh does not require one — tier 2 authenticates its own callers natively, and an identity
provider is a service the mesh hosts like any other. The substrate is four services, not five.
The test earned its keep here by turning a vague unease into one answerable question — *does the
control plane delegate authentication?* — rather than a debate about how important identity
feels.
## Becoming self-hosting — the forge and the registries
Self-improvement means the mesh hosts the things it improves itself with: a forge, an image
registry, a package registry. The obvious worry is that these duplicate — a `hal-mesh-gitea`
for the mesh and a `gitea` for everyone else. **They do not, and the reason is worth stating
carefully, because it is the same reason the bootstrap keeps failing today.**
### They are not substrate
Run the test with the sharpened verb. *Can the control plane start without a forge?* **Yes.** It
comes up, holds inventory, answers questions and manages nodes with the modules it already has.
What it cannot do is **change itself**. That is a capability, not a precondition.
So: forge, image registry and package registry are **tier 4 workloads**. One module each, in the
catalogue, exactly like the media server. There is no mesh-specific copy.
This is not a technicality. It buys a property worth having: **if the forge dies, the mesh keeps
running.** Nodes stay managed, services stay up, only self-modification stops. Putting the forge
in the substrate would make losing it fatal, for no gain.
### But delivery needs them — is that not an upward dependency?
It would be, stated naively, and that would break the one rule the whole skeleton rests on.
It is resolved the way the constitution already says to resolve it — **depend on abstractions,
not on concrete dependencies**. Tier 2's delivery context does not require *the forge module*.
It declares requirements:
| Delivery requires | Satisfied by |
|---|---|
| a source of record for module code | whichever module provides it |
| somewhere to publish images | whichever module provides it |
| somewhere to publish packages | whichever module provides it |
| somewhere to put build artifacts | the substrate's object store |
Tier 2 defines the requirement; tier 4 provides the implementation; the binding is data. The
dependency points **downward from the provider to the interface**, which is legal, and the
control plane never names a concrete module.
The mechanism for this already exists and is the mesh's most valuable one: **provisioning**. A
module declares what it provides; a consumer declares what it requires; the mesh binds them.
The only new idea is that **the control plane is itself a consumer** — it has requirements, and
they are satisfied the same way a workload's are.
That generalisation is significant enough to need its own study, and is not settled here.
### Self-hosting is a state the mesh reaches, not a precondition
This is the part today's mesh gets wrong, and it explains a recurring class of pain.
A first node comes up from **pinned external artifacts** — upstream images, by digest, carried
in the bundle. It has to: the mesh's own registry does not exist yet, and cannot. The mesh at
this point is running and manages nodes, and is not yet self-hosting.
Self-hosting is then **reached**: the forge is installed as an ordinary workload, the mesh's own
source moves into it, the registries come up, and delivery's requirements are re-bound from
external providers to internal ones. From that point the mesh builds and deploys itself.
Stated as a lifecycle:
```
pinned external artifacts ─► mesh runs, manages nodes
│
│ forge + registries installed as workloads
│ delivery's requirements re-bound
▼
mesh builds and deploys itself
```
**Today's mesh assumes the second state from the first moment.** Its source, its packages and
its images are all expected to be self-hosted before there is anything to host them — which is
why raising a first node needs a script that exists solely to paper over the impossibility, and
why that script is the least-exercised path in the system.
Making the transition explicit has a second benefit: it is reversible. A mesh whose forge is
broken can re-bind delivery to external providers and keep improving itself while it repairs
the forge. Today that escape hatch does not exist, because the dependency is not expressed
anywhere it could be changed.
### So, concretely
One `gitea` module. One image-registry module. One package-registry module. Each a tier-4
workload. The mesh's own instances are distinguished from any other instance **by what they are
bound to, not by being different modules** — precisely the same answer as postgres, arrived at
by the same test.
## The five fates of a module
@@ -145,7 +236,6 @@ hal-substrate/ TIER 1
bus/<broker>/
objects/<object-store>/
images/<registry>/
identity/<idp>/ boundary case — see above
hal-mesh/ TIER 2
record/ the event log contexts integrate through