01-RESEARCH/006-mesh-from-scratch/00-overview.md

@@ -40,6 +40,17 @@ look unsolvable from inside that frame:
 Designing from nothing is a way to find out which parts of the current shape are requirements
 and which are residue.
 
+**And there is more residue than expected, from a knowable source.** This began as a
+**dotfiles repository** — the first two days of history adopt dotfiles, add per-node dotfile
+overrides, and introduce service symlinking with an ignore file. The flat one-directory-per-tool
+catalogue, linking rather than copying, adoption of already-configured machines, per-node
+overrides, and the desktop modules are all inherited from that, not chosen for a mesh. Recorded
+in [`03-DESIGN/00-as-is/10-module-catalogue.md`](../../03-DESIGN/00-as-is/10-module-catalogue.md).
+
+That makes this effort's question sharper than "what would we do differently": much of what
+looks like design is a generalisation of *place files on my machines*, never revisited because
+it was never stated as an assumption.
+
 ## The requirements this is designed against
 
 Stated by the operator, recorded here so the skeleton can be checked against them rather than

01-RESEARCH/006-mesh-from-scratch/code-skeleton.md

@@ -15,7 +15,9 @@ A decision procedure, so placement is answerable rather than argued. Ask in orde
 wins:
 
 1. **Does it apply state on a machine?** → tier 0, inside the host.
-2. **Can the control plane exist without it?** If *no* → tier 1, substrate.
+2. **Can the control plane *start* without it?** If *no* → tier 1, substrate. Note the verb:
+   *start*, not *function fully*. A capability the control plane loses without something is not
+   the same as a thing it cannot come up without — see "becoming self-hosting" below.
 3. **Does it decide what should be true across nodes?** → tier 2, a control-plane context.
 4. **Is it a way to talk to tier 2, holding no logic of its own?** → tier 3, a surface.
 5. **Otherwise** → tier 4, a workload.
@@ -50,13 +52,102 @@ inverse — substrate reaching into the catalogue for a module — would not be,
 the naive "twice" answer would have created.
 
 The same reasoning places the rest of the substrate: the bus, the object store, the image
-registry. Each fails step 2, each lives once, each is pinned.
+registry. Each fails step 2, each lives once, each is pinned. That is the whole substrate: four
+services, and the deliberate absence of a fifth.
 
-**One genuine boundary case, flagged rather than decided.** The identity provider passes step 2
-only if the control plane delegates authentication rather than doing it natively. If tier 2
-authenticates callers itself, the identity provider drops to tier 4 and the substrate has four
-services instead of five. The test does not answer this; it turns it into a question with a
-clear shape, which is what a test is for.
+**The identity provider was raised as a boundary case and is settled: it is not substrate.**
+The mesh does not require one — tier 2 authenticates its own callers natively, and an identity
+provider is a service the mesh hosts like any other. The substrate is four services, not five.
+
+The test earned its keep here by turning a vague unease into one answerable question — *does the
+control plane delegate authentication?* — rather than a debate about how important identity
+feels.
+
+## Becoming self-hosting — the forge and the registries
+
+Self-improvement means the mesh hosts the things it improves itself with: a forge, an image
+registry, a package registry. The obvious worry is that these duplicate — a `hal-mesh-gitea`
+for the mesh and a `gitea` for everyone else. **They do not, and the reason is worth stating
+carefully, because it is the same reason the bootstrap keeps failing today.**
+
+### They are not substrate
+
+Run the test with the sharpened verb. *Can the control plane start without a forge?* **Yes.** It
+comes up, holds inventory, answers questions and manages nodes with the modules it already has.
+What it cannot do is **change itself**. That is a capability, not a precondition.
+
+So: forge, image registry and package registry are **tier 4 workloads**. One module each, in the
+catalogue, exactly like the media server. There is no mesh-specific copy.
+
+This is not a technicality. It buys a property worth having: **if the forge dies, the mesh keeps
+running.** Nodes stay managed, services stay up, only self-modification stops. Putting the forge
+in the substrate would make losing it fatal, for no gain.
+
+### But delivery needs them — is that not an upward dependency?
+
+It would be, stated naively, and that would break the one rule the whole skeleton rests on.
+
+It is resolved the way the constitution already says to resolve it — **depend on abstractions,
+not on concrete dependencies**. Tier 2's delivery context does not require *the forge module*.
+It declares requirements:
+
+| Delivery requires | Satisfied by |
+|---|---|
+| a source of record for module code | whichever module provides it |
+| somewhere to publish images | whichever module provides it |
+| somewhere to publish packages | whichever module provides it |
+| somewhere to put build artifacts | the substrate's object store |
+
+Tier 2 defines the requirement; tier 4 provides the implementation; the binding is data. The
+dependency points **downward from the provider to the interface**, which is legal, and the
+control plane never names a concrete module.
+
+The mechanism for this already exists and is the mesh's most valuable one: **provisioning**. A
+module declares what it provides; a consumer declares what it requires; the mesh binds them.
+The only new idea is that **the control plane is itself a consumer** — it has requirements, and
+they are satisfied the same way a workload's are.
+
+That generalisation is significant enough to need its own study, and is not settled here.
+
+### Self-hosting is a state the mesh reaches, not a precondition
+
+This is the part today's mesh gets wrong, and it explains a recurring class of pain.
+
+A first node comes up from **pinned external artifacts** — upstream images, by digest, carried
+in the bundle. It has to: the mesh's own registry does not exist yet, and cannot. The mesh at
+this point is running and manages nodes, and is not yet self-hosting.
+
+Self-hosting is then **reached**: the forge is installed as an ordinary workload, the mesh's own
+source moves into it, the registries come up, and delivery's requirements are re-bound from
+external providers to internal ones. From that point the mesh builds and deploys itself.
+
+Stated as a lifecycle:
+
+```
+pinned external artifacts  ─►  mesh runs, manages nodes
+                                      │
+                                      │  forge + registries installed as workloads
+                                      │  delivery's requirements re-bound
+                                      ▼
+                               mesh builds and deploys itself
+```
+
+**Today's mesh assumes the second state from the first moment.** Its source, its packages and
+its images are all expected to be self-hosted before there is anything to host them — which is
+why raising a first node needs a script that exists solely to paper over the impossibility, and
+why that script is the least-exercised path in the system.
+
+Making the transition explicit has a second benefit: it is reversible. A mesh whose forge is
+broken can re-bind delivery to external providers and keep improving itself while it repairs
+the forge. Today that escape hatch does not exist, because the dependency is not expressed
+anywhere it could be changed.
+
+### So, concretely
+
+One `gitea` module. One image-registry module. One package-registry module. Each a tier-4
+workload. The mesh's own instances are distinguished from any other instance **by what they are
+bound to, not by being different modules** — precisely the same answer as postgres, arrived at
+by the same test.
 
 ## The five fates of a module
 
@@ -145,7 +236,6 @@ hal-substrate/                     TIER 1
     bus/<broker>/
     objects/<object-store>/
     images/<registry>/
-    identity/<idp>/                boundary case — see above
 
 hal-mesh/                          TIER 2
     record/                        the event log contexts integrate through

01-RESEARCH/007-provisioning-as-the-core/00-overview.md

@@ -0,0 +1,51 @@
+---
+status: active
+initiated: 2026-08-23
+touches:
+  - 03-DESIGN/00-as-is/03-provisioning.md
+  - 02-DECISIONS/0005-capabilities-are-provisioned-on-declaration.md
+  - 01-RESEARCH/006-mesh-from-scratch/code-skeleton.md
+became: []
+---
+
+# 007 — Provisioning as the mesh's core mechanism
+
+## What is being investigated
+
+Provisioning is the mechanism the whole mesh rests on: a module declares what it needs, and the
+mesh makes it exist, generates the credential, records the grant, and puts the values where the
+module will read them. [ADR 0005](../../02-DECISIONS/0005-capabilities-are-provisioned-on-declaration.md)
+calls it the mesh's core concern rather than its plumbing.
+
+[Research 006](../006-mesh-from-scratch/code-skeleton.md) then asks it to carry **more**: the
+control plane becomes a consumer with its own requirements — a source of record, an image
+registry, a package registry — satisfied by the same mechanism. That generalisation is only
+safe if the mechanism is sound, and the as-is record says it is not, in named ways.
+
+## Why now
+
+Four weaknesses are already documented in
+[`03-DESIGN/00-as-is/03-provisioning.md`](../../03-DESIGN/00-as-is/03-provisioning.md), each
+observed rather than theorised:
+
+- **Rotation has no fan-out.** A shared credential can be rotated without telling the peers
+  holding the old one. This has locked the mesh out of its own broker.
+- **A grant is not a check.** The record says a resource was provisioned. Nothing verifies it
+  still exists, still has that credential, or is reachable from where the consumer runs.
+- **A frozen password outlives its generation.** A generated secret written once diverges from a
+  persistent data directory initialised earlier, and presents as an authentication error.
+- **No requirements is indistinguishable from provisioning that did not run.** A module that
+  declares nothing skips the stage, which is correct, and looks identical to failure.
+
+Generalising a mechanism with these properties to the control plane's own dependencies would
+make each of them fatal rather than annoying.
+
+## The questions
+
+| Question | Why it matters |
+|---|---|
+| What does a **grant** mean, exactly — a record that a resource was created, or a claim about the world that is continuously reconciled? | The difference between the current model and one where "provisioned" is checkable. Almost every weakness above is a symptom of the first answer. |
+| How is a credential **rotated** with fan-out to every holder? | The mechanism grants easily and regrants not at all. This is the most damaging gap and it has taken the mesh down. |
+| Can the **control plane** hold requirements, and what satisfies them before anything is installed? | The generalisation research 006 needs. Ties directly to the self-hosting transition. |
+| What happens when a requirement **cannot** be satisfied — no provider, provider on an unreachable node, provider not yet installed? | Today this is silent or a stall. It should be a stated, visible state. |
+| Does a requirement belong to a **module** or to one of its **parts**? | Research 006 splits `feature` into artifact and part. A part-scoped requirement means a database is not provisioned where the part that needs it is not installed. |

01-RESEARCH/008-delivery-coordinator/00-overview.md

@@ -0,0 +1,53 @@
+---
+status: active
+initiated: 2026-08-23
+touches:
+  - 03-DESIGN/00-as-is/04-delivery.md
+  - 02-DECISIONS/0014-build-publish-and-deploy-are-three-silos.md
+  - 02-DECISIONS/0013-an-artifact-is-build-output.md
+  - 01-RESEARCH/006-mesh-from-scratch/code-skeleton.md
+became: []
+---
+
+# 008 — The coordinator: a change checked in becomes a deployed state
+
+## What is being investigated
+
+The mesh's own continuous delivery: a change is committed, and the mesh ends up in the state
+that change describes — across every node the change touches, with a verdict that says whether
+it worked.
+
+The coordinator is what orchestrates that, and it is the mesh's most consequential machinery:
+everything reaches every node through it.
+
+## Why now
+
+The as-is record ([`03-DESIGN/00-as-is/04-delivery.md`](../../03-DESIGN/00-as-is/04-delivery.md))
+names problems that are structural rather than incidental:
+
+- **A green pipeline proves transport, not effect.** The stages report that a message was
+  dispatched and accepted, which is not the same as the thing running, correct, or present.
+  This is the mesh's single most consistent failure shape.
+- **Detection is the most fragile input.** A merge that creates no pipeline, with nothing saying
+  so, is the characteristic bad outcome — and it has happened for reasons unrelated to the
+  change.
+- **The fan-out point is asymmetric.** The build node has already passed two silos when work
+  fans out, and code that knew only about the first parked it forever while every other node
+  deployed cleanly.
+- **There is no end-to-end coverage.** The harness has not built since 2026-06-04
+  ([`04-ISSUES/005`](../../04-ISSUES/005-pipeline-test-harness-unbuildable/00-report.md)).
+
+Research 006 adds a requirement the current design does not have: the coordinator must work
+**before the mesh is self-hosting**, when source and artifacts come from outside, and keep
+working across the transition to self-hosted providers.
+
+## The questions
+
+| Question | Why it matters |
+|---|---|
+| What is a **deployed state**, and how does the mesh know it is in one? | Everything follows from this. If a stage reports transport, "deployed" is a claim nobody checked. A desired-state model with reconciliation gives a different answer from a job-completion model. |
+| Does the coordinator dispatch **stages**, or converge nodes on a **declaration**? | The current model is a state machine over stages. The alternative is that a node is told what should be true and reports what is. The second makes drift visible; the first cannot see it. |
+| How does a change **become** a pipeline, reliably? | Detection has failed for reasons unrelated to the change, silently. |
+| What produces a **verdict**, and what is it a verdict about? | Ties to the lab ([ADR 0016](../../02-DECISIONS/0016-a-lab-node-is-a-virtual-machine.md)) and to a module carrying its own assertions. |
+| How does delivery work **before self-hosting**, and across the transition? | From research 006: source and artifacts start external and are re-bound to internal providers. The coordinator has to be indifferent to which. |
+| Does the **three-silo** split survive the artifact/part split? | [ADR 0014](../../02-DECISIONS/0014-build-publish-and-deploy-are-three-silos.md) is cardinality-driven, and research 006 renames the thing the cardinality is about. |

03-DESIGN/00-as-is/10-module-catalogue.md

@@ -62,6 +62,42 @@ addressed in principle by
 [ADR 0017](../../02-DECISIONS/0017-modules-outside-the-core-are-grouped-by-domain.md), which
 deliberately does not yet settle the domain list.
 
+## Where the shape came from
+
+The catalogue's shape is not arbitrary and it is not a series of mistakes. **This repository
+began as a dotfiles repository**, and most of what looks inexplicable is inherited from that
+directly.
+
+The evidence is in the first two days of history: *adopt desktop dotfiles and scripts from the
+original dotfiles repository*; *per-node dotfile overrides*; *service symlinking, node
+`.dfignore`, headless server support* — all on 2026-02-24 and 2026-02-25, before anything
+resembling a mesh existed. Forty-five commits mention dotfiles.
+
+Read that way, several things stop being puzzles:
+
+| Feature of today's shape | Dotfiles ancestor |
+|---|---|
+| One flat directory per piece of software | Exactly how a dotfiles repository is organised |
+| **Linking rather than copying** as a stated design principle | How every dotfiles manager works — the whole category is built on it |
+| **Adoption** — taking over a machine that already exists, with its own configuration | The core dotfiles verb, and the reason a machine could be brought in at all |
+| **Per-node overrides** | Per-host dotfile overrides, generalised into per-node module settings |
+| Desktop and workstation modules — browser, file manager, editors, session, audio, personal scripts | Dotfiles content that became "modules" when modules became the only unit |
+| An ignore file controlling what is placed on a machine | A dotfiles manager's ignore file |
+
+The generalisation from *place files on my machines* to *manage a mesh of nodes* was the right
+move and it worked. What it did not do is revisit the assumptions underneath, because they were
+never stated as assumptions — they were just how the thing already worked.
+
+**This is the most useful single fact for anyone changing the catalogue**, and it is why the
+linking principle in particular reads as a deliberate architectural choice when it is an
+inheritance. See [ADR 0018](../../02-DECISIONS/0018-the-mesh-creates-no-symlinks.md), whose case
+this strengthens: the argument for links was never made *for a mesh*.
+
+It also explains the measurement in
+[research 005](../../01-RESEARCH/005-domain-grouping/analysis.md). Fifty modules that never
+change alongside anything are not fifty missing domains — many are dotfiles-era entries for one
+piece of software, which never shared a domain because they never had one.
+
 ## Two properties worth keeping
 
 Whatever replaces the shape, two things about it are right.