diff --git a/01-RESEARCH/005-domain-grouping/00-overview.md b/01-RESEARCH/005-domain-grouping/00-overview.md index ae54aae..7a12a48 100644 --- a/01-RESEARCH/005-domain-grouping/00-overview.md +++ b/01-RESEARCH/005-domain-grouping/00-overview.md @@ -58,3 +58,4 @@ open questions below. | Whether provider modules group at all, and if so what a consumer's requirement names instead of a module. | The provisioning reference is load-bearing; getting it wrong is expensive. Opinion and evidence in the analysis; not yet decided. | | Whether applications group into domains now and leave the monorepo later as a unit, or leave first. | Decided in principle — group first, then split — but the migration order has real cost either way. | | What to do with the ~50 modules that co-change with nothing. | The evidence gives no grouping signal for them at all. That may mean they are correctly sized already. | +| Whether "group or leave" is even the right pair of options. | [Research 006](../006-mesh-from-scratch/code-skeleton.md) finds a third fate — **absorbed into the node host**, ceasing to be a module at all — and argues it is the correct answer for the reachability cluster this effort measured. If so, the cluster this effort found is evidence for absorption rather than for grouping. | diff --git a/01-RESEARCH/006-mesh-from-scratch/00-overview.md b/01-RESEARCH/006-mesh-from-scratch/00-overview.md index 1a5877b..a8ccd05 100644 --- a/01-RESEARCH/006-mesh-from-scratch/00-overview.md +++ b/01-RESEARCH/006-mesh-from-scratch/00-overview.md @@ -17,7 +17,9 @@ What the mesh would look like if it were laid out today, with the requirements k of the accumulated shape — expressed as a **skeleton**: repositories at the root, modules inside them, and whatever turns out to be the right leaf unit below that. -The deliverable is [`skeleton.md`](skeleton.md). +The deliverables are [`skeleton.md`](skeleton.md) — tiers, repositories and the four design +moves — and [`code-skeleton.md`](code-skeleton.md) — the tier test, what a module looks like on +disk, and where today's catalogue lands. ## Why @@ -76,5 +78,6 @@ the catalogue where modules genuinely change together under one intent. The skel | Does the record — the event log contexts integrate through — belong to the substrate or the control plane? | It is infrastructure by shape and domain by content. Placing it wrong reintroduces a circularity. | | One repository per tier, or per context? | Already open from ADR 0015 as "catalogue destination — one repository or many". The skeleton assumes per tier and does not settle it. | | Does an unprivileged node earn a place in the inventory, or only a presence? | Decides whether "node" means one thing or two. | +| Does absorbing overlay, filtering, packages, supervision and the container runtime make the host too large? | It is the skeleton's biggest unproven claim. A binary whose whole argument is that it has no dependencies now carries six concerns. | +| Four substrate services or five? | The identity provider passes the tier test only if the control plane delegates authentication rather than doing it natively. | | Does `feature` survive? | The skeleton splits it in two and argues the conflation is what makes the delivery pipeline hard to reason about. Unproven. | -| Is `tier` the right word? | It is this document's coinage, not established vocabulary, and it did not land on first reading. `boot order` and `ring` are the alternatives. The concept is settled; the word is not. | diff --git a/01-RESEARCH/006-mesh-from-scratch/code-skeleton.md b/01-RESEARCH/006-mesh-from-scratch/code-skeleton.md new file mode 100644 index 0000000..97c46cf --- /dev/null +++ b/01-RESEARCH/006-mesh-from-scratch/code-skeleton.md @@ -0,0 +1,182 @@ +--- +effort: 006-mesh-from-scratch +updated: 2026-08-23 +--- + +# The code skeleton + +[`skeleton.md`](skeleton.md) laid out tiers and repositories. This is the level below: what a +module looks like on disk, and — the question that forced a correction — where a given piece of +today's catalogue actually lands. + +## The tier test + +A decision procedure, so placement is answerable rather than argued. Ask in order; first match +wins: + +1. **Does it apply state on a machine?** → tier 0, inside the host. +2. **Can the control plane exist without it?** If *no* → tier 1, substrate. +3. **Does it decide what should be true across nodes?** → tier 2, a control-plane context. +4. **Is it a way to talk to tier 2, holding no logic of its own?** → tier 3, a surface. +5. **Otherwise** → tier 4, a workload. + +## Worked example — where postgres ends up + +The obvious answer is "twice": once as the mesh's own database in the substrate, once as a +hosted database in the catalogue. That answer is wrong, and seeing why fixes something. + +Run the test. *Can the control plane exist without a relational store?* No. **Postgres is +tier 1.** It lives once: + +``` +hal-substrate/store/postgres/ +``` + +There is no second copy in the catalogue, because the thing that differs between the mesh's own +database and a project's database is **not the module**. It is how that instance is brought up: + +| | The mesh's own instance | A project's database | +|---|---|---| +| Brought up by | the host, from the pinned bundle, with no control plane present | the ordinary delivery and provisioning path | +| Declared in | `hal-substrate/bundle.yml` | the consuming module's `requires:` | +| Exists because | the control plane cannot start without it | something asked for it | + +Same module, two roles. **Tier is a property of the module** — what must exist before what — and +**the bundle is a property of the mesh's own instance.** + +This is also legal under the dependency rule, which is worth checking rather than assuming: a +tier-4 workload that requires a database depends on tier 1, which points *downward*. The +inverse — substrate reaching into the catalogue for a module — would not be, and is the shape +the naive "twice" answer would have created. + +The same reasoning places the rest of the substrate: the bus, the object store, the image +registry. Each fails step 2, each lives once, each is pinned. + +**One genuine boundary case, flagged rather than decided.** The identity provider passes step 2 +only if the control plane delegates authentication rather than doing it natively. If tier 2 +authenticates callers itself, the identity provider drops to tier 4 and the substrate has four +services instead of five. The test does not answer this; it turns it into a question with a +clear shape, which is what a test is for. + +## The five fates of a module + +Research 005 asked whether a catalogue module should be **grouped into a domain** or **leave +the repository**. Working the tier test across the catalogue surfaces a third answer that +neither option covers, and it is the most common one. + +| Fate | Means | Examples from today | +|---|---|---| +| **Absorbed into the host** | It is not a module at all. It is part of what "managing a machine" means, and belongs in tier 0. | overlay membership, packet filtering, package management, service supervision, container runtime, filesystem management | +| **Substrate** | The control plane cannot exist without it. Pinned, host-applied. | relational store, bus, object store, image registry | +| **Control-plane context** | It decides something across nodes. | connectivity policy, inventory, delivery, provisioning, observability | +| **Workload module** | The mesh hosts it. Grouped per [ADR 0017](../../02-DECISIONS/0017-modules-outside-the-core-are-grouped-by-domain.md). | media library, desktop session, collaboration tooling | +| **Leaves the repository** | A standalone application, per [ADR 0010](../../02-DECISIONS/0010-applications-live-in-their-own-repository.md). | the applications identified in research 005 | + +**The first fate is the finding.** Research 005 measured the reachability cluster — proxy, +resolver, firewall, overlay — as the only place in the catalogue where modules genuinely change +together under one intent. The skeleton explains *why*: they are not four modules that ought to +be one domain module. They are four facets of one thing the host should own, currently +expressed as modules because a module was the only unit available. + +Under this skeleton the overlay module and the firewall module **stop existing**. The host holds +membership and applies filtering; tier 2 decides the policy; the swappable backends stay +modules. That is a different and better answer than grouping them, and it was not visible from +inside the current frame. + +It also partly answers research 005's other open question — the fifty modules that co-change +with nothing. Several are host concerns rather than domains: package management, container +runtime, filesystem tooling. Silence was the right signal after all; the wrong conclusion was +that grouping was the only available fix. + +## What a module looks like on disk + +Per [`skeleton.md`](skeleton.md) Move 4, a module declares **parts** — independently selectable +pieces of desired state — and produces **artifacts** — things built once per version. + +``` +/ + module.yml identity, what it provides, what it requires, + which host profiles it can land on + parts/ + service/ desired state: container, volumes, exposure + provisioner/ how it grants its resource to consumers + migrations/ its own persistent state + tools/ capabilities it contributes + artifacts/ + / source for something built and published +``` + +A module with no artifacts consumes an upstream image and builds nothing. A module with no +parts is not a module. + +Worked through for the substrate's relational store: + +``` +hal-substrate/store/postgres/ + module.yml provides: database · profiles: [managed] + parts/ + service/ the container, its volume, its network exposure + provisioner/ grants a database and role to a consumer + migrations/ none — it holds no state of its own + artifacts/ none — upstream image, pinned by digest in bundle.yml +``` + +## The tree, at file level + +``` +hal-host/ TIER 0 + cmd/host/ + internal/ + apply/ reconcile declared state + inventory/ what this machine is and can do + link/ outbound connection to the control plane + overlay/ membership: address, keys, tunnel + filter/ packet filtering from tier-2 policy + packages/ package management + services/ supervision + containers/ container runtime + store/ embedded local state + profile/ managed · user · edge + substrate.lock pinned tier-1 descriptor + +hal-substrate/ TIER 1 + bundle.yml the pinned set, by digest + store/postgres/ + bus// + objects// + images// + identity// boundary case — see above + +hal-mesh/ TIER 2 + record/ the event log contexts integrate through + inventory/ nodes · modules · assignments · versions + config/ settings · secrets · derivation + connectivity/ addresses · resolution · exposure · filtering · certificates + provisioning/ grants between modules + delivery/ source → artifact → node + observability/ health · logs · metrics + identity/ agents · humans · services · authorisation + work/ tasks · workflows · runs + knowledge/ memory · documents · retrieval + api/ the one interface surfaces speak to + +hal-surfaces/ TIER 3 + tools/ web/ cli/ + +hal-catalog/ TIER 4 + // layout as above + +hal-lab/ hal-sdk/ hal-hq/ +``` + +## What this does not settle + +- **The identity boundary case.** Four substrate services or five. +- **Where the record lives.** Still the open question from + [`00-overview.md`](00-overview.md), and the tier test does not resolve it: the record is + needed by tier 2 and is *of* tier 2, which is exactly the shape that produces a circularity. +- **Whether absorbing into the host makes the host too large.** Six internal concerns is already + a lot for a binary whose whole argument is that it has no dependencies. The counter-argument + is that each is small and none can be optional — but this is the skeleton's biggest unproven + claim, and it should be tested by writing the host's interface before anything else. +- **The migration.** Nothing here says how today becomes this.