novox/hq
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

research 006: the mesh designed from nothing

Browse Source 
A skeleton laid out against the stated requirements rather than derived
from the current shape: four tiers, repositories at the root, and a
dependency rule that only points downward.

Four moves the current shape does not have.

The substrate is applied by the agent from a pinned bundle, not delivered
by the pipeline. That is the bootstrap circularity removed rather than
worked around — the first node is the ordinary path with no control plane
on the other end, which also makes it the cheapest lab scenario instead of
the one nobody exercises.

One agent binary with a detected capability profile — managed, user, edge.
A phone becomes a capability question rather than a platform question, so
it needs no second implementation. Modules declare which profiles they can
land on, and an impossible assignment fails at declaration.

Connectivity becomes a context. ADR 0015 names nine and none owns the
overlay, resolver, firewall or ingress, while research 005 measured
reachability as the only cluster in the catalogue that genuinely changes
together under one intent. Gap and evidence point the same way. That is an
addition to an accepted record, so it needs its own record and is not
written here.

Feature splits into artifact (built once per version) and part (selected
per node). The conflation of those two cardinalities under one word is
what makes the delivery pipeline hard to reason about.

Also makes explicit in how-we-build that the main-branch rule covers this
repository too. The rule already said 'without exception'; nothing was
amended, so nothing is recorded.
This commit is contained in:
jschoubben
2026-08-23 19:25:29 +02:00
parent 143f8ab2f1
commit b4365d8aa4
3 changed files with 282 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
01-RESEARCH/006-mesh-from-scratch/skeleton.md
+202
View File
@@ -0,0 +1,202 @@
---
effort: 006-mesh-from-scratch
updated: 2026-08-23
---
# The skeleton
Repositories at the root, modules inside them, parts at the leaf. Four tiers, and a dependency
rule that only points downward.
## The tree
```
hal-agent/ TIER 0 — the only thing ever installed by hand
apply/ reconcile declared state on this machine
inventory/ what this node is, has, and is capable of
link/ the single outbound connection to the control plane
store/ embedded local state — authoritative while disconnected
profile/ capability detection: managed · user · edge
substrate.lock pinned tier-1 descriptor, appliable with no mesh present
hal-substrate/ TIER 1 — declarations only, no logic of its own
store/ relational state
bus/ commands and events
objects/ blobs and build artifacts
images/ container images
identity/ the identity provider
bundle.yml the pinned set tier 0 can raise alone
hal-mesh/ TIER 2 — the control plane
record/ the event log every context integrates through
inventory/ nodes · modules · assignments · versions
config/ settings · secrets · derivation onto nodes
connectivity/ overlay · resolution · exposure · filtering · certificates
provisioning/ resource grants between modules
delivery/ source → artifact → node
observability/ health · logs · metrics · alerts
identity/ agents · humans · services · authorisation
work/ tasks · workflows · runs
knowledge/ memory · documents · retrieval
api/ the one interface every surface speaks to
hal-surfaces/ TIER 3 — thin; no logic lives here
tools/ the agent-facing tool surface
web/ the operator-facing interface
cli/ the shell-facing interface
hal-catalog/ TIER 4 — what the mesh hosts
<domain>/ grouped per ADR 0017, list per research 005
hal-lab/ the whole mesh, disposable, on one machine
hal-sdk/ contracts shared across tiers — types, not behaviour
hal-hq/ this repository
```
## The dependency rule
**A tier may depend only on tiers below it.** Substrate never references the control plane.
The control plane never reaches into a node except through the agent. A surface holds no logic
a second surface would have to reimplement.
This is the whole of the bootstrap answer, and per this repository's own rule it must say how
it is checked: a dependency-direction lint in the build, failing on an upward import. A tier
rule enforced by intention is the same as no tier rule — that is
[ADR 0008](../../02-DECISIONS/0008-a-failed-step-fails-the-job.md) applied to architecture.
## Move 1 — the substrate is applied, not delivered
**The problem.** The mesh needs a database, a bus, an object store, a registry and an identity
provider. Today those are modules, and modules are installed by the delivery pipeline, which
needs the database and the bus. The first node is therefore raised by a special script that
exists only because of the circularity, and every later change to the substrate has to pretend
the circularity is not there.
**The move.** The agent can apply a declaration without anyone telling it to. The substrate is
a **pinned bundle** the agent carries: a fixed, versioned, self-contained descriptor of the
five services and nothing else. Raising a first node is `agent apply substrate.lock` — not a
special path, just the ordinary one with no control plane on the other end.
The circularity disappears rather than being worked around: **the substrate is applied by tier
0, the mesh is delivered by tier 2, and they are different mechanisms on purpose.**
The price is real and should be named: the substrate is upgraded by bumping a pin and
re-applying, not by the pipeline. It gets less machinery than everything else — no per-node
selection, no provisioning, no fan-out — and that is the point. Five services justify a
simpler mechanism than a hundred.
## Move 2 — the agent is one binary with capability profiles
**The problem.** Everything assumes root on a machine whose packages, services and network the
mesh owns. A phone cannot offer that, and neither can a work laptop. The current answer would
be a lightweight fork, which means two implementations and one of them rotting.
**The move.** One agent, one binary, and a **profile** it detects rather than is told:
| Profile | Can | Typical |
|---|---|---|
| `managed` | packages, services, network, filesystem — the full surface | a machine the mesh owns |
| `user` | user-level services and tools; no package or network management | a shared or administered machine |
| `edge` | report presence, relay, expose a tool surface; hold nothing | a phone |
A module declares which profiles it can land on. Assignment to an incapable node fails at
declaration time, not at deploy time — a phone is not a machine that fails to install a
firewall; it is a node the firewall module cannot be assigned to.
This makes the phone case a **capability question rather than a platform question**, which is
what keeps it from becoming a second implementation. It also removes the current unstated
assumption that every node is equivalent — already false, and today handled by remembering.
## Move 3 — connectivity becomes a context
[ADR 0015](../../02-DECISIONS/0015-mesh-brokers-nodes-host-agents-think.md) names nine contexts
and none of them owns the overlay, the resolver, the firewall or the ingress. `config` owns
PKI, which is the closest thing, and it is not close.
Meanwhile [research 005](../005-domain-grouping/analysis.md) measured the whole catalogue and
found that reachability is the **only** place where modules genuinely change together under one
intent — the proxy with the resolver, the firewall with the overlay, repeatedly, because *how a
node is reachable* is one question asked in four places.
So the evidence and the gap point the same way. `connectivity` owns:
- the overlay every node joins, and the addresses on it
- name resolution, internal and public
- exposure — which services answer from outside, on which names
- filtering — what may reach a node at all
- certificates for both name spaces
This is an addition to an accepted record, so it is a decision, not a drafting choice. It
belongs in a new record that extends ADR 0015 the way
[ADR 0017](../../02-DECISIONS/0017-modules-outside-the-core-are-grouped-by-domain.md) does —
not written here.
## Move 4 — `feature` splits in two
The invitation was to check whether the concept survives. It does not, in one piece.
Today a **feature** means both *a thing built once* and *a thing selected per node*, and the
delivery pipeline is hard to reason about precisely because those have different cardinality
and one word ([ADR 0014](../../02-DECISIONS/0014-build-publish-and-deploy-are-three-silos.md)
is the pipeline half of the same confusion).
Split it:
| Concept | Is | Cardinality |
|---|---|---|
| **artifact** | something built and published — an image, a bundle, a package | once per module version |
| **part** | an independently selectable piece of a module's desired state | chosen per node, per assignment |
A module declares desired state in parts, and produces artifacts. An assignment names the node
and the parts. Delivery builds artifacts once and applies parts per node — and the two words
now carry the two cardinalities that the pipeline already has.
This keeps what features are genuinely for — per-node opt-in of *some* of a module, which the
work breakdown already calls for — and drops the conflation that makes the current model
confusing.
## What the mesh is, versus what it hosts
Tiers 0–3 are the mesh. Tier 4 is everything it carries, and the boundary is stated by
requirement rather than by taste: **a module is part of the mesh if removing it stops the mesh
managing nodes.** A media server does not. An identity provider does — which is why identity
sits in the substrate and not the catalogue, despite being, in every other respect, an
application like any other.
That test also settles the IT-company goal without a special category. Development, design and
deployment tooling are **workloads** — tier 4, hosted, provisioned, delivered like anything
else. The mesh does not grow a "company" feature; it hosts the tools a company runs on, and
the fact that it runs its own development on them is dogfooding, not architecture.
## What agents are, structurally
Self-improvement and self-healing are not a tier. Agents are participants
([ADR 0012](../../02-DECISIONS/0012-agents-are-persistent-employees.md)) that hold identity in
tier 2, act through tier 3 like any other caller, and run as workloads in tier 4.
This matters for one reason: **an agent must not have a privileged path**. Anything an agent
can do to the mesh, a person can do through the same surface, and anything it cannot express
through the tool surface is a gap in the surface rather than a reason for a back door. Self-
healing built on a private channel is unreviewable, and would be the one part of the mesh with
no human checkpoint.
## How this is tested
The lab ([ADR 0016](../../02-DECISIONS/0016-a-lab-node-is-a-virtual-machine.md)) raises the
tree above on one machine: virtual machines as nodes, a real overlay between them, the real
substrate bundle, the real control plane, the real delivery path.
The tier rule is what makes that affordable. A scenario needing only tiers 0 and 1 is one
virtual machine and a pinned bundle — which is also, exactly, the bootstrap path. **The
hardest thing to test becomes the cheapest scenario to run**, and the first-node path stops
being the one thing nobody exercises until it breaks.
## What this skeleton does not answer
- Where the record lives. It is infrastructure by shape and domain by content, and putting it
in the substrate risks recreating a circularity in the one place the design just removed one.
- Whether tier 2's contexts are one repository or several. Open from ADR 0015 already.
- Whether an `edge` node is in the inventory or merely present — which decides whether "node"
is one concept or two.
- The migration. Nothing here says how today's mesh becomes this, and the skeleton is worth
little until that is costed.