78a2274baff9ff1bb44162b07a3e6ce92bbe2c6e
15
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 jschoubben
|
1111bd84d7 |
Establish the repo for the completed Phase 0-3 build
Settles the design repository now that the self-upgrade build is on main: - Records the two decisions that shipped without a record — ADR 0077 (the controller/foundation/node vocabulary) and ADR 0078 (the store and broker are ordinary modules); accepts ADR 0075 and 0076, which shipped work rests on. - Fills issue 051's amended-design and wires ADR 0078 into 07-the-foundation. - Sweeps the repo rename (mesh-control -> mesh-controller) into the mutable docs now that the forge repo is renamed; updates the glossary note and repos.md. - Fixes the six broken links from the design-doc renames, indexes the glossary, regenerates the decisions reading order. Both checks (records.py, index.py) are green. Statuses stay honest: the build is on main and lab-proven but not deployed as the production mesh, so the to-be docs remain in-progress and the as-is layer (the hal mesh) is unchanged — graduation to implemented + as-is belongs to deployment, not merge. https://claude.ai/code/session_01D6qtiYU3P9jk3pnAXyAFyx |
||
|
jschoubben
|
333356cff3 |
Order the records the way the system is learned
Jochen asked whether the order made sense. It did not -- it followed when things happened to be decided, which after consolidation is fictional anyway since record 5 alone folds decisions taken across a week. Concretely wrong before: the domain statement sat at 8, after five engineering rules; the constitution was scattered across 5, 12 and 17; the tiers landed at 15, 16, 21 and 22 with process records in between. Now it walks: what the mesh is (1-3), its tiers from the bottom up (4-8), what runs on them and how it gets there (9-10), how it is built (11-16), how it is checked (17-18), how we work (19-23). Two things made this safe rather than free. It is a permutation, not a compaction, so the renames go through temporary names -- otherwise two files want one slot and one is lost. And the reference rewrite is a single simultaneous pass, because almost every number moved into a slot another number was vacating; replacing one at a time would have cascaded and pointed things at the wrong record while still resolving. Verified: 284 [ADR NNNN](path) links across the repository, all with matching text and target. The ordering principle is now stated in 19 rather than left implicit -- the repository already said "the numbering is the flow" about its folders, and there was no reason for the records to be the exception. |
||
|
jschoubben
|
e1febe8e0f |
Renumber the records 1 to 23
The consolidation left a sparse sequence -- 1, 4, 6, 7, 9, 10, 12, 15, 16, 18, 19, 25, 34, 35, 36, 37, 40, 42, 44, 45, 48, 49, 58 -- where the gaps were only the archaeology of what used to be there. Renumbered contiguously. Renames run in ascending order, so every target number is already free and no two files ever collide. The reference rewrite is one simultaneous pass rather than a sequence of replacements. Numbers moved into slots other numbers were vacating -- the node host went 37 to 16 while the lab went 16 to 9 -- so replacing one at a time would have cascaded and silently pointed things at the wrong record. Seven plain-text references survived the merges as prose rather than links, naming records that no longer existed: the enrolment token, the link boundary, what a declaration is, reachability, the repository structure. Each mapped to the consolidated record that now holds it. Verified rather than assumed: every [ADR NNNN](path) link now has matching text and target, checked across the whole repository, and the checker passes. Frontmatter `consolidates:` lists dropped -- they named records that are gone, and each consolidated record already says in prose what it absorbed. |
||
|
jschoubben
|
77f3a4cea7 |
Consolidate: 65 decision records to 23
Every remaining cluster merged. Each was one design that had been split across
several records because it was worked out over days rather than at once.
the node host 8 -> 1 applies not decides, depends on nothing,
per operating system, root service, the
launcher, episodic, what a declaration is,
actions from the bundle only
a node and how it joins 4 -> 1 what a node is, joining, the link as
security boundary, the enrolment token
modules and the graph 7 -> 1 everything is a module, no domain modules,
three edges, provisioning, the core library
substrate and control 6 -> 1 the test, seven contexts, one control plane,
plane the authority is not a database, the named
products, the pinned bundle
connectivity 3 -> 1 a route is a grant, reachability declared,
filter rules
delivery 5 -> 1 reconciliation not a pipeline, artifacts,
the three silos, a failed step, the verdict
the lab 5 -> 1 (earlier)
how this repository 10 -> 1 (earlier)
works
Nothing was dropped. Each consolidated record carries the reasoning of the ones
it absorbs -- the measurements, the incidents, the alternatives rejected --
because that reasoning is the only reason to keep a record at all. What is gone
is the fragmentation: eight files to read to understand tier 0, when tier 0 is
one component.
The four superseded records went too. They existed to point at their
successors, and the successors now contain what they said.
The checker made this safe. Each merge left dangling links -- 38 files after
the host merge alone -- and it named every one. Nothing was found by reading,
and a manual pass would certainly have missed some, including references inside
AGENTS.md which every session loads.
|
||
|
jschoubben
|
5e83ac2c22 |
Consolidate: 65 decision records to 52
Jochen: a normal application has 3-5 ADRs, maybe 10 for a large one, and we are at 65. Fair, and the cause is mine -- I recorded every FINDING as a decision rather than every fork in the road. Two merges, both cases where one decision had been split across many records because it was taken over several days rather than at once. 0019 absorbs ten records about how this repository works: what it is and that it is public, the folder flow, the two design layers, the issue front door, status in frontmatter, playbooks, the naming rule, the product name. Those were never ten decisions -- they were one, seen from ten angles as the repository took shape. 0016 absorbs the five about the lab: a node is a virtual machine, a router is scenery, a scenario declares the underlay, a scenario is a closed address space, and the two scenario classes. Same pattern -- one design, split by the order it was worked out in. The consolidated 0019 also raises the bar for what earns a record, since that is what produced 65: a record is warranted when there is a genuine fork -- a direction reversed, an alternative that will be proposed again, something contested. A finding is not a decision, and a bug is certainly not. Everything else belongs in the design document where the reasoning is actually read. The checker earned its place here. Deleting nine records left 13 dangling links across the repository and it named every one, including in AGENTS.md. Nothing was found by reading. Remaining clusters worth the same treatment: the host (8 records), delivery (5), modules (6), connectivity (4), substrate and control plane (4). That would be 52 down to roughly 30. |
||
|
jschoubben
|
60aea14935 |
Define the substrate, and answer 006's four-or-five conditionally
Same gap as the control plane: load-bearing and unpinned. The substrate is what the control plane CONSUMES AND CANNOT GRANT ITSELF. Every module needing a database asks provisioning for one; the control plane needs one too and cannot ask itself, because it is not running yet. That circularity is not an awkwardness to work around — it is the definition, and anything on the wrong side of it must be raised by the bundle the host carries. Which answers 006's open question in the honest form rather than with a number. The identity provider is substrate only if the control plane DELEGATES authentication — then it cannot serve anybody before the provider exists and cannot grant itself a client. If it authenticates natively, the provider is an ordinary hosted service. So the count follows from a decision not yet taken, and asserting four was asserting that decision. The test also rules out the tempting wrong answer: an identity provider, a mail server and an analytics service are all infrastructure by any ordinary reading, and none are substrate, because the control plane starts and runs without them. Important is not the test. Records why the bundle is pinned by hand — it is applied when no mesh exists, so nothing can resolve a version or ask a registry — and why it must be self-contained, which makes it an artifact built on a machine with a network for a machine that may have none. |
||
|
jschoubben
|
b9facf9375 |
Design the node host
Playbook 02 step 3, on four recorded decisions. Tier 0 has one job — apply declared state on this machine — and the six absorbed concerns are instances of it, not additions to it. Specifies the six parts and what each owns, and the two properties that make apply trustworthy rather than merely present: every applier reads back, because setting a value is not evidence the value took; and what was applied is recorded after it works, never before, because a failed apply leaves the machine wherever it reached and nothing must claim otherwise. Build order is staged so each stage is verifiable in the lab before the next exists. Stage 1 is profile and inventory — no control plane, no declarations, no network — and it is deliberately the smallest useful thing, because `place:` has nothing to place and the lab therefore raises empty machines. Stage 1 ends that, and every later stage is tested by a lab that already works. Stage 2 is the one that could invalidate the tier boundary: whether one host can raise the substrate alone is Move 1's assumption and has never been proved. Every decision the design rests on is given the test that asserts it, per 0034 — including the dependency-direction lint, which is what makes "the host never queries the mesh database" a rule rather than an intention. Six things left open and named, including the one that host-size.md could not measure: zero dependencies, but still six vocabularies. |
||
|
jschoubben
|
72b22830f3 |
ADRs 0036, 0037, 0038 — what a node is, what the host does, how one joins
0036 (accepted): a node is a managed machine, and disconnection is a situation. The open question posed a class distinction — full nodes and lesser presences. There is none. Reachability is state, not kind, which promotes the host's local store from a component to a requirement: it is what makes disconnection ordinary rather than exceptional. The reduced contract the question reached for is real but it is capability, and that belongs in the profile. 0037 (accepted): the host applies, it does not decide. Measured rather than argued — the absorption is smaller than the machinery that already applies state, and eight of ten adapters carry no dependency to move. The two that do open a Postgres connection to the control plane, which inside tier 0 is the one thing the tier rule exists to forbid. So each concern splits: deciding needs every other node and stays in tier 2; applying needs root and locality and goes to tier 0. The host carries ONE concern, of which the six are instances. 0038 (proposed): a node joins by linking first. The operator's two-modes proposal, adopted as intent and corrected as structure. Two modes is two code paths where the first runs once per mesh and rots — and the mesh already has that fault in its worst form, as three hand-run shell scripts. Instead: one behaviour, two sources of declaration. The first node is not a different kind of node, it is a node whose mesh is not up yet, and its specialness is temporary and self-erasing. 0038 also shrinks the migration 0037 called expensive: a joining node never needs mesh-wide state, because the hard part of the overlay is only needed to compute the WHOLE mesh. It needs one peer. The rest arrives. Left open and said so: what may be pushed over the link and how a joining node proves it is entitled to join, and whether one host can raise the substrate alone. |
||
|
jschoubben
|
42bce02bba |
006: answer the host-size question by measuring it
The skeleton's biggest unproven claim was that absorbing six concerns makes a binary whose whole argument is having no dependencies carry six of them. Measured against origin/main, and the question turns out to ask about the wrong axis. By size the absorption is SMALLER than the machinery that already applies state on a node — 2755 lines of adapters against 3059 lines of meshware, env-sync and config-sync. The host is not a new large thing; it already exists, spread across three core modules. The real risk is direction, and it is two modules wide rather than six concerns wide. Eight of ten adapters already receive derived state and only apply it, so absorbing them moves code that has no dependency to move. Two — wireguard and traefik — open a Postgres connection to the control plane and compute their own configuration, which inside tier 0 would be an upward dependency and is exactly what the tier rule forbids. And the split has already been happening without being named: dnsmasq-app needs the same node data as wireguard and does not query for it, because hand- duplicated state went wrong and someone derived it centrally instead. Eight of ten adapters are on the far side of that migration. So the absorption is not a move, it is a split: deciding stays in tier 2, applying goes to tier 0. The claim survives with its scope corrected — the host carries ONE concern, apply declared state on this machine, of which the six are instances. Stated open rather than glossed: the two unsplit modules are the two hardest, six concerns is still six vocabularies even at zero dependencies, and what the host must carry versus find is issue 007 and unresolved. Question B also recorded as answered by the operator — a node is a managed machine, and a disconnected node is still a node in a different situation. The question posed a class distinction; there is none, and what varies is state. |
||
|
jschoubben
|
09489a298c |
ADR 0030: the repository structure, and the rule that names them
The tiers were settled and the product was named, but the repositories themselves existed only in a research sketch. That had already caused two problems. ADR 0029 makes the lab phase 0 of the migration and could not say where it lives, because no record named a repository. And the sketch contradicted an accepted record: it listed mesh-hq while ADR 0028 had decided novox/hq and explicitly rejected that name. A design resting on research is resting on something that can change without a decision. Corrected in the research too. The naming rule, which both earlier records implied and neither stated: a repository belonging to a product carries that product's prefix; a company-scoped one does not. That is why this repository is hq and the mesh's are mesh-*. Seven repositories recorded — host, substrate, control, surfaces, sdk, lab, and this one. The lab gets its own: it ships to nobody, outlives any single tier, and drives virtualisation on a workstation, which nothing else does. Inside the host it would couple development tooling to a shipped component; inside the control plane the bootstrap scenario would depend on a tier that does not exist when it is needed. Tier 4 is deliberately not decided. Whether the catalogue is one repository, one per domain or one per application stays open from ADR 0015 and is blocked on research 005 — how many repositories hold domains cannot be answered before knowing what the domains are. mesh-catalog appears in the sketch and is not decided by this record. The cost is stated rather than glossed: seven release cadences where there is one, and cross-repository changes that used to be one commit. |
||
|
jschoubben
|
87f4f29cc6 |
Novox Mesh, Nox, and HQ becomes company-scoped
ADR 0027 — the product is Novox Mesh, shortened to mesh internally. HAL was never chosen: it arrived with the dotfiles repository this grew out of, it is borrowed, and it is borrowed from the canonical untrustworthy machine intelligence, which is an odd flag for infrastructure trusted with credentials. Timing is the substance of the decision, not an aside — the skeleton is not built, so renaming costs a search and replace now and a migration later. Nox is an identity of Novox, and specifically the agent of the MESH rather than of a node. Nodes keep their own identities. Nox addresses them, and a human mostly talks to Nox — which makes it the concrete form of the mission's vision: state an intent, and the mesh works out which node holds the thing. It holds no private channel. The gap this opens is recorded: ADR 0012 binds every agent to a home node, and a mesh-scoped agent has none, so the model needs extending. ADR 0028 — HQ is company-scoped, novox/hq, with the mesh as its first product. Checked rather than assumed: the company organisation already holds live projects that the mesh builds and deploys, so they are tenants rather than peers, and the mesh is the ground they stand on. There is also company work outside the mesh already, which strengthens the case and means the eventual split is closer than "some day" — so each document's scope is fixed now, in a table, making that split mechanical instead of archaeological. The folders are deliberately not restructured yet. The skeleton takes the new vocabulary: mesh-host, mesh-substrate, mesh-control, mesh-surfaces, mesh-catalog. Substrate drops to four services now that identity is a hosted workload rather than a dependency. Research 009 opens the migration, with the reframing that lowers its risk: replace the control plane, do not move the workloads. Their data never moves, so it is re-declared rather than adopted — which keeps adoption out of scope, as the lab design requires. Self-hosting is the last phase, or a failed cutover takes away the means to fix it. |
||
|
jschoubben
|
daf3e17c32 |
self-hosting, provisioning and delivery efforts, and the dotfiles origin
The identity provider is settled as not-substrate: the mesh does not require one, tier 2 authenticates natively, and it is a hosted service like any other. Four substrate services, not five. The tier test's second step gains the verb that matters — can the control plane START without it, not function fully without it. That verb answers the forge and the registries. They are not substrate and they are not duplicated: the control plane starts and manages nodes without a forge, it just cannot change itself. One gitea module, tier 4, and the mesh's own instance is distinguished by what it is bound to rather than by being a different module — the same answer as postgres, from the same test. It also buys a property worth having: if the forge dies the mesh keeps running. Delivery needing them is not an upward dependency, resolved the way the constitution already says to: tier 2 declares requirements, tier 4 provides implementations, the binding is data. The mechanism is provisioning, and the new idea is that the control plane is itself a consumer. Self-hosting therefore becomes a state the mesh REACHES, not a precondition. A first node comes up from pinned external artifacts and re-binds to internal providers once they exist. Today's mesh assumes the second state from the first moment, which is why the first-node path needs a script that papers over an impossibility and is the least-exercised code in the system. Made explicit, the transition is also reversible. Research 007 and 008 opened for the two areas flagged as important and complex, scoped from the weaknesses the as-is layer already documents rather than started blank. And the origin: this began as a dotfiles repository. The first two days adopt dotfiles, add per-node overrides, and introduce service symlinking with an ignore file. The flat one-directory-per-tool catalogue, linking over copying, adoption of already-configured machines, per-node overrides and the desktop modules are all inherited rather than chosen for a mesh. That is the single most useful fact for anyone changing the catalogue, it strengthens ADR 0018 — the case for links was never made for a mesh — and it explains research 005's silent fifty: dotfiles-era entries for one tool never shared a domain because they never had one. |
||
|
jschoubben
|
7a20358113 |
research 006: the code skeleton, and where postgres lands
A tier test as a decision procedure — five ordered questions, first match wins — so placement is answerable rather than argued. Postgres was the test case and the naive answer is wrong. Not twice, once: the control plane cannot exist without a relational store, so it is tier 1 and lives in hal-substrate/store/postgres. What differs between the mesh's own database and a project's is not the module but how that instance is brought up — pinned bundle applied by the host, versus the ordinary delivery and provisioning path. Tier is a property of the module; the bundle is a property of the mesh's own instance. The naive answer would also have made substrate reach up into the catalogue, which the dependency rule forbids. Working the test across the catalogue surfaces a third fate that neither of research 005's options covers, and it is the most common one: absorbed into the host, ceasing to be a module at all. That explains 005's one positive measurement rather than confirming it — the reachability cluster is not four modules that should be one domain module, it is four facets of one thing the host should own, expressed as modules because a module was the only unit available. Under this skeleton the overlay and firewall modules stop existing. It also partly answers the silent fifty: several are host concerns, so silence was the right signal and grouping was the wrong inference. Flags rather than settles: the identity provider is a genuine boundary case (four substrate services or five), and absorbing six concerns into a binary whose argument is that it has no dependencies is the skeleton's biggest unproven claim. |
||
|
jschoubben
|
00b8398d07 |
skeleton: hal-agent -> hal-host, and the two gaps the questions found
Agent is a first-class concept here — a participant, some of whom are human, holding identity and memory. Using it for the tier-0 node binary put both meanings in one document: hal-agent at tier 0, agents at tier 2. That is the anatomy-naming failure again, an evocative domain word pointing at infrastructure, and how-we-build 4 exists to catch it. ADR 0015's own title supplies the fix: the mesh brokers, nodes host, agents think. So the control plane brokers (hal-mesh), the tier-0 binary hosts (hal-host), the participant thinks (agents, untouched). hal-node was rejected — Node is the inventory aggregate, the binary is what runs on it. Recorded in the document as a near-miss rather than quietly corrected. Tiers now explained before the tree, as a boot narrative, with the point they were carrying made explicit: dependencies point only downward, that is the whole bootstrap answer, and the current mesh violates it — the database is a module, modules come from the pipeline, the pipeline needs the database. Connectivity gains the part that was missing. Naming a context says who decides, not who runs it: overlay membership is tier 0 in the host, policy is tier 2, machinery is tier 4 modules. And the host's link to the control plane deliberately does not run over the overlay, or the overlay would have to exist before a node could be told how to join it. 'tier' is this document's coinage and did not land on first reading; that is now an open question rather than settled vocabulary. |
||
|
jschoubben
|
b4365d8aa4 |
research 006: the mesh designed from nothing
A skeleton laid out against the stated requirements rather than derived from the current shape: four tiers, repositories at the root, and a dependency rule that only points downward. Four moves the current shape does not have. The substrate is applied by the agent from a pinned bundle, not delivered by the pipeline. That is the bootstrap circularity removed rather than worked around — the first node is the ordinary path with no control plane on the other end, which also makes it the cheapest lab scenario instead of the one nobody exercises. One agent binary with a detected capability profile — managed, user, edge. A phone becomes a capability question rather than a platform question, so it needs no second implementation. Modules declare which profiles they can land on, and an impossible assignment fails at declaration. Connectivity becomes a context. ADR 0015 names nine and none owns the overlay, resolver, firewall or ingress, while research 005 measured reachability as the only cluster in the catalogue that genuinely changes together under one intent. Gap and evidence point the same way. That is an addition to an accepted record, so it needs its own record and is not written here. Feature splits into artifact (built once per version) and part (selected per node). The conflation of those two cardinalities under one word is what makes the delivery pipeline hard to reason about. Also makes explicit in how-we-build that the main-branch rule covers this repository too. The rule already said 'without exception'; nothing was amended, so nothing is recorded. |