Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 jochen
|
e710abd9b8 | ||
|
jochen
|
479b8fe72d | ||
|
jochen
|
e3a5862c5a |
@@ -23,13 +23,10 @@ and says they disagree.
|
||||
|
||||
## Amending the to-be layer
|
||||
|
||||
1. Propose the decision record in the knowledge base (`hq-graduate`, steps 2 and 3). If it reverses an
|
||||
earlier one, it `supersedes` it, or `changes` the clauses it replaces — the earlier record's text is
|
||||
never edited.
|
||||
2. Edit the design, add `"ADR NNNN"` to its `decisions:`, and set `updated:` to today.
|
||||
3. If this came from an issue, give the issue the document in the issue tracker: `amended-design` on its
|
||||
`issue-tracker.move` to `resolved`, or `issue-tracker.link` with kind `design` before then. Never in the
|
||||
report's frontmatter (ADR 0275).
|
||||
1. Write the decision record. If it reverses an earlier one, that record gets
|
||||
`status: superseded` and `superseded-by:` — its text is never edited.
|
||||
2. Edit the design; set `updated:` to today.
|
||||
3. If this came from an issue, set that issue's `amended-design:`.
|
||||
|
||||
## Amending the as-is layer
|
||||
|
||||
|
||||
@@ -1,44 +0,0 @@
|
||||
---
|
||||
name: hq-cleanup
|
||||
description: Use when the operator asks to clean up the forge and the working copies — stale or merged branches, old pull requests, finished worktrees, leftover folders, stashes. Triggers on "cleanup", "clean up our repos", "stale branches", "old worktrees", "tidy the forge". Reads first, deletes only what the operator approves, and reports what it cleaned, kept and left.
|
||||
---
|
||||
|
||||
# hq-cleanup
|
||||
|
||||
Runs playbook [09 — Cleanup](../../../00-META/process/09-cleanup.md); the playbook is authoritative. This skill
|
||||
adds the mechanics.
|
||||
|
||||
## 1. Take stock (one read-only agent, smaller model)
|
||||
|
||||
Brief it to change nothing and to write one report to the session's scratchpad:
|
||||
|
||||
- **Forge.** `gitea.gitea_list_repos`; per repository its default branch and **every branch from the git refs**
|
||||
(`gitea.gitea_api` GET `/repos/{owner}/{repo}/git/refs/heads`; the list tool misses branches, issue 949);
|
||||
`gitea.gitea_branch_protection_get`; open pull requests (`gitea.gitea_open_pulls`, titles and update times
|
||||
from `gitea.gitea_api` `/repos/{owner}/{repo}/pulls?state=open`). Per branch: contained in the default branch
|
||||
(`gitea.gitea_contains`, with the default branch's sha), patch-equal (`git cherry` in a local clone), tip date.
|
||||
- **Local.** Under the projects folder: each clone's `git worktree list --porcelain`, `git branch -vv`,
|
||||
`git branch --merged origin/<default>`, `git stash list`, `git status --porcelain` per worktree; folders and
|
||||
loose files beside the clones with `du -sh` and newest file. A worktree under another session's scratchpad is
|
||||
Kept.
|
||||
- A **Missing tools** section: each read it did without a mesh tool.
|
||||
|
||||
## 2. Classify and ask
|
||||
|
||||
Classes as playbook 09 step 2: Safe, Likely safe, Kept, Unknown. Tell the operator the counts and the report's
|
||||
place, and ask one question: delete the Safe list? Likely-safe and Unknown items are asked separately.
|
||||
|
||||
## 3. Delete the approved list only
|
||||
|
||||
- Forge branch: `gitea.gitea_delete_branch`.
|
||||
- Worktree: `git worktree remove <path>` (never `--force`), then `git worktree prune`.
|
||||
- Local branch: `git branch -d <name>` (never `-D`).
|
||||
Once design 58 milestone 7 is built (issue 884), each deletion is a call held as an ask: the operator approves
|
||||
the exact list on Telegram and the mesh carries it out on the warrant. Until then a session's permission check
|
||||
refuses these; hand the operator the approved list as a file, with the commands, and do not work around the
|
||||
refusal.
|
||||
|
||||
## 4. Report
|
||||
|
||||
Four lists: **cleaned** (with its evidence), **kept** (why), **unknown** (what it looks like), **the operator
|
||||
kept** (their word). Open each missing tool with `issue-tracker.open`, or note it on an existing issue.
|
||||
@@ -18,7 +18,7 @@ Record it as **one memory file** in Claude's persistent memory directory for thi
|
||||
path is given in the session's memory instructions), with `metadata.type: project`, plus a
|
||||
one-line pointer in `MEMORY.md`.
|
||||
|
||||
**Not** in `04-ISSUES`, the knowledge base, or anywhere else in the repository:
|
||||
**Not** in `04-ISSUES`, `01-RESEARCH` or anywhere else in the repository:
|
||||
|
||||
- A parked thought is not an issue or a research effort. Giving it a number asserts it has been
|
||||
triaged, which is exactly what deferring says has not happened.
|
||||
|
||||
@@ -23,19 +23,14 @@ result is not "nothing to learn" — it is the reason the next person repeats th
|
||||
|
||||
## Steps
|
||||
|
||||
1. Read the issue: `mesh_call issue-tracker.get` with its number, and `00-report.md` when it has a
|
||||
folder. Before you start, move it to `diagnosing` with `issue-tracker.move` and `by`, your name and node as
|
||||
who@node (such as `fix-399@laptop`). The tracker refuses it while another worker is current, naming them.
|
||||
2. Investigate. The trail goes in `issue-tracker.note`, **dated** in its text, including what was
|
||||
ruled out and how; for an issue with a folder, in `01-diagnosis.md` there as well. Archaeology —
|
||||
which commit, which pull request, which date a behaviour changed — is the most valuable content.
|
||||
3. When the owner is known, `issue-tracker.move` to `located` with `located-in`: entries of the
|
||||
tracker's registered list of owners (repositories, and modules as `mesh-catalog modules/<module>`);
|
||||
the tracker refuses an owner it does not know.
|
||||
4. Before you start the fix: `issue-tracker.move` to `fixing` with `by`.
|
||||
5. On resolution: `issue-tracker.move` to `resolved`, taken only from `fixing`, with `fixed-by`, and `replay` or `replay-none`
|
||||
for a core issue. If the root cause was a design gap, run `hq-graduate` for the amendment and give
|
||||
`amended-design`. Nothing of this is written in the report's frontmatter.
|
||||
1. Read `00-report.md`. Move `status:` to `diagnosing`.
|
||||
2. Investigate. Write `01-diagnosis.md` in the same folder: the trail, **dated**, including
|
||||
what was ruled out and how. Archaeology — which commit, which pull request, which date a
|
||||
behaviour changed — is the most valuable content here.
|
||||
3. When the owner is known, set `status: located` and fill `located-in:` with repositories or
|
||||
modules from [`00-META/repos.md`](../../../00-META/repos.md).
|
||||
4. On resolution: `status: resolved`, fill `fixed-by:`. If the root cause was a design gap, run
|
||||
`hq-graduate` for the amendment and fill `amended-design:`.
|
||||
|
||||
## Rules
|
||||
|
||||
|
||||
@@ -8,34 +8,25 @@ description: Use when a research effort in HQ concludes and becomes design, or w
|
||||
Closes a research effort into a decision and a design, or amends an existing design.
|
||||
**Authoritative playbook:**
|
||||
[`00-META/process/02-graduation.md`](../../../00-META/process/02-graduation.md) — read
|
||||
it; this skill adds only the scaffolding. Decision records and research efforts live whole in the
|
||||
knowledge base (seat `knowledge-base`, ADR 0290), reached through the mesh MCP server; nothing of them is
|
||||
written into this repository.
|
||||
it; this skill adds only the scaffolding.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Check against GENESIS** — `mission.md`, `context.md`, `effect.md`. If the conclusion
|
||||
conflicts, say which is wrong, in writing, before proceeding.
|
||||
2. **Reserve the record's number.** `mesh_call knowledge-base.reserve {"kind": "decision", "for": "<title>"}`.
|
||||
3. **Propose the decision record.** `mesh_call knowledge-base.propose` with the reserved `number`, `title`,
|
||||
`topic` (one of the six in [`02-DECISIONS/README.md`](../../../02-DECISIONS/README.md)), `date`,
|
||||
`deciders`, `clauses` (each `"**Lead.** the statement"`), `sections` (Context, Considered Options,
|
||||
Consequences, How it is checked, References), `links` (`extends`, `rests-on`, `supersedes`, `changes` with
|
||||
`"NNNN §N"`, `cites`), `homes` (the design's path), and `needs-operator: true` when the operator must
|
||||
approve it. A refusal names every rule broken: fix them all and propose again. The knowledge base
|
||||
reviews it itself; on `changes` revise and propose again with `why`. `knowledge-base.get` shows its
|
||||
status, reviews and the operator's ask.
|
||||
4. **Write the design** under `03-DESIGN/01-to-be/` with `layer: to-be`, `status: designed`,
|
||||
empty `code:`, today's `updated:`, and `decisions:` listing `"ADR NNNN"`.
|
||||
5. **Close the effort** — `knowledge-base.propose` the effort again, whole, with `status: graduated` and
|
||||
`became` links to the record (`"NNNN"`) and the design (its path).
|
||||
2. **Write the decision record** in `02-DECISIONS/`, next free number, frontmatter per
|
||||
[`02-DECISIONS/README.md`](../../../02-DECISIONS/README.md). `reconstructed: false` — this one is being taken,
|
||||
not recovered. Record the rejected options.
|
||||
3. **Write the design** under `03-DESIGN/01-to-be/` with `layer: to-be`, `status: designed`,
|
||||
empty `code:`, today's `updated:`, and `decisions:` pointing at the record.
|
||||
4. **Close the effort** — the research `00-overview.md` gets `status: graduated` and `became:`
|
||||
pointing at both.
|
||||
|
||||
## Amending an existing design
|
||||
|
||||
A design changes only through a decision. Propose the record first; if it reverses an earlier one,
|
||||
it `supersedes` it (or `changes` the clauses it replaces) — **never edit the earlier record's text**;
|
||||
a correction of fact is a progressive insight (`propose` the accepted record with `insight`). Then edit
|
||||
the design, add `"ADR NNNN"` to its `decisions:`, and set `updated:`.
|
||||
A design changes only through a decision. Write the record first; if it reverses an earlier
|
||||
one, set that record's `status: superseded` and `superseded-by:` — **never edit its text**.
|
||||
Then edit the design and set `updated:`.
|
||||
|
||||
## When something ships
|
||||
|
||||
@@ -51,4 +42,3 @@ cannot be checked, it is `in-progress`.
|
||||
- Do not move a to-be document into `00-as-is/`.
|
||||
- Do not edit an as-is document to describe an intention.
|
||||
- Do not change a decision record's meaning. Supersede it.
|
||||
- Do not write a decision record, or a copy of one, into this repository.
|
||||
|
||||
@@ -10,8 +10,8 @@ Hands a settled design to a code repository. **Authoritative playbook:**
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Confirm it is settled.** `status: designed`, and every claim traceable to a decision record in the knowledge base
|
||||
(`knowledge-base.get` for each entry of its `decisions:`). An open question in the text is a reason to run `hq-new-research`, not to build
|
||||
1. **Confirm it is settled.** `status: designed`, and every claim traceable to a record in
|
||||
`02-DECISIONS/`. An open question in the text is a reason to run `hq-new-research`, not to build
|
||||
around it.
|
||||
2. **Name the owner.** Set `code:` from
|
||||
[`00-META/repos.md`](../../../00-META/repos.md). If the repository does not exist yet,
|
||||
|
||||
@@ -21,46 +21,27 @@ Opens a numbered issue. **Authoritative playbook:**
|
||||
there, this is not an issue — it is a lookup. If the answer is a general lesson, it belongs in
|
||||
both.
|
||||
|
||||
## The number comes from the issue tracker
|
||||
|
||||
Every issue is opened in the issue tracker (seat `issue-tracker`, module `mesh-issues`, ADR 0275). **Never take a
|
||||
number by reading `main` or the open pull requests** — that is how numbers collided. If the tracker cannot be
|
||||
reached, take no number and say so.
|
||||
|
||||
Decide the shape first:
|
||||
|
||||
- **Design or governance issue** — its narrative belongs in hq: it gets a folder `04-ISSUES/NNN-short-name/`.
|
||||
- **Anything else** — it lives wholly in the tracker, and nothing is written in hq.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Load `mesh_call` and `mesh_describe` (deferred tools of the `mesh` MCP server) and open the issue:
|
||||
`mesh_call issue-tracker.open` with `title` (the symptom in one line), `report` (what was seen, what was expected,
|
||||
how it was found — the evidence: what was run, what came back, when), optionally `located-in` when the owner is
|
||||
already known, and — **only for an issue with a folder** — `record: short-name` (lower-case words joined by
|
||||
hyphens, no number). The answer carries the number `NNN`.
|
||||
2. **Issue wholly in the tracker:** done. Its trail goes in `issue-tracker.note`, its status changes through
|
||||
`issue-tracker.move`, its links through `issue-tracker.link`.
|
||||
3. **Issue with a folder:** on a branch off `origin/main`, create `04-ISSUES/NNN-short-name/00-report.md`, `NNN` being
|
||||
exactly the tracker's number and `short-name` exactly the `record` passed:
|
||||
1. Next free number. Create `04-ISSUES/NNN-short-name/00-report.md` with:
|
||||
|
||||
```yaml
|
||||
---
|
||||
status: open
|
||||
opened: YYYY-MM-DD
|
||||
located-in: []
|
||||
fixed-by:
|
||||
amended-design:
|
||||
---
|
||||
```
|
||||
|
||||
then `# NNN. The title`, the title given to the tracker. **No status key**: the issue's status, owner and fix
|
||||
live in the tracker (ADR 0275), and `cycle.py` refuses `status:`, `located-in:`, `fixed-by:`,
|
||||
`amended-design:`, `replay:` or `replay-none:` in any report. Notes and links go to the
|
||||
tracker.
|
||||
4. Write the symptom **as observed**, in plain terms, with the evidence that it happened.
|
||||
5. Say why it matters beyond the instance. An issue that is only one occurrence is a knowledge
|
||||
2. Write the symptom **as observed**, in plain terms, with the evidence that it happened —
|
||||
what was run, what came back, when.
|
||||
3. Say why it matters beyond the instance. An issue that is only one occurrence is a knowledge
|
||||
base entry.
|
||||
6. End with open questions rather than a proposed fix. Diagnosis is a separate step.
|
||||
4. End with open questions rather than a proposed fix. Diagnosis is a separate step.
|
||||
|
||||
## Do not
|
||||
|
||||
- Do not name nodes, domains, addresses or paths. This repository is public.
|
||||
- Do not guess the owner — `located-in:` is filled by diagnosis, not by opening.
|
||||
- Do not renumber a folder or reuse a number: a number never moves.
|
||||
|
||||
@@ -5,37 +5,38 @@ description: Use when starting a new research effort in HQ — an idea, technolo
|
||||
|
||||
# hq-new-research
|
||||
|
||||
Starts a new research effort in the knowledge base. **Authoritative playbook:**
|
||||
Scaffolds a new research effort. **Authoritative playbook:**
|
||||
[`00-META/process/01-research.md`](../../../00-META/process/01-research.md) — read it;
|
||||
this skill only does the mechanical setup. A research effort lives whole in the knowledge base (seat
|
||||
`knowledge-base`, ADR 0290); nothing of it is written into this repository.
|
||||
this skill only does the mechanical setup.
|
||||
|
||||
## Steps
|
||||
|
||||
All through the mesh MCP server (`mesh_describe` gives each verb's arguments).
|
||||
1. Find the next free number: list `01-RESEARCH/NNN-*`, take the highest plus one, zero-padded
|
||||
to three digits.
|
||||
2. Create `01-RESEARCH/NNN-descriptive-name/` (kebab-case from the topic).
|
||||
3. Create `00-overview.md` in it with exactly this frontmatter, then a prose summary of **what** is
|
||||
being investigated, **why**, and **what it touches**:
|
||||
|
||||
1. **Read what is known.** `mesh_call knowledge-base.ask {"question": "<the question in plain words>"}`;
|
||||
read an effort or record it names with `knowledge-base.get` (`kind: research` for an effort,
|
||||
`reasoning: true` for its documents).
|
||||
2. **Reserve the number.** `mesh_call knowledge-base.reserve {"kind": "research", "for": "<title>"}`. The
|
||||
number is spent; never pick one by hand.
|
||||
3. **Propose the effort.** `mesh_call knowledge-base.propose` with `kind: research`, the reserved `number`,
|
||||
`title`, `status: active`, `date` (today), `sections` (the documents in order, each `name` and `text`;
|
||||
the first `00-overview.md`: **what** is investigated, **why**, and **what it touches**), and `links`
|
||||
of kind `touches` for every record, effort, issue or design it touches (`cites` for one it only names).
|
||||
4. **Do the research** in further documents of the same effort. Each change is `propose` again with the
|
||||
effort whole and `why`. A refusal names every rule broken: fix them all and propose again. On review
|
||||
`changes`, revise and propose again; `knowledge-base.get` shows the reviews.
|
||||
```yaml
|
||||
---
|
||||
status: active
|
||||
initiated: YYYY-MM-DD
|
||||
touches: []
|
||||
became: []
|
||||
---
|
||||
```
|
||||
|
||||
4. Do the research in further documents in the same folder. Keep the summary current.
|
||||
|
||||
## Do not
|
||||
|
||||
- Do not restate the status in prose. It is the effort's `status`, in one place.
|
||||
- Do not give a `became` link while the effort is open — `hq-graduate` gives it at closure.
|
||||
- Do not restate the status in prose. It lives in frontmatter, in one place.
|
||||
- Do not fill `became:` while the effort is open — `hq-graduate` sets it at closure.
|
||||
- Do not skip or reuse a sequence number.
|
||||
- Do not write into `03-DESIGN` from an open effort.
|
||||
- Do not write the effort, or a copy of it, into this repository.
|
||||
- Do not name the mesh being observed. Evidence is required; identification is forbidden.
|
||||
|
||||
## Closing
|
||||
|
||||
An effort never just stops. It closes through `hq-graduate` as `graduated` or `abandoned`,
|
||||
always with a `became` link to what it turned into. Nothing is deleted.
|
||||
always with `became:` pointing at what it turned into. Nothing is deleted.
|
||||
|
||||
@@ -5,7 +5,7 @@ description: Use when you need a cross-cutting view of where HQ stands — resea
|
||||
|
||||
# hq-status
|
||||
|
||||
Generates a cross-cutting view **from frontmatter, the knowledge base and the issue tracker**. This is a read-and-render skill, not a
|
||||
Generates a cross-cutting view **from frontmatter**. This is a read-and-render skill, not a
|
||||
workflow — HQ has **no central status file by design** (decision 35). Every view is
|
||||
generated on demand and never written back to disk.
|
||||
|
||||
@@ -13,37 +13,29 @@ generated on demand and never written back to disk.
|
||||
|
||||
| Section | Files | Frontmatter |
|
||||
|---|---|---|
|
||||
| Research | the knowledge base, not a file: `knowledge-base.ask`, and `knowledge-base.get` with `kind: research` and a number | `status` (`active` / `graduated` / `abandoned`), `date`, links (`became`, `touches`, `cites`) |
|
||||
| Research | `01-RESEARCH/NNN-*/00-overview.md` | `status` (`active` / `graduated` / `abandoned`), `initiated`, `touches`, `became` |
|
||||
| Design | `03-DESIGN/**/*.md` (not READMEs) | `layer` (`as-is` / `to-be`), `status` (`designed` / `in-progress` / `implemented` / `abandoned`), `code`, `updated`, `decisions` |
|
||||
| Decisions | the knowledge base, not a file: `knowledge-base.holds` with a topic (the clauses in force, with their records), `knowledge-base.get` with a number, `knowledge-base.links` | `status` (`proposed` / `accepted` / `superseded`), `topic`, `date`, `deciders`, each clause's status, links both ways, what changed it since |
|
||||
| Issues | the issue tracker, not a file: `mesh_call issue-tracker.list` (optionally `status`, `owner`, `phrase`) and `issue-tracker.get` | `status` (`open` / `diagnosing` / `located` / `fixing` / `resolved` / `wontfix`), `worker` and `stale` while `diagnosing` or `fixing`, `opened`, `located-in`, `fixed-by`, `amended-design`, `record` (the folder here, when it has one) |
|
||||
| Decisions | `02-DECISIONS/NNNN-*.md` | `status` (`proposed` / `accepted` / `superseded`), `date`, `deciders`, `reconstructed`, `superseded-by`, `extends` |
|
||||
| Issues | `04-ISSUES/NNN-*/00-report.md` | `status` (`open` / `diagnosing` / `located` / `resolved` / `wontfix`), `opened`, `located-in`, `fixed-by`, `amended-design` |
|
||||
|
||||
## Steps
|
||||
|
||||
1. Read the designs' frontmatter (a grep across `03-DESIGN/` is enough, no need to load bodies), the
|
||||
issues from the tracker, and the records from the knowledge base. An issue report carries no status;
|
||||
where an issue stands is only in the tracker (ADR 0275). A decision record's or research effort's
|
||||
status is only in the knowledge base (ADR 0290).
|
||||
2. Render what was asked as Markdown tables. Group and sort sensibly. The **ADR index** by topic is
|
||||
`knowledge-base.holds` with each topic of the reading order `02-DECISIONS/README.md` writes, in that
|
||||
order, with a `bound` large enough for the topic (it says what it cut); it is the only place the list is
|
||||
shown (ADR 0248). The knowledge base has no verb that lists every record by number: a view by number
|
||||
is built from `get` per number, and says which numbers it read.
|
||||
1. Read the frontmatter block from every file above — a grep across each tree is enough, no
|
||||
need to load bodies.
|
||||
2. Render what was asked as Markdown tables. Group and sort sensibly. The **ADR index** is one
|
||||
of these views: records ordered by number, with title, date and status, and reconstructed
|
||||
ones marked.
|
||||
3. Flag anything inconsistent at the end, as flags — do not silently correct the render:
|
||||
- a `graduated` or `abandoned` effort with no `became` link
|
||||
- a `graduated` or `abandoned` effort with an empty `became:`
|
||||
- an `implemented` design with an empty `code:`
|
||||
- a `superseded` record nothing `supersedes`
|
||||
- a `superseded` record with no `superseded-by:`
|
||||
- an `open` or `diagnosing` issue with no activity
|
||||
- a to-be design whose `decisions:` names a record `knowledge-base.get` does not find, or one that is
|
||||
not `accepted`
|
||||
- a to-be design whose `decisions:` points at a record that does not exist
|
||||
- an as-is design whose claims are older than its `updated:` date suggests
|
||||
|
||||
## Do not
|
||||
|
||||
- **Do not write a status file.** Central status files are explicitly rejected. The view is
|
||||
always generated, always ephemeral. This includes the ADR index — the hand-written one had
|
||||
already drifted after a single addition, which is why it was removed, and the generated one
|
||||
written into `02-DECISIONS/README.md` made every two open decisions conflict (issue 297), which
|
||||
is why that went too. `index.py` fails if the list is stored there again.
|
||||
- Do not infer status from prose. Trust only a design's frontmatter, the tracker for an issue and the
|
||||
knowledge base for a record; if one is wrong, flag it.
|
||||
already drifted after a single addition, which is why it was removed.
|
||||
- Do not infer status from prose. Trust only the frontmatter; if it is wrong, flag it.
|
||||
|
||||
@@ -27,9 +27,7 @@ the claim "HQ is the source" is checked.
|
||||
3. Publish it to the knowledge base under the constitution slug, replacing the body.
|
||||
4. **Read it back and verify the change is present.** A publish that reported success and did
|
||||
nothing is exactly the failure class this repository exists to name — do not skip this.
|
||||
5. Note the sync in the decision record's Consequences (in the knowledge base: while the record is
|
||||
proposed, `knowledge-base.propose` it again with the note and `why`; once it is accepted, `propose` it with
|
||||
`insight`, saying what stood before).
|
||||
5. Note the sync in the decision record's Consequences.
|
||||
|
||||
## Do not
|
||||
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
__pycache__/
|
||||
+3
-3
@@ -35,7 +35,7 @@ comparing it against the code rather than by anyone noticing:
|
||||
|
||||
- It described the pipeline as having a separate builder process and a build stage that
|
||||
packages. Neither was true after 2026-08-04; the documents stayed stale until 2026-08-06
|
||||
(ADR 0010).
|
||||
([ADR 0010](../02-DECISIONS/0010-delivery.md)).
|
||||
- It listed the mesh as spanning a fixed number of named machines, which is exactly the
|
||||
content this repository cannot carry.
|
||||
|
||||
@@ -45,10 +45,10 @@ symlinks at all — the rule is not merely "only the installer may link", and a
|
||||
elevating linking to a principle points the opposite way from where this is going.
|
||||
|
||||
What exists today is that the installer owns and reconciles every link
|
||||
(ADR 0012) — an as-is fact, recorded in
|
||||
([ADR 0012](../02-DECISIONS/0012-the-mesh-creates-no-symlinks.md)) — an as-is fact, recorded in
|
||||
[`03-DESIGN/00-as-is/05-runtime-and-installation.md`](../03-DESIGN/00-as-is/05-runtime-and-installation.md).
|
||||
Centralising who may link narrowed the incident class; it did not close it. The intent is to
|
||||
remove the mechanism, recorded as ADR 0012.
|
||||
remove the mechanism, recorded as [ADR 0012](../02-DECISIONS/0012-the-mesh-creates-no-symlinks.md).
|
||||
|
||||
A founding document contradicting the direction of travel is precisely the failure this folder
|
||||
exists to prevent.
|
||||
|
||||
+28
-85
@@ -1,12 +1,9 @@
|
||||
# Checks
|
||||
|
||||
```
|
||||
python3 00-META/checks/records.py structure: links, no record written here, citations in the form "ADR NNNN"
|
||||
python3 00-META/checks/index.py the reading order's topics are written, and no list of records is stored
|
||||
python3 00-META/checks/cycle.py the development cycle: a design's layer, status, decisions and code; an issue report's frontmatter
|
||||
python3 00-META/checks/words.py the glossary's retired words are not used, no daemon's line is written, and no word is defined twice
|
||||
python3 00-META/checks/words.py --list tools the words the catalogue's copy must list
|
||||
python3 00-META/checks/words.py --list lines the daemons' lines the catalogue's copy must list
|
||||
python3 00-META/checks/records.py structure: links, citations, supersession, topics
|
||||
python3 00-META/checks/index.py the reading order in 02-DECISIONS/README.md is current
|
||||
python3 00-META/checks/index.py --write regenerate it
|
||||
```
|
||||
|
||||
Non-zero exit on any problem, so it can be a gate rather than a report.
|
||||
@@ -23,49 +20,29 @@ indistinguishable from one that cannot.
|
||||
| Check | Asserts | Found |
|
||||
|---|---|---|
|
||||
| `links` | every relative link resolves | — (run ad hoc during authoring; now permanent) |
|
||||
| `no-records` | `02-DECISIONS/` and `01-RESEARCH/` hold their README and nothing else: a decision record or research effort lives in the knowledge base (ADR 0290 rule 9) | written with the change that removed the records (issue 426); it failed on a record file put back in `02-DECISIONS/` before it passed |
|
||||
| `decisions-form` | every entry of a design's `decisions:` is "ADR NNNN", four digits | written with issue 426; it failed on an entry `ADR 290` before it passed |
|
||||
| `mention-form` | every "ADR N" in the text has four digits, and every "research N" three; a code span is an example and is not read | written with issue 426; it failed on `ADR 12` and `research 45` before it passed |
|
||||
| *(index.py)* | the reading order's six topics are written in `02-DECISIONS/README.md`, none twice, and the README stores no list of records (ADR 0248). The knowledge base reads these lines and refuses a record whose topic is not one of them | a stored list made every two open decisions conflict ([issue 297](../../04-ISSUES/297-two-open-decisions-always-conflict/00-report.md)); it failed on the README of `main` before the list was removed |
|
||||
| `rests-on` | `decisions:` and `extends:` name records that exist and are **accepted** | the class behind both incidents |
|
||||
| `live-citation` | a governing document citing a **superseded** record names its replacement in the same paragraph | `01-to-be/README.md` citing ADR 0022 as live guidance |
|
||||
| `supersession` | if A says it was superseded by B, B says it supersedes A | ADR 0012 never declared that it superseded 0011 |
|
||||
| `numbering` | the number in the filename is the number in the heading | — |
|
||||
| `topics` | every record names a topic the index knows | — |
|
||||
| *(index.py)* | the written reading order matches what the records say | — |
|
||||
| `status-vs-code` | a to-be document naming specific code is not still `designed` | **ten documents**, several with a *What was built* section, describing lab-proven code |
|
||||
|
||||
## Where the rules on records went
|
||||
|
||||
The decision records and research efforts left this repository for the knowledge base (ADR 0290, issue 426),
|
||||
and the rules these checks held over them left with them. The knowledge base holds each as a rule of its own,
|
||||
named in its refusal: `knowledge-base.propose` refuses a record that breaks one of the rules that judge one
|
||||
record, naming every reason at once, and the four rules that judge the whole record set run after every
|
||||
acceptance and raise a condition when one fails (mesh-knowledge's `rules.go`, one table-test case per rule).
|
||||
|
||||
| Rule here until issue 426 | Where it lives now |
|
||||
|---|---|
|
||||
| `records.py` `rests-on`, for a record's `extends:` | the knowledge base's `rests-on` at propose, and the whole-set rule `rests-on-live` |
|
||||
| `records.py` `rests-on`, for a design's `decisions:` naming an accepted record | **nowhere yet**: this repository cannot read the knowledge base. The reviewer checks each new entry with `knowledge-base.get`. Designs become the knowledge base's own kind under issue 428 |
|
||||
| `records.py` `live-citation`, in a record | the knowledge base's `live-citation` |
|
||||
| `records.py` `live-citation`, in `00-META/` and `03-DESIGN/01-to-be/` | **nowhere yet**, as above: the reviewer checks a cited record's status with `knowledge-base.get`; issue 428 |
|
||||
| `records.py` `supersession` | the knowledge base's `supersedes-accepted` and `self-link`, and the whole-set rule `supersession` |
|
||||
| `records.py` `partial-supersession` | the knowledge base's `changes` link to a clause, `changes-in-force` and `supersedes-accepted`, and the whole-set rule `partial` |
|
||||
| `records.py` `clauses` | the knowledge base's `has-clause`, `clause-numbering`, `clause-lead` and `clause-size` |
|
||||
| `records.py` `numbering` | the knowledge base's `title` and `reserved` |
|
||||
| `records.py` `topics`, `index.py` (a record's topic) | the knowledge base's `topic`, against the reading order `index.py` still checks |
|
||||
| `records.py` `insights` | the knowledge base's `insight-form`; an accepted record changes by a new version marked as a progressive insight |
|
||||
| `index.py --print` | `knowledge-base.holds` with a topic |
|
||||
| `cycle.py` research: a known status, `became:` when graduated | the knowledge base's `research-status`, `overview` and `became` |
|
||||
| `cycle.py` no two decision records share a number | the knowledge base's `reserved`, and the issue tracker's counter behind `knowledge-base.reserve` |
|
||||
| `cycle.py` an accepted decision cited by nothing | the knowledge base's whole-set rule `cited`: a record accepted there is cited by another record or names a home (`homes` at propose) |
|
||||
| `cycle.py` a deferral names the issue that tracks it | the knowledge base's `deferral` |
|
||||
| `words.py` over research efforts from 2026-10-07, and `kept` in `words-allowed.md` | the knowledge base's `retired-word` and `daemon-line`, from this glossary |
|
||||
|
||||
## What is deliberately not checked
|
||||
|
||||
- **Whether a record a document cites exists, or is still accepted.** A checkout cannot read the knowledge
|
||||
base, and a check that called the mesh would make every merge here depend on it. Until designs become the
|
||||
knowledge base's own kind (issue 428), the reviewer asks `knowledge-base.get` for each record a change
|
||||
cites in `00-META/` or `03-DESIGN/01-to-be/` (the documents that tell somebody what to do). An as-is
|
||||
document may rest on a superseded record: it describes what runs, and what runs was built under whatever
|
||||
was decided at the time (ADR 0006: *as-is describing a superseded decision is exactly what as-is is for*).
|
||||
- **Whether a citation's prose is still true.** A document can cite an accepted record and describe it
|
||||
wrongly, and nothing here notices.
|
||||
- **`02-DECISIONS/` and `01-RESEARCH/` may cite superseded records freely.** A decision record
|
||||
discusses history; research records what was observed. Flagging those would produce noise on
|
||||
correct documents, and a check that cries wolf gets suppressed — which costs more than not
|
||||
having it.
|
||||
- **`03-DESIGN/00-as-is/` may rest on a superseded record.** It describes what runs, and what
|
||||
runs was built under whatever was decided at the time
|
||||
([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md):
|
||||
*as-is describing a superseded decision is exactly what as-is is for*).
|
||||
- **Whether a citation's prose is still true.** Only whether the record it points at is live.
|
||||
A document can cite an accepted record and describe it wrongly, and nothing here notices.
|
||||
|
||||
So "governing" means `00-META/` and `03-DESIGN/01-to-be/` — the documents that tell somebody
|
||||
what to do.
|
||||
|
||||
## Adding a check
|
||||
|
||||
@@ -74,43 +51,9 @@ whose failure has never been observed is a guess about its own correctness.
|
||||
|
||||
## cycle.py
|
||||
|
||||
The development cycle, checked (ADR 0080):
|
||||
every design parses, its `layer:` matches its folder and its `status:` is known; a to-be design names a
|
||||
decision in `decisions:`; an in-progress or implemented design names its owning code; and no two issue
|
||||
records share a number (issue 155 — the number is how a record is cited, and `main` lags every open pull
|
||||
request, so two people reading it allocate the same one). An issue report carries the day it was `opened:`
|
||||
and no status key — `status:`, `located-in:`, `fixed-by:`, `amended-design:`, `replay:`, `replay-none:` or
|
||||
`resolved:` — because an issue's status lives in the issue tracker (ADR 0275); it failed on all 361 reports
|
||||
that carried them before they were removed. The rules on an issue's status this check held until then — an
|
||||
owner for a located issue, a fix for a resolved one, and a core issue's `replay` or `replay-none` (ADR 0237)
|
||||
— are refused at the write by `issue-tracker.move`. Its rules on research efforts and decision records left
|
||||
with them (the table above). `python3 00-META/checks/cycle.py`
|
||||
|
||||
## words.py
|
||||
|
||||
One word per thing, checked (ADR 0244).
|
||||
It reads the glossary's *Not:* lines (the retired words, each with its scope) and *Identifier until
|
||||
renamed* lines, and fails on a retired word or a bare identifier in running prose — what is left once code,
|
||||
quotations, struck-through text, link targets, comments and frontmatter are taken out — in `00-META/`,
|
||||
`03-DESIGN/`, `AGENTS.md`, `README.md`, and issues opened from 2026-10-07. Decision records and research
|
||||
efforts live in the knowledge base, which holds a proposed record to the same words (the table above). It
|
||||
also fails when a glossary head word heads two entries or is also retired.
|
||||
[`words-allowed.md`](words-allowed.md) names a document that could not be reworded at once, with a date. With `MESH_CATALOG_DIR` set it compares the catalogue's copy
|
||||
of the tools' retired words (`retired-words`) with the glossary.
|
||||
|
||||
It also fails on a **daemon's line** (ADR 0272,
|
||||
its insight of 2026-10-09): the operator types one command, `nox`, and a document names a `nox` line, never the
|
||||
controller's or the node-engine's binary followed by one of its commands. The lines are the check's own list
|
||||
(`DAEMON_LINES`); a command that is a line only with an argument after it is matched as a line is written. Looked
|
||||
for in the same documents, from 2026-10-09 for issues, in the document's own words — code blocks and
|
||||
spans included, quotations and block quotes taken out. With `MESH_CATALOG_DIR` set it compares the catalogue's copy
|
||||
(`retired-lines`) with the list.
|
||||
|
||||
It failed on something real before it passed: 581 uses of retired words in 63 documents on its first run, besides the glossary itself and research 034 —
|
||||
"the host" for the node-engine in 30 designs, "control plane" in 10, the glossary's own entry for the tool runner —
|
||||
and two glossary contradictions (the "console", the deprecated broker's seat), fixed in the change that
|
||||
added it. A retired word's plural (`s` or `es` on its last word) is matched as the word: until 2026-10-07
|
||||
it was not, so "alerts", "rollouts" and "the hosts" (the node-engines) passed while their singulars
|
||||
failed; the plural failed on 15 uses in 10 documents before it passed. A verb that reads like a plural
|
||||
("a node hosts") is a finding too, and is reworded. Homonyms (*plan*, *gate*, *tier*, *ask*, *store*, *record*, *check*) are not checked by it: a
|
||||
word list cannot tell one sense from another, so they are reviewed.
|
||||
The development cycle, checked ([ADR 0080](../../02-DECISIONS/0080-the-development-cycle-is-checked.md)):
|
||||
a to-be design names a decision, an in-progress/implemented design names its owning code, a
|
||||
located/fixed issue names its owner, a fixed/resolved issue says what fixed it, a graduated
|
||||
research overview says what it became, and no two issue records share a number (issue 155 — the
|
||||
number is how a record is cited, and `main` lags every open pull request, so two people reading it
|
||||
allocate the same one). `python3 00-META/checks/cycle.py`
|
||||
|
||||
Binary file not shown.
+88
-35
@@ -4,7 +4,7 @@
|
||||
The knowledge flow (00-META/process/00-overview.md) says work moves idea -> research ->
|
||||
decision -> to-be design -> code, and symptom -> issue -> diagnosis -> fix. Those are rules,
|
||||
and a rule states how it is checked (AGENTS.md) -- this is how. Everything here reads only
|
||||
frontmatter, because a status this repository holds lives in frontmatter and nowhere else.
|
||||
frontmatter, because status lives in frontmatter and nowhere else.
|
||||
|
||||
What is enforced:
|
||||
|
||||
@@ -12,25 +12,20 @@ What is enforced:
|
||||
known `status:`. A TO-BE doc names at least one decision (`decisions:`) -- no
|
||||
design without a decision -- and once `in-progress` or `implemented` it names
|
||||
its owning code (`code:`) -- no development without a design that says where.
|
||||
issues every report has frontmatter, and no two records share a number -- the number is
|
||||
how a record is cited. An issue's status, owner, fix and replay live in the issue
|
||||
tracker since the import of slice 3 (ADR 0275, to-be 53), where `issue-tracker.move`
|
||||
refuses at the write what this check refused at merge until then: a status outside
|
||||
the five, `located` without an owner, `resolved` without a fix, a core issue
|
||||
resolved without its replay (ADR 0237), `wontfix` without a reason. A report
|
||||
carries none of those keys, only the day it was `opened:` -- the tracker holds
|
||||
them, and two places holding one fact drift. The keys the reports opened before
|
||||
the import carried were removed once the import held them; the commit it read
|
||||
is on every imported issue.
|
||||
issues a known `status:`; once `located`, `located-in:` names the owner;
|
||||
once `resolved`, `fixed-by:` says what fixed it (prose counts --
|
||||
"nothing, the capability existed" is an answer). And no two records share a
|
||||
number -- the number is how a record is cited.
|
||||
research a known `status:`; a `graduated` overview says what it `became:`, and every
|
||||
target it names exists.
|
||||
decisions every accepted record is REACHABLE from the cycle: cited by a design doc's
|
||||
frontmatter, a research overview, an issue report, a 00-META document, or another
|
||||
record's extends/supersedes chain. A decision nothing points at is one nobody will
|
||||
find by following pointers -- which is how records go stale in people's heads.
|
||||
|
||||
The decision records and research efforts live in the knowledge base (ADR 0290, issue 426), and the
|
||||
rules this check held over them are the knowledge base's now: a research effort's status and what it
|
||||
became, and a decision's duplicate number, are refused by `knowledge-base.propose`; an accepted decision
|
||||
cited by nothing is the knowledge base's whole-set rule `cited`; a deferral that names no issue is its
|
||||
rule `deferral`. README.md beside this file says where each went.
|
||||
|
||||
Deliberately NOT enforced: as-is docs need no decisions (they describe what exists, not what
|
||||
was decided).
|
||||
Deliberately NOT enforced: `resolved` issues may leave `located-in` empty (a symptom that
|
||||
turned out not to be a defect has no owner), and as-is docs need no decisions (they
|
||||
describe what exists, not what was decided).
|
||||
|
||||
python3 00-META/checks/cycle.py
|
||||
"""
|
||||
@@ -43,11 +38,8 @@ import sys
|
||||
ROOT = os.path.normpath(os.path.join(os.path.dirname(__file__), "..", ".."))
|
||||
|
||||
DESIGN_STATUSES = {"proposed", "designed", "in-progress", "implemented", "abandoned"}
|
||||
|
||||
# The keys an issue report carried while its status lived here. The issue tracker holds them now (ADR 0275,
|
||||
# slice 3 of to-be 53), and `issue-tracker.move` enforces the cycle's rules on them at the write. A report
|
||||
# carrying one of them is a status written in two places, whenever it was opened.
|
||||
TRACKER_KEYS = ("status", "located-in", "fixed-by", "amended-design", "replay", "replay-none", "resolved")
|
||||
ISSUE_STATUSES = {"open", "diagnosing", "located", "resolved", "wontfix"}
|
||||
RESEARCH_STATUSES = {"active", "graduated", "abandoned"}
|
||||
|
||||
|
||||
def rel(path):
|
||||
@@ -139,27 +131,88 @@ def main():
|
||||
else:
|
||||
seen[number] = name
|
||||
|
||||
# Since the import of slice 3 (ADR 0275), an issue's status, owner, fix and replay live in the issue
|
||||
# tracker, and its `move` refuses at the write what this check refused at merge. A report carries none
|
||||
# of those keys.
|
||||
# And decision records, which 155's fix left out: on 2026-10-02 two ADRs numbered 0169 landed
|
||||
# on main from two sessions within the hour, and every check passed.
|
||||
seen_records = {}
|
||||
for path in sorted(glob.glob(os.path.join(ROOT, "02-DECISIONS", "[0-9]*.md"))):
|
||||
name = os.path.basename(path)
|
||||
number = name.split("-", 1)[0]
|
||||
if not number.isdigit():
|
||||
continue
|
||||
if number in seen_records:
|
||||
bad(os.path.join("02-DECISIONS", name),
|
||||
"is numbered %s, and so is %s -- a record's number is how it is cited. Take the next "
|
||||
"free number across main AND every open pull request; the branch that lands last "
|
||||
"renumbers" % (number, seen_records[number]))
|
||||
else:
|
||||
seen_records[number] = name
|
||||
|
||||
for path in sorted(glob.glob(os.path.join(ROOT, "04-ISSUES", "*", "00-report.md"))):
|
||||
front = frontmatter(path)
|
||||
if front is None:
|
||||
bad(path, "no frontmatter")
|
||||
continue
|
||||
opened = str(front.get("opened") or "")
|
||||
if not opened:
|
||||
bad(path, "opened: says nothing -- the day the issue was opened, YYYY-MM-DD")
|
||||
status = front.get("status")
|
||||
if status not in ISSUE_STATUSES:
|
||||
bad(path, "status %r is not one of %s" % (status, sorted(ISSUE_STATUSES)))
|
||||
if status in ("located", "resolved") and not listy(front, "located-in"):
|
||||
bad(path, "status %s but located-in is empty" % status)
|
||||
if status == "resolved" and not listy(front, "fixed-by"):
|
||||
bad(path, "status %s but fixed-by says nothing" % status)
|
||||
|
||||
# ---- research ----------------------------------------------------------------------
|
||||
for path in sorted(glob.glob(os.path.join(ROOT, "01-RESEARCH", "*", "00-overview.md"))):
|
||||
front = frontmatter(path)
|
||||
if front is None:
|
||||
bad(path, "no frontmatter")
|
||||
continue
|
||||
carried = [key for key in TRACKER_KEYS if key in front]
|
||||
if carried:
|
||||
bad(path, "carries %s, but an issue's status, owner, fix and replay live in the issue tracker "
|
||||
"(ADR 0275): move it with `issue-tracker.move`, and keep no copy here"
|
||||
% ", ".join("`%s:`" % key for key in carried))
|
||||
status = front.get("status")
|
||||
if status not in RESEARCH_STATUSES:
|
||||
bad(path, "status %r is not one of %s" % (status, sorted(RESEARCH_STATUSES)))
|
||||
if status == "graduated":
|
||||
became = listy(front, "became")
|
||||
if not became:
|
||||
bad(path, "graduated but became: names nothing")
|
||||
for target in became:
|
||||
if not os.path.exists(os.path.join(ROOT, target)):
|
||||
bad(path, "became names %s, which does not exist" % target)
|
||||
|
||||
# ---- decisions -------------------------------------------------------------------
|
||||
records = {}
|
||||
for path in sorted(glob.glob(os.path.join(ROOT, "02-DECISIONS", "[0-9]*.md"))):
|
||||
front = frontmatter(path)
|
||||
records[os.path.basename(path)] = (path, (front or {}).get("status"))
|
||||
|
||||
cited = set()
|
||||
sources = (glob.glob(os.path.join(ROOT, "03-DESIGN", "*", "*.md"))
|
||||
+ glob.glob(os.path.join(ROOT, "01-RESEARCH", "*", "00-overview.md"))
|
||||
+ glob.glob(os.path.join(ROOT, "04-ISSUES", "*", "00-report.md"))
|
||||
+ glob.glob(os.path.join(ROOT, "00-META", "**", "*.md"), recursive=True))
|
||||
for path in sources:
|
||||
text = open(path, encoding="utf-8").read()
|
||||
if os.sep + "03-DESIGN" + os.sep in path:
|
||||
# A design doc's governing citations live in frontmatter; a prose mention is
|
||||
# commentary, not a home.
|
||||
m = re.match(r"^---\n(.*?)\n---", text, re.S)
|
||||
text = m.group(1) if m else ""
|
||||
for m in re.finditer(r"([0-9]{4}-[^\s\)\],#]+\.md)", text):
|
||||
cited.add(os.path.basename(m.group(1)))
|
||||
for name in records:
|
||||
front = frontmatter(records[name][0]) or {}
|
||||
for key in ("extends", "supersedes", "superseded-by"):
|
||||
v = front.get(key)
|
||||
if isinstance(v, str) and v:
|
||||
cited.add(os.path.basename(v))
|
||||
for name, (path, status) in records.items():
|
||||
if status == "accepted" and name not in cited:
|
||||
bad(path, "an accepted decision nothing in the cycle cites -- give it a home in a "
|
||||
"design doc's decisions:, a research became:, an issue, or 00-META")
|
||||
|
||||
checked = (
|
||||
len(glob.glob(os.path.join(ROOT, "03-DESIGN", "0*", "*.md")))
|
||||
+ len(glob.glob(os.path.join(ROOT, "04-ISSUES", "*", "00-report.md")))
|
||||
+ len(glob.glob(os.path.join(ROOT, "01-RESEARCH", "*", "00-overview.md")))
|
||||
+ len(records)
|
||||
)
|
||||
if failures:
|
||||
print("cycle: %d document(s) break the development cycle:" % len(failures))
|
||||
|
||||
+82
-48
@@ -1,72 +1,106 @@
|
||||
#!/usr/bin/env python3
|
||||
"""The reading order of the decision records: its topics are written here, its records are not.
|
||||
"""Generate the decision index, and check the written one still matches.
|
||||
|
||||
A number identifies a record and never changes, so the reading order is not the order of the numbers.
|
||||
It is the six topics `02-DECISIONS/README.md` writes down, in order, and each record's own topic. The
|
||||
records live in the knowledge base (ADR 0290, issue 426), which reads these lines through the records
|
||||
module and refuses a record whose topic is not one of them; the list of records under a topic is its
|
||||
answer to `knowledge-base.holds` with that topic, never stored here (ADR 0248, hq issue 297).
|
||||
A number identifies a record and never changes, so the folder listing is creation order rather
|
||||
than reading order. The index is what carries the path — and it is written rather than only
|
||||
generated on demand, because a reader on a forge sees the folder and not a command.
|
||||
|
||||
python3 00-META/checks/index.py check
|
||||
The objection to a written index is that it drifts. That objection is answered by checking it
|
||||
rather than by refusing to write one, which is `how-we-build` §5: a rule states how it is
|
||||
checked.
|
||||
|
||||
It fails when the README's reading order is missing or names a topic twice, and when the README stores
|
||||
a list of records again — the index markers, or a line of a generated list. The last one is what keeps
|
||||
the conflict of issue 297 from coming back. Whether every record's topic is one of the six is the
|
||||
knowledge base's rule `topic`, judged when a record is proposed.
|
||||
python3 00-META/checks/index.py --write regenerate it
|
||||
python3 00-META/checks/index.py fail if it is stale
|
||||
"""
|
||||
|
||||
import glob
|
||||
import io
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
README = "02-DECISIONS/README.md"
|
||||
START = "<!-- index:start -->"
|
||||
END = "<!-- index:end -->"
|
||||
|
||||
# A topic in the README's reading order, e.g.
|
||||
# 1. **What the mesh is** — `topic: the mesh`. What it is for, ...
|
||||
# The knowledge base (mesh-knowledge, reference.go) reads the same line, so the form is a contract.
|
||||
TOPIC_LINE = re.compile(r"^\d+\.\s+\*\*(?P<label>[^*]+)\*\*\s+—\s+`topic:\s*(?P<topic>[^`]+)`", re.M)
|
||||
|
||||
# What a stored list looks like: the old markers, or a line of the list index.py used to print.
|
||||
STORED = [
|
||||
(re.compile(r"<!--\s*index:(start|end)\s*-->"), "an index marker"),
|
||||
(re.compile(r"^- \*\*\d{4}\*\* — \[", re.M), "a line of a generated list"),
|
||||
# The reading order. Topics a record may belong to, in the order somebody would learn the system.
|
||||
TOPICS = [
|
||||
("the mesh", "What the mesh is"),
|
||||
("the tiers", "Its tiers, from the bottom up"),
|
||||
("what runs on it", "What runs on them, and how it gets there"),
|
||||
("building it", "How it is built"),
|
||||
("checking it", "How it is checked"),
|
||||
("how we work", "How we work"),
|
||||
]
|
||||
|
||||
|
||||
def reading_order(text=None):
|
||||
"""The topics, in order, as (topic, label) pairs, from the README."""
|
||||
if text is None:
|
||||
text = io.open(README, encoding='utf-8').read()
|
||||
return [(m.group("topic").strip(), m.group("label").strip()) for m in TOPIC_LINE.finditer(text)]
|
||||
def field(text, name):
|
||||
m = re.search(r'^%s:\s*(.+)$' % name, text, re.M)
|
||||
return m.group(1).strip() if m else None
|
||||
|
||||
|
||||
def records():
|
||||
out = []
|
||||
for path in sorted(glob.glob('02-DECISIONS/0*.md')):
|
||||
text = io.open(path, encoding='utf-8').read()
|
||||
heading = re.search(r'^# \d+\.\s*(.+)$', text, re.M)
|
||||
out.append({
|
||||
"file": os.path.basename(path),
|
||||
"number": os.path.basename(path)[:4],
|
||||
"title": heading.group(1).strip() if heading else "(no heading)",
|
||||
"topic": field(text, "topic"),
|
||||
"status": field(text, "status"),
|
||||
})
|
||||
return out
|
||||
|
||||
|
||||
def render(rs):
|
||||
known = {t for t, _ in TOPICS}
|
||||
lines = [START, ""]
|
||||
for topic, label in TOPICS:
|
||||
rows = [r for r in rs if r["topic"] == topic]
|
||||
if not rows:
|
||||
continue
|
||||
lines.append("### %s" % label)
|
||||
lines.append("")
|
||||
for r in rows:
|
||||
mark = "" if r["status"] == "accepted" else " *(%s)*" % r["status"]
|
||||
lines.append("- **%s** — [%s](%s)%s" % (r["number"], r["title"], r["file"], mark))
|
||||
lines.append("")
|
||||
stray = [r for r in rs if r["topic"] not in known]
|
||||
if stray:
|
||||
lines.append("### Unfiled")
|
||||
lines.append("")
|
||||
for r in stray:
|
||||
lines.append("- **%s** — [%s](%s) — `topic:` is %r, which is not one of %s" % (
|
||||
r["number"], r["title"], r["file"], r["topic"], ", ".join(sorted(known))))
|
||||
lines.append("")
|
||||
lines.append(END)
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def main():
|
||||
if "--print" in sys.argv:
|
||||
print("index: the records live in the knowledge base (ADR 0290). Read a topic's with "
|
||||
"`knowledge-base.holds` and that topic, through the mesh MCP server.")
|
||||
return 2
|
||||
|
||||
text = io.open(README, encoding='utf-8').read()
|
||||
topics = reading_order(text)
|
||||
wanted = render(records())
|
||||
|
||||
problems = []
|
||||
if not topics:
|
||||
problems.append("%s writes no reading order: no line of the form "
|
||||
"'1. **Label** — `topic: name`. ...'; the knowledge base refuses every "
|
||||
"record without it" % README)
|
||||
names = [t for t, _ in topics]
|
||||
if len(set(names)) != len(names):
|
||||
problems.append("%s names a topic twice in its reading order" % README)
|
||||
for pattern, what in STORED:
|
||||
if pattern.search(text):
|
||||
problems.append("%s stores a list of records (%s). The records live in the knowledge base, "
|
||||
"and a topic's list is `knowledge-base.holds` with that topic" % (README, what))
|
||||
|
||||
for p in problems:
|
||||
print("index: " + p)
|
||||
if problems:
|
||||
if START not in text or END not in text:
|
||||
print("index: %s has no index markers (%s / %s)" % (README, START, END))
|
||||
return 1
|
||||
print("index: %d topics in the reading order, no stored list" % len(topics))
|
||||
|
||||
current = text[text.index(START):text.index(END) + len(END)]
|
||||
if "--write" in sys.argv:
|
||||
if current == wanted:
|
||||
print("index: already current")
|
||||
return 0
|
||||
io.open(README, 'w', encoding='utf-8').write(text.replace(current, wanted, 1))
|
||||
print("index: written")
|
||||
return 0
|
||||
|
||||
if current != wanted:
|
||||
print("index: %s is stale. Regenerate it:\n"
|
||||
" python3 00-META/checks/index.py --write" % README)
|
||||
return 1
|
||||
print("index: current")
|
||||
return 0
|
||||
|
||||
|
||||
|
||||
+259
-69
@@ -1,5 +1,5 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Structural checks over HQ's own documents.
|
||||
"""Structural checks over HQ's own records.
|
||||
|
||||
Every check here exists because the thing it checks for actually happened. See README.md
|
||||
for which incident is behind which check. Run from the repository root:
|
||||
@@ -7,11 +7,6 @@ for which incident is behind which check. Run from the repository root:
|
||||
python3 00-META/checks/records.py
|
||||
|
||||
Exits non-zero if anything fails, so it can be a gate rather than a report.
|
||||
|
||||
The decision records and research efforts live in the knowledge base (ADR 0290, issue 426), and the
|
||||
rules that judged them here are the knowledge base's now: `propose` refuses a record that breaks one.
|
||||
README.md beside this file says where each rule went. What stays here judges what this repository
|
||||
still holds, and keeps the records from coming back.
|
||||
"""
|
||||
|
||||
import os
|
||||
@@ -20,17 +15,14 @@ import sys
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
# Where a citation is *guidance* rather than history. A document here tells somebody what to
|
||||
# do, so a link to a superseded record is an instruction to follow a withdrawn decision.
|
||||
# 02-DECISIONS and 01-RESEARCH are deliberately absent: they record what was decided and what
|
||||
# was observed, and both legitimately discuss superseded records at length.
|
||||
GOVERNING = ("00-META/", "03-DESIGN/01-to-be/")
|
||||
|
||||
LINK = re.compile(r"\[[^\]]*\]\((?!https?:|mailto:)([^)]+)\)")
|
||||
|
||||
# The two folders the records left. Each keeps a README that points at the knowledge base, and
|
||||
# 02-DECISIONS/README.md also writes the reading order's topics, which the knowledge base reads.
|
||||
RECORD_FOLDERS = ("02-DECISIONS", "01-RESEARCH")
|
||||
POINTERS = {"02-DECISIONS/README.md", "01-RESEARCH/README.md"}
|
||||
|
||||
# A decision record is named "ADR NNNN", four digits, and a research effort "research NNN", three.
|
||||
DECISION_ENTRY = re.compile(r"^ADR \d{4}$")
|
||||
ADR_MENTION = re.compile(r"\bADRs?\s+(\d+)\b")
|
||||
RESEARCH_MENTION = re.compile(r"\b[Rr]esearch\s+(\d+)\b")
|
||||
ADR_FILE = re.compile(r"^(\d{4})-")
|
||||
|
||||
|
||||
def markdown_files():
|
||||
@@ -53,7 +45,7 @@ def read(path):
|
||||
def frontmatter(text):
|
||||
"""Minimal frontmatter reader — enough for the fields these checks use.
|
||||
|
||||
Not a YAML parser on purpose: a dependency in a repository that has none, to read a few
|
||||
Not a YAML parser on purpose: a dependency in a repository that has none, to read four
|
||||
scalar fields and one list, would cost more than it returns.
|
||||
"""
|
||||
if not text.startswith("---\n"):
|
||||
@@ -82,11 +74,24 @@ def frontmatter(text):
|
||||
return fields
|
||||
|
||||
|
||||
def unquote(value):
|
||||
value = value.strip()
|
||||
if len(value) >= 2 and value[0] == value[-1] and value[0] in "\"'":
|
||||
return value[1:-1]
|
||||
return value
|
||||
def load_records():
|
||||
"""Every decision record, by its four-digit number."""
|
||||
records = {}
|
||||
folder = os.path.join(ROOT, "02-DECISIONS")
|
||||
for name in sorted(os.listdir(folder)):
|
||||
match = ADR_FILE.match(name)
|
||||
if not name.endswith(".md") or not match:
|
||||
continue
|
||||
path = os.path.join(folder, name)
|
||||
text = read(path)
|
||||
records[match.group(1)] = {
|
||||
"number": match.group(1),
|
||||
"name": name,
|
||||
"path": path,
|
||||
"text": text,
|
||||
"front": frontmatter(text),
|
||||
}
|
||||
return records
|
||||
|
||||
|
||||
class Failures:
|
||||
@@ -124,62 +129,242 @@ def check_links(failures):
|
||||
failures.add("links", f"{rel(path)}:{number}", f"link does not resolve: {target}")
|
||||
|
||||
|
||||
def check_no_records(failures):
|
||||
"""No decision record or research effort is written into this repository again (ADR 0290 rule 9).
|
||||
def check_rests_on(failures, records):
|
||||
"""A document's `decisions:` and a record's `extends:` must name a live record.
|
||||
|
||||
The knowledge base is the only place a record is read; a file here would be a second copy that
|
||||
drifts from it, and nothing would read it. The folders keep their pointer READMEs and nothing else.
|
||||
These are the load-bearing citations: the document declares that it rests on that
|
||||
decision. Resting on a withdrawn one is the defect this whole check set exists for.
|
||||
"""
|
||||
for folder in RECORD_FOLDERS:
|
||||
base = os.path.join(ROOT, folder)
|
||||
if not os.path.isdir(base):
|
||||
failures.add("no-records", folder, "the folder is gone; it keeps a README that points at the "
|
||||
"knowledge base (and, for 02-DECISIONS, the reading order the knowledge base reads)")
|
||||
continue
|
||||
for current, dirs, files in os.walk(base):
|
||||
for name in sorted(files + [d + "/" for d in dirs]):
|
||||
path = rel(os.path.join(current, name))
|
||||
if path.rstrip("/") in POINTERS:
|
||||
for path in markdown_files():
|
||||
front = frontmatter(read(path))
|
||||
cited = list(front.get("decisions", []) or [])
|
||||
extends = front.get("extends")
|
||||
if isinstance(extends, str) and extends:
|
||||
cited.append(extends)
|
||||
|
||||
for entry in cited:
|
||||
match = ADR_FILE.match(os.path.basename(entry))
|
||||
if not match:
|
||||
failures.add("rests-on", rel(path), f"not a decision record: {entry}")
|
||||
continue
|
||||
number = match.group(1)
|
||||
if number not in records:
|
||||
failures.add("rests-on", rel(path), f"no such record: {entry}")
|
||||
continue
|
||||
status = records[number]["front"].get("status")
|
||||
if status != "accepted":
|
||||
# A proposed record may extend another proposed one. Decisions are drafted in
|
||||
# chains -- 0059 extends 0057 while both await review -- and refusing that would
|
||||
# mean either drafting out of order or marking records accepted to satisfy a
|
||||
# check, which is the failure this repository already made once.
|
||||
if frontmatter(read(path)).get("status") == "proposed":
|
||||
continue
|
||||
# An as-is document describes what runs, and what runs was built under
|
||||
# whatever was decided at the time. ADR 0056: "as-is describing a superseded
|
||||
# decision is exactly what as-is is for."
|
||||
if rel(path).startswith("03-DESIGN/00-as-is/"):
|
||||
continue
|
||||
# A withdrawn record's citations are history. It instructs nobody -- every reader
|
||||
# is sent to its superseder -- so what it was built on may itself be withdrawn.
|
||||
# Refusing that would mean rewriting the lineage of a record whose reasoning is
|
||||
# the thing the immutability rule protects.
|
||||
if frontmatter(read(path)).get("status") == "superseded":
|
||||
continue
|
||||
# An extension that supersedes legitimately names what it replaced.
|
||||
this = ADR_FILE.match(os.path.basename(path))
|
||||
supersedes = records[number]["front"].get("superseded-by", "")
|
||||
if this and supersedes and os.path.basename(path) in str(supersedes):
|
||||
continue
|
||||
failures.add(
|
||||
"no-records",
|
||||
path,
|
||||
"a decision record or research effort lives in the knowledge base, never here: "
|
||||
"write it with knowledge-base.reserve and knowledge-base.propose (playbooks 01 and 02)",
|
||||
"rests-on",
|
||||
rel(path),
|
||||
f"rests on ADR {number}, which is '{status}' — a document may not rest on a "
|
||||
f"record that is not accepted",
|
||||
)
|
||||
|
||||
|
||||
def check_decisions_entries(failures):
|
||||
"""A design's `decisions:` names each record as "ADR NNNN".
|
||||
def check_live_citations(failures, records):
|
||||
"""In a governing document, a link to a superseded record must name its replacement.
|
||||
|
||||
A to-be design rests on the records it names, and the knowledge base answers for a record by its
|
||||
number; an entry in another form is a citation nothing can follow.
|
||||
The reader of a rule needs to know the rule was withdrawn, and needs somewhere to go.
|
||||
Naming the superseder in the same paragraph is both, and it is what a person would
|
||||
write anyway.
|
||||
"""
|
||||
superseded = {
|
||||
number: record["front"].get("superseded-by", "")
|
||||
for number, record in records.items()
|
||||
if record["front"].get("status") == "superseded"
|
||||
}
|
||||
|
||||
for path in markdown_files():
|
||||
if not rel(path).startswith("03-DESIGN/"):
|
||||
if not any(rel(path).startswith(prefix) for prefix in GOVERNING):
|
||||
continue
|
||||
for entry in frontmatter(read(path)).get("decisions", []) or []:
|
||||
if not DECISION_ENTRY.match(unquote(entry)):
|
||||
failures.add("decisions-form", rel(path),
|
||||
f"decisions: entry {entry!r} is not 'ADR NNNN' (four digits)")
|
||||
|
||||
|
||||
def check_mentions(failures):
|
||||
"""In running text, a decision record is "ADR NNNN" (four digits) and a research effort "research NNN"
|
||||
(three): the form the knowledge base and the issue tracker read a citation in. A code span is an example,
|
||||
and is not read."""
|
||||
for path in markdown_files():
|
||||
text = read(path)
|
||||
for number, line in enumerate(text.split("\n"), 1):
|
||||
line = re.sub(r"`[^`]*`", "", line)
|
||||
for m in ADR_MENTION.finditer(line):
|
||||
if len(m.group(1)) != 4:
|
||||
failures.add("mention-form", f"{rel(path)}:{number}",
|
||||
f"{m.group(0)!r}: a decision record is named 'ADR NNNN', four digits")
|
||||
for m in RESEARCH_MENTION.finditer(line):
|
||||
if len(m.group(1)) != 3:
|
||||
failures.add("mention-form", f"{rel(path)}:{number}",
|
||||
f"{m.group(0)!r}: a research effort is named 'research NNN', three digits")
|
||||
offset = 0
|
||||
for paragraph in text.split("\n\n"):
|
||||
line_no = text[:offset].count("\n") + 1
|
||||
offset += len(paragraph) + 2
|
||||
targets = LINK.findall(paragraph)
|
||||
for target in targets:
|
||||
match = ADR_FILE.match(os.path.basename(target.split("#")[0]))
|
||||
if not match or match.group(1) not in superseded:
|
||||
continue
|
||||
number = match.group(1)
|
||||
replacement = os.path.basename(str(superseded[number]))
|
||||
if not replacement:
|
||||
failures.add(
|
||||
"live-citation",
|
||||
f"{rel(path)}:{line_no}",
|
||||
f"cites superseded ADR {number}, which names no superseder",
|
||||
)
|
||||
continue
|
||||
if not any(replacement in t for t in targets):
|
||||
failures.add(
|
||||
"live-citation",
|
||||
f"{rel(path)}:{line_no}",
|
||||
f"cites superseded ADR {number} without naming its replacement "
|
||||
f"({replacement}) in the same paragraph",
|
||||
)
|
||||
|
||||
|
||||
def check_supersession_symmetry(failures, records):
|
||||
"""If A says it was superseded by B, B must say it supersedes A."""
|
||||
for number, record in records.items():
|
||||
front = record["front"]
|
||||
status = front.get("status")
|
||||
by = os.path.basename(str(front.get("superseded-by", "")))
|
||||
|
||||
if status == "superseded" and not by:
|
||||
failures.add("supersession", rel(record["path"]), "marked superseded but names no superseder")
|
||||
continue
|
||||
if by and status != "superseded":
|
||||
failures.add("supersession", rel(record["path"]), f"names a superseder but status is '{status}'")
|
||||
if not by:
|
||||
continue
|
||||
|
||||
match = ADR_FILE.match(by)
|
||||
if not match or match.group(1) not in records:
|
||||
failures.add("supersession", rel(record["path"]), f"superseder does not exist: {by}")
|
||||
continue
|
||||
other = records[match.group(1)]
|
||||
# `supersedes:` may name one record or several. One decision replacing two is a real
|
||||
# situation -- two records that built and refined the same wrong mechanism are withdrawn
|
||||
# by the one record that removes it -- and a check that allows only one would force
|
||||
# either a chain of pro-forma records or an unmarked supersession.
|
||||
claimed = other["front"].get("supersedes", "")
|
||||
if isinstance(claimed, str):
|
||||
claimed = [claimed] if claimed else []
|
||||
claims = [os.path.basename(str(entry)) for entry in claimed]
|
||||
if record["name"] not in claims:
|
||||
failures.add(
|
||||
"supersession",
|
||||
rel(other["path"]),
|
||||
f"ADR {number} says this supersedes it; this record does not say so "
|
||||
f"(supersedes: {', '.join(claims) or 'absent'})",
|
||||
)
|
||||
|
||||
|
||||
def check_topics(failures, records):
|
||||
"""Every record names a topic the index knows.
|
||||
|
||||
The topic is what puts a record in the reading order, so a record without one — or with one
|
||||
nobody defined — disappears from the index rather than appearing in the wrong place. That is
|
||||
the quiet failure, so it is the one checked.
|
||||
"""
|
||||
known = {"the mesh", "the tiers", "what runs on it", "building it", "checking it",
|
||||
"how we work"}
|
||||
for number, record in sorted(records.items()):
|
||||
topic = record["front"].get("topic")
|
||||
if not topic:
|
||||
failures.add("topics", rel(record["path"]),
|
||||
"no topic, so it has no place in the reading order")
|
||||
elif topic not in known:
|
||||
failures.add("topics", rel(record["path"]),
|
||||
"topic %r is not one of: %s" % (topic, ", ".join(sorted(known))))
|
||||
|
||||
|
||||
def check_numbering(failures, records):
|
||||
"""The number in the filename is the number in the heading."""
|
||||
for number, record in records.items():
|
||||
heading = re.search(r"^# (\d+)\.", record["text"], re.M)
|
||||
if not heading:
|
||||
failures.add("numbering", rel(record["path"]), "no '# N. Title' heading")
|
||||
elif heading.group(1) != number.lstrip("0"):
|
||||
failures.add(
|
||||
"numbering",
|
||||
rel(record["path"]),
|
||||
f"filename says {number}, heading says {heading.group(1)}",
|
||||
)
|
||||
|
||||
|
||||
def check_progressive_insights(failures, records):
|
||||
"""A correction made inside a record is marked and dated, or it is a silent rewrite.
|
||||
|
||||
A record may be corrected in place when a *fact* in it went stale and the decision still
|
||||
stands (`02-DECISIONS/README.md`, "Progressive insight"). The whole safety of that allowance
|
||||
is that the correction is legible in the record rather than only in a diff nobody reads, so
|
||||
the form is what is checked here: every mention of an insight is the marker, the marker
|
||||
carries an ISO date, and that date is not earlier than the decision's own — an insight
|
||||
predating the decision it corrects is a copied marker, not a correction.
|
||||
|
||||
What this cannot check is an edit made with no marker at all. Nothing mechanical can; that
|
||||
one is the reviewer's, reading the diff. The check keeps the *marked* path honest so that an
|
||||
unmarked change stands out as the anomaly it is.
|
||||
"""
|
||||
phrase = re.compile(r"progressive insight", re.I)
|
||||
# Both patterns stay on one line: a bold run does not span paragraphs, and `[^*]*` across
|
||||
# newlines will happily join an unrelated `**` far above to the marker below, reporting the
|
||||
# whole span between them. It did exactly that the first time this ran.
|
||||
# Trailing words after the date are allowed — "— 2026-09-26, correcting the one above." — so
|
||||
# an insight can say what it relates to. Only the date's presence and position are fixed.
|
||||
marker = re.compile(r"\*\*Progressive insights?[ \t]*[\u2014\u2013-][ \t]*(\d{4}-\d{2}-\d{2})[^*\n]*\*\*")
|
||||
loose = re.compile(r"\*\*[^*\n]*[Pp]rogressive insights?[^*\n]*\*\*")
|
||||
iso = re.compile(r"^\d{4}-\d{2}-\d{2}$")
|
||||
|
||||
for number, record in sorted(records.items()):
|
||||
text = record["text"]
|
||||
if not phrase.search(text):
|
||||
continue
|
||||
decided = str(record["front"].get("date", ""))
|
||||
good = [(m.start(), m.end(), m.group(1)) for m in marker.finditer(text)]
|
||||
|
||||
for m in loose.finditer(text):
|
||||
if any(s <= m.start() and m.end() <= e for s, e, _ in good):
|
||||
continue
|
||||
# A bold run carrying a link is discussing an insight — usually another record's —
|
||||
# rather than marking one. A marker never needs to cite anything.
|
||||
if "](" in m.group(0):
|
||||
continue
|
||||
failures.add("insights", rel(record["path"]),
|
||||
"a progressive insight is not in the dated marked form "
|
||||
"'**Progressive insight \u2014 YYYY-MM-DD.**': %s" % m.group(0))
|
||||
|
||||
for _, _, stamp in good:
|
||||
if decided and iso.match(decided) and stamp < decided:
|
||||
failures.add("insights", rel(record["path"]),
|
||||
"a progressive insight dated %s predates the decision (%s)"
|
||||
% (stamp, decided))
|
||||
|
||||
covered = [(s, e) for s, e, _ in good]
|
||||
for m in phrase.finditer(text):
|
||||
if any(s <= m.start() and m.end() <= e for s, e in covered):
|
||||
continue
|
||||
line = text.rfind("\n", 0, m.start()) + 1
|
||||
end = text.find("\n", m.end())
|
||||
whole = text[line:end if end != -1 else len(text)]
|
||||
if whole.lstrip().startswith("#"):
|
||||
continue
|
||||
# A line that also carries a link is discussing the rule, not marking a correction:
|
||||
# a marker never needs to cite anything, and a record that reasons about the policy
|
||||
# must be able to name it. Bare prose with no citation is the informal marking this
|
||||
# is here to catch.
|
||||
if "](" in whole:
|
||||
continue
|
||||
if loose.search(text, line, text.find("\n", m.end()) + 1 or len(text)):
|
||||
continue
|
||||
failures.add("insights", rel(record["path"]),
|
||||
"'progressive insight' appears unmarked; a correction is marked and "
|
||||
"dated, or it is a silent rewrite")
|
||||
|
||||
|
||||
def check_status_against_code(failures):
|
||||
@@ -215,11 +400,16 @@ def check_status_against_code(failures):
|
||||
|
||||
def main():
|
||||
failures = Failures()
|
||||
records = load_records()
|
||||
check_links(failures)
|
||||
check_no_records(failures)
|
||||
check_decisions_entries(failures)
|
||||
check_mentions(failures)
|
||||
check_rests_on(failures, records)
|
||||
check_live_citations(failures, records)
|
||||
check_supersession_symmetry(failures, records)
|
||||
check_numbering(failures, records)
|
||||
check_topics(failures, records)
|
||||
check_status_against_code(failures)
|
||||
check_progressive_insights(failures, records)
|
||||
print(f"records: {len(records)} decision records checked")
|
||||
return failures.report()
|
||||
|
||||
|
||||
|
||||
@@ -1,10 +0,0 @@
|
||||
# Words allowed — documents `words.py` does not hold to the glossary's retired words
|
||||
|
||||
Read by [`words.py`](words.py) (ADR 0244).
|
||||
One row per document. *Until* is the date by which it is reworded — past it, the row fails like the words
|
||||
themselves. A row whose document no longer uses a retired word fails too, so this list only shrinks. The
|
||||
research efforts this list kept as written (research 034 and 042) live in the knowledge base now (ADR 0290),
|
||||
which keeps a record's words as it was imported.
|
||||
|
||||
| Document | Until | Why |
|
||||
|---|---|---|
|
||||
@@ -1,377 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""One word per thing, checked (ADR 0244).
|
||||
|
||||
The glossary (00-META/glossary.md) is the authority on the mesh's words. It names the words it
|
||||
retired on one fixed kind of line, so a program can read them:
|
||||
|
||||
*Not:* ~~control plane~~, ~~overlay~~ (hq)
|
||||
|
||||
A struck word with no scope is retired everywhere; `(hq)` retires it in this repository's prose only;
|
||||
`(tools)` in the descriptions of the mesh's tools only. And it names code that still carries an old
|
||||
name until that code is renamed:
|
||||
|
||||
*Identifier until renamed:* `mesh-host` — the repository, binary and service unit
|
||||
|
||||
An identifier is allowed inside a code span and nowhere else.
|
||||
|
||||
What is enforced:
|
||||
|
||||
retired no retired word of scope `hq` (or none), and no identifier, in running prose of a
|
||||
checked document. Running prose is what is left once code blocks, code spans, block
|
||||
quotes, text in quotation marks, struck-through text, link targets, HTML comments and
|
||||
frontmatter are taken out: a quotation keeps the words it quotes, a link target is a file
|
||||
name, and an identifier lives in a code span. A word's plural is the word: "alerts" fails
|
||||
as "alert" does.
|
||||
unique no head word (a bolded word opening a glossary entry) heads two entries, and no head
|
||||
word is also struck through on a *Not:* line.
|
||||
tools list with `--list tools` it prints the words retired in the tools' descriptions, which is the
|
||||
copy the catalogue's own check keeps (mesh-catalog `retired-words`). With
|
||||
MESH_CATALOG_DIR set to a checkout of the catalogue it also fails when that copy differs.
|
||||
|
||||
lines no daemon's line in a checked document (ADR 0272, its insight of 2026-10-09). The operator
|
||||
types one command, `nox` (`mesh-cli`); the node-engine (`nox-mesh-host`) and the
|
||||
controller (`mesh-controller`) are daemons, and a person never types their names — so a
|
||||
document names a `nox` line, never `mesh-controller push` or `mesh-host hand-over`. The
|
||||
lines are DAEMON_LINES below: each daemon's commands, and which of them are a line only
|
||||
with an argument after them ("the mesh-controller module" is prose; `mesh-controller
|
||||
module check …` is a line). A line is looked for in the document's own words — code
|
||||
blocks and code spans included, since that is where a line to type is written — with
|
||||
quotations, block quotes, link targets and comments taken out. `--list lines` prints the
|
||||
list, which the catalogue's own check keeps a copy of (mesh-catalog `retired-lines`); with
|
||||
MESH_CATALOG_DIR set it also fails when that copy differs. The node-engine's install-time
|
||||
subcommands (`nox-mesh-host enrol`, `key`, `reconcile`, …) are not lines here: they are
|
||||
typed at a machine no mesh reaches yet, and the engine's usage says so.
|
||||
|
||||
Checked documents: 00-META/, 03-DESIGN/ (both layers), AGENTS.md and README.md, always; an issue opened
|
||||
on or after FROM (on or after LINES_FROM for the lines). The decision records and research efforts live in
|
||||
the knowledge base (ADR 0290, issue 426), which holds a record proposed there to the same words through its
|
||||
rules `retired-word` and `daemon-line`, read from this glossary.
|
||||
|
||||
A document the check fails on that cannot be reworded in the change that found it is named, with a date
|
||||
and a reason, in words-allowed.md beside this file. An entry past its date fails like the word itself, and
|
||||
an entry for a document that no longer needs it fails too.
|
||||
|
||||
python3 00-META/checks/words.py
|
||||
python3 00-META/checks/words.py --list tools
|
||||
python3 00-META/checks/words.py --list lines
|
||||
"""
|
||||
|
||||
import datetime
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
ROOT = os.path.normpath(os.path.join(os.path.dirname(__file__), "..", ".."))
|
||||
GLOSSARY = "00-META/glossary.md"
|
||||
ALLOWED = "00-META/checks/words-allowed.md"
|
||||
|
||||
# The day the rule began (ADR 0244): issues opened from it on are held to it.
|
||||
FROM = "2026-10-07"
|
||||
|
||||
ALWAYS = ("00-META/", "03-DESIGN/", "AGENTS.md", "README.md")
|
||||
|
||||
# The daemons' lines (ADR 0272, its insight of 2026-10-09), and the day that rule began. "alone" is a
|
||||
# command that is a line by itself; "with an argument" one that is a line only when written as one.
|
||||
LINES_FROM = "2026-10-09"
|
||||
DAEMON_LINES = {
|
||||
"mesh-controller": {
|
||||
"alone": ("adopt assign broker cancel cleanup collect conditions converge doctor durations hand-act "
|
||||
"hand-acts healers identity kill licence migrate operator pause pin plan prepare push rebuild "
|
||||
"rehearse replay resume retire rollout rotate secret settings status token unassign unpin "
|
||||
"upgrade").split(),
|
||||
"with an argument": ("artifacts ask builds bus collection data delivery images mirrors module node "
|
||||
"overlay plans queue seat seats take version").split(),
|
||||
},
|
||||
"mesh-host": {
|
||||
"alone": "apply bundle enrol enroll hand-over help inventory key overlay owned profile reconcile run version".split(),
|
||||
"with an argument": [],
|
||||
},
|
||||
"nox-mesh-host": {"alone": ["hand-over"], "with an argument": []},
|
||||
}
|
||||
|
||||
NOT_LINE = re.compile(r"^\s*(?:[-*]\s+)?\*Not:\*(.*)$", re.M)
|
||||
STRUCK = re.compile(r"~~([^~]+)~~(?:\s*\((hq|tools)\))?")
|
||||
IDENT_LINE = re.compile(r"^\s*(?:[-*]\s+)?\*Identifier until renamed:\*(.*)$", re.M)
|
||||
CODE_SPAN = re.compile(r"`([^`]+)`")
|
||||
HEAD = re.compile(r"^\s*-\s+\*\*([^*]+)\*\*", re.M)
|
||||
|
||||
|
||||
def rel(path):
|
||||
return os.path.relpath(path, ROOT)
|
||||
|
||||
|
||||
def read(path):
|
||||
with open(os.path.join(ROOT, path), encoding="utf-8") as handle:
|
||||
return handle.read()
|
||||
|
||||
|
||||
def frontmatter_field(text, name):
|
||||
if not text.startswith("---\n"):
|
||||
return None
|
||||
end = text.find("\n---", 4)
|
||||
m = re.search(r"^%s:\s*(\S+)" % name, text[4:end], re.M)
|
||||
return m.group(1) if m else None
|
||||
|
||||
|
||||
def glossary():
|
||||
"""The retired words with their scope, the identifiers, and the head words."""
|
||||
text = read(GLOSSARY)
|
||||
retired = []
|
||||
for line in NOT_LINE.findall(text):
|
||||
for word, scope in STRUCK.findall(line):
|
||||
for one in word.split(" / "):
|
||||
retired.append((one.strip(), scope or "all"))
|
||||
identifiers = []
|
||||
for line in IDENT_LINE.findall(text):
|
||||
identifiers += [i.strip() for i in CODE_SPAN.findall(line)]
|
||||
heads = []
|
||||
for head in HEAD.findall(text):
|
||||
heads += [h.strip() for h in head.split(" / ")]
|
||||
return retired, identifiers, heads
|
||||
|
||||
|
||||
def pattern(word):
|
||||
"""A whole-word, case-insensitive match; a space in the word matches a space, a line break or a hyphen.
|
||||
|
||||
The plural is the word too: its last part may end in `s` or `es` ("alerts", "control planes"), since
|
||||
a retired word does not come back by being counted. Anything else joined on is another word.
|
||||
"""
|
||||
parts = [re.escape(p) for p in word.split()]
|
||||
return re.compile(r"(?i)(?<![\w-])" + r"[\s-]+".join(parts) + r"(?:e?s)?(?![\w-])")
|
||||
|
||||
|
||||
def blank(match):
|
||||
return re.sub(r"[^\n]", " ", match.group(0))
|
||||
|
||||
|
||||
def prose(text, glossary_file=False):
|
||||
"""The text with everything that is not running prose blanked, keeping offsets and line numbers."""
|
||||
if text.startswith("---\n"):
|
||||
end = text.find("\n---", 4)
|
||||
if end != -1:
|
||||
text = re.sub(r"[^\n]", " ", text[: end + 4]) + text[end + 4:]
|
||||
steps = [
|
||||
r"(?ms)^\s*```.*?^\s*```", # code blocks
|
||||
r"(?s)<!--.*?-->", # comments
|
||||
r"(?m)^\s*>.*$", # block quotes
|
||||
r"`[^`\n]+`", # code spans
|
||||
r"\]\([^)\s]*\)", # link targets
|
||||
r"~~[^~\n]+~~", # struck through: a word named as retired
|
||||
r"\"[^\"\n]*\"", # "quoted"
|
||||
r"\u201c[^\u201d]*\u201d", # curly double quotes
|
||||
r"\u2018[^\u2019\n]*\u2019", # curly single quotes
|
||||
]
|
||||
if glossary_file:
|
||||
steps[:0] = [NOT_LINE.pattern, IDENT_LINE.pattern]
|
||||
for step in steps:
|
||||
text = re.sub(step, blank, text)
|
||||
return text
|
||||
|
||||
|
||||
def issues(since=FROM):
|
||||
"""The issue reports opened on or after `since`, file by file."""
|
||||
out = []
|
||||
base = os.path.join(ROOT, "04-ISSUES")
|
||||
if not os.path.isdir(base):
|
||||
return out
|
||||
for issue in sorted(os.listdir(base)):
|
||||
head = os.path.join(base, issue, "00-report.md")
|
||||
if not os.path.isfile(head):
|
||||
continue
|
||||
date = frontmatter_field(read(rel(head)), "opened")
|
||||
if not date or date < since:
|
||||
continue
|
||||
for name in sorted(os.listdir(os.path.join(base, issue))):
|
||||
if name.endswith(".md"):
|
||||
out.append("04-ISSUES/%s/%s" % (issue, name))
|
||||
return out
|
||||
|
||||
|
||||
def checked_documents(since=FROM):
|
||||
docs = []
|
||||
for entry in ALWAYS:
|
||||
path = os.path.join(ROOT, entry)
|
||||
if os.path.isfile(path):
|
||||
docs.append(entry)
|
||||
continue
|
||||
for base, dirs, files in os.walk(path):
|
||||
dirs.sort()
|
||||
for name in sorted(files):
|
||||
if name.endswith(".md"):
|
||||
docs.append(rel(os.path.join(base, name)))
|
||||
return docs + issues(since)
|
||||
|
||||
|
||||
def allowed():
|
||||
"""words-allowed.md: one row per document — | `document` | until | why |.
|
||||
|
||||
`until` is a date, after which the allowance fails like the words themselves.
|
||||
"""
|
||||
out, problems = {}, []
|
||||
if not os.path.isfile(os.path.join(ROOT, ALLOWED)):
|
||||
return out, problems
|
||||
today = datetime.date.today().isoformat()
|
||||
for line in read(ALLOWED).splitlines():
|
||||
cells = [c.strip() for c in line.strip().strip("|").split("|")]
|
||||
if len(cells) != 3 or not cells[0].startswith("`"):
|
||||
continue
|
||||
doc, until, why = cells[0].strip("`"), cells[1], cells[2]
|
||||
if not why:
|
||||
problems.append("%s: the allowance for %s gives no reason" % (ALLOWED, doc))
|
||||
if not re.match(r"^\d{4}-\d{2}-\d{2}$", until):
|
||||
problems.append("%s: the allowance for %s has no date" % (ALLOWED, doc))
|
||||
continue
|
||||
elif until < today:
|
||||
problems.append("%s: the allowance for %s ran out on %s — reword it" % (ALLOWED, doc, until))
|
||||
continue
|
||||
out[doc] = until
|
||||
return out, problems
|
||||
|
||||
|
||||
def check_retired(retired, identifiers):
|
||||
problems = []
|
||||
allowance, problems_allowed = allowed()
|
||||
problems += problems_allowed
|
||||
words = [(w, pattern(w), "retired") for w, scope in retired if scope in ("all", "hq")]
|
||||
words += [(i, pattern(i), "an identifier, allowed only in a code span") for i in identifiers]
|
||||
used = set()
|
||||
for doc in checked_documents():
|
||||
text = prose(read(doc), glossary_file=(doc == GLOSSARY))
|
||||
for word, rx, why in words:
|
||||
for m in rx.finditer(text):
|
||||
if doc in allowance:
|
||||
used.add(doc)
|
||||
continue
|
||||
line = text.count("\n", 0, m.start()) + 1
|
||||
problems.append("%s:%d: %r is %s (glossary)" % (doc, line, m.group(0).replace("\n", " "), why))
|
||||
for doc in allowance:
|
||||
if doc not in used:
|
||||
problems.append("%s: %s no longer uses a retired word — remove its allowance" % (ALLOWED, doc))
|
||||
return problems
|
||||
|
||||
|
||||
def lines_list():
|
||||
"""The daemons' lines as the catalogue's copy lists them: `<daemon> <command>`, ` …` when an argument must follow."""
|
||||
out = []
|
||||
for daemon, kinds in DAEMON_LINES.items():
|
||||
out += ["%s %s" % (daemon, c) for c in kinds["alone"]]
|
||||
out += ["%s %s …" % (daemon, c) for c in kinds["with an argument"]]
|
||||
return sorted(out)
|
||||
|
||||
|
||||
def line_pattern(daemon, command, with_argument):
|
||||
"""The line whole: the daemon, white space, the command. One that is a line only with an argument is matched
|
||||
as a line is written — at the start of a line or a code span, a prompt allowed, an argument after it."""
|
||||
d, c = re.escape(daemon), re.escape(command)
|
||||
if with_argument:
|
||||
return re.compile(r"(?m)(?:^|`)[ \t]*(?:\$[ \t]*)?(" + d + r"\s+" + c + r"\s+[^\s`])")
|
||||
return re.compile(r"(?<![\w-])" + d + r"\s+" + c + r"(?![\w-])")
|
||||
|
||||
|
||||
def own_words(text):
|
||||
"""The text with what is not the document's own words blanked: as prose(), but a code block and a code span
|
||||
are kept, since a line to type is written in one."""
|
||||
if text.startswith("---\n"):
|
||||
end = text.find("\n---", 4)
|
||||
if end != -1:
|
||||
text = re.sub(r"[^\n]", " ", text[: end + 4]) + text[end + 4:]
|
||||
for step in (
|
||||
r"(?s)<!--.*?-->", # comments
|
||||
r"(?m)^\s*>.*$", # block quotes
|
||||
r"\]\([^)\s]*\)", # link targets
|
||||
r"~~[^~\n]+~~", # struck through
|
||||
r"\"[^\"\n]*\"", # "quoted"
|
||||
r"\u201c[^\u201d]*\u201d", # curly double quotes
|
||||
r"\u2018[^\u2019\n]*\u2019", # curly single quotes
|
||||
):
|
||||
text = re.sub(step, blank, text)
|
||||
return text
|
||||
|
||||
|
||||
def check_lines():
|
||||
problems = []
|
||||
patterns = []
|
||||
for daemon, kinds in DAEMON_LINES.items():
|
||||
patterns += [(daemon, c, line_pattern(daemon, c, False)) for c in kinds["alone"]]
|
||||
patterns += [(daemon, c, line_pattern(daemon, c, True)) for c in kinds["with an argument"]]
|
||||
for doc in checked_documents(LINES_FROM):
|
||||
text = own_words(read(doc))
|
||||
for daemon, command, rx in patterns:
|
||||
for m in rx.finditer(text):
|
||||
line = text.count("\n", 0, m.start()) + 1
|
||||
problems.append("%s:%d: `%s %s` is a daemon's line — the operator types `nox …` (ADR 0272)"
|
||||
% (doc, line, daemon, command))
|
||||
return problems
|
||||
|
||||
|
||||
def check_unique(retired, heads):
|
||||
problems = []
|
||||
seen = {}
|
||||
for head in heads:
|
||||
key = head.lower()
|
||||
if key in seen:
|
||||
problems.append("%s: %r heads two entries" % (GLOSSARY, head))
|
||||
seen[key] = True
|
||||
for word, _ in retired:
|
||||
if word.lower() in seen:
|
||||
problems.append("%s: %r is a head word and also retired" % (GLOSSARY, word))
|
||||
return problems
|
||||
|
||||
|
||||
def tools_list(retired):
|
||||
return sorted({w for w, scope in retired if scope in ("all", "tools")}, key=str.lower)
|
||||
|
||||
|
||||
def check_copy(retired):
|
||||
catalogue = os.environ.get("MESH_CATALOG_DIR")
|
||||
if not catalogue:
|
||||
print("NOT COMPARED: MESH_CATALOG_DIR is not set, so the catalogue's copy of the list was not read")
|
||||
return []
|
||||
path = os.path.join(catalogue, "retired-words")
|
||||
try:
|
||||
with open(path, encoding="utf-8") as handle:
|
||||
copy = [l.strip() for l in handle if l.strip() and not l.startswith("#")]
|
||||
except OSError as err:
|
||||
return ["the catalogue's copy of the retired words cannot be read: %s" % err]
|
||||
want = tools_list(retired)
|
||||
problems = []
|
||||
if sorted(copy, key=str.lower) != want:
|
||||
problems.append("the catalogue's retired-words differs from the glossary: it should list %s" % ", ".join(want))
|
||||
path = os.path.join(catalogue, "retired-lines")
|
||||
try:
|
||||
with open(path, encoding="utf-8") as handle:
|
||||
copy = [l.strip() for l in handle if l.strip() and not l.startswith("#")]
|
||||
except OSError as err:
|
||||
return problems + ["the catalogue's copy of the daemons' lines cannot be read: %s" % err]
|
||||
if sorted(copy) != lines_list():
|
||||
problems.append("the catalogue's retired-lines differs from this check's list: regenerate it with "
|
||||
"`python3 00-META/checks/words.py --list lines`")
|
||||
return problems
|
||||
|
||||
|
||||
def main(argv):
|
||||
retired, identifiers, heads = glossary()
|
||||
if argv[1:2] == ["--list"]:
|
||||
scope = argv[2] if len(argv) > 2 else "tools"
|
||||
if scope == "lines":
|
||||
words = lines_list()
|
||||
else:
|
||||
words = tools_list(retired) if scope == "tools" else sorted({w for w, s in retired if s in ("all", scope)})
|
||||
print("\n".join(words))
|
||||
return 0
|
||||
if not retired:
|
||||
print("words: the glossary names no retired word on a *Not:* line — the check would pass on anything")
|
||||
return 1
|
||||
problems = check_unique(retired, heads) + check_retired(retired, identifiers) + check_lines() + check_copy(retired)
|
||||
for p in problems:
|
||||
print(p)
|
||||
if problems:
|
||||
print("words: %d problem(s)" % len(problems))
|
||||
return 1
|
||||
print("words: %d retired words and %d identifiers, none in running prose; %d daemons' lines, none written; "
|
||||
"%d head words, each once" % (len(retired), len(identifiers), len(lines_list()), len(heads)))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
+1
-1
@@ -11,7 +11,7 @@ Imagine the mesh works as intended. What is different?
|
||||
|
||||
An agent says what it wants — from a terminal, a phone, a message — and the mesh takes it
|
||||
from there. It works out which nodes are involved, does the work, and returns a
|
||||
result. Nobody opens a terminal, recalls which node holds what, or follows a runbook
|
||||
result. Nobody opens a console, recalls which node holds what, or follows a runbook
|
||||
written months ago.
|
||||
|
||||
The interface is intent. The mesh handles the rest.
|
||||
|
||||
+84
-839
@@ -1,857 +1,102 @@
|
||||
# Glossary — the words of the mesh, by domain, and the ones it stopped using
|
||||
# Glossary — the words this repository uses, and the ones it stopped using
|
||||
|
||||
One name per thing. This page is the authority; where an older record says something else, that
|
||||
record keeps its words and this page says how to read them. It exists because the words kept drifting
|
||||
in conversation — "control plane", "controller" and "master" for one thing, "substrate" and
|
||||
"foundation" for another — and a mesh you cannot name precisely is a mesh two people describe differently. The rule and
|
||||
its check are ADR 0244;
|
||||
the domains are drawn in [to-be 49](../03-DESIGN/01-to-be/49-the-mesh-in-domains.md).
|
||||
record is being superseded, not this page. It exists because the terms kept drifting in
|
||||
conversation — control plane / controller / master / hub for one thing, substrate / foundation for
|
||||
another — and a mesh you cannot name precisely is a mesh two people describe differently.
|
||||
|
||||
## How to read this page
|
||||
## The mesh and its machines
|
||||
|
||||
A **domain** is an area of the mesh that owns a set of concepts: inside it each concept has one word,
|
||||
and the domain decides what that word means. Other domains use the word as it is defined here and never
|
||||
change its meaning. Every word is defined once, in the domain that owns it; a section lists the words
|
||||
it uses from other domains under *Uses*, with no second definition.
|
||||
- **node** — a machine in the mesh. There are 0..n of them, and each runs the host agent. A node is
|
||||
just a machine that has joined; being one implies nothing about what it runs.
|
||||
- **operator account** — the login name of the person who works on a node, stated on the node
|
||||
record; empty for a machine nobody logs into. Everything the mesh places under a person's home is
|
||||
resolved against this account's home and owned by it
|
||||
([ADR 0176](../02-DECISIONS/0176-the-operator-account-is-a-node-fact-and-a-home-is-a-placement-root.md)).
|
||||
Not "the user" (ambiguous with a module's own account) and not a name a definition carries.
|
||||
- **control-node** — the one node that also holds the `mesh-controller` seat. There is exactly one
|
||||
per mesh. "control-node" is not a separate kind of machine — it is a node that additionally runs
|
||||
the controller (and, today, the foundation). Lose it and the other nodes keep running what they
|
||||
were last told; they simply cannot be told anything new.
|
||||
- ~~master / slave~~, ~~hub / peer~~ — not used. The relationship is *controller and nodes*, and no
|
||||
node is subordinate: a node applies declarations on its own and survives the control-node dying.
|
||||
|
||||
Two fixed lines follow an entry where they apply, and `00-META/checks/words.py` reads them:
|
||||
## What runs the mesh
|
||||
|
||||
- *Not:* followed by struck-through words — the words this entry replaced. A struck word with no scope
|
||||
is retired everywhere; *(hq)* retires it in this repository's prose only; *(tools)* in the
|
||||
descriptions of the mesh's tools and seat verbs only. A retired word may be quoted, never used.
|
||||
- *Identifier until renamed:* followed by a name in a code span — code that still carries an old name.
|
||||
It may stand in a code span and nowhere else until the code is renamed, and then the line goes.
|
||||
|
||||
Nothing else on this page is struck through. A word that means different things in different domains
|
||||
is listed under [Homonyms](#homonyms), with the qualified form each domain uses.
|
||||
|
||||
## Words every domain uses
|
||||
|
||||
- **mesh** — the whole: the nodes, the modules assigned to them, the controller that decides and the
|
||||
records it keeps. One mesh per operator; a second mesh is a second everything.
|
||||
- **domain** — an area of the mesh that owns a set of concepts and the one word for each, as described
|
||||
in the section above (ADR 0244).
|
||||
ADR 0006 called seven of them the controller's *contexts*; they carry over as domains under the same
|
||||
names, except *observability*, which is now **Health and repair**. Where a domain's records live in
|
||||
the controller, the domain owns that store alone (ADR 0008).
|
||||
*Context* in that sense is not used in new writing; the word stays ordinary English, so this is
|
||||
checked by review, not by `words.py`.
|
||||
- **machine** — any computer: hardware, a virtual machine, before, during or outside its membership of
|
||||
the mesh. Prose about the computer itself — its disks, its kernel, the network it sits on, what was on
|
||||
it before the mesh — says machine.
|
||||
- **node** — a machine the mesh has adopted and owns. A machine becomes a node when it joins
|
||||
(ADR 0004), and from then on it runs the node-engine
|
||||
and is either **adopted** (the mesh holds what it found there as found) or **converged** (the mesh made
|
||||
it what it is) — ADR 0100.
|
||||
Prose about a member of the mesh says node. A node is just a machine that has joined; being one
|
||||
implies nothing about what it runs.
|
||||
- **module** — the unit the mesh assigns: one named thing, defined by its manifest, that a node runs —
|
||||
a service, a container, files, packages, its own code — and the tools it serves
|
||||
(ADR 0040). Everything configurable on a node is a module.
|
||||
- **seat** — a named role at a scope (node / site / mesh), held by a module assignment, from a **closed
|
||||
set** the mesh defines: a claim naming a seat outside the set is refused. A seat may **deliver a
|
||||
provision**, and its holder is then the mesh's answer for it when several modules provide it
|
||||
(ADR 0126). The set, with who holds each
|
||||
seat, is the overview of what a mesh has ([26 — The seats](../03-DESIGN/01-to-be/26-the-seats.md)).
|
||||
A seat carries **verbs** — the tools every holder must serve. A verb added to a held seat is first
|
||||
promised as optional, then served, then required (ADR 0246).
|
||||
A node seat may also name **the paths its holder owns**, such as `/etc/hosts`: a claim whose module does
|
||||
not declare them is refused, and only a claimant declares them (ADR 0287).
|
||||
- **operator** — the person who runs the mesh, and the one the mesh talks to.
|
||||
*Not:* ~~master~~ (hq)
|
||||
- **person** — any human, as against the mesh acting unattended: *a person's word* releases a held
|
||||
delivery, *a person* deletes a consumer's data. The operator is one; the word says that a human, not
|
||||
the mesh, acted.
|
||||
|
||||
## Module — what a module is and declares
|
||||
|
||||
**Purpose.** To say what one module is, in the one document every other domain reads.
|
||||
**Recorded in** the catalogue, one manifest per module. **Upstream of** every other domain; its words
|
||||
are a published vocabulary, fixed by the manifest's schema. **Decided by** ADR 0040, 0126, 0174, 0188,
|
||||
0207, 0236, 0262.
|
||||
**Uses:** module, seat, config value (Config), provision (Provisioning), data class (Data), health (Health
|
||||
and repair).
|
||||
|
||||
- **manifest** — the file a module is defined by (`module.json`): what it is, what it provides and
|
||||
requires, the seats it claims, the resources it declares, its tools, its data and its health. Say
|
||||
manifest in prose, not the file's name.
|
||||
*Not:* ~~module definition~~ (hq)
|
||||
- **resource** — one thing a manifest declares a node must have: a file, a directory, a service, a
|
||||
container, a package, a bundle, an archive. A module **declares** resources; the verb *declares* is
|
||||
this domain's and means what a manifest states.
|
||||
- **claim** — a module taking a spot on a seat: `claims: [{name, scope}]` in a manifest. A mesh-scoped
|
||||
exclusive claim is how the mesh says "there is one of me". A foundation seat is named after the role
|
||||
it guards: the `mesh-controller`, `postgres` and `nats` modules claim the `mesh-controller`,
|
||||
`mesh-store` and `mesh-broker` seats (ADR 0079,
|
||||
ADR 0116).
|
||||
- **kept region** — a marked block in a managed file the mesh writes *into*, where the operator's own
|
||||
lines survive every send and are given back when the module goes (ADR 0174). The other way a node
|
||||
varies a module, besides a config value; a **managed** file's marked block is one (Provisioning).
|
||||
- **bundle** — the artifact a module's own code is built into — its tools, a seat's implementation, a
|
||||
daemon — in any language the mesh has a toolchain for; never an image. A **tools bundle** speaks MCP to
|
||||
the tool runner. One module may declare several
|
||||
(ADR 0188).
|
||||
- **tool** — one capability a module serves through the tool runner, addressed `<module>.<tool>`. A
|
||||
seat's **verb** is the same thing carried by a seat rather than a module.
|
||||
- **mesh-sdk** — the library a module's own code is written against, including the SDK harness that serves a
|
||||
tools bundle to the tool runner. The SDK harness is part of it, not a library of its own.
|
||||
*Not:* ~~tools-sdk~~
|
||||
- **invokes** — the manifest word for the tools a module calls, `<module>.<tool>` each or `*` for every
|
||||
one. A grant on the publish side and nothing else; a module that declares none calls nothing.
|
||||
- **upgrade policy** — how a module's new builds reach its nodes: `roll` (one node first, judged, then
|
||||
the rest), `together`, or `record` (register the build, send it nowhere)
|
||||
(ADR 0236).
|
||||
The controller's verb `upgrade` shows it.
|
||||
|
||||
## Config — the values someone may change
|
||||
|
||||
**Purpose.** To hold, check and resolve every value a module reads that someone may change, and to say
|
||||
where each resolved value came from. **Recorded by** the `mesh-config` module, in its buckets on the bus,
|
||||
with its store as the history behind them. **Upstream of** Provisioning, which places the values;
|
||||
downstream of Module and Placement. **Decided by** ADR 0262, 0277, 0348.
|
||||
**Uses:** module, node, manifest (Module), assignment (Placement), provision (Provisioning), proposal
|
||||
(Operator and conversation).
|
||||
|
||||
- **config value** — anything a module reads that someone may change: a font size, a domain, an address.
|
||||
A module declares every one it reads under `config` in its manifest, with a type, a description,
|
||||
validation, whether it is trusted, optionally the predefined key it falls back to, and a default or
|
||||
`required`; it lands in a template file as `${config:<key>}` or is read by the module's code at runtime.
|
||||
It resolves from four sources, the most specific winning: the default in the manifest; **mesh config**,
|
||||
the value of the predefined key it falls back to for the whole mesh; **node config**, that key's value
|
||||
on one node; and **module-assignment config**, the module's own value on one node. There is no module
|
||||
value for the whole mesh. *Config* is the concept, the verb (`config`) and the manifest field
|
||||
(ADR 0348). Mesh config is a source level, the value set for the whole mesh; `mesh-config` is the module
|
||||
that keeps every level. The controller holds, mirrors and resolves no config value (ADR 0348 §9).
|
||||
Older records and documents say "setting" for a config value and "settings" for the concept; new writing
|
||||
says config value and config. The word goes on this entry's *Not:* line once the documents in force that
|
||||
still use it in running prose are reworded, which is issue 683; until then it is checked by review. The
|
||||
manifest field and the verb "settings" are kept as aliases until every module has moved (ADR 0348 §1,
|
||||
issue 630), and are named on the line below with the rest once issue 683 is done.
|
||||
*Identifier until renamed:* `takes-settings`, `${setting:}` — manifest forms kept as aliases until every
|
||||
module has moved to their config names (ADR 0348 §1, issue 630).
|
||||
What a "flavor" once varied is a config value or a separate module.
|
||||
*Not:* ~~flavor~~
|
||||
- **predefined key** — a key of the catalogue's `predefined.json`, the fixed list of keys for which mesh
|
||||
config and node config hold values, each with its type, description, validation and whether it is
|
||||
trusted, such as `font-size`. A config value may name the predefined key it falls back to, of its own
|
||||
type; one that names none resolves from its default and the module's own value only. The list is the
|
||||
catalogue build's: a change to it is validated with every manifest naming a key, and removing a key still
|
||||
named or still holding a value is refused (ADR 0348).
|
||||
- **trusted** — said of a config value that decides what runs or who can reach what. At every level it is
|
||||
set only at the controller's terminal or through `config` with propose, on the operator's Approve; a
|
||||
module value that falls back to a trusted predefined key is itself trusted (ADR 0277, ADR 0348 §8). Not
|
||||
the trusted sender of an operator message (see [Homonyms](#homonyms)).
|
||||
- **preference** — a config value whose manifest gives it a default, because its value is true of the
|
||||
software and only tuned per node: a font size, a width
|
||||
(ADR 0262).
|
||||
A config value with no default is `required`: the operator's own value, such as a domain or an
|
||||
identity, and while nothing sets it the module's files are not written (ADR 0348).
|
||||
*Not:* ~~tunable~~ (hq)
|
||||
- **`mesh-config`** — the module that owns every config value, and not the source level mesh config: it
|
||||
holds the seat `mesh-config` once for the mesh, validates, takes a trusted value only from the
|
||||
controller's terminal or the operator's Approve, keeps the history with who set it and why, and resolves
|
||||
the four sources (ADR 0348).
|
||||
|
||||
## Core — the mesh's own machinery
|
||||
|
||||
**Purpose.** To keep the controller, the bus, the store and every node's engine running and agreed, so
|
||||
every other domain has something to run on. **Recorded by** the controller and the store.
|
||||
**Upstream of** every domain but Module; the others use the bus and the store as they are.
|
||||
**Decided by** ADR 0005, 0006, 0067, 0106, 0141, 0175, 0227, 0229, 0289.
|
||||
**Uses:** node, module, seat.
|
||||
|
||||
- **controller** — the component that decides what each node should be, holds the mesh's records, and
|
||||
tells nodes over the bus. The relationship is *controller and nodes*, and no node is subordinate: a
|
||||
node applies its declaration on its own and survives the control-node dying.
|
||||
*Not:* ~~control plane~~, ~~slave~~ (hq), ~~mesh-control~~ (hq)
|
||||
- **mesh-controller** — the module that runs the controller. It claims the `mesh-controller` seat at
|
||||
mesh scope, which is what makes it singular.
|
||||
- **control-node** — the one node that also holds the `mesh-controller` seat. There is exactly one per
|
||||
mesh. It is not a separate kind of node — it is a node that additionally runs the controller (and,
|
||||
today, the foundation). Lose it and the other nodes keep running what they were last told; they simply
|
||||
cannot be told anything new.
|
||||
- **node-engine** — the program on every node that applies what the controller declares: it receives
|
||||
the node's declaration, writes the files, runs the services and containers, and reports what it did.
|
||||
It is the engine, not a module: it owns no file's content, and every file it writes belongs to the
|
||||
module that declared it. *Agent* is avoided because that word means a coding agent here. A record
|
||||
written before the rename keeps the old name. "The host" is retired in this repository's prose only:
|
||||
in the tools' descriptions it is also an SSH server's or a container runtime's own word, so a
|
||||
description that means the node-engine by it is found by review.
|
||||
*Not:* ~~host agent~~, ~~the host~~ (hq), ~~node host~~
|
||||
*Identifier until renamed:* `mesh-host` — the repository, its binary and its service unit; the rename is
|
||||
[issue 354](../04-ISSUES/354-a-decided-rename-was-never-scheduled-and-the-operator-believed-it-live/00-report.md)
|
||||
- **tool runner** — the one program on each node that loads every assigned module's tools bundle and
|
||||
serves every tool and every held seat's verb on the subjects the memberships issue; the node-engine
|
||||
supervises it as a process beside the services it runs, never a container
|
||||
(ADR 0175,
|
||||
which called it "node tools"). Its mode on loopback is the mesh MCP server.
|
||||
*Not:* ~~node tools~~, ~~tool runtime~~ (hq)
|
||||
*Identifier until renamed:* `node-tools` — the module, its unit and its bus account; the rename is
|
||||
[issue 354](../04-ISSUES/354-a-decided-rename-was-never-scheduled-and-the-operator-believed-it-live/00-report.md)
|
||||
- **foundation** — the store and the bus, raised at genesis before any module system exists. The
|
||||
foundation is not a third thing beside the store and the bus — it *is* those two, named together.
|
||||
*Not:* ~~substrate~~
|
||||
- **genesis** — raising the foundation and the controller on an empty mesh, by the one installation
|
||||
done by hand (ADR 0067).
|
||||
- **store** — the one postgres server. It holds the controller's own databases (`inventory`,
|
||||
`identity`, `licences`, each owned by one domain, ADR 0008) and every module's own database. One server,
|
||||
many databases — never one shared mesh database. Bare *store* means this and nothing else
|
||||
(see [Homonyms](#homonyms)).
|
||||
- **controller** — the component that decides what each node should be, holds the mesh's records,
|
||||
and tells nodes over the broker. Replaces **"control plane"** (borrowed from networking's
|
||||
control-plane/data-plane, and opaque here).
|
||||
- **mesh-controller** — the module that runs the controller. It **claims** the `mesh-controller`
|
||||
seat at mesh scope, which is what makes it singular. Replaces the module name **`mesh-control`**.
|
||||
(The git repository has been renamed `mesh-control` -> `mesh-controller` on the forge; the module,
|
||||
container and image it produces are `mesh-controller`.)
|
||||
- **foundation** — the store and the broker, raised at genesis before any module system exists.
|
||||
Replaces **"substrate"** (a biology metaphor that landed for no one). The foundation is not a
|
||||
third thing beside the store and broker — it *is* those two, named together.
|
||||
- **store** — the one postgres server. It holds the controller's own context databases
|
||||
(`inventory`, `identity`, `licences` — a context owns its store, [ADR 0008](../02-DECISIONS/0008-a-context-owns-its-store.md))
|
||||
and every module's own database. One server, many databases — never one shared "mesh database".
|
||||
- **bus** — the mesh's own nervous system: NATS, one per mesh, carrying every link the mesh has —
|
||||
control, declarations, builds, events, tool calls (ADR 0106).
|
||||
A module reaches it by requiring `mesh-bus` (ADR 0128);
|
||||
one that does not require it has no account on it. Held by the `mesh-broker` seat, which is named
|
||||
after the role rather than the server, so the server can change without the seat doing so. The `nats`
|
||||
module holds it.
|
||||
- **the deprecated broker** — the lavinmq module. It was the mesh's bus and is not any more. It keeps
|
||||
running as an **ordinary provider** of the `amqp` provision, for modules that need a message broker
|
||||
of their own the way something needs a database
|
||||
(ADR 0131) — it claims no
|
||||
seat, is not foundation, is never raised at genesis, and a mesh that never installs it is complete.
|
||||
Say *the deprecated broker*, not *the compatibility broker* (it serves the mesh's own modules, not only
|
||||
the predecessor's) and not *the AMQP broker* (naming it after a protocol invites describing the bus by
|
||||
contrast with it, which is backwards).
|
||||
- **layer** — one of the four levels the mesh is built in, from the bottom: the node-engine, the
|
||||
foundation, the controller, the surfaces. The records' `topic: the tiers` and `repos.md` say *tier* for
|
||||
this; new prose says layer (see [Homonyms](#homonyms)).
|
||||
- **lease** and **epoch** — which controller may send (the lease, held in the bus and renewed), and the
|
||||
order of what it sent (the epoch, which a node reads before it accepts a declaration)
|
||||
(ADR 0229).
|
||||
control, declarations, builds, events, tool calls
|
||||
([ADR 0106](../02-DECISIONS/0106-the-bus-is-nats.md)). A module reaches it by requiring
|
||||
`mesh-bus` ([ADR 0128](../02-DECISIONS/0128-the-mesh-bus-is-required-not-ambient.md)); one that
|
||||
does not require it has no account on it. Held by the `mesh-broker` seat, which is named after
|
||||
the *role* rather than the server, so the server can change without the seat doing so.
|
||||
- **the deprecated broker** — the lavinmq module. It was the mesh's bus and is not any more. It
|
||||
keeps running as an **ordinary provider** of the `amqp` provision, for modules that need a
|
||||
message broker of their own the way something needs a database
|
||||
([ADR 0127](../02-DECISIONS/0127-amqp-is-a-provision-not-the-bus.md) (superseded by [ADR 0131](../02-DECISIONS/0131-everything-on-the-mesh-speaks-to-the-broker-seat.md))) — no seat, not foundation,
|
||||
never raised at genesis, and a mesh that never installs it is complete.
|
||||
|
||||
- **record mirror** — a key-value bucket on the bus that the controller writes from one kind of its
|
||||
records (nodes, settings, declared config, predefined keys), after each change commits in its store and only
|
||||
when the value differs, with one event per write. The store stays the record; the self-check compares
|
||||
the two and raises `mirror-drifted` on any difference
|
||||
(ADR 0289, ADR 0356). Builds, rollouts and assignments are no mirrors: they are
|
||||
the chain, kept on the bus as their only source (ADR 0408). Shortened to *mirror* where the
|
||||
text is about the controller's records; the builder's *mirrored* `upstream` image (Change and delivery) is
|
||||
a different thing, a copy of an outside image in the artifact store.
|
||||
Say *the deprecated broker*, not "the compatibility broker" (it serves the mesh's own modules,
|
||||
not only the predecessor's) and not "the AMQP broker" (naming it after a protocol invites
|
||||
describing the bus by contrast with it, which is backwards: the bus is the mesh's nervous
|
||||
system and this is a module).
|
||||
|
||||
## Placement — what runs on which node
|
||||
## What the mesh stores and serves
|
||||
|
||||
**Purpose.** To decide, send and apply what each node runs, and to say why.
|
||||
**Recorded by** the controller's `inventory` database. **Upstream of** Provisioning, Connectivity, Data
|
||||
and Health and repair. **Decided by** ADR 0100, 0126, 0176, 0181, 0207, 0221, 0252, 0261.
|
||||
**Uses:** node, machine, module, seat, manifest (Module), config value (Config).
|
||||
- **package** — what code resolves when it is **compiled**: an npm/cargo/pypi dependency, by
|
||||
**version**. Served by the **package-registry** (gitea). Only a builder talks to it.
|
||||
- **artifact** — anything a build produces and the mesh delivers to a machine by **digest**: an
|
||||
`image`, a mirrored `upstream` image, a `bundle` of the module's own code, an `archive`. Served by
|
||||
the **artifact-store**, an OCI registry that holds every kind as content-addressed blobs
|
||||
([ADR 0156](../02-DECISIONS/0156-an-artifact-is-what-a-build-produces-and-the-store-is-named-for-its-scope.md)).
|
||||
Every node pulls from it. An image is one kind of artifact, and a module is not an image.
|
||||
- These are two protocols, not one store being weak — see [ADR 0075](../02-DECISIONS/0075-two-stores-and-which-provides-what.md).
|
||||
|
||||
- **assignment** — a module put on a node. Its config values on that node (module-assignment config,
|
||||
Config) are `mesh-config`'s, never part of what the controller records (ADR 0348 §9).
|
||||
- **pending assignment** — an assignment of a module whose build is still running, kept, and made by the
|
||||
controller when that build registers the module, or ended with why when it does not
|
||||
(ADR 0261).
|
||||
- **scope** — where a seat or a claim holds: `node`, `site` or `mesh`. `node` is also the scope's
|
||||
value in a manifest.
|
||||
- **capacity** — how many holders a seat takes. A capacity-1 seat is exclusive. A higher-capacity seat
|
||||
is a **bench**: several holders coexist. A **replicated** bench has one holder per node, each on record
|
||||
and each answering the same, like `mesh-dns-resolver`
|
||||
(ADR 0223). A
|
||||
**kinded** bench has holders that are different modules, each claiming one **kind**, and a verb's
|
||||
subject carries the kind; `channel` and `intake` are the only ones (ADR 0234).
|
||||
- **installed / holding** — a module may be assigned (its package installed, its files placed) without
|
||||
holding the seat its family declares; *holding* is being the one — the login shell, the display session
|
||||
— on that node (ADR 0176).
|
||||
The module that holds a seat is its **holder**.
|
||||
- **depends on a seat** — a module needing a seat held on its node by some module, without holding it.
|
||||
Derived from the resources it declares, never stated: a `service` depends on `node-service-manager`, a
|
||||
`package` on `node-package-manager`, a `container` on `node-container-runtime`
|
||||
(ADR 0207).
|
||||
Not a claim: a module claims a seat it holds and declares resources.
|
||||
- **declaration** — what the controller sends one node: every resource of every module assigned there,
|
||||
composed. The noun is this domain's; what a manifest says is that it *declares* (Module), and a manifest
|
||||
is never called a declaration. The controller's verb `plan` previews a node's declaration.
|
||||
- **send** — the controller giving a node its declaration. The controller's verb `push` asks for a
|
||||
send (of one node, or of every node); prose says send. A git push and a phone's push notification are
|
||||
other things (see [Homonyms](#homonyms)).
|
||||
- **apply** — the node-engine making its node match its declaration; **reconcile** is doing so again
|
||||
until nothing differs.
|
||||
- **operator account** — the login name of the person who works on a node, stated on the node record;
|
||||
empty for a node nobody logs into. Everything the mesh places under a person's home is resolved
|
||||
against this account's home and owned by it
|
||||
(ADR 0181).
|
||||
Not a name a manifest carries.
|
||||
- **agent account** — the login every coding agent on a node runs as, when the node names one of its own;
|
||||
stated on the node record at the controller's terminal, never by a verb or a config value. Empty means agents
|
||||
run as the operator account. It cannot become root without a person, and the node-engine judges that on
|
||||
every look (ADR 0266).
|
||||
- **account group** — a group of the machine's group database that a module puts an account in, by
|
||||
declaring the account with that group and nothing else of it. Several modules may each add one to the
|
||||
same account; the mesh takes back only an account group it added
|
||||
(ADR 0252).
|
||||
Never *membership*, which is the bus's word.
|
||||
|
||||
## Provisioning — one module serving another
|
||||
|
||||
**Purpose.** To resolve which module serves a provision for which consumer, to wire the two, and to
|
||||
place every value a module needs into its files on its node. **Recorded by** the controller and
|
||||
`mesh-provisioner`. **Upstream of** Data; downstream of Module, Placement, Config and Identity.
|
||||
**Decided by** ADR 0027, 0084, 0138, 0225, 0230, 0263, 0349.
|
||||
**Uses:** module, seat, node-engine (Core), declaration (Placement), config value (Config), credential
|
||||
and vault (Identity and access).
|
||||
## How modules relate to the mesh
|
||||
|
||||
- **seat** — a named role at a scope (node / site / mesh), held by a module assignment, from a
|
||||
**closed set** the mesh defines: a claim naming a seat outside the set is refused. A seat may
|
||||
**deliver a provision**, and its holder is then the mesh's answer for it when several modules
|
||||
provide it ([ADR 0126](../02-DECISIONS/0126-a-module-declares-its-own-seats.md) (superseding [ADR 0110](../02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md))).
|
||||
The set, with who holds each seat, is the overview of what a mesh has
|
||||
([26 — The seats](../03-DESIGN/01-to-be/26-the-seats.md)). A seat has a **capacity**: a
|
||||
capacity-1 seat is exclusive (one holder); a higher-capacity seat is a **bench** (several holders
|
||||
coexist).
|
||||
- **claim** — a module taking a spot on a seat. `claims: [{name, scope}]` in a manifest. A
|
||||
mesh-scoped exclusive claim is how the mesh says "there is one of me". A foundation seat is
|
||||
named after the server it guards: the `mesh-controller`, `postgres` and `lavinmq` modules claim
|
||||
the `mesh-controller`, `mesh-store` and `mesh-broker` seats ([ADR 0079](../02-DECISIONS/0079-the-foundation-seats-are-named-after-their-servers.md)).
|
||||
- **provision** — a service one module `provides` and others `require`; the mesh resolves a provider
|
||||
and wires the two with an endpoint and a credential. A provision is a service you offer, a seat is a
|
||||
role you occupy, and the two meet where a seat delivers a provision: occupying the seat is what makes
|
||||
a module *the* provider of it.
|
||||
Every value a module needs is a provision from the domain that owns it: placement from the controller,
|
||||
config values from `mesh-config`, shared secrets from the vault, a consumer's values from the module
|
||||
providing it, an item of a collection from its contributor. A module says only what it needs and where
|
||||
it lands (ADR 0349).
|
||||
- **provider** and **consumer** — the module that provides a provision, and the module that requires
|
||||
it. Which of several providers serves a consumer is resolved
|
||||
(ADR 0084).
|
||||
- **pin** — a person's choice of provider for a consumer, which resolution then respects.
|
||||
- **endpoint** — where a consumer reaches its provider; an assignment binds it and says how far it
|
||||
reaches (ADR 0138).
|
||||
- **retire** — a provider stops serving a consumer the mesh no longer asks for; what it kept is deleted
|
||||
only by a person (ADR 0230).
|
||||
- **share** — a folder one node offers to the mesh's other nodes over the private network: the provision
|
||||
`nfs-share`, provided by the holder of the node seat `node-nfs-server`, one share per folder
|
||||
(ADR 0263). A share may also be granted to the machine's local network (ADR 0341). An **SMB share** is
|
||||
a folder the holder of the node seat `node-smb-server` serves to devices on the local network, and is
|
||||
not a provision (ADR 0341).
|
||||
- **mount** — a share, or an occasional source, made visible at a path on one node by the holder of the
|
||||
node seat `node-mounts`, as a mount unit and an automount unit, mounted on access (ADR 0263).
|
||||
- **occasional source** — a share on a device that is not a node and comes and goes, such as a USB disk;
|
||||
its absence is a state, *not present*, never a fault (ADR 0263).
|
||||
- **mesh-provisioner** — the module that places provisions on the nodes: it holds the seat
|
||||
`mesh-provisioner` once for the mesh, keeping which domain answers which kind of placeholder, what each
|
||||
module on each node needs, what is applied where and the items of every collection, and the seat
|
||||
`node-provisioner` on every node, which its provisioner client serves (ADR 0349).
|
||||
- **provisioner client** — mesh-provisioner's part on one node. For each module there it reads the
|
||||
placeholders of its template files, records the needs, watches each owner's values for the node, fills a
|
||||
template only once every value is there, has the node-engine write the file through a narrow local verb,
|
||||
makes the program take the change as the file declares (`restart-on`, `reload-on`, `read-on-use`), and
|
||||
reports `prov.applied`. It runs as a service with a bus account of its own, never in the tool runner
|
||||
(ADR 0349).
|
||||
- **collection** — a provision with many providers. The **collector**, a module that takes additions from
|
||||
others (a shell's snippets, a bar's blocks), requires it; each **contributor** ships its **item** as a
|
||||
template in its own repository and provides it. The provisioner client fills the collector's template
|
||||
from the items of the contributors placed on its node, and an item goes when its contributor leaves the
|
||||
node (ADR 0349).
|
||||
- **owned** — a placed file that is the mesh's whole: the provisioner writes it back whenever it differs.
|
||||
It is one of three answers a manifest gives for each file it places (`owner-at-runtime`): **owned**,
|
||||
**seed** or **managed**. Whichever it is, the file has one writer: the node-engine writes it only on the
|
||||
provisioner client's word (ADR 0349).
|
||||
- **seed** — a placed file the provisioner writes once; from then on it is the program's.
|
||||
- **managed** — a placed file of which only its named keys or a marked block (a kept region, Module) are
|
||||
the mesh's; the program keeps the rest.
|
||||
and wires the two with an endpoint and a credential. A provision is a service you offer, a seat
|
||||
is a role you occupy, and the two meet where a seat delivers a provision: occupying the seat is
|
||||
what makes a module *the* provider of it.
|
||||
|
||||
## Identity and access — who may do what
|
||||
## The surfaces
|
||||
|
||||
**Purpose.** To issue, hold, rotate and check the credentials and grants by which modules, nodes, agents
|
||||
and the operator act. **Recorded by** the controller's `identity` and `licences` databases and the vault.
|
||||
**Upstream of** Provisioning, Change and delivery, and Operator and conversation.
|
||||
**Decided by** ADR 0085, 0113, 0183, 0225, 0234, 0386, 0387, 0388.
|
||||
**Uses:** module, node, seat, tool (Module), operator, consumer and provision (Provisioning), agent and ask (Operator and conversation).
|
||||
|
||||
- **credential** — anything that proves an identity: a password, a key, a token. A **pair credential**
|
||||
is the two ends of one credential, the consumer's and the provider's.
|
||||
- **secret** — a credential or other value the vault keeps sealed and the node-engine unseals into a
|
||||
file at apply (ADR 0085).
|
||||
- **secret family** — a module's own secrets declared as one name ending in `-*`; each member is the prefix
|
||||
and a name (`smb-password-<source>`, one per source the module's config values name), given by its full name
|
||||
and never made by the mesh
|
||||
(ADR 0283).
|
||||
- **vault** — the module that owns every secret (`mesh-vault`,
|
||||
[24 — The secrets vault](../03-DESIGN/01-to-be/24-the-secrets-vault.md)).
|
||||
- **identity** — a person, an agent or a module as the mesh knows it: a name, a kind, a role, and whether it
|
||||
is switched on. Every identity and role is held by the identity seat, in the bus's key-value store, and
|
||||
changes only through its verbs (ADR 0386).
|
||||
- **role** — what an identity may call, as a set of grants (ADR 0386). Not a seat: a bare *role* is a
|
||||
homonym (see [Homonyms](#homonyms)).
|
||||
- **grant** — one line of a role: an effect everywhere, an effect on one seat, or one named tool, and may be
|
||||
limited to named machines; a dangerous tool is granted only by name (ADR 0386). A module's role
|
||||
is made from what its manifest declares it calls (ADR 0386 §4).
|
||||
- **effect** — a tool's mark, set by its module in its manifest: **reads**, **acts** or **dangerous**; a
|
||||
tool served without one counts as dangerous (ADR 0387).
|
||||
- **break-glass agent** — the one agent that holds every tool and never asks: sealed until the operator, or
|
||||
a condition the operator named, opens it, open until the operator closes it, and every act it takes told
|
||||
at once (ADR 0387).
|
||||
- **signed login** — a bus login signed by a key the bus trusts, carrying its expiry, which the bus checks by
|
||||
its signature instead of reading a list (ADR 0388). The identity seat signs with one **signing key** per
|
||||
role.
|
||||
- **root key** — the key at the top of the signed logins, which vouches for the signing keys: sealed in the
|
||||
vault and used only to replace them, on the operator's Approve (ADR 0388).
|
||||
- **bus account** and **membership** — a module's or a node's account on the bus, and the subjects it
|
||||
is issued. The bus server calls its accounts *users*; the mesh says bus account.
|
||||
- **rotate** — replacing a credential without a consumer holding one the provider does not know.
|
||||
- **licence** — a model-access account the licence manager holds and **binds** to a consumer, the
|
||||
coding agent on a node (ADR 0183).
|
||||
Taking a consumer off a licence is **unbinding** it; the licence manager's verb for it is still
|
||||
`release`, an identifier to rename.
|
||||
- **proof** — what an authorising answer carries: a verified sender, a one-time code, a security key's
|
||||
touch (ADR 0234). How much proof a kind of ask needs is its **assurance level**.
|
||||
- **controller's terminal** — a command line the controller runs as typed by its operator, not asked through a
|
||||
verb, so a change that is the terminal's alone is taken (issue 339, issue 340) — or, since
|
||||
ADR 0277,
|
||||
proposed through a verb and set on the operator's warrant. In the controller's code it is a
|
||||
line run without `MESH_VERB`. The operator reaches it with `mesh-cli` from their own account on a node where no
|
||||
agent can run as that account: the control-node, and a node whose agents run under an account of their own
|
||||
(ADR 0272).
|
||||
|
||||
## Change and delivery — a commit on its way to the nodes
|
||||
|
||||
**Purpose.** To take one commit from its pull request to every node that should run it, and to put it
|
||||
back when it fails. **Recorded by** the `mesh-delivery` module (deliveries) and the controller (the
|
||||
planner, the rollout). **Downstream of** Module, Core, Health and repair and Data.
|
||||
**Decided by** ADR 0162, 0218, 0236, 0237, 0238, 0239, 0250, 0267, 0270, 0276, 0323, 0401–0408.
|
||||
**Uses:** module, node, declaration and send (Placement), health (Health and repair), person.
|
||||
|
||||
- **merge check** — the two statuses a pull request carries before it may merge: the **merge gate**
|
||||
(`mesh/merge-gate`, the build seat's composed check of the modules the change touches) and the
|
||||
**repository check** (`mesh/repo-check`, the repository's own `merge-check.sh`)
|
||||
(ADR 0238). Both judge the merge of the pull request's head into a named **trunk commit**, made in the
|
||||
build seat's own clone and never pushed (ADR 0323).
|
||||
- **trunk commit** — the commit of the trunk a merge check's verdict was judged against: the trunk's head when
|
||||
the build seat cloned it. The verdict holds for that commit only, and goes out of date when the trunk moves
|
||||
(ADR 0323).
|
||||
- **build seat** — the seat whose holder builds modules, `node-build-agent`; its holder on a node is the
|
||||
**builder**. Any node may hold the seat, so there is no build node by nature
|
||||
(ADR 0237).
|
||||
*Not:* ~~build machine~~
|
||||
- **build queue** — the builds waiting to run, kept by the bus as three work queues a builder takes in order:
|
||||
delivery builds, then checks of pull requests marked ready, then every other check; each check queue keeps one
|
||||
request per pull request, the newest (ADR 0406); one entry in it is a
|
||||
**build request**.
|
||||
The queue's verbs still say *ask* for an entry; the conversation's ask is another thing
|
||||
(see [Homonyms](#homonyms)).
|
||||
- **package** — what code resolves when it is **compiled**: an npm/cargo/pypi dependency, by **version**.
|
||||
Served by the **package-registry** (gitea). Only a builder talks to it.
|
||||
- **artifact** — anything a build produces and the mesh delivers to a node by **digest**: an `image`, a
|
||||
mirrored `upstream` image, a `bundle` of the module's own code, an `archive`. Served by the
|
||||
**artifact store**, an OCI registry that holds every kind as content-addressed blobs
|
||||
(ADR 0156).
|
||||
An image is one kind of artifact, and a module is not an image. A package and an artifact are two
|
||||
protocols, not one store being weak (ADR 0075).
|
||||
- **build source** — the files one artifact is built from, in one or more repositories: for a Go program
|
||||
its import closure (the directories of its packages, its Go module files) and its recipe; for any other
|
||||
artifact the paths its manifest names, by default its module's own directory. Each artifact of a build
|
||||
has its own, and a change rebuilds the artifacts whose build source it touches
|
||||
(ADR 0267).
|
||||
Older records (ADR 0015, ADR 0111) and designs say *build source* for the repository a module
|
||||
is built from; this entry narrows it to the files within, and that use reads as *the repository holding
|
||||
the build source*. Sharing a repository is not a dependency, and no build source is a whole repository
|
||||
holding another module's files.
|
||||
- **catalogue** — what the mesh holds of every module, at which commit; and the repository the
|
||||
modules live in. `mesh-catalog` and `catalog_modules` are identifiers and keep their spelling.
|
||||
*Not:* ~~catalog~~ (hq)
|
||||
- **delivery** — one commit in one repository on its way to the nodes, from its pull request's head
|
||||
being announced to one of its final states: one pull request, one status, one note
|
||||
(ADR 0239).
|
||||
From its merge on it is delivered by the rollout of the batch that answers it, and ends with that rollout
|
||||
(ADR 0276).
|
||||
Its states are one table (`proposed`, `checked`, `ready` or `rejected`, `published`, `delivering`,
|
||||
`held`, and the final six: `delivered`, `failed`, `superseded`, `stopped`, `retired` and `covered`;
|
||||
ADR 0250,
|
||||
ADR 0270). Not *change* (a diff). *Pipeline* is the predecessor's build-and-deploy
|
||||
mechanism, which the as-is designs describe; it is never a word for a delivery.
|
||||
*Not:* ~~deployment~~ (hq)
|
||||
- **covered** — the end of a delivery that a later merge containing it delivered: every module it moves was
|
||||
sent to every machine running it by the rollout of a later merge into the same trunk, delivered on its own
|
||||
passing check, whose merge commit the forge says contains this one. Not *superseded*, which is a delivery
|
||||
whose rollout a newer delivery took over before it ended; nor *delivered*, which its own rollout or an empty plan
|
||||
ends in (ADR 0270).
|
||||
- **delivery group** — two or more deliveries sharing a pull request head branch name across the mesh's
|
||||
repositories, delivered as one unit in an order declared (`after:`) or inferred by the planner. One
|
||||
level: a group holds deliveries, never groups. Its state is derived from its members, never set.
|
||||
- **hotfix** — one pull request in one repository, delivered at its merge (ADR 0401).
|
||||
- **feature** — one branch name across its repositories: in two or more, its pull requests are one delivery group,
|
||||
merged together and delivered member by member in the group's order; in one, a single pull request delivered as a
|
||||
lone delivery (ADR 0401). Work is never held back on a branch other than a trunk: the trunk is what runs.
|
||||
*Not:* ~~release branch~~ (hq)
|
||||
- **toolchain** — a language's compiler and tools, installed by that language's build module on every node holding
|
||||
the build seat, pinned to one version that is the same on every builder; its version is part of a build's reuse
|
||||
key (ADR 0405). No longer an image.
|
||||
- **delivery plan** — what a delivery does to the mesh: its **build plan** (the modules it moves and
|
||||
their dependents, in tiers), its **deploy plan** (per node, what it receives and what waits for a
|
||||
person) and its verdict (the composed nodes, the replays). Computed by the controller's planner from a
|
||||
diffset; the controller's verb `delivery-plan` shows it. *Deploy* is used in this one place.
|
||||
*Not:* ~~change plan~~, ~~release plan~~
|
||||
- **merge window** — the rolling wait in which the controller collects merges before it plans them: each
|
||||
merge restarts it, and it closes when none arrived for its length (90 seconds by default) or at the latest
|
||||
10 minutes after its first merge (ADR 0276). Kept as two keys with a time to live on the bus; the first to
|
||||
expire closes the batch (ADR 0407).
|
||||
- **batch** — the merges one merge window collected, planned once as one rollout. For a repository merged more
|
||||
than once in it, the rollout takes the latest commit, and the earlier merges are answered by it. Not *group*,
|
||||
which is a delivery group.
|
||||
- **mesh-delivery** — the module that owns deliveries and delivery groups, holding the mesh-scoped seat
|
||||
of the same name. It records and decides; the controller sends, judges and rolls back when it is asked.
|
||||
- **rollout** — the controller's sending of one batch's builds across nodes, at most one trunk commit per
|
||||
repository, tier by tier, to every node that runs each module at once (ADR 0276, ADR 0360). Rollouts that share a
|
||||
repository, a module or a graph edge run one after another; others run side by side (ADR 0407). It names every
|
||||
merge it answers, and its record is one key of the bus's rollouts bucket, a link of the chain (ADR 0408). A primitive the delivering stage
|
||||
asks for, not an object anyone manages. A **tier** is one step of a rollout: a module, then the modules
|
||||
built against it. The controller's verb `plans` lists the rollouts. A module is rolled out by its `roll`
|
||||
policy.
|
||||
*Not:* ~~walk~~ (hq)
|
||||
- **first-node gate** — the judgement on a delivery's first node: healthy three times, no witness put it
|
||||
back, no condition raised since the send (ADR 0236 §2). Not *the gate* bare (see [Homonyms](#homonyms)).
|
||||
- **core module** and **leaf module** — a core module is one on the controller's own path (the controller,
|
||||
the node-engine, the tool runner, the bus, mesh-delivery) or the mesh's resolver, whose judgement protects
|
||||
the mesh's ability to reach a node at all; a leaf module is every other module. Decided by
|
||||
ADR 0282.
|
||||
Not a *core component* (ADR 0236's controller, node-engine, tool runner and bus, each with a health probe):
|
||||
the core modules add mesh-delivery and the resolver.
|
||||
- **delivery time** — from a delivery's merge to every node of its deploy plan that is heard from running its
|
||||
modules' builds and reporting them healthy. Its **delivery budget** is what it is held to: three minutes for
|
||||
a leaf module (ADR 0304), ten for a core module (ADR 0282). Not the rollout's time, which ends at the send. A
|
||||
measure inside the fix budget, not the operator's target (ADR 0313).
|
||||
- **fix time** — from an issue's first move to `fixing` to the moment the last of its fixes is delivered: the
|
||||
operator's measure of a fix, the agent's work, the review and the checks included, the delivery time its last
|
||||
part. Its **fix budget** is what it is held to: five minutes for a leaf module, ten for a core module, counted
|
||||
from the move to `fixing`; an issue whose fixes move no module has none (ADR 0313). Not the delivery time,
|
||||
which starts at the merge.
|
||||
- **release** — a person's word that a held delivery goes on; the verb `mesh-delivery.release`. Used in
|
||||
this sense only.
|
||||
- **rollback** — putting a node back on the build it ran before, by something other than the build being
|
||||
judged (a witness, ADR 0236).
|
||||
- **the lab**, **scenario**, **replay** — a real mesh raised to run a change against before it reaches
|
||||
nodes; what a lab run declares; and a recorded failure run again to prove a fix (ADR 0016, ADR 0237).
|
||||
|
||||
## Health and repair — what is wrong, and putting it right
|
||||
|
||||
**Purpose.** To notice what is wrong with the mesh, say it once, repair what may be repaired unattended,
|
||||
and record what was done by hand. **Recorded by** the controller's condition store.
|
||||
**Upstream of** Change and delivery (the first-node gate) and Operator and conversation (a condition
|
||||
becomes a message). **Decided by** ADR 0227, 0231, 0240, 0252, 0283.
|
||||
**Uses:** node, module, node-engine (Core).
|
||||
|
||||
- **health** — a module's or a core component's statement of how it is alive and ready, which the
|
||||
node-engine judges (ADR 0240).
|
||||
- **relogin needed** — an account's health when the group database lists it in an account group and its
|
||||
running session, started before, does not have that group yet. Said by the node-engine as the module's
|
||||
resource of kind `account`; it clears at the first look after a new login (ADR 0252).
|
||||
- **waiting** — a resource's health when its module's tool check says it waits for the operator: a named own
|
||||
secret or config value of the module, not given yet, without which a part cannot work. Not a fault: the
|
||||
controller checks it and says it as the condition `needs-operator`, naming the act, and the first-node gate
|
||||
reads it as *waits for a person*
|
||||
(ADR 0283).
|
||||
- **probe** — one look at one thing's health, run by whoever owns the verdict; what a probe returns is a
|
||||
**finding**, before it becomes a condition.
|
||||
- **signal** and **watchdog** — something that must keep happening (a heartbeat, a report after a send),
|
||||
and the watcher that raises a condition when it stops (to-be 45 §3).
|
||||
- **condition** — one open fact about something the mesh owns that is wrong, with a key and a severity;
|
||||
raised and cleared only by observation
|
||||
(ADR 0231).
|
||||
What a wrapped dashboard raises is its own; the mesh's notion is a condition.
|
||||
*Not:* ~~alert~~ (hq)
|
||||
- **self-check** — the controller's own examination of the mesh, run on demand and on a schedule
|
||||
(to-be 45 §4). Its verb is `doctor`, an identifier; prose says self-check.
|
||||
*Not:* ~~doctor~~ (hq)
|
||||
- **healer** — a registered response to one condition kind: its **repair** (the ordinary path again),
|
||||
its **budget**, its back-off and its brake. A healer may not withdraw, delete or recreate data.
|
||||
- **drill** — something broken on purpose to see the mesh raise and clear its condition; a drill is
|
||||
never counted as a repair.
|
||||
- **rehearsal** — a question the operator starts at the controller's terminal (`rehearse`) and answers on a
|
||||
channel, to see an answer travel the whole way: its approval performs nothing and is recorded as the
|
||||
operator's decision (ADR 0259).
|
||||
Not a drill: nothing is broken.
|
||||
- **hand-act** — a repair a person made by hand, recorded so the mesh knows it happened.
|
||||
- **witness** — what rolls a core component back when its new build does not become healthy: never the
|
||||
component itself (to-be 45 §8).
|
||||
|
||||
## Data — what the mesh keeps, and getting it back
|
||||
|
||||
**Purpose.** To know every item of data a module holds, how precious it is, and to keep it recoverable.
|
||||
**Recorded by** the manifests' `data` declarations and every node's `node-backup` seat.
|
||||
**Upstream of** Change and delivery and Health and repair. **Decided by** ADR 0232, 0233, 0235.
|
||||
**Uses:** module, node, provider and consumer (Provisioning), person.
|
||||
|
||||
- **data class** — how precious an item of data is, ranked by the operator: `irreplaceable`,
|
||||
`valuable`, `rebuildable`, `cache`; and `none` for a provision that keeps nothing of anybody's
|
||||
(ADR 0233).
|
||||
- **backup** — a copy kept elsewhere by the node's `node-backup` holder; **restore point** — one backup
|
||||
at one moment; **restore** — bringing it back beside the live data, never over it.
|
||||
- **stream snapshot** — the bus's own backup of each stream, taken by the module holding `mesh-broker`
|
||||
(ADR 0235).
|
||||
- **sticky binding** — a consumer's tie to the data a provider keeps for it, which moves only by a
|
||||
person (ADR 0232).
|
||||
|
||||
## Connectivity — how nodes and modules reach one another
|
||||
|
||||
**Purpose.** To make every node and module reachable by name where it should be, and unreachable where
|
||||
it should not. **Recorded by** the controller. **Upstream of** Provisioning and Health and repair.
|
||||
**Decided by** ADR 0007, 0117, 0138, 0223, 0226, 0247.
|
||||
**Uses:** node, machine, assignment and endpoint.
|
||||
|
||||
- **private network** — the mesh's own encrypted network between its nodes, on which every node has an
|
||||
address and a mesh name (ADR 0226).
|
||||
The network the machine sits on without it is the **underlay**.
|
||||
*Not:* ~~overlay~~ (hq)
|
||||
- **anchor** and **hub** — the roles a node plays for the private network: the anchor is reachable from
|
||||
outside and every node reaches it; a hub relays for nodes that cannot reach each other directly.
|
||||
- **resolver** — a seat holder that answers the mesh's names; a node lists only the mesh's resolvers
|
||||
(ADR 0223), or, where it holds one, its own resolver, which asks them. **uplink** — a node's
|
||||
connection to the outside network, a seat (ADR 0117). **hostname** — a node's own name,
|
||||
a seat.
|
||||
- **split DNS** — resolving names by domain on one machine: a VPN's domains through the VPN's servers
|
||||
over its link, every other name through the mesh's resolvers. The provision `split-dns`, provided by
|
||||
the holder of the node seat `node-resolver`, the machine's **own resolver**, which exists only where
|
||||
something requires it (ADR 0247).
|
||||
- **proxy** and **public name** — the module that answers a public name and forwards it to an
|
||||
endpoint on the private network.
|
||||
- **packet filter** — what the mesh enforces on a node about which packets pass, the
|
||||
`node-packet-filter` seat and its verb `rules`. A **found firewall** is a program the mesh found on a
|
||||
machine and keeps retired or in force; *firewall* says that, and a wrapped program's firewall is its own.
|
||||
- **intrusion prevention** and **ban** — refusing a source that misbehaved, and the refusal itself.
|
||||
- **reach** — how far an assignment's endpoint may be reached from: the node, the private network, or
|
||||
the outside (ADR 0138).
|
||||
|
||||
## Operator and conversation — the person the mesh works for
|
||||
|
||||
**Purpose.** To let the mesh and its operator talk: tell, ask, answer, and act only on an answer it can
|
||||
trust. **Recorded by** the router (asks, warrants and the operator's identities).
|
||||
**Downstream of** Health and repair, Change and delivery, and Identity and access. An authorising
|
||||
answer acts in whichever domain asked, through that domain's own verb; the conversation owns the
|
||||
asking, never the act. **Decided by** ADR 0152, 0175, 0234, 0259, 0272, 0383, 0384, 0385, 0390.
|
||||
**Uses:** operator, person, node, module, seat, tool (Module), proof and identity (Identity and access), release (Change and delivery).
|
||||
|
||||
- **mesh MCP server** — the endpoint on a node's loopback through which every agent and person on that
|
||||
node reaches the mesh: the MCP server named `mesh`, with its five tools `mesh_overview`,
|
||||
`mesh_machine`, `mesh_search`, `mesh_describe` and `mesh_call`. It is the tool runner's loopback mode,
|
||||
not a module of its own. "Console" suggested a terminal or a shell, and the module `mesh-console` it
|
||||
once named no longer exists; "the tool bridge" and "the brain" named the predecessor's program.
|
||||
*Not:* ~~console~~ (hq), ~~mesh-console~~, ~~tool bridge~~
|
||||
- **mesh-cli** — the module on every node that places the operator's command, and the command itself:
|
||||
`/usr/local/bin/mesh-cli <command line>` asks the controller to run one of its command lines. It asks the
|
||||
node-engine on its own machine, which says to the controller which account is asking; the controller runs the
|
||||
line as the **controller's terminal** or as an ordinary call
|
||||
(ADR 0272).
|
||||
- **nox** — `mesh-cli`'s alias, in interactive zsh only. A script, `sudo` and a document meant to work everywhere
|
||||
say `mesh-cli`. The operator also calls the mesh itself *nox mesh*; on this page and in the records *nox* is the
|
||||
alias alone. It is the one command the operator types: every line the mesh addresses to the operator — a
|
||||
condition's words, a verb's answer, a README, a design — names a `nox` line, never a daemon's (the controller's
|
||||
or the node-engine's binary followed by a command), which `words.py` and the catalogue's check refuse
|
||||
(ADR 0272,
|
||||
its insight of 2026-10-09).
|
||||
- **channel / intake** — the two kinded benches the mesh talks to its operator through: `channel`
|
||||
sends, `intake` turns what arrives into one envelope. A holder of either declares **capabilities** from
|
||||
the fixed vocabulary `channel-capabilities/1`. Not *notifier* (that is the desktop's node seat, one
|
||||
holder of kind `desktop`) and not *bot* (that is one service's account).
|
||||
- **router** — the module that holds the `operator-channel` seat, orders the channels and checks every
|
||||
sender (ADR 0234).
|
||||
- **headline / verdict / explanation / resolved line** — what a condition says to the operator in plain
|
||||
words: a few words naming the thing and what is wrong; "Nothing for you to do." or "Needs you:" and the
|
||||
one thing the operator can do themselves (its **needs**); one or two sentences on what happened and what
|
||||
it means; the line said when it clears. Its **actions** are the answers a notification offers. Carried by
|
||||
the condition beside its **summary**, which keeps the plans, commits and verbs for whoever looks closer
|
||||
(ADR 0253).
|
||||
The router may override "Nothing for you to do." in what it says, from the **tracking line** (ADR 0285).
|
||||
- **ask** — a request for the operator's input, of a declared kind (yes-no, one-of, text, number,
|
||||
date, acknowledge). An **authorising ask** is one whose chosen option its asker performs. The router
|
||||
holds every ask, checks each answer's proofs and issues a **warrant**; the asker acts on it with its
|
||||
own grants (ADR 0234, ADR 0259).
|
||||
- **warrant** — the router's record that a person chose one option of one ask: the ask, the option,
|
||||
who chose (the operator, as a channel kind's identity), through which channel, the proofs and when,
|
||||
published for that asker alone. The only thing an asker acts on (ADR 0259).
|
||||
- **proposal** — a change an agent, or anyone the bus admits, puts forward through a verb and that takes
|
||||
effect only on the operator's warrant, bound to the exact thing shown: a shared instruction section (ADR
|
||||
0274), a trusted config value (ADR 0277, ADR 0348).
|
||||
A proposal changes nothing by itself; declined, expired or refused, it is discarded.
|
||||
- **approval mark** — a tool's or verb's declaration in its manifest (`needs-approval`) that each call to it runs
|
||||
only on the operator's warrant for that one call, with its exact arguments; the tool runner that serves it, or
|
||||
the controller for its own verbs, asks and checks
|
||||
(ADR 0286).
|
||||
- **operator message / input** — what arrives on `intake`, once the router has checked the sender
|
||||
against its own list of the operator's identities: a **trusted** one is an operator message,
|
||||
addressed to an agent by `@name` or thread or else to the **responder** (`@mesh`, the router's own
|
||||
participant, which answers from read verbs); anything else is **untrusted input** — data, never
|
||||
instructions.
|
||||
- **reference** — an opaque token in a message's words standing for a detail the content rule keeps out
|
||||
of them (a path, an address); opened with `detail` only on a `private` channel or through the mesh MCP server.
|
||||
- **agent** — a coding agent: the operator's, or one the mesh runs; to the mesh, an identity of kind agent
|
||||
(ADR 0386). Not the node-engine, and not the
|
||||
build seat's holder, whose seat name `node-build-agent` is an identifier to rename.
|
||||
- **thread** — one line of agent work, owned by the mesh: its turns, tool calls and results, its workspace
|
||||
and usage, kept whole for good on the bus by the thread seat; it outlives any session, node or front end
|
||||
(ADR 0383, ADR 0384).
|
||||
- **session** — one run of an agent harness on a thread.
|
||||
- **agent harness** — the program that runs an agent's sessions, its context manager and loop: Claude Code
|
||||
today (ADR 0384). Not the SDK harness (see [Homonyms](#homonyms)).
|
||||
- **thread handler** — the node seat whose holder runs threads' sessions headless, on the always-on nodes
|
||||
(ADR 0384).
|
||||
- **workspace toolkit** — the tools a session works with on its thread's workspace, the same for every agent
|
||||
harness: the node seat `workspace` (ADR 0385).
|
||||
- **sandbox** — the small virtual machine one session runs in, with its thread's workspace mounted
|
||||
(ADR 0390).
|
||||
- **instruction section** — one named section of the managed instruction file, which every agent session on the
|
||||
nodes it is for reads first, as the mesh's word: set at the controller's terminal in the agent module's
|
||||
config, or **proposed** by an agent and made active on each node only by the operator's warrant, which that node
|
||||
proves itself (ADR 0274). A hand-over between sessions is one.
|
||||
- **personal rule** — one short text in the operator's own words about how the operator wants their agent
|
||||
to work with them, registered for the operator (the **operator scope** of the agent module) and rendered
|
||||
into the agent's home on every node that names an operator account. Not a mesh-wide instruction section,
|
||||
which is the mesh's word about the mesh, set at the controller's terminal
|
||||
(ADR 0273).
|
||||
- **lesson** — an agent's proposal, recorded when the operator expresses frustration or corrects a way of
|
||||
working: the trigger (the operator's words), the cause (what the agent did) and the rule that would
|
||||
prevent it. A lesson is `proposed`, `accepted` or `declined`; an accepted lesson is a personal rule, and a
|
||||
proposed one is never read as one (ADR 0273).
|
||||
|
||||
## Media — what a person watches, listens to and reads
|
||||
|
||||
**Purpose.** To find, fetch and file copies of the works a person watches, listens to and reads, each in
|
||||
the library its kind names. **Recorded by** the `media-importer` module's config values and its memory of what
|
||||
it was told, and the servarr apps' own libraries. **Upstream of** nothing. **Decided by** ADR 0295, 0296,
|
||||
0302.
|
||||
**Uses:** module, seat, config value (Config).
|
||||
|
||||
- **content** — a work a person watches, listens to or reads: a film, an album and its tracks, an
|
||||
episode, a book, a session of a sport event. It is known by an identity from a catalogue outside the
|
||||
mesh, never by a file's name, and a work with no copy anywhere is still content (ADR 0302).
|
||||
- **kind** — a kind of content, declared as configuration, never code: its parts, the metadata fields
|
||||
that identify a work, the file extensions of its copies, and its folder and file templates. The first
|
||||
are music, movie, series, audiobook, ebook and sport event.
|
||||
- **media release** — one copy of a work an engine can fetch, such as an NZB or a video. Downloaded, it
|
||||
sits in the downloads folder, not yet filed. ADR 0302 says *release*.
|
||||
- **library** — one folder of one kind, filed by one importer: the media-importer's, named in its
|
||||
config, or a servarr app's. Several libraries may hold one kind, as TV Shows and Anime both hold
|
||||
series.
|
||||
- **media file** — a file on disk together with its metadata: its kind, the fields that identify the
|
||||
work, and what part of the work the file is. It is what the media-importer takes.
|
||||
- **media-importer** — the module that files a media file into its library at the place its kind's
|
||||
templates make of its metadata. It files only the downloads it was told about, and never into a
|
||||
servarr app's library (ADR 0302).
|
||||
- **download client** — a module that fetches a media release into the downloads folder and emits
|
||||
`download.added` and `download.completed` naming the download's id, its category and the path of its
|
||||
files: `nzbget`, `qbittorrent`, `youtube`.
|
||||
- **engine** — a module that searches for media releases: the holder of `usenet-search` (`nzbgeek`,
|
||||
ADR 0295) or of `youtube-search` (`youtube`, ADR 0296).
|
||||
- **grab** — an engine handing one media release to a download client. It names the work it is for, so
|
||||
the media-importer knows where the copy belongs.
|
||||
|
||||
## The record — how this repository works
|
||||
|
||||
Not a domain of the mesh but of the way it is built, listed because its words meet the mesh's.
|
||||
**Recorded in** this repository, the issue tracker and the knowledge base. **Decided by** ADR 0019, 0080, 0244, 0275,
|
||||
0284, 0290, 0291.
|
||||
|
||||
- **research effort**, **graduation**, **hand-off**, **playbook** — an investigation, held whole in the
|
||||
knowledge base and named "research NNN";
|
||||
its closing into a decision and a design; a design given to a code repository; a documented workflow
|
||||
in `00-META/process/`.
|
||||
- **decision record** — one numbered record in the knowledge base, named "ADR NNNN", never rewritten:
|
||||
**superseded** by a later record, or corrected by a marked **progressive insight**. Always qualified in this repository: bare
|
||||
*record* has other meanings (see [Homonyms](#homonyms)).
|
||||
- **clause** — one numbered statement a decision record makes, with its own status (`proposed`, `in force`,
|
||||
`changed`, `superseded`): a later record may change one clause and leave the rest
|
||||
(ADR 0290). Not a *rule*, which is a line of
|
||||
`00-META/how-we-build.md`.
|
||||
- **knowledge base** — what holds the decision records and research efforts whole (number, status, topic,
|
||||
typed links, clauses and reasoning, with every version), takes them as proposals, has them reviewed and accepted,
|
||||
and answers what holds: the seat `knowledge-base`, held by the module `mesh-knowledge`
|
||||
([55 — The knowledge base](../03-DESIGN/01-to-be/55-the-knowledge-base.md)). Not **the record**, the literal reader
|
||||
of this repository.
|
||||
- **design** — a document in `03-DESIGN/`: **as-is** (what runs) or **to-be** (what is being built).
|
||||
- **milestone** — one step of a design's work breakdown, usable on its own once delivered and named by what the
|
||||
operator or the mesh can then do; delivered as one feature (playbook 04, ADR 0401).
|
||||
- **epic** — a run of milestones in one design, released milestone by milestone (ADR 0401).
|
||||
- **issue** — a question about the mesh that someone must answer, numbered and tracked by the issue tracker, which
|
||||
holds it whole: its report, diagnosis and resolution included (ADR 0291);
|
||||
the folders in `04-ISSUES/` are imported there and removed.
|
||||
An **incident** is a past event that taught a rule.
|
||||
- **issue tracker** — what holds the mesh's issues and hands out their numbers: the seat `issue-tracker`,
|
||||
held by the module `mesh-issues` ([53 — The issue tracker](../03-DESIGN/01-to-be/53-the-issue-tracker.md)).
|
||||
Not hits's *item*; bare *ticket* is a homonym (see [Homonyms](#homonyms)).
|
||||
*Not:* ~~ticketing system~~ (hq), ~~ticket system~~ (hq)
|
||||
- **duplicate check** — the issue tracker's judgement, when an issue is opened, of whether it asks a question an
|
||||
older issue not resolved already asks; a certain duplicate is closed `wontfix` with its report kept on the older
|
||||
issue (ADR 0278).
|
||||
Its pass over the backlog is the verb `groom`.
|
||||
- **triage** — the issue tracker's judgement of an opened issue, made by the agent with no tools: whether it is a
|
||||
duplicate (the **duplicate check**), its **severity** and what it **needs**, a suggested owner, and the issues it
|
||||
depends on or relates to; it fills only what nobody gave (ADR 0279).
|
||||
- **ready** — of an issue: every issue it depends on is wontfix, or resolved with its fix delivered.
|
||||
- **worker** — of an issue: the agent or person, with its node (who@node), that moved it to `diagnosing` or `fixing`
|
||||
and works on it; another is refused while the worker is current, unless it gives a reason to **take it over**. A
|
||||
worker with nothing done on the issue for a day is **stale**. **Fixing** is the status an issue is at while its fix
|
||||
is worked on; an issue is resolved only from it (ADR 0284).
|
||||
Not a *claim* or a *holder*, which are a module's on a seat.
|
||||
- **abandoned** — of an issue: at `diagnosing` or `fixing`, its worker stale and no pull request of it open; another
|
||||
worker takes it over without a reason (ADR 0294).
|
||||
Not a *stale claim*: a claim is a module's on a seat.
|
||||
- **fix** — of an issue: a pull request its `fixed-by` names or it links; one closed without merging is
|
||||
**withdrawn**. An issue at `fixing` with every fix delivered is **delivered, not resolved** (ADR 0294).
|
||||
- **tracks** — of an issue and a condition: the issue links the condition's key with a link of kind `condition`. The
|
||||
**tracking line** is the line the router says in every message about a condition: the issues that track it and
|
||||
where each stands, or that none does (ADR 0285).
|
||||
- **hq check** — one of `00-META/checks/`, run by `merge-check.sh` as this repository's check.
|
||||
|
||||
## Homonyms
|
||||
|
||||
A word below means different things in different domains. In a governing document it is never bare;
|
||||
it is always the qualified form. **Checked by review, not by `words.py`:** a word list cannot tell one
|
||||
sense from another, so the reviewer looks for the bare word in the diff. Where a homonym is settled by
|
||||
renaming one sense, the old sense moves to a *Not:* line and becomes mechanical.
|
||||
|
||||
| Word | Sense | Say | Domain |
|
||||
|---|---|---|---|
|
||||
| plan | what the controller would send one node | that node's **declaration** (verb `plan`) | Placement |
|
||||
| | what a delivery would do to the mesh | **delivery plan** | Change and delivery |
|
||||
| | the sending of one batch of commits across nodes | **rollout** (verb `plans`) | Change and delivery |
|
||||
| | a step a person starts, like the bus's | **planned step** | Core |
|
||||
| membership | the subjects a bus account is issued | **membership** | Provisioning |
|
||||
| | an account in a group of the machine | **account group** | Placement |
|
||||
| push | the controller giving nodes their declarations | **send** (verb `push`) | Placement |
|
||||
| | a git push | **git push** | The record |
|
||||
| | a phone notification | **push notification** | Operator and conversation |
|
||||
| release | a person letting a held delivery go on | **release** | Change and delivery |
|
||||
| | taking a consumer off a licence | **unbind** (verb `release`, to rename) | Identity and access |
|
||||
| | a copy of a work an engine can fetch | **media release** | Media |
|
||||
| gate | the judgement on a delivery's first node | **first-node gate** | Change and delivery |
|
||||
| | the pull request status | **merge gate** | Change and delivery |
|
||||
| | a failed step stopping its module's later steps (ADR 0136) | *a failed step holds its module* | Placement |
|
||||
| tier | a level of the mesh | **layer** | Core |
|
||||
| | a step of a rollout | **tier** | Change and delivery |
|
||||
| | how much proof an ask needs | **assurance level** | Identity and access |
|
||||
| harness | the part of `mesh-sdk` that serves a tools bundle | **SDK harness** | Module |
|
||||
| | the program that runs an agent's sessions | **agent harness** | Operator and conversation |
|
||||
| | the TypeScript `provisioner` helper for database providers | **`provisioner`**, renamed by design 57 | Provisioning |
|
||||
| role | what a seat names, at a scope | **seat**; the bare word in older designs is tracked by issue 872 | Module |
|
||||
| | what an identity may call | **role** | Identity and access |
|
||||
| ask | a request for the operator's input | **ask** | Operator and conversation |
|
||||
| | an entry in the build queue | **build request** | Change and delivery |
|
||||
| store | the one database server | **store** | Core |
|
||||
| | the OCI registry | **artifact store** | Change and delivery |
|
||||
| | a node's own copy of its last declaration | **last declaration** | Placement |
|
||||
| record | a numbered decision | **decision record** | The record |
|
||||
| | the literal reader of this repository, searched first for a symptom | **the record** (the `records` module) | Operator and conversation |
|
||||
| | the upgrade policy that sends nowhere | `record` | Module |
|
||||
| knowledge base | the module holding the decision records and research efforts whole | **knowledge base** | The record |
|
||||
| | the predecessor's operational memory, in older playbooks (issue 430) | none: it no longer exists | — |
|
||||
| check | a pull request's two statuses | **merge check**, **merge gate**, **repository check** | Change and delivery |
|
||||
| | a delivery group's verdict | **composed check** | Change and delivery |
|
||||
| | the controller's own examination | **self-check** | Health and repair |
|
||||
| | a file in `00-META/checks/` | **hq check** | The record |
|
||||
| agent | a coding agent | **agent** | Operator and conversation |
|
||||
| | the build seat's holder | **builder** | Change and delivery |
|
||||
| the user | an account of a wrapped program, or of the bus server | **its account**, **bus account** | Identity and access |
|
||||
| | a human | **operator** or **person** | words every domain uses |
|
||||
| firewall | what the mesh enforces | **packet filter** | Connectivity |
|
||||
| | what was found on a machine | **found firewall** | Connectivity |
|
||||
| ticket | an answer's one-time token (ADR 0258) | **answer ticket** | Operator and conversation |
|
||||
| | a question someone must answer | **issue** | The record |
|
||||
| deploy | the per-node part of a delivery plan | **deploy plan** | Change and delivery |
|
||||
| | a wrapped program's own deploy | its own word | — |
|
||||
| managed | a placed file of which only named keys or a marked block are the mesh's | **managed** (`owner-at-runtime`) | Provisioning |
|
||||
| | a file the mesh writes, such as the agent's instruction file or `managed_settings` | the file's own name, or *a file the mesh places* | — |
|
||||
| trusted | a config value that decides what runs or who can reach what | **trusted** config value | Config |
|
||||
| | a sender on the operator's own list | **trusted** sender, its message an **operator message** | Operator and conversation |
|
||||
- **console** — the module (`mesh-console`) that puts the mesh's tools in front of whoever is on a
|
||||
machine: an MCP endpoint on the machine's loopback for an agent, the same endpoint for a person. It
|
||||
is assigned like any module, holds a credential the mesh minted, and calls tools under a grant its
|
||||
manifest declares (`invokes`). Loopback is the authority boundary: whoever is on the machine owns the
|
||||
mesh there ([ADR 0152](../02-DECISIONS/0152-the-operators-surface-is-a-module-the-console.md)).
|
||||
Not "the tool bridge", "the brain" or "the MCP server" — those name the predecessor's program or a
|
||||
protocol, and the console is a module.
|
||||
- **invokes** — the manifest word for the tools a module calls, `<module>.<tool>` each or `*` for
|
||||
every one. A grant on the publish side and nothing else; a module that declares none calls nothing.
|
||||
|
||||
## How this page is kept
|
||||
|
||||
A new word for an existing thing lands here first, in the domain that owns it, in the same change that
|
||||
introduces it in code. A word moves to another domain only with a decision record. A retired word goes
|
||||
on a *Not:* line and nowhere else on this page, and `words.py` then fails on it in running prose. A
|
||||
decision record keeps whatever word it was written with — those are immutable — so a word
|
||||
retired here may still appear in one, and the *Not:* lines are how to read it.
|
||||
|
||||
**How it is checked** (ADR 0244):
|
||||
`python3 00-META/checks/words.py`, run by `merge-check.sh` on every pull request, fails on a retired
|
||||
word or a bare identifier in running prose in `00-META/`, `03-DESIGN/`, `AGENTS.md`, `README.md`, and
|
||||
issues opened from 2026-10-07 — code spans, quotations and link targets excepted — and on a
|
||||
head word that heads two entries or is also retired. The words retired with no scope or with *(tools)*
|
||||
are copied into the catalogue as `retired-words`, whose own repository check holds the descriptions of
|
||||
the mesh's tools to them; a change here that retires such a word changes that copy in the same delivery.
|
||||
A decision record or research effort proposed to the knowledge base is held to the same words by its
|
||||
rules `retired-word` and `daemon-line`, which read this page (ADR 0290). Homonyms are checked by review.
|
||||
A new name for an existing thing lands here first, in the same change that introduces it in code. A
|
||||
record under `02-DECISIONS/` keeps whatever word it was written with — those are immutable — so a
|
||||
term retired here may still appear there, and the mapping above is how to read it.
|
||||
|
||||
+17
-42
@@ -1,10 +1,9 @@
|
||||
---
|
||||
status: canonical
|
||||
updated: 2026-10-16
|
||||
derives: the constitution page
|
||||
updated: 2026-08-23
|
||||
derives: knowledge-base constitution page
|
||||
decisions:
|
||||
- "ADR 0020"
|
||||
- "ADR 0391"
|
||||
- 02-DECISIONS/0020-the-mesh-is-governed-by-a-constitution.md
|
||||
---
|
||||
|
||||
# How we build
|
||||
@@ -38,13 +37,13 @@ incident behind it is not written down, and the fix is to write it down, not to
|
||||
| Rule | What it means |
|
||||
|---|---|
|
||||
| **Never write to a production database directly** | No insert, update, delete or schema statement executed against production by hand. Schema changes go through numbered migrations; data changes go through application code or the module's own capabilities. Raw statements skip every side effect the proper path has — events, audit, cache invalidation, fan-out. |
|
||||
| **Every schema change is a migration** | Numbered, in the module's own language, compiled with it. Both a baseline for a fresh installation *and* an incremental migration for installations that already exist. If code references a column, the migration creating it must exist. ADR 0013 |
|
||||
| **Every schema change is a migration** | Numbered, in the module's own language, compiled with it. Both a baseline for a fresh installation *and* an incremental migration for installations that already exist. If code references a column, the migration creating it must exist. [ADR 0013](../02-DECISIONS/0013-schema-changes-are-numbered-migrations.md) |
|
||||
| **Never bypass the pipeline** | No manual database edit, no manual restart as a workaround. Fix the cause and deploy. A workaround that works is a workaround that is never removed, and the next person cannot tell the node from its declaration. |
|
||||
| **Never create a symlink** | A hand-made link caused production data loss through container volume resolution, and the judgement needed to make a safe exception is exactly the judgement unavailable at the moment it matters. **The mesh creates none at all** (ADR 0012, which supersedes ADR 0012). The links the installer still reconciles are a migration, not a permission. |
|
||||
| **Never create a symlink** | A hand-made link caused production data loss through container volume resolution, and the judgement needed to make a safe exception is exactly the judgement unavailable at the moment it matters. **The mesh creates none at all** ([ADR 0012](../02-DECISIONS/0012-the-mesh-creates-no-symlinks.md), which supersedes [ADR 0012](../02-DECISIONS/0012-the-mesh-creates-no-symlinks.md)). The links the installer still reconciles are a migration, not a permission. |
|
||||
| **Never push directly to the main branch** | Branch, push, review, merge. Every merge is a human checkpoint, without exception — **including in this repository**. A documentation repository is not a lower tier of care; a decision record lands the same way a service does. |
|
||||
| **One change per pull request, and never merge unapproved work** | Unrelated improvements bundled together cannot be reviewed or reverted separately. And the checkpoint is **a person deciding, not a person clicking** — work may be merged by whoever wrote it once a human has explicitly approved *that merge*, and never on a standing permission, an instruction to do the work, silence, or the author's own judgement that it is ready. ADR 0023 |
|
||||
| **One change per pull request, and never merge unapproved work** | Unrelated improvements bundled together cannot be reviewed or reverted separately. And the checkpoint is **a person deciding, not a person clicking** — work may be merged by whoever wrote it once a human has explicitly approved *that merge*, and never on a standing permission, an instruction to do the work, silence, or the author's own judgement that it is ready. [ADR 0023](../02-DECISIONS/0023-approval-is-the-checkpoint.md) |
|
||||
| **Never open a pull request unprompted** | A permissions list saying it is allowed is not a request. |
|
||||
| **A failed step fails the job** | A sequence that continues past a failure does the next thing in the wrong place. Gate each step on the last. ADR 0010, and §5. |
|
||||
| **A failed step fails the job** | A sequence that continues past a failure does the next thing in the wrong place. Gate each step on the last. [ADR 0010](../02-DECISIONS/0010-delivery.md), and §5. |
|
||||
|
||||
### A failed step must stop the steps after it — how it was earned
|
||||
|
||||
@@ -70,7 +69,7 @@ reported failure, nothing stopped, and the damage happened somewhere nobody was
|
||||
- **Every runtime variable is declared.** A variable the module reads and the manifest does not
|
||||
declare is invisible to the mesh: it will not be generated, injected, or audited.
|
||||
- **Provisioned credentials arrive through declared requirements**, never hardcoded in code,
|
||||
compose files or scripts. ADR 0009
|
||||
compose files or scripts. [ADR 0009](../02-DECISIONS/0009-modules-and-the-graph.md)
|
||||
- **Never install a package by hand.** A package is declared in the manifest and arrives the
|
||||
way every other package does. A hand-installed package is invisible to the mesh: it is not
|
||||
declared, not reproduced on the next node, and not present after a rebuild — and the node
|
||||
@@ -83,7 +82,7 @@ reported failure, nothing stopped, and the damage happened somewhere nobody was
|
||||
- **Every standalone application gets its own repository**, with a manifest at its root,
|
||||
registered as a build source. Creating an application directory in the monorepo is a
|
||||
convention violation and reviewers reject it.
|
||||
ADR 0015
|
||||
[ADR 0015](../02-DECISIONS/0015-applications-live-in-their-own-repository.md)
|
||||
|
||||
### Migrations
|
||||
|
||||
@@ -97,7 +96,7 @@ reported failure, nothing stopped, and the damage happened somewhere nobody was
|
||||
surface is regenerated from the mesh database; a local edit survives one synchronisation and is
|
||||
then silently overwritten, bringing back whatever it fixed. Use the mesh operation that owns
|
||||
the value. If unsure whether a file is managed, ask the tooling — the answer is not visible
|
||||
from the file. ADR 0011
|
||||
from the file. [ADR 0011](../02-DECISIONS/0011-managed-files-are-generated-never-edited.md)
|
||||
|
||||
---
|
||||
|
||||
@@ -122,7 +121,7 @@ for them. Do not merge them into one module: they are delivered to different nod
|
||||
that must be assigned where half of it is unwanted is not a boundary either.
|
||||
|
||||
Coherence is a context. Delivery is a module. Relationships are edges, not folders.
|
||||
ADR 0009
|
||||
[ADR 0009](../02-DECISIONS/0009-modules-and-the-graph.md)
|
||||
|
||||
### Contexts integrate through the record, never through a shared schema
|
||||
|
||||
@@ -173,30 +172,6 @@ quietly stop being true, and nobody will learn that from a document.
|
||||
Not test-driven development as a blanket rule — a test written first against undiscovered
|
||||
behaviour asserts a guess. The obligation is that every decision has a defender.
|
||||
|
||||
### Tests run at the lowest level that proves them, within a time budget
|
||||
|
||||
Three levels, by how much real system a test uses (ADR 0391):
|
||||
|
||||
| Level | Uses | Budget | Runs |
|
||||
|---|---|---|---|
|
||||
| **Unit** | one piece of logic alone: no database, network, bus, running service or other process | well under a second per test | locally, the tests of what changed, while working; and in the forge's checks |
|
||||
| **Integration** | the real thing beside it: a database, the bus, a catalogue tree | the repository's whole suite in seconds, never over sixty | only in the forge's checks, once per push |
|
||||
| **End to end** | the whole mesh, or a lab mesh, from the outside | — | only when the operator asks |
|
||||
|
||||
- **Test each behaviour at the lowest level that can prove it.** A defect found higher up gets a test at the
|
||||
lowest level that reproduces it, before its fix.
|
||||
- **A test that needs longer than a second, or a database, the network or a service, is not a unit test.**
|
||||
- **A suite over sixty seconds is a defect with its own issue**: it is made faster or split, not accepted.
|
||||
- **Never run a merge check, a throwaway database or an integration suite locally.** The forge's checks do,
|
||||
on the builder.
|
||||
- **A report names its tests and how long they took**, so a test over its budget is seen when it is added.
|
||||
|
||||
### Tests within a time budget — how it was earned
|
||||
|
||||
On 2026-10-16 a feature's fixing agents ran the catalogue's whole merge check (about ten minutes), a
|
||||
throwaway database and a locally built gate during local development. Each round cost tens of minutes and
|
||||
caught neither real failure; the forge's checks found both.
|
||||
|
||||
### A report is read from the system, never from what asked for it
|
||||
|
||||
The same rule as the two above, pointed at reporting rather than at verification. **Anything
|
||||
@@ -223,8 +198,8 @@ declaration was never wrong. Nothing ever asked the system.
|
||||
### Search the record before forming a hypothesis
|
||||
|
||||
The first action on any error message, failing service or unexpected behaviour is to search the
|
||||
record for the literal error text (`records.records_search`) — before a hypothesis, not after one fails.
|
||||
The search matches the words as written, so search with the error's own words.
|
||||
operational memory for the literal error text — before a hypothesis, not after one fails. The
|
||||
knowledge base is indexed on symptoms.
|
||||
|
||||
This fires hardest on *familiar* ground, where a confident trail feels like progress. Two
|
||||
entries were each rediscovered from scratch over several hours in a single session because the
|
||||
@@ -239,7 +214,7 @@ This document is governed. It does not change by commit message or unilateral de
|
||||
1. **Propose** — a change stating what rule is changing, why the current wording is
|
||||
inadequate, and what reviewed it.
|
||||
2. **Review** — sign-off by reviewers who are not the proposer.
|
||||
3. **Record** — the change is a decision and gets a decision record in the knowledge base (playbook [02](process/02-graduation.md)), because a
|
||||
3. **Record** — the change is a decision and gets a record in [`02-DECISIONS/`](../02-DECISIONS/), because a
|
||||
rule the mesh enforces is architecturally significant.
|
||||
4. **Sync** — playbook [`process/05-constitution-sync.md`](process/05-constitution-sync.md)
|
||||
publishes the derived page. An unsynced rule is a rule the mesh does not enforce, whatever
|
||||
@@ -251,7 +226,7 @@ No drive-by edits. Every change traces to a recorded decision.
|
||||
a design meeting with at least two node operators — which has never been met and cannot be, as
|
||||
there is one operator. A rule that cannot be satisfied is not a high standard; it is a rule
|
||||
everything silently violates. Recorded here as resolved in favour of what is achievable, and
|
||||
what has in fact been practised (ADR 0022).
|
||||
what has in fact been practised ([ADR 0022](../02-DECISIONS/0022-the-constitution-absorbs-what-is-enforced.md)).
|
||||
|
||||
---
|
||||
|
||||
@@ -268,7 +243,7 @@ process. Absence of an override means these rules apply unmodified.
|
||||
## 8. Code quality
|
||||
|
||||
*Absorbed 2026-08-26 from the enforced page, which carried these rules while this document did
|
||||
not — ADR 0022.*
|
||||
not — [ADR 0022](../02-DECISIONS/0022-the-constitution-absorbs-what-is-enforced.md).*
|
||||
|
||||
**These rules are recorded because they are enforced, not because this repository earned them.**
|
||||
Every other rule here states the incident or measurement behind it. These state nothing,
|
||||
@@ -297,7 +272,7 @@ Data access, business logic and the interface layer are separate.
|
||||
|
||||
*Scope: the mesh's services and surfaces. Tier 0 is a statically linked binary that must depend
|
||||
on nothing installed first, and is written in Go —
|
||||
ADR 0005.*
|
||||
[ADR 0005](../02-DECISIONS/0005-the-node-host.md).*
|
||||
|
||||
- TypeScript throughout; no new untyped JavaScript.
|
||||
- Strict, with no implicit `any` and no unchecked index access.
|
||||
|
||||
+1
-1
@@ -11,7 +11,7 @@ updated: 2026-08-22
|
||||
|
||||
An agent states an intent — in words, from wherever they already are — and the mesh
|
||||
carries it out. It takes the request in, works out what it means, does the work across
|
||||
whichever nodes it needs, and returns a result. No terminal to open, no runbook to follow,
|
||||
whichever nodes it needs, and returns a result. No console to open, no runbook to follow,
|
||||
no remembering which node holds which thing.
|
||||
|
||||
Not automation, which does what it was told to do in advance. Self-control: the mesh
|
||||
|
||||
@@ -3,8 +3,8 @@
|
||||
How work moves through HQ, and who may do what. In industry terms this is **spec-driven
|
||||
development, with provenance**: the decision is the why, the design doc is the spec, `code:`
|
||||
names the implementation, and the lab beds are the conformance tests — and unlike the common
|
||||
form, the chain itself is checked (ADR 0080,
|
||||
ADR 0081). Every other document in this folder is a
|
||||
form, the chain itself is checked ([ADR 0080](../../02-DECISIONS/0080-the-development-cycle-is-checked.md),
|
||||
[0081](../../02-DECISIONS/0081-a-decision-nothing-cites-is-not-yet-in-the-chain.md)). Every other document in this folder is a
|
||||
playbook: trigger, who runs it, steps, outputs. Engineers and agents follow the same
|
||||
playbooks; agents must not act outside them.
|
||||
|
||||
@@ -19,13 +19,11 @@ playbooks; agents must not act outside them.
|
||||
## The knowledge flow
|
||||
|
||||
```
|
||||
idea ──► research ──► decision ──► 03-DESIGN/01-to-be ──► built (code repo)
|
||||
│ │ │
|
||||
│ │ └─► 03-DESIGN/00-as-is once shipped
|
||||
idea ──► 01-RESEARCH ──► decision (02-DECISIONS/) ──► 03-DESIGN/01-to-be ──► built (code repo)
|
||||
│ │ │
|
||||
│ │ └─► 03-DESIGN/00-as-is once shipped
|
||||
│ └────► abandoned (recorded, kept)
|
||||
└─(small/obvious, still a decision record)──────────► 03-DESIGN directly
|
||||
|
||||
research and decisions: in the knowledge base (knowledge-base.reserve, propose, ask, get, holds)
|
||||
└─(small/obvious, still recorded in 02-DECISIONS)──────────► 03-DESIGN directly
|
||||
|
||||
symptom ──► 04-ISSUES ──► diagnosis ──► code-repo fix and/or design amendment
|
||||
|
||||
@@ -39,7 +37,7 @@ how-we-build.md ──► constitution sync ──► knowledge base ──► i
|
||||
| Layer | What it is | Changes when |
|
||||
|---|---|---|
|
||||
| `00-as-is/` | The mesh that exists today. Describes shipped behaviour, including behaviour nobody would choose again. | Something ships, or an as-is claim is found to be wrong. |
|
||||
| `01-to-be/` | The mesh being built toward. Every statement traceable to a decision record in the knowledge base. | A decision is taken or amended. |
|
||||
| `01-to-be/` | The mesh being built toward. Every statement traceable to a record in `02-DECISIONS/`. | A decision is taken or amended. |
|
||||
|
||||
A to-be document that ships does not move. Its as-is counterpart is written or updated, the
|
||||
to-be document's frontmatter goes to `implemented`, and both stand — one describing what runs,
|
||||
@@ -50,39 +48,30 @@ the expensive half.
|
||||
|
||||
| # | Playbook | Trigger |
|
||||
|---|---|---|
|
||||
| [01](01-research.md) | Research | An idea from the operator, or one worth investigating before committing to design: setup questions, findings, a direction the operator confirms |
|
||||
| [02](02-graduation.md) | Graduation & design change | Research concludes (its direction confirmed), or a design must change: decision records, the to-be design |
|
||||
| [01](01-research.md) | Research | An idea worth investigating before committing to design |
|
||||
| [02](02-graduation.md) | Graduation & design change | Research concludes, or a design must change |
|
||||
| [03](03-issues.md) | Issues | Something is wrong — often with the owner unknown |
|
||||
| [04](04-build-handoff.md) | Work breakdown and handoff | A design is ready to be built: milestones usable on their own, issues per repository, each milestone one feature step |
|
||||
| [04](04-build-handoff.md) | Build handoff | A design is ready to be built |
|
||||
| [05](05-constitution-sync.md) | Constitution sync | `how-we-build.md` changed a rule the mesh enforces |
|
||||
| [06](06-writing-a-module.md) | Writing a module | Something that runs today must run on the mesh |
|
||||
| [07](07-feature-branches.md) | Features | Work that changes code across issues or repositories: prepared context, local development, one review, one check, one delivery |
|
||||
| [08](08-small-fixes.md) | Small fixes | One issue in one repository, about an hour of work: one review, one push, one delivery |
|
||||
| [09](09-cleanup.md) | Cleanup | Merged branches, stale pull requests, finished worktrees and folders: read first, delete only what the operator approves, report the rest |
|
||||
| [07](07-feature-branches.md) | Feature branches across repos | Work that changes code, in one repo or several at once |
|
||||
|
||||
## The cycle is checked
|
||||
|
||||
The flow above is a rule, and a rule states how it is checked:
|
||||
[`00-META/checks/cycle.py`](../checks/cycle.py) refuses a to-be design that names no
|
||||
decision, an `in-progress`/`implemented` design that names no owning code, two issue folders
|
||||
sharing a number or an issue report carrying a status key; a `graduated` research effort that
|
||||
does not say what it became is refused by the knowledge base's `propose` (ADR 0290). Run it with `records.py`, `index.py` and
|
||||
`words.py` before any HQ merge (`sh merge-check.sh` runs all four). The issue rules the check
|
||||
once held — `located` with an owner, `resolved` with a fix, a core issue with its replay — are
|
||||
refused at the write by the issue tracker's `move`
|
||||
(ADR 0275). What the checks cannot see — that code work actually started from a
|
||||
decision, an `in-progress`/`implemented` design that names no owning code, an issue marked
|
||||
`located`/`fixed` with no owner or `fixed`/`resolved` with no fix, and a `graduated`
|
||||
research overview that does not say what it became. Run it with `records.py` and `index.py`
|
||||
before any HQ merge. What the checks cannot see — that code work actually started from a
|
||||
handoff — is held by playbooks [04](04-build-handoff.md) and [07](07-feature-branches.md):
|
||||
a feature branch exists because a design or an issue sent it.
|
||||
|
||||
## Status lives in frontmatter, an issue's in the tracker
|
||||
## Status lives in frontmatter
|
||||
|
||||
Design docs carry their status as YAML frontmatter (schema in playbook [02](02-graduation.md)).
|
||||
Research efforts and decision records carry theirs in the knowledge base, which refuses a wrong one at
|
||||
`propose` (ADR 0290). An issue's status lives in the
|
||||
issue tracker, which refuses a wrong status at the write, and its report here carries only
|
||||
the day it was opened (playbook [03](03-issues.md)). There are **no central status
|
||||
Research overviews, design docs, issue reports and decision records each carry their status as
|
||||
YAML frontmatter (schemas in the section READMEs and playbooks). There are **no central status
|
||||
files** and no decision ledger. **Every decision is a record in
|
||||
the knowledge base** — if it is worth recording it is worth a record, and if
|
||||
it is not worth a record it is not recorded. Cross-cutting views are generated on demand by the
|
||||
`hq-status` skill and never written to disk (ADR 0248); the decisions of a topic are
|
||||
`knowledge-base.holds` with that topic.
|
||||
[`02-DECISIONS`](../../02-DECISIONS/)** — if it is worth recording it is worth a record, and if
|
||||
it is not worth a record it is not recorded. Cross-cutting views, the decision index included,
|
||||
are generated on demand by the `hq-status` skill and never written to disk.
|
||||
|
||||
+29
-121
@@ -1,140 +1,48 @@
|
||||
# Playbook 01 — Research
|
||||
|
||||
**Trigger.** An idea the operator expresses, or an idea, technology or approach worth investigating before
|
||||
it is committed to design. Also: an as-is document that raises a question nobody can answer. A symptom (a
|
||||
bug, something broken) is not research: it follows playbook [03](03-issues.md).
|
||||
**Trigger.** An idea, technology or approach worth investigating before it is committed to
|
||||
design. Also: an as-is document that raises a question nobody can answer.
|
||||
|
||||
**Who runs it.** Anyone; in an agent session, the session that heard the idea leads it.
|
||||
|
||||
**Where it lives.** A research effort lives whole in the knowledge base, the seat `knowledge-base`, reached
|
||||
through the mesh MCP server from any node and any folder (ADR 0290). This repository holds none.
|
||||
|
||||
## The cycle it starts
|
||||
|
||||
```
|
||||
idea ─► research (01) ─► decisions and design (02) ─► work breakdown in milestones (04)
|
||||
─► each milestone built and delivered as one feature step (07) ─► what is reported afterwards (03, 08)
|
||||
```
|
||||
|
||||
An idea arrives in one of two forms, and the first step differs:
|
||||
|
||||
- **A new idea**, where only the mesh's own context matters: read what the knowledge base holds on it.
|
||||
- **An idea for an existing application**: read that application's to-be and as-is designs, its decision
|
||||
records and its open issues first; the effort `touches` them.
|
||||
**Who runs it.** Anyone.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Read what is known.** `knowledge-base.ask` with the idea in plain words returns the decision records and
|
||||
research efforts that bear on it; `knowledge-base.get` reads one (`reasoning: true`, or `section`, for its
|
||||
documents). Read the issue tracker for open issues in the same area (`issue-tracker.list`).
|
||||
2. **Ask the operator to set it up.** A few questions, one at a time, each answerable with a choice and
|
||||
carrying a recommendation: what comes first, what is out of scope, which options must be tried, what
|
||||
done looks like. Three to five questions is usual; the answers are the effort's first findings.
|
||||
- **Check every option against the mesh's own rules before offering it.** An option that breaks one
|
||||
(the core values in [`mission.md`](../mission.md), among them *the mesh owns the truth* and
|
||||
*sovereignty: external dependencies are deliberate and few*; the rules in
|
||||
[`how-we-build.md`](../how-we-build.md); and everything between modules over the bus, research 036) is
|
||||
dropped, or offered only with the rule it breaks named. A recommendation the operator has to argue back
|
||||
costs a version.
|
||||
- **Explain each question with a concrete picture** in the operator's words: what they would see, do or
|
||||
lose under each option. No internal names (a record number, a verb, a word the glossary lacks) without
|
||||
what it is beside it.
|
||||
3. **Take a number.** `knowledge-base.reserve` with `kind: research` and `for`: the effort's title. The number
|
||||
is spent whether or not the effort is ever proposed; one never proposed is withdrawn after a week. Never
|
||||
pick one by hand.
|
||||
4. **Propose it.** `knowledge-base.propose` with `kind: research`, the `number`, a one-line `title`,
|
||||
`status: active`, `date` (the day it began), and `sections`, each a `name` and its markdown `text`:
|
||||
- `00-overview.md`: the idea in the operator's words, what exists, the operator's answers (dated), and the
|
||||
words the effort uses that the glossary lacks, each defined in one line;
|
||||
- `01-questions.md`: the questions to answer, the first ones first;
|
||||
- `02-method.md`: what will be tried and measured, and on what.
|
||||
1. Take the next free number: highest `01-RESEARCH/NNN-*` plus one, zero-padded to three
|
||||
digits. Never skip or reuse a number.
|
||||
2. Create `01-RESEARCH/NNN-descriptive-name/` and a `00-overview.md` in it carrying:
|
||||
|
||||
Name what it touches as `links` of kind `touches` (`"0276"`, `"research 045"`, `"issue 365"`, or a
|
||||
design's path), and what it names for another reason as `cites`.
|
||||
5. **Do the research.** Try, measure and read, smallest question first. Each finding goes into
|
||||
`04-findings.md`: what was tried, what it showed (with numbers), what it means. Each change of the effort
|
||||
is `knowledge-base.propose` again, whole, with `why`; every version is kept, and `knowledge-base.history`
|
||||
lists them and gives the difference between two. Until issue 844 is delivered, the knowledge base
|
||||
takes a new version of an accepted, active effort only as a progressive insight: give `insight`, saying
|
||||
what the effort held before the new finding, and mark the finding in the text with its date.
|
||||
- **Propose a version per settled set, not per answer.** While the operator answers a run of questions,
|
||||
or while experiments run, keep the answers and findings in the session's save point and propose once
|
||||
when the set is settled. Each version is reviewed and resent whole, so a version per answer multiplies
|
||||
the reviews.
|
||||
- **Pre-check before proposing**, against what the review sends back most: every record, issue or design
|
||||
the text names is in `links`; work put off names the issue that tracks it (an acceptance test is not
|
||||
put-off work: it is what the build proves, step 7); each figure is given as measured (a range, not a
|
||||
rounded claim); no word on a glossary *Not:* line.
|
||||
- **Try at the cheapest level that answers the question.** A program's own behaviour (a library, a
|
||||
server, a protocol) is tried in process, on the machine's loopback, in minutes. Lab runs (about 12 to
|
||||
16 minutes fresh) are for the mesh's own code; before that code exists, a lab run tests nothing an
|
||||
in-process test does not, and its lab scenario is written as the build's acceptance test instead.
|
||||
6. **Place it in the domains.** Before the direction, write `03-domains.md`: the domain or domains the idea
|
||||
belongs to (the glossary's sections, ADR 0244); the modules and seats it lands in, each new, changed or
|
||||
retired; what it does to each domain's words, each word added (with its domain and a one-line meaning),
|
||||
changed or retired, and none on a *Not:* line or already meaning something else (the glossary's homonyms);
|
||||
and whether a domain grows or shrinks, a concept moving between domains included. The direction names its
|
||||
decisions in those words. Asked when the idea is set up, too: an option that lands a concept in the wrong
|
||||
domain is offered with that cost named, like one that breaks a rule.
|
||||
7. **Settle the direction with the operator.** When the questions are answered, write `05-direction.md`:
|
||||
the proposed decisions in a few lines each, the options rejected and why, and the build's acceptance
|
||||
tests: what the build must prove before it is done, carried by its milestones (playbook
|
||||
[04](04-build-handoff.md)). Nothing else is left open: each question closes by a test, by a decision the
|
||||
effort records with its reason (a small design limit is the lead's), or by the operator. Ask the
|
||||
operator to confirm or change the direction, one point at a time, then propose it once. Then the effort
|
||||
graduates through playbook [02](02-graduation.md).
|
||||
```yaml
|
||||
---
|
||||
status: active
|
||||
initiated: YYYY-MM-DD
|
||||
touches: [] # areas, subsystems or as-is design docs the effort bears on
|
||||
---
|
||||
```
|
||||
|
||||
## Bounds and models
|
||||
|
||||
| Step | Bound | Model |
|
||||
|---|---|---|
|
||||
| reading and the setup questions (1, 2) | 30 minutes | the session's own, larger |
|
||||
| proposing (3, 4) | 15 minutes | larger |
|
||||
| each experiment or reading task (5) | 30 minutes | smaller, one agent at a time unless the operator allows more |
|
||||
| placing it in the domains (6) | 15 minutes | larger |
|
||||
| the direction (7) | 30 minutes | larger |
|
||||
|
||||
A step past its bound is said to the operator at once, with what is being done about it. The lead reports
|
||||
as playbook 07 does: every ten minutes and at once when something is found, stops or needs the operator,
|
||||
with the allowance used.
|
||||
Then a short prose summary: **what** is being investigated, **why**, and **what it
|
||||
touches**.
|
||||
3. Do the research in further documents in the same folder — notes, option analyses, evidence,
|
||||
draft designs. Anything goes. Keep the summary in `00-overview.md` current as the effort changes
|
||||
shape.
|
||||
|
||||
## What makes research worth reading
|
||||
|
||||
State evidence, not assertion. *"Zero of 124 modules declare `brain` as a dependency"* outranks *"the
|
||||
dependency rule is not followed"*. An effort that measured nothing has not finished.
|
||||
State evidence, not assertion. *"Zero of 124 modules declare `brain` as a dependency"*
|
||||
outranks *"the dependency rule is not followed"*. An effort that measured nothing has not
|
||||
finished.
|
||||
|
||||
Research describes real observations but never identifies the mesh it observed. The shape of a finding
|
||||
survives anonymisation intact — *a node publicly named but behind a household NAT* carries the whole
|
||||
lesson without naming anything.
|
||||
|
||||
`propose` refuses an effort that breaks a rule that judges one record (an unknown status, a link to
|
||||
nothing, a retired word, a daemon's line), naming every reason at once and writing nothing. The knowledge
|
||||
base then reviews the proposal itself; on `changes` the effort stays proposed with the findings and its
|
||||
author proposes it again, at most three times before the findings go to the operator with a
|
||||
recommendation. An effort that needs the operator's word is proposed with `needs-operator: true` and waits
|
||||
for the operator's Approve on a channel that proves who answered.
|
||||
Research describes real observations but never identifies the mesh it observed. The shape of a
|
||||
finding survives anonymisation intact — *a node publicly named but behind a household NAT*
|
||||
carries the whole lesson without naming anything.
|
||||
|
||||
## Do not
|
||||
|
||||
- Do not put status in prose. It is the effort's `status`, and the prose must not restate it.
|
||||
- Do not give a `became` link while the effort is open — playbook 02 gives it at closure.
|
||||
- Do not put status in prose. It lives in `00-overview.md`'s frontmatter, and the prose must not restate it.
|
||||
- Do not set `became:` while the effort is open — playbook 02 sets it at closure.
|
||||
- Do not write into `03-DESIGN` from an open effort.
|
||||
- Do not write an effort, or a copy of one, into this repository: `records.py` refuses a new path under
|
||||
`01-RESEARCH/`.
|
||||
|
||||
## Closing
|
||||
|
||||
An effort never just stops. It closes through playbook [02](02-graduation.md) as `graduated` or
|
||||
`abandoned`, always with a `became` link to what it turned into. Nothing is deleted — what was rejected,
|
||||
and why, is the more expensive half to rediscover.
|
||||
|
||||
## How it is checked
|
||||
|
||||
The rules on an effort are the knowledge base's: `propose` refuses one that breaks a rule that judges one
|
||||
record (the module's table test holds one case per rule), and the rules that judge the whole record set run
|
||||
after every acceptance and raise a condition when one fails. The habits of steps 2, 5, 6 and 7 (options checked
|
||||
against the rules, one version per settled set, the cheapest level, nothing left open) are the lead's, read by
|
||||
the reviewer; of the pre-check, `propose` refuses a link to nothing and a retired word, and the review returns
|
||||
a record named but not linked, an untracked deferral and a figure not as measured. That this repository holds
|
||||
no effort is `records.py`'s `no-records` check, on every pull request. An active effort with no new version
|
||||
for seven days is the smell of research left behind: its lead closes or continues it.
|
||||
An effort never just stops. It closes through playbook [02](02-graduation.md) as `graduated`
|
||||
or `abandoned`, always with `became:` pointing at what it turned into. Nothing is deleted —
|
||||
what was rejected, and why, is the more expensive half to rediscover.
|
||||
|
||||
@@ -4,39 +4,16 @@
|
||||
|
||||
**Who runs it.** Anyone, with the decision recorded before the design moves.
|
||||
|
||||
**Where a decision lives.** A decision record lives whole in the knowledge base, the seat `knowledge-base`,
|
||||
reached through the mesh MCP server from any node and any folder (ADR 0290): its number, title, topic,
|
||||
status, date, deciders, its clauses, its typed links, its reasoning and every version. This repository holds
|
||||
none, and keeps no copy. What was decided is read with `knowledge-base.ask` (a question in plain words),
|
||||
`knowledge-base.get` (one record by number; `reasoning: true` for its why) and `knowledge-base.holds` (the
|
||||
clauses in force for a topic or a record).
|
||||
|
||||
## Graduating research
|
||||
|
||||
The effort's `05-direction.md`, confirmed by the operator (playbook [01](01-research.md), step 7), is what
|
||||
graduates. The lead writes the records and the design on the larger model; it is one sitting, bounded at
|
||||
about an hour, and said to the operator if it runs past.
|
||||
|
||||
1. **Check it against GENESIS.** An effort graduates only if its conclusion is traceable to
|
||||
[`mission.md`](../mission.md), [`context.md`](../context.md) and
|
||||
[`effect.md`](../effect.md). If it conflicts, either the effort is wrong or GENESIS is —
|
||||
say which, in writing, before proceeding.
|
||||
2. **Record the decisions** (below, *Writing a decision record*): one record per choice that could be
|
||||
reversed on its own, not one record for the whole effort.
|
||||
- **Write them together, not one after another.** Reserve every number first, then propose each record
|
||||
as soon as it is written, without waiting for the review of the one before; the reviews run side by
|
||||
side, and the operator's asks for the records that need them arrive together. A record that
|
||||
`rests-on` or `extends` a sibling is proposed once that sibling is accepted.
|
||||
- **Take the clauses from the confirmed direction** and cite the effort (`cites`, `"research NNN"`) for
|
||||
the evidence instead of retyping its findings: a record's Context holds the few facts its decision
|
||||
rests on, with the effort named for the rest.
|
||||
- **Pre-check each record before proposing it**, as for research (playbook [01](01-research.md), step
|
||||
5): every record, issue or design it names is in `links`; each deferral names its issue; each figure
|
||||
is given as measured; no word on a glossary *Not:* line.
|
||||
3. **Write the design.** **One design per thing** (the operator, 2026-10-16): when a to-be design already
|
||||
covers what the decisions change, they rewrite that design, as *Amending an existing design* below says,
|
||||
and no second design is written beside it that refers to it. Only for something no design covers, create
|
||||
the document under `03-DESIGN/01-to-be/` with frontmatter:
|
||||
2. **Record the decision.** Write a record in [`02-DECISIONS/`](../../02-DECISIONS/) taking the next free
|
||||
number. Format and rules are in [`02-DECISIONS/README.md`](../../02-DECISIONS/README.md). State evidence,
|
||||
not assertion, and record the options that were rejected — that is the half worth keeping.
|
||||
3. **Write the design.** Create the document under `03-DESIGN/01-to-be/` with frontmatter:
|
||||
|
||||
```yaml
|
||||
---
|
||||
@@ -44,93 +21,25 @@ about an hour, and said to the operator if it runs past.
|
||||
status: designed
|
||||
code: [] # owning code repo(s); set at build handoff, empty before
|
||||
updated: YYYY-MM-DD
|
||||
decisions:
|
||||
- "ADR NNNN"
|
||||
decisions: [02-DECISIONS/NNNN-....md]
|
||||
---
|
||||
```
|
||||
|
||||
The design lists the direction's acceptance tests: what the build proves before it is done.
|
||||
|
||||
The words of the effort's `03-domains.md` go into the glossary in the same change, each in its domain: a
|
||||
word added gets its entry, a word whose meaning changed gets its entry rewritten, and a retired word moves to
|
||||
its replacement's *Not:* line. A word that already means something else in another domain is named as a
|
||||
homonym with the qualified form each domain uses.
|
||||
|
||||
4. **Hand it to the work breakdown.** The design goes on through playbook [04](04-build-handoff.md):
|
||||
milestones, issues, and each milestone built as one feature step (playbook [07](07-feature-branches.md)).
|
||||
5. **Close the effort.** `knowledge-base.propose` the effort again, whole, with `status: graduated` and a
|
||||
`became` link to the decision record (`"NNNN"`) and to the design document (its path). An effort that
|
||||
stops without a decision is proposed with `status: abandoned` and a `became` link to what replaced it.
|
||||
|
||||
## Writing a decision record
|
||||
|
||||
1. **Take a number.** `knowledge-base.reserve` with `kind: decision` and `for`: the record's title. The
|
||||
number comes from the issue tracker's counter and is spent whether or not the record is accepted; one
|
||||
never proposed is withdrawn after a week. Never pick a number by hand.
|
||||
2. **Propose it.** `knowledge-base.propose` with the `number`, a one-line `title` in plain language, its
|
||||
`topic` (one of the six of the reading order, [`02-DECISIONS/README.md`](../../02-DECISIONS/README.md)),
|
||||
`date` (when the decision was taken, not when it was written down), `deciders`, and:
|
||||
- `clauses`: what was decided, in order, each one statement opening with a bold lead
|
||||
(`"**What is decided.** …"`);
|
||||
- `sections`: the reasoning, in order: `Context` (what was true, with evidence), `Considered Options`
|
||||
(numbered, each with why it was rejected), `Consequences` (what follows, including what got harder),
|
||||
`How it is checked` (a rule states how it is checked), `References`;
|
||||
- `links`: `extends` for a record it builds on, `rests-on` for a decision it stands on, `supersedes` for one it replaces whole, `changes` with
|
||||
the clause it replaces part of (`"0276 §4"`, and `from` for the clause of this record that does it),
|
||||
`cites` for any record, effort, issue or design it names;
|
||||
- `homes`: the documents of this repository that cite it, such as the design's path;
|
||||
- `needs-operator: true` when it holds only once the operator approves it.
|
||||
|
||||
State evidence, not assertion. *"Zero of 124 modules declare `brain` as a dependency"* outranks *"the
|
||||
dependency rule is not followed"*. Record the options that were rejected — that is the half worth keeping.
|
||||
3. **A record that puts work off** — a rename, a check, a case it does not handle — names the issue that
|
||||
tracks it, in the sentence that defers it ("tracked by issue N"), or the deferral is not accepted: a
|
||||
deferral nothing tracks reads as done
|
||||
([issue 354](../../04-ISSUES/354-a-decided-rename-was-never-scheduled-and-the-operator-believed-it-live/00-report.md),
|
||||
ADR 0244's code renames). `propose` refuses it (rule `deferral`).
|
||||
4. **Review and acceptance.** `propose` refuses, naming every reason at once and writing nothing, a record
|
||||
that breaks one of the rules that judge one record: a link to nothing, a clause without a bold lead, a
|
||||
superseded record cited alone, an unknown topic, a retired word of the glossary, a daemon's line, a
|
||||
deferral no issue tracks. The knowledge base then reviews the proposal itself, through a judgement that
|
||||
sees only the record, the records it links and the rules, and answers `approve` or `changes`. On
|
||||
`changes` the record stays proposed with the findings: revise it by proposing it again, whole, with `why`;
|
||||
at most three rounds, then the findings go to the operator with a recommendation.
|
||||
On `approve` it is accepted, or, when it says `needs-operator`, it waits for the operator's Approve on a
|
||||
channel that proves who answered. Nothing proposed is in force. `knowledge-base.get` shows a record's
|
||||
reviews and the operator's ask when it waits for one.
|
||||
|
||||
## Changing a decision
|
||||
|
||||
**A record's meaning is never rewritten.** A decision is changed by a new record, and a correction of fact
|
||||
by a progressive insight.
|
||||
|
||||
- **Reversing a decision, changing its scope, rejecting an option it accepted, or making a consequence false
|
||||
that a later record cites** is a new record that `supersedes` the old one whole, or `changes` the clauses
|
||||
it replaces. On its acceptance the old record or clauses become `superseded` or `changed`, and every
|
||||
answer says by what. The test is not how large the edit looks; it is whether a reader who acted on the
|
||||
old text would now be wrong about *what was decided* rather than about *a detail the decision did not
|
||||
rest on*.
|
||||
- **A progressive insight** corrects a fact a record asserted that turned out wrong while the decision it
|
||||
supports stays right: a count taken before anyone measured, a file named that does not exist. It is
|
||||
`knowledge-base.propose` of the accepted record with `insight`: what was found, what the record said
|
||||
before, and what it says instead. It is a new version, marked as one, dated, and reviewed like a
|
||||
proposal; it adds and never quietly replaces; the decision, the options weighed and the consequences
|
||||
stand untouched. If the correction changes what was decided, it is not an insight — write the new record.
|
||||
- **Where a later record changes the mechanism an older one states** without reversing it, the later
|
||||
record `changes` the older one's clause, so a reader who arrives at the older record by a citation sees
|
||||
what changed it since.
|
||||
4. **Close the effort.** Set the effort's `00-overview.md` frontmatter to `status: graduated` and
|
||||
`became:` pointing at the design document and the decision record.
|
||||
|
||||
## Amending an existing design
|
||||
|
||||
A design changes only through a decision.
|
||||
|
||||
1. Write the decision record (above). If it reverses an earlier one, it `supersedes` it.
|
||||
2. Edit the to-be design document, add the record to its `decisions:` as `"ADR NNNN"`, and set `updated:`
|
||||
to today.
|
||||
3. If the amendment came from an issue, give the issue the document path in the issue tracker:
|
||||
`amended-design` on its `issue-tracker.move` to `resolved`, or `issue-tracker.link` with kind
|
||||
`design` before then — never in the report's frontmatter
|
||||
(ADR 0275).
|
||||
1. Write the decision record. If it reverses an earlier one, the earlier record's `status:`
|
||||
becomes `superseded-by: 02-DECISIONS/NNNN-....md` — **its reasoning is never rewritten**. If the
|
||||
earlier record is sound and only a *fact* in it went stale, that is a **progressive insight**,
|
||||
corrected in place and marked in the record rather than superseded
|
||||
([`02-DECISIONS/README.md`](../../02-DECISIONS/README.md)).
|
||||
2. Edit the to-be design document and set `updated:` to today.
|
||||
3. If the amendment came from an issue, set that issue's `amended-design:` to the document
|
||||
path.
|
||||
|
||||
## When something ships
|
||||
|
||||
@@ -150,17 +59,3 @@ Implementation state is a third axis, independent of both design and decision.
|
||||
- Do not edit an as-is document to describe an intention. That is what the to-be layer is for.
|
||||
- Do not change a decision record's meaning. Supersede it. Correcting a fact it got wrong, while
|
||||
the decision stands, is a progressive insight — marked and dated in the record, never silent.
|
||||
- Do not write a decision record, or a copy of one, into this repository: `records.py` refuses a new path
|
||||
under `02-DECISIONS/` other than its README.
|
||||
|
||||
## How it is checked
|
||||
|
||||
The rules on a record are the knowledge base's: `propose` refuses one that breaks a rule that judges one
|
||||
record (its module's table test holds one case per rule), and the four rules that judge the whole record set
|
||||
(an accepted decision cited by nothing, a supersession named by one side only, a changed clause nothing
|
||||
changes, an accepted record extending a superseded one) run after every acceptance and raise a condition
|
||||
when one fails. In this repository, `records.py` checks that a design's `decisions:` entries and every
|
||||
"ADR NNNN" in the text are well formed, and that no record is written here; `index.py` checks the reading
|
||||
order's six topics, which the knowledge base reads. The three habits of graduating step 2 (side by side, clauses
|
||||
from the direction, the pre-check) are the lead's, read by the reviewer; `propose` refuses a link to nothing,
|
||||
a deferral no issue tracks and a retired word, and the review returns a record named but not linked.
|
||||
|
||||
@@ -8,115 +8,53 @@ design permits.
|
||||
|
||||
## What belongs here, and what does not
|
||||
|
||||
| Belongs in `04-ISSUES` | Belongs with the fix's own lesson, not here |
|
||||
| Belongs in `04-ISSUES` | Belongs in the knowledge base |
|
||||
|---|---|
|
||||
| The design permits a failure to be silent | How to fix one occurrence of it |
|
||||
| A documented rule is enforced by nothing | A command that works around it |
|
||||
| A stated invariant is false in practice | A node-specific quirk |
|
||||
| The owning component is unknown and finding it needs the whole mesh in view | Symptom → fix, once the answer is known |
|
||||
|
||||
A fix's lesson (symptom → fix) is written where a search by the symptom's literal words finds it
|
||||
(`records.records_search`). This folder is not a second copy of it. An issue here is a question HQ must **answer**, not an
|
||||
The knowledge base already holds the operational record and is indexed on symptoms. This
|
||||
folder is not a second copy of it. An issue here is a question HQ must **answer**, not an
|
||||
incident someone must **clear**.
|
||||
|
||||
## The issue tracker holds the number and the status
|
||||
|
||||
Every issue is opened in the **issue tracker**, the module `mesh-issues` holding the seat `issue-tracker` on the
|
||||
control-node (ADR 0275,
|
||||
[to-be 53](../../03-DESIGN/01-to-be/53-the-issue-tracker.md)). Its number comes from one counter advanced by
|
||||
compare-and-set, so two authors opening at the same moment never get the same one. **A number is never taken by
|
||||
reading `main` or the open pull requests** — that is how numbers collided (issue 155: two records sharing a number
|
||||
reached `main` with every check passing). A session that cannot reach the tracker takes no number and says so.
|
||||
|
||||
**The tracker is the truth for every issue's status, owner, fix and links**, whether or not the issue has a folder
|
||||
here. `issue-tracker.get NNN` says where an issue stands; `issue-tracker.list` says what is open. Nothing here
|
||||
says it: a report's frontmatter carries no status.
|
||||
|
||||
An issue takes one of two shapes:
|
||||
|
||||
- **With a folder here** — a design or governance issue, whose narrative belongs in this public record and is
|
||||
reviewed in a pull request. The folder holds the report and the diagnosis; the tracker holds everything else.
|
||||
- **Wholly in the tracker** — every other issue. Nothing is written here.
|
||||
|
||||
**`04-ISSUES/` stays, for the narrative only.** It keeps the public narrative of a design or governance issue: the
|
||||
symptom as observed, the diagnosis, what was ruled out, and why. A plain bug — a defect whose fix needs no design
|
||||
question answered — needs no folder; it is whole in the tracker.
|
||||
|
||||
**The reports written before the tracker** (the folders numbered 001 to 364) carried `status:`, `located-in:`,
|
||||
`fixed-by:`, `amended-design:`, `replay:` and `replay-none:` in their frontmatter until the import of slice 3 had
|
||||
taken them into the tracker and been checked against it, field by field; then one pull request removed them, so
|
||||
that no status is held in two places. What they said that day is in this repository's history at the commit the
|
||||
import read, which every imported issue names (`imported.commit` in `issue-tracker.get`).
|
||||
|
||||
## Steps
|
||||
|
||||
1. Open the issue in the tracker, through the mesh MCP server:
|
||||
`mesh_call issue-tracker.open` with `title`, `report` (the symptom **as observed**, in plain terms, with the
|
||||
evidence that it happened), optionally `located-in` (entries of the tracker's registered list of owners), and, for
|
||||
an issue that gets a folder here, `record` — the folder's name without its number, `short-name`. The tracker
|
||||
answers with the number `NNN`; a refused open spends none.
|
||||
|
||||
**An issue with a folder** then creates `04-ISSUES/NNN-short-name/00-report.md` under exactly that number, on a
|
||||
branch:
|
||||
1. Take the next free number — **across `main` and every open pull request**, not `main` alone.
|
||||
Work sits on unmerged branches for days, so two people both reading `main` allocate the same
|
||||
number; it happened twice in one hour between two machines, and the second collision reached
|
||||
`main` with every check passing (issue 155). `cycle.py` now refuses two records sharing a number,
|
||||
which catches a collision but does not prevent one. Create
|
||||
`04-ISSUES/NNN-short-name/00-report.md`:
|
||||
|
||||
```yaml
|
||||
---
|
||||
status: open
|
||||
opened: YYYY-MM-DD
|
||||
located-in: [] # owning repo(s)/module(s), filled by diagnosis
|
||||
fixed-by: # PR or commit reference, filled at resolution
|
||||
amended-design: # design doc path, when the root cause was a design gap
|
||||
---
|
||||
```
|
||||
|
||||
Then `# NNN. The title`, the title as the tracker holds it, and the symptom as observed, as in the tracker's
|
||||
report. **No status key**: `cycle.py` refuses `status:`, `located-in:`, `fixed-by:`, `amended-design:`,
|
||||
`replay:` or `replay-none:` in any report, because the tracker holds them and two places holding one fact
|
||||
drift.
|
||||
|
||||
**An issue about a condition** links the condition's key at once: `issue-tracker.link` with kind `condition` and
|
||||
the key `mesh-controller.conditions` gives. Every message about that condition then names the issue and where it
|
||||
stands; one no issue links says "No issue tracks this yet." An agent that silences a condition because an issue
|
||||
tracks it links that issue first (ADR 0285).
|
||||
|
||||
**An issue wholly in the tracker** is done at this step.
|
||||
2. Investigate. **Before you start, move the issue to `diagnosing` with `by`**: who works on it and its node, as
|
||||
who@node (`fix-399@laptop`, `operator@workstation`). The tracker keeps you as the issue's **worker**, and `get` and `list`
|
||||
show it with since when. The trail goes in `issue-tracker.note`, and, for a folder-backed issue, in
|
||||
`01-diagnosis.md` in the same folder as well — dated, including what was ruled out. A note keeps you the worker; a
|
||||
worker with nothing done on the issue for a day is stale. Then move it to `located` with `located-in`, which the
|
||||
tracker refuses without a registered owner; that ends your work on it.
|
||||
3. Fix. **Before you start the fix, move the issue to `fixing` with `by`**, even when you diagnosed it yourself. The
|
||||
tracker refuses the move while another worker is current, naming them: ask them, or take it over with
|
||||
`take-over` saying why, which is noted on the issue. A stale worker may be taken over from without a reason.
|
||||
Name the fix in `fixed-by` when it merges, with a move to `fixing` that gives `by` again; the issue stays
|
||||
`fixing` until the acceptance is proven live
|
||||
(ADR 0284,
|
||||
ADR 0281).
|
||||
4. Resolve. `issue-tracker.move` to `resolved`, which the tracker takes only from `fixing`, with `fixed-by`, and, if the root cause was a design gap,
|
||||
run playbook [02](02-graduation.md) and give `amended-design`. **A core issue** — its `located-in`
|
||||
names the controller, the node-engine, the tool runner, the SDK, or the catalogue's bus, forge or build
|
||||
agent — resolves with `replay`, the id of its replay in mesh-lab's replays register, proved to fail on
|
||||
the commit before the fix and pass on it, or with `replay-none` saying why none is possible
|
||||
(ADR 0237);
|
||||
`issue-tracker.move` refuses the move without it. `wontfix` needs a `reason`.
|
||||
Then the symptom **as observed**, in plain terms, with the evidence that it happened.
|
||||
2. Investigate in `01-diagnosis.md` in the same folder — the trail, dated, including what was
|
||||
ruled out. Move `status:` to `diagnosing`, then `located` once the owner is known.
|
||||
3. Resolve. Set `status: resolved`, fill `fixed-by:`, and if the root cause was a design gap,
|
||||
run playbook [02](02-graduation.md) and fill `amended-design:`.
|
||||
|
||||
## Rules
|
||||
|
||||
- Closed issues are never deleted — they are the mesh's symptom-to-component memory.
|
||||
- `fixed-by` names something that will still exist: a commit or a pull request, never a branch. A
|
||||
- `fixed-by:` names something that will still exist: a commit or a pull request, never a branch. A
|
||||
branch is deleted when it merges, so a branch name there is a pointer that resolves to nothing by
|
||||
the time anybody follows it.
|
||||
- A fix that turns out to have broken something else is written back into the record that asked for
|
||||
it, pointing at the new issue. Somebody arriving at a record to learn why the code is the way it
|
||||
is must not have to already know there was a sequel.
|
||||
- A number never moves. A folder carries the number the tracker gave; a folder whose number collides with
|
||||
another (one numbered before the tracker, or by hand) is a fault, and `cycle.py` refuses two records
|
||||
sharing a number.
|
||||
- An issue whose answer is a general lesson should also have that lesson written where a search by
|
||||
the symptom's literal words finds it, so the next person searching it finds it. Both, not either.
|
||||
- `wontfix` is legitimate and requires a sentence saying why; the tracker refuses it without one.
|
||||
- **An agent says when it starts work on an issue**, by moving it to `diagnosing` or `fixing` with `by`. Two agents
|
||||
never start the same work without knowing: the second is refused, naming the first.
|
||||
- **A rule states how it is checked.** The cycle's rules on an issue — a status one of the six, `diagnosing` and
|
||||
`fixing` with a worker and no other current one, `located` with an owner, `resolved` only from `fixing` and with a
|
||||
fix, a core issue with its replay, `wontfix` with a reason — are refused at the write by
|
||||
`issue-tracker.move` (mesh-catalog `modules/mesh-issues`, its table test over the rules). `cycle.py` checks what
|
||||
this repository still holds: no two folders share a number, every report has frontmatter with the day it was
|
||||
`opened:`, and no report carries a status key.
|
||||
- Renumbering a collision happens once, in the branch that lands last. Renumbering a branch whose
|
||||
author is still pushing only moves the race.
|
||||
- An issue whose answer is a general lesson should also be written to the knowledge base, so
|
||||
the next person searching a symptom finds it. Both, not either.
|
||||
- `status: wontfix` is legitimate and requires a sentence saying why.
|
||||
|
||||
@@ -1,67 +1,30 @@
|
||||
# Playbook 04 — Work breakdown and handoff
|
||||
# Playbook 04 — Build handoff
|
||||
|
||||
**Trigger.** A to-be design is settled (playbook [02](02-graduation.md)) and its building is about to start.
|
||||
**Trigger.** A to-be design is settled and work is about to start in a code repository.
|
||||
|
||||
**Who runs it.** The lead who will run the feature, on the larger model. It is one sitting, bounded at about
|
||||
an hour, and said to the operator if it runs past.
|
||||
|
||||
## What it produces
|
||||
|
||||
A design becomes **milestones**, and each milestone becomes **issues**:
|
||||
|
||||
- **A milestone is usable on its own.** Once its delivery plan is done, the operator or the mesh can do
|
||||
something new, said in one sentence ("a Claude Code session is mirrored into a thread on the bus").
|
||||
A milestone that changes nothing anyone can use is not a milestone; fold it into the next.
|
||||
- **A milestone is one feature step** of playbook [07](07-feature-branches.md): prepared, developed in one to
|
||||
two hours, reviewed, checked and delivered as one. A milestone larger than that is split.
|
||||
- **An issue is one repository's part of a milestone,** sized so a fixing agent finishes it inside the
|
||||
milestone's development time. Issues that must land in order are linked `depends-on`.
|
||||
**Who runs it.** Whoever starts the build.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Confirm the design is settled.** Its frontmatter reads `status: designed`, and every claim in it traces
|
||||
to a decision record in the knowledge base (`knowledge-base.get` for each entry of its `decisions:`). An
|
||||
open question in the text is a reason to go back to playbook [01](01-research.md), not to build around it.
|
||||
2. **Name the owners.** Set `code:` in the design's frontmatter to the repositories from
|
||||
[`repos.md`](../repos.md). A repository that does not exist yet is added to `repos.md` in the same change.
|
||||
3. **Check the as-is.** Read the matching `03-DESIGN/00-as-is/` document. What is being replaced is stated
|
||||
there; if it is not, write it before changing it. Building against an undocumented as-is is how a shipped
|
||||
behaviour gets lost.
|
||||
4. **Break it into milestones.** Write them into the design, in a section *Milestones*: for each, its number,
|
||||
its one-sentence outcome, the acceptance tests of the design it proves, the repositories it touches, and
|
||||
what it depends on. Order them so each is usable as soon as it is delivered, the smallest useful one
|
||||
first. A milestone is named by what it lets someone do; how it ships is shaped by the plan its changes would
|
||||
have, asked through `mesh-delivery.what-if`: the modules it moves and their fan-out (ADR 0401 §6).
|
||||
5. **Ask the operator to confirm the milestones**, in a short table: number, outcome, the acceptance tests it
|
||||
proves, repositories, the parts each repository builds. Change what they change.
|
||||
6. **Open the issues.** For each confirmed milestone, `issue-tracker.open` one issue per repository part, its
|
||||
title the symptom or capability in plain words, its report naming the design, the milestone and the
|
||||
decisions it builds with; link the order with `depends-on`. Write the issue numbers into the milestone's
|
||||
entry in the design.
|
||||
7. **Flip the status.** `status: in-progress`, `updated:` today; the design change goes through playbook
|
||||
[08](08-small-fixes.md) (one hq pull request).
|
||||
8. **Hand each milestone to playbook 07, one at a time.** The milestone is the feature step; its issues are the
|
||||
step's issues; the next milestone starts when the previous one's delivery plan is done.
|
||||
9. **On completion of the last milestone**, run the "when something ships" section of playbook
|
||||
1. **Confirm the design is settled.** Its frontmatter reads `status: designed`, and every
|
||||
claim in it traces to a record in [`02-DECISIONS/`](../../02-DECISIONS/). An open question in the text is a
|
||||
reason to run playbook [01](01-research.md), not to start building around it.
|
||||
2. **Name the owner.** Set `code:` in the design's frontmatter to the repositories from
|
||||
[`repos.md`](../repos.md). If the repository does not exist yet, add it to `repos.md` in
|
||||
the same change.
|
||||
3. **Check the as-is.** Read the matching `03-DESIGN/00-as-is/` document. What is being
|
||||
replaced is stated there; if it is not, write it before changing it. Building against an
|
||||
undocumented as-is is how a shipped behaviour gets lost.
|
||||
4. **Flip the status.** `status: in-progress`, `updated:` today.
|
||||
5. **Build in the code repository.** HQ is not a code repository and never carries
|
||||
implementation.
|
||||
6. **On completion**, run the "when something ships" section of playbook
|
||||
[02](02-graduation.md).
|
||||
|
||||
## Rules
|
||||
|
||||
- What the handoff leaves for later is an issue with an owner, named in the design or the decision that leaves
|
||||
it — never a sentence saying "later". The design's `code:` names who owns what is built; the issue names who
|
||||
owns what is not. The knowledge base's rule `deferral` checks the decision record when it is proposed
|
||||
([issue 354](../../04-ISSUES/354-a-decided-rename-was-never-scheduled-and-the-operator-believed-it-live/00-report.md)).
|
||||
- HQ is not a code repository and never carries implementation.
|
||||
- How the building is done — worktrees, agents, review, checks, the ready mark and the delivery — is playbook
|
||||
07's, not this one's.
|
||||
|
||||
## How it is checked
|
||||
|
||||
Read by the lead and the reviewer today; a check in hq's merge-check is tracked by issue 845.
|
||||
|
||||
- A design in `status: in-progress` has a *Milestones* section, each milestone with its outcome and its
|
||||
issues; an in-progress design without one is the smell of building without a breakdown.
|
||||
- Every issue a milestone names exists in the tracker (`issue-tracker.get`), and an issue in `fixing` names
|
||||
its milestone in its report.
|
||||
- A milestone whose issues are all `resolved` has had its delivery plan done; its outcome sentence and
|
||||
acceptance tests are proven on the mesh before the next milestone starts.
|
||||
- Every merge is a human checkpoint, without exception.
|
||||
- Never open a pull request unprompted. A permissions list saying it is allowed is not a
|
||||
request.
|
||||
- Work in an isolated worktree, never a shared checkout. A failed `cd` in a shared checkout
|
||||
commits to the wrong branch, and the error scrolls past.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Playbook 05 — Constitution sync
|
||||
|
||||
Implements ADR 0021: HQ is
|
||||
the source of the constitution, and the constitution page is derived, never edited.
|
||||
Implements [ADR 0021](../../02-DECISIONS/0021-hq-is-the-source-of-the-constitution.md): HQ is
|
||||
the source of the constitution, and the knowledge-base page is derived, never edited.
|
||||
|
||||
**Trigger.** [`how-we-build.md`](../how-we-build.md) changed a rule that the mesh enforces at
|
||||
runtime.
|
||||
@@ -24,7 +24,7 @@ and, per the repository's own rule, it is how the rule "HQ is the source" is che
|
||||
the derived page carries the rule alone.
|
||||
2. Record the change as a decision — a rule the mesh enforces is architecturally significant.
|
||||
Playbook [02](02-graduation.md).
|
||||
3. Publish the derived page where the mesh keeps the constitution page, under its slug, replacing its
|
||||
3. Publish the derived page to the knowledge base under the constitution slug, replacing its
|
||||
body. Keep the section numbering stable: the meeting orchestrator and the review fragments
|
||||
cite sections by number.
|
||||
4. Verify the derived page reads back with the change present. A publish that reported success
|
||||
|
||||
@@ -25,7 +25,7 @@ Three questions, answered from the machine:
|
||||
## The steps
|
||||
|
||||
1. **Name what it provides and requires**, if anything. A name is what a consumer is coupled to,
|
||||
not the role it plays (ADR 0027):
|
||||
not the role it plays ([ADR 0027](../../02-DECISIONS/0027-a-provision-names-what-the-consumer-is-coupled-to.md)):
|
||||
`postgres-database`, not `database`. Most modules provide nothing and require nothing — an
|
||||
application is usually a leaf.
|
||||
|
||||
@@ -52,7 +52,7 @@ Three questions, answered from the machine:
|
||||
the container env-file: [/var/lib/postgres/superuser.env]
|
||||
```
|
||||
|
||||
The node-engine fills the hole on the machine, which is the only place both halves exist — the mesh
|
||||
The host fills the hole on the machine, which is the only place both halves exist — the mesh
|
||||
discarded the value
|
||||
([credentials and their rotation](../../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md)).
|
||||
A **provisioner** is the exception: it reads a password file, so it mounts the `.secret`
|
||||
@@ -89,7 +89,7 @@ Six attempts, one real bug. Recorded because the ratio is the lesson: **the mesh
|
||||
time and the scaffolding was not.**
|
||||
|
||||
- A shape existed in the language and no host implemented it, so every declaration carrying one
|
||||
was refused whole — correctly, and the node-engine said exactly that. **Nobody was reading the node-engine's
|
||||
was refused whole — correctly, and the host said exactly that. **Nobody was reading the host's
|
||||
log.** Read it first; it is the only place that says why a machine did nothing.
|
||||
- A blind find-and-replace renamed a provision in quotes and missed the same word bare.
|
||||
- A command was tested only for the invocations that should fail, so it rejected every real one
|
||||
@@ -99,9 +99,9 @@ time and the scaffolding was not.**
|
||||
|
||||
## Rules
|
||||
|
||||
- **Read the node-engine's log before theorising.** A declaration that was sent and not applied says so
|
||||
- **Read the host's log before theorising.** A declaration that was sent and not applied says so
|
||||
there and nowhere else.
|
||||
- **A failing test is kept, not skipped.** It is the reproduction.
|
||||
- **Never rotate during an adoption.** Rotation is a separate act, afterwards, deliberately.
|
||||
- **A data directory is never removed by the mesh** (ADR 0030),
|
||||
- **A data directory is never removed by the mesh** ([ADR 0030](../../02-DECISIONS/0030-data-outlives-the-mesh-that-declared-it.md)),
|
||||
and that protects against the mesh only — not against a disk or a mistaken command.
|
||||
|
||||
@@ -1,259 +1,69 @@
|
||||
# Playbook 07 — Features
|
||||
# Playbook 07 — Feature branches across repos
|
||||
|
||||
**Trigger.** A feature: work that changes code in one repository or several (`mesh-sdk`, `mesh-controller`,
|
||||
`mesh-catalog`, `mesh-host`, `mesh-lab`, and `hq` when a design rides along), too large to be one small fix.
|
||||
A small fix (one repository, one issue, about an hour of work) follows
|
||||
[playbook 08](08-small-fixes.md) instead.
|
||||
**Trigger.** Work that changes code — in one code repo or in several at once (`mesh-sdk`,
|
||||
`mesh-controller`, `mesh-catalog`, `mesh-host`, `mesh-lab`, and `hq` when a decision rides along).
|
||||
|
||||
**Who runs it.** Anyone who writes code, engineers and agents alike. Agents follow it exactly.
|
||||
**Who runs it.** Anyone who writes code, engineers and agents alike. Agents follow it exactly —
|
||||
it is the guard against the failure it was written for.
|
||||
|
||||
## The failures it prevents
|
||||
## The failure it prevents
|
||||
|
||||
- **Pull requests as scratchpads.** A feature worked as one pull request per unit of thought left sixteen
|
||||
stacked pull requests unmerged, consolidated by hand afterwards. A pull request is the end of a feature,
|
||||
not a step in it.
|
||||
- **Waiting on checks mid-feature.** A feature worked as one pull request per issue into protected
|
||||
branches spent most of a day waiting for merge checks while the code was ready.
|
||||
- **Context bought again and again.** A fresh agent per small issue spends most of its run reading the
|
||||
playbook, the issue and the code before it writes a line. On 2026-10-14 an issue sized for five to ten
|
||||
minutes took forty-one.
|
||||
- **Finished work that never ships.** A feature that kept taking new issues held thirteen landed issues
|
||||
undelivered for a day: an issue done at 21:09 was still not on the machines at 15:00 the next day.
|
||||
- **Work that waits and nobody notices.** Most lost time was not work at all, but a check, a delivery plan
|
||||
or an agent waiting on something that had stopped, unseen.
|
||||
- **Review that keeps hardening.** One fix went through seven review rounds in five and a half hours, each
|
||||
asking for more hardening than its issue asked (issue 704, 2026-10-14). A round is for what makes the
|
||||
change wrong; everything else is an issue of its own.
|
||||
- **The first check after the whole branch is written.** A feature's catalogue branch, pushed once after
|
||||
hours of local work, met its first check on 2026-10-16 and failed on 29 problems, TypeScript errors among
|
||||
them, that a unit suite run continuously while working finds at once.
|
||||
- **A green pull request nobody marks.** An approved, green pull request that was not marked ready waited
|
||||
12 h 17 min for someone to merge it (mesh-catalog #300, 2026-10-14).
|
||||
- **A module proven only after it reached the machines.** The mail modules were delivered with unit tests
|
||||
only, then failed live three times (issues 859, 865, 868) and took five more hours of fixes.
|
||||
A feature was worked as a branch-and-MR per *unit of thought* — one per decision, one per
|
||||
stacked increment — and each MR was treated as finished when it was *opened*, not when it was
|
||||
*merged*. Across repos the same feature took a different branch name in each. The MRs piled up
|
||||
unmerged: one session left **sixteen** stacked intermediate MRs that had to be consolidated and
|
||||
closed by hand. An MR is a review checkpoint, not a scratchpad.
|
||||
|
||||
## The shape
|
||||
## The rule
|
||||
|
||||
One feature is **one branch name**, **one worktree per repository**, **one pull request per repository**,
|
||||
opened **once**, after the review. It runs in five phases, and its time is bounded:
|
||||
One feature is **one branch name**, **one worktree per repo**, **one MR per repo**, opened
|
||||
**once, at the end**.
|
||||
|
||||
| Phase | What happens | Bound |
|
||||
|---|---|---|
|
||||
| 0. Prepare | the lead settles the open questions and writes one context file per repository | 15 minutes |
|
||||
| 1. Develop | one fixing agent per repository builds every issue locally, unit tests only | 1 to 2 hours |
|
||||
| 2. Review | one reviewer reads all of it; reports go both ways; at most 3 rounds | 15 minutes |
|
||||
| 3. Check | push; the forge's merge checks run the integration tests; no further review | one check run |
|
||||
| 4. Deliver | merge; the delivery plan builds, publishes and sends the whole feature | its delivery plan |
|
||||
1. **Name the feature once.** `feat/<slug>`. The *same* branch name in every repo the feature
|
||||
touches — never a different name per repo, never a fresh branch per increment within the
|
||||
feature.
|
||||
2. **Isolate each repo.** One git worktree per touched repo under `.work/<slug>/<repo>`, branched
|
||||
off `main`:
|
||||
```
|
||||
git worktree add .work/<slug>/<repo> -b feat/<slug> origin/main
|
||||
```
|
||||
Parallel features never collide, and no shared checkout is edited.
|
||||
|
||||
A feature that cannot be developed in about two hours is split into **steps**. Each step runs all five
|
||||
phases on its own; the next step starts only when the previous step's delivery plan is done. A step is the
|
||||
unit that ships, so nothing finished waits for work that is not.
|
||||
**Add the untouched siblings the lab reads.** The lab beds find the other repositories by
|
||||
sibling path from the lab checkout (`../mesh-tools/module.json`, `../mesh-sdk`, …), the way
|
||||
the main layout has them. A `.work/<slug>/` directory holding only the touched repos fails a
|
||||
bed at once with `no manifest for mesh-tools at …/.work/<slug>/mesh-tools/module.json`, after
|
||||
genesis has already passed. Give the directory those repos as **detached worktrees on `main`**
|
||||
— never symlinks:
|
||||
```
|
||||
git worktree add --detach .work/<slug>/mesh-tools main
|
||||
```
|
||||
3. **Commit as you go — locally.** Increments land on the one branch. Nothing is pushed and no
|
||||
MR is opened mid-feature.
|
||||
4. **Finish, then publish.** When the whole feature is done — every repo, tests green — push
|
||||
every branch and open **one MR per touched repo**, together.
|
||||
5. **Merge promptly, once approved.** Every merge into `main` is notified and approved
|
||||
([ADR 0023](../../02-DECISIONS/0023-approval-is-the-checkpoint.md)); once it is, merge —
|
||||
do not leave it sitting. The branch is deleted on merge.
|
||||
6. **Leave nothing behind.** After the MRs merge, no `feat/<slug>` branch and no `.work/<slug>`
|
||||
worktree survive.
|
||||
|
||||
**Scope is fixed at phase 0.** An issue found during a feature goes to the next step or the next feature,
|
||||
never into the running one, unless it blocks the running one.
|
||||
## What this is not
|
||||
|
||||
## Phase 0 — Prepare (the lead, larger model)
|
||||
|
||||
1. **Name the feature once:** `feat/<slug>`, the same branch name in every repository it touches. The
|
||||
branch is never protected.
|
||||
2. **Make the worktrees:** one per touched repository under `.work/<slug>/<repo>`, branched off `main`
|
||||
(`git worktree add .work/<slug>/<repo> -b feat/<slug> origin/main`). The lab beds find the other
|
||||
repositories by sibling path, so add the untouched siblings the lab reads as detached worktrees on
|
||||
`main` (`git worktree add --detach .work/<slug>/mesh-tools main`), never symlinks.
|
||||
3. **Settle every open question** in the feature's issues before any agent starts: on the issue, through
|
||||
the knowledge base, or by asking the operator. A fixing agent that meets an undecided design question
|
||||
stops, and its run is spent again.
|
||||
4. **Write one context file per repository**, `.work/<slug>/<repo>.context.md`. It is what a fixing agent
|
||||
reads instead of rediscovering the code, and it is reused by every agent and every round:
|
||||
- the issues it builds, in order, each with: the files and functions it changes, the decisions it builds
|
||||
with (quoted, with where each was decided), the test to write first, done-when in one sentence;
|
||||
- the repository's map for this feature: the packages and files involved, with one line each on what they
|
||||
hold, and the patterns to copy (file:line);
|
||||
- what not to touch, and what lands beside it in other repositories;
|
||||
- the local facts: the worktree, the branch, the check commands, a temporary directory for tests outside
|
||||
the home (`TMPDIR`), and the unit tests to run (never a database server: tests that need one are
|
||||
integration tests, which the forge's checks run).
|
||||
When the reading is large, a read-only agent on the smaller model answers the lead's exact questions with
|
||||
file:line, and the lead writes the file from its answers.
|
||||
5. **Move the issues to `fixing`**, `by` naming the feature's agent for that repository.
|
||||
|
||||
## Phase 1 — Develop (one fixing agent per repository, smaller model)
|
||||
|
||||
- **One agent per repository, kept for the whole feature.** It reads its context file once and builds the
|
||||
issues one after another, committing each locally with a message naming the issue ("Keep the assignments
|
||||
on the bus (issue 652)"). It is not replaced per issue: the lead continues the same agent (by its id) with
|
||||
the next instruction, so its context is paid for once.
|
||||
- **Unit tests only, of what changed, each well under a second** (how-we-build §5, ADR 0391). Each issue's
|
||||
first test fails before its change and passes after; `gofmt`, `go vet`
|
||||
and the repository's unit tests stay green. The integration tests (`merge-check.sh` with the store and the
|
||||
bus) wait for phase 3.
|
||||
- **Nothing is pushed** and no pull request is opened in this phase. Never land on a red check: run the
|
||||
checks, read the result, then land in a separate step.
|
||||
- **The fixing agent's brief** (the lead gives it with the context file): name your issue in every step and
|
||||
message ("issue 805: landing in mesh-catalog"); reach the mesh only through the mesh MCP server, never ssh;
|
||||
never push, open, merge or close a pull request; never write to a store's database by hand, a schema change
|
||||
is a numbered migration and the consolidated schema; Go only unless the context file says otherwise; never
|
||||
move another issue. Report back in under 150 words: commits, tests added and that they failed first, the
|
||||
check results, anything you stopped on.
|
||||
- **An agent that finds the code differs from its context file stops and reports** (file:line); the lead
|
||||
settles it, updates the context file, and continues the agent.
|
||||
- **Two repositories that depend on each other** keep their order in the context files: the reader before the
|
||||
writer, the controller's new manifest word before the catalogue manifest that uses it.
|
||||
- **Several agents in one repository** (only for a large repository with independent areas): each works on a
|
||||
local branch of its own from the feature branch's head and lands by fast-forward under a lock
|
||||
(`flock <feature worktree>.lock git -C <feature worktree> merge --ff-only <its branch>`), after its unit
|
||||
tests pass; if another landed first, it merges again and repeats. In a shared clone, fetch the feature
|
||||
branch by name before branching from it. Never resolve a conflict with a blind `git merge-file --union`.
|
||||
|
||||
## Phase 2 — Review (one reviewer, larger model; at most 3 rounds, about 15 minutes)
|
||||
|
||||
1. **Bring every feature branch up to the current trunk** (fetch by name, then merge), so the reviewed tree is
|
||||
the tree that is pushed.
|
||||
2. **One reviewer reads every repository's local commits** together, against the context files and the
|
||||
decisions. Its report is per repository: each finding as file:line, the failure it causes, and the fix.
|
||||
It says which round it is in ("round 1 of at most 3").
|
||||
3. **The report goes to the same fixing agents** that wrote the code, continued by their ids, so they fix
|
||||
with the context they already hold. Each answers the reviewer: what it changed, per finding, or why not.
|
||||
4. **The reviewer reads the answers and the new commits**, and approves or reports again (step 5 says what a round
|
||||
may ask). After round 3 the lead stops and brings the operator what is left, with a recommendation.
|
||||
5. **A finding outside the issues' done-when is never a change asked, in any round.** More hardening, a
|
||||
further case, a nicer form: the reviewer opens it with `issue-tracker.open` and names the number in its
|
||||
report, and the round does not wait for it. What a round asks is only what makes the change wrong against
|
||||
its issues and its decisions.
|
||||
|
||||
## Phase 3 — Check (no review)
|
||||
|
||||
1. **Push every feature branch and open one pull request per repository, together.**
|
||||
The integration tests
|
||||
run only in the merge check (the operator, 2026-10-16), never before. Pull requests in two or
|
||||
more repositories that share the branch name are one delivery group (ADR 0239 §2).
|
||||
Where a session pushes through the forge tool `gitea_commit` (one commit on the forge's current base), the
|
||||
reviewer first confirms that commit's tree equals the reviewed one.
|
||||
2. **The forge's merge checks run the integration tests once** (`mesh/merge-gate`, `mesh/repo-check`). No
|
||||
second review: phase 2 reviewed this tree.
|
||||
3. **A check that fails** goes back to the reviewer with the failure, for **one more cycle** of phase 2 with
|
||||
the same fixing agents; then push once again.
|
||||
4. **The step's acceptance is proven before it is marked ready.** A step that adds or changes a module
|
||||
carries an integration test that starts the module whole, its process against the real bus and store, and
|
||||
shows the step's acceptance sentence; the forge's checks run it (ADR 0391 §1, §6). Unit tests alone are not
|
||||
that proof; the mesh has no end-to-end tests yet, and the lab is not used (the operator, 2026-10-16).
|
||||
5. **The reviewer marks each pull request ready as soon as the acceptance is shown** with
|
||||
`mesh-delivery.mark-ready` (the delivery of the head it approved, who reviewed it and why), which writes the
|
||||
label `mesh/ready` and then `ready at <full sha>`, the order the forge counts (issue 913); mesh-delivery merges it when its checks pass
|
||||
and the operator approved the merge into main on Telegram, the channel that proves who answered today (ADR 0364,
|
||||
ADR 0398 §3; an Approve at the terminal is issue 953). Beyond that
|
||||
Approve, the operator's word is waited for only when the step changes what the operator sees, removes
|
||||
data, or needs a step only the operator's terminal can take; the lead says which before the step starts.
|
||||
|
||||
**One exception: a catalogue change that uses a new manifest word.** The catalogue's merge check reads
|
||||
manifests with the controller the mesh runs, so the controller's pull request is merged and delivered first;
|
||||
then `mesh-delivery.recheck` the catalogue's and merge it on the new verdict. Plan such a feature as two steps.
|
||||
The exception ends when milestone 3 of design 47 is delivered (issue 977, ADR 0402, ADR 0403): the group's composed
|
||||
check, judged by the group's controller, is then each member's gate, and the feature is one group.
|
||||
|
||||
## Phase 4 — Deliver
|
||||
|
||||
The merge starts the delivery plan: build, publish, send. A merge into a branch other than `main` is not a
|
||||
delivery: no machine receives it (issue 889 while it still starts one). The lead follows it to done
|
||||
(`mesh-controller.plans` with `wait` and `until: done`), then resolves the feature's issues with `fixed-by`
|
||||
naming its pull requests, and removes what is left: no `feat/<slug>` branch and no `.work/<slug>` worktree
|
||||
survive.
|
||||
|
||||
## Models
|
||||
|
||||
| Who | Model | Why |
|
||||
|---|---|---|
|
||||
| The lead (phase 0 decisions, the context files, following and reporting) | larger | judgement |
|
||||
| Read-only questions about the code for the context files | smaller | reading, answered with file:line |
|
||||
| Fixing agents (phase 1, and the fixing in phases 2 and 3) | smaller | building to a written context |
|
||||
| The reviewer (phase 2, and the extra cycle in phase 3) | larger | judgement, once per round |
|
||||
| Decision records | larger | judgement |
|
||||
|
||||
Every agent the mesh runs draws on one weekly allowance shared by every session, and the larger model
|
||||
spends it many times faster. Up to five agents on the smaller model at once, at most two subagents on the
|
||||
larger one (the lead not counted).
|
||||
|
||||
## Following and reporting
|
||||
|
||||
Most lost time is waiting nobody saw. The lead therefore keeps a **wait list**: everything the feature waits
|
||||
on, with when it started and its bound.
|
||||
|
||||
| Waiting on | Bound | Past the bound |
|
||||
|---|---|---|
|
||||
| a fixing agent's issue | 30 minutes | ask it where it stands; split or take over |
|
||||
| the review | 15 minutes a round | ask the reviewer; stop at round 3 |
|
||||
| a merge check | 15 minutes | read the build queue (`mesh-controller.queue`) and the check's log |
|
||||
| the ready mark | 15 minutes after the acceptance is shown | the reviewer marks it, or the lead does |
|
||||
| a merge after ready | 5 minutes after the checks pass | read `mesh-delivery.show` for the delivery |
|
||||
| a delivery plan | its class's budget (`mesh-delivery.times`), at most 30 minutes | read `mesh-controller.plans`, then `mesh-delivery.stalled` |
|
||||
|
||||
**Every five minutes, every two minutes while a delivery plan runs, and at once when something lands, stops or
|
||||
needs the operator,** the lead reports, in one of the two layouts below (the operator, 2026-10-16: an overview
|
||||
of the issues, often). Times are local and absolute, from the clock (`date`), never guessed; every number has its
|
||||
unit and its bound or baseline; an issue is named by its number and a few words, never by its number alone.
|
||||
|
||||
**The full layout** (in the session), the issues as a tree: grouped by repository, each issue under the one it
|
||||
waits for, marked ✓ landed, ▶ fixing, ⏸ waiting, ✗ stopped:
|
||||
|
||||
```
|
||||
Feature <slug> · step <n> of <m> · phase <name>, <minutes used> of <bound> min
|
||||
Issues: <landed> of <total> landed · <fixing> fixing · <waiting> waiting
|
||||
|
||||
mesh-controller (agent: fixing, 2 of 3 landed)
|
||||
├── ✓ 652 assignments on the bus landed 14:02
|
||||
├── ▶ 736 placement words leave the layers fixing since 13:51, 23 of 30 min
|
||||
│ └── ⏸ 751 config keys named (mesh-catalog) waits for 736
|
||||
└── ✓ 654 a file names its template landed 13:20
|
||||
mesh-catalog (agent: waiting)
|
||||
└── ⏸ 751 (above, under 736)
|
||||
|
||||
Opened along the way: <each issue opened during the feature, its number and a few words, and where it goes> — or "none"
|
||||
Past its bound: <the wait, how long, what is being done> — or "none"
|
||||
Needs you: <one line per question, with my recommendation> — or "nothing"
|
||||
Allowance: week <n> % (<+d> since the last report), session <n> %
|
||||
Next: <the next action, one line>
|
||||
```
|
||||
|
||||
**The short layout** (on a channel such as Telegram, or when nothing changed but time):
|
||||
|
||||
```
|
||||
<slug> step <n>/<m> · <phase> <used>/<bound> min · <landed>/<total> landed · past bound: <none | what> · needs you: <nothing | what> · week <n> %
|
||||
```
|
||||
|
||||
A report says what changed since the last one first. In the session every report is the full layout, the whole
|
||||
tree of the step's issues included, also when only time passed: the operator reads the state from it without
|
||||
asking. The short layout is for a channel.
|
||||
|
||||
## The context of the lead's own session
|
||||
|
||||
At the end of each phase, and whenever the session's context grows large or spread over unrelated work, the
|
||||
lead prepares a `/clear` as the mesh's instruction "prepare before clear" says: a save point with the
|
||||
feature's phase, its branches and heads, the context files' paths, the agents' ids, the wait list and the
|
||||
next action; then the prompt the operator pastes. The context files make the feature resumable after a clear.
|
||||
|
||||
## Issues that stop
|
||||
|
||||
An issue stays in `fixing` only while an agent works on it. A step that stops, for a wait past its bound, a
|
||||
hand-over, or the session's end, moves each of its issues not delivered back to `open` with
|
||||
`issue-tracker.move` and a note saying how far it got (its branch, its last commit, what is left). A list of
|
||||
issues in `fixing` that nobody works on reads to the operator as stuck work.
|
||||
- **Not a licence to batch unbounded work.** A feature is a *bounded* unit; if it sprawls for
|
||||
days, end-of-feature bloat merely replaces per-increment bloat. Split it into features, each
|
||||
its own branch and MR.
|
||||
- **Not a second trunk.** Every repo branches off `main`. There is no longer an `initialization`
|
||||
trunk.
|
||||
|
||||
## How it is checked
|
||||
|
||||
- A feature's issues show their commits on the feature branch (`git log --grep 'issue <n>'`), and no pull
|
||||
request into a feature branch exists. A pull request per issue, or an agent waiting on a merge check in
|
||||
phase 1, is the failure this playbook prevents.
|
||||
- A context file exists per touched repository under `.work/<slug>/` while the feature runs.
|
||||
- After a feature's delivery, `git branch -r | grep feat/<slug>` and `git worktree list` return nothing for it.
|
||||
- More than one open pull request in a repository that share no feature name, or a feature step whose issues
|
||||
stay in `fixing` for more than a working day, is the smell: stop and bring it to the operator.
|
||||
- A review report that asks a change outside the issues' done-when, or a fourth round, is the failure of the
|
||||
review rules; `issue-tracker.list` with `flag: abandoned` lists issues left in `fixing` with nobody on them.
|
||||
- A pull request reviewed and green but not marked ready for more than 15 minutes is a lead that did not
|
||||
mark it.
|
||||
- A step that adds or changes a module marked ready with no integration test that starts the module whole is
|
||||
the failure of phase 3's fourth step.
|
||||
- A bed run from `.work/<slug>/mesh-lab` that fails naming a `.work/<slug>/<repo>/…` path it cannot find is a
|
||||
missing sibling worktree, not a mesh fault.
|
||||
The end state is visible, and its absence is the smell:
|
||||
|
||||
When this playbook meets a problem, the lesson is added here, so the next feature avoids it.
|
||||
- After a feature merges, `git branch -r | grep feat/<slug>` and `git worktree list` return
|
||||
nothing for it. A surviving branch or worktree means step 6 was skipped.
|
||||
- A bed run from `.work/<slug>/mesh-lab` that fails naming a `.work/<slug>/<repo>/…` path it
|
||||
cannot find is a missing sibling worktree, not a mesh fault.
|
||||
- More than one open MR in a repo that share no feature name, or a stack of MRs none of which is
|
||||
merged, is the failure this playbook exists to prevent — stop and consolidate before opening
|
||||
more.
|
||||
|
||||
@@ -1,101 +0,0 @@
|
||||
# Playbook 08 — Small fixes
|
||||
|
||||
**Trigger.** One issue in one repository, small enough to build in about an hour: a bug with a known cause,
|
||||
a missing check, a wording, a playbook or design edit. Anything larger, or anything that needs a change in a
|
||||
second repository first, is a feature and follows [playbook 07](07-feature-branches.md). A feature is the
|
||||
larger kind: it touches several modules and has a research effort or a design before it. **Needing a decision
|
||||
does not make a fix a feature**: the decision is settled first (step 1) and the fix stays small (the operator,
|
||||
2026-10-16).
|
||||
|
||||
**Related issues in one fix.** A small fix may cover two or three related issues when they are in the same
|
||||
repository, mostly in the same module, and together still fit in about an hour: they touch the same files, or
|
||||
one would be wrong or unsafe without the other (switching a clipboard's history on before its history tool stops
|
||||
handing out what was copied). Each issue is named in the commit and the pull request and moved to `fixing`,
|
||||
and each is resolved on its own once the fix is delivered. More issues, a second repository, or more than an
|
||||
hour is a feature.
|
||||
|
||||
**Who runs it.** Anyone who writes code, engineers and agents alike.
|
||||
|
||||
## The aim
|
||||
|
||||
A small fix spends its time on the fix, and runs the long part once: **one review, one push, one check run,
|
||||
one delivery**. Everything before the push is local and fast.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Read the issue and the code it names** (the session's own model, or one read-only agent on the smaller
|
||||
model answering exact questions with file:line). If a design question is open, it is settled before any
|
||||
code is written: on the issue, through the knowledge base, or by the operator. A fix that turns out to
|
||||
need a second repository, or more than about an hour, stops here and becomes a feature; one that needs a
|
||||
decision stays a small fix.
|
||||
2. **Move the issue to `fixing`** with `by` as who@node.
|
||||
3. **Build on a local branch** `fix/<issue>-<slug>`, in a worktree of its own branched from `main`. Write the
|
||||
test that fails today first; make the change; keep `gofmt`, `go vet` and the unit tests of what changed green
|
||||
(each well under a second; never a merge check or a database locally: how-we-build §5, ADR 0391).
|
||||
Commit, naming the issue. For a playbook or design edit, run the repository's own checks (in hq,
|
||||
`sh merge-check.sh`).
|
||||
4. **One review of the local commit, before anything is pushed** (a reviewer that did not write it, larger
|
||||
model; at most 3 rounds, as playbook 07; from round 2 only what is wrong; a finding outside the issue's
|
||||
done-when is opened as an issue of its own, never asked as a change). The reviewer's findings go back to the same agent that wrote the fix, continued by
|
||||
its id, so the fix keeps its context. A playbook or design edit of a few lines may skip the reviewer: the
|
||||
operator's word on the pull request is its review, and the lead marks it ready on that word.
|
||||
5. **Push once and open the pull request.** The trunk is what runs and nothing holds merged work back
|
||||
(ADR 0401), so a fix waits for no other change's delivery. The forge's merge
|
||||
check runs the integration tests, the only
|
||||
place they run (the operator, 2026-10-16). A failed check
|
||||
goes back to the same agent with the failure, and the reviewer reads only the change it makes.
|
||||
6. **The reviewer marks it ready at once** (for an edit that skipped the reviewer, the lead, on the
|
||||
operator's word) with `mesh-delivery.mark-ready` (the delivery of the head it approved, who reviewed it and
|
||||
why), which writes the label `mesh/ready` and then `ready at <full sha>`, the order the forge counts (issue 913);
|
||||
mesh-delivery merges it when its checks pass. A fix to a module that changes how it runs carries an
|
||||
integration test starting the module whole against the real bus and store, run by the forge's checks
|
||||
(ADR 0391 §1, §6); unit tests alone are not that proof, and the lab is not used. The operator's word is waited for only when the fix changes what the operator sees, removes data,
|
||||
or needs the operator's terminal.
|
||||
7. **Follow its delivery plan to done** (`mesh-controller.plans` with `wait` and `until: done`), prove the fix on
|
||||
the mesh (the symptom gone, measured), and resolve the issue with `located-in` and `fixed-by` the same hour.
|
||||
|
||||
A fix that stops before its delivery moves its issue back to `open` with a note saying how far it got
|
||||
(branch, last commit, what is left), as playbook [07](07-feature-branches.md) says.
|
||||
|
||||
**A second fault found by the delivery.** When a fix's delivery shows a second fault that keeps the issue's
|
||||
done-when from holding, and that fault is in the same files, it is taken into the same issue as a second pull
|
||||
request with a review round of its own. A fault in other code is opened as an issue of its own, and the first
|
||||
issue waits for it (issue 937: mesh-catalog #335 was taken in; issue 943 was opened).
|
||||
|
||||
**A fix that moves a file to a new filler compares the values first.** When a fix makes a module's file filled a
|
||||
new way for the first time (the provisioner client in place of the controller, a provider's entry in place of a
|
||||
composed value), the fix lists every value the old way put into that file on each machine it reaches and shows,
|
||||
in its review, that the new way gives the same values, or says which differ and why. A difference nobody looked
|
||||
at is an outage: on 2026-10-16 the forge and the identity provider on the control node were down for 17 minutes
|
||||
because postgres's consumers were served the port inside its container instead of the port it is published on
|
||||
(issue 957). The comparison is read from the machines (the file as it is today) before the merge, not from the
|
||||
code.
|
||||
|
||||
## Bounds
|
||||
|
||||
| Waiting on | Bound | Past the bound |
|
||||
|---|---|---|
|
||||
| building the fix | 1 hour | it is a feature: stop and plan it under playbook 07 |
|
||||
| the review | 15 minutes | ask the reviewer where it stands |
|
||||
| the ready mark | 15 minutes after the review approved | the reviewer marks it, or the lead does |
|
||||
| the merge check | 15 minutes | read `mesh-controller.queue` and the check's log |
|
||||
| the merge after ready | 5 minutes after the checks pass | read `mesh-delivery.show` for the delivery |
|
||||
| the delivery plan | its class's budget (`mesh-delivery.times`), at most 30 minutes | read `mesh-controller.plans`, then `mesh-delivery.stalled` |
|
||||
|
||||
A wait past its bound is said to the operator at once, with what is being done about it.
|
||||
|
||||
## Models
|
||||
|
||||
The fixing agent runs on the smaller model; the reviewer on the larger. A session fixing alone does the
|
||||
reading itself and uses one subagent only for the review.
|
||||
|
||||
## How it is checked
|
||||
|
||||
- A small fix has one pull request with one or few commits naming its issue, opened after its review: a pull
|
||||
request with many heads is a fix reviewed after its push.
|
||||
- Its issue moves to `resolved` within the hour its delivery plan ends.
|
||||
- A pull request reviewed and green but unmarked
|
||||
for more than 15 minutes, a fix to a module's running marked ready with no integration test of the module whole, or a review report
|
||||
asking a change outside the issue's done-when, is the failure of steps 4 to 6.
|
||||
- An issue in `fixing` with nobody on it shows in `issue-tracker.list` with `flag: abandoned`; a stopped fix
|
||||
moves it back to `open` with its note.
|
||||
@@ -1,49 +0,0 @@
|
||||
# Playbook 09 — Cleanup
|
||||
|
||||
**Trigger.** The operator asks for a cleanup, or the forge and the working copies have gathered what nobody
|
||||
uses: merged branches, stale pull requests, worktrees and folders of finished work.
|
||||
|
||||
**Who runs it.** A session on the operator's word, with the operator deciding every deletion. Agents read and
|
||||
propose; a deletion is the operator's act or is done on the operator's word for the list it was shown.
|
||||
|
||||
## The aim
|
||||
|
||||
Remove what is provably done, and nothing else. A cleanup that deletes one piece of unfinished work costs more
|
||||
than every stale branch it removed. So every item is shown with its evidence, and anything not provably safe is
|
||||
reported and left.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Take stock, reading only.** One agent on the smaller model reads, changing nothing:
|
||||
- the forge: every repository, every branch **as git refs** (the branch list tool misses some, issue 949),
|
||||
every open pull request with its head, title and last update;
|
||||
- each working copy on the machine: its worktrees, local branches, stashes, and the folders beside the
|
||||
clones, with size and newest file.
|
||||
It writes one report with each item, its class and its evidence, and a list of the tools it lacked.
|
||||
2. **Classify each item**, by evidence, never by name or age alone:
|
||||
- **Safe** — a forge branch whose tip is contained in its repository's default branch, with no open pull
|
||||
request, not protected and not a long-lived branch the mesh treats as a trunk; a worktree that is clean,
|
||||
whose branch is contained in the remote default branch, untouched for two hours and not in another
|
||||
session's working folder; a local branch contained in the remote default branch.
|
||||
- **Likely safe, not proven** — a branch whose every commit is patch-equal to one on the default branch
|
||||
(the forge lands some commits under new hashes): listed, deleted only on the operator's word for it.
|
||||
- **Kept** — unmerged commits, a dirty worktree, a stash, an open pull request, a protected or trunk-like
|
||||
branch, a repository of another owner, another session's worktree.
|
||||
- **Unknown** — a folder or file outside git: described, never opened where a secret may be, left.
|
||||
Stale open pull requests (no update for three days) are reported with their author's issue, never closed.
|
||||
3. **Ask the operator** with the counts per class and the report's place. One question: delete the safe
|
||||
items as listed? The likely-safe and unknown items are separate questions, item by item when few.
|
||||
4. **Delete only the approved list**, each with the command that refuses an unsafe case (a worktree removal
|
||||
that refuses a dirty one, a branch deletion that refuses an unmerged one). A refusal is reported, never
|
||||
forced. The operator approves a deletion on a channel that proves who answered, as an ask bound to that
|
||||
exact list and carried out on the warrant (ADR 0387, design 58 milestone 7, issue 884). Until that is built,
|
||||
an agent session's permission check refuses these deletions, and the approved list is handed to the
|
||||
operator to run.
|
||||
5. **Report**: what was cleaned, what was kept and why, what was unknown, what the operator kept, and the
|
||||
missing tools, each opened as an issue (`issue-tracker.open`) or added to an existing one.
|
||||
|
||||
## How it is checked
|
||||
|
||||
- The report names, for every deleted item, the evidence it was safe (contained, clean, no pull request).
|
||||
- Nothing classed Kept or Unknown is deleted without the operator's word for that item.
|
||||
- A cleanup's missing tools are issues, not notes in a session.
|
||||
+9
-9
@@ -15,30 +15,30 @@ and a forge address is an operational detail (see [`README`](../README.md)).
|
||||
| Repository | Owns |
|
||||
|---|---|
|
||||
| `hal` | The monorepo — the node runtime, the module catalogue, the delivery machinery, and the bootstrap scripts. Every core module lives here. |
|
||||
| `hq` | This repository, under the company organisation — mission, research, design, decisions, issue diagnosis. Company-scoped (ADR 0019); the mesh is its first product. The source of truth for *why*. Carries no implementation. |
|
||||
| `hq` | This repository, under the company organisation — mission, research, design, decisions, issue diagnosis. Company-scoped ([ADR 0019](../02-DECISIONS/0019-how-this-repository-works.md)); the mesh is its first product. The source of truth for *why*. Carries no implementation. |
|
||||
| *(one per application)* | Every standalone application, site or side-project gets its own repository, with `module.yml` at the root. Registered with the mesh as a build source; built and deployed by the same pipeline as anything in the monorepo. |
|
||||
| `migration` | **Private.** The record of one installation replacing the predecessor mesh with this one: the runbook, a dated log of every step and what it cost, the per-service data procedures, the readiness checks, and the scripts. Private because it is the opposite of this repository in every way that matters — it names machines, addresses, ports and paths, because a procedure that cannot be followed is not one. Where hq asks *what did we decide and why*, that repository answers *what happened on the machines, in what order, and what to do next*. Its `HANDOFF.md` is where somebody picking the work up starts. |
|
||||
|
||||
## What the mesh becomes
|
||||
|
||||
ADR 0019 records the repositories the
|
||||
[ADR 0019](../02-DECISIONS/0019-how-this-repository-works.md) records the repositories the
|
||||
monorepo decomposes into. **`mesh-host`, `mesh-controller`, `mesh-catalog`, `mesh-lab`, `mesh-sdk` and `mesh-tools` exist
|
||||
so far** — the lab was built first (ADR 0016). The tiered
|
||||
so far** — the lab was built first ([ADR 0016](../02-DECISIONS/0016-the-lab.md)). The tiered
|
||||
decomposition below is the planned shape; the repositories built to date do not map onto it
|
||||
one-for-one — `mesh-catalog`, `mesh-sdk` and `mesh-tools` exist where the table names
|
||||
`mesh-foundation` and `mesh-surfaces`, and reconciling the two is itself still ahead.
|
||||
|
||||
| Repository | Tier | Holds |
|
||||
|---|---|---|
|
||||
| `mesh-host` | 0 | **exists.** The node-engine — one statically linked binary, requiring nothing present (ADR 0005) |
|
||||
| `mesh-host` | 0 | **exists.** The node host — one statically linked binary, requiring nothing present ([ADR 0005](../02-DECISIONS/0005-the-node-host.md)) |
|
||||
| `mesh-foundation` | 1 | the four pinned services, as declarations |
|
||||
| `mesh-controller` | 2 | **exists.** The controller and its contexts — one of seven built (ADR 0006) |
|
||||
| `mesh-controller` | 2 | **exists.** The controller and its contexts — one of seven built ([ADR 0006](../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)) |
|
||||
| `mesh-surfaces` | 3 | tools, web, cli |
|
||||
| `mesh-sdk` | — | the stable spine modules build against — the tool-serving SDK harness, the messaging/event framework, the contracts and core primitives. Holds nothing per-module and nothing volatile (ADR 0039). |
|
||||
| `mesh-sdk` | — | the stable spine modules build against — the tool-serving harness, the messaging/event framework, the contracts and core primitives. Holds nothing per-module and nothing volatile ([ADR 0039](../02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md)). |
|
||||
| `mesh-lab` | — | **exists.** The lab — scenario lifecycle, networking, placement. Ships to nobody; runs on a workstation. |
|
||||
|
||||
Tier 4's shape is open, and deliberately so: see ADR 0019 and
|
||||
research 005.
|
||||
[research 005](../01-RESEARCH/005-domain-grouping/00-overview.md).
|
||||
|
||||
## What lives where inside the monorepo
|
||||
|
||||
@@ -56,7 +56,7 @@ Named by role, because the layout is itself part of the as-is design — see
|
||||
## Why applications do not live in the monorepo
|
||||
|
||||
A standalone application in the monorepo is a convention violation, and reviewers reject it.
|
||||
The reasoning is recorded in ADR 0015:
|
||||
The reasoning is recorded in [`02-DECISIONS/0010`](../02-DECISIONS/0015-applications-live-in-their-own-repository.md):
|
||||
the mesh installs, provisions for, and ships an application through exactly the same machinery
|
||||
whether or not its source sits beside the mesh's own — so co-location buys nothing and costs
|
||||
the monorepo's review cadence.
|
||||
@@ -67,7 +67,7 @@ Each module is a standalone package that consumes its dependencies from the priv
|
||||
not from a sibling directory. The workspace was removed after it caused build-versus-development
|
||||
divergence — a workspace member importing another resolved to local unbuilt source in the
|
||||
pipeline and to a published version in development. Recorded in
|
||||
ADR 0014.
|
||||
[`02-DECISIONS/0007`](../02-DECISIONS/0014-no-npm-workspace.md).
|
||||
|
||||
Consequence, and it is a real one: a cross-package change is two steps — publish, then consume
|
||||
— and a repository-wide `npm install` does not exist.
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
---
|
||||
status: active
|
||||
initiated: 2026-08-22
|
||||
touches: [03-DESIGN/00-as-is/02-modules-and-manifests.md, 03-DESIGN/00-as-is/10-module-catalogue.md, 03-DESIGN/01-to-be/00-work-breakdown.md]
|
||||
became: [02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md]
|
||||
---
|
||||
|
||||
# 001 — Module domain decomposition
|
||||
|
||||
- **Initiated by:** jochen, 2026-08-22
|
||||
- **Areas touched:** every `hal/*` and `noxflow/*` module; the pipeline's dependency
|
||||
graph; the knowledge base; agent identity and credentials.
|
||||
|
||||
## Summary
|
||||
|
||||
HAL has 124 modules. That number is not a maintenance problem in itself — it is the
|
||||
**symptom of missing bounded contexts**. Modules are split not because they model
|
||||
different domains, but because splitting is the only lever the platform offers:
|
||||
|
||||
- no way to run one daemon on one node without making it a module
|
||||
(`hal/claude-licences` — one daemon, single-node)
|
||||
- no way to expose two of a kind from one module
|
||||
- no namespace separating the mesh from the software it runs
|
||||
|
||||
This effort establishes the **current state**, the **ideal state**, and the sequence
|
||||
between them.
|
||||
|
||||
## Trigger
|
||||
|
||||
A night of debugging that produced four fixes and one conclusion. Every fault was a
|
||||
boundary fault:
|
||||
|
||||
- Per-agent Claude credentials had to be written by the *noxflow runtime*, because
|
||||
`agents.claude_account` is in noxflow's database — even though agent identity is a
|
||||
mesh concept and node identity already lives in the mesh registry.
|
||||
- Whether `hal/brain` may depend on noxflow took three attempts to answer, twice
|
||||
wrongly, because the ownership boundary was never stated.
|
||||
- Authoritative documentation existed in `mesh_docs` and was not found, while a
|
||||
proposal in repo markdown was invisible to search entirely.
|
||||
|
||||
## Decisions taken (2026-08-22)
|
||||
|
||||
| Question | Decision |
|
||||
|---|---|
|
||||
| What should noxflow become? | Decompose into `hal/*` modules; noxflow returns to tasks/workflows |
|
||||
| Who owns agent identity? | `hal/agents` — a mesh concept, alongside nodes |
|
||||
| Where do third-party apps live? | Out of this repo. They run *on* the mesh; they are not *of* it |
|
||||
| Knowledge structure | Modelled on `papa-hq`; implementation choice left open |
|
||||
|
||||
## Open questions
|
||||
|
||||
Tracked in [`analysis.md`](analysis.md) under "Open questions".
|
||||
|
||||
## Deliberately not decided
|
||||
|
||||
Recorded so they are not mistaken for oversights. Each is open, and each comes out of
|
||||
[ADR 0001](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md); this effort stays
|
||||
`active` until they are answered.
|
||||
|
||||
| Question | Status |
|
||||
|---|---|
|
||||
| `hal/scheduler` — infrastructure, or part of the work context. | Open. |
|
||||
| Which context owns the executor. | Open. |
|
||||
| Catalogue destination — one repository or many. | Open. Phase 4. |
|
||||
| What the shared library keeps after extraction. | Open. Phase 3. |
|
||||
| Where human agent modality is recorded — which user, on which node, a human agent acts as. | Open. Required by the model; not yet stored. |
|
||||
| Which domains the modules outside the platform core group into. | Open, from [ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md), which settles the principle and deliberately not the list. |
|
||||
@@ -0,0 +1,242 @@
|
||||
---
|
||||
effort: 001-module-domain-decomposition
|
||||
updated: 2026-08-22
|
||||
---
|
||||
|
||||
# Current state → ideal state
|
||||
|
||||
## 1. The count is a symptom
|
||||
|
||||
124 modules. The number is not the problem; the reason for it is. Modules are split
|
||||
because splitting is the **only granularity lever the platform offers**:
|
||||
|
||||
| symptom | module | why it exists |
|
||||
|---|---|---|
|
||||
| one daemon must run on one node | `hal/claude-licences` | no per-node feature opt-in |
|
||||
| a library must not drag a 450 MB dep | `hal/claude` | no way to expose two npm packages |
|
||||
| a domain needs a daemon *and* a library | `hal/claude-code` + `hal/claude` | one feature of each type per module |
|
||||
| a handful of SDK verbs need a home | `infra` (10 lines) | tools must belong to *some* module |
|
||||
|
||||
A feature is a **handler type**, discovered by directory presence, singleton per module
|
||||
(`FEATURE_HANDLERS` in `modules/hal/sdk/src/feature-handlers/index.ts`). So "one more
|
||||
deployable unit" always means "one more module".
|
||||
|
||||
Splitting also fragments domains. Claude licence logic sits in three modules —
|
||||
resolution in `hal/claude`, health in `hal/claude-code`, refresh in
|
||||
`hal/claude-licences` — so following one token means reading three.
|
||||
|
||||
## 2. Two things are conflated in one repo
|
||||
|
||||
`noxflow` is 45 tables. Its original purpose — a HAL-native Jira — is 7 of them:
|
||||
|
||||
| domain | tables |
|
||||
|---|---:|
|
||||
| tasks (the original concept) | 7 |
|
||||
| agents / HR | 13 |
|
||||
| knowledge (a Confluence) | 8 |
|
||||
| meetings | 7 |
|
||||
| scheduling / ops | 2 |
|
||||
| other | 8 |
|
||||
|
||||
Agents alone are nearly double the concept the module was built for. This is the direct
|
||||
cause of concrete faults: per-agent Claude credentials had to be written by the noxflow
|
||||
runtime purely because `agents.claude_account` lives in noxflow's database, while node
|
||||
identity — the same kind of fact — lives in the mesh registry.
|
||||
|
||||
Separately, 91 of the 124 modules are third-party software (`plex`, `sonarr`,
|
||||
`firefox`, `postgres`). These **run on the mesh; they are not part of it**. They belong
|
||||
outside this repository. The pipeline and provisioning system are deliberately generic,
|
||||
so HAL's own modules dogfood exactly what external modules use — that property is what
|
||||
makes the separation safe.
|
||||
|
||||
## 3. Knowledge is split three ways
|
||||
|
||||
| store | owner | size | search |
|
||||
|---|---|---:|---|
|
||||
| `mesh_docs` (+ revisions) | `hal/hippocampus`, in the mesh DB | 144 docs, 73 touched in 30d | yes |
|
||||
| `knowledge_*` (8 tables) | `noxflow/runtime`, in the noxflow DB | 20 pages | yes |
|
||||
| repo markdown | git | `VISION.md`, `FLAVOR-PROPOSAL.md`, `docs/` | **no** |
|
||||
|
||||
The librarian is split across two of them: `librarian_file_document` writes via
|
||||
hippocampus, `librarian_ask` and `librarian_retrieve` read via noxflow. Filing and
|
||||
retrieval, different modules, different databases.
|
||||
|
||||
Both DB stores have revisions; only noxflow has a review workflow (`_proposals`,
|
||||
`_promotion_requests`). Neither has file access. Git has file access and history but no
|
||||
search integration. **Every piece needed already exists — none of them together.**
|
||||
|
||||
## 4. `hal/developer` and the missing local mesh
|
||||
|
||||
`hal/developer` (≈2,600 lines) exposes `dev_stage`, `dev_diff`, `dev_validate`,
|
||||
`dev_typecheck`, `dev_verify_flavor`, `dev_bootstrap`, `dev_build`, `dev_up`, `dev_down`,
|
||||
`dev_env`. It is the tooling that both reflects and enforces the module model, so it
|
||||
moves with any change to that model.
|
||||
|
||||
`dev_up` builds a dev environment **for a single module**: resolves `requires:`, starts
|
||||
providers "via systemctl (same as production)", provisions workspace-namespaced
|
||||
databases. It borrows the host node's services.
|
||||
|
||||
**There is no way to run a mesh locally.** Consequently anything that only manifests
|
||||
*between* nodes — credential rotation reaching a running session, provision credentials
|
||||
fanning out, cascade ordering, artifact packaging — is discoverable only in production.
|
||||
Every fault fixed on 2026-08-22 was of that kind.
|
||||
|
||||
A containerised mesh (N nodes, a broker, a registry, a coordinator) is therefore not a
|
||||
convenience. It is the precondition for a refactor of this size being verifiable at all.
|
||||
|
||||
## 5. `hal/sdk` is where domains go to hide
|
||||
|
||||
155 files, 34,636 lines. It contains code from essentially every bounded context:
|
||||
|
||||
| file | domain it belongs to |
|
||||
|---|---|
|
||||
| `installer-core.ts` (1,495) | runtime |
|
||||
| `tools/provisions.ts` (1,311) | provisioning |
|
||||
| `tools/deploy-service.ts` (1,081) | deploy |
|
||||
| `workflow-engine.ts` (970), `task-commands.ts` (588) | **work** — noxflow's domain, inside HAL's SDK |
|
||||
| `artifact-manager.ts` (948), `build-executor.ts` (760), `feature-handlers/*` | delivery |
|
||||
| `env-generator.ts` (944) | config |
|
||||
| `amqp-client.ts` (899) | comms |
|
||||
| `meshware.ts` (799) | runtime |
|
||||
| `module-registry.ts` (743) | mesh |
|
||||
| `claude-credentials.ts` | **ai** |
|
||||
|
||||
Nothing here was misplaced carelessly. Each landed in the SDK because the SDK is the one
|
||||
package every module may import — so putting a shared type there is the way to avoid a
|
||||
circular dependency between two modules that both need it.
|
||||
|
||||
The result is that the dependency graph is trivially acyclic and completely
|
||||
uninformative: everything depends on one 34k-line package, so a change anywhere in it
|
||||
rebuilds everything. On 2026-08-22 a three-line fix in `feature-handlers/migrations.ts`
|
||||
produced a 1,445-job, 26-minute mesh-wide cascade.
|
||||
|
||||
The last row is from that same day, and is the clearest illustration: per-agent Claude
|
||||
config-directory helpers were added to `@hal/sdk/claude-credentials.ts` because it was
|
||||
the only place both `hal/claude` and the noxflow runtime could import from. The
|
||||
alternative — `hal/ai` owning it and noxflow depending on `hal/ai` — was unavailable
|
||||
without answering the ownership question this effort exists to answer.
|
||||
|
||||
**Extraction is therefore blocked on the decomposition, not the reverse.** A domain
|
||||
module can only own its shared code once the domain has a module. The SDK should retain
|
||||
only what is genuinely cross-cutting: transport, manifest types, logging, process
|
||||
helpers.
|
||||
|
||||
## 6. The crux: two notions of "agent"
|
||||
|
||||
The original vision was **a node is an agent with thinking abilities**. Later, noxflow
|
||||
implemented actual agents — employees with skills, workspaces and tasks. Both survive,
|
||||
and they collide.
|
||||
|
||||
The collision is visible in the data. There are **two agent rows per node**:
|
||||
|
||||
| agent | display name | skills | bound to |
|
||||
|---|---|---|---|
|
||||
| one named after each node | the node's name | `{deploy,verify,operate}` | its node |
|
||||
| `hal-<node>`, one per node | "HAL (\<node\>)" | `{}` | its node |
|
||||
|
||||
One carries the **work**; the other carries only **identity** — no skills at all, existing
|
||||
to hold a licence for HAL's own sessions (`resolveHalModuleEnv`). The node-as-agent idea
|
||||
was implemented twice, half each.
|
||||
|
||||
The same split appears in the schema: `nodes.hal_claude_account` and
|
||||
`agents.claude_account` are the same fact in two tables in two databases. What was
|
||||
documented as "three licence touchpoints" (NODE, HAL, AGENT) is **one concept modelled
|
||||
three times**.
|
||||
|
||||
### Resolution
|
||||
|
||||
**Nodes and agents are decoupled.** The agents module owns agents; agents run on nodes.
|
||||
A node is a place where an agent can run — that is the whole of the relationship. There
|
||||
is no resident agent, no node-owned identity, no ownership in either direction.
|
||||
|
||||
Agents named after a node still exist, but they are **ordinary agents defined through the
|
||||
agents module** that happen to hold infra skills and be named after the node they usually run
|
||||
on. Nothing about them is structural.
|
||||
|
||||
Two consequences follow:
|
||||
|
||||
1. **`nodes.node_license` and `nodes.hal_claude_account` should not exist.** A node does
|
||||
not authenticate to a model provider — agents do. Both collapse into
|
||||
`agents.claude_account`, and what was documented as three licence touchpoints becomes
|
||||
one.
|
||||
2. **The pair of agent rows per node merges.** `hal-<node>` exists only to hold a licence
|
||||
for HAL's own sessions; once licences belong to agents and any module may employ an
|
||||
agent, it has no reason to be separate from `<node>`.
|
||||
|
||||
What remains unaccounted for is the operator's own interactive session:
|
||||
`~/.claude/.credentials.json` on a node serves a **human at a prompt**, not an agent.
|
||||
Under this decoupling that is not a node property either — it is the operator's
|
||||
credential on whichever machine they are sitting at. Whether the operator is modelled as
|
||||
a persona (soulstream treats humans and agents as peers with identical credentials) is an
|
||||
open question, and the last remaining reason `nodes` carries a licence column today.
|
||||
|
||||
## 7. The record, not "meetings"
|
||||
|
||||
soulstream models work as **one signed log** in which humans and agents apply identical
|
||||
changes, with work arriving by **mention**. Memory and decisions live in that single
|
||||
auditable stream — no hidden state between components. Its stated economics,
|
||||
*"cheap awareness and expensive thinking"*, is the same principle as the budget guard:
|
||||
constant monitoring, selective deep processing.
|
||||
|
||||
Under that model "meetings" is not a domain. A meeting is a **thread in the record**.
|
||||
Which means six of today's modules and table groups are one thing seen from different
|
||||
angles:
|
||||
|
||||
`hal/axon` (DMs, rooms) · `hal/cortex` · `hal/synapse` · `hal/notifications` ·
|
||||
`conversations` · `meetings`
|
||||
|
||||
A notification, in particular, is just a mention not yet read — which is why it needed
|
||||
its own table, its own id column, and its own failure mode.
|
||||
|
||||
## 8. Ideal state
|
||||
|
||||
The mesh repository contains only the mesh, decomposed by domain rather than by
|
||||
deployment accident:
|
||||
|
||||
| context | what it is | absorbs today's |
|
||||
|---|---|---|
|
||||
| `hal/infra` | **the room** — nodes, registry, provisioning, pipeline, node runtime | `hal/mesh`, `hal/brain`, `hal/coordinator`, `hal/meshware`, `hal/developer`, `hal/bootstrap` |
|
||||
| `hal/agents` | **the name and the thinking** — Agent owns identity, licence, runs, memory, thoughts | noxflow agents (13 tables), `hal/thoughts` |
|
||||
| `hal/stream` | **the record** — mentions, threads, meetings, notifications | `hal/axon`, `hal/cortex`, `hal/synapse`, `hal/notifications`, meetings (7), conversations |
|
||||
| `hal/knowledge` | documents, spaces, revisions, review, search | `hal/hippocampus` + noxflow `knowledge_*` |
|
||||
| `hal/work` | tasks, workflows, boards — the original noxflow | noxflow tasks (7 tables) |
|
||||
| `hal/ai` | provider integration, flavored per vendor | `hal/claude*`, `claude-code` |
|
||||
| `hal/config` | env and config distribution, secrets, PKI | `hal/env-sync`, `hal/config-sync`, `hal/secrets`, `mesh-ca` |
|
||||
| `hal/observability` | health, dashboards, logs | `hal/health`, `hal/meshboard` |
|
||||
|
||||
Eight contexts instead of 33 platform modules, with the catalogue elsewhere.
|
||||
|
||||
The naming corrects itself in the process: what is called "brain" today is
|
||||
infrastructure, and the word survives only as a concept an Agent owns — never again as a
|
||||
module name.
|
||||
|
||||
This shape depends on **named features with per-node opt-in** — without it, every
|
||||
independently-deployable unit inside a context becomes a module again and the count
|
||||
returns. That mechanism is the subject of a separate ADR.
|
||||
|
||||
## Decided since
|
||||
|
||||
| question | decision |
|
||||
|---|---|
|
||||
| Meetings — own context or part of agents? | **Neither.** A meeting is a thread in the record; `hal/stream` absorbs it. The category was wrong. |
|
||||
| Thoughts — autonomy or agent behaviour? | `hal/agents`. Any agent can have a thought. |
|
||||
| `hal/axon` vs `hal/cortex` | `hal/cortex` is vestigial — no `daemon/` source, unit inactive on every node, description copy-pasted from axon. Messaging belongs to `hal/stream`. |
|
||||
| `hal/hypothalamus` | Orphaned build output. A `dist/` directory with no `module.yml`, so not a module at all. |
|
||||
| Who owns agent identity? | `hal/agents`. Nodes and agents are decoupled; a node is only a place an agent can run. |
|
||||
| Where do third-party modules live? | Outside this repository. They run *on* the mesh, not *of* it. |
|
||||
|
||||
## Open questions
|
||||
|
||||
1. **`hal/scheduler`** — infrastructure (fire an event later, belongs to `hal/mesh`) or
|
||||
part of `hal/work`?
|
||||
2. **The executor.** Pulling and routing work items is `hal/work`; spawning a session is
|
||||
`hal/agents`; providing the place to run is `hal/mesh`. It currently straddles all
|
||||
three, which is the same kind of straddle that put credential-writing in the noxflow
|
||||
runtime.
|
||||
3. **Catalogue destination** — one repository, or per-application repositories, given the
|
||||
pipeline resolves dependencies across the registry rather than the filesystem?
|
||||
4. **SDK residue** — after extraction, does `hal/sdk` keep transport (`amqp-client`), or
|
||||
does that belong to `hal/stream`? Everything imports it, which argues both ways.
|
||||
5. **Human agent modality.** ADR 0001 requires a fact the mesh does not record: which
|
||||
user, on which node, a human agent acts as. Where does it live — an attribute of the
|
||||
agent, or of the agent-node binding?
|
||||
@@ -0,0 +1,59 @@
|
||||
---
|
||||
status: graduated
|
||||
initiated: 2026-08-22
|
||||
touches: [03-DESIGN/00-as-is/04-delivery.md, 03-DESIGN/00-as-is/05-runtime-and-installation.md]
|
||||
became: [03-DESIGN/01-to-be/01-end-to-end-testing.md, 02-DECISIONS/0016-the-lab.md]
|
||||
---
|
||||
|
||||
# 002 — A mesh that runs locally
|
||||
|
||||
- **Initiated by:** jochen, 2026-08-22
|
||||
- **Areas touched:** `install.d/`, `hal/meshware`, `hal/coordinator`, `hal/brain`,
|
||||
`hal/developer` (`dev_up`), `hal/sdk` (env generation, feature handlers, artifact
|
||||
manager), `test/pipeline/`, `test/dev-mesh/`, the provisioning path in
|
||||
`modules/postgres/`.
|
||||
|
||||
## Summary
|
||||
|
||||
Phase 0 of [`03-DESIGN/00-work-breakdown.md`](../../03-DESIGN/01-to-be/00-work-breakdown.md) requires
|
||||
a mesh that comes up in containers, runs its own pipeline, and reproduces known faults on
|
||||
demand. Nothing else in the decomposition starts until it exists, because every fault the
|
||||
decomposition addresses was found in production — there was nowhere else to find it.
|
||||
|
||||
This effort establishes what already runs in a container, what is welded to the host, and
|
||||
what it would take to close the gap. It does **not** choose an approach: the central
|
||||
question — how a containerised node executes a module service, when a module service is
|
||||
defined today as a systemd unit shelling to `docker compose` in `/services/` — is not
|
||||
answered by ADR 0001 and is recorded below rather than decided.
|
||||
|
||||
## What was established
|
||||
|
||||
- The two existing container harnesses are **neither of them a mesh**, and one of them has
|
||||
not been able to build since 2026-06-04.
|
||||
- Node identity is already portable — a single environment variable, no host handshake.
|
||||
- Four concrete host couplings block a containerised node, all with known locations.
|
||||
- All three Phase 0 fixtures are reproducible; one of them is documented in the knowledge
|
||||
base with an open root cause and is the cheapest place to start.
|
||||
|
||||
Detail and evidence in [`analysis.md`](analysis.md).
|
||||
|
||||
## Questions — all settled 2026-08-22
|
||||
|
||||
**Phase 0 is a development environment**, not a fixture rig — it is what the host-borrowing
|
||||
dev tooling becomes.
|
||||
|
||||
**The trigger is a real source-forge container**, because the webhook relay is part of what
|
||||
is under test.
|
||||
|
||||
**A node is a system container**, promotable to a virtual machine per node. This answered
|
||||
the question the effort was stuck on, and dissolved it rather than solving it: against a
|
||||
real node with a real init, the four host couplings catalogued in `analysis.md` §3 are not
|
||||
couplings — they are how a node works. They were obstacles only to a node modelled as an
|
||||
application container.
|
||||
|
||||
Consequently [`003-service-supervision`](../003-service-supervision/) **no longer blocks
|
||||
Phase 0**. It remains a live architecture question, on its own timeline.
|
||||
|
||||
The remaining questions in `analysis.md` — what replaces host paths in a container, and how
|
||||
faithful the lab must be — are answered in the design: nothing replaces them, because the
|
||||
paths are real; and the divergences are enumerated rather than discovered.
|
||||
@@ -0,0 +1,263 @@
|
||||
---
|
||||
effort: 002-local-mesh
|
||||
updated: 2026-08-22
|
||||
---
|
||||
|
||||
# A mesh that runs locally — current state and obstacles
|
||||
|
||||
Evidence for Phase 0. Every claim here is either a file location or something measured on
|
||||
2026-08-22; where a claim was checked and found false, that is recorded too.
|
||||
|
||||
---
|
||||
|
||||
## 1. What exists today, and why neither is a mesh
|
||||
|
||||
### `test/pipeline/` — the only containerised HAL node, and it is dead
|
||||
|
||||
This harness builds `@hal/sdk` and the `hal/brain` daemon into a `node:23-alpine` image,
|
||||
compiles `hal/coordinator`'s tools against it, and runs `brainstem.js` with
|
||||
`HAL_MODE=daemon` against a containerised postgres and LavinMQ
|
||||
(`test/pipeline/Dockerfile`, `test/pipeline/docker-compose.yml`). It proves the valuable
|
||||
thing: **a node runtime needs no systemd, no `/services/`, and no nvm** — environment
|
||||
variables and a seeded `nodes` row are sufficient.
|
||||
|
||||
It also cannot build. Measured 2026-08-22:
|
||||
|
||||
```
|
||||
Step 7/16 : RUN npm run build -w modules/hal/sdk
|
||||
npm error No workspaces found:
|
||||
npm error --workspace=modules/hal/sdk
|
||||
```
|
||||
|
||||
The npm workspace it depends on was removed on 2026-06-04 (`21ef4a4e`, "kill npm
|
||||
workspace"), and the root `package.json` now carries a comment explaining why it will not
|
||||
come back. The harness has not been touched since before that commit.
|
||||
|
||||
So the repository's only end-to-end pipeline test has been unrunnable for two and a half
|
||||
months and nothing reported it — which is the same shape as the faults Phase 0 exists to
|
||||
catch. **A test nobody runs is indistinguishable from a test that passes.**
|
||||
|
||||
### `test/dev-mesh/` — a work plane, not a control plane
|
||||
|
||||
Four `runtime-*` services, a noxflow API, a meshboard and a shared postgres + LavinMQ on
|
||||
one network (`test/dev-mesh/docker-compose.yml`). It is genuinely useful and the topology
|
||||
is a good precedent: one network, per-service `HAL_NODE`, `EXECUTOR_STUB=1` so dispatch and
|
||||
run lifecycle are real while the model call is faked.
|
||||
|
||||
It is the wrong layer for Phase 0, on three counts:
|
||||
|
||||
| | dev-mesh does | Phase 0 needs |
|
||||
|---|---|---|
|
||||
| state | restores `pg_dump`s (`test/dev-mesh/init.sh`) | schema built by **migrations**, or fixture 3 is untestable by construction |
|
||||
| code | bind-mounts pre-built `dist/` from the host | artifacts **built and delivered** by the pipeline |
|
||||
| scope | noxflow runtime, API, meshboard | `hal/coordinator`, `hal/meshware`, MinIO, provisioning |
|
||||
|
||||
No meshware, no coordinator, no artifact store, no provisioning, no build. It exercises
|
||||
what the mesh *runs*; Phase 0 needs what the mesh *is*.
|
||||
|
||||
### `dev_up` — confirmed host-coupled
|
||||
|
||||
The claim in the work breakdown holds. `startDevProvider()` starts providers through the
|
||||
host's user systemd instance and reads credentials from a host path
|
||||
(`modules/hal/developer/tools/dev-env.ts:146-178`):
|
||||
|
||||
```ts
|
||||
const unitName = `hal-module@${provider}.service`;
|
||||
execSync(`systemctl --user start ${unitName}`, { timeout: 60_000, stdio: "pipe" });
|
||||
const creds = resolveRunningProviderCreds(provider); // reads /services/<provider>/.env
|
||||
```
|
||||
|
||||
It borrows the host. There is no mesh to stand up.
|
||||
|
||||
---
|
||||
|
||||
## 2. What is already portable
|
||||
|
||||
- **Node identity is one environment variable.** `resolveNodeName()` is
|
||||
`process.env.HAL_NODE || hostname()`, lowercased (`modules/hal/sdk/src/mesh-config.ts:215`).
|
||||
No file, no registration handshake, no host coupling.
|
||||
- **Registration is a plain upsert.** `mesh_node_register` inserts into `nodes`
|
||||
(`modules/hal/mesh/tools/index.ts:490-540`); mandatory fields are `name` and `user_name`.
|
||||
- **The daemons are AMQP loops.** `hal/meshware` consumes `cmd.feature.{build,install,configure,start}`
|
||||
(`modules/hal/meshware/daemon/src/cerebellum.ts:780-857`); `hal/coordinator` consumes
|
||||
`event.gitea.push` and drives the cascade. Both are configured entirely by environment.
|
||||
- **`hal/brain` in daemon mode already runs in a plain node image**, per `test/pipeline/`.
|
||||
|
||||
---
|
||||
|
||||
## 3. The obstacles, located
|
||||
|
||||
Four couplings stand between the daemons and a container. All are in code, none are
|
||||
mysterious.
|
||||
|
||||
| # | coupling | location |
|
||||
|---|---|---|
|
||||
| 1 | meshware restarts itself via the host: `execFileSync("systemctl", ["--user","restart","hal-meshware.service"])` | `modules/hal/meshware/daemon/src/cerebellum.ts:826` |
|
||||
| 2 | meshware restarts the broker via a hardcoded host path: `execFileSync("docker",["compose","-f","/services/lavinmq/docker-compose.yml","restart"])` | `cerebellum.ts:843-844` |
|
||||
| 3 | **a module service *is* a systemd unit** — `hal-module@.service` runs `docker compose --project-directory /services/%i` | `modules/hal/meshware/systemd/hal-module@.service:10-11` |
|
||||
| 4 | env generation writes `homedir()`-relative host paths — `~/.config/hal/env`, `~/.config/hal/modules/*.env`, `/services/*/.env` | `modules/hal/sdk/src/env-generator.ts:188-223`, `:538-568`, `:584-606` |
|
||||
|
||||
1, 2 and 4 are small: a guard and a configurable root. **3 is the architectural one**, and
|
||||
it is the first open question below — in a container, "systemd unit that runs docker
|
||||
compose" has no natural translation.
|
||||
|
||||
The bootstrap scripts add their own host assumptions — nvm for node resolution, `sudo
|
||||
mkdir -p /services && chown`, and a `~/dotfiles` repository that is not in this repo
|
||||
(`install.d/adopt.sh:165-170`) — but Phase 0 does not have to run them. A container image
|
||||
can be built the way `test/pipeline/Dockerfile` builds one, bypassing the bootstrap path
|
||||
entirely. That is a deliberate divergence to record: **the local mesh would not test the
|
||||
bootstrap**, only the running mesh.
|
||||
|
||||
---
|
||||
|
||||
## 4. External services, and whether they can be local
|
||||
|
||||
| service | used for | local substitute |
|
||||
|---|---|---|
|
||||
| PostgreSQL | registry + pipeline state | yes — already the pattern in both harnesses |
|
||||
| LavinMQ | all coordinator↔meshware messaging | yes — `cloudamqp/lavinmq`, already used |
|
||||
| MinIO | per-feature artifact tarballs, `modules/{name}/{version}/{feature}.tar.gz` (`modules/hal/sdk/src/artifact-manager.ts:21,65`) | yes — `minio/minio`, needs only `REGISTRY_MINIO_*` |
|
||||
| Gitea | source, push webhook, **and** the `@hal/*` npm registry | container exists, but heavy; see open question 2 |
|
||||
| Docker registry | `docker push` for modules declaring `docker:` (`build-executor.ts:195-231`) | `registry:2`, or avoid modules that need it |
|
||||
| Traefik | writes routing config at install; does not gate success | omit |
|
||||
|
||||
Only Gitea is awkward, and only because it carries two roles at once.
|
||||
|
||||
---
|
||||
|
||||
## 5. The fixtures
|
||||
|
||||
Phase 0.5 asks for at least one reproducible known fault. All three are reachable; they
|
||||
differ sharply in cost.
|
||||
|
||||
### Fixture B — a provider deploy rotates a shared credential without fanning out
|
||||
|
||||
**Cheapest, best understood, and the root cause is still open.** Documented at
|
||||
`troubleshooting/provision-adoption-rotates-live-credential`. The mechanism is two lines:
|
||||
|
||||
```ts
|
||||
async provision(project, username, options) {
|
||||
const password = generatePassword(); // ALWAYS a fresh password
|
||||
```
|
||||
|
||||
and, in the adoption branch of `provisionDatabase`, an `ALTER ROLE ... WITH PASSWORD` that
|
||||
rotates the live secret while updating only the provider's own `mesh_provisions` row
|
||||
(`modules/postgres/tools/index.ts`). Every consumer sharing that role keeps a stale
|
||||
password and fails permanently; the provider recovers alone, and that asymmetry is the
|
||||
tell.
|
||||
|
||||
Reproduction needs one provider node and two consumer nodes — the minimum interesting
|
||||
mesh. It re-fires on every provider deploy, so it does not need to be provoked, only
|
||||
observed. Confirmation is a timestamp comparison, not a hash: the stored values are
|
||||
`enc:v1:` with a random IV, so identical plaintexts hash differently.
|
||||
|
||||
Measured in production 2026-08-22: three nodes failing since 2026-08-20 14:15, 399 errors
|
||||
each; the provider failed for 22 minutes and recovered by itself.
|
||||
|
||||
### Fixture C — a migration ships nothing while the pipeline reports success
|
||||
|
||||
Three independent silent-skip points, any one of which produces it:
|
||||
|
||||
1. **Not-applicable and succeeded are the same status.** `MigrationsHandler.detect()` is
|
||||
`existsSync(join(moduleDir, "migrations"))` against the freshly cloned workspace
|
||||
(`modules/hal/sdk/src/feature-handlers/migrations.ts:32-41`). If the directory is not
|
||||
there — uncommitted, ignored, or a `module_path` that does not line up — the build
|
||||
reports `success` (`cerebellum.ts:634-638`).
|
||||
2. **Upload failure is a warning.** `uploadFeatureArtifact()` returns `{sha256: ""}` with a
|
||||
`warn` when none of the listed files exist (`artifact-manager.ts:41-44`), and callers
|
||||
catch and warn rather than throw (`cerebellum.ts:672-673`). The symmetric download
|
||||
failure on the target node is also non-fatal (`artifact-manager.ts:93-98`,
|
||||
`cerebellum.ts:426-431`, commented "Non-fatal: feature may work without artifact").
|
||||
3. **Missing packaged files are skipped in silence.** `addFileEntries()` does
|
||||
`if (!existsSync(abs)) continue;` (`module-builder-core.ts:134-135`) — an explicit
|
||||
`package:` entry that is not on disk simply is not in the tarball.
|
||||
|
||||
This is the same class as `troubleshooting/flavors-never-packaged` (`flavors/` was never
|
||||
staged, so a node kept its first copy forever) and is catalogued in
|
||||
`troubleshooting/deploy-reports-transport-not-effect`, which counted **0 of 119 modules
|
||||
implementing `verify`** — the stage that exists, is dispatched, has a working handler, and
|
||||
would turn every one of these into a red pipeline.
|
||||
|
||||
### Fixture A — a credential rotation does not reach a running session
|
||||
|
||||
The most valuable and the most work: it needs a *running session* to rotate underneath,
|
||||
which means the local mesh must be able to start one. The mechanism is understood — an
|
||||
`EnvironmentFile` is read once and `process.env` is a snapshot — and it is why credentials
|
||||
became files that get re-read. Defer it behind B and C.
|
||||
|
||||
**Recommendation:** B first. It needs three nodes and a provision, no build and no session,
|
||||
and it is the one whose root cause is still open — so reproducing it locally has value
|
||||
beyond proving the harness.
|
||||
|
||||
---
|
||||
|
||||
## 6. A claim that was checked and found false
|
||||
|
||||
The survey behind this document suspected that the live AMQP pipeline never writes
|
||||
`deployments` or `node_modules.installed_version`, because the code that does so sits in
|
||||
`installer-core.ts:1045-1126` on what is commented as the "CLI path".
|
||||
|
||||
Measured against production, 2026-08-22 17:30 UTC — deployments in the preceding 24 hours:
|
||||
|
||||
| node | deploys | newest |
|
||||
|---|---|---|
|
||||
| 1 | 24 | 17:26:17 |
|
||||
| 2 | 27 | 17:22:22 |
|
||||
| 3 | 47 | 17:25:54 |
|
||||
| 4 | 26 | 17:22:23 |
|
||||
|
||||
The record is live on all four nodes and minutes old. **The claim is refuted.** Recorded
|
||||
because the reasoning was plausible and someone will retrace it.
|
||||
|
||||
---
|
||||
|
||||
## 7. Questions
|
||||
|
||||
### Settled (jochen, 2026-08-22)
|
||||
|
||||
**Phase 0 is a development environment, not a fixture rig.** It is built as a supported
|
||||
surface to work in daily, and it is what `dev_up` should become. The definition of done —
|
||||
"demonstrated in the local mesh" — is therefore meant literally.
|
||||
|
||||
**The trigger is a real Gitea container.** The webhook relay is part of what is under test,
|
||||
including the changed-file enrichment that exists because the webhook truncates at 20
|
||||
commits (`modules/hal/gitea/tools/index.ts:136-195`). A synthetic `event.gitea.push` would
|
||||
skip it. Gitea also carries the `@hal/*` npm registry, so the local mesh needs it twice
|
||||
over either way.
|
||||
|
||||
### Open — blocking
|
||||
|
||||
1. **How does a node supervise a module service?** Today a module service is a systemd unit
|
||||
running `docker compose` against `/services/<module>`
|
||||
(`modules/hal/meshware/systemd/hal-module@.service:10-11`), which has no direct
|
||||
translation inside a container. The question raised in response is the better one, and
|
||||
is broader than Phase 0: **what would it cost to stop using systemd altogether and have
|
||||
the mesh supervise its own services?** That is a mesh-level architectural question, not
|
||||
a local-mesh implementation detail — if the answer is that the mesh should own
|
||||
supervision, Phase 0 should not build a container-only workaround first. Under
|
||||
investigation; findings will land in [`003-service-supervision`](../003-service-supervision/)
|
||||
and, if it goes ahead, a decision record of its own.
|
||||
|
||||
2. **What replaces `~/.config/hal/env` and `/services/` inside a container?** A configurable
|
||||
root keeps one code path; container-specific targets keep the host paths untouched. This
|
||||
decides whether env generation gets a seam or a conditional.
|
||||
|
||||
3. **How faithful must the local mesh be to be trusted?** It will not run the bootstrap
|
||||
scripts, and it will run one OS where the real mesh is heterogeneous by design
|
||||
(`00-META/context.md`). Stating the divergence up front is what stops "it works
|
||||
locally" from becoming its own class of silent failure. Now sharper, because a
|
||||
development environment people use daily is trusted far more than a rig — and drifting
|
||||
from production costs correspondingly more.
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- [`02-DECISIONS/0001`](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md) — the decision this
|
||||
phase unblocks
|
||||
- [`03-DESIGN/00-work-breakdown.md`](../../03-DESIGN/01-to-be/00-work-breakdown.md) — Phase 0 tasks
|
||||
and checkpoint
|
||||
- `troubleshooting/provision-adoption-rotates-live-credential` — fixture B, root cause open
|
||||
- `troubleshooting/deploy-reports-transport-not-effect` — the nine defects that shipped
|
||||
green, and the unused `verify` stage
|
||||
- `troubleshooting/flavors-never-packaged` — fixture C, previously seen in the wild
|
||||
@@ -0,0 +1,79 @@
|
||||
---
|
||||
status: graduated
|
||||
initiated: 2026-08-22
|
||||
touches: [03-DESIGN/00-as-is/05-runtime-and-installation.md]
|
||||
became:
|
||||
- 02-DECISIONS/0005-the-node-host.md
|
||||
- 03-DESIGN/01-to-be/05-the-node-host.md
|
||||
---
|
||||
|
||||
# 003 — Who supervises a service
|
||||
|
||||
**No longer blocks Phase 0** (see below).
|
||||
- **Initiated by:** jochen, 2026-08-22, in response to
|
||||
[`002-local-mesh`](../002-local-mesh/analysis.md) open question 1
|
||||
- **Areas touched:** every module shipping a `systemd/` directory (14), the
|
||||
`hal-module@` template, `hal/sdk` feature handlers, `dev_up`, `log_tail` /
|
||||
`systemd_journal`, the bootstrap scripts.
|
||||
|
||||
## The question
|
||||
|
||||
`002` asked how a containerised node runs a module service, given that a module service is
|
||||
defined today as a systemd unit running `docker compose` against `/services/`. The response
|
||||
was the better question:
|
||||
|
||||
> If it's possible to run systemd inside a container, that's the way to go I think. However,
|
||||
> what would the cost be to step away from systemd to run our services and set it up in a
|
||||
> different way? More hal mesh approach.
|
||||
|
||||
This effort answers the cost half. It does not choose.
|
||||
|
||||
## Summary of findings
|
||||
|
||||
- **The question splits in two**, and the halves have opposite answers. Supervising
|
||||
`docker compose` stacks through systemd is largely **redundant** — 44 of 44 module compose
|
||||
files already declare a restart policy, so Docker is already the supervisor. Supervising
|
||||
HAL's **9 long-running Node daemons** is not redundant: `Restart=always` is currently the
|
||||
only thing between a crash and a dead node.
|
||||
- **The hard part is fate-sharing, not systemd.** meshware already cannot restart itself and
|
||||
carries a documented workaround for it. Any mesh-native supervisor inherits that problem
|
||||
recursively unless it sits outside the mesh's own process tree — at which point it is an
|
||||
OS-level supervisor again, just reinvented.
|
||||
- **There is a third option neither of us named**, and it is the one that also solves Phase 0:
|
||||
run HAL's own daemons as containers, making Docker the supervisor for everything. Local and
|
||||
production then have the same shape rather than a translation layer between them.
|
||||
- **It cannot be all-or-nothing**, and ADR 0001 already says why: a human agent acts through a
|
||||
shell and a desktop. Those parts are on the host by definition.
|
||||
- One incidental finding: the automatic node rescue that documentation describes **does not
|
||||
exist**. No unit declares `OnFailure=`, and nothing calls `hal-rescue.sh` on a timer.
|
||||
|
||||
Detail and costs in [`analysis.md`](analysis.md).
|
||||
|
||||
## What it became
|
||||
|
||||
*Closed 2026-08-28.* The decision this effort asked for was taken — and taken without citing it,
|
||||
which is why the effort sat `active` for five days after being answered. Recorded here because
|
||||
finding that is the point of a sweep.
|
||||
|
||||
**The third option is what the mesh adopted.** `Docker is the supervisor for everything` is
|
||||
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md): the
|
||||
host is a plain process on the machine and everything above tier 0 is a container. The substrate
|
||||
bootstrap declares no service at all — it is package, container, action, container — so the
|
||||
44-of-44 restart policies this effort counted are the supervision, exactly as it argued.
|
||||
|
||||
**Fate-sharing was the hard part, and it is solved the way this effort predicted.** It said any
|
||||
mesh-native supervisor inherits the problem *unless it sits outside the mesh's own process
|
||||
tree*. [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) puts
|
||||
the launcher there: it supervises the host as a child and shares no code with it, so a host that
|
||||
cannot start is still recovered.
|
||||
|
||||
**It is not all-or-nothing, as this effort insisted.** A human agent acts through a shell and a
|
||||
desktop, and those are on the host. So is the host itself — the one thing an init starts.
|
||||
|
||||
## What is not closed
|
||||
|
||||
**The automatic node rescue the documentation describes does not exist.** No unit declares
|
||||
`OnFailure=`, and nothing calls the rescue script on a timer. That is a documented behaviour
|
||||
which never happens, and it outlives this effort — filed as
|
||||
[`04-ISSUES/008`](../../04-ISSUES/008-the-documented-node-rescue-does-not-exist/00-report.md)
|
||||
rather than closed with it.
|
||||
@@ -0,0 +1,230 @@
|
||||
---
|
||||
effort: 003-service-supervision
|
||||
updated: 2026-08-22
|
||||
---
|
||||
|
||||
# Who supervises a service — the cost of leaving systemd
|
||||
|
||||
Measured 2026-08-22. Every claim is a file location or a count.
|
||||
|
||||
---
|
||||
|
||||
## 1. What systemd actually does for HAL
|
||||
|
||||
**14 modules** ship a `systemd/` directory. The units divide cleanly into three kinds:
|
||||
|
||||
| kind | count | what it is |
|
||||
|---|---|---|
|
||||
| long-running daemons | 9 | `hal/brain`, `hal/meshware`, `hal/coordinator`, `hal/cortex`, `hal/env-sync`, `hal/file-share`, `hal/thoughts`, `noxflow/runtime`, `desktop_notifications` — all `Restart=always`, `RestartSec=10` |
|
||||
| scheduled one-shots | 6 timers | docker-prune, docker-registry maintenance, claude-code sessions + usage, health, mailu cert-sync |
|
||||
| the template | 1 | `hal-module@.service` — the per-module Docker lifecycle |
|
||||
|
||||
Only `mailu` is system-scope. Everything else is a user unit under
|
||||
`~/.config/systemd/user`, auto-detected from directory presence — the module author names
|
||||
the file and that filename *is* the unit name
|
||||
(`modules/hal/sdk/src/feature-handlers/systemd.ts:31-42`).
|
||||
|
||||
The template is the piece `002` tripped over
|
||||
(`modules/hal/meshware/systemd/hal-module@.service`):
|
||||
|
||||
```
|
||||
ExecStart=docker compose --project-directory /services/%i --project-name %i up --remove-orphans --pull always
|
||||
ExecStop=docker compose --project-directory /services/%i --project-name %i down
|
||||
Restart=on-failure
|
||||
StartLimitIntervalSec=120
|
||||
StartLimitBurst=5
|
||||
```
|
||||
|
||||
Systemd's roles, then: restart-on-failure, start at login, env-file loading, ordering,
|
||||
per-module lifecycle, timers, and — via journald — **the only log store the 9 Node daemons
|
||||
have** (`modules/hal/sdk/src/tools/log-tail.ts:50-53`).
|
||||
|
||||
---
|
||||
|
||||
## 2. The question splits in two, and the halves disagree
|
||||
|
||||
### For `docker compose` stacks, systemd is mostly redundant
|
||||
|
||||
**44 of 44** module `docker-compose.yml` files declare a container-level `restart:` policy —
|
||||
`always` or `unless-stopped`, none missing. Docker's own daemon already restarts crashed
|
||||
containers, independently of systemd.
|
||||
|
||||
`hal-module@.service` does not supervise the containers. It supervises the **`docker compose
|
||||
up` foreground process**. It is a second layer on top of a restart policy that already
|
||||
works. What it genuinely adds is narrower than it looks:
|
||||
|
||||
- one uniform verb for every module type — `systemctl --user start hal-module@X` rather than
|
||||
remembering each module's compose invocation, relied on across a dozen call sites in
|
||||
`meshware.ts:124-154`, `dev-env.ts:142-163`, `installer-core.ts`, `infra.ts:166`
|
||||
- recovery when the `docker compose up --pull always` process itself dies — a failed image
|
||||
pull, not a crashed container
|
||||
- rate-limited restart (`StartLimitBurst=5`) so a broken stack does not spin
|
||||
|
||||
That is real, but it is a convenience layer, not a safety layer. **This half could go at
|
||||
moderate cost.**
|
||||
|
||||
### For the 9 Node daemons, systemd is load-bearing
|
||||
|
||||
There is no alternative supervisor anywhere in the repo. No PM2, no forever, no nodemon, no
|
||||
watchdog loop — all checked, zero hits. `Restart=always` is the only thing standing between
|
||||
a crashed daemon and a dead node.
|
||||
|
||||
**This half is the actual question.**
|
||||
|
||||
---
|
||||
|
||||
## 3. The hard part is fate-sharing
|
||||
|
||||
The difficulty is not systemd. It is that **a supervisor must not share fate with what it
|
||||
supervises**, and the codebase already has a scar from exactly this.
|
||||
|
||||
meshware cannot restart itself mid-request: killing the process before it ACKs the AMQP
|
||||
message loses the message. So it defers its own restart by two seconds after closing the
|
||||
connection (`modules/hal/meshware/daemon/src/cerebellum.ts:815-828`) — a commented
|
||||
workaround for a problem that only exists because the thing being restarted is the thing
|
||||
doing the restarting.
|
||||
|
||||
Any mesh-native supervisor inherits this recursively. Something has to be the outermost
|
||||
always-alive process, and if it is written by the mesh, the mesh must supervise it, and so
|
||||
on. The recursion only terminates at a process the mesh does not own. Today that is
|
||||
systemd. **A "more mesh" supervisor that is itself a mesh process is not a smaller problem;
|
||||
it is the same problem with a new name.**
|
||||
|
||||
This is the strongest argument for the status quo, and it is worth stating plainly before
|
||||
looking at alternatives.
|
||||
|
||||
---
|
||||
|
||||
## 4. The option neither of us named
|
||||
|
||||
There is a third answer that terminates the recursion in something that is not systemd and
|
||||
not written by us: **run HAL's own daemons as containers.**
|
||||
|
||||
Docker is already the outermost supervisor for 44 of 44 module stacks. It does not share
|
||||
fate with the mesh. It already has restart policies, backoff, and a log store that
|
||||
`log_tail` already speaks (`log-tail.ts:43-47` reads `docker logs` for Docker modules
|
||||
today). Extending it from "the things the mesh runs" to "the mesh itself" is not new
|
||||
machinery — it is applying machinery the mesh already trusts to one more case.
|
||||
|
||||
What that buys, beyond supervision:
|
||||
|
||||
- **Phase 0 stops being a translation.** A containerised node becomes the same shape as a
|
||||
production node, rather than a local approximation with a systemd-shaped hole in it. The
|
||||
divergence that `002` open question 3 worries about largely disappears.
|
||||
- **`/services/` and `~/.config/hal/` stop being special.** Mounts, not host paths.
|
||||
- **The GENESIS "dogfood everything" value gets easier**, not harder: the mesh's own
|
||||
components would ship and run exactly like everything else it carries.
|
||||
|
||||
What it costs, honestly:
|
||||
|
||||
- **journald → docker logs** for the 9 daemons. `log_tail` already handles both, but
|
||||
`systemd_journal` and the health checks that read unit state
|
||||
(`modules/hal/mesh/health.sh:34-40`, `modules/hal/health/hal-health.sh:335-395`) would
|
||||
need a container-aware path.
|
||||
- **Boot start** becomes Docker's `restart: always` plus the Docker daemon being enabled at
|
||||
boot — which is still one systemd unit, but the OS's own, not ours.
|
||||
- **A container needs the host to be reachable** for anything that touches the node itself.
|
||||
Some of these daemons exist precisely to write host files.
|
||||
|
||||
---
|
||||
|
||||
## 5. Why it cannot be all-or-nothing — and ADR 0001 already says so
|
||||
|
||||
Some of what runs under systemd today **cannot** be containerised, and the reason is
|
||||
already in the domain model. ADR 0001:
|
||||
|
||||
> a non-human agent acts through a spawned session — a human agent acts through a shell or
|
||||
> desktop
|
||||
|
||||
`hal/brain` has two modes (`modules/hal/brain/daemon/src/brainstem.ts:7-13`): a daemon mode
|
||||
that is an AMQP relay, and a **cortex mode that is an MCP server over stdio for an
|
||||
interactive Claude Code session**. The second is a human agent's modality. It runs in the
|
||||
human's shell, on the human's node, against the human's `~/.claude`. Containerising it is
|
||||
not a hard engineering problem, it is a category error.
|
||||
|
||||
The same holds for `desktop_notifications` and everything `hal/desktop-environment` touches.
|
||||
|
||||
So the line is not "systemd or not". It is:
|
||||
|
||||
| | belongs where |
|
||||
|---|---|
|
||||
| mesh daemons — meshware, coordinator, env-sync, thoughts, file-share, brain **in daemon mode** | supervisable by Docker; candidates to containerise |
|
||||
| human-modality surfaces — brain **in cortex mode**, desktop notifications, desktop environment | on the host, by definition |
|
||||
| module stacks | already Docker; systemd layer is the redundant part |
|
||||
|
||||
This split is not a compromise between the options. It is what the domain model implies,
|
||||
and it is a decent sign that the model is doing work.
|
||||
|
||||
---
|
||||
|
||||
## 6. Options, with costs
|
||||
|
||||
| | option | cost | what it buys |
|
||||
|---|---|---|---|
|
||||
| **A** | **Keep systemd; systemd-in-container for Phase 0** | privileged containers, heavy images, slow iteration; local mesh keeps a shape production does not have | nothing changes in production; smallest change to the mesh |
|
||||
| **B** | **Drop the `hal-module@` layer only** — let Docker's restart policies supervise stacks directly | reimplement uniform start/stop across ~12 call sites; lose rate-limited restart and pull-failure recovery | removes the redundant layer; does **not** solve Phase 0 on its own, since the daemons still need supervising |
|
||||
| **C** | **Containerise the mesh daemons; Docker supervises** | container-aware `systemd_journal`/health; host access for daemons that write host files; the human-modality surfaces stay on the host regardless | terminates the fate-sharing recursion without writing a supervisor; makes local and production the same shape; Phase 0 becomes much less of a special case |
|
||||
| **D** | **Write a mesh-native supervisor** | the fate-sharing recursion (§3), plus matching systemd's maturity — backoff, resource limits, clean SIGTERM (`noxflow/runtime` already depends on `TimeoutStopSec=60`) — on machines that are somebody's daily driver | most "mesh"; least justified by the evidence |
|
||||
|
||||
**D is the option the phrasing "more hal mesh approach" points at, and the evidence argues
|
||||
against it.** Supervision is not a domain concern the mesh is better placed to solve than
|
||||
the OS; the mesh's distinguishing feature is brokering capabilities, not restarting
|
||||
processes. C gets the benefit D is reaching for — the mesh not depending on host-specific
|
||||
init — without the recursion.
|
||||
|
||||
**C and B compose.** C is the one that pays for Phase 0.
|
||||
|
||||
Worth noting what is *not* in this table: the dozens of `systemctl` call sites that
|
||||
configure the **host OS's own** units — NetworkManager, resolved, oomd, zram, docker.service,
|
||||
fail2ban, sshd, zfs, across the vendor-firmware and power modules, `wireguard`, `sshd`, `zfs`
|
||||
and others. Those are not HAL supervising itself; they are HAL configuring an Arch box. They
|
||||
are out of scope for every option above and do not go away under any of them.
|
||||
|
||||
---
|
||||
|
||||
## 7. An incidental finding
|
||||
|
||||
Documentation describes an automatic node rescue: `hal-rescue.sh:20` states it is "triggered
|
||||
automatically by `hal-health.timer` when hal-meshware is failed".
|
||||
|
||||
**It is not.** `hal-health.sh` contains no call to `hal-rescue.sh` (checked, zero matches),
|
||||
and **no unit in the repository declares `OnFailure=`** (checked, zero matches). The only
|
||||
real triggers are the manual `rescue_node` tool
|
||||
(`modules/hal/sdk/src/tools/deploy-rescue.ts:77-92`) and running `install.d/rescue.sh` by
|
||||
hand.
|
||||
|
||||
This is worth recording for two reasons. It weakens any argument that systemd-adjacent
|
||||
self-healing is already wired — it is not. And it is another instance of the pattern this
|
||||
whole refactor is about: **a documented mechanism that does not exist, believed because it
|
||||
was written down.** `00-META/how-we-build.md` calls this out as a rule; here it is again,
|
||||
found by grep.
|
||||
|
||||
---
|
||||
|
||||
## 8. Open question
|
||||
|
||||
**Which supervision model does the mesh adopt?** A, B, C, D or a combination.
|
||||
|
||||
**This no longer gates Phase 0.** When this was written, the local mesh was assumed to be
|
||||
built from application containers, which forced the question — there is no natural way to
|
||||
run an init system inside one. The decision of 2026-08-22 to build development nodes as
|
||||
**system containers** (see [`03-DESIGN/01-end-to-end-testing.md`](../../03-DESIGN/01-to-be/01-end-to-end-testing.md))
|
||||
removes that pressure entirely: a system container runs a real init, so the existing model
|
||||
works unmodified and the lab needs no answer here to exist.
|
||||
|
||||
What remains is the question on its own merits, which is worth keeping open because the
|
||||
evidence above still holds: option B removes a layer that 44 of 44 module stacks have already
|
||||
made redundant, and option C would let the mesh stop depending on host-specific init. Neither
|
||||
is urgent. Both are now cheap to *try*, because there is somewhere to try them.
|
||||
|
||||
Option D — a mesh-written supervisor — remains the one the evidence argues against, for the
|
||||
fate-sharing reason in §3.
|
||||
|
||||
## References
|
||||
|
||||
- [`002-local-mesh`](../002-local-mesh/analysis.md) — the effort this came out of
|
||||
- [`02-DECISIONS/0001`](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md) — agent modality, which
|
||||
decides what cannot leave the host
|
||||
- `modules/hal/meshware/daemon/src/cerebellum.ts:815-828` — the self-restart workaround
|
||||
- `modules/hal/meshware/systemd/hal-module@.service` — the per-module Docker lifecycle
|
||||
- `modules/hal/sdk/src/feature-handlers/systemd.ts` — detection, install, start
|
||||
@@ -0,0 +1,67 @@
|
||||
---
|
||||
status: graduated
|
||||
initiated: 2026-08-22
|
||||
touches: [03-DESIGN/00-as-is/01-mesh-and-transport.md, 03-DESIGN/01-to-be/01-end-to-end-testing.md]
|
||||
became:
|
||||
- 02-DECISIONS/0016-the-lab.md
|
||||
- 02-DECISIONS/0016-the-lab.md
|
||||
- 02-DECISIONS/0016-the-lab.md
|
||||
- 03-DESIGN/01-to-be/02-scenario-declaration.md
|
||||
- 03-DESIGN/01-to-be/01-end-to-end-testing.md
|
||||
---
|
||||
|
||||
# 004 — Reproducing the mesh network in a lab
|
||||
|
||||
- **Initiated by:** jochen, 2026-08-22 — *"the most difficult part of our VM setup will be
|
||||
the networking part"*
|
||||
- **Areas touched:** `modules/wireguard`, `modules/dnsmasq-app`, `modules/traefik`,
|
||||
`modules/mesh-ca`, `node_accessors`, `nodes.site` / `nodes.underlay_addr`.
|
||||
|
||||
## Summary
|
||||
|
||||
The network is **entirely generated from mesh-DB rows by module hooks**. `install.d` performs
|
||||
no network configuration whatsoever — no WireGuard, no DNS, no firewall. That makes a faithful
|
||||
lab primarily a *data* problem rather than a networking problem, and means the lab exercises
|
||||
the real code path instead of a reimplementation of it.
|
||||
|
||||
One constraint decides whether the lab works at all: the WireGuard endpoint rule tests the
|
||||
underlay address against an RFC1918 regex to decide reachability. **A simulated public segment
|
||||
addressed from RFC1918 space silently prevents the mesh from forming** — no endpoint is written
|
||||
for the hub, so nothing can ever initiate. The simulated public segment must therefore use
|
||||
TEST-NET-3 (`203.0.113.0/24`).
|
||||
|
||||
With that one substitution the lab reproduces the production topology exactly, including the
|
||||
case that is hardest to get right: a node that is publicly *named* but sits behind NAT, whose
|
||||
endpoint the hub can only learn from a handshake.
|
||||
|
||||
Detail in [`analysis.md`](analysis.md).
|
||||
|
||||
## Settled
|
||||
|
||||
**The lab issues its own certificates.** Public names are certified by an ACME server on the
|
||||
lab's wan segment; `.internal` names keep the mesh CA. The lab preserves production's two-CA
|
||||
split rather than collapsing it, because a single-CA lab would hide any bug living in that
|
||||
split. It also makes the router's port forward load-bearing — HTTP-01 must reach the
|
||||
published-but-NATed node on port 80, so a broken forward becomes a reproducible certificate
|
||||
failure instead of a mystery.
|
||||
|
||||
Requires one change: `caServer` is not set on the reverse proxy today, so it defaults to the
|
||||
public authority's **production** endpoint. It must become configurable, defaulting to
|
||||
production so real nodes are unaffected.
|
||||
|
||||
## Open
|
||||
|
||||
*Closed 2026-08-25.* The lab is stood up. The topology this effort described raises, and the
|
||||
substitution it turned on — a simulated public segment addressed from documentation space
|
||||
rather than RFC1918 — is enforced by the declaration validator before anything is raised
|
||||
rather than left as a thing to remember.
|
||||
|
||||
The certificate conclusion above is carried by
|
||||
[`01-end-to-end-testing.md`](../../03-DESIGN/01-to-be/01-end-to-end-testing.md), which
|
||||
specifies the lab's own ACME issuer on the public segment. It is **designed and not built** —
|
||||
implementation state is a third axis, and the effort graduates on its conclusions, not on
|
||||
their delivery.
|
||||
|
||||
One item leaves this effort without a home and is recorded here so it is not lost: the reverse
|
||||
proxy does not set `caServer`, so it defaults to the public authority's production endpoint.
|
||||
That is a fact about what runs today, not about the lab.
|
||||
@@ -0,0 +1,203 @@
|
||||
---
|
||||
effort: 004-lab-network
|
||||
updated: 2026-08-22
|
||||
---
|
||||
|
||||
# Reproducing the mesh network in a lab
|
||||
|
||||
Established 2026-08-22 by reading the generating code and the live mesh DB. Every claim is a
|
||||
file location or a queried row.
|
||||
|
||||
---
|
||||
|
||||
## 1. The network is data, not configuration
|
||||
|
||||
`install.d/mesh-init.sh` and `install.d/adopt.sh` perform **no network configuration at
|
||||
all** — no WireGuard, no DNS, no firewall, no `/etc/hosts`. Every part of the network layer is
|
||||
generated by module hooks from mesh-DB rows:
|
||||
|
||||
| layer | generated by | from |
|
||||
|---|---|---|
|
||||
| WireGuard interface + peers | `modules/wireguard/hooks/index.ts` (`postConfigure`) | `node_wg_keys`, `nodes.site`, `nodes.underlay_addr`, `module_env.WG_ADDRESS` |
|
||||
| `.internal` name resolution | `modules/dnsmasq-app/hooks/index.ts` | mesh config peers → `internal_domain` + `wg_address` |
|
||||
| public routing / vhosts | `modules/hal/sdk/src/vhost-gen.ts`, `feature-handlers/vhost.ts` | `vhosts:` manifests + `node_accessors` |
|
||||
| internal TLS | `modules/mesh-ca/hooks/index.ts` | a singleton CA row in the mesh DB |
|
||||
|
||||
**Consequence:** a faithful lab is mostly a matter of writing the right rows. The network that
|
||||
results is produced by the same code production runs, which is the difference between testing
|
||||
the network and testing a model of it.
|
||||
|
||||
---
|
||||
|
||||
## 2. The constraint that decides whether the lab works
|
||||
|
||||
`modules/wireguard/hooks/index.ts:225-240` decides, per pair, whether to write an `Endpoint`:
|
||||
|
||||
```js
|
||||
const isPrivate = (a) => /^(10\.|127\.|192\.168\.|172\.(1[6-9]|2\d|3[01])\.)/.test(a);
|
||||
|
||||
if (coLocated && underlay) Endpoint = `${underlay}:${port}` // same LAN
|
||||
else if (underlay && !isPrivate(underlay)) Endpoint = `${underlay}:${port}` // public
|
||||
// else: no Endpoint — the peer must initiate, and we learn its endpoint from the handshake
|
||||
```
|
||||
|
||||
A simulated public segment addressed out of RFC1918 space — `10.200.0.0/24`, say — makes the
|
||||
hub's underlay test as **private**. No spoke writes an `Endpoint` for the hub. Nothing can
|
||||
initiate. **No handshake ever occurs and the mesh silently never forms**, presenting as a
|
||||
WireGuard fault rather than an addressing choice.
|
||||
|
||||
**The simulated public segment must therefore be `203.0.113.0/24`** — TEST-NET-3, reserved by
|
||||
RFC 5737 for documentation, guaranteed never to route on the real internet, and not matched by
|
||||
that regex. The code then treats it exactly as it treats a real hosting provider address.
|
||||
|
||||
This is the single most important fact in this document.
|
||||
|
||||
---
|
||||
|
||||
## 3. The topology being reproduced
|
||||
|
||||
The shape below is what a mesh of this kind looks like: one node with a routable address, one
|
||||
publicly named but behind a household NAT, one stationary workstation, one that roams.
|
||||
Addresses use the documentation ranges of RFC 5737 and RFC 1918 throughout.
|
||||
|
||||
| node | profile | site | underlay | WG | accessors |
|
||||
|---|---|---|---|---|---|
|
||||
| `anchor` | server | `dc` | `203.0.113.10` (routable) | `10.10.0.1/24` | `anchor.example` (public, primary) + `anchor.internal` (lan) |
|
||||
| `home-server` | server | `home` | `192.168.1.135` | `10.10.0.2/24` | `home-server.example` (public, primary) + `home-server.internal` (lan) |
|
||||
| `workstation` | workstation | `home` | `192.168.1.250` | `10.10.0.3/24` | `workstation.internal` (lan, primary) |
|
||||
| `laptop` | workstation | `NULL` | `NULL` | `10.10.0.4/24` | `laptop.internal` (lan, primary) |
|
||||
|
||||
**Hub election is by convention, not by flag.** The hub is the node whose `profile='server'`
|
||||
*and* whose `WG_ADDRESS` begins `10.10.0.1` (`hooks/index.ts:188`). A lab must assign
|
||||
`10.10.0.1` to the node it intends as hub or there will be no hub.
|
||||
|
||||
**`site` drives direct peering** (`hooks/index.ts:197-203`). Two nodes with the same non-null
|
||||
`site` peer directly with a `/32`; everything else routes through the hub's `/24`. A `NULL`
|
||||
site means roaming and hub-only — deliberately, because WireGuard has no failover and a more
|
||||
specific `/32` route to a dead endpoint blackholes rather than falling back.
|
||||
|
||||
The four interesting pairs, all of which the lab must reproduce:
|
||||
|
||||
| pair | behaviour | branch taken |
|
||||
|---|---|---|
|
||||
| anything → `anchor` | `Endpoint` written | underlay non-private |
|
||||
| `home-server` ↔ `workstation` | direct peer, LAN endpoints, keepalive | co-located |
|
||||
| **`anchor` → `home-server`** | **no `Endpoint`; learned from handshake** | not co-located, underlay private |
|
||||
| `laptop` → anything | hub only, always initiates | `site` is `NULL` |
|
||||
|
||||
The third is the one worth building the lab for. The code comments at `hooks/index.ts:206-224`
|
||||
record what it cost to get right: testing `profile === "server"` was tried and was wrong,
|
||||
because a home-hosted node **is** a server yet is not publicly reachable — *"role does not
|
||||
imply reachability; the address does."* An earlier version aimed the hub at that node's public
|
||||
name, which hairpinned off the household NAT: 1.77 MiB sent, 0 B received, no handshake.
|
||||
|
||||
---
|
||||
|
||||
## 4. The lab
|
||||
|
||||
```
|
||||
br-wan 203.0.113.0/24 TEST-NET-3 — non-private, so the code treats it as public
|
||||
│
|
||||
├── hub 203.0.113.10 profile=server site=dc WG 10.10.0.1
|
||||
│
|
||||
└── router VM 203.0.113.1 / 192.168.1.1
|
||||
│ NAT, plus one forwarded port to reproduce a published-but-NATed node
|
||||
│
|
||||
br-lan 192.168.1.0/24 identical to production, same host addresses
|
||||
├── a 192.168.1.135 profile=server site=home WG 10.10.0.2
|
||||
└── b 192.168.1.250 profile=workstation site=home WG 10.10.0.3
|
||||
|
||||
c — attach to br-lan, or br-wan ("away"), or detach ("asleep")
|
||||
underlay NULL profile=workstation site=NULL WG 10.10.0.4
|
||||
```
|
||||
|
||||
Kept **byte-identical** to production: the LAN subnet and its host addresses, and the entire
|
||||
WireGuard plan. Only the public segment is substituted, and only because it must be.
|
||||
|
||||
**The router earns its own VM.** It is what makes the published-but-NATed case real: that node
|
||||
is reachable from outside only through a forwarded port, and the hub must learn its endpoint.
|
||||
It also gives somewhere to break things — drop the forward and observe whether the mesh
|
||||
notices or whether the public name simply stops working.
|
||||
|
||||
**Names.** A resolver on the wan side is authoritative for the public zone. `.internal` names
|
||||
need nothing extra: `dnsmasq-app` generates them from mesh config on each node, and writes an
|
||||
`/etc/hosts` block as a floor underneath, because a node must reach the mesh DB before its own
|
||||
DNS exists.
|
||||
|
||||
---
|
||||
|
||||
## 5. What a node needs before any of this works
|
||||
|
||||
Rows in the mesh DB — `nodes` (`name`, `user_name`, `profile`, `site`, `underlay_addr`),
|
||||
`node_accessors`, `module_env.WG_ADDRESS` (the hook hard-fails without it,
|
||||
`hooks/index.ts:114-116`), and `node_modules` assigning at least `wireguard`, `dnsmasq-app`,
|
||||
`mesh-ca`, and `traefik` where it serves.
|
||||
|
||||
On disk beforehand, because the node must reach the mesh DB before it can read any of the
|
||||
above: registry database and object-store host and credentials, plus an npm token. This
|
||||
ordering — contact the mesh before the mesh has configured you — is itself worth reproducing,
|
||||
and is why the `/etc/hosts` floor exists.
|
||||
|
||||
Everything else is generated: the WireGuard keypair locally (the private key never leaves the
|
||||
node; the public key is published to `node_wg_keys`), the peer list, the DNS records, the TLS
|
||||
leaf.
|
||||
|
||||
---
|
||||
|
||||
## 6. Certificates — the lab issues its own
|
||||
|
||||
**Settled 2026-08-22: the lab runs its own ACME issuer.**
|
||||
|
||||
Public certificates use ACME **HTTP-01** via the reverse proxy, which requires genuine public
|
||||
reachability, so an isolated lab cannot use the real issuer. Rather than forgo certificate
|
||||
testing, the lab stands up an ACME server on its wan segment.
|
||||
|
||||
**The lab keeps production's two-CA split rather than collapsing it.** Production issues
|
||||
public names from a public authority and internal names from the mesh CA; a lab with one CA
|
||||
would hide any bug living in that split. So:
|
||||
|
||||
| | production | lab |
|
||||
|---|---|---|
|
||||
| public names | a public ACME authority | a test ACME server on the wan segment |
|
||||
| `.internal` names | `mesh-ca` | `mesh-ca`, unchanged |
|
||||
|
||||
**A test issuer is the right shape, not a shortcut.** Purpose-built ACME test servers
|
||||
deliberately vary their behaviour — validation timing, nonce handling, chain composition — to
|
||||
expose assumptions a well-behaved authority would let pass. A lab CA that is *too* polite
|
||||
tests less than the real thing, not more.
|
||||
|
||||
### It also exercises the port forward
|
||||
|
||||
HTTP-01 means the issuer must reach the node being certified on port 80. In the lab:
|
||||
|
||||
- the hub is directly reachable on the wan segment — straightforward
|
||||
- **the published-but-NATed node is reachable only through the router's forwarded port**
|
||||
|
||||
So certificate issuance for that node passes only if the forward is correct. That is exactly
|
||||
why its certificate works in production, and it makes "the forward is missing" a reproducible
|
||||
failure rather than a mystery.
|
||||
|
||||
### Required change: `caServer` must be configurable
|
||||
|
||||
`modules/traefik/docker-compose.yml:17-19` sets the challenge entrypoint, the contact address
|
||||
and the storage path — but **no `caServer`**, so Traefik defaults to the public authority's
|
||||
*production* endpoint. Pointing the lab at its own issuer requires adding a `caServer` flag
|
||||
fed by an environment value, defaulting to production so real nodes are unaffected and the lab
|
||||
overrides it per node.
|
||||
|
||||
Worth noting independently of the lab: aiming at the production endpoint rather than a staging
|
||||
one means every certificate experiment on a real node consumes production issuance quota, and
|
||||
a retry loop can exhaust it for a week. The lab issuer removes that exposure.
|
||||
|
||||
---
|
||||
|
||||
## 7. Incidental finding: `scope:` is read by nothing
|
||||
|
||||
Several manifests declare `scope: public` on firewall rules — `wireguard`, `traefik`, `gitea`,
|
||||
`mailu`, `qbittorrent`. It is **not part of the rule type** (`module-registry.ts:17-29`) and is
|
||||
**referenced by no code** in the firewall path. Real scoping is done with `from:`, as
|
||||
`modules/unifi/module.yml:52-93` does deliberately.
|
||||
|
||||
So a manifest can appear to restrict a port to the public scope and in fact restrict nothing.
|
||||
This is the same shape as the rule in `00-META/how-we-build.md` — *an unenforced rule is
|
||||
indistinguishable from a wrong one, and costs more, because people believe it.*
|
||||
@@ -0,0 +1,87 @@
|
||||
---
|
||||
status: graduated
|
||||
initiated: 2026-08-23
|
||||
touches:
|
||||
- 02-DECISIONS/0009-modules-and-the-graph.md
|
||||
- 03-DESIGN/00-as-is/10-module-catalogue.md
|
||||
- 03-DESIGN/00-as-is/02-modules-and-manifests.md
|
||||
became:
|
||||
- 02-DECISIONS/0009-modules-and-the-graph.md
|
||||
- 02-DECISIONS/0009-modules-and-the-graph.md
|
||||
---
|
||||
|
||||
# 005 — Which domains the catalogue groups into
|
||||
|
||||
[ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md) settles
|
||||
that modules outside the platform core are grouped by domain rather than by single function,
|
||||
and deliberately does not settle the list. This effort settles the list — and, first, tests
|
||||
whether the premise survives measurement.
|
||||
|
||||
## What is being investigated
|
||||
|
||||
Eighty-nine modules sit outside the platform core. The question is which of them belong
|
||||
together, and the method is evidence rather than intuition: **which modules actually change
|
||||
together**, measured across the full history of the code repository.
|
||||
|
||||
## Why
|
||||
|
||||
The argument in ADR 0022 is that the catalogue's shape records what was installed rather than
|
||||
what anything is for — that four modules constituting "how a node is reachable" have no
|
||||
relationship the mesh can see, so a change to connectivity is made four times.
|
||||
|
||||
That argument is testable. If those modules genuinely change together, the grouping is
|
||||
justified by more than tidiness. If they do not, the premise needs revising before a list is
|
||||
drawn from it.
|
||||
|
||||
## What it touches
|
||||
|
||||
The catalogue's shape, the manifest, and — for one candidate grouping — the provisioning
|
||||
reference itself, since a requirement names a provider **module**. Grouping providers would
|
||||
change what a consumer names.
|
||||
|
||||
## Status
|
||||
|
||||
Measurement is done and is in [`analysis.md`](analysis.md). It **partly contradicts the
|
||||
premise**, in a way that narrows the effort usefully:
|
||||
|
||||
- Non-platform modules overwhelmingly change **alone** — 10% of commits touch more than one,
|
||||
and 50 of 89 never co-change with anything.
|
||||
- Two clusters do exist. One of them, reachability, holds up as a domain.
|
||||
- The other, the provisioned infrastructure providers, co-changes for a reason that argues
|
||||
**against** grouping rather than for it.
|
||||
|
||||
The remaining work is the list itself, for the modules where grouping is justified, plus the
|
||||
open questions below.
|
||||
|
||||
## What it became
|
||||
|
||||
*Closed 2026-08-28.* Three of the four questions are answered, and by records that did not cite
|
||||
this effort — which is why it stayed open after being resolved.
|
||||
|
||||
**Whether provider modules group at all** — *no.*
|
||||
[ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md):
|
||||
there is no `networking` thing to install, there are concrete modules named individually. Folders
|
||||
assert relationships; edges record them. *Provider* stops being a category at the same time.
|
||||
|
||||
**Whether "group or leave" is even the right pair of options** — *it was not*, and that is the
|
||||
useful finding. [ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md)
|
||||
reframes it: things that change together share an **authority**, not a package. This effort's own
|
||||
measurement is what that record rests on — reachability being the *only* place modules genuinely
|
||||
co-change is why connectivity is a context and why nothing else needed one.
|
||||
|
||||
**What to do with the ~50 modules that co-change with nothing** — *nothing.* They are modules.
|
||||
Grouping is a tag and a query over the graph, neither of which anybody keeps true by hand.
|
||||
|
||||
**What remains is a sequencing question, not a grouping one**, and it moves rather than closes:
|
||||
*whether applications leave the monorepo before or after they group* is
|
||||
[research 009](../009-migration/00-overview.md)'s, because it is about how to get from here to
|
||||
there rather than about what the shape is.
|
||||
|
||||
## Open (superseded by the above) questions
|
||||
|
||||
| Question | Why it is open |
|
||||
|---|---|
|
||||
| Whether provider modules group at all, and if so what a consumer's requirement names instead of a module. | The provisioning reference is load-bearing; getting it wrong is expensive. Opinion and evidence in the analysis; not yet decided. |
|
||||
| Whether applications group into domains now and leave the monorepo later as a unit, or leave first. | Decided in principle — group first, then split — but the migration order has real cost either way. |
|
||||
| What to do with the ~50 modules that co-change with nothing. | The evidence gives no grouping signal for them at all. That may mean they are correctly sized already. |
|
||||
| Whether "group or leave" is even the right pair of options. | [Research 006](../006-mesh-from-scratch/code-skeleton.md) finds a third fate — **absorbed into the node host**, ceasing to be a module at all — and argues it is the correct answer for the reachability cluster this effort measured. If so, the cluster this effort found is evidence for absorption rather than for grouping. |
|
||||
@@ -0,0 +1,144 @@
|
||||
---
|
||||
effort: 005-domain-grouping
|
||||
updated: 2026-08-23
|
||||
---
|
||||
|
||||
# Which modules actually change together
|
||||
|
||||
## Method
|
||||
|
||||
Every commit in the code repository's main branch that touches the module catalogue, reduced
|
||||
to the set of modules it touched. Platform-namespace modules are excluded — their
|
||||
decomposition is settled by
|
||||
[ADR 0001](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md). Modules that no
|
||||
longer exist are excluded, because pre-rename names dominate the raw signal and describe a
|
||||
catalogue nobody works in.
|
||||
|
||||
Commits touching more than eight modules are excluded from pair counting: a sweep across
|
||||
forty modules says the sweep happened, not that the modules are related.
|
||||
|
||||
## Finding 1 — these modules overwhelmingly change alone
|
||||
|
||||
| Measure | Value |
|
||||
|---|---|
|
||||
| Non-platform modules today | 89 |
|
||||
| Commits touching at least one | 312 |
|
||||
| Commits touching two to eight together | 33 — **10%** |
|
||||
| Modules that never co-change with anything | **50 of 89** |
|
||||
|
||||
Nine commits in ten touch exactly one module. More than half the catalogue has never been
|
||||
edited alongside anything else.
|
||||
|
||||
This is the first thing a grouping proposal has to survive, and it is not what the premise
|
||||
predicts.
|
||||
|
||||
## Finding 2 — two clusters exist
|
||||
|
||||
Everything above two co-changes falls into one of two groups.
|
||||
|
||||
**The provisioned infrastructure providers.** The densest cluster: the relational stores with
|
||||
each other and with the cache, the broker, the object store and the image registry — a
|
||||
strongly connected set, six to two co-changes per pair.
|
||||
|
||||
**Reachability.** The reverse proxy with the resolver, the firewall and the VPN; the firewall
|
||||
with the intrusion filter. Three to two co-changes per pair.
|
||||
|
||||
Outside those: a torrent client with the VPN, twice. A login manager with a media player,
|
||||
twice. Nothing else reaches two.
|
||||
|
||||
## Finding 3 — the provider cluster is manifest churn, not cohesion
|
||||
|
||||
This is the result that matters, and it is only visible by reading the commits rather than
|
||||
counting them.
|
||||
|
||||
Every multi-provider commit is a **cross-cutting change to the machinery, applied N times**:
|
||||
|
||||
| Date | What the commit did |
|
||||
|---|---|
|
||||
| 2026-04-15 | modules own their tools — decentralised tool serving |
|
||||
| 2026-04-24 | filesystem-driven feature detection; manifest boilerplate removed |
|
||||
| 2026-04-27 | remove runtime dependencies that were only build dependencies |
|
||||
| 2026-05-01 | hooks move into their own directory with their own package |
|
||||
| 2026-05-05 | hooks renamed to a stage-and-feature convention; legacy install scripts removed |
|
||||
| 2026-08-06 | verify **by shape in the SDK**, not by copying a script into every module |
|
||||
| 2026-08-11 | bind declared volume paths that leaked an anonymous volume per deploy |
|
||||
| 2026-08-23 | scope services to the local network and mesh; respect the profile gate |
|
||||
|
||||
**Not one of them is a change to what a database is.** They are all "the manifest contract
|
||||
changed, therefore every manifest changed".
|
||||
|
||||
Two consequences follow.
|
||||
|
||||
Merging the providers into one module would not have prevented a single one of these commits.
|
||||
It would have made the same edit land in one file rather than six — which is a diff-size
|
||||
improvement, not a boundary.
|
||||
|
||||
And the history already shows the correct fix being applied, repeatedly and successfully:
|
||||
*verify by shape in the SDK, not by copying a script into every module*. When the same edit
|
||||
must be made in every provider, the answer that worked was **moving the concern into the
|
||||
machinery**, not merging the modules that carry it.
|
||||
|
||||
Co-change here measures coupling to the manifest contract. It does not measure domain
|
||||
cohesion, and using it as though it did would group the catalogue by which modules are most
|
||||
boilerplate-heavy.
|
||||
|
||||
## Finding 4 — reachability holds up
|
||||
|
||||
The same reading applied to the reachability cluster gives a different answer. Most of its
|
||||
twenty-five multi-module commits are the same machinery churn — but not all, and the
|
||||
remainder are genuine:
|
||||
|
||||
| Date | What the commit did |
|
||||
|---|---|
|
||||
| 2026-07-29 | let the mesh's proxy coexist with another listener on the same port |
|
||||
| 2026-08-03 | derive public split-DNS from node accessors so requests stop hairpinning |
|
||||
| 2026-08-06 | firewall mesh-only by default, public by declaration |
|
||||
|
||||
Each is one intent — *change how a node is reachable* — landing across the proxy, the
|
||||
resolver, the firewall and the VPN together. That is exactly the shape ADR 0022 describes, and
|
||||
it is the only place in the catalogue where the measurement finds it.
|
||||
|
||||
The 2026-08-23 scoping commit is the sharpest case: it spans the reachability cluster **and**
|
||||
two providers, because "which network is this exposed on" is a reachability question asked of
|
||||
a database.
|
||||
|
||||
## What this means for ADR 0022
|
||||
|
||||
The record's principle stands, and its scope needs narrowing. Grouping by domain is:
|
||||
|
||||
- **Justified by evidence** for reachability. One intent, several modules, repeatedly.
|
||||
- **Argued against by evidence** for the providers. The coupling is to the manifest contract,
|
||||
and the demonstrated fix is to move the concern into the machinery.
|
||||
- **Unsupported either way** for the fifty modules that co-change with nothing. Silence is not
|
||||
evidence of independence — many are simply rarely touched — but there is no measured basis
|
||||
for grouping them, and a proposal that groups them is drawn from intuition. It should say so.
|
||||
|
||||
The catalogue's shape is still wrong in the way the record describes. Measurement says
|
||||
grouping is the right fix in fewer places than the record implies, and that a second fix —
|
||||
moving cross-cutting concerns into the machinery — accounts for most of what looks like
|
||||
grouping pressure.
|
||||
|
||||
## Recommendation on the provisioning reference
|
||||
|
||||
Asked directly, and stated as an opinion because it is not yet decided.
|
||||
|
||||
**Do not group the provider modules.** Three reasons, in order of weight:
|
||||
|
||||
1. **The evidence for grouping them is the wrong evidence.** Finding 3.
|
||||
2. **A requirement names a provider module.** Grouping providers means a consumer names a
|
||||
resource type and something else chooses the implementation. That is not a folder move; it
|
||||
is implementation selection, a substantially larger design with its own failure modes, and
|
||||
nothing currently asks for it.
|
||||
3. **It would hide which implementation serves a requirement** — the one place the mesh most
|
||||
needs to be explicit, and precisely the indirection ADR 0022 warns grouping causes.
|
||||
|
||||
A provider module is already exactly one purpose: it provisions one resource type. That is a
|
||||
boundary, not an accident of installation.
|
||||
|
||||
## Still to do
|
||||
|
||||
- Draw the list for the cases where grouping is justified, and say plainly which entries rest
|
||||
on measurement and which on judgement.
|
||||
- Decide the ~50 silent modules: correctly sized, or unmeasured?
|
||||
- Test the reachability grouping against the migration cost — the modules in it are among the
|
||||
most-changed in the catalogue, so churn during a move is not hypothetical.
|
||||
@@ -0,0 +1,94 @@
|
||||
---
|
||||
status: active
|
||||
initiated: 2026-08-23
|
||||
touches:
|
||||
- 02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md
|
||||
- 02-DECISIONS/0009-modules-and-the-graph.md
|
||||
- 03-DESIGN/00-as-is/00-overview.md
|
||||
- 03-DESIGN/01-to-be/00-work-breakdown.md
|
||||
became: []
|
||||
---
|
||||
|
||||
# 006 — The mesh designed from nothing
|
||||
|
||||
## What is being investigated
|
||||
|
||||
What the mesh would look like if it were laid out today, with the requirements known and none
|
||||
of the accumulated shape — expressed as a **skeleton**: repositories at the root, modules
|
||||
inside them, and whatever turns out to be the right leaf unit below that.
|
||||
|
||||
The deliverables are [`skeleton.md`](skeleton.md) — tiers, repositories and the four design
|
||||
moves — and [`code-skeleton.md`](code-skeleton.md) — the tier test, what a module looks like on
|
||||
disk, and where today's catalogue lands.
|
||||
|
||||
## Why
|
||||
|
||||
Every structural decision so far has been a **correction**: eight contexts replacing thirty-three
|
||||
modules ([ADR 0001](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md)), domains
|
||||
replacing single-function modules
|
||||
([ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md)). A
|
||||
correction inherits the frame of the thing it corrects, and two of the mesh's oldest problems
|
||||
look unsolvable from inside that frame:
|
||||
|
||||
- **The bootstrap circularity.** The mesh needs a database, a bus, a registry and an identity
|
||||
provider. Those are modules the mesh installs. The mesh cannot install them before it exists.
|
||||
This has been worked around repeatedly and never designed away.
|
||||
- **Participation requires privilege.** Everything assumes root on a machine whose packages and
|
||||
services the mesh owns. A phone cannot participate on those terms, and neither can a machine
|
||||
someone else administers.
|
||||
|
||||
Designing from nothing is a way to find out which parts of the current shape are requirements
|
||||
and which are residue.
|
||||
|
||||
**And there is more residue than expected, from a knowable source.** This began as a
|
||||
**dotfiles repository** — the first two days of history adopt dotfiles, add per-node dotfile
|
||||
overrides, and introduce service symlinking with an ignore file. The flat one-directory-per-tool
|
||||
catalogue, linking rather than copying, adoption of already-configured machines, per-node
|
||||
overrides, and the desktop modules are all inherited from that, not chosen for a mesh. Recorded
|
||||
in [`03-DESIGN/00-as-is/10-module-catalogue.md`](../../03-DESIGN/00-as-is/10-module-catalogue.md).
|
||||
|
||||
That makes this effort's question sharper than "what would we do differently": much of what
|
||||
looks like design is a generalisation of *place files on my machines*, never revisited because
|
||||
it was never stated as an assumption.
|
||||
|
||||
## The requirements this is designed against
|
||||
|
||||
Stated by the operator, recorded here so the skeleton can be checked against them rather than
|
||||
against taste:
|
||||
|
||||
1. The mesh manages multiple computers — **full control**, through modules installed to nodes.
|
||||
2. Mesh state lives in a **database**: which modules on which nodes, logs, configuration.
|
||||
3. Configuration has **several touchpoints** — tool surface, web interface, others — all hosted
|
||||
by the mesh itself.
|
||||
4. **Connectivity** is core: every node reachable from every other over a shared overlay, some
|
||||
nodes publicly exposed, firewalls configured.
|
||||
5. The mesh **hosts applications** — and requires some of them itself. This is the circularity.
|
||||
6. **Arch Linux only for now**; ideally any device, including phones, on lighter terms.
|
||||
7. The end goal is to **operate an IT company** on it — development, design, deployment, full
|
||||
circle, self-hosted. Personal cloud infrastructure.
|
||||
8. **Agents make it self-improving and self-healing.**
|
||||
9. It is **end-to-end testable on one machine**
|
||||
([ADR 0016](../../02-DECISIONS/0016-the-lab.md)).
|
||||
|
||||
## Status
|
||||
|
||||
A first skeleton exists, with four design moves that the current shape does not have. It is
|
||||
`active` because two of them are unproven and one contradicts a record that is already
|
||||
accepted.
|
||||
|
||||
**Finding worth stating up front:** [ADR 0001](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md)
|
||||
names nine bounded contexts and **none of them owns connectivity** — no overlay, no resolution,
|
||||
no firewall, no ingress. Requirement 4 has no home in the accepted decomposition, while
|
||||
[research 005](../005-domain-grouping/analysis.md) found reachability to be the *only* part of
|
||||
the catalogue where modules genuinely change together under one intent. The skeleton adds it.
|
||||
|
||||
## Open questions
|
||||
|
||||
| Question | Why it is open |
|
||||
|---|---|
|
||||
| Does the record — the event log contexts integrate through — belong to the substrate or the control plane? | It is infrastructure by shape and domain by content. Placing it wrong reintroduces a circularity. |
|
||||
| One repository per tier, or per context? | Already open from ADR 0001 as "catalogue destination — one repository or many". The skeleton assumes per tier and does not settle it. |
|
||||
| ~~Does an unprivileged node earn a place in the inventory, or only a presence?~~ | **Answered 2026-08-25** by the operator: a node is a *managed machine inside the mesh*, not an unprivileged something — and a disconnected node is still a node, in a different situation. The question posed a class distinction; the answer is that there is none, and what varies is **state**. Recorded as [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md). |
|
||||
| ~~Does absorbing overlay, filtering, packages, supervision and the container runtime make the host too large?~~ | **Answered 2026-08-25** — [`host-size.md`](host-size.md). Measured: the absorption is smaller than the machinery that already applies state, and eight of ten adapters already carry no dependency. The risk is not size but direction, and it is two modules wide. The claim survives with its scope corrected — the host carries one concern, *apply declared state on this machine*, of which the six are instances. Recorded as [ADR 0005](../../02-DECISIONS/0005-the-node-host.md), designed in [`05-the-node-host.md`](../../03-DESIGN/01-to-be/05-the-node-host.md). |
|
||||
| ~~Four substrate services or five?~~ | **Answered conditionally**, which is the honest form — [`07-the-foundation.md`](../../03-DESIGN/01-to-be/07-the-foundation.md). The substrate is *what the control plane consumes and cannot grant itself*. The identity provider qualifies only if the control plane delegates authentication; if it authenticates natively it is an ordinary hosted service. The count follows from a decision not yet taken, and asserting four was asserting that decision. |
|
||||
| Does `feature` survive? | The skeleton splits it in two and argues the conflation is what makes the delivery pipeline hard to reason about. Unproven. |
|
||||
@@ -0,0 +1,272 @@
|
||||
---
|
||||
effort: 006-mesh-from-scratch
|
||||
updated: 2026-08-23
|
||||
---
|
||||
|
||||
# The code skeleton
|
||||
|
||||
[`skeleton.md`](skeleton.md) laid out tiers and repositories. This is the level below: what a
|
||||
module looks like on disk, and — the question that forced a correction — where a given piece of
|
||||
today's catalogue actually lands.
|
||||
|
||||
## The tier test
|
||||
|
||||
A decision procedure, so placement is answerable rather than argued. Ask in order; first match
|
||||
wins:
|
||||
|
||||
1. **Does it apply state on a machine?** → tier 0, inside the host.
|
||||
2. **Can the control plane *start* without it?** If *no* → tier 1, substrate. Note the verb:
|
||||
*start*, not *function fully*. A capability the control plane loses without something is not
|
||||
the same as a thing it cannot come up without — see "becoming self-hosting" below.
|
||||
3. **Does it decide what should be true across nodes?** → tier 2, a control-plane context.
|
||||
4. **Is it a way to talk to tier 2, holding no logic of its own?** → tier 3, a surface.
|
||||
5. **Otherwise** → tier 4, a workload.
|
||||
|
||||
## Worked example — where postgres ends up
|
||||
|
||||
The obvious answer is "twice": once as the mesh's own database in the substrate, once as a
|
||||
hosted database in the catalogue. That answer is wrong, and seeing why fixes something.
|
||||
|
||||
Run the test. *Can the control plane exist without a relational store?* No. **Postgres is
|
||||
tier 1.** It lives once:
|
||||
|
||||
```
|
||||
mesh-substrate/store/postgres/
|
||||
```
|
||||
|
||||
There is no second copy in the catalogue, because the thing that differs between the mesh's own
|
||||
database and a project's database is **not the module**. It is how that instance is brought up:
|
||||
|
||||
| | The mesh's own instance | A project's database |
|
||||
|---|---|---|
|
||||
| Brought up by | the host, from the pinned bundle, with no control plane present | the ordinary delivery and provisioning path |
|
||||
| Declared in | `mesh-substrate/bundle.yml` | the consuming module's `requires:` |
|
||||
| Exists because | the control plane cannot start without it | something asked for it |
|
||||
|
||||
Same module, two roles. **Tier is a property of the module** — what must exist before what — and
|
||||
**the bundle is a property of the mesh's own instance.**
|
||||
|
||||
This is also legal under the dependency rule, which is worth checking rather than assuming: a
|
||||
tier-4 workload that requires a database depends on tier 1, which points *downward*. The
|
||||
inverse — substrate reaching into the catalogue for a module — would not be, and is the shape
|
||||
the naive "twice" answer would have created.
|
||||
|
||||
The same reasoning places the rest of the substrate: the bus, the object store, the image
|
||||
registry. Each fails step 2, each lives once, each is pinned. That is the whole substrate: four
|
||||
services, and the deliberate absence of a fifth.
|
||||
|
||||
**The identity provider was raised as a boundary case and is settled: it is not substrate.**
|
||||
The mesh does not require one — tier 2 authenticates its own callers natively, and an identity
|
||||
provider is a service the mesh hosts like any other. The substrate is four services, not five.
|
||||
|
||||
The test earned its keep here by turning a vague unease into one answerable question — *does the
|
||||
control plane delegate authentication?* — rather than a debate about how important identity
|
||||
feels.
|
||||
|
||||
## Becoming self-hosting — the forge and the registries
|
||||
|
||||
Self-improvement means the mesh hosts the things it improves itself with: a forge, an image
|
||||
registry, a package registry. The obvious worry is that these duplicate — a `mesh-gitea`
|
||||
for the mesh and a `gitea` for everyone else. **They do not, and the reason is worth stating
|
||||
carefully, because it is the same reason the bootstrap keeps failing today.**
|
||||
|
||||
### They are not substrate
|
||||
|
||||
Run the test with the sharpened verb. *Can the control plane start without a forge?* **Yes.** It
|
||||
comes up, holds inventory, answers questions and manages nodes with the modules it already has.
|
||||
What it cannot do is **change itself**. That is a capability, not a precondition.
|
||||
|
||||
So: forge, image registry and package registry are **tier 4 workloads**. One module each, in the
|
||||
catalogue, exactly like the media server. There is no mesh-specific copy.
|
||||
|
||||
This is not a technicality. It buys a property worth having: **if the forge dies, the mesh keeps
|
||||
running.** Nodes stay managed, services stay up, only self-modification stops. Putting the forge
|
||||
in the substrate would make losing it fatal, for no gain.
|
||||
|
||||
### But delivery needs them — is that not an upward dependency?
|
||||
|
||||
It would be, stated naively, and that would break the one rule the whole skeleton rests on.
|
||||
|
||||
It is resolved the way the constitution already says to resolve it — **depend on abstractions,
|
||||
not on concrete dependencies**. Tier 2's delivery context does not require *the forge module*.
|
||||
It declares requirements:
|
||||
|
||||
| Delivery requires | Satisfied by |
|
||||
|---|---|
|
||||
| a source of record for module code | whichever module provides it |
|
||||
| somewhere to publish images | whichever module provides it |
|
||||
| somewhere to publish packages | whichever module provides it |
|
||||
| somewhere to put build artifacts | the substrate's object store |
|
||||
|
||||
Tier 2 defines the requirement; tier 4 provides the implementation; the binding is data. The
|
||||
dependency points **downward from the provider to the interface**, which is legal, and the
|
||||
control plane never names a concrete module.
|
||||
|
||||
The mechanism for this already exists and is the mesh's most valuable one: **provisioning**. A
|
||||
module declares what it provides; a consumer declares what it requires; the mesh binds them.
|
||||
The only new idea is that **the control plane is itself a consumer** — it has requirements, and
|
||||
they are satisfied the same way a workload's are.
|
||||
|
||||
That generalisation is significant enough to need its own study, and is not settled here.
|
||||
|
||||
### Self-hosting is a state the mesh reaches, not a precondition
|
||||
|
||||
This is the part today's mesh gets wrong, and it explains a recurring class of pain.
|
||||
|
||||
A first node comes up from **pinned external artifacts** — upstream images, by digest, carried
|
||||
in the bundle. It has to: the mesh's own registry does not exist yet, and cannot. The mesh at
|
||||
this point is running and manages nodes, and is not yet self-hosting.
|
||||
|
||||
Self-hosting is then **reached**: the forge is installed as an ordinary workload, the mesh's own
|
||||
source moves into it, the registries come up, and delivery's requirements are re-bound from
|
||||
external providers to internal ones. From that point the mesh builds and deploys itself.
|
||||
|
||||
Stated as a lifecycle:
|
||||
|
||||
```
|
||||
pinned external artifacts ─► mesh runs, manages nodes
|
||||
│
|
||||
│ forge + registries installed as workloads
|
||||
│ delivery's requirements re-bound
|
||||
▼
|
||||
mesh builds and deploys itself
|
||||
```
|
||||
|
||||
**Today's mesh assumes the second state from the first moment.** Its source, its packages and
|
||||
its images are all expected to be self-hosted before there is anything to host them — which is
|
||||
why raising a first node needs a script that exists solely to paper over the impossibility, and
|
||||
why that script is the least-exercised path in the system.
|
||||
|
||||
Making the transition explicit has a second benefit: it is reversible. A mesh whose forge is
|
||||
broken can re-bind delivery to external providers and keep improving itself while it repairs
|
||||
the forge. Today that escape hatch does not exist, because the dependency is not expressed
|
||||
anywhere it could be changed.
|
||||
|
||||
### So, concretely
|
||||
|
||||
One `gitea` module. One image-registry module. One package-registry module. Each a tier-4
|
||||
workload. The mesh's own instances are distinguished from any other instance **by what they are
|
||||
bound to, not by being different modules** — precisely the same answer as postgres, arrived at
|
||||
by the same test.
|
||||
|
||||
## The five fates of a module
|
||||
|
||||
Research 005 asked whether a catalogue module should be **grouped into a domain** or **leave
|
||||
the repository**. Working the tier test across the catalogue surfaces a third answer that
|
||||
neither option covers, and it is the most common one.
|
||||
|
||||
| Fate | Means | Examples from today |
|
||||
|---|---|---|
|
||||
| **Absorbed into the host** | It is not a module at all. It is part of what "managing a machine" means, and belongs in tier 0. | overlay membership, packet filtering, package management, service supervision, container runtime, filesystem management |
|
||||
| **Substrate** | The control plane cannot exist without it. Pinned, host-applied. | relational store, bus, object store, image registry |
|
||||
| **Control-plane context** | It decides something across nodes. | connectivity policy, inventory, delivery, provisioning, observability |
|
||||
| **Workload module** | The mesh hosts it. Grouped per [ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md). | media library, desktop session, collaboration tooling |
|
||||
| **Leaves the repository** | A standalone application, per [ADR 0015](../../02-DECISIONS/0015-applications-live-in-their-own-repository.md). | the applications identified in research 005 |
|
||||
|
||||
**The first fate is the finding.** Research 005 measured the reachability cluster — proxy,
|
||||
resolver, firewall, overlay — as the only place in the catalogue where modules genuinely change
|
||||
together under one intent. The skeleton explains *why*: they are not four modules that ought to
|
||||
be one domain module. They are four facets of one thing the host should own, currently
|
||||
expressed as modules because a module was the only unit available.
|
||||
|
||||
Under this skeleton the overlay module and the firewall module **stop existing**. The host holds
|
||||
membership and applies filtering; tier 2 decides the policy; the swappable backends stay
|
||||
modules. That is a different and better answer than grouping them, and it was not visible from
|
||||
inside the current frame.
|
||||
|
||||
It also partly answers research 005's other open question — the fifty modules that co-change
|
||||
with nothing. Several are host concerns rather than domains: package management, container
|
||||
runtime, filesystem tooling. Silence was the right signal after all; the wrong conclusion was
|
||||
that grouping was the only available fix.
|
||||
|
||||
## What a module looks like on disk
|
||||
|
||||
Per [`skeleton.md`](skeleton.md) Move 4, a module declares **parts** — independently selectable
|
||||
pieces of desired state — and produces **artifacts** — things built once per version.
|
||||
|
||||
```
|
||||
<module>/
|
||||
module.yml identity, what it provides, what it requires,
|
||||
which host profiles it can land on
|
||||
parts/
|
||||
service/ desired state: container, volumes, exposure
|
||||
provisioner/ how it grants its resource to consumers
|
||||
migrations/ its own persistent state
|
||||
tools/ capabilities it contributes
|
||||
artifacts/
|
||||
<name>/ source for something built and published
|
||||
```
|
||||
|
||||
A module with no artifacts consumes an upstream image and builds nothing. A module with no
|
||||
parts is not a module.
|
||||
|
||||
Worked through for the substrate's relational store:
|
||||
|
||||
```
|
||||
mesh-substrate/store/postgres/
|
||||
module.yml provides: database · profiles: [managed]
|
||||
parts/
|
||||
service/ the container, its volume, its network exposure
|
||||
provisioner/ grants a database and role to a consumer
|
||||
migrations/ none — it holds no state of its own
|
||||
artifacts/ none — upstream image, pinned by digest in bundle.yml
|
||||
```
|
||||
|
||||
## The tree, at file level
|
||||
|
||||
```
|
||||
mesh-host/ TIER 0
|
||||
cmd/host/
|
||||
internal/
|
||||
apply/ reconcile declared state
|
||||
inventory/ what this machine is and can do
|
||||
link/ outbound connection to the control plane
|
||||
overlay/ membership: address, keys, tunnel
|
||||
filter/ packet filtering from tier-2 policy
|
||||
packages/ package management
|
||||
services/ supervision
|
||||
containers/ container runtime
|
||||
store/ embedded local state
|
||||
profile/ managed · user · edge
|
||||
substrate.lock pinned tier-1 descriptor
|
||||
|
||||
mesh-substrate/ TIER 1
|
||||
bundle.yml the pinned set, by digest
|
||||
store/postgres/
|
||||
bus/<broker>/
|
||||
objects/<object-store>/
|
||||
images/<registry>/
|
||||
|
||||
mesh-control/ TIER 2
|
||||
record/ the event log contexts integrate through
|
||||
inventory/ nodes · modules · assignments · versions
|
||||
config/ settings · secrets · derivation
|
||||
connectivity/ addresses · resolution · exposure · filtering · certificates
|
||||
provisioning/ grants between modules
|
||||
delivery/ source → artifact → node
|
||||
observability/ health · logs · metrics
|
||||
identity/ agents · humans · services · authorisation
|
||||
work/ tasks · workflows · runs
|
||||
knowledge/ memory · documents · retrieval
|
||||
api/ the one interface surfaces speak to
|
||||
|
||||
mesh-surfaces/ TIER 3
|
||||
tools/ web/ cli/
|
||||
|
||||
mesh-catalog/ TIER 4
|
||||
<domain>/<module>/ layout as above
|
||||
|
||||
mesh-lab/ mesh-sdk/ hq/ (company-scoped — ADR 0019)
|
||||
```
|
||||
|
||||
## What this does not settle
|
||||
|
||||
- **The identity boundary case.** Four substrate services or five.
|
||||
- **Where the record lives.** Still the open question from
|
||||
[`00-overview.md`](00-overview.md), and the tier test does not resolve it: the record is
|
||||
needed by tier 2 and is *of* tier 2, which is exactly the shape that produces a circularity.
|
||||
- **Whether absorbing into the host makes the host too large.** Six internal concerns is already
|
||||
a lot for a binary whose whole argument is that it has no dependencies. The counter-argument
|
||||
is that each is small and none can be optional — but this is the skeleton's biggest unproven
|
||||
claim, and it should be tested by writing the host's interface before anything else.
|
||||
- **The migration.** Nothing here says how today becomes this.
|
||||
@@ -0,0 +1,126 @@
|
||||
# Is the host too large?
|
||||
|
||||
The skeleton absorbs overlay membership, packet filtering, package management, service
|
||||
supervision, the container runtime and filesystem management into tier 0, and
|
||||
[`00-overview.md`](00-overview.md) calls this *"the skeleton's biggest unproven claim. A binary
|
||||
whose whole argument is that it has no dependencies now carries six concerns."*
|
||||
|
||||
This is that claim, measured.
|
||||
|
||||
## Method
|
||||
|
||||
Against `origin/main` of the monorepo at 2026-08-25 — read through git refs rather than a
|
||||
checkout, which sits on a feature branch 1031 commits behind with uncommitted work.
|
||||
|
||||
For each module that implements one of the six concerns: how much code it is, and **what it
|
||||
depends on to do its job**. The second question turned out to be the one that matters.
|
||||
|
||||
## Finding 1 — by size, the concern is misplaced
|
||||
|
||||
| Absorbed into the host | Lines |
|
||||
|---|---|
|
||||
| `dnsmasq-app` | 706 |
|
||||
| `traefik` | 426 |
|
||||
| `ufw` | 385 |
|
||||
| `mesh-ca` | 353 |
|
||||
| `wireguard` | 313 |
|
||||
| `incus` | 285 |
|
||||
| `package-manager` | 118 |
|
||||
| `zfs` | 67 |
|
||||
| `fail2ban` | 60 |
|
||||
| `docker-app` | 42 |
|
||||
| **total** | **2 755** |
|
||||
|
||||
| Machinery that already applies state on a node | Lines |
|
||||
|---|---|
|
||||
| `hal/meshware` | 1 829 |
|
||||
| `hal/env-sync` | 626 |
|
||||
| `hal/config-sync` | 604 |
|
||||
| **total** | **3 059** |
|
||||
|
||||
**Everything being absorbed is smaller than the machinery that already exists to apply it.**
|
||||
The six concerns are not six subsystems; they are ten thin adapters averaging 275 lines, most
|
||||
of which is rendering a config file and running a command.
|
||||
|
||||
The host is not a new large thing. It already exists, spread across three core modules, and
|
||||
the absorption adds less code than those three already contain.
|
||||
|
||||
Size is therefore the wrong axis, and the open question asked about the wrong thing.
|
||||
|
||||
## Finding 2 — the real risk is direction, and it is narrow
|
||||
|
||||
What each adapter needs in order to decide what to write:
|
||||
|
||||
| Module | Gets its inputs from | Applier only? |
|
||||
|---|---|---|
|
||||
| `ufw` | `~/.hal/modules/*/module.yml` on the local disk | yes |
|
||||
| `dnsmasq-app` | `process.env.DNSMASQ_*`, derived centrally by `env-sync` | yes |
|
||||
| `fail2ban`, `mesh-ca`, `package-manager`, `docker-app`, `incus`, `zfs` | local state and declared env | yes |
|
||||
| `wireguard` | **direct `pg` connection** — `nodes`, `node_accessors`, `node_wg_keys`, `module_env` | **no** |
|
||||
| `traefik` | **direct `pg` connection** — `nodes`, `mesh_ca` | **no** |
|
||||
|
||||
**Eight of ten are already pure appliers.** They receive derived state and put it on the
|
||||
machine. Absorbing those into tier 0 moves no dependency at all — it moves code that already
|
||||
has none.
|
||||
|
||||
**Two reach upward.** `wireguard` and `traefik` open a connection to the control plane's
|
||||
database and compute their own configuration from it. Absorbing them *as they are* would put a
|
||||
Postgres client and knowledge of the mesh schema inside tier 0 — an upward dependency, which
|
||||
is precisely what
|
||||
[the tier rule](skeleton.md) forbids and what the entire bootstrap argument rests on.
|
||||
|
||||
So the danger in the absorption is real, and it is two modules wide rather than six concerns
|
||||
wide.
|
||||
|
||||
## Finding 3 — the split has already been happening, unnamed
|
||||
|
||||
`dnsmasq-app` is the same *kind* of module as `wireguard`: it needs every node's addresses and
|
||||
names. It does not query for them. Its own comments record why:
|
||||
|
||||
> *"the mesh DB already holds [this] in `node_accessors` and the WireGuard address, duplicated
|
||||
> by hand on all four nodes. Renaming the namespace then meant editing four override rows
|
||||
> nobody [knew about]."*
|
||||
|
||||
and
|
||||
|
||||
> *"now derived from `node_accessors`"*
|
||||
|
||||
That is one module having already made the move the skeleton proposes — deciding centrally,
|
||||
applying locally — for the ordinary reason that hand-duplicated state went wrong. Nobody named
|
||||
it as an architectural direction; it was reached by fixing a bug.
|
||||
|
||||
Eight of ten adapters are on the far side of that migration. Two are not.
|
||||
|
||||
## What this means for the open question
|
||||
|
||||
**The absorption is not a move. It is a split, and it is mostly already done.**
|
||||
|
||||
The question "does the host become too large" assumed six concerns would arrive whole. They do
|
||||
not. Each divides:
|
||||
|
||||
- **deciding** — what this node's overlay, names, exposure and filtering should be, which needs
|
||||
every other node and therefore belongs in tier 2;
|
||||
- **applying** — putting that on the machine, which needs root and locality and therefore
|
||||
belongs in tier 0.
|
||||
|
||||
Tier 0 absorbs the applying. That is ~2 755 lines today, most of it already dependency-free,
|
||||
against 3 059 lines of apply machinery the host needs regardless.
|
||||
|
||||
**The claim survives, with its scope corrected.** The host does not carry six concerns; it
|
||||
carries one — *apply declared state on this machine* — of which the six are instances. That is
|
||||
the skeleton's own tier-0 test (*"does it apply state on a machine?"*) applied to itself.
|
||||
|
||||
## What remains open
|
||||
|
||||
- **The two unsplit modules are the two hardest.** Overlay needs every node's key, address,
|
||||
site and endpoint reachability; the proxy needs certificates and every node's exposed names.
|
||||
These are where "derive centrally, apply locally" is most work, and neither has been done.
|
||||
The measurement says the design is right; it does not say the migration is cheap.
|
||||
- **Six concerns is still six vocabularies.** Absorbing them adds no dependencies but does add
|
||||
surface: the host must know what a WireGuard peer, an nftables rule, a package, a unit, a
|
||||
container and a dataset *are*. Nothing here measures that cost, and it is the residue of the
|
||||
original worry.
|
||||
- **What the host must carry versus what it must find.** The host manages `wg`, `nft`,
|
||||
`pacman`, `docker`; it does not contain them.
|
||||
[Issue 007](../../04-ISSUES/007-an-installed-package-is-not-a-capability/00-report.md) is
|
||||
exactly this question and is unresolved.
|
||||
@@ -0,0 +1,281 @@
|
||||
---
|
||||
effort: 006-mesh-from-scratch
|
||||
updated: 2026-08-23
|
||||
---
|
||||
|
||||
# The skeleton
|
||||
|
||||
Repositories at the root, modules inside them, parts at the leaf. Four tiers, and a dependency
|
||||
rule that only points downward.
|
||||
|
||||
## What a tier is
|
||||
|
||||
A tier answers one question: **what has to exist before this can exist?** It is not importance,
|
||||
and it is not a layer in the networking sense. It is bootstrap order, made explicit.
|
||||
|
||||
Walk a bare machine to a running mesh and the tiers fall out of the story:
|
||||
|
||||
```
|
||||
bare machine
|
||||
│ one command lands ONE binary. Nothing else exists. TIER 0 host
|
||||
│
|
||||
│ it reads a pinned file it already carries and raises a
|
||||
│ database, a bus, an object store and an image
|
||||
│ registry — locally, alone. TIER 1 substrate
|
||||
│
|
||||
│ on those, the mesh's brain starts: which nodes exist,
|
||||
│ what runs where, what is reachable. TIER 2 control plane
|
||||
│
|
||||
│ ways to talk to that brain. TIER 3 surfaces
|
||||
│
|
||||
└ everything the mesh then carries. TIER 4 workloads
|
||||
```
|
||||
|
||||
**Dependencies point only downward.** The substrate never references the control plane. That
|
||||
one constraint is the entire bootstrap answer, because it guarantees there is always a place to
|
||||
start.
|
||||
|
||||
Why it earns its keep here: **the current mesh violates this, and that is the circularity that
|
||||
keeps recurring.** The mesh database is a module; modules are installed by the delivery
|
||||
pipeline; the pipeline needs the database. No order works, so a first-node script exists to
|
||||
paper over it, and every later substrate change has to pretend the problem is not there.
|
||||
|
||||
A tier is not a repository and not a bounded context. Those are different cuts: a context says
|
||||
*who owns this concept*, a tier says *what must already be running*.
|
||||
|
||||
## The tree
|
||||
|
||||
```
|
||||
mesh-host/ TIER 0 — the only thing ever installed by hand
|
||||
apply/ reconcile declared state on this machine
|
||||
inventory/ what this node is, has, and is capable of
|
||||
link/ the single outbound connection to the control plane
|
||||
store/ embedded local state — authoritative while disconnected
|
||||
profile/ capability detection: managed · user · edge
|
||||
substrate.lock pinned tier-1 descriptor, appliable with no mesh present
|
||||
|
||||
mesh-substrate/ TIER 1 — declarations only, no logic of its own
|
||||
store/ relational state
|
||||
bus/ commands and events
|
||||
objects/ blobs and build artifacts
|
||||
images/ container images
|
||||
bundle.yml the pinned set tier 0 can raise alone
|
||||
|
||||
mesh-control/ TIER 2 — the control plane
|
||||
record/ the event log every context integrates through
|
||||
inventory/ nodes · modules · assignments · versions
|
||||
config/ settings · secrets · derivation onto nodes
|
||||
connectivity/ overlay · resolution · exposure · filtering · certificates
|
||||
provisioning/ resource grants between modules
|
||||
delivery/ source → artifact → node
|
||||
observability/ health · logs · metrics · alerts
|
||||
identity/ agents · humans · services · authorisation
|
||||
work/ tasks · workflows · runs
|
||||
knowledge/ memory · documents · retrieval
|
||||
api/ the one interface every surface speaks to
|
||||
|
||||
mesh-surfaces/ TIER 3 — thin; no logic lives here
|
||||
tools/ the agent-facing tool surface
|
||||
web/ the operator-facing interface
|
||||
cli/ the shell-facing interface
|
||||
|
||||
mesh-catalog/ TIER 4 — what the mesh hosts
|
||||
<domain>/ grouped per ADR 0022, list per research 005
|
||||
|
||||
mesh-lab/ the whole mesh, disposable, on one machine
|
||||
mesh-sdk/ contracts shared across tiers — types, not behaviour
|
||||
|
||||
hq/ company-scoped, not a mesh repository — ADR 0019
|
||||
```
|
||||
|
||||
## The dependency rule
|
||||
|
||||
**A tier may depend only on tiers below it.** Substrate never references the control plane.
|
||||
The control plane never reaches into a node except through the host. A surface holds no logic
|
||||
a second surface would have to reimplement.
|
||||
|
||||
This is the whole of the bootstrap answer, and per this repository's own rule it must say how
|
||||
it is checked: a dependency-direction lint in the build, failing on an upward import. A tier
|
||||
rule enforced by intention is the same as no tier rule — that is
|
||||
[ADR 0010](../../02-DECISIONS/0010-delivery.md) applied to architecture.
|
||||
|
||||
## Move 1 — the substrate is applied, not delivered
|
||||
|
||||
**The problem.** The mesh needs a database, a bus, an object store and an image registry.
|
||||
Today those are modules, and modules are installed by the delivery pipeline, which
|
||||
needs the database and the bus. The first node is therefore raised by a special script that
|
||||
exists only because of the circularity, and every later change to the substrate has to pretend
|
||||
the circularity is not there.
|
||||
|
||||
**The move.** The host can apply a declaration without anyone telling it to. The substrate is
|
||||
a **pinned bundle** the host carries: a fixed, versioned, self-contained descriptor of the
|
||||
four services and nothing else. Raising a first node is `host apply substrate.lock` — not a
|
||||
special path, just the ordinary one with no control plane on the other end.
|
||||
|
||||
The circularity disappears rather than being worked around: **the substrate is applied by tier
|
||||
0, the mesh is delivered by tier 2, and they are different mechanisms on purpose.**
|
||||
|
||||
The price is real and should be named: the substrate is upgraded by bumping a pin and
|
||||
re-applying, not by the pipeline. It gets less machinery than everything else — no per-node
|
||||
selection, no provisioning, no fan-out — and that is the point. Five services justify a
|
||||
simpler mechanism than a hundred.
|
||||
|
||||
## Move 2 — the host is one binary with capability profiles
|
||||
|
||||
**The problem.** Everything assumes root on a machine whose packages, services and network the
|
||||
mesh owns. A phone cannot offer that, and neither can a work laptop. The current answer would
|
||||
be a lightweight fork, which means two implementations and one of them rotting.
|
||||
|
||||
**The move.** One host binary, and a **profile** it detects rather than is told:
|
||||
|
||||
| Profile | Can | Typical |
|
||||
|---|---|---|
|
||||
| `managed` | packages, services, network, filesystem — the full surface | a machine the mesh owns |
|
||||
| `user` | user-level services and tools; no package or network management | a shared or administered machine |
|
||||
| `edge` | report presence, relay, expose a tool surface; hold nothing | a phone |
|
||||
|
||||
A module declares which profiles it can land on. Assignment to an incapable node fails at
|
||||
declaration time, not at deploy time — a phone is not a machine that fails to install a
|
||||
firewall; it is a node the firewall module cannot be assigned to.
|
||||
|
||||
This makes the phone case a **capability question rather than a platform question**, which is
|
||||
what keeps it from becoming a second implementation. It also removes the current unstated
|
||||
assumption that every node is equivalent — already false, and today handled by remembering.
|
||||
|
||||
## Move 3 — connectivity becomes a context
|
||||
|
||||
[ADR 0001](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md) names nine contexts
|
||||
and none of them owns the overlay, the resolver, the firewall or the ingress. `config` owns
|
||||
PKI, which is the closest thing, and it is not close.
|
||||
|
||||
Meanwhile [research 005](../005-domain-grouping/analysis.md) measured the whole catalogue and
|
||||
found that reachability is the **only** place where modules genuinely change together under one
|
||||
intent — the proxy with the resolver, the firewall with the overlay, repeatedly, because *how a
|
||||
node is reachable* is one question asked in four places.
|
||||
|
||||
So the evidence and the gap point the same way. `connectivity` owns:
|
||||
|
||||
- the overlay every node joins, and the addresses on it
|
||||
- name resolution, internal and public
|
||||
- exposure — which services answer from outside, on which names
|
||||
- filtering — what may reach a node at all
|
||||
- certificates for both name spaces
|
||||
|
||||
This is an addition to an accepted record, so it is a decision, not a drafting choice. It
|
||||
belongs in a new record that extends ADR 0001 the way
|
||||
[ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md) does —
|
||||
not written here.
|
||||
|
||||
### Where the networking actually lives
|
||||
|
||||
A context is an authority, not a running thing, so naming one does not say who brings the
|
||||
overlay up. That splits three ways, and the split is the design:
|
||||
|
||||
| Concern | Tier | Why there |
|
||||
|---|---|---|
|
||||
| **Overlay membership** — this node joins, holds an address, keeps the tunnel up | **0, in the host** | Everything cross-node needs it *before* the substrate is reachable from elsewhere. A module cannot provide it, because installing a module is itself a cross-node operation. |
|
||||
| **Policy** — who holds which address, what resolves, what is exposed, what is filtered | **2, `connectivity`** | Bookkeeping and authority. It decides; it runs nothing. |
|
||||
| **Machinery** — resolver, reverse proxy, firewall backend, certificate issuance | **4, modules** | Swappable, and not every node needs them. A node without a reverse proxy is still a node. |
|
||||
|
||||
There is a second circularity hiding here, and it has to be closed explicitly: **the host's link
|
||||
to the control plane does not run over the overlay.** If it did, the overlay would have to be up
|
||||
before the host could be told how to join it. The link is ordinary outbound internet to a public
|
||||
endpoint; the overlay carries node-to-node traffic only.
|
||||
|
||||
Joining is therefore: host lands → links out with a join token → control plane returns an
|
||||
address and keys → host raises membership → the substrate on other nodes becomes reachable.
|
||||
|
||||
This also makes the `edge` profile honest rather than special-cased. A phone can hold overlay
|
||||
membership in userspace without privilege, and cannot run the machinery. That is the profile
|
||||
distinction doing its job.
|
||||
|
||||
## Move 4 — `feature` splits in two
|
||||
|
||||
The invitation was to check whether the concept survives. It does not, in one piece.
|
||||
|
||||
Today a **feature** means both *a thing built once* and *a thing selected per node*, and the
|
||||
delivery pipeline is hard to reason about precisely because those have different cardinality
|
||||
and one word ([ADR 0010](../../02-DECISIONS/0010-delivery.md)
|
||||
is the pipeline half of the same confusion).
|
||||
|
||||
Split it:
|
||||
|
||||
| Concept | Is | Cardinality |
|
||||
|---|---|---|
|
||||
| **artifact** | something built and published — an image, a bundle, a package | once per module version |
|
||||
| **part** | an independently selectable piece of a module's desired state | chosen per node, per assignment |
|
||||
|
||||
A module declares desired state in parts, and produces artifacts. An assignment names the node
|
||||
and the parts. Delivery builds artifacts once and applies parts per node — and the two words
|
||||
now carry the two cardinalities that the pipeline already has.
|
||||
|
||||
This keeps what features are genuinely for — per-node opt-in of *some* of a module, which the
|
||||
work breakdown already calls for — and drops the conflation that makes the current model
|
||||
confusing.
|
||||
|
||||
## What the mesh is, versus what it hosts
|
||||
|
||||
Tiers 0–3 are the mesh. Tier 4 is everything it carries, and the boundary is stated by
|
||||
requirement rather than by taste: **a module is part of the mesh if removing it stops the mesh
|
||||
managing nodes.** A media server does not. A relational store does — which is why it sits in the
|
||||
substrate and not the catalogue, despite being, in every other respect, an application like any
|
||||
other. An identity provider, notably, does **not**: the control plane authenticates its own
|
||||
callers, so identity is a hosted service like the media server.
|
||||
|
||||
That test also settles the IT-company goal without a special category. Development, design and
|
||||
deployment tooling are **workloads** — tier 4, hosted, provisioned, delivered like anything
|
||||
else. The mesh does not grow a "company" feature; it hosts the tools a company runs on, and
|
||||
the fact that it runs its own development on them is dogfooding, not architecture.
|
||||
|
||||
## What agents are, structurally
|
||||
|
||||
Self-improvement and self-healing are not a tier. Agents are participants
|
||||
([ADR 0003](../../02-DECISIONS/0003-agents-are-persistent-employees.md)) that hold identity in
|
||||
tier 2, act through tier 3 like any other caller, and run as workloads in tier 4.
|
||||
|
||||
This matters for one reason: **an agent must not have a privileged path**. Anything an agent
|
||||
can do to the mesh, a person can do through the same surface, and anything it cannot express
|
||||
through the tool surface is a gap in the surface rather than a reason for a back door. Self-
|
||||
healing built on a private channel is unreviewable, and would be the one part of the mesh with
|
||||
no human checkpoint.
|
||||
|
||||
## How this is tested
|
||||
|
||||
The lab ([ADR 0016](../../02-DECISIONS/0016-the-lab.md)) raises the
|
||||
tree above on one machine: virtual machines as nodes, a real overlay between them, the real
|
||||
substrate bundle, the real control plane, the real delivery path.
|
||||
|
||||
The tier rule is what makes that affordable. A scenario needing only tiers 0 and 1 is one
|
||||
virtual machine and a pinned bundle — which is also, exactly, the bootstrap path. **The
|
||||
hardest thing to test becomes the cheapest scenario to run**, and the first-node path stops
|
||||
being the one thing nobody exercises until it breaks.
|
||||
|
||||
## What this skeleton does not answer
|
||||
|
||||
- Where the record lives. It is infrastructure by shape and domain by content, and putting it
|
||||
in the substrate risks recreating a circularity in the one place the design just removed one.
|
||||
- Whether tier 2's contexts are one repository or several. Open from ADR 0001 already.
|
||||
- Whether an `edge` node is in the inventory or merely present — which decides whether "node"
|
||||
is one concept or two.
|
||||
- The migration. Nothing here says how today's mesh becomes this, and the skeleton is worth
|
||||
little until that is costed.
|
||||
|
||||
## A naming near-miss, recorded
|
||||
|
||||
The tier-0 binary was first called `mesh-agent`, because "node agent" is the reflex everywhere
|
||||
else in the industry. That is wrong here, and wrong in the specific way
|
||||
[`how-we-build.md`](../../00-META/how-we-build.md) §4 exists to catch: **Agent** is a
|
||||
first-class concept in this mesh — a participant, some of whom are human, holding identity and
|
||||
memory ([ADR 0003](../../02-DECISIONS/0003-agents-are-persistent-employees.md)). One document
|
||||
carried both meanings.
|
||||
|
||||
It is the same failure as the anatomy naming in the current runtime: an evocative domain word
|
||||
pointing at infrastructure.
|
||||
|
||||
[ADR 0001](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md) supplies the fix in
|
||||
its own title — *the mesh brokers capabilities; nodes host; agents think.* Three verbs, three
|
||||
components: the control plane **brokers** (`mesh-control`), the tier-0 binary **hosts**
|
||||
(`mesh-host`), the participant **thinks** (`agents`, untouched).
|
||||
|
||||
`mesh-node` was the alternative and was rejected: *Node* is the aggregate in the inventory — the
|
||||
record of a machine — while the binary is what runs on it and does the hosting.
|
||||
@@ -0,0 +1,51 @@
|
||||
---
|
||||
status: active
|
||||
initiated: 2026-08-23
|
||||
touches:
|
||||
- 03-DESIGN/00-as-is/03-provisioning.md
|
||||
- 02-DECISIONS/0009-modules-and-the-graph.md
|
||||
- 01-RESEARCH/006-mesh-from-scratch/code-skeleton.md
|
||||
became: []
|
||||
---
|
||||
|
||||
# 007 — Provisioning as the mesh's core mechanism
|
||||
|
||||
## What is being investigated
|
||||
|
||||
Provisioning is the mechanism the whole mesh rests on: a module declares what it needs, and the
|
||||
mesh makes it exist, generates the credential, records the grant, and puts the values where the
|
||||
module will read them. [ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md)
|
||||
calls it the mesh's core concern rather than its plumbing.
|
||||
|
||||
[Research 006](../006-mesh-from-scratch/code-skeleton.md) then asks it to carry **more**: the
|
||||
control plane becomes a consumer with its own requirements — a source of record, an image
|
||||
registry, a package registry — satisfied by the same mechanism. That generalisation is only
|
||||
safe if the mechanism is sound, and the as-is record says it is not, in named ways.
|
||||
|
||||
## Why now
|
||||
|
||||
Four weaknesses are already documented in
|
||||
[`03-DESIGN/00-as-is/03-provisioning.md`](../../03-DESIGN/00-as-is/03-provisioning.md), each
|
||||
observed rather than theorised:
|
||||
|
||||
- **Rotation has no fan-out.** A shared credential can be rotated without telling the peers
|
||||
holding the old one. This has locked the mesh out of its own broker.
|
||||
- **A grant is not a check.** The record says a resource was provisioned. Nothing verifies it
|
||||
still exists, still has that credential, or is reachable from where the consumer runs.
|
||||
- **A frozen password outlives its generation.** A generated secret written once diverges from a
|
||||
persistent data directory initialised earlier, and presents as an authentication error.
|
||||
- **No requirements is indistinguishable from provisioning that did not run.** A module that
|
||||
declares nothing skips the stage, which is correct, and looks identical to failure.
|
||||
|
||||
Generalising a mechanism with these properties to the control plane's own dependencies would
|
||||
make each of them fatal rather than annoying.
|
||||
|
||||
## The questions
|
||||
|
||||
| Question | Why it matters |
|
||||
|---|---|
|
||||
| What does a **grant** mean, exactly — a record that a resource was created, or a claim about the world that is continuously reconciled? | The difference between the current model and one where "provisioned" is checkable. Almost every weakness above is a symptom of the first answer. |
|
||||
| How is a credential **rotated** with fan-out to every holder? | The mechanism grants easily and regrants not at all. This is the most damaging gap and it has taken the mesh down. |
|
||||
| Can the **control plane** hold requirements, and what satisfies them before anything is installed? | The generalisation research 006 needs. Ties directly to the self-hosting transition. |
|
||||
| What happens when a requirement **cannot** be satisfied — no provider, provider on an unreachable node, provider not yet installed? | Today this is silent or a stall. It should be a stated, visible state. |
|
||||
| Does a requirement belong to a **module** or to one of its **parts**? | Research 006 splits `feature` into artifact and part. A part-scoped requirement means a database is not provisioned where the part that needs it is not installed. |
|
||||
@@ -0,0 +1,101 @@
|
||||
---
|
||||
status: graduated
|
||||
initiated: 2026-08-23
|
||||
touches:
|
||||
- 03-DESIGN/00-as-is/04-delivery.md
|
||||
- 02-DECISIONS/0010-delivery.md
|
||||
- 02-DECISIONS/0010-delivery.md
|
||||
- 01-RESEARCH/006-mesh-from-scratch/code-skeleton.md
|
||||
became:
|
||||
- 02-DECISIONS/0010-delivery.md
|
||||
- 02-DECISIONS/0010-delivery.md
|
||||
---
|
||||
|
||||
# 008 — The coordinator: a change checked in becomes a deployed state
|
||||
|
||||
## What is being investigated
|
||||
|
||||
The mesh's own continuous delivery: a change is committed, and the mesh ends up in the state
|
||||
that change describes — across every node the change touches, with a verdict that says whether
|
||||
it worked.
|
||||
|
||||
The coordinator is what orchestrates that, and it is the mesh's most consequential machinery:
|
||||
everything reaches every node through it.
|
||||
|
||||
## Why now
|
||||
|
||||
The as-is record ([`03-DESIGN/00-as-is/04-delivery.md`](../../03-DESIGN/00-as-is/04-delivery.md))
|
||||
names problems that are structural rather than incidental:
|
||||
|
||||
- **A green pipeline proves transport, not effect.** The stages report that a message was
|
||||
dispatched and accepted, which is not the same as the thing running, correct, or present.
|
||||
This is the mesh's single most consistent failure shape.
|
||||
- **Detection is the most fragile input.** A merge that creates no pipeline, with nothing saying
|
||||
so, is the characteristic bad outcome — and it has happened for reasons unrelated to the
|
||||
change.
|
||||
- **The fan-out point is asymmetric.** The build node has already passed two silos when work
|
||||
fans out, and code that knew only about the first parked it forever while every other node
|
||||
deployed cleanly.
|
||||
- **There is no end-to-end coverage.** The harness has not built since 2026-06-04
|
||||
([`04-ISSUES/005`](../../04-ISSUES/005-pipeline-test-harness-unbuildable/00-report.md)).
|
||||
|
||||
Research 006 adds a requirement the current design does not have: the coordinator must work
|
||||
**before the mesh is self-hosting**, when source and artifacts come from outside, and keep
|
||||
working across the transition to self-hosted providers.
|
||||
|
||||
## What it became
|
||||
|
||||
*Closed 2026-08-28.* All six questions are answered, by two records, and the second exists
|
||||
because the first was honest about what it did not fix.
|
||||
|
||||
**Does the coordinator dispatch stages, or converge nodes on a declaration?** — *Converge.*
|
||||
[ADR 0010](../../02-DECISIONS/0010-delivery.md): a pipeline ends when the
|
||||
declaration is updated, and the host applies it and reads back — so the reporter is the applier.
|
||||
|
||||
**Does the three-silo split survive?** — *Yes, with the third redefined.* The cardinality
|
||||
observation holds; the third silo is not a stage any more.
|
||||
|
||||
**How does a change become a pipeline, reliably?** — *It does not become a pipeline at all.*
|
||||
[ADR 0010](../../02-DECISIONS/0010-delivery.md) applies 0058's
|
||||
move one level up: the control plane holds what source exists and what has been built, and builds
|
||||
the difference. **An event makes it fast; nothing makes it necessary.** The failures this effort
|
||||
catalogued — a truncated commit list, a broken path match — become latency rather than silence.
|
||||
|
||||
**What is a deployed state?** — *Two comparisons, not an event.* Does every node's reported state
|
||||
match what is declared, and is what is declared built from current source? A milestone can be
|
||||
claimed by something that did not check; a comparison cannot.
|
||||
|
||||
**What produces a verdict, and what is it about?** — *An artifact, and it gates eligibility.* The
|
||||
mesh must not converge onto something broken, so an artifact may be declared only once the lab
|
||||
has judged it fit. Sharper than the question expected: a verdict is a property an artifact has,
|
||||
not a report about a run.
|
||||
|
||||
**How does delivery work before self-hosting?** — *It mostly stops being a question.* A reconciler
|
||||
reads source and writes artifacts; where those live is a binding, external first and internal
|
||||
later. The transition looked hard because a pipeline's stages name their targets.
|
||||
|
||||
## What this effort was right about
|
||||
|
||||
Its first question — *what is a deployed state, and how does the mesh know it is in one* — was
|
||||
marked "everything follows from this", and everything did. Both records above are answers to it:
|
||||
0058 makes the applier the reporter, and 0063 makes currency a comparison. The effort put the
|
||||
load-bearing question first.
|
||||
|
||||
## What is NOT closed by this
|
||||
|
||||
[ADR 0010](../../02-DECISIONS/0010-delivery.md) names four costs
|
||||
and one of them is a real risk rather than a trade: **a reconciler that cannot reach its target
|
||||
retries forever, and without something that notices, the failure is silence** — which is the
|
||||
fault this effort exists to catalogue, reintroduced in a new place. That belongs to observability
|
||||
and it is not designed.
|
||||
|
||||
## The questions (all answered above)
|
||||
|
||||
| Question | Why it matters |
|
||||
|---|---|
|
||||
| What is a **deployed state**, and how does the mesh know it is in one? | Everything follows from this. If a stage reports transport, "deployed" is a claim nobody checked. A desired-state model with reconciliation gives a different answer from a job-completion model. |
|
||||
| Does the coordinator dispatch **stages**, or converge nodes on a **declaration**? | The current model is a state machine over stages. The alternative is that a node is told what should be true and reports what is. The second makes drift visible; the first cannot see it. |
|
||||
| How does a change **become** a pipeline, reliably? | Detection has failed for reasons unrelated to the change, silently. |
|
||||
| What produces a **verdict**, and what is it a verdict about? | Ties to the lab ([ADR 0016](../../02-DECISIONS/0016-the-lab.md)) and to a module carrying its own assertions. |
|
||||
| How does delivery work **before self-hosting**, and across the transition? | From research 006: source and artifacts start external and are re-bound to internal providers. The coordinator has to be indifferent to which. |
|
||||
| Does the **three-silo** split survive the artifact/part split? | [ADR 0010](../../02-DECISIONS/0010-delivery.md) is cardinality-driven, and research 006 renames the thing the cardinality is about. |
|
||||
@@ -0,0 +1,95 @@
|
||||
---
|
||||
status: active
|
||||
initiated: 2026-08-23
|
||||
touches:
|
||||
- 01-RESEARCH/006-mesh-from-scratch/code-skeleton.md
|
||||
- 03-DESIGN/01-to-be/01-end-to-end-testing.md
|
||||
- 02-DECISIONS/0016-the-lab.md
|
||||
- 03-DESIGN/00-as-is/00-overview.md
|
||||
became: []
|
||||
---
|
||||
|
||||
# 009 — Getting from the mesh that exists to the mesh that is designed
|
||||
|
||||
## What is being investigated
|
||||
|
||||
How a running mesh becomes the one in
|
||||
[research 006](../006-mesh-from-scratch/code-skeleton.md), without losing what it currently
|
||||
carries.
|
||||
|
||||
The proposed shape: **build tiers 0, 1 and 2, then replace the current setup in one move.**
|
||||
|
||||
## Why big-bang is the right instinct here
|
||||
|
||||
Recorded because incremental is the reflex answer and it is wrong in this case.
|
||||
|
||||
- **The two models are structurally incompatible.** The tier rule, the host absorbing what are
|
||||
now modules, the artifact/part split, provisioning generalised to the control plane's own
|
||||
requirements — none of these can half-apply. Running both models at once means the old one's
|
||||
assumptions keep constraining the new one, which is how a migration becomes permanent.
|
||||
- **Nothing external depends on it.** No users outside the operator, no service level to hold.
|
||||
- **The lab exists precisely for this** ([ADR 0016](../../02-DECISIONS/0016-the-lab.md)).
|
||||
A big-bang that has been rehearsed end to end, repeatedly, on identical machines is not the
|
||||
same risk as one performed for the first time on the real mesh. This is also why the lab is
|
||||
phase 0 rather than a verification step later: the new mesh is *developed* inside it, so by
|
||||
cutover the procedure has been run hundreds of times rather than rehearsed a few.
|
||||
- **Incremental would carry the rot forward.** The as-is layer documents silent failure paths,
|
||||
a dead test harness and unenforced rules. A gradual migration preserves them by definition.
|
||||
|
||||
## The distinction that lowers the risk
|
||||
|
||||
**Replace the control plane; do not move the workloads.**
|
||||
|
||||
The things that would hurt to lose — mail, media, source, databases, their data directories —
|
||||
are not the mesh. They are what the mesh manages. They sit in container volumes on nodes, and
|
||||
they do not need to move for the control plane above them to be replaced.
|
||||
|
||||
So the big-bang is: the old control plane stops managing these nodes, and the new one starts —
|
||||
with the workload data untouched, in place, and re-declared rather than migrated.
|
||||
|
||||
That reframing turns "replace the mesh" into "replace the part with no persistent state of its
|
||||
own", which is a materially smaller act than it first sounds.
|
||||
|
||||
## The tension this exposes
|
||||
|
||||
Taking over already-running workloads is **adoption**, and adoption was ruled out of scope —
|
||||
recorded as a legacy path in
|
||||
[`03-DESIGN/01-to-be/01-end-to-end-testing.md`](../../03-DESIGN/01-to-be/01-end-to-end-testing.md).
|
||||
The migration appears to need exactly the capability the design declared it would not have.
|
||||
|
||||
The way out, to be tested: the new mesh does not adopt anything. It **declares** the workloads
|
||||
from scratch and points them at data directories that already exist. Nothing inspects a running
|
||||
machine to learn what is there; the declarations are written from the as-is layer, which is what
|
||||
that layer is for. Data survives because it was never touched, not because it was adopted.
|
||||
|
||||
If that holds, adoption stays out of scope and the migration is ordinary declaration. If it does
|
||||
not, adoption needs a one-time, explicitly unsupported tool, and that should be a decision rather
|
||||
than a discovery.
|
||||
|
||||
## Sequencing
|
||||
|
||||
| Phase | What | Done when |
|
||||
|---|---|---|
|
||||
| **0** | **Build the lab's bootstrap scenario** ([ADR 0016](../../02-DECISIONS/0016-the-lab.md)) — virtual machines, a network, a way to place a binary, snapshot and reset. No forge, no coordinator, no pipeline. | A machine can be raised from nothing, reset, and raised again, repeatably. |
|
||||
| A | Build tier 0, **inside the lab**. The host's interface first — it carries the skeleton's biggest unproven claim. | A bare machine becomes a managed node with no mesh present. |
|
||||
| B | Build tier 1 and 2. The bootstrap scenario grows into the full one by addition — the same machines, with more placed inside them. | The lab raises a full mesh from nothing, repeatedly, from pinned external artifacts. |
|
||||
| C | Enough of tier 3 to operate it. | The mesh can be driven without direct database access. |
|
||||
| D | Declare the existing workloads against the new model. | The lab runs them, with copies of real data shapes. |
|
||||
| E | **Rehearse the cutover in the lab** against a mesh built to resemble the real one. | Repeatable, and repeatably reversible. |
|
||||
| F | Cut over. | The real nodes are managed by the new control plane. |
|
||||
| G | Reach self-hosting — re-bind delivery from external providers to the mesh's own forge and registries. | The mesh builds and deploys itself. |
|
||||
|
||||
Phase G is deliberately last. Per research 006, self-hosting is a state the mesh **reaches**;
|
||||
attempting the cutover and the self-hosting transition in the same move recreates exactly the
|
||||
circularity the skeleton removes — and would mean a failed cutover could take away the means to
|
||||
fix it.
|
||||
|
||||
## The questions
|
||||
|
||||
| Question | Why it matters |
|
||||
|---|---|
|
||||
| What state must **survive** the cutover, versus be re-created? | Workload data must. Provisioned credentials could be re-issued. The mesh's own inventory could be re-declared. Each answer changes the risk. |
|
||||
| What is the **way back**? | A cutover with no rollback is not a plan. If workload data is untouched, reverting may be as small as re-pointing the old control plane at it — to be verified, not assumed. |
|
||||
| How is the cutover **rehearsed** against something resembling the real mesh, without copying the real mesh into a repository? | The lab must be able to model the real topology's shape without carrying its identity. |
|
||||
| Does anything have to keep running **during** the cutover? | Mail and source are the obvious candidates. If yes, "big-bang" is really "big-bang with exceptions", and the exceptions should be named now. |
|
||||
| Is the forge inside or outside the cutover? | If the mesh's own forge goes down with the old control plane, the means of deploying a fix goes with it. This is the self-hosting circularity appearing as a migration risk. |
|
||||
@@ -0,0 +1,60 @@
|
||||
---
|
||||
status: active
|
||||
initiated: 2026-08-24
|
||||
touches:
|
||||
- 03-DESIGN/01-to-be/03-scenario-lifecycle.md
|
||||
- 02-DECISIONS/0016-the-lab.md
|
||||
- 02-DECISIONS/0016-the-lab.md
|
||||
became: []
|
||||
---
|
||||
|
||||
# 010 — What the lab's inner loop actually costs
|
||||
|
||||
## What is being investigated
|
||||
|
||||
[The lifecycle design](../../03-DESIGN/01-to-be/03-scenario-lifecycle.md) closes on an open
|
||||
question that is measurable rather than arguable:
|
||||
|
||||
> **What a snapshot costs.** Whole-scenario snapshots of several machines are the operation the
|
||||
> inner loop repeats most, so their cost sets the loop's speed. If restoring is slow, the loop
|
||||
> is slow, and everything above is theory.
|
||||
|
||||
Measured on a workstation, 2026-08-24. Numbers in [`measurements.md`](measurements.md).
|
||||
|
||||
## Why it matters
|
||||
|
||||
[ADR 0016](../../02-DECISIONS/0016-the-lab.md) makes the
|
||||
bootstrap scenario the inner development loop for tiers 0 and 1 — the argument being that
|
||||
raising a node from nothing stops being the least-exercised path and becomes the most-exercised
|
||||
one. **That argument is only true if raising and resetting are cheap.** A loop that costs
|
||||
minutes is a loop people avoid, and the least-exercised path stays least-exercised.
|
||||
|
||||
## Status
|
||||
|
||||
Measured, and the finding is a blocker rather than a data point.
|
||||
|
||||
**A snapshot on the current host is a full copy of the machine's disk.** 1.6 GB and ten seconds
|
||||
for one small virtual machine at best — and over two minutes when observed a second time. Cost
|
||||
scales with the number of machines and the size of their disks, not with what changed.
|
||||
|
||||
**With copy-on-write it is 0.13 seconds and costs the delta.** Verified, not assumed. The
|
||||
projected four-machine reset cycle falls from roughly ninety seconds to roughly fifteen, of
|
||||
which almost all is a boot that cannot be avoided.
|
||||
|
||||
The cause is not virtual machines and not incus. It is that the host offers incus exactly one
|
||||
storage driver, `dir`, which has no copy-on-write and therefore no cheap snapshot. The kernel
|
||||
supports btrfs; the userspace tool that would let incus use it is simply not installed.
|
||||
|
||||
So the question *"is the lab's inner loop fast enough"* currently answers itself the wrong way,
|
||||
for a reason that is one declared package away from being fixed — and declaring packages is
|
||||
something the mesh already does.
|
||||
|
||||
## Open questions
|
||||
|
||||
| Question | Why it matters |
|
||||
|---|---|
|
||||
| ~~How much does a copy-on-write pool actually improve it?~~ **Measured: snapshot 9.9 s → 0.13 s, restore 10.4 s → 0.80 s, three snapshots sharing 1.36 GB rather than costing 4.8 GB.** The projected four-machine cycle falls from ~90 s to ~15 s. | Answered. The inner-loop argument holds *with* copy-on-write and did not without it. |
|
||||
| How does the lab install its own prerequisites on a clean machine? | The lab needs a virtualisation daemon, copy-on-write tooling and a pool before it can do anything — and it cannot depend on the mesh for them, since it is where the mesh is built. |
|
||||
| Why was the second snapshot more than twelve times slower than the first? | If snapshot cost is unpredictable rather than merely high, that is worse — a loop with a variable multi-minute step is one nobody trusts. |
|
||||
| Does a scenario snapshot need the machines stopped? | Stateless snapshots of a running virtual machine capture the disk but not memory. Whether a mesh restored that way is coherent is not established. |
|
||||
| What is the cost at scenario scale — four machines rather than one? | Only single-machine numbers were taken. If the operation is serial, four machines is four times the wait. |
|
||||
@@ -0,0 +1,141 @@
|
||||
---
|
||||
effort: 010-lab-inner-loop-cost
|
||||
updated: 2026-08-24
|
||||
---
|
||||
|
||||
# Measurements
|
||||
|
||||
Taken 2026-08-24 on a workstation with hardware virtualisation available, an NVMe-backed ext4
|
||||
root, and 300 GB free. One virtual machine, 1 GiB memory, 2 CPUs, from a cached distribution
|
||||
image.
|
||||
|
||||
## The environment, before anything ran
|
||||
|
||||
| Fact | Value | Consequence |
|
||||
|---|---|---|
|
||||
| Hardware virtualisation | present | virtual machines run at native speed; the choice in [ADR 0016](../../02-DECISIONS/0016-the-lab.md) is not paying an emulation penalty |
|
||||
| Storage drivers the daemon offers | **`dir` only** | no copy-on-write, therefore no cheap snapshot |
|
||||
| Host filesystems | ext4 throughout | nothing copy-on-write to put a pool on |
|
||||
| btrfs kernel module | **available** | the kernel can do it |
|
||||
| `btrfs-progs` | **not installed** | which is the entire reason the driver is absent |
|
||||
|
||||
The last two rows are the finding. The daemon advertises only `dir` because the userspace tool
|
||||
for anything better is missing — not because the host cannot do better.
|
||||
|
||||
## Raising a machine
|
||||
|
||||
| Step | Time |
|
||||
|---|---|
|
||||
| launch call returns | **3.4 s** |
|
||||
| machine actually usable — a command executes on it | **14.3 s** |
|
||||
|
||||
The gap matters for the lifecycle design: `raise` returning is not the same as the scenario
|
||||
being ready, so the verb has to wait for the second number, not report the first. Reporting
|
||||
the first would be the mesh's own recurring failure — transport reported as effect.
|
||||
|
||||
## Snapshot and restore
|
||||
|
||||
| Operation | Time | Disk |
|
||||
|---|---|---|
|
||||
| snapshot, first | **9.9 s** | **+1.6 GB** |
|
||||
| snapshot, second | **> 120 s — did not complete** | — |
|
||||
| restore call returns | **10.4 s** | — |
|
||||
| machine usable again | **20.1 s** total | — |
|
||||
|
||||
Instance on disk before snapshotting: 1.5 GB. Snapshot directory afterwards: 1.6 GB. **A `dir`
|
||||
snapshot is a full copy** — the storage cost equals the instance, and nothing is shared.
|
||||
|
||||
Implied copy throughput on the first snapshot is roughly 160 MB/s, which is far below what the
|
||||
underlying NVMe can do and is consistent with a real, durable copy rather than a metadata
|
||||
operation.
|
||||
|
||||
**The second snapshot is the more troubling number.** It exceeded two minutes and was still
|
||||
running when the observation was cut off; only the first snapshot exists. Whatever the cause —
|
||||
page cache exhausted by the preceding restore, writeback contention — the practical
|
||||
consequence is that snapshot cost here is **not merely high, it is unpredictable**.
|
||||
|
||||
## What this projects to
|
||||
|
||||
A four-machine scenario, taking the optimistic single-machine numbers and assuming the
|
||||
operations are serial:
|
||||
|
||||
| | one machine | four machines |
|
||||
|---|---|---|
|
||||
| raise, to usable | 14 s | ~57 s |
|
||||
| snapshot | 10 s, 1.6 GB | ~40 s, 6.4 GB |
|
||||
| restore, to usable | 20 s | ~80 s |
|
||||
|
||||
A reset-and-rerun cycle is therefore **around a minute and a half at best**, and unbounded at
|
||||
worst, before any of the mesh's own work begins.
|
||||
|
||||
## The judgement
|
||||
|
||||
**This is too slow for an inner loop**, and the reason is not the design.
|
||||
|
||||
[ADR 0016](../../02-DECISIONS/0016-the-lab.md) argues that
|
||||
making the bootstrap path the inner development loop turns the least-exercised code in the
|
||||
system into the most-exercised. That argument holds only while resetting is cheap. At a minute
|
||||
and a half a cycle, with occasional multi-minute stalls, the loop is one a person works around
|
||||
— and the path stays under-exercised for exactly the reason it always was.
|
||||
|
||||
Nothing about virtual machines causes this. Hardware virtualisation is present and the machines
|
||||
boot in fourteen seconds. **The cost is entirely the storage driver**, and the driver is absent
|
||||
because one userspace package is not installed on the host.
|
||||
|
||||
The mesh already has the mechanism for that: a module declares a package, and a hook makes it a
|
||||
working capability — which is precisely what was just done for the virtualisation daemon
|
||||
itself, and what [`04-ISSUES/007`](../../04-ISSUES/007-an-installed-package-is-not-a-capability/00-report.md)
|
||||
is about.
|
||||
|
||||
## The fix, measured
|
||||
|
||||
The comparison was subsequently run. One package — `btrfs-progs`, no dependencies — installed by
|
||||
hand, the daemon restarted so it re-detected drivers, a copy-on-write pool created on a loop
|
||||
file, and the identical image launched onto it.
|
||||
|
||||
| Operation | `dir` | copy-on-write | |
|
||||
|---|---|---|---|
|
||||
| snapshot | 9.9 s, then **> 120 s** | **0.13 s** | ~76× faster, and *consistent* |
|
||||
| snapshot again | — | 0.12 s | |
|
||||
| snapshot a third time | — | 0.13 s | |
|
||||
| restore call | 10.4 s | **0.80 s** | ~13× faster |
|
||||
| restore, to usable | 20.1 s | **10.5 s** | the remainder is boot, which is irreducible |
|
||||
| three snapshots, storage | ~4.8 GB | **1.36 GB total, shared** | cost is the delta, not the disk |
|
||||
|
||||
**The fix is real, and larger than expected.** Snapshot goes from ten seconds to a tenth of a
|
||||
second, and — more importantly — from *wildly variable* to *flat*. Three consecutive snapshots
|
||||
took 0.13, 0.12 and 0.13 seconds. On `dir` the second snapshot never finished.
|
||||
|
||||
Storage stops scaling with the machine and starts scaling with what changed: three snapshots of
|
||||
a 1.5 GB instance occupied 1.36 GB in total, because they share.
|
||||
|
||||
### What it projects to
|
||||
|
||||
A four-machine reset-and-rerun cycle, the operation the inner loop repeats most:
|
||||
|
||||
| | `dir` | copy-on-write |
|
||||
|---|---|---|
|
||||
| snapshot the scenario | ~40 s, 6.4 GB | **~0.5 s**, delta-sized |
|
||||
| restore it | ~40 s + boot | **~3 s** + boot |
|
||||
| **cycle** | **~90 s, unbounded at worst** | **~15 s, dominated by boot** |
|
||||
|
||||
At fifteen seconds, dominated by a boot that cannot be avoided, the inner loop is viable and
|
||||
[ADR 0016](../../02-DECISIONS/0016-the-lab.md)'s argument holds.
|
||||
At ninety it did not.
|
||||
|
||||
### One honest counter-observation
|
||||
|
||||
Launching onto the fresh copy-on-write pool was **slower** — 20.2 s to usable against 14.3 s —
|
||||
because the image had to be unpacked into a pool that had never seen it. That cost is paid once
|
||||
per pool, not per scenario, and it is dwarfed by what snapshotting saves. But it is a real
|
||||
number and it went the other way.
|
||||
|
||||
### State this left behind
|
||||
|
||||
Recorded because hand-made state is exactly what the mesh's rules exist to prevent, and it must
|
||||
be declared properly rather than left as an artefact of a measurement:
|
||||
|
||||
- `btrfs-progs` installed by hand. Its installation regenerated the boot initramfs, a side
|
||||
effect worth knowing about.
|
||||
- The daemon restarted once, to re-detect drivers.
|
||||
- The test pool and instance were **removed**; the pool the lab actually needs does not exist.
|
||||
@@ -0,0 +1,203 @@
|
||||
---
|
||||
status: graduated
|
||||
initiated: 2026-08-25
|
||||
became:
|
||||
- 02-DECISIONS/0009-modules-and-the-graph.md
|
||||
- 02-DECISIONS/0008-a-context-owns-its-store.md
|
||||
- 03-DESIGN/01-to-be/06-the-controller.md
|
||||
- 03-DESIGN/01-to-be/07-the-foundation.md
|
||||
touches:
|
||||
- 02-DECISIONS/0009-modules-and-the-graph.md
|
||||
- 02-DECISIONS/0009-modules-and-the-graph.md
|
||||
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
|
||||
- 03-DESIGN/00-as-is/02-modules-and-manifests.md
|
||||
- 03-DESIGN/00-as-is/10-module-catalogue.md
|
||||
- 04-ISSUES/007-an-installed-package-is-not-a-capability/00-report.md
|
||||
---
|
||||
|
||||
# 011 — The module graph
|
||||
|
||||
> **A third edge was found after this graduated.** This effort established *presence* and
|
||||
> *instantiation*, and both are **runtime** edges — they answer *what does this need in order to
|
||||
> run*. Delivery needs a different question answered — *what has to be rebuilt when this changes*
|
||||
> — and that is a **build** edge, fixed inside an artifact rather than negotiated when it runs.
|
||||
> Recorded by [ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md), which also
|
||||
> notes what this effort's three entities turn out to be good for
|
||||
> ([ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md)).
|
||||
|
||||
## What is being investigated
|
||||
|
||||
Whether the catalogue's missing structure is a **graph** — modules declaring what they need,
|
||||
what they offer, and what they exclude — and what that replaces.
|
||||
|
||||
**[`worked-provider.md`](worked-provider.md) works one module through completely**, and breaks
|
||||
the tidy version. A database is nine things, not one — and a small game asking the mesh for its
|
||||
own database shows there are **two kinds of edge**: *presence*, where the thing must exist, and
|
||||
*instantiation*, where a provider makes something for a consumer and hands back credentials.
|
||||
Instantiation implies presence and not the reverse. The current system already had this split
|
||||
and the design had collapsed it.
|
||||
|
||||
**[`features.md`](features.md) answers what happens to `feature`.** It is one word for four
|
||||
things spanning three tiers — artifacts built once per version, resources applied to a machine,
|
||||
actions run against something that is not this machine, and checks that are requirements in
|
||||
disguise. Measured: every one of the twenty-one handlers implements all six stages, so `configs`
|
||||
has a build stage with nothing to build and `npm` has a start stage with nothing to start. That
|
||||
emptiness is the conflation, and it is why a stage that did nothing and a stage that failed look
|
||||
alike. Nothing replaces it, because it was never one concept. **One property is worth keeping:
|
||||
content is detected, relationships are declared.**
|
||||
|
||||
**[`cases.md`](cases.md) enumerates what a module can be** — twenty kinds of thing the mesh has
|
||||
to install, run, own or know about — and extracts the axes a manifest must express. Two of those
|
||||
axes appear in no current thinking: **how many instances** a thing may have, and **whether two
|
||||
can coexist**.
|
||||
|
||||
**The design is in [`proposal.md`](proposal.md): one kind of edge.** A module provides names
|
||||
and requires names, and that single relation absorbs requiring a module, requiring a resource,
|
||||
and the interface-and-adapter idea. An abstract name is legitimate **only where providers are
|
||||
genuinely substitutable** — `terminal` passes, `database` does not, because a consumer speaking
|
||||
Postgres does not speak MongoDB. The adapter is what creates an interface; without one there is
|
||||
a **tag**, which describes and does not bind.
|
||||
A **node provides names too**, which makes capability checking stop being a separate mechanism
|
||||
and makes the host's own capability report an input to resolution rather than something a
|
||||
person reads.
|
||||
|
||||
[`analysis.md`](analysis.md) measured the current catalogue. Its value to the design is two
|
||||
lessons rather than its machinery — a field that means *depends on* should say so, and
|
||||
placement does not belong in a manifest — and the rest is recorded as as-is evidence.
|
||||
|
||||
**Measured, and the premise was wrong for the existing system: the graph is not missing there.**
|
||||
[`analysis.md`](analysis.md) — 126 manifests, 103 edges, no cycles, nothing dangling, and a
|
||||
resolver that topologically sorts them, already called by the tool loader, the installer and
|
||||
the delivery coordinator. What the effort assumed would need building is a thing to call.
|
||||
|
||||
What survives is narrower: three declarations that do not exist (`excludes`, a required node
|
||||
capability, an interface with adapters), and two defects worth fixing whatever else is
|
||||
concluded — `provider:` is a dependency edge that is not read as one, which makes the closure
|
||||
for a working mesh come out without a database; and the resolver continues past a cycle and
|
||||
past a missing dependency, contrary to ADR 0001.
|
||||
|
||||
[ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md) proposes
|
||||
grouping modules by domain. [Research 005](../005-domain-grouping/analysis.md) measured that
|
||||
proposal and found its evidence holds in exactly one place — reachability — which
|
||||
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md) has since absorbed
|
||||
into the host. The measured case for domain grouping has therefore been consumed by a decision
|
||||
taken for unrelated reasons, and what remains is fifty modules that co-change with nothing.
|
||||
|
||||
That leaves the original complaint unanswered: the catalogue records **what was installed**
|
||||
rather than **what anything is for**, and nothing in the system can see a relationship between
|
||||
two modules. Grouping asserts relationships. A graph records them.
|
||||
|
||||
## Why now
|
||||
|
||||
A proposal to split modules into *provisioning services* and *applications* was worked through
|
||||
and abandoned in favour of one concept with facets, for a reason worth keeping:
|
||||
|
||||
- The split cannot be filed consistently. A git forge is consumed as a service *and* operated
|
||||
through a web interface. An analytics service grants tracking identity *and* is a dashboard
|
||||
somebody reads. An identity provider grants authentication *and* has an admin console.
|
||||
- The operator's correction is the sharper form: **what runs on the machine is a supervised
|
||||
container, not something a user started.** That is a fact about *how a thing runs*, not about
|
||||
what kind of thing it is — so it is a facet, not a taxonomy.
|
||||
|
||||
Filing decisions that follow from nothing are the disease research 005 measured. A second
|
||||
taxonomy would reproduce it.
|
||||
|
||||
So [ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md) survives, and the question
|
||||
becomes what a module must be able to **declare**.
|
||||
|
||||
## The shape being investigated
|
||||
|
||||
Five declarations, of which two exist today.
|
||||
|
||||
| Declaration | Today | Notes |
|
||||
|---|---|---|
|
||||
| **requires a resource** — a database, a bucket | yes | provisioning, [ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md) |
|
||||
| **provides a resource** | yes | as above |
|
||||
| **requires another module** | **no** | the dependency edge — the graph's substance |
|
||||
| **excludes another module** | **no** | installing A makes B unavailable |
|
||||
| **requires a node capability** | **no** | a graphical session, a container runtime, an architecture |
|
||||
|
||||
And one structural idea on top: **an interface module carries no implementation.** Adapters
|
||||
provide it. An assistant interface with several model-provider adapters; a terminal interface
|
||||
with several terminal adapters. A dependent names the interface and never an implementation.
|
||||
|
||||
**Prior art to measure against, not invent past.** This is a package manager's model, and the
|
||||
platform's own package manager already has all of it: `depends` is the dependency edge,
|
||||
`conflicts` is exclusion, and `provides` is the interface — several packages provide one
|
||||
virtual name, and a dependent names the virtual one. That the design arrived at the same shape
|
||||
independently is evidence for it. It is also a warning: dependency resolution, version
|
||||
constraints, conflict handling and rollback are a long-solved and easily-botched problem, and
|
||||
the effort should establish what to **delegate** rather than reimplement.
|
||||
|
||||
## Capabilities, and what may be installed
|
||||
|
||||
The operator's formulation: *system specs are capabilities, and capabilities unlock installable
|
||||
modules — you cannot install a graphical application on a node with no display server.*
|
||||
|
||||
The question that follows is whether the mesh may install a capability. The effort's working
|
||||
position, to be tested:
|
||||
|
||||
- **Intrinsic capabilities** — hardware, architecture, network position — are facts about a
|
||||
machine. They are detected, never installed, and a module requiring one it does not have is
|
||||
not unresolved but **impossible** on that node.
|
||||
- **Provided capabilities** — a display server, a container runtime — are not a separate kind
|
||||
of thing at all. They are modules that provide a capability, and requiring one is an ordinary
|
||||
edge the graph resolves by installing it.
|
||||
|
||||
If that holds, "may the mesh install a capability" is not a policy question. It is dependency
|
||||
resolution, and the only genuinely new thing is detection.
|
||||
|
||||
Which is where [issue 007](../../04-ISSUES/007-an-installed-package-is-not-a-capability/00-report.md)
|
||||
bears directly: *an installed package is not a capability*. A provided capability is not real
|
||||
because a package is present — it is real when it is present, running and working, and the
|
||||
difference is exactly the class of fault this repository keeps recording.
|
||||
|
||||
## What it touches beyond the catalogue
|
||||
|
||||
The operator's assessment is that the machinery around a module is wanted and its integration
|
||||
is not: **scheduled tasks, hooks, migrations, configuration and environment settings are worth
|
||||
keeping; seeds are not; and the current integration is wrong enough to need a major refactor.**
|
||||
|
||||
That is a claim to test rather than adopt. Research 005 already found supporting evidence from
|
||||
a different direction — that the densest apparent coupling in the catalogue is manifest
|
||||
boilerplate churn, cross-cutting changes to the machinery applied N times — which is what an
|
||||
integration being wrong looks like from the outside.
|
||||
|
||||
## Open questions
|
||||
|
||||
Struck-through rows are answered, with where. The rest are live.
|
||||
|
||||
### Settled
|
||||
|
||||
| Question | Answer |
|
||||
|---|---|
|
||||
| ~~What does the graph **delete**?~~ | For the existing system: nothing, it is already there ([`analysis.md`](analysis.md)). For the design: the module/resource distinction, the interface as a kind of thing, capability checking as a separate mechanism, domain grouping, and — the first clear deletion — **grant kinds**, once a module may only be granted what it exclusively owns ([`worked-provider.md`](worked-provider.md)). |
|
||||
| ~~Is an interface a module, or a name?~~ | A **name**, and only where providers are genuinely substitutable. The adapter is what creates one; without an adapter there is a **tag**, which describes and does not bind ([`proposal.md`](proposal.md)). |
|
||||
| ~~Where do domain modules fit?~~ | They do not. There is core infrastructure — concrete modules named individually, not flavourable, nothing standing in front of them. |
|
||||
| ~~What happens to domain grouping?~~ | Superseded. Folders assert relationships; edges record them. What grouping was for is a tag and a query. [ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md) is `proposed` and should be superseded rather than narrowed. |
|
||||
| ~~Is there one kind of edge?~~ | **No — two.** *Presence*, where a thing must exist, and *instantiation*, where a provider makes something for a consumer and hands back credentials. Instantiation implies presence, not the reverse. |
|
||||
| ~~When two modules provide one name, who chooses?~~ | Neither the consumer naming a node nor the consumer not caring. The consumer declares the **scope of its own need** — shared across its instances, or one each — the mesh binds, and the binding is written down and sticky. Where it is written follows the scope. |
|
||||
| ~~Can several modules share one database?~~ | **No.** A module is granted only what it exclusively owns — no shared writes and no read role on another's store, because reading couples you to its layout just as firmly. |
|
||||
| ~~What about a dashboard reading a dozen stores?~~ | **The rule is about contexts, not processes.** The mesh's own board reading the mesh's own store is the mesh showing its own data — not a boundary crossing. Everything inside a context reads its store freely; what is forbidden is a *different* context reading it. An earlier answer here was wrong. |
|
||||
| ~~Can every registry consumer be served another way?~~ | **Largely dissolves.** Of eighteen direct consumers, the owner keeps its database, node appliers are already stopped by ADR 0005, and the bulk are **foreign tenants** — thirteen tables across three contexts — who need to move out rather than read differently. |
|
||||
|
||||
### Live
|
||||
|
||||
| Question | Why it is open |
|
||||
|---|---|
|
||||
| How many instances should a module have? | Not derivable and not global: one-per-mesh is right for the broker and wrong for a store a disconnected node needs. It is per-module and nothing in the schema says it. |
|
||||
| Can two of something coexist? | `excludes` covers part of it. Two terminals are fine, two things wanting one port are not, two brokers might be either. |
|
||||
| What does a provider hand back? | Credentials and an address for a store; a command for a terminal. Same relation, different shape crossing it. |
|
||||
| Is provisioning one mechanism or two? | The mesh's own registry is provisioned **before there is a mesh**, so provisioning is part of the bootstrap and part of what the carried bundle expresses. At bootstrap the store is local; afterwards it is on another node. Same operation, both sides of a tier boundary. |
|
||||
| Is a tool surface one relation with two audiences, or two? | 56 of 126 modules carry tools — more than carry a service — and what consumes them is an **agent**, not a module. |
|
||||
| ~~Do the remaining cross-context reads want an interface or events?~~ | **Derived, not chosen.** Neither is SQL — that only ever runs against your own store. ADR 0004 makes disconnection ordinary, so anything that must work while disconnected cannot use a request and needs a local copy: a subscription. Anything where a stale answer is worse than none cannot use a subscription. |
|
||||
| What does a consumer do about events it missed while disconnected? | Replay from a point, ask once for a full picture and resume, or rebuild. The question every projection has, and unanswered here. |
|
||||
| What happens to a grant when its consumer is removed? | Dropping is data loss; keeping is a leak. ADR 0005's removal rule does not obviously carry, because the thing lives inside another module's state. |
|
||||
| Is a declaration composed per node, from what that node reported? | Some configuration follows the hardware. Either the host fills a blank — deciding, against ADR 0005 — or the control plane composes from the node's inventory first. |
|
||||
| Would `excludes` and capability requirements actually be used? | Zero manifests declare either, which is equally consistent with *nobody needs them* and *nobody can express them*. |
|
||||
| What does an exclusion mean for something already installed? | Refuse the install, or surface the conflict and let it be decided. |
|
||||
| Are tiers a view of the graph, or a constraint on it? | If a tier is a computed level the word is a convenience. If *a tier may depend only on tiers below it* is to be enforced, it is a constraint and must be stated as one. |
|
||||
| Where does resolution happen — the mesh, or the platform's package manager? | The mesh must model mesh-level edges. Whether it also resolves operating-system packages decides whether a solver gets written. |
|
||||
| How far may the control plane be split? | A single board over several contexts works because they are contexts *inside* one control plane with one interface. If a context becomes its own deployable with its own interface, the board is coupled to N of them and the composition has nowhere to live that tier 3 permits. A constraint on splitting, worth knowing before splitting. |
|
||||
| What does the pipeline schedule, once features are gone? | It schedules features today. The four categories they split into have different lifecycles, so the unit of work differs for each and needs naming. |
|
||||
| Should placement leave the catalogue? | A provision pins itself to a named node in the manifest. Placement is an inventory decision, and having it in the catalogue means a second node cannot provide the mesh's store without editing its consumer. |
|
||||
@@ -0,0 +1,138 @@
|
||||
# The graph is not missing
|
||||
|
||||
Measured against `origin/main` of the code repository, 2026-08-26. Every manifest, read through
|
||||
git refs rather than a checkout.
|
||||
|
||||
The effort was opened to ask whether the catalogue's missing structure is a graph. It is not
|
||||
missing. **It exists, it is healthy, and three separate parts of the system already use it.**
|
||||
|
||||
That is the finding, and it changes what is worth asking.
|
||||
|
||||
## Finding 1 — the graph is already declared, and it is clean
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| manifests | 126 |
|
||||
| declare a dependency on another module | 64 |
|
||||
| declare a requirement on a provision | 11 |
|
||||
| **edges** | **103** |
|
||||
| declare neither | 62 — 49% |
|
||||
| **cycles** | **0** |
|
||||
| **dependencies declared but absent** | **0** |
|
||||
| deepest chain | 5 |
|
||||
|
||||
Half the catalogue is unconnected, which matches
|
||||
[research 005](../005-domain-grouping/analysis.md)'s finding that fifty modules co-change with
|
||||
nothing. The connected half is well formed: no cycles, nothing dangling.
|
||||
|
||||
## Finding 2 — it is already resolved, and already used
|
||||
|
||||
The platform SDK carries a dependency resolver that topologically sorts modules, and it is
|
||||
called from three places: the tool loader at startup, the installer when syncing modules onto a
|
||||
node, and the delivery coordinator when expanding what a change affects.
|
||||
|
||||
It also already does something the effort assumed would need designing: **a requirement on
|
||||
another module's provision is treated as an implicit edge to that module**, so a consumer does
|
||||
not have to declare the same relationship twice.
|
||||
|
||||
So *ordering by the graph* — which
|
||||
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md) says
|
||||
the control plane will do — is not a thing to build. It is a thing to call.
|
||||
|
||||
## Finding 3 — the most important edges in the mesh are invisible
|
||||
|
||||
The one place the graph is wrong, and it is wrong about the substrate.
|
||||
|
||||
A module that needs a database declares it like this:
|
||||
|
||||
```yaml
|
||||
provisions:
|
||||
- name: mesh-db
|
||||
provider: postgres # ← a dependency on the postgres module
|
||||
node: <a named node> # ← and where it must run
|
||||
```
|
||||
|
||||
`provider:` names a module. It is a dependency, declared, in the manifest — and it sits inside
|
||||
`provisions:`, which is what a module *offers*. The resolver reads `dependencies:` and
|
||||
`requires:`, so it never sees it.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| provider references that name a real module | 4 |
|
||||
| **invisible to the resolver** | **3, across 2 modules** |
|
||||
|
||||
Three edges is nothing, and they are the mesh's own database, the mesh's own broker, and the
|
||||
work engine's database. The most load-bearing relationships in the system are the ones the
|
||||
graph cannot see.
|
||||
|
||||
**The consequence is measurable.** Computing what a working mesh needs, from the declared
|
||||
graph:
|
||||
|
||||
```
|
||||
registry → sdk → mesh → meshware 4 modules, 4 levels
|
||||
```
|
||||
|
||||
No database. No broker. A closure that is arithmetically correct and obviously wrong, and wrong
|
||||
for exactly one reason: a field that means *depends on* is not read as one.
|
||||
|
||||
This is [04-ISSUES/003](../../04-ISSUES/003-firewall-scope-is-read-by-no-code/00-report.md)
|
||||
again, in a new place — not a key nothing reads, but a key read as something other than what it
|
||||
means.
|
||||
|
||||
## Finding 4 — the resolver continues past faults it should stop on
|
||||
|
||||
Two behaviours, both contrary to
|
||||
[ADR 0010](../../02-DECISIONS/0010-delivery.md):
|
||||
|
||||
- **A cycle warns and falls back to input order.** A cycle means no correct order exists; the
|
||||
resolver proceeds with an arbitrary one and logs a line.
|
||||
- **A dependency that does not exist warns and continues.** The validation is documented as
|
||||
*non-fatal, logged as warnings*.
|
||||
|
||||
Neither has fired in the current catalogue — there are no cycles and nothing dangling — which
|
||||
is why nobody has noticed. They are latent, and they are in the component that
|
||||
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md) makes
|
||||
responsible for the ordering a host will apply without question.
|
||||
|
||||
## Finding 5 — placement is decided in the catalogue
|
||||
|
||||
`node:` in a provision pins it to a named node, in the manifest. Two modules do this today,
|
||||
and they are the substrate ones.
|
||||
|
||||
Which node runs what is an inventory and placement decision — tier 2 by the skeleton's own
|
||||
test. Having it in a manifest means the catalogue decides placement, and a second node cannot
|
||||
provide the mesh's database without editing the module that consumes it.
|
||||
|
||||
## What this means for the effort
|
||||
|
||||
**The opening question — "what does the graph delete?" — has an answer: nothing, because the
|
||||
graph is already there.** The premise was wrong, and finding that out is the effort's first
|
||||
result rather than a setback.
|
||||
|
||||
The questions that survive are narrower and answerable:
|
||||
|
||||
| Missing declaration | Manifests using it today |
|
||||
|---|---|
|
||||
| `excludes` — installing A makes B unavailable | **0** |
|
||||
| a required node capability | **0** |
|
||||
| an interface, with adapters providing it | **0** |
|
||||
|
||||
Those are what a graph would *add*. What it would delete is a different and smaller list, and
|
||||
the honest version of it is: nothing yet.
|
||||
|
||||
**And two defects worth fixing regardless of what else this effort concludes:**
|
||||
|
||||
1. `provider:` is a dependency edge and is not read as one. Fixing it makes the closure correct
|
||||
— which is what [research 012](../012-the-minimum-viable-node/00-overview.md) needs in order
|
||||
to answer what a one-node mesh requires.
|
||||
2. The resolver continues past a cycle and past a missing dependency. Both should refuse.
|
||||
|
||||
## What was not measured
|
||||
|
||||
- **Whether the missing declarations would be used.** Zero manifests declare exclusions or
|
||||
capabilities, and that is equally consistent with *nobody needs them* and *nobody can express
|
||||
them*. Nothing here separates those.
|
||||
- **Whether the interface-and-adapter idea has a consumer.** It is a good shape, and it is
|
||||
argued for rather than measured.
|
||||
- **What the closure should be.** Finding 3 says the computed one is wrong. It does not say
|
||||
what the right one is; that needs the fix first.
|
||||
@@ -0,0 +1,128 @@
|
||||
# What a module can be
|
||||
|
||||
Every kind of thing the mesh has to install, run, own or know about, before deciding what a
|
||||
manifest says. Written to be argued with: a case here that turns out not to exist should be
|
||||
struck, and one that is missing is a hole in whatever schema follows.
|
||||
|
||||
The hard cases are at the end, and they are the point.
|
||||
|
||||
## The ordinary cases
|
||||
|
||||
**1 — A supervised service.** A container the mesh runs and keeps running. Nobody starts it;
|
||||
it is simply up. *A relational store, a message broker, an object store, a dashboard.*
|
||||
|
||||
**2 — A system package with configuration.** Not a container. Installed into the machine,
|
||||
configured through files, run by the service manager. *A firewall, a resolver, an overlay.*
|
||||
Note: [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) says applying
|
||||
these is the host's job — so what the module contributes is the *deciding*, not the doing.
|
||||
|
||||
**3 — An application a person launches.** Installed on a node, started by a human, running only
|
||||
while they use it. *An editor, a chat client, a file manager, a terminal.*
|
||||
|
||||
**4 — A command-line tool.** Installed, on the path, run when invoked. No service, no window.
|
||||
*A formatter, a query client, a backup utility.*
|
||||
|
||||
**5 — A library.** Never runs at all. Consumed at build time by other modules. *An SDK.*
|
||||
|
||||
**6 — A one-shot task.** Runs once, changes something, exits. *A schema migration, a data
|
||||
import, a seed.*
|
||||
|
||||
**7 — A scheduled task.** Runs repeatedly on a timer, exits each time. *A backup, a prune, a
|
||||
report.*
|
||||
|
||||
**8 — An adapter.** Exists to make several unlike things look alike behind one name. *A model
|
||||
provider behind an assistant interface.*
|
||||
|
||||
**9 — A standalone application in its own repository.** Same shape as any of the above; the
|
||||
difference is only where its source lives
|
||||
([ADR 0015](../../02-DECISIONS/0015-applications-live-in-their-own-repository.md)). Worth
|
||||
listing because a schema that assumes a monorepo path would exclude it.
|
||||
|
||||
## The cases that break a naive schema
|
||||
|
||||
**10 — Something that is a service *and* an application.** A git forge is consumed by other
|
||||
modules as a remote and a registry, *and* operated by a person through a web interface. A web
|
||||
analytics service grants a tracking identity *and* is a dashboard somebody reads. Neither is a
|
||||
service-or-application choice; both are true simultaneously.
|
||||
|
||||
**11 — Something that provides to others *and* consumes from others.** The store provides
|
||||
databases and needs a filesystem. The forge provides a registry and needs a database. Provider
|
||||
and consumer are not kinds of module; they are ends of edges.
|
||||
|
||||
**12 — Something the mesh installs that then becomes a node capability.** The container runtime
|
||||
is installed *by* the mesh, and once it works, the node **provides** `container-runtime` to
|
||||
everything else. So a module can change what its node provides. The node's provides-list is
|
||||
therefore partly derived from what is installed on it, not only detected from what was already
|
||||
there — and the two have to agree.
|
||||
|
||||
**13 — Something that must be adopted rather than installed.** The machine already has the
|
||||
package manager, the container runtime, possibly the version control system, each with
|
||||
configuration somebody chose. The module does not install it; it takes it over
|
||||
([research 012](../012-the-minimum-viable-node/00-overview.md)).
|
||||
|
||||
**14 — Something that is a set, not a thing.** *Core infrastructure* is not installable — it is
|
||||
a name for the concrete modules a working node needs. Whether that is a module whose only
|
||||
content is `requires`, or a query over the graph, or a pinned list outside the catalogue, is
|
||||
undecided and is one of the sharper questions here.
|
||||
|
||||
**15 — Something with one instance for the whole mesh.** There is one mesh database, not one per
|
||||
node. Assigning it to two nodes is not redundancy, it is two meshes. Contrast with a terminal,
|
||||
where per-node is the only sensible reading.
|
||||
|
||||
**16 — Something that may be installed several times over.** Two terminals coexist happily. Two
|
||||
things wanting port 443 do not. Two message brokers might be fine or might be a split brain,
|
||||
and nothing in *provides* and *requires* distinguishes those.
|
||||
|
||||
**17 — Something that is not software at all.** A firewall policy. A DNS record. A certificate.
|
||||
It owns no binary, runs nothing, and is entirely *desired state* — which is the one case that
|
||||
fits the host's declaration model exactly and fits an installable-package model not at all.
|
||||
|
||||
**18 — An agent.** The mesh's own premise is that agents are participants. An agent has an
|
||||
identity, a licence, a node it runs on, and work it does. Whether that is a module, a record in
|
||||
the control plane, or something else is not obvious, and getting it wrong shapes everything
|
||||
about how agents are assigned.
|
||||
|
||||
**19 — The host itself.** Tier 0 installs everything else and is installed by hand. It is not a
|
||||
module, and a schema that cannot say so has a bootstrap problem hiding in it.
|
||||
|
||||
**20 — Something the mesh depends on and does not control.** A domain registrar, an upstream
|
||||
resolver, a certificate authority, an electricity supply. Almost certainly not modules — but
|
||||
the mesh's health depends on them, and they are the reason a node can be perfectly configured
|
||||
and still not work.
|
||||
|
||||
## What varies across the cases
|
||||
|
||||
The list above matters less than this. These are the axes a manifest has to express, and each
|
||||
one is a question the schema must answer or deliberately refuse.
|
||||
|
||||
| Axis | Range | Sharpest case |
|
||||
|---|---|---|
|
||||
| **Does it run?** | supervised · launched by a person · once · on a timer · never | 5, 6, 7, 17 |
|
||||
| **Who starts it?** | the mesh · a human · nothing | 1 vs 3 |
|
||||
| **Where does it come from?** | container image · system package · our source · already on the machine | 12, 13 |
|
||||
| **Does it provide to other modules?** | a resource · an abstract name · nothing | 8, 11 |
|
||||
| **Does it hold state?** | yes, and it matters where · no | 1 vs 3 |
|
||||
| **How many instances?** | one per mesh · one per node · many per node | 15, 16 |
|
||||
| **Can two coexist?** | yes · no · only with different settings | 16 |
|
||||
| **Is it installed or adopted?** | installed · adopted · either, depending on the machine | 13 |
|
||||
| **Does installing it change what the node provides?** | yes · no | 12 |
|
||||
|
||||
**Two of these are not in any current thinking**, and both come from the hard cases:
|
||||
*how many instances* (15) and *can two coexist* (16). `excludes` covers part of the second and
|
||||
nothing covers the first.
|
||||
|
||||
## Questions the cases raise
|
||||
|
||||
- **Is "runs" a property or a kind?** The axes suggest a property — one schema, with a field
|
||||
saying how it runs, `never` included. The alternative is several kinds of module with
|
||||
different schemas, which is the taxonomy [research 011](00-overview.md) already rejected once
|
||||
for services and applications.
|
||||
- **Is an agent a module?** (18) If yes, the schema carries identity and licensing. If no, the
|
||||
mesh has two catalogues.
|
||||
- **Is core infrastructure a module?** (14) A module whose only content is `requires` is either
|
||||
elegant or a category pretending to be a thing — the same trap `database` was.
|
||||
- **What names one-per-mesh?** (15) Nothing in provides, requires or excludes says it, and
|
||||
getting it wrong means two of something that must be one.
|
||||
- **Where does a policy live?** (17) Pure desired state fits the host's declaration exactly.
|
||||
Whether it is a module at all, or something the control plane derives and no catalogue entry
|
||||
exists for, is open.
|
||||
@@ -0,0 +1,131 @@
|
||||
# What a feature is, and what it splits into
|
||||
|
||||
Measured against `origin/main` of the code repository, 2026-08-26.
|
||||
|
||||
The operator wants the feature concept gone.
|
||||
[Research 006](../006-mesh-from-scratch/00-overview.md) left it open — *"does `feature`
|
||||
survive? The skeleton splits it in two and argues the conflation is what makes the delivery
|
||||
pipeline hard to reason about."*
|
||||
|
||||
This is what it actually is, and what it turns into.
|
||||
|
||||
## What it is today
|
||||
|
||||
A **feature** is a kind of content a module can carry, detected from what its directory
|
||||
contains rather than declared. Twenty-one of them, each with a handler that owns its whole
|
||||
lifecycle:
|
||||
|
||||
```
|
||||
configs dist-assets events hooks migrations migration-artifact npm npm-install
|
||||
prerequisite-env prerequisite-packages prerequisite-provision provision-migrations
|
||||
provision-seeds seeds service systemd tools verifiers verify vhost detect
|
||||
```
|
||||
|
||||
Each moves through **six stages**: build, publish, install, configure, start, verify.
|
||||
|
||||
## Finding — every handler implements every stage
|
||||
|
||||
The structural evidence, and it is not a style problem.
|
||||
|
||||
| handler | stages it implements |
|
||||
|---|---|
|
||||
| `configs` | build · configure · install · start · verify |
|
||||
| `service` | build · configure · install · start · verify |
|
||||
| `systemd` | build · configure · install · start · verify |
|
||||
| `tools` | build · configure · install · start · verify |
|
||||
| `vhost` | build · configure · install · start · verify |
|
||||
| `migrations` | build · configure · install · start · verify |
|
||||
| `npm` | build · publish · install · configure · start |
|
||||
|
||||
`configs` writes files onto a node. It has nothing to build, and it has a build stage.
|
||||
`npm` publishes a package to a registry. It has nothing to start, and it has a start stage.
|
||||
|
||||
**One interface spans build-time and apply-time, so every kind of content must implement both
|
||||
halves and most of them do nothing in one.** That is the conflation, and the emptiness is what
|
||||
makes the pipeline hard to reason about: a stage that does nothing and a stage that failed to
|
||||
do anything look identical from outside.
|
||||
|
||||
## What it splits into
|
||||
|
||||
The twenty-one are not one kind of thing. They are four, and they belong to different tiers.
|
||||
|
||||
**Artifacts — built once per version, then published.** `npm`, `dist-assets`,
|
||||
`migration-artifact`. Nothing about a node is involved; the output is a thing that exists in a
|
||||
registry. **Tier 2, delivery.**
|
||||
|
||||
**Resources — desired state on a machine.** `configs`, `service`, `systemd`, `vhost`, `tools`.
|
||||
Applied, converged, idempotent — which is exactly what
|
||||
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md)
|
||||
already describes and what the host already does. **Tier 0.**
|
||||
|
||||
**Actions — run once, against something that is not this machine.** `migrations`, `seeds`,
|
||||
`provision-migrations`, `provision-seeds`, `hooks`. A migration runs against a database, and the
|
||||
database may be on another node entirely. Neither an artifact nor node state, which is why they
|
||||
sit awkwardly in a scheme built for both. **Tier 2, and the operator wants seeds gone.**
|
||||
|
||||
**Checks — assertions, not changes.** `prerequisite-env`, `prerequisite-packages`,
|
||||
`prerequisite-provision`, `verify`, `verifiers`, `detect`. The prerequisites are **requirements
|
||||
in disguise** — a module saying what must be true before it can be installed, which is precisely
|
||||
what an edge in the graph says. The verifiers are the read-back the host already performs.
|
||||
|
||||
## So the answer is: it splits, and the split is a tier boundary
|
||||
|
||||
**`feature` is one word for four things spanning three tiers.** That is why every handler
|
||||
implements every stage, why half of them are empty, and why the pipeline is hard to reason
|
||||
about.
|
||||
|
||||
Nothing replaces it, because it was never one concept:
|
||||
|
||||
| Was a feature | Becomes | Whose |
|
||||
|---|---|---|
|
||||
| npm, dist-assets, migration-artifact | an **artifact** | delivery |
|
||||
| configs, service, systemd, vhost, tools | a **resource** in a declaration | the host |
|
||||
| migrations, hooks | an **action** against something else | delivery |
|
||||
| seeds | *nothing* — the operator wants them gone | — |
|
||||
| prerequisite-* | an **edge** in the graph | the catalogue |
|
||||
| verify, verifiers | the host's **read-back** | the host |
|
||||
|
||||
## Tools are a facet, and their consumer is not a module
|
||||
|
||||
The most common content in the catalogue: 56 of 126 modules carry a tool surface, more than
|
||||
carry a service.
|
||||
|
||||
It survives the split, and it does not fit either half cleanly. A tool is not an artifact and
|
||||
not node state — it is a **contract the mesh publishes on a module's behalf**, and what consumes
|
||||
it is an **agent**, not another module.
|
||||
|
||||
That is a second audience, and the design has only described one. A module `provides` things
|
||||
other modules require; a module also `provides` things agents call. Same word, different
|
||||
consumer, different lifecycle — a tool appears when the module is assigned somewhere and
|
||||
disappears when it is not, and nothing in the graph edges says so.
|
||||
|
||||
Whether that is one relation with two audiences or two relations is undecided, and it is the
|
||||
kind of question that is cheap now and expensive later.
|
||||
|
||||
## What is lost, and should not be
|
||||
|
||||
**Detection.** Features are detected from the directory rather than declared, and the reason is
|
||||
good: *a declared list and the directory it describes drift, and the directory is the one that
|
||||
is true.* That property is worth keeping whatever the concept is called — a module that says it
|
||||
has migrations and has none, or has them and does not say so, is a fault nobody sees until it
|
||||
matters.
|
||||
|
||||
Under the split, detection still applies: what a module **contains** is read from what is there.
|
||||
What it **requires**, **provides** and **excludes** is declared, because none of that is visible
|
||||
in a directory.
|
||||
|
||||
That line is worth stating precisely, because it is the one the current design got right:
|
||||
**content is detected, relationships are declared.**
|
||||
|
||||
## Open
|
||||
|
||||
- **Is an action a resource?** A migration is not node state and not an artifact. It could be a
|
||||
resource type the host applies with the target being a database rather than a machine — which
|
||||
would collapse the third category into the second, at the cost of the host reaching something
|
||||
that is not the machine it is on. That cost looks too high, but it has not been argued.
|
||||
- **Where do hooks go?** They are arbitrary code a module runs at a stage — the escape hatch. A
|
||||
design with no escape hatch is either very good or has not met reality yet, and this one has
|
||||
not.
|
||||
- **What does the pipeline schedule, once features are gone?** It currently schedules features.
|
||||
If the four categories have different lifecycles, the unit of work is different for each, and
|
||||
what the coordinator orchestrates needs naming.
|
||||
@@ -0,0 +1,167 @@
|
||||
# One kind of edge
|
||||
|
||||
> **Superseded in part by [`worked-provider.md`](worked-provider.md).** Working postgres through
|
||||
> completely shows there are **two** kinds of edge, not one: *presence* — the thing exists and is
|
||||
> reachable, nothing created — and *instantiation* — the provider is asked to make something for
|
||||
> this consumer and hands back credentials. Instantiation implies presence; presence does not
|
||||
> imply instantiation. Everything else below stands; the claim in the title does not.
|
||||
|
||||
A design, not an account of what exists. [`analysis.md`](analysis.md) measured the current
|
||||
catalogue and its value here is two lessons rather than its machinery: a field that means
|
||||
*depends on* should say so, and placement does not belong in a manifest.
|
||||
|
||||
## The idea
|
||||
|
||||
**A module provides names. A module requires names. That is the only edge.**
|
||||
|
||||
Everything the effort listed as separate declarations turns out to be one relation with
|
||||
different kinds of name on either end.
|
||||
|
||||
```
|
||||
postgres provides postgres
|
||||
kitty provides kitty, terminal
|
||||
xterm provides xterm, terminal
|
||||
anthropic provides anthropic, ai-assistant
|
||||
|
||||
meshboard requires postgres, lavinmq
|
||||
vscode requires terminal, display-server
|
||||
```
|
||||
|
||||
A name is either **concrete** — a module's own name, so `requires: postgres` means that module
|
||||
and no other — or **abstract**, so `requires: terminal` means whatever provides it.
|
||||
|
||||
That single relation absorbs three things the effort had listed separately:
|
||||
|
||||
| Was going to be | Is |
|
||||
|---|---|
|
||||
| requires another module | requires a concrete name |
|
||||
| requires a resource | requires the module that provides it, by name |
|
||||
| an interface, with adapters providing it | an abstract name, legitimate only where an adapter makes providers substitutable |
|
||||
|
||||
The interface stops being a kind of module. It is a name with more than one provider *and a
|
||||
contract they all satisfy*, and nothing has to declare that it is one.
|
||||
|
||||
### An abstract name is only legitimate when providers are actually substitutable
|
||||
|
||||
The test, and it is a strict one: **can a consumer be switched from one provider to another
|
||||
without changing?**
|
||||
|
||||
`terminal` passes. Anything that runs a command in a terminal works, and a consumer never
|
||||
learns which one it got.
|
||||
|
||||
**`database` fails, and it is the example worth keeping.** A module connecting to Postgres does
|
||||
not connect to MongoDB, or to SQL Server, or to MariaDB. Different wire protocol, different
|
||||
dialect, different driver. A consumer that declared `requires: database` and was handed any of
|
||||
them would break — so the name promises something no provider can deliver, and the resolver
|
||||
would satisfy a requirement that is not satisfied.
|
||||
|
||||
That failure has a shape this repository already knows: something declared, accepted, and not
|
||||
true. It is worse here than usual because the graph would report success.
|
||||
|
||||
**The adapter is what creates an interface.** `ai-assistant` is a legitimate abstract name
|
||||
exactly when adapters exist to normalise the providers behind it. Without an adapter there is
|
||||
no interface — there is a category, and a category is not an edge.
|
||||
|
||||
So `postgres` is required by name, and a module that could genuinely work with several stores
|
||||
requires whichever it actually speaks to.
|
||||
|
||||
### Categories are catalogue metadata, not structure
|
||||
|
||||
*Database* is still a useful word — for finding things, for a person browsing what the mesh can
|
||||
host, for grouping in an interface. It is a **tag**.
|
||||
|
||||
Tags describe. Edges bind. Keeping them apart is what stops the catalogue acquiring a second
|
||||
kind of relationship that looks like a dependency and is not — which is what a folder named
|
||||
after a domain already was.
|
||||
|
||||
## The node provides names too
|
||||
|
||||
The move that makes capabilities stop being a separate system.
|
||||
|
||||
A node's profile is a set of provided names. `display-server`. `container-runtime`.
|
||||
`amd64`. A module requiring `display-server` is satisfied by **the node**, exactly as a module
|
||||
requiring `postgres` is satisfied by another module.
|
||||
|
||||
So there is one resolution rather than two: *is this name provided by anything available here?*
|
||||
A graphical application cannot be installed on a node with no display server for the same
|
||||
reason, and through the same code, that it cannot be installed without its libraries.
|
||||
|
||||
And it means the host's capability detection — which already reports what a machine can be
|
||||
asked to do, with a reason for each verdict — **is the node's provides-list**. It was built to
|
||||
be read by a person; it turns out to be an input to resolution.
|
||||
|
||||
## What a module declares
|
||||
|
||||
```yaml
|
||||
name: meshboard
|
||||
|
||||
provides: [meshboard]
|
||||
requires: [postgres, lavinmq, container-runtime] # by name: it speaks their protocols
|
||||
excludes: []
|
||||
tags: [observability] # for finding it, never for resolving it
|
||||
|
||||
runs: # what applying it means
|
||||
- container: ...
|
||||
constraints: # what must be true of a node, never which node
|
||||
- architecture: amd64
|
||||
```
|
||||
|
||||
`postgres` and `lavinmq` are concrete because this module speaks their wire protocols and would
|
||||
break against anything else. `container-runtime` is abstract and satisfied by the **node**.
|
||||
|
||||
**`excludes`** is the one genuinely new relation: naming something that cannot coexist with
|
||||
this. It is not derivable from requires and provides, and without it two modules that both
|
||||
provide `terminal` — or two that both want port 443 — look independent right up until
|
||||
installing the second breaks the first.
|
||||
|
||||
**`constraints` are not placement.** They say what must be true of a node, never which node.
|
||||
Which node runs what is an inventory decision, and the measurement found the current catalogue
|
||||
deciding it in the manifest — a module pinning its database to a named node, so that a second
|
||||
node cannot provide it without editing the module that consumes it. That is the mistake this
|
||||
separation exists to avoid.
|
||||
|
||||
## What it deletes
|
||||
|
||||
The question the effort opened with, answered for the design rather than for what exists.
|
||||
|
||||
- **The distinction between requiring a module and requiring a resource.** One relation.
|
||||
- **The interface as a kind of thing.** A name with several providers.
|
||||
- **Capability checking as a separate mechanism.** The node is a provider.
|
||||
- **The domain module** — a `networking` module that exists to gather a firewall, a resolver and
|
||||
a proxy under one name. It came from an older shape and does not fit: there is no such thing
|
||||
to install. There is **core infrastructure**, which is a set of concrete modules named
|
||||
individually — a firewall, a store, a resolver — with no flavour and no grouping module
|
||||
standing in front of them.
|
||||
- **Domain grouping as structure** ([ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md)).
|
||||
Folders assert relationships; edges record them. What grouping was for — finding things,
|
||||
seeing what belongs together — is a **tag** and a *query* over the graph, neither of which
|
||||
anybody has to keep true by hand.
|
||||
- **Tiers as a separate concept**, possibly. A tier is a level in the graph, and levels are
|
||||
computed. Whether the coarse boundary is still worth naming is below.
|
||||
|
||||
## What it does not delete, and should not
|
||||
|
||||
**A resolver still has to exist**, and it will need version constraints, conflict handling and
|
||||
a story for when two modules provide one name and nothing says which to use. That is the
|
||||
long-solved and easily-botched part, and the design should say what it **delegates** to the
|
||||
platform's package manager rather than reimplementing.
|
||||
|
||||
## Open
|
||||
|
||||
- **When two modules provide one abstract name, who chooses?** A node with both `kitty` and
|
||||
`xterm` satisfies `terminal` twice. Whether the choice is a node setting, a mesh setting, or
|
||||
an explicit pin is the substance of the interface idea, and this proposal does not settle it.
|
||||
It is a smaller question than it was: the strict substitutability test means the consumer
|
||||
genuinely does not care which it gets, so the choice is about preference rather than
|
||||
correctness.
|
||||
- **Are tiers a view of the graph, or a boundary that survives it?** If tiers are levels, the
|
||||
concept is derived and the word is a convenience. If the tier rule — *a tier may depend only
|
||||
on tiers below it* — is meant to be enforceable, it is a constraint on the graph rather than
|
||||
a description of it, and it has to be stated as one.
|
||||
- **What does a provider hand back?** A consumer requiring `postgres` needs credentials and an
|
||||
address; one requiring `terminal` needs a command. The requirement is satisfied by the same
|
||||
relation in both cases, and what flows across it is not the same shape. Whether that lives in
|
||||
the name, beside it, or in what the provider returns is undecided.
|
||||
- **What does a node provide that is not a capability?** Its architecture is a provided name
|
||||
under this scheme, and so is its operating system. That may be elegant or may be one
|
||||
abstraction too far; nothing here tests it.
|
||||
@@ -0,0 +1,491 @@
|
||||
# A provider, all the way through
|
||||
|
||||
One module, worked out completely, because it is the case that breaks the tidy version. It
|
||||
looks like *a container that runs a database* and it is at least nine things.
|
||||
|
||||
Postgres is the example. **The shape is not specific to it** — see the end.
|
||||
|
||||
## What it carries
|
||||
|
||||
**1 — A supervised container.** An image, a version, a data volume, and configuration. Easy,
|
||||
and the only part the phrase "a docker service" describes.
|
||||
|
||||
**2 — Persistent state, and where it lives matters.** The volume is the database. Moving this
|
||||
module between nodes is not rescheduling; it is a migration. Almost nothing else in the
|
||||
catalogue has this property, and nothing in `provides` / `requires` expresses it.
|
||||
|
||||
**3 — Configuration that is partly the machine's.** Tuning follows the hardware — memory,
|
||||
storage. A declaration generated centrally cannot know those, and
|
||||
[research 012](../012-the-minimum-viable-node/00-overview.md) says the machine's own values win
|
||||
on conflict. So some of this module's configuration is *derived from the node it lands on*.
|
||||
|
||||
**4 — A tool surface.** It exposes capabilities agents can call — query, list, provision. That
|
||||
is not a resource on a machine and not an artifact; it is a contract the mesh publishes on the
|
||||
module's behalf.
|
||||
|
||||
**5 — A provisioner.** The part that matters, and the one below.
|
||||
|
||||
**6 — Its own bookkeeping.** The provisioner must remember what it granted to whom, or it
|
||||
cannot revoke, rotate or clean up. So a module that provides state to others *also* holds state
|
||||
about its providing — and that state is not the database's data.
|
||||
|
||||
**7 — An exposure decision, per node it runs on.** Reachable from the machine only, from the
|
||||
local network, or publicly. That is a property of *this assignment*, not of the module: the
|
||||
same module on two nodes may answer differently.
|
||||
|
||||
**8 — Credentials it generates.** Per consumer, and they have to reach the consumer. Which
|
||||
means a provisioning edge carries a payload, and the payload is a secret.
|
||||
|
||||
**9 — Health that is not "the container is up".** A container running and a database accepting
|
||||
connections are different facts, and the second is the one anything cares about. This is the
|
||||
host's read-back rule, at a distance.
|
||||
|
||||
## The provisioner is a second kind of edge
|
||||
|
||||
The tidy version of this effort said: *a module provides names, a module requires names, that is
|
||||
the only edge.* A small game wanting to store data shows it is not.
|
||||
|
||||
```
|
||||
my-cool-game requires postgres # I speak its protocol, it must exist
|
||||
my-cool-game requires a database FROM postgres, called my-cool-game
|
||||
```
|
||||
|
||||
The first is **presence**: the thing exists and is reachable. Nothing is created; nothing flows
|
||||
back. `vscode requires terminal` is this, and so is `requires container-runtime`.
|
||||
|
||||
The second is **instantiation**: the provider is asked to make something *for this consumer*,
|
||||
and hands back what the consumer needs to use it. A database, a user, a password, an address.
|
||||
|
||||
They differ in every way that matters:
|
||||
|
||||
| | presence | instantiation |
|
||||
|---|---|---|
|
||||
| creates something | no | yes, one per consumer |
|
||||
| carries a payload back | no | credentials, an address |
|
||||
| can be revoked | — | yes, and must be when the consumer goes |
|
||||
| provider holds state about it | no | yes — who was granted what |
|
||||
| satisfied by | anything providing the name | that provider, specifically |
|
||||
|
||||
**This is the mesh's actual power**, in the operator's words: a small game declares it wants a
|
||||
database and the mesh makes one. Nobody creates a user by hand, nobody pastes a connection
|
||||
string. That is worth being precise about rather than folding into a single relation because
|
||||
one relation is prettier.
|
||||
|
||||
## What that costs the design
|
||||
|
||||
**The proposal's "one kind of edge" is wrong**, and the current system already knew: it has
|
||||
`dependencies` for presence and `requires: provision:` for instantiation, with the resolver
|
||||
deriving a presence edge from every instantiation edge. [`analysis.md`](analysis.md) recorded
|
||||
that derivation as a convenience. It is not — it is the correct relationship between two
|
||||
genuinely different relations.
|
||||
|
||||
So: **two kinds of edge, one graph.** Instantiation implies presence. Presence does not imply
|
||||
instantiation.
|
||||
|
||||
## Which provider, when two nodes run one
|
||||
|
||||
Asked directly, because two nodes can each run a relational store and a consumer has to be
|
||||
served by one of them. Neither obvious answer is right.
|
||||
|
||||
**Not "the consumer names the node."** That is placement in the consumer's manifest — a small
|
||||
game edited because a database moved, which is the fault
|
||||
[`proposal.md`](proposal.md) separates constraints from placement to avoid.
|
||||
|
||||
**Not "the consumer does not care" either.** For presence it genuinely does not: a terminal is a
|
||||
terminal. For instantiation it cares permanently, because the data lands in exactly one store
|
||||
and the wrong choice is discovered long afterwards.
|
||||
|
||||
**What the consumer does know is the scope of its own need.** Not which node — how many of the
|
||||
thing it wants, relative to itself:
|
||||
|
||||
| Scope | Means | Example |
|
||||
|---|---|---|
|
||||
| **shared** | one instance serves every instance of this consumer | the mesh's own registry: every node reads the same rows |
|
||||
| **per instance** | each instance of this consumer gets its own | a local cache, a per-node queue |
|
||||
|
||||
That is a property of the consumer, expressible without naming anything. And it is the thing
|
||||
that actually decides: a shared need cannot be satisfied by a provider each node runs
|
||||
separately, and a per-instance need should not be satisfied by a shared one.
|
||||
|
||||
**Then the mesh binds, and the binding is written down.** Not recomputed: a resolver that
|
||||
re-derives which store serves a consumer will one day derive a different answer and relocate a
|
||||
database, so the binding is made once and changed only deliberately.
|
||||
|
||||
**Where it is written down follows the scope.** A shared grant belongs to the *module* and every
|
||||
assignment of it references the same one — which is the answer for one module installed on two
|
||||
nodes wanting one database between them. A per-instance grant belongs to the *assignment*. Same
|
||||
relation, two homes, and which home is not a detail: it is what makes two instances share
|
||||
something or not.
|
||||
|
||||
The pieces that follow, none of them settled here:
|
||||
|
||||
- **When several providers satisfy the scope**, something chooses — most plausibly locality,
|
||||
preferring a provider on the same node. That is a default, and it must be overridable, because
|
||||
the reason to override it is exactly the reason nobody anticipated it.
|
||||
- **A binding is a thing that can be wrong.** Once recorded it can be inspected, and a consumer
|
||||
bound to a store on a node that no longer exists is a question somebody can be asked rather
|
||||
than a failure at connect time.
|
||||
- **Moving a binding moves data.** Whatever the mechanism, changing it is a migration and not a
|
||||
configuration change, and a design that lets it look like the latter will lose something.
|
||||
|
||||
## Provisioning is early, not late
|
||||
|
||||
An assumption worth killing: that provisioning is something the control plane does for
|
||||
consumers once a mesh is running.
|
||||
|
||||
**The mesh's own registry is a provisioned database.** So is its virtual host on the broker.
|
||||
Neither exists until something creates them, and nothing in the mesh works until they do. The
|
||||
order is:
|
||||
|
||||
```
|
||||
1 the store runs from the bundle the host carries
|
||||
2 a database is created in it a provisioning step
|
||||
3 the mesh's own schema is applied a migration, against that database
|
||||
4 the control plane starts and only now is there a mesh
|
||||
5 everything else is provisioned the ordinary path
|
||||
```
|
||||
|
||||
Steps 2 and 3 happen **before there is a mesh to do them**. So provisioning is not a
|
||||
control-plane service that consumers use; it is part of the bootstrap, and part of what the
|
||||
carried bundle has to be able to express.
|
||||
|
||||
**Which strains what a declaration is.** [ADR 0005](../../02-DECISIONS/0005-the-node-host.md)
|
||||
has the host applying *declared state on this machine*. A database inside a running store is not
|
||||
a file or a unit — and at bootstrap it is, at least, local: the store is on the same machine as
|
||||
the host applying the bundle.
|
||||
|
||||
Later it is not. A consumer on one node provisioned from a store on another is the ordinary
|
||||
case, and reaching it is not the host's job.
|
||||
|
||||
**Resolved as two mechanisms, which is the answer rather than a compromise**
|
||||
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)). The host
|
||||
runs bootstrap actions locally from the bundle; the control plane provisions across the mesh
|
||||
afterwards. Different actors, different scopes, different trust paths — so there is no single
|
||||
operation with a tier boundary running through it.
|
||||
|
||||
## Several modules, one database — and the case for refusing
|
||||
|
||||
Asked, then reconsidered by the operator: *maybe we should not allow it.*
|
||||
|
||||
The permissive version was a per-consumer **schema** inside a shared database — its own
|
||||
namespace, its own migrations, revocable by dropping the schema, with a cross-context join
|
||||
possible but deliberate.
|
||||
|
||||
**The stricter version is better, and it goes further than schemas.**
|
||||
|
||||
> **A module is only ever granted a resource it exclusively owns.**
|
||||
|
||||
No shared writes. And **no read-only role on another module's database either** — reading
|
||||
another context's tables couples you to its layout exactly as firmly as writing them does, and
|
||||
the coupling is harder to see because nothing breaks until the owner changes a column.
|
||||
|
||||
That is what [`how-we-build`](../../00-META/how-we-build.md) §4 already says: *contexts integrate
|
||||
through the record, never through a shared schema.* The permissive version kept the letter of it
|
||||
and left the temptation in place, and the path of least resistance wins eventually. A boundary
|
||||
that is merely inconvenient to cross is a boundary that gets crossed.
|
||||
|
||||
### What it costs
|
||||
|
||||
**Cross-module reporting.** Anything wanting to know what several modules hold can no longer
|
||||
join across them. It consumes their events, or calls their interface, and neither is as
|
||||
immediate as a query.
|
||||
|
||||
That cost is the point rather than a regrettable side effect — it is §4's whole argument, and
|
||||
the mesh already has both mechanisms: an event stream every context publishes to, and a tool
|
||||
surface every module exposes. What gets harder is the thing that was making work belonging to
|
||||
one context keep having to be implemented in another.
|
||||
|
||||
**One more connection per consumer.** A dozen modules means a dozen databases rather than a
|
||||
dozen schemas in one. For a relational store this is unremarkable; it is worth stating only so
|
||||
nobody discovers it as a surprise.
|
||||
|
||||
### What it deletes
|
||||
|
||||
The effort has been looking for what the design *removes* rather than adds, and this is the
|
||||
first clear instance:
|
||||
|
||||
- **Grant kinds.** There is one — an exclusive resource. No schema grants, no read roles, no
|
||||
scoping rules for who may see what inside a shared thing.
|
||||
- **The question of who owns which table**, and with it the guessing at revocation time.
|
||||
- **Cross-module migration ordering.** Two modules migrating one database need their migrations
|
||||
ordered against each other. Exclusive ownership means a module's migrations are ordered only
|
||||
against itself.
|
||||
- **A whole class of permission modelling** that a shared store would otherwise need.
|
||||
|
||||
### The rule is about contexts, not processes
|
||||
|
||||
An earlier version of this file argued that a dashboard reading a dozen stores was caught by the
|
||||
rule, because a dashboard is a surface and surfaces speak to an interface. **That was wrong, and
|
||||
it drew the line in the wrong place.**
|
||||
|
||||
The mesh's own board showing nodes, modules and deployments is not a separate context reaching
|
||||
across a boundary. It is the mesh showing its own data. Reading that store is not a violation
|
||||
however it is done, and requiring it to go through an interface to reach facts its own context
|
||||
owns would be ceremony.
|
||||
|
||||
**The rule is: a context is granted what it exclusively owns.** Everything inside that context —
|
||||
its service, its surface, its tools — reads it freely. What is forbidden is a *different*
|
||||
context reading it.
|
||||
|
||||
Which is exactly what the count below shows: the problem was never surfaces. It was three other
|
||||
contexts keeping their tables in the mesh's database.
|
||||
|
||||
**And one surface over several contexts is normal.** The board visualises the mesh, the work
|
||||
engine, the knowledge base and more, and the alternative — a separate web application per
|
||||
context — is worse for everyone who uses it. That is not a compromise with the rule; composing
|
||||
several sources into one view is what a surface *is*.
|
||||
|
||||
What it changes is only where it reads from: each context's **interface**, not each context's
|
||||
**store**. Most of that already exists — 56 of 126 modules carry a tool surface, more than carry
|
||||
a service.
|
||||
|
||||
**And the unified board is what keeps those interfaces honest.** If a view cannot be built from
|
||||
a context's interface, that interface is inadequate — discovered in the one place where it is
|
||||
cheap to notice, rather than the first time something else needs the same data and quietly
|
||||
reaches for the store instead.
|
||||
|
||||
**But "the board calls each context's interface" is not quite it either**, and the objection is
|
||||
right: if every context runs its own service with its own interface, the board is coupled to N
|
||||
of them instead of N schemas, something has to compose them, and composition is logic — which
|
||||
tier 3 says a surface does not hold. That moves the problem up a layer rather than solving it.
|
||||
|
||||
**The skeleton already answers this and the argument above talked past it.** `work` and
|
||||
`knowledge` are not separate services; they are **contexts inside the control plane**, alongside
|
||||
the record, inventory, delivery and the rest — and `api` is listed there as *the one interface
|
||||
every surface speaks to*.
|
||||
|
||||
So the board speaks to **one** interface. Behind it the contexts stay separate: separate stores,
|
||||
integrating through the record. But they are one tier, one repository, one deployable, and
|
||||
coupling *within* a tier is not what the tier rule forbids.
|
||||
|
||||
Which resolves the objection rather than deflecting it: the problem does move up a layer, and
|
||||
the layer it moves to already exists and has this as its job.
|
||||
|
||||
**The caveat is load-bearing.** This holds only while the contexts are not separate deployables.
|
||||
The moment one becomes its own service with its own interface, the board is back to N clients,
|
||||
something must compose them, and the composition has nowhere to live that tier 3 permits. That
|
||||
is a real constraint on how far the control plane may be split, and it is worth knowing now
|
||||
rather than discovering it by splitting.
|
||||
|
||||
If composing even one interface turns out too slow, the answer is a projection the board owns
|
||||
and keeps current from events — not access to somebody else's tables.
|
||||
|
||||
### Checked against the real consumers
|
||||
|
||||
The rule's survival turned on whether every reader of the mesh's own registry could be served
|
||||
some other way. **Eighteen consumers open a direct connection to it.** Four groups, and only one
|
||||
of them is work.
|
||||
|
||||
| Group | What happens under the rule |
|
||||
|---|---|
|
||||
| **The owner and its machinery** — the mesh module, the SDK, the environment and configuration synchronisers, secrets | Nothing. It owns the database. |
|
||||
| **Node appliers** — the overlay, the shell daemon, the resolver | **Already resolved.** [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) stops the host querying the mesh database, decided for tier reasons with nothing to do with this. |
|
||||
| **Foreign tenants** — the work engine (10 tables), the knowledge base (2), pipeline logs (1) | They need **their own database**. They are not reading the registry; they are storing their own data in it. |
|
||||
| **Genuine cross-context reads** — the work engine reads `nodes`; two others read a handful | The only ones needing an interface or events. |
|
||||
|
||||
**Thirteen foreign tables live in the mesh's registry database**, belonging to three separate
|
||||
contexts. That is [`how-we-build`](../../00-META/how-we-build.md) §4's shared schema, counted.
|
||||
|
||||
**So the question was the wrong shape.** The bulk of the problem is not readers needing a new
|
||||
route to data — it is **tenants needing to move out**. Tasks, agents and teams have nothing to do
|
||||
with nodes and modules; they are co-located by history. Give that context its own database and
|
||||
its dependency on the registry shrinks to a single table.
|
||||
|
||||
What remains is a handful of genuine cross-context reads, small enough to enumerate rather than
|
||||
estimate. **The rule holds.**
|
||||
|
||||
### Request or subscription, and what decides
|
||||
|
||||
The remaining cross-context reads need one or the other. **Neither is SQL** — under exclusive
|
||||
ownership a module runs SQL against its own database and nothing else, whatever transport a
|
||||
query might travel over. Both options are the mesh's own channel, and both ride the broker
|
||||
([ADR 0002](../../02-DECISIONS/0002-nodes-communicate-over-a-broker.md)), so the transport is
|
||||
not the distinction.
|
||||
|
||||
**The distinction is where the answer lives when you need it.**
|
||||
|
||||
| | request | subscription |
|
||||
|---|---|---|
|
||||
| you ask | at the moment you need to know | never — you are told |
|
||||
| the answer lives | on the other side | in your own store |
|
||||
| freshness | always current | as current as the last event you received |
|
||||
| when the other side is down | you cannot answer | you answer from your copy |
|
||||
| what you must handle | a round trip that can fail | events you missed while you were down |
|
||||
|
||||
**What decides is not taste.** [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)
|
||||
makes disconnection an ordinary situation rather than an exception. So:
|
||||
|
||||
> **Anything that must keep working while disconnected cannot use a request** — there is nobody
|
||||
> to ask. It needs a local copy, which means a subscription.
|
||||
|
||||
And the converse: anything that must be *correct at the instant of asking*, where a stale answer
|
||||
is worse than no answer, cannot use a subscription. A display can lag. A decision about whether
|
||||
a grant is still valid cannot.
|
||||
|
||||
That turns an apparently open question into a derived one. What remains genuinely open is
|
||||
narrower: **what a consumer does about the events it missed** while it was disconnected — replay
|
||||
from a point, ask once for a full picture and resume, or rebuild from scratch. That is the same
|
||||
question every projection has, and nothing in the record answers it yet.
|
||||
|
||||
## A migration belongs to the consumer and runs on the provider
|
||||
|
||||
A game defines migrations. They run against the database the store granted **it**. So the
|
||||
migration is:
|
||||
|
||||
- **owned** by the consumer — it is that module's schema, versioned with that module;
|
||||
- **hosted** by the provider — it runs inside something the consumer does not control;
|
||||
- **ordered** after the provisioning edge — there is nothing to migrate until the grant exists;
|
||||
- **scoped** to the grant — the consumer's migrations touch its database and no other.
|
||||
|
||||
Ownership crosses the edge, which nothing in *provides* and *requires* expresses. And it is the
|
||||
*action* category from [`features.md`](features.md) made concrete: not an artifact, not node
|
||||
state, and not something the host can apply, because the thing it changes is not the machine.
|
||||
|
||||
It also gives a consumer's own install an internal order — **provisioned, then migrated, then
|
||||
started** — that depends on an edge rather than on the module's contents.
|
||||
|
||||
## What still has no answer
|
||||
|
||||
**How many instances of postgres should exist?** One per mesh is wrong — a node that must work
|
||||
while disconnected cannot depend on a database elsewhere. One per node is wrong — the mesh's own
|
||||
registry is one thing, not one per node. So the answer is per-module, and nothing in the schema
|
||||
says it. This is [`cases.md`](cases.md) axis *how many instances*, and postgres is the case that
|
||||
proves it cannot be a global rule.
|
||||
|
||||
**What happens to a grant when the consumer is removed?** The game is uninstalled. Its database
|
||||
still exists, holding its data. Dropping it silently is data loss; keeping it forever is a leak.
|
||||
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md) says
|
||||
the host removes what it applied and no longer declares — but this is not on the host, it is
|
||||
inside another module's state, and the same reasoning does not obviously carry.
|
||||
|
||||
**Where does node-derived configuration come from?** (3) The control plane composes a
|
||||
declaration, and cannot know this machine's memory. Either the host fills in a blank the
|
||||
declaration leaves — which makes the host decide something, against
|
||||
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md) — or the control
|
||||
plane reads the node's inventory first and composes with it. The second is consistent and means
|
||||
a declaration is composed *per node from what the node reported*, which is a stronger claim than
|
||||
anything recorded so far.
|
||||
|
||||
|
||||
## Providing is not a substrate thing
|
||||
|
||||
The four pinned services are the obvious providers, and they are not a category.
|
||||
|
||||
| Service | What a consumer asks it for |
|
||||
|---|---|
|
||||
| a relational store | a database, a user, credentials |
|
||||
| another relational store, different vendor | a database — **and not the same one** |
|
||||
| a message broker | a virtual host, a user, permissions |
|
||||
| an object store | a bucket and keys |
|
||||
| an image registry | a repository |
|
||||
| an identity provider | a client, a realm, a secret |
|
||||
| an analytics service | a site, and a tracking identity |
|
||||
| a low-code data platform | a base, and a token |
|
||||
| an application platform | a project, which is several of the above at once |
|
||||
| a mail server | a mailbox, an alias, credentials |
|
||||
|
||||
**Any hosted service can be a factory.** Providing is a facet a module may have, not a kind of
|
||||
module it is — which is the same conclusion the effort reached about services and applications,
|
||||
arriving from the other direction.
|
||||
|
||||
That kills the last reason to keep *provider* as a category. A module runs something, or grants
|
||||
something, or both, or neither.
|
||||
|
||||
### Two stores, and why `database` still is not a name
|
||||
|
||||
Two relational stores from different vendors both grant *a database*. They are the sharpest
|
||||
possible test of the substitutability rule from [`proposal.md`](proposal.md), and they fail it
|
||||
completely: different wire protocol, different dialect, different driver, different client
|
||||
library compiled into the consumer.
|
||||
|
||||
A consumer declaring `requires: database` and being handed either would break against one of
|
||||
them. So the name promises what no provider delivers — and now with two real providers in the
|
||||
catalogue rather than a thought experiment.
|
||||
|
||||
*Database* remains a **tag**. It is how a person finds both. It is not how a consumer names what
|
||||
it needs.
|
||||
|
||||
## The same shape, three more times
|
||||
|
||||
The message broker has all nine. So does the object store, and so does the image registry. They
|
||||
differ in what a consumer asks for — a database, a virtual host, a bucket, a repository — and in
|
||||
nothing structural.
|
||||
|
||||
**A substrate service is a service plus a factory.** That is the whole pattern, and there are
|
||||
four of them. It generalises past the substrate too: anything that grants something per consumer
|
||||
has this shape, and anything that does not is the simpler case.
|
||||
|
||||
But two things differ *between* them, and both matter more than the similarity.
|
||||
|
||||
### The broker cannot be managed over the broker
|
||||
|
||||
[ADR 0002](../../02-DECISIONS/0002-nodes-communicate-over-a-broker.md) makes the broker the
|
||||
channel every node takes work from, and
|
||||
[ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md) makes it the security
|
||||
boundary — everything a node applies arrives through it.
|
||||
|
||||
So the module providing the broker is also **the way modules are managed**. A declaration cannot
|
||||
be delivered to it over itself, and reconfiguring it is done through the thing being
|
||||
reconfigured. Nothing else in the catalogue has that property; the store is consumed by the
|
||||
control plane but is not how the control plane *reaches* anything.
|
||||
|
||||
This is exactly what the carried bundle exists for
|
||||
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)): the broker is raised from
|
||||
what the host carries, before there is a channel, because there is no other way to raise it.
|
||||
Recorded here because it is a constraint on *one module*, not a general rule, and a schema with
|
||||
no way to say so hides it.
|
||||
|
||||
### Two modules of identical shape want different instance counts
|
||||
|
||||
The broker is one per mesh — a single point of failure and a single point of trust, by decision
|
||||
rather than by accident. The store cannot be: a node that must keep working while disconnected
|
||||
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)) cannot depend on a database
|
||||
somewhere else.
|
||||
|
||||
Same nine properties, opposite answers. Which settles something the cases file left open: **how
|
||||
many instances is not derivable from what a module is.** It is a decision per module, it has to
|
||||
be declared, and nothing in `provides`, `requires` or `excludes` says it.
|
||||
|
||||
### And revocation differs in consequence
|
||||
|
||||
Revoking a database leaves data behind until something drops it — a leak, and recoverable.
|
||||
Revoking a virtual host drops whatever had not been delivered — not recoverable, and silent.
|
||||
|
||||
The relation is the same and the blast radius is not, which is an argument for the provider
|
||||
deciding what revocation means rather than the mesh applying one rule to all of them.
|
||||
|
||||
|
||||
## The assignment is a third thing
|
||||
|
||||
Recorded because the operator tried the alternative and abandoned it: **modules were once
|
||||
node-agnostic**, and it did not survive contact.
|
||||
|
||||
The worked example says why. Several of a provider's nine properties are not properties of the
|
||||
module at all:
|
||||
|
||||
- **where its state lives** — a volume on a particular machine;
|
||||
- **how it is reached** — the same module on two nodes may answer locally, on the network, or
|
||||
publicly, and that is a per-assignment decision;
|
||||
- **configuration derived from the hardware** — tuning follows the memory and storage of the
|
||||
machine it landed on;
|
||||
- **whether this instance is the one** a given consumer is provisioned from.
|
||||
|
||||
None of those belong in the catalogue, because they differ per node. None belong in the node,
|
||||
because they are about this module. **They belong to the pairing**, and a design with only
|
||||
modules and nodes has nowhere to put them — which is what "node-agnostic" ran out of.
|
||||
|
||||
So there are three entities, not two:
|
||||
|
||||
> **a module** · **a node** · **an assignment**, which is a module on a node and carries its own
|
||||
> configuration
|
||||
|
||||
The current system already has this, arrived at the same way: environment values are stored per
|
||||
module *and per node*, so a module's settings differ between the machines running it.
|
||||
|
||||
**This does not put placement back in the manifest.** A module still says what must be true of a
|
||||
node and never which node ([`proposal.md`](proposal.md)). What changes is that the *result* of
|
||||
placing it is a thing with its own state, rather than a fact recorded on one of the two ends.
|
||||
|
||||
And it makes the composed declaration question from above answerable: a declaration is built
|
||||
from the module, the node's inventory, and the assignment between them. Three inputs, which is
|
||||
why two were never enough.
|
||||
@@ -0,0 +1,213 @@
|
||||
---
|
||||
status: active
|
||||
initiated: 2026-08-26
|
||||
touches:
|
||||
- 02-DECISIONS/0005-the-node-host.md
|
||||
- 02-DECISIONS/0011-managed-files-are-generated-never-edited.md
|
||||
- 03-DESIGN/01-to-be/05-the-node-host.md
|
||||
- 03-DESIGN/00-as-is/05-runtime-and-installation.md
|
||||
- 01-RESEARCH/011-the-module-graph/00-overview.md
|
||||
- 02-DECISIONS/0078-the-store-and-broker-are-modules.md
|
||||
- 02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md
|
||||
- 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
|
||||
---
|
||||
|
||||
# 012 — The minimum viable node, and adopting what is already there
|
||||
|
||||
## What is being investigated
|
||||
|
||||
Two questions that turn out to be one:
|
||||
|
||||
**What is the bare minimum to run a one-node mesh?** Not the tiers as asserted, but the actual
|
||||
closure — take the thing that must run, walk what it needs, and the set that comes back is the
|
||||
answer.
|
||||
|
||||
**And how does a machine that is already in use become that?** A candidate node is not empty. It
|
||||
has a package manager, probably a container runtime, possibly a git installation, each with
|
||||
configuration somebody chose. The mesh must **own** those, and owning is not the same as finding
|
||||
them present.
|
||||
|
||||
## Why
|
||||
|
||||
Building tier 0 reached a wall that looked like a packaging problem and is not.
|
||||
|
||||
The host can be told to run a container or install a package. Both need a file — an image, an
|
||||
archive — and the question was where the host gets it. That framing produced a bad trilemma:
|
||||
carry everything in the bundle, download at apply time, or have something push the files in
|
||||
first. Downloading fails on the first node, which cannot fetch the image registry from the image
|
||||
registry it is trying to start.
|
||||
|
||||
> **Qualified by [ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md).** The
|
||||
> reframing below still holds for what a *tailored installer* contains — the missing pieces for a
|
||||
> given machine. It does **not** have to hold for container images: the installer fetches those
|
||||
> by digest, because a real machine has a network and the sealed case is the lab.
|
||||
|
||||
**The reframing:** the machine is not offline. What matters is *when* the fetching happens. Move
|
||||
it from apply time to **build time** — build the installer on a machine that has a network,
|
||||
tailored to the target, and apply it on a target that then needs nothing. That is the same move
|
||||
the lab already made for its router image, and the same property the delivery design already
|
||||
claims: what ships is self-contained and a deploy touches no network.
|
||||
|
||||
Which makes the interesting question not *where do artifacts come from* but **what is missing
|
||||
from this particular machine**, and that needs both of the questions above answered.
|
||||
|
||||
## What tailoring implies
|
||||
|
||||
- **The binary stays generic; the payload is tailored.** One static host per architecture. What
|
||||
is machine-specific is the bundle it applies. Rebuilding the host per machine would buy
|
||||
nothing.
|
||||
- **Detection is the input, not a report.** What the host already reports about a machine —
|
||||
its profile and inventory — is what the difference is computed against. This is the first use
|
||||
of stage 1 by something other than a person reading it.
|
||||
|
||||
## Adoption
|
||||
|
||||
**Simply having a package installed is not enough.** If the mesh manages the container runtime,
|
||||
it decides that runtime's configuration; a runtime found already installed carries settings
|
||||
somebody chose, and those cannot be discovered by noticing the binary exists.
|
||||
|
||||
So a machine already in use is **adopted**: what is there is read, taken over, and thereafter
|
||||
generated.
|
||||
|
||||
**This was the original path.** [`00-as-is/05`](../../03-DESIGN/00-as-is/05-runtime-and-installation.md)
|
||||
records adoption of a pre-existing machine's configuration as the original mechanism, since made
|
||||
legacy and explicitly out of scope for the lab. It returns here for a different reason than it
|
||||
was dropped for, which is a thing to notice rather than to gloss.
|
||||
|
||||
**It creates a state that does not exist today.** [ADR 0011](../../02-DECISIONS/0011-managed-files-are-generated-never-edited.md)
|
||||
has managed files generated and never edited; adoption needs a one-time import before that rule
|
||||
starts applying. Three states, and the middle one is new:
|
||||
|
||||
> unmanaged → **adopted once** → generated
|
||||
|
||||
**And it crosses a boundary just drawn.** [ADR 0005](../../02-DECISIONS/0005-the-node-host.md)
|
||||
says the host never touches what it did not create — the rule that stops a converger deleting
|
||||
what the mesh never put there. Adoption is the deliberate act of taking ownership of exactly
|
||||
that. The rule needs a companion rather than an exception: *never, unless adoption made it the
|
||||
host's*, with adoption being explicit, recorded, and visible in what the host says it owns.
|
||||
|
||||
## Nothing is taken over without keeping what was there
|
||||
|
||||
**Before adoption touches a file, the original is kept.** Adoption happens on machines somebody
|
||||
is already using, and the configuration being taken over is configuration somebody chose. A
|
||||
one-way door on a working machine is not an installation, it is a risk nobody agreed to.
|
||||
|
||||
This is a *never* rule rather than a courtesy, and it earns that by the same incident the mesh's
|
||||
strongest rule already carries: the worst loss in this record came from a tool acting on a path
|
||||
it did not own. Adoption is that act, made deliberate — which makes the safeguard obligatory
|
||||
rather than optional.
|
||||
|
||||
What that requires, and what remains open: where the copy lives, whether it is recorded in what
|
||||
the node knows about itself so that adoption is *visibly* reversible, and whether the mesh keeps
|
||||
it forever or hands it back when it stops managing the thing.
|
||||
|
||||
## Adoption produces a briefing, not just a result
|
||||
|
||||
Proposed by the operator, and it answers a question this effort had open with two bad answers.
|
||||
|
||||
Adoption meets things a script cannot decide. A container runtime configured with one storage
|
||||
driver and a mesh wanting another. A package pinned to a version somebody chose for a reason.
|
||||
Local settings the mesh has no opinion about and no business discarding. Silently winning is
|
||||
wrong in both directions; refusing outright makes a machine in use unadoptable.
|
||||
|
||||
**So adoption has two outputs.** What it did — mechanical, recorded, in the node's state. And a
|
||||
**briefing**: what it found, what it took over, and what it could not resolve, written to be
|
||||
read by a person or an agent, which is the first thing a session on that node has to work with.
|
||||
|
||||
Conflicts are **flagged, not resolved**. That is the same principle the declaration parser
|
||||
already applies — name every problem at once, to somebody who can act on it — at a larger
|
||||
scale, and applied to a case where refusing wholesale would be worse than proceeding.
|
||||
|
||||
### On conflict, the machine's configuration is kept
|
||||
|
||||
Decided by the operator, after first deciding the opposite — recorded that way because the
|
||||
reasoning for each direction is the useful part.
|
||||
|
||||
Where the existing configuration and the mesh's disagree, **what is already on the machine
|
||||
stays**, the conflict is flagged, and it is reconciled afterwards. Adoption always **completes**
|
||||
— flags inform, they do not block — and *adopted with open questions* prevents nothing. The
|
||||
node is a node.
|
||||
|
||||
**What this buys.** Adoption becomes non-destructive by construction. The class of conflict that
|
||||
made the opposite rule dangerous — a storage driver against the filesystem it is actually on, a
|
||||
data directory pointing at a mount that exists — cannot arise, because nothing tied to the
|
||||
machine's physical reality is ever overwritten. A machine in use keeps working exactly as it
|
||||
did.
|
||||
|
||||
**What it exposes, which is the mirror of what it fixes.** The mesh's configuration is not only
|
||||
preference. Some of it is what a module needs in order to function at all. Keeping the machine's
|
||||
version there produces a module that is installed and does not work — *an installed package is
|
||||
not a capability*
|
||||
([04-ISSUES/007](../../04-ISSUES/007-an-installed-package-is-not-a-capability/00-report.md))
|
||||
arriving from a direction that issue did not anticipate. And a fleet where every node kept its
|
||||
own settings is a fleet where a module works on one node and fails on another with nothing in
|
||||
the mesh able to say why.
|
||||
|
||||
**The distinction that dissolves both rules.** Neither direction is right as a blanket, because
|
||||
the question is not *whose configuration wins*. It is whether the module **requires** the
|
||||
setting or merely **prefers** it — required contradictions cannot be kept without breaking the
|
||||
module, and preferences should always yield to what is already there.
|
||||
|
||||
That is a property of the module's own declaration rather than of the adoption algorithm, which
|
||||
makes it one more thing the graph would carry
|
||||
([research 011](../011-the-module-graph/00-overview.md)). Until modules can say which of their
|
||||
settings are load-bearing, adoption is choosing a default in the dark, and the default chosen
|
||||
here is the one that does not break the machine it is adopting.
|
||||
|
||||
### The briefing carries an outcome, and the outcome is derived
|
||||
|
||||
Proposed by the operator: the report states plainly whether adoption succeeded, partly
|
||||
succeeded or failed, and each line carries its own severity.
|
||||
|
||||
**Each line is marked, and the overall is the worst mark present.** Derived rather than stated
|
||||
alongside, because two fields written independently drift — and a briefing reading *full
|
||||
success* while carrying a failed line is exactly the fault this record keeps cataloguing. An
|
||||
outcome computed from its lines cannot disagree with them.
|
||||
|
||||
| Mark | Means |
|
||||
|---|---|
|
||||
| `ok` | done, and verified |
|
||||
| `kept` | a disagreement; the machine's value was kept and somebody should look |
|
||||
| `unknown` | could not be determined |
|
||||
| `failed` | could not be done — the node is not what was asked for |
|
||||
|
||||
**`unknown` is not a shade of success.** Adoption will meet configuration it cannot parse and
|
||||
state it cannot read, and folding those into *fine* is the same move as reporting an installed
|
||||
package as a capability. A thing nobody could determine is a thing nobody can rely on, and it
|
||||
gets its own mark for the same reason a capability detector reports *why*.
|
||||
|
||||
**And this opens something the earlier rule did not cover.** *Flags inform, they do not block*
|
||||
was decided about **conflicts** — where the mesh chose, deliberately, and the machine still
|
||||
works. A **failure** is different in kind: not *we chose* but *we could not*. Treating both the
|
||||
same makes a node where something the mesh needed never happened indistinguishable from one
|
||||
where a log level differed. Whether a failed line still lets adoption complete is therefore
|
||||
reopened by adding severity, and is not decided here.
|
||||
|
||||
## Adoption as a mode, for the migration
|
||||
|
||||
*2026-09-22.* The migration from the predecessor mesh gave the middle state a length. A machine
|
||||
running the predecessor is **adopted** when the mesh comes up on it — the predecessor's control
|
||||
stopped, the machine's firewall and files kept in force, the mesh opening what it needs through
|
||||
them — and stays adopted while its modules migrate one at a time, until the operator **converges**
|
||||
it. Measured on the control-node, and weighed against the alternatives, in
|
||||
[*migrating a node that is in use*](migrating-a-node-in-use.md); decided in
|
||||
[ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md).
|
||||
|
||||
## Open questions
|
||||
|
||||
| Question | Why it is open |
|
||||
|---|---|
|
||||
| What is the closure for a one-node mesh? | The skeleton asserts four pinned services. [Research 006](../006-mesh-from-scratch/00-overview.md) already asks whether it is four or five and does not answer. A graph gives a computed answer instead of an asserted one, which is [research 011](../011-the-module-graph/00-overview.md). |
|
||||
| Is "tier" the same thing as a graph level? | Tiers were named as a bootstrap order. If the closure is computed, tiers may be a derived view of the graph rather than a separate concept — or they may be a coarser boundary that survives for a different reason. |
|
||||
| ~~What happens when existing configuration contradicts what the mesh needs?~~ | **Answered** — the machine's configuration is kept, the conflict is flagged, and it is reconciled afterwards. |
|
||||
| ~~Do flags block, or only inform?~~ | **Answered** — they inform. Adoption always completes, and the node is a node. |
|
||||
| Can a module say which of its settings are load-bearing? | The question that dissolves the conflict rule rather than choosing a side. A setting the module *requires* cannot be kept from the machine without producing something installed and broken; a setting it merely *prefers* should always yield. Until a module can say which is which, adoption is defaulting in the dark. Belongs with the graph. |
|
||||
| Does a `failed` line still let adoption complete? | *Flags inform, they do not block* was decided about conflicts, where the mesh chose and the machine works. A failure is *we could not*, which is different in kind — and treating them alike hides the worse one behind the commoner one. |
|
||||
| How is a flagged conflict reconciled, and by whom? | The briefing hands it to a session. What that session is empowered to change, and whether the resolution is recorded so the next adoption does not re-raise it, is undecided. |
|
||||
| ~~How long is a machine adopted?~~ | **Decided** ([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)) — for as long as it is being migrated: a mode per node, recorded, ended by an explicit and previewed flip. |
|
||||
| Where does the kept original live, and for how long? | Whether it is recorded in the node's state so adoption is visibly reversible, and whether it is returned when the mesh stops managing the thing. |
|
||||
| What shape is a briefing? | Structured enough to be acted on, prose enough to be read. It is the first thing a session on a new node sees, which makes it an interface rather than a log. |
|
||||
| Does owning a package mean owning its version? | Owning configuration and owning the package are different scopes. The second means the mesh decides which version is installed, and that decision then has to survive the machine's own package manager updating it. |
|
||||
| How does a bundle stay true between building and applying? | It is built against a scan of the target. The machine can move between the scan and the apply, so the host has to verify rather than assume — and fail plainly when the bundle no longer fits. |
|
||||
| What cannot be precomputed at all? | Anything built from source on the target still needs a toolchain and a network at that moment. Tailoring moves that cost rather than removing it, and *minimal viable* has to be honest about what it cannot ship ahead. |
|
||||
| Does presence differencing understate the gap? | Knowing a package manager is installed does not say it is the version the mesh needs. A difference computed on presence alone is optimistic. |
|
||||
@@ -0,0 +1,154 @@
|
||||
# Migrating a node that is in use — adoption as a mode, not a moment
|
||||
|
||||
*2026-09-22. Measured on the machine that will be the control-node, which is running the
|
||||
predecessor mesh today. Nothing below names it; the counts are its own.*
|
||||
|
||||
## The question
|
||||
|
||||
The mesh replaces a predecessor mesh that is running, on the same machines, with the services
|
||||
people use. The control-node is decided: it is the machine that already carries the predecessor's
|
||||
broker and build pipeline. So the question is not *where* the mesh starts but **how a machine
|
||||
running the predecessor becomes a node of the mesh without its services noticing** — and then how
|
||||
each of the other machines follows.
|
||||
|
||||
## The operator's proposal
|
||||
|
||||
Proposed by the operator, 2026-09-22, and the shape this document tests:
|
||||
|
||||
1. **Stop the predecessor's control on a machine** — its daemons that write configuration: the
|
||||
network and firewall configuration above all, which decide what is reachable and what is
|
||||
blocked. Its services keep running; only the control over their configuration stops.
|
||||
2. **Bring the mesh up on that machine in adoption mode.** It takes custody of those files and
|
||||
keeps what it finds in force.
|
||||
3. **Migrate the modules one at a time**, data preserved, per the cutover procedure.
|
||||
4. **Move to the next machine and repeat** — adopted first, keeping its local configuration, then
|
||||
migrated.
|
||||
5. **When every machine is migrated, flip adoption mode**, and the mesh takes full control of the
|
||||
configuration it has been holding.
|
||||
|
||||
This is the research above made concrete. It keeps the conflict rule already decided here — *on
|
||||
conflict, what is on the machine stays* — and gives the middle state, *adopted*, a length: not a
|
||||
one-time import before generating starts, but a mode that lasts for as long as the machine is
|
||||
being migrated, ended by an explicit act.
|
||||
|
||||
## What the machine actually looks like
|
||||
|
||||
Measured, read-only:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Containers running | 60, all the predecessor's services and their stores |
|
||||
| The predecessor's control | user-level daemons, separate from the services; none of the 20 running system services is the predecessor's control |
|
||||
| Firewall | the predecessor's, active: 54 incoming rules and 52 forwarding rules, each served port allowed explicitly — how a default-deny firewall reads |
|
||||
| Files the predecessor's configuration sync writes | 12, of which 3 are system files (an ssh server drop-in, the package manager's configuration, one service's configuration); the rest are the operator's shell and agent files |
|
||||
| Per-service configuration | environment and composition files per service, written by the predecessor's service tooling |
|
||||
|
||||
**Stopping the predecessor's control stops nothing that serves.** The services are containers and
|
||||
system units that run without it; what stops is the rewriting of their configuration. Nothing
|
||||
changes on the machine until something else writes.
|
||||
|
||||
## What collides, measured rather than assumed
|
||||
|
||||
An earlier note assumed the mesh's foundation could not stand beside the predecessor because both
|
||||
want the store's and the broker's standard ports. **The measurement says otherwise.** The
|
||||
predecessor publishes its own store on a non-standard port and its broker on another; the standard
|
||||
ports the foundation binds for its store and its bus are free.
|
||||
|
||||
What does collide:
|
||||
|
||||
| The mesh wants | Held by | When |
|
||||
|---|---|---|
|
||||
| the registry's port | the predecessor's registry | at genesis — the foundation raises a registry |
|
||||
| the broker's management port, on loopback | the predecessor's broker | at genesis |
|
||||
| the private network's port | the predecessor's own tunnel | at genesis on a control-node that is the private network's hub; otherwise when the node is placed on it |
|
||||
| the web ports | the predecessor's reverse proxy | when the route proxy is assigned |
|
||||
| the resolver's port | a resolver the predecessor runs | when a resolver module is assigned |
|
||||
|
||||
On the control-node, which is the private network's hub, three of these fall at genesis and cannot
|
||||
be deferred; two only when a particular module is taken, the moment its predecessor stops anyway.
|
||||
Today the foundation's ports are **fixed**: written in the installer's bundle and in the
|
||||
catalogue's manifests, so a collision is found when a container fails to bind, not before — and a
|
||||
port changed at genesis would be changed back when the foundation is adopted as modules, since the
|
||||
applier recreates a container whose declared spec differs. The private network's address range
|
||||
must also stay clear of the range the predecessor's tunnel uses; on the machines measured they are
|
||||
distinct.
|
||||
|
||||
## What would break if the mesh came up as it is today
|
||||
|
||||
**The firewall.** Genesis loads a base ruleset — drop anything undeclared — in its own table
|
||||
([ADR 0088](../../02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md)). The
|
||||
predecessor's firewall is a different table. The kernel runs every base chain registered at the
|
||||
same hook, in priority order: an accept ends only its own chain and the packet goes on to the next,
|
||||
and a drop in any is final — whether the other firewall's chains are nftables or legacy iptables.
|
||||
So the
|
||||
mesh's base ruleset would drop everything the predecessor's firewall allows and the mesh has not
|
||||
declared — every web, mail and database port in the table above — the moment genesis ran.
|
||||
|
||||
**The files.** The host writes a declared file whatever it finds at the path, reporting it as
|
||||
updated. A file the predecessor left — the ssh server drop-in, the resolver's configuration — is
|
||||
replaced the first time a module declaring that path is assigned, before that module's service
|
||||
has moved.
|
||||
|
||||
**The container names.** The applier keys a container on its name. A catalogue module whose
|
||||
container carries the same name as the predecessor's service it replaces takes that container over
|
||||
the moment it is assigned: today, *assigning a module is migrating it*, never a preparation.
|
||||
|
||||
**The published ports.** The foundation publishes its ports on every interface, and a published
|
||||
container port reaches the container through the forwarded path, not the incoming one. A firewall
|
||||
that filters only incoming traffic never sees it. The base ruleset is what keeps the store
|
||||
unreachable from outside today — and it is the thing that cannot be loaded on this machine.
|
||||
|
||||
## The pipeline freezes while the control-node migrates
|
||||
|
||||
The predecessor's build pipeline and its coordinator run on the control-node. Stopping its control
|
||||
there stops the predecessor's updates for **every** machine it manages, until the migration is
|
||||
done. Their services keep running; they receive nothing new. That is the price of the proposal and
|
||||
it is worth stating, not a reason against it: the migration is the period in which the predecessor
|
||||
is being replaced, and it does not need to keep changing.
|
||||
|
||||
## What adoption mode has to mean
|
||||
|
||||
For the proposal to hold, *adopted* must be a **state the mesh records per node**, not an
|
||||
intention, and each thing the mesh would otherwise take must say what it does in that state:
|
||||
|
||||
- **What is found is kept until its module is taken.** *Found* is precise: present at a declared
|
||||
path or name with no record in the host's store. A found file or container is held, its original
|
||||
recorded, until the operator **takes** the module on that node — the cutover, done when the
|
||||
module's data has moved. Assigning prepares; taking migrates. Without the distinction the rule
|
||||
never fires: the host only ever sees what assigned modules declare.
|
||||
- **The firewall found on the machine stays in force.** The mesh loads no table on an adopted node
|
||||
that drops by default or holds an accept. What it needs open it declares as openings the host converges
|
||||
*through the found firewall*, on the incoming and the forwarded path, marked as the mesh's and
|
||||
re-checked on every reconcile so a reload or reboot does not lose them. An accept in a table of
|
||||
its own would not help: the found firewall's drop would still be final. What a table of its own
|
||||
*can* do is refuse, and a refusal is final too — so the mesh guards the store and the broker's
|
||||
management port from everyone but the private network and the machine itself, in a table that
|
||||
only refuses, ahead of the container runtime's redirect — which the found firewall does not do
|
||||
and cannot undo. The bus, the registry and the hub's port stay open to
|
||||
anywhere: a node enrols before it has a private-network address.
|
||||
- **The foundation's ports are the node's to give** — set at genesis, checked free, and kept as
|
||||
that node's settings, read everywhere they are used, so adopting the foundation as modules does
|
||||
not move them back.
|
||||
- **The flip is per node and previewed** from what is actually reachable — listening sockets and
|
||||
published ports, not the found firewall's allow list, which does not see what a container runtime
|
||||
forwards.
|
||||
|
||||
## Options weighed
|
||||
|
||||
| Option | Verdict |
|
||||
|---|---|
|
||||
| Cut the machine over in one go: stop the predecessor's store, broker, registry and proxy, raise the foundation in their place | Rejected. Every predecessor service goes down until it has migrated, and the predecessor's other machines lose their broker. The rollback is restarting the predecessor, which is a recovery, not a step. |
|
||||
| A separate machine as the control-node | Rejected. Contradicts the decision that the control-node is the machine that already carries the predecessor's broker and pipeline. |
|
||||
| Converge on joining, as the mesh does today | Rejected. The base ruleset closes every predecessor port at genesis, and found files are replaced before their services move. |
|
||||
| Make only the foundation's ports configurable and otherwise converge | Rejected as insufficient. It solves the bind collisions and none of the firewall or file ones. |
|
||||
| Treat assigning a module as migrating it | Rejected on review. The rule that keeps found files would never fire — the host only sees what assigned modules declare — and an assignment on an adopted node would be an outage rather than a preparation. |
|
||||
| **Adoption as a mode, per node, ended by an explicit flip** | The proposal. It is the conflict rule already decided here, given a duration. |
|
||||
|
||||
## What this leaves open
|
||||
|
||||
Answered by the decision record this feeds — [ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)
|
||||
— only for the migration's needs. The general questions above stay open: whether a module can say
|
||||
which settings are load-bearing, what shape a briefing takes, how a flagged conflict is reconciled.
|
||||
One is newly sharp: the mesh opening ports through a firewall it did not install needs to speak that
|
||||
firewall. There is one kind on the machines measured; a machine with another is not covered until
|
||||
someone writes for it.
|
||||
@@ -0,0 +1,54 @@
|
||||
---
|
||||
status: active
|
||||
initiated: 2026-09-23
|
||||
touches:
|
||||
- 02-DECISIONS/0075-two-stores-and-which-provides-what.md
|
||||
- 02-DECISIONS/0079-the-foundation-seats-are-named-after-their-servers.md
|
||||
- 02-DECISIONS/0071-genesis-builds-from-a-mesh-that-already-exists.md
|
||||
- 02-DECISIONS/0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md
|
||||
- 04-ISSUES/085-the-packages-port-given-at-genesis-is-not-a-setting/00-report.md
|
||||
- 04-ISSUES/090-the-forge-module-does-not-take-over-the-forge-genesis-raised/00-report.md
|
||||
---
|
||||
|
||||
# 013 — The forge and the registries: what a seat is for, and what it is not
|
||||
|
||||
**The question, asked during the first migration:** the mesh runs a forge that serves git and
|
||||
packages, an image registry, and a bootstrap forge genesis raises before any module exists. Should
|
||||
there be a mesh-scoped **git seat**, the way there is one for the store and the broker?
|
||||
|
||||
**The short answer: no, and the question points at a different hole.** A seat answers *is there
|
||||
exactly one of you*. Every symptom around the forge is about **an address and a credential nobody
|
||||
resolves**, which a seat does not answer. The mechanism that would answer it — a provision, with
|
||||
`serves` and a grant — already exists, is already used for the image store, and is already
|
||||
declared for packages. It simply has no consumer: the one module that needs it carries a literal
|
||||
instead.
|
||||
|
||||
See [the survey](the-survey.md) for what the code does today, with evidence.
|
||||
|
||||
## What was found
|
||||
|
||||
- **A seat does exactly two things**: it refuses a second claimant at resolution, and in one place
|
||||
it answers *where is the broker*. It is a resolution-time predicate; no host ever hears of it.
|
||||
- **Four mesh-scoped seats exist**, all named after a singular server the mesh runs **for its own
|
||||
working** — controller, store, broker, catalogue. A forge is an application the world made, not
|
||||
part of that set.
|
||||
- **Nothing in the mesh requires git.** There is no provision for it, and the forge's git service —
|
||||
over https and ssh — is declared nowhere. Only its npm half is a provision.
|
||||
- **`package-registry` is a fully built provision with zero consumers.** The builder, its only
|
||||
real consumer, bypasses it with a file naming the module, the address `127.0.0.1` and the port.
|
||||
- **The builder is told where source lives, per build, by a person.** The clone URL is an opaque
|
||||
string; each module remembers its own; no credential is ever attached; nothing polls a forge.
|
||||
- **A seat would forbid something the mesh should allow**: a second forge — one for the mesh's own
|
||||
source, one for something else — which ADR 0075 already argues for against a node-scoped claim.
|
||||
|
||||
## What this leaves open
|
||||
|
||||
- Should the forge's git service become a **provision** (`source-forge`), so the builder resolves
|
||||
the address and a credential the way it already resolves the image store? That is what would let
|
||||
a forge move, be renamed, or be replaced without editing every module's stored URL, and it is the
|
||||
unanswered half of [issue 085](../../04-ISSUES/085-the-packages-port-given-at-genesis-is-not-a-setting/00-report.md).
|
||||
- The obstacle is known and written down: such a requirement **may go unanswered during genesis**,
|
||||
and the mesh has no optional requirement. Deciding that is the real work.
|
||||
- Should the mesh learn when a source moves? It records the fact and never discovers it: nothing
|
||||
polls, and there is no receiver for a forge's push. `build --behind` answers a question only a
|
||||
person can currently make true.
|
||||
@@ -0,0 +1,130 @@
|
||||
# The survey — seats, the forge, the registries, as the code has them
|
||||
|
||||
*2026-09-23. Read from the code and the records, not from memory. Every claim here was checked
|
||||
against a file.*
|
||||
|
||||
## What a seat is
|
||||
|
||||
The glossary calls a seat *a named position at a scope with a capacity*, and a claim *a module
|
||||
taking a spot on it*. **Capacity does not exist in the code.** There is no capacity field and no
|
||||
bench; every claim is exclusive. The "shared seat" of the design is the word `provides` wearing
|
||||
that name.
|
||||
|
||||
Claiming does two things and no others:
|
||||
|
||||
- **It refuses a second claimant when a node's modules are resolved** — within one node at any
|
||||
scope, and against every other node for mesh and site scope.
|
||||
- **It answers one question, once:** the controller finds where the broker is by looking for the
|
||||
module claiming the broker's seat.
|
||||
|
||||
No host is ever told about a seat. It is a resolution-time predicate.
|
||||
|
||||
Two mechanical asymmetries matter. A mesh-scoped seat's exclusivity is defended only by nodes
|
||||
that *resolve*: a node whose modules fail to resolve contributes nothing, so a broken node does
|
||||
not hold its seat against a second claimant. And a site-scoped seat does nothing at all when
|
||||
either node has no site.
|
||||
|
||||
## The seats that exist
|
||||
|
||||
Twelve claims, twelve names. Four are mesh-scoped: the controller, the store, the broker, the
|
||||
catalogue. All four are things the mesh runs **for its own working**, and three of them are raised
|
||||
by genesis and adopted in place — which is the argument of the record that named them.
|
||||
|
||||
The node-scoped ones split in two: a **role on this machine** (the build machine, the packet
|
||||
filter, the intrusion prevention, the private network) and **a scarce machine resource** (the
|
||||
resolver's configuration file, the DNS port). The second kind is a claim for the reason a port is:
|
||||
there is one of it on the machine.
|
||||
|
||||
Two oddities worth stating:
|
||||
|
||||
- The image store claims its seat at **node** scope while providing its service at **mesh** scope.
|
||||
The record that discussed this left open whether it should hold that claim at all.
|
||||
- **Nothing claims the two public ports.** The proxy binds them and claims nothing, which is the
|
||||
hole the route handover fell into.
|
||||
|
||||
## The forge, and what it serves
|
||||
|
||||
The forge serves three things: git over https, git over ssh on its own port, and an npm registry.
|
||||
**Only the npm half is in the mesh's vocabulary** — the forge provides `package-registry` and says
|
||||
where it answers. Git is served and declared nowhere: no provision, no `serves`, nothing that can
|
||||
require it. The only trace is the public label in its route contribution, which the mesh is
|
||||
explicitly not meant to interpret.
|
||||
|
||||
The image store is a separate module and a separate provision, plain HTTP, trusted because it is
|
||||
reachable only over the private network.
|
||||
|
||||
A second module also provides `package-registry`. Two providers of one provision is not a refusal
|
||||
but an ambiguity, and the mesh already has the command that settles it: pin one.
|
||||
|
||||
## Where the builder's three addresses come from
|
||||
|
||||
| What it needs | How it finds it |
|
||||
|---|---|
|
||||
| the broker | a sealed secret of its own |
|
||||
| the image store | **a real provision binding**, resolved by the mesh |
|
||||
| the package registry | **a file in its own manifest**, naming the module, `127.0.0.1` and the port |
|
||||
| the source repository | nothing at all — a person types a clone URL per build |
|
||||
|
||||
The installer says this plainly in a comment: *the one binding nothing resolves: the builder dials
|
||||
the forge by a number it carries.* Genesis can override the port of that literal, and nothing
|
||||
else: not the forge's identity, not its address.
|
||||
|
||||
`package-registry` therefore has **zero consumers**. The builder does not require it. The
|
||||
provision is fully built — two providers, grants, a served description including the registry's
|
||||
path — and nothing asks for it.
|
||||
|
||||
## How a module's source is found
|
||||
|
||||
It is not found; it is supplied. The clone URL arrives as an argument, travels to the build machine
|
||||
and reaches `git clone` unexamined. No credential is attached, so a private forge works only if the
|
||||
build machine's own git configuration already authenticates. Each module remembers the URL it was
|
||||
built from, so there are as many forge addresses as modules, each frozen at whatever was typed.
|
||||
|
||||
At genesis there are four independent source URLs, one per flag, unrelated to each other.
|
||||
|
||||
If the forge moves or is renamed: every stored URL goes stale independently, each failing at its
|
||||
own clone, and there is no command to re-point them. Genesis keeps working, because it takes URLs
|
||||
as flags — which is the existing record's position: the forge is reached *by a name outside the
|
||||
mesh*, and failover is repointing that name.
|
||||
|
||||
## What genesis's forge is
|
||||
|
||||
A bare container run before any module exists, because the toolchain resolves the mesh's own
|
||||
packages by version from an npm registry. It holds **no seat, no manifest, no provision**. The
|
||||
mesh is told one thing about it — a port — and only when that port is not the default. **On an
|
||||
ordinary genesis the mesh never learns the bootstrap forge exists.**
|
||||
|
||||
Its successor module differs from it in five ways: container name, network, data directory, the
|
||||
address it is told to call itself, and — contradicting a comment that says they are pinned
|
||||
identically — **the image digest**. So assigning the module raises a second forge beside the first.
|
||||
|
||||
A seat cannot fix that. A seat is checked between manifests; the bootstrap forge has no manifest,
|
||||
so no seat can see it. The check that is actually wanted — same name, same data, same network — is
|
||||
between an installer constant and a manifest.
|
||||
|
||||
## Would a git seat be coherent?
|
||||
|
||||
In form, yes: add the claim and a second forge is refused. In effect it buys one refusal nobody
|
||||
has hit, and forbids an arrangement the mesh should allow — a second forge for something other
|
||||
than the mesh's own source.
|
||||
|
||||
Measured against the four mesh seats, it does not fit: those are singular servers the mesh runs
|
||||
for itself, and a forge is an application. Nothing would *use* it, because the only seat consumer
|
||||
answers "where is the broker", and a forge address is not one fact: a module's source is a
|
||||
repository **and a path and a ref**.
|
||||
|
||||
## What the code already supports and nobody uses
|
||||
|
||||
- **`package-registry` with no consumers.** The builder's literal file is a hand-rolled copy of
|
||||
the binding the mesh would have written for it.
|
||||
- **Pinning a provider** already expresses *this* forge, of several, without forbidding the second.
|
||||
- **Grants** on the forge are the credential mechanism that would hand the builder an account.
|
||||
Today the installer creates that account directly, because nothing requires the provision and so
|
||||
the grant has no consumer to write for.
|
||||
|
||||
## The reading
|
||||
|
||||
The question *should there be a git seat* is the wrong shape for the problem under it. Every
|
||||
symptom — the literal address, the port that would not follow a node, the takeover that is not a
|
||||
takeover — is about **an address and a credential nobody resolves**. A seat resolves neither. The
|
||||
provision mechanism does, and is already built.
|
||||
@@ -0,0 +1,108 @@
|
||||
---
|
||||
status: graduated
|
||||
became: 02-DECISIONS/0106-the-bus-is-nats.md
|
||||
initiated: 2026-09-23
|
||||
touches:
|
||||
- 02-DECISIONS/0002-nodes-communicate-over-a-broker.md
|
||||
- 02-DECISIONS/0033-the-substrate-is-a-store-and-a-broker.md
|
||||
- 02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md
|
||||
- 02-DECISIONS/0041-events-are-a-relationship.md
|
||||
- 02-DECISIONS/0042-the-shape-of-an-event-on-the-wire.md
|
||||
- 02-DECISIONS/0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md
|
||||
- 02-DECISIONS/0078-the-store-and-broker-are-modules.md
|
||||
- 02-DECISIONS/0083-one-push-leaves-the-mesh-consistent.md
|
||||
---
|
||||
|
||||
# 014 — The bus on NATS: replace the broker, and when
|
||||
|
||||
**The question, asked mid-migration:** the operator wants the mesh's bus — today an AMQP broker —
|
||||
replaced by NATS, with everything the broker does today. Do we finish the migration on AMQP and
|
||||
move to NATS after, or go to NATS directly?
|
||||
|
||||
**The short answer: decide NATS now, build it in the lab in parallel, and cut the mesh's bus over
|
||||
in one rehearsed rollout — after the migration's core is done and never underneath it.** "First
|
||||
everything on AMQP" is already the state and costs nothing more: the predecessor's broker was
|
||||
merged into the mesh's tonight, and every module converted from here on targets the sdk's broker
|
||||
contract, which names no protocol. The only code that speaks AMQP is the mesh's own, in three
|
||||
places, and it is swapped once.
|
||||
|
||||
## What was measured
|
||||
|
||||
**Where AMQP is spoken** (non-test files): the controller's `internal/link` (5 files, ~2.4k lines
|
||||
with tests) and the builder's main; the host's `internal/link` (~1.5k); the tool runtime's
|
||||
`broker-amqp.ts` (~600 with its main). **The sdk speaks none** — its `Broker` is `request`,
|
||||
`handle`, `publish`, `subscribe`, `close` ([ADR 0039](../../02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md)
|
||||
put the client in the runtime for exactly this). **Every module's tools and events go through the
|
||||
sdk**; one module (`amqp-ping`, a probe) talks AMQP on purpose.
|
||||
|
||||
**What the bus carries:** a control queue (`control`, `.upgrades`, `.catchup`), one queue per node
|
||||
(`node.<name>`), `builds`, the enrolment/report/alive/built flows, a topic exchange of events
|
||||
(`mesh.events.<module>.<event>`, dead-lettered to `mesh.events.dead`), an RPC exchange (`mesh.rpc`)
|
||||
and per-tool service queues (`serve.<module>.<tool>`), plus an MQTT exchange.
|
||||
|
||||
**What it relies on:** TLS for the bus a node enrols over; prefetch with reject/nack so the control
|
||||
queue **holds messages unacknowledged while the store restarts** and retries
|
||||
([ADR 0083](../../02-DECISIONS/0083-one-push-leaves-the-mesh-consistent.md)); publisher confirms and
|
||||
mandatory routing; a dead-letter exchange; **per-module accounts scoped by `emits`/`consumes`**
|
||||
([ADR 0043](../../02-DECISIONS/0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md)),
|
||||
minted through the broker's management API; the broker as a module holding the `mesh-broker` seat
|
||||
([ADR 0078](../../02-DECISIONS/0078-the-store-and-broker-are-modules.md)).
|
||||
|
||||
**The predecessor's world**, now on the same broker: ~50 AMQP connections per machine from four
|
||||
machines, 595 queues, four vhosts, six users. All of it AMQP, all of it retiring module by module.
|
||||
NATS speaks no AMQP: that world cannot move; it does not need to.
|
||||
|
||||
## How NATS answers each of those
|
||||
|
||||
| The mesh relies on | NATS | Note |
|
||||
|---|---|---|
|
||||
| topic routing keys | subjects with wildcards | same shape (`mesh.events.>`) |
|
||||
| RPC and tool invocation over an exchange + reply queue | request/reply, native | simpler than today |
|
||||
| competing consumers | queue groups | same |
|
||||
| durable control queue, hold-unacked-and-retry, catch-up | JetStream: streams, durable consumers, ack/nak with delay, replay | the 0083 guarantee moves to JetStream; core NATS alone is at-most-once and would not do |
|
||||
| dead-letter | max-deliver + advisories, or a stream fed from them | different mechanism, same effect |
|
||||
| TLS bus | TLS | same |
|
||||
| accounts scoped by emits/consumes, vhosts | accounts (isolation) with users and per-subject publish/subscribe permissions | stronger than today; the controller writes an auth config the host declares and the server reloads, instead of calling a management API |
|
||||
| management API | `nats` CLI and an HTTP monitoring endpoint | no vhost concept — accounts instead |
|
||||
| MQTT | built in | same |
|
||||
| a module seat `mesh-broker` | unchanged — the seat is the server, the module changes | ADR 0079 |
|
||||
| multi-node | clusters and leaf nodes | not needed now; a leaf per node is a later question |
|
||||
|
||||
Nothing the mesh needs is missing. The differences are in the shape of durability (JetStream must
|
||||
be declared, streams and consumers are objects) and of accounts (configuration, not API calls).
|
||||
|
||||
## The cost, honestly
|
||||
|
||||
The bus is the mesh's nervous system. Moving it means: a new `nats` module in the catalogue taking
|
||||
the `mesh-broker` seat; the controller's and the host's link packages rewritten; the tool runtime's
|
||||
client swapped behind the unchanged contract; enrolment, reports, builds and the guard's ports
|
||||
re-derived; the lab beds that prove the bus (store window, enrolment, upgrades) re-run on the new
|
||||
one; and eight decisions amended or superseded. Weeks, not days — and none of it can be done
|
||||
halfway on a live mesh: the controller, every host and every tool runtime move together.
|
||||
|
||||
## The sequencing question, answered
|
||||
|
||||
1. **NATS first, directly.** Stalls the migration for the length of the build; the mesh's bus
|
||||
changes under a half-migrated node; the predecessor's clients still need AMQP, so a second
|
||||
broker runs anyway, plus a bridge for whatever crosses. Rejected.
|
||||
2. **Finish on AMQP, NATS after.** Wastes nothing — no module written from here on speaks AMQP —
|
||||
but leaves the decision unmade while modules are written, and the runtimes idle. Adequate.
|
||||
3. **Decide NATS now; build it in the lab in parallel; cut the mesh's bus over in one rehearsed
|
||||
rollout after the core is migrated.** The predecessor's clients never notice: their broker is
|
||||
the one the mesh adopted, kept as a compatibility module with an end date — the day the last
|
||||
AMQP client is gone. **Recommended.**
|
||||
|
||||
## What this leaves open
|
||||
|
||||
- The **decision itself**, as a record: the bus is NATS; the AMQP broker becomes the predecessor's
|
||||
compatibility broker and retires with the last AMQP client. Written when the operator says so.
|
||||
- **JetStream's shape for the control plane**: one stream per concern (control, nodes, builds,
|
||||
events) or one with subjects; retention; what the store window guarantee looks like as ack-wait
|
||||
and nak-delay. Measured in the lab, not designed on paper.
|
||||
- **Accounts as configuration**: the controller writes users and permissions into a file the host
|
||||
declares, reloaded on change — which is the [ADR 0102](../../04-ISSUES/102-an-address-recorded-at-genesis-or-build-does-not-follow-the-nodes-ports/00-report.md)
|
||||
discipline applied from the start — or the JWT/operator model. The first is simpler and matches
|
||||
how the mesh already writes everything.
|
||||
- **The MCP bridge** the operator asked for the same day is written against the sdk's contract, so
|
||||
it moves with the bus and is not written twice.
|
||||
- Whether a **leaf node per machine** replaces the hub-and-spoke bus later — out of scope here.
|
||||
@@ -0,0 +1,142 @@
|
||||
---
|
||||
status: active
|
||||
initiated: 2026-09-24
|
||||
touches:
|
||||
- 02-DECISIONS/0028-the-substrate-supplies-the-control-plane-and-nothing-else.md
|
||||
- 02-DECISIONS/0033-the-substrate-is-a-store-and-a-broker.md
|
||||
- 02-DECISIONS/0048-a-provider-creates-the-credential-the-mesh-minted.md
|
||||
- 02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md
|
||||
- 02-DECISIONS/0078-the-store-and-broker-are-modules.md
|
||||
- 02-DECISIONS/0084-which-provider-serves-a-consumer.md
|
||||
- 03-DESIGN/00-as-is/03-provisioning.md
|
||||
- 03-DESIGN/01-to-be/07-the-foundation.md
|
||||
- 04-ISSUES/113-the-object-stores-images-were-withdrawn-upstream/00-report.md
|
||||
---
|
||||
|
||||
# 015 — The object store after MinIO: which S3 implementation, and how the data moves
|
||||
|
||||
**The question.** The mesh's object store is MinIO. Its community edition is archived upstream,
|
||||
its server and client images have been deleted from every public registry, and the pinned release
|
||||
is four and a half years old and will never be patched
|
||||
([issue 113](../../04-ISSUES/113-the-object-stores-images-were-withdrawn-upstream/00-report.md)).
|
||||
Which S3-compatible implementation replaces it, and what is the migration track for the data and
|
||||
the provisioning model that sit on top of it?
|
||||
|
||||
**Why now, and why not sooner.** Nothing is on fire: nodes that already hold the images keep
|
||||
running, and issue 113 establishes that the deploy path tolerates an unfetchable-but-present
|
||||
image by design. The forcing function is not an outage but a one-way door — **no node that does
|
||||
not already hold the images can ever provision the module again**, so the mesh's ability to stand
|
||||
a node up from its declarations is already broken for this module, and silently.
|
||||
|
||||
**The direction is not a departure from the design; it is the design.** The foundation document
|
||||
already states the commitment:
|
||||
|
||||
> The dependency is on the **protocol**, not the product: AMQP for the bus, S3 for the object
|
||||
> store, the OCI protocol for the registry. That is what keeps the naming safe rather than a
|
||||
> commitment that cannot be revisited.
|
||||
|
||||
The object store is also **not** a foundation service — ADR 0028 removed it, and it is an
|
||||
ordinary module required through the module graph by whatever wants one. (The "exception that is
|
||||
not a swap" in that passage is the relational store, whose provisioning model borrows PostgreSQL's
|
||||
own meaning of databases, roles and schemas. The object store carries no such coupling: a bucket
|
||||
is a bucket.) So this effort is an instantiation of an existing principle, not a redesign — which
|
||||
is the cheapest kind of decision to make and the strongest kind to cite.
|
||||
|
||||
## What the replacement has to carry, measured
|
||||
|
||||
Taken from the module's manifest, its composition, its tool surface, and a search for its
|
||||
consumers across the catalogue — not from assumption.
|
||||
|
||||
| Requirement | Evidence in the module today |
|
||||
|---|---|
|
||||
| S3 API | The protocol every consumer speaks; already the design's stated dependency. |
|
||||
| ~~OIDC login against the mesh's identity provider~~ | **Struck 2026-09-24. Not a requirement, and it never worked.** Six variables are wired and an entrypoint blocks on the provider, which reads as a live feature. The module's own hook comment records the end state as *"policy claim missing"* — a failing login. See [01](01-candidate-comparison.md). |
|
||||
| **Per-application access keys, each scoped to a bucket** | The real requirement. A "user" of the store is normally an application; the mesh already mints a credential per provisioned bucket. |
|
||||
| **One live consumer using it as opaque primary storage** | A file-sync application, since early 2023: objects named by internal id, metadata in its own database. Highest-risk consumer — a live copy drifts, and its bucket name must be preserved. |
|
||||
| Erasure-coded multi-node topology | Four server nodes with two data directories each, behind a load balancer. |
|
||||
| A single-node form | Declared as a flavour, for development and small nodes. |
|
||||
| Buckets as a typed provision | The module declares a provision type of `bucket` on a named network; the mesh mints the credential and the provider creates it (ADRs 0048, 0084). |
|
||||
| A tool surface | Bucket create/list/delete, object list/info/delete, presigned URL, and provisioning. |
|
||||
| A console | Published on its own subdomain through the reverse proxy, with an unlimited request-body middleware for uploads. |
|
||||
|
||||
**Consumers, counted:** one application module, one capture module that takes a private bucket per
|
||||
node, one workflow module's tools, and the delivery/rescue internals of the shared library. The
|
||||
surface is small — the cost is concentrated in the provisioning handler, the tool handlers and the
|
||||
OIDC story, not spread across the catalogue.
|
||||
|
||||
## Candidates
|
||||
|
||||
**Four candidates, not three.** The comparison was briefly narrowed to SeaweedFS on the strength of
|
||||
console single sign-on; that axis turned out not to be a requirement, and the incumbent's own
|
||||
maintained fork had been omitted altogether. Both errors, and why they happened, are recorded in
|
||||
[01 — the candidates measured](01-candidate-comparison.md), which carries the evidence and the
|
||||
requirement-by-requirement detail.
|
||||
|
||||
In short, and only in short:
|
||||
|
||||
- **The maintained fork of the incumbent** — the community edition was archived and its images
|
||||
deleted, but a fork publishes, tracks CVEs, and preserves the on-disk format, S3 API and
|
||||
environment surface. Costs **an image reference** where every other option costs a data
|
||||
migration, two rewrites and a maintenance window. Does not end the dependence on an abandoned
|
||||
codebase; buys time to choose deliberately.
|
||||
- **Garage** — its permission model *is* the requirement (per access key, per bucket), its admin
|
||||
API is the closest match to how the mesh provisions, and the highest-risk consumer is
|
||||
first-party documented against it. Remaining cost: no object versioning, no server-side
|
||||
encryption or object locking, partial lifecycle — **unmeasured against the ten buckets, and the
|
||||
one thing that could still disqualify it**.
|
||||
- **SeaweedFS** — longest field record and erasure coding. Its console sign-on is a paid feature,
|
||||
which is now beside the point. What weighs against it is narrower: its S3 surface is a gateway
|
||||
translating onto its own file-system API, with no first-party support for the opaque consumer.
|
||||
- **RustFS** — closest in shape to the incumbent, so the least porting. But it reached general
|
||||
availability eight days before this was written, and carries an open defect in the credential
|
||||
path. Two earlier claims about it are corrected in 01: it is **not** a drop-in that retains
|
||||
existing data.
|
||||
- **Ceph RGW** — remains rejected as disproportionate where the object store is an ordinary
|
||||
module rather than a platform.
|
||||
|
||||
**This is now two decisions, not one:** whether to repoint to the fork or migrate, and — if
|
||||
migrating — to which. Repointing does not foreclose migrating, which is the argument for taking it
|
||||
first. On the corrected requirement the migration ranking is Garage, then SeaweedFS, and not yet
|
||||
RustFS. Two measurements gate any graduation: **which S3 endpoints the consumers actually call**
|
||||
(Garage cannot be ranked fairly until counted), and **whether the fork can read the incumbent's
|
||||
on-disk format in place** — tested on a copy, because the migration between them is one-way. Both
|
||||
are in [01](01-candidate-comparison.md#what-is-still-unmeasured).
|
||||
|
||||
## The migration track, in outline
|
||||
|
||||
Data movement is the easy half, and deliberately reversible.
|
||||
|
||||
1. **Stand the replacement up beside the incumbent**, on its own ports, its own provision type and
|
||||
**its own data directory**. Nothing removed. The data directory matters: reusing one the
|
||||
incumbent already holds would put a fresh single-drive store on top of a live erasure set.
|
||||
2. **Copy bucket by bucket with a neutral tool.** `rclone` rather than the incumbent's own client
|
||||
— the client has been withdrawn upstream too, so building the migration on it would inherit
|
||||
the same dependency this effort exists to remove.
|
||||
3. **Verify per bucket** — object counts and checksums, not a transfer exit code.
|
||||
4. **Repoint consumers through the connection the module already publishes.** Consumers read an
|
||||
API URL from the module's declared connections rather than addressing the store directly, so
|
||||
the cutover surface is that value plus the provisioning and tool handlers.
|
||||
5. **Freeze writes, final incremental sync, flip**, and keep the incumbent read-only as the
|
||||
rollback until confidence is earned. For the opaque consumer this is **not optional and not
|
||||
instant**: it stores objects by internal id with metadata in its own database, so a copy taken
|
||||
while it runs will drift. It needs a maintenance window for the final sync, and the window is
|
||||
proportional to 82,496 objects rather than to 230 GiB.
|
||||
6. **Retire**, and only then remove the module.
|
||||
|
||||
The genuinely new work is not the copy. It is the **provisioning handler** and the **tool
|
||||
handlers**, both written against the incumbent's admin API. *The OIDC wiring was previously listed
|
||||
here and is struck: it is not a requirement and it never worked.*
|
||||
|
||||
## Open questions
|
||||
|
||||
- ~~How much of the OIDC requirement survives, and in which build?~~ **Answered, and it was the
|
||||
wrong question.** The console requirement does not exist, and the login it referred to never
|
||||
worked. What replaced it: which S3 endpoints consumers actually call, and whether the fork reads
|
||||
the incumbent's format in place.
|
||||
- Does the mesh's bucket provision translate to the candidate's identity model without weakening
|
||||
what ADR 0049 says about a consumer's identity fitting the tightest backend?
|
||||
- Should this effort also answer issue 113's general question — mirroring third-party images into
|
||||
the mesh's own registry — or is that a separate decision? Replacing one withdrawn product with
|
||||
another unmirrored upstream leaves the same one-way door in place, just further from the hinge.
|
||||
- Is the four-node erasure-coded topology still warranted, or was it inherited? Worth re-asking
|
||||
while the product is being chosen, rather than reproducing a shape by default.
|
||||
@@ -0,0 +1,176 @@
|
||||
# 015 / 01 — The candidates measured
|
||||
|
||||
*Rewritten 2026-09-24. An earlier version of this document ranked the candidates on whether they
|
||||
preserved single-sign-on to the object store's **console**. That was the wrong axis — it is not a
|
||||
requirement — and a fourth candidate was missing entirely. Both errors are recorded at the end,
|
||||
because how a comparison came to be ranked on the wrong thing is worth more than the ranking was.*
|
||||
|
||||
## The requirement, corrected
|
||||
|
||||
Taken from the operator and from the running system, not from the module's shape.
|
||||
|
||||
**A "user" of the object store is normally an application.** The requirement is therefore
|
||||
**per-application access keys, each scoped to its own bucket** — not per-human single sign-on. The
|
||||
mesh already works this way: it mints a credential for every provisioned bucket, and the consumer
|
||||
reads an endpoint from the module's declared connection rather than addressing the store directly.
|
||||
|
||||
**The console is not a requirement.** It was the axis the previous version ranked on, and it should
|
||||
not have been.
|
||||
|
||||
**The identity-provider login never worked.** The predecessor's module wires six OIDC variables and
|
||||
blocks startup until the provider answers, which reads like a working feature. It is not: the
|
||||
module's own hook comment records the end state as *"policy claim missing"* — a **failing** login,
|
||||
written up as progress because it proved the provider had registered. The identity provider emits no
|
||||
such claim, nothing in the module creates the mapper, and the configured scope alone would not carry
|
||||
a custom one. Two days of logs show no genuine login attempts, only internet scanners failing on an
|
||||
STS API version. **Nothing should be carried forward on the assumption this works**, and no
|
||||
candidate should be credited or penalised for matching it.
|
||||
|
||||
**One consumer is live, opaque, and holds real user files.** A file-sync application has used the
|
||||
store as its **primary storage** since early 2023: objects named by an internal id, with all
|
||||
metadata in its own database. Three consequences — a copy taken while it runs will drift, its bucket
|
||||
name must be preserved or its database references break, and it is the highest-risk consumer of the
|
||||
lot.
|
||||
|
||||
## What is actually stored, measured
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Logical | **230 GiB, 82,496 objects, 10 buckets** |
|
||||
| Raw on disk | **468 GiB** — eight drive directories at 59 GiB each |
|
||||
| Implied scheme | 468 ÷ 230 = **2.03×**, confirming erasure coding at half parity |
|
||||
| Headroom | ~1.3 TiB free on the filesystem holding it |
|
||||
|
||||
**All eight "drives" are directories on one filesystem on one machine.** The erasure coding is
|
||||
therefore not buying independent-drive redundancy; the real failure domain is the array underneath,
|
||||
which has its own. This single fact decides more of the comparison than any product feature: a
|
||||
scheme's redundancy model is close to irrelevant here, and what remains is its storage overhead.
|
||||
|
||||
At 230 GiB with 1.3 TiB free, **storage overhead is not a deciding cost either.** Replication at
|
||||
three copies would run ~690 GiB against the present 468 GiB — about **+222 GiB**, comfortably
|
||||
absorbed. Erasure coding at a wider stripe would *save* roughly 146 GiB. Both are rounding errors
|
||||
against the headroom, and neither should decide this.
|
||||
|
||||
## The candidates
|
||||
|
||||
Four, not three. The previous version omitted the first.
|
||||
|
||||
### The maintained fork of the incumbent
|
||||
|
||||
The community edition was archived upstream and its images deleted
|
||||
([issue 113](../../04-ISSUES/113-the-object-stores-images-were-withdrawn-upstream/00-report.md)),
|
||||
but **a fork is maintained and publishing** — `pgsty/minio`, from the Pigsty project. It restores
|
||||
the console stripped from the community
|
||||
build, rebuilt image and package distribution, tracks CVEs, and states that it preserves the on-disk
|
||||
format, the S3 API and the environment-variable surface. Verified by pulling it: it reports a
|
||||
current release, permissive-to-copyleft licensing unchanged from upstream, and identifies itself as
|
||||
a community fork. Adoption is real — the server image has been pulled three quarters of a million
|
||||
times.
|
||||
|
||||
**Why it reorders the comparison.** Every other candidate costs a data migration, a provisioning
|
||||
handler rewritten against a different admin API, a tool surface ported, and a maintenance window for
|
||||
the opaque consumer. The fork costs **an image reference**. It also closes the issue's one-way door:
|
||||
a node holding nothing can provision the module again, and patches resume.
|
||||
|
||||
**What it does not do** is end the dependence on a codebase its original authors abandoned. It is
|
||||
maintenance mode, largely one project's effort, with no new features intended. It buys time to
|
||||
choose deliberately rather than under pressure — which is worth a great deal, and is not the same as
|
||||
a decision.
|
||||
|
||||
### Garage
|
||||
|
||||
**The best fit for how the mesh provisions.** Its permission model is *per access key, per bucket,
|
||||
read/write/owner* — which is the requirement above stated verbatim rather than approximated. Its
|
||||
admin API is a first-class REST surface with tokens scopeable to exactly the two operations a bucket
|
||||
provision performs. The opaque consumer is **first-party documented** against it, for primary
|
||||
storage, including client-side encryption support.
|
||||
|
||||
Its previously-recorded penalties mostly dissolve under the corrected requirement: it has no console
|
||||
and no identity-provider integration, neither of which is wanted; and it replaces AWS-style ACLs and
|
||||
bucket policies with its own per-key-per-bucket model, which is the thing being asked for.
|
||||
|
||||
**What genuinely remains.** It replicates rather than erasure-codes — immaterial at this volume and
|
||||
on a single array, as above. It does **not implement the full span of S3 endpoints**: object
|
||||
versioning is absent, object locking and server-side encryption endpoints are absent, and lifecycle
|
||||
is partial. **Whether any of the ten buckets depends on those is unmeasured, and it is the one thing
|
||||
that could still disqualify it.**
|
||||
|
||||
### SeaweedFS
|
||||
|
||||
Longest field record of the group, permissive licence, erasure coding, and identity-provider
|
||||
integration on the S3 API through token exchange. **Its console sign-on is a paid feature** — the
|
||||
admin UI itself is open, its identity integration is not. That finding is what falsified the
|
||||
previous version's narrowing, and it is now largely beside the point, since the console is not a
|
||||
requirement.
|
||||
|
||||
What weighs against it here is narrower and more specific: its S3 surface is a **gateway
|
||||
translating onto its own file-system API**, with acknowledged divergence from AWS behaviour at the
|
||||
edges, and there is no first-party documentation for the opaque consumer. For a store already
|
||||
holding real user files in an opaque layout, first-party support is worth more than a feature list.
|
||||
It also carries more moving parts than a single-machine deployment needs.
|
||||
|
||||
### RustFS
|
||||
|
||||
Closest in shape to the incumbent — a similar admin API and client compatibility, so the existing
|
||||
handlers would port with least effort — under a permissive licence, with erasure coding and a
|
||||
console that does integrate an identity provider.
|
||||
|
||||
Two things were recorded about it earlier that were **wrong, and are corrected here**: it is *not* a
|
||||
binary-level drop-in that retains existing data (API compatibility and on-disk compatibility are
|
||||
separate paths, and the on-disk one is preview-scoped with documented encryption limits), and it
|
||||
therefore offers no shortcut around the migration. It also carries **an open defect in the exact
|
||||
area the mesh depends on** — an access key created by an identity-provider user reported denied on
|
||||
all S3 operations.
|
||||
|
||||
Decisively for now: **it reached general availability eight days before this was written.** For a
|
||||
component holding 230 GiB of real user files, field record is a feature, and it does not have one.
|
||||
|
||||
### Ceph RGW
|
||||
|
||||
Remains rejected, for the reason already recorded: disproportionate where the object store is an
|
||||
ordinary module rather than a platform.
|
||||
|
||||
## Where this leaves it
|
||||
|
||||
**The decision is no longer "which product replaces the incumbent".** It is two decisions, and they
|
||||
can be taken in either order but should not be confused:
|
||||
|
||||
1. **Repoint to the maintained fork, or migrate now?** Repointing is an image reference and it
|
||||
closes the issue. Migrating now costs a data copy, two rewrites and a maintenance window, and
|
||||
buys independence from an abandoned codebase sooner.
|
||||
2. **If migrating, which?** On the corrected requirement the ranking is **Garage first** — its
|
||||
permission model *is* the requirement, its provisioning API is the closest match, and the
|
||||
highest-risk consumer is first-party supported. SeaweedFS second, on field record, with a
|
||||
translation-layer caveat that matters more here than its feature list. RustFS not yet, on age.
|
||||
|
||||
Taking (1) does not foreclose (2), and that asymmetry is the argument for taking (1) first.
|
||||
|
||||
## What is still unmeasured
|
||||
|
||||
1. **Whether any of the ten buckets needs object versioning, server-side encryption or lifecycle.**
|
||||
This gates Garage specifically and nothing else here answers it.
|
||||
2. **Whether the fork's release can actually read the incumbent's on-disk format in place.** The
|
||||
format is claimed compatible across a multi-year gap; the migration between them is one-way, so
|
||||
this is tested on a copy or not at all.
|
||||
3. **Whether the opaque consumer's maintenance window is acceptable**, and how long it actually is
|
||||
at 82,496 objects.
|
||||
4. **Whether the eight-drive erasure-coded shape is warranted at all.** The evidence above says it
|
||||
is not buying what it appears to: eight directories, one array, one machine. It looks inherited.
|
||||
|
||||
**Nothing graduates to a decision before 1 and 2.**
|
||||
|
||||
## Two errors in the previous version of this document
|
||||
|
||||
Recorded because the shape of both survives anonymisation and neither is unique to this effort.
|
||||
|
||||
**It ranked on a requirement that did not exist.** Console single sign-on was treated as the axis
|
||||
because the module's configuration showed it wired up, and a wired-up configuration was read as a
|
||||
used feature. It was neither used nor working. *A configured feature is not an observed one*, and
|
||||
the evidence needed was the operator's answer and the logs — both cheap, neither consulted before
|
||||
the ranking was written.
|
||||
|
||||
**It omitted the incumbent's own fork.** The whole effort began because an upstream withdrew its
|
||||
images; whether anyone had continued that upstream was the first question to ask and it was not
|
||||
asked. The candidate list was assembled from a search for *alternatives*, which by construction
|
||||
returns things that are not the incumbent. *When a dependency dies, "who took it over" precedes
|
||||
"what replaces it".*
|
||||
@@ -0,0 +1,57 @@
|
||||
---
|
||||
status: graduated
|
||||
became:
|
||||
- 02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md
|
||||
- 03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md
|
||||
initiated: 2026-09-26
|
||||
touches:
|
||||
- 02-DECISIONS/0113-the-vault-makes-every-secret.md
|
||||
- 02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md
|
||||
- 02-DECISIONS/0048-a-provider-creates-the-credential-the-mesh-minted.md
|
||||
- 03-DESIGN/01-to-be/13-credentials-and-their-rotation.md
|
||||
- 03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md
|
||||
- 04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md
|
||||
---
|
||||
|
||||
# 016 — How a credential can be rotated
|
||||
|
||||
**What.** Which rotation mechanisms the mesh's providers can actually support, measured against
|
||||
their code rather than assumed. Every provider in the catalogue was read, found by listing every definition that provides something: how it names what it
|
||||
makes for a consumer, what its remove destroys, whether it re-applies a password, whether its
|
||||
backend can hold two secrets for one login or two logins on one resource, and how its own
|
||||
administrative credential is set. The consumer side was read too: when a module reads a secret, and
|
||||
what makes it read a new one.
|
||||
|
||||
**Why.** [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md), as first drafted,
|
||||
chose *overlap*: add a second login beside the first, move every reader, then remove the old one,
|
||||
"through the adapter's existing create and remove", with "no consumer changes". A review showed that
|
||||
claim false. In most providers the consumer's data is named after its login, and remove drops the data
|
||||
with the login. Overlap as written would have deleted every consumer's database on its first
|
||||
rotation. The mechanism has to be chosen on what the providers do.
|
||||
|
||||
**What it touches.** Rotation in 0113 and [to-be 27](../../03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md),
|
||||
which [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md) decided on
|
||||
these findings. The identity budget in
|
||||
[ADR 0049](../../02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md), if a consumer
|
||||
gets two logins. The rotation already implemented, which [to-be 13](../../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md)
|
||||
describes.
|
||||
|
||||
**Documents.**
|
||||
|
||||
- [01 — The providers](01-the-providers.md): the survey, one row per provider, and what it shows.
|
||||
- [02 — The readers](02-the-readers.md): how a secret reaches a running process, and what already
|
||||
recreates it.
|
||||
- [03 — The options](03-the-options.md): each rotation mechanism against those facts, and a
|
||||
recommendation.
|
||||
|
||||
**Finding, in one paragraph.** All nine credential providers already re-apply a consumer's password
|
||||
in place on every create, and the controller's `rotate` command relies on that. It is a working
|
||||
rotation with a stated window. Eight of the nine name the consumer's resource after its login, and five
|
||||
destroy the consumer's data when they remove the login. The harness, keyed by login, would do the same
|
||||
on any change of login. Only one backend holds two passwords on one login, and two more hold several
|
||||
tokens. Eight backends can grant two logins the same rights over one resource; the ninth can give one
|
||||
login a second token. So every provider can hold **two credentials** over one resource, but only after
|
||||
each adapter separates *the consumer's resource* from *the credential that reaches it*. In postgres
|
||||
that also means the resource belongs to a role no login owns. Administrative credentials are a
|
||||
different case. They have one party and a fixed name, and five backends take them only at first
|
||||
initialisation, so changing one needs the old and the new value at once.
|
||||
@@ -0,0 +1,103 @@
|
||||
# 01 — The providers
|
||||
|
||||
Read from the catalogue's main branch: each provider's provisioner adapter (`create`, `remove`),
|
||||
the client functions they call, and each definition's own credentials. The providers were found by
|
||||
listing every definition that provides something and has a provisioner, not from memory. A first pass
|
||||
of this survey worked from memory and missed one, mailu.
|
||||
|
||||
**The provisioner harness** in `mesh-sdk` calls `create` for a consumer when its contribution appears
|
||||
or changes (its login, password or values), and after the provisioner restarts. It calls `remove` for
|
||||
a login it applied earlier in the same process that is no longer contributed. Its record of what was
|
||||
applied is kept in memory and keyed by login. Two things follow:
|
||||
|
||||
- a consumer whose derived login changes is removed under the old login and created under the new one,
|
||||
in one pass;
|
||||
- a contribution that disappears while the provisioner is down is never removed, and is left behind.
|
||||
|
||||
## The credential providers
|
||||
|
||||
`login` is the consumer's derived identity, which the adapter receives as `as`
|
||||
([ADR 0049](../../02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md)).
|
||||
|
||||
| provider | the consumer's resource is named | remove destroys | create re-applies the password | two secrets on one login | two logins on one resource |
|
||||
|---|---|---|---|---|---|
|
||||
| postgres | a database named `login`, owned by the role `login` | the database and the role | yes, `ALTER ROLE … PASSWORD` when the role exists | no: a role has one password | yes, but only through a role that cannot log in owning the database, with each login working as it. Otherwise whatever one login creates is its own, and dropping that login means handing its objects over first. Not done today |
|
||||
| mssql | a database named `login`, with the login mapped into it | the database and the login | yes, `ALTER LOGIN … WITH PASSWORD` | no: a login has one password | yes, two logins mapped to users in `db_owner`. A user owning a schema cannot be dropped, and a login with an open session cannot. Not done today |
|
||||
| mongodb | a database named `login`, with a user holding `dbOwner` | the database and the user | yes, `updateUser` with the new password | no: a user has one credential | yes, two users with `dbOwner` on one database. Not done today |
|
||||
| redis | the key prefix `login:` on an ACL user named `login` | the user, **not** its keys | yes: `ACL SETUSER … reset … >password` replaces all of them | **yes**: an ACL user holds several passwords, added with `>` and removed with `<`. Today's `reset` discards all but the new one | yes, two users on one key prefix, once the prefix is not the login |
|
||||
| minio | a bucket derived from `login`, and a service account whose access key is `login` | the access key; the bucket **only if empty**. A bucket holding objects is left, and the failure logged | yes, by removing the access key and adding it again, which leaves a moment with no key | no, but an access key *is* the login: a second key is a second login | yes, two service accounts with one bucket policy. The access key is capped at 20 characters |
|
||||
| lavinmq | a virtual host named `login`, and a user named `login` with permissions on it | the virtual host, with any queued messages, and the user | yes, the user is written again with the password | no: a user has one password | yes, permissions for two users on one virtual host |
|
||||
| mosquitto | a client named `login`, with a role named for it on the topic prefix `login/#` | the client and its role | yes, the password is set when the client exists | no: a client has one password | yes, two clients holding one role, once the prefix is not the login. The MQTT client identifier is chosen by the consumer, not tied to the login; a duplicate one takes the older session over |
|
||||
| mailu | a mailbox `login@domain`, unless the consumer contributes its own account name | the mailbox with its mail, for a login-named one; a contributed name is left for an operator | yes, the password is set when the user exists | no for the password; a user can hold several authentication tokens, per the backend's documentation | **no**: a mail user *is* its mailbox |
|
||||
| gitea (npm) | a user named `login` on a team of an organisation that owns every package | the user; **packages survive**, because the organisation owns them | yes, the user's password is set on every run | no for the password; a user can hold several access tokens | yes, trivially: a second member of the same team |
|
||||
|
||||
## The other providers
|
||||
|
||||
| provider | answers with | credential |
|
||||
|---|---|---|
|
||||
| umami | a website, found by its public name | none. The site id it makes has no way back to the consumer today |
|
||||
| cloudflare-dns | a public name derived from `login` | none handed to the consumer; its own API token is an operator value |
|
||||
| showcase | a route | none |
|
||||
| mesh-vault | custody: it records and withdraws sealed values in a ledger | it holds secrets; it makes none today |
|
||||
|
||||
verdaccio provides the npm registry too, and has no provisioner.
|
||||
|
||||
## Each provider's own administrative credential
|
||||
|
||||
| provider | identity | how the backend takes it |
|
||||
|---|---|---|
|
||||
| postgres | a fixed superuser | from a file **only at first initialisation** |
|
||||
| mssql | `sa` | from the environment at first setup. The image documents no file form, and the definition records that as a declared exception |
|
||||
| mongodb | a fixed `root` | from a file **only at first initialisation**, when the data directory is empty |
|
||||
| mosquitto | a fixed admin client | seeded into the broker's dynamic-security file **once**; the seeding step skips when the file exists |
|
||||
| lavinmq | a fixed admin name | per its own bootstrap code, **only on a first boot** with an empty data directory. No resource in the definition runs that bootstrap; what sets it on a running mesh is outside the catalogue |
|
||||
| redis | the default user | from `requirepass` in a configuration the mesh renders, read when the server starts |
|
||||
| minio | a fixed root user | from a file, read when the server starts |
|
||||
|
||||
**In five of seven, a new administrative value takes effect only through a command run with the old
|
||||
one.** The credential file is mounted directly into both the server and the provisioner. So replacing
|
||||
it recreates the provisioner, which then holds only the new value while the backend still expects the
|
||||
old one, and the provisioner is locked out. That is worse than changing nothing.
|
||||
|
||||
Every provider module also has its own bus account, an own secret, read at start.
|
||||
|
||||
## What the tables show
|
||||
|
||||
1. **Every credential provider already rotates in place.** All nine re-apply the password on the
|
||||
same login each time `create` runs. The controller's `rotate` command relies on that: it replaces
|
||||
the credential in the inventory and sends both ends in one push. Its own comments state the window,
|
||||
between the provider applying and the consumer restarting, in which the consumer cannot
|
||||
authenticate.
|
||||
2. **Eight of nine name the consumer's resource after its login.** Only gitea separates them,
|
||||
because an organisation owns the packages. A second login therefore has no resource of its own to
|
||||
reach, and cannot share the first one's without the adapter granting it.
|
||||
3. **Five of nine destroy the consumer's data when they remove the login**: postgres, mssql and
|
||||
mongodb drop the database, lavinmq drops the virtual host with its queued messages, and mailu
|
||||
deletes the mailbox with its mail. minio drops only an empty bucket, and redis leaves the keys. In
|
||||
those five, *retire a login* and *delete the consumer's data* are one call. With the harness keyed
|
||||
by login, a changed login triggers it too.
|
||||
4. **One backend holds two passwords on one login** (redis). Two hold several tokens beside one
|
||||
password (gitea and mailu). A rotation built on two secrets per login would work for three
|
||||
providers out of nine.
|
||||
5. **Eight of nine can give two logins the same rights over one resource.** Group roles in postgres,
|
||||
database roles in mssql and mongodb, permissions in lavinmq, a shared role in mosquitto, a shared
|
||||
policy in minio, a shared key prefix in redis, a shared team in gitea. mailu cannot, because its
|
||||
user is its mailbox, but it can give one user a second token. So every provider can hold **two
|
||||
credentials** over one resource, though not every one as two logins. No adapter does either today.
|
||||
6. **Ownership is a trap in two backends.** In postgres whatever a login creates is that login's, so a
|
||||
second login cannot alter the first one's tables, and the first cannot be dropped while it owns
|
||||
them. The one-step way out deletes them. In mssql, a login cannot be dropped with a session open,
|
||||
nor its user while it owns a schema.
|
||||
7. **The administrative credentials have one party and a fixed name**, and five backends take them
|
||||
only at first initialisation. The provisioner needs the old and the new value at once to change
|
||||
them. Today nothing can give it both.
|
||||
8. **A consumer's identity is already the resource's name.** The login is derived from the
|
||||
assignment, which is a module on a node, so the current login and "the consumer" are the same
|
||||
string today. A second login would need a new name. The resource can keep the one it has.
|
||||
|
||||
## Seen on the way
|
||||
|
||||
The redis configuration names no ACL file, so a consumer's ACL user exists only in memory. A restart
|
||||
of the redis server erases every consumer's user. The provisioner does not create them again until it
|
||||
restarts itself, because its in-memory record says they are done. That is not a rotation finding, but
|
||||
it is a live fault, and it is recorded here so it is not lost.
|
||||
@@ -0,0 +1,46 @@
|
||||
# 02 — The readers
|
||||
|
||||
How a secret reaches a running process, and what makes the process take a new one.
|
||||
|
||||
## No module watches a secret
|
||||
|
||||
A search of every module's code in the catalogue found no file watching of any kind, and no
|
||||
re-reading of a secret while running. **Every reader reads a secret when it starts.** There is no
|
||||
consumer that takes a new value live, so every rotation that changes what a consumer presents ends in
|
||||
the consumer restarting.
|
||||
|
||||
## The host already recreates what read a changed file
|
||||
|
||||
The node host records, for every long-running container, the digest of each file it read when it
|
||||
was created: its env-files, and every file bind-mounted into it directly. When a digest changes, the
|
||||
host recreates the container, even though its spec is otherwise unchanged. This is the fix for
|
||||
[issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md).
|
||||
It is on the host's main branch, while the issue is still recorded as located, not fixed.
|
||||
|
||||
Two cases are deliberately left out and need `restart-on` in the definition:
|
||||
|
||||
- a file read out of a **mounted directory**, because the host cannot know whether the service reads
|
||||
it once or watches it (a route proxy re-reads its routes live; a provisioner polls what it receives);
|
||||
- a **process** rather than a container.
|
||||
|
||||
## What the catalogue does with it
|
||||
|
||||
22 definitions declare a secret they receive. In 16 of them it reaches the service through a
|
||||
rendered file, usually an env-file. That case the host already covers. 14 declare `restart-on` for
|
||||
something. Whether each of the 22 is fully covered depends on how its secret travels: through an
|
||||
env-file or a direct mount, which the host covers, or through a directory or into a process, which
|
||||
needs `restart-on`. **That was not classified module by module.** It is the check to run before a
|
||||
rotation mechanism relies on it.
|
||||
|
||||
The count covers only the `secrets` field. The 49 modules with their own bus account, and 54 with any
|
||||
own secret, are readers too, and their bus accounts are rotated like any credential two parties hold.
|
||||
Their files are mounted directly, which the host covers, but the classification has to name them.
|
||||
|
||||
## What this means for rotation
|
||||
|
||||
- The *read at start* half of 0113's recipient model is already true, and mostly already handled by
|
||||
the host. The restart is derived from the files a container reads, not declared per secret.
|
||||
- Any mechanism, in place or overlapping, ends with the reader being recreated. What differs is
|
||||
whether the credential it held until then still works.
|
||||
- For a single-party secret, a module's own, the reader is also the only holder. There is nobody to
|
||||
overlap with, and delivering the new file recreates the reader.
|
||||
@@ -0,0 +1,79 @@
|
||||
# 03 — The options
|
||||
|
||||
Three mechanisms, weighed against [01](01-the-providers.md) and [02](02-the-readers.md).
|
||||
|
||||
## A. In place, as today
|
||||
|
||||
The vault makes a new value. Every applier re-applies it on the same login, which all nine
|
||||
providers already do. Every reader is recreated by the host.
|
||||
|
||||
- **Works with:** every provider, unchanged. It is what `rotate` does now.
|
||||
- **Costs:** a window per consumer, from the provider applying to the consumer being recreated. They
|
||||
are on different machines, and nothing orders them. A reader whose machine is unreachable from the
|
||||
mesh but still reaches its provider stays locked out until the mesh reaches it again.
|
||||
- **Admin credentials:** the natural form. The provider module is the only party, and it has to
|
||||
apply the new value with the old one anyway (finding 6).
|
||||
|
||||
## B. Two secrets on one login
|
||||
|
||||
The applier adds the new password beside the old one, readers move, and the old one is removed.
|
||||
|
||||
- **Works with:** redis natively, and gitea and mailu through tokens. **Not** with the other six, whose
|
||||
backends hold one password per login (finding 4).
|
||||
- **Verdict:** not a mechanism, a special case. Using it where it exists and something else
|
||||
elsewhere is the "this way or that way" the design is trying to remove.
|
||||
|
||||
## C. Two credentials per consumer, over one resource
|
||||
|
||||
The consumer has two credentials and uses one at a time. The applier ensures the other with the new
|
||||
value and gives it the same rights over the consumer's resource. Readers move to it, and then the old
|
||||
credential is retired, which removes the credential only, never the resource. **What a credential is,
|
||||
is the adapter's**: a second login for eight providers (finding 5), a second token on the same login
|
||||
for mailu. The mesh sees one mechanism.
|
||||
|
||||
- **Works with:** every provider, **after** each adapter changes:
|
||||
- the resource is named after the consumer, not the login. Today the two are the same string (finding
|
||||
8), so existing resources keep their names, and the current login stays one of the two;
|
||||
- the resource is owned by the resource, not by a login. In postgres that is a role no one logs in
|
||||
as, which each login works as, and ownership of an existing database moves to it once (finding 6);
|
||||
- both credentials get the same rights, over data and structure;
|
||||
- *retire a credential* and *remove the consumer* become two operations. Today they are one call, and
|
||||
in five providers that call destroys data (finding 3). The harness must key by consumer, so that a
|
||||
changed login is not a removal. This is the whole of the danger, and it has to be split, whatever
|
||||
else is chosen.
|
||||
- **Costs:**
|
||||
- every credential adapter changes;
|
||||
- the harness learns the alternation and a confirmation per step, and rotation state has to live
|
||||
somewhere that survives a restart, which the harness's memory does not;
|
||||
- the second login's name must fit the tightest backend. That is 20 characters for a minio access
|
||||
key ([ADR 0049](../../02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md)), and
|
||||
a suffix spends part of it;
|
||||
- retiring an mssql login has to end its sessions first.
|
||||
- **Gains:** no window. A reader that cannot be reached keeps a working credential until it can.
|
||||
- **Does not apply to** single-party secrets: admin credentials and a module's own secrets. There is
|
||||
no second party to overlap with.
|
||||
|
||||
## Independent of the choice
|
||||
|
||||
- **Split remove, and key the harness by consumer.** Retiring a credential, or a login changing, must
|
||||
never be able to destroy a consumer's data. That holds under A too, because A's remove is the same
|
||||
call.
|
||||
- **Admin credentials are applied by their own provider**, using the old value, with the new one staged
|
||||
beside it. Five backends take the value only at first initialisation. Replacing the file first locks
|
||||
the provisioner out (finding 7).
|
||||
- **Classify the readers** ([02](02-the-readers.md)), bus-account readers included, before relying on
|
||||
derived restarts.
|
||||
|
||||
## Recommendation
|
||||
|
||||
- **Two-party credentials, consumer credentials and bus accounts: C**, because it is the only
|
||||
mechanism every provider supports, and it closes the window instead of shortening it. Its prerequisite, separating the resource from the login and
|
||||
retiring a login from removing a consumer, is worth doing on its own, because it removes a
|
||||
data-loss path that exists today.
|
||||
- **Single-party secrets (admin credentials, a module's own): A, staged.** In place, applied by the
|
||||
provider that holds them, with the new value beside the old until it has taken.
|
||||
- **Until the adapters are changed, A stays** as `rotate` implements it, with its window stated. It is
|
||||
not replaced by a mechanism the providers cannot yet carry.
|
||||
|
||||
This is two mechanisms, split by a property of the secret rather than by provider: whether it has one
|
||||
party or two. Every provider is treated the same way for the same kind of secret.
|
||||
@@ -0,0 +1,47 @@
|
||||
---
|
||||
status: active
|
||||
initiated: 2026-09-26
|
||||
touches:
|
||||
- 00-META/mission.md
|
||||
- 02-DECISIONS/0106-the-bus-is-nats.md
|
||||
- 02-DECISIONS/0010-delivery.md
|
||||
- 02-DECISIONS/0083-one-push-leaves-the-mesh-consistent.md
|
||||
- 03-DESIGN/01-to-be/06-the-controller.md
|
||||
- 03-DESIGN/01-to-be/09-the-node-lifecycle.md
|
||||
- 03-DESIGN/00-as-is/09-interfaces-and-observability.md
|
||||
---
|
||||
|
||||
# 017 — A mesh that heals itself
|
||||
|
||||
**What.** The behaviour the operator wants: a mesh that runs itself. It notices what is wrong,
|
||||
repairs what it can, and hands what it cannot repair to someone who can, with the reason. This effort
|
||||
writes that wish down as intended behaviour, designed for the bus the mesh is moving to
|
||||
([ADR 0106](../../02-DECISIONS/0106-the-bus-is-nats.md): NATS). It also records what can be done
|
||||
pragmatically before that move.
|
||||
|
||||
**Why.** The mission is *a mesh that controls itself* ([mission](../../00-META/mission.md)). The
|
||||
mesh can tell whether it is up. It cannot tell whether it is right. The as-is page on observability says so
|
||||
([as-is 09](../../03-DESIGN/00-as-is/09-interfaces-and-observability.md)). To-be 06 names an
|
||||
`observability` context in the controller and leaves its store undecided. Nothing routes a condition
|
||||
the mesh cannot fix to anyone. The cost is measurable: **46 of the 116 issue reports in this
|
||||
repository describe a failure that was silent.** A mesh that heals itself is, first, a mesh that stops
|
||||
failing silently.
|
||||
|
||||
**What it touches.** The controller's observability context, the node lifecycle's liveness, delivery
|
||||
([ADR 0010](../../02-DECISIONS/0010-delivery.md), [ADR 0083](../../02-DECISIONS/0083-one-push-leaves-the-mesh-consistent.md)),
|
||||
the provisioner harness, and rotation, which is proposed alongside to-be 27 as ADR 0114.
|
||||
|
||||
**Documents.**
|
||||
|
||||
- [01 — The intended behaviour](01-the-intended-behaviour.md): the wish, as principles and as how
|
||||
the mesh behaves once the bus is NATS.
|
||||
- [02 — Now, pragmatically](02-now-pragmatically.md): what is done before NATS, why it does not
|
||||
build anything the move would throw away, and what has been done already.
|
||||
|
||||
**Next.** Two measurements this effort owes before it can graduate:
|
||||
|
||||
1. **Every loop in the mesh**: what it converges, and whether it compares against observed state or
|
||||
against its own memory. Issue 120 found the provisioner harness trusting memory. The same pattern is
|
||||
expected elsewhere.
|
||||
2. **The 46 silent failures, classified**: a missing observation, a loop trusting memory, or a missing
|
||||
escalation. That shows which mechanism removes the most of them.
|
||||
@@ -0,0 +1,85 @@
|
||||
# 01 — The intended behaviour
|
||||
|
||||
The operator's wish, written as behaviour: what a person or an agent sees the mesh doing. This is a
|
||||
target to design toward, not a design. Every part of it is to be decided through a record before it
|
||||
is built.
|
||||
|
||||
## Principles
|
||||
|
||||
**1. Every loop compares what should be with what is, never with what it did.** Desired state is the
|
||||
mesh's: assignments, requirements, seats. Observed state is read from the thing itself: the container,
|
||||
the backend, the node. A loop that compares against its own memory of what it applied is blind to
|
||||
anything that changed behind its back. That is issue 120, and it is the pattern this whole effort is
|
||||
written against.
|
||||
|
||||
**2. Healing is the ordinary path run again, never a second path.** Repairing a lost login is
|
||||
provisioning it. Repairing a dead container is converging the node. Repairing a stale declaration is
|
||||
delivering it. A repair that needs its own code is a second way of doing something, which is exactly
|
||||
what the mesh is removing everywhere else.
|
||||
|
||||
**3. A repair never destroys.** Healing may recreate, re-provision, re-deliver and restart. It may
|
||||
never delete a consumer's data, retire a credential someone still uses, or pick a winner between two
|
||||
contradictory states. Where the only repair is destructive, it is escalated.
|
||||
|
||||
**4. Nothing fails silently.** Every condition the mesh cannot repair within its budget becomes
|
||||
visible. It is named, it says since when, why, and who can resolve it. It is visible until it is
|
||||
resolved, and resolved by observation, not by someone clicking it away.
|
||||
|
||||
**5. What the mesh cannot fix goes to an agent.** Per the mission, an agent may be human or not. A
|
||||
condition that needs judgement is handed to one, as work, with what the mesh knows. It is not handed
|
||||
over as a notification that someone may or may not read.
|
||||
|
||||
**6. Correctness, not only liveness.** A running process that authenticates with a dead credential,
|
||||
serves an old version, or routes nowhere is not healthy. What a provision's contract promises is what
|
||||
is checked: the credential authenticates, the route answers, the version is the declared one.
|
||||
|
||||
## The loop, everywhere
|
||||
|
||||
Every part of the mesh that owns something runs the same loop:
|
||||
|
||||
1. **know** what should be true: from assignments, requirements and seats;
|
||||
2. **observe** what is true: from the thing itself, on its own cadence;
|
||||
3. **repair** the difference by running the ordinary path again, within a budget of attempts and
|
||||
time;
|
||||
4. **raise** a *condition* when the budget is spent or the only repair is destructive;
|
||||
5. **clear** the condition when observation shows it resolved.
|
||||
|
||||
A **condition** is a durable fact about something the mesh owns, such as a node, an assignment, a
|
||||
provision, a seat or a rotation: what is wrong, since when, the evidence, what was tried, and who can
|
||||
resolve it. Conditions are the one thing a person or an agent looks at to know whether the mesh is
|
||||
right. `status` is the list of open conditions. When it is empty, the mesh is right, not just up.
|
||||
|
||||
## On NATS
|
||||
|
||||
[ADR 0106](../../02-DECISIONS/0106-the-bus-is-nats.md) moves the bus to NATS, and NATS makes most of
|
||||
this cheaper, because observation becomes something every component publishes rather than something
|
||||
a central process polls.
|
||||
|
||||
| the wish needs | on NATS |
|
||||
|---|---|
|
||||
| every component says it is alive | a heartbeat on a subject per node and assignment; silence past its interval is a condition, and nobody polls |
|
||||
| every component says what it observed | observations published on subjects (`mesh.observed.<node>.<assignment>`, for instance), consumed by whoever owns the comparison |
|
||||
| the last known state survives restarts | a JetStream key-value bucket of observed state per owner; the provisioner's "what I applied" and a rotation's step live there, not in memory |
|
||||
| conditions are durable and watchable | conditions as entries in a key-value bucket, watched by anyone who cares: a surface, an agent, the controller |
|
||||
| the bus itself is observed | the server's advisories (a consumer exceeding its deliveries, a slow consumer, a client disconnecting) and its monitoring endpoint become observations like any other |
|
||||
| a repair is retried, not lost | JetStream redelivery with delay, which is the same mechanism [ADR 0083](../../02-DECISIONS/0083-one-push-leaves-the-mesh-consistent.md)'s guarantee moves to |
|
||||
| work handed to an agent | a condition that needs judgement published as a task on a subject an agent's queue group consumes |
|
||||
|
||||
**Who compares.** Each owner compares its own: the host for its node's containers and files, a
|
||||
provisioner for its backend, the vault for rotations, the controller for delivery and seats. The
|
||||
controller's observability context does not repair anything. It holds conditions, their history,
|
||||
and the view across the mesh. It notices what no owner can see about itself: an owner gone silent.
|
||||
|
||||
## What stays human
|
||||
|
||||
Some repairs need the operator's key, and the mesh says so rather than pretending otherwise:
|
||||
re-raising the vault or the broker, and recovering a node's identity. These are conditions too, with
|
||||
the procedure named, and they are the only ones that can never clear themselves.
|
||||
|
||||
## Open
|
||||
|
||||
- The budgets: how many attempts, over how long, per kind of repair.
|
||||
- How a condition that needs judgement reaches an agent, and how the agent's action is recorded.
|
||||
- Where the observability context stores history (to-be 06 left it open; volume argues against the
|
||||
relational store).
|
||||
- Which correctness probe each provision's contract offers, and how often it runs.
|
||||
@@ -0,0 +1,46 @@
|
||||
# 02 — Now, pragmatically
|
||||
|
||||
The intended behaviour lands on NATS. The bus moves after the migration's core
|
||||
([ADR 0106](../../02-DECISIONS/0106-the-bus-is-nats.md)). Until then, work toward it is chosen by one
|
||||
test:
|
||||
|
||||
**Does it survive the move?** A change to what a loop compares against, or to what an adapter can
|
||||
tell about its backend, survives, because it is independent of the bus. A new AMQP queue for health
|
||||
reports, a poller written against the broker's management API, or a condition store built on the
|
||||
current broker does not survive, and is not built.
|
||||
|
||||
## Done
|
||||
|
||||
**The provisioner asks the backend, not memory** (issue 120). The SDK's harness gained an optional
|
||||
`holds` on the adapter, asked for every applied consumer every minute. A consumer the backend no
|
||||
longer holds is provisioned again. Being unable to ask is not treated as loss. The cache module
|
||||
implements it first, because its server keeps its users in memory and forgets them all on a restart.
|
||||
That was verified against a real server: a restart erases every consumer's user, and `holds` answers
|
||||
correctly for absent, present, wrong-password, disabled and deleted.
|
||||
Changes: mesh-sdk PR #7 (0.1.1) and mesh-catalog PR #84.
|
||||
|
||||
This is principle 1 applied to one loop. It survives the move unchanged. On NATS, the harness's
|
||||
record of what it applied moves from memory into a key-value bucket, and `holds` stays as it is.
|
||||
|
||||
## Next, in order of silent failures removed
|
||||
|
||||
1. **`holds` for the other credential providers.** Each backend can answer whether a login exists
|
||||
with the mesh's password without changing anything. Where a backend cannot check a password without
|
||||
logging in, logging in is the check.
|
||||
2. **The harness's other blind spot.** A consumer that goes away while its provisioner is down is never
|
||||
removed. The fix is the same principle in reverse: list what the backend holds, and compare it with
|
||||
what the mesh asks for. Removal stays subject to ADR 0114's rule that it never follows from a login
|
||||
changing.
|
||||
3. **`status` reports what owners already know.** The host knows which containers it recreated and
|
||||
why. A rotation knows who it waits on. Delivery knows what is outstanding. Surfacing those as
|
||||
conditions in the existing `status` needs no new transport. It is the shape the NATS condition store
|
||||
will hold.
|
||||
4. **The loop inventory and the classification** in [00](00-overview.md). They decide what comes after
|
||||
these three.
|
||||
|
||||
## Not now
|
||||
|
||||
- Heartbeats, observation subjects, key-value state, advisories: all NATS, all after the move.
|
||||
- Handing conditions to agents: designed with NATS, where a task on a subject is native.
|
||||
- Choosing the observability store: decided when there is something to store, which is after the
|
||||
move.
|
||||
@@ -0,0 +1,80 @@
|
||||
---
|
||||
status: active
|
||||
initiated: 2026-10-02
|
||||
touches:
|
||||
- 02-DECISIONS/0040-what-a-module-is.md
|
||||
- 02-DECISIONS/0011-managed-files-are-generated-never-edited.md
|
||||
- 02-DECISIONS/0132-a-seat-carries-the-tools-its-holder-must-serve.md
|
||||
- 02-DECISIONS/0150-a-modules-own-code-runs-as-supervised-processes-under-one-account.md
|
||||
- 02-DECISIONS/0152-the-operators-surface-is-a-module-the-console.md
|
||||
- 02-DECISIONS/0161-what-deserves-a-seat.md
|
||||
- 03-DESIGN/01-to-be/05-the-node-host.md
|
||||
- 03-DESIGN/01-to-be/29-a-node-has-operator-accounts.md
|
||||
- 03-DESIGN/01-to-be/33-the-tools-the-mesh-answers.md
|
||||
- 03-DESIGN/01-to-be/34-the-console.md
|
||||
- 03-DESIGN/00-as-is/10-module-catalogue.md
|
||||
- 04-ISSUES/160-a-machine-says-little-about-itself-and-only-when-asked/00-report.md
|
||||
- 04-ISSUES/168-a-setting-reaches-every-file-and-contribution/00-report.md
|
||||
became: []
|
||||
---
|
||||
|
||||
# 018 — The operator's machine as modules
|
||||
|
||||
**What.** The mesh owns the whole machine, not only the services on it. Everything a person
|
||||
configures on a node — the login manager, the display server, the window manager, the shell, the
|
||||
terminal, the launcher, the notifier, the audio setup, the boot images, the downloads folder, the
|
||||
agent at the terminal — is a module: a package, the files it owns under `/etc` and under the
|
||||
operator's home, the seat it holds, the tools it serves. One default configuration per module,
|
||||
varied per node only through settings rendered into the file or a kept operator region, never
|
||||
through an edit. The servers take the universal modules (shell, prompt, git, the agent); the
|
||||
workstations take those and the graphical stack, which a capability the machine reports gates.
|
||||
This effort writes that behaviour down, measures what the predecessor's desktop modules actually
|
||||
contain, and settles what the mesh must gain before the first of them can be written.
|
||||
|
||||
**Why.** The predecessor is retired on every node. What it still owned on the two workstations —
|
||||
about thirty modules' worth of dotfiles, user units and `/etc` files — is now owned by nothing:
|
||||
no generator regenerates them, and a fix to one of them is a hand edit that nothing records. The
|
||||
migration scoped these modules out as *the workstation's own environment*, and
|
||||
[to-be 29](../../03-DESIGN/01-to-be/29-a-node-has-operator-accounts.md) names them as the last
|
||||
thing the predecessor was keeping alive. To-be 29 covers one directory, `~/.ssh`, and draws a
|
||||
boundary inside it. The operator wants no boundary: the machine is the mesh's, as far as it makes
|
||||
sense to configure it. That is a wider scope than any design states, and it reaches three records
|
||||
that were written for services: what a module is, where a module's tools run, and what a managed
|
||||
file may be.
|
||||
|
||||
**What it touches.** The module definition ([ADR 0040](../../02-DECISIONS/0040-what-a-module-is.md)),
|
||||
seats and their contracts ([ADR 0132](../../02-DECISIONS/0132-a-seat-carries-the-tools-its-holder-must-serve.md)),
|
||||
where a module's tools run ([ADR 0150](../../02-DECISIONS/0150-a-modules-own-code-runs-as-supervised-processes-under-one-account.md),
|
||||
[ADR 0152](../../02-DECISIONS/0152-the-operators-surface-is-a-module-the-console.md),
|
||||
[to-be 33](../../03-DESIGN/01-to-be/33-the-tools-the-mesh-answers.md) §6), the host's vocabulary
|
||||
([to-be 05](../../03-DESIGN/01-to-be/05-the-node-host.md)), managed files and settings
|
||||
([ADR 0011](../../02-DECISIONS/0011-managed-files-are-generated-never-edited.md),
|
||||
[issue 168](../../04-ISSUES/168-a-setting-reaches-every-file-and-contribution/00-report.md)),
|
||||
and the catalogue's shape ([as-is 10](../../03-DESIGN/00-as-is/10-module-catalogue.md)).
|
||||
|
||||
**Documents.**
|
||||
|
||||
- [01 — The intended behaviour](01-the-intended-behaviour.md): the operator's wish, written as
|
||||
how the mesh behaves, in the mesh's own words.
|
||||
- [02 — What exists, and what is missing](02-what-exists-and-what-is-missing.md): the
|
||||
predecessor's desktop modules measured; which records already say what is wanted; the gaps.
|
||||
- [03 — One tool executor per node](03-one-tool-executor-per-node.md): where a module's tools
|
||||
run. The direction the operator set, the evidence for it, and what it supersedes.
|
||||
- [04 — The seats of the environment](04-the-seats-of-the-environment.md): the roles a machine
|
||||
has once, their candidate contracts, and what gates each.
|
||||
|
||||
**What this must settle before it graduates.**
|
||||
|
||||
1. A module is one *managed thing*, software or not, and every module may serve tools — or ADR
|
||||
0040 already says this and only its examples are narrow.
|
||||
2. One tool executor per node, host-side, module-agnostic; which records it supersedes and
|
||||
in what form the console continues.
|
||||
3. Per-node variation is a setting rendered into the file or a kept region, never an edit —
|
||||
ADR 0011 stands — and issue 168 is fixed before any environment module carries a setting.
|
||||
4. User-scoped units on the host's `service` shape, and a service-manager seat whose holder
|
||||
serves the tools about them.
|
||||
5. The operator account stated on every node; today no node record carries one.
|
||||
6. The seats of the environment and their verbs, one record per seat, slowly, because a
|
||||
seat's tools bind every future holder.
|
||||
7. Where the environment modules live: this catalogue, or one of their own as the media chain
|
||||
has; and whether a third-party organisation's tooling belongs in a public catalogue at all.
|
||||
@@ -0,0 +1,99 @@
|
||||
# 01 — The intended behaviour
|
||||
|
||||
*Written 2026-10-02 from the operator's words, in the mesh's words. What is wanted, before what
|
||||
exists. Where a sentence restates a record, the record is named; where it goes further, that is
|
||||
said.*
|
||||
|
||||
## The machine is the mesh's
|
||||
|
||||
**Everything configurable on a node is declared by a module.** Not only the services the mesh
|
||||
runs: the login manager, the display server, the window manager, the bar, the launcher, the
|
||||
notifier, the compositor, the lock screen, the terminal emulator, the clipboard, the shell and its
|
||||
prompt, the editor, the audio setup, the boot images, the package manager's configuration, the
|
||||
agent a person runs at a terminal, and the folders a person works in — a downloads folder that is
|
||||
tidied, backed up, distributed to other nodes and asked questions of. System folders and the
|
||||
operator's home alike. The operator is the only person on every node, so the mesh manages the
|
||||
person's machine, not a machine with a person on it.
|
||||
|
||||
This is [ADR 0040](../../02-DECISIONS/0040-what-a-module-is.md)'s definition applied without
|
||||
the service bias its examples carry. A module is one managed thing, named once, described
|
||||
completely by its manifest. It may have a package, files, a container, a unit, a binary, a seat it
|
||||
holds, and tools it serves — any one of these, or all, or two. There is **no kind of module**: zsh
|
||||
has a package, files, a seat claim and the tools that claim obliges it to serve; downloads has a
|
||||
folder, a process and tools; nftables has a package, files, a service, a seat and tools. The
|
||||
difference is what each declares, not what each is.
|
||||
|
||||
**The home has no boundary.** [To-be 29](../../03-DESIGN/01-to-be/29-a-node-has-operator-accounts.md)
|
||||
owns one directory under the home and draws a line inside it between the mesh's and the person's.
|
||||
Here the line is drawn only by what the modules declare: every file some module places is the
|
||||
mesh's; what no module declares is found and left alone, exactly as the adoption rules already
|
||||
say for a machine. The reach is bounded by sense, not by a rule — the mesh configures what can
|
||||
be configured, and a person's documents, projects and history are data under
|
||||
[ADR 0051](../../02-DECISIONS/0051-shared-data-is-the-operators.md), not configuration.
|
||||
|
||||
**A module names no node and no path.** The operator account is a node fact and the home is
|
||||
derived from it ([to-be 29](../../03-DESIGN/01-to-be/29-a-node-has-operator-accounts.md) §1–2,
|
||||
shipped in the controller; its record is proposed in an open change). A module places a file
|
||||
*under the home, owned by the account*, and the same manifest lands on a server and a laptop.
|
||||
|
||||
## One default, varied by settings, never by edits
|
||||
|
||||
**One module, one default configuration.** The window manager module ships the configuration
|
||||
that is right for every node. There are no flavors: the predecessor's one desktop module carried
|
||||
four, one per class of machine, and what differed between them is what settings are for.
|
||||
|
||||
**A node varies a module in exactly two ways.** A **setting**, declared by the module with its
|
||||
type, meaning and default (proposed alongside the container-runtime records), set for the mesh
|
||||
or for one node, and rendered into the file at composition — the value is in the file, not in an
|
||||
environment variable the file reads. Or a **kept region**: a block in a file the mesh writes
|
||||
*into*, where the operator's own lines survive every push
|
||||
([ADR 0102](../../02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md)). An
|
||||
edit to a managed file outside such a region is not a third way; it is overwritten, as
|
||||
[ADR 0011](../../02-DECISIONS/0011-managed-files-are-generated-never-edited.md) says, and the
|
||||
predecessor's habit of adopting disk drift back into its database is not carried over.
|
||||
|
||||
The predecessor's theming — some ninety environment variables substituted into templates at sync
|
||||
time, with tools to list and set them — is the same idea with the wrong rendering. The knobs
|
||||
become declared settings; the file carries the value.
|
||||
|
||||
## Roles a machine has once are seats, and seats carry tools
|
||||
|
||||
**A role a machine fills at most once is a node-scoped seat**, declared by a module
|
||||
([ADR 0121](../../02-DECISIONS/0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md),
|
||||
[ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md)): the login shell, the
|
||||
display session, the display server, the terminal emulator, the launcher, the notifier, the
|
||||
compositor, the lock screen, the service manager, the boot loader. Several modules may be able to
|
||||
hold one — zsh, fish and bash can all hold the login shell — and the assignment on each node says
|
||||
which does. Installing a shell is installing software; holding the seat is being *the* shell.
|
||||
|
||||
**A seat's contract is its tools** ([ADR 0132](../../02-DECISIONS/0132-a-seat-carries-the-tools-its-holder-must-serve.md)).
|
||||
Every holder of the login-shell seat serves `execute`, which takes one string, the command, and
|
||||
runs it on the node the seat is scoped to. Every holder of the boot seat serves "rebuild the boot
|
||||
images", so *"rebuild your boot images"* is a verb addressed to a machine, not a one-off step in
|
||||
a hook. Every holder of the service-manager seat answers for the units on the machine, system and
|
||||
user scope. A module may serve its own tools beside the seat's
|
||||
([ADR 0170](../../02-DECISIONS/0170-the-firewall-seat-serves-its-verbs.md) §2): show the rendered
|
||||
configuration, set a theme value, report status.
|
||||
|
||||
**Any tool may be called from any node.** The operator's statement, and the grant model it
|
||||
implies: the executor on each node may call everything, as the console already may. A verb that
|
||||
needs root on the machine is the module's concern — the tool escalates, the executor and the
|
||||
caller do not know.
|
||||
|
||||
## Servers and workstations differ by capability, not by catalogue
|
||||
|
||||
The same catalogue serves every node. A module declares what it needs — a graphical session, a
|
||||
display server, a container runtime — and the machine reports what it has, as the profile already
|
||||
reports eight capabilities today ([issue 160](../../04-ISSUES/160-a-machine-says-little-about-itself-and-only-when-asked/00-report.md)).
|
||||
Assignment refuses the wrong placement by name
|
||||
([ADR 0161](../../02-DECISIONS/0161-what-deserves-a-seat.md) §3). So every node takes the shell,
|
||||
the prompt, git and the agent; only a node with a graphical session can take the display server,
|
||||
and only a node holding the display server can take a window manager. Nothing in a module says
|
||||
"workstation".
|
||||
|
||||
## What the operator would say to the mesh
|
||||
|
||||
*Set the login shell on the build node to fish. Rebuild the laptop's boot images. Show me the
|
||||
window manager's effective configuration on the desktop and where each value comes from. Give
|
||||
the downloads folder on the laptop to the home server. Run `uptime` on every node.* Each of these
|
||||
is a seat verb or a module tool, addressed to a node, answered by whatever holds the role there.
|
||||
+91
@@ -0,0 +1,91 @@
|
||||
# 02 — What exists, and what is missing
|
||||
|
||||
*Measured 2026-10-02 on one installation: two workstations, two servers, all four converged to
|
||||
the mesh; the predecessor retired on the last workstation the day before. Numbers are from the
|
||||
machines and the repositories, not from memory.*
|
||||
|
||||
## 1. What the predecessor's desktop looks like
|
||||
|
||||
The predecessor's catalogue on the laptop held **34 modules**, of which **28** are the operator's
|
||||
environment rather than services. By what they declare:
|
||||
|
||||
| shape | count | examples |
|
||||
|---|---|---|
|
||||
| package only | 9 | browser, mail client, process monitor, media player, file manager, chat |
|
||||
| package + `/etc` files + system service | 5 | login manager, display server, power and thermal daemons, package manager configuration |
|
||||
| package + files under the home | 6 | shell and prompt, the agent at the terminal, scripts, the sync client, a music player |
|
||||
| files under the home + user units + hooks | 2 | the desktop environment, audio |
|
||||
| third-party organisation tooling | 6 | out of scope here |
|
||||
|
||||
**The desktop module alone** declares **88 files**, **4 flavors** (the window-manager stack, and
|
||||
one per class of machine), **2 user units** with a hook to enable them, 8 files under `/etc`, a
|
||||
wallpaper shipped as an asset, and reads **about 90 environment variables** as theme knobs,
|
||||
substituted into its templates at sync time and set through a theming tool. Its hook exists
|
||||
because *shipping a unit file does not run it*: one unit had been deployed for months and ran on
|
||||
one machine only, because somebody had enabled it there by hand.
|
||||
|
||||
**The shell module** ships `~/.zshrc`, the prompt configuration, an `~/.ssh/config` that the
|
||||
predecessor generated from its registry, and a `LOGIN_SHELL` variable applied with `chsh` by a
|
||||
hook. Two flavors: the prompt theme, and autocompletion.
|
||||
|
||||
**Other modules write into the desktop module's files.** The chat client places i3 and notifier
|
||||
snippets into `config.d` directories the desktop module owns, and its launch flags, window
|
||||
placement and notification colours are each a variable with a default.
|
||||
|
||||
**One-off steps live in hooks** across the set: enable user units, `chsh`, create a swap file,
|
||||
`mkinitcpio`, enable a vendor VPN service the package ships disabled. Every one is state the
|
||||
host could declare or a verb a seat could serve; none is today.
|
||||
|
||||
## 2. What the migration did with them
|
||||
|
||||
The migration's module to-do scoped the whole set out as *desktop / workstation ricing — the
|
||||
workstation's own environment* and *node/OS tooling — managed on the node, never catalogue*. The
|
||||
last workstation's runbook then split the same set three ways: **A**, system scope, which the
|
||||
host's vocabulary can express today (the login manager, the display server, the power daemons,
|
||||
the package manager, the container runtime); **B**, under a home or a user unit, waiting on
|
||||
to-be 29; **C**, package only, the operator's call. The migration log closes the workstation with
|
||||
*the operator's desktop awaiting its design*.
|
||||
|
||||
Two things followed from scoping them out. Nothing regenerates those files now, so a fix is a hand
|
||||
edit — the login manager's session script was fixed this way on the day of writing, and recorded
|
||||
in a repository nothing deploys from. And the one piece of this family written as a mesh module,
|
||||
the ssh client, was closed on hold in the catalogue until the controller carried the account fact.
|
||||
|
||||
## 3. What the records already give
|
||||
|
||||
| wanted | record | state |
|
||||
|---|---|---|
|
||||
| one module per managed thing; every module may have tools | [ADR 0040](../../02-DECISIONS/0040-what-a-module-is.md) | accepted; examples are services, and the shell is named as a *shared* seat |
|
||||
| a module declares its own node-scoped seat | [ADR 0121](../../02-DECISIONS/0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md), [ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md) | accepted |
|
||||
| a seat's contract is its tools; a holder may add its own | [ADR 0132](../../02-DECISIONS/0132-a-seat-carries-the-tools-its-holder-must-serve.md), [ADR 0170](../../02-DECISIONS/0170-the-firewall-seat-serves-its-verbs.md) | accepted; one node seat serves verbs live |
|
||||
| a capability the machine reports gates a holder | [ADR 0161](../../02-DECISIONS/0161-what-deserves-a-seat.md) §3 | accepted; the profile already reports `graphical-session` |
|
||||
| the account as a node fact; a file under the home owned by it | [to-be 29](../../03-DESIGN/01-to-be/29-a-node-has-operator-accounts.md) §1–2 | built in the controller; its record proposed in an open change |
|
||||
| inside a home: owned, written into, written by the module, found | proposed in the same change | proposed |
|
||||
| a setting declared with type, meaning, default and cost | proposed with the container-runtime records | proposed |
|
||||
| a managed file is derived; an edit is overwritten | [ADR 0011](../../02-DECISIONS/0011-managed-files-are-generated-never-edited.md) | accepted |
|
||||
| the mesh writes into a shared file, never over it | [ADR 0102](../../02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md) | accepted |
|
||||
| a module names no path; the host resolves the home | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md) | accepted |
|
||||
| the `user` shape: a login shell is declared state | [to-be 05](../../03-DESIGN/01-to-be/05-the-node-host.md) | designed; used by no module |
|
||||
|
||||
## 4. What is missing
|
||||
|
||||
1. **The account is recorded nowhere.** The node record has the column; on all four nodes it
|
||||
is empty. Every home-scoped module is unassignable until the operator states it.
|
||||
2. **User-scoped units.** The host's `service` shape has no user scope. To-be 29 says it
|
||||
plainly: *a workstation's per-user daemons have no form the mesh can send.* The desktop
|
||||
module's two units, the audio masks, the power module's memory guard and the thermal
|
||||
daemon's profile switcher all need it.
|
||||
3. **One-off steps.** `mkinitcpio`, `chsh`, creating a swap file. Each is either declared
|
||||
state the host lacks a shape for, or a verb a seat should serve. An action in a declaration
|
||||
is refused over the link, and rightly.
|
||||
4. **Settings leak** ([issue 168](../../04-ISSUES/168-a-setting-reaches-every-file-and-contribution/00-report.md)):
|
||||
a setting reaches every mergeable file and every contribution of its module. Ninety theme
|
||||
knobs on that mechanism would reach ninety files. The proposed settings record says a setting
|
||||
names the file it lands in; that has to ship first.
|
||||
5. **Where tools run.** Every module that serves a tool today does so from its own container
|
||||
per node. See [03](03-one-tool-executor-per-node.md).
|
||||
6. **A seat's verbs are undecided for every seat but three.** To-be 33 leaves which verbs each
|
||||
seat serves as *a decision per seat, slowly*. The environment adds a dozen seats.
|
||||
7. **Catalogue placement.** The media chain left this catalogue for its own; whether the
|
||||
environment does the same, and whether a third-party organisation's tooling belongs in a
|
||||
public catalogue, are unasked.
|
||||
@@ -0,0 +1,88 @@
|
||||
# 03 — One tool executor per node
|
||||
|
||||
*The direction the operator set on 2026-10-02, the evidence it rests on, and what it supersedes.
|
||||
A direction, not yet a decision: the record is written when this effort graduates.*
|
||||
|
||||
## Where tools are served today
|
||||
|
||||
[To-be 33](../../03-DESIGN/01-to-be/33-the-tools-the-mesh-answers.md) names three families — a
|
||||
role's tools on the seat, a module's own tools on the module, the mesh's own verbs on the
|
||||
controller seat — and [ADR 0160](../../02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md)
|
||||
says a runtime serves the subjects its membership issues. What *runs* that runtime is
|
||||
[ADR 0150](../../02-DECISIONS/0150-a-modules-own-code-runs-as-supervised-processes-under-one-account.md):
|
||||
one supervised process per module, under the module's own account, carrying that module's
|
||||
compiled tools. Measured on the live mesh:
|
||||
|
||||
| who answers | how it runs | count |
|
||||
|---|---|---|
|
||||
| the mesh's own verbs | the controller binary, on its node | 17 verbs |
|
||||
| the store seat | the store's own runtime | 2 verbs |
|
||||
| the packet-filter seat | **a container per node**, built on the tool-runtime base image, with the network namespace and `NET_ADMIN`, on all four nodes | 3 verbs and 1 own tool |
|
||||
| every module's own tools | the module's container, one per node it runs on | 67 tools across the catalogue |
|
||||
| the console | a container per node, loopback MCP, `invokes: *` | serves none, calls all |
|
||||
| the host | — | serves nothing; answers no question about the machine |
|
||||
|
||||
**The packet-filter holder is the case to look at.** The module is a package, three files and a
|
||||
system service. To serve three verbs it also declares a built image and a container on every
|
||||
node whose only job is to answer them. Scaled to the environment — a shell, a prompt, a launcher,
|
||||
a notifier, a compositor, a login manager, a service manager, a boot loader, a downloads folder —
|
||||
that is one container per module per node for software that is itself not a container, and the
|
||||
operator's judgement is that tools should not run inside a container at all.
|
||||
|
||||
## The direction
|
||||
|
||||
**One tool executor per node, on the host side.** A process the host supervises, the way the
|
||||
launcher supervises the host ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)): not a
|
||||
container, one bus credential for the node, module-agnostic. It loads the tool code of every
|
||||
module assigned to the node and serves each module's tools and each held seat's verbs on the
|
||||
subjects the membership issues — nothing changes in what [ADR 0159](../../02-DECISIONS/0159-a-tool-call-names-the-machine-and-a-holder-serves-its-seats-verbs.md)
|
||||
and [ADR 0160](../../02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md)
|
||||
say about subjects, grants and memberships; what changes is that one process subscribes for the
|
||||
node instead of one per module.
|
||||
|
||||
- **A module brings its tools as a built artifact**, a bundle the pipeline produces, never an
|
||||
image. The executor knows bundles and subjects; it knows nothing of zsh or nftables.
|
||||
- **A tool is code the module wrote**, one function behind an MCP verb. `execute` on the shell
|
||||
seat is a function with a string argument. The executor does not declare, template or
|
||||
interpret tools; it runs them.
|
||||
- **Root is the module's concern.** A tool that must change the packet filter or rebuild boot
|
||||
images escalates itself. The executor does not run as root for everyone, and the caller does
|
||||
not know.
|
||||
- **Any node may call any tool on any node.** The executor's credential may call everything,
|
||||
as the console's already does; per-module grants on the calling side are not kept.
|
||||
- **The mesh's own verbs stay with the controller** ([ADR 0154](../../02-DECISIONS/0154-the-meshs-own-verbs-are-the-controller-seats-tools.md)),
|
||||
and a mesh-scoped seat's verbs run on the node that holds it
|
||||
([ADR 0121](../../02-DECISIONS/0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md)).
|
||||
No hub is added; the controller's node is already one.
|
||||
|
||||
**The console is the executor, renamed.** It already runs on every node with a credential that
|
||||
may call everything, and it already serves the mesh's tools to whoever is on the machine over
|
||||
MCP on loopback ([ADR 0152](../../02-DECISIONS/0152-the-operators-surface-is-a-module-the-console.md)).
|
||||
It moves out of its container into the host's process tree, gains the serving half, and takes a
|
||||
name that says what it is — *the node's tool runtime* or simply *node tools*; "console" names
|
||||
the operator's half only.
|
||||
|
||||
## What it supersedes, and what it keeps
|
||||
|
||||
| record | effect |
|
||||
|---|---|
|
||||
| [ADR 0047](../../02-DECISIONS/0047-a-module-runs-its-code-as-its-own-process-with-its-own-account.md), [ADR 0150](../../02-DECISIONS/0150-a-modules-own-code-runs-as-supervised-processes-under-one-account.md) | superseded *for tools*: one process per node runs every module's tool code, under one account. A module's long-running service — a daemon, a container — is untouched; the executor runs tools, not services. The record must say why one account for every module's tools is acceptable: every tool may be called from every node anyway, and root is taken by the tool, not granted to the process |
|
||||
| [ADR 0152](../../02-DECISIONS/0152-the-operators-surface-is-a-module-the-console.md) | kept in substance — a module, assigned per node, loopback MCP, the machine's login is the authority — changed in form: host-side, not a container; serves as well as calls; renamed |
|
||||
| [to-be 33](../../03-DESIGN/01-to-be/33-the-tools-the-mesh-answers.md) §6, [to-be 34](../../03-DESIGN/01-to-be/34-the-console.md) | amended the same way |
|
||||
| [ADR 0170](../../02-DECISIONS/0170-the-firewall-seat-serves-its-verbs.md) §3, *a container may ask for a capability* | moot for that holder: the verbs run on the host side and escalate as they need |
|
||||
| the container-runtime seat, proposed in an open change: *the holder runs as a supervised process and serves the verbs locally to the host and on the bus* | consistent — a supervised process serving verbs is what the executor is; the open question is whether that holder keeps its own process or serves through the executor like everyone else |
|
||||
| the tool-runtime base image | no longer the way tools reach a node; may remain the way a module's *service* is built |
|
||||
|
||||
## What stays open
|
||||
|
||||
- **The executor's language.** The host is a static Go binary and loads no plugins, so the
|
||||
executor is a sibling process, and its language decides the language of every tool bundle.
|
||||
One decision, taken once.
|
||||
- **How a bundle reaches the node.** An artifact of the module's build, delivered as the host
|
||||
delivers everything else; whether it is a file resource in the declaration or a thing the
|
||||
executor fetches by digest.
|
||||
- **Reload.** A push that adds or upgrades a module's bundle reaches a running executor as a
|
||||
reload, not a restart, or every tool on the node blinks on every push.
|
||||
- **The host's own questions.** [Issue 160](../../04-ISSUES/160-a-machine-says-little-about-itself-and-only-when-asked/00-report.md)
|
||||
wants a machine to say more about itself. With an executor on every node, "what is this
|
||||
machine made of" is a seat verb like any other, served there.
|
||||
@@ -0,0 +1,65 @@
|
||||
# 04 — The seats of the environment
|
||||
|
||||
*Candidates, not decisions. To-be 33 says which verbs a seat serves is a decision per seat,
|
||||
taken slowly, because a seat's tools bind every future holder. This document lists the roles the
|
||||
operator's machine has once, who could hold each, what gates it, and a first verb or two — so
|
||||
each record has a starting point.*
|
||||
|
||||
## The rule for what is a seat here
|
||||
|
||||
A role the machine fills **at most once** is a node-scoped seat, declared by the module family
|
||||
that fills it ([ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md)). A thing
|
||||
several of which coexist without contention — editors, browsers, media players — is not a seat;
|
||||
each is a module with its own tools, and nothing is singular about it. A seat is held by one
|
||||
assignment per node; other modules of the same family may be installed beside it without
|
||||
holding it ([ADR 0040](../../02-DECISIONS/0040-what-a-module-is.md) §1, read with the sharper
|
||||
distinction: *installed* is not *holding*).
|
||||
|
||||
## Candidate seats
|
||||
|
||||
| seat | holders | gated by | first verbs |
|
||||
|---|---|---|---|
|
||||
| **login shell** | zsh, fish, bash | nothing: universal | `execute(command)`; `show-config`; the holding itself sets the account's login shell through the host's `user` shape |
|
||||
| **service manager** | systemd | the `service-manager` capability the profile reports | units: list, status, start, stop, restart, enable, journal; **user scope** on each |
|
||||
| **boot** | grub, systemd-boot | a machine that boots itself (not a container host) | `rebuild-images`; `entries` |
|
||||
| **package manager** | pacman, apt | the `package-manager` capability | search, installed, upgrade, orphans; today a capability the host uses, not a seat anyone holds |
|
||||
| **display server** | xorg, wayland compositors that are their own server | the `graphical-session` capability | `displays`; `layout` |
|
||||
| **display session** | i3, sway | the display server seat held on the node; i3 needs x11, sway needs wayland | `reload`; `workspaces`; `windows`; `move` |
|
||||
| **terminal emulator** | xterm, alacritty, foot | display session | `open`; `font` |
|
||||
| **launcher** | rofi, dmenu | display session | `show`; `theme` |
|
||||
| **notifier** | dunst, mako | display session | `send`; `history`; `rule` |
|
||||
| **compositor** | picom | display server (x11 only) | `restart`; `effects` |
|
||||
| **lock screen** | i3lock, swaylock | display session | `lock` |
|
||||
| **bar** | i3status-rust, waybar | display session | `reload`; `blocks` |
|
||||
| **login manager** | lemurs, greetd | graphical session | `sessions`; `default-session` |
|
||||
| **audio** | pipewire, pulseaudio | the machine reports a sound device | `sinks`, `sources`, `default`, `volume`, `mute` |
|
||||
| **clipboard** | greenclip, cliphist | display session | `history`; `clear` |
|
||||
|
||||
Not seats, modules with their own tools: the editor, the browser, the mail client, the file
|
||||
manager, the media player, the chat client, the agent at the terminal, the downloads folder, the
|
||||
scripts folder, the sync client, the power and thermal daemons that are specific to one machine's
|
||||
hardware.
|
||||
|
||||
## What the table implies
|
||||
|
||||
**Capabilities come first.** `graphical-session`, `service-manager` and `package-manager` are
|
||||
reported today. *A display server is held* is not a capability but a seat being held, and a
|
||||
module that needs it declares a dependency on the seat, not on a capability: *i3 needs the
|
||||
display server seat held by xorg*. Whether a held seat can gate another's assignment is a
|
||||
question for the controller's resolver, and the first environment module after the shell will
|
||||
ask it.
|
||||
|
||||
**The service manager comes early.** Four of the predecessor's modules ship user units, and the
|
||||
executor itself is a unit. User scope on the host's `service` shape is a host change whichever
|
||||
module holds the seat; the seat's holder answers the questions about units, it does not apply
|
||||
them — the host does, as it does for every declared resource.
|
||||
|
||||
**The shell comes first.** Universal, no capability, one verb that is immediately useful on
|
||||
every node, and the `user` shape already makes the login shell declared state. It is the module
|
||||
that proves the pattern: a package, files under the home owned by the account, a seat claim,
|
||||
tools served by the executor, settings for the few things that vary per node, and a kept region
|
||||
for the operator's own lines.
|
||||
|
||||
**The login manager is the first system-scope one**, because it needs nothing new: a package,
|
||||
two files under `/etc`, a service — the same shape the ssh daemon module has today — and the
|
||||
session script it owns is the file that was hand-fixed the day this effort opened.
|
||||
+44
-10
@@ -1,12 +1,46 @@
|
||||
# 01-RESEARCH
|
||||
|
||||
**The research efforts live in the knowledge base, not here** (ADR 0290). The knowledge base is the module
|
||||
`mesh-knowledge`, holding the seat `knowledge-base`; it holds every effort whole — its number, title, status,
|
||||
the day it began, its documents, its links and every version — and it is the only place an effort is read.
|
||||
This repository keeps no copy. From any node and any folder, through the mesh MCP server:
|
||||
`knowledge-base.ask` with a question in plain words, or `knowledge-base.get` with `kind: research` and a
|
||||
number (`reasoning: true`, or `section` with a document's name, for its documents). An effort is started
|
||||
with `knowledge-base.reserve` and `knowledge-base.propose`, as playbook
|
||||
[01](../00-META/process/01-research.md) says, and closed through playbook
|
||||
[02](../00-META/process/02-graduation.md). The efforts left this folder at hq commit 76b346af,
|
||||
the commit the knowledge base's records name as their source (issue 426).
|
||||
Investigations that have not yet hardened into design.
|
||||
|
||||
## Structure
|
||||
|
||||
Each effort lives in `NNN-descriptive-name/` and **must** contain `00-overview.md`, carrying its
|
||||
state in YAML frontmatter and a prose summary below it:
|
||||
|
||||
```yaml
|
||||
---
|
||||
status: active | graduated | abandoned
|
||||
initiated: YYYY-MM-DD
|
||||
touches: [] # design docs, subsystems or areas the effort bears on
|
||||
became: [] # required when status is terminal — what it turned into
|
||||
---
|
||||
```
|
||||
|
||||
The prose says what is being investigated, why, and what it touches. It does not restate the
|
||||
status — status lives in one place, and two places is one too many.
|
||||
|
||||
Further documents in the same folder hold the work itself: notes, evidence, option analyses,
|
||||
draft designs.
|
||||
|
||||
## Lifecycle
|
||||
|
||||
| status | Meaning |
|
||||
|---|---|
|
||||
| `active` | Investigation in progress. |
|
||||
| `graduated` | Checked against `00-META`, decided in `02-DECISIONS/`, and specified in `03-DESIGN` — see `became:`. |
|
||||
| `abandoned` | Stopped or superseded. Nothing is deleted. |
|
||||
|
||||
An effort graduates by producing a decision record **and** a `03-DESIGN` entry. It is abandoned
|
||||
in place — never deleted. What was rejected, and why, is the more expensive half to
|
||||
rediscover.
|
||||
|
||||
Starting and closing efforts is playbook territory:
|
||||
[`00-META/process/01-research.md`](../00-META/process/01-research.md) and
|
||||
[`02-graduation.md`](../00-META/process/02-graduation.md).
|
||||
|
||||
## Rules
|
||||
|
||||
- Markdown only. Do not skip or reuse a sequence number.
|
||||
- **Evidence, not assertion.** An effort that measured nothing has not finished.
|
||||
- Research describes real observations but **never identifies the mesh it observed**. The shape
|
||||
of a finding survives anonymisation intact.
|
||||
|
||||
@@ -0,0 +1,235 @@
|
||||
---
|
||||
topic: the mesh
|
||||
status: accepted
|
||||
date: 2026-08-22
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
---
|
||||
|
||||
# 1. The mesh brokers capabilities; nodes host; agents think
|
||||
|
||||
## Context
|
||||
|
||||
HAL has 124 modules. The count is not the problem — it is the symptom. Modules are split
|
||||
because splitting is the only granularity the platform offers, and domains are merged
|
||||
because a shared database is the only integration it offers. Both pressures push in the
|
||||
same direction: boundaries end up drawn by deployment accident rather than by domain.
|
||||
|
||||
Three observations establish the state.
|
||||
|
||||
**The word "agent" means two different things.** The original design treated a node as an
|
||||
agent with thinking abilities. Later, noxflow implemented agents as employees with
|
||||
skills, workspaces and tasks. Both survive. The collision is visible in the data — there
|
||||
are **two agent rows per node**:
|
||||
|
||||
| agent | skills |
|
||||
|---|---|
|
||||
| one named after the node | `{deploy,verify,operate}` |
|
||||
| one named `hal-<node>` | `{}` |
|
||||
|
||||
One carries the work; the other carries only identity, existing to hold a licence for
|
||||
HAL's own sessions. The same split appears in the schema: `nodes.hal_claude_account` and
|
||||
`agents.claude_account` are one fact in two tables in two databases, and what was
|
||||
documented as "three licence touchpoints" is one concept modelled three times.
|
||||
|
||||
**Domains integrate by sharing a schema.** `noxflow` is 45 tables spanning five domains —
|
||||
tasks (7), agents (13), knowledge (8), meetings (7), scheduling (2). Its original purpose
|
||||
is 7 of 45. Because agent identity lives in noxflow's database, work that belongs
|
||||
elsewhere must be implemented there: per-agent Claude credentials had to be written by
|
||||
the noxflow runtime, even though node identity — the same kind of fact — lives in the
|
||||
mesh registry.
|
||||
|
||||
**The core domain has no context to live in.** The mesh's distinguishing feature is that
|
||||
a module declares `requires: postgres/database` and never learns where the database
|
||||
lives, who owns the credential, or how it rotates. That is capability brokering, and it
|
||||
is what makes this a mesh rather than four machines with a configuration manager. Yet the
|
||||
logic implementing it sits in `modules/postgres/tools/index.ts` — a provider module,
|
||||
where "the mesh brokers credentials" cannot be expressed. On 2026-08-22 three of its
|
||||
invariants were found violated simultaneously (see Consequences).
|
||||
|
||||
## Considered Options
|
||||
|
||||
1. **Keep the current layout; fix bugs as they surface.** Rejected. The faults are not
|
||||
independent. Every incident on 2026-08-22 — credentials written by the wrong module, a
|
||||
dependency question answered wrongly twice, an invariant enforced nowhere — traced to
|
||||
a boundary that was never stated. Fixing them individually leaves the generator intact.
|
||||
|
||||
2. **Merge aggressively into few large modules.** Fewer names, same problem: a shared
|
||||
schema across domains is what produced noxflow, and doing it deliberately would
|
||||
produce it again at larger scale.
|
||||
|
||||
3. **Decompose by bounded context, with the mesh as a broker.** Name contexts after their
|
||||
aggregates, integrate through a published record rather than a shared schema, and let
|
||||
deployment granularity be a feature-level concern rather than a reason to create a
|
||||
module. **Adopted.**
|
||||
|
||||
## Decision
|
||||
|
||||
### The domain, in one sentence
|
||||
|
||||
**The mesh brokers capabilities. Nodes are places where work runs. Agents are personas
|
||||
that think and act.** Everything else supports one of those three.
|
||||
|
||||
### What this is, plainly — and what "mesh" does not mean
|
||||
|
||||
*Written 2026-08-29, from working through connectivity and asking whether the word still fits.*
|
||||
|
||||
Four layers. Naming them honestly is worth more than the word on the tin:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **machines are linked by a private network** | and every machine reaches every other over it |
|
||||
| **one node holds knowledge of all of them** | the control plane, and only it |
|
||||
| **modules are how anything is built and delivered** | this *is* the CI/CD, not something beside it ([ADR 0010](0010-delivery.md)) |
|
||||
| **every node runs a session you can message** | a feature of the node, remembering across callers; any node can message any other ([ADR 0004](0004-a-node-and-how-it-joins.md)) |
|
||||
| **workers are hired onto nodes to do tasks** | employees, with a lifecycle — a different thing from the row above ([ADR 0003](0003-agents-are-persistent-employees.md)) |
|
||||
|
||||
**The last two rows are not the same thing and the vocabulary of one does not describe the other.**
|
||||
A node's session comes with the machine: nobody hires it, it holds no tasks, it is never
|
||||
reassigned, and it goes when the node leaves. A worker is an employee — named, hired, drained,
|
||||
retired, movable. They are built from the same parts and run on entirely different terms, and
|
||||
collapsing them is how the employee vocabulary ends up stretched over something it does not fit.
|
||||
|
||||
**Neither makes the node itself a thinking thing.** A node is a machine; both of these run *on*
|
||||
one, which is why *a node does not authenticate to a model provider, agents do* is unaffected by
|
||||
either.
|
||||
|
||||
**Where the value is** is that both can reach across the whole set: a shell, a service, a file, or
|
||||
simply a question to another node. Not machines that can be configured centrally, which is
|
||||
ordinary, but a set of machines that can be worked across as though they were one.
|
||||
|
||||
**This is not a mesh in the peer-to-peer sense and will not become one.** The word describes what
|
||||
machines can reach, not how they are governed:
|
||||
|
||||
| | a mesh? |
|
||||
|---|---|
|
||||
| what a machine can reach | **yes** — genuinely any to any |
|
||||
| how the traffic travels | no — anything crossing sites transits the hub |
|
||||
| who decides | no. One node, declared |
|
||||
|
||||
**And *master* overstates it in the other direction.** A master implies the others need it in
|
||||
order to function. They do not: every node holds what it was last told and runs from that copy
|
||||
*always* — not as a fallback, as the only mode it has. So the control plane being gone is every
|
||||
node in the ordinary disconnected situation at once, and **what is lost is change, not
|
||||
operation.**
|
||||
|
||||
The accurate phrase is **one authority, no failover**, and both halves are deliberate
|
||||
([ADR 0006](0006-the-substrate-and-the-control-plane.md)).
|
||||
|
||||
### Nodes and agents are decoupled
|
||||
|
||||
A node is a place where an agent can run — that is the entire relationship. There is no
|
||||
resident agent, no node-owned identity, no ownership in either direction. Agents named
|
||||
after a node remain, as **ordinary agents** that happen to hold infra skills.
|
||||
|
||||
Consequently `nodes.node_license` and `nodes.hal_claude_account` cease to exist: a node
|
||||
does not authenticate to a model provider, agents do. The two agent rows per node merge.
|
||||
|
||||
### There is one kind of participant, and some are human
|
||||
|
||||
Human and non-human participants are both **agents**. Both hold identity and credentials;
|
||||
both act, remember and coordinate. What differs is **modality** — how an agent acts:
|
||||
|
||||
| modality | credential is delivered to |
|
||||
|---|---|
|
||||
| spawned session | that agent's own config directory |
|
||||
| shell or desktop | that agent's home on the node it acts from |
|
||||
|
||||
A node holds no licence. **An agent holds credentials, and delivery follows that agent's
|
||||
node bindings and modality.** A human agent's grant arrives in the home directory of the
|
||||
user it acts as, on the nodes it is bound to — the same rule that puts a spawned agent's
|
||||
grant in its config directory, with a different target.
|
||||
|
||||
This requires one fact the mesh does not record today: which user, on which node, a given
|
||||
human agent acts as. Adding it is what removes the node licence — the node is currently
|
||||
standing in for an identity the mesh cannot name. It is also what makes a second human
|
||||
agent require no new mechanism.
|
||||
|
||||
### Contexts
|
||||
|
||||
| context | aggregate | subdomain |
|
||||
|---|---|---|
|
||||
| `hal/mesh` | **Provision**, Node, Module — brokering and its bookkeeping | core |
|
||||
| `hal/agents` | **Agent** — identity, licence, runs, memory, thoughts | core |
|
||||
| `hal/work` | **Task** — workflows, bindings | core |
|
||||
| `hal/stream` | **Thread** — mentions, messages, meetings, notifications | core |
|
||||
| `hal/delivery` | **Pipeline** — jobs, artifacts, features | supporting |
|
||||
| `hal/knowledge` | **Document** — spaces, revisions, review | supporting |
|
||||
| `hal/ai` | **Licence** — provider grants and rotation | supporting |
|
||||
| `hal/observability` | **Check** | supporting |
|
||||
| `hal/config` | **Setting** — env, secrets, PKI | generic |
|
||||
|
||||
A *brain* — memory, thoughts, cognition — is a concept the Agent aggregate owns. It is
|
||||
not a module. Anatomy makes attractive names and poor boundaries; today `hal/brain` names
|
||||
infrastructure and `hal/cortex` describes itself as messaging while running nowhere.
|
||||
|
||||
### Provisioning is the core domain, not plumbing
|
||||
|
||||
`hal/mesh` is a broker; the registry is its bookkeeping. Its invariants are explicit and
|
||||
owned:
|
||||
|
||||
- one rotation source per resource
|
||||
- a credential change fans out to every consumer
|
||||
- a consumer never holds a credential the provider does not know about
|
||||
|
||||
### Contexts integrate through the record, never a shared schema
|
||||
|
||||
`hal/stream` is the published language. A context publishes; it does not join across a
|
||||
boundary. This is what dissolves "meetings" as a domain — a meeting is a thread, and a
|
||||
notification is a mention not yet read.
|
||||
|
||||
### Third-party software leaves the repository
|
||||
|
||||
`plex`, `sonarr`, `postgres`, `verdaccio`, `docker-registry` and 88 others run **on** the
|
||||
mesh; they are not **of** it. The pipeline and provisioning are deliberately
|
||||
module-agnostic, so HAL's own modules dogfood exactly what external modules use — which
|
||||
is what makes the separation safe rather than merely tidy.
|
||||
|
||||
## Consequences
|
||||
|
||||
**The invariants now have an owner, and were measurably unowned before.** On 2026-08-22,
|
||||
`provision_ensure` — documented as "NEVER rotates an existing secret" — was found to mint
|
||||
a new password on every adoption and update only the provider's row. `hal_notifications`
|
||||
consumers on three nodes held dead credentials for two days; `hal_transcripts` had two
|
||||
rows written 216 ms apart, so at most one could match the live role.
|
||||
|
||||
**noxflow dissolves.** `hal/work` inherits tasks and workflows — the concept it was built
|
||||
for. Agents, knowledge, meetings and scheduling return to their contexts. The name goes
|
||||
away.
|
||||
|
||||
**`hal/sdk` shrinks.** 155 files, 34,636 lines, containing code from every context —
|
||||
including `workflow-engine.ts` and `task-commands.ts`, work-domain logic in the kernel
|
||||
every module imports. Each landed there to avoid a cycle between modules that both needed
|
||||
it; a domain module can only own its shared code once the domain has a module. Extraction
|
||||
is therefore downstream of this decision, not independent of it.
|
||||
|
||||
**Two mechanisms are prerequisites, not follow-ups.**
|
||||
|
||||
- *Named features with per-node opt-in.* Without it, every independently deployable unit
|
||||
inside a context becomes a module again and the count returns. `hal/claude-licences`
|
||||
exists solely because one daemon must run on one node.
|
||||
- *A local mesh in containers.* `dev_up` starts providers "via systemctl (same as
|
||||
production)" — it borrows the host, and no mesh can be stood up locally. Everything that
|
||||
manifests **between** nodes is therefore discoverable only in production, which is where
|
||||
every fault of 2026-08-22 was found. A refactor of this size is otherwise unverifiable.
|
||||
|
||||
**Credential delivery becomes uniform.** Per-agent credential directories, built for
|
||||
spawned sessions, extend to human agents unchanged. A file previously scoped to a node
|
||||
becomes scoped to an agent — which would have prevented the class of failure where a
|
||||
rotation reached one node of four while the mesh reported success.
|
||||
|
||||
**Migration is incremental and long.** Contexts can be extracted one at a time behind the
|
||||
existing pipeline. Nothing here requires a flag day, and nothing here is cheap.
|
||||
|
||||
## References
|
||||
|
||||
- [`01-RESEARCH/001-module-domain-decomposition`](../01-RESEARCH/001-module-domain-decomposition/analysis.md)
|
||||
— current-state evidence, table counts, open questions
|
||||
- [`00-META/how-we-build.md`](../00-META/how-we-build.md) — naming and integration rules
|
||||
- `modules/hal/sdk/src/feature-handlers/index.ts` — `FEATURE_HANDLERS`, the fixed handler
|
||||
array that makes a feature a singleton per module
|
||||
- `modules/postgres/tools/index.ts` — the adoption path that rotates a shared credential
|
||||
- Mediahuis `papa-hq`, ADR 0020 *Composable, independently-shippable modules* — the
|
||||
constraints that make a unit independently shippable, applicable unchanged to features
|
||||
- impire.io / soulstream — *the record* as integration substrate, personas over services,
|
||||
and "cheap awareness and expensive thinking"
|
||||
@@ -0,0 +1,69 @@
|
||||
---
|
||||
topic: the mesh
|
||||
status: accepted
|
||||
date: 2026-02-25
|
||||
deciders: jochen
|
||||
reconstructed: true
|
||||
---
|
||||
|
||||
# 2. Nodes communicate over a message broker, not over HTTP
|
||||
|
||||
> Reconstructed after the fact from the evidence cited below. The decision was taken in
|
||||
> implementation, not in a record; this document states what was decided and why, not a
|
||||
> deliberation that happened.
|
||||
|
||||
## Context
|
||||
|
||||
The mesh is a set of machines that must call each other's capabilities. On the day the
|
||||
repository was founded there was no inter-node transport at all — each node was configured
|
||||
independently and shared nothing at runtime.
|
||||
|
||||
Three properties were required and are visible in everything built since:
|
||||
|
||||
- A node behind a household NAT must participate fully. It can dial out; nothing can dial in.
|
||||
- A node that is asleep, rebooting or upgrading must not cause a caller to fail — the request
|
||||
should wait, not error.
|
||||
- Adding a node must not require editing anything on the nodes that already exist.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **HTTP APIs between nodes.** Rejected. Every node becomes a server that every other node
|
||||
must be able to reach, which the NAT case makes impossible without inbound tunnels to each
|
||||
participant. It also makes node liveness a caller's problem: a request to a sleeping node
|
||||
is an error rather than a wait.
|
||||
2. **Polling a shared database.** Rejected. Latency is the poll interval, load is constant and
|
||||
independent of demand, and request/reply has to be built on top of it by hand.
|
||||
3. **A central message broker with per-node exchanges.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
All inter-node communication goes through a message broker. Every node owns a topic exchange
|
||||
named for itself and a request queue; a shared mesh exchange carries commands and events that
|
||||
are not addressed to one node.
|
||||
|
||||
Three message shapes, and only three:
|
||||
|
||||
- **RPC** — request/reply, for calling a capability that lives on another node.
|
||||
- **Commands** — instructions to do a stage of work, addressed by what is to be done.
|
||||
- **Events** — statements that something happened, addressed to nobody.
|
||||
|
||||
Every node dials the broker outbound. Nothing dials a node.
|
||||
|
||||
## Consequences
|
||||
|
||||
- NAT stops being an architectural concern. A node's reachability is a property of the broker
|
||||
connection, not of its network position.
|
||||
- A call to a node that is down waits in that node's queue instead of failing. This is usually
|
||||
right and occasionally the wrong thing entirely — a queued command for a node that never
|
||||
returns is a stall with no error, which is the failure shape this mesh keeps rediscovering.
|
||||
- The broker is a single point of failure and a single point of trust. Its credential is
|
||||
mesh-wide, so rotating it is a mesh-wide operation.
|
||||
- Tools never leave the host: a remote call proxies over the broker and the credentials stay
|
||||
where the capability is.
|
||||
|
||||
## References
|
||||
|
||||
- The broker was stood up on 2026-02-25, the second day of the repository.
|
||||
- Knowledge base: `mesh` (transport, exchanges, queue naming), `troubleshooting/amqp-credential-rotation`.
|
||||
- The stall shape is recorded in `troubleshooting/empty-pipeline-blocks-the-queue` and
|
||||
`troubleshooting/daemon-and-tool-server-share-a-request-queue`.
|
||||
@@ -0,0 +1,75 @@
|
||||
---
|
||||
topic: the mesh
|
||||
status: accepted
|
||||
date: 2026-07-12
|
||||
deciders: jochen
|
||||
reconstructed: true
|
||||
---
|
||||
|
||||
# 3. An agent is a persistent employee, not an instance of a pool
|
||||
|
||||
> Reconstructed after the fact from the evidence cited below.
|
||||
|
||||
## Context
|
||||
|
||||
Agents were originally a **pool**: a named kind of worker, scaled to some number of
|
||||
interchangeable instances. Work went to whichever instance was free.
|
||||
|
||||
That model has no place to put the things that turn out to matter. An agent that accumulates
|
||||
knowledge of a domain cannot keep it, because the next task lands on a different instance. An
|
||||
agent cannot own a workspace, because there are several of it. It cannot be held to a policy —
|
||||
warned for a violation, then dismissed — because there is no continuing subject to warn.
|
||||
|
||||
Scaling was also solving a problem the mesh does not have. Instances were being multiplied to
|
||||
get concurrency, when concurrency is a property of how much work one agent may hold at once.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **Keep the pool, attach memory to the pool.** Rejected: shared memory across
|
||||
interchangeable workers is a knowledge base, not an agent's experience, and the mesh
|
||||
already has one.
|
||||
2. **Keep the pool, make instances sticky.** Rejected as a pool pretending to be identities —
|
||||
identity by scheduling accident, lost on any restart.
|
||||
3. **One agent is one persistent identity, with concurrency as a property of it.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
An agent is a **singular, named, persistent identity**: a home node, a workspace on that node,
|
||||
accumulating memory, and a lifecycle — hired, active, draining, retired. Not a pool member.
|
||||
|
||||
Concurrency is a property of the agent, not a count of copies: an agent has a cap on how many
|
||||
sessions it may hold at once.
|
||||
|
||||
Lifecycle is explicit and has verbs. An agent is hired onto a node; it may be reassigned while
|
||||
idle; it is retired by draining first, and forced only deliberately. Retired agents are not
|
||||
deleted.
|
||||
|
||||
Surge capacity is expressed within the model rather than against it: a template agent is a
|
||||
blueprint, cloned into a real agent with a lifetime when a queue grows, drained and retired
|
||||
when it expires. A temporary employee is still an employee.
|
||||
|
||||
Some agents are **human**. What differs is modality — how the agent acts — not category. A node
|
||||
itself is an agent of a kind exempt from the hiring lifecycle.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Memory, workspace and reputation have a subject to belong to. Policy becomes possible: an
|
||||
agent that violates a rule can be warned, and warned agents can be dismissed.
|
||||
- The mesh gained a hiring model, and with it the question of who may hire.
|
||||
- Scaling by adding instances is gone. If one agent is saturated, either its session cap rises
|
||||
or another agent is hired — both deliberate acts.
|
||||
- The transition was not free. Lifecycle columns had to reach every query that selects an
|
||||
agent, and the ones that were missed failed at the moment of hiring rather than at startup.
|
||||
- This is the decision [ADR 0001](0001-mesh-brokers-nodes-host-agents-think.md) generalises:
|
||||
one kind of participant, differing only in modality.
|
||||
|
||||
## References
|
||||
|
||||
- `docs(adr): agents as persistent employees + MINERVA librarian` (#495), 2026-07-12 — the
|
||||
original record, in the code repository.
|
||||
- `feat(B4): one persistent employee, N sessions — rename max_instances → max_sessions` (#547)
|
||||
and `feat(noxflow): B3 — workspace provisioner for agent employee model` (#549), 2026-07-20.
|
||||
- `feat(noxflow): warn-then-fire agents who merge to main without review` (#209), 2026-06-01 —
|
||||
policy that presumes a continuing subject, predating the model that provides one.
|
||||
- Knowledge base: `agents/employee-lifecycle`, `agents/temp-surge`, `agents/workspace-layout`.
|
||||
- The migration cost: `troubleshooting`/`noxflow-agent-enriched-select-missing-lifecycle-columns` (#546).
|
||||
@@ -0,0 +1,263 @@
|
||||
---
|
||||
topic: the tiers
|
||||
status: accepted
|
||||
date: 2026-08-28
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
---
|
||||
|
||||
# 4. A node, and how it joins
|
||||
|
||||
*Consolidated 2026-08-28 from four records.*
|
||||
|
||||
## What a node is
|
||||
|
||||
**A managed machine inside the mesh.** Not a device that is merely known about, not an
|
||||
unprivileged something. If the mesh does not manage it, it is not a node — it is a client, a
|
||||
peer, or a thing on the network, and those want their own names rather than a weakened version of
|
||||
this one.
|
||||
|
||||
**A disconnected node is still a node, in a different situation.** Reachability is **state, not
|
||||
class**. A machine switched off, roaming, or behind a connection that has dropped has not become
|
||||
a lesser kind of thing; it has a last-known state and a pending set of declarations.
|
||||
|
||||
The distinction people reach for is real, but it is **capability** — what this machine can be
|
||||
asked to do — and that belongs in the host's profile rather than in the definition of a node.
|
||||
|
||||
**This is the rule that does the most work elsewhere.** A single control plane is tolerable
|
||||
because its absence is every node in the ordinary disconnected situation at once. An episodic
|
||||
host on a phone is that situation more often. Neither needed a new mechanism.
|
||||
|
||||
### A node runs one agent session
|
||||
|
||||
*Written 2026-08-29. It runs on every node today and appeared in no record, which is how something
|
||||
deliberate comes to look accidental.*
|
||||
|
||||
**A node is a machine. The session is a feature of it** — one of the things running there, like the
|
||||
host, like any workload. The node does not think; something on the node does. Which is why
|
||||
[ADR 0001](0001-mesh-brokers-nodes-host-agents-think.md)'s *a node does not authenticate to a model
|
||||
provider, agents do* holds unchanged: the session authenticates, and it is not the machine.
|
||||
|
||||
**It is permanent, and it remembers.** Anything in the mesh can send it a message; it replies; and
|
||||
what it was asked ten minutes ago is still there next week, alongside what everything else asked in
|
||||
between — the same way both sides of any conversation remember it.
|
||||
|
||||
Its system prompt is the node's **engram** — what makes one node's replies recognisably its own
|
||||
rather than generic.
|
||||
|
||||
**It has its own tools**, and fewer than a session a person is driving directly. So a question can
|
||||
be answered by going and looking: *what is in our forge*, not only *what is your battery*.
|
||||
|
||||
**Messages travel the broker like everything else** ([ADR 0002](0002-nodes-communicate-over-a-broker.md)).
|
||||
There is no second transport and nothing is dialled.
|
||||
|
||||
**Any node can message any node, and this is the one part of the system that is genuinely a mesh**
|
||||
— symmetric, with no centre. A node that is asked something it does not have can ask another, and
|
||||
how it passes the question on is its own business: it may say who wants to know, or simply ask. A
|
||||
person relaying a question makes the same choice, and it follows from the engram rather than from a
|
||||
message format.
|
||||
|
||||
**There is no authorisation between nodes.** Every node is the operator's own, so a message from
|
||||
one is a message from them, and asking a node something is asking a colleague rather than
|
||||
presenting credentials. Stated once so it is not discovered later: **the mesh boundary is therefore
|
||||
the security boundary** — anything inside can reach what any node can reach, which is what puts the
|
||||
whole perimeter on the token and the overlay
|
||||
([ADR 0007](0007-connectivity.md)).
|
||||
|
||||
**It can be switched off, and switched off it still answers.** A node whose session is disabled
|
||||
replies saying so, at once, with no model involved — the queue is still read, and the state is the
|
||||
reply. That is deliberate and it is the same rule the host follows about a service that does not
|
||||
exist: **absence must never be indistinguishable from a failure to answer.** A node with nothing
|
||||
there is a silence somebody has to go and diagnose; a node that says *I am switched off* is not.
|
||||
|
||||
**One per node, always, and it cannot be moved to another machine.** Two and nothing decides which
|
||||
replies; none and the node is mute; moved, and one machine is answering as another.
|
||||
|
||||
**It is not an employee** ([ADR 0003](0003-agents-are-persistent-employees.md)). Nobody hires it,
|
||||
it holds no tasks, it drains nothing and it is never reassigned — that vocabulary was written for
|
||||
workers and does not describe this. It exists because the node does, and it is gone when the node
|
||||
leaves.
|
||||
|
||||
## How it joins
|
||||
|
||||
**The host has one behaviour and two sources of declaration.** What differs between the first
|
||||
node and the fiftieth is not what the host does but where the declaration comes from — and, as
|
||||
above, that is a situation rather than a class.
|
||||
|
||||
| | declaration comes from |
|
||||
|---|---|
|
||||
| no mesh reachable | the pinned bundle the host carries |
|
||||
| mesh reachable | the control plane, over the link |
|
||||
|
||||
**The first node is not a different kind of node.** It is a node whose mesh is not up yet. It
|
||||
applies the bundle it carries, the control plane comes up on top of it, and from that moment it
|
||||
takes declarations like everything else. **Its specialness is temporary and self-erasing**, which
|
||||
is what the hand-run bootstrap scripts never were.
|
||||
|
||||
**A joining node does the minimum to be reachable and nothing else** — an identity, an address,
|
||||
and one peer to reach. It does **not** compute the overlay: the whole peer set is derived
|
||||
centrally and pushed down.
|
||||
|
||||
That is also why the migration is smaller than it looked. The hard part of the overlay — every
|
||||
node's key, address, site and reachability — is only needed to compute the *whole* mesh, and a
|
||||
joining node needs one peer.
|
||||
|
||||
## The link is the security boundary
|
||||
|
||||
**Everything reaching a node arrives one way**, and four properties make that a boundary rather
|
||||
than a pipe.
|
||||
|
||||
**It is outbound and node-initiated.** The node dials the control plane; nothing dials a node. Not
|
||||
only defensive — most nodes sit behind a household connection with no forwarded port, so an
|
||||
inbound control channel would work for one node and not the rest, and the difference would be
|
||||
invisible until it mattered. **A node has no listening control surface at all.**
|
||||
|
||||
**A node holds its own identity and nothing else.** No shared secret, no credential to anything it
|
||||
does not own. **Compromise of a node is compromise of that node** — which the current arrangement
|
||||
does not have, because every node permanently holds the same database and object-store
|
||||
credentials, and there is no mechanism that rotates one and informs everything holding it.
|
||||
|
||||
### What that identity is: a keypair the node generates
|
||||
|
||||
*Written 2026-08-29. This is the same rule as the sentence above, and it had been treated as an
|
||||
open question for weeks because of a word.*
|
||||
|
||||
**The node generates a keypair. The private half never leaves the machine. The mesh records the
|
||||
public half.** Ed25519, the same as the control plane's signing key, in the other direction:
|
||||
the mesh proves itself to a node by signing, and a node proves itself to the mesh by signing.
|
||||
|
||||
**This was never open.** [`08-connectivity.md`](../03-DESIGN/01-to-be/08-connectivity.md) already
|
||||
says it of the overlay keys, in these words: *each node generates its own keypair, the private key
|
||||
never leaves the machine, the public key is published to the mesh* — and adds that this **is**
|
||||
ADR 0004's *a node holds its own identity*, applied. What was missing was applying it to the thing
|
||||
this record is about.
|
||||
|
||||
**The word that caused it:** the lifecycle says a joining node *receives* its own durable identity,
|
||||
which reads as the mesh issuing something, and then the question becomes *issuing what*. It does
|
||||
not issue anything. The node arrives holding its identity; what it receives is **being known**.
|
||||
Enrolment is the moment the mesh writes down a public key it will believe, and the one-time secret
|
||||
is what buys the right to have it written down.
|
||||
|
||||
**Everything above then holds literally.** Nothing is stored that could be stolen and replayed: the
|
||||
mesh's copy is a public key, so a copy of the mesh's database grants nothing. *Compromise of a node
|
||||
is compromise of that node* becomes true rather than aspirational, because the only secret on a
|
||||
machine is the one that identifies it.
|
||||
|
||||
### What "connecting to the mesh" is, concretely
|
||||
|
||||
*Written 2026-08-29, because it was asked and this record had never said it.*
|
||||
|
||||
**One outbound AMQP connection from the node to the broker, held open.** That is all of it. There
|
||||
is no second connection and nothing is ever dialled *at* a node. Being in the mesh, operationally,
|
||||
means that connection is up; being disconnected means it is not
|
||||
([`09-the-node-lifecycle.md`](../03-DESIGN/01-to-be/09-the-node-lifecycle.md)).
|
||||
|
||||
**Two different things ride on it, and conflating them is what made this confusing:**
|
||||
|
||||
| | what it answers | who issues it |
|
||||
|---|---|---|
|
||||
| **an AMQP account** | may this connection be accepted at all | **the mesh, at enrolment** |
|
||||
| **the node's keypair** | which node is speaking, on every message | **the node**, above |
|
||||
|
||||
**The account is the mesh's to issue**, and per node. The broker has to authenticate somebody
|
||||
before a connection exists, and a shared account would let any node consume another's queue —
|
||||
which is the shared-credential fault this record exists to remove, reappearing at the transport.
|
||||
So enrolment creates that node's account and hands it over, and it is rotatable without touching
|
||||
the node's identity.
|
||||
|
||||
**The keypair is not made redundant by it.** With only an account, the control plane would know
|
||||
which node is speaking *because the broker says so* — and that is the same transitive authority
|
||||
this record refuses in the other direction. A compromised broker could then attribute reports to
|
||||
whichever node it liked, and the control plane would act on them. Signing is what removes the
|
||||
broker from the question in both directions.
|
||||
|
||||
**So a node holds two things after enrolment**: a credential the mesh issued for reaching the
|
||||
broker, and a key it generated itself that the mesh only ever sees the public half of. Both are
|
||||
its own, neither reaches anything else, and *compromise of a node is compromise of that node*
|
||||
still holds.
|
||||
|
||||
### Its own key, not the machine's SSH host key
|
||||
|
||||
Reusing the host key is the obvious economy and it is refused, for reasons that are operational
|
||||
rather than fastidious:
|
||||
|
||||
- **It is regenerated by ordinary events.** A reinstall, an image cloned, `ssh-keygen -A` on a
|
||||
rebuild — each silently un-enrols the node, and the failure appears as an authentication problem
|
||||
with no cause anybody changed.
|
||||
- **It is managed by something else.** Its lifecycle belongs to the machine's SSH daemon, and an
|
||||
identity the mesh depends on should not rotate on a schedule the mesh does not know about.
|
||||
- **Not every node has one.** A partial host has no SSH daemon
|
||||
([ADR 0005](0005-the-node-host.md)), and an identity scheme that excludes a supported kind of
|
||||
node is not one.
|
||||
|
||||
**The mesh should still know the host key** — it knows every node, so it can distribute host keys
|
||||
the same way it distributes authorised keys
|
||||
([ADR 0006](0006-the-substrate-and-the-control-plane.md)), and node-to-node SSH stops depending on
|
||||
trust-on-first-use. That is the good half of the idea, kept.
|
||||
|
||||
**Authority is mutual.** The node proves it may join, and the control plane proves it is the
|
||||
mesh. One-way is not enough: the host applies whatever the link delivers, so a node that cannot
|
||||
tell the mesh from something impersonating it will apply that something's declarations.
|
||||
|
||||
**What may be pushed is bounded by form, not by trust.** Declarations of known shape, never a
|
||||
command to run. Stated honestly, **this bounds form and not impact**: a compromised control plane
|
||||
can declare harmful state and the host will apply it faithfully, because that is what it is for.
|
||||
What the property buys is that the blast radius is *describable* — exactly what the declaration
|
||||
language can express, which can be reviewed. An arbitrary command channel has no such bound.
|
||||
|
||||
## The enrolment token carries the mesh
|
||||
|
||||
Mutual authority needs the node to verify something before it trusts anything, and that is a
|
||||
circle: verifying the mesh needs the mesh's certificate authority, and obtaining one means
|
||||
trusting whoever hands it over. There is a second circle beside it — a node must reach the mesh
|
||||
before the mesh has configured it, so it can resolve no mesh name.
|
||||
|
||||
**Both are the same shape: a node needs a fact about the mesh before it has any trustworthy way
|
||||
to obtain one.** So that fact arrives by a path other than the mesh.
|
||||
|
||||
**The token carries five things**, and it is the only thing a joining node needs:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **who it is** | the name the mesh calls this machine |
|
||||
| **where** | the broker's **address**, not a name — there is no resolution yet, and this is why none is needed |
|
||||
| **what it is connecting to** | the fingerprint of the broker's certificate |
|
||||
| **who it will believe** | the control plane's signing identity |
|
||||
| **the right to join** | a one-time secret, useless once used and useless after it expires |
|
||||
|
||||
*The first row was added 2026-08-30, from raising a mesh end to end for the first time.* It reads
|
||||
like an oversight and is not: **the node cannot work its own name out.** The name is the mesh's,
|
||||
chosen when the record was created, and the broker account the node authenticates as is named
|
||||
after it — so it must be known *before* the mesh can tell the node anything. It is not a secret,
|
||||
and whoever issues the token already has it.
|
||||
|
||||
Without it, enrolment fails at the broker with an empty username and a message about credentials,
|
||||
which points at everything except the cause. **A missing fact that surfaces as an authentication
|
||||
error is worse than one that surfaces as a missing fact.**
|
||||
|
||||
**Carried out of band**, by the person adopting the machine. That is what breaks both circles:
|
||||
its authenticity comes from the channel it travelled, not from anything the node can check
|
||||
afterwards. **Trust on first use, with the first use moved out of band** — the difference between
|
||||
a pin and a guess.
|
||||
|
||||
**The endpoint and the authority are two identities.** A node connects to the broker and takes
|
||||
instruction from the control plane behind it. Pinning only the broker would make the control
|
||||
plane's authority *transitive*, and a compromised broker could then forge declarations — which,
|
||||
since the host applies whatever the link delivers, is the whole machine. So the transport is
|
||||
verified once at connect, and **each declaration is verified by its signature, every time**.
|
||||
|
||||
**What this settles:** the mesh's certificate authority is not a bootstrap concern — it certifies
|
||||
internal names once a node is a member. Nothing needs name resolution before the link. And
|
||||
nothing is placed on disk beforehand except the token, which is the first moment *a node holds
|
||||
only its own identity* becomes true rather than aspirational.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **Declarations must be signed**, and the host must tell *this is not from the mesh I joined*
|
||||
apart from *this is malformed*. Rotating the signing identity is a fleet-wide operation with an
|
||||
overlapping rollover, and that is the cost of not trusting the broker.
|
||||
- **The token becomes security-critical**, because it carries the pin. Tampering substitutes the
|
||||
mesh — which is strictly better than the alternative, where there is nothing to tamper with and
|
||||
the node trusts the first answer unconditionally.
|
||||
- **A rejoining node is ordinary.** There is no long-lived secret to recover, so a node that lost
|
||||
its identity gets a new token.
|
||||
@@ -0,0 +1,155 @@
|
||||
---
|
||||
topic: the tiers
|
||||
status: accepted
|
||||
date: 2026-08-28
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
---
|
||||
|
||||
# 5. The node host
|
||||
|
||||
*Consolidated 2026-08-28 from eight records. Tier 0 is one component and was decided over a
|
||||
week; the reasoning is kept, the fragmentation is not.*
|
||||
|
||||
## It applies; it does not decide
|
||||
|
||||
**The host makes a machine match what it was told, and never works out what that should be.**
|
||||
|
||||
This is the line the whole tier rests on, and it is not about privilege — it is about what a
|
||||
single machine can *know*. Deciding needs knowledge the machine does not have: which nodes should
|
||||
run a store, which peers belong in an overlay, whether a node has been unreachable for a week.
|
||||
Anything needing a second node is the control plane's.
|
||||
|
||||
The practical form: **the host never queries the mesh's database and holds no credential to it.**
|
||||
Two modules in the current mesh do, and they are the reason every node permanently carries a
|
||||
database credential.
|
||||
|
||||
## It depends on nothing that must be installed first
|
||||
|
||||
**A statically linked binary. Copy it onto a machine and run it — that is the whole
|
||||
installation.** Written in Go, because the job is system-level and because a runtime that must be
|
||||
installed first would make the host depend on the thing it exists to install.
|
||||
|
||||
**What it needs from the machine is not a dependency in this sense.** An init is not installed;
|
||||
it is what the machine already is. A package manager is the distribution. Those are what a
|
||||
machine *is*, not what must be put on it before the host works.
|
||||
|
||||
## It is built per operating system
|
||||
|
||||
**`systemd` and `pacman` are the Arch host's implementation, not abstractions the mesh grows.**
|
||||
They are not independent choices: a machine has pacman *because* it is Arch, and the package
|
||||
manager, service manager and packaging format arrive together as one decision somebody made at
|
||||
install time.
|
||||
|
||||
```
|
||||
mesh-host-arch pacman · systemctl · a container runtime
|
||||
mesh-host-alpine apk · rc-service
|
||||
mesh-host-android neither — a partial host
|
||||
```
|
||||
|
||||
**Abstracting them was rejected on correctness, not effort.** The service applier reads systemd's
|
||||
`LoadState` to tell *not installed* apart from *stopped* — which is what stops it reporting
|
||||
absence as success — and OpenRC has no equivalent. An interface spanning both must drop it, and
|
||||
the lowest common denominator is exactly where that fault lives.
|
||||
|
||||
**Almost all of it is shared.** The declaration vocabulary, the store, the apply loop, the
|
||||
read-back discipline, the refusal model and the link are portable. Two appliers differ.
|
||||
|
||||
**A host that cannot implement a shape refuses it.** Android has no package manager it may drive
|
||||
and no init it may register with, so it implements `file`, `directory` and `action` and refuses
|
||||
the rest — the same refusal an unknown type gets, with a different reason. Those three are the
|
||||
portable floor, and they are what makes a partial host a real thing rather than a broken one.
|
||||
|
||||
## It is a root service, and it never manages its own unit
|
||||
|
||||
**Root**, because no useful part of the job is unprivileged: it writes under `/etc`, installs
|
||||
packages, manages units and runs containers.
|
||||
|
||||
**It cannot run in a container**, and the reason is decisive rather than stylistic: installing the
|
||||
container runtime is a step of the bootstrap, so a host inside a container would need the thing
|
||||
it exists to install. Everything above tier 0 is a container; the host is not. That split is the
|
||||
tier boundary made concrete.
|
||||
|
||||
**The installation owns the host; the host owns everything else.** It manages `service` resources
|
||||
and its own unit is one — the temptation is obvious and it ends with a host stopping itself half
|
||||
way through an apply, leaving a machine with nothing running to fix it.
|
||||
|
||||
## An init is asked for one thing
|
||||
|
||||
**Start this at boot.** That is all, and every init can express it — systemd, OpenRC, runit, s6.
|
||||
|
||||
**Everything else is a launcher the host ships**, which supervises it: restart it when it exits,
|
||||
count consecutive failures, roll back after too many, halt after that. Policy in a unit file can
|
||||
only be read and hoped for; a script with a counter can be tested, and this is the one piece that
|
||||
must work on a machine where the host does not.
|
||||
|
||||
**The launcher does not exec the host, it supervises it** — so restarting is ours rather than the
|
||||
init's. The cost is signals: a supervisor that exits while its child runs leaves the host to be
|
||||
*killed* rather than to *stop*, and an apply interrupted that way is the half-configured machine
|
||||
this design is about. So it traps the shutdown signal, passes it down, and waits.
|
||||
|
||||
**A clean exit is the upgrade path**, and it is the easiest thing to get wrong — twice now. The
|
||||
host stands aside for a new binary by exiting zero, so anything supervising must restart on a
|
||||
zero exit and must not count it as a failure.
|
||||
|
||||
**Recovery is local, and detection is the mesh's.** Nothing dials a node and a host that cannot
|
||||
start cannot report, so the node must recover itself. But a local supervisor sees one process
|
||||
failing and cannot tell a broken machine from a broken release — only something watching every
|
||||
node can, which is why a host rollout is staged and stops when nodes go quiet.
|
||||
|
||||
## A host may be episodic
|
||||
|
||||
**Resident or episodic, and both are hosts.** A phone has no init to register with and nothing
|
||||
worth supervising, because a supervisor would be killed alongside what it supervises. So it runs
|
||||
when the platform allows and is killed when the platform wants the memory — **and that is
|
||||
disconnection**, which is already an ordinary situation.
|
||||
|
||||
It needs no keep-alive and no new mechanism: the store is already authoritative while
|
||||
disconnected, reconcile already happens on start, and *last heard from* is already reported
|
||||
rather than alarmed on. An episodic host cannot be the first node, because every bootstrap step
|
||||
is a shape it refuses.
|
||||
|
||||
## What a declaration is
|
||||
|
||||
**An ordered list of resources the host owns.** JSON, because Go's standard library carries a
|
||||
JSON parser and no YAML, and the one binary whose argument is that it needs nothing must not
|
||||
gain a parser to buy authoring comfort in a machine-written document.
|
||||
|
||||
**Ordered, because ordering is a decision.** The host does not sort and does not resolve
|
||||
dependencies — that would be deciding, and deciding the thing most likely to differ between what
|
||||
the control plane intended and what the machine does.
|
||||
|
||||
**Every resource has a stable identity** — a name the control plane keeps across declarations, not
|
||||
a position and not a hash of content. It is what lets the store say *this is the same resource I
|
||||
applied last time*, which is what makes removal possible at all.
|
||||
|
||||
**Unknown is refused, whole.** A field the host does not know is something the control plane
|
||||
believes it asked for. A declaration naming one is rejected entirely, naming every problem at
|
||||
once — a host that applied the parts it understood would leave a machine that looks configured
|
||||
and is not.
|
||||
|
||||
**Six shapes:** `file`, `directory`, `service`, `package`, `container`, `action`. Every addition
|
||||
widens what a compromised control plane can express, so the list is a security artefact and grows
|
||||
deliberately.
|
||||
|
||||
### The bundle may carry actions; the link may not
|
||||
|
||||
An `action` runs a command, and the host never learns what it means. It is needed because the
|
||||
bootstrap creates a database before there is any mesh to ask for one, and the host must not learn
|
||||
what a database is.
|
||||
|
||||
**Permitted from the bundle, refused from the link**, and the asymmetry is the whole point: a
|
||||
bundle arrives *with* the binary, so anyone able to put a hostile action there could have put it
|
||||
in the host itself — refusing it buys nothing and costs the bootstrap. The link is a separate
|
||||
party, reachable separately, and an action there is an unbounded blast radius.
|
||||
|
||||
**An action must carry its own verification**, which is also its idempotency check. The host does
|
||||
not know what a database is, so *is it already there* is a question only the declaration can ask.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **The migration is smaller than it looks.** A joining node never needs mesh-wide state — it
|
||||
needs an identity, an address and one peer, and the rest arrives as declarations.
|
||||
- **What is applied is recorded after it works, never before.** A failed apply leaves the machine
|
||||
in whatever state it reached, and nothing must claim otherwise.
|
||||
- **A second operating system is additive**: two appliers and a four-line init file.
|
||||
@@ -0,0 +1,279 @@
|
||||
---
|
||||
topic: the tiers
|
||||
status: accepted
|
||||
date: 2026-08-28
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
---
|
||||
|
||||
# 6. The substrate and the control plane
|
||||
|
||||
*Consolidated 2026-08-28 from six records. Extended 2026-08-29, by building it: the language, and
|
||||
what must be running before the control plane starts — which this record had left not established
|
||||
and could not have settled the way it was asking.*
|
||||
|
||||
## The control plane is what needs to know about more than one node
|
||||
|
||||
That is the whole test, and it follows from the host applying rather than deciding: **deciding
|
||||
needs knowledge a single machine does not have.**
|
||||
|
||||
| question | whose |
|
||||
|---|---|
|
||||
| write this file, with this content, with this mode | the **host** |
|
||||
| which nodes should run the store | the **control plane** |
|
||||
| is this unit running | the **host** |
|
||||
| which peers belong in this node's overlay | the **control plane** |
|
||||
| has this node been unreachable for a week | the **control plane** — nobody else is watching |
|
||||
|
||||
**Anything a single machine could answer alone is not the control plane's.**
|
||||
|
||||
### Seven contexts and one interface
|
||||
|
||||
**inventory, config, connectivity, provisioning, delivery, observability, identity** — plus
|
||||
`api`, the one interface every surface speaks to. Each earns its place by the test above rather
|
||||
than by being ours.
|
||||
|
||||
**`work`, `knowledge` and `stream` are mesh-hosted applications, not control plane.** A task does
|
||||
not need to know a node exists. *Being ours does not make something infrastructure.*
|
||||
|
||||
**`identity` owns SSH access.** *Written 2026-08-29, on noticing it was assumed everywhere and
|
||||
stated nowhere.* SSH appears three times across this design and every time as something that
|
||||
*uses* the overlay — "the way back in", "every node reaches every other: SSH, services, ordinary
|
||||
traffic" — while nothing said who hands out the keys. Nobody else could: the mesh is the only
|
||||
thing that knows which humans and agents exist and which nodes they may reach, which is
|
||||
`identity`'s definition. The node end already works, since an `authorized_keys` file is a file.
|
||||
|
||||
It is three questions wearing one name, and only two of them are the mesh's:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **humans** | their key, on the nodes they are allowed on |
|
||||
| **agents** | the same, with a lifetime — and revocation that has to actually work |
|
||||
| **an agent reaching another node** | **this is the point of the mesh, not an exception to it** — see below |
|
||||
|
||||
**There is no such thing as node-to-node SSH here, and that is a clarification rather than a
|
||||
restriction.** The actor is always an **agent**; a node is only where it happens to be running —
|
||||
[ADR 0001](0001-mesh-brokers-nodes-host-agents-think.md): *a node is a place where an agent can
|
||||
run, that is the entire relationship.* An agent hired onto one node reaching another to do work is
|
||||
the capability the whole arrangement exists to provide.
|
||||
|
||||
**The credential is the agent's, never the node's.** It lives in the agent's own credential
|
||||
directory ([ADR 0001](0001-mesh-brokers-nodes-host-agents-think.md)), so a node's
|
||||
`authorized_keys` lists **agents** and never nodes. Three things follow, and they are why this
|
||||
shape is better rather than merely allowed:
|
||||
|
||||
- [ADR 0004](0004-a-node-and-how-it-joins.md)'s *a node holds its own identity and nothing else*
|
||||
stays true — no node holds a key that reaches another node;
|
||||
- a compromised node costs the credentials of the agents that were on it, not a way into
|
||||
everything;
|
||||
- **who may reach what is a mesh-wide fact**, which is exactly why it is `identity`'s and not
|
||||
something arranged locally.
|
||||
|
||||
**And it does not conflict with the host having no inbound control surface.** That rule is about
|
||||
how a node's *declared state* changes: over the broker, never by being dialled. An agent with a
|
||||
shell is not the mesh reconfiguring a machine — it is what a person with a terminal has always
|
||||
been, and this design already depends on it working
|
||||
([ADR 0007](0007-connectivity.md): the overlay is *the way back in*). What such a session can leave
|
||||
behind is drift, and drift is what reconciliation is for.
|
||||
|
||||
**Where the record lives is deliberately open.** Contexts integrate through it, which makes it
|
||||
load-bearing, and putting it in the substrate risks recreating the circularity the tiers just
|
||||
removed. Listing it as an eighth context would settle by naming what has not been settled by
|
||||
arguing.
|
||||
|
||||
### One node runs it, and nothing takes over
|
||||
|
||||
**Declared, never elected.** No promotion, no quorum, no fencing, no split brain — none of it
|
||||
built, so none of it can be subtly wrong.
|
||||
|
||||
#### The option that would make it a real mesh, and why not
|
||||
|
||||
*Written 2026-08-29. It had been rejected by never being written down, which is the weakest way
|
||||
to reject anything.*
|
||||
|
||||
A genuine peer-to-peer mesh means **no node is special**, and that has a concrete price:
|
||||
|
||||
- every node holds the **whole inventory**, so there is a replication process between them;
|
||||
- replication needs a writer, so one node is elected **master**, and something promotes a new one
|
||||
when it drops — Redis Sentinel and its whole family of problems;
|
||||
- and it still would not deliver what the name promises, because **application databases are not
|
||||
replicated.** A workload's store lives where the workload lives.
|
||||
|
||||
That last point is the one that settles it. To make the mesh genuinely peer-to-peer we would have
|
||||
to become **a replicated database system for everything running on it** — not for our own
|
||||
inventory, for every consumer's data too. That is a product, and a much larger one than the thing
|
||||
it would be supporting.
|
||||
|
||||
**So there are three central roles, not one**, and it is worth seeing them separately because
|
||||
only the third costs operation:
|
||||
|
||||
| | its loss costs |
|
||||
|---|---|
|
||||
| **the control plane** | nothing can be *changed*. Nothing stops running |
|
||||
| **the broker** | nothing can be told anything, or report anything |
|
||||
| **the hub** | nodes in different places **cannot reach each other** ([ADR 0007](0007-connectivity.md)) |
|
||||
|
||||
**Whether these are one node is not decided here.** All three must be dialable by every node, which
|
||||
pushes toward one; nothing says they must be.
|
||||
|
||||
**That is sound rather than merely cheap**, because the design already tolerates its absence by
|
||||
construction: a node reconciles from its own store and never needed to ask anybody to hold the
|
||||
state it was last given. **The control plane being down is not a new failure mode — it is every
|
||||
node in the ordinary disconnected situation at once.** What is lost is *change*, not *operation*.
|
||||
|
||||
The honest half: this node is a single point of failure, recovery is **restore rather than
|
||||
failover** — which makes backup the availability mechanism rather than hygiene — and
|
||||
**certificate renewal is the clock.** An outage outlasting a renewal window expires every public
|
||||
name, which turns an inconvenience into an outage on a timer. Nothing measures that today.
|
||||
|
||||
## The authority is the control plane, not a database
|
||||
|
||||
**There is no single mesh database.** Each context owns its store exclusively, and *the mesh
|
||||
database* names a thing that will not exist.
|
||||
|
||||
**No node reads any of them** — not for writes, not for reads. A node is *told* what to own, over
|
||||
the link, in a bounded vocabulary; it **states** what it applied, and the owning context writes.
|
||||
The difference is the security boundary: something that can write cannot be prevented from
|
||||
writing anything.
|
||||
|
||||
**A node runs from its own store always, not as a fallback.** The current arrangement's nastiest
|
||||
property is that *a node running from cache looks identical to a node running from the database*,
|
||||
with no age on the cache and nothing reporting divergence. Under this there is no second mode to
|
||||
be mistaken for the first.
|
||||
|
||||
**What survives from the original decision:** the repository defines what exists, the mesh defines
|
||||
what runs where, and no node-to-module mapping is ever committed. That is what makes the
|
||||
repositories node-agnostic and why anything about the mesh can be published at all.
|
||||
|
||||
**The error underneath was a category error**: *source of truth* named a storage location when it
|
||||
meant an **authority**. Once the store is the answer, *which database* becomes the question, and
|
||||
shared schemas follow.
|
||||
|
||||
## The substrate is what the control plane consumes and cannot grant itself
|
||||
|
||||
Every module needing a database asks provisioning for one. The control plane needs a database too
|
||||
and cannot ask itself, because it is not running yet. **That circularity is the definition**, and
|
||||
anything on the wrong side of it is raised from the bundle the host carries.
|
||||
|
||||
| role | product | |
|
||||
|---|---|---|
|
||||
| relational store | **PostgreSQL** | its own state lives there |
|
||||
| message bus | **LavinMQ** | it cannot grant itself a virtual host — and **precedes it**, below |
|
||||
| object store | **MinIO** | it cannot grant itself a bucket |
|
||||
| image registry | **an OCI registry** | it cannot grant itself a repository |
|
||||
| identity provider | — | **conditional**: substrate only if the control plane delegates authentication, which is undecided |
|
||||
|
||||
**The role and the product are both written.** The role is what the argument turns on; the product
|
||||
is what gets installed and pinned, and a design that names only the role does not record that the
|
||||
choice was made. **The dependency is on the protocol** — AMQP, S3, OCI — which is what keeps
|
||||
naming them safe. The store is the exception: the provisioning model uses databases, roles and
|
||||
schemas as PostgreSQL means them.
|
||||
|
||||
**A container runtime is detected, not chosen** — docker or podman, because a machine that
|
||||
already has one keeps it. Only the version probe differs between them; the behavioural difference
|
||||
(podman has no daemon, so containers do not return after a reboot unless a unit is enabled)
|
||||
belongs in the declaration rather than the host.
|
||||
|
||||
**Being substrate and being in the bundle are different questions.** PostgreSQL and LavinMQ must
|
||||
precede the control plane; the object store and the registry are substrate by role and ordinary by
|
||||
delivery, provisioned once there is a control plane to do it.
|
||||
|
||||
### Why the broker precedes it too
|
||||
|
||||
Written after the fact, because this record first left it *not established* and framed it as
|
||||
turning on whether the control plane's own contexts talk to each other over the bus.
|
||||
|
||||
**They do not** — they are one process and dispatch internally. Under that framing the broker is
|
||||
provisioned like anything else and the bundle stays at one image.
|
||||
|
||||
**The framing cannot answer the question.** What decides it is not how the contexts reach each
|
||||
other. It is how the control plane reaches a *node* — and that is settled above: never except over
|
||||
the link, and the link is AMQP ([ADR 0002](0002-nodes-communicate-over-a-broker.md),
|
||||
[ADR 0004](0004-a-node-and-how-it-joins.md)). So:
|
||||
|
||||
```
|
||||
the bundle raises the control plane
|
||||
the control plane provisions the broker ← by telling a host to run it
|
||||
telling a host happens over the link
|
||||
the link is the broker
|
||||
```
|
||||
|
||||
**And it is not avoided by the first node being local.** `enrol` dials the broker at the address in
|
||||
its token, which is the first node's own third step. A machine that raised the mesh still joins it
|
||||
the ordinary way, and that was deliberate — its specialness lasts two commands. Making it join by
|
||||
some other route would buy a smaller bundle by giving up the property the design was built to have.
|
||||
|
||||
The broker precedes the control plane for the same reason PostgreSQL does: **the control plane
|
||||
cannot grant itself the thing it would need in order to grant it.**
|
||||
|
||||
**What it costs.** Two images rather than one, against the wish above to keep the bundle at roughly
|
||||
one so a person can read it — two is still readable, four would not be. And three things that are
|
||||
not images, each an action the bundle declares and the host runs, the way the database already is:
|
||||
a virtual host, a credential on it, and **a certificate**. That last is the awkward one: a token
|
||||
pins the fingerprint a host must expect *before it sends anything*, so the broker needs a
|
||||
certificate at a moment when there is no mesh to issue one and no public name to obtain one for.
|
||||
Self-signed and pinned is the shape that fits; how it is later replaced by the certificates in
|
||||
[ADR 0007](0007-connectivity.md) is not decided here.
|
||||
|
||||
## The control plane is written in Go
|
||||
|
||||
The same language as the host, so tiers 0 and 2 are one language and not two.
|
||||
|
||||
The reason that decides it is not familiarity. **Its image is pinned by digest in the bundle**,
|
||||
which means it is fetched and run on a machine where no mesh exists yet — nothing to check it
|
||||
against, nothing watching, and a person expected to have read the bundle and believed it. A
|
||||
statically linked binary makes that image the program and nothing else: no interpreter, no package
|
||||
tree, no transitive dependency that arrived because something needed a date library. Everything
|
||||
under that line is something somebody would have to audit, on the one image the whole mesh is
|
||||
raised from.
|
||||
|
||||
A second reason, smaller and still real: the control plane runs a reconcile loop of its own
|
||||
([ADR 0010](0010-delivery.md) — artifacts against source, as the host reconciles machine state
|
||||
against declarations). Two loops of the same shape are cheaper to hold in one head when they are
|
||||
also the same language.
|
||||
|
||||
**The option rejected** is TypeScript, matching the lab and the surfaces that will speak to this.
|
||||
The argument for it is that tier 3 is web and CLI, so a TypeScript control plane would share types
|
||||
with its callers rather than generating a contract. True, and it does not reach far enough:
|
||||
`mesh-sdk` is *contracts shared across tiers* and **tier 0 is Go**, so the contracts cross a
|
||||
language boundary whatever tier 2 is written in. The choice is between generating them for one
|
||||
consumer or for two.
|
||||
|
||||
**What it costs, plainly:** the control plane can import nothing that exists today, and a person
|
||||
moving between tier 2 and tier 3 changes language. Neither is recovered later — the language is the
|
||||
most expensive thing in this record to reverse.
|
||||
|
||||
## The installer fetches what it pins
|
||||
|
||||
`substrate.lock` carries **references, not payload** — an image name and a **digest**, fetched at
|
||||
apply time. A tag moves; a digest does not, and reproducibility comes from pinning the identity of
|
||||
a thing rather than carrying its bytes.
|
||||
|
||||
The assumption that a machine might have no network came from the lab and was wrong: a machine
|
||||
being adopted has one, and the sealed case is the lab.
|
||||
|
||||
**The lab places images by raising a registry inside the scenario**, which is what a real node
|
||||
pulls from anyway — so it tests the real path rather than a stand-in for it. The digests that
|
||||
registry serves are its own, and that satisfies this rule: what is required is a reference that
|
||||
is **exact and cannot move**, and one it assigned is both. Assuming an upstream digest had to be
|
||||
preserved is what made this look impossible for a while
|
||||
([04-ISSUES/009](../04-ISSUES/009-a-digest-pinned-image-cannot-be-placed-in-the-lab/00-report.md)).
|
||||
|
||||
**Its contents are per operating system** even though its mechanism is not — package names, unit
|
||||
names and service names all differ, so an Arch host embeds an Arch bundle.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **The bundle stays small and reviewable.** A list of pinned references is something a person can
|
||||
read; a bundle containing images is not.
|
||||
- **An apply can fail because something is unreachable**, which a self-contained artifact could
|
||||
not. That must fail *legibly*, naming what could not be fetched and from where.
|
||||
- **Cross-context reporting is harder, and that is the point.** Anything wanting to see across
|
||||
contexts consumes their events or calls their interfaces.
|
||||
- **A queue with no limit grows until the broker's disk is full**, and the broker is what every
|
||||
node depends on. The bound is per queue and is not decided.
|
||||
- **The bundle carries two images and four actions**, and the substrate bootstrap grows a step.
|
||||
- **Nothing in the first node's path is special-cased.** Enrolment is walked on node one.
|
||||
- **The broker's certificate at bootstrap has no answer yet**, and is named as unfinished rather
|
||||
than assumed. It is the first thing that will be wanted when the link is built.
|
||||
- **The language cannot be revisited cheaply.** It is the one line here close to irreversible.
|
||||
@@ -0,0 +1,191 @@
|
||||
---
|
||||
topic: the tiers
|
||||
status: accepted
|
||||
date: 2026-08-28
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
---
|
||||
|
||||
# 7. Connectivity
|
||||
|
||||
*Consolidated 2026-08-28 from three records. Overlay, resolution, exposure, filtering and
|
||||
certificates are one design.*
|
||||
|
||||
## Why it is control-plane work
|
||||
|
||||
Apply the test — *everything that needs to know about more than one node* — and not one of the
|
||||
five can be answered by a machine on its own:
|
||||
|
||||
| | needs to know |
|
||||
|---|---|
|
||||
| **overlay** — who peers with whom | every node, and which can be dialled |
|
||||
| **resolution** — which name is which node | every node |
|
||||
| **exposure** — which public name reaches which container | which node is publicly reachable |
|
||||
| **filtering** — which port is open, to whom | what is assigned here, and the overlay's shape |
|
||||
| **certificates** — who may present which name | which name belongs to which node |
|
||||
|
||||
That is exactly what the current arrangement gets wrong, by computing all five on the node from a
|
||||
direct database connection. Two modules do this, and they are the only two left holding a
|
||||
credential to the control plane's database.
|
||||
|
||||
**The shape of the fix, once for all five:** the connectivity context computes the configuration;
|
||||
it arrives over the link as `file` resources; the service reads files and knows nothing about the
|
||||
mesh. **This costs no new host vocabulary.**
|
||||
|
||||
## A route is a grant
|
||||
|
||||
**Ingress is not substrate.** The control plane does not need a route to start — it listens
|
||||
locally — and no node needs one to reach it, because the node dials out and has no listening
|
||||
control surface. It grants itself a route afterwards, the way it grants itself a bucket.
|
||||
|
||||
The strongest objection deserves stating: the `api` is the one interface every surface speaks to,
|
||||
so eventually it *does* want a public name. But **wanting one later is not needing one to
|
||||
start**, and that distinction is the entire substrate test.
|
||||
|
||||
**A module that must be reachable declares it needs a route; the proxy provides one.** Ordinary
|
||||
instantiation, with the direction mirrored — the consumer supplies a target and receives a name.
|
||||
|
||||
**Exposure is three facts at two scopes**, which is why it cannot live on the node:
|
||||
|
||||
| the fact | scope |
|
||||
|---|---|
|
||||
| the public name resolves to an address | **mesh** — which node is publicly reachable |
|
||||
| a certificate valid for that name exists | **mesh** — issued once, used on one node |
|
||||
| the proxy maps that name to that container | **node** |
|
||||
|
||||
**A node without a public address is proxied by one that has**, across the overlay. Most nodes sit
|
||||
behind a connection with no forwarded port, so exposure cannot assume the workload's node is
|
||||
reachable.
|
||||
|
||||
## Reachability is declared, not inferred
|
||||
|
||||
The overlay's peer graph is computed from whether a node can be dialled, and that was inferred
|
||||
from a regular expression over the address. **The address is evidence of reachability; it is not
|
||||
the fact**, and the gap has already cost:
|
||||
|
||||
| address | the regex says | actually |
|
||||
|---|---|---|
|
||||
| `100.64.0.0/10` — carrier-grade NAT | **public** | **not reachable.** An endpoint is written to an address nothing can reach |
|
||||
| any IPv6 address | public | the test is v4 shapes only |
|
||||
| a routable address behind a closed firewall | public | not reachable |
|
||||
| a documentation range standing in for a public segment | private | reachable — this is the lab bug |
|
||||
|
||||
**A test environment having to choose its addresses to satisfy a regex is the regex telling us it
|
||||
is not a fact.**
|
||||
|
||||
So: **an endpoint, or none** — declared. And **the hub is declared, never derived from an address
|
||||
prefix**, because an election decided by the first four characters of an address fails silently,
|
||||
cannot be queried, and makes a renumbering an outage.
|
||||
|
||||
**The address remains evidence and stops being the fact.** Where an observed endpoint disagrees
|
||||
with a declared one, the disagreement is a **reportable condition**, not a silent correction.
|
||||
|
||||
**What does not change** is the lesson underneath: role does not imply reachability — a
|
||||
home-hosted node is a server that cannot be dialled. This keeps that and stops encoding it as a
|
||||
pattern match.
|
||||
|
||||
### Some nodes must be reachable, and this had not been said
|
||||
|
||||
*Written 2026-08-29, on being asked and finding no answer.*
|
||||
|
||||
Everything above treats reachability as a **fact to record** — which node can be dialled, so that
|
||||
exposure and certificate issuance can be placed. It never said the converse, and the converse is a
|
||||
hard requirement:
|
||||
|
||||
| role | dialled by | so it needs |
|
||||
|---|---|---|
|
||||
| **the node running the broker** | every node, outbound ([ADR 0002](0002-nodes-communicate-over-a-broker.md)) | to be reachable from wherever nodes are, at a **stable address** |
|
||||
| **the hub** | every node not co-located with its peer | the same |
|
||||
|
||||
**Across the internet, "reachable from wherever nodes are" means publicly reachable.** For a mesh
|
||||
confined to one network it does not — the requirement is about the nodes that exist, not about the
|
||||
public internet.
|
||||
|
||||
**Stable is the sharper half.** A token carries the broker's *address, not a name*, because there
|
||||
is no resolution before joining ([ADR 0004](0004-a-node-and-how-it-joins.md)). A broker node whose
|
||||
address moves invalidates every token issued for it, and a node that was disconnected across the
|
||||
change cannot get back.
|
||||
|
||||
**A mesh whose nodes are all behind NAT cannot be raised.** That is a real precondition and it
|
||||
belongs with the others rather than being discovered.
|
||||
|
||||
### The link stays on the underlay, and that is a repair channel
|
||||
|
||||
The obvious objection is that all traffic should run over the overlay. Nearly all of it does — SSH,
|
||||
services, node to node — and the exception is each node's own outbound link to the broker.
|
||||
|
||||
**At join time it is forced**: a node has no overlay yet, so it cannot use one to ask for one.
|
||||
**Afterwards it is a choice**, and the reason is that the link is how a broken node is fixed. A
|
||||
repair channel carried over the thing being repaired is not a repair channel: a node whose only
|
||||
path home was the overlay is gone the moment an overlay declaration is wrong.
|
||||
|
||||
**What it does not cost is confidentiality.** The link is already authenticated and encrypted
|
||||
against a pinned fingerprint ([ADR 0004](0004-a-node-and-how-it-joins.md)), so moving it onto the
|
||||
overlay would not protect traffic that is unprotected today.
|
||||
|
||||
## A filter rule names its source
|
||||
|
||||
`scope: public` is declared in five manifests, is part of no rule type, and is **referenced by no
|
||||
code**. So five manifests appear to restrict a port and restrict nothing — on the modules most
|
||||
worth restricting.
|
||||
|
||||
**A rule names its source. `from:` is the only way to scope one, and a rule without one is open**
|
||||
— which it must say plainly rather than appear to deny.
|
||||
|
||||
**`scope:` is removed rather than implemented**, because giving it meaning would leave two ways to
|
||||
express one thing. And the general fix is that **an unknown key is refused**: the host's
|
||||
declaration parser already works this way, and manifests are the layer where that discipline is
|
||||
missing. `scope:` survived because nothing rejected it, and it spread by copying to five
|
||||
manifests.
|
||||
|
||||
## Order, and what it costs
|
||||
|
||||
**The link runs on the underlay and never on the overlay.** The overlay is configured by the mesh,
|
||||
so a link requiring it could never be established on a new node.
|
||||
|
||||
**The first declaration is the overlay and nothing else** — because a node's address and peers are
|
||||
*assigned* so it cannot come earlier, and because it is the way back in. A node reachable over the
|
||||
overlay can be fixed by hand if a later declaration breaks the machine; **a large first
|
||||
declaration risks a node that is broken and unreachable at once.**
|
||||
|
||||
**Reachable is not the same as having a control surface.** Every node reaches every other over the
|
||||
overlay — SSH, services, ordinary traffic — and every node consumes from the broker. What is
|
||||
forbidden is a listening thing that accepts instructions and changes the machine.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **The last two direct database connections leave the nodes**, and with them the database
|
||||
credential every node carries.
|
||||
- **The `/etc/hosts` floor goes**, along with the bootstrap circularity it patched.
|
||||
- **Two certificate authorities stay separate on purpose**: a public one for public names, the
|
||||
mesh's own for internal ones. A single-CA lab would hide any bug living in the split.
|
||||
- **What happens when the hub is down**: nothing takes over. Non-co-located paths stop; co-located
|
||||
peers and every assigned workload keep running.
|
||||
|
||||
## Open — the link over the overlay, with a fallback
|
||||
|
||||
*Raised 2026-08-29 and deliberately left open, because the honest gain is smaller than it looks
|
||||
and it is a decision rather than a derivation.*
|
||||
|
||||
The proposal: a node prefers the overlay for its link and drops to the underlay when the overlay
|
||||
is not working — so ordinary operation is private and the underlay stays as the way back.
|
||||
|
||||
**Two things it would have to get right:**
|
||||
|
||||
- **The trigger cannot be "is the overlay up".** A WireGuard interface has no link state; once
|
||||
configured it is up whether or not the far end exists. So there is no flag to read, and failing
|
||||
over means *try, fail, time out, retry elsewhere*.
|
||||
- **Running on the fallback has to be visible.** A node that quietly drops to the underlay is a
|
||||
node whose overlay is broken with nothing to say so, and it will stay broken because everything
|
||||
still works. That is this repository's recurring fault — a failure that reads as success — and a
|
||||
fallback is the easiest place in the design to reintroduce it.
|
||||
|
||||
**What stops it being an obvious win:** if the underlay path must stay available for the fallback,
|
||||
the broker stays exposed on it. So the exposure is unchanged and the traffic was already encrypted
|
||||
— the gain is which network carries bytes, not what an attacker can reach.
|
||||
|
||||
**The version that would buy something is overlay-only**, with the broker firewalled to the overlay
|
||||
in steady state, accepting that a node whose overlay breaks needs hands-on recovery. That is a real
|
||||
trade: it exchanges the automatic way back in for a closed port.
|
||||
|
||||
Not decided either way here.
|
||||
@@ -0,0 +1,114 @@
|
||||
---
|
||||
topic: the tiers
|
||||
status: accepted
|
||||
date: 2026-08-26
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0009-modules-and-the-graph.md
|
||||
---
|
||||
|
||||
# 8. A context owns its store, exclusively
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0009](0009-modules-and-the-graph.md) settles what a module
|
||||
declares. This settles what a grant may be, and it is the half that **removes** things.
|
||||
|
||||
`how-we-build` §4 already says *contexts integrate through the record, never through a shared
|
||||
schema*, and states the cost: several domains share one forty-five-table schema, which is why
|
||||
work belonging to one context keeps having to be implemented in another.
|
||||
|
||||
That was written as a principle. Counted, it is thirteen foreign tables belonging to three
|
||||
separate contexts, living in the mesh's own registry database.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **A schema per consumer inside a shared database.** Namespaced, revocable by dropping the
|
||||
schema, with a cross-context join possible but deliberate. Rejected: it keeps the letter of
|
||||
§4 and leaves the temptation in place, and a boundary that is merely inconvenient to cross
|
||||
gets crossed.
|
||||
2. **Read-only roles on another context's store.** Rejected for the same reason and one worse:
|
||||
reading another context's tables couples you to its layout exactly as firmly as writing them,
|
||||
and the coupling is invisible until the owner changes a column.
|
||||
3. **Exclusive ownership.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
> **A context is granted only what it exclusively owns.**
|
||||
|
||||
No shared writes. No read-only role on another context's store. If you need what another context
|
||||
holds, you ask it or you subscribe to it.
|
||||
|
||||
**The unit is the context, not the process.** Everything inside a context — its service, its
|
||||
surface, its tools — reads its own store freely. A board showing the mesh's own nodes and
|
||||
modules is the mesh showing its own data, not a boundary crossing. What is forbidden is a
|
||||
*different* context reading it.
|
||||
|
||||
### Asking or subscribing is derived, not chosen
|
||||
|
||||
[ADR 0004](0004-a-node-and-how-it-joins.md) makes disconnection an ordinary situation. So:
|
||||
|
||||
- **Anything that must keep working while disconnected cannot ask** — there is nobody to ask. It
|
||||
keeps a local copy, which means subscribing.
|
||||
- **Anything where a stale answer is worse than none cannot subscribe.** A display may lag; a
|
||||
decision about whether a grant is still valid may not.
|
||||
|
||||
Neither is a query against another store, whatever transport it travels over.
|
||||
|
||||
### What that is, concretely
|
||||
|
||||
*Written 2026-08-29, on building the first one — the rule above was clear and what to type was not.*
|
||||
|
||||
**One PostgreSQL database per context, named for the context.** A separate database rather than a
|
||||
separate schema is the whole point: a cross-schema join is a qualified name away, and a
|
||||
cross-database join needs a foreign data wrapper somebody has to install and explain.
|
||||
|
||||
**And one credential per context, held only by it.** There is no mesh-wide connection setting and
|
||||
no way to ask for one, so reaching another context's store is not a matter of restraint — a process
|
||||
has no address for it and nothing to present. That is also how this rule is *checked*: what a
|
||||
context can reach is the list of variables the declaration running it grants, and it is read there
|
||||
rather than audited in code.
|
||||
|
||||
**Contexts that do not exist yet do not get a database.** The bootstrap creates the ones there are.
|
||||
|
||||
**How a context added later gets its database is open**, and it is a real question: by then there is
|
||||
a control plane, but a control plane holding a credential that can create databases is holding
|
||||
rather more than the thing it exclusively owns.
|
||||
|
||||
## What this removes
|
||||
|
||||
The first clear list of what the design deletes rather than adds:
|
||||
|
||||
- **Grant kinds.** There is one: an exclusive resource. No schema grants, no read roles, no
|
||||
rules about who may see what inside a shared thing.
|
||||
- **The question of who owns which table**, and the guessing at revocation time. Removing a
|
||||
consumer drops what it was granted, whole.
|
||||
- **Cross-context migration ordering.** Two contexts migrating one database must be ordered
|
||||
against each other. Exclusive ownership means a context's migrations are ordered only against
|
||||
itself.
|
||||
- **A class of permission modelling** a shared store would otherwise need.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **Cross-context reporting is harder, and that is the point.** Anything wanting to see across
|
||||
contexts consumes their events or calls their interfaces. That is §4's argument, and the cost
|
||||
it names is the one already paid.
|
||||
- **A single surface over several contexts still works** — that is what a surface is. It reads
|
||||
interfaces, not stores. This holds while the contexts sit behind **one** interface; splitting
|
||||
a context into its own deployable costs that, and the composition would have nowhere to live
|
||||
that tier 3 permits. **A real constraint on how far the control plane may be split.**
|
||||
- **Three contexts must move out of the registry database**, taking thirteen tables with them.
|
||||
Their dependency on the registry then shrinks to almost nothing — one of them needs a single
|
||||
table.
|
||||
- **The node appliers were already handled.** [ADR 0005](0005-the-node-host.md)
|
||||
stopped the host querying the mesh database for tier reasons unrelated to this, and it removes
|
||||
most of the remaining direct readers as a side effect.
|
||||
- **What a consumer does about events missed while disconnected is not decided** — replay from a
|
||||
point, ask once and resume, or rebuild. The question every projection has.
|
||||
|
||||
## References
|
||||
|
||||
- [`how-we-build.md`](../00-META/how-we-build.md) §4 — the rule this makes enforceable.
|
||||
- [Research 011](../01-RESEARCH/011-the-module-graph/worked-provider.md) — the count, the worked
|
||||
provider, and the dashboard case.
|
||||
- [ADR 0004](0004-a-node-and-how-it-joins.md) — why asking or subscribing is derived.
|
||||
@@ -0,0 +1,441 @@
|
||||
---
|
||||
topic: what runs on it
|
||||
status: accepted
|
||||
date: 2026-08-28
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
---
|
||||
|
||||
# 9. Modules and the graph
|
||||
|
||||
*Consolidated 2026-08-28 from six records.*
|
||||
|
||||
## Everything is a module
|
||||
|
||||
One kind of thing, one manifest describing all of them. A database, a web application, a window
|
||||
manager and a firewall rule set are all modules — not because they are alike, but because
|
||||
**anything else means a second kind of thing with its own rules, and then a third.**
|
||||
|
||||
**A module is the unit of delivery**: assignable to a node, versionable, replaceable on its own.
|
||||
|
||||
## There are no domain modules
|
||||
|
||||
An earlier decision grouped modules by domain — four things constituting *how a node is
|
||||
reachable* becoming one `networking` module. **That was wrong, and the correction is worth
|
||||
keeping** because the observation behind it was right.
|
||||
|
||||
The measurement holds: reachability is the **only** place in the catalogue where modules
|
||||
genuinely change together under one intent. What did not hold is the conclusion. Tight coupling
|
||||
means they share an **authority** — one place that decides for all of them — and not that they
|
||||
should be one artifact. `wireguard` and the proxy are deployed to different sets of nodes, so a
|
||||
module containing both would be assigned where half of it is unwanted.
|
||||
|
||||
> **Coherence is a context. Delivery is a module.**
|
||||
|
||||
**Folders assert relationships; edges record them.** What grouping was for — finding things,
|
||||
seeing what belongs together — is a tag and a query, neither of which anybody has to keep true by
|
||||
hand.
|
||||
|
||||
### What a domain module turns out to be, and why it is not the one refused above
|
||||
|
||||
*Written 2026-08-29, from building it. The heading above reads as a contradiction of what now
|
||||
exists and is not one — but only if the difference is stated, so it is stated here.*
|
||||
|
||||
**What was refused contains things. What exists contains nothing.**
|
||||
|
||||
| | `networking` as refused | `networking` as built |
|
||||
|---|---|---|
|
||||
| what is in it | WireGuard, a proxy, a firewall — artifacts | nothing at all |
|
||||
| what it says | *these ship together* | *I want a private network and names* |
|
||||
| what is assigned | one module, half of it unwanted | whatever answers each requirement, each on its own |
|
||||
|
||||
The objection above is untouched by this and still correct: a module holding WireGuard and a
|
||||
proxy is assigned where half of it is unwanted. **A module holding nothing cannot be, because
|
||||
there is no half.** It is requirements and a name, and every artifact it leads to is still an
|
||||
ordinary module assigned on its own terms.
|
||||
|
||||
**Why it is worth having.** Most people want the network working and do not want to choose a VPN.
|
||||
`assign networking` finds one answer to each requirement and takes it without asking, because
|
||||
with one candidate there was never a question — the rule below about refusing does the work.
|
||||
Somebody who does care assigns the VPN they want, and *that is the whole of choosing*: there is no
|
||||
flavor field, no variant syntax, and no second verb. **Picking an implementation is assigning a
|
||||
module.**
|
||||
|
||||
**What it costs, stated because it is real.** Adding a second implementation to the catalogue
|
||||
turns a settled question into an open one for **everyone using the bundle**, not only for whoever
|
||||
wanted the alternative. Every node assigned `networking` refuses until somebody says which. That
|
||||
is [the refusing rule](#a-requirement-with-several-answers-is-refused-never-guessed) applied
|
||||
consistently, and the alternative is a default — which is the flavor field returning under a
|
||||
better name. The cost is one assignment per node, and the message names the candidates.
|
||||
|
||||
**A consequence that had to be found by running it.** A bundle can drag an implementation in
|
||||
through a requirement nobody looked at. Choosing a different VPN still installed WireGuard,
|
||||
because the names module needed addresses only WireGuard hands out, and nobody was told. Two VPNs
|
||||
on one machine is not always wrong — a machine may run one for another purpose — but being **the**
|
||||
network the mesh runs over is singular, so that is a claim, and the collision is refused by name.
|
||||
**The general rule: what a bundle pulls in is only as safe as the claims on what it pulls in
|
||||
from.**
|
||||
|
||||
## Three edges
|
||||
|
||||
| edge | means | declared? | satisfied |
|
||||
|---|---|---|---|
|
||||
| **presence** | that thing must exist and be reachable here | yes | at provisioning |
|
||||
| **instantiation** | that thing makes something for me and hands back credentials — a database, a bucket, a route | yes | at provisioning, and again whenever it must be |
|
||||
| **build** | I was compiled against that artifact | **no — read from imports** | **at build, once** |
|
||||
|
||||
**Instantiation implies presence; presence does not imply instantiation.**
|
||||
|
||||
**A route is an instantiation edge**, and it is worth noticing because the direction is the mirror
|
||||
of a database: the consumer supplies a target and receives a *name*, rather than supplying nothing
|
||||
and receiving credentials. Same edge.
|
||||
|
||||
**Provider stops being a category.** Any hosted thing can be a factory — an identity provider
|
||||
grants clients, a mail server grants mailboxes. It is a facet, not a kind.
|
||||
|
||||
**A module may also declare what it claims**, because some things cannot coexist and that is a
|
||||
fact about the module rather than about a particular node. What that means precisely is below.
|
||||
|
||||
### Where the answer to a requirement is allowed to live
|
||||
|
||||
*Written 2026-08-29, from building it. The table above distinguishes **presence** from
|
||||
**instantiation** and this is the half of that distinction nobody had noticed was missing: not
|
||||
what the edge hands over, but **where the thing on the other end is.***
|
||||
|
||||
Two different things were both being written as a requirement:
|
||||
|
||||
| | *a shell*, *a display server*, *a private network* | *a database*, *an object store*, *an identity provider* |
|
||||
|---|---|---|
|
||||
| where the answer lives | **this machine** | **somewhere in the mesh** |
|
||||
| how it is answered | install another module here | find the node already running it |
|
||||
| what is missing if absent | a module to assign here | **a decision about where**, which is nobody's to make silently |
|
||||
|
||||
Answering the second like the first installs a database on every machine that uses one, which is
|
||||
what it did.
|
||||
|
||||
**So a provided name carries a scope**, the same idea a claim already has, and written short in
|
||||
the ordinary case so the few that are not node-scoped stand out rather than drowning. Scope is a
|
||||
property of **the name, not of each provider**: two modules disagreeing about whether a database
|
||||
is local would make one requirement mean two things depending on which happened to answer it, so
|
||||
that is refused.
|
||||
|
||||
**A requirement answered from the mesh is never satisfied by installing it here.** Nothing, and
|
||||
the mesh refuses and says which module to assign somewhere. Two, and it refuses and says how to
|
||||
choose — the same rule as everywhere else, for the same reason: picking is guessing, and the wrong
|
||||
guess puts somebody's data on a machine they did not choose.
|
||||
|
||||
**Choosing is recorded per node**, because that is the granularity the choice actually has — two
|
||||
machines may reasonably use two different databases and a mesh-wide answer could not say so. A
|
||||
choice pointing at a machine that does not provide the thing is refused rather than quietly
|
||||
replaced by one that does, and a single available provider does not override a choice either.
|
||||
**Both are the same rule: the mesh does not overrule a person, and it does not move data without
|
||||
being told to.**
|
||||
|
||||
**What this is a prerequisite for.** Knowing *which node* answers is the first half of handing a
|
||||
credential back — you cannot be given a database's password before it is settled whose database it
|
||||
is. So a node's resolution now records what it takes from elsewhere, which is both the only part
|
||||
of its set that stops working when a *different* machine goes away, and the place a credential
|
||||
will hang.
|
||||
|
||||
### An edge has two directions, and only one of them is built
|
||||
|
||||
*Written 2026-08-29, from building it. The row above already says a consumer **supplies a target
|
||||
and receives a name**; what it did not say is that those are two separate mechanisms, and that
|
||||
having one without the other is what forced two modules outside the system entirely.*
|
||||
|
||||
| direction | the consumer says | who needs it |
|
||||
|---|---|---|
|
||||
| **contribution** | *publish me at this name, on this port* | the proxy, the DNS server, a firewall |
|
||||
| **binding** | *and give me back a credential to it* | the database, the object store, the identity provider |
|
||||
|
||||
**Contribution is built.** A module declares what it contributes to a requirement; the control
|
||||
plane collects every contribution on a node and writes them to a path the provider named, as a
|
||||
file, in the mesh's own shape. **Contributing to something is requiring it** — asking to be
|
||||
published means a publisher must exist, and a module that had to say both would eventually say
|
||||
one, with the failure appearing as a machine where nothing serves the route.
|
||||
|
||||
**The control plane does not know what a reverse proxy is**, and does not write one's
|
||||
configuration. It delivers the facts; the module turns them into whatever it runs. That boundary
|
||||
is what makes swapping the proxy cost nothing in any module that publishes through it, and it is
|
||||
[the same separation](0001-mesh-brokers-nodes-host-agents-think.md) that keeps third-party
|
||||
software *on* the mesh rather than *of* it. It also costs the host nothing: a received file is a
|
||||
file, which was checked by putting the control plane's output through the host's own parser rather
|
||||
than by asserting it.
|
||||
|
||||
**Binding is built except for the secret**, and that turned out to be the useful way to cut it.
|
||||
|
||||
A provider says what a consumer needs in order to use it — a port, a driver, a realm — and a
|
||||
consumer says where it wants to be told. The mesh adds the half only it has: **which machine, and
|
||||
what that machine is called on the private network.** So an application on one node is handed the
|
||||
address of its database on another, as a file, and reaches it by a name the mesh also created.
|
||||
|
||||
**The file states that it carries no credential, and why.** A missing field looks like a bug; a
|
||||
stated absence looks like a boundary, and somebody wiring this up should not spend an afternoon
|
||||
looking for a password that was never going to be there.
|
||||
|
||||
### And the secret, which is delivered without ever being held
|
||||
|
||||
*Written 2026-08-30, after looking at how the existing mesh does it. The design here is a reaction
|
||||
to a measurement, not a preference.*
|
||||
|
||||
**The obvious arrangement is a credentials column, encrypted at rest.** It exists, and its own
|
||||
tooling records what it bought:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| the tool for finding a secret matches **by value**, not by name | because one password is in the provisions table, the environment table, each node's environment file in plain text, and **inside every connection string composed from it** — copies its documentation calls *"often the only copies actually in use"* |
|
||||
| a query against the encrypted column **returns zero rows and proves nothing** | so auditing moved to the decrypted copies on the machines |
|
||||
|
||||
**Two faults, and encryption at rest addresses neither.** The control plane can read what it
|
||||
stores, so a copy of its database is a copy of every credential in the mesh. And one secret has
|
||||
many homes with nothing tracking them — **composition is what mints the untracked ones**, because
|
||||
building a connection string centrally creates a new secret-bearing value no rotation path knows
|
||||
about.
|
||||
|
||||
**So the value is sealed to the node that will use it before it is stored.** With a key that node
|
||||
generated and whose private half the mesh has never seen — a third key beside the identity and the
|
||||
overlay, for the same reason those are two rather than one. What is stored is unusable by whoever
|
||||
holds it, the mesh included, and the broker relays a blob it cannot read. This is what makes
|
||||
[ADR 0004](0004-a-node-and-how-it-joins.md)'s *compromise of a node is compromise of that node*
|
||||
true of secrets rather than true of identity and quietly false of everything that matters.
|
||||
|
||||
**And nothing is composed centrally.** A connection string is assembled on the machine that needs
|
||||
one, if at all. The mesh delivers parts.
|
||||
|
||||
**What it costs, stated because it is real:** the mesh cannot audit by value. That is the right
|
||||
trade rather than an oversight — a query over an encrypted column could not either, so the audit
|
||||
was never real. What *is* answerable is which node holds what, which is the question rotation
|
||||
actually asks.
|
||||
|
||||
**A consequence that shapes the mechanism.** The mesh discarded the plaintext, so it cannot
|
||||
compose a file containing it. The credential is therefore **its own file**, holding the value and
|
||||
nothing else, beside the readable one. That is better than the alternative it was forced into:
|
||||
the readable half stays readable in the declaration, and the secret half changes only when the
|
||||
secret does, so a service reloading on it reloads for a real reason.
|
||||
|
||||
**Rotation is generating a new one**, because reading the old one back is not possible. Both ends
|
||||
are re-sealed and reach their machines in the same push — which removes the window where half the
|
||||
mesh holds a dead credential, the failure
|
||||
[recorded in ADR 0001](0001-mesh-brokers-nodes-host-agents-think.md) as consumers on three nodes
|
||||
holding one for two days.
|
||||
|
||||
### The provisioner, which is where the mesh stops
|
||||
|
||||
*Written 2026-08-30, from building one and running it against a real database.*
|
||||
|
||||
**A password nothing was told to create authenticates nowhere.** The mesh generates one, seals it
|
||||
to both ends and cannot read it — so it cannot tell the software to start accepting it either.
|
||||
Something on the providing machine reads what arrived and makes it true. That is a provisioner.
|
||||
|
||||
**It belongs to the module, not to the mesh**, and the boundary is the same one that keeps
|
||||
third-party software running *on* the mesh rather than being *of* it
|
||||
([ADR 0001](0001-mesh-brokers-nodes-host-agents-think.md)). The control plane decides and never
|
||||
touches a machine. **What the mesh owns is the contract**, which is two files the host writes from
|
||||
an ordinary declaration:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| the manifest | every consumer, what it asked for, and **where** its credential is |
|
||||
| one file per consumer | that credential, alone in it |
|
||||
|
||||
Two files because the mesh discarded the value and cannot compose a document containing it. As
|
||||
before, the constraint produces the better shape: the readable half stays readable and auditable
|
||||
in the declaration, and the secret half changes only when the secret does.
|
||||
|
||||
**It reconciles; it is never told what changed.** It runs after every declaration and must reach
|
||||
the same state from wherever it starts. Three consequences, and each of them is a fault that has
|
||||
been shipped somewhere:
|
||||
|
||||
- **the password is set every time, not only on creation** — otherwise the role already exists,
|
||||
nothing happens, and a rotation reports success while changing nothing
|
||||
- **what it made and nobody asks for any more is removed** — otherwise a consumer that left keeps
|
||||
a working login for ever and nothing ever says so. This is the same rule the host follows about
|
||||
[removing what it declared and no longer declares](../04-ISSUES/010-the-first-declaration-destroys-the-substrate/00-report.md)
|
||||
- **what it did not make is left alone** — otherwise it cannot be run on a system that predates
|
||||
it, which is every system anybody would want to adopt
|
||||
|
||||
**A missing credential is refused rather than worked around.** A role created without one is a
|
||||
login nothing can use, and nothing would report it until something tried to connect.
|
||||
|
||||
**This is where the mesh stops**, and saying so is the point of the section. It decides, delivers
|
||||
and can prove what it delivered; the last inch belongs to whoever knows what `create role` means.
|
||||
|
||||
**One check that only became possible now.** Two machines wired together across no private network
|
||||
is a mesh that reports itself configured and does not work, and the failure surfaces as a
|
||||
connection timing out — the slowest place to find anything. It is refused, and it is only
|
||||
*checkable* because the network became [something a machine is
|
||||
given](#what-a-domain-module-turns-out-to-be-and-why-it-is-not-the-one-refused-above) rather than
|
||||
something it has by virtue of holding an address.
|
||||
|
||||
**What the absence cost, measured.** Exactly two modules opened a direct connection to the control
|
||||
plane's database — the proxy and the VPN — and they are the reason every node permanently holds a
|
||||
credential to it. Both were doing by hand what this edge is for. The VPN's half is closed by being
|
||||
[a module whose files are computed](../03-DESIGN/01-to-be/08-connectivity.md); the proxy's is
|
||||
closed by contribution. **Neither needed a new kind of thing, and both had been outside the model
|
||||
for as long as there was one.**
|
||||
|
||||
### Why the build edge is a different kind
|
||||
|
||||
It is fixed inside an artifact rather than negotiated when something runs, and **its only remedy
|
||||
is a rebuild** — nothing can re-provision it.
|
||||
|
||||
It is also **derived rather than declared**, and the asymmetry is deliberate: a runtime edge is an
|
||||
*intention* somebody has about how the mesh should be wired, and only a person can state it. A
|
||||
build edge is a *fact about code that already exists*, and a declared list of dependencies drifts
|
||||
from the imports it describes.
|
||||
|
||||
**An artifact is out of date when its source moved, or when anything it was built against moved.**
|
||||
So what is recorded is a commit *and the identity of every artifact it was built against*, which
|
||||
is what makes the rebuild set computable and *is this current?* answerable without building.
|
||||
|
||||
**The graph measures design quality, not just build order.** A module with many inbound build
|
||||
edges is one whose every change is expensive — and that is readable before anything is built. The
|
||||
current shared library is exactly that, and nobody could see it because nothing drew the edges.
|
||||
|
||||
## Provisioning is declared, never configured by hand
|
||||
|
||||
A module declares what it **provides** and what it **requires**. The mesh satisfies it: a
|
||||
provisioner belonging to the provider creates the resource and its credential, records the grant,
|
||||
and the values are derived onto the consumer. **Neither the credential nor the topology is ever
|
||||
written by hand.** A requirement may name a provider on another node, so cross-node wiring is the
|
||||
same declaration.
|
||||
|
||||
## What a module claims, and why it is not a list of rivals
|
||||
|
||||
*Written 2026-08-29, replacing pairwise exclusion.*
|
||||
|
||||
**Exclusivity is not a property of a module. It is a property of a singular resource the module
|
||||
takes over.** Two shells do not compete for anything and any number may be installed. Two display
|
||||
servers both want the seat, and only one may have it.
|
||||
|
||||
> **A module declares what it *claims*. Two modules claiming the same thing cannot both be
|
||||
> assigned within that claim's scope.**
|
||||
|
||||
**Not "xorg conflicts with wayland".** Pairwise exclusion has a property that only shows up later:
|
||||
adding a third display server means **editing xorg and wayland to know about it**. Every new
|
||||
module requires changing modules nobody who wrote it owns, and the edits grow as the square of
|
||||
the count. With a claim, the third one says `claims: the seat` and nothing else changes anywhere.
|
||||
**The new module is the only thing that has to know anything** — which is the difference between
|
||||
a catalogue that grows and one that calcifies.
|
||||
|
||||
The pattern is common enough to be worth listing, because seeing it is most of understanding it:
|
||||
|
||||
| these coexist | these claim one thing |
|
||||
|---|---|
|
||||
| shells — bash, zsh, fish | display servers — xorg, wayland (*the seat*) |
|
||||
| editors — vim, emacs, helix | init — systemd, openrc (*pid 1*) |
|
||||
| language runtimes | container runtime — docker, podman |
|
||||
| terminal emulators | reverse proxies — nginx, caddy, traefik (*ports 80/443*) |
|
||||
| browsers | time — chrony, timesyncd, ntpd (*the clock*) |
|
||||
| | resolvers — resolved, dnsmasq, unbound (*`/etc/resolv.conf`*) |
|
||||
| | network management — NetworkManager, networkd, netctl |
|
||||
| | mail — postfix, exim, msmtp (*port 25*) |
|
||||
| | audio — pipewire, pulseaudio (*the device*) |
|
||||
|
||||
**A claim has a scope**, because not everything singular is singular per machine:
|
||||
|
||||
| scope | example |
|
||||
|---|---|
|
||||
| **node** | the seat, pid 1, port 443 |
|
||||
| **site** | a DHCP server on a segment |
|
||||
| **mesh** | the hub, the control plane |
|
||||
|
||||
The last is not new — the mesh already enforces exactly one hub with a unique index
|
||||
([ADR 0007](0007-connectivity.md)). Scope is that idea, said once rather than hard-coded per case.
|
||||
|
||||
**Some conflicts need no claim at all.** Two modules declaring the same file, or binding the same
|
||||
port, are visible from *what they declare* — the mesh already holds every resource of every
|
||||
declaration. So a claim is only written for the abstract ones, where nothing in the declaration
|
||||
reveals the clash. That keeps the manifest small, which is worth protecting.
|
||||
|
||||
## A requirement with several answers is refused, never guessed
|
||||
|
||||
A module requiring *a shell* may be satisfied by three. The mesh does not pick.
|
||||
|
||||
| candidates | what happens |
|
||||
|---|---|
|
||||
| exactly one | assigned, silently — there was no choice to make |
|
||||
| none | refused, naming what is missing |
|
||||
| several | **refused, naming them**, and a person chooses |
|
||||
|
||||
**This is what makes a solver unnecessary.** Counting candidates is a few lines and has no
|
||||
surprising behaviour; a solver that picks has to be understood before its answer can be trusted,
|
||||
and it is understood by whoever is debugging it at the time. Nothing here is lost by waiting —
|
||||
a solver can be added later without changing a single manifest, and the reverse is not true.
|
||||
|
||||
**Requiring a module and requiring a capability are different fields**, because the remedies
|
||||
differ and the message should say which:
|
||||
|
||||
- *i3 needs xorg, which is not assigned here* — assign it.
|
||||
- *this machine has no seat* — wrong machine; nothing can be installed to fix it.
|
||||
|
||||
### A capability may carry a value, and that is not a new idea
|
||||
|
||||
A capability is a named fact about a machine, **detected and never assumed**. Its presence gates
|
||||
an assignment; its detail can also carry a value — `seat: card1-DP-1`, `panel: oled`, an
|
||||
architecture, an amount of memory. Nothing new is needed for that: a verdict has always had a
|
||||
detail beside its yes or no.
|
||||
|
||||
So *can this run here* and *what should it be configured as* are answered by the same fact, read
|
||||
two ways. A module that must not be assigned without an OLED panel and one that dims itself
|
||||
differently on one are reading the same line.
|
||||
|
||||
**What keeps the set from sprawling is the cost of adding one.** A capability must be detected,
|
||||
and the detector must say how it knows — so nobody can add one they cannot check, which is the
|
||||
whole of [04-ISSUES/007](../04-ISSUES/007-an-installed-package-is-not-a-capability/00-report.md):
|
||||
an installed package was treated as a capability and a node was assigned work it could not do.
|
||||
|
||||
**And detectors ship inside the host**, which is one statically linked binary. Adding a capability
|
||||
means shipping a new host to every node that needs it. That is a real cost and it argues for
|
||||
keeping the vocabulary small and general — `seat`, not `has-nvidia-with-two-outputs`.
|
||||
|
||||
## "Flavor" is retired
|
||||
|
||||
It was carrying three unrelated meanings — variants of a thing, a subset of one module a node
|
||||
installs, and whatever the current system does, which earned two knowledge-base entries about
|
||||
going wrong. **A word with three meanings cannot be reasoned about**, and every attempt to design
|
||||
around it produced a rule that was right for one meaning and wrong for the others.
|
||||
|
||||
What it was reaching for is two ordinary things:
|
||||
|
||||
- **Different modules that provide the same thing.** `zsh` and `fish` both provide *a shell*. They
|
||||
are two modules, not one module with a switch: they share a name and nothing else — different
|
||||
packages, different configuration, different everything.
|
||||
- **One module with a setting.** A monitoring module that is an agent here and a server there is
|
||||
one module, configured. Nothing varies but a value.
|
||||
|
||||
If something is neither, it is probably two modules.
|
||||
|
||||
**A third thing it was reaching for, added 2026-08-29:** *I want this working and I do not care
|
||||
which one.* That is a module with requirements and no files —
|
||||
[a domain module](#what-a-domain-module-turns-out-to-be-and-why-it-is-not-the-one-refused-above) —
|
||||
and it is what makes "different modules that provide the same thing" bearable for somebody who
|
||||
does not want to know there is a choice.
|
||||
|
||||
## The core library is the mesh's domain
|
||||
|
||||
One module everything may depend on. It holds **what is true of the mesh regardless of which
|
||||
context you are in**: a module, a node, an assignment.
|
||||
|
||||
The test: *would this still mean the same thing in a context that had never heard of the one it
|
||||
came from?* A node would. A pipeline stage would not — that is delivery's.
|
||||
|
||||
**Types ship with the module that owns them**, not here. A consumer needing `inventory`'s types
|
||||
depends on `inventory` — one narrow, visible edge — rather than everything depending on a hub
|
||||
where the relationship cannot be seen. **A library everything depends on is expensive to change
|
||||
whether it holds types or code; the fan-in is what makes it expensive**, which is why *types, not
|
||||
behaviour* was the wrong guard.
|
||||
|
||||
**It stays small on its own.** A domain model changes when what the mesh *is* changes, which is
|
||||
rare. A drawer labelled *shared* changes whenever anybody writes something reusable, which is
|
||||
constantly — and *who else might want this* always answers yes, which is how the current one grew.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **Fewer things will be shared, and some code will be written twice.** That is the trade: the
|
||||
current library exists because sharing felt free. Two similar functions in two modules is often
|
||||
the better answer.
|
||||
- **The check is a measurement rather than a prohibition.** Inbound build edges say when something
|
||||
is becoming a hub, while it is happening rather than after.
|
||||
- **Reading build edges needs a language-aware tool per language**, which is the real cost and the
|
||||
reason declaring them looks tempting. It is still wrong.
|
||||
@@ -0,0 +1,161 @@
|
||||
---
|
||||
topic: what runs on it
|
||||
status: accepted
|
||||
date: 2026-08-28
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
---
|
||||
|
||||
# 10. Delivery
|
||||
|
||||
*Consolidated 2026-08-28 from five records.*
|
||||
|
||||
## Delivery is a comparison, not a pipeline
|
||||
|
||||
**The control plane holds what source exists and what has been built from it, and builds the
|
||||
difference.** A change becomes a build because source is **ahead of artifacts** — answerable at
|
||||
any moment — rather than because a message arrived.
|
||||
|
||||
**An event makes it fast. Nothing makes it necessary.** A missed notification costs latency and
|
||||
cannot cost correctness.
|
||||
|
||||
That is the same shape the host uses on a machine, one layer up:
|
||||
|
||||
| | reconciles | against |
|
||||
|---|---|---|
|
||||
| the control plane | artifacts | source |
|
||||
| the host | machine state | declarations |
|
||||
|
||||
**This is not the current coordinator repaired.** That is a state machine over stages; the value
|
||||
of it here is as a catalogue of the ways this fails, and it has been used for exactly that.
|
||||
|
||||
### The module system is the CI/CD
|
||||
|
||||
*Written 2026-08-29, because this was the intention throughout and was never stated in one line.*
|
||||
|
||||
**There is no pipeline product beside the mesh, and there is not going to be one.** A module
|
||||
declares what it is ([ADR 0009](0009-modules-and-the-graph.md)); the control plane notices its
|
||||
source is ahead of its artifacts and builds it; the graph says what else that invalidates; the
|
||||
node that should run it is told. Build, test, publish and deploy are the same reconciliation seen
|
||||
at four points, not four stages wired together.
|
||||
|
||||
**Which is why the module system is the core of the setup rather than one component of it.** Every
|
||||
other layer is carried by it: the substrate is modules the bundle raises before there is a mesh,
|
||||
the control plane is a module, and an application is a module with a different manifest. A thing
|
||||
that cannot be expressed as a module cannot be delivered at all — that is a real constraint, and it
|
||||
is the one keeping a second delivery mechanism from growing beside this one.
|
||||
|
||||
**What disappears is the pipeline as a state machine** — no stage list something can be omitted
|
||||
from, which is how a verify stage was built and never scheduled, and no run to lose.
|
||||
|
||||
### Currency is the whole input closure
|
||||
|
||||
**An artifact is out of date when its source moved, or anything it was built against moved.** So a
|
||||
shared library changing invalidates everything with a transitive build edge to it, in dependency
|
||||
order, because a module cannot be built against a new library until it exists.
|
||||
|
||||
**The module graph is a prerequisite of this, not an enabler of it.** Without it there is no
|
||||
rebuild set and no ordering, and this cannot be implemented.
|
||||
|
||||
## An artifact is build output, never a source tree
|
||||
|
||||
Compiled and bundled with its dependency graph inlined. **A deploy is extract-and-run and touches
|
||||
no network.**
|
||||
|
||||
The consequence is the whole cost of the decision: **anything not in the build output does not
|
||||
ship.** Every file kind had to be brought into that rule separately, and each was discovered by
|
||||
something silently not happening after a deploy — migrations reading a source layout,
|
||||
provisioning scripts reading a source layout, selection files never packaged at all.
|
||||
|
||||
## Three silos, and the third is not a stage
|
||||
|
||||
The cardinality observation holds and is what the split is for:
|
||||
|
||||
| silo | runs | ends with |
|
||||
|---|---|---|
|
||||
| **build** | once per module | a self-contained artifact |
|
||||
| **publish** | once per module | that artifact addressable — an image by digest, a package in the mesh's repository |
|
||||
| **deploy** | **once, not once per node** | the affected nodes' **declarations updated** |
|
||||
|
||||
**Deploy stops sending commands to nodes.** It changes what the control plane says each node
|
||||
should be, which is one write. What happens on the machines is the host's ordinary reconcile.
|
||||
|
||||
**Why this fixes the failure class rather than patching it.** Every recorded fault shares one
|
||||
shape: *the thing that reported success was not the thing that did the work.* A coordinator
|
||||
dispatching a command can only report on dispatch. Under this the reporter **is** the applier —
|
||||
which already refuses to record a resource until it read it back, and already fails the whole
|
||||
apply on one failed step.
|
||||
|
||||
**The verify stage disappears as a stage**, which is the strongest evidence for the shape:
|
||||
verification stops being a step that can be omitted from a list and becomes a property of applying
|
||||
at all.
|
||||
|
||||
**There is no fan-out**, so the defect class that came from the build node having passed through
|
||||
two silos while others had not cannot arise.
|
||||
|
||||
## A step that fails must fail the job
|
||||
|
||||
A step that fails and lets the job continue **reports success for work that did not happen**.
|
||||
Absence of an error is not evidence of an effect.
|
||||
|
||||
This is the mesh's most consistent failure shape, and it is not incidental — it is what stage
|
||||
reporting measured. Documented instances: a service reported started when the container command
|
||||
merely returned; an image pull failure that did not fail the deploy; a package install that 404'd
|
||||
from every mirror while the job went green; a node left on old code after a failed download with
|
||||
a version marker that had already advanced.
|
||||
|
||||
## The verdict is tiered
|
||||
|
||||
An artifact may not be declared until something has judged it fit. **Two tiers, because one gate
|
||||
would be both slow and unreliable:**
|
||||
|
||||
| | judged by | when |
|
||||
|---|---|---|
|
||||
| the module's own tests | the build | **always** — this is most of it |
|
||||
| the lab | a raised scenario | when an assertion genuinely needs a mesh |
|
||||
|
||||
A lab scenario takes tens of seconds and can fail for reasons that have nothing to do with the
|
||||
artifact, and a shared-library change produces a cascade of dozens. One expensive
|
||||
non-deterministic gate fails in both directions: a flaky run marks a good artifact unfit, a lucky
|
||||
one marks a bad artifact fit, and **neither failure looks like itself.**
|
||||
|
||||
**A run that failed environmentally is not a verdict.** A machine that would not boot says nothing
|
||||
about the artifact, and recording it as *unfit* is the same untruth as recording a dispatch as a
|
||||
deploy.
|
||||
|
||||
## What a result means
|
||||
|
||||
> **The declaration is updated, and here is which nodes have applied it.**
|
||||
|
||||
A pipeline does not wait for every node — one may be legitimately switched off for a week, and a
|
||||
delivery mechanism that blocks on a sleeping laptop is one nobody will use.
|
||||
|
||||
```
|
||||
delivered declaration updated for 5 nodes
|
||||
applied 3 of 5
|
||||
outstanding 2 — last seen 4 days ago, 20 minutes ago
|
||||
```
|
||||
|
||||
**Outstanding is not failure**, and conflating them is how the old system produced a stall with no
|
||||
error anywhere.
|
||||
|
||||
## What must exist first
|
||||
|
||||
1. **The module graph, with build edges.** No graph, no rebuild set and no ordering.
|
||||
2. **A recorded input closure per artifact**, so currency is answerable without building.
|
||||
3. **Something that notices a reconciler is not converging.** Below.
|
||||
|
||||
## Open, and the first is the real risk
|
||||
|
||||
- **A loop that will not converge is harder to debug than a job that failed.** A failed job stops
|
||||
and names its step; a reconciler retries forever. Without something that notices *this has been
|
||||
trying for an hour*, the failure is **silence** — the fault this removes, reintroduced in a new
|
||||
place.
|
||||
- **The run identity people use is lost.** *Did my change go out?* is answerable today by opening
|
||||
a pipeline. Something must replace that or this is worse to live with, whatever its properties.
|
||||
- **Does a fit artifact declare itself?** If it does, merging to main deploys to production —
|
||||
which may be wanted and is far too large a property to acquire by omission.
|
||||
- **Rebuild storms are mostly behaviourally empty.** Reproducible builds would stop a cascade at
|
||||
the first module whose output did not move; without them one commit redeploys the fleet for no
|
||||
change in behaviour.
|
||||
- **Detection stays the fragile input for latency**, though no longer for correctness.
|
||||
@@ -0,0 +1,64 @@
|
||||
---
|
||||
topic: building it
|
||||
status: accepted
|
||||
date: 2026-04-03
|
||||
deciders: jochen
|
||||
reconstructed: true
|
||||
---
|
||||
|
||||
# 11. Managed files are generated onto nodes and never edited there
|
||||
|
||||
> Reconstructed after the fact from the evidence cited below.
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0006](0006-the-substrate-and-the-control-plane.md) put every binding in the mesh
|
||||
database. But the things that consume those bindings — environment files, service
|
||||
definitions, daemon configuration, firewall rules — are files on a node's disk, because that
|
||||
is what the software reading them requires.
|
||||
|
||||
So the same value exists twice: authoritatively in the database, and materialised in a file.
|
||||
Any edit to the file is a change to a copy. Before this decision, environment values could be
|
||||
pushed from a node back into the database, which made the direction ambiguous in both
|
||||
directions at once.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **Bidirectional sync** — a node's edits flow back to the database. Rejected, and removed.
|
||||
Two writers and no arbiter: whichever synced last wins, and neither is authority.
|
||||
2. **Files are authoritative; the database is a cache of them.** Rejected — it inverts
|
||||
ADR 0013 and returns to state that cannot be reconciled across nodes.
|
||||
3. **Strictly one-directional: the database is written, files are generated.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
Every managed file is **derived**. A synchroniser regenerates it from the mesh database
|
||||
whenever the underlying values change. The write path is the mesh tool that owns the value;
|
||||
the file is an output.
|
||||
|
||||
This applies to generated environment files, service definitions, managed configuration, and
|
||||
anything else a synchroniser lists as its own.
|
||||
|
||||
An edit to a managed file survives until the next synchronisation and is then overwritten,
|
||||
without a warning, taking whatever it was fixing with it.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **A file edited on a node is a bug with a delay on it.** This is now one of the mesh's core
|
||||
values, and it is a consequence of this decision rather than a stance taken independently.
|
||||
- To change a value you must know which tool owns it. That is a real cost, paid every time,
|
||||
and the reason the mesh provides a way to ask whether a given file is managed.
|
||||
- Debugging by editing a file no longer works, and fails in the most confusing way available:
|
||||
it works, and then stops working later for no locally visible reason.
|
||||
- Recovery is cheap. A node's entire managed surface can be regenerated from the database.
|
||||
- Values resolve by precedence — database override, then existing file value, then generated,
|
||||
then manifest default — which means an unset override does not clobber a generated
|
||||
password. The subtlety is real and has caused its own confusion.
|
||||
|
||||
## References
|
||||
|
||||
- `Extract hal/env-sync module, remove syncEnvToDb`, 2026-04-03 — the commit that removed the
|
||||
node-to-database direction.
|
||||
- Knowledge base: `conventions/no-direct-file-mutation`, `provisioning` (resolution priority).
|
||||
- Regeneration gaps: `troubleshooting/changed-manifest-default-not-rerendered`,
|
||||
`troubleshooting/config-removed-from-manifest-not-pruned`.
|
||||
@@ -0,0 +1,101 @@
|
||||
---
|
||||
topic: building it
|
||||
status: accepted
|
||||
date: 2026-08-23
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
---
|
||||
|
||||
# 12. The mesh creates no symlinks — a derived file is a copy
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0012](0012-the-mesh-creates-no-symlinks.md) responded to production data loss — a hand-made
|
||||
link, resolved through a container engine's volume handling, pointing a mount somewhere it
|
||||
should not have — by centralising linking in the installer and forbidding it everywhere else.
|
||||
|
||||
That narrowed the incident class. It did not close it. The hazard is not *who* made the link;
|
||||
it is that a path can resolve somewhere other than where it appears to. A link made by the
|
||||
installer resolves exactly the same way as a link made by hand. The rule made the mechanism
|
||||
rarer and better-governed while leaving the mechanism in place.
|
||||
|
||||
Two things have changed since, and together they remove the argument that kept it.
|
||||
|
||||
**The original case for linking was staleness.** A copy of a service definition goes stale
|
||||
silently while the catalogue moves on, so a link was the cheap way to guarantee the running
|
||||
node reads a current definition. That argument assumes the node's copy is unmanaged.
|
||||
|
||||
**It is not.** [ADR 0011](0011-managed-files-are-generated-never-edited.md) established that
|
||||
everything on a node's disk is derived from the mesh and regenerated when its inputs change,
|
||||
and the installer already **reconciles** links rather than assuming them — repointing stale
|
||||
ones, adopting real files it finds where a link belongs. Reconciling content is the same
|
||||
operation as reconciling a pointer, plus a comparison.
|
||||
|
||||
So the mesh already has the machinery that makes a copy safe, and is using a link to solve a
|
||||
problem that machinery solves better. Worse, a link is conceptually the wrong shape: it makes
|
||||
the node's runtime state a *pointer into source*, which is the one thing
|
||||
[ADR 0006](0006-the-substrate-and-the-control-plane.md) and ADR 0011 exist to prevent.
|
||||
State is derived onto nodes; it does not reach back.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **Keep ADR 0019 as the final position** — centralised linking, forbidden elsewhere.
|
||||
Rejected as the status quo. It governs the mechanism rather than removing it, and the
|
||||
failure it was written for remains reachable by any code path the installer trusts.
|
||||
2. **Keep links but harden them** — canonicalise before mounting, refuse a link that escapes
|
||||
an expected root. Rejected: it is a check bolted onto a hazard, and it has to be correct in
|
||||
every consumer, including container engines the mesh does not control.
|
||||
3. **Copy, reconciled by the installer, with staleness detected rather than assumed away.**
|
||||
Proposed here.
|
||||
|
||||
## Decision
|
||||
|
||||
*Accepted 2026-08-25. The position is settled; the migration is not designed. See "Open" below.*
|
||||
|
||||
**The mesh creates no symlinks.** A file a node needs is placed on that node as a real file,
|
||||
derived from the mesh and reconciled by the installer like every other managed file
|
||||
([ADR 0011](0011-managed-files-are-generated-never-edited.md)).
|
||||
|
||||
The prohibition in ADR 0019 stands and widens: it ceases to be "only the installer may link"
|
||||
and becomes "nothing links, the installer included".
|
||||
|
||||
When this is accepted, ADR 0019 becomes superseded rather than edited — its reasoning is why
|
||||
the rule exists at all, and the incident behind it is the reason anyone believes either record.
|
||||
|
||||
## Consequences
|
||||
|
||||
- The path-resolution hazard is removed rather than governed. There is no link for a container
|
||||
engine to resolve, so the class of failure that cost production data is closed by
|
||||
construction.
|
||||
- A node's runtime state stops pointing into source. What a node holds is derived output, which
|
||||
is what the mesh's model already says it is everywhere else.
|
||||
- **Staleness becomes a real problem that must be answered, not assumed away.** This is the
|
||||
cost, and it is the whole cost: today a link cannot be stale, and a copy can. The answer has
|
||||
to be detection — the installer comparing what is on disk against what the mesh says should
|
||||
be — and it must be loud, because a silently stale definition is exactly the failure shape
|
||||
this mesh keeps producing ([ADR 0010](0010-delivery.md)).
|
||||
- Reconciliation gets more expensive: comparing content rather than checking a pointer's
|
||||
target, on every module, on every node.
|
||||
- Disk usage rises, trivially, and is not a consideration.
|
||||
- Existing links must be converted. A node mid-migration holds both forms, so reconciliation
|
||||
has to handle finding a link where a file now belongs — the mirror image of the adoption it
|
||||
already does.
|
||||
|
||||
## Open
|
||||
|
||||
- **How staleness is detected.** Content hash, version marker, or regeneration on every
|
||||
reconcile. This is the decision that makes or breaks the change and it is not taken here.
|
||||
- **Whether anything must keep a link** for reasons outside the mesh's control. If something
|
||||
does, that is a finding worth recording rather than an exception worth granting quietly.
|
||||
- **Migration order.** Converting a node's links is a change to how its services resolve their
|
||||
own definitions, which is not a change to make everywhere at once.
|
||||
|
||||
Until those are answered this record stays `proposed`, and ADR 0019 remains the governing rule.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0012](0012-the-mesh-creates-no-symlinks.md) — the incident, and the rule this widens.
|
||||
- [ADR 0011](0011-managed-files-are-generated-never-edited.md) — the machinery that makes a
|
||||
copy safe.
|
||||
- [`03-DESIGN/00-as-is/05-runtime-and-installation.md`](../03-DESIGN/00-as-is/05-runtime-and-installation.md)
|
||||
— what the installer does today, including reconciliation and adoption.
|
||||
@@ -0,0 +1,69 @@
|
||||
---
|
||||
topic: building it
|
||||
status: accepted
|
||||
date: 2026-05-14
|
||||
deciders: jochen
|
||||
reconstructed: true
|
||||
---
|
||||
|
||||
# 13. Schema and state changes are numbered migrations, in the same language as the code
|
||||
|
||||
> Reconstructed after the fact from the evidence cited below.
|
||||
|
||||
## Context
|
||||
|
||||
Modules own persistent state. That state has to change as they change, across nodes that are
|
||||
at different versions, some of which have data that predates the change.
|
||||
|
||||
Two things were being done that do not survive contact with a second node. Schema was created
|
||||
at startup, so what a table looked like depended on which version last started. And migrations
|
||||
were shell scripts, so they could not use the types, connection handling or helpers the module
|
||||
already had, and were not compiled or checked with it.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **Startup SQL / create-if-missing.** Rejected. It converges only for a node that started
|
||||
with the newest version. A node that never restarts never migrates; a node that restarts on
|
||||
an old version can undo a change.
|
||||
2. **Shell migrations.** Rejected. Unchecked, untyped, and a separate dialect from the module
|
||||
they belong to. Also, as later discovered, packaged differently — and therefore
|
||||
occasionally not packaged at all.
|
||||
3. **Numbered migrations in the module's own language, compiled with it.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
Every schema or state change is a numbered migration file, written in the same language as the
|
||||
module and compiled with it. There is no startup schema creation and no ad-hoc statement.
|
||||
|
||||
Rules that come with it:
|
||||
|
||||
- The initial migration is **frozen** once it has run anywhere. It is never modified; a change
|
||||
is a new number.
|
||||
- Every statement is **idempotent** — guarded so that re-running is safe.
|
||||
- A change needs **both** a baseline for a fresh installation and an incremental migration for
|
||||
installations that already exist. Code referencing a column requires that the migration
|
||||
creating it exists.
|
||||
- Migration numbers are unique. A duplicate prefix is a defect, not a style issue.
|
||||
|
||||
## Consequences
|
||||
|
||||
- A node at any version converges to the current schema by running the migrations it has not
|
||||
run.
|
||||
- Migrations are checked by the same compiler as the code, and a migration that does not
|
||||
compile fails the build rather than the deployment.
|
||||
- The rules are enforced unevenly. Duplicate prefixes have shipped repeatedly and been fixed by
|
||||
renumbering afterwards; the mesh now validates for them, which is the check this rule needed
|
||||
in order to be real.
|
||||
- A migration directory is a feature like any other, which means it is packaged like any other
|
||||
— and when packaging is wrong, migrations silently do not ship. This has happened.
|
||||
- Freezing the initial migration means a fresh installation replays the entire history. That
|
||||
cost grows and nothing currently bounds it.
|
||||
|
||||
## References
|
||||
|
||||
- `fix(agents): convert workflow migrations to TypeScript` (#50), 2026-05-14.
|
||||
- `feat(dev_validate): guard against duplicate migration numeric prefixes` (#379), 2026-06-26 —
|
||||
the rule acquiring a check.
|
||||
- Knowledge base: `migrations/schema-drift`, `noxflow/troubleshooting/migration-number-collision`.
|
||||
- Packaging failures: `troubleshooting/shell-migrations-never-packaged`,
|
||||
`troubleshooting/provision-migration-never-applied`.
|
||||
@@ -0,0 +1,66 @@
|
||||
---
|
||||
topic: building it
|
||||
status: accepted
|
||||
date: 2026-06-04
|
||||
deciders: jochen
|
||||
reconstructed: true
|
||||
---
|
||||
|
||||
# 14. No workspace — each module is a standalone package consuming published dependencies
|
||||
|
||||
> Reconstructed after the fact from the evidence cited below.
|
||||
|
||||
## Context
|
||||
|
||||
Modules depend on each other, above all on the shared library every module builds against.
|
||||
A workspace was the obvious way to express that: sibling packages, resolved locally, one
|
||||
install at the root.
|
||||
|
||||
It produced a divergence that is worth stating precisely, because it is not obvious. In
|
||||
development, a workspace member importing a sibling resolves to that sibling's **local source**.
|
||||
In the pipeline, each module is built alone, from a clone, without its siblings present — so
|
||||
the same import resolves to the **published version**. The two environments were therefore
|
||||
building different code from identical source, and the failure appeared only in the pipeline,
|
||||
in a module that had not been touched.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **Keep the workspace and make the pipeline replicate it** — clone every module, build the
|
||||
graph. Rejected: it makes every build a whole-repository build, which is the cost the
|
||||
per-module pipeline exists to avoid, and it does not extend to modules in their own
|
||||
repositories.
|
||||
2. **Keep the workspace and pin siblings to published versions.** Rejected as the worst of
|
||||
both: the workspace's local resolution silently overrides the pin, so the divergence
|
||||
remains while looking solved.
|
||||
3. **No workspace. Every module is standalone and consumes published dependencies.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
There is no workspace. Each module is an independent package that declares its dependencies
|
||||
and consumes them from the private registry, including the mesh's own shared library.
|
||||
|
||||
A cross-package change is therefore two steps: publish the producer, then consume it. The
|
||||
pipeline does the first on push and resolves the levels so that a module always builds against
|
||||
its dependencies' freshly published versions.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Development and the pipeline resolve imports identically. The divergence is gone by
|
||||
construction rather than by discipline.
|
||||
- A module in its own repository is not a special case. It builds exactly as a module in the
|
||||
monorepo does — which is what makes [ADR 0015](0015-applications-live-in-their-own-repository.md)
|
||||
cheap.
|
||||
- A cross-package change costs a publish-and-consume round trip. This is the real price, paid
|
||||
on every shared-library change.
|
||||
- There is no repository-wide install and no repository-wide build. Anything that assumed one
|
||||
broke, and one thing that assumed one has stayed broken: the end-to-end pipeline harness has
|
||||
not built since this decision landed. See
|
||||
[`04-ISSUES/005`](../04-ISSUES/005-pipeline-test-harness-unbuildable/00-report.md).
|
||||
|
||||
## References
|
||||
|
||||
- `fix(noxflow): kill npm workspace, restore encryption inside PgAdminRepo` (#240),
|
||||
2026-06-04. The reason is recorded in the root package manifest, which still carries the
|
||||
note explaining why no workspace exists.
|
||||
- The divergence it fixed is named there: workspace members importing each other resolved to
|
||||
local unbuilt source in the pipeline.
|
||||
@@ -0,0 +1,65 @@
|
||||
---
|
||||
topic: building it
|
||||
status: accepted
|
||||
date: 2026-07-10
|
||||
deciders: jochen
|
||||
reconstructed: true
|
||||
---
|
||||
|
||||
# 15. Applications live in their own repository; the monorepo is for the mesh
|
||||
|
||||
> Reconstructed after the fact from the evidence cited below.
|
||||
|
||||
## Context
|
||||
|
||||
The module system makes adding anything to the monorepo trivial — a directory and a manifest.
|
||||
That ease is the problem. Standalone applications, sites and side-projects accumulated beside
|
||||
the mesh's own components, and once there they inherited the monorepo's review cadence, its
|
||||
pipeline detection, and its history.
|
||||
|
||||
The mesh's own code and an application that merely runs on the mesh have nothing in common
|
||||
except the manifest format. They change for different reasons, are reviewed by different
|
||||
criteria, and have no reason to share a branch.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **Everything in the monorepo.** Rejected — it is what existed. The monorepo becomes an
|
||||
inventory of one installation's applications, and every application change queues behind
|
||||
mesh review.
|
||||
2. **A second monorepo for applications.** Rejected: the same coupling with an extra name.
|
||||
Applications have no more in common with each other than with the mesh.
|
||||
3. **One repository per application, registered with the mesh as a build source.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
Every standalone application, site or side-project lives in its own repository, with a manifest
|
||||
at the root. It registers with the mesh as a build source and is then built, provisioned,
|
||||
deployed and verified by exactly the same pipeline as anything in the monorepo.
|
||||
|
||||
The monorepo holds the mesh: the runtime, the core modules, the delivery machinery, and the
|
||||
shared infrastructure the mesh itself provisions against.
|
||||
|
||||
Creating an application directory in the monorepo is a convention violation, and reviewers
|
||||
reject it.
|
||||
|
||||
## Consequences
|
||||
|
||||
- An application's cadence is its own. It is not reviewed as mesh code and does not queue
|
||||
behind mesh work.
|
||||
- The separation is safe **only because** the pipeline and provisioning are identical either
|
||||
side of it — which [ADR 0014](0014-no-npm-workspace.md) is what makes true. Without
|
||||
standalone packages this decision would fork the build.
|
||||
- The monorepo stops being an inventory of the installation, which is a precondition for
|
||||
publishing anything about it.
|
||||
- Discovery gets harder: there is no single listing of everything the mesh runs, and the
|
||||
registry of build sources becomes the closest thing to one.
|
||||
- A module source that is not registered is silently skipped by the pipeline. The cost of
|
||||
being outside the monorepo is that being forgotten is possible.
|
||||
|
||||
## References
|
||||
|
||||
- The rule is stated in the governed constitution page authored 2026-07-10, §3, as a
|
||||
convention violation reviewers must reject.
|
||||
- [ADR 0001](0001-mesh-brokers-nodes-host-agents-think.md) decision 4 extends this from *new*
|
||||
applications to the modules already in the monorepo.
|
||||
- Knowledge base: `troubleshooting/unregistered-module-source`.
|
||||
@@ -0,0 +1,92 @@
|
||||
---
|
||||
topic: building it
|
||||
status: accepted
|
||||
date: 2026-08-28
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
---
|
||||
|
||||
# 16. The lab
|
||||
|
||||
*Consolidated 2026-08-28 from five records. The lab is one design and was split across five
|
||||
decisions taken over three days; the reasoning is kept, the fragmentation is not.*
|
||||
|
||||
The environment a change is run against before it reaches real machines.
|
||||
|
||||
> **Still the lab, no longer the test bed — 2026-09-30, by [ADR 0149](0149-the-live-mesh-is-the-test-bed.md).**
|
||||
> Everything here stands. What changed is what the lab is *for*: a change is verified against the mesh
|
||||
> that is running, because the faults that cost the most are faults of a mesh that already exists —
|
||||
> bound consumers, containers made against an older roster, an adopted machine — and a bed is by
|
||||
> construction a mesh that does not. Raising a mesh from bare is now the lab's whole job, which is the
|
||||
> one thing the live mesh cannot be asked to do. 0149 also supersedes
|
||||
> [ADR 0068](0068-the-lab-takes-requests.md), which extended this one and was never built.
|
||||
|
||||
## A node in the lab is a virtual machine
|
||||
|
||||
It boots a stock Linux image, runs the real install, and becomes a node. **It is not a model of
|
||||
a node**, so no question arises about how good the model is — which is the whole reason for
|
||||
paying the cost of virtual machines rather than containers.
|
||||
|
||||
The lab is driven by **incus**, and a scenario is raised from a declaration.
|
||||
|
||||
## A router is scenery, and is therefore a container
|
||||
|
||||
**Nothing under test runs on a router.** It is not a participant, holds no identity, has nothing
|
||||
installed on it by the mesh, and no assertion is ever made about its internals. It exists so that
|
||||
packets between machines behave the way they behave in the world.
|
||||
|
||||
The fidelity argument that makes a node a virtual machine does not reach it: what a router *is*
|
||||
does not matter, only what it *does to traffic*. So a router is a system container, and the lab
|
||||
is cheaper for it.
|
||||
|
||||
## A scenario declares the underlay, and only the underlay
|
||||
|
||||
**What a hosting provider and a home router would have provided**, before any of our software
|
||||
touched the machine:
|
||||
|
||||
- which segments exist, and their address ranges
|
||||
- which machine sits on which segment, at which address
|
||||
- what NAT sits between them, and which ports are forwarded through it
|
||||
- which machines are detached, and may be attached or detached during a run
|
||||
|
||||
**A scenario declares nothing about the overlay** — no overlay addresses, no hub, no peering, no
|
||||
names, no certificates. Those are the mesh's job, and a scenario that supplied them would be
|
||||
testing itself.
|
||||
|
||||
> A scenario provides what a hosting provider and a home router would provide, and nothing our
|
||||
> software is responsible for.
|
||||
|
||||
## A scenario is a closed address space
|
||||
|
||||
Every segment materialises as its own isolated link belonging to one scenario instance. **Two
|
||||
scenarios raised from the same declaration hold the same addresses and never meet**, because
|
||||
nothing joins their links. The declaration therefore keeps its literal addresses and they mean
|
||||
exactly what they say.
|
||||
|
||||
**The consequence that constrains everything else: the lab never reaches into a scenario over
|
||||
IP.** It talks to a machine through the virtualisation layer's own channel — the way one would
|
||||
use a console rather than the network. That is what makes two identical scenarios able to run at
|
||||
once, and it is why placing anything inside a machine is a hypervisor operation rather than a
|
||||
network one.
|
||||
|
||||
## Two scenario classes, and the first has no pipeline
|
||||
|
||||
| | **bootstrap** | **full** |
|
||||
|---|---|---|
|
||||
| contains | machines, the host binary, a pinned substrate bundle | a complete mesh: forge, coordinator, delivery, modules |
|
||||
| verdict from | what the host reports about the state it reconciled | a delivery result ending in verification |
|
||||
| exercises | tiers 0 and 1 | tiers 2 and above, and modules |
|
||||
|
||||
**The bootstrap class comes first**, because it is what develops the node host, and because a
|
||||
full scenario needs tiers that do not exist yet. A lab that could only raise the larger class
|
||||
would be a lab nobody could use until everything else was built.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **The lab tests the real code path**, not a reimplementation of it. The network a scenario
|
||||
produces is generated by the same code production runs.
|
||||
- **Isolation is what makes it usable in parallel**, and it costs the ability to reach in over
|
||||
IP. Everything the lab puts inside a machine — a binary, an image, a file — goes through the
|
||||
hypervisor.
|
||||
- **A sealed scenario cannot fetch anything**, which is a real limit rather than an inconvenience:
|
||||
it is why images have to be placed and why a container runtime has to be in the base image.
|
||||
@@ -0,0 +1,92 @@
|
||||
---
|
||||
topic: checking it
|
||||
status: accepted
|
||||
date: 2026-08-24
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
---
|
||||
|
||||
# 17. A test defends a decision
|
||||
|
||||
## Context
|
||||
|
||||
[`how-we-build.md`](../00-META/how-we-build.md) §5 already says that **if a document states a
|
||||
rule about the mesh, it says how the rule is verified**, on the grounds that an unenforced rule
|
||||
is indistinguishable from a wrong one and costs more, because people believe it.
|
||||
|
||||
That rule is applied to prose and to acceptance criteria. It has never been applied to
|
||||
**decisions**, and it should be — a decision record states something that must be true, which
|
||||
is the same kind of claim.
|
||||
|
||||
The gap was found by review. The lab reached 2,128 lines with 1,072 of them untested, and
|
||||
**no stated rule was broken.** There is no testing posture in `how-we-build.md` at all: no
|
||||
expectation, no gate, no definition of done. Every decision the lab embodies — the underlay
|
||||
boundary, the closed address space, routers as scenery, waiting for usable rather than for a
|
||||
call to return — was verified by hand, by running scenarios and reading output, and none of
|
||||
that survives the terminal it was run in.
|
||||
|
||||
Which is the fault the mesh already has catalogued at scale: an end-to-end harness that has
|
||||
not built since 2026-06-04, and nothing said so
|
||||
([`04-ISSUES/005`](../04-ISSUES/005-pipeline-test-harness-unbuildable/00-report.md)). Coverage
|
||||
assumed rather than checked.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **A coverage percentage.** Rejected. It measures how much code a test touched, not whether
|
||||
anything important is defended, and it is satisfied by tests that assert nothing. A number
|
||||
would have been met by testing the parser harder while the hypervisor integration stayed
|
||||
unasserted.
|
||||
2. **Test-driven development as a hard rule.** Rejected, and not because it is wrong in general.
|
||||
Half the lab's implementation was discovery: that the hypervisor CLI reads a definition from
|
||||
stdin and hangs, that it assigns a MAC without recording it, that a stock image's boot-time
|
||||
networking flushes a static address. A test written first against undiscovered behaviour
|
||||
asserts a guess.
|
||||
3. **A test defends a decision.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
**Every decision record states something that must be true. A test asserts it.**
|
||||
|
||||
A decision with no test is a decision that will quietly stop being true, and nobody will find
|
||||
out from a document. Concretely:
|
||||
|
||||
- Where a decision is about **structure or logic** — what a declaration may say, what is
|
||||
refused, how a name is derived — the test is a unit test, and it is **written first**, because
|
||||
the behaviour is knowable before the code.
|
||||
- Where a decision is about **behaviour against a real system** — a hypervisor, a broker, a
|
||||
daemon — the test runs against the real thing, and is written **alongside**, because the
|
||||
behaviour is discovered rather than known.
|
||||
- **Mocking the boundary is forbidden.** A test that fakes a hypervisor asserts that the fake
|
||||
behaves as expected, which is the shape of test this whole effort exists to stop shipping.
|
||||
- **The gate is blocking, and green is the definition of done.** A change that has not run its
|
||||
tests is not finished, whatever its diff looks like.
|
||||
|
||||
A test names the decision it defends. Not as ceremony: it is what makes the pairing checkable,
|
||||
so a decision without one can be *found* rather than noticed.
|
||||
|
||||
## Consequences
|
||||
|
||||
- The question *"which tests matter"* has an answer that is not a number. The decisions are the
|
||||
list, and they are already written down.
|
||||
- **A new decision costs a test.** That is the intended friction — a decision nobody will assert
|
||||
is one worth reconsidering.
|
||||
- Integration tests need real infrastructure and are slow. That cost is accepted: a fast test
|
||||
suite that mocks the boundary would tell us nothing about the boundary, which is where every
|
||||
interesting fault in this session actually was.
|
||||
- Some decisions are not mechanically assertable — *the mesh brokers capabilities; nodes host;
|
||||
agents think* is a shape, not a predicate. Those should say so in the record rather than being
|
||||
quietly exempt, so the exemption is visible.
|
||||
- Records 0001–0033 were made before this rule. They are not retroactively invalid, but each
|
||||
should acquire a test or an explicit note that it cannot have one, and until then this rule
|
||||
is aspirational for them — which is exactly the state §5 warns about, recorded rather than
|
||||
hidden.
|
||||
|
||||
## References
|
||||
|
||||
- [`how-we-build.md`](../00-META/how-we-build.md) §5 — the rule this extends from prose to
|
||||
decisions.
|
||||
- [`04-ISSUES/005`](../04-ISSUES/005-pipeline-test-harness-unbuildable/00-report.md) — coverage
|
||||
assumed rather than checked, for two and a half months.
|
||||
- The sibling HQ repository for the PAPA platform states the same boundary rule — the contract
|
||||
is tested against the real system, mocking the client is forbidden, and a blocking gate is the
|
||||
definition of done. This record adopts that posture and adds the decision pairing.
|
||||
@@ -0,0 +1,99 @@
|
||||
---
|
||||
topic: checking it
|
||||
status: accepted
|
||||
date: 2026-08-24
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 0017-a-test-defends-a-decision.md
|
||||
---
|
||||
|
||||
# 18. A picture of a system is read from the system, never from what asked for it
|
||||
|
||||
## Context
|
||||
|
||||
A scenario declaration is a file. A raised scenario is a set of machines, links and rulesets.
|
||||
The two are supposed to correspond, and the entire value of the lab rests on noticing when
|
||||
they do not — [ADR 0017](0017-a-test-defends-a-decision.md) says a claim nothing checks is a
|
||||
claim that will quietly stop being true.
|
||||
|
||||
Drawing a scenario makes that concrete, and forces a choice that looks cosmetic and is not.
|
||||
A diagram of a running system can be produced two ways: parse the declaration and lay it out,
|
||||
or interrogate the system and lay *that* out. The first is far easier — the declaration is
|
||||
already parsed, already validated, already in memory.
|
||||
|
||||
It is also worthless for the only question worth asking of such a picture: *is what is running
|
||||
what I asked for?* A drawing built from the request and captioned **as raised** answers that
|
||||
question with the request, which always agrees with itself.
|
||||
|
||||
This is the same fault as
|
||||
[04-ISSUES/003](../04-ISSUES/003-firewall-scope-is-read-by-no-code/00-report.md) — a firewall key declared in
|
||||
five manifests and read by no code, so a manifest appears to restrict a port and restricts
|
||||
nothing. The declaration was never wrong. Nothing ever asked the system.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **Draw the declaration, and label it honestly.** Cheap, and useful for review before
|
||||
anything is raised. Insufficient alone: it can never disagree with itself.
|
||||
2. **Draw the system, inferring the rest from the declaration where the system is silent.**
|
||||
The tempting middle. Rejected — a picture where some facts are observed and some are
|
||||
assumed has no honest caption, and the assumed ones are exactly the interesting ones.
|
||||
3. **Two pictures, one layout, neither borrowing from the other.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
**A picture captioned *as raised* reads only the running system.** It never opens the
|
||||
declaration, not even for a fact the system happens not to record.
|
||||
|
||||
Where the hypervisor does not natively hold a fact the picture needs — whether a segment is
|
||||
public, what a gateway translates, whether a machine refuses inbound — **the raise records it
|
||||
on the resource** as metadata, and the picture reads it back from there.
|
||||
|
||||
That recording carries its own rule, which is the substance of this decision rather than an
|
||||
implementation note:
|
||||
|
||||
> **A behavioural tag is written after the behaviour works, never when the resource is
|
||||
> created.**
|
||||
|
||||
Written at creation, a tag restates the request in a new location and inherits none of the
|
||||
authority of having happened. A failed raise deliberately leaves its wreckage standing, so a
|
||||
tag written up front would let a picture of that wreckage badge translation the gateway was
|
||||
never configured to do — reproducing, inside the tool built to catch the fault, exactly the
|
||||
fault.
|
||||
|
||||
So: the gateway is tagged after its ruleset applies; the machine after the read-back proves
|
||||
its firewall loaded.
|
||||
|
||||
Both pictures render through **one layout**, so they can be put side by side and the
|
||||
difference read off directly.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **It earned itself on the first comparison.** Drawn side by side, every virtual machine in
|
||||
the live picture held no addresses at all. A container's interface carries the name of the
|
||||
device it was configured as; a virtual machine names its own — so joining addresses to
|
||||
devices by name attached every address to a container and none to a VM. Nothing failed;
|
||||
a whole class of machine silently lost its addresses. The two pictures disagreed, so it was
|
||||
visible in seconds. It is now joined on MAC.
|
||||
- Raise does more work, and writes metadata it does not itself consume. Accepted: the cost is
|
||||
a few config keys, and it is what makes a raised instance self-describing.
|
||||
- A resource raised before a tag existed is missing it. The reader says so rather than filling
|
||||
the gap from the declaration — an untagged link draws as unknown, not as what the file said
|
||||
it should be.
|
||||
- **The rule generalises past diagrams.** Anything reporting on the mesh — a status view, an
|
||||
inventory, a health check — is subject to it. A report assembled from the intended state is
|
||||
not a report.
|
||||
- The declared picture stays, and stays useful: it is review before raising, and it is one half
|
||||
of the comparison. It carries no runtime status, because it cannot know any.
|
||||
|
||||
- **The constitution sync is now owed.** Accepted 2026-08-25, so §5 carries the rule
|
||||
unqualified and playbook [05](../00-META/process/05-constitution-sync.md) is due. Until it
|
||||
runs, the mesh does not enforce this — an unsynced rule is a rule the mesh does not enforce,
|
||||
whatever this document says.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0017](0017-a-test-defends-a-decision.md) — a claim nothing checks stops being true.
|
||||
- [ADR 0016](0016-the-lab.md) — why the lab must not supply what the
|
||||
mesh is responsible for; the same instinct, applied to facts rather than to configuration.
|
||||
- [04-ISSUES/003](../04-ISSUES/003-firewall-scope-is-read-by-no-code/00-report.md) — the fault in production
|
||||
form.
|
||||
@@ -0,0 +1,132 @@
|
||||
---
|
||||
topic: how we work
|
||||
status: accepted
|
||||
date: 2026-08-28
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
---
|
||||
|
||||
# 19. How this repository works
|
||||
|
||||
*Consolidated 2026-08-28 from ten records that were one decision seen from ten angles. The
|
||||
reasoning is kept; the fragmentation is not.*
|
||||
|
||||
## The repository
|
||||
|
||||
**`novox/hq` is Novox's headquarters, and it is public.**
|
||||
|
||||
Company-scoped, not the mesh's. Today almost everything in it is about the mesh, because the
|
||||
mesh is what Novox is building — a fact about the present rather than a definition. A second
|
||||
product would live here too.
|
||||
|
||||
**Public** means written for a reader who is not its author and has no access to the mesh it
|
||||
describes. Nothing here may contain routable addresses, real domain names, node names, absolute
|
||||
paths, usernames or credentials. The test: *would this paragraph still teach a stranger running
|
||||
an entirely different mesh?*
|
||||
|
||||
**Separate from the code** because the cadence differs — a decision changes when thinking
|
||||
changes, not when code changes — and because a public repository cannot be a private one's
|
||||
subdirectory.
|
||||
|
||||
**The naming rule:** a repository belonging to a product carries that product's prefix. A
|
||||
company-scoped one does not. So this is `hq` and the mesh's are `mesh-*`.
|
||||
|
||||
| Repository | Tier | Holds |
|
||||
|---|---|---|
|
||||
| `novox/mesh-host` | 0 | the node host — the one binary installed by hand |
|
||||
| `novox/mesh-substrate` | 1 | the pinned tier-1 services, as declarations |
|
||||
| `novox/mesh-control` | 2 | the control plane and its contexts |
|
||||
| `novox/mesh-surfaces` | 3 | tools, web, cli — thin, no logic |
|
||||
| `novox/mesh-sdk` | — | the mesh's own domain ([ADR 0009](0009-modules-and-the-graph.md)) |
|
||||
| `novox/mesh-lab` | — | the lab: scenario lifecycle, networking, placement |
|
||||
| `novox/hq` | — | this one |
|
||||
|
||||
**The product is `Novox Mesh`**, shortened to `mesh` in internal use — repository names, the
|
||||
module namespace, environment variables, paths. **`Nox` is an identity of Novox**, an agent
|
||||
participant within the mesh's own model, not a second system.
|
||||
|
||||
## The folders, and why they are numbered
|
||||
|
||||
**The numbering is the flow.** Research produces a decision; the decision authorises a design.
|
||||
Following the numbers walks the process in the order it happens.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `01-RESEARCH` | an open question, while it is open |
|
||||
| `02-DECISIONS` | what was decided, and why |
|
||||
| `03-DESIGN` | what is being built |
|
||||
| `04-ISSUES` | something wrong at the level of design or governance |
|
||||
|
||||
**`03-DESIGN` has two layers and they are never mixed.** `00-as-is/` describes the mesh that
|
||||
exists, written from the implementation and the operational record. `01-to-be/` describes the one
|
||||
being built toward. Every document says which it is. A statement about the future does not belong
|
||||
in an as-is document, and an as-is document is never edited to describe an intention.
|
||||
|
||||
**`04-ISSUES` is for design-level faults** — a rule enforced by nothing, a stated invariant that
|
||||
is false, a failure the design permits to be silent. Not an operational ticket queue.
|
||||
|
||||
## What a decision record is, and is not
|
||||
|
||||
**If a decision is worth recording, it is worth a record. If it is not worth a record, it is not
|
||||
recorded.**
|
||||
|
||||
That bar has been read too generously. A *finding* is not a decision. A bug is not a decision.
|
||||
**A record is warranted when there is a genuine fork**: a direction reversed, an alternative
|
||||
seriously considered and likely to be proposed again, or something contested that needs to stay
|
||||
settled. Everything else belongs in the design document, where the reasoning is read.
|
||||
|
||||
**There is no ledger** — no separate document summarising, ranking or tracking decisions. A
|
||||
chronological view is generated from frontmatter, which is what a ledger was actually for.
|
||||
|
||||
**A number identifies a record and never changes.** It is not a position, and it cannot be
|
||||
both — a position moves when the set changes, and an identity that moves is not one.
|
||||
|
||||
That is not a preference. Records are referenced from **outside** this repository: code
|
||||
comments, commit messages, the knowledge base. Renumbering once cost 96 references across two
|
||||
code repositories, and nothing in either would have failed to compile — the comments would
|
||||
simply have pointed at the wrong reasoning, which is worse than a broken link because nothing
|
||||
reports it.
|
||||
|
||||
**So the reading order lives in a generated index**, from each record's `topic:` — what the mesh
|
||||
is, then its tiers from the bottom up, then what runs on them and how it gets there, then how it
|
||||
is built, how it is checked, and how we work.
|
||||
|
||||
**And the index is written, not only generated on demand.** A reader looking at the folder on a
|
||||
forge sees the folder, not a command. The objection to a written index is that it drifts, and
|
||||
that is answered by **checking** it rather than by refusing to write one — which is §5's own
|
||||
rule: a rule states how it is checked. A record with no topic, or a topic nobody defined, fails
|
||||
the same check, because the quiet failure is a record that vanishes from the order rather than
|
||||
appearing in the wrong place.
|
||||
|
||||
**The design layer is what you read.** These records explain *why* a thing is as it is. They are
|
||||
not a description of the system, and needing to read them to understand it would mean the design
|
||||
documents had failed.
|
||||
|
||||
## Status, and views over it
|
||||
|
||||
**Every document carries its state in YAML frontmatter** — research overviews, design documents,
|
||||
decision records, issue reports.
|
||||
|
||||
**There are no central status files.** Every cross-cutting view — a status matrix, a decision
|
||||
index, an open-issue list — is generated from frontmatter when asked for, never written to disk.
|
||||
Two places holding one fact drift, and the written one wins by being closer to hand.
|
||||
|
||||
**Prose does not restate status.** One place, and two is one too many.
|
||||
|
||||
## Workflows are playbooks
|
||||
|
||||
Every workflow is a playbook in [`00-META/process/`](../00-META/process/): trigger, who runs it,
|
||||
steps, outputs. People and agents follow the same ones, and **agents do not act outside them**.
|
||||
|
||||
Each is wrapped by a thin skill that defers to the playbook as authoritative and adds only the
|
||||
mechanical scaffolding — so the process has one definition rather than a document and an
|
||||
implementation that disagree.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **A reader has one place per thing.** The design layer describes the system; these records say
|
||||
why; the playbooks say how work is done.
|
||||
- **Records will accumulate more slowly**, because the bar is a fork rather than a finding. This
|
||||
record is itself the correction: ten records became one because they were one decision.
|
||||
- **The public rule constrains everything written here**, permanently and at every commit. It is
|
||||
the reason research describes real observations without identifying the mesh it observed.
|
||||
@@ -0,0 +1,64 @@
|
||||
---
|
||||
topic: how we work
|
||||
status: accepted
|
||||
date: 2026-07-10
|
||||
deciders: jochen
|
||||
reconstructed: true
|
||||
---
|
||||
|
||||
# 20. The mesh is governed by a constitution, injected where work is decided
|
||||
|
||||
> Reconstructed after the fact from the evidence cited below.
|
||||
|
||||
## Context
|
||||
|
||||
By mid-2026 the mesh was doing a large share of its own design and implementation work through
|
||||
agents. The rules those agents were expected to follow existed — in operating instructions, in
|
||||
convention documents, in the knowledge base — but they were **retrieved**: an agent had to know
|
||||
a rule existed in order to look it up.
|
||||
|
||||
Rules that must be looked up are followed by whoever already knows them, which is precisely the
|
||||
population that does not need them. The rules being violated were the ones nobody thought to
|
||||
search for.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **Documentation plus review.** Rejected — it is what existed. Review catches a violation
|
||||
after the work is done, and only if the reviewer knows the rule.
|
||||
2. **Lint and automated checks only.** Rejected as insufficient, not wrong. A check catches
|
||||
what can be expressed mechanically; most of these rules are about judgement — what belongs
|
||||
in a repository, when a criterion counts as verified.
|
||||
3. **A canonical rule set, injected into context wherever work is decided, with a check phase
|
||||
before output is accepted.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
A single canonical document states the mesh's non-negotiable rules. It is **injected
|
||||
proactively** into every eligible design and analysis session — agents do not fetch it, it
|
||||
arrives — and a check phase verifies the session's output against it before the work proceeds.
|
||||
|
||||
It is a governed document, not a page. Changing it requires a proposal, sign-off by reviewers
|
||||
who are not the proposer, and a recorded decision. Drive-by edits are reverted.
|
||||
|
||||
Scoped override pages may **tighten** it for a team or product. They may never relax it.
|
||||
|
||||
## Consequences
|
||||
|
||||
- A rule reaches the work whether or not anyone remembered it existed.
|
||||
- The check phase makes a violation a blocking outcome rather than a review comment.
|
||||
- Two copies of the same rules now exist: this document, and the reasoning in HQ that earned
|
||||
them. The enforced copy wins by default, so the reasoned copy quietly stops being true —
|
||||
which is why [`00-META/how-we-build.md`](../00-META/how-we-build.md) is now the source
|
||||
and the governed page is derived from it, via playbook
|
||||
[`05-constitution-sync.md`](../00-META/process/05-constitution-sync.md).
|
||||
- Injection costs context on every eligible turn, and grows with the document. Nothing
|
||||
currently bounds that.
|
||||
- The amendment process requires two reviewers, which a mesh with one human operator satisfies
|
||||
only by counting agents. That tension is real and unresolved.
|
||||
|
||||
## References
|
||||
|
||||
- The governed page was authored 2026-07-10 and carries its own amendment process.
|
||||
- `feat(noxflow): HAL architectural conformance gate for reviewer + architect` (#296),
|
||||
2026-06-10 — the check phase, predating the document it checks against.
|
||||
- Knowledge base: `platform/constitution`.
|
||||
@@ -0,0 +1,68 @@
|
||||
---
|
||||
topic: how we work
|
||||
status: accepted
|
||||
date: 2026-08-23
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
---
|
||||
|
||||
# 21. HQ is the source of the mesh constitution
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0020](0020-the-mesh-is-governed-by-a-constitution.md) established a canonical rule set,
|
||||
injected into every eligible design session and checked before output is accepted. It lives in
|
||||
the knowledge base, where the orchestrator reads it.
|
||||
|
||||
HQ separately carried a document stating overlapping rules with the reasoning that earned each
|
||||
one. Two texts, one enforced and one not.
|
||||
|
||||
That arrangement has a predictable outcome and it is not a tie. The enforced copy wins by
|
||||
default, because it is the one that blocks work. The reasoned copy quietly stops being true,
|
||||
and the rules survive without the incidents that justify them — at which point a rule reads as
|
||||
arbitrary, and an arbitrary rule is the kind people route around.
|
||||
|
||||
## Considered options
|
||||
|
||||
1. **The knowledge-base page is the source; HQ points at it.** Rejected, though it is the
|
||||
honest description of what was already happening. It leaves the reasoning downstream of the
|
||||
rule, and the reasoning is the part that makes a rule survive a challenge.
|
||||
2. **Accept the overlap and let both stand.** Rejected: two authorities is no authority, and
|
||||
the drift is silent.
|
||||
3. **HQ is the source; the governed page is derived and published from it.** Chosen.
|
||||
|
||||
## Decision
|
||||
|
||||
[`00-META/how-we-build.md`](../00-META/how-we-build.md) is the source. The governed page the
|
||||
mesh injects is **derived** from it — the rules without the reasoning — and is never edited
|
||||
directly.
|
||||
|
||||
Publishing is a playbook step, not a manual act, and it ends with **reading the page back and
|
||||
verifying the change is present**. A publish that reported success and did nothing is exactly
|
||||
the failure class this mesh keeps producing
|
||||
([ADR 0010](0010-delivery.md)).
|
||||
|
||||
Section numbering is stable, because the orchestrator and the review fragments cite sections by
|
||||
number.
|
||||
|
||||
## Consequences
|
||||
|
||||
- One source, many surfaces — the same argument HQ's separation already rests on
|
||||
([ADR 0019](0019-how-this-repository-works.md)), applied to the rules themselves.
|
||||
- Each rule keeps the incident that earned it, in a place that is reviewed as a diff.
|
||||
- An edit to the derived page survives until the next sync and then vanishes. The playbook says
|
||||
so, and nothing mechanically prevents it.
|
||||
- **The sync is manual and is the weak point.** An unsynced rule is a rule the mesh does not
|
||||
enforce, whatever the source says — so the playbook requires the failure to be stated rather
|
||||
than passed over. This is the same class of gap as
|
||||
[`04-ISSUES/006`](../04-ISSUES/006-hq-is-not-indexed-into-the-knowledge-base/00-report.md),
|
||||
and it is worth watching for the same reason.
|
||||
- The document grew from four rules to seven sections, because it now has to carry everything
|
||||
the mesh enforces rather than only what someone thought to write down.
|
||||
|
||||
## References
|
||||
|
||||
- [`00-META/process/05-constitution-sync.md`](../00-META/process/05-constitution-sync.md) —
|
||||
the sync, including the read-back.
|
||||
- [ADR 0020](0020-the-mesh-is-governed-by-a-constitution.md) — the governed page and why it
|
||||
exists.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user