Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 jochen
|
26ff767a7b | ||
|
jochen
|
4fed95b3d7 | ||
|
jochen
|
e2ff68004e |
@@ -23,13 +23,10 @@ and says they disagree.
|
||||
|
||||
## Amending the to-be layer
|
||||
|
||||
1. Propose the decision record in the knowledge base (`hq-graduate`, steps 2 and 3). If it reverses an
|
||||
earlier one, it `supersedes` it, or `changes` the clauses it replaces — the earlier record's text is
|
||||
never edited.
|
||||
2. Edit the design, add `"ADR NNNN"` to its `decisions:`, and set `updated:` to today.
|
||||
3. If this came from an issue, give the issue the document in the issue tracker: `amended-design` on its
|
||||
`issue-tracker.move` to `resolved`, or `issue-tracker.link` with kind `design` before then. Never in the
|
||||
report's frontmatter (ADR 0275).
|
||||
1. Write the decision record. If it reverses an earlier one, that record gets
|
||||
`status: superseded` and `superseded-by:` — its text is never edited.
|
||||
2. Edit the design; set `updated:` to today.
|
||||
3. If this came from an issue, set that issue's `amended-design:`.
|
||||
|
||||
## Amending the as-is layer
|
||||
|
||||
|
||||
@@ -1,44 +0,0 @@
|
||||
---
|
||||
name: hq-cleanup
|
||||
description: Use when the operator asks to clean up the forge and the working copies — stale or merged branches, old pull requests, finished worktrees, leftover folders, stashes. Triggers on "cleanup", "clean up our repos", "stale branches", "old worktrees", "tidy the forge". Reads first, deletes only what the operator approves, and reports what it cleaned, kept and left.
|
||||
---
|
||||
|
||||
# hq-cleanup
|
||||
|
||||
Runs playbook [09 — Cleanup](../../../00-META/process/09-cleanup.md); the playbook is authoritative. This skill
|
||||
adds the mechanics.
|
||||
|
||||
## 1. Take stock (one read-only agent, smaller model)
|
||||
|
||||
Brief it to change nothing and to write one report to the session's scratchpad:
|
||||
|
||||
- **Forge.** `gitea.gitea_list_repos`; per repository its default branch and **every branch from the git refs**
|
||||
(`gitea.gitea_api` GET `/repos/{owner}/{repo}/git/refs/heads`; the list tool misses branches, issue 949);
|
||||
`gitea.gitea_branch_protection_get`; open pull requests (`gitea.gitea_open_pulls`, titles and update times
|
||||
from `gitea.gitea_api` `/repos/{owner}/{repo}/pulls?state=open`). Per branch: contained in the default branch
|
||||
(`gitea.gitea_contains`, with the default branch's sha), patch-equal (`git cherry` in a local clone), tip date.
|
||||
- **Local.** Under the projects folder: each clone's `git worktree list --porcelain`, `git branch -vv`,
|
||||
`git branch --merged origin/<default>`, `git stash list`, `git status --porcelain` per worktree; folders and
|
||||
loose files beside the clones with `du -sh` and newest file. A worktree under another session's scratchpad is
|
||||
Kept.
|
||||
- A **Missing tools** section: each read it did without a mesh tool.
|
||||
|
||||
## 2. Classify and ask
|
||||
|
||||
Classes as playbook 09 step 2: Safe, Likely safe, Kept, Unknown. Tell the operator the counts and the report's
|
||||
place, and ask one question: delete the Safe list? Likely-safe and Unknown items are asked separately.
|
||||
|
||||
## 3. Delete the approved list only
|
||||
|
||||
- Forge branch: `gitea.gitea_delete_branch`.
|
||||
- Worktree: `git worktree remove <path>` (never `--force`), then `git worktree prune`.
|
||||
- Local branch: `git branch -d <name>` (never `-D`).
|
||||
Once design 58 milestone 7 is built (issue 884), each deletion is a call held as an ask: the operator approves
|
||||
the exact list on Telegram and the mesh carries it out on the warrant. Until then a session's permission check
|
||||
refuses these; hand the operator the approved list as a file, with the commands, and do not work around the
|
||||
refusal.
|
||||
|
||||
## 4. Report
|
||||
|
||||
Four lists: **cleaned** (with its evidence), **kept** (why), **unknown** (what it looks like), **the operator
|
||||
kept** (their word). Open each missing tool with `issue-tracker.open`, or note it on an existing issue.
|
||||
@@ -18,7 +18,7 @@ Record it as **one memory file** in Claude's persistent memory directory for thi
|
||||
path is given in the session's memory instructions), with `metadata.type: project`, plus a
|
||||
one-line pointer in `MEMORY.md`.
|
||||
|
||||
**Not** in `04-ISSUES`, the knowledge base, or anywhere else in the repository:
|
||||
**Not** in `04-ISSUES`, `01-RESEARCH` or anywhere else in the repository:
|
||||
|
||||
- A parked thought is not an issue or a research effort. Giving it a number asserts it has been
|
||||
triaged, which is exactly what deferring says has not happened.
|
||||
|
||||
@@ -23,19 +23,14 @@ result is not "nothing to learn" — it is the reason the next person repeats th
|
||||
|
||||
## Steps
|
||||
|
||||
1. Read the issue: `mesh_call issue-tracker.get` with its number, and `00-report.md` when it has a
|
||||
folder. Before you start, move it to `diagnosing` with `issue-tracker.move` and `by`, your name and node as
|
||||
who@node (such as `fix-399@laptop`). The tracker refuses it while another worker is current, naming them.
|
||||
2. Investigate. The trail goes in `issue-tracker.note`, **dated** in its text, including what was
|
||||
ruled out and how; for an issue with a folder, in `01-diagnosis.md` there as well. Archaeology —
|
||||
which commit, which pull request, which date a behaviour changed — is the most valuable content.
|
||||
3. When the owner is known, `issue-tracker.move` to `located` with `located-in`: entries of the
|
||||
tracker's registered list of owners (repositories, and modules as `mesh-catalog modules/<module>`);
|
||||
the tracker refuses an owner it does not know.
|
||||
4. Before you start the fix: `issue-tracker.move` to `fixing` with `by`.
|
||||
5. On resolution: `issue-tracker.move` to `resolved`, taken only from `fixing`, with `fixed-by`, and `replay` or `replay-none`
|
||||
for a core issue. If the root cause was a design gap, run `hq-graduate` for the amendment and give
|
||||
`amended-design`. Nothing of this is written in the report's frontmatter.
|
||||
1. Read `00-report.md`. Move `status:` to `diagnosing`.
|
||||
2. Investigate. Write `01-diagnosis.md` in the same folder: the trail, **dated**, including
|
||||
what was ruled out and how. Archaeology — which commit, which pull request, which date a
|
||||
behaviour changed — is the most valuable content here.
|
||||
3. When the owner is known, set `status: located` and fill `located-in:` with repositories or
|
||||
modules from [`00-META/repos.md`](../../../00-META/repos.md).
|
||||
4. On resolution: `status: resolved`, fill `fixed-by:`. If the root cause was a design gap, run
|
||||
`hq-graduate` for the amendment and fill `amended-design:`.
|
||||
|
||||
## Rules
|
||||
|
||||
|
||||
@@ -8,34 +8,25 @@ description: Use when a research effort in HQ concludes and becomes design, or w
|
||||
Closes a research effort into a decision and a design, or amends an existing design.
|
||||
**Authoritative playbook:**
|
||||
[`00-META/process/02-graduation.md`](../../../00-META/process/02-graduation.md) — read
|
||||
it; this skill adds only the scaffolding. Decision records and research efforts live whole in the
|
||||
knowledge base (seat `knowledge-base`, ADR 0290), reached through the mesh MCP server; nothing of them is
|
||||
written into this repository.
|
||||
it; this skill adds only the scaffolding.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Check against GENESIS** — `mission.md`, `context.md`, `effect.md`. If the conclusion
|
||||
conflicts, say which is wrong, in writing, before proceeding.
|
||||
2. **Reserve the record's number.** `mesh_call knowledge-base.reserve {"kind": "decision", "for": "<title>"}`.
|
||||
3. **Propose the decision record.** `mesh_call knowledge-base.propose` with the reserved `number`, `title`,
|
||||
`topic` (one of the six in [`02-DECISIONS/README.md`](../../../02-DECISIONS/README.md)), `date`,
|
||||
`deciders`, `clauses` (each `"**Lead.** the statement"`), `sections` (Context, Considered Options,
|
||||
Consequences, How it is checked, References), `links` (`extends`, `rests-on`, `supersedes`, `changes` with
|
||||
`"NNNN §N"`, `cites`), `homes` (the design's path), and `needs-operator: true` when the operator must
|
||||
approve it. A refusal names every rule broken: fix them all and propose again. The knowledge base
|
||||
reviews it itself; on `changes` revise and propose again with `why`. `knowledge-base.get` shows its
|
||||
status, reviews and the operator's ask.
|
||||
4. **Write the design** under `03-DESIGN/01-to-be/` with `layer: to-be`, `status: designed`,
|
||||
empty `code:`, today's `updated:`, and `decisions:` listing `"ADR NNNN"`.
|
||||
5. **Close the effort** — `knowledge-base.propose` the effort again, whole, with `status: graduated` and
|
||||
`became` links to the record (`"NNNN"`) and the design (its path).
|
||||
2. **Write the decision record** in `02-DECISIONS/`, next free number, frontmatter per
|
||||
[`02-DECISIONS/README.md`](../../../02-DECISIONS/README.md). `reconstructed: false` — this one is being taken,
|
||||
not recovered. Record the rejected options.
|
||||
3. **Write the design** under `03-DESIGN/01-to-be/` with `layer: to-be`, `status: designed`,
|
||||
empty `code:`, today's `updated:`, and `decisions:` pointing at the record.
|
||||
4. **Close the effort** — the research `00-overview.md` gets `status: graduated` and `became:`
|
||||
pointing at both.
|
||||
|
||||
## Amending an existing design
|
||||
|
||||
A design changes only through a decision. Propose the record first; if it reverses an earlier one,
|
||||
it `supersedes` it (or `changes` the clauses it replaces) — **never edit the earlier record's text**;
|
||||
a correction of fact is a progressive insight (`propose` the accepted record with `insight`). Then edit
|
||||
the design, add `"ADR NNNN"` to its `decisions:`, and set `updated:`.
|
||||
A design changes only through a decision. Write the record first; if it reverses an earlier
|
||||
one, set that record's `status: superseded` and `superseded-by:` — **never edit its text**.
|
||||
Then edit the design and set `updated:`.
|
||||
|
||||
## When something ships
|
||||
|
||||
@@ -51,4 +42,3 @@ cannot be checked, it is `in-progress`.
|
||||
- Do not move a to-be document into `00-as-is/`.
|
||||
- Do not edit an as-is document to describe an intention.
|
||||
- Do not change a decision record's meaning. Supersede it.
|
||||
- Do not write a decision record, or a copy of one, into this repository.
|
||||
|
||||
@@ -10,8 +10,8 @@ Hands a settled design to a code repository. **Authoritative playbook:**
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Confirm it is settled.** `status: designed`, and every claim traceable to a decision record in the knowledge base
|
||||
(`knowledge-base.get` for each entry of its `decisions:`). An open question in the text is a reason to run `hq-new-research`, not to build
|
||||
1. **Confirm it is settled.** `status: designed`, and every claim traceable to a record in
|
||||
`02-DECISIONS/`. An open question in the text is a reason to run `hq-new-research`, not to build
|
||||
around it.
|
||||
2. **Name the owner.** Set `code:` from
|
||||
[`00-META/repos.md`](../../../00-META/repos.md). If the repository does not exist yet,
|
||||
|
||||
@@ -21,46 +21,27 @@ Opens a numbered issue. **Authoritative playbook:**
|
||||
there, this is not an issue — it is a lookup. If the answer is a general lesson, it belongs in
|
||||
both.
|
||||
|
||||
## The number comes from the issue tracker
|
||||
|
||||
Every issue is opened in the issue tracker (seat `issue-tracker`, module `mesh-issues`, ADR 0275). **Never take a
|
||||
number by reading `main` or the open pull requests** — that is how numbers collided. If the tracker cannot be
|
||||
reached, take no number and say so.
|
||||
|
||||
Decide the shape first:
|
||||
|
||||
- **Design or governance issue** — its narrative belongs in hq: it gets a folder `04-ISSUES/NNN-short-name/`.
|
||||
- **Anything else** — it lives wholly in the tracker, and nothing is written in hq.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Load `mesh_call` and `mesh_describe` (deferred tools of the `mesh` MCP server) and open the issue:
|
||||
`mesh_call issue-tracker.open` with `title` (the symptom in one line), `report` (what was seen, what was expected,
|
||||
how it was found — the evidence: what was run, what came back, when), optionally `located-in` when the owner is
|
||||
already known, and — **only for an issue with a folder** — `record: short-name` (lower-case words joined by
|
||||
hyphens, no number). The answer carries the number `NNN`.
|
||||
2. **Issue wholly in the tracker:** done. Its trail goes in `issue-tracker.note`, its status changes through
|
||||
`issue-tracker.move`, its links through `issue-tracker.link`.
|
||||
3. **Issue with a folder:** on a branch off `origin/main`, create `04-ISSUES/NNN-short-name/00-report.md`, `NNN` being
|
||||
exactly the tracker's number and `short-name` exactly the `record` passed:
|
||||
1. Next free number. Create `04-ISSUES/NNN-short-name/00-report.md` with:
|
||||
|
||||
```yaml
|
||||
---
|
||||
status: open
|
||||
opened: YYYY-MM-DD
|
||||
located-in: []
|
||||
fixed-by:
|
||||
amended-design:
|
||||
---
|
||||
```
|
||||
|
||||
then `# NNN. The title`, the title given to the tracker. **No status key**: the issue's status, owner and fix
|
||||
live in the tracker (ADR 0275), and the cycle test refuses `status:`, `located-in:`, `fixed-by:`,
|
||||
`amended-design:`, `replay:` or `replay-none:` in any report. Notes and links go to the
|
||||
tracker.
|
||||
4. Write the symptom **as observed**, in plain terms, with the evidence that it happened.
|
||||
5. Say why it matters beyond the instance. An issue that is only one occurrence is a knowledge
|
||||
2. Write the symptom **as observed**, in plain terms, with the evidence that it happened —
|
||||
what was run, what came back, when.
|
||||
3. Say why it matters beyond the instance. An issue that is only one occurrence is a knowledge
|
||||
base entry.
|
||||
6. End with open questions rather than a proposed fix. Diagnosis is a separate step.
|
||||
4. End with open questions rather than a proposed fix. Diagnosis is a separate step.
|
||||
|
||||
## Do not
|
||||
|
||||
- Do not name nodes, domains, addresses or paths. This repository is public.
|
||||
- Do not guess the owner — `located-in:` is filled by diagnosis, not by opening.
|
||||
- Do not renumber a folder or reuse a number: a number never moves.
|
||||
|
||||
@@ -5,37 +5,38 @@ description: Use when starting a new research effort in HQ — an idea, technolo
|
||||
|
||||
# hq-new-research
|
||||
|
||||
Starts a new research effort in the knowledge base. **Authoritative playbook:**
|
||||
Scaffolds a new research effort. **Authoritative playbook:**
|
||||
[`00-META/process/01-research.md`](../../../00-META/process/01-research.md) — read it;
|
||||
this skill only does the mechanical setup. A research effort lives whole in the knowledge base (seat
|
||||
`knowledge-base`, ADR 0290); nothing of it is written into this repository.
|
||||
this skill only does the mechanical setup.
|
||||
|
||||
## Steps
|
||||
|
||||
All through the mesh MCP server (`mesh_describe` gives each verb's arguments).
|
||||
1. Find the next free number: list `01-RESEARCH/NNN-*`, take the highest plus one, zero-padded
|
||||
to three digits.
|
||||
2. Create `01-RESEARCH/NNN-descriptive-name/` (kebab-case from the topic).
|
||||
3. Create `00-overview.md` in it with exactly this frontmatter, then a prose summary of **what** is
|
||||
being investigated, **why**, and **what it touches**:
|
||||
|
||||
1. **Read what is known.** `mesh_call knowledge-base.ask {"question": "<the question in plain words>"}`;
|
||||
read an effort or record it names with `knowledge-base.get` (`kind: research` for an effort,
|
||||
`reasoning: true` for its documents).
|
||||
2. **Reserve the number.** `mesh_call knowledge-base.reserve {"kind": "research", "for": "<title>"}`. The
|
||||
number is spent; never pick one by hand.
|
||||
3. **Propose the effort.** `mesh_call knowledge-base.propose` with `kind: research`, the reserved `number`,
|
||||
`title`, `status: active`, `date` (today), `sections` (the documents in order, each `name` and `text`;
|
||||
the first `00-overview.md`: **what** is investigated, **why**, and **what it touches**), and `links`
|
||||
of kind `touches` for every record, effort, issue or design it touches (`cites` for one it only names).
|
||||
4. **Do the research** in further documents of the same effort. Each change is `propose` again with the
|
||||
effort whole and `why`. A refusal names every rule broken: fix them all and propose again. On review
|
||||
`changes`, revise and propose again; `knowledge-base.get` shows the reviews.
|
||||
```yaml
|
||||
---
|
||||
status: active
|
||||
initiated: YYYY-MM-DD
|
||||
touches: []
|
||||
became: []
|
||||
---
|
||||
```
|
||||
|
||||
4. Do the research in further documents in the same folder. Keep the summary current.
|
||||
|
||||
## Do not
|
||||
|
||||
- Do not restate the status in prose. It is the effort's `status`, in one place.
|
||||
- Do not give a `became` link while the effort is open — `hq-graduate` gives it at closure.
|
||||
- Do not restate the status in prose. It lives in frontmatter, in one place.
|
||||
- Do not fill `became:` while the effort is open — `hq-graduate` sets it at closure.
|
||||
- Do not skip or reuse a sequence number.
|
||||
- Do not write into `03-DESIGN` from an open effort.
|
||||
- Do not write the effort, or a copy of it, into this repository.
|
||||
- Do not name the mesh being observed. Evidence is required; identification is forbidden.
|
||||
|
||||
## Closing
|
||||
|
||||
An effort never just stops. It closes through `hq-graduate` as `graduated` or `abandoned`,
|
||||
always with a `became` link to what it turned into. Nothing is deleted.
|
||||
always with `became:` pointing at what it turned into. Nothing is deleted.
|
||||
|
||||
@@ -5,7 +5,7 @@ description: Use when you need a cross-cutting view of where HQ stands — resea
|
||||
|
||||
# hq-status
|
||||
|
||||
Generates a cross-cutting view **from frontmatter, the knowledge base and the issue tracker**. This is a read-and-render skill, not a
|
||||
Generates a cross-cutting view **from frontmatter**. This is a read-and-render skill, not a
|
||||
workflow — HQ has **no central status file by design** (decision 35). Every view is
|
||||
generated on demand and never written back to disk.
|
||||
|
||||
@@ -13,29 +13,28 @@ generated on demand and never written back to disk.
|
||||
|
||||
| Section | Files | Frontmatter |
|
||||
|---|---|---|
|
||||
| Research | the knowledge base, not a file: `knowledge-base.ask`, and `knowledge-base.get` with `kind: research` and a number | `status` (`active` / `graduated` / `abandoned`), `date`, links (`became`, `touches`, `cites`) |
|
||||
| Research | `01-RESEARCH/NNN-*/00-overview.md` | `status` (`active` / `graduated` / `abandoned`), `initiated`, `touches`, `became` |
|
||||
| Design | `03-DESIGN/**/*.md` (not READMEs) | `layer` (`as-is` / `to-be`), `status` (`designed` / `in-progress` / `implemented` / `abandoned`), `code`, `updated`, `decisions` |
|
||||
| Decisions | the knowledge base, not a file: `knowledge-base.holds` with a topic (the clauses in force, with their records), `knowledge-base.get` with a number, `knowledge-base.links` | `status` (`proposed` / `accepted` / `superseded`), `topic`, `date`, `deciders`, each clause's status, links both ways, what changed it since |
|
||||
| Issues | the issue tracker, not a file: `mesh_call issue-tracker.list` (optionally `status`, `owner`, `phrase`) and `issue-tracker.get` | `status` (`open` / `diagnosing` / `located` / `fixing` / `resolved` / `wontfix`), `worker` and `stale` while `diagnosing` or `fixing`, `opened`, `located-in`, `fixed-by`, `amended-design`, `record` (the folder here, when it has one) |
|
||||
| Decisions | `02-DECISIONS/NNNN-*.md` | `status` (`proposed` / `accepted` / `superseded`), `date`, `deciders`, `reconstructed`, `superseded-by`, `extends` |
|
||||
| Issues | `04-ISSUES/NNN-*/00-report.md` | `status` (`open` / `diagnosing` / `located` / `resolved` / `wontfix`), `opened`, `located-in`, `fixed-by`, `amended-design` |
|
||||
|
||||
## Steps
|
||||
|
||||
1. Read the designs' frontmatter (a grep across `03-DESIGN/` is enough, no need to load bodies), the
|
||||
issues from the tracker, and the records from the knowledge base. An issue report carries no status;
|
||||
where an issue stands is only in the tracker (ADR 0275). A decision record's or research effort's
|
||||
status is only in the knowledge base (ADR 0290).
|
||||
2. Render what was asked as Markdown tables. Group and sort sensibly. The **ADR index** by topic is
|
||||
`knowledge-base.holds` with each topic of the reading order `02-DECISIONS/README.md` writes, in that
|
||||
order, with a `bound` large enough for the topic (it says what it cut); it is the only place the list is
|
||||
shown (ADR 0248). The knowledge base has no verb that lists every record by number: a view by number
|
||||
is built from `get` per number, and says which numbers it read.
|
||||
1. Read the frontmatter block from every file above — a grep across each tree is enough, no
|
||||
need to load bodies.
|
||||
2. Render what was asked as Markdown tables. Group and sort sensibly. The **ADR index** is one
|
||||
of these views, and it is the only place the list of records is shown
|
||||
([ADR 0248](../../../02-DECISIONS/0248-the-decision-index-is-generated-when-it-is-read-and-never-stored.md)):
|
||||
run `python3 00-META/checks/index.py --print` for the records grouped by topic in the reading
|
||||
order `02-DECISIONS/README.md` writes, each with its status when not `accepted`. Asked for it by
|
||||
number instead, render records ordered by number, with title, date and status, and reconstructed
|
||||
ones marked. Off this checkout, the records module's `records_decisions` answers the same list.
|
||||
3. Flag anything inconsistent at the end, as flags — do not silently correct the render:
|
||||
- a `graduated` or `abandoned` effort with no `became` link
|
||||
- a `graduated` or `abandoned` effort with an empty `became:`
|
||||
- an `implemented` design with an empty `code:`
|
||||
- a `superseded` record nothing `supersedes`
|
||||
- a `superseded` record with no `superseded-by:`
|
||||
- an `open` or `diagnosing` issue with no activity
|
||||
- a to-be design whose `decisions:` names a record `knowledge-base.get` does not find, or one that is
|
||||
not `accepted`
|
||||
- a to-be design whose `decisions:` points at a record that does not exist
|
||||
- an as-is design whose claims are older than its `updated:` date suggests
|
||||
|
||||
## Do not
|
||||
@@ -44,6 +43,5 @@ generated on demand and never written back to disk.
|
||||
always generated, always ephemeral. This includes the ADR index — the hand-written one had
|
||||
already drifted after a single addition, which is why it was removed, and the generated one
|
||||
written into `02-DECISIONS/README.md` made every two open decisions conflict (issue 297), which
|
||||
is why that went too. the index test fails if the list is stored there again.
|
||||
- Do not infer status from prose. Trust only a design's frontmatter, the tracker for an issue and the
|
||||
knowledge base for a record; if one is wrong, flag it.
|
||||
is why that went too. `index.py` fails if the list is stored there again.
|
||||
- Do not infer status from prose. Trust only the frontmatter; if it is wrong, flag it.
|
||||
|
||||
@@ -27,9 +27,7 @@ the claim "HQ is the source" is checked.
|
||||
3. Publish it to the knowledge base under the constitution slug, replacing the body.
|
||||
4. **Read it back and verify the change is present.** A publish that reported success and did
|
||||
nothing is exactly the failure class this repository exists to name — do not skip this.
|
||||
5. Note the sync in the decision record's Consequences (in the knowledge base: while the record is
|
||||
proposed, `knowledge-base.propose` it again with the note and `why`; once it is accepted, `propose` it with
|
||||
`insight`, saying what stood before).
|
||||
5. Note the sync in the decision record's Consequences.
|
||||
|
||||
## Do not
|
||||
|
||||
|
||||
+4
-4
@@ -15,7 +15,7 @@ works — plus the engineering practice that holds across everything Novox build
|
||||
|
||||
## Rules
|
||||
|
||||
- Markdown only, but for the checks and one drawn overview of the playbooks (`process/playbooks.html`).
|
||||
- Markdown only.
|
||||
- **Stable by nature.** Changes here reflect a genuine shift in intent, not iteration. The one
|
||||
exception is `how-we-build.md`, which changes whenever a rule is earned — and only through
|
||||
its amendment process.
|
||||
@@ -35,7 +35,7 @@ comparing it against the code rather than by anyone noticing:
|
||||
|
||||
- It described the pipeline as having a separate builder process and a build stage that
|
||||
packages. Neither was true after 2026-08-04; the documents stayed stale until 2026-08-06
|
||||
(ADR 0010).
|
||||
([ADR 0010](../02-DECISIONS/0010-delivery.md)).
|
||||
- It listed the mesh as spanning a fixed number of named machines, which is exactly the
|
||||
content this repository cannot carry.
|
||||
|
||||
@@ -45,10 +45,10 @@ symlinks at all — the rule is not merely "only the installer may link", and a
|
||||
elevating linking to a principle points the opposite way from where this is going.
|
||||
|
||||
What exists today is that the installer owns and reconciles every link
|
||||
(ADR 0012) — an as-is fact, recorded in
|
||||
([ADR 0012](../02-DECISIONS/0012-the-mesh-creates-no-symlinks.md)) — an as-is fact, recorded in
|
||||
[`03-DESIGN/00-as-is/05-runtime-and-installation.md`](../03-DESIGN/00-as-is/05-runtime-and-installation.md).
|
||||
Centralising who may link narrowed the incident class; it did not close it. The intent is to
|
||||
remove the mechanism, recorded as ADR 0012.
|
||||
remove the mechanism, recorded as [ADR 0012](../02-DECISIONS/0012-the-mesh-creates-no-symlinks.md).
|
||||
|
||||
A founding document contradicting the direction of travel is precisely the failure this folder
|
||||
exists to prevent.
|
||||
|
||||
+41
-76
@@ -1,19 +1,14 @@
|
||||
# Checks
|
||||
|
||||
The four checks are Go tests (ADR 0527), in this folder (`*_test.go`) with `go.mod` at the root: run
|
||||
`go test ./...` from the root. The checker runs the same as `mesh/check` on every pull request. Each check is a
|
||||
function over a file tree that returns its problems, tested on hq's tree and on a deliberately broken tree built
|
||||
in memory. The Python files and `merge-check.sh` are gone (issue 1296).
|
||||
|
||||
```
|
||||
records_test.go structure: links, no record written here, citations in the form "ADR NNNN"
|
||||
index_test.go the reading order's topics are written, and no list of records is stored
|
||||
cycle_test.go the development cycle: a design's layer, status, decisions and code; an issue report's frontmatter
|
||||
words_test.go the glossary's retired words are not used, no daemon's line is written, and no word is defined twice
|
||||
python3 00-META/checks/records.py structure: links, citations, supersession, topics
|
||||
python3 00-META/checks/index.py every record's topic is in the reading order, and no list of records is stored
|
||||
python3 00-META/checks/index.py --print the records, by topic, in reading order (generated, never written)
|
||||
python3 00-META/checks/words.py the glossary's retired words are not used, and no word is defined twice
|
||||
python3 00-META/checks/words.py --list tools the words the catalogue's copy must list
|
||||
```
|
||||
|
||||
A failing test fails `go test`, so it can be a gate rather than a report. With `MESH_CATALOG_DIR` set, the words
|
||||
test also compares the catalogue's copies of the retired words and the daemons' lines with the glossary.
|
||||
Non-zero exit on any problem, so it can be a gate rather than a report.
|
||||
|
||||
**Why this exists.** Until now nothing in this repository was verified by anything but reading,
|
||||
which is how a superseded decision stayed live in the constitution for days and in
|
||||
@@ -27,89 +22,59 @@ indistinguishable from one that cannot.
|
||||
| Check | Asserts | Found |
|
||||
|---|---|---|
|
||||
| `links` | every relative link resolves | — (run ad hoc during authoring; now permanent) |
|
||||
| `no-records` | `02-DECISIONS/` and `01-RESEARCH/` hold their README and nothing else: a decision record or research effort lives in the knowledge base (ADR 0290 rule 9) | written with the change that removed the records (issue 426); it failed on a record file put back in `02-DECISIONS/` before it passed |
|
||||
| `decisions-form` | every entry of a design's `decisions:` is "ADR NNNN", four digits | written with issue 426; it failed on an entry `ADR 290` before it passed |
|
||||
| `mention-form` | every "ADR N" in the text has four digits, and every "research N" three; a code span is an example and is not read | written with issue 426; it failed on `ADR 12` and `research 45` before it passed |
|
||||
| *(index test)* | the reading order's six topics are written in `02-DECISIONS/README.md`, none twice, and the README stores no list of records (ADR 0248). The knowledge base reads these lines and refuses a record whose topic is not one of them | a stored list made every two open decisions conflict ([issue 297](../../04-ISSUES/297-two-open-decisions-always-conflict/00-report.md)); it failed on the README of `main` before the list was removed |
|
||||
| `rests-on` | `decisions:` and `extends:` name records that exist and are **accepted** | the class behind both incidents |
|
||||
| `live-citation` | a governing document citing a **superseded** record names its replacement in the same paragraph | `01-to-be/README.md` citing ADR 0022 as live guidance |
|
||||
| `supersession` | if A says it was superseded by B, B says it supersedes A | ADR 0012 never declared that it superseded 0011 |
|
||||
| `numbering` | the number in the filename is the number in the heading | — |
|
||||
| `topics` | every record names a topic of the reading order in `02-DECISIONS/README.md` | — |
|
||||
| *(index.py)* | the reading order is written, every record's topic is one of it, and the README stores no list of records ([ADR 0248](../../02-DECISIONS/0248-the-decision-index-is-generated-when-it-is-read-and-never-stored.md)) | a stored list made every two open decisions conflict ([issue 297](../../04-ISSUES/297-two-open-decisions-always-conflict/00-report.md)); it failed on the README of `main` before the list was removed |
|
||||
| `status-vs-code` | a to-be document naming specific code is not still `designed` | **ten documents**, several with a *What was built* section, describing lab-proven code |
|
||||
|
||||
## Where the rules on records went
|
||||
|
||||
The decision records and research efforts left this repository for the knowledge base (ADR 0290, issue 426),
|
||||
and the rules these checks held over them left with them. The knowledge base holds each as a rule of its own,
|
||||
named in its refusal: `knowledge-base.propose` refuses a record that breaks one of the rules that judge one
|
||||
record, naming every reason at once, and the four rules that judge the whole record set run after every
|
||||
acceptance and raise a condition when one fails (mesh-knowledge's `rules.go`, one table-test case per rule).
|
||||
|
||||
| Rule here until issue 426 (the Python checks, deleted with issue 1296) | Where it lives now |
|
||||
|---|---|
|
||||
| `records.py` `rests-on`, for a record's `extends:` | the knowledge base's `rests-on` at propose, and the whole-set rule `rests-on-live` |
|
||||
| `records.py` `rests-on`, for a design's `decisions:` naming an accepted record | **nowhere yet**: this repository cannot read the knowledge base. The reviewer checks each new entry with `knowledge-base.get`. Designs become the knowledge base's own kind under issue 428 |
|
||||
| `records.py` `live-citation`, in a record | the knowledge base's `live-citation` |
|
||||
| `records.py` `live-citation`, in `00-META/` and `03-DESIGN/01-to-be/` | **nowhere yet**, as above: the reviewer checks a cited record's status with `knowledge-base.get`; issue 428 |
|
||||
| `records.py` `supersession` | the knowledge base's `supersedes-accepted` and `self-link`, and the whole-set rule `supersession` |
|
||||
| `records.py` `partial-supersession` | the knowledge base's `changes` link to a clause, `changes-in-force` and `supersedes-accepted`, and the whole-set rule `partial` |
|
||||
| `records.py` `clauses` | the knowledge base's `has-clause`, `clause-numbering`, `clause-lead` and `clause-size` |
|
||||
| `records.py` `numbering` | the knowledge base's `title` and `reserved` |
|
||||
| `records.py` `topics`, `index.py` (a record's topic) | the knowledge base's `topic`, against the reading order the index test still checks |
|
||||
| `records.py` `insights` | the knowledge base's `insight-form`; an accepted record changes by a new version marked as a progressive insight |
|
||||
| `index.py --print` | `knowledge-base.holds` with a topic |
|
||||
| `cycle.py` research: a known status, `became:` when graduated | the knowledge base's `research-status`, `overview` and `became` |
|
||||
| `cycle.py` no two decision records share a number | the knowledge base's `reserved`, and the issue tracker's counter behind `knowledge-base.reserve` |
|
||||
| `cycle.py` an accepted decision cited by nothing | the knowledge base's whole-set rule `cited`: a record accepted there is cited by another record or names a home (`homes` at propose) |
|
||||
| `cycle.py` a deferral names the issue that tracks it | the knowledge base's `deferral` |
|
||||
| `words.py` over research efforts from 2026-10-07, and `kept` in `words-allowed.md` | the knowledge base's `retired-word` and `daemon-line`, from this glossary |
|
||||
|
||||
## What is deliberately not checked
|
||||
|
||||
- **Whether a record a document cites exists, or is still accepted.** A checkout cannot read the knowledge
|
||||
base, and a check that called the mesh would make every merge here depend on it. Until designs become the
|
||||
knowledge base's own kind (issue 428), the reviewer asks `knowledge-base.get` for each record a change
|
||||
cites in `00-META/` or `03-DESIGN/01-to-be/` (the documents that tell somebody what to do). An as-is
|
||||
document may rest on a superseded record: it describes what runs, and what runs was built under whatever
|
||||
was decided at the time (ADR 0006: *as-is describing a superseded decision is exactly what as-is is for*).
|
||||
- **Whether a citation's prose is still true.** A document can cite an accepted record and describe it
|
||||
wrongly, and nothing here notices.
|
||||
- **`02-DECISIONS/` and `01-RESEARCH/` may cite superseded records freely.** A decision record
|
||||
discusses history; research records what was observed. Flagging those would produce noise on
|
||||
correct documents, and a check that cries wolf gets suppressed — which costs more than not
|
||||
having it.
|
||||
- **`03-DESIGN/00-as-is/` may rest on a superseded record.** It describes what runs, and what
|
||||
runs was built under whatever was decided at the time
|
||||
([ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md):
|
||||
*as-is describing a superseded decision is exactly what as-is is for*).
|
||||
- **Whether a citation's prose is still true.** Only whether the record it points at is live.
|
||||
A document can cite an accepted record and describe it wrongly, and nothing here notices.
|
||||
|
||||
So "governing" means `00-META/` and `03-DESIGN/01-to-be/` — the documents that tell somebody
|
||||
what to do.
|
||||
|
||||
## Adding a check
|
||||
|
||||
State what incident it would have caught, and make it fail before you make it pass. A check
|
||||
whose failure has never been observed is a guess about its own correctness.
|
||||
|
||||
## The cycle test
|
||||
## cycle.py
|
||||
|
||||
The development cycle, checked (ADR 0080):
|
||||
every design parses, its `layer:` matches its folder and its `status:` is known; a to-be design names a
|
||||
decision in `decisions:`; an in-progress or implemented design names its owning code; and no two issue
|
||||
records share a number (issue 155 — the number is how a record is cited, and `main` lags every open pull
|
||||
request, so two people reading it allocate the same one). An issue report carries the day it was `opened:`
|
||||
and no status key — `status:`, `located-in:`, `fixed-by:`, `amended-design:`, `replay:`, `replay-none:` or
|
||||
`resolved:` — because an issue's status lives in the issue tracker (ADR 0275); it failed on all 361 reports
|
||||
that carried them before they were removed. The rules on an issue's status this check held until then — an
|
||||
owner for a located issue, a fix for a resolved one, and a core issue's `replay` or `replay-none` (ADR 0237)
|
||||
— are refused at the write by `issue-tracker.move`. Its rules on research efforts and decision records left
|
||||
with them (the table above).
|
||||
The development cycle, checked ([ADR 0080](../../02-DECISIONS/0080-the-development-cycle-is-checked.md)):
|
||||
a to-be design names a decision, an in-progress/implemented design names its owning code, a
|
||||
located/fixed issue names its owner, a fixed/resolved issue says what fixed it, a graduated
|
||||
research overview says what it became, and no two issue records share a number (issue 155 — the
|
||||
number is how a record is cited, and `main` lags every open pull request, so two people reading it
|
||||
allocate the same one). And a core issue — one opened from 2026-10-07 whose `located-in` names a core
|
||||
repository — resolves only with `replay:` (an id in mesh-lab's replays register) or `replay-none:` saying
|
||||
why none is possible ([ADR 0237](../../02-DECISIONS/0237-a-change-is-judged-against-the-mesh-that-runs-before-it-merges-on-the-build-seat.md));
|
||||
it failed on a resolved core issue carrying neither before it passed. `python3 00-META/checks/cycle.py`
|
||||
|
||||
## The words test
|
||||
## words.py
|
||||
|
||||
One word per thing, checked (ADR 0244).
|
||||
One word per thing, checked ([ADR 0244](../../02-DECISIONS/0244-the-mesh-is-described-in-domains-and-one-word-names-one-thing.md)).
|
||||
It reads the glossary's *Not:* lines (the retired words, each with its scope) and *Identifier until
|
||||
renamed* lines, and fails on a retired word or a bare identifier in running prose — what is left once code,
|
||||
quotations, struck-through text, link targets, comments and frontmatter are taken out — in `00-META/`,
|
||||
`03-DESIGN/`, `AGENTS.md`, `README.md`, and issues opened from 2026-10-07. Decision records and research
|
||||
efforts live in the knowledge base, which holds a proposed record to the same words (the table above). It
|
||||
also fails when a glossary head word heads two entries or is also retired.
|
||||
[`words-allowed.md`](words-allowed.md) names a document that could not be reworded at once, with a date. With `MESH_CATALOG_DIR` set it compares the catalogue's copy
|
||||
`03-DESIGN/`, `AGENTS.md`, `README.md`, and research and issues dated from 2026-10-07. Decision records are
|
||||
never checked. It also fails when a glossary head word heads two entries or is also retired.
|
||||
[`words-allowed.md`](words-allowed.md) names a document that could not be reworded at once, with a date, or
|
||||
a graduated research effort kept as written. With `MESH_CATALOG_DIR` set it compares the catalogue's copy
|
||||
of the tools' retired words (`retired-words`) with the glossary.
|
||||
|
||||
It also fails on a **daemon's line** (ADR 0272,
|
||||
its insight of 2026-10-09): the operator types one command, `nox`, and a document names a `nox` line, never the
|
||||
controller's or the node-engine's binary followed by one of its commands. The lines are the check's own list
|
||||
(`daemonLines` in the words test); a command that is a line only with an argument after it is matched as a line is written. Looked
|
||||
for in the same documents, from 2026-10-09 for issues, in the document's own words — code blocks and
|
||||
spans included, quotations and block quotes taken out. With `MESH_CATALOG_DIR` set it compares the catalogue's copy
|
||||
(`retired-lines`) with the list.
|
||||
|
||||
It failed on something real before it passed: 581 uses of retired words in 63 documents on its first run, besides the glossary itself and research 034 —
|
||||
"the host" for the node-engine in 30 designs, "control plane" in 10, the glossary's own entry for the tool runner —
|
||||
and two glossary contradictions (the "console", the deprecated broker's seat), fixed in the change that
|
||||
|
||||
@@ -0,0 +1,251 @@
|
||||
#!/usr/bin/env python3
|
||||
"""The development cycle, checked.
|
||||
|
||||
The knowledge flow (00-META/process/00-overview.md) says work moves idea -> research ->
|
||||
decision -> to-be design -> code, and symptom -> issue -> diagnosis -> fix. Those are rules,
|
||||
and a rule states how it is checked (AGENTS.md) -- this is how. Everything here reads only
|
||||
frontmatter, because status lives in frontmatter and nowhere else.
|
||||
|
||||
What is enforced:
|
||||
|
||||
design every 03-DESIGN doc parses, carries `layer:` matching its directory, and a
|
||||
known `status:`. A TO-BE doc names at least one decision (`decisions:`) -- no
|
||||
design without a decision -- and once `in-progress` or `implemented` it names
|
||||
its owning code (`code:`) -- no development without a design that says where.
|
||||
issues a known `status:`; once `located`, `located-in:` names the owner;
|
||||
once `resolved`, `fixed-by:` says what fixed it (prose counts --
|
||||
"nothing, the capability existed" is an answer). And no two records share a
|
||||
number -- the number is how a record is cited.
|
||||
replays a core issue -- one opened from 2026-10-07 whose `located-in:` names a core repository --
|
||||
resolves only with `replay:` (the replay's id in mesh-lab's replays register) or
|
||||
`replay-none:` (why no replay is possible): ADR 0237, to-be 45 §9. A replay is what
|
||||
proves a fix fails before and passes after; without one, a fixed class comes back
|
||||
through a different door (research 031: four such chains in one week).
|
||||
research a known `status:`; a `graduated` overview says what it `became:`, and every
|
||||
target it names exists.
|
||||
decisions every accepted record is REACHABLE from the cycle: cited by a design doc's
|
||||
frontmatter, a research overview, an issue report, a 00-META document, or another
|
||||
record's extends/supersedes chain. A decision nothing points at is one nobody will
|
||||
find by following pointers -- which is how records go stale in people's heads.
|
||||
|
||||
Deliberately NOT enforced: `resolved` issues may leave `located-in` empty (a symptom that
|
||||
turned out not to be a defect has no owner), and as-is docs need no decisions (they
|
||||
describe what exists, not what was decided).
|
||||
|
||||
python3 00-META/checks/cycle.py
|
||||
"""
|
||||
|
||||
import glob
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
ROOT = os.path.normpath(os.path.join(os.path.dirname(__file__), "..", ".."))
|
||||
|
||||
DESIGN_STATUSES = {"proposed", "designed", "in-progress", "implemented", "abandoned"}
|
||||
ISSUE_STATUSES = {"open", "diagnosing", "located", "resolved", "wontfix"}
|
||||
RESEARCH_STATUSES = {"active", "graduated", "abandoned"}
|
||||
|
||||
# The core, as to-be 45 names it, by where its code lives: the controller, the node-engine, the node
|
||||
# tools and the console, the SDK's loops, and the catalogue's bus, forge (the announcer of merges) and
|
||||
# build agent. A `located-in:` entry naming any of these makes an issue a core issue.
|
||||
CORE = re.compile(r"\b(mesh-controller|mesh-host|mesh-tools|mesh-sdk|node-tools|"
|
||||
r"mesh-catalog\s+modules/(nats|gitea|build-agent))\b")
|
||||
|
||||
# The day the rule began (ADR 0237): issues opened before it are not held to it.
|
||||
REPLAYS_FROM = "2026-10-07"
|
||||
REPLAY_ID = re.compile(r"^R\d+\b")
|
||||
|
||||
|
||||
def rel(path):
|
||||
return os.path.relpath(path, ROOT)
|
||||
|
||||
|
||||
def frontmatter(path):
|
||||
"""The YAML block between the first two --- lines, as {key: raw-value-string}.
|
||||
|
||||
Minimal on purpose, like records.py: enough for the fields these checks read. A list
|
||||
value (block or inline) is joined into its items; a scalar stays a string.
|
||||
"""
|
||||
text = open(path, encoding="utf-8").read()
|
||||
m = re.match(r"^---\n(.*?)\n---", text, re.S)
|
||||
if not m:
|
||||
return None
|
||||
front, out, key = m.group(1), {}, None
|
||||
for line in front.split("\n"):
|
||||
item = re.match(r"^\s+-\s*(.+?)\s*$", line)
|
||||
if item and key:
|
||||
out[key].append(item.group(1))
|
||||
continue
|
||||
kv = re.match(r"^([A-Za-z-]+):\s*(.*)$", line)
|
||||
if not kv:
|
||||
continue
|
||||
key, value = kv.group(1), kv.group(2).strip()
|
||||
if value.startswith("[") and value.endswith("]"):
|
||||
out[key] = [v.strip() for v in value[1:-1].split(",") if v.strip()]
|
||||
elif value == "":
|
||||
out[key] = [] # a block list may follow; stays [] if nothing does
|
||||
else:
|
||||
out[key] = value
|
||||
return out
|
||||
|
||||
|
||||
def listy(front, key):
|
||||
v = front.get(key)
|
||||
if v is None:
|
||||
return []
|
||||
return v if isinstance(v, list) else ([v] if str(v).strip() else [])
|
||||
|
||||
|
||||
def main():
|
||||
failures = []
|
||||
|
||||
def bad(path, why):
|
||||
failures.append(" %s: %s" % (rel(path), why))
|
||||
|
||||
# ---- design ------------------------------------------------------------------------
|
||||
for layer, name in (("00-as-is", "as-is"), ("01-to-be", "to-be")):
|
||||
for path in sorted(glob.glob(os.path.join(ROOT, "03-DESIGN", layer, "*.md"))):
|
||||
if os.path.basename(path) == "README.md":
|
||||
continue
|
||||
front = frontmatter(path)
|
||||
if front is None:
|
||||
bad(path, "no frontmatter")
|
||||
continue
|
||||
if front.get("layer") != name:
|
||||
bad(path, "layer is %r; this directory is %s" % (front.get("layer"), name))
|
||||
status = front.get("status")
|
||||
if status not in DESIGN_STATUSES:
|
||||
bad(path, "status %r is not one of %s" % (status, sorted(DESIGN_STATUSES)))
|
||||
if name == "to-be":
|
||||
if not listy(front, "decisions"):
|
||||
bad(path, "names no decisions -- no design without a decision")
|
||||
if status in ("in-progress", "implemented") and not listy(front, "code"):
|
||||
bad(path, "status %s but code: names no owner -- no development "
|
||||
"without a design that says where" % status)
|
||||
|
||||
# ---- issues ------------------------------------------------------------------------
|
||||
# Two records may not share a number. Numbers are taken as "next free after main", and work
|
||||
# sits on unmerged branches for days -- so two people reading the same main allocate the same
|
||||
# number, and nothing said so. It happened twice in one evening between two machines, and the
|
||||
# second collision landed on main with all three checks passing (issue 155). An issue number is
|
||||
# how every other record cites this one; two records answering to it means a pointer that
|
||||
# resolves to whichever the reader happened to open.
|
||||
seen = {}
|
||||
for folder in sorted(glob.glob(os.path.join(ROOT, "04-ISSUES", "*", ""))):
|
||||
name = os.path.basename(os.path.normpath(folder))
|
||||
number = name.split("-", 1)[0]
|
||||
if not number.isdigit():
|
||||
continue
|
||||
if number in seen:
|
||||
bad(os.path.join("04-ISSUES", name),
|
||||
"is numbered %s, and so is %s -- an issue number is how it is cited, and two "
|
||||
"records answering to one means a citation that resolves to whichever the reader "
|
||||
"opened. Take the next free number across main AND every open pull request"
|
||||
% (number, seen[number]))
|
||||
else:
|
||||
seen[number] = name
|
||||
|
||||
# And decision records, which 155's fix left out: on 2026-10-02 two ADRs numbered 0169 landed
|
||||
# on main from two sessions within the hour, and every check passed.
|
||||
seen_records = {}
|
||||
for path in sorted(glob.glob(os.path.join(ROOT, "02-DECISIONS", "[0-9]*.md"))):
|
||||
name = os.path.basename(path)
|
||||
number = name.split("-", 1)[0]
|
||||
if not number.isdigit():
|
||||
continue
|
||||
if number in seen_records:
|
||||
bad(os.path.join("02-DECISIONS", name),
|
||||
"is numbered %s, and so is %s -- a record's number is how it is cited. Take the next "
|
||||
"free number across main AND every open pull request; the branch that lands last "
|
||||
"renumbers" % (number, seen_records[number]))
|
||||
else:
|
||||
seen_records[number] = name
|
||||
|
||||
for path in sorted(glob.glob(os.path.join(ROOT, "04-ISSUES", "*", "00-report.md"))):
|
||||
front = frontmatter(path)
|
||||
if front is None:
|
||||
bad(path, "no frontmatter")
|
||||
continue
|
||||
status = front.get("status")
|
||||
if status not in ISSUE_STATUSES:
|
||||
bad(path, "status %r is not one of %s" % (status, sorted(ISSUE_STATUSES)))
|
||||
if status in ("located", "resolved") and not listy(front, "located-in"):
|
||||
bad(path, "status %s but located-in is empty" % status)
|
||||
if status == "resolved" and not listy(front, "fixed-by"):
|
||||
bad(path, "status %s but fixed-by says nothing" % status)
|
||||
# A core issue resolves with its replay, or with why none is possible (ADR 0237).
|
||||
replay = " ".join(listy(front, "replay"))
|
||||
if replay and not REPLAY_ID.match(replay):
|
||||
bad(path, "replay: %r names no replay -- an id from mesh-lab's replays register, R<issue>" % replay)
|
||||
core = any(CORE.search(entry) for entry in listy(front, "located-in"))
|
||||
opened = str(front.get("opened") or "")
|
||||
if (status == "resolved" and core and opened >= REPLAYS_FROM and not replay
|
||||
and not " ".join(listy(front, "replay-none")).strip()):
|
||||
bad(path, "a core issue resolved with no replay: name it in `replay:` (mesh-lab replays "
|
||||
"register.go) or say in `replay-none:` why none is possible (ADR 0237)")
|
||||
|
||||
# ---- research ----------------------------------------------------------------------
|
||||
for path in sorted(glob.glob(os.path.join(ROOT, "01-RESEARCH", "*", "00-overview.md"))):
|
||||
front = frontmatter(path)
|
||||
if front is None:
|
||||
bad(path, "no frontmatter")
|
||||
continue
|
||||
status = front.get("status")
|
||||
if status not in RESEARCH_STATUSES:
|
||||
bad(path, "status %r is not one of %s" % (status, sorted(RESEARCH_STATUSES)))
|
||||
if status == "graduated":
|
||||
became = listy(front, "became")
|
||||
if not became:
|
||||
bad(path, "graduated but became: names nothing")
|
||||
for target in became:
|
||||
if not os.path.exists(os.path.join(ROOT, target)):
|
||||
bad(path, "became names %s, which does not exist" % target)
|
||||
|
||||
# ---- decisions -------------------------------------------------------------------
|
||||
records = {}
|
||||
for path in sorted(glob.glob(os.path.join(ROOT, "02-DECISIONS", "[0-9]*.md"))):
|
||||
front = frontmatter(path)
|
||||
records[os.path.basename(path)] = (path, (front or {}).get("status"))
|
||||
|
||||
cited = set()
|
||||
sources = (glob.glob(os.path.join(ROOT, "03-DESIGN", "*", "*.md"))
|
||||
+ glob.glob(os.path.join(ROOT, "01-RESEARCH", "*", "00-overview.md"))
|
||||
+ glob.glob(os.path.join(ROOT, "04-ISSUES", "*", "00-report.md"))
|
||||
+ glob.glob(os.path.join(ROOT, "00-META", "**", "*.md"), recursive=True))
|
||||
for path in sources:
|
||||
text = open(path, encoding="utf-8").read()
|
||||
if os.sep + "03-DESIGN" + os.sep in path:
|
||||
# A design doc's governing citations live in frontmatter; a prose mention is
|
||||
# commentary, not a home.
|
||||
m = re.match(r"^---\n(.*?)\n---", text, re.S)
|
||||
text = m.group(1) if m else ""
|
||||
for m in re.finditer(r"([0-9]{4}-[^\s\)\],#]+\.md)", text):
|
||||
cited.add(os.path.basename(m.group(1)))
|
||||
for name in records:
|
||||
front = frontmatter(records[name][0]) or {}
|
||||
for key in ("extends", "supersedes", "superseded-by"):
|
||||
v = front.get(key)
|
||||
if isinstance(v, str) and v:
|
||||
cited.add(os.path.basename(v))
|
||||
for name, (path, status) in records.items():
|
||||
if status == "accepted" and name not in cited:
|
||||
bad(path, "an accepted decision nothing in the cycle cites -- give it a home in a "
|
||||
"design doc's decisions:, a research became:, an issue, or 00-META")
|
||||
|
||||
checked = (
|
||||
len(glob.glob(os.path.join(ROOT, "03-DESIGN", "0*", "*.md")))
|
||||
+ len(glob.glob(os.path.join(ROOT, "04-ISSUES", "*", "00-report.md")))
|
||||
+ len(glob.glob(os.path.join(ROOT, "01-RESEARCH", "*", "00-overview.md")))
|
||||
+ len(records)
|
||||
)
|
||||
if failures:
|
||||
print("cycle: %d document(s) break the development cycle:" % len(failures))
|
||||
print("\n".join(failures))
|
||||
return 1
|
||||
print("cycle: %d documents checked, the chain holds" % checked)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -1,169 +0,0 @@
|
||||
package checks
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"path"
|
||||
"sort"
|
||||
"strings"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
)
|
||||
|
||||
// The development cycle, checked (cycle.py until ADR 0527; ADR 0080). Everything here reads frontmatter only,
|
||||
// because a status this repository holds lives in frontmatter and nowhere else. The decision records and research
|
||||
// efforts live in the knowledge base (ADR 0290), whose rules judge them.
|
||||
|
||||
var designStatuses = map[string]bool{"proposed": true, "designed": true, "in-progress": true, "implemented": true, "abandoned": true}
|
||||
|
||||
// The keys an issue report carried while its status lived here. The issue tracker holds them now (ADR 0275), and
|
||||
// `issue-tracker.move` enforces the cycle's rules at the write. A report carrying one is a status in two places.
|
||||
var trackerKeys = []string{"status", "located-in", "fixed-by", "amended-design", "replay", "replay-none", "resolved"}
|
||||
|
||||
// checkDesigns: every design parses, carries `layer:` matching its folder and a known `status:`; a to-be design
|
||||
// names a decision, and once in-progress or implemented names its owning code. As-is designs need no decisions.
|
||||
func checkDesigns(fsys fs.FS) []string {
|
||||
var out []string
|
||||
for _, layer := range []struct{ folder, name string }{{"00-as-is", "as-is"}, {"01-to-be", "to-be"}} {
|
||||
entries, _ := fs.ReadDir(fsys, "03-DESIGN/"+layer.folder)
|
||||
for _, e := range entries {
|
||||
if e.IsDir() || !strings.HasSuffix(e.Name(), ".md") || e.Name() == "README.md" {
|
||||
continue
|
||||
}
|
||||
p := path.Join("03-DESIGN", layer.folder, e.Name())
|
||||
front := frontmatter(readFile(fsys, p))
|
||||
if front == nil {
|
||||
out = append(out, "cycle: "+p+": no frontmatter")
|
||||
continue
|
||||
}
|
||||
if front.scalar("layer") != layer.name {
|
||||
out = append(out, fmt.Sprintf("cycle: %s: layer is %q; this directory is %s", p, front.scalar("layer"), layer.name))
|
||||
}
|
||||
status := front.scalar("status")
|
||||
if !designStatuses[status] {
|
||||
var known []string
|
||||
for s := range designStatuses {
|
||||
known = append(known, s)
|
||||
}
|
||||
sort.Strings(known)
|
||||
out = append(out, fmt.Sprintf("cycle: %s: status %q is not one of %v", p, status, known))
|
||||
}
|
||||
if layer.name == "to-be" {
|
||||
if len(front.list("decisions")) == 0 {
|
||||
out = append(out, "cycle: "+p+": names no decisions -- no design without a decision")
|
||||
}
|
||||
if (status == "in-progress" || status == "implemented") && len(front.list("code")) == 0 {
|
||||
out = append(out, fmt.Sprintf("cycle: %s: status %s but code: names no owner -- no development without a design that says where", p, status))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// checkIssues: no two issue folders share a number (issue 155: the number is how a record is cited), every report
|
||||
// has frontmatter with `opened:`, and none carries a key the tracker holds.
|
||||
func checkIssues(fsys fs.FS) []string {
|
||||
var out []string
|
||||
entries, _ := fs.ReadDir(fsys, "04-ISSUES")
|
||||
seen := map[string]string{}
|
||||
for _, e := range entries {
|
||||
if !e.IsDir() {
|
||||
continue
|
||||
}
|
||||
number := strings.SplitN(e.Name(), "-", 2)[0]
|
||||
if number == "" || strings.Trim(number, "0123456789") != "" {
|
||||
continue
|
||||
}
|
||||
if first, ok := seen[number]; ok {
|
||||
out = append(out, fmt.Sprintf("cycle: 04-ISSUES/%s: is numbered %s, and so is %s -- an issue number is how it is cited, and two records "+
|
||||
"answering to one means a citation that resolves to whichever the reader opened. Take the next free number across main AND every open pull request", e.Name(), number, first))
|
||||
} else {
|
||||
seen[number] = e.Name()
|
||||
}
|
||||
}
|
||||
for _, e := range entries {
|
||||
p := path.Join("04-ISSUES", e.Name(), "00-report.md")
|
||||
if !e.IsDir() {
|
||||
continue
|
||||
}
|
||||
if _, err := fs.Stat(fsys, p); err != nil {
|
||||
continue
|
||||
}
|
||||
front := frontmatter(readFile(fsys, p))
|
||||
if front == nil {
|
||||
out = append(out, "cycle: "+p+": no frontmatter")
|
||||
continue
|
||||
}
|
||||
if front.scalar("opened") == "" {
|
||||
out = append(out, "cycle: "+p+": opened: says nothing -- the day the issue was opened, YYYY-MM-DD")
|
||||
continue
|
||||
}
|
||||
var carried []string
|
||||
for _, k := range trackerKeys {
|
||||
if _, ok := front[k]; ok {
|
||||
carried = append(carried, "`"+k+":`")
|
||||
}
|
||||
}
|
||||
if len(carried) > 0 {
|
||||
out = append(out, fmt.Sprintf("cycle: %s: carries %s, but an issue's status, owner, fix and replay live in the issue tracker (ADR 0275): "+
|
||||
"move it with `issue-tracker.move`, and keep no copy here", p, strings.Join(carried, ", ")))
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func TestCycleDesignsHoldTheChain(t *testing.T) { expectNone(t, checkDesigns(rootFS())) }
|
||||
func TestCycleIssuesHoldTheChain(t *testing.T) { expectNone(t, checkIssues(rootFS())) }
|
||||
|
||||
func design(layer, status, extra string) *fstest.MapFile {
|
||||
return doc("---\nlayer: " + layer + "\nstatus: " + status + "\n" + extra + "---\nbody\n")
|
||||
}
|
||||
|
||||
func TestCycleDesignsCatchesEachBreak(t *testing.T) {
|
||||
broken := fstest.MapFS{
|
||||
"03-DESIGN/00-as-is/a-nofront.md": doc("no frontmatter"),
|
||||
"03-DESIGN/00-as-is/b-wronglayer.md": design("to-be", "implemented", ""),
|
||||
"03-DESIGN/00-as-is/c-badstatus.md": design("as-is", "done", ""),
|
||||
"03-DESIGN/01-to-be/d-nodecision.md": design("to-be", "designed", ""),
|
||||
"03-DESIGN/01-to-be/e-nocode.md": design("to-be", "in-progress", "decisions: [ADR 0001]\n"),
|
||||
"03-DESIGN/01-to-be/README.md": doc("not a design"),
|
||||
}
|
||||
got := checkDesigns(broken)
|
||||
expectProblem(t, got, "a-nofront.md: no frontmatter")
|
||||
expectProblem(t, got, `b-wronglayer.md: layer is "to-be"; this directory is as-is`)
|
||||
expectProblem(t, got, `c-badstatus.md: status "done" is not one of`)
|
||||
expectProblem(t, got, "d-nodecision.md: names no decisions")
|
||||
expectProblem(t, got, "e-nocode.md: status in-progress but code: names no owner")
|
||||
if len(got) != 5 {
|
||||
t.Errorf("exactly the five breaks, as-is needs no decisions and a README is no design: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCycleDesignsAcceptsASoundPair(t *testing.T) {
|
||||
sound := fstest.MapFS{
|
||||
"03-DESIGN/00-as-is/a.md": design("as-is", "implemented", ""),
|
||||
"03-DESIGN/01-to-be/b.md": design("to-be", "in-progress", "decisions: [ADR 0001]\ncode: [mesh-sdk]\n"),
|
||||
}
|
||||
expectNone(t, checkDesigns(sound))
|
||||
}
|
||||
|
||||
func TestCycleIssuesCatchesEachBreak(t *testing.T) {
|
||||
report := func(front string) *fstest.MapFile { return doc("---\n" + front + "---\nbody\n") }
|
||||
broken := fstest.MapFS{
|
||||
"04-ISSUES/100-a/00-report.md": report("opened: 2026-10-01\n"),
|
||||
"04-ISSUES/100-b/00-report.md": report("opened: 2026-10-01\n"),
|
||||
"04-ISSUES/101-noopened/00-report.md": report("title: x\n"),
|
||||
"04-ISSUES/102-tracker/00-report.md": report("opened: 2026-10-01\nstatus: open\nreplay: none\n"),
|
||||
"04-ISSUES/103-nofront/00-report.md": doc("plain"),
|
||||
"04-ISSUES/notes/readme.md": doc("not an issue"),
|
||||
}
|
||||
got := checkIssues(broken)
|
||||
expectProblem(t, got, "04-ISSUES/100-b: is numbered 100, and so is 100-a")
|
||||
expectProblem(t, got, "101-noopened/00-report.md: opened: says nothing")
|
||||
expectProblem(t, got, "102-tracker/00-report.md: carries `status:`, `replay:`")
|
||||
expectProblem(t, got, "103-nofront/00-report.md: no frontmatter")
|
||||
if len(got) != 4 {
|
||||
t.Errorf("exactly the four breaks: %q", got)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,115 @@
|
||||
#!/usr/bin/env python3
|
||||
"""The reading order of the decision records: its topics are written, its list is generated.
|
||||
|
||||
A number identifies a record and never changes, so the folder listing is creation order rather
|
||||
than reading order. The reading order is the six topics `02-DECISIONS/README.md` writes down, in
|
||||
order, and each record's own `topic:`. The list of records under each topic is generated from
|
||||
that frontmatter when somebody reads it, and never stored (ADR 0248, hq issue 297): a stored
|
||||
list was a line every decision pull request added to one file, so any two open at once conflicted.
|
||||
|
||||
python3 00-META/checks/index.py check
|
||||
python3 00-META/checks/index.py --print print the list, by topic, in reading order
|
||||
|
||||
It fails when the README's reading order is missing, when a record's topic is not one of its
|
||||
topics, and when the README stores a list of records again — the index markers, or a line of
|
||||
the generated list. The last one is what keeps the conflict from coming back.
|
||||
"""
|
||||
|
||||
import glob
|
||||
import io
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
README = "02-DECISIONS/README.md"
|
||||
|
||||
# A topic in the README's reading order, e.g.
|
||||
# 1. **What the mesh is** — `topic: the mesh`. What it is for, ...
|
||||
# The records module's `records_decisions` reads the same line, so the form is a contract.
|
||||
TOPIC_LINE = re.compile(r"^\d+\.\s+\*\*(?P<label>[^*]+)\*\*\s+—\s+`topic:\s*(?P<topic>[^`]+)`", re.M)
|
||||
|
||||
# What a stored list looks like: the old markers, or a line of the list this script prints.
|
||||
STORED = [
|
||||
(re.compile(r"<!--\s*index:(start|end)\s*-->"), "an index marker"),
|
||||
(re.compile(r"^- \*\*\d{4}\*\* — \[", re.M), "a line of the generated list"),
|
||||
]
|
||||
|
||||
|
||||
def reading_order(text=None):
|
||||
"""The topics, in order, as (topic, label) pairs, from the README."""
|
||||
if text is None:
|
||||
text = io.open(README, encoding='utf-8').read()
|
||||
return [(m.group("topic").strip(), m.group("label").strip()) for m in TOPIC_LINE.finditer(text)]
|
||||
|
||||
|
||||
def field(text, name):
|
||||
m = re.search(r'^%s:\s*(.+)$' % name, text, re.M)
|
||||
return m.group(1).strip() if m else None
|
||||
|
||||
|
||||
def records():
|
||||
out = []
|
||||
for path in sorted(glob.glob('02-DECISIONS/0*.md')):
|
||||
text = io.open(path, encoding='utf-8').read()
|
||||
heading = re.search(r'^# \d+\.\s*(.+)$', text, re.M)
|
||||
out.append({
|
||||
"file": os.path.basename(path),
|
||||
"number": os.path.basename(path)[:4],
|
||||
"title": heading.group(1).strip() if heading else "(no heading)",
|
||||
"topic": field(text, "topic"),
|
||||
"status": field(text, "status"),
|
||||
})
|
||||
return out
|
||||
|
||||
|
||||
def render(topics, rs):
|
||||
lines = []
|
||||
for topic, label in topics:
|
||||
rows = [r for r in rs if r["topic"] == topic]
|
||||
if not rows:
|
||||
continue
|
||||
lines.append("### %s" % label)
|
||||
lines.append("")
|
||||
for r in rows:
|
||||
mark = "" if r["status"] == "accepted" else " *(%s)*" % r["status"]
|
||||
lines.append("- **%s** — [%s](%s)%s" % (r["number"], r["title"], r["file"], mark))
|
||||
lines.append("")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def main():
|
||||
text = io.open(README, encoding='utf-8').read()
|
||||
topics = reading_order(text)
|
||||
rs = records()
|
||||
|
||||
if "--print" in sys.argv:
|
||||
print(render(topics, rs))
|
||||
return 0
|
||||
|
||||
problems = []
|
||||
if not topics:
|
||||
problems.append("%s writes no reading order: no line of the form "
|
||||
"'1. **Label** — `topic: name`. ...'" % README)
|
||||
names = [t for t, _ in topics]
|
||||
if len(set(names)) != len(names):
|
||||
problems.append("%s names a topic twice in its reading order" % README)
|
||||
for pattern, what in STORED:
|
||||
if pattern.search(text):
|
||||
problems.append("%s stores the list of records (%s). The list is generated when read "
|
||||
"(ADR 0248): `python3 00-META/checks/index.py --print`" % (README, what))
|
||||
known = set(names)
|
||||
for r in rs if known else []: # with no reading order, every record would be named; one line says why
|
||||
if r["topic"] not in known:
|
||||
problems.append("%s: topic %r is not one of the reading order's: %s"
|
||||
% (r["file"], r["topic"], ", ".join(names)))
|
||||
|
||||
for p in problems:
|
||||
print("index: " + p)
|
||||
if problems:
|
||||
return 1
|
||||
print("index: %d topics, %d records, no stored list" % (len(topics), len(rs)))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -1,93 +0,0 @@
|
||||
package checks
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
)
|
||||
|
||||
// The reading order of the decision records (index.py until ADR 0527): its topics are written in
|
||||
// 02-DECISIONS/README.md, its records are not. The knowledge base reads these lines and refuses a record whose
|
||||
// topic is not one of them, so the form is a contract; a topic's records are `knowledge-base.holds` with that
|
||||
// topic, never stored here (ADR 0248, issue 297).
|
||||
|
||||
const readmeOfDecisions = "02-DECISIONS/README.md"
|
||||
|
||||
// A topic line: `1. **Label** — `topic: name`. ...`
|
||||
var topicLineRx = regexp.MustCompile(`(?m)^\d+\.\s+\*\*([^*]+)\*\*\s+—\s+` + "`topic:\\s*([^`]+)`")
|
||||
|
||||
// What a stored list looks like: the old markers, or a line of the list index.py used to print.
|
||||
var storedLists = []struct {
|
||||
rx *regexp.Regexp
|
||||
what string
|
||||
}{
|
||||
{regexp.MustCompile(`<!--\s*index:(start|end)\s*-->`), "an index marker"},
|
||||
{regexp.MustCompile(`(?m)^- \*\*\d{4}\*\* — \[`), "a line of a generated list"},
|
||||
}
|
||||
|
||||
func readingOrder(text string) []string {
|
||||
var topics []string
|
||||
for _, m := range topicLineRx.FindAllStringSubmatch(text, -1) {
|
||||
topics = append(topics, strings.TrimSpace(m[2]))
|
||||
}
|
||||
return topics
|
||||
}
|
||||
|
||||
func checkIndex(fsys fs.FS) []string {
|
||||
text := readFile(fsys, readmeOfDecisions)
|
||||
topics := readingOrder(text)
|
||||
var out []string
|
||||
if len(topics) == 0 {
|
||||
out = append(out, "index: "+readmeOfDecisions+" writes no reading order: no line of the form '1. **Label** — `topic: name`. ...'; "+
|
||||
"the knowledge base refuses every record without it")
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
for _, topic := range topics {
|
||||
if seen[topic] {
|
||||
out = append(out, fmt.Sprintf("index: %s names a topic twice in its reading order: %s", readmeOfDecisions, topic))
|
||||
break
|
||||
}
|
||||
seen[topic] = true
|
||||
}
|
||||
for _, s := range storedLists {
|
||||
if s.rx.MatchString(text) {
|
||||
out = append(out, fmt.Sprintf("index: %s stores a list of records (%s). The records live in the knowledge base, and a topic's list is "+
|
||||
"`knowledge-base.holds` with that topic", readmeOfDecisions, s.what))
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func TestIndexReadingOrderIsWrittenAndStoresNoList(t *testing.T) {
|
||||
expectNone(t, checkIndex(rootFS()))
|
||||
if n := len(readingOrder(readFile(rootFS(), readmeOfDecisions))); n == 0 {
|
||||
t.Error("the reading order has no topic")
|
||||
}
|
||||
}
|
||||
|
||||
const sound = "1. **One** — `topic: one`. x\n2. **Two** — `topic: two`. y\n"
|
||||
|
||||
func TestIndexCatchesAMissingReadingOrder(t *testing.T) {
|
||||
expectProblem(t, checkIndex(fstest.MapFS{readmeOfDecisions: doc("nothing written here\n")}), "writes no reading order")
|
||||
}
|
||||
|
||||
func TestIndexCatchesADuplicateTopic(t *testing.T) {
|
||||
broken := fstest.MapFS{readmeOfDecisions: doc(sound + "3. **Again** — `topic: one`. z\n")}
|
||||
expectProblem(t, checkIndex(broken), "names a topic twice")
|
||||
}
|
||||
|
||||
func TestIndexCatchesAStoredList(t *testing.T) {
|
||||
for what, stored := range map[string]string{
|
||||
"an index marker": "<!-- index:start -->\n",
|
||||
"a line of a generated list": "- **0001** — [a title](x.md)\n",
|
||||
} {
|
||||
expectProblem(t, checkIndex(fstest.MapFS{readmeOfDecisions: doc(sound + stored)}), what)
|
||||
}
|
||||
}
|
||||
|
||||
func TestIndexAcceptsASoundReadme(t *testing.T) {
|
||||
expectNone(t, checkIndex(fstest.MapFS{readmeOfDecisions: doc(sound)}))
|
||||
}
|
||||
@@ -1 +0,0 @@
|
||||
These tests read the repository's documents (rootFS), outside this directory: mesh-build-go never keeps this package's verdict (issue 1935).
|
||||
@@ -0,0 +1,418 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Structural checks over HQ's own records.
|
||||
|
||||
Every check here exists because the thing it checks for actually happened. See README.md
|
||||
for which incident is behind which check. Run from the repository root:
|
||||
|
||||
python3 00-META/checks/records.py
|
||||
|
||||
Exits non-zero if anything fails, so it can be a gate rather than a report.
|
||||
"""
|
||||
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
# Where a citation is *guidance* rather than history. A document here tells somebody what to
|
||||
# do, so a link to a superseded record is an instruction to follow a withdrawn decision.
|
||||
# 02-DECISIONS and 01-RESEARCH are deliberately absent: they record what was decided and what
|
||||
# was observed, and both legitimately discuss superseded records at length.
|
||||
GOVERNING = ("00-META/", "03-DESIGN/01-to-be/")
|
||||
|
||||
LINK = re.compile(r"\[[^\]]*\]\((?!https?:|mailto:)([^)]+)\)")
|
||||
ADR_FILE = re.compile(r"^(\d{4})-")
|
||||
|
||||
|
||||
def markdown_files():
|
||||
for base, dirs, files in os.walk(ROOT):
|
||||
dirs[:] = [d for d in dirs if d not in (".git", ".claude")]
|
||||
for name in sorted(files):
|
||||
if name.endswith(".md"):
|
||||
yield os.path.join(base, name)
|
||||
|
||||
|
||||
def rel(path):
|
||||
return os.path.relpath(path, ROOT)
|
||||
|
||||
|
||||
def read(path):
|
||||
with open(path, encoding="utf-8") as handle:
|
||||
return handle.read()
|
||||
|
||||
|
||||
def frontmatter(text):
|
||||
"""Minimal frontmatter reader — enough for the fields these checks use.
|
||||
|
||||
Not a YAML parser on purpose: a dependency in a repository that has none, to read four
|
||||
scalar fields and one list, would cost more than it returns.
|
||||
"""
|
||||
if not text.startswith("---\n"):
|
||||
return {}
|
||||
end = text.find("\n---", 4)
|
||||
if end == -1:
|
||||
return {}
|
||||
fields, key = {}, None
|
||||
for line in text[4:end].split("\n"):
|
||||
item = re.match(r"^\s+-\s+(.*)$", line)
|
||||
if item and key:
|
||||
fields.setdefault(key, []).append(item.group(1).strip())
|
||||
continue
|
||||
pair = re.match(r"^([A-Za-z_-]+):\s*(.*)$", line)
|
||||
if pair:
|
||||
key = pair.group(1)
|
||||
value = pair.group(2).strip()
|
||||
if value.startswith("[") and value.endswith("]"):
|
||||
# inline list: `decisions: [a, b]`, `code: []`
|
||||
inner = value[1:-1].strip()
|
||||
fields[key] = [v.strip() for v in inner.split(",") if v.strip()]
|
||||
elif value:
|
||||
fields[key] = value
|
||||
else:
|
||||
fields[key] = []
|
||||
return fields
|
||||
|
||||
|
||||
def load_records():
|
||||
"""Every decision record, by its four-digit number."""
|
||||
records = {}
|
||||
folder = os.path.join(ROOT, "02-DECISIONS")
|
||||
for name in sorted(os.listdir(folder)):
|
||||
match = ADR_FILE.match(name)
|
||||
if not name.endswith(".md") or not match:
|
||||
continue
|
||||
path = os.path.join(folder, name)
|
||||
text = read(path)
|
||||
records[match.group(1)] = {
|
||||
"number": match.group(1),
|
||||
"name": name,
|
||||
"path": path,
|
||||
"text": text,
|
||||
"front": frontmatter(text),
|
||||
}
|
||||
return records
|
||||
|
||||
|
||||
class Failures:
|
||||
def __init__(self):
|
||||
self.items = []
|
||||
|
||||
def add(self, check, location, message):
|
||||
self.items.append((check, location, message))
|
||||
|
||||
def report(self):
|
||||
if not self.items:
|
||||
print("records: all checks passed")
|
||||
return 0
|
||||
by_check = {}
|
||||
for check, location, message in self.items:
|
||||
by_check.setdefault(check, []).append((location, message))
|
||||
for check in sorted(by_check):
|
||||
print(f"\n{check} — {len(by_check[check])} problem(s)")
|
||||
for location, message in by_check[check]:
|
||||
print(f" {location}\n {message}")
|
||||
print(f"\nrecords: {len(self.items)} problem(s)")
|
||||
return 1
|
||||
|
||||
|
||||
def check_links(failures):
|
||||
"""Every relative link resolves to something that exists."""
|
||||
for path in markdown_files():
|
||||
folder = os.path.dirname(path)
|
||||
for number, line in enumerate(read(path).split("\n"), 1):
|
||||
for target in LINK.findall(line):
|
||||
target = target.split("#")[0].strip()
|
||||
if not target:
|
||||
continue
|
||||
if not os.path.exists(os.path.normpath(os.path.join(folder, target))):
|
||||
failures.add("links", f"{rel(path)}:{number}", f"link does not resolve: {target}")
|
||||
|
||||
|
||||
def check_rests_on(failures, records):
|
||||
"""A document's `decisions:` and a record's `extends:` must name a live record.
|
||||
|
||||
These are the load-bearing citations: the document declares that it rests on that
|
||||
decision. Resting on a withdrawn one is the defect this whole check set exists for.
|
||||
"""
|
||||
for path in markdown_files():
|
||||
front = frontmatter(read(path))
|
||||
cited = list(front.get("decisions", []) or [])
|
||||
extends = front.get("extends")
|
||||
if isinstance(extends, str) and extends:
|
||||
cited.append(extends)
|
||||
|
||||
for entry in cited:
|
||||
match = ADR_FILE.match(os.path.basename(entry))
|
||||
if not match:
|
||||
failures.add("rests-on", rel(path), f"not a decision record: {entry}")
|
||||
continue
|
||||
number = match.group(1)
|
||||
if number not in records:
|
||||
failures.add("rests-on", rel(path), f"no such record: {entry}")
|
||||
continue
|
||||
status = records[number]["front"].get("status")
|
||||
if status != "accepted":
|
||||
# A proposed record may extend another proposed one. Decisions are drafted in
|
||||
# chains -- 0059 extends 0057 while both await review -- and refusing that would
|
||||
# mean either drafting out of order or marking records accepted to satisfy a
|
||||
# check, which is the failure this repository already made once.
|
||||
if frontmatter(read(path)).get("status") == "proposed":
|
||||
continue
|
||||
# An as-is document describes what runs, and what runs was built under
|
||||
# whatever was decided at the time. ADR 0056: "as-is describing a superseded
|
||||
# decision is exactly what as-is is for."
|
||||
if rel(path).startswith("03-DESIGN/00-as-is/"):
|
||||
continue
|
||||
# A withdrawn record's citations are history. It instructs nobody -- every reader
|
||||
# is sent to its superseder -- so what it was built on may itself be withdrawn.
|
||||
# Refusing that would mean rewriting the lineage of a record whose reasoning is
|
||||
# the thing the immutability rule protects.
|
||||
if frontmatter(read(path)).get("status") == "superseded":
|
||||
continue
|
||||
# An extension that supersedes legitimately names what it replaced.
|
||||
this = ADR_FILE.match(os.path.basename(path))
|
||||
supersedes = records[number]["front"].get("superseded-by", "")
|
||||
if this and supersedes and os.path.basename(path) in str(supersedes):
|
||||
continue
|
||||
failures.add(
|
||||
"rests-on",
|
||||
rel(path),
|
||||
f"rests on ADR {number}, which is '{status}' — a document may not rest on a "
|
||||
f"record that is not accepted",
|
||||
)
|
||||
|
||||
|
||||
def check_live_citations(failures, records):
|
||||
"""In a governing document, a link to a superseded record must name its replacement.
|
||||
|
||||
The reader of a rule needs to know the rule was withdrawn, and needs somewhere to go.
|
||||
Naming the superseder in the same paragraph is both, and it is what a person would
|
||||
write anyway.
|
||||
"""
|
||||
superseded = {
|
||||
number: record["front"].get("superseded-by", "")
|
||||
for number, record in records.items()
|
||||
if record["front"].get("status") == "superseded"
|
||||
}
|
||||
|
||||
for path in markdown_files():
|
||||
if not any(rel(path).startswith(prefix) for prefix in GOVERNING):
|
||||
continue
|
||||
text = read(path)
|
||||
offset = 0
|
||||
for paragraph in text.split("\n\n"):
|
||||
line_no = text[:offset].count("\n") + 1
|
||||
offset += len(paragraph) + 2
|
||||
targets = LINK.findall(paragraph)
|
||||
for target in targets:
|
||||
match = ADR_FILE.match(os.path.basename(target.split("#")[0]))
|
||||
if not match or match.group(1) not in superseded:
|
||||
continue
|
||||
number = match.group(1)
|
||||
replacement = os.path.basename(str(superseded[number]))
|
||||
if not replacement:
|
||||
failures.add(
|
||||
"live-citation",
|
||||
f"{rel(path)}:{line_no}",
|
||||
f"cites superseded ADR {number}, which names no superseder",
|
||||
)
|
||||
continue
|
||||
if not any(replacement in t for t in targets):
|
||||
failures.add(
|
||||
"live-citation",
|
||||
f"{rel(path)}:{line_no}",
|
||||
f"cites superseded ADR {number} without naming its replacement "
|
||||
f"({replacement}) in the same paragraph",
|
||||
)
|
||||
|
||||
|
||||
def check_supersession_symmetry(failures, records):
|
||||
"""If A says it was superseded by B, B must say it supersedes A."""
|
||||
for number, record in records.items():
|
||||
front = record["front"]
|
||||
status = front.get("status")
|
||||
by = os.path.basename(str(front.get("superseded-by", "")))
|
||||
|
||||
if status == "superseded" and not by:
|
||||
failures.add("supersession", rel(record["path"]), "marked superseded but names no superseder")
|
||||
continue
|
||||
if by and status != "superseded":
|
||||
failures.add("supersession", rel(record["path"]), f"names a superseder but status is '{status}'")
|
||||
if not by:
|
||||
continue
|
||||
|
||||
match = ADR_FILE.match(by)
|
||||
if not match or match.group(1) not in records:
|
||||
failures.add("supersession", rel(record["path"]), f"superseder does not exist: {by}")
|
||||
continue
|
||||
other = records[match.group(1)]
|
||||
# `supersedes:` may name one record or several. One decision replacing two is a real
|
||||
# situation -- two records that built and refined the same wrong mechanism are withdrawn
|
||||
# by the one record that removes it -- and a check that allows only one would force
|
||||
# either a chain of pro-forma records or an unmarked supersession.
|
||||
claimed = other["front"].get("supersedes", "")
|
||||
if isinstance(claimed, str):
|
||||
claimed = [claimed] if claimed else []
|
||||
claims = [os.path.basename(str(entry)) for entry in claimed]
|
||||
if record["name"] not in claims:
|
||||
failures.add(
|
||||
"supersession",
|
||||
rel(other["path"]),
|
||||
f"ADR {number} says this supersedes it; this record does not say so "
|
||||
f"(supersedes: {', '.join(claims) or 'absent'})",
|
||||
)
|
||||
|
||||
|
||||
def check_topics(failures, records):
|
||||
"""Every record names a topic of the reading order.
|
||||
|
||||
The topic is what puts a record in the reading order, so a record without one — or with one
|
||||
nobody defined — disappears from the generated list rather than appearing in the wrong place.
|
||||
That is the quiet failure, so it is the one checked. The topics are the ones
|
||||
`02-DECISIONS/README.md` writes down, read the way `index.py` reads them, so there is one list.
|
||||
"""
|
||||
from index import reading_order
|
||||
known = {topic for topic, _ in reading_order()}
|
||||
for number, record in sorted(records.items()):
|
||||
topic = record["front"].get("topic")
|
||||
if not topic:
|
||||
failures.add("topics", rel(record["path"]),
|
||||
"no topic, so it has no place in the reading order")
|
||||
elif topic not in known:
|
||||
failures.add("topics", rel(record["path"]),
|
||||
"topic %r is not one of: %s" % (topic, ", ".join(sorted(known))))
|
||||
|
||||
|
||||
def check_numbering(failures, records):
|
||||
"""The number in the filename is the number in the heading."""
|
||||
for number, record in records.items():
|
||||
heading = re.search(r"^# (\d+)\.", record["text"], re.M)
|
||||
if not heading:
|
||||
failures.add("numbering", rel(record["path"]), "no '# N. Title' heading")
|
||||
elif heading.group(1) != number.lstrip("0"):
|
||||
failures.add(
|
||||
"numbering",
|
||||
rel(record["path"]),
|
||||
f"filename says {number}, heading says {heading.group(1)}",
|
||||
)
|
||||
|
||||
|
||||
def check_progressive_insights(failures, records):
|
||||
"""A correction made inside a record is marked and dated, or it is a silent rewrite.
|
||||
|
||||
A record may be corrected in place when a *fact* in it went stale and the decision still
|
||||
stands (`02-DECISIONS/README.md`, "Progressive insight"). The whole safety of that allowance
|
||||
is that the correction is legible in the record rather than only in a diff nobody reads, so
|
||||
the form is what is checked here: every mention of an insight is the marker, the marker
|
||||
carries an ISO date, and that date is not earlier than the decision's own — an insight
|
||||
predating the decision it corrects is a copied marker, not a correction.
|
||||
|
||||
What this cannot check is an edit made with no marker at all. Nothing mechanical can; that
|
||||
one is the reviewer's, reading the diff. The check keeps the *marked* path honest so that an
|
||||
unmarked change stands out as the anomaly it is.
|
||||
"""
|
||||
phrase = re.compile(r"progressive insight", re.I)
|
||||
# Both patterns stay on one line: a bold run does not span paragraphs, and `[^*]*` across
|
||||
# newlines will happily join an unrelated `**` far above to the marker below, reporting the
|
||||
# whole span between them. It did exactly that the first time this ran.
|
||||
# Trailing words after the date are allowed — "— 2026-09-26, correcting the one above." — so
|
||||
# an insight can say what it relates to. Only the date's presence and position are fixed.
|
||||
marker = re.compile(r"\*\*Progressive insights?[ \t]*[\u2014\u2013-][ \t]*(\d{4}-\d{2}-\d{2})[^*\n]*\*\*")
|
||||
loose = re.compile(r"\*\*[^*\n]*[Pp]rogressive insights?[^*\n]*\*\*")
|
||||
iso = re.compile(r"^\d{4}-\d{2}-\d{2}$")
|
||||
|
||||
for number, record in sorted(records.items()):
|
||||
text = record["text"]
|
||||
if not phrase.search(text):
|
||||
continue
|
||||
decided = str(record["front"].get("date", ""))
|
||||
good = [(m.start(), m.end(), m.group(1)) for m in marker.finditer(text)]
|
||||
|
||||
for m in loose.finditer(text):
|
||||
if any(s <= m.start() and m.end() <= e for s, e, _ in good):
|
||||
continue
|
||||
# A bold run carrying a link is discussing an insight — usually another record's —
|
||||
# rather than marking one. A marker never needs to cite anything.
|
||||
if "](" in m.group(0):
|
||||
continue
|
||||
failures.add("insights", rel(record["path"]),
|
||||
"a progressive insight is not in the dated marked form "
|
||||
"'**Progressive insight \u2014 YYYY-MM-DD.**': %s" % m.group(0))
|
||||
|
||||
for _, _, stamp in good:
|
||||
if decided and iso.match(decided) and stamp < decided:
|
||||
failures.add("insights", rel(record["path"]),
|
||||
"a progressive insight dated %s predates the decision (%s)"
|
||||
% (stamp, decided))
|
||||
|
||||
covered = [(s, e) for s, e, _ in good]
|
||||
for m in phrase.finditer(text):
|
||||
if any(s <= m.start() and m.end() <= e for s, e in covered):
|
||||
continue
|
||||
line = text.rfind("\n", 0, m.start()) + 1
|
||||
end = text.find("\n", m.end())
|
||||
whole = text[line:end if end != -1 else len(text)]
|
||||
if whole.lstrip().startswith("#"):
|
||||
continue
|
||||
# A line that also carries a link is discussing the rule, not marking a correction:
|
||||
# a marker never needs to cite anything, and a record that reasons about the policy
|
||||
# must be able to name it. Bare prose with no citation is the informal marking this
|
||||
# is here to catch.
|
||||
if "](" in whole:
|
||||
continue
|
||||
if loose.search(text, line, text.find("\n", m.end()) + 1 or len(text)):
|
||||
continue
|
||||
failures.add("insights", rel(record["path"]),
|
||||
"'progressive insight' appears unmarked; a correction is marked and "
|
||||
"dated, or it is a silent rewrite")
|
||||
|
||||
|
||||
def check_status_against_code(failures):
|
||||
"""A design document naming specific code may not still call itself `designed`.
|
||||
|
||||
**Naming a file is a claim that the file implements this**, so the two fields have to agree.
|
||||
They drifted: ten to-be documents named working, lab-proven code — several with a *What was
|
||||
built* or *Raised, and observed* section — while still saying nothing had been built.
|
||||
|
||||
Deliberately weak, and that is the point of it being mechanical. It cannot tell whether the
|
||||
prose is true, only that a document has stopped claiming to be unbuilt once it points at
|
||||
something. `code: [mesh-controller]` — a repository with no path — is a plan and stays
|
||||
`designed`.
|
||||
"""
|
||||
for path in markdown_files():
|
||||
if not rel(path).startswith("03-DESIGN/01-to-be/") or path.endswith("README.md"):
|
||||
continue
|
||||
front = frontmatter(read(path))
|
||||
if front.get("status") != "designed":
|
||||
continue
|
||||
for entry in front.get("code") or []:
|
||||
named = re.sub(r"\s*\(.*\)$", "", entry).strip().split(None, 1)
|
||||
if len(named) > 1:
|
||||
failures.add(
|
||||
"status-vs-code",
|
||||
rel(path),
|
||||
f"`designed`, but names {named[1]!r} in {named[0]}. Naming a file claims "
|
||||
f"it implements this — use `in-progress`, or `implemented` once it is "
|
||||
f"defensible from that repository's main branch.",
|
||||
)
|
||||
break
|
||||
|
||||
|
||||
def main():
|
||||
failures = Failures()
|
||||
records = load_records()
|
||||
check_links(failures)
|
||||
check_rests_on(failures, records)
|
||||
check_live_citations(failures, records)
|
||||
check_supersession_symmetry(failures, records)
|
||||
check_numbering(failures, records)
|
||||
check_topics(failures, records)
|
||||
check_status_against_code(failures)
|
||||
check_progressive_insights(failures, records)
|
||||
print(f"records: {len(records)} decision records checked")
|
||||
return failures.report()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -1,206 +0,0 @@
|
||||
package checks
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"path"
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
)
|
||||
|
||||
// Structural checks over hq's own documents (records.py until ADR 0527). The decision records and research
|
||||
// efforts live in the knowledge base (ADR 0290, issue 426); what stays here judges what this repository still
|
||||
// holds, and keeps the records from coming back.
|
||||
|
||||
var (
|
||||
linkRx = regexp.MustCompile(`\[[^\]]*\]\(([^)]+)\)`)
|
||||
// A decision record is named "ADR NNNN", four digits, and a research effort "research NNN", three.
|
||||
decisionEntryRx = regexp.MustCompile(`^ADR \d{4}$`)
|
||||
adrMentionRx = regexp.MustCompile(`\bADRs?\s+(\d+)\b`)
|
||||
researchMentionRx = regexp.MustCompile(`\b[Rr]esearch\s+(\d+)\b`)
|
||||
codeSpanRx = regexp.MustCompile("`[^`]*`")
|
||||
recordFolders = []string{"02-DECISIONS", "01-RESEARCH"}
|
||||
recordFolderPoints = map[string]bool{"02-DECISIONS/README.md": true, "01-RESEARCH/README.md": true}
|
||||
)
|
||||
|
||||
// checkLinks: every relative link resolves to something that exists.
|
||||
func checkLinks(fsys fs.FS) []string {
|
||||
var out []string
|
||||
for _, p := range markdownFiles(fsys) {
|
||||
for n, line := range strings.Split(readFile(fsys, p), "\n") {
|
||||
for _, m := range linkRx.FindAllStringSubmatch(line, -1) {
|
||||
target := m[1]
|
||||
if strings.HasPrefix(target, "http:") || strings.HasPrefix(target, "https:") || strings.HasPrefix(target, "mailto:") {
|
||||
continue
|
||||
}
|
||||
target = strings.TrimSpace(strings.SplitN(target, "#", 2)[0])
|
||||
if target == "" {
|
||||
continue
|
||||
}
|
||||
if _, err := fs.Stat(fsys, path.Join(path.Dir(p), target)); err != nil {
|
||||
out = append(out, fmt.Sprintf("links: %s:%d: link does not resolve: %s", p, n+1, target))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// checkNoRecords: the two folders the records left hold their pointer README and nothing else (ADR 0290 rule 9).
|
||||
func checkNoRecords(fsys fs.FS) []string {
|
||||
var out []string
|
||||
for _, folder := range recordFolders {
|
||||
if _, err := fs.Stat(fsys, folder); err != nil {
|
||||
out = append(out, "no-records: "+folder+": the folder is gone; it keeps a README that points at the knowledge base")
|
||||
continue
|
||||
}
|
||||
_ = fs.WalkDir(fsys, folder, func(p string, d fs.DirEntry, err error) error {
|
||||
if err != nil || p == folder || recordFolderPoints[p] {
|
||||
return nil
|
||||
}
|
||||
out = append(out, "no-records: "+p+": a decision record or research effort lives in the knowledge base, never here: "+
|
||||
"write it with knowledge-base.reserve and knowledge-base.propose (playbooks 01 and 02)")
|
||||
return nil
|
||||
})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// checkDecisionsEntries: a design's `decisions:` names each record as "ADR NNNN".
|
||||
func checkDecisionsEntries(fsys fs.FS) []string {
|
||||
var out []string
|
||||
for _, p := range markdownFiles(fsys) {
|
||||
if !strings.HasPrefix(p, "03-DESIGN/") {
|
||||
continue
|
||||
}
|
||||
for _, e := range frontmatter(readFile(fsys, p)).list("decisions") {
|
||||
if !decisionEntryRx.MatchString(unquote(e)) {
|
||||
out = append(out, fmt.Sprintf("decisions-form: %s: decisions: entry %q is not 'ADR NNNN' (four digits)", p, e))
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// checkMentions: in running text a decision record is "ADR NNNN" and a research effort "research NNN"; a code
|
||||
// span is an example and is not read.
|
||||
func checkMentions(fsys fs.FS) []string {
|
||||
var out []string
|
||||
for _, p := range markdownFiles(fsys) {
|
||||
for n, line := range strings.Split(readFile(fsys, p), "\n") {
|
||||
line = codeSpanRx.ReplaceAllString(line, "")
|
||||
for _, m := range adrMentionRx.FindAllStringSubmatch(line, -1) {
|
||||
if len(m[1]) != 4 {
|
||||
out = append(out, fmt.Sprintf("mention-form: %s:%d: %q: a decision record is named 'ADR NNNN', four digits", p, n+1, m[0]))
|
||||
}
|
||||
}
|
||||
for _, m := range researchMentionRx.FindAllStringSubmatch(line, -1) {
|
||||
if len(m[1]) != 3 {
|
||||
out = append(out, fmt.Sprintf("mention-form: %s:%d: %q: a research effort is named 'research NNN', three digits", p, n+1, m[0]))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
var codeEntryRx = regexp.MustCompile(`\s*\(.*\)$`)
|
||||
|
||||
// checkStatusAgainstCode: a to-be design naming specific code (a path after the repository) may not still be
|
||||
// `designed`. Deliberately weak: it cannot tell whether the prose is true, only that a document stopped claiming
|
||||
// to be unbuilt once it points at something. `code: [mesh-controller]` is a plan and stays `designed`.
|
||||
func checkStatusAgainstCode(fsys fs.FS) []string {
|
||||
var out []string
|
||||
for _, p := range markdownFiles(fsys) {
|
||||
if !strings.HasPrefix(p, "03-DESIGN/01-to-be/") || strings.HasSuffix(p, "README.md") {
|
||||
continue
|
||||
}
|
||||
front := frontmatter(readFile(fsys, p))
|
||||
if front.scalar("status") != "designed" {
|
||||
continue
|
||||
}
|
||||
for _, e := range front.list("code") {
|
||||
named := strings.SplitN(strings.TrimSpace(codeEntryRx.ReplaceAllString(e, "")), " ", 2)
|
||||
if len(named) > 1 {
|
||||
out = append(out, fmt.Sprintf("status-vs-code: %s: `designed`, but names %q in %s. Naming a file claims it implements this "+
|
||||
"- use `in-progress`, or `implemented` once it is defensible from that repository's main branch.", p, strings.TrimSpace(named[1]), named[0]))
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func doc(s string) *fstest.MapFile { return &fstest.MapFile{Data: []byte(s)} }
|
||||
|
||||
// The real tree.
|
||||
|
||||
func TestRecordsLinksResolve(t *testing.T) { expectNone(t, checkLinks(rootFS())) }
|
||||
func TestRecordsFoldersHoldOnlyPointers(t *testing.T) { expectNone(t, checkNoRecords(rootFS())) }
|
||||
func TestRecordsDecisionsEntriesForm(t *testing.T) { expectNone(t, checkDecisionsEntries(rootFS())) }
|
||||
func TestRecordsMentionForm(t *testing.T) { expectNone(t, checkMentions(rootFS())) }
|
||||
func TestRecordsStatusAgainstCode(t *testing.T) { expectNone(t, checkStatusAgainstCode(rootFS())) }
|
||||
|
||||
// The checks themselves, each against a broken fixture, and against a sound one.
|
||||
|
||||
func TestRecordsLinksCatchesABrokenLink(t *testing.T) {
|
||||
broken := fstest.MapFS{
|
||||
"a/one.md": doc("see [two](two.md) and [gone](gone.md#x) and [web](https://example.com/x)\n"),
|
||||
"a/two.md": doc("[up](../a/one.md#top) [anchor](#here)\n"),
|
||||
}
|
||||
got := checkLinks(broken)
|
||||
expectProblem(t, got, "a/one.md:1: link does not resolve: gone.md")
|
||||
if len(got) != 1 {
|
||||
t.Errorf("only the broken link is a problem, got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRecordsNoRecordsCatchesARecordPutBack(t *testing.T) {
|
||||
broken := fstest.MapFS{
|
||||
"02-DECISIONS/README.md": doc("pointer"),
|
||||
"02-DECISIONS/0001-x.md": doc("a record"),
|
||||
"01-RESEARCH/README.md": doc("pointer"),
|
||||
"01-RESEARCH/045/00-x.md": doc("an effort"),
|
||||
}
|
||||
got := checkNoRecords(broken)
|
||||
expectProblem(t, got, "02-DECISIONS/0001-x.md")
|
||||
expectProblem(t, got, "01-RESEARCH/045")
|
||||
expectProblem(t, checkNoRecords(fstest.MapFS{"02-DECISIONS/README.md": doc("p")}), "01-RESEARCH: the folder is gone")
|
||||
}
|
||||
|
||||
func TestRecordsDecisionsFormCatchesAShortEntry(t *testing.T) {
|
||||
broken := fstest.MapFS{
|
||||
"03-DESIGN/01-to-be/1.md": doc("---\nlayer: to-be\ndecisions:\n - ADR 0290\n - ADR 290\n---\n"),
|
||||
"00-META/x.md": doc("---\ndecisions: [ADR 5]\n---\n"),
|
||||
}
|
||||
got := checkDecisionsEntries(broken)
|
||||
expectProblem(t, got, `entry "ADR 290"`)
|
||||
if len(got) != 1 {
|
||||
t.Errorf("only a design's entries are read, and ADR 0290 is sound: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRecordsMentionFormCatchesShortNumbers(t *testing.T) {
|
||||
broken := fstest.MapFS{"x.md": doc("fine ADR 0290 and research 045\nbad ADR 12 and research 45\nexample `ADR 7` and `research 9`\n")}
|
||||
got := checkMentions(broken)
|
||||
expectProblem(t, got, "x.md:2: \"ADR 12\"")
|
||||
expectProblem(t, got, "x.md:2: \"research 45\"")
|
||||
if len(got) != 2 {
|
||||
t.Errorf("code spans and sound forms are not problems: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRecordsStatusAgainstCodeCatchesADesignedDesignNamingAFile(t *testing.T) {
|
||||
broken := fstest.MapFS{
|
||||
"03-DESIGN/01-to-be/9.md": doc("---\nstatus: designed\ncode:\n - mesh-controller internal/x.go (the thing)\n---\n"),
|
||||
"03-DESIGN/01-to-be/8.md": doc("---\nstatus: designed\ncode: [mesh-controller]\n---\n"),
|
||||
"03-DESIGN/01-to-be/7.md": doc("---\nstatus: in-progress\ncode: [mesh-controller internal/x.go]\n---\n"),
|
||||
}
|
||||
got := checkStatusAgainstCode(broken)
|
||||
expectProblem(t, got, "03-DESIGN/01-to-be/9.md: `designed`, but names \"internal/x.go\" in mesh-controller")
|
||||
if len(got) != 1 {
|
||||
t.Errorf("a plan without a path and an in-progress design are sound: %q", got)
|
||||
}
|
||||
}
|
||||
@@ -1,39 +0,0 @@
|
||||
package checks
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
)
|
||||
|
||||
func TestMarkdownFilesSkipsToolFolders(t *testing.T) {
|
||||
fsys := fstest.MapFS{
|
||||
"a.md": {Data: []byte("x")},
|
||||
"b/c.md": {Data: []byte("x")},
|
||||
"b/c.txt": {Data: []byte("x")},
|
||||
".git/d.md": {Data: []byte("x")},
|
||||
".claude/e.md": {Data: []byte("x")},
|
||||
".work/f/g.md": {Data: []byte("x")},
|
||||
}
|
||||
got := markdownFiles(fsys)
|
||||
want := []string{"a.md", "b/c.md"}
|
||||
if !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("markdownFiles = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFrontmatterReadsScalarsAndLists(t *testing.T) {
|
||||
f := frontmatter("---\nlayer: to-be\ndecisions:\n - ADR 0001\n - ADR 0002\ncode: []\ncodes: [a, b]\n---\nbody\n")
|
||||
if f.scalar("layer") != "to-be" {
|
||||
t.Errorf("layer = %q", f.scalar("layer"))
|
||||
}
|
||||
if !reflect.DeepEqual(f.list("decisions"), []string{"ADR 0001", "ADR 0002"}) {
|
||||
t.Errorf("decisions = %q", f.list("decisions"))
|
||||
}
|
||||
if len(f.list("code")) != 0 || !reflect.DeepEqual(f.list("codes"), []string{"a", "b"}) {
|
||||
t.Errorf("code = %q, codes = %q", f.list("code"), f.list("codes"))
|
||||
}
|
||||
if frontmatter("no frontmatter") != nil || frontmatter("---\nunclosed") != nil {
|
||||
t.Error("a document without a closed block has no frontmatter")
|
||||
}
|
||||
}
|
||||
@@ -1,150 +0,0 @@
|
||||
// Package checks holds hq's four checks as Go tests (ADR 0527): records, index, cycle and words.
|
||||
//
|
||||
// The package has tests only. Each check is a function over an fs.FS that returns its problems, so
|
||||
// the test of the real tree reads the repository's root, and the test of the check itself reads a
|
||||
// deliberately broken tree built in memory and must see the problem named (every check here failed on
|
||||
// a broken fixture before it passed). The shared reading of Markdown and frontmatter is in this file.
|
||||
package checks
|
||||
|
||||
import (
|
||||
"io/fs"
|
||||
"os"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// rootFS is hq's tree: the checks run from 00-META/checks, two folders under the root.
|
||||
func rootFS() fs.FS { return os.DirFS("../..") }
|
||||
|
||||
// skipDirs are folders no check reads: git's, the agent tooling's and a session's scratch folder.
|
||||
var skipDirs = map[string]bool{".git": true, ".claude": true, ".work": true}
|
||||
|
||||
// markdownFiles lists every Markdown file of the tree, sorted, in slash paths from the root.
|
||||
func markdownFiles(fsys fs.FS) []string {
|
||||
var out []string
|
||||
_ = fs.WalkDir(fsys, ".", func(p string, d fs.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
if d.IsDir() {
|
||||
if p != "." && skipDirs[d.Name()] {
|
||||
return fs.SkipDir
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if strings.HasSuffix(p, ".md") {
|
||||
out = append(out, p)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
func readFile(fsys fs.FS, p string) string {
|
||||
b, err := fs.ReadFile(fsys, p)
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
return string(b)
|
||||
}
|
||||
|
||||
// Front is a document's frontmatter: a scalar field holds a string, a list field a []string.
|
||||
type Front map[string]any
|
||||
|
||||
func (f Front) scalar(key string) string {
|
||||
s, _ := f[key].(string)
|
||||
return s
|
||||
}
|
||||
|
||||
func (f Front) list(key string) []string {
|
||||
switch v := f[key].(type) {
|
||||
case []string:
|
||||
return v
|
||||
case string:
|
||||
if strings.TrimSpace(v) != "" {
|
||||
return []string{v}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
var (
|
||||
fmItem = regexp.MustCompile(`^\s+-\s*(.+?)\s*$`)
|
||||
fmPair = regexp.MustCompile(`^([A-Za-z_-]+):\s*(.*)$`)
|
||||
)
|
||||
|
||||
// frontmatter reads the block between the first two --- lines. It is not a YAML parser on purpose: a
|
||||
// dependency to read a few scalar fields and one list would cost more than it returns. It returns nil
|
||||
// when the document has no frontmatter.
|
||||
func frontmatter(text string) Front {
|
||||
if !strings.HasPrefix(text, "---\n") {
|
||||
return nil
|
||||
}
|
||||
end := strings.Index(text[4:], "\n---")
|
||||
if end == -1 {
|
||||
return nil
|
||||
}
|
||||
out := Front{}
|
||||
key := ""
|
||||
for _, line := range strings.Split(text[4:4+end], "\n") {
|
||||
if m := fmItem.FindStringSubmatch(line); m != nil && key != "" {
|
||||
list, _ := out[key].([]string)
|
||||
out[key] = append(list, m[1])
|
||||
continue
|
||||
}
|
||||
m := fmPair.FindStringSubmatch(line)
|
||||
if m == nil {
|
||||
continue
|
||||
}
|
||||
key = m[1]
|
||||
value := strings.TrimSpace(m[2])
|
||||
switch {
|
||||
case strings.HasPrefix(value, "[") && strings.HasSuffix(value, "]"):
|
||||
list := []string{}
|
||||
for _, v := range strings.Split(value[1:len(value)-1], ",") {
|
||||
if v = strings.TrimSpace(v); v != "" {
|
||||
list = append(list, v)
|
||||
}
|
||||
}
|
||||
out[key] = list
|
||||
case value == "":
|
||||
out[key] = []string{}
|
||||
default:
|
||||
out[key] = value
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func unquote(v string) string {
|
||||
v = strings.TrimSpace(v)
|
||||
if len(v) >= 2 && v[0] == v[len(v)-1] && (v[0] == '"' || v[0] == '\'') {
|
||||
return v[1 : len(v)-1]
|
||||
}
|
||||
return v
|
||||
}
|
||||
|
||||
// lineOf is the 1-based line of byte offset i in text.
|
||||
func lineOf(text string, i int) int { return strings.Count(text[:i], "\n") + 1 }
|
||||
|
||||
// expectNone fails the test with every problem a check found in the real tree.
|
||||
func expectNone(t *testing.T, problems []string) {
|
||||
t.Helper()
|
||||
for _, p := range problems {
|
||||
t.Error(p)
|
||||
}
|
||||
}
|
||||
|
||||
// expectProblem fails unless some problem contains want: the broken fixture must be caught, by name.
|
||||
func expectProblem(t *testing.T, problems []string, want string) {
|
||||
t.Helper()
|
||||
for _, p := range problems {
|
||||
if strings.Contains(p, want) {
|
||||
return
|
||||
}
|
||||
}
|
||||
t.Errorf("no problem mentions %q; got %q", want, problems)
|
||||
}
|
||||
@@ -1,22 +1,16 @@
|
||||
# Words allowed — documents the words test does not hold to the glossary's retired words
|
||||
# Words allowed — documents `words.py` does not hold to the glossary's retired words
|
||||
|
||||
Read by the words test (`words_test.go`) (ADR 0244).
|
||||
Read by [`words.py`](words.py) ([ADR 0244](../../02-DECISIONS/0244-the-mesh-is-described-in-domains-and-one-word-names-one-thing.md)).
|
||||
One row per document. *Until* is the date by which it is reworded — past it, the row fails like the words
|
||||
themselves. A row whose document no longer uses a retired word fails too, so this list only shrinks. The
|
||||
research efforts this list kept as written (research 034 and 042) live in the knowledge base now (ADR 0290),
|
||||
which keeps a record's words as it was imported.
|
||||
themselves — or `kept` for a graduated research effort, which records what was said and keeps its words
|
||||
the way a decision record does. A row whose document no longer uses a retired word fails too, so this
|
||||
list only shrinks.
|
||||
|
||||
| Document | Until | Why |
|
||||
|---|---|---|
|
||||
| `04-ISSUES/285-the-merge-gate-compared-broken-to-broken-and-passed/00-report.md` | 2027-04-30 | history: the issue's narrative keeps the word of its day, "merge gate" (issue 1303); it is not rewritten |
|
||||
| `04-ISSUES/300-a-new-module-is-not-built-at-its-merge/00-report.md` | 2027-04-30 | history: the issue's narrative keeps the word of its day, "merge gate" (issue 1303); it is not rewritten |
|
||||
| `04-ISSUES/302-the-labs-go-replays-were-never-checked-before-merge/01-diagnosis.md` | 2027-04-30 | history: the issue's narrative keeps the word of its day, "merge gate" (issue 1303); it is not rewritten |
|
||||
| `04-ISSUES/305-a-recheck-left-the-old-verdict-standing/00-report.md` | 2027-04-30 | history: the issue's narrative keeps the word of its day, "merge gate" (issue 1303); it is not rewritten |
|
||||
| `04-ISSUES/336-a-send-held-for-the-bus-step-raises-nothing/00-report.md` | 2027-04-30 | history: the issue's narrative keeps the word of its day, "merge gate" (issue 1303); it is not rewritten |
|
||||
| `04-ISSUES/336-a-send-held-for-the-bus-step-raises-nothing/01-diagnosis.md` | 2027-04-30 | history: the issue's narrative keeps the word of its day, "merge gate" (issue 1303); it is not rewritten |
|
||||
| `04-ISSUES/338-a-module-built-from-a-shared-repository-moves-on-every-merge-to-it/01-diagnosis.md` | 2027-04-30 | history: the issue's narrative keeps the word of its day, "merge gate" (issue 1303); it is not rewritten |
|
||||
| `04-ISSUES/341-a-merge-check-judges-the-head-not-what-merging-it-makes/01-diagnosis.md` | 2027-04-30 | history: the issue's narrative keeps the word of its day, "merge gate" (issue 1303); it is not rewritten |
|
||||
| `04-ISSUES/353-the-controller-asked-before-the-bus-held-its-grant/00-report.md` | 2027-04-30 | history: the issue's narrative keeps the word of its day, "merge gate" (issue 1303); it is not rewritten |
|
||||
| `04-ISSUES/354-a-decided-rename-was-never-scheduled-and-the-operator-believed-it-live/01-diagnosis.md` | 2027-04-30 | history: the issue's narrative keeps the word of its day, "merge gate" (issue 1303); it is not rewritten |
|
||||
| `04-ISSUES/362-two-merges-seconds-apart-left-their-delivery-waiting-for-a-person/01-diagnosis.md` | 2027-04-30 | history: the issue's narrative keeps the word of its day, "merge gate" (issue 1303); it is not rewritten |
|
||||
| `04-ISSUES/364-the-gate-refuses-a-module-that-uses-a-seat-another-module-declares/00-report.md` | 2027-04-30 | history: the issue's narrative keeps the word of its day, "merge gate" (issue 1303); it is not rewritten |
|
||||
| `01-RESEARCH/034-the-mesh-in-domains/00-overview.md` | kept | the effort that studied these words; its findings quote and propose them as they stood before ADR 0244 |
|
||||
| `01-RESEARCH/034-the-mesh-in-domains/01-the-concepts-in-use.md` | kept | as above: the inventory of the words in use |
|
||||
| `01-RESEARCH/034-the-mesh-in-domains/02-the-domains.md` | kept | as above: the proposed domains, in the words of the day |
|
||||
| `01-RESEARCH/034-the-mesh-in-domains/03-the-clashes.md` | kept | as above: every clash names the words that clashed |
|
||||
| `01-RESEARCH/034-the-mesh-in-domains/04-how-the-glossary-is-checked.md` | kept | as above: the check's own examples |
|
||||
| `01-RESEARCH/034-the-mesh-in-domains/05-a-glossary-by-domain.md` | kept | as above: the proposed glossary, with the entries of the day |
|
||||
|
||||
@@ -0,0 +1,283 @@
|
||||
#!/usr/bin/env python3
|
||||
"""One word per thing, checked (ADR 0244).
|
||||
|
||||
The glossary (00-META/glossary.md) is the authority on the mesh's words. It names the words it
|
||||
retired on one fixed kind of line, so a program can read them:
|
||||
|
||||
*Not:* ~~control plane~~, ~~overlay~~ (hq)
|
||||
|
||||
A struck word with no scope is retired everywhere; `(hq)` retires it in this repository's prose only;
|
||||
`(tools)` in the descriptions of the mesh's tools only. And it names code that still carries an old
|
||||
name until that code is renamed:
|
||||
|
||||
*Identifier until renamed:* `mesh-host` — the repository, binary and service unit
|
||||
|
||||
An identifier is allowed inside a code span and nowhere else.
|
||||
|
||||
What is enforced:
|
||||
|
||||
retired no retired word of scope `hq` (or none), and no identifier, in running prose of a
|
||||
checked document. Running prose is what is left once code blocks, code spans, block
|
||||
quotes, text in quotation marks, struck-through text, link targets, HTML comments and
|
||||
frontmatter are taken out: a quotation keeps the words it quotes, a link target is a file
|
||||
name, and an identifier lives in a code span. A word's plural is the word: "alerts" fails
|
||||
as "alert" does.
|
||||
unique no head word (a bolded word opening a glossary entry) heads two entries, and no head
|
||||
word is also struck through on a *Not:* line.
|
||||
tools list with `--list tools` it prints the words retired in the tools' descriptions, which is the
|
||||
copy the catalogue's own check keeps (mesh-catalog `retired-words`). With
|
||||
MESH_CATALOG_DIR set to a checkout of the catalogue it also fails when that copy differs.
|
||||
|
||||
Checked documents: 00-META/, 03-DESIGN/ (both layers), AGENTS.md and README.md, always; a research
|
||||
effort initiated, or an issue opened, on or after FROM. Never 02-DECISIONS/: a decision record keeps the
|
||||
words it was written with.
|
||||
|
||||
A document the check fails on that cannot be reworded in the change that found it is named, with a date
|
||||
and a reason, in words-allowed.md beside this file. An entry past its date fails like the word itself, and
|
||||
an entry for a document that no longer needs it fails too. `kept` instead of a date is allowed only for a
|
||||
graduated research effort, which is a record of what was said, like a decision record.
|
||||
|
||||
python3 00-META/checks/words.py
|
||||
python3 00-META/checks/words.py --list tools
|
||||
"""
|
||||
|
||||
import datetime
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
ROOT = os.path.normpath(os.path.join(os.path.dirname(__file__), "..", ".."))
|
||||
GLOSSARY = "00-META/glossary.md"
|
||||
ALLOWED = "00-META/checks/words-allowed.md"
|
||||
|
||||
# The day the rule began (ADR 0244): research and issues from it on are held to it.
|
||||
FROM = "2026-10-07"
|
||||
|
||||
ALWAYS = ("00-META/", "03-DESIGN/", "AGENTS.md", "README.md")
|
||||
|
||||
NOT_LINE = re.compile(r"^\s*(?:[-*]\s+)?\*Not:\*(.*)$", re.M)
|
||||
STRUCK = re.compile(r"~~([^~]+)~~(?:\s*\((hq|tools)\))?")
|
||||
IDENT_LINE = re.compile(r"^\s*(?:[-*]\s+)?\*Identifier until renamed:\*(.*)$", re.M)
|
||||
CODE_SPAN = re.compile(r"`([^`]+)`")
|
||||
HEAD = re.compile(r"^\s*-\s+\*\*([^*]+)\*\*", re.M)
|
||||
|
||||
|
||||
def rel(path):
|
||||
return os.path.relpath(path, ROOT)
|
||||
|
||||
|
||||
def read(path):
|
||||
with open(os.path.join(ROOT, path), encoding="utf-8") as handle:
|
||||
return handle.read()
|
||||
|
||||
|
||||
def frontmatter_field(text, name):
|
||||
if not text.startswith("---\n"):
|
||||
return None
|
||||
end = text.find("\n---", 4)
|
||||
m = re.search(r"^%s:\s*(\S+)" % name, text[4:end], re.M)
|
||||
return m.group(1) if m else None
|
||||
|
||||
|
||||
def glossary():
|
||||
"""The retired words with their scope, the identifiers, and the head words."""
|
||||
text = read(GLOSSARY)
|
||||
retired = []
|
||||
for line in NOT_LINE.findall(text):
|
||||
for word, scope in STRUCK.findall(line):
|
||||
for one in word.split(" / "):
|
||||
retired.append((one.strip(), scope or "all"))
|
||||
identifiers = []
|
||||
for line in IDENT_LINE.findall(text):
|
||||
identifiers += [i.strip() for i in CODE_SPAN.findall(line)]
|
||||
heads = []
|
||||
for head in HEAD.findall(text):
|
||||
heads += [h.strip() for h in head.split(" / ")]
|
||||
return retired, identifiers, heads
|
||||
|
||||
|
||||
def pattern(word):
|
||||
"""A whole-word, case-insensitive match; a space in the word matches a space, a line break or a hyphen.
|
||||
|
||||
The plural is the word too: its last part may end in `s` or `es` ("alerts", "control planes"), since
|
||||
a retired word does not come back by being counted. Anything else joined on is another word.
|
||||
"""
|
||||
parts = [re.escape(p) for p in word.split()]
|
||||
return re.compile(r"(?i)(?<![\w-])" + r"[\s-]+".join(parts) + r"(?:e?s)?(?![\w-])")
|
||||
|
||||
|
||||
def blank(match):
|
||||
return re.sub(r"[^\n]", " ", match.group(0))
|
||||
|
||||
|
||||
def prose(text, glossary_file=False):
|
||||
"""The text with everything that is not running prose blanked, keeping offsets and line numbers."""
|
||||
if text.startswith("---\n"):
|
||||
end = text.find("\n---", 4)
|
||||
if end != -1:
|
||||
text = re.sub(r"[^\n]", " ", text[: end + 4]) + text[end + 4:]
|
||||
steps = [
|
||||
r"(?ms)^\s*```.*?^\s*```", # code blocks
|
||||
r"(?s)<!--.*?-->", # comments
|
||||
r"(?m)^\s*>.*$", # block quotes
|
||||
r"`[^`\n]+`", # code spans
|
||||
r"\]\([^)\s]*\)", # link targets
|
||||
r"~~[^~\n]+~~", # struck through: a word named as retired
|
||||
r"\"[^\"\n]*\"", # "quoted"
|
||||
r"\u201c[^\u201d]*\u201d", # curly double quotes
|
||||
r"\u2018[^\u2019\n]*\u2019", # curly single quotes
|
||||
]
|
||||
if glossary_file:
|
||||
steps[:0] = [NOT_LINE.pattern, IDENT_LINE.pattern]
|
||||
for step in steps:
|
||||
text = re.sub(step, blank, text)
|
||||
return text
|
||||
|
||||
|
||||
def research_and_issues():
|
||||
"""The research efforts and issue reports dated on or after FROM, file by file."""
|
||||
out = []
|
||||
for folder, key, first in (("01-RESEARCH", "initiated", "00-overview.md"), ("04-ISSUES", "opened", "00-report.md")):
|
||||
base = os.path.join(ROOT, folder)
|
||||
for effort in sorted(os.listdir(base)):
|
||||
head = os.path.join(base, effort, first)
|
||||
if not os.path.isfile(head):
|
||||
continue
|
||||
date = frontmatter_field(read(rel(head)), key)
|
||||
if not date or date < FROM:
|
||||
continue
|
||||
for name in sorted(os.listdir(os.path.join(base, effort))):
|
||||
if name.endswith(".md"):
|
||||
out.append("%s/%s/%s" % (folder, effort, name))
|
||||
return out
|
||||
|
||||
|
||||
def checked_documents():
|
||||
docs = []
|
||||
for entry in ALWAYS:
|
||||
path = os.path.join(ROOT, entry)
|
||||
if os.path.isfile(path):
|
||||
docs.append(entry)
|
||||
continue
|
||||
for base, dirs, files in os.walk(path):
|
||||
dirs.sort()
|
||||
for name in sorted(files):
|
||||
if name.endswith(".md"):
|
||||
docs.append(rel(os.path.join(base, name)))
|
||||
return docs + research_and_issues()
|
||||
|
||||
|
||||
def allowed():
|
||||
"""words-allowed.md: one row per document — | `document` | until | why |.
|
||||
|
||||
`until` is a date, after which the allowance fails like the words themselves, or `kept` for a
|
||||
document that records what was said and must keep its words: only a research effort that has
|
||||
graduated may be kept, because once it has become a decision it is a record like one.
|
||||
"""
|
||||
out, problems = {}, []
|
||||
if not os.path.isfile(os.path.join(ROOT, ALLOWED)):
|
||||
return out, problems
|
||||
today = datetime.date.today().isoformat()
|
||||
for line in read(ALLOWED).splitlines():
|
||||
cells = [c.strip() for c in line.strip().strip("|").split("|")]
|
||||
if len(cells) != 3 or not cells[0].startswith("`"):
|
||||
continue
|
||||
doc, until, why = cells[0].strip("`"), cells[1], cells[2]
|
||||
if not why:
|
||||
problems.append("%s: the allowance for %s gives no reason" % (ALLOWED, doc))
|
||||
if until == "kept":
|
||||
overview = os.path.join(os.path.dirname(doc), "00-overview.md")
|
||||
status = frontmatter_field(read(overview), "status") if os.path.isfile(os.path.join(ROOT, overview)) else None
|
||||
if not doc.startswith("01-RESEARCH/") or status != "graduated":
|
||||
problems.append("%s: %s is kept, but only a graduated research effort may be" % (ALLOWED, doc))
|
||||
continue
|
||||
elif not re.match(r"^\d{4}-\d{2}-\d{2}$", until):
|
||||
problems.append("%s: the allowance for %s has neither a date nor `kept`" % (ALLOWED, doc))
|
||||
continue
|
||||
elif until < today:
|
||||
problems.append("%s: the allowance for %s ran out on %s — reword it" % (ALLOWED, doc, until))
|
||||
continue
|
||||
out[doc] = until
|
||||
return out, problems
|
||||
|
||||
|
||||
def check_retired(retired, identifiers):
|
||||
problems = []
|
||||
allowance, problems_allowed = allowed()
|
||||
problems += problems_allowed
|
||||
words = [(w, pattern(w), "retired") for w, scope in retired if scope in ("all", "hq")]
|
||||
words += [(i, pattern(i), "an identifier, allowed only in a code span") for i in identifiers]
|
||||
used = set()
|
||||
for doc in checked_documents():
|
||||
text = prose(read(doc), glossary_file=(doc == GLOSSARY))
|
||||
for word, rx, why in words:
|
||||
for m in rx.finditer(text):
|
||||
if doc in allowance:
|
||||
used.add(doc)
|
||||
continue
|
||||
line = text.count("\n", 0, m.start()) + 1
|
||||
problems.append("%s:%d: %r is %s (glossary)" % (doc, line, m.group(0).replace("\n", " "), why))
|
||||
for doc in allowance:
|
||||
if doc not in used:
|
||||
problems.append("%s: %s no longer uses a retired word — remove its allowance" % (ALLOWED, doc))
|
||||
return problems
|
||||
|
||||
|
||||
def check_unique(retired, heads):
|
||||
problems = []
|
||||
seen = {}
|
||||
for head in heads:
|
||||
key = head.lower()
|
||||
if key in seen:
|
||||
problems.append("%s: %r heads two entries" % (GLOSSARY, head))
|
||||
seen[key] = True
|
||||
for word, _ in retired:
|
||||
if word.lower() in seen:
|
||||
problems.append("%s: %r is a head word and also retired" % (GLOSSARY, word))
|
||||
return problems
|
||||
|
||||
|
||||
def tools_list(retired):
|
||||
return sorted({w for w, scope in retired if scope in ("all", "tools")}, key=str.lower)
|
||||
|
||||
|
||||
def check_copy(retired):
|
||||
catalogue = os.environ.get("MESH_CATALOG_DIR")
|
||||
if not catalogue:
|
||||
print("NOT COMPARED: MESH_CATALOG_DIR is not set, so the catalogue's copy of the list was not read")
|
||||
return []
|
||||
path = os.path.join(catalogue, "retired-words")
|
||||
try:
|
||||
with open(path, encoding="utf-8") as handle:
|
||||
copy = [l.strip() for l in handle if l.strip() and not l.startswith("#")]
|
||||
except OSError as err:
|
||||
return ["the catalogue's copy of the retired words cannot be read: %s" % err]
|
||||
want = tools_list(retired)
|
||||
if sorted(copy, key=str.lower) != want:
|
||||
return ["the catalogue's retired-words differs from the glossary: it should list %s" % ", ".join(want)]
|
||||
return []
|
||||
|
||||
|
||||
def main(argv):
|
||||
retired, identifiers, heads = glossary()
|
||||
if argv[1:2] == ["--list"]:
|
||||
scope = argv[2] if len(argv) > 2 else "tools"
|
||||
words = tools_list(retired) if scope == "tools" else sorted({w for w, s in retired if s in ("all", scope)})
|
||||
print("\n".join(words))
|
||||
return 0
|
||||
if not retired:
|
||||
print("words: the glossary names no retired word on a *Not:* line — the check would pass on anything")
|
||||
return 1
|
||||
problems = check_unique(retired, heads) + check_retired(retired, identifiers) + check_copy(retired)
|
||||
for p in problems:
|
||||
print(p)
|
||||
if problems:
|
||||
print("words: %d problem(s)" % len(problems))
|
||||
return 1
|
||||
print("words: %d retired words and %d identifiers, none in running prose; %d head words, each once"
|
||||
% (len(retired), len(identifiers), len(heads)))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
@@ -1,756 +0,0 @@
|
||||
package checks
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path"
|
||||
"regexp"
|
||||
"runtime"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
"time"
|
||||
"unicode"
|
||||
)
|
||||
|
||||
// One word per thing, checked (words.py until ADR 0527; ADR 0244). The glossary (00-META/glossary.md) is the
|
||||
// authority on the mesh's words. It names the words it retired on one fixed kind of line, so a program can read them:
|
||||
//
|
||||
// *Not:* ~~control plane~~, ~~overlay~~ (hq)
|
||||
//
|
||||
// A struck word with no scope is retired everywhere; `(hq)` retires it in this repository's prose only; `(tools)`
|
||||
// in the descriptions of the mesh's tools only. And it names code that still carries an old name until it is renamed:
|
||||
//
|
||||
// *Identifier until renamed:* `mesh-host` - the repository, binary and service unit
|
||||
//
|
||||
// An identifier is allowed inside a code span and nowhere else.
|
||||
//
|
||||
// What is enforced:
|
||||
//
|
||||
// retired no retired word of scope hq (or none), and no identifier, in running prose of a checked document.
|
||||
// Running prose is what is left once code blocks, code spans, block quotes, text in quotation marks,
|
||||
// struck-through text, link targets, HTML comments and frontmatter are taken out. A word's plural is the word.
|
||||
// unique no head word heads two entries, and none is also struck through on a *Not:* line.
|
||||
// lines no daemon's line in a checked document (ADR 0272, its insight of 2026-10-09): the operator types `nox`.
|
||||
// copy with MESH_CATALOG_DIR set to a checkout of the catalogue, its copies of the retired words and the
|
||||
// daemons' lines equal this glossary's and this list (skipped, with its reason, when unset).
|
||||
//
|
||||
// Checked documents: 00-META/, 03-DESIGN/ (both layers), AGENTS.md and README.md, always; an issue opened on or after
|
||||
// wordsFrom (on or after linesFrom for the lines). A document that cannot be reworded in the change that found it is
|
||||
// named, with a date and a reason, in words-allowed.md beside this file; an entry past its date fails like the word
|
||||
// itself, and so does an entry for a document that no longer needs it.
|
||||
|
||||
const (
|
||||
glossaryFile = "00-META/glossary.md"
|
||||
allowedFile = "00-META/checks/words-allowed.md"
|
||||
// The day the rule began (ADR 0244): issues opened from it on are held to it.
|
||||
wordsFrom = "2026-10-07"
|
||||
// The day the daemons' lines rule began.
|
||||
linesFrom = "2026-10-09"
|
||||
)
|
||||
|
||||
var wordsAlways = []string{"00-META/", "03-DESIGN/", "AGENTS.md", "README.md"}
|
||||
|
||||
// daemonLines are the daemons' lines (ADR 0272): each daemon's commands. "alone" is a command that is a line by
|
||||
// itself; "with an argument" one that is a line only when written as one ("the mesh-controller module" is prose;
|
||||
// `mesh-controller module check ...` is a line). The node-engine's install-time subcommands are not lines here.
|
||||
var daemonLines = []struct {
|
||||
daemon, alone, withArgument string
|
||||
}{
|
||||
{"mesh-controller",
|
||||
"adopt assign broker cancel cleanup collect conditions converge doctor durations hand-act hand-acts healers identity kill licence " +
|
||||
"migrate operator pause pin plan prepare push rebuild rehearse replay resume retire rollout rotate secret settings status token " +
|
||||
"unassign unpin upgrade",
|
||||
"artifacts ask builds bus collection data delivery images mirrors module node overlay plans queue seat seats take version"},
|
||||
{"mesh-host", "apply bundle enrol enroll hand-over help inventory key overlay owned profile reconcile run version", ""},
|
||||
{"nox-mesh-host", "hand-over", ""},
|
||||
}
|
||||
|
||||
var (
|
||||
notLineRx = regexp.MustCompile(`(?m)^\s*(?:[-*]\s+)?\*Not:\*(.*)$`)
|
||||
struckRx = regexp.MustCompile(`~~([^~]+)~~(?:\s*\((hq|tools)\))?`)
|
||||
identLineRx = regexp.MustCompile(`(?m)^\s*(?:[-*]\s+)?\*Identifier until renamed:\*(.*)$`)
|
||||
spanRx = regexp.MustCompile("`([^`]+)`")
|
||||
headRx = regexp.MustCompile(`(?m)^\s*-\s+\*\*([^*]+)\*\*`)
|
||||
openedRx = regexp.MustCompile(`(?m)^opened:\s*(\S+)`)
|
||||
)
|
||||
|
||||
type retiredWord struct{ word, scope string }
|
||||
|
||||
// parseGlossary reads the retired words with their scope, the identifiers and the head words from the glossary.
|
||||
func parseGlossary(text string) (retired []retiredWord, identifiers, heads []string) {
|
||||
for _, m := range notLineRx.FindAllStringSubmatch(text, -1) {
|
||||
for _, s := range struckRx.FindAllStringSubmatch(m[1], -1) {
|
||||
scope := s[2]
|
||||
if scope == "" {
|
||||
scope = "all"
|
||||
}
|
||||
for _, one := range strings.Split(s[1], " / ") {
|
||||
retired = append(retired, retiredWord{strings.TrimSpace(one), scope})
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, m := range identLineRx.FindAllStringSubmatch(text, -1) {
|
||||
for _, s := range spanRx.FindAllStringSubmatch(m[1], -1) {
|
||||
identifiers = append(identifiers, strings.TrimSpace(s[1]))
|
||||
}
|
||||
}
|
||||
for _, m := range headRx.FindAllStringSubmatch(text, -1) {
|
||||
for _, h := range strings.Split(m[1], " / ") {
|
||||
heads = append(heads, strings.TrimSpace(h))
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
func isWordRune(r rune) bool {
|
||||
return unicode.IsLetter(r) || unicode.IsDigit(r) || r == '_' || r == '-'
|
||||
}
|
||||
|
||||
// findBounded returns the start offsets of the matches of rx that are whole words: no word character or hyphen
|
||||
// before or after (the lookarounds of the Python check, which RE2 has not).
|
||||
func findBounded(rx *regexp.Regexp, text string) []int {
|
||||
var out []int
|
||||
pos := 0
|
||||
for pos <= len(text) {
|
||||
loc := rx.FindStringIndex(text[pos:])
|
||||
if loc == nil {
|
||||
break
|
||||
}
|
||||
start, end := pos+loc[0], pos+loc[1]
|
||||
before, after := true, true
|
||||
if start > 0 {
|
||||
r := []rune(text[max(0, start-4):start])
|
||||
before = !isWordRune(r[len(r)-1])
|
||||
}
|
||||
if end < len(text) {
|
||||
r := []rune(text[end:min(len(text), end+4)])
|
||||
after = !isWordRune(r[0])
|
||||
}
|
||||
if before && after {
|
||||
out = append(out, start)
|
||||
pos = end
|
||||
if end == start {
|
||||
pos++
|
||||
}
|
||||
} else {
|
||||
pos = start + 1
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// wordRegexp matches a word whole, case-insensitively; a space in the word matches a space, a line break or a hyphen.
|
||||
// The plural is the word too: its last part may end in s or es ("alerts", "control planes").
|
||||
func wordRegexp(word string) *regexp.Regexp {
|
||||
var parts []string
|
||||
for _, p := range strings.Fields(word) {
|
||||
parts = append(parts, regexp.QuoteMeta(p))
|
||||
}
|
||||
return regexp.MustCompile(`(?i)` + strings.Join(parts, `[\s-]+`) + `(?:e?s)?`)
|
||||
}
|
||||
|
||||
// blank replaces every byte of s but the line breaks, so offsets and line numbers hold.
|
||||
func blank(s string) string {
|
||||
b := []byte(s)
|
||||
for i, c := range b {
|
||||
if c != '\n' {
|
||||
b[i] = ' '
|
||||
}
|
||||
}
|
||||
return string(b)
|
||||
}
|
||||
|
||||
func blankFrontmatter(text string) string {
|
||||
if strings.HasPrefix(text, "---\n") {
|
||||
if end := strings.Index(text[4:], "\n---"); end != -1 {
|
||||
end += 4
|
||||
return blank(text[:end+4]) + text[end+4:]
|
||||
}
|
||||
}
|
||||
return text
|
||||
}
|
||||
|
||||
func blankAll(text string, steps []*regexp.Regexp) string {
|
||||
for _, rx := range steps {
|
||||
text = rx.ReplaceAllStringFunc(text, blank)
|
||||
}
|
||||
return text
|
||||
}
|
||||
|
||||
func compileAll(patterns ...string) []*regexp.Regexp {
|
||||
var out []*regexp.Regexp
|
||||
for _, p := range patterns {
|
||||
out = append(out, regexp.MustCompile(p))
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// What is not the document's own words, in the order it is taken out.
|
||||
const (
|
||||
stepComment = `(?s)<!--.*?-->`
|
||||
stepBlockQuote = `(?m)^\s*>.*$`
|
||||
stepLinkTarget = `\]\([^)\s]*\)`
|
||||
stepStruck = `~~[^~\n]+~~`
|
||||
stepQuoted = `"[^"\n]*"`
|
||||
stepCurlyDouble = `\x{201c}[^\x{201d}]*\x{201d}`
|
||||
stepCurlySingle = `\x{2018}[^\x{2019}\n]*\x{2019}`
|
||||
)
|
||||
|
||||
var (
|
||||
proseSteps = compileAll("(?ms)^\\s*```.*?^\\s*```", stepComment, stepBlockQuote, "`[^`\\n]+`", stepLinkTarget, stepStruck,
|
||||
stepQuoted, stepCurlyDouble, stepCurlySingle)
|
||||
// In the glossary the *Not:* and *Identifier until renamed:* lines name the retired words, so they are taken out first.
|
||||
glossaryProseSteps = append(compileAll(notLineRx.String(), identLineRx.String()), proseSteps...)
|
||||
// The daemons' lines are looked for in the document's own words: a code block and a code span are kept, since a
|
||||
// line to type is written in one.
|
||||
ownWordsSteps = compileAll(stepComment, stepBlockQuote, stepLinkTarget, stepStruck, stepQuoted, stepCurlyDouble, stepCurlySingle)
|
||||
)
|
||||
|
||||
// prose is the text with everything that is not running prose blanked, keeping offsets and line numbers.
|
||||
func prose(text string, glossaryDoc bool) string {
|
||||
text = blankFrontmatter(text)
|
||||
if glossaryDoc {
|
||||
return blankAll(text, glossaryProseSteps)
|
||||
}
|
||||
return blankAll(text, proseSteps)
|
||||
}
|
||||
|
||||
func ownWords(text string) string { return blankAll(blankFrontmatter(text), ownWordsSteps) }
|
||||
|
||||
// issueDocuments lists the issue reports' files opened on or after since.
|
||||
func issueDocuments(fsys fs.FS, since string) []string {
|
||||
var out []string
|
||||
entries, _ := fs.ReadDir(fsys, "04-ISSUES")
|
||||
for _, e := range entries {
|
||||
if !e.IsDir() {
|
||||
continue
|
||||
}
|
||||
head := readFile(fsys, path.Join("04-ISSUES", e.Name(), "00-report.md"))
|
||||
front := ""
|
||||
if strings.HasPrefix(head, "---\n") {
|
||||
if end := strings.Index(head[4:], "\n---"); end != -1 {
|
||||
if m := openedRx.FindStringSubmatch(head[4 : 4+end]); m != nil {
|
||||
front = m[1]
|
||||
}
|
||||
}
|
||||
}
|
||||
if front == "" || front < since {
|
||||
continue
|
||||
}
|
||||
files, _ := fs.ReadDir(fsys, path.Join("04-ISSUES", e.Name()))
|
||||
for _, f := range files {
|
||||
if strings.HasSuffix(f.Name(), ".md") {
|
||||
out = append(out, path.Join("04-ISSUES", e.Name(), f.Name()))
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// perDocument runs fn over the documents on all processors (the unit level is short) and joins the problems in
|
||||
// the documents' order.
|
||||
func perDocument(docs []string, fn func(doc string) []string) []string {
|
||||
results := make([][]string, len(docs))
|
||||
var wg sync.WaitGroup
|
||||
sem := make(chan struct{}, runtime.NumCPU())
|
||||
for i, d := range docs {
|
||||
wg.Add(1)
|
||||
sem <- struct{}{}
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
results[i] = fn(d)
|
||||
<-sem
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
var out []string
|
||||
for _, r := range results {
|
||||
out = append(out, r...)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func checkedDocuments(fsys fs.FS, since string) []string {
|
||||
var docs []string
|
||||
for _, entry := range wordsAlways {
|
||||
if strings.HasSuffix(entry, "/") {
|
||||
_ = fs.WalkDir(fsys, strings.TrimSuffix(entry, "/"), func(p string, d fs.DirEntry, err error) error {
|
||||
if err == nil && !d.IsDir() && strings.HasSuffix(p, ".md") {
|
||||
docs = append(docs, p)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
} else if _, err := fs.Stat(fsys, entry); err == nil {
|
||||
docs = append(docs, entry)
|
||||
}
|
||||
}
|
||||
return append(docs, issueDocuments(fsys, since)...)
|
||||
}
|
||||
|
||||
var dateRx = regexp.MustCompile(`^\d{4}-\d{2}-\d{2}$`)
|
||||
|
||||
// parseAllowed reads words-allowed.md: one row per document, | `document` | until | why |. Until is a date, after
|
||||
// which the allowance fails like the words themselves. The clock is the caller's: today is YYYY-MM-DD.
|
||||
func parseAllowed(text, today string) (allowed map[string]string, problems []string) {
|
||||
allowed = map[string]string{}
|
||||
for _, line := range strings.Split(text, "\n") {
|
||||
cells := strings.Split(strings.Trim(strings.TrimSpace(line), "|"), "|")
|
||||
if len(cells) != 3 {
|
||||
continue
|
||||
}
|
||||
for i := range cells {
|
||||
cells[i] = strings.TrimSpace(cells[i])
|
||||
}
|
||||
if !strings.HasPrefix(cells[0], "`") {
|
||||
continue
|
||||
}
|
||||
doc, until, why := strings.Trim(cells[0], "`"), cells[1], cells[2]
|
||||
if why == "" {
|
||||
problems = append(problems, fmt.Sprintf("%s: the allowance for %s gives no reason", allowedFile, doc))
|
||||
}
|
||||
if !dateRx.MatchString(until) {
|
||||
problems = append(problems, fmt.Sprintf("%s: the allowance for %s has no date", allowedFile, doc))
|
||||
continue
|
||||
}
|
||||
if until < today {
|
||||
problems = append(problems, fmt.Sprintf("%s: the allowance for %s ran out on %s — reword it", allowedFile, doc, until))
|
||||
continue
|
||||
}
|
||||
allowed[doc] = until
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
func checkRetired(fsys fs.FS, retired []retiredWord, identifiers []string, today string) []string {
|
||||
allowance, out := parseAllowed(readFile(fsys, allowedFile), today)
|
||||
type matcher struct {
|
||||
word string
|
||||
rx *regexp.Regexp
|
||||
why string
|
||||
}
|
||||
var words []matcher
|
||||
for _, r := range retired {
|
||||
if r.scope == "all" || r.scope == "hq" {
|
||||
words = append(words, matcher{r.word, wordRegexp(r.word), "retired"})
|
||||
}
|
||||
}
|
||||
for _, i := range identifiers {
|
||||
words = append(words, matcher{i, wordRegexp(i), "an identifier, allowed only in a code span"})
|
||||
}
|
||||
var mu sync.Mutex
|
||||
used := map[string]bool{}
|
||||
out = append(out, perDocument(checkedDocuments(fsys, wordsFrom), func(d string) []string {
|
||||
var found []string
|
||||
text := prose(readFile(fsys, d), d == glossaryFile)
|
||||
lower := strings.ToLower(text)
|
||||
for _, w := range words {
|
||||
// Every match holds the word's first part literally, so a text without it is not searched (the unit level is short).
|
||||
if !strings.Contains(lower, strings.ToLower(strings.Fields(w.word)[0])) {
|
||||
continue
|
||||
}
|
||||
for _, start := range findBounded(w.rx, text) {
|
||||
if _, ok := allowance[d]; ok {
|
||||
mu.Lock()
|
||||
used[d] = true
|
||||
mu.Unlock()
|
||||
continue
|
||||
}
|
||||
m := w.rx.FindString(text[start:])
|
||||
found = append(found, fmt.Sprintf("%s:%d: %q is %s (glossary)", d, lineOf(text, start), strings.ReplaceAll(m, "\n", " "), w.why))
|
||||
}
|
||||
}
|
||||
return found
|
||||
})...)
|
||||
var unused []string
|
||||
for d := range allowance {
|
||||
if !used[d] {
|
||||
unused = append(unused, d)
|
||||
}
|
||||
}
|
||||
sort.Strings(unused)
|
||||
for _, d := range unused {
|
||||
out = append(out, fmt.Sprintf("%s: %s no longer uses a retired word — remove its allowance", allowedFile, d))
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func checkUnique(retired []retiredWord, heads []string) []string {
|
||||
var out []string
|
||||
seen := map[string]bool{}
|
||||
for _, h := range heads {
|
||||
k := strings.ToLower(h)
|
||||
if seen[k] {
|
||||
out = append(out, fmt.Sprintf("%s: %q heads two entries", glossaryFile, h))
|
||||
}
|
||||
seen[k] = true
|
||||
}
|
||||
for _, r := range retired {
|
||||
if seen[strings.ToLower(r.word)] {
|
||||
out = append(out, fmt.Sprintf("%s: %q is a head word and also retired", glossaryFile, r.word))
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// linesList is the daemons' lines as the catalogue's copy lists them: "<daemon> <command>", with " …" when an
|
||||
// argument must follow.
|
||||
func linesList() []string {
|
||||
var out []string
|
||||
for _, d := range daemonLines {
|
||||
for _, c := range strings.Fields(d.alone) {
|
||||
out = append(out, d.daemon+" "+c)
|
||||
}
|
||||
for _, c := range strings.Fields(d.withArgument) {
|
||||
out = append(out, d.daemon+" "+c+" …")
|
||||
}
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
func checkLines(fsys fs.FS) []string {
|
||||
type pat struct {
|
||||
daemon, command string
|
||||
rx *regexp.Regexp
|
||||
bounded bool
|
||||
}
|
||||
var pats []pat
|
||||
for _, d := range daemonLines {
|
||||
dq := regexp.QuoteMeta(d.daemon)
|
||||
for _, c := range strings.Fields(d.alone) {
|
||||
pats = append(pats, pat{d.daemon, c, regexp.MustCompile(dq + `\s+` + regexp.QuoteMeta(c)), true})
|
||||
}
|
||||
for _, c := range strings.Fields(d.withArgument) {
|
||||
// A line only when written as one: at the start of a line or a code span, a prompt allowed, an argument after.
|
||||
pats = append(pats, pat{d.daemon, c, regexp.MustCompile("(?m)(?:^|`)[ \\t]*(?:\\$[ \\t]*)?(" + dq + `\s+` + regexp.QuoteMeta(c) + "\\s+[^\\s`])"), false})
|
||||
}
|
||||
}
|
||||
return perDocument(checkedDocuments(fsys, linesFrom), func(d string) []string {
|
||||
var out []string
|
||||
text := ownWords(readFile(fsys, d))
|
||||
for _, p := range pats {
|
||||
if !strings.Contains(text, p.daemon) {
|
||||
continue
|
||||
}
|
||||
var starts []int
|
||||
if p.bounded {
|
||||
starts = findBounded(p.rx, text)
|
||||
} else {
|
||||
for _, loc := range p.rx.FindAllStringSubmatchIndex(text, -1) {
|
||||
starts = append(starts, loc[2])
|
||||
}
|
||||
}
|
||||
for _, s := range starts {
|
||||
out = append(out, fmt.Sprintf("%s:%d: `%s %s` is a daemon's line — the operator types `nox …` (ADR 0272)", d, lineOf(text, s), p.daemon, p.command))
|
||||
}
|
||||
}
|
||||
return out
|
||||
})
|
||||
}
|
||||
|
||||
func toolsList(retired []retiredWord) []string {
|
||||
seen := map[string]bool{}
|
||||
var out []string
|
||||
for _, r := range retired {
|
||||
if (r.scope == "all" || r.scope == "tools") && !seen[r.word] {
|
||||
seen[r.word] = true
|
||||
out = append(out, r.word)
|
||||
}
|
||||
}
|
||||
sort.Slice(out, func(i, j int) bool { return strings.ToLower(out[i]) < strings.ToLower(out[j]) })
|
||||
return out
|
||||
}
|
||||
|
||||
func readList(file string) ([]string, error) {
|
||||
b, err := os.ReadFile(file)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var out []string
|
||||
for _, l := range strings.Split(string(b), "\n") {
|
||||
if l = strings.TrimSpace(l); l != "" && !strings.HasPrefix(l, "#") {
|
||||
out = append(out, l)
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// checkCopy compares the catalogue's copies of the retired words and the daemons' lines with this glossary's. A
|
||||
// difference names what the copy should list, so regenerating it is copying that from the failure.
|
||||
func checkCopy(catalogue string, retired []retiredWord) []string {
|
||||
var out []string
|
||||
copyWords, err := readList(path.Join(catalogue, "retired-words"))
|
||||
if err != nil {
|
||||
return []string{"the catalogue's copy of the retired words cannot be read: " + err.Error()}
|
||||
}
|
||||
sort.Slice(copyWords, func(i, j int) bool { return strings.ToLower(copyWords[i]) < strings.ToLower(copyWords[j]) })
|
||||
want := toolsList(retired)
|
||||
if strings.Join(copyWords, "\n") != strings.Join(want, "\n") {
|
||||
out = append(out, "the catalogue's retired-words differs from the glossary: it should list "+strings.Join(want, ", "))
|
||||
}
|
||||
copyLines, err := readList(path.Join(catalogue, "retired-lines"))
|
||||
if err != nil {
|
||||
return append(out, "the catalogue's copy of the daemons' lines cannot be read: "+err.Error())
|
||||
}
|
||||
sort.Strings(copyLines)
|
||||
if strings.Join(copyLines, "\n") != strings.Join(linesList(), "\n") {
|
||||
out = append(out, "the catalogue's retired-lines differs from this check's list: it should list, one per line:\n"+strings.Join(linesList(), "\n"))
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// The real tree.
|
||||
|
||||
func today() string { return time.Now().Format("2006-01-02") }
|
||||
|
||||
func TestWordsGlossaryNamesRetiredWords(t *testing.T) {
|
||||
retired, identifiers, heads := parseGlossary(readFile(rootFS(), glossaryFile))
|
||||
if len(retired) == 0 {
|
||||
t.Fatal("the glossary names no retired word on a *Not:* line — the check would pass on anything")
|
||||
}
|
||||
if len(identifiers) == 0 || len(heads) == 0 {
|
||||
t.Errorf("the glossary names %d identifiers and %d head words; both are expected", len(identifiers), len(heads))
|
||||
}
|
||||
}
|
||||
|
||||
func TestWordsEachHeadWordHeadsOneEntry(t *testing.T) {
|
||||
retired, _, heads := parseGlossary(readFile(rootFS(), glossaryFile))
|
||||
expectNone(t, checkUnique(retired, heads))
|
||||
}
|
||||
|
||||
func TestWordsNoRetiredWordInRunningProse(t *testing.T) {
|
||||
retired, identifiers, _ := parseGlossary(readFile(rootFS(), glossaryFile))
|
||||
expectNone(t, checkRetired(rootFS(), retired, identifiers, today()))
|
||||
}
|
||||
|
||||
func TestWordsNoDaemonLineWritten(t *testing.T) { expectNone(t, checkLines(rootFS())) }
|
||||
|
||||
func TestWordsCatalogueCopyAgrees(t *testing.T) {
|
||||
catalogue := os.Getenv("MESH_CATALOG_DIR")
|
||||
if catalogue == "" {
|
||||
t.Skip("NOT COMPARED: MESH_CATALOG_DIR is not set, so the catalogue's copy of the list was not read")
|
||||
}
|
||||
retired, _, _ := parseGlossary(readFile(rootFS(), glossaryFile))
|
||||
expectNone(t, checkCopy(catalogue, retired))
|
||||
}
|
||||
|
||||
// The checks themselves, against broken trees (H5a: the glossary).
|
||||
|
||||
const glossaryFixture = `# Glossary
|
||||
|
||||
- **control plane** — the retired one's replacement is below.
|
||||
*Not:* ~~control plane~~, ~~overlay~~ (hq), ~~alert~~ (tools), ~~a / b~~
|
||||
- **controller** — decides.
|
||||
*Not:* ~~control plane~~
|
||||
- **Controller** — again.
|
||||
- *Identifier until renamed:* ` + "`mesh-host`, `nox-old`" + ` — the repository
|
||||
`
|
||||
|
||||
func TestGlossaryReadsNotLinesScopesIdentifiersAndHeads(t *testing.T) {
|
||||
retired, identifiers, heads := parseGlossary(glossaryFixture)
|
||||
want := []retiredWord{{"control plane", "all"}, {"overlay", "hq"}, {"alert", "tools"}, {"a", "all"}, {"b", "all"}, {"control plane", "all"}}
|
||||
if fmt.Sprint(retired) != fmt.Sprint(want) {
|
||||
t.Errorf("retired = %v, want %v", retired, want)
|
||||
}
|
||||
if fmt.Sprint(identifiers) != "[mesh-host nox-old]" {
|
||||
t.Errorf("identifiers = %v", identifiers)
|
||||
}
|
||||
if fmt.Sprint(heads) != "[control plane controller Controller]" {
|
||||
t.Errorf("heads = %v", heads)
|
||||
}
|
||||
got := checkUnique(retired, heads)
|
||||
expectProblem(t, got, `"Controller" heads two entries`)
|
||||
expectProblem(t, got, `"control plane" is a head word and also retired`)
|
||||
}
|
||||
|
||||
func TestToolsListHoldsTheWordsOfScopeAllAndTools(t *testing.T) {
|
||||
retired, _, _ := parseGlossary(glossaryFixture)
|
||||
if got := fmt.Sprint(toolsList(retired)); got != "[a alert b control plane]" {
|
||||
t.Errorf("toolsList = %s", got)
|
||||
}
|
||||
}
|
||||
|
||||
// H5b: prose blanking and matching.
|
||||
|
||||
func TestProseBlanksWhatIsNotRunningProse(t *testing.T) {
|
||||
text := "---\nfm: overlay\n---\n" +
|
||||
"bare overlay one\n" +
|
||||
"```\noverlay in a block\n```\n" +
|
||||
"span `overlay` x\n" +
|
||||
"> quoted overlay\n" +
|
||||
"say \"overlay\" and “overlay” and ‘overlay’\n" +
|
||||
"~~overlay~~ struck\n" +
|
||||
"[link](overlay.md)\n" +
|
||||
"<!-- overlay\nover lines -->\n" +
|
||||
"bare overlay two\n"
|
||||
got := prose(text, false)
|
||||
if strings.Count(got, "overlay") != 2 {
|
||||
t.Errorf("only the two bare uses stay: %q", got)
|
||||
}
|
||||
if strings.Count(got, "\n") != strings.Count(text, "\n") || len(got) != len(text) {
|
||||
t.Error("blanking keeps offsets and line numbers")
|
||||
}
|
||||
}
|
||||
|
||||
func TestProseOfTheGlossaryBlanksNotAndIdentifierLines(t *testing.T) {
|
||||
got := prose("- **x** — y\n *Not:* control plane, plain\n- *Identifier until renamed:* mesh-host\nthe control plane\n", true)
|
||||
if strings.Count(got, "control plane") != 1 || strings.Contains(got, "mesh-host") {
|
||||
t.Errorf("only the running use stays: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWordMatchesWholeWordsPluralsAndLineBreaks(t *testing.T) {
|
||||
rx := wordRegexp("control plane")
|
||||
count := func(s string) int { return len(findBounded(rx, s)) }
|
||||
for s, want := range map[string]int{
|
||||
"the control plane runs": 1,
|
||||
"Control Planes": 1,
|
||||
"control-plane and control\nplane": 2,
|
||||
"uncontrol plane": 0,
|
||||
"control planet": 0,
|
||||
"control plane-x": 0,
|
||||
"x-control plane": 0,
|
||||
"control planexs": 0,
|
||||
} {
|
||||
if got := count(s); got != want {
|
||||
t.Errorf("%q: %d matches, want %d", s, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRetiredCatchesAWordInProseAndNamesDocumentAndLine(t *testing.T) {
|
||||
retired := []retiredWord{{"control plane", "all"}, {"overlay", "hq"}, {"alert", "tools"}}
|
||||
broken := mapOf(map[string]string{
|
||||
"AGENTS.md": "fine\nthe control plane here\n",
|
||||
"03-DESIGN/01-to-be/a.md": "an overlay and \"a control plane quoted\" and `overlay` alert\n",
|
||||
"README.md": "mesh-host runs; `mesh-host` too\n",
|
||||
})
|
||||
got := checkRetired(broken, retired, []string{"mesh-host"}, "2026-10-23")
|
||||
expectProblem(t, got, `AGENTS.md:2: "control plane" is retired`)
|
||||
expectProblem(t, got, `03-DESIGN/01-to-be/a.md:1: "overlay" is retired`)
|
||||
expectProblem(t, got, `README.md:1: "mesh-host" is an identifier, allowed only in a code span`)
|
||||
if len(got) != 3 {
|
||||
t.Errorf("a quotation, a code span and a word of scope tools are not problems: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestIssuesAreHeldToTheRuleFromItsDay(t *testing.T) {
|
||||
retired := []retiredWord{{"overlay", "all"}}
|
||||
fsys := mapOf(map[string]string{
|
||||
"04-ISSUES/1-old/00-report.md": "---\nopened: 2026-10-06\n---\noverlay\n",
|
||||
"04-ISSUES/2-new/00-report.md": "---\nopened: 2026-10-07\n---\nfine\n",
|
||||
"04-ISSUES/2-new/01-notes.md": "an overlay\n",
|
||||
"04-ISSUES/3-nodate/00-report.md": "overlay\n",
|
||||
})
|
||||
got := checkRetired(fsys, retired, nil, "2026-10-23")
|
||||
expectProblem(t, got, "04-ISSUES/2-new/01-notes.md:1")
|
||||
if len(got) != 1 {
|
||||
t.Errorf("only the issue opened from 2026-10-07 on is read: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
// H5c: allowances, with an injected clock.
|
||||
|
||||
const allowedFixture = `# Words allowed
|
||||
|
||||
| Document | Until | Why |
|
||||
|---|---|---|
|
||||
| ` + "`a.md`" + ` | 2026-12-01 | quotes what was |
|
||||
| ` + "`b.md`" + ` | 2026-10-01 | ran out |
|
||||
| ` + "`c.md`" + ` | soon | no date |
|
||||
| ` + "`d.md`" + ` | 2026-12-01 | |
|
||||
`
|
||||
|
||||
func TestAllowedReadsRowsAndTheInjectedClock(t *testing.T) {
|
||||
allowed, problems := parseAllowed(allowedFixture, "2026-10-23")
|
||||
if _, ok := allowed["a.md"]; !ok || len(allowed) != 2 {
|
||||
t.Errorf("a.md and d.md are in date: %v", allowed)
|
||||
}
|
||||
expectProblem(t, problems, "the allowance for b.md ran out on 2026-10-01")
|
||||
expectProblem(t, problems, "the allowance for c.md has no date")
|
||||
expectProblem(t, problems, "the allowance for d.md gives no reason")
|
||||
// The same text on an earlier day: b.md is still in date.
|
||||
if early, _ := parseAllowed(allowedFixture, "2026-09-30"); len(early) != 3 {
|
||||
t.Errorf("on 2026-09-30 a, b and d are in date: %v", early)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAllowanceHoldsAWordBackUntilItsDateAndShrinks(t *testing.T) {
|
||||
retired := []retiredWord{{"overlay", "all"}}
|
||||
allowedRow := "| `AGENTS.md` | 2026-12-01 | quotes what was |\n"
|
||||
tree := func(agents, allowed string) map[string]string {
|
||||
return map[string]string{"AGENTS.md": agents, allowedFile: allowed}
|
||||
}
|
||||
expectNone(t, checkRetired(mapOf(tree("an overlay\n", allowedRow)), retired, nil, "2026-10-23"))
|
||||
expectProblem(t, checkRetired(mapOf(tree("an overlay\n", allowedRow)), retired, nil, "2026-12-02"), "ran out on 2026-12-01")
|
||||
expectProblem(t, checkRetired(mapOf(tree("clean\n", allowedRow)), retired, nil, "2026-10-23"), "AGENTS.md no longer uses a retired word")
|
||||
expectProblem(t, checkRetired(mapOf(tree("an overlay\n", "")), retired, nil, "2026-10-23"), `AGENTS.md:1: "overlay" is retired`)
|
||||
}
|
||||
|
||||
// H5d: the daemons' lines and the catalogue comparison.
|
||||
|
||||
func TestLinesCatchesADaemonsLineButNotProse(t *testing.T) {
|
||||
fsys := mapOf(map[string]string{
|
||||
"AGENTS.md": "the mesh-controller module is prose\n" +
|
||||
"run `mesh-controller module check x` now\n" +
|
||||
"or:\n```\n$ mesh-controller push\n```\n" +
|
||||
"a quotation \"mesh-controller status\" stays\n" +
|
||||
"> mesh-controller doctor in a block quote\n" +
|
||||
"a mesh-controller pushed is another word\n",
|
||||
"04-ISSUES/1/00-report.md": "---\nopened: 2026-10-08\n---\nmesh-host apply\n",
|
||||
"04-ISSUES/2/00-report.md": "---\nopened: 2026-10-09\n---\nmesh-host apply\n",
|
||||
})
|
||||
got := checkLines(fsys)
|
||||
expectProblem(t, got, "AGENTS.md:2: `mesh-controller module` is a daemon's line")
|
||||
expectProblem(t, got, "AGENTS.md:5: `mesh-controller push` is a daemon's line")
|
||||
expectProblem(t, got, "04-ISSUES/2/00-report.md:4: `mesh-host apply`")
|
||||
if len(got) != 3 {
|
||||
t.Errorf("prose, quotations, block quotes and issues before 2026-10-09 are not problems: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLinesListNamesEachLineOnce(t *testing.T) {
|
||||
list := linesList()
|
||||
seen := map[string]bool{}
|
||||
for _, l := range list {
|
||||
if seen[l] {
|
||||
t.Errorf("%q listed twice", l)
|
||||
}
|
||||
seen[l] = true
|
||||
}
|
||||
if !seen["mesh-controller push"] || !seen["mesh-controller module …"] || !seen["nox-mesh-host hand-over"] {
|
||||
t.Errorf("the list lacks a known line: %v", list[:5])
|
||||
}
|
||||
}
|
||||
|
||||
func TestCatalogueCopyDifferenceIsCaughtAndNamesTheList(t *testing.T) {
|
||||
retired := []retiredWord{{"overlay", "all"}, {"alert", "tools"}, {"hq only", "hq"}}
|
||||
dir := t.TempDir()
|
||||
write := func(name, content string) {
|
||||
if err := os.WriteFile(path.Join(dir, name), []byte(content), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
write("retired-words", "# copy\nalert\noverlay\n")
|
||||
write("retired-lines", strings.Join(linesList(), "\n")+"\n")
|
||||
expectNone(t, checkCopy(dir, retired))
|
||||
|
||||
write("retired-words", "alert\n")
|
||||
write("retired-lines", "mesh-controller push\n")
|
||||
got := checkCopy(dir, retired)
|
||||
expectProblem(t, got, "retired-words differs from the glossary: it should list alert, overlay")
|
||||
expectProblem(t, got, "retired-lines differs from this check's list")
|
||||
expectProblem(t, checkCopy(t.TempDir(), retired), "cannot be read")
|
||||
}
|
||||
|
||||
func mapOf(files map[string]string) fs.FS {
|
||||
m := fstest.MapFS{}
|
||||
for k, v := range files {
|
||||
m[k] = doc(v)
|
||||
}
|
||||
return m
|
||||
}
|
||||
+159
-704
File diff suppressed because it is too large
Load Diff
+17
-58
@@ -1,10 +1,9 @@
|
||||
---
|
||||
status: canonical
|
||||
updated: 2026-10-16
|
||||
derives: the constitution page
|
||||
updated: 2026-08-23
|
||||
derives: knowledge-base constitution page
|
||||
decisions:
|
||||
- "ADR 0020"
|
||||
- "ADR 0391"
|
||||
- 02-DECISIONS/0020-the-mesh-is-governed-by-a-constitution.md
|
||||
---
|
||||
|
||||
# How we build
|
||||
@@ -38,13 +37,13 @@ incident behind it is not written down, and the fix is to write it down, not to
|
||||
| Rule | What it means |
|
||||
|---|---|
|
||||
| **Never write to a production database directly** | No insert, update, delete or schema statement executed against production by hand. Schema changes go through numbered migrations; data changes go through application code or the module's own capabilities. Raw statements skip every side effect the proper path has — events, audit, cache invalidation, fan-out. |
|
||||
| **Every schema change is a migration** | Numbered, in the module's own language, compiled with it. Both a baseline for a fresh installation *and* an incremental migration for installations that already exist. If code references a column, the migration creating it must exist. ADR 0013 |
|
||||
| **Every schema change is a migration** | Numbered, in the module's own language, compiled with it. Both a baseline for a fresh installation *and* an incremental migration for installations that already exist. If code references a column, the migration creating it must exist. [ADR 0013](../02-DECISIONS/0013-schema-changes-are-numbered-migrations.md) |
|
||||
| **Never bypass the pipeline** | No manual database edit, no manual restart as a workaround. Fix the cause and deploy. A workaround that works is a workaround that is never removed, and the next person cannot tell the node from its declaration. |
|
||||
| **Never create a symlink** | A hand-made link caused production data loss through container volume resolution, and the judgement needed to make a safe exception is exactly the judgement unavailable at the moment it matters. **The mesh creates none at all** (ADR 0012, which supersedes ADR 0012). The links the installer still reconciles are a migration, not a permission. |
|
||||
| **Never create a symlink** | A hand-made link caused production data loss through container volume resolution, and the judgement needed to make a safe exception is exactly the judgement unavailable at the moment it matters. **The mesh creates none at all** ([ADR 0012](../02-DECISIONS/0012-the-mesh-creates-no-symlinks.md), which supersedes [ADR 0012](../02-DECISIONS/0012-the-mesh-creates-no-symlinks.md)). The links the installer still reconciles are a migration, not a permission. |
|
||||
| **Never push directly to the main branch** | Branch, push, review, merge. Every merge is a human checkpoint, without exception — **including in this repository**. A documentation repository is not a lower tier of care; a decision record lands the same way a service does. |
|
||||
| **One change per pull request, and never merge unapproved work** | Unrelated improvements bundled together cannot be reviewed or reverted separately. And the checkpoint is **a person deciding, not a person clicking** — work may be merged by whoever wrote it once a human has explicitly approved *that merge*, and never on a standing permission, an instruction to do the work, silence, or the author's own judgement that it is ready. ADR 0023 |
|
||||
| **One change per pull request, and never merge unapproved work** | Unrelated improvements bundled together cannot be reviewed or reverted separately. And the checkpoint is **a person deciding, not a person clicking** — work may be merged by whoever wrote it once a human has explicitly approved *that merge*, and never on a standing permission, an instruction to do the work, silence, or the author's own judgement that it is ready. [ADR 0023](../02-DECISIONS/0023-approval-is-the-checkpoint.md) |
|
||||
| **Never open a pull request unprompted** | A permissions list saying it is allowed is not a request. |
|
||||
| **A failed step fails the job** | A sequence that continues past a failure does the next thing in the wrong place. Gate each step on the last. ADR 0010, and §5. |
|
||||
| **A failed step fails the job** | A sequence that continues past a failure does the next thing in the wrong place. Gate each step on the last. [ADR 0010](../02-DECISIONS/0010-delivery.md), and §5. |
|
||||
|
||||
### A failed step must stop the steps after it — how it was earned
|
||||
|
||||
@@ -70,7 +69,7 @@ reported failure, nothing stopped, and the damage happened somewhere nobody was
|
||||
- **Every runtime variable is declared.** A variable the module reads and the manifest does not
|
||||
declare is invisible to the mesh: it will not be generated, injected, or audited.
|
||||
- **Provisioned credentials arrive through declared requirements**, never hardcoded in code,
|
||||
compose files or scripts. ADR 0009
|
||||
compose files or scripts. [ADR 0009](../02-DECISIONS/0009-modules-and-the-graph.md)
|
||||
- **Never install a package by hand.** A package is declared in the manifest and arrives the
|
||||
way every other package does. A hand-installed package is invisible to the mesh: it is not
|
||||
declared, not reproduced on the next node, and not present after a rebuild — and the node
|
||||
@@ -83,7 +82,7 @@ reported failure, nothing stopped, and the damage happened somewhere nobody was
|
||||
- **Every standalone application gets its own repository**, with a manifest at its root,
|
||||
registered as a build source. Creating an application directory in the monorepo is a
|
||||
convention violation and reviewers reject it.
|
||||
ADR 0015
|
||||
[ADR 0015](../02-DECISIONS/0015-applications-live-in-their-own-repository.md)
|
||||
|
||||
### Migrations
|
||||
|
||||
@@ -97,7 +96,7 @@ reported failure, nothing stopped, and the damage happened somewhere nobody was
|
||||
surface is regenerated from the mesh database; a local edit survives one synchronisation and is
|
||||
then silently overwritten, bringing back whatever it fixed. Use the mesh operation that owns
|
||||
the value. If unsure whether a file is managed, ask the tooling — the answer is not visible
|
||||
from the file. ADR 0011
|
||||
from the file. [ADR 0011](../02-DECISIONS/0011-managed-files-are-generated-never-edited.md)
|
||||
|
||||
---
|
||||
|
||||
@@ -122,7 +121,7 @@ for them. Do not merge them into one module: they are delivered to different nod
|
||||
that must be assigned where half of it is unwanted is not a boundary either.
|
||||
|
||||
Coherence is a context. Delivery is a module. Relationships are edges, not folders.
|
||||
ADR 0009
|
||||
[ADR 0009](../02-DECISIONS/0009-modules-and-the-graph.md)
|
||||
|
||||
### Contexts integrate through the record, never through a shared schema
|
||||
|
||||
@@ -173,46 +172,6 @@ quietly stop being true, and nobody will learn that from a document.
|
||||
Not test-driven development as a blanket rule — a test written first against undiscovered
|
||||
behaviour asserts a guess. The obligation is that every decision has a defender.
|
||||
|
||||
### A bus permission is proven under the real permission
|
||||
|
||||
A change that adds or widens what a mesh process reads or writes on the bus (a subscription, a publish, a stream or
|
||||
key-value read, a consumer) carries an integration test that runs it on a bus user with exactly the production
|
||||
permissions of that account (mesh-sdk's composition of them), against a real bus server, and that fails with the
|
||||
bus's permissions violation when the widest new access is not granted. It runs in the forge's check like every
|
||||
integration test. A reviewer reading code does not see a grant, and a fake bus grants everything (ADR 0560).
|
||||
|
||||
How it is checked: a review that approves such a change whose pull request names no such test is the rule skipped.
|
||||
|
||||
### A bus permission under the real permission — how it was earned
|
||||
|
||||
On 2026-10-27 a change widened what every node agent watches; two reviews read its code and its tests ran against a
|
||||
fake bus. Live, every node's agent was refused that read by its grant and stopped composing, and the same day the
|
||||
grants part's user list dropped nine seats' subscriptions (issues 1126 and 1945).
|
||||
|
||||
### Tests run at the lowest level that proves them, within a time budget
|
||||
|
||||
Three levels, by how much real system a test uses (ADR 0391):
|
||||
|
||||
| Level | Uses | Budget | Runs |
|
||||
|---|---|---|---|
|
||||
| **Unit** | one piece of logic alone: no database, network, bus, running service or other process | well under a second per test | locally, the tests of what changed, while working; and in the forge's checks |
|
||||
| **Integration** | the real thing beside it: a database, the bus, a catalogue tree | the repository's whole suite in seconds, never over sixty | only in the forge's checks, once per push |
|
||||
| **End to end** | the whole mesh, or a lab mesh, from the outside | — | only when the operator asks |
|
||||
|
||||
- **Test each behaviour at the lowest level that can prove it.** A defect found higher up gets a test at the
|
||||
lowest level that reproduces it, before its fix.
|
||||
- **A test that needs longer than a second, or a database, the network or a service, is not a unit test.**
|
||||
- **A suite over sixty seconds is a defect with its own issue**: it is made faster or split, not accepted.
|
||||
- **Never run a merge check, a throwaway database or an integration suite locally.** The forge's checks do,
|
||||
on the builder.
|
||||
- **A report names its tests and how long they took**, so a test over its budget is seen when it is added.
|
||||
|
||||
### Tests within a time budget — how it was earned
|
||||
|
||||
On 2026-10-16 a feature's fixing agents ran the catalogue's whole merge check (about ten minutes), a
|
||||
throwaway database and a locally built gate during local development. Each round cost tens of minutes and
|
||||
caught neither real failure; the forge's checks found both.
|
||||
|
||||
### A report is read from the system, never from what asked for it
|
||||
|
||||
The same rule as the two above, pointed at reporting rather than at verification. **Anything
|
||||
@@ -239,8 +198,8 @@ declaration was never wrong. Nothing ever asked the system.
|
||||
### Search the record before forming a hypothesis
|
||||
|
||||
The first action on any error message, failing service or unexpected behaviour is to search the
|
||||
record for the literal error text (`records.records_search`) — before a hypothesis, not after one fails.
|
||||
The search matches the words as written, so search with the error's own words.
|
||||
operational memory for the literal error text — before a hypothesis, not after one fails. The
|
||||
knowledge base is indexed on symptoms.
|
||||
|
||||
This fires hardest on *familiar* ground, where a confident trail feels like progress. Two
|
||||
entries were each rediscovered from scratch over several hours in a single session because the
|
||||
@@ -255,7 +214,7 @@ This document is governed. It does not change by commit message or unilateral de
|
||||
1. **Propose** — a change stating what rule is changing, why the current wording is
|
||||
inadequate, and what reviewed it.
|
||||
2. **Review** — sign-off by reviewers who are not the proposer.
|
||||
3. **Record** — the change is a decision and gets a decision record in the knowledge base (playbook [02](process/02-graduation.md)), because a
|
||||
3. **Record** — the change is a decision and gets a record in [`02-DECISIONS/`](../02-DECISIONS/), because a
|
||||
rule the mesh enforces is architecturally significant.
|
||||
4. **Sync** — playbook [`process/05-constitution-sync.md`](process/05-constitution-sync.md)
|
||||
publishes the derived page. An unsynced rule is a rule the mesh does not enforce, whatever
|
||||
@@ -267,7 +226,7 @@ No drive-by edits. Every change traces to a recorded decision.
|
||||
a design meeting with at least two node operators — which has never been met and cannot be, as
|
||||
there is one operator. A rule that cannot be satisfied is not a high standard; it is a rule
|
||||
everything silently violates. Recorded here as resolved in favour of what is achievable, and
|
||||
what has in fact been practised (ADR 0022).
|
||||
what has in fact been practised ([ADR 0022](../02-DECISIONS/0022-the-constitution-absorbs-what-is-enforced.md)).
|
||||
|
||||
---
|
||||
|
||||
@@ -284,7 +243,7 @@ process. Absence of an override means these rules apply unmodified.
|
||||
## 8. Code quality
|
||||
|
||||
*Absorbed 2026-08-26 from the enforced page, which carried these rules while this document did
|
||||
not — ADR 0022.*
|
||||
not — [ADR 0022](../02-DECISIONS/0022-the-constitution-absorbs-what-is-enforced.md).*
|
||||
|
||||
**These rules are recorded because they are enforced, not because this repository earned them.**
|
||||
Every other rule here states the incident or measurement behind it. These state nothing,
|
||||
@@ -313,7 +272,7 @@ Data access, business logic and the interface layer are separate.
|
||||
|
||||
*Scope: the mesh's services and surfaces. Tier 0 is a statically linked binary that must depend
|
||||
on nothing installed first, and is written in Go —
|
||||
ADR 0005.*
|
||||
[ADR 0005](../02-DECISIONS/0005-the-node-host.md).*
|
||||
|
||||
- TypeScript throughout; no new untyped JavaScript.
|
||||
- Strict, with no implicit `any` and no unchecked index access.
|
||||
|
||||
@@ -3,8 +3,8 @@
|
||||
How work moves through HQ, and who may do what. In industry terms this is **spec-driven
|
||||
development, with provenance**: the decision is the why, the design doc is the spec, `code:`
|
||||
names the implementation, and the lab beds are the conformance tests — and unlike the common
|
||||
form, the chain itself is checked (ADR 0080,
|
||||
ADR 0081). Every other document in this folder is a
|
||||
form, the chain itself is checked ([ADR 0080](../../02-DECISIONS/0080-the-development-cycle-is-checked.md),
|
||||
[0081](../../02-DECISIONS/0081-a-decision-nothing-cites-is-not-yet-in-the-chain.md)). Every other document in this folder is a
|
||||
playbook: trigger, who runs it, steps, outputs. Engineers and agents follow the same
|
||||
playbooks; agents must not act outside them.
|
||||
|
||||
@@ -19,13 +19,11 @@ playbooks; agents must not act outside them.
|
||||
## The knowledge flow
|
||||
|
||||
```
|
||||
idea ──► research ──► decision ──► 03-DESIGN/01-to-be ──► built (code repo)
|
||||
│ │ │
|
||||
│ │ └─► 03-DESIGN/00-as-is once shipped
|
||||
idea ──► 01-RESEARCH ──► decision (02-DECISIONS/) ──► 03-DESIGN/01-to-be ──► built (code repo)
|
||||
│ │ │
|
||||
│ │ └─► 03-DESIGN/00-as-is once shipped
|
||||
│ └────► abandoned (recorded, kept)
|
||||
└─(small/obvious, still a decision record)──────────► 03-DESIGN directly
|
||||
|
||||
research and decisions: in the knowledge base (knowledge-base.reserve, propose, ask, get, holds)
|
||||
└─(small/obvious, still recorded in 02-DECISIONS)──────────► 03-DESIGN directly
|
||||
|
||||
symptom ──► 04-ISSUES ──► diagnosis ──► code-repo fix and/or design amendment
|
||||
|
||||
@@ -39,7 +37,7 @@ how-we-build.md ──► constitution sync ──► knowledge base ──► i
|
||||
| Layer | What it is | Changes when |
|
||||
|---|---|---|
|
||||
| `00-as-is/` | The mesh that exists today. Describes shipped behaviour, including behaviour nobody would choose again. | Something ships, or an as-is claim is found to be wrong. |
|
||||
| `01-to-be/` | The mesh being built toward. Every statement traceable to a decision record in the knowledge base. | A decision is taken or amended. |
|
||||
| `01-to-be/` | The mesh being built toward. Every statement traceable to a record in `02-DECISIONS/`. | A decision is taken or amended. |
|
||||
|
||||
A to-be document that ships does not move. Its as-is counterpart is written or updated, the
|
||||
to-be document's frontmatter goes to `implemented`, and both stand — one describing what runs,
|
||||
@@ -48,45 +46,33 @@ the expensive half.
|
||||
|
||||
## The playbooks
|
||||
|
||||
A drawn overview of them, the cycle, a feature's steps, the review loop and an issue's life, is
|
||||
[`playbooks.html`](playbooks.html): a picture of these playbooks, never their authority.
|
||||
|
||||
| # | Playbook | Trigger |
|
||||
|---|---|---|
|
||||
| [01](01-research.md) | Research | An idea from the operator, or one worth investigating before committing to design: setup questions, findings, a direction the operator confirms |
|
||||
| [02](02-graduation.md) | Graduation & design change | Research concludes (its direction confirmed), or a design must change: decision records, the to-be design |
|
||||
| [01](01-research.md) | Research | An idea worth investigating before committing to design |
|
||||
| [02](02-graduation.md) | Graduation & design change | Research concludes, or a design must change |
|
||||
| [03](03-issues.md) | Issues | Something is wrong — often with the owner unknown |
|
||||
| [04](04-build-handoff.md) | Work breakdown and handoff | A design is ready to be built: milestones usable on their own, issues per repository, each milestone one feature step |
|
||||
| [04](04-build-handoff.md) | Build handoff | A design is ready to be built |
|
||||
| [05](05-constitution-sync.md) | Constitution sync | `how-we-build.md` changed a rule the mesh enforces |
|
||||
| [06](06-writing-a-module.md) | Writing a module | Something that runs today must run on the mesh |
|
||||
| [07](07-feature-branches.md) | Features | Work that changes code across issues or repositories: prepared context, local development, one review, one check, one delivery |
|
||||
| [08](08-small-fixes.md) | Small fixes | One issue in one repository, about an hour of work: one review, one check, one delivery |
|
||||
| [09](09-cleanup.md) | Cleanup | Merged branches, stale pull requests, finished worktrees and folders: read first, delete only what the operator approves, report the rest |
|
||||
| [10](10-visual-design.md) | Visual design | A page the operator will use is about to be designed or rebuilt: mockups with fixed data, the operator picks, the pick is the reference |
|
||||
| [07](07-feature-branches.md) | Feature branches across repos | Work that changes code, in one repo or several at once |
|
||||
|
||||
## The cycle is checked
|
||||
|
||||
The flow above is a rule, and a rule states how it is checked:
|
||||
the cycle test in [`00-META/checks/`](../checks/) refuses a to-be design that names no
|
||||
decision, an `in-progress`/`implemented` design that names no owning code, two issue folders
|
||||
sharing a number or an issue report carrying a status key; a `graduated` research effort that
|
||||
does not say what it became is refused by the knowledge base's `propose` (ADR 0290). Run it with the records, index and
|
||||
words tests before any HQ merge: `go test ./...` at the root runs all four, and the checker runs the same as `mesh/check` (ADR 0527). The issue rules the check
|
||||
once held — `located` with an owner, `resolved` with a fix, a core issue with its replay — are
|
||||
refused at the write by the issue tracker's `move`
|
||||
(ADR 0275). What the checks cannot see — that code work actually started from a
|
||||
[`00-META/checks/cycle.py`](../checks/cycle.py) refuses a to-be design that names no
|
||||
decision, an `in-progress`/`implemented` design that names no owning code, an issue marked
|
||||
`located`/`fixed` with no owner or `fixed`/`resolved` with no fix, and a `graduated`
|
||||
research overview that does not say what it became. Run it with `records.py`, `index.py` and
|
||||
`words.py` before any HQ merge (`sh merge-check.sh` runs all four). What the checks cannot see — that code work actually started from a
|
||||
handoff — is held by playbooks [04](04-build-handoff.md) and [07](07-feature-branches.md):
|
||||
a feature branch exists because a design or an issue sent it.
|
||||
|
||||
## Status lives in frontmatter, an issue's in the tracker
|
||||
## Status lives in frontmatter
|
||||
|
||||
Design docs carry their status as YAML frontmatter (schema in playbook [02](02-graduation.md)).
|
||||
Research efforts and decision records carry theirs in the knowledge base, which refuses a wrong one at
|
||||
`propose` (ADR 0290). An issue's status lives in the
|
||||
issue tracker, which refuses a wrong status at the write, and its report here carries only
|
||||
the day it was opened (playbook [03](03-issues.md)). There are **no central status
|
||||
Research overviews, design docs, issue reports and decision records each carry their status as
|
||||
YAML frontmatter (schemas in the section READMEs and playbooks). There are **no central status
|
||||
files** and no decision ledger. **Every decision is a record in
|
||||
the knowledge base** — if it is worth recording it is worth a record, and if
|
||||
it is not worth a record it is not recorded. Cross-cutting views are generated on demand by the
|
||||
`hq-status` skill and never written to disk (ADR 0248); the decisions of a topic are
|
||||
`knowledge-base.holds` with that topic.
|
||||
[`02-DECISIONS`](../../02-DECISIONS/)** — if it is worth recording it is worth a record, and if
|
||||
it is not worth a record it is not recorded. Cross-cutting views, the decision index included,
|
||||
are generated on demand by the `hq-status` skill and never written to disk
|
||||
([ADR 0248](../../02-DECISIONS/0248-the-decision-index-is-generated-when-it-is-read-and-never-stored.md)).
|
||||
|
||||
+29
-121
@@ -1,140 +1,48 @@
|
||||
# Playbook 01 — Research
|
||||
|
||||
**Trigger.** An idea the operator expresses, or an idea, technology or approach worth investigating before
|
||||
it is committed to design. Also: an as-is document that raises a question nobody can answer. A symptom (a
|
||||
bug, something broken) is not research: it follows playbook [03](03-issues.md).
|
||||
**Trigger.** An idea, technology or approach worth investigating before it is committed to
|
||||
design. Also: an as-is document that raises a question nobody can answer.
|
||||
|
||||
**Who runs it.** Anyone; in an agent session, the session that heard the idea leads it.
|
||||
|
||||
**Where it lives.** A research effort lives whole in the knowledge base, the seat `knowledge-base`, reached
|
||||
through the mesh MCP server from any node and any folder (ADR 0290). This repository holds none.
|
||||
|
||||
## The cycle it starts
|
||||
|
||||
```
|
||||
idea ─► research (01) ─► decisions and design (02) ─► work breakdown in milestones (04)
|
||||
─► each milestone built and delivered as one feature step (07) ─► what is reported afterwards (03, 08)
|
||||
```
|
||||
|
||||
An idea arrives in one of two forms, and the first step differs:
|
||||
|
||||
- **A new idea**, where only the mesh's own context matters: read what the knowledge base holds on it.
|
||||
- **An idea for an existing application**: read that application's to-be and as-is designs, its decision
|
||||
records and its open issues first; the effort `touches` them.
|
||||
**Who runs it.** Anyone.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Read what is known.** `knowledge-base.ask` with the idea in plain words returns the decision records and
|
||||
research efforts that bear on it; `knowledge-base.get` reads one (`reasoning: true`, or `section`, for its
|
||||
documents). Read the issue tracker for open issues in the same area (`issue-tracker.list`).
|
||||
2. **Ask the operator to set it up.** A few questions, one at a time, each answerable with a choice and
|
||||
carrying a recommendation: what comes first, what is out of scope, which options must be tried, what
|
||||
done looks like. Three to five questions is usual; the answers are the effort's first findings.
|
||||
- **Check every option against the mesh's own rules before offering it.** An option that breaks one
|
||||
(the core values in [`mission.md`](../mission.md), among them *the mesh owns the truth* and
|
||||
*sovereignty: external dependencies are deliberate and few*; the rules in
|
||||
[`how-we-build.md`](../how-we-build.md); and everything between modules over the bus, research 036) is
|
||||
dropped, or offered only with the rule it breaks named. A recommendation the operator has to argue back
|
||||
costs a version.
|
||||
- **Explain each question with a concrete picture** in the operator's words: what they would see, do or
|
||||
lose under each option. No internal names (a record number, a verb, a word the glossary lacks) without
|
||||
what it is beside it.
|
||||
3. **Take a number.** `knowledge-base.reserve` with `kind: research` and `for`: the effort's title. The number
|
||||
is spent whether or not the effort is ever proposed; one never proposed is withdrawn after a week. Never
|
||||
pick one by hand.
|
||||
4. **Propose it.** `knowledge-base.propose` with `kind: research`, the `number`, a one-line `title`,
|
||||
`status: active`, `date` (the day it began), and `sections`, each a `name` and its markdown `text`:
|
||||
- `00-overview.md`: the idea in the operator's words, what exists, the operator's answers (dated), and the
|
||||
words the effort uses that the glossary lacks, each defined in one line;
|
||||
- `01-questions.md`: the questions to answer, the first ones first;
|
||||
- `02-method.md`: what will be tried and measured, and on what.
|
||||
1. Take the next free number: highest `01-RESEARCH/NNN-*` plus one, zero-padded to three
|
||||
digits. Never skip or reuse a number.
|
||||
2. Create `01-RESEARCH/NNN-descriptive-name/` and a `00-overview.md` in it carrying:
|
||||
|
||||
Name what it touches as `links` of kind `touches` (`"0276"`, `"research 045"`, `"issue 365"`, or a
|
||||
design's path), and what it names for another reason as `cites`.
|
||||
5. **Do the research.** Try, measure and read, smallest question first. Each finding goes into
|
||||
`04-findings.md`: what was tried, what it showed (with numbers), what it means. Each change of the effort
|
||||
is `knowledge-base.propose` again, whole, with `why`; every version is kept, and `knowledge-base.history`
|
||||
lists them and gives the difference between two. Until issue 844 is delivered, the knowledge base
|
||||
takes a new version of an accepted, active effort only as a progressive insight: give `insight`, saying
|
||||
what the effort held before the new finding, and mark the finding in the text with its date.
|
||||
- **Propose a version per settled set, not per answer.** While the operator answers a run of questions,
|
||||
or while experiments run, keep the answers and findings in the session's save point and propose once
|
||||
when the set is settled. Each version is reviewed and resent whole, so a version per answer multiplies
|
||||
the reviews.
|
||||
- **Pre-check before proposing**, against what the review sends back most: every record, issue or design
|
||||
the text names is in `links`; work put off names the issue that tracks it (an acceptance test is not
|
||||
put-off work: it is what the build proves, step 7); each figure is given as measured (a range, not a
|
||||
rounded claim); no word on a glossary *Not:* line.
|
||||
- **Try at the cheapest level that answers the question.** A program's own behaviour (a library, a
|
||||
server, a protocol) is tried in process, on the machine's loopback, in minutes. Lab runs (about 12 to
|
||||
16 minutes fresh) are for the mesh's own code; before that code exists, a lab run tests nothing an
|
||||
in-process test does not, and its lab scenario is written as the build's acceptance test instead.
|
||||
6. **Place it in the domains.** Before the direction, write `03-domains.md`: the domain or domains the idea
|
||||
belongs to (the glossary's sections, ADR 0244); the modules and seats it lands in, each new, changed or
|
||||
retired; what it does to each domain's words, each word added (with its domain and a one-line meaning),
|
||||
changed or retired, and none on a *Not:* line or already meaning something else (the glossary's homonyms);
|
||||
and whether a domain grows or shrinks, a concept moving between domains included. The direction names its
|
||||
decisions in those words. Asked when the idea is set up, too: an option that lands a concept in the wrong
|
||||
domain is offered with that cost named, like one that breaks a rule.
|
||||
7. **Settle the direction with the operator.** When the questions are answered, write `05-direction.md`:
|
||||
the proposed decisions in a few lines each, the options rejected and why, and the build's acceptance
|
||||
tests: what the build must prove before it is done, carried by its milestones (playbook
|
||||
[04](04-build-handoff.md)). Nothing else is left open: each question closes by a test, by a decision the
|
||||
effort records with its reason (a small design limit is the lead's), or by the operator. Ask the
|
||||
operator to confirm or change the direction, one point at a time, then propose it once. Then the effort
|
||||
graduates through playbook [02](02-graduation.md).
|
||||
```yaml
|
||||
---
|
||||
status: active
|
||||
initiated: YYYY-MM-DD
|
||||
touches: [] # areas, subsystems or as-is design docs the effort bears on
|
||||
---
|
||||
```
|
||||
|
||||
## Bounds and models
|
||||
|
||||
| Step | Bound | Model |
|
||||
|---|---|---|
|
||||
| reading and the setup questions (1, 2) | 30 minutes | the session's own, larger |
|
||||
| proposing (3, 4) | 15 minutes | larger |
|
||||
| each experiment or reading task (5) | 30 minutes | smaller, one agent at a time unless the operator allows more |
|
||||
| placing it in the domains (6) | 15 minutes | larger |
|
||||
| the direction (7) | 30 minutes | larger |
|
||||
|
||||
A step past its bound is said to the operator at once, with what is being done about it. The lead reports
|
||||
as playbook 07 does: every ten minutes and at once when something is found, stops or needs the operator,
|
||||
with the allowance used.
|
||||
Then a short prose summary: **what** is being investigated, **why**, and **what it
|
||||
touches**.
|
||||
3. Do the research in further documents in the same folder — notes, option analyses, evidence,
|
||||
draft designs. Anything goes. Keep the summary in `00-overview.md` current as the effort changes
|
||||
shape.
|
||||
|
||||
## What makes research worth reading
|
||||
|
||||
State evidence, not assertion. *"Zero of 124 modules declare `brain` as a dependency"* outranks *"the
|
||||
dependency rule is not followed"*. An effort that measured nothing has not finished.
|
||||
State evidence, not assertion. *"Zero of 124 modules declare `brain` as a dependency"*
|
||||
outranks *"the dependency rule is not followed"*. An effort that measured nothing has not
|
||||
finished.
|
||||
|
||||
Research describes real observations but never identifies the mesh it observed. The shape of a finding
|
||||
survives anonymisation intact — *a node publicly named but behind a household NAT* carries the whole
|
||||
lesson without naming anything.
|
||||
|
||||
`propose` refuses an effort that breaks a rule that judges one record (an unknown status, a link to
|
||||
nothing, a retired word, a daemon's line), naming every reason at once and writing nothing. The knowledge
|
||||
base then reviews the proposal itself; on `changes` the effort stays proposed with the findings and its
|
||||
author proposes it again, at most three times before the findings go to the operator with a
|
||||
recommendation. An effort that needs the operator's word is proposed with `needs-operator: true` and waits
|
||||
for the operator's Approve on a channel that proves who answered.
|
||||
Research describes real observations but never identifies the mesh it observed. The shape of a
|
||||
finding survives anonymisation intact — *a node publicly named but behind a household NAT*
|
||||
carries the whole lesson without naming anything.
|
||||
|
||||
## Do not
|
||||
|
||||
- Do not put status in prose. It is the effort's `status`, and the prose must not restate it.
|
||||
- Do not give a `became` link while the effort is open — playbook 02 gives it at closure.
|
||||
- Do not put status in prose. It lives in `00-overview.md`'s frontmatter, and the prose must not restate it.
|
||||
- Do not set `became:` while the effort is open — playbook 02 sets it at closure.
|
||||
- Do not write into `03-DESIGN` from an open effort.
|
||||
- Do not write an effort, or a copy of one, into this repository: the records test refuses a new path under
|
||||
`01-RESEARCH/`.
|
||||
|
||||
## Closing
|
||||
|
||||
An effort never just stops. It closes through playbook [02](02-graduation.md) as `graduated` or
|
||||
`abandoned`, always with a `became` link to what it turned into. Nothing is deleted — what was rejected,
|
||||
and why, is the more expensive half to rediscover.
|
||||
|
||||
## How it is checked
|
||||
|
||||
The rules on an effort are the knowledge base's: `propose` refuses one that breaks a rule that judges one
|
||||
record (the module's table test holds one case per rule), and the rules that judge the whole record set run
|
||||
after every acceptance and raise a condition when one fails. The habits of steps 2, 5, 6 and 7 (options checked
|
||||
against the rules, one version per settled set, the cheapest level, nothing left open) are the lead's, read by
|
||||
the reviewer; of the pre-check, `propose` refuses a link to nothing and a retired word, and the review returns
|
||||
a record named but not linked, an untracked deferral and a figure not as measured. That this repository holds
|
||||
no effort is the records test's `no-records` check, on every pull request. An active effort with no new version
|
||||
for seven days is the smell of research left behind: its lead closes or continues it.
|
||||
An effort never just stops. It closes through playbook [02](02-graduation.md) as `graduated`
|
||||
or `abandoned`, always with `became:` pointing at what it turned into. Nothing is deleted —
|
||||
what was rejected, and why, is the more expensive half to rediscover.
|
||||
|
||||
@@ -4,39 +4,16 @@
|
||||
|
||||
**Who runs it.** Anyone, with the decision recorded before the design moves.
|
||||
|
||||
**Where a decision lives.** A decision record lives whole in the knowledge base, the seat `knowledge-base`,
|
||||
reached through the mesh MCP server from any node and any folder (ADR 0290): its number, title, topic,
|
||||
status, date, deciders, its clauses, its typed links, its reasoning and every version. This repository holds
|
||||
none, and keeps no copy. What was decided is read with `knowledge-base.ask` (a question in plain words),
|
||||
`knowledge-base.get` (one record by number; `reasoning: true` for its why) and `knowledge-base.holds` (the
|
||||
clauses in force for a topic or a record).
|
||||
|
||||
## Graduating research
|
||||
|
||||
The effort's `05-direction.md`, confirmed by the operator (playbook [01](01-research.md), step 7), is what
|
||||
graduates. The lead writes the records and the design on the larger model; it is one sitting, bounded at
|
||||
about an hour, and said to the operator if it runs past.
|
||||
|
||||
1. **Check it against GENESIS.** An effort graduates only if its conclusion is traceable to
|
||||
[`mission.md`](../mission.md), [`context.md`](../context.md) and
|
||||
[`effect.md`](../effect.md). If it conflicts, either the effort is wrong or GENESIS is —
|
||||
say which, in writing, before proceeding.
|
||||
2. **Record the decisions** (below, *Writing a decision record*): one record per choice that could be
|
||||
reversed on its own, not one record for the whole effort.
|
||||
- **Write them together, not one after another.** Reserve every number first, then propose each record
|
||||
as soon as it is written, without waiting for the review of the one before; the reviews run side by
|
||||
side, and the operator's asks for the records that need them arrive together. A record that
|
||||
`rests-on` or `extends` a sibling is proposed once that sibling is accepted.
|
||||
- **Take the clauses from the confirmed direction** and cite the effort (`cites`, `"research NNN"`) for
|
||||
the evidence instead of retyping its findings: a record's Context holds the few facts its decision
|
||||
rests on, with the effort named for the rest.
|
||||
- **Pre-check each record before proposing it**, as for research (playbook [01](01-research.md), step
|
||||
5): every record, issue or design it names is in `links`; each deferral names its issue; each figure
|
||||
is given as measured; no word on a glossary *Not:* line.
|
||||
3. **Write the design.** **One design per thing** (the operator, 2026-10-16): when a to-be design already
|
||||
covers what the decisions change, they rewrite that design, as *Amending an existing design* below says,
|
||||
and no second design is written beside it that refers to it. Only for something no design covers, create
|
||||
the document under `03-DESIGN/01-to-be/` with frontmatter:
|
||||
2. **Record the decision.** Write a record in [`02-DECISIONS/`](../../02-DECISIONS/) taking the next free
|
||||
number. Format and rules are in [`02-DECISIONS/README.md`](../../02-DECISIONS/README.md). State evidence,
|
||||
not assertion, and record the options that were rejected — that is the half worth keeping.
|
||||
3. **Write the design.** Create the document under `03-DESIGN/01-to-be/` with frontmatter:
|
||||
|
||||
```yaml
|
||||
---
|
||||
@@ -44,96 +21,25 @@ about an hour, and said to the operator if it runs past.
|
||||
status: designed
|
||||
code: [] # owning code repo(s); set at build handoff, empty before
|
||||
updated: YYYY-MM-DD
|
||||
decisions:
|
||||
- "ADR NNNN"
|
||||
decisions: [02-DECISIONS/NNNN-....md]
|
||||
---
|
||||
```
|
||||
|
||||
The design lists the direction's acceptance tests: what the build proves before it is done.
|
||||
|
||||
The words of the effort's `03-domains.md` go into the glossary in the same change, each in its domain: a
|
||||
word added gets its entry, a word whose meaning changed gets its entry rewritten, and a retired word moves to
|
||||
its replacement's *Not:* line. A word that already means something else in another domain is named as a
|
||||
homonym with the qualified form each domain uses.
|
||||
|
||||
4. **Hand it to the work breakdown.** The design goes on through playbook [04](04-build-handoff.md):
|
||||
milestones, issues, and each milestone built as one feature step (playbook [07](07-feature-branches.md)).
|
||||
5. **Close the effort.** `knowledge-base.propose` the effort again, whole, with `status: graduated` and a
|
||||
`became` link to the decision record (`"NNNN"`) and to the design document (its path). An effort that
|
||||
stops without a decision is proposed with `status: abandoned` and a `became` link to what replaced it.
|
||||
|
||||
## Writing a decision record
|
||||
|
||||
1. **Take a number.** `knowledge-base.reserve` with `kind: decision` and `for`: the record's title. The
|
||||
number comes from the issue tracker's counter and is spent whether or not the record is accepted; one
|
||||
never proposed is withdrawn after a week. Never pick a number by hand.
|
||||
2. **Propose it.** `knowledge-base.propose` with the `number`, a one-line `title` in plain language, its
|
||||
`topic` (one of the six of the reading order, [`02-DECISIONS/README.md`](../../02-DECISIONS/README.md)),
|
||||
`date` (when the decision was taken, not when it was written down), `deciders`, and:
|
||||
- `clauses`: what was decided, in order, each one statement opening with a bold lead
|
||||
(`"**What is decided.** …"`);
|
||||
- `sections`: the reasoning, in order: `Context` (what was true, with evidence), `Considered Options`
|
||||
(numbered, each with why it was rejected), `Consequences` (what follows, including what got harder),
|
||||
`How it is checked` (a rule states how it is checked), `References`;
|
||||
- `links`: `extends` for a record it builds on, `rests-on` for a decision it stands on, `supersedes` for one it replaces whole, `changes` with
|
||||
the clause it replaces part of (`"0276 §4"`, and `from` for the clause of this record that does it),
|
||||
`cites` for any record, effort, issue or design it names;
|
||||
- `homes`: the documents of this repository that cite it, such as the design's path;
|
||||
- `needs-operator: true` when it holds only once the operator approves it.
|
||||
|
||||
State evidence, not assertion. *"Zero of 124 modules declare `brain` as a dependency"* outranks *"the
|
||||
dependency rule is not followed"*. Record the options that were rejected — that is the half worth keeping.
|
||||
3. **A record that puts work off** — a rename, a check, a case it does not handle — names the issue that
|
||||
tracks it, in the sentence that defers it ("tracked by issue N"), or the deferral is not accepted: a
|
||||
deferral nothing tracks reads as done
|
||||
([issue 354](../../04-ISSUES/354-a-decided-rename-was-never-scheduled-and-the-operator-believed-it-live/00-report.md),
|
||||
ADR 0244's code renames). `propose` refuses it (rule `deferral`).
|
||||
4. **Review and acceptance.** `propose` refuses, naming every reason at once and writing nothing, a record
|
||||
that breaks one of the rules that judge one record: a link to nothing, a clause without a bold lead, a
|
||||
superseded record cited alone, an unknown topic, a retired word of the glossary, a daemon's line, a
|
||||
deferral no issue tracks. The knowledge base then reviews the proposal itself, through a judgement that
|
||||
sees only the record, the records it links and the rules, and answers `approve` or `changes`. On
|
||||
`changes` the record stays proposed with the findings: revise it by proposing it again, whole, with `why`;
|
||||
after three rounds the lead does not stop to ask, but fixes each further finding itself and proposes again
|
||||
(the operator, 2026-10-18: ADR 0429 went past three rounds, each finding narrower than the last, and asking
|
||||
first only added a wait). Only a finding that would change what was decided, rather than how the record
|
||||
says it, goes to the operator, with a recommendation.
|
||||
On `approve` it is accepted, or, when it says `needs-operator`, it waits for the operator's Approve on a
|
||||
channel that proves who answered. Nothing proposed is in force. `knowledge-base.get` shows a record's
|
||||
reviews and the operator's ask when it waits for one.
|
||||
|
||||
## Changing a decision
|
||||
|
||||
**A record's meaning is never rewritten.** A decision is changed by a new record, and a correction of fact
|
||||
by a progressive insight.
|
||||
|
||||
- **Reversing a decision, changing its scope, rejecting an option it accepted, or making a consequence false
|
||||
that a later record cites** is a new record that `supersedes` the old one whole, or `changes` the clauses
|
||||
it replaces. On its acceptance the old record or clauses become `superseded` or `changed`, and every
|
||||
answer says by what. The test is not how large the edit looks; it is whether a reader who acted on the
|
||||
old text would now be wrong about *what was decided* rather than about *a detail the decision did not
|
||||
rest on*.
|
||||
- **A progressive insight** corrects a fact a record asserted that turned out wrong while the decision it
|
||||
supports stays right: a count taken before anyone measured, a file named that does not exist. It is
|
||||
`knowledge-base.propose` of the accepted record with `insight`: what was found, what the record said
|
||||
before, and what it says instead. It is a new version, marked as one, dated, and reviewed like a
|
||||
proposal; it adds and never quietly replaces; the decision, the options weighed and the consequences
|
||||
stand untouched. If the correction changes what was decided, it is not an insight — write the new record.
|
||||
- **Where a later record changes the mechanism an older one states** without reversing it, the later
|
||||
record `changes` the older one's clause, so a reader who arrives at the older record by a citation sees
|
||||
what changed it since.
|
||||
4. **Close the effort.** Set the effort's `00-overview.md` frontmatter to `status: graduated` and
|
||||
`became:` pointing at the design document and the decision record.
|
||||
|
||||
## Amending an existing design
|
||||
|
||||
A design changes only through a decision.
|
||||
|
||||
1. Write the decision record (above). If it reverses an earlier one, it `supersedes` it.
|
||||
2. Edit the to-be design document, add the record to its `decisions:` as `"ADR NNNN"`, and set `updated:`
|
||||
to today.
|
||||
3. If the amendment came from an issue, give the issue the document path in the issue tracker:
|
||||
`amended-design` on its `issue-tracker.move` to `resolved`, or `issue-tracker.link` with kind
|
||||
`design` before then — never in the report's frontmatter
|
||||
(ADR 0275).
|
||||
1. Write the decision record. If it reverses an earlier one, the earlier record's `status:`
|
||||
becomes `superseded-by: 02-DECISIONS/NNNN-....md` — **its reasoning is never rewritten**. If the
|
||||
earlier record is sound and only a *fact* in it went stale, that is a **progressive insight**,
|
||||
corrected in place and marked in the record rather than superseded
|
||||
([`02-DECISIONS/README.md`](../../02-DECISIONS/README.md)).
|
||||
2. Edit the to-be design document and set `updated:` to today.
|
||||
3. If the amendment came from an issue, set that issue's `amended-design:` to the document
|
||||
path.
|
||||
|
||||
## When something ships
|
||||
|
||||
@@ -153,19 +59,3 @@ Implementation state is a third axis, independent of both design and decision.
|
||||
- Do not edit an as-is document to describe an intention. That is what the to-be layer is for.
|
||||
- Do not change a decision record's meaning. Supersede it. Correcting a fact it got wrong, while
|
||||
the decision stands, is a progressive insight — marked and dated in the record, never silent.
|
||||
- Do not write a decision record, or a copy of one, into this repository: the records test refuses a new path
|
||||
under `02-DECISIONS/` other than its README.
|
||||
|
||||
## How it is checked
|
||||
|
||||
The rules on a record are the knowledge base's: `propose` refuses one that breaks a rule that judges one
|
||||
record (its module's table test holds one case per rule), and the four rules that judge the whole record set
|
||||
(an accepted decision cited by nothing, a supersession named by one side only, a changed clause nothing
|
||||
changes, an accepted record extending a superseded one) run after every acceptance and raise a condition
|
||||
when one fails. In this repository, the records test checks that a design's `decisions:` entries and every
|
||||
"ADR NNNN" in the text are well formed, and that no record is written here; the index test checks the reading
|
||||
order's six topics, which the knowledge base reads. The three habits of graduating step 2 (side by side, clauses
|
||||
from the direction, the pre-check) are the lead's, read by the reviewer; `propose` refuses a link to nothing,
|
||||
a deferral no issue tracks and a retired word, and the review returns a record named but not linked.
|
||||
The rounds are read in `knowledge-base.get` (its reviews, one per version): a question to the operator about a
|
||||
record's wording after its third round is the failure of step 4 of *Writing a decision record*.
|
||||
|
||||
@@ -8,115 +8,57 @@ design permits.
|
||||
|
||||
## What belongs here, and what does not
|
||||
|
||||
| Belongs in `04-ISSUES` | Belongs with the fix's own lesson, not here |
|
||||
| Belongs in `04-ISSUES` | Belongs in the knowledge base |
|
||||
|---|---|
|
||||
| The design permits a failure to be silent | How to fix one occurrence of it |
|
||||
| A documented rule is enforced by nothing | A command that works around it |
|
||||
| A stated invariant is false in practice | A node-specific quirk |
|
||||
| The owning component is unknown and finding it needs the whole mesh in view | Symptom → fix, once the answer is known |
|
||||
|
||||
A fix's lesson (symptom → fix) is written where a search by the symptom's literal words finds it
|
||||
(`records.records_search`). This folder is not a second copy of it. An issue here is a question HQ must **answer**, not an
|
||||
The knowledge base already holds the operational record and is indexed on symptoms. This
|
||||
folder is not a second copy of it. An issue here is a question HQ must **answer**, not an
|
||||
incident someone must **clear**.
|
||||
|
||||
## The issue tracker holds the number and the status
|
||||
|
||||
Every issue is opened in the **issue tracker**, the module `mesh-issues` holding the seat `issue-tracker` on the
|
||||
control-node (ADR 0275,
|
||||
[to-be 53](../../03-DESIGN/01-to-be/53-the-issue-tracker.md)). Its number comes from one counter advanced by
|
||||
compare-and-set, so two authors opening at the same moment never get the same one. **A number is never taken by
|
||||
reading `main` or the open pull requests** — that is how numbers collided (issue 155: two records sharing a number
|
||||
reached `main` with every check passing). A session that cannot reach the tracker takes no number and says so.
|
||||
|
||||
**The tracker is the truth for every issue's status, owner, fix and links**, whether or not the issue has a folder
|
||||
here. `issue-tracker.get NNN` says where an issue stands; `issue-tracker.list` says what is open. Nothing here
|
||||
says it: a report's frontmatter carries no status.
|
||||
|
||||
An issue takes one of two shapes:
|
||||
|
||||
- **With a folder here** — a design or governance issue, whose narrative belongs in this public record and is
|
||||
reviewed in a pull request. The folder holds the report and the diagnosis; the tracker holds everything else.
|
||||
- **Wholly in the tracker** — every other issue. Nothing is written here.
|
||||
|
||||
**`04-ISSUES/` stays, for the narrative only.** It keeps the public narrative of a design or governance issue: the
|
||||
symptom as observed, the diagnosis, what was ruled out, and why. A plain bug — a defect whose fix needs no design
|
||||
question answered — needs no folder; it is whole in the tracker.
|
||||
|
||||
**The reports written before the tracker** (the folders numbered 001 to 364) carried `status:`, `located-in:`,
|
||||
`fixed-by:`, `amended-design:`, `replay:` and `replay-none:` in their frontmatter until the import of slice 3 had
|
||||
taken them into the tracker and been checked against it, field by field; then one pull request removed them, so
|
||||
that no status is held in two places. What they said that day is in this repository's history at the commit the
|
||||
import read, which every imported issue names (`imported.commit` in `issue-tracker.get`).
|
||||
|
||||
## Steps
|
||||
|
||||
1. Open the issue in the tracker, through the mesh MCP server:
|
||||
`mesh_call issue-tracker.open` with `title`, `report` (the symptom **as observed**, in plain terms, with the
|
||||
evidence that it happened), optionally `located-in` (entries of the tracker's registered list of owners), and, for
|
||||
an issue that gets a folder here, `record` — the folder's name without its number, `short-name`. The tracker
|
||||
answers with the number `NNN`; a refused open spends none.
|
||||
|
||||
**An issue with a folder** then creates `04-ISSUES/NNN-short-name/00-report.md` under exactly that number, on a
|
||||
branch:
|
||||
1. Take the next free number — **across `main` and every open pull request**, not `main` alone.
|
||||
Work sits on unmerged branches for days, so two people both reading `main` allocate the same
|
||||
number; it happened twice in one hour between two machines, and the second collision reached
|
||||
`main` with every check passing (issue 155). `cycle.py` now refuses two records sharing a number,
|
||||
which catches a collision but does not prevent one. Create
|
||||
`04-ISSUES/NNN-short-name/00-report.md`:
|
||||
|
||||
```yaml
|
||||
---
|
||||
status: open
|
||||
opened: YYYY-MM-DD
|
||||
located-in: [] # owning repo(s)/module(s), filled by diagnosis
|
||||
fixed-by: # PR or commit reference, filled at resolution
|
||||
amended-design: # design doc path, when the root cause was a design gap
|
||||
---
|
||||
```
|
||||
|
||||
Then `# NNN. The title`, the title as the tracker holds it, and the symptom as observed, as in the tracker's
|
||||
report. **No status key**: the cycle test refuses `status:`, `located-in:`, `fixed-by:`, `amended-design:`,
|
||||
`replay:` or `replay-none:` in any report, because the tracker holds them and two places holding one fact
|
||||
drift.
|
||||
|
||||
**An issue about a condition** links the condition's key at once: `issue-tracker.link` with kind `condition` and
|
||||
the key `mesh-controller.conditions` gives. Every message about that condition then names the issue and where it
|
||||
stands; one no issue links says "No issue tracks this yet." An agent that silences a condition because an issue
|
||||
tracks it links that issue first (ADR 0285).
|
||||
|
||||
**An issue wholly in the tracker** is done at this step.
|
||||
2. Investigate. **Before you start, move the issue to `diagnosing` with `by`**: who works on it and its node, as
|
||||
who@node (`fix-399@laptop`, `operator@workstation`). The tracker keeps you as the issue's **worker**, and `get` and `list`
|
||||
show it with since when. The trail goes in `issue-tracker.note`, and, for a folder-backed issue, in
|
||||
`01-diagnosis.md` in the same folder as well — dated, including what was ruled out. A note keeps you the worker; a
|
||||
worker with nothing done on the issue for a day is stale. Then move it to `located` with `located-in`, which the
|
||||
tracker refuses without a registered owner; that ends your work on it.
|
||||
3. Fix. **Before you start the fix, move the issue to `fixing` with `by`**, even when you diagnosed it yourself. The
|
||||
tracker refuses the move while another worker is current, naming them: ask them, or take it over with
|
||||
`take-over` saying why, which is noted on the issue. A stale worker may be taken over from without a reason.
|
||||
Name the fix in `fixed-by` when it merges, with a move to `fixing` that gives `by` again; the issue stays
|
||||
`fixing` until the acceptance is proven live
|
||||
(ADR 0284,
|
||||
ADR 0281).
|
||||
4. Resolve. `issue-tracker.move` to `resolved`, which the tracker takes only from `fixing`, with `fixed-by`, and, if the root cause was a design gap,
|
||||
run playbook [02](02-graduation.md) and give `amended-design`. **A core issue** — its `located-in`
|
||||
Then the symptom **as observed**, in plain terms, with the evidence that it happened.
|
||||
2. Investigate in `01-diagnosis.md` in the same folder — the trail, dated, including what was
|
||||
ruled out. Move `status:` to `diagnosing`, then `located` once the owner is known.
|
||||
3. Resolve. Set `status: resolved`, fill `fixed-by:`, and if the root cause was a design gap,
|
||||
run playbook [02](02-graduation.md) and fill `amended-design:`. **A core issue** — its `located-in`
|
||||
names the controller, the node-engine, the tool runner, the SDK, or the catalogue's bus, forge or build
|
||||
agent — resolves with `replay`, the id of its replay in mesh-lab's replays register, proved to fail on
|
||||
the commit before the fix and pass on it, or with `replay-none` saying why none is possible
|
||||
(ADR 0237);
|
||||
`issue-tracker.move` refuses the move without it. `wontfix` needs a `reason`.
|
||||
agent — resolves with `replay:`, the id of its replay in mesh-lab's replays register, proved to fail on
|
||||
the commit before the fix and pass on it, or with `replay-none:` saying why none is possible
|
||||
([ADR 0237](../../02-DECISIONS/0237-a-change-is-judged-against-the-mesh-that-runs-before-it-merges-on-the-build-seat.md); `cycle.py` checks it).
|
||||
|
||||
## Rules
|
||||
|
||||
- Closed issues are never deleted — they are the mesh's symptom-to-component memory.
|
||||
- `fixed-by` names something that will still exist: a commit or a pull request, never a branch. A
|
||||
- `fixed-by:` names something that will still exist: a commit or a pull request, never a branch. A
|
||||
branch is deleted when it merges, so a branch name there is a pointer that resolves to nothing by
|
||||
the time anybody follows it.
|
||||
- A fix that turns out to have broken something else is written back into the record that asked for
|
||||
it, pointing at the new issue. Somebody arriving at a record to learn why the code is the way it
|
||||
is must not have to already know there was a sequel.
|
||||
- A number never moves. A folder carries the number the tracker gave; a folder whose number collides with
|
||||
another (one numbered before the tracker, or by hand) is a fault, and the cycle test refuses two records
|
||||
sharing a number.
|
||||
- An issue whose answer is a general lesson should also have that lesson written where a search by
|
||||
the symptom's literal words finds it, so the next person searching it finds it. Both, not either.
|
||||
- `wontfix` is legitimate and requires a sentence saying why; the tracker refuses it without one.
|
||||
- **An agent says when it starts work on an issue**, by moving it to `diagnosing` or `fixing` with `by`. Two agents
|
||||
never start the same work without knowing: the second is refused, naming the first.
|
||||
- **A rule states how it is checked.** The cycle's rules on an issue — a status one of the six, `diagnosing` and
|
||||
`fixing` with a worker and no other current one, `located` with an owner, `resolved` only from `fixing` and with a
|
||||
fix, a core issue with its replay, `wontfix` with a reason — are refused at the write by
|
||||
`issue-tracker.move` (mesh-catalog `modules/mesh-issues`, its table test over the rules). the cycle test checks what
|
||||
this repository still holds: no two folders share a number, every report has frontmatter with the day it was
|
||||
`opened:`, and no report carries a status key.
|
||||
- Renumbering a collision happens once, in the branch that lands last. Renumbering a branch whose
|
||||
author is still pushing only moves the race.
|
||||
- An issue whose answer is a general lesson should also be written to the knowledge base, so
|
||||
the next person searching a symptom finds it. Both, not either.
|
||||
- `status: wontfix` is legitimate and requires a sentence saying why.
|
||||
|
||||
@@ -1,79 +1,30 @@
|
||||
# Playbook 04 — Work breakdown and handoff
|
||||
# Playbook 04 — Build handoff
|
||||
|
||||
**Trigger.** A to-be design is settled (playbook [02](02-graduation.md)) and its building is about to start.
|
||||
**Trigger.** A to-be design is settled and work is about to start in a code repository.
|
||||
|
||||
**Who runs it.** The lead who will run the feature, on the larger model. It is one sitting, bounded at about
|
||||
an hour, and said to the operator if it runs past.
|
||||
|
||||
## What it produces
|
||||
|
||||
A design becomes **milestones**, and each milestone becomes **issues**:
|
||||
|
||||
- **A milestone is usable on its own.** Once it is delivered, the operator or the mesh can do
|
||||
something new, said in one sentence ("a Claude Code session is mirrored into a thread on the bus").
|
||||
A milestone that changes nothing anyone can use is not a milestone; fold it into the next.
|
||||
- **A milestone is one release** of playbook [07](07-feature-branches.md): prepared, developed whole, reviewed,
|
||||
checked and delivered as one, never cut for its size or for time (the operator, 2026-10-17).
|
||||
- **A milestone is one piece of functionality, never cut by an order between its parts** (the operator, 2026-10-21:
|
||||
a milestone may span several modules, repositories and dependencies such as an SDK change; 2026-10-22: "a milestone
|
||||
is a milestone, we shouldn't cut it"). Which of its modules reaches the machines first (a reader of a new manifest
|
||||
word before the modules that use it, a provider before its consumers) is the assigner's rules' (design 47); an SDK
|
||||
change is built into its users. The parts such an order touches are found in playbook [07](07-feature-branches.md)
|
||||
step 0 item 6 and become issues of the same milestone; an order those rules cannot keep is an issue of the build
|
||||
and delivery system, not a cut.
|
||||
Until 2026-10-22 this rule cut milestones by release order (design 47's release 1-2 became six milestones).
|
||||
- **An issue is one repository's part of a milestone,** sized so a fixing agent finishes it inside the
|
||||
milestone's development time. Issues that must land in order are linked `depends-on`.
|
||||
**Who runs it.** Whoever starts the build.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Confirm the design is settled.** Its frontmatter reads `status: designed`, and every claim in it traces
|
||||
to a decision record in the knowledge base (`knowledge-base.get` for each entry of its `decisions:`). An
|
||||
open question in the text is a reason to go back to playbook [01](01-research.md), not to build around it.
|
||||
2. **Name the owners.** Set `code:` in the design's frontmatter to the repositories from
|
||||
[`repos.md`](../repos.md). A repository that does not exist yet is added to `repos.md` in the same change.
|
||||
3. **Check the as-is.** Read the matching `03-DESIGN/00-as-is/` document. What is being replaced is stated
|
||||
there; if it is not, write it before changing it. Building against an undocumented as-is is how a shipped
|
||||
behaviour gets lost.
|
||||
4. **Break it into milestones.** Write them into the design, in a section *Milestones*: for each, its number,
|
||||
its one-sentence outcome, the acceptance tests of the design it proves, the repositories it touches, and
|
||||
what it depends on. Order them so each is usable as soon as it is delivered, the smallest useful one
|
||||
first. A milestone is named by what it lets someone do; how it ships is shaped by the modules its changes move
|
||||
and the nodes that run them (ADR 0401 §6), read from the indexer's fingerprints through the status view's
|
||||
`indexer_modules` and `indexer_candidates`, and from the nodes' haves (`node_haves`).
|
||||
5. **Ask the operator to confirm the milestones**, in a short table: number, outcome, the acceptance tests it
|
||||
proves, repositories, the parts each repository builds. Change what they change.
|
||||
6. **Open the issues.** For each confirmed milestone, `issue-tracker.open` one issue per repository part, its
|
||||
title the symptom or capability in plain words, its report naming the design, the milestone and the
|
||||
decisions it builds with; link the order with `depends-on`. Write the issue numbers into the milestone's
|
||||
entry in the design.
|
||||
7. **Flip the status.** `status: in-progress`, `updated:` today; the design change goes through playbook
|
||||
[08](08-small-fixes.md) (one hq pull request).
|
||||
8. **Hand each milestone to playbook 07, one at a time.** The milestone is one release of playbook 07; its issues
|
||||
are the release's issues; the next milestone starts when the previous one is delivered. A design waits
|
||||
while another design's milestones are being built: one design is built at a time (playbook
|
||||
[07](07-feature-branches.md)), so this step starts only when the design before it is delivered.
|
||||
9. **On completion of the last milestone**, run the "when something ships" section of playbook
|
||||
1. **Confirm the design is settled.** Its frontmatter reads `status: designed`, and every
|
||||
claim in it traces to a record in [`02-DECISIONS/`](../../02-DECISIONS/). An open question in the text is a
|
||||
reason to run playbook [01](01-research.md), not to start building around it.
|
||||
2. **Name the owner.** Set `code:` in the design's frontmatter to the repositories from
|
||||
[`repos.md`](../repos.md). If the repository does not exist yet, add it to `repos.md` in
|
||||
the same change.
|
||||
3. **Check the as-is.** Read the matching `03-DESIGN/00-as-is/` document. What is being
|
||||
replaced is stated there; if it is not, write it before changing it. Building against an
|
||||
undocumented as-is is how a shipped behaviour gets lost.
|
||||
4. **Flip the status.** `status: in-progress`, `updated:` today.
|
||||
5. **Build in the code repository.** HQ is not a code repository and never carries
|
||||
implementation.
|
||||
6. **On completion**, run the "when something ships" section of playbook
|
||||
[02](02-graduation.md).
|
||||
|
||||
## Rules
|
||||
|
||||
- What the handoff leaves for later is an issue with an owner, named in the design or the decision that leaves
|
||||
it — never a sentence saying "later". The design's `code:` names who owns what is built; the issue names who
|
||||
owns what is not. The knowledge base's rule `deferral` checks the decision record when it is proposed
|
||||
([issue 354](../../04-ISSUES/354-a-decided-rename-was-never-scheduled-and-the-operator-believed-it-live/00-report.md)).
|
||||
- HQ is not a code repository and never carries implementation.
|
||||
- How the building is done — worktrees, agents, review, checks, the Approve and the delivery — is playbook
|
||||
07's, not this one's.
|
||||
|
||||
## How it is checked
|
||||
|
||||
Read by the lead and the reviewer today; a check among hq's Go tests is tracked by issue 845.
|
||||
|
||||
- A design in `status: in-progress` has a *Milestones* section, each milestone with its outcome and its
|
||||
issues; an in-progress design without one is the smell of building without a breakdown.
|
||||
- Every issue a milestone names exists in the tracker (`issue-tracker.get`), and an issue in `fixing` names
|
||||
its milestone in its report.
|
||||
- A milestone whose issues are all `resolved` is delivered (the status view said `delivered` for its source
|
||||
revision); its outcome sentence and
|
||||
acceptance tests are proven on the mesh before the next milestone starts.
|
||||
- Every merge is a human checkpoint, without exception.
|
||||
- Never open a pull request unprompted. A permissions list saying it is allowed is not a
|
||||
request.
|
||||
- Work in an isolated worktree, never a shared checkout. A failed `cd` in a shared checkout
|
||||
commits to the wrong branch, and the error scrolls past.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Playbook 05 — Constitution sync
|
||||
|
||||
Implements ADR 0021: HQ is
|
||||
the source of the constitution, and the constitution page is derived, never edited.
|
||||
Implements [ADR 0021](../../02-DECISIONS/0021-hq-is-the-source-of-the-constitution.md): HQ is
|
||||
the source of the constitution, and the knowledge-base page is derived, never edited.
|
||||
|
||||
**Trigger.** [`how-we-build.md`](../how-we-build.md) changed a rule that the mesh enforces at
|
||||
runtime.
|
||||
@@ -24,7 +24,7 @@ and, per the repository's own rule, it is how the rule "HQ is the source" is che
|
||||
the derived page carries the rule alone.
|
||||
2. Record the change as a decision — a rule the mesh enforces is architecturally significant.
|
||||
Playbook [02](02-graduation.md).
|
||||
3. Publish the derived page where the mesh keeps the constitution page, under its slug, replacing its
|
||||
3. Publish the derived page to the knowledge base under the constitution slug, replacing its
|
||||
body. Keep the section numbering stable: the meeting orchestrator and the review fragments
|
||||
cite sections by number.
|
||||
4. Verify the derived page reads back with the change present. A publish that reported success
|
||||
|
||||
@@ -25,7 +25,7 @@ Three questions, answered from the machine:
|
||||
## The steps
|
||||
|
||||
1. **Name what it provides and requires**, if anything. A name is what a consumer is coupled to,
|
||||
not the role it plays (ADR 0027):
|
||||
not the role it plays ([ADR 0027](../../02-DECISIONS/0027-a-provision-names-what-the-consumer-is-coupled-to.md)):
|
||||
`postgres-database`, not `database`. Most modules provide nothing and require nothing — an
|
||||
application is usually a leaf.
|
||||
|
||||
@@ -103,25 +103,5 @@ time and the scaffolding was not.**
|
||||
there and nowhere else.
|
||||
- **A failing test is kept, not skipped.** It is the reproduction.
|
||||
- **Never rotate during an adoption.** Rotation is a separate act, afterwards, deliberately.
|
||||
- **A data directory is never removed by the mesh** (ADR 0030),
|
||||
- **A data directory is never removed by the mesh** ([ADR 0030](../../02-DECISIONS/0030-data-outlives-the-mesh-that-declared-it.md)),
|
||||
and that protects against the mesh only — not against a disk or a mistaken command.
|
||||
|
||||
## Moving a module's directory
|
||||
|
||||
When a fix moves what a container mounts to a new path (issue 1843: only-office's working directories
|
||||
moved into a root-only box), the old directory does not go away by itself, and it is closed in three steps.
|
||||
|
||||
1. **The move.** The new directory gets a new id at its new path. The old one keeps **its old id at its old
|
||||
path** for as long as it holds data, with the mode it should now have: the node-engine leaves a directory
|
||||
it finds at a path under a new id as it found it, and sets no mode when the owner differs (issue 339), so
|
||||
only the old id closes it. The node-engine removes a directory that is no longer declared only when it is
|
||||
empty; a non-empty one is kept on disk with no further cleanup.
|
||||
2. **The deletion, on the operator's word.** Its contents are data, so they are removed only after the
|
||||
operator's yes, through `<node>/node-files.delete`. That tool moves into the node's trash: the disk space
|
||||
comes back only when the trash is emptied (issue 1870: no tool does that yet).
|
||||
3. **The declaration leaves** in a second small fix, once the directory is empty (issue 1854).
|
||||
|
||||
Plan the three in the first fix's issue, so the second fix is not found later by accident.
|
||||
|
||||
A service that starts on empty working directories may need minutes before it answers (OnlyOffice: about
|
||||
two): the live check after the delivery waits for it before calling the move broken.
|
||||
|
||||
@@ -1,655 +1,69 @@
|
||||
# Playbook 07 — Features
|
||||
# Playbook 07 — Feature branches across repos
|
||||
|
||||
**Trigger.** A feature: work that changes code in one repository or several (`mesh-sdk`, `mesh-controller`,
|
||||
`mesh-catalog`, `mesh-host`, `mesh-lab`, and `hq` when a design rides along), too large to be one small fix.
|
||||
A small fix (one repository, one issue, about an hour of work) follows
|
||||
[playbook 08](08-small-fixes.md) instead.
|
||||
**Trigger.** Work that changes code — in one code repo or in several at once (`mesh-sdk`,
|
||||
`mesh-controller`, `mesh-catalog`, `mesh-host`, `mesh-lab`, and `hq` when a decision rides along).
|
||||
|
||||
**Who runs it.** Anyone who writes code, engineers and agents alike. Agents follow it exactly.
|
||||
**Who runs it.** Anyone who writes code, engineers and agents alike. Agents follow it exactly —
|
||||
it is the guard against the failure it was written for.
|
||||
|
||||
## The failures it prevents
|
||||
## The failure it prevents
|
||||
|
||||
- **Pull requests as scratchpads.** A feature worked as one pull request per unit of thought left sixteen
|
||||
stacked pull requests unmerged, consolidated by hand afterwards. A pull request is the end of a feature,
|
||||
not a step in it.
|
||||
- **Waiting on checks mid-feature.** A feature worked as one pull request per issue into protected
|
||||
branches spent most of a day waiting for merge checks while the code was ready.
|
||||
- **Context bought again and again.** A fresh agent per small issue spends most of its run reading the
|
||||
playbook, the issue and the code before it writes a line. On 2026-10-14 an issue sized for five to ten
|
||||
minutes took forty-one.
|
||||
- **Finished work that never ships.** A feature that kept taking new issues held thirteen landed issues
|
||||
undelivered for a day: an issue done at 21:09 was still not on the machines at 15:00 the next day.
|
||||
- **Work that waits and nobody notices.** Most lost time was not work at all, but a check, a delivery
|
||||
or an agent waiting on something that had stopped, unseen.
|
||||
- **Review that keeps hardening.** One fix went through seven review rounds in five and a half hours, each
|
||||
asking for more hardening than its issue asked (issue 704, 2026-10-14). A round is for what makes the
|
||||
change wrong; everything else is an issue of its own.
|
||||
- **The first check after the whole branch is written.** A feature's catalogue branch, pushed once after
|
||||
hours of local work, met its first check on 2026-10-16 and failed on 29 problems, TypeScript errors among
|
||||
them, that a unit suite run continuously while working finds at once.
|
||||
- **A green pull request nobody marks.** An approved, green pull request that was not marked ready waited
|
||||
12 h 17 min for someone to merge it (mesh-catalog #300, 2026-10-14). There is no ready mark since ADR 0421:
|
||||
the operator's Approve is asked as soon as the checks pass.
|
||||
- **Review rounds spent on what a run or a search would have shown.** On 2026-10-19 milestone R3b of design 47
|
||||
took two review rounds and one more cycle after a failed try. A test that "failed first" only by not compiling
|
||||
proved nothing, and dropping the field it was for passed every test. A reader of the changed record that the
|
||||
survey never listed ended a module's conditions at each sync. The reviewer judged the bus tests by reasoning, and
|
||||
wrongly, because they ran only after the review.
|
||||
- **A check asked again and again.** Build capacity is the mesh's scarcest resource ("we don't have more
|
||||
processing power", the operator, 2026-10-26), and a release that ran a try before its review, a try after it
|
||||
and a check per fix round on an open pull request spent it several times over: every push to an open pull
|
||||
request asks a new check, of every member for a group. A release runs one check ("every release should only
|
||||
run 1 check or 1 try", the operator, 2026-10-26; ADR 0548).
|
||||
- **A node agent's bus reads, proven only by review.** On 2026-10-27 a change widened what every node agent watches
|
||||
on the bus; two reviews read the code, no test ran it under a node's real permissions, and once it was live every
|
||||
node's agent was refused that read and stopped composing (issue 1126). How-we-build's rule "A bus permission is
|
||||
proven under the real permission" (ADR 0560) is what the review now asks for.
|
||||
- **A module proven only after it reached the machines.** The mail modules were delivered with unit tests
|
||||
only, then failed live three times (issues 859, 865, 868) and took five more hours of fixes.
|
||||
A feature was worked as a branch-and-MR per *unit of thought* — one per decision, one per
|
||||
stacked increment — and each MR was treated as finished when it was *opened*, not when it was
|
||||
*merged*. Across repos the same feature took a different branch name in each. The MRs piled up
|
||||
unmerged: one session left **sixteen** stacked intermediate MRs that had to be consolidated and
|
||||
closed by hand. An MR is a review checkpoint, not a scratchpad.
|
||||
|
||||
## The shape
|
||||
## The rule
|
||||
|
||||
One feature is **one branch name**, **one worktree per repository**, **one pull request per repository**,
|
||||
opened **once**, after the review, and **one check** (ADR 0548). It runs in five steps, and its time is bounded. A milestone has no phases
|
||||
(the operator, 2026-10-17): these are its steps, and a report names the step it is in:
|
||||
One feature is **one branch name**, **one worktree per repo**, **one MR per repo**, opened
|
||||
**once, at the end**.
|
||||
|
||||
| Step | What happens | Bound |
|
||||
|---|---|---|
|
||||
| 0. Prepare | the lead settles the open questions and writes one context file per repository | 15 minutes |
|
||||
| 1. Develop | one fixing agent per repository builds every issue locally, the whole unit suite before every commit, the integration tests compiled; nothing is pushed | the milestone's issues, 30 minutes each |
|
||||
| 2. Review | one reviewer reads all of it, the local commits, judging the integration tests by reading them against the code; reports go both ways; at most 3 rounds | 15 minutes |
|
||||
| 3. Check | push the reviewed heads once and open the pull requests together; the checker's one check runs the integration tests; no further review unless the check fails | one check run |
|
||||
| 4. Deliver | merge; the source revision moves, each module is built once and wanted on its nodes, its canary first | until the view says the source revision `delivered` |
|
||||
|
||||
**One milestone is one release** (the operator, 2026-10-17). A milestone is developed whole, reviewed once,
|
||||
pushed once, checked once and delivered once. It is never cut into steps for its size or for time: every cut adds a review, a
|
||||
merge check, an Approve and a delivery of waiting, so the fewest cuts is the fastest. Nor is it cut for an
|
||||
order between its parts (the operator, 2026-10-22: "a milestone is a milestone, we shouldn't cut it"): which module
|
||||
reaches the machines before which is the assigner's rules' (a build and a composition before the want, a provider
|
||||
before its consumers, the canary before the rest; design 47), and that is what they are for. An order they cannot
|
||||
keep is a fault of the build and delivery system, opened as an issue there, never a cut and never a step a session
|
||||
takes by hand.
|
||||
A milestone is one piece of functionality, possibly spanning several modules, repositories and dependencies such as
|
||||
an SDK change (the operator, 2026-10-21). Its pull requests in every repository share the milestone's branch name,
|
||||
so they are one delivery group (ADR 0482, which holds that a change across repositories is one group, one plan and
|
||||
one Approve, and that its developer orders no release). Nothing of the milestone merges until every issue of it is
|
||||
built, reviewed and checked; then the one Approve releases it all as one source revision, the assigner's rules
|
||||
keep a provider before its consumers, and an SDK change is built into the modules that use it. A small fix found during the milestone rides in this
|
||||
group, as playbook [08](08-small-fixes.md) says.
|
||||
|
||||
**One design is built at a time** (the operator, 2026-10-17). While the milestones of one design are being
|
||||
built, no milestone of another design starts: its issues stay `open`, and nobody moves them to `diagnosing` or
|
||||
`fixing`. The next design starts when the last milestone of the one being built is delivered. A
|
||||
lead who finds another design's milestone already in `fixing` asks its owner before starting its own. A small
|
||||
fix ([playbook 08](08-small-fixes.md)) and an issue that is in no design's milestones are not held by this rule.
|
||||
|
||||
**Scope is fixed at the prepare step.** An issue found during a feature never goes into the running milestone, unless it
|
||||
blocks the running one: it is fixed in between or right after the delivery, as the next paragraph says, or goes to the
|
||||
next milestone or the next feature.
|
||||
|
||||
**An issue opened along the way is fixed by the session that opened it** (the operator, 2026-10-21): it is never only
|
||||
listed under "Opened along the way" and left. A small one (one repository, about an hour) is fixed in between, as a
|
||||
small fix ([playbook 08](08-small-fixes.md)) while the milestone waits on a check, an Approve or a delivery; the
|
||||
rest are fixed right after the milestone's delivery, before the next milestone starts, together on one branch where
|
||||
they share a repository. Either way it stays out of the running milestone's branch and pull request, unless it blocks
|
||||
the running milestone, so the scope above holds. Only an issue in another design's milestones (one design is built at
|
||||
a time), or one that needs a decision or the operator first, waits, and the report says which and why. Each report's line "Opened along the way" says for every
|
||||
issue whether it is being fixed in between, after the delivery, or waits, and on what.
|
||||
|
||||
## Step 0 — Prepare (the lead, larger model)
|
||||
|
||||
1. **Name the feature once:** `feat/<slug>`, the same branch name in every repository it touches (but see the
|
||||
exception for a new manifest word in phase 3: a release that must merge first has a name of its own). The
|
||||
branch is never protected. **One milestone is one branch per repository** (the operator, 2026-10-17), with one
|
||||
folder `.work/<slug>/`; never a second name for part of it (`-2`, `-step2`).
|
||||
2. **Make the worktrees:** one per touched repository under `.work/<slug>/<repo>`, branched off `main`
|
||||
(`git worktree add .work/<slug>/<repo> -b feat/<slug> origin/main`). The lab beds find the other
|
||||
repositories by sibling path, so add the untouched siblings the lab reads as detached worktrees on
|
||||
`main` (`git worktree add --detach .work/<slug>/mesh-tools main`), never symlinks. **A repository's own
|
||||
clone stays on `main`** (the operator, 2026-10-17): no work is done in it and no branch is checked out
|
||||
there, so a session that opens in it reads what holds now; fetch and fast-forward it before reading.
|
||||
3. **Settle every open question** in the feature's issues before any agent starts: on the issue, through
|
||||
the knowledge base, or by asking the operator. **Read what is decided before designing** (`knowledge-base.ask`):
|
||||
a question about delivery, checks or builds is most likely answered in design 47 and its decisions already (the operator,
|
||||
2026-10-26: "that is simply our delivery plan and milestones, we've already researched it"); propose new research
|
||||
only for what no record decides. A fixing agent that meets an undecided design question
|
||||
stops, and its run is spent again.
|
||||
4. **Write one context file per repository**, `.work/<slug>/<repo>.context.md`. It is what a fixing agent
|
||||
reads instead of rediscovering the code, and it is reused by every agent and every round:
|
||||
- the issues it builds, in order, each with: the files and functions it changes, the decisions it builds
|
||||
with (quoted, with where each was decided), and its acceptance criteria (item 5), each with the test that
|
||||
proves it;
|
||||
- the repository's map for this feature: the packages and files involved, with one line each on what they
|
||||
hold, and the patterns to copy (file:line);
|
||||
- what not to touch, and what lands beside it in other repositories;
|
||||
- the local facts: the worktree, the branch, the check commands, a temporary directory for tests outside
|
||||
the home (`TMPDIR`), and the unit tests to run (never a database server: tests that need one are
|
||||
integration tests, which the pull request's one check runs).
|
||||
When the reading is large, a read-only agent on the smaller model answers the lead's exact questions with
|
||||
file:line, and the lead writes the file from its answers.
|
||||
5. **Write the findings into the issues** (the operator, 2026-10-17: the better the content of the issues, the
|
||||
better work the agents do, and the less review is needed). Before any agent starts, each issue carries, with
|
||||
`issue-tracker.note`: every file and test that touches the change, found by search and not from memory
|
||||
(file:line); **every other reader and writer of each value the change adds or changes** (a field, a key, a kind
|
||||
of record), each with what it does with the new value, and a criterion for each that could treat it wrongly
|
||||
(2026-10-19: the router's sync with the controller ended every condition the controller did not list, and the
|
||||
survey, which followed only the new condition's own path, never named it); how the change is checked and delivered end to end (which test proves its done-when, which
|
||||
modules its merge moves, read from the indexer's candidates of its group, and to which machines); and for a
|
||||
group, its merge order. The context file
|
||||
points to these notes; an issue reread by another session or a later agent then holds what was found.
|
||||
When the reading is large, it starts with a read-only survey agent on the smaller model.
|
||||
**Each issue carries its acceptance criteria, written by the lead** (the operator, 2026-10-18: the larger
|
||||
model writes them, and they guide every test). Each criterion is one sentence, given, when, then, about what
|
||||
the module or the operator observes, never about how the code does it, and names the level of the test that
|
||||
proves it: unit, integration (a bus, a store or a process started) or end to end (none exist yet, so such a
|
||||
criterion is proven live after the delivery and said so). Together they cover the issue's done-when and the
|
||||
cases its life passes through: on 2026-10-18 mesh-delivery's group life (a group of three, a member's own
|
||||
check, a close in a group, a second pull request at the same commit, a stop and a resend) had no criterion,
|
||||
and each case was found by a failure on the live mesh. Each criterion is numbered in its note and cited as
|
||||
`<issue>/AC<n>` (`1118/AC2`). They are an `issue-tracker.note` headed `Acceptance criteria`, a revision a new note that replaces the older one, until the tracker holds them as a
|
||||
field (issue 1118). A criterion the reviewer adds in step 2 is written onto the issue the same way.
|
||||
**The survey sizes each issue, and an issue sized over an hour is split before any agent starts** (the operator,
|
||||
2026-10-17), into issues of about 30 minutes each, one change each, in the order they land: the bound of a
|
||||
fixing agent's issue is then a real bound. On 2026-10-17 one issue held four changes sized at four to six
|
||||
hours; it ran past its 30-minute bound for over an hour, and the reports could only say "still on part D".
|
||||
6. **List every part the change reaches, before any agent starts**, so each is an issue of the milestone and
|
||||
nothing is found on the way. Read, for each repository of the milestone, what it takes from the others and how:
|
||||
the SDK's users (they pin an SDK commit, and a pin to a commit no branch holds is a download that fails later,
|
||||
issue 1148); every program that parses a manifest strictly (search `ParseManifest` across every repository, with
|
||||
the SDK commit each pins), which must take an SDK that knows a new manifest word (2026-10-21: design 60's
|
||||
catalogue milestone found mesh-build-go and the delivery's indexer reading manifests with an SDK from before the
|
||||
word); the SDK's list of a defined seat's verbs, which must name a verb a module adds (2026-10-21: node-files' two
|
||||
new verbs were refused on the claim); the node-engine's decoder for a new key on the wire; the route by which a
|
||||
repository's first module registers; and a part that needs something the bus or a manifest cannot yet give, which
|
||||
needs its decision first. Each such part is an issue of the milestone and its pull request a member of the group;
|
||||
the order in which they reach the machines is the assigner's rules' (the operator, 2026-10-22), never a cut.
|
||||
Where those rules cannot keep an order (a node told of a key before it reads it, a module using a word before its
|
||||
readers are delivered), that is an issue of the build and delivery system, opened with `issue-tracker.open` and
|
||||
said to the operator.
|
||||
7. **Write fixtures from the shape the other side writes, never from memory** (2026-10-18: a watchman's tests
|
||||
mirrored a build key the controller never writes, and every test passed): copy a value from the writer's code or
|
||||
from the live bus, and name where it came from.
|
||||
8. **Answer three questions in the milestone file** (`.work/<slug>/milestone.md`; research 071, issue 1941), written by the
|
||||
lead from this template before any agent starts. Each answer is a fact found, not a hope:
|
||||
1. **Name the feature once.** `feat/<slug>`. The *same* branch name in every repo the feature
|
||||
touches — never a different name per repo, never a fresh branch per increment within the
|
||||
feature.
|
||||
2. **Isolate each repo.** One git worktree per touched repo under `.work/<slug>/<repo>`, branched
|
||||
off `main`:
|
||||
```
|
||||
Lifetime of what the work creates (research 071, a):
|
||||
- <bucket key or stream> | written by <module> | deleted by <module> on <event>, or time to live <duration>
|
||||
(one line for every key-value bucket key and stream the work creates or starts writing; "none created" if none)
|
||||
Trying the switch-over on one node first (b):
|
||||
- <which node goes first, and what is read there before the rest>
|
||||
- or: the operator chose no first trial on one node, on <date>; what that risks: <one sentence>
|
||||
Integration level (c):
|
||||
- <the one check that runs it: the group's pull requests, one check each, none before the review>
|
||||
- what the check does not cover: <the parts of the group that meet only live>
|
||||
git worktree add .work/<slug>/<repo> -b feat/<slug> origin/main
|
||||
```
|
||||
A key or stream with no one who deletes it and no time to live is a leak and is settled before work starts.
|
||||
9. **Move the issues to `fixing`**, `by` naming the feature's agent for that repository.
|
||||
Parallel features never collide, and no shared checkout is edited.
|
||||
|
||||
## Step 1 — Develop (one fixing agent per repository, smaller model)
|
||||
**Add the untouched siblings the lab reads.** The lab beds find the other repositories by
|
||||
sibling path from the lab checkout (`../mesh-tools/module.json`, `../mesh-sdk`, …), the way
|
||||
the main layout has them. A `.work/<slug>/` directory holding only the touched repos fails a
|
||||
bed at once with `no manifest for mesh-tools at …/.work/<slug>/mesh-tools/module.json`, after
|
||||
genesis has already passed. Give the directory those repos as **detached worktrees on `main`**
|
||||
— never symlinks:
|
||||
```
|
||||
git worktree add --detach .work/<slug>/mesh-tools main
|
||||
```
|
||||
3. **Commit as you go — locally.** Increments land on the one branch. Nothing is pushed and no
|
||||
MR is opened mid-feature.
|
||||
4. **Finish, then publish.** When the whole feature is done — every repo, tests green — push
|
||||
every branch and open **one MR per touched repo**, together.
|
||||
5. **Merge promptly, once approved.** Every merge into `main` is notified and approved
|
||||
([ADR 0023](../../02-DECISIONS/0023-approval-is-the-checkpoint.md)); once it is, merge —
|
||||
do not leave it sitting. The branch is deleted on merge.
|
||||
6. **Leave nothing behind.** After the MRs merge, no `feat/<slug>` branch and no `.work/<slug>`
|
||||
worktree survive.
|
||||
|
||||
- **One agent per repository, kept for the whole feature.** It reads its context file once and builds the
|
||||
issues one after another, committing each locally with a message naming the issue ("Keep the assignments
|
||||
on the bus (issue 652)"). It is not replaced per issue: the lead continues the same agent (by its id) with
|
||||
the next instruction, so its context is paid for once.
|
||||
- **Unit tests only, each well under a second** (how-we-build §5, ADR 0391). **The whole unit suite runs before
|
||||
every commit** (the operator, 2026-10-17), not only the tests of what changed: it takes under a second, and a
|
||||
change breaks tests in packages it never touched. Each issue's first test fails before its change, on an assertion about what the
|
||||
code does, and passes after. **A test that fails only because it does not compile has not failed first**: where
|
||||
the test calls what the change adds, the agent takes out the change's effect (the line that does it, not the new
|
||||
function), sees the test fail, and puts it back (2026-10-19: a check request's new field was proven by a test of
|
||||
its helper alone, and dropping the field from the request passed every test); `gofmt`, `go vet` and the whole unit suite stay green. **The tests are written from the issue's
|
||||
acceptance criteria**: one test or more per criterion at the level it names, each test's comment citing
|
||||
the criterion it proves by its label (`// 1118/AC2`), and no criterion left without its test; an integration criterion is a test
|
||||
that compiles and is run by the pull request's one check. The integration tests (those that start a store, a bus
|
||||
or a process) are compiled with `go vet -tags integration ./...` while developing and run by that check, nowhere
|
||||
else.
|
||||
- **Nothing is pushed and no pull request is opened in this step, and there is no try** (ADR 0548; the operator,
|
||||
2026-10-26: "every release should only run 1 check or 1 try"). When every issue is committed, the lead brings each
|
||||
feature branch up to the current trunk (fetch by name, then merge) and the review reads the local commits. Never call
|
||||
`mesh-controller.try`, `mesh-delivery-checker.try` or `mesh-delivery-checker.checker_try`; one is run only when the operator asks for it. A reviewer
|
||||
or a session that wants the code on the forge may push a branch that has no pull request: it runs nothing. Never land
|
||||
on a red check: read the result, then land in a separate step.
|
||||
- **The fixing agent's brief** (the lead gives it with the context file): name your issue in every step and
|
||||
message ("issue 805: landing in mesh-catalog"); reach the mesh only through the mesh MCP server, never ssh;
|
||||
never push, open, merge or close a pull request; never call a try; never write to a store's database by hand, a schema change
|
||||
is a numbered migration and the consolidated schema; Go only unless the context file says otherwise; never
|
||||
move another issue; **leave the integration tests to the check** (no store or bus started by hand): compile them
|
||||
with `go vet -tags integration ./...`. A single integration test may be run locally against an embedded bus when
|
||||
it is the fastest way to find a cause, said in the review report and the pull request. After a failed check it
|
||||
usually pays to run that repository's whole integration level locally (`go test -tags integration -count=1 ./...` in each
|
||||
module), by the lead or by the fixing agent on the lead's word, before the one more push: a verdict names only the first failed
|
||||
test (issue 1921), and on 2026-10-23 R4g's faults surfaced one per run until a local run listed them all in a minute (a
|
||||
guideline, the operator). Report back in under 150 words: commits, tests added and that they failed first, the
|
||||
local results, anything you stopped on.
|
||||
- **An agent that finds the code differs from its context file stops and reports** (file:line); the lead
|
||||
settles it, updates the context file, and continues the agent.
|
||||
- **Two repositories that depend on each other** keep their order in the context files: the reader before the
|
||||
writer, the controller's new manifest word before the catalogue manifest that uses it.
|
||||
- **Several agents in one repository** (only for a large repository with independent areas): each works on a
|
||||
local branch of its own from the feature branch's head and lands by fast-forward under a lock
|
||||
(`flock <feature worktree>.lock git -C <feature worktree> merge --ff-only <its branch>`), after its unit
|
||||
tests pass; if another landed first, it merges again and repeats. In a shared clone, fetch the feature
|
||||
branch by name before branching from it. Never resolve a conflict with a blind `git merge-file --union`.
|
||||
## What this is not
|
||||
|
||||
## Step 2 — Review (one reviewer, larger model; at most 3 rounds, about 15 minutes)
|
||||
|
||||
1. **The reviewed tree is the tree that is pushed**: every feature branch was brought up to the current trunk
|
||||
before the review (step 1). The check runs on the merge with the trunk as it is then, so a
|
||||
trunk that moves later is met by the forge's merge check, not by another merge here. Every verdict names the
|
||||
trunk it merged onto ("pass <head> on <trunk>"), and the lead reads that trunk against the repository's main
|
||||
at once: a trunk behind main means the source revision (source.main) lags, and every check in that
|
||||
repository is judged on it until it moves, so the operator's Approve will merge nothing. Say it then, not at the
|
||||
merge (issue 1726: on 2026-10-24 the gate dropped two merges it heard before it held its seat, and a
|
||||
reviewed, approved pull request in each of the two repositories was refused at its Approve).
|
||||
2. **One reviewer reads every repository's local commits** together, against the context files and the
|
||||
decisions. **It is one agent, kept for the whole release** like the fixing agents: the lead continues it by
|
||||
its id for every later round, for the extra cycle after a failed check (step 3) and for its review record, so it
|
||||
reads the decisions, the issues, the context files and the commits once (issue 1042). A fresh reviewer per
|
||||
round or per mark pays for all of that again. Its report is per repository: each finding as file:line, the failure it causes, and the fix.
|
||||
It says which round it is in ("round 1 of at most 3").
|
||||
3. **The report goes to the same fixing agents** that wrote the code, continued by their ids, so they fix
|
||||
with the context they already hold. Each answers the reviewer: what it changed, per finding, or why not.
|
||||
4. **Every new integration test is read against the code it drives.** Fixing agents only compile it, and no run precedes the review (step 1), so the reviewer judges it by reading; the check that follows shows whether it passed but not whether it proves what it says. The reviewer traces each step of each new integration test through the code (fixtures against the manifest rules, what the store keeps and reads back, who holds a seat), and says which steps it could not trace. On 2026-10-17 a new test failed on its first two runs, on a fixture and then on a fault of the change; reading it against the code after the first failure found another fault, which would have failed a run too. A review that finds a fault only by reading is the cheaper run. The reviewer also checks that each new integration test enters through the same path the live trigger takes, not through a state it writes by hand: on 2026-10-17 milestone 4's test saved its delivery plan directly, so it skipped the cut from a batch that every live plan takes, and it passed while the backup step never ran (issue 1086).
|
||||
5. **The reviewer reads the answers and the new commits**, and approves or reports again (step 7 says what a round
|
||||
may ask). After round 3 the lead stops and brings the operator what is left, with a recommendation.
|
||||
6. **The reviewer reads every acceptance criterion against its test**: each criterion has a test at its level
|
||||
that would fail without the change, and no test proves less than its criterion says. A criterion missing
|
||||
for a case the issue's done-when needs is written onto the issue (step 0, item 5) and is a finding; a test
|
||||
missing for a criterion is a finding.
|
||||
**A rule keyed on a property is checked against everything else on the mesh that has it.** When a change
|
||||
decides by a property the bus or the store gives (a stream's retention, a folder's name, an account's kind),
|
||||
the reviewer lists what else already has that property, live, and asks what the rule does to it. On
|
||||
2026-10-19 release 1 of feature mail-sender made a work-queue stream's dead letters drop-only by the bus's
|
||||
work-queue retention; three review rounds passed it, and only its try showed that every seat's work stream has
|
||||
that retention too, so dead asks across the mesh would no longer have been delivered again (issue 1178, fixed
|
||||
before its pull request by a mark only a declared module work queue carries).
|
||||
6a. **The reviewer reads the milestone file's three answers** (step 0, item 8) against the change: it lists every bucket
|
||||
key and stream the diff creates or writes (search the diff for the bucket and stream names) and finds each in the
|
||||
lifetime answer with a deleter and event, or a time to live; it reads the switch-over answer and checks it was done as
|
||||
written or that the operator's choice is recorded; it reads the integration-level answer against the check the pull requests will ask. A missing answer, or a key or stream the diff creates that the answer omits, is a change asked.
|
||||
7. **A finding outside the issues' done-when and their acceptance criteria is never a change asked, in any round.** More hardening, a
|
||||
further case, a nicer form: the reviewer opens it with `issue-tracker.open` and names the number in its
|
||||
report, and the round does not wait for it. What a round asks is only what makes the change wrong against
|
||||
its issues, their acceptance criteria and its decisions.
|
||||
8. **A timer that writes state, or a rule fixed in code that overrides what a person sets, makes the change
|
||||
wrong.** The mesh is NATS-native: a part says what happened as an event and acts on events; nothing polls
|
||||
and rewrites state on an interval, and a policy the operator can change is a setting the module declares,
|
||||
never a constant. The reviewer reads every new or changed interval, ticker, sleep loop and periodic
|
||||
run-once in the change and asks of each: what event should drive it instead, and what setting turns it off.
|
||||
One with no answer is a change asked. On 2026-10-23 the forge module's loop rewrote every repository's
|
||||
branch protection every ten minutes from rules in its code and undid the operator's changes at the forge
|
||||
(issue 1903).
|
||||
9. **At the second instance of a fault kind in a review, the developer searches for every instance and names each in the
|
||||
report.** On 2026-10-25 a "busy, return at once" guard that dropped a newer request was found in three places,
|
||||
one review after another. The developer writes the kind in one line, searches the repository (`git grep`) for
|
||||
its shape, fixes every hit in one change and lists each file and line in the report. How it is checked: a
|
||||
report to the second round naming a kind and one place only, followed by a third round on the same kind, is
|
||||
this rule skipped.
|
||||
|
||||
## Step 3 — Check (no review)
|
||||
|
||||
1. **Push the reviewed heads once, then open one pull request per repository, together** (ADR 0548; the operator,
|
||||
2026-10-26: "every release should only run 1 check or 1 try"). **No pull request, draft or not, exists before the
|
||||
review approves the heads**, and no fix round is pushed to an open pull request: every push to an open pull request
|
||||
asks a new check, of every member for a group, and the mesh's build capacity is spent on it ("we don't have more
|
||||
processing power"). The integration tests run once, in the checker's merge check on the merge with the trunk's head, at the unit
|
||||
and the integration level, through the build module of the repository's language and by that language's convention
|
||||
(ADR 0445, ADR 0529); the check waits on the language's check stream until a build node has a free slot
|
||||
(ADR 0525 §1). **No try:** never call `mesh-controller.try`, `mesh-delivery-checker.try` or `mesh-delivery-checker.checker_try`, except when the
|
||||
operator asks for one. While developing, no `go test -tags integration` run of a whole level, and no store or bus started
|
||||
by hand on the machine (the operator, 2026-10-18).
|
||||
**A group is one check.** Before the push, every member pins the group's own mesh-sdk head exactly (the checker refuses an
|
||||
older commit of the same branch), then all members are pushed and opened together, so the group's check runs once. A head the review replaces
|
||||
counts for nothing against a running check; say so, and ask no other until the new head is reviewed.
|
||||
**Before a session trusts an agent's "pushed", it reads the forge branch's head** (`git ls-remote origin refs/heads/<branch>`) and compares it with the commit reviewed; an agent may commit on a detached HEAD and report a push that never happened (2026-10-26: the first check of a reviewed fix failed "not our ref"). The pull request is opened only on that read head. Pull requests in two or
|
||||
more repositories that share the branch name are one delivery group (ADR 0239 §2).
|
||||
**A temporary mode that merges by hand** (the lead merges because the normal Approve and merge path is not
|
||||
running): the pull request's one check still runs the integration level, and the lead names its verdict in the
|
||||
pull request's description. A try with tests `.` is asked only when the operator asks for one, and then named there with its id and verdict (research 071).
|
||||
Where a session pushes through the forge tool `gitea_commit` (one commit on the forge's current base), the
|
||||
reviewer first confirms that commit's tree equals the reviewed one. **A large change is pushed by a pusher agent** on the smaller
|
||||
model: `gitea_commit` patches of about 30 KB each, a file copied from another path sent as a copy (`copy_from`)
|
||||
and not as its bytes, and the forge's tree compared with the local one after the last part (2026-10-20: R4c-2's
|
||||
two repositories went up in parts this way, and the lead's own context stayed free for the review). **Each pull request's description carries
|
||||
the review record** (ADR 0421 §5), one line the lead writes from the reviewer's approval:
|
||||
`Reviewed-by: <who@node> at <full sha>` (the issue 1022 note gives its form). mesh-delivery reads it at every new
|
||||
head and puts it in the Approve's ask; a missing line, or one naming another commit, is said first in the ask.
|
||||
It is never a gate. **Beside it, the description names the session that pushed the head**, one line
|
||||
`Pushed-by: <who@node>` (ADR 0505 §5): when the head cannot merge because its check failed after the Approve (a
|
||||
conflict, a failed check, a rejected group), the cure is that session's new head, and mesh-delivery tells the
|
||||
operator only after two hours with no new head, naming the branch's author and this session. The session that
|
||||
pushed watches its own pull request (the instruction follow-what-you-start) and pushes the new head well inside
|
||||
those two hours. A description without the line names the branch alone.
|
||||
**The description names its issues as `issue <n>`, and only its own** (2026-10-21: okioki #9 listed its four
|
||||
issues bare, `Issues: 1608, 1646, …`, and named a milestone-2 issue as `issue 1654` and a follow-up as
|
||||
`issues 1665, 1666, 1667`, so mesh-delivery carried 1654 and 1665 as the pull request's issues and the Approve's
|
||||
ask named them): mesh-delivery takes a pull request's issues from the words `issue <n>` and `issues <n>` in its
|
||||
description (the plural yields its first number only), so write `issue 1608, issue 1646`, and name any other
|
||||
issue in words, never by `issue <n>` or `issues <n>`.
|
||||
**A catalogue pull request that removes a module, or a subject a module names, is also judged by the
|
||||
controller's unit tests against the catalogue's branch**, because those tests read the catalogue and no check
|
||||
of the catalogue runs them: on 2026-10-20 mesh-catalog #381 removed the old delivery module, and every
|
||||
controller check failed afterwards on a test of the catalogue's subjects until issue 1406 removed the name
|
||||
mesh-issues still listened to. A controller's check does not find it, since a check reads the catalogue's main
|
||||
(also in a delivery group that holds the catalogue's pull request); the lead runs mesh-controller's unit
|
||||
suite with `MESH_CHECK_BESIDE` naming a directory that holds the catalogue's branch, as mesh-controller's
|
||||
`internal/beside` says.
|
||||
2. **The checker's merge check runs the integration tests once, as the gate** (its one status, `mesh/check`). No
|
||||
second review: step 2 reviewed this tree.
|
||||
3. **A check that fails** goes back to the same reviewer, continued by its id, with the failure, for **one more cycle** of step 2 with
|
||||
the same fixing agents; then push **once** again (the one more push of ADR 0548). The check is **read, never re-asked to see if it passes**: a
|
||||
check asked again without a new, reviewed head spends the build capacity for nothing. **The failure is read from the check's whole log**, never
|
||||
reasoned from its summary: `mesh-delivery.show` names the check's build id, and `mesh-controller.builds` with
|
||||
`log` set to that id prints everything the builder said. A check's summary picks only some failing lines;
|
||||
on 2026-10-17 it showed a store test's `FAIL` without its cause, and a fix was first guessed from the code.
|
||||
**A failure the check hides or loses** (a verdict naming only the first failed test, a log not kept) is a fault of the
|
||||
checking tool: it is fixed there, the same turn (instruction fix-a-bug-met-twice), not asked around.
|
||||
**A change to the Go build module's recipe is judged by the build module the nodes run**, not by the changed one, so a
|
||||
test the change adds must pass under either version. **A change that needs the new recipe to judge it ships the recipe
|
||||
first, as a small fix of its own**, and the rest waits for its delivery: on 2026-10-17 milestone 3 of design 47 moved its
|
||||
integration tests behind a build tag in the same pull request as the script that ran that tag, and main's script would
|
||||
have skipped every tagged test, its acceptance tests among them. Since R4g no repository holds such a script (ADR 0529):
|
||||
which tests run at which level is the build tag's (ADR 0445 §2), and hq's checks are Go tests like any repository's
|
||||
(ADR 0527). The prepare step looks for this: a change to what the checker's build module selects (its recipe, a build
|
||||
tag convention) is cut ahead of the milestone.
|
||||
4. **The milestone's acceptance is proven by its check.** A milestone that adds or changes a module
|
||||
carries an integration test that starts the module whole, its process against the real bus and store, and
|
||||
shows the milestone's acceptance sentence; the forge's checks run it (ADR 0391 §1, §6). Unit tests alone are not
|
||||
that proof; the mesh has no end-to-end tests yet, and the lab is not used (the operator, 2026-10-16).
|
||||
**A new manifest word's test goes through the first composition that meets it, not only the verb that fills it.**
|
||||
On 2026-10-19 release 1 of feature mail-sender tested a process's own bus account through `module issue`
|
||||
alone; the first push after mesh-mail declared the process composed a random 40-character stand-in for the
|
||||
account file before anyone issued it, and the process crash-looped until the account was issued and pushed by
|
||||
hand (issue 1226).
|
||||
5. **The operator's Approve is asked as soon as the checks pass** (ADR 0421). Nobody marks, labels, comments or
|
||||
rechecks: mesh-delivery asks the Approve on Telegram, the channel that proves who answered today (ADR 0398 §3;
|
||||
an Approve at the terminal is issue 953), once its one check (`mesh/check`) passed, for a delivery group once for all its
|
||||
members, naming the review record. On the Approve it merges the approved heads, a group's in one pass
|
||||
(ADR 0418). A new head ends the Approve and is asked again once its checks pass, which is why a head is pushed only after its review. A watcher follows the checks,
|
||||
the Approve and the merge, and wakes the lead. Beyond that
|
||||
Approve, the operator's word is waited for only when the milestone changes what the operator sees, removes
|
||||
data, or needs a step only the operator's terminal can take; the lead says which before the milestone starts.
|
||||
**A pull request that depends on an earlier one outside its delivery group is opened only once the earlier one is delivered**, its build
|
||||
running on every machine it should reach, not merely merged. A note in the description is no safeguard: the
|
||||
operator's Approve merges whatever is asked. On 2026-10-25 a second change merged before the first was delivered
|
||||
and machines lost modules. How it is checked: `mesh-delivery.deliveries` shows the earlier change's source
|
||||
revision `delivered` before the later pull request exists (`gitea.gitea_get_pull_request`, its creation time);
|
||||
a later one older than that is this rule skipped. Members of one delivery group are the exception: they open together, with one check, one Approve and one merge in one pass (item 1).
|
||||
**A reviewed follow-up to a change the operator has not yet approved is folded into that pull request** (its head
|
||||
moved to include it): one Approve, one delivery. The old Approve ask closes itself on the new head and the new
|
||||
head is asked once its checks pass. How it is checked: a follow-up that sits in a second pull request while the
|
||||
first still waits shows in `gitea.gitea_list_pull_requests` as two open pull requests of one lead.
|
||||
|
||||
**One exception: a catalogue change that uses a new manifest word.** The catalogue's merge check reads
|
||||
manifests with the controller the mesh runs, so the controller's pull request is merged and delivered first;
|
||||
then `mesh-delivery.recheck` the catalogue's and merge it on the new verdict. Such work is two releases, the controller's first, and playbook 04 cuts its milestones that way.
|
||||
**A new key the node-engine checks is such a word too** (issue 1051): the controller judges every machine with the
|
||||
node-engine's own validator, vendored at the commit `go.mod`'s `replace` pins, so a key added to the node-engine is
|
||||
refused by the gate until the controller's pin moves to the delivered node-engine (`go mod vendor`); the feature is
|
||||
then three releases, the node-engine's, the controller's, the catalogue's. **Every release that must merge before
|
||||
the catalogue's is on a branch name of its own** (`feat/<slug>-node-engine`, `feat/<slug>-controller`): on the
|
||||
feature's name it joins the catalogue's delivery group, and a group merges only whole, so neither merges.
|
||||
The exception ends when milestone 3 of design 47 is delivered (issue 977, ADR 0402, ADR 0403): the group's composed
|
||||
check, judged by the group's controller, is then each member's gate, and the feature is one group.
|
||||
|
||||
## Step 4 — Deliver
|
||||
|
||||
The merge moves the source revision; there is no delivery plan (design 47, ADR 0539). The indexer writes each
|
||||
module's new fingerprint, the builder builds it once, the controller writes its compositions, credential and grant,
|
||||
and the assigner writes each node's want when its seven rules hold: the module's canary first, its other nodes once
|
||||
it runs healthy there. A merge into a branch other than `main` is not a delivery: no machine receives it.
|
||||
|
||||
**The view follows a delivery; nobody asks the controller.** The lead's watcher waits on the status view's events:
|
||||
`moving` (a module's want on a node differs from its have), `settled` (the have reached the want, running and
|
||||
healthy), `stuck` (held, a failed health check, or past the watchman's thirty minutes) and `delivered` (every want of
|
||||
the source revision equals its fingerprint and every have its want). After each wake it reads the state again from
|
||||
the view's wants page or its tools: `assigner_wants` and `assigner_want` (a want not yet at the newest fingerprint
|
||||
names, in `newest` and `rule`, the rule that holds it back, and its words say why), and `node_haves` (the node
|
||||
agent's haves: fingerprint, composition digest, phase and health). The delivery is done when the view says
|
||||
`delivered` for its source revision; then the lead resolves the feature's issues with `fixed-by` naming its pull
|
||||
requests, and removes what is left: no `feat/<slug>` branch and no `.work/<slug>` worktree survive.
|
||||
|
||||
**What a merge moves is read before the operator is told.** A change to mesh-sdk rebuilds every module that takes
|
||||
the SDK, with their backups and their asks for a person (issue 721): on 2026-10-20 the lead said an SDK merge moved
|
||||
nothing, and its delivery moved mesh-sdk with 41 dependents to all four machines. The lead reads the indexer's
|
||||
candidates of the group (the view's `indexer_candidates`: each module whose fingerprint the change moves) and the
|
||||
nodes that run them, and names what it moves before asking an Approve.
|
||||
|
||||
**A held move is the operator's answer, asked by the assigner.** A module of policy `record` whose move would
|
||||
recreate a container or move data, a failed backup before an apply, and a data module whose new fingerprint failed
|
||||
its health check on the canary each wait for the operator's answer to the assigner's ask; the view says them
|
||||
`stuck`, and a condition names the node and the module. The lead reads the ask on the router
|
||||
(`operator-channel.asks`) before telling the operator it waits, and adds it to the wait list. No session writes a
|
||||
want to get past a hold.
|
||||
|
||||
**A new module's first assignment is its first real composition.** The composition check composes every machine as
|
||||
it is assigned, so a module that no machine runs yet is composed by no check: on 2026-10-21 the build modules' first
|
||||
assignment found two faults no merge check could see (issue 1537, a template the controller filled for a value the
|
||||
forge serves only later; issue 1542, the node-engine refusing the module's own `.ssh`). So a new module is assigned
|
||||
to one machine first, its have read after the node agent applies it (`node_haves`: phase and health) and its node's
|
||||
answer read for a failed resource (`mesh-controller.node`), and taken back at once when that machine cannot compose
|
||||
or apply, so the machine keeps applying everything else; the other machines follow once it runs there.
|
||||
|
||||
## Deliver the controller, the builder or a build module without the builder
|
||||
|
||||
The builder delivers its own module, the build modules and the controller from its keys, and nothing falls back to another builder
|
||||
(ADR 0524 §7). When the builder or a build module cannot write references, or a controller cannot take its own fix (issue 1886: the
|
||||
controller's plan could never finish, and no plan it made could deliver its own fix), the way is ADR 0526's, in this order, each
|
||||
step recorded in the hand-act log with its reason:
|
||||
|
||||
1. **Revert, with no build.** Revert the change that broke the builder or the build module: the earlier source's fingerprint has a done key naming
|
||||
references, so the revert's delivery builds nothing. Read first that the artifact store still holds the earlier artifact
|
||||
(`mesh-controller.artifacts`): it keeps a recorded build while it is among its module's five newest or a definition names it, and the
|
||||
controller keeps its composition while it is among the five newest too, so the assigner's want delivers it.
|
||||
2. **A build by a session's call.** Call the build module's fetch and build tools on one named build node (for Go, `build_go_fetch` and then
|
||||
`build_go_build` with `publish: true`); the build module pushes the artifact and answers its reference. `mesh-pack` in mesh-sdk gives a
|
||||
hand build the build module's form; write the Go version of each hand build down, since another version gives another digest.
|
||||
3. **Register by reference, at the controller's terminal only.** The operator names the module, its repository, its path and each artifact's
|
||||
reference there; no verb reaches it. Say before you ask that this is a terminal step, and which notifications it raises; check that the
|
||||
command exists on the node before you hand it over.
|
||||
4. **A controller that cannot take its own fix** is put back by keys the operator writes at the terminal, with the operator's administrator
|
||||
account (the operator's credential), never by a declaration sent by hand (ADR 0539 §10). To an earlier controller: the operator writes
|
||||
`want.<control-node>.mesh-controller` naming a fingerprint whose composition is kept and that composition's digest; a controller
|
||||
composes the one it replaced at its start, so that one is always kept. To a hand build never composed: register it as in step 3, then
|
||||
`mesh-hand-compose` (in mesh-sdk, beside `mesh-pack`) composes the module for the control-node from the controller's kept inputs, signs it
|
||||
with the operator's key, and with `-write` and the operator's credential writes the composition into `composed-<node>` and its digest
|
||||
into `mesh-controller_composed`; the operator then writes the want naming that digest. Read the composition `mesh-hand-compose` prints
|
||||
before `-write`. Each act goes into the hand-act log with its reason; say before you ask that these are terminal steps, and which
|
||||
notifications they raise.
|
||||
5. **Before a switch that moves the controller, the builder or a build module,** the lead keeps the running versions' artifacts outside the
|
||||
artifact store, by digest on two nodes, with module, artifact, commit, Go version and the steps that put each back; the store's nightly
|
||||
collection removes a blob the controller did not record.
|
||||
|
||||
Nothing here is a work-around for a delivery that can run normally: while the builder delivers, no hand act replaces it.
|
||||
|
||||
## Models
|
||||
|
||||
| Who | Model | Why |
|
||||
|---|---|---|
|
||||
| The lead (step 0 decisions, the context files, following and reporting) | larger | judgement |
|
||||
| Read-only questions about the code for the context files | smaller | reading, answered with file:line |
|
||||
| Fixing agents (step 1, and the fixing in steps 2 and 3) | smaller | building to a written context |
|
||||
| The reviewer (step 2, the extra cycle in step 3 and the review record; one agent, continued by its id) | larger | judgement, once per round |
|
||||
| Decision records | larger | judgement |
|
||||
|
||||
Every agent the mesh runs draws on one weekly allowance shared by every session, and the larger model
|
||||
spends it many times faster. Up to five agents on the smaller model at once, at most two subagents on the
|
||||
larger one (the lead not counted).
|
||||
|
||||
## Following and reporting
|
||||
|
||||
Most lost time is waiting nobody saw. The lead therefore keeps a **wait list**: everything the feature waits
|
||||
on, with when it started and its bound.
|
||||
|
||||
| Waiting on | Bound | Past the bound |
|
||||
|---|---|---|
|
||||
| a fixing agent's issue | 30 minutes | ask it where it stands; split or take over |
|
||||
| the review | 15 minutes a round | ask the reviewer; stop at round 3 |
|
||||
| a merge check | 15 minutes | read `mesh-delivery.show` (its state and verdict), then the check's log; the check stream's backlog and the age of its oldest ask are in the delivery view (ADR 0525 §9) |
|
||||
| the Approve's ask | 5 minutes after the check (`mesh/check`) passes | read `mesh-delivery.show`: an ask not raised is a fault, said to the operator |
|
||||
| a merge after the Approve | 5 minutes | read `mesh-delivery.show` for the delivery |
|
||||
| a delivery | its class's budget (3 minutes for a leaf module, 10 for a core module), at most 30 minutes | read the view's wants page, or `assigner_wants` against `node_haves`: a want held back names its rule, a `stuck` module its words |
|
||||
|
||||
**Every wait on the list has a watcher** (the operator, 2026-10-17). A session runs only when a message reaches
|
||||
it, so a check, a merge, a delivery, a record's review or the operator's Approve on a channel that lands
|
||||
while the lead's turn has ended wakes nobody: on 2026-10-17 an approved group sat half merged for 14 minutes
|
||||
unseen (issue 1032). For each wait the lead starts a watcher, an agent on the smaller model that changes nothing,
|
||||
never the smallest (the operator, 2026-10-17).
|
||||
It loops `mesh_wait` on the event families of what it watches (`mesh-delivery-gate.transition` and
|
||||
`mesh-delivery-gate.group` for a merge, `mesh-delivery-view.moving`, `.settled`, `.stuck` and `.delivered` for its
|
||||
delivery to the nodes, for example; the families a watcher may wait on are those the mesh granted
|
||||
the tool runner, and a refusal names them) with a timeout under 60 seconds, since the call itself ends near
|
||||
a minute. After every answer or timeout it reads the state from the verb that owns it (`mesh-delivery.show`,
|
||||
the view's `assigner_wants` and `node_haves`, `knowledge-base.get`), because an event only wakes it (ADR 0367). It finishes, and so wakes
|
||||
the lead, when the state changes or the wait passes its bound. The lead never sleeps or polls by hand to wait.
|
||||
**A watcher watches one wait and ends with it.** It is never re-pointed at the next pull request or delivery: on
|
||||
2026-10-19 one watcher sent on to three pull requests in turn missed that the third's checks had passed at 04:58,
|
||||
and the lead told the operator for 15 minutes that they were still running. The next wait gets a fresh watcher,
|
||||
and before a report the watcher reads the owning verb once more and says only what it says now.
|
||||
**An ask is said to exist only after it is read on the router.** Before the lead tells the operator that an
|
||||
Approve, a Retry or a Release waits for them, it reads `operator-channel.asks` and finds that ask open by its key;
|
||||
on 2026-10-20 an ask was named to the operator that the router did not hold. A merge the gate passed but whose
|
||||
ask is not on the router is the fault of the Approve's wait in the table above.
|
||||
|
||||
**Every ten minutes (the operator, 2026-10-17), every two minutes while a delivery moves (the view says modules `moving`), and at once when something lands, stops or
|
||||
needs the operator,** the lead reports, in one of the two layouts below (the operator, 2026-10-16: an overview
|
||||
of the issues, often). Times are local and absolute, from the clock (`date`), never guessed; every number has its
|
||||
unit and its bound or baseline; an issue is named by its number and a few words, never by its number alone.
|
||||
|
||||
**The full layout** (in the session) is the whole design as a tree (the operator, 2026-10-17: always the full
|
||||
overview): every milestone of the design with its state, and under the running milestone its repositories and
|
||||
their issues, each issue under the one it waits for, marked ✓ landed or delivered, ▶ fixing, ⏸ waiting, ✗ stopped:
|
||||
|
||||
```
|
||||
Feature <slug> · milestone <n> of <m> · step <name>, <minutes used> of <bound> min
|
||||
Issues: <landed> of <total> landed · <fixing> fixing · <waiting> waiting
|
||||
|
||||
Design <nn> · <its title>
|
||||
├── ✓ milestone 1 <its outcome, a few words> delivered 12:58
|
||||
├── ▶ milestone 2 <its outcome, a few words>
|
||||
│ ├── mesh-controller (agent: fixing, 2 of 3 landed)
|
||||
│ │ ├── ✓ 652 assignments on the bus landed 14:02
|
||||
│ │ ├── ▶ 736 placement words leave the layers fixing since 13:51, 23 of 30 min
|
||||
│ │ │ └── ⏸ 751 config keys named (mesh-catalog) waits for 736
|
||||
│ │ └── ✓ 654 a file names its template landed 13:20
|
||||
│ └── mesh-catalog (agent: waiting)
|
||||
│ └── ⏸ 751 (above, under 736)
|
||||
└── ⏸ milestone 3 <its outcome, a few words> <its issues>
|
||||
|
||||
Opened along the way: <each issue opened during the feature, its number and a few words, and when it is fixed: in between, after the delivery, or waits on what> — or "none"
|
||||
Past its bound: <the wait, how long, what is being done> — or "none"
|
||||
Needs you: <one line per question, with my recommendation> — or "nothing"
|
||||
Allowance: week <n> % (<+d> since the last report), session <n> %
|
||||
Next: <the next actions up to the delivery, in order>
|
||||
```
|
||||
|
||||
**The tree is read, not assembled.** `knowledge-base.progress` with `of: "design <nn>"` answers the tree's
|
||||
milestones, issues, pull requests and held groups in one call, each with when it last moved: `detail: minimal` for the
|
||||
five-minute report, `detailed` about every thirty minutes and whenever something stops (issue 1947). What it cannot
|
||||
read for certain from the design's milestone table it says in `warnings`; the lead fixes the table, it does not
|
||||
correct the tree by hand. How it is checked: a report's tree that disagrees with the verb's answer of the same minute
|
||||
is a tree assembled by hand.
|
||||
|
||||
**The short layout** (on a channel such as Telegram, or when nothing changed but time):
|
||||
|
||||
```
|
||||
<slug> milestone <n>/<m> · <step> <used>/<bound> min · <landed>/<total> landed · past bound: <none | what> · needs you: <nothing | what> · week <n> %
|
||||
```
|
||||
|
||||
A report says what changed since the last one first. In the session every report is the full layout, the whole
|
||||
tree of the milestone's issues included, also when only time passed: the operator reads the state from it without
|
||||
asking. The short layout is for a channel.
|
||||
|
||||
## The context of the lead's own session
|
||||
|
||||
At the end of each step, and whenever the session's context grows large or spread over unrelated work, the
|
||||
lead prepares a `/clear` as the mesh's instruction "prepare before clear" says: a save point with the
|
||||
feature's step, its branches and heads, the context files' paths, the agents' ids, the wait list and the
|
||||
next action; then the prompt the operator pastes. The context files make the feature resumable after a clear.
|
||||
|
||||
## The design's state is written back
|
||||
|
||||
Other sessions know what is built only from the design's frontmatter and the issue tracker; what the lead
|
||||
knows and does not write back, nobody else knows. A design merged in one delivery with three of its tasks
|
||||
left, its frontmatter still saying `designed`, read to another session as not built, and a second design
|
||||
was started beside it (issue 1009, 2026-10-17). So the lead writes the design's state back at each of these
|
||||
moments, the same hour, an hq change going through [playbook 08](08-small-fixes.md):
|
||||
|
||||
- **The first milestone starts:** the design's `status: in-progress` ([playbook 04](04-build-handoff.md) step 7),
|
||||
if it is not already.
|
||||
- **A milestone is delivered** (the view says `delivered` for its source revision): its issues are resolved (step 4), with `fixed-by` naming its pull
|
||||
requests. The design's milestone entry is not changed: it names the milestone's issues, and whether they are
|
||||
delivered, when and by what is read from the issue tracker when it is asked, never copied into the design
|
||||
(issue 1038; AGENTS.md: cross-cutting views are generated when they are read).
|
||||
- **The building pauses with work left** (another design takes its turn, a hand-over, the session's end): the
|
||||
design stays `in-progress`, and each issue left gets a note with `issue-tracker.note` saying what it waits
|
||||
for and what comes after it ("built after design N is delivered, before design M"). The first issue of
|
||||
a design that waits its turn gets the same note.
|
||||
- **The last milestone is delivered:** [playbook 02](02-graduation.md), "When something ships": the as-is is
|
||||
written and the design says `implemented`.
|
||||
|
||||
## Issues that stop
|
||||
|
||||
An issue stays in `fixing` only while an agent works on it. A milestone that stops, for a wait past its bound, a
|
||||
hand-over, or the session's end, moves each of its issues not delivered back to `open` with
|
||||
`issue-tracker.move` and a note saying how far it got (its branch, its last commit, what is left). A list of
|
||||
issues in `fixing` that nobody works on reads to the operator as stuck work.
|
||||
- **Not a licence to batch unbounded work.** A feature is a *bounded* unit; if it sprawls for
|
||||
days, end-of-feature bloat merely replaces per-increment bloat. Split it into features, each
|
||||
its own branch and MR.
|
||||
- **Not a second trunk.** Every repo branches off `main`. There is no longer an `initialization`
|
||||
trunk.
|
||||
|
||||
## How it is checked
|
||||
|
||||
- A feature's issues show their commits on the feature branch (`git log --grep 'issue <n>'`), and no pull
|
||||
request into a feature branch exists. A pull request per issue, or an agent waiting on a merge check in
|
||||
step 1, is the failure this playbook prevents.
|
||||
- A context file exists per touched repository under `.work/<slug>/` while the feature runs.
|
||||
- No issue of a running milestone is sized over an hour in its prepare note; an issue whose fixing agent passes
|
||||
twice its bound was not split.
|
||||
- Every issue of a running milestone has a note from the prepare step naming its files and tests (file:line), how
|
||||
it is checked and delivered, and for a group its merge order (`issue-tracker.get`); an issue moved to `fixing`
|
||||
without one is a prepare step skipped.
|
||||
- Every issue of a running milestone has a note headed `Acceptance criteria` before it moves to `fixing`
|
||||
(`issue-tracker.get`), each criterion naming its test level, and every criterion's label is cited by a test in the
|
||||
pushed tree (`git grep -n '<issue>/AC<n>'`); an issue without them is a prepare step skipped, and a
|
||||
criterion no test names is a review that missed it (until issue 1118 makes the tracker refuse it).
|
||||
- Every fixing agent's report gives the whole unit suite's result and time; a report naming only some packages'
|
||||
tests is the whole-suite rule not kept.
|
||||
- The lead's save point names each watcher with its model; a watcher on the smallest model is the failure of
|
||||
"Following and reporting".
|
||||
- A report or a save point that names a "phase" of a milestone is the old wording: a milestone has steps.
|
||||
- Every repository's own clone says `main` (`git -C <clone> branch --show-current`); another branch there is
|
||||
work done outside a worktree (issue 1010).
|
||||
- After a feature's delivery, `git branch -r | grep feat/<slug>` and `git worktree list` return nothing for it.
|
||||
- A design whose milestone issues are partly resolved says `in-progress`, and one whose issues are all
|
||||
resolved and delivered says `implemented`: read its Milestones section against `issue-tracker.get` for each
|
||||
issue. A design that says `designed` with a resolved milestone issue is a state not written back.
|
||||
- A design's Milestones section names each milestone's issues and nothing of their state: a "delivered", a date
|
||||
or a pull request in an entry is a copy of the tracker's state (issue 1038), read with
|
||||
`grep -n 'delivered' <design>` over its Milestones section.
|
||||
- Every open issue of a design that says `in-progress` and has no agent on it shows, in `issue-tracker.get`, a
|
||||
note saying what it waits for; one without such a note is a pause not written back.
|
||||
- The milestone issues in `fixing` (`issue-tracker.list`) belong to one design at most: milestone issues
|
||||
of two designs in `fixing` at once is the failure of "one design is built at a time".
|
||||
- More than one open pull request in a repository that share no feature name, or a milestone whose issues
|
||||
stay in `fixing` for more than a working day, is the smell: stop and bring it to the operator.
|
||||
- A review report that asks a change outside the issues' done-when and acceptance criteria, or a fourth round, is the failure of the
|
||||
review rules; `issue-tracker.list` with `flag: abandoned` lists issues left in `fixing` with nobody on them.
|
||||
- A pull request green for more than 5 minutes with no Approve asked is a fault of mesh-delivery
|
||||
(`mesh-delivery.show`), said to the operator.
|
||||
- Every review record of a release names one reviewer (`mesh-delivery.show`, the review record), and the lead's
|
||||
save point names one reviewer id per release; a second reviewer within one release is a reviewer not kept
|
||||
(issue 1042). A pull request whose ask says it has no review record at its head is a record not written.
|
||||
- A running milestone's file holds the three answers of step 0 item 8 (`cat .work/<slug>/milestone.md`), and the
|
||||
review report says it read them (step 2, item 6a): a bucket key or stream in the diff (`git diff main --
|
||||
| grep -i 'bucket\|stream'`) that the lifetime answer omits is the failure of that item (research 071).
|
||||
- A pull request merged by hand in a temporary mode names, in its description on the forge (`gitea.gitea_get_pull_request`), the
|
||||
verdict of its one check; a try with tests `.` is named only where the operator asked for one.
|
||||
- **One release, one check** (ADR 0548): `gitea.gitea_get_pull_request` shows each pull request created after its
|
||||
review record's time, and `mesh-delivery.show` shows one check per head and, for a group, one head per member; a pull
|
||||
request that existed before its review approved it, a second head with no failed check before it, or a check of a
|
||||
head asked more than once, is this rule skipped. A call of `mesh-controller.try`, `mesh-delivery-checker.try` or `checker_try` that the operator
|
||||
did not ask for shows in the checker's asks and is the same breach.
|
||||
- A milestone that adds or changes a module whose pull request has no integration test that starts the module
|
||||
whole is the failure of the check step's fourth item.
|
||||
- A bed run from `.work/<slug>/mesh-lab` that fails naming a `.work/<slug>/<repo>/…` path it cannot find is a
|
||||
missing sibling worktree, not a mesh fault.
|
||||
The end state is visible, and its absence is the smell:
|
||||
|
||||
When this playbook meets a problem, the lesson is added here, so the next feature avoids it.
|
||||
- After a feature merges, `git branch -r | grep feat/<slug>` and `git worktree list` return
|
||||
nothing for it. A surviving branch or worktree means step 6 was skipped.
|
||||
- A bed run from `.work/<slug>/mesh-lab` that fails naming a `.work/<slug>/<repo>/…` path it
|
||||
cannot find is a missing sibling worktree, not a mesh fault.
|
||||
- More than one open MR in a repo that share no feature name, or a stack of MRs none of which is
|
||||
merged, is the failure this playbook exists to prevent — stop and consolidate before opening
|
||||
more.
|
||||
|
||||
@@ -1,154 +0,0 @@
|
||||
# Playbook 08 — Small fixes
|
||||
|
||||
**Trigger.** One issue in one repository, small enough to build in about an hour: a bug with a known cause,
|
||||
a missing check, a wording, a playbook or design edit. Anything larger, or anything that needs a change in a
|
||||
second repository first, is a feature and follows [playbook 07](07-feature-branches.md). A feature is the
|
||||
larger kind: it touches several modules and has a research effort or a design before it. **Needing a decision
|
||||
does not make a fix a feature**: the decision is settled first (step 1) and the fix stays small (the operator,
|
||||
2026-10-16).
|
||||
|
||||
**Related issues in one fix.** A small fix may cover two or three related issues when they are in the same
|
||||
repository, mostly in the same module, and together still fit in about an hour: they touch the same files, or
|
||||
one would be wrong or unsafe without the other (switching a clipboard's history on before its history tool stops
|
||||
handing out what was copied). Each issue is named in the commit and the pull request and moved to `fixing`,
|
||||
and each is resolved on its own once the fix is delivered. More issues, a second repository, or more than an
|
||||
hour is a feature.
|
||||
|
||||
**Who runs it.** Anyone who writes code, engineers and agents alike.
|
||||
|
||||
**A fix found during a milestone rides in that milestone's group** (the operator, 2026-10-21, after a day with ten
|
||||
mesh-sdk merges, each rebuilding about forty dependent modules, and eight mesh-controller merges, each sending a
|
||||
delivery to every node). Playbook [07](07-feature-branches.md) makes a milestone one piece of functionality
|
||||
released once as one delivery group (ADR 0482). A small fix in a repository the running milestone touches, mesh-sdk
|
||||
and mesh-controller above all, is built as below but on the milestone's branch, so its pull request joins the group
|
||||
and merges with it. The exception is a fault that is broken for the operator now (something they see failing at
|
||||
this moment): it goes alone as a small fix, and the lead's report says why.
|
||||
|
||||
## The aim
|
||||
|
||||
A small fix spends its time on the fix, and runs the long part once: **one review, one push, one check run,
|
||||
one delivery** (ADR 0548; the operator, 2026-10-26: "every release should only run 1 check or 1 try"; "we don't have more
|
||||
processing power"). Everything before the push is local and fast, and no try is run.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Read the issue and the code it names** (the session's own model, or one read-only agent on the smaller
|
||||
model answering exact questions with file:line). If a design question is open, it is settled before any
|
||||
code is written: on the issue, through the knowledge base (`knowledge-base.ask` first: what is decided is read before anything is designed), or by the operator. A fix that turns out to
|
||||
need a second repository, or more than about an hour, stops here and becomes a feature; one that needs a
|
||||
decision stays a small fix. **The session's own model (the larger) then writes the issue's acceptance
|
||||
criteria** as playbook [07](07-feature-branches.md) step 0 item 5 says: each one sentence, given, when, then,
|
||||
naming the level of the test that proves it, as an `issue-tracker.note` headed `Acceptance criteria`. A
|
||||
playbook or design edit has none: its check is `go test ./...` at hq's root, which runs the four checks (ADR 0527).
|
||||
2. **Move the issue to `fixing`** with `by` as who@node.
|
||||
3. **Build on a local branch** `fix/<issue>-<slug>`, in a worktree of its own branched from `main`, never in
|
||||
the repository's own clone, which stays on `main` (playbook [07](07-feature-branches.md), step 0, item 2). Write the
|
||||
test that fails today first, on an assertion about what the code does: a test that fails only because it
|
||||
does not compile has not failed first (playbook [07](07-feature-branches.md) step 1); make the change; keep `gofmt`, `go vet` and the repository's whole unit suite green
|
||||
before every commit (each test well under a second; never a merge check or a database locally: how-we-build §5, ADR 0391).
|
||||
Check that it compiles with `go vet`, never `go build` in a module's folder: with one main package it writes the
|
||||
program beside `go.mod`, and `git add -A` takes it (issue 1577: two 5-10 MB programs committed on 2026-10-21,
|
||||
caught only by the review); read `git show --stat` before pushing.
|
||||
The tests are written from the acceptance criteria, at least one per criterion at its level, each citing the
|
||||
criterion it proves by its label (`<issue>/AC<n>`, playbook 07 step 0 item 5). Commit, naming the issue. For a playbook or design edit, run the repository's own checks (in hq,
|
||||
`go test ./...` at the root). The integration tests are compiled (`go vet -tags integration ./...`) and not run:
|
||||
nothing is pushed, no pull request is opened and no try (`mesh-controller.try`, `mesh-delivery-checker.try`, `mesh-delivery-checker.checker_try`) is called, unless the operator asks for
|
||||
one. A single integration test may be run locally against an embedded bus when it is the fastest way to find a cause.
|
||||
4. **One review of the local commit** (a reviewer that did not write it, larger
|
||||
model; at most 3 rounds, as playbook 07; from round 2 only what is wrong; it reads each acceptance criterion
|
||||
against its test, as playbook 07 step 2 item 6; a finding outside the issue's done-when and its acceptance
|
||||
criteria is opened as an issue of its own, never asked as a change; it judges the integration tests by reading them against the code, as playbook 07 step 2 item 4 says). The reviewer's findings go back to the same agent that wrote the fix, continued by
|
||||
its id, so the fix keeps its context. A playbook or design edit of a few lines may skip the reviewer: the
|
||||
operator's Approve of its merge is its review, and the review record says so.
|
||||
5. **Push the reviewed head once and open the pull request, only after the review approves.** The trunk is what runs and nothing holds merged work back
|
||||
(ADR 0401), so a fix waits for no other change's delivery. No pull request, draft or not, exists before the review
|
||||
approves the head, and no fix round is pushed to an open pull request: every push to an open pull request asks a new
|
||||
check. The checker's merge check runs the unit and the integration level once, as the gate (playbook [07](07-feature-branches.md) step 3); the pushed head is read from the forge first, as playbook 07 step 3 item 1 says.
|
||||
A failed check is read from its whole log (never re-asked to see if it passes), then the repository's whole integration level
|
||||
is run locally (`go test -tags integration -count=1 ./...` in each module) so every failure shows at once, where a verdict
|
||||
names only the first (issue 1921); the failures go back to the same agent, fixed in one change, the reviewer reads only the change
|
||||
it makes, and the head is pushed once more.
|
||||
**A temporary mode that merges by hand** (the lead merges because the normal Approve and merge path is not
|
||||
running): the one check still runs, and the lead names its verdict in the pull request's description; a try with tests `.` is asked only when the operator asks for one (research 071; playbook
|
||||
[07](07-feature-branches.md) step 3).
|
||||
6. **The pull request's description carries the review record** (`Reviewed-by: <who@node> at <full sha>`,
|
||||
playbook [07](07-feature-branches.md) step 3) and the session that pushed it (`Pushed-by: <who@node>`, ADR 0505 §5), and mesh-delivery asks the operator's Approve as soon as its
|
||||
checks pass and merges the approved head (ADR 0421); nobody marks, labels, comments or rechecks. A fix to a module that changes how it runs carries an
|
||||
integration test starting the module whole against the real bus and store, run by the checker's one merge check
|
||||
(ADR 0391 §1, §6); unit tests alone are not that proof, and the lab is not used. The operator's word is waited for only when the fix changes what the operator sees, removes data,
|
||||
or needs the operator's terminal.
|
||||
**A fix that depends on another pull request outside its delivery group opens only once that one is delivered**, not merely merged (members of one group open together, as playbook 07 step 3 item 1 says) (a note
|
||||
in the description does not hold it back: the Approve merges whatever is asked; 2026-10-25, machines lost modules
|
||||
when a second change merged first). **A reviewed follow-up to a fix still waiting for the Approve is folded into
|
||||
that pull request** (head moved): one Approve, one delivery. Both are checked as playbook
|
||||
[07](07-feature-branches.md) step 3 says.
|
||||
**At the second instance of a fault kind found in a review, the developer searches for every instance** and names
|
||||
each in the report (playbook [07](07-feature-branches.md) step 2, item 9).
|
||||
7. **Follow its delivery to done**: a watcher waits on the status view's events (`mesh-delivery-view.moving`,
|
||||
`.settled`, `.stuck` and `.delivered`) and reads the view's `assigner_wants` and `node_haves` after each wake, as
|
||||
playbook [07](07-feature-branches.md) step 4 says; the delivery is done when the view says `delivered` for its
|
||||
source revision. Then prove the fix on the mesh (the symptom gone, measured), and resolve the issue with
|
||||
`located-in` and `fixed-by` the same hour.
|
||||
|
||||
A fix that stops before its delivery moves its issue back to `open` with a note saying how far it got
|
||||
(branch, last commit, what is left), as playbook [07](07-feature-branches.md) says.
|
||||
|
||||
**An issue opened along the way is fixed too** (the operator, 2026-10-21). A fix's review, check or delivery that
|
||||
opens an issue (a remark outside the done-when, a fault met live) leaves it to the same session to fix: right after
|
||||
this fix's delivery, as a small fix of its own, or folded into the next one in the same repository. It waits only
|
||||
when another running milestone owns it, or it needs a decision or the operator first; the report then says on what.
|
||||
|
||||
**A second fault found by the delivery.** When a fix's delivery shows a second fault that keeps the issue's
|
||||
done-when from holding, and that fault is in the same files, it is taken into the same issue as a second pull
|
||||
request with a review round of its own. A fault in other code is opened as an issue of its own, and the first
|
||||
issue waits for it (issue 937: mesh-catalog #335 was taken in; issue 943 was opened).
|
||||
|
||||
**A fix that moves a file to a new filler compares the values first.** When a fix makes a module's file filled a
|
||||
new way for the first time (the provisioner client in place of the controller, a provider's entry in place of a
|
||||
composed value), the fix lists every value the old way put into that file on each machine it reaches and shows,
|
||||
in its review, that the new way gives the same values, or says which differ and why. A difference nobody looked
|
||||
at is an outage: on 2026-10-16 the forge and the identity provider on the control node were down for 17 minutes
|
||||
because postgres's consumers were served the port inside its container instead of the port it is published on
|
||||
(issue 957). The comparison is read from the machines (the file as it is today) before the merge, not from the
|
||||
code.
|
||||
|
||||
## Bounds
|
||||
|
||||
| Waiting on | Bound | Past the bound |
|
||||
|---|---|---|
|
||||
| building the fix | 1 hour | it is a feature: stop and plan it under playbook 07 |
|
||||
| the review | 15 minutes | ask the reviewer where it stands |
|
||||
| the merge check | 30 minutes (it waits behind the asks before it for a build node's free slot, ADR 0525) | read `mesh-delivery.show`, the check's log and the check stream's backlog in the delivery view |
|
||||
| the Approve's ask | 5 minutes after the check (`mesh/check`) passes | read `mesh-delivery.show`: an ask not raised is a fault, said to the operator |
|
||||
| the merge after the Approve | 5 minutes | read `mesh-delivery.show` for the delivery |
|
||||
| the delivery | its class's budget (3 minutes for a leaf module, 10 for a core module), at most 30 minutes | read the view's wants page, or `assigner_wants` against `node_haves`: a want held back names its rule, a `stuck` module its words |
|
||||
|
||||
A wait past its bound is said to the operator at once, with what is being done about it.
|
||||
|
||||
## Models
|
||||
|
||||
The fixing agent runs on the smaller model; the reviewer on the larger. A session fixing alone does the
|
||||
reading itself and uses one subagent only for the review.
|
||||
|
||||
## How it is checked
|
||||
|
||||
- A small fix has one pull request with one or few commits naming its issue, opened after its review: a pull
|
||||
request with many heads is a fix reviewed after its pull request was opened.
|
||||
- A pull request merged by hand in a temporary mode names the verdict of its one check in its description
|
||||
on the forge (`gitea.gitea_get_pull_request`); one without is the step skipped.
|
||||
- One fix, one check (ADR 0548): a pull request that existed before its review approved the head, or a head whose check was
|
||||
asked more than once with no new reviewed head, is steps 4 and 5 skipped; so is a `mesh-controller.try` or `mesh-delivery-checker.try` the operator did not ask for.
|
||||
- Its issue moves to `resolved` within the hour the view says its source revision `delivered`.
|
||||
- A pull request with no review record at its head (the Approve's ask says so first), a fix to a module's
|
||||
running pushed with no integration test of the module whole, or a review report
|
||||
asking a change outside the issue's done-when and acceptance criteria, is the failure of steps 4 to 6.
|
||||
- A fixed code issue shows a note headed `Acceptance criteria` (`issue-tracker.get`) older than its first
|
||||
commit, and each criterion's label is cited by a test in the pushed tree
|
||||
(`git grep -n '<issue>/AC<n>'`); one without is step 1 skipped.
|
||||
- An issue in `fixing` with nobody on it shows in `issue-tracker.list` with `flag: abandoned`; a stopped fix
|
||||
moves it back to `open` with its note.
|
||||
- A fix found during a milestone is named in the lead's report with the group it rides in, or with the reason it
|
||||
went alone. `mesh-delivery.deliveries` shows the milestone's members in one group, merged as one source revision
|
||||
per milestone; a merge of mesh-sdk or mesh-controller outside the group, with no reported reason, is this rule skipped.
|
||||
- The repository's own clone says `main` (`git -C <clone> branch --show-current`); another branch there is a
|
||||
fix built outside a worktree (issue 1010).
|
||||
@@ -1,49 +0,0 @@
|
||||
# Playbook 09 — Cleanup
|
||||
|
||||
**Trigger.** The operator asks for a cleanup, or the forge and the working copies have gathered what nobody
|
||||
uses: merged branches, stale pull requests, worktrees and folders of finished work.
|
||||
|
||||
**Who runs it.** A session on the operator's word, with the operator deciding every deletion. Agents read and
|
||||
propose; a deletion is the operator's act or is done on the operator's word for the list it was shown.
|
||||
|
||||
## The aim
|
||||
|
||||
Remove what is provably done, and nothing else. A cleanup that deletes one piece of unfinished work costs more
|
||||
than every stale branch it removed. So every item is shown with its evidence, and anything not provably safe is
|
||||
reported and left.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Take stock, reading only.** One agent on the smaller model reads, changing nothing:
|
||||
- the forge: every repository, every branch **as git refs** (the branch list tool misses some, issue 949),
|
||||
every open pull request with its head, title and last update;
|
||||
- each working copy on the machine: its worktrees, local branches, stashes, and the folders beside the
|
||||
clones, with size and newest file.
|
||||
It writes one report with each item, its class and its evidence, and a list of the tools it lacked.
|
||||
2. **Classify each item**, by evidence, never by name or age alone:
|
||||
- **Safe** — a forge branch whose tip is contained in its repository's default branch, with no open pull
|
||||
request, not protected and not a long-lived branch the mesh treats as a trunk; a worktree that is clean,
|
||||
whose branch is contained in the remote default branch, untouched for two hours and not in another
|
||||
session's working folder; a local branch contained in the remote default branch.
|
||||
- **Likely safe, not proven** — a branch whose every commit is patch-equal to one on the default branch
|
||||
(the forge lands some commits under new hashes): listed, deleted only on the operator's word for it.
|
||||
- **Kept** — unmerged commits, a dirty worktree, a stash, an open pull request, a protected or trunk-like
|
||||
branch, a repository of another owner, another session's worktree.
|
||||
- **Unknown** — a folder or file outside git: described, never opened where a secret may be, left.
|
||||
Stale open pull requests (no update for three days) are reported with their author's issue, never closed.
|
||||
3. **Ask the operator** with the counts per class and the report's place. One question: delete the safe
|
||||
items as listed? The likely-safe and unknown items are separate questions, item by item when few.
|
||||
4. **Delete only the approved list**, each with the command that refuses an unsafe case (a worktree removal
|
||||
that refuses a dirty one, a branch deletion that refuses an unmerged one). A refusal is reported, never
|
||||
forced. The operator approves a deletion on a channel that proves who answered, as an ask bound to that
|
||||
exact list and carried out on the warrant (ADR 0387, design 58 milestone 7, issue 884). Until that is built,
|
||||
an agent session's permission check refuses these deletions, and the approved list is handed to the
|
||||
operator to run.
|
||||
5. **Report**: what was cleaned, what was kept and why, what was unknown, what the operator kept, and the
|
||||
missing tools, each opened as an issue (`issue-tracker.open`) or added to an existing one.
|
||||
|
||||
## How it is checked
|
||||
|
||||
- The report names, for every deleted item, the evidence it was safe (contained, clean, no pull request).
|
||||
- Nothing classed Kept or Unknown is deleted without the operator's word for that item.
|
||||
- A cleanup's missing tools are issues, not notes in a session.
|
||||
@@ -1,66 +0,0 @@
|
||||
# Playbook 10 — Visual design
|
||||
|
||||
**Trigger.** A page or a screen the operator will use is about to be designed or rebuilt, or the operator judges
|
||||
one unusable. It runs before the decisions about its look are written and before any of its code: a page built
|
||||
first and judged after it ships is rebuilt (the operator, 2026-10-22, on design 59's mail page, judged unusable
|
||||
the day it was delivered).
|
||||
|
||||
**Who runs it.** The lead of the design, on the larger model. Mockups may be drawn by an agent on the smaller
|
||||
model from the lead's brief. The operator picks; nobody else does.
|
||||
|
||||
## What it produces
|
||||
|
||||
- **The reference**: one mockup the operator picked, kept with the design as `NN-mockup/` beside its to-be
|
||||
document. It is a direction, not a pixel specification: the build matches what it shows the operator doing,
|
||||
in the order and the weight it shows, not its exact pixels.
|
||||
- **The decisions it needs**: what the pick changes about what the mesh keeps, shows or does, written as decision
|
||||
records (playbook [02](02-graduation.md)) before the design's milestones (playbook [04](04-build-handoff.md)).
|
||||
- **An acceptance test in the operator's words**: the page is done when the operator uses it, on the devices they
|
||||
named, for the work the mockup shows.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Gather what the page must carry.** A page whose direction needs research first starts at playbook
|
||||
[01](01-research.md); this playbook then draws it. Read the as-is of what it replaces and the real shape of its data from the
|
||||
mesh (how many items, how long a line, what is missing in practice), the operator's complaint in their own
|
||||
words, and a short survey of how others solve it (a handful of products, one line each on what changes for
|
||||
the person). Write it into the work folder; nothing goes on the mesh.
|
||||
2. **Draw three to five mockups with fixed data.** Each is one static HTML file that opens from disk: invented
|
||||
data shaped like the real (names, amounts, lengths, a long subject, an empty sender), no backend, a mock script
|
||||
only where an interaction is the point. Each mockup is a different direction, not a variation of one; each
|
||||
shows a phone's width and a desktop's. One index page links them all.
|
||||
3. **Show them and ask one question.** Put the index on the machine the operator is at, and ask which direction to
|
||||
keep, one question at a time with the lead's recommendation. A round's answer may be a pick, a combination of
|
||||
two, or a change; the next round draws only that. Three rounds without a pick mean something the operator wants
|
||||
is missing: ask what, before drawing again.
|
||||
4. **Make the pick the reference.** Copy it into the design's `NN-mockup/`, with every name, address, employer,
|
||||
mail address, domain, provider and amount of the operator's own replaced by invented ones: hq is public. Say in the design that
|
||||
the reference is a direction, and link it.
|
||||
5. **Write what is new as decisions,** each with the operator's Approve where it changes what they see, then the
|
||||
design and its milestones. The design's acceptance tests end with the operator using the page live.
|
||||
6. **Hold the build to it.** The lead gives playbook [07](07-feature-branches.md)'s reviewer the reference with
|
||||
the built page, at a phone's and a desktop's width, as part of what the review reads against the issue. The
|
||||
operator using the page is the milestone's live proof, made before the next milestone starts (playbook
|
||||
[04](04-build-handoff.md)).
|
||||
|
||||
## Bounds
|
||||
|
||||
| Waiting on | Bound | Past the bound |
|
||||
|---|---|---|
|
||||
| a round of mockups | 1 hour | say to the operator what is taking long |
|
||||
| the operator's pick | the next exchange | ask once more, one question, with the recommendation |
|
||||
| rounds without a pick | 3 | ask what is missing before drawing again |
|
||||
|
||||
## Rules
|
||||
|
||||
- No code reaches the mesh during a visual design: mockups live in the work folder until one is picked.
|
||||
- Mockups use invented data, never a copy of the operator's mail, files or accounts.
|
||||
- The operator's pick stands over the lead's taste; the lead's recommendation is said once, with its reason.
|
||||
|
||||
## How it is checked
|
||||
|
||||
- A to-be design of a page the operator uses links its reference mockup in `NN-mockup/`, and that mockup holds
|
||||
none of the operator's own names, addresses, domains, providers or amounts (read by the reviewer of the design's pull request).
|
||||
- The milestone's review read the built page beside the reference at a phone's and a desktop's width (its
|
||||
report says so).
|
||||
- The design's last acceptance test is the operator using the page, proven before the next milestone starts.
|
||||
@@ -1,521 +0,0 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>Novox Playbooks</title>
|
||||
<style>
|
||||
:root{
|
||||
--bg:#f7f6f2; --card:#ffffff; --ink:#1d1f23; --mute:#5d6470; --line:#d9d6cc; --soft:#efede6;
|
||||
--idea:#2f5fa7; --sym:#b4462b; --dec:#7a4bb0; --build:#3c8a5a; --mesh:#0f6f7a; --op:#c2410c; --side:#8a6d1f;
|
||||
--ok:#15803d; --okbg:#e6f4ea; --warn:#a16207; --warnbg:#fdf3d7; --bad:#b91c1c; --badbg:#fde8e8;
|
||||
}
|
||||
@media (prefers-color-scheme: dark){
|
||||
:root:not([data-theme="light"]){
|
||||
--bg:#15171b; --card:#1e2127; --ink:#e8e6e1; --mute:#9aa1ad; --line:#353a43; --soft:#262a31;
|
||||
--idea:#7aa7ef; --sym:#f08a6c; --dec:#b993ef; --build:#6fcf97; --mesh:#4fc3cf; --op:#fb923c; --side:#d9b44a;
|
||||
--ok:#4ade80; --okbg:#173222; --warn:#facc15; --warnbg:#3a3016; --bad:#f87171; --badbg:#3a1d1d;
|
||||
}
|
||||
}
|
||||
:root[data-theme="dark"]{
|
||||
--bg:#15171b; --card:#1e2127; --ink:#e8e6e1; --mute:#9aa1ad; --line:#353a43; --soft:#262a31;
|
||||
--idea:#7aa7ef; --sym:#f08a6c; --dec:#b993ef; --build:#6fcf97; --mesh:#4fc3cf; --op:#fb923c; --side:#d9b44a;
|
||||
--ok:#4ade80; --okbg:#173222; --warn:#facc15; --warnbg:#3a3016; --bad:#f87171; --badbg:#3a1d1d;
|
||||
}
|
||||
*{box-sizing:border-box}
|
||||
body{margin:0;background:var(--bg);color:var(--ink);font:15px/1.55 system-ui,-apple-system,"Segoe UI",sans-serif}
|
||||
main{max-width:1120px;margin:0 auto;padding:28px 16px 72px}
|
||||
h1{font-size:28px;margin:0 0 4px}
|
||||
h2{font-size:20px;margin:44px 0 6px}
|
||||
h3{font-size:16px;margin:0 0 6px}
|
||||
p{margin:6px 0 10px}
|
||||
.sub{color:var(--mute);margin:0 0 18px}
|
||||
.lead{max-width:760px}
|
||||
nav{display:flex;flex-wrap:wrap;gap:8px;margin:14px 0 6px}
|
||||
nav a,.chip{display:inline-block;padding:4px 11px;border:1px solid var(--line);border-radius:999px;color:var(--ink);text-decoration:none;font-size:13px;background:var(--card);cursor:pointer}
|
||||
nav a:hover,.chip:hover{border-color:var(--mute)}
|
||||
.chip.on{background:var(--ink);color:var(--bg);border-color:var(--ink)}
|
||||
.box{background:var(--card);border:1px solid var(--line);border-radius:12px;padding:16px;margin:12px 0}
|
||||
.scroll{overflow-x:auto}
|
||||
svg{display:block;max-width:100%;height:auto;font-family:inherit}
|
||||
svg text{fill:var(--ink);font-size:13px}
|
||||
svg .mute{fill:var(--mute);font-size:11.5px}
|
||||
svg .t{font-weight:600}
|
||||
svg .edge{stroke:var(--mute);stroke-width:1.6;fill:none}
|
||||
svg .dash{stroke-dasharray:5 4}
|
||||
.legend{display:flex;flex-wrap:wrap;gap:14px;font-size:13px;color:var(--mute);margin-top:8px}
|
||||
.legend i{display:inline-block;width:12px;height:12px;border-radius:3px;margin-right:6px;vertical-align:-1px}
|
||||
.grid{display:grid;grid-template-columns:repeat(auto-fill,minmax(320px,1fr));gap:14px}
|
||||
.pb{background:var(--card);border:1px solid var(--line);border-radius:12px;padding:14px 16px;border-top:4px solid var(--c,var(--line))}
|
||||
.pb .num{font:600 12px/1 ui-monospace,monospace;color:var(--c);letter-spacing:.04em}
|
||||
.pb h3{margin:4px 0 4px;font-size:17px}
|
||||
.pb .trig{color:var(--mute);font-size:13.5px;margin:0 0 8px}
|
||||
.pb details{border-top:1px solid var(--line);padding-top:6px;margin-top:6px}
|
||||
.pb summary{cursor:pointer;font-size:13.5px;font-weight:600;color:var(--ink);list-style:none}
|
||||
.pb summary::before{content:"▸ ";color:var(--mute)}
|
||||
.pb details[open] summary::before{content:"▾ "}
|
||||
.pb ol,.pb ul{margin:6px 0 4px;padding-left:20px;font-size:13.5px}
|
||||
.pb li{margin:2px 0}
|
||||
.kv{display:grid;grid-template-columns:96px 1fr;gap:4px 10px;font-size:13px;margin-top:6px}
|
||||
.kv b{color:var(--mute);font-weight:500}
|
||||
.tag{display:inline-block;font-size:11.5px;padding:1px 7px;border-radius:999px;background:var(--soft);color:var(--mute);margin:0 4px 4px 0}
|
||||
.new{background:var(--okbg);color:var(--ok);font-weight:600}
|
||||
table{border-collapse:collapse;width:100%;font-size:14px}
|
||||
th,td{text-align:left;padding:7px 9px;border-bottom:1px solid var(--line);vertical-align:top}
|
||||
th{color:var(--mute);font-weight:600;font-size:12.5px;text-transform:uppercase;letter-spacing:.03em}
|
||||
.note{border-left:3px solid var(--ok);background:var(--okbg);padding:9px 12px;border-radius:6px;font-size:14px}
|
||||
.two{display:grid;grid-template-columns:1fr 1fr;gap:14px}
|
||||
@media (max-width:760px){.two{grid-template-columns:1fr}}
|
||||
footer{color:var(--mute);font-size:13px;margin-top:48px}
|
||||
code{font:12.5px ui-monospace,monospace;background:var(--soft);padding:1px 5px;border-radius:4px}
|
||||
.theme{float:right}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<main>
|
||||
<button class="chip theme" id="theme" type="button">Theme</button>
|
||||
<h1>Novox Playbooks</h1>
|
||||
<p class="sub">How work is done on the mesh, from an idea or a symptom to a delivered change. Drawn from hq's <code>00-META/process/</code> at main <code>f269e83</code>, 2026-10-19, amended 2026-10-26 for ADR 0548 (one release, one check: no try).</p>
|
||||
|
||||
<nav>
|
||||
<a href="#cycle">The cycle</a>
|
||||
<a href="#feature">A feature, step by step</a>
|
||||
<a href="#tests">When tests run</a>
|
||||
<a href="#review">The review loop</a>
|
||||
<a href="#issues">An issue's life</a>
|
||||
<a href="#where">Where things live</a>
|
||||
<a href="#all">All playbooks</a>
|
||||
</nav>
|
||||
|
||||
<!-- ============ THE CYCLE ============ -->
|
||||
<h2 id="cycle">The development cycle</h2>
|
||||
<p class="lead">Work enters as an <b>idea</b> or a <b>symptom</b>. An idea becomes a decision, a design and then milestones; a symptom becomes an issue. Both end as code delivered to the machines, and the as-is design is updated. <b>No design without a decision; no development without a design or an issue that sends it.</b></p>
|
||||
<div class="box scroll">
|
||||
<svg viewBox="0 0 1080 470" role="img" aria-label="The development cycle across playbooks 01 to 09">
|
||||
<defs>
|
||||
<marker id="ar" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse"><path d="M0,0L10,5L0,10z" fill="var(--mute)"/></marker>
|
||||
</defs>
|
||||
<!-- entry points -->
|
||||
<g>
|
||||
<rect x="16" y="40" width="120" height="50" rx="25" fill="none" stroke="var(--idea)" stroke-width="2"/>
|
||||
<text x="76" y="70" text-anchor="middle" class="t">Idea</text>
|
||||
<rect x="16" y="300" width="120" height="50" rx="25" fill="none" stroke="var(--sym)" stroke-width="2"/>
|
||||
<text x="76" y="330" text-anchor="middle" class="t">Symptom</text>
|
||||
</g>
|
||||
<!-- idea path -->
|
||||
<g>
|
||||
<rect x="176" y="30" width="150" height="70" rx="10" fill="var(--card)" stroke="var(--idea)" stroke-width="2"/>
|
||||
<text x="251" y="58" text-anchor="middle" class="t">01 Research</text>
|
||||
<text x="251" y="78" text-anchor="middle" class="mute">questions, findings,</text>
|
||||
<text x="251" y="92" text-anchor="middle" class="mute">direction confirmed</text>
|
||||
|
||||
<rect x="366" y="30" width="160" height="70" rx="10" fill="var(--card)" stroke="var(--dec)" stroke-width="2"/>
|
||||
<text x="446" y="58" text-anchor="middle" class="t">02 Graduation</text>
|
||||
<text x="446" y="78" text-anchor="middle" class="mute">decisions in the</text>
|
||||
<text x="446" y="92" text-anchor="middle" class="mute">knowledge base · to-be</text>
|
||||
|
||||
<rect x="566" y="30" width="160" height="70" rx="10" fill="var(--card)" stroke="var(--dec)" stroke-width="2"/>
|
||||
<text x="646" y="58" text-anchor="middle" class="t">04 Breakdown</text>
|
||||
<text x="646" y="78" text-anchor="middle" class="mute">milestones by release,</text>
|
||||
<text x="646" y="92" text-anchor="middle" class="mute">issues per repository</text>
|
||||
|
||||
<rect x="766" y="30" width="150" height="70" rx="10" fill="var(--card)" stroke="var(--build)" stroke-width="2.5"/>
|
||||
<text x="841" y="58" text-anchor="middle" class="t">07 Feature</text>
|
||||
<text x="841" y="78" text-anchor="middle" class="mute">one milestone =</text>
|
||||
<text x="841" y="92" text-anchor="middle" class="mute">one release</text>
|
||||
</g>
|
||||
<!-- symptom path -->
|
||||
<g>
|
||||
<rect x="176" y="290" width="150" height="70" rx="10" fill="var(--card)" stroke="var(--sym)" stroke-width="2"/>
|
||||
<text x="251" y="318" text-anchor="middle" class="t">03 Issues</text>
|
||||
<text x="251" y="338" text-anchor="middle" class="mute">opened in the tracker,</text>
|
||||
<text x="251" y="352" text-anchor="middle" class="mute">diagnosed, located</text>
|
||||
|
||||
<rect x="566" y="290" width="160" height="70" rx="10" fill="var(--card)" stroke="var(--build)" stroke-width="2.5"/>
|
||||
<text x="646" y="318" text-anchor="middle" class="t">08 Small fix</text>
|
||||
<text x="646" y="338" text-anchor="middle" class="mute">one issue, one repository,</text>
|
||||
<text x="646" y="352" text-anchor="middle" class="mute">about an hour</text>
|
||||
</g>
|
||||
<!-- delivered + as-is -->
|
||||
<g>
|
||||
<rect x="950" y="160" width="116" height="74" rx="10" fill="var(--card)" stroke="var(--mesh)" stroke-width="2.5"/>
|
||||
<text x="1008" y="188" text-anchor="middle" class="t">Delivered</text>
|
||||
<text x="1008" y="206" text-anchor="middle" class="mute">Approve, merge,</text>
|
||||
<text x="1008" y="220" text-anchor="middle" class="mute">wants and haves</text>
|
||||
|
||||
<rect x="766" y="160" width="150" height="74" rx="10" fill="var(--card)" stroke="var(--dec)" stroke-width="2" stroke-dasharray="5 4"/>
|
||||
<text x="841" y="188" text-anchor="middle" class="t">As-is updated</text>
|
||||
<text x="841" y="206" text-anchor="middle" class="mute">02 "when it ships":</text>
|
||||
<text x="841" y="220" text-anchor="middle" class="mute">to-be → implemented</text>
|
||||
</g>
|
||||
<!-- side playbooks -->
|
||||
<g>
|
||||
<rect x="176" y="400" width="190" height="56" rx="10" fill="var(--soft)" stroke="var(--side)"/>
|
||||
<text x="271" y="424" text-anchor="middle" class="t">05 Constitution sync</text>
|
||||
<text x="271" y="442" text-anchor="middle" class="mute">how-we-build → derived page</text>
|
||||
<rect x="396" y="400" width="190" height="56" rx="10" fill="var(--soft)" stroke="var(--side)"/>
|
||||
<text x="491" y="424" text-anchor="middle" class="t">06 Writing a module</text>
|
||||
<text x="491" y="442" text-anchor="middle" class="mute">what runs today → a module</text>
|
||||
<rect x="876" y="400" width="190" height="56" rx="10" fill="var(--soft)" stroke="var(--side)"/>
|
||||
<text x="971" y="424" text-anchor="middle" class="t">09 Cleanup</text>
|
||||
<text x="971" y="442" text-anchor="middle" class="mute">branches, worktrees, on the word</text>
|
||||
</g>
|
||||
<!-- edges -->
|
||||
<path class="edge" d="M136,65 H174" marker-end="url(#ar)"/>
|
||||
<path class="edge" d="M326,65 H364" marker-end="url(#ar)"/>
|
||||
<path class="edge" d="M526,65 H564" marker-end="url(#ar)"/>
|
||||
<path class="edge" d="M726,65 H764" marker-end="url(#ar)"/>
|
||||
<path class="edge" d="M916,65 C990,65 1008,100 1008,158" marker-end="url(#ar)"/>
|
||||
<path class="edge" d="M136,325 H174" marker-end="url(#ar)"/>
|
||||
<path class="edge" d="M326,325 H564" marker-end="url(#ar)"/>
|
||||
<text x="445" y="317" text-anchor="middle" class="mute">a fix of about an hour</text>
|
||||
<path class="edge" d="M726,325 C990,325 1008,290 1008,236" marker-end="url(#ar)"/>
|
||||
<path class="edge" d="M948,197 H918" marker-end="url(#ar)"/>
|
||||
<path class="edge dash" d="M251,290 C251,200 420,170 440,102" marker-end="url(#ar)"/>
|
||||
<text x="300" y="196" class="mute">a design gap: decision first</text>
|
||||
<path class="edge dash" d="M326,300 C560,250 720,170 800,102" marker-end="url(#ar)"/>
|
||||
<text x="612" y="236" class="mute">larger than an hour: a feature</text>
|
||||
<path class="edge dash" d="M1008,236 C1008,330 990,360 971,398" marker-end="url(#ar)"/>
|
||||
</svg>
|
||||
<div class="legend">
|
||||
<span><i style="background:var(--idea)"></i>from an idea</span>
|
||||
<span><i style="background:var(--sym)"></i>from a symptom</span>
|
||||
<span><i style="background:var(--dec)"></i>decisions and designs</span>
|
||||
<span><i style="background:var(--build)"></i>code changes</span>
|
||||
<span><i style="background:var(--mesh)"></i>the mesh delivers</span>
|
||||
<span><i style="background:var(--side)"></i>alongside</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- ============ FEATURE SWIMLANE ============ -->
|
||||
<h2 id="feature">A feature, step by step (playbook 07)</h2>
|
||||
<p class="lead">One milestone is one release: one branch name in every repository it touches, one worktree per repository, one fixing agent per repository kept for the whole milestone, one reviewer, and one pull request per repository, opened once. Nothing is pushed or tried before the review (ADR 0548): the branches are pushed <b>once, after the review approves</b>, and the one check runs the integration tests.</p>
|
||||
<div class="box scroll">
|
||||
<svg viewBox="0 0 1080 430" role="img" aria-label="Playbook 07 in lanes: lead, fixing agents, reviewer, the mesh, the operator">
|
||||
<defs>
|
||||
<marker id="ar2" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse"><path d="M0,0L10,5L0,10z" fill="var(--mute)"/></marker>
|
||||
</defs>
|
||||
<!-- step columns -->
|
||||
<g class="mute">
|
||||
<text x="190" y="20" text-anchor="middle" class="t" style="fill:var(--mute)">0 · Prepare</text>
|
||||
<text x="190" y="34" text-anchor="middle" class="mute">15 min</text>
|
||||
<text x="410" y="20" text-anchor="middle" class="t" style="fill:var(--mute)">1 · Develop</text>
|
||||
<text x="410" y="34" text-anchor="middle" class="mute">30 min per issue · local only</text>
|
||||
<text x="630" y="20" text-anchor="middle" class="t" style="fill:var(--mute)">2 · Review</text>
|
||||
<text x="630" y="34" text-anchor="middle" class="mute">≤ 3 rounds · 15 min</text>
|
||||
<text x="830" y="20" text-anchor="middle" class="t" style="fill:var(--mute)">3 · Check</text>
|
||||
<text x="830" y="34" text-anchor="middle" class="mute">push once · 1 check</text>
|
||||
<text x="1000" y="20" text-anchor="middle" class="t" style="fill:var(--mute)">4 · Deliver</text>
|
||||
<text x="1000" y="34" text-anchor="middle" class="mute">until delivered</text>
|
||||
</g>
|
||||
<!-- lanes -->
|
||||
<g>
|
||||
<rect x="0" y="46" width="1080" height="72" fill="var(--soft)" opacity=".55"/>
|
||||
<rect x="0" y="190" width="1080" height="72" fill="var(--soft)" opacity=".55"/>
|
||||
<rect x="0" y="334" width="1080" height="72" fill="var(--soft)" opacity=".55"/>
|
||||
<text x="10" y="86" class="t">Lead</text><text x="10" y="102" class="mute">larger model</text>
|
||||
<text x="10" y="158" class="t">Fixing agents</text><text x="10" y="174" class="mute">smaller model</text>
|
||||
<text x="10" y="230" class="t">Reviewer</text><text x="10" y="246" class="mute">larger model</text>
|
||||
<text x="10" y="302" class="t">The mesh</text><text x="10" y="318" class="mute">the one check, wants</text>
|
||||
<text x="10" y="374" class="t">Operator</text><text x="10" y="390" class="mute">Telegram</text>
|
||||
</g>
|
||||
<!-- lead -->
|
||||
<rect x="110" y="56" width="160" height="52" rx="8" fill="var(--card)" stroke="var(--dec)" stroke-width="2"/>
|
||||
<text x="190" y="78" text-anchor="middle" class="t">Survey · criteria</text>
|
||||
<text x="190" y="96" text-anchor="middle" class="mute">context file per repo</text>
|
||||
<rect x="340" y="56" width="140" height="52" rx="8" fill="var(--card)" stroke="var(--dec)" stroke-width="2"/>
|
||||
<text x="410" y="78" text-anchor="middle" class="t">Up to trunk</text>
|
||||
<text x="410" y="96" text-anchor="middle" class="mute">each branch, local</text>
|
||||
<rect x="760" y="56" width="140" height="52" rx="8" fill="var(--card)" stroke="var(--dec)" stroke-width="2"/>
|
||||
<text x="830" y="78" text-anchor="middle" class="t">Push reviewed heads</text>
|
||||
<text x="830" y="96" text-anchor="middle" class="mute">once, PRs together</text>
|
||||
<rect x="930" y="56" width="140" height="52" rx="8" fill="var(--card)" stroke="var(--dec)" stroke-width="2"/>
|
||||
<text x="1000" y="78" text-anchor="middle" class="t">Follow to done</text>
|
||||
<text x="1000" y="96" text-anchor="middle" class="mute">resolve the issues</text>
|
||||
<!-- fixing -->
|
||||
<rect x="300" y="128" width="220" height="52" rx="8" fill="var(--card)" stroke="var(--build)" stroke-width="2"/>
|
||||
<text x="410" y="150" text-anchor="middle" class="t">Build each issue, commit</text>
|
||||
<text x="410" y="168" text-anchor="middle" class="mute">unit suite before every commit</text>
|
||||
<rect x="560" y="128" width="140" height="52" rx="8" fill="var(--card)" stroke="var(--build)" stroke-width="2"/>
|
||||
<text x="630" y="150" text-anchor="middle" class="t">Fix findings</text>
|
||||
<text x="630" y="168" text-anchor="middle" class="mute">the same agents</text>
|
||||
<!-- reviewer -->
|
||||
<rect x="560" y="200" width="140" height="52" rx="8" fill="var(--card)" stroke="var(--idea)" stroke-width="2"/>
|
||||
<text x="630" y="222" text-anchor="middle" class="t">Read all repos</text>
|
||||
<text x="630" y="240" text-anchor="middle" class="mute">reads tests · every criterion</text>
|
||||
<!-- mesh -->
|
||||
<rect x="340" y="272" width="140" height="52" rx="8" fill="var(--card)" stroke="var(--mesh)" stroke-width="2"/>
|
||||
<text x="410" y="294" text-anchor="middle" class="t">no try</text>
|
||||
<text x="410" y="312" text-anchor="middle" class="mute">nothing pushed yet</text>
|
||||
<rect x="760" y="272" width="140" height="52" rx="8" fill="var(--card)" stroke="var(--mesh)" stroke-width="2"/>
|
||||
<text x="830" y="294" text-anchor="middle" class="t">Forge checks</text>
|
||||
<text x="830" y="312" text-anchor="middle" class="mute">one check · mesh/check</text>
|
||||
<rect x="930" y="272" width="140" height="52" rx="8" fill="var(--card)" stroke="var(--mesh)" stroke-width="2"/>
|
||||
<text x="1000" y="294" text-anchor="middle" class="t">Merge · wants</text>
|
||||
<text x="1000" y="312" text-anchor="middle" class="mute">backups first, canary first</text>
|
||||
<!-- operator -->
|
||||
<rect x="860" y="344" width="140" height="52" rx="8" fill="var(--card)" stroke="var(--op)" stroke-width="2.5"/>
|
||||
<text x="930" y="366" text-anchor="middle" class="t">Approve</text>
|
||||
<text x="930" y="384" text-anchor="middle" class="mute">once, for the group</text>
|
||||
<!-- edges -->
|
||||
<path class="edge" d="M190,108 C190,140 260,154 298,154" marker-end="url(#ar2)"/>
|
||||
<path class="edge" d="M410,128 V110" marker-end="url(#ar2)"/>
|
||||
<path class="edge" d="M630,200 V182" marker-end="url(#ar2)"/>
|
||||
<path class="edge dash" d="M610,182 V198" marker-end="url(#ar2)"/>
|
||||
<text x="514" y="196" class="mute">round n+1</text>
|
||||
<path class="edge" d="M700,226 C740,226 740,82 758,82" marker-end="url(#ar2)"/>
|
||||
<text x="712" y="140" class="mute">approved</text>
|
||||
<path class="edge" d="M830,108 V270" marker-end="url(#ar2)"/>
|
||||
<path class="edge" d="M900,298 C915,298 920,330 925,342" marker-end="url(#ar2)"/>
|
||||
<path class="edge" d="M1000,344 V326" marker-end="url(#ar2)"/>
|
||||
<path class="edge" d="M1000,272 V110" marker-end="url(#ar2)"/>
|
||||
</svg>
|
||||
</div>
|
||||
<div class="two">
|
||||
<div class="box">
|
||||
<h3>Step 0 · what the lead writes before any agent starts</h3>
|
||||
<ul style="margin:6px 0;padding-left:20px;font-size:14px">
|
||||
<li>Every open question settled: on the issue, in the knowledge base, or by the operator.</li>
|
||||
<li>Into each issue: every file and test it touches (file:line), how it is checked and delivered, a group's merge order.</li>
|
||||
<li><span class="tag new">new</span>Every other reader and writer of each value the change touches, each with a criterion.</li>
|
||||
<li>Acceptance criteria: given / when / then, each naming its test level.</li>
|
||||
<li>Every part the change reaches is an issue of the milestone; their order is the assigner's rules', never a cut.</li>
|
||||
<li>Fixtures copied from what the other side really writes, never from memory.</li>
|
||||
<li>An issue sized over an hour is split before work starts.</li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="box">
|
||||
<h3>Step 1 · the fixing agent's brief</h3>
|
||||
<ul style="margin:6px 0;padding-left:20px;font-size:14px">
|
||||
<li>One commit per issue, naming it; tests cite their criterion (<code>// 1217/AC3</code>).</li>
|
||||
<li><span class="tag new">new</span>A test fails first on what the code does. A compile error is no failure: take out the change's effect and see an assertion fail.</li>
|
||||
<li>The whole unit suite before every commit; each test well under a second.</li>
|
||||
<li>Integration tests are compiled only: never run on the machine.</li>
|
||||
<li>Never push, open or merge; never move an issue; never ssh.</li>
|
||||
<li>Code that differs from the context file: stop and report, file:line.</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- ============ TESTS ============ -->
|
||||
<h2 id="tests">When tests run</h2>
|
||||
<p class="lead">Two levels. Unit tests run all the time on the machine; integration tests (a bus, a store or a process started) are compiled there and never run: the checker's one merge check, asked by the one push after the review, runs them as the gate. There is no try (ADR 0548).</p>
|
||||
<div class="box scroll">
|
||||
<svg viewBox="0 0 1080 170" role="img" aria-label="Test levels over a feature's timeline">
|
||||
<defs><marker id="ar3" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto"><path d="M0,0L10,5L0,10z" fill="var(--mute)"/></marker></defs>
|
||||
<path class="edge" d="M120,140 H1060" marker-end="url(#ar3)"/>
|
||||
<text x="120" y="162" class="mute">develop</text><text x="470" y="162" class="mute">end of step 1</text><text x="640" y="162" class="mute">review</text><text x="800" y="162" class="mute">step 3</text><text x="960" y="162" class="mute">merge</text>
|
||||
<text x="10" y="44" class="t">Unit</text><text x="10" y="60" class="mute">on the machine</text>
|
||||
<g fill="var(--build)">
|
||||
<rect x="120" y="36" width="700" height="16" rx="8" opacity=".25"/>
|
||||
<circle cx="160" cy="44" r="6"/><circle cx="220" cy="44" r="6"/><circle cx="290" cy="44" r="6"/><circle cx="350" cy="44" r="6"/><circle cx="420" cy="44" r="6"/><circle cx="600" cy="44" r="6"/><circle cx="680" cy="44" r="6"/>
|
||||
</g>
|
||||
<text x="830" y="42" class="mute">whole suite before every commit</text><text x="830" y="56" class="mute">each test under a second</text>
|
||||
<text x="10" y="104" class="t">Integration</text><text x="10" y="120" class="mute">never on the machine</text>
|
||||
<g>
|
||||
<rect x="120" y="96" width="340" height="16" rx="8" fill="var(--mute)" opacity=".18"/>
|
||||
<text x="290" y="108" text-anchor="middle" class="mute">compiled only: go vet -tags integration</text>
|
||||
<rect x="770" y="90" width="120" height="28" rx="6" fill="none" stroke="var(--mesh)" stroke-width="2" stroke-dasharray="5 4"/><text x="830" y="109" text-anchor="middle" class="mute">push once</text>
|
||||
<rect x="920" y="90" width="120" height="28" rx="6" fill="var(--mesh)"/><text x="980" y="109" text-anchor="middle" style="fill:var(--bg)" class="t">merge check</text>
|
||||
</g>
|
||||
</svg>
|
||||
</div>
|
||||
|
||||
<!-- ============ REVIEW LOOP ============ -->
|
||||
<h2 id="review">The review loop</h2>
|
||||
<div class="two">
|
||||
<div class="box scroll">
|
||||
<svg viewBox="0 0 520 300" role="img" aria-label="The review loop: at most three rounds">
|
||||
<defs><marker id="ar4" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto"><path d="M0,0L10,5L0,10z" fill="var(--mute)"/></marker></defs>
|
||||
<rect x="20" y="20" width="150" height="54" rx="8" fill="var(--card)" stroke="var(--idea)" stroke-width="2"/>
|
||||
<text x="95" y="44" text-anchor="middle" class="t">Round n of ≤ 3</text><text x="95" y="62" text-anchor="middle" class="mute">code + criteria</text>
|
||||
<polygon points="300,20 380,47 300,74 220,47" fill="var(--card)" stroke="var(--mute)"/>
|
||||
<text x="300" y="51" text-anchor="middle" class="t">Wrong?</text>
|
||||
<rect x="400" y="20" width="110" height="54" rx="8" fill="var(--okbg)" stroke="var(--ok)" stroke-width="2"/>
|
||||
<text x="455" y="44" text-anchor="middle" class="t">Approve</text><text x="455" y="62" text-anchor="middle" class="mute">Reviewed-by line</text>
|
||||
<rect x="210" y="130" width="180" height="54" rx="8" fill="var(--card)" stroke="var(--build)" stroke-width="2"/>
|
||||
<text x="300" y="154" text-anchor="middle" class="t">Same fixing agents fix</text><text x="300" y="172" text-anchor="middle" class="mute">answer per finding</text>
|
||||
<rect x="20" y="130" width="160" height="54" rx="8" fill="var(--soft)" stroke="var(--side)"/>
|
||||
<text x="100" y="154" text-anchor="middle" class="t">issue-tracker.open</text><text x="100" y="172" text-anchor="middle" class="mute">anything beyond the issue</text>
|
||||
<rect x="300" y="236" width="210" height="54" rx="8" fill="var(--warnbg)" stroke="var(--warn)" stroke-width="2"/>
|
||||
<text x="405" y="260" text-anchor="middle" class="t">After round 3: the operator</text><text x="405" y="278" text-anchor="middle" class="mute">what is left, with a recommendation</text>
|
||||
<path class="edge" d="M170,47 H218" marker-end="url(#ar4)"/>
|
||||
<path class="edge" d="M380,47 H398" marker-end="url(#ar4)"/><text x="384" y="38" class="mute">no</text>
|
||||
<path class="edge" d="M300,74 V128" marker-end="url(#ar4)"/><text x="306" y="104" class="mute">yes</text>
|
||||
<path class="edge" d="M210,157 C140,157 95,120 95,76" marker-end="url(#ar4)"/><text x="120" y="112" class="mute">n+1</text>
|
||||
<path class="edge dash" d="M250,66 C200,100 170,120 150,128" marker-end="url(#ar4)"/><text x="178" y="96" class="mute">hardening,</text><text x="178" y="108" class="mute">a nicer form</text>
|
||||
<path class="edge dash" d="M360,184 C380,210 400,220 405,234" marker-end="url(#ar4)"/>
|
||||
</svg>
|
||||
</div>
|
||||
<div class="box">
|
||||
<h3>What a round may ask</h3>
|
||||
<p style="font-size:14px">Only what makes the change <b>wrong</b> against its issues, their acceptance criteria and its decisions. Anything more (hardening, a further case, a nicer form) becomes a new issue, and the round does not wait for it.</p>
|
||||
<h3 style="margin-top:12px">What every round reads</h3>
|
||||
<ul style="margin:6px 0;padding-left:20px;font-size:14px">
|
||||
<li>Every criterion against its test: a test at its level that would fail without the change.</li>
|
||||
<li>Every new integration test traced through the code by reading it: no run exists before the check.</li>
|
||||
<li>That each integration test enters by the path the live trigger takes.</li>
|
||||
</ul>
|
||||
<div class="note" style="margin-top:10px">Why reviews took more than one round (R3b, 2026-10-19): a test that "failed" only by not compiling, a reader of the changed record the survey missed, and bus tests judged by reasoning before they ran. ADR 0548 and playbook 07 step 2 item 4 change the last: bus tests are judged by reading them against the code, and the one check runs them after the review.</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- ============ ISSUE LIFE ============ -->
|
||||
<h2 id="issues">An issue's life (playbook 03)</h2>
|
||||
<p class="lead">The issue tracker (seat <code>issue-tracker</code>) holds every issue's number, status, owner, fix and links. Its <code>move</code> refuses a wrong move at the write.</p>
|
||||
<div class="box scroll">
|
||||
<svg viewBox="0 0 1080 230" role="img" aria-label="Issue statuses and the moves between them">
|
||||
<defs><marker id="ar5" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto"><path d="M0,0L10,5L0,10z" fill="var(--mute)"/></marker></defs>
|
||||
<g>
|
||||
<rect x="20" y="70" width="120" height="50" rx="25" fill="var(--card)" stroke="var(--sym)" stroke-width="2"/><text x="80" y="100" text-anchor="middle" class="t">open</text>
|
||||
<rect x="220" y="70" width="140" height="50" rx="25" fill="var(--card)" stroke="var(--warn)" stroke-width="2"/><text x="290" y="100" text-anchor="middle" class="t">diagnosing</text>
|
||||
<rect x="440" y="70" width="120" height="50" rx="25" fill="var(--card)" stroke="var(--dec)" stroke-width="2"/><text x="500" y="100" text-anchor="middle" class="t">located</text>
|
||||
<rect x="640" y="70" width="120" height="50" rx="25" fill="var(--card)" stroke="var(--build)" stroke-width="2"/><text x="700" y="100" text-anchor="middle" class="t">fixing</text>
|
||||
<rect x="860" y="70" width="140" height="50" rx="25" fill="var(--okbg)" stroke="var(--ok)" stroke-width="2.5"/><text x="930" y="100" text-anchor="middle" class="t">resolved</text>
|
||||
<rect x="440" y="170" width="120" height="44" rx="22" fill="var(--soft)" stroke="var(--mute)"/><text x="500" y="197" text-anchor="middle" class="t">wontfix</text>
|
||||
</g>
|
||||
<path class="edge" d="M140,95 H218" marker-end="url(#ar5)"/><text x="146" y="86" class="mute">by</text>
|
||||
<path class="edge" d="M360,95 H438" marker-end="url(#ar5)"/><text x="366" y="86" class="mute">located-in</text>
|
||||
<path class="edge" d="M560,95 H638" marker-end="url(#ar5)"/><text x="590" y="86" class="mute">by</text>
|
||||
<path class="edge" d="M760,95 H858" marker-end="url(#ar5)"/><text x="770" y="138" class="mute">fixed-by, delivered,</text><text x="770" y="152" class="mute">proven · core: replay</text>
|
||||
<path class="edge dash" d="M140,105 C300,150 520,150 640,108" marker-end="url(#ar5)"/><text x="300" y="150" class="mute">straight to fixing, with by</text>
|
||||
<path class="edge dash" d="M500,120 V168" marker-end="url(#ar5)"/><text x="508" y="150" class="mute">reason</text>
|
||||
<path class="edge dash" d="M700,70 C700,30 600,30 590,30 C400,30 120,30 90,68" marker-end="url(#ar5)"/><text x="320" y="24" class="mute">stopping: back to open, with a note of branch, last commit and what is left</text>
|
||||
</svg>
|
||||
<div class="legend"><span>A worker idle for a day may be taken over without a reason; otherwise <code>take-over</code> with a reason. An issue about a condition is linked to it, so the alert names the issue.</span></div>
|
||||
</div>
|
||||
|
||||
<!-- ============ WHERE THINGS LIVE ============ -->
|
||||
<h2 id="where">Where things live</h2>
|
||||
<div class="box scroll">
|
||||
<table>
|
||||
<tr><th>What</th><th>Lives in</th><th>Written with</th><th>Never</th></tr>
|
||||
<tr><td>Decisions (ADRs) and research efforts</td><td>the knowledge base (seat <code>knowledge-base</code>)</td><td><code>reserve</code>, then <code>propose</code> the record whole</td><td>a file in hq</td></tr>
|
||||
<tr><td>Issues: status, owner, fix, links</td><td>the issue tracker</td><td><code>open</code>, <code>move</code>, <code>note</code>, <code>link</code></td><td>a status key in a report</td></tr>
|
||||
<tr><td>An issue's narrative (design or governance only)</td><td>hq <code>04-ISSUES/NNN-…/</code></td><td>a pull request</td><td>its status</td></tr>
|
||||
<tr><td>Designs, as-is and to-be</td><td>hq <code>03-DESIGN/</code>, status in frontmatter</td><td>a pull request</td><td>the two layers mixed</td></tr>
|
||||
<tr><td>Playbooks, glossary, how-we-build</td><td>hq <code>00-META/</code></td><td>a pull request</td><td>a copy in a session's memory</td></tr>
|
||||
<tr><td>The rules every session keeps</td><td>instruction sections (module <code>claude-code</code>)</td><td><code>claude_code_instructions_propose</code>, the operator's Approve</td><td>a hand-edited file</td></tr>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
<!-- ============ ALL PLAYBOOKS ============ -->
|
||||
<h2 id="all">All playbooks</h2>
|
||||
<p class="sub">Open a card for its steps, what it produces and how it is checked.</p>
|
||||
<div id="filters" style="display:flex;flex-wrap:wrap;gap:8px;margin-bottom:12px"></div>
|
||||
<div class="grid" id="cards"></div>
|
||||
|
||||
<footer>Drawn from hq main f269e83 on 2026-10-19 and kept in hq beside the playbooks. The playbooks are the authority; this page is a picture of them, and a change to a playbook's steps updates it in the same pull request (not yet checked: issue 1222).</footer>
|
||||
</main>
|
||||
|
||||
<script>
|
||||
const P = [
|
||||
{n:"01",name:"Research",kind:"idea",c:"--idea",
|
||||
trig:"An idea worth investigating before it is committed to design. A symptom is not research.",
|
||||
who:"The session that heard the idea leads.",
|
||||
steps:["Read what is known: ask, get, the tracker's list.","Ask 3 to 5 setup questions, one at a time, each with a recommendation.","Check every option against the mission and how-we-build first.","Reserve a number (kind research), never pick one by hand.","Propose the effort: overview, questions, method, with links.","Research; findings in 04-findings, one version per settled set.","Try at the cheapest level that answers the question.","Write 03-domains: modules, seats, glossary words added or retired.","Write 05-direction: decisions, rejected options, acceptance tests.","Ask the operator to confirm the direction, one point at a time.","If the review asks changes: propose again, at most three times."],
|
||||
prod:"A research effort in the knowledge base, status active, every version kept. Nothing in hq.",
|
||||
next:"02",bounds:"Setup 30 min · proposing 15 min · each experiment 30 min · direction 30 min · report every 10 min",
|
||||
prev:["Research that measured nothing","A recommendation the operator must argue back","Efforts copied into hq"],
|
||||
check:"propose refuses rule breaks; the knowledge base reviews; the records test on every PR."},
|
||||
{n:"02",name:"Graduation & design change",kind:"idea",c:"--dec",
|
||||
trig:"A research effort concludes, or an existing design must change.",
|
||||
who:"The lead, on the larger model; the decision is recorded before the design moves.",
|
||||
steps:["Check the conclusion against mission, context and effect.","Reserve a number per decision.","Propose one record per choice that can be reversed on its own.","Pre-check, then propose siblings together.","On changes: revise whole; after three rounds fix findings yourself.","A finding that would change what was decided: ask the operator.","needs-operator: wait for the operator's Approve.","Amend the design, or create a to-be design with status designed.","Add the decisions, updated:, and the effort's glossary words.","Hand the design to the breakdown (04).","Close the effort: graduated with became links, or abandoned.","When it ships: write the as-is, set the to-be implemented."],
|
||||
prod:"Decision records; a to-be design with frontmatter; glossary entries; later the as-is.",
|
||||
next:"04",bounds:"One sitting of about an hour",
|
||||
prev:["A decision rewritten silently","A deferral nothing tracks","A design amended without a decision"],
|
||||
check:"propose refuses; the records and index tests; set-wide rules raise conditions."},
|
||||
{n:"03",name:"Issues",kind:"symptom",c:"--sym",
|
||||
trig:"Something is wrong: an unenforced rule, a stated behaviour that does not happen, a silent failure.",
|
||||
who:"Anyone opens; a worker claims it by moving it.",
|
||||
steps:["Open it in the tracker: title, report; take the number it gives.","Tracker unreachable: take no number, say so.","A design or governance issue: a folder in hq with the report.","About a condition: link the condition's key.","Move to diagnosing with by, before investigating.","Note the trail; a folder issue also gets a dated diagnosis.","Move to located with located-in, a registered owner.","Move to fixing with by, before fixing.","Name fixed-by at the merge; stay fixing until proven live.","Move to resolved; a core issue gives replay or replay-none.","A design gap: run 02 and give amended-design.","Not going to be fixed: wontfix with a reason."],
|
||||
prod:"A tracker record with notes and links; for a design issue, its narrative folder in hq.",
|
||||
next:"02 · 07 · 08",bounds:"A worker idle for a day is stale",
|
||||
prev:["Numbers that collide","Two agents starting the same work","Status held in two places"],
|
||||
check:"The tracker's move refuses at the write; the cycle test checks the folders."},
|
||||
{n:"04",name:"Work breakdown & handoff",kind:"idea",c:"--dec",
|
||||
trig:"A to-be design is settled and its building is about to start.",
|
||||
who:"The lead who will run the feature, on the larger model.",
|
||||
steps:["Confirm status designed; each decision traces to a record.","An open question remains: back to research.","Set code: to the owning repositories.","Read the as-is; write it first if it does not say what is replaced.","Write the Milestones: outcome, acceptance tests, repositories.","Cut milestones by functionality, never by an order between parts; fold any not usable alone.","Ask the operator to confirm the milestones.","Open one issue per repository part, linked depends-on.","Write the issue numbers into the design; flip to in-progress.","Hand each milestone to 07, one at a time, one design at a time."],
|
||||
prod:"Milestones in the design; issues per repository part; the design in-progress.",
|
||||
next:"07",bounds:"One sitting · one design built at a time · a milestone is one release",
|
||||
prev:["Building without a breakdown","Building against an undocumented as-is","Deferrals with no owning issue"],
|
||||
check:"Read by the lead and the reviewer; a check is tracked by issue 845."},
|
||||
{n:"05",name:"Constitution sync",kind:"side",c:"--side",
|
||||
trig:"how-we-build.md changed a rule the mesh enforces at runtime.",
|
||||
who:"Whoever made the change.",
|
||||
steps:["Edit how-we-build; the reasoning stays there.","Record the change as a decision (02).","Publish the derived page under its slug.","Keep the section numbering stable.","Read the page back; verify the change is there.","Note the sync in the record's consequences."],
|
||||
prod:"A decision record; the derived constitution page injected into design sessions.",
|
||||
next:"02",bounds:"no time bound",
|
||||
prev:["Drift between the reasoned and the enforced copy","A publish that said success and did nothing","Edits to the derived page that vanish"],
|
||||
check:"The read-back in step 5."},
|
||||
{n:"06",name:"Writing a module",kind:"side",c:"--side",
|
||||
trig:"Something that runs today must run on the mesh.",
|
||||
who:"Whoever ports or writes it.",
|
||||
steps:["Read what runs: compose file, environment, where data sits.","Name what it provides and requires.","Declare capabilities for machine facts.","Write resources in the order they must happen.","Every secret in a file, never in env.","Pin every image by digest.","New: generate the credential; adopted: accept it.","Prove it against the real software: auth, refusal, restart."],
|
||||
prod:"A module declaration, optionally a provisioner, and its proof.",
|
||||
next:"07",bounds:"no time bound",
|
||||
prev:["A service running with no password","A new password locking out adopted data","A test that cannot fail"],
|
||||
check:"Example tests refuse the env-file mistake; review."},
|
||||
{n:"07",name:"Features",kind:"build",c:"--build",
|
||||
trig:"Work that changes code across issues or repositories, too large for a small fix.",
|
||||
who:"A lead (larger model), one fixing agent per repository (smaller), one reviewer (larger).",
|
||||
steps:["Prepare: name the branch, make the worktrees, settle every question.","Write one context file per repository.","Write the findings and acceptance criteria into each issue.","Survey every other reader and writer of each changed value. (new)","Answer three questions in the milestone file: who deletes each new key and stream, how the switch-over is tried on one node first, what the one check does not cover. The review checks them. (new)","List every part the change reaches; copy fixtures from the real writer.","Develop: one agent per repository, one commit per issue.","A test fails first on behaviour, not on compiling. (new)","Bring each branch up to the trunk; nothing is pushed, no try runs. (new)","Review all of it, reading the integration tests against the code; at most 3 rounds.","A group pins the group's own mesh-sdk head in every member. (new)","Push the reviewed heads once and read each forge head against the reviewed commit; no draft pull request, no fix round on an open one. (new)","Open one pull request per repository, together, with Reviewed-by and Pushed-by; the one check runs. A failed check is read, fixed, reviewed, pushed once more. (new)","The operator's Approve on Telegram; the group merges in one pass.","Follow the delivery in the status view (moving, settled, stuck, delivered) to done; resolve the issues; clean up. (new)","When the builder cannot deliver its own fix: revert, a build by a session's call, a registration at the terminal, copies kept outside the store; a broken controller is put back by a want the operator writes, a hand build by mesh-hand-compose and a want. (new)"],
|
||||
prod:"One pull request per repository, one source revision delivered, resolved issues, no branch or worktree left.",
|
||||
next:"02 (when it ships)",bounds:"Prepare 15 min · 30 min per issue · review 15 min, ≤ 3 rounds · one check, pushed once",
|
||||
prev:["Pull requests as scratchpads","Context bought again and again","Review rounds spent on what a run would show","A node agent's bus reads, proven only by review"],
|
||||
check:"The checker's one status, mesh/check; the review record; the tracker's move."},
|
||||
{n:"08",name:"Small fixes",kind:"build",c:"--build",
|
||||
trig:"One issue in one repository, about an hour of work.",
|
||||
who:"The session, with one reviewer on the larger model.",
|
||||
steps:["Read the issue and the code; settle any open question.","Write the acceptance criteria.","Move the issue to fixing.","Build on fix/<issue>-<slug> in its own worktree.","A test that fails today first, on behaviour. (new)","Nothing is pushed and no try runs before the review. (new)","One review of the local commit.","Push the reviewed head once; only then open the pull request. (new)","The operator's Approve; the merge; the delivery, followed in the status view. (new)","Prove the fix live and resolve the same hour.","During a milestone: ride in its delivery group, unless broken for the operator now. (new)"],
|
||||
prod:"One pull request, one delivery, a resolved issue.",
|
||||
next:"—",bounds:"About an hour · one review, one push, one check",
|
||||
prev:["A fix reviewed after its pull request opened","A long part run twice","A fix waiting on someone else's delivery","Ten SDK merges and eight controller merges in a day, one per fix"],
|
||||
check:"The checker's status; the review record; the tracker's move; one delivery group and one source revision per milestone."},
|
||||
{n:"09",name:"Cleanup",kind:"side",c:"--side",
|
||||
trig:"The operator asks, or merged branches, stale pull requests and old worktrees pile up.",
|
||||
who:"A session on the operator's word; the operator decides every deletion.",
|
||||
steps:["Take stock, read-only.","Write one report: item, class, evidence.","Classify: safe, likely safe, kept, unknown.","Report stale pull requests; never close them.","Ask: delete the safe items as listed?","Delete only the approved list; never force.","Report what was cleaned, kept and unknown."],
|
||||
prod:"A stock report, the approved deletions, issues for missing tools.",
|
||||
next:"03",bounds:"A worktree is safe only if untouched for 2 h · a PR is stale after 3 days",
|
||||
prev:["Deleting unfinished work","Deletion by name or age alone","Missing tools left as session notes"],
|
||||
check:"The report names the evidence for each deletion."},
|
||||
{n:"10",name:"Visual design",kind:"idea",c:"--idea",
|
||||
trig:"A page the operator will use is about to be designed or rebuilt, or the operator judges one unusable.",
|
||||
who:"The design's lead; the operator picks.",
|
||||
steps:["Gather what the page must carry: the as-is, the real data's shape, the operator's words, how others do it.","Draw three to five mockups with fixed, invented data, phone and desktop. (new)","Show them; ask one question per round. (new)","Make the pick the reference, with the operator's own details replaced. (new)","Write what is new as decisions, then the design and its milestones.","Hold the build to the reference; done when the operator uses it."],
|
||||
prod:"The reference mockup beside the design, its decisions, an acceptance test of live use.",
|
||||
next:"02, then 04",bounds:"A round 1 h · 3 rounds without a pick: ask what is missing",
|
||||
prev:["A page judged unusable the day it ships","Rebuilding after delivery"],
|
||||
check:"The design links its mockup; the review shows the build beside it; the operator's use is the last test."}
|
||||
];
|
||||
const kinds={all:"All",idea:"From an idea",symptom:"From a symptom",build:"Code changes",side:"Alongside"};
|
||||
let current="all";
|
||||
function esc(s){return s.replace(/&/g,"&").replace(/</g,"<")}
|
||||
function render(){
|
||||
const g=document.getElementById("cards"); g.innerHTML="";
|
||||
P.filter(p=>current==="all"||p.kind===current).forEach(p=>{
|
||||
const d=document.createElement("article"); d.className="pb"; d.style.setProperty("--c",`var(${p.c})`);
|
||||
d.innerHTML=`<div class="num">PLAYBOOK ${p.n}</div><h3>${esc(p.name)}</h3><p class="trig">${esc(p.trig)}</p>
|
||||
<div>${p.bounds.split(" · ").map(b=>`<span class="tag">${esc(b)}</span>`).join("")}</div>
|
||||
<details><summary>Steps (${p.steps.length})</summary><ol>${p.steps.map(s=>`<li>${esc(s).replace("(new)",'<span class="tag new">new</span>')}</li>`).join("")}</ol></details>
|
||||
<details><summary>Produces, prevents, checked by</summary><div class="kv">
|
||||
<b>Who</b><span>${esc(p.who)}</span><b>Produces</b><span>${esc(p.prod)}</span>
|
||||
<b>Then</b><span>${esc(p.next)}</span><b>Prevents</b><span>${p.prev.map(esc).join("; ")}</span>
|
||||
<b>Checked by</b><span>${esc(p.check)}</span></div></details>`;
|
||||
g.appendChild(d);
|
||||
});
|
||||
}
|
||||
const f=document.getElementById("filters");
|
||||
Object.entries(kinds).forEach(([k,v])=>{const b=document.createElement("button");b.type="button";b.className="chip"+(k===current?" on":"");b.textContent=v;
|
||||
b.onclick=()=>{current=k;[...f.children].forEach(x=>x.classList.remove("on"));b.classList.add("on");render()};f.appendChild(b)});
|
||||
render();
|
||||
const root=document.documentElement;
|
||||
try{const t=localStorage.getItem("pb-theme");if(t)root.dataset.theme=t}catch(e){}
|
||||
document.getElementById("theme").onclick=()=>{const dark=root.dataset.theme?root.dataset.theme==="dark":matchMedia("(prefers-color-scheme: dark)").matches;
|
||||
root.dataset.theme=dark?"light":"dark";try{localStorage.setItem("pb-theme",root.dataset.theme)}catch(e){}};
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
+9
-10
@@ -15,31 +15,30 @@ and a forge address is an operational detail (see [`README`](../README.md)).
|
||||
| Repository | Owns |
|
||||
|---|---|
|
||||
| `hal` | The monorepo — the node runtime, the module catalogue, the delivery machinery, and the bootstrap scripts. Every core module lives here. |
|
||||
| `hq` | This repository, under the company organisation — mission, research, design, decisions, issue diagnosis. Company-scoped (ADR 0019); the mesh is its first product. The source of truth for *why*. Carries no implementation. |
|
||||
| `hq` | This repository, under the company organisation — mission, research, design, decisions, issue diagnosis. Company-scoped ([ADR 0019](../02-DECISIONS/0019-how-this-repository-works.md)); the mesh is its first product. The source of truth for *why*. Carries no implementation. |
|
||||
| *(one per application)* | Every standalone application, site or side-project gets its own repository, with `module.yml` at the root. Registered with the mesh as a build source; built and deployed by the same pipeline as anything in the monorepo. |
|
||||
| `migration` | **Private.** The record of one installation replacing the predecessor mesh with this one: the runbook, a dated log of every step and what it cost, the per-service data procedures, the readiness checks, and the scripts. Private because it is the opposite of this repository in every way that matters — it names machines, addresses, ports and paths, because a procedure that cannot be followed is not one. Where hq asks *what did we decide and why*, that repository answers *what happened on the machines, in what order, and what to do next*. Its `HANDOFF.md` is where somebody picking the work up starts. |
|
||||
|
||||
## What the mesh becomes
|
||||
|
||||
ADR 0019 records the repositories the
|
||||
[ADR 0019](../02-DECISIONS/0019-how-this-repository-works.md) records the repositories the
|
||||
monorepo decomposes into. **`mesh-host`, `mesh-controller`, `mesh-catalog`, `mesh-lab`, `mesh-sdk` and `mesh-tools` exist
|
||||
so far** — the lab was built first (ADR 0016). The tiered
|
||||
so far** — the lab was built first ([ADR 0016](../02-DECISIONS/0016-the-lab.md)). The tiered
|
||||
decomposition below is the planned shape; the repositories built to date do not map onto it
|
||||
one-for-one — `mesh-catalog`, `mesh-sdk` and `mesh-tools` exist where the table names
|
||||
`mesh-foundation` and `mesh-surfaces`, and reconciling the two is itself still ahead.
|
||||
|
||||
| Repository | Tier | Holds |
|
||||
|---|---|---|
|
||||
| `mesh-host` | 0 | **exists.** The node-engine — one statically linked binary, requiring nothing present (ADR 0005) |
|
||||
| `mesh-host` | 0 | **exists.** The node-engine — one statically linked binary, requiring nothing present ([ADR 0005](../02-DECISIONS/0005-the-node-host.md)) |
|
||||
| `mesh-foundation` | 1 | the four pinned services, as declarations |
|
||||
| `mesh-controller` | 2 | **exists.** The controller and its contexts — one of seven built (ADR 0006) |
|
||||
| `mesh-controller` | 2 | **exists.** The controller and its contexts — one of seven built ([ADR 0006](../02-DECISIONS/0006-the-substrate-and-the-control-plane.md)) |
|
||||
| `mesh-surfaces` | 3 | tools, web, cli |
|
||||
| `mesh-sdk` | — | the stable spine modules build against — the tool-serving SDK harness, the messaging/event framework, the contracts and core primitives. Holds nothing per-module and nothing volatile (ADR 0039). |
|
||||
| `mesh-sdk` | — | the stable spine modules build against — the tool-serving harness, the messaging/event framework, the contracts and core primitives. Holds nothing per-module and nothing volatile ([ADR 0039](../02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md)). |
|
||||
| `mesh-lab` | — | **exists.** The lab — scenario lifecycle, networking, placement. Ships to nobody; runs on a workstation. |
|
||||
| `mesh-delivery` | — | the delivery domain: the parts that build, check and deliver a change by watching and writing keys on the bus, nine modules each whole in it, prefixed `mesh-delivery-` (ADR 0429). Made by the forge module from its config (ADR 0428). |
|
||||
|
||||
Tier 4's shape is open, and deliberately so: see ADR 0019 and
|
||||
research 005.
|
||||
[research 005](../01-RESEARCH/005-domain-grouping/00-overview.md).
|
||||
|
||||
## What lives where inside the monorepo
|
||||
|
||||
@@ -57,7 +56,7 @@ Named by role, because the layout is itself part of the as-is design — see
|
||||
## Why applications do not live in the monorepo
|
||||
|
||||
A standalone application in the monorepo is a convention violation, and reviewers reject it.
|
||||
The reasoning is recorded in ADR 0015:
|
||||
The reasoning is recorded in [`02-DECISIONS/0010`](../02-DECISIONS/0015-applications-live-in-their-own-repository.md):
|
||||
the mesh installs, provisions for, and ships an application through exactly the same machinery
|
||||
whether or not its source sits beside the mesh's own — so co-location buys nothing and costs
|
||||
the monorepo's review cadence.
|
||||
@@ -68,7 +67,7 @@ Each module is a standalone package that consumes its dependencies from the priv
|
||||
not from a sibling directory. The workspace was removed after it caused build-versus-development
|
||||
divergence — a workspace member importing another resolved to local unbuilt source in the
|
||||
pipeline and to a published version in development. Recorded in
|
||||
ADR 0014.
|
||||
[`02-DECISIONS/0007`](../02-DECISIONS/0014-no-npm-workspace.md).
|
||||
|
||||
Consequence, and it is a real one: a cross-package change is two steps — publish, then consume
|
||||
— and a repository-wide `npm install` does not exist.
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
---
|
||||
status: active
|
||||
initiated: 2026-08-22
|
||||
touches: [03-DESIGN/00-as-is/02-modules-and-manifests.md, 03-DESIGN/00-as-is/10-module-catalogue.md, 03-DESIGN/01-to-be/00-work-breakdown.md]
|
||||
became: [02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md]
|
||||
---
|
||||
|
||||
# 001 — Module domain decomposition
|
||||
|
||||
- **Initiated by:** jochen, 2026-08-22
|
||||
- **Areas touched:** every `hal/*` and `noxflow/*` module; the pipeline's dependency
|
||||
graph; the knowledge base; agent identity and credentials.
|
||||
|
||||
## Summary
|
||||
|
||||
HAL has 124 modules. That number is not a maintenance problem in itself — it is the
|
||||
**symptom of missing bounded contexts**. Modules are split not because they model
|
||||
different domains, but because splitting is the only lever the platform offers:
|
||||
|
||||
- no way to run one daemon on one node without making it a module
|
||||
(`hal/claude-licences` — one daemon, single-node)
|
||||
- no way to expose two of a kind from one module
|
||||
- no namespace separating the mesh from the software it runs
|
||||
|
||||
This effort establishes the **current state**, the **ideal state**, and the sequence
|
||||
between them.
|
||||
|
||||
## Trigger
|
||||
|
||||
A night of debugging that produced four fixes and one conclusion. Every fault was a
|
||||
boundary fault:
|
||||
|
||||
- Per-agent Claude credentials had to be written by the *noxflow runtime*, because
|
||||
`agents.claude_account` is in noxflow's database — even though agent identity is a
|
||||
mesh concept and node identity already lives in the mesh registry.
|
||||
- Whether `hal/brain` may depend on noxflow took three attempts to answer, twice
|
||||
wrongly, because the ownership boundary was never stated.
|
||||
- Authoritative documentation existed in `mesh_docs` and was not found, while a
|
||||
proposal in repo markdown was invisible to search entirely.
|
||||
|
||||
## Decisions taken (2026-08-22)
|
||||
|
||||
| Question | Decision |
|
||||
|---|---|
|
||||
| What should noxflow become? | Decompose into `hal/*` modules; noxflow returns to tasks/workflows |
|
||||
| Who owns agent identity? | `hal/agents` — a mesh concept, alongside nodes |
|
||||
| Where do third-party apps live? | Out of this repo. They run *on* the mesh; they are not *of* it |
|
||||
| Knowledge structure | Modelled on `papa-hq`; implementation choice left open |
|
||||
|
||||
## Open questions
|
||||
|
||||
Tracked in [`analysis.md`](analysis.md) under "Open questions".
|
||||
|
||||
## Deliberately not decided
|
||||
|
||||
Recorded so they are not mistaken for oversights. Each is open, and each comes out of
|
||||
[ADR 0001](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md); this effort stays
|
||||
`active` until they are answered.
|
||||
|
||||
| Question | Status |
|
||||
|---|---|
|
||||
| `hal/scheduler` — infrastructure, or part of the work context. | Open. |
|
||||
| Which context owns the executor. | Open. |
|
||||
| Catalogue destination — one repository or many. | Open. Phase 4. |
|
||||
| What the shared library keeps after extraction. | Open. Phase 3. |
|
||||
| Where human agent modality is recorded — which user, on which node, a human agent acts as. | Open. Required by the model; not yet stored. |
|
||||
| Which domains the modules outside the platform core group into. | Open, from [ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md), which settles the principle and deliberately not the list. |
|
||||
@@ -0,0 +1,242 @@
|
||||
---
|
||||
effort: 001-module-domain-decomposition
|
||||
updated: 2026-08-22
|
||||
---
|
||||
|
||||
# Current state → ideal state
|
||||
|
||||
## 1. The count is a symptom
|
||||
|
||||
124 modules. The number is not the problem; the reason for it is. Modules are split
|
||||
because splitting is the **only granularity lever the platform offers**:
|
||||
|
||||
| symptom | module | why it exists |
|
||||
|---|---|---|
|
||||
| one daemon must run on one node | `hal/claude-licences` | no per-node feature opt-in |
|
||||
| a library must not drag a 450 MB dep | `hal/claude` | no way to expose two npm packages |
|
||||
| a domain needs a daemon *and* a library | `hal/claude-code` + `hal/claude` | one feature of each type per module |
|
||||
| a handful of SDK verbs need a home | `infra` (10 lines) | tools must belong to *some* module |
|
||||
|
||||
A feature is a **handler type**, discovered by directory presence, singleton per module
|
||||
(`FEATURE_HANDLERS` in `modules/hal/sdk/src/feature-handlers/index.ts`). So "one more
|
||||
deployable unit" always means "one more module".
|
||||
|
||||
Splitting also fragments domains. Claude licence logic sits in three modules —
|
||||
resolution in `hal/claude`, health in `hal/claude-code`, refresh in
|
||||
`hal/claude-licences` — so following one token means reading three.
|
||||
|
||||
## 2. Two things are conflated in one repo
|
||||
|
||||
`noxflow` is 45 tables. Its original purpose — a HAL-native Jira — is 7 of them:
|
||||
|
||||
| domain | tables |
|
||||
|---|---:|
|
||||
| tasks (the original concept) | 7 |
|
||||
| agents / HR | 13 |
|
||||
| knowledge (a Confluence) | 8 |
|
||||
| meetings | 7 |
|
||||
| scheduling / ops | 2 |
|
||||
| other | 8 |
|
||||
|
||||
Agents alone are nearly double the concept the module was built for. This is the direct
|
||||
cause of concrete faults: per-agent Claude credentials had to be written by the noxflow
|
||||
runtime purely because `agents.claude_account` lives in noxflow's database, while node
|
||||
identity — the same kind of fact — lives in the mesh registry.
|
||||
|
||||
Separately, 91 of the 124 modules are third-party software (`plex`, `sonarr`,
|
||||
`firefox`, `postgres`). These **run on the mesh; they are not part of it**. They belong
|
||||
outside this repository. The pipeline and provisioning system are deliberately generic,
|
||||
so HAL's own modules dogfood exactly what external modules use — that property is what
|
||||
makes the separation safe.
|
||||
|
||||
## 3. Knowledge is split three ways
|
||||
|
||||
| store | owner | size | search |
|
||||
|---|---|---:|---|
|
||||
| `mesh_docs` (+ revisions) | `hal/hippocampus`, in the mesh DB | 144 docs, 73 touched in 30d | yes |
|
||||
| `knowledge_*` (8 tables) | `noxflow/runtime`, in the noxflow DB | 20 pages | yes |
|
||||
| repo markdown | git | `VISION.md`, `FLAVOR-PROPOSAL.md`, `docs/` | **no** |
|
||||
|
||||
The librarian is split across two of them: `librarian_file_document` writes via
|
||||
hippocampus, `librarian_ask` and `librarian_retrieve` read via noxflow. Filing and
|
||||
retrieval, different modules, different databases.
|
||||
|
||||
Both DB stores have revisions; only noxflow has a review workflow (`_proposals`,
|
||||
`_promotion_requests`). Neither has file access. Git has file access and history but no
|
||||
search integration. **Every piece needed already exists — none of them together.**
|
||||
|
||||
## 4. `hal/developer` and the missing local mesh
|
||||
|
||||
`hal/developer` (≈2,600 lines) exposes `dev_stage`, `dev_diff`, `dev_validate`,
|
||||
`dev_typecheck`, `dev_verify_flavor`, `dev_bootstrap`, `dev_build`, `dev_up`, `dev_down`,
|
||||
`dev_env`. It is the tooling that both reflects and enforces the module model, so it
|
||||
moves with any change to that model.
|
||||
|
||||
`dev_up` builds a dev environment **for a single module**: resolves `requires:`, starts
|
||||
providers "via systemctl (same as production)", provisions workspace-namespaced
|
||||
databases. It borrows the host node's services.
|
||||
|
||||
**There is no way to run a mesh locally.** Consequently anything that only manifests
|
||||
*between* nodes — credential rotation reaching a running session, provision credentials
|
||||
fanning out, cascade ordering, artifact packaging — is discoverable only in production.
|
||||
Every fault fixed on 2026-08-22 was of that kind.
|
||||
|
||||
A containerised mesh (N nodes, a broker, a registry, a coordinator) is therefore not a
|
||||
convenience. It is the precondition for a refactor of this size being verifiable at all.
|
||||
|
||||
## 5. `hal/sdk` is where domains go to hide
|
||||
|
||||
155 files, 34,636 lines. It contains code from essentially every bounded context:
|
||||
|
||||
| file | domain it belongs to |
|
||||
|---|---|
|
||||
| `installer-core.ts` (1,495) | runtime |
|
||||
| `tools/provisions.ts` (1,311) | provisioning |
|
||||
| `tools/deploy-service.ts` (1,081) | deploy |
|
||||
| `workflow-engine.ts` (970), `task-commands.ts` (588) | **work** — noxflow's domain, inside HAL's SDK |
|
||||
| `artifact-manager.ts` (948), `build-executor.ts` (760), `feature-handlers/*` | delivery |
|
||||
| `env-generator.ts` (944) | config |
|
||||
| `amqp-client.ts` (899) | comms |
|
||||
| `meshware.ts` (799) | runtime |
|
||||
| `module-registry.ts` (743) | mesh |
|
||||
| `claude-credentials.ts` | **ai** |
|
||||
|
||||
Nothing here was misplaced carelessly. Each landed in the SDK because the SDK is the one
|
||||
package every module may import — so putting a shared type there is the way to avoid a
|
||||
circular dependency between two modules that both need it.
|
||||
|
||||
The result is that the dependency graph is trivially acyclic and completely
|
||||
uninformative: everything depends on one 34k-line package, so a change anywhere in it
|
||||
rebuilds everything. On 2026-08-22 a three-line fix in `feature-handlers/migrations.ts`
|
||||
produced a 1,445-job, 26-minute mesh-wide cascade.
|
||||
|
||||
The last row is from that same day, and is the clearest illustration: per-agent Claude
|
||||
config-directory helpers were added to `@hal/sdk/claude-credentials.ts` because it was
|
||||
the only place both `hal/claude` and the noxflow runtime could import from. The
|
||||
alternative — `hal/ai` owning it and noxflow depending on `hal/ai` — was unavailable
|
||||
without answering the ownership question this effort exists to answer.
|
||||
|
||||
**Extraction is therefore blocked on the decomposition, not the reverse.** A domain
|
||||
module can only own its shared code once the domain has a module. The SDK should retain
|
||||
only what is genuinely cross-cutting: transport, manifest types, logging, process
|
||||
helpers.
|
||||
|
||||
## 6. The crux: two notions of "agent"
|
||||
|
||||
The original vision was **a node is an agent with thinking abilities**. Later, noxflow
|
||||
implemented actual agents — employees with skills, workspaces and tasks. Both survive,
|
||||
and they collide.
|
||||
|
||||
The collision is visible in the data. There are **two agent rows per node**:
|
||||
|
||||
| agent | display name | skills | bound to |
|
||||
|---|---|---|---|
|
||||
| one named after each node | the node's name | `{deploy,verify,operate}` | its node |
|
||||
| `hal-<node>`, one per node | "HAL (\<node\>)" | `{}` | its node |
|
||||
|
||||
One carries the **work**; the other carries only **identity** — no skills at all, existing
|
||||
to hold a licence for HAL's own sessions (`resolveHalModuleEnv`). The node-as-agent idea
|
||||
was implemented twice, half each.
|
||||
|
||||
The same split appears in the schema: `nodes.hal_claude_account` and
|
||||
`agents.claude_account` are the same fact in two tables in two databases. What was
|
||||
documented as "three licence touchpoints" (NODE, HAL, AGENT) is **one concept modelled
|
||||
three times**.
|
||||
|
||||
### Resolution
|
||||
|
||||
**Nodes and agents are decoupled.** The agents module owns agents; agents run on nodes.
|
||||
A node is a place where an agent can run — that is the whole of the relationship. There
|
||||
is no resident agent, no node-owned identity, no ownership in either direction.
|
||||
|
||||
Agents named after a node still exist, but they are **ordinary agents defined through the
|
||||
agents module** that happen to hold infra skills and be named after the node they usually run
|
||||
on. Nothing about them is structural.
|
||||
|
||||
Two consequences follow:
|
||||
|
||||
1. **`nodes.node_license` and `nodes.hal_claude_account` should not exist.** A node does
|
||||
not authenticate to a model provider — agents do. Both collapse into
|
||||
`agents.claude_account`, and what was documented as three licence touchpoints becomes
|
||||
one.
|
||||
2. **The pair of agent rows per node merges.** `hal-<node>` exists only to hold a licence
|
||||
for HAL's own sessions; once licences belong to agents and any module may employ an
|
||||
agent, it has no reason to be separate from `<node>`.
|
||||
|
||||
What remains unaccounted for is the operator's own interactive session:
|
||||
`~/.claude/.credentials.json` on a node serves a **human at a prompt**, not an agent.
|
||||
Under this decoupling that is not a node property either — it is the operator's
|
||||
credential on whichever machine they are sitting at. Whether the operator is modelled as
|
||||
a persona (soulstream treats humans and agents as peers with identical credentials) is an
|
||||
open question, and the last remaining reason `nodes` carries a licence column today.
|
||||
|
||||
## 7. The record, not "meetings"
|
||||
|
||||
soulstream models work as **one signed log** in which humans and agents apply identical
|
||||
changes, with work arriving by **mention**. Memory and decisions live in that single
|
||||
auditable stream — no hidden state between components. Its stated economics,
|
||||
*"cheap awareness and expensive thinking"*, is the same principle as the budget guard:
|
||||
constant monitoring, selective deep processing.
|
||||
|
||||
Under that model "meetings" is not a domain. A meeting is a **thread in the record**.
|
||||
Which means six of today's modules and table groups are one thing seen from different
|
||||
angles:
|
||||
|
||||
`hal/axon` (DMs, rooms) · `hal/cortex` · `hal/synapse` · `hal/notifications` ·
|
||||
`conversations` · `meetings`
|
||||
|
||||
A notification, in particular, is just a mention not yet read — which is why it needed
|
||||
its own table, its own id column, and its own failure mode.
|
||||
|
||||
## 8. Ideal state
|
||||
|
||||
The mesh repository contains only the mesh, decomposed by domain rather than by
|
||||
deployment accident:
|
||||
|
||||
| context | what it is | absorbs today's |
|
||||
|---|---|---|
|
||||
| `hal/infra` | **the room** — nodes, registry, provisioning, pipeline, node runtime | `hal/mesh`, `hal/brain`, `hal/coordinator`, `hal/meshware`, `hal/developer`, `hal/bootstrap` |
|
||||
| `hal/agents` | **the name and the thinking** — Agent owns identity, licence, runs, memory, thoughts | noxflow agents (13 tables), `hal/thoughts` |
|
||||
| `hal/stream` | **the record** — mentions, threads, meetings, notifications | `hal/axon`, `hal/cortex`, `hal/synapse`, `hal/notifications`, meetings (7), conversations |
|
||||
| `hal/knowledge` | documents, spaces, revisions, review, search | `hal/hippocampus` + noxflow `knowledge_*` |
|
||||
| `hal/work` | tasks, workflows, boards — the original noxflow | noxflow tasks (7 tables) |
|
||||
| `hal/ai` | provider integration, flavored per vendor | `hal/claude*`, `claude-code` |
|
||||
| `hal/config` | env and config distribution, secrets, PKI | `hal/env-sync`, `hal/config-sync`, `hal/secrets`, `mesh-ca` |
|
||||
| `hal/observability` | health, dashboards, logs | `hal/health`, `hal/meshboard` |
|
||||
|
||||
Eight contexts instead of 33 platform modules, with the catalogue elsewhere.
|
||||
|
||||
The naming corrects itself in the process: what is called "brain" today is
|
||||
infrastructure, and the word survives only as a concept an Agent owns — never again as a
|
||||
module name.
|
||||
|
||||
This shape depends on **named features with per-node opt-in** — without it, every
|
||||
independently-deployable unit inside a context becomes a module again and the count
|
||||
returns. That mechanism is the subject of a separate ADR.
|
||||
|
||||
## Decided since
|
||||
|
||||
| question | decision |
|
||||
|---|---|
|
||||
| Meetings — own context or part of agents? | **Neither.** A meeting is a thread in the record; `hal/stream` absorbs it. The category was wrong. |
|
||||
| Thoughts — autonomy or agent behaviour? | `hal/agents`. Any agent can have a thought. |
|
||||
| `hal/axon` vs `hal/cortex` | `hal/cortex` is vestigial — no `daemon/` source, unit inactive on every node, description copy-pasted from axon. Messaging belongs to `hal/stream`. |
|
||||
| `hal/hypothalamus` | Orphaned build output. A `dist/` directory with no `module.yml`, so not a module at all. |
|
||||
| Who owns agent identity? | `hal/agents`. Nodes and agents are decoupled; a node is only a place an agent can run. |
|
||||
| Where do third-party modules live? | Outside this repository. They run *on* the mesh, not *of* it. |
|
||||
|
||||
## Open questions
|
||||
|
||||
1. **`hal/scheduler`** — infrastructure (fire an event later, belongs to `hal/mesh`) or
|
||||
part of `hal/work`?
|
||||
2. **The executor.** Pulling and routing work items is `hal/work`; spawning a session is
|
||||
`hal/agents`; providing the place to run is `hal/mesh`. It currently straddles all
|
||||
three, which is the same kind of straddle that put credential-writing in the noxflow
|
||||
runtime.
|
||||
3. **Catalogue destination** — one repository, or per-application repositories, given the
|
||||
pipeline resolves dependencies across the registry rather than the filesystem?
|
||||
4. **SDK residue** — after extraction, does `hal/sdk` keep transport (`amqp-client`), or
|
||||
does that belong to `hal/stream`? Everything imports it, which argues both ways.
|
||||
5. **Human agent modality.** ADR 0001 requires a fact the mesh does not record: which
|
||||
user, on which node, a human agent acts as. Where does it live — an attribute of the
|
||||
agent, or of the agent-node binding?
|
||||
@@ -0,0 +1,59 @@
|
||||
---
|
||||
status: graduated
|
||||
initiated: 2026-08-22
|
||||
touches: [03-DESIGN/00-as-is/04-delivery.md, 03-DESIGN/00-as-is/05-runtime-and-installation.md]
|
||||
became: [03-DESIGN/01-to-be/01-end-to-end-testing.md, 02-DECISIONS/0016-the-lab.md]
|
||||
---
|
||||
|
||||
# 002 — A mesh that runs locally
|
||||
|
||||
- **Initiated by:** jochen, 2026-08-22
|
||||
- **Areas touched:** `install.d/`, `hal/meshware`, `hal/coordinator`, `hal/brain`,
|
||||
`hal/developer` (`dev_up`), `hal/sdk` (env generation, feature handlers, artifact
|
||||
manager), `test/pipeline/`, `test/dev-mesh/`, the provisioning path in
|
||||
`modules/postgres/`.
|
||||
|
||||
## Summary
|
||||
|
||||
Phase 0 of [`03-DESIGN/00-work-breakdown.md`](../../03-DESIGN/01-to-be/00-work-breakdown.md) requires
|
||||
a mesh that comes up in containers, runs its own pipeline, and reproduces known faults on
|
||||
demand. Nothing else in the decomposition starts until it exists, because every fault the
|
||||
decomposition addresses was found in production — there was nowhere else to find it.
|
||||
|
||||
This effort establishes what already runs in a container, what is welded to the host, and
|
||||
what it would take to close the gap. It does **not** choose an approach: the central
|
||||
question — how a containerised node executes a module service, when a module service is
|
||||
defined today as a systemd unit shelling to `docker compose` in `/services/` — is not
|
||||
answered by ADR 0001 and is recorded below rather than decided.
|
||||
|
||||
## What was established
|
||||
|
||||
- The two existing container harnesses are **neither of them a mesh**, and one of them has
|
||||
not been able to build since 2026-06-04.
|
||||
- Node identity is already portable — a single environment variable, no host handshake.
|
||||
- Four concrete host couplings block a containerised node, all with known locations.
|
||||
- All three Phase 0 fixtures are reproducible; one of them is documented in the knowledge
|
||||
base with an open root cause and is the cheapest place to start.
|
||||
|
||||
Detail and evidence in [`analysis.md`](analysis.md).
|
||||
|
||||
## Questions — all settled 2026-08-22
|
||||
|
||||
**Phase 0 is a development environment**, not a fixture rig — it is what the host-borrowing
|
||||
dev tooling becomes.
|
||||
|
||||
**The trigger is a real source-forge container**, because the webhook relay is part of what
|
||||
is under test.
|
||||
|
||||
**A node is a system container**, promotable to a virtual machine per node. This answered
|
||||
the question the effort was stuck on, and dissolved it rather than solving it: against a
|
||||
real node with a real init, the four host couplings catalogued in `analysis.md` §3 are not
|
||||
couplings — they are how a node works. They were obstacles only to a node modelled as an
|
||||
application container.
|
||||
|
||||
Consequently [`003-service-supervision`](../003-service-supervision/) **no longer blocks
|
||||
Phase 0**. It remains a live architecture question, on its own timeline.
|
||||
|
||||
The remaining questions in `analysis.md` — what replaces host paths in a container, and how
|
||||
faithful the lab must be — are answered in the design: nothing replaces them, because the
|
||||
paths are real; and the divergences are enumerated rather than discovered.
|
||||
@@ -0,0 +1,263 @@
|
||||
---
|
||||
effort: 002-local-mesh
|
||||
updated: 2026-08-22
|
||||
---
|
||||
|
||||
# A mesh that runs locally — current state and obstacles
|
||||
|
||||
Evidence for Phase 0. Every claim here is either a file location or something measured on
|
||||
2026-08-22; where a claim was checked and found false, that is recorded too.
|
||||
|
||||
---
|
||||
|
||||
## 1. What exists today, and why neither is a mesh
|
||||
|
||||
### `test/pipeline/` — the only containerised HAL node, and it is dead
|
||||
|
||||
This harness builds `@hal/sdk` and the `hal/brain` daemon into a `node:23-alpine` image,
|
||||
compiles `hal/coordinator`'s tools against it, and runs `brainstem.js` with
|
||||
`HAL_MODE=daemon` against a containerised postgres and LavinMQ
|
||||
(`test/pipeline/Dockerfile`, `test/pipeline/docker-compose.yml`). It proves the valuable
|
||||
thing: **a node runtime needs no systemd, no `/services/`, and no nvm** — environment
|
||||
variables and a seeded `nodes` row are sufficient.
|
||||
|
||||
It also cannot build. Measured 2026-08-22:
|
||||
|
||||
```
|
||||
Step 7/16 : RUN npm run build -w modules/hal/sdk
|
||||
npm error No workspaces found:
|
||||
npm error --workspace=modules/hal/sdk
|
||||
```
|
||||
|
||||
The npm workspace it depends on was removed on 2026-06-04 (`21ef4a4e`, "kill npm
|
||||
workspace"), and the root `package.json` now carries a comment explaining why it will not
|
||||
come back. The harness has not been touched since before that commit.
|
||||
|
||||
So the repository's only end-to-end pipeline test has been unrunnable for two and a half
|
||||
months and nothing reported it — which is the same shape as the faults Phase 0 exists to
|
||||
catch. **A test nobody runs is indistinguishable from a test that passes.**
|
||||
|
||||
### `test/dev-mesh/` — a work plane, not a control plane
|
||||
|
||||
Four `runtime-*` services, a noxflow API, a meshboard and a shared postgres + LavinMQ on
|
||||
one network (`test/dev-mesh/docker-compose.yml`). It is genuinely useful and the topology
|
||||
is a good precedent: one network, per-service `HAL_NODE`, `EXECUTOR_STUB=1` so dispatch and
|
||||
run lifecycle are real while the model call is faked.
|
||||
|
||||
It is the wrong layer for Phase 0, on three counts:
|
||||
|
||||
| | dev-mesh does | Phase 0 needs |
|
||||
|---|---|---|
|
||||
| state | restores `pg_dump`s (`test/dev-mesh/init.sh`) | schema built by **migrations**, or fixture 3 is untestable by construction |
|
||||
| code | bind-mounts pre-built `dist/` from the host | artifacts **built and delivered** by the pipeline |
|
||||
| scope | noxflow runtime, API, meshboard | `hal/coordinator`, `hal/meshware`, MinIO, provisioning |
|
||||
|
||||
No meshware, no coordinator, no artifact store, no provisioning, no build. It exercises
|
||||
what the mesh *runs*; Phase 0 needs what the mesh *is*.
|
||||
|
||||
### `dev_up` — confirmed host-coupled
|
||||
|
||||
The claim in the work breakdown holds. `startDevProvider()` starts providers through the
|
||||
host's user systemd instance and reads credentials from a host path
|
||||
(`modules/hal/developer/tools/dev-env.ts:146-178`):
|
||||
|
||||
```ts
|
||||
const unitName = `hal-module@${provider}.service`;
|
||||
execSync(`systemctl --user start ${unitName}`, { timeout: 60_000, stdio: "pipe" });
|
||||
const creds = resolveRunningProviderCreds(provider); // reads /services/<provider>/.env
|
||||
```
|
||||
|
||||
It borrows the host. There is no mesh to stand up.
|
||||
|
||||
---
|
||||
|
||||
## 2. What is already portable
|
||||
|
||||
- **Node identity is one environment variable.** `resolveNodeName()` is
|
||||
`process.env.HAL_NODE || hostname()`, lowercased (`modules/hal/sdk/src/mesh-config.ts:215`).
|
||||
No file, no registration handshake, no host coupling.
|
||||
- **Registration is a plain upsert.** `mesh_node_register` inserts into `nodes`
|
||||
(`modules/hal/mesh/tools/index.ts:490-540`); mandatory fields are `name` and `user_name`.
|
||||
- **The daemons are AMQP loops.** `hal/meshware` consumes `cmd.feature.{build,install,configure,start}`
|
||||
(`modules/hal/meshware/daemon/src/cerebellum.ts:780-857`); `hal/coordinator` consumes
|
||||
`event.gitea.push` and drives the cascade. Both are configured entirely by environment.
|
||||
- **`hal/brain` in daemon mode already runs in a plain node image**, per `test/pipeline/`.
|
||||
|
||||
---
|
||||
|
||||
## 3. The obstacles, located
|
||||
|
||||
Four couplings stand between the daemons and a container. All are in code, none are
|
||||
mysterious.
|
||||
|
||||
| # | coupling | location |
|
||||
|---|---|---|
|
||||
| 1 | meshware restarts itself via the host: `execFileSync("systemctl", ["--user","restart","hal-meshware.service"])` | `modules/hal/meshware/daemon/src/cerebellum.ts:826` |
|
||||
| 2 | meshware restarts the broker via a hardcoded host path: `execFileSync("docker",["compose","-f","/services/lavinmq/docker-compose.yml","restart"])` | `cerebellum.ts:843-844` |
|
||||
| 3 | **a module service *is* a systemd unit** — `hal-module@.service` runs `docker compose --project-directory /services/%i` | `modules/hal/meshware/systemd/hal-module@.service:10-11` |
|
||||
| 4 | env generation writes `homedir()`-relative host paths — `~/.config/hal/env`, `~/.config/hal/modules/*.env`, `/services/*/.env` | `modules/hal/sdk/src/env-generator.ts:188-223`, `:538-568`, `:584-606` |
|
||||
|
||||
1, 2 and 4 are small: a guard and a configurable root. **3 is the architectural one**, and
|
||||
it is the first open question below — in a container, "systemd unit that runs docker
|
||||
compose" has no natural translation.
|
||||
|
||||
The bootstrap scripts add their own host assumptions — nvm for node resolution, `sudo
|
||||
mkdir -p /services && chown`, and a `~/dotfiles` repository that is not in this repo
|
||||
(`install.d/adopt.sh:165-170`) — but Phase 0 does not have to run them. A container image
|
||||
can be built the way `test/pipeline/Dockerfile` builds one, bypassing the bootstrap path
|
||||
entirely. That is a deliberate divergence to record: **the local mesh would not test the
|
||||
bootstrap**, only the running mesh.
|
||||
|
||||
---
|
||||
|
||||
## 4. External services, and whether they can be local
|
||||
|
||||
| service | used for | local substitute |
|
||||
|---|---|---|
|
||||
| PostgreSQL | registry + pipeline state | yes — already the pattern in both harnesses |
|
||||
| LavinMQ | all coordinator↔meshware messaging | yes — `cloudamqp/lavinmq`, already used |
|
||||
| MinIO | per-feature artifact tarballs, `modules/{name}/{version}/{feature}.tar.gz` (`modules/hal/sdk/src/artifact-manager.ts:21,65`) | yes — `minio/minio`, needs only `REGISTRY_MINIO_*` |
|
||||
| Gitea | source, push webhook, **and** the `@hal/*` npm registry | container exists, but heavy; see open question 2 |
|
||||
| Docker registry | `docker push` for modules declaring `docker:` (`build-executor.ts:195-231`) | `registry:2`, or avoid modules that need it |
|
||||
| Traefik | writes routing config at install; does not gate success | omit |
|
||||
|
||||
Only Gitea is awkward, and only because it carries two roles at once.
|
||||
|
||||
---
|
||||
|
||||
## 5. The fixtures
|
||||
|
||||
Phase 0.5 asks for at least one reproducible known fault. All three are reachable; they
|
||||
differ sharply in cost.
|
||||
|
||||
### Fixture B — a provider deploy rotates a shared credential without fanning out
|
||||
|
||||
**Cheapest, best understood, and the root cause is still open.** Documented at
|
||||
`troubleshooting/provision-adoption-rotates-live-credential`. The mechanism is two lines:
|
||||
|
||||
```ts
|
||||
async provision(project, username, options) {
|
||||
const password = generatePassword(); // ALWAYS a fresh password
|
||||
```
|
||||
|
||||
and, in the adoption branch of `provisionDatabase`, an `ALTER ROLE ... WITH PASSWORD` that
|
||||
rotates the live secret while updating only the provider's own `mesh_provisions` row
|
||||
(`modules/postgres/tools/index.ts`). Every consumer sharing that role keeps a stale
|
||||
password and fails permanently; the provider recovers alone, and that asymmetry is the
|
||||
tell.
|
||||
|
||||
Reproduction needs one provider node and two consumer nodes — the minimum interesting
|
||||
mesh. It re-fires on every provider deploy, so it does not need to be provoked, only
|
||||
observed. Confirmation is a timestamp comparison, not a hash: the stored values are
|
||||
`enc:v1:` with a random IV, so identical plaintexts hash differently.
|
||||
|
||||
Measured in production 2026-08-22: three nodes failing since 2026-08-20 14:15, 399 errors
|
||||
each; the provider failed for 22 minutes and recovered by itself.
|
||||
|
||||
### Fixture C — a migration ships nothing while the pipeline reports success
|
||||
|
||||
Three independent silent-skip points, any one of which produces it:
|
||||
|
||||
1. **Not-applicable and succeeded are the same status.** `MigrationsHandler.detect()` is
|
||||
`existsSync(join(moduleDir, "migrations"))` against the freshly cloned workspace
|
||||
(`modules/hal/sdk/src/feature-handlers/migrations.ts:32-41`). If the directory is not
|
||||
there — uncommitted, ignored, or a `module_path` that does not line up — the build
|
||||
reports `success` (`cerebellum.ts:634-638`).
|
||||
2. **Upload failure is a warning.** `uploadFeatureArtifact()` returns `{sha256: ""}` with a
|
||||
`warn` when none of the listed files exist (`artifact-manager.ts:41-44`), and callers
|
||||
catch and warn rather than throw (`cerebellum.ts:672-673`). The symmetric download
|
||||
failure on the target node is also non-fatal (`artifact-manager.ts:93-98`,
|
||||
`cerebellum.ts:426-431`, commented "Non-fatal: feature may work without artifact").
|
||||
3. **Missing packaged files are skipped in silence.** `addFileEntries()` does
|
||||
`if (!existsSync(abs)) continue;` (`module-builder-core.ts:134-135`) — an explicit
|
||||
`package:` entry that is not on disk simply is not in the tarball.
|
||||
|
||||
This is the same class as `troubleshooting/flavors-never-packaged` (`flavors/` was never
|
||||
staged, so a node kept its first copy forever) and is catalogued in
|
||||
`troubleshooting/deploy-reports-transport-not-effect`, which counted **0 of 119 modules
|
||||
implementing `verify`** — the stage that exists, is dispatched, has a working handler, and
|
||||
would turn every one of these into a red pipeline.
|
||||
|
||||
### Fixture A — a credential rotation does not reach a running session
|
||||
|
||||
The most valuable and the most work: it needs a *running session* to rotate underneath,
|
||||
which means the local mesh must be able to start one. The mechanism is understood — an
|
||||
`EnvironmentFile` is read once and `process.env` is a snapshot — and it is why credentials
|
||||
became files that get re-read. Defer it behind B and C.
|
||||
|
||||
**Recommendation:** B first. It needs three nodes and a provision, no build and no session,
|
||||
and it is the one whose root cause is still open — so reproducing it locally has value
|
||||
beyond proving the harness.
|
||||
|
||||
---
|
||||
|
||||
## 6. A claim that was checked and found false
|
||||
|
||||
The survey behind this document suspected that the live AMQP pipeline never writes
|
||||
`deployments` or `node_modules.installed_version`, because the code that does so sits in
|
||||
`installer-core.ts:1045-1126` on what is commented as the "CLI path".
|
||||
|
||||
Measured against production, 2026-08-22 17:30 UTC — deployments in the preceding 24 hours:
|
||||
|
||||
| node | deploys | newest |
|
||||
|---|---|---|
|
||||
| 1 | 24 | 17:26:17 |
|
||||
| 2 | 27 | 17:22:22 |
|
||||
| 3 | 47 | 17:25:54 |
|
||||
| 4 | 26 | 17:22:23 |
|
||||
|
||||
The record is live on all four nodes and minutes old. **The claim is refuted.** Recorded
|
||||
because the reasoning was plausible and someone will retrace it.
|
||||
|
||||
---
|
||||
|
||||
## 7. Questions
|
||||
|
||||
### Settled (jochen, 2026-08-22)
|
||||
|
||||
**Phase 0 is a development environment, not a fixture rig.** It is built as a supported
|
||||
surface to work in daily, and it is what `dev_up` should become. The definition of done —
|
||||
"demonstrated in the local mesh" — is therefore meant literally.
|
||||
|
||||
**The trigger is a real Gitea container.** The webhook relay is part of what is under test,
|
||||
including the changed-file enrichment that exists because the webhook truncates at 20
|
||||
commits (`modules/hal/gitea/tools/index.ts:136-195`). A synthetic `event.gitea.push` would
|
||||
skip it. Gitea also carries the `@hal/*` npm registry, so the local mesh needs it twice
|
||||
over either way.
|
||||
|
||||
### Open — blocking
|
||||
|
||||
1. **How does a node supervise a module service?** Today a module service is a systemd unit
|
||||
running `docker compose` against `/services/<module>`
|
||||
(`modules/hal/meshware/systemd/hal-module@.service:10-11`), which has no direct
|
||||
translation inside a container. The question raised in response is the better one, and
|
||||
is broader than Phase 0: **what would it cost to stop using systemd altogether and have
|
||||
the mesh supervise its own services?** That is a mesh-level architectural question, not
|
||||
a local-mesh implementation detail — if the answer is that the mesh should own
|
||||
supervision, Phase 0 should not build a container-only workaround first. Under
|
||||
investigation; findings will land in [`003-service-supervision`](../003-service-supervision/)
|
||||
and, if it goes ahead, a decision record of its own.
|
||||
|
||||
2. **What replaces `~/.config/hal/env` and `/services/` inside a container?** A configurable
|
||||
root keeps one code path; container-specific targets keep the host paths untouched. This
|
||||
decides whether env generation gets a seam or a conditional.
|
||||
|
||||
3. **How faithful must the local mesh be to be trusted?** It will not run the bootstrap
|
||||
scripts, and it will run one OS where the real mesh is heterogeneous by design
|
||||
(`00-META/context.md`). Stating the divergence up front is what stops "it works
|
||||
locally" from becoming its own class of silent failure. Now sharper, because a
|
||||
development environment people use daily is trusted far more than a rig — and drifting
|
||||
from production costs correspondingly more.
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- [`02-DECISIONS/0001`](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md) — the decision this
|
||||
phase unblocks
|
||||
- [`03-DESIGN/00-work-breakdown.md`](../../03-DESIGN/01-to-be/00-work-breakdown.md) — Phase 0 tasks
|
||||
and checkpoint
|
||||
- `troubleshooting/provision-adoption-rotates-live-credential` — fixture B, root cause open
|
||||
- `troubleshooting/deploy-reports-transport-not-effect` — the nine defects that shipped
|
||||
green, and the unused `verify` stage
|
||||
- `troubleshooting/flavors-never-packaged` — fixture C, previously seen in the wild
|
||||
@@ -0,0 +1,79 @@
|
||||
---
|
||||
status: graduated
|
||||
initiated: 2026-08-22
|
||||
touches: [03-DESIGN/00-as-is/05-runtime-and-installation.md]
|
||||
became:
|
||||
- 02-DECISIONS/0005-the-node-host.md
|
||||
- 03-DESIGN/01-to-be/05-the-node-host.md
|
||||
---
|
||||
|
||||
# 003 — Who supervises a service
|
||||
|
||||
**No longer blocks Phase 0** (see below).
|
||||
- **Initiated by:** jochen, 2026-08-22, in response to
|
||||
[`002-local-mesh`](../002-local-mesh/analysis.md) open question 1
|
||||
- **Areas touched:** every module shipping a `systemd/` directory (14), the
|
||||
`hal-module@` template, `hal/sdk` feature handlers, `dev_up`, `log_tail` /
|
||||
`systemd_journal`, the bootstrap scripts.
|
||||
|
||||
## The question
|
||||
|
||||
`002` asked how a containerised node runs a module service, given that a module service is
|
||||
defined today as a systemd unit running `docker compose` against `/services/`. The response
|
||||
was the better question:
|
||||
|
||||
> If it's possible to run systemd inside a container, that's the way to go I think. However,
|
||||
> what would the cost be to step away from systemd to run our services and set it up in a
|
||||
> different way? More hal mesh approach.
|
||||
|
||||
This effort answers the cost half. It does not choose.
|
||||
|
||||
## Summary of findings
|
||||
|
||||
- **The question splits in two**, and the halves have opposite answers. Supervising
|
||||
`docker compose` stacks through systemd is largely **redundant** — 44 of 44 module compose
|
||||
files already declare a restart policy, so Docker is already the supervisor. Supervising
|
||||
HAL's **9 long-running Node daemons** is not redundant: `Restart=always` is currently the
|
||||
only thing between a crash and a dead node.
|
||||
- **The hard part is fate-sharing, not systemd.** meshware already cannot restart itself and
|
||||
carries a documented workaround for it. Any mesh-native supervisor inherits that problem
|
||||
recursively unless it sits outside the mesh's own process tree — at which point it is an
|
||||
OS-level supervisor again, just reinvented.
|
||||
- **There is a third option neither of us named**, and it is the one that also solves Phase 0:
|
||||
run HAL's own daemons as containers, making Docker the supervisor for everything. Local and
|
||||
production then have the same shape rather than a translation layer between them.
|
||||
- **It cannot be all-or-nothing**, and ADR 0001 already says why: a human agent acts through a
|
||||
shell and a desktop. Those parts are on the host by definition.
|
||||
- One incidental finding: the automatic node rescue that documentation describes **does not
|
||||
exist**. No unit declares `OnFailure=`, and nothing calls `hal-rescue.sh` on a timer.
|
||||
|
||||
Detail and costs in [`analysis.md`](analysis.md).
|
||||
|
||||
## What it became
|
||||
|
||||
*Closed 2026-08-28.* The decision this effort asked for was taken — and taken without citing it,
|
||||
which is why the effort sat `active` for five days after being answered. Recorded here because
|
||||
finding that is the point of a sweep.
|
||||
|
||||
**The third option is what the mesh adopted.** `Docker is the supervisor for everything` is
|
||||
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md): the
|
||||
host is a plain process on the machine and everything above tier 0 is a container. The substrate
|
||||
bootstrap declares no service at all — it is package, container, action, container — so the
|
||||
44-of-44 restart policies this effort counted are the supervision, exactly as it argued.
|
||||
|
||||
**Fate-sharing was the hard part, and it is solved the way this effort predicted.** It said any
|
||||
mesh-native supervisor inherits the problem *unless it sits outside the mesh's own process
|
||||
tree*. [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) puts
|
||||
the launcher there: it supervises the host as a child and shares no code with it, so a host that
|
||||
cannot start is still recovered.
|
||||
|
||||
**It is not all-or-nothing, as this effort insisted.** A human agent acts through a shell and a
|
||||
desktop, and those are on the host. So is the host itself — the one thing an init starts.
|
||||
|
||||
## What is not closed
|
||||
|
||||
**The automatic node rescue the documentation describes does not exist.** No unit declares
|
||||
`OnFailure=`, and nothing calls the rescue script on a timer. That is a documented behaviour
|
||||
which never happens, and it outlives this effort — filed as
|
||||
[`04-ISSUES/008`](../../04-ISSUES/008-the-documented-node-rescue-does-not-exist/00-report.md)
|
||||
rather than closed with it.
|
||||
@@ -0,0 +1,230 @@
|
||||
---
|
||||
effort: 003-service-supervision
|
||||
updated: 2026-08-22
|
||||
---
|
||||
|
||||
# Who supervises a service — the cost of leaving systemd
|
||||
|
||||
Measured 2026-08-22. Every claim is a file location or a count.
|
||||
|
||||
---
|
||||
|
||||
## 1. What systemd actually does for HAL
|
||||
|
||||
**14 modules** ship a `systemd/` directory. The units divide cleanly into three kinds:
|
||||
|
||||
| kind | count | what it is |
|
||||
|---|---|---|
|
||||
| long-running daemons | 9 | `hal/brain`, `hal/meshware`, `hal/coordinator`, `hal/cortex`, `hal/env-sync`, `hal/file-share`, `hal/thoughts`, `noxflow/runtime`, `desktop_notifications` — all `Restart=always`, `RestartSec=10` |
|
||||
| scheduled one-shots | 6 timers | docker-prune, docker-registry maintenance, claude-code sessions + usage, health, mailu cert-sync |
|
||||
| the template | 1 | `hal-module@.service` — the per-module Docker lifecycle |
|
||||
|
||||
Only `mailu` is system-scope. Everything else is a user unit under
|
||||
`~/.config/systemd/user`, auto-detected from directory presence — the module author names
|
||||
the file and that filename *is* the unit name
|
||||
(`modules/hal/sdk/src/feature-handlers/systemd.ts:31-42`).
|
||||
|
||||
The template is the piece `002` tripped over
|
||||
(`modules/hal/meshware/systemd/hal-module@.service`):
|
||||
|
||||
```
|
||||
ExecStart=docker compose --project-directory /services/%i --project-name %i up --remove-orphans --pull always
|
||||
ExecStop=docker compose --project-directory /services/%i --project-name %i down
|
||||
Restart=on-failure
|
||||
StartLimitIntervalSec=120
|
||||
StartLimitBurst=5
|
||||
```
|
||||
|
||||
Systemd's roles, then: restart-on-failure, start at login, env-file loading, ordering,
|
||||
per-module lifecycle, timers, and — via journald — **the only log store the 9 Node daemons
|
||||
have** (`modules/hal/sdk/src/tools/log-tail.ts:50-53`).
|
||||
|
||||
---
|
||||
|
||||
## 2. The question splits in two, and the halves disagree
|
||||
|
||||
### For `docker compose` stacks, systemd is mostly redundant
|
||||
|
||||
**44 of 44** module `docker-compose.yml` files declare a container-level `restart:` policy —
|
||||
`always` or `unless-stopped`, none missing. Docker's own daemon already restarts crashed
|
||||
containers, independently of systemd.
|
||||
|
||||
`hal-module@.service` does not supervise the containers. It supervises the **`docker compose
|
||||
up` foreground process**. It is a second layer on top of a restart policy that already
|
||||
works. What it genuinely adds is narrower than it looks:
|
||||
|
||||
- one uniform verb for every module type — `systemctl --user start hal-module@X` rather than
|
||||
remembering each module's compose invocation, relied on across a dozen call sites in
|
||||
`meshware.ts:124-154`, `dev-env.ts:142-163`, `installer-core.ts`, `infra.ts:166`
|
||||
- recovery when the `docker compose up --pull always` process itself dies — a failed image
|
||||
pull, not a crashed container
|
||||
- rate-limited restart (`StartLimitBurst=5`) so a broken stack does not spin
|
||||
|
||||
That is real, but it is a convenience layer, not a safety layer. **This half could go at
|
||||
moderate cost.**
|
||||
|
||||
### For the 9 Node daemons, systemd is load-bearing
|
||||
|
||||
There is no alternative supervisor anywhere in the repo. No PM2, no forever, no nodemon, no
|
||||
watchdog loop — all checked, zero hits. `Restart=always` is the only thing standing between
|
||||
a crashed daemon and a dead node.
|
||||
|
||||
**This half is the actual question.**
|
||||
|
||||
---
|
||||
|
||||
## 3. The hard part is fate-sharing
|
||||
|
||||
The difficulty is not systemd. It is that **a supervisor must not share fate with what it
|
||||
supervises**, and the codebase already has a scar from exactly this.
|
||||
|
||||
meshware cannot restart itself mid-request: killing the process before it ACKs the AMQP
|
||||
message loses the message. So it defers its own restart by two seconds after closing the
|
||||
connection (`modules/hal/meshware/daemon/src/cerebellum.ts:815-828`) — a commented
|
||||
workaround for a problem that only exists because the thing being restarted is the thing
|
||||
doing the restarting.
|
||||
|
||||
Any mesh-native supervisor inherits this recursively. Something has to be the outermost
|
||||
always-alive process, and if it is written by the mesh, the mesh must supervise it, and so
|
||||
on. The recursion only terminates at a process the mesh does not own. Today that is
|
||||
systemd. **A "more mesh" supervisor that is itself a mesh process is not a smaller problem;
|
||||
it is the same problem with a new name.**
|
||||
|
||||
This is the strongest argument for the status quo, and it is worth stating plainly before
|
||||
looking at alternatives.
|
||||
|
||||
---
|
||||
|
||||
## 4. The option neither of us named
|
||||
|
||||
There is a third answer that terminates the recursion in something that is not systemd and
|
||||
not written by us: **run HAL's own daemons as containers.**
|
||||
|
||||
Docker is already the outermost supervisor for 44 of 44 module stacks. It does not share
|
||||
fate with the mesh. It already has restart policies, backoff, and a log store that
|
||||
`log_tail` already speaks (`log-tail.ts:43-47` reads `docker logs` for Docker modules
|
||||
today). Extending it from "the things the mesh runs" to "the mesh itself" is not new
|
||||
machinery — it is applying machinery the mesh already trusts to one more case.
|
||||
|
||||
What that buys, beyond supervision:
|
||||
|
||||
- **Phase 0 stops being a translation.** A containerised node becomes the same shape as a
|
||||
production node, rather than a local approximation with a systemd-shaped hole in it. The
|
||||
divergence that `002` open question 3 worries about largely disappears.
|
||||
- **`/services/` and `~/.config/hal/` stop being special.** Mounts, not host paths.
|
||||
- **The GENESIS "dogfood everything" value gets easier**, not harder: the mesh's own
|
||||
components would ship and run exactly like everything else it carries.
|
||||
|
||||
What it costs, honestly:
|
||||
|
||||
- **journald → docker logs** for the 9 daemons. `log_tail` already handles both, but
|
||||
`systemd_journal` and the health checks that read unit state
|
||||
(`modules/hal/mesh/health.sh:34-40`, `modules/hal/health/hal-health.sh:335-395`) would
|
||||
need a container-aware path.
|
||||
- **Boot start** becomes Docker's `restart: always` plus the Docker daemon being enabled at
|
||||
boot — which is still one systemd unit, but the OS's own, not ours.
|
||||
- **A container needs the host to be reachable** for anything that touches the node itself.
|
||||
Some of these daemons exist precisely to write host files.
|
||||
|
||||
---
|
||||
|
||||
## 5. Why it cannot be all-or-nothing — and ADR 0001 already says so
|
||||
|
||||
Some of what runs under systemd today **cannot** be containerised, and the reason is
|
||||
already in the domain model. ADR 0001:
|
||||
|
||||
> a non-human agent acts through a spawned session — a human agent acts through a shell or
|
||||
> desktop
|
||||
|
||||
`hal/brain` has two modes (`modules/hal/brain/daemon/src/brainstem.ts:7-13`): a daemon mode
|
||||
that is an AMQP relay, and a **cortex mode that is an MCP server over stdio for an
|
||||
interactive Claude Code session**. The second is a human agent's modality. It runs in the
|
||||
human's shell, on the human's node, against the human's `~/.claude`. Containerising it is
|
||||
not a hard engineering problem, it is a category error.
|
||||
|
||||
The same holds for `desktop_notifications` and everything `hal/desktop-environment` touches.
|
||||
|
||||
So the line is not "systemd or not". It is:
|
||||
|
||||
| | belongs where |
|
||||
|---|---|
|
||||
| mesh daemons — meshware, coordinator, env-sync, thoughts, file-share, brain **in daemon mode** | supervisable by Docker; candidates to containerise |
|
||||
| human-modality surfaces — brain **in cortex mode**, desktop notifications, desktop environment | on the host, by definition |
|
||||
| module stacks | already Docker; systemd layer is the redundant part |
|
||||
|
||||
This split is not a compromise between the options. It is what the domain model implies,
|
||||
and it is a decent sign that the model is doing work.
|
||||
|
||||
---
|
||||
|
||||
## 6. Options, with costs
|
||||
|
||||
| | option | cost | what it buys |
|
||||
|---|---|---|---|
|
||||
| **A** | **Keep systemd; systemd-in-container for Phase 0** | privileged containers, heavy images, slow iteration; local mesh keeps a shape production does not have | nothing changes in production; smallest change to the mesh |
|
||||
| **B** | **Drop the `hal-module@` layer only** — let Docker's restart policies supervise stacks directly | reimplement uniform start/stop across ~12 call sites; lose rate-limited restart and pull-failure recovery | removes the redundant layer; does **not** solve Phase 0 on its own, since the daemons still need supervising |
|
||||
| **C** | **Containerise the mesh daemons; Docker supervises** | container-aware `systemd_journal`/health; host access for daemons that write host files; the human-modality surfaces stay on the host regardless | terminates the fate-sharing recursion without writing a supervisor; makes local and production the same shape; Phase 0 becomes much less of a special case |
|
||||
| **D** | **Write a mesh-native supervisor** | the fate-sharing recursion (§3), plus matching systemd's maturity — backoff, resource limits, clean SIGTERM (`noxflow/runtime` already depends on `TimeoutStopSec=60`) — on machines that are somebody's daily driver | most "mesh"; least justified by the evidence |
|
||||
|
||||
**D is the option the phrasing "more hal mesh approach" points at, and the evidence argues
|
||||
against it.** Supervision is not a domain concern the mesh is better placed to solve than
|
||||
the OS; the mesh's distinguishing feature is brokering capabilities, not restarting
|
||||
processes. C gets the benefit D is reaching for — the mesh not depending on host-specific
|
||||
init — without the recursion.
|
||||
|
||||
**C and B compose.** C is the one that pays for Phase 0.
|
||||
|
||||
Worth noting what is *not* in this table: the dozens of `systemctl` call sites that
|
||||
configure the **host OS's own** units — NetworkManager, resolved, oomd, zram, docker.service,
|
||||
fail2ban, sshd, zfs, across the vendor-firmware and power modules, `wireguard`, `sshd`, `zfs`
|
||||
and others. Those are not HAL supervising itself; they are HAL configuring an Arch box. They
|
||||
are out of scope for every option above and do not go away under any of them.
|
||||
|
||||
---
|
||||
|
||||
## 7. An incidental finding
|
||||
|
||||
Documentation describes an automatic node rescue: `hal-rescue.sh:20` states it is "triggered
|
||||
automatically by `hal-health.timer` when hal-meshware is failed".
|
||||
|
||||
**It is not.** `hal-health.sh` contains no call to `hal-rescue.sh` (checked, zero matches),
|
||||
and **no unit in the repository declares `OnFailure=`** (checked, zero matches). The only
|
||||
real triggers are the manual `rescue_node` tool
|
||||
(`modules/hal/sdk/src/tools/deploy-rescue.ts:77-92`) and running `install.d/rescue.sh` by
|
||||
hand.
|
||||
|
||||
This is worth recording for two reasons. It weakens any argument that systemd-adjacent
|
||||
self-healing is already wired — it is not. And it is another instance of the pattern this
|
||||
whole refactor is about: **a documented mechanism that does not exist, believed because it
|
||||
was written down.** `00-META/how-we-build.md` calls this out as a rule; here it is again,
|
||||
found by grep.
|
||||
|
||||
---
|
||||
|
||||
## 8. Open question
|
||||
|
||||
**Which supervision model does the mesh adopt?** A, B, C, D or a combination.
|
||||
|
||||
**This no longer gates Phase 0.** When this was written, the local mesh was assumed to be
|
||||
built from application containers, which forced the question — there is no natural way to
|
||||
run an init system inside one. The decision of 2026-08-22 to build development nodes as
|
||||
**system containers** (see [`03-DESIGN/01-end-to-end-testing.md`](../../03-DESIGN/01-to-be/01-end-to-end-testing.md))
|
||||
removes that pressure entirely: a system container runs a real init, so the existing model
|
||||
works unmodified and the lab needs no answer here to exist.
|
||||
|
||||
What remains is the question on its own merits, which is worth keeping open because the
|
||||
evidence above still holds: option B removes a layer that 44 of 44 module stacks have already
|
||||
made redundant, and option C would let the mesh stop depending on host-specific init. Neither
|
||||
is urgent. Both are now cheap to *try*, because there is somewhere to try them.
|
||||
|
||||
Option D — a mesh-written supervisor — remains the one the evidence argues against, for the
|
||||
fate-sharing reason in §3.
|
||||
|
||||
## References
|
||||
|
||||
- [`002-local-mesh`](../002-local-mesh/analysis.md) — the effort this came out of
|
||||
- [`02-DECISIONS/0001`](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md) — agent modality, which
|
||||
decides what cannot leave the host
|
||||
- `modules/hal/meshware/daemon/src/cerebellum.ts:815-828` — the self-restart workaround
|
||||
- `modules/hal/meshware/systemd/hal-module@.service` — the per-module Docker lifecycle
|
||||
- `modules/hal/sdk/src/feature-handlers/systemd.ts` — detection, install, start
|
||||
@@ -0,0 +1,67 @@
|
||||
---
|
||||
status: graduated
|
||||
initiated: 2026-08-22
|
||||
touches: [03-DESIGN/00-as-is/01-mesh-and-transport.md, 03-DESIGN/01-to-be/01-end-to-end-testing.md]
|
||||
became:
|
||||
- 02-DECISIONS/0016-the-lab.md
|
||||
- 02-DECISIONS/0016-the-lab.md
|
||||
- 02-DECISIONS/0016-the-lab.md
|
||||
- 03-DESIGN/01-to-be/02-scenario-declaration.md
|
||||
- 03-DESIGN/01-to-be/01-end-to-end-testing.md
|
||||
---
|
||||
|
||||
# 004 — Reproducing the mesh network in a lab
|
||||
|
||||
- **Initiated by:** jochen, 2026-08-22 — *"the most difficult part of our VM setup will be
|
||||
the networking part"*
|
||||
- **Areas touched:** `modules/wireguard`, `modules/dnsmasq-app`, `modules/traefik`,
|
||||
`modules/mesh-ca`, `node_accessors`, `nodes.site` / `nodes.underlay_addr`.
|
||||
|
||||
## Summary
|
||||
|
||||
The network is **entirely generated from mesh-DB rows by module hooks**. `install.d` performs
|
||||
no network configuration whatsoever — no WireGuard, no DNS, no firewall. That makes a faithful
|
||||
lab primarily a *data* problem rather than a networking problem, and means the lab exercises
|
||||
the real code path instead of a reimplementation of it.
|
||||
|
||||
One constraint decides whether the lab works at all: the WireGuard endpoint rule tests the
|
||||
underlay address against an RFC1918 regex to decide reachability. **A simulated public segment
|
||||
addressed from RFC1918 space silently prevents the mesh from forming** — no endpoint is written
|
||||
for the hub, so nothing can ever initiate. The simulated public segment must therefore use
|
||||
TEST-NET-3 (`203.0.113.0/24`).
|
||||
|
||||
With that one substitution the lab reproduces the production topology exactly, including the
|
||||
case that is hardest to get right: a node that is publicly *named* but sits behind NAT, whose
|
||||
endpoint the hub can only learn from a handshake.
|
||||
|
||||
Detail in [`analysis.md`](analysis.md).
|
||||
|
||||
## Settled
|
||||
|
||||
**The lab issues its own certificates.** Public names are certified by an ACME server on the
|
||||
lab's wan segment; `.internal` names keep the mesh CA. The lab preserves production's two-CA
|
||||
split rather than collapsing it, because a single-CA lab would hide any bug living in that
|
||||
split. It also makes the router's port forward load-bearing — HTTP-01 must reach the
|
||||
published-but-NATed node on port 80, so a broken forward becomes a reproducible certificate
|
||||
failure instead of a mystery.
|
||||
|
||||
Requires one change: `caServer` is not set on the reverse proxy today, so it defaults to the
|
||||
public authority's **production** endpoint. It must become configurable, defaulting to
|
||||
production so real nodes are unaffected.
|
||||
|
||||
## Open
|
||||
|
||||
*Closed 2026-08-25.* The lab is stood up. The topology this effort described raises, and the
|
||||
substitution it turned on — a simulated public segment addressed from documentation space
|
||||
rather than RFC1918 — is enforced by the declaration validator before anything is raised
|
||||
rather than left as a thing to remember.
|
||||
|
||||
The certificate conclusion above is carried by
|
||||
[`01-end-to-end-testing.md`](../../03-DESIGN/01-to-be/01-end-to-end-testing.md), which
|
||||
specifies the lab's own ACME issuer on the public segment. It is **designed and not built** —
|
||||
implementation state is a third axis, and the effort graduates on its conclusions, not on
|
||||
their delivery.
|
||||
|
||||
One item leaves this effort without a home and is recorded here so it is not lost: the reverse
|
||||
proxy does not set `caServer`, so it defaults to the public authority's production endpoint.
|
||||
That is a fact about what runs today, not about the lab.
|
||||
@@ -0,0 +1,203 @@
|
||||
---
|
||||
effort: 004-lab-network
|
||||
updated: 2026-08-22
|
||||
---
|
||||
|
||||
# Reproducing the mesh network in a lab
|
||||
|
||||
Established 2026-08-22 by reading the generating code and the live mesh DB. Every claim is a
|
||||
file location or a queried row.
|
||||
|
||||
---
|
||||
|
||||
## 1. The network is data, not configuration
|
||||
|
||||
`install.d/mesh-init.sh` and `install.d/adopt.sh` perform **no network configuration at
|
||||
all** — no WireGuard, no DNS, no firewall, no `/etc/hosts`. Every part of the network layer is
|
||||
generated by module hooks from mesh-DB rows:
|
||||
|
||||
| layer | generated by | from |
|
||||
|---|---|---|
|
||||
| WireGuard interface + peers | `modules/wireguard/hooks/index.ts` (`postConfigure`) | `node_wg_keys`, `nodes.site`, `nodes.underlay_addr`, `module_env.WG_ADDRESS` |
|
||||
| `.internal` name resolution | `modules/dnsmasq-app/hooks/index.ts` | mesh config peers → `internal_domain` + `wg_address` |
|
||||
| public routing / vhosts | `modules/hal/sdk/src/vhost-gen.ts`, `feature-handlers/vhost.ts` | `vhosts:` manifests + `node_accessors` |
|
||||
| internal TLS | `modules/mesh-ca/hooks/index.ts` | a singleton CA row in the mesh DB |
|
||||
|
||||
**Consequence:** a faithful lab is mostly a matter of writing the right rows. The network that
|
||||
results is produced by the same code production runs, which is the difference between testing
|
||||
the network and testing a model of it.
|
||||
|
||||
---
|
||||
|
||||
## 2. The constraint that decides whether the lab works
|
||||
|
||||
`modules/wireguard/hooks/index.ts:225-240` decides, per pair, whether to write an `Endpoint`:
|
||||
|
||||
```js
|
||||
const isPrivate = (a) => /^(10\.|127\.|192\.168\.|172\.(1[6-9]|2\d|3[01])\.)/.test(a);
|
||||
|
||||
if (coLocated && underlay) Endpoint = `${underlay}:${port}` // same LAN
|
||||
else if (underlay && !isPrivate(underlay)) Endpoint = `${underlay}:${port}` // public
|
||||
// else: no Endpoint — the peer must initiate, and we learn its endpoint from the handshake
|
||||
```
|
||||
|
||||
A simulated public segment addressed out of RFC1918 space — `10.200.0.0/24`, say — makes the
|
||||
hub's underlay test as **private**. No spoke writes an `Endpoint` for the hub. Nothing can
|
||||
initiate. **No handshake ever occurs and the mesh silently never forms**, presenting as a
|
||||
WireGuard fault rather than an addressing choice.
|
||||
|
||||
**The simulated public segment must therefore be `203.0.113.0/24`** — TEST-NET-3, reserved by
|
||||
RFC 5737 for documentation, guaranteed never to route on the real internet, and not matched by
|
||||
that regex. The code then treats it exactly as it treats a real hosting provider address.
|
||||
|
||||
This is the single most important fact in this document.
|
||||
|
||||
---
|
||||
|
||||
## 3. The topology being reproduced
|
||||
|
||||
The shape below is what a mesh of this kind looks like: one node with a routable address, one
|
||||
publicly named but behind a household NAT, one stationary workstation, one that roams.
|
||||
Addresses use the documentation ranges of RFC 5737 and RFC 1918 throughout.
|
||||
|
||||
| node | profile | site | underlay | WG | accessors |
|
||||
|---|---|---|---|---|---|
|
||||
| `anchor` | server | `dc` | `203.0.113.10` (routable) | `10.10.0.1/24` | `anchor.example` (public, primary) + `anchor.internal` (lan) |
|
||||
| `home-server` | server | `home` | `192.168.1.135` | `10.10.0.2/24` | `home-server.example` (public, primary) + `home-server.internal` (lan) |
|
||||
| `workstation` | workstation | `home` | `192.168.1.250` | `10.10.0.3/24` | `workstation.internal` (lan, primary) |
|
||||
| `laptop` | workstation | `NULL` | `NULL` | `10.10.0.4/24` | `laptop.internal` (lan, primary) |
|
||||
|
||||
**Hub election is by convention, not by flag.** The hub is the node whose `profile='server'`
|
||||
*and* whose `WG_ADDRESS` begins `10.10.0.1` (`hooks/index.ts:188`). A lab must assign
|
||||
`10.10.0.1` to the node it intends as hub or there will be no hub.
|
||||
|
||||
**`site` drives direct peering** (`hooks/index.ts:197-203`). Two nodes with the same non-null
|
||||
`site` peer directly with a `/32`; everything else routes through the hub's `/24`. A `NULL`
|
||||
site means roaming and hub-only — deliberately, because WireGuard has no failover and a more
|
||||
specific `/32` route to a dead endpoint blackholes rather than falling back.
|
||||
|
||||
The four interesting pairs, all of which the lab must reproduce:
|
||||
|
||||
| pair | behaviour | branch taken |
|
||||
|---|---|---|
|
||||
| anything → `anchor` | `Endpoint` written | underlay non-private |
|
||||
| `home-server` ↔ `workstation` | direct peer, LAN endpoints, keepalive | co-located |
|
||||
| **`anchor` → `home-server`** | **no `Endpoint`; learned from handshake** | not co-located, underlay private |
|
||||
| `laptop` → anything | hub only, always initiates | `site` is `NULL` |
|
||||
|
||||
The third is the one worth building the lab for. The code comments at `hooks/index.ts:206-224`
|
||||
record what it cost to get right: testing `profile === "server"` was tried and was wrong,
|
||||
because a home-hosted node **is** a server yet is not publicly reachable — *"role does not
|
||||
imply reachability; the address does."* An earlier version aimed the hub at that node's public
|
||||
name, which hairpinned off the household NAT: 1.77 MiB sent, 0 B received, no handshake.
|
||||
|
||||
---
|
||||
|
||||
## 4. The lab
|
||||
|
||||
```
|
||||
br-wan 203.0.113.0/24 TEST-NET-3 — non-private, so the code treats it as public
|
||||
│
|
||||
├── hub 203.0.113.10 profile=server site=dc WG 10.10.0.1
|
||||
│
|
||||
└── router VM 203.0.113.1 / 192.168.1.1
|
||||
│ NAT, plus one forwarded port to reproduce a published-but-NATed node
|
||||
│
|
||||
br-lan 192.168.1.0/24 identical to production, same host addresses
|
||||
├── a 192.168.1.135 profile=server site=home WG 10.10.0.2
|
||||
└── b 192.168.1.250 profile=workstation site=home WG 10.10.0.3
|
||||
|
||||
c — attach to br-lan, or br-wan ("away"), or detach ("asleep")
|
||||
underlay NULL profile=workstation site=NULL WG 10.10.0.4
|
||||
```
|
||||
|
||||
Kept **byte-identical** to production: the LAN subnet and its host addresses, and the entire
|
||||
WireGuard plan. Only the public segment is substituted, and only because it must be.
|
||||
|
||||
**The router earns its own VM.** It is what makes the published-but-NATed case real: that node
|
||||
is reachable from outside only through a forwarded port, and the hub must learn its endpoint.
|
||||
It also gives somewhere to break things — drop the forward and observe whether the mesh
|
||||
notices or whether the public name simply stops working.
|
||||
|
||||
**Names.** A resolver on the wan side is authoritative for the public zone. `.internal` names
|
||||
need nothing extra: `dnsmasq-app` generates them from mesh config on each node, and writes an
|
||||
`/etc/hosts` block as a floor underneath, because a node must reach the mesh DB before its own
|
||||
DNS exists.
|
||||
|
||||
---
|
||||
|
||||
## 5. What a node needs before any of this works
|
||||
|
||||
Rows in the mesh DB — `nodes` (`name`, `user_name`, `profile`, `site`, `underlay_addr`),
|
||||
`node_accessors`, `module_env.WG_ADDRESS` (the hook hard-fails without it,
|
||||
`hooks/index.ts:114-116`), and `node_modules` assigning at least `wireguard`, `dnsmasq-app`,
|
||||
`mesh-ca`, and `traefik` where it serves.
|
||||
|
||||
On disk beforehand, because the node must reach the mesh DB before it can read any of the
|
||||
above: registry database and object-store host and credentials, plus an npm token. This
|
||||
ordering — contact the mesh before the mesh has configured you — is itself worth reproducing,
|
||||
and is why the `/etc/hosts` floor exists.
|
||||
|
||||
Everything else is generated: the WireGuard keypair locally (the private key never leaves the
|
||||
node; the public key is published to `node_wg_keys`), the peer list, the DNS records, the TLS
|
||||
leaf.
|
||||
|
||||
---
|
||||
|
||||
## 6. Certificates — the lab issues its own
|
||||
|
||||
**Settled 2026-08-22: the lab runs its own ACME issuer.**
|
||||
|
||||
Public certificates use ACME **HTTP-01** via the reverse proxy, which requires genuine public
|
||||
reachability, so an isolated lab cannot use the real issuer. Rather than forgo certificate
|
||||
testing, the lab stands up an ACME server on its wan segment.
|
||||
|
||||
**The lab keeps production's two-CA split rather than collapsing it.** Production issues
|
||||
public names from a public authority and internal names from the mesh CA; a lab with one CA
|
||||
would hide any bug living in that split. So:
|
||||
|
||||
| | production | lab |
|
||||
|---|---|---|
|
||||
| public names | a public ACME authority | a test ACME server on the wan segment |
|
||||
| `.internal` names | `mesh-ca` | `mesh-ca`, unchanged |
|
||||
|
||||
**A test issuer is the right shape, not a shortcut.** Purpose-built ACME test servers
|
||||
deliberately vary their behaviour — validation timing, nonce handling, chain composition — to
|
||||
expose assumptions a well-behaved authority would let pass. A lab CA that is *too* polite
|
||||
tests less than the real thing, not more.
|
||||
|
||||
### It also exercises the port forward
|
||||
|
||||
HTTP-01 means the issuer must reach the node being certified on port 80. In the lab:
|
||||
|
||||
- the hub is directly reachable on the wan segment — straightforward
|
||||
- **the published-but-NATed node is reachable only through the router's forwarded port**
|
||||
|
||||
So certificate issuance for that node passes only if the forward is correct. That is exactly
|
||||
why its certificate works in production, and it makes "the forward is missing" a reproducible
|
||||
failure rather than a mystery.
|
||||
|
||||
### Required change: `caServer` must be configurable
|
||||
|
||||
`modules/traefik/docker-compose.yml:17-19` sets the challenge entrypoint, the contact address
|
||||
and the storage path — but **no `caServer`**, so Traefik defaults to the public authority's
|
||||
*production* endpoint. Pointing the lab at its own issuer requires adding a `caServer` flag
|
||||
fed by an environment value, defaulting to production so real nodes are unaffected and the lab
|
||||
overrides it per node.
|
||||
|
||||
Worth noting independently of the lab: aiming at the production endpoint rather than a staging
|
||||
one means every certificate experiment on a real node consumes production issuance quota, and
|
||||
a retry loop can exhaust it for a week. The lab issuer removes that exposure.
|
||||
|
||||
---
|
||||
|
||||
## 7. Incidental finding: `scope:` is read by nothing
|
||||
|
||||
Several manifests declare `scope: public` on firewall rules — `wireguard`, `traefik`, `gitea`,
|
||||
`mailu`, `qbittorrent`. It is **not part of the rule type** (`module-registry.ts:17-29`) and is
|
||||
**referenced by no code** in the firewall path. Real scoping is done with `from:`, as
|
||||
`modules/unifi/module.yml:52-93` does deliberately.
|
||||
|
||||
So a manifest can appear to restrict a port to the public scope and in fact restrict nothing.
|
||||
This is the same shape as the rule in `00-META/how-we-build.md` — *an unenforced rule is
|
||||
indistinguishable from a wrong one, and costs more, because people believe it.*
|
||||
@@ -0,0 +1,87 @@
|
||||
---
|
||||
status: graduated
|
||||
initiated: 2026-08-23
|
||||
touches:
|
||||
- 02-DECISIONS/0009-modules-and-the-graph.md
|
||||
- 03-DESIGN/00-as-is/10-module-catalogue.md
|
||||
- 03-DESIGN/00-as-is/02-modules-and-manifests.md
|
||||
became:
|
||||
- 02-DECISIONS/0009-modules-and-the-graph.md
|
||||
- 02-DECISIONS/0009-modules-and-the-graph.md
|
||||
---
|
||||
|
||||
# 005 — Which domains the catalogue groups into
|
||||
|
||||
[ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md) settles
|
||||
that modules outside the platform core are grouped by domain rather than by single function,
|
||||
and deliberately does not settle the list. This effort settles the list — and, first, tests
|
||||
whether the premise survives measurement.
|
||||
|
||||
## What is being investigated
|
||||
|
||||
Eighty-nine modules sit outside the platform core. The question is which of them belong
|
||||
together, and the method is evidence rather than intuition: **which modules actually change
|
||||
together**, measured across the full history of the code repository.
|
||||
|
||||
## Why
|
||||
|
||||
The argument in ADR 0022 is that the catalogue's shape records what was installed rather than
|
||||
what anything is for — that four modules constituting "how a node is reachable" have no
|
||||
relationship the mesh can see, so a change to connectivity is made four times.
|
||||
|
||||
That argument is testable. If those modules genuinely change together, the grouping is
|
||||
justified by more than tidiness. If they do not, the premise needs revising before a list is
|
||||
drawn from it.
|
||||
|
||||
## What it touches
|
||||
|
||||
The catalogue's shape, the manifest, and — for one candidate grouping — the provisioning
|
||||
reference itself, since a requirement names a provider **module**. Grouping providers would
|
||||
change what a consumer names.
|
||||
|
||||
## Status
|
||||
|
||||
Measurement is done and is in [`analysis.md`](analysis.md). It **partly contradicts the
|
||||
premise**, in a way that narrows the effort usefully:
|
||||
|
||||
- Non-platform modules overwhelmingly change **alone** — 10% of commits touch more than one,
|
||||
and 50 of 89 never co-change with anything.
|
||||
- Two clusters do exist. One of them, reachability, holds up as a domain.
|
||||
- The other, the provisioned infrastructure providers, co-changes for a reason that argues
|
||||
**against** grouping rather than for it.
|
||||
|
||||
The remaining work is the list itself, for the modules where grouping is justified, plus the
|
||||
open questions below.
|
||||
|
||||
## What it became
|
||||
|
||||
*Closed 2026-08-28.* Three of the four questions are answered, and by records that did not cite
|
||||
this effort — which is why it stayed open after being resolved.
|
||||
|
||||
**Whether provider modules group at all** — *no.*
|
||||
[ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md):
|
||||
there is no `networking` thing to install, there are concrete modules named individually. Folders
|
||||
assert relationships; edges record them. *Provider* stops being a category at the same time.
|
||||
|
||||
**Whether "group or leave" is even the right pair of options** — *it was not*, and that is the
|
||||
useful finding. [ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md)
|
||||
reframes it: things that change together share an **authority**, not a package. This effort's own
|
||||
measurement is what that record rests on — reachability being the *only* place modules genuinely
|
||||
co-change is why connectivity is a context and why nothing else needed one.
|
||||
|
||||
**What to do with the ~50 modules that co-change with nothing** — *nothing.* They are modules.
|
||||
Grouping is a tag and a query over the graph, neither of which anybody keeps true by hand.
|
||||
|
||||
**What remains is a sequencing question, not a grouping one**, and it moves rather than closes:
|
||||
*whether applications leave the monorepo before or after they group* is
|
||||
[research 009](../009-migration/00-overview.md)'s, because it is about how to get from here to
|
||||
there rather than about what the shape is.
|
||||
|
||||
## Open (superseded by the above) questions
|
||||
|
||||
| Question | Why it is open |
|
||||
|---|---|
|
||||
| Whether provider modules group at all, and if so what a consumer's requirement names instead of a module. | The provisioning reference is load-bearing; getting it wrong is expensive. Opinion and evidence in the analysis; not yet decided. |
|
||||
| Whether applications group into domains now and leave the monorepo later as a unit, or leave first. | Decided in principle — group first, then split — but the migration order has real cost either way. |
|
||||
| What to do with the ~50 modules that co-change with nothing. | The evidence gives no grouping signal for them at all. That may mean they are correctly sized already. |
|
||||
| Whether "group or leave" is even the right pair of options. | [Research 006](../006-mesh-from-scratch/code-skeleton.md) finds a third fate — **absorbed into the node host**, ceasing to be a module at all — and argues it is the correct answer for the reachability cluster this effort measured. If so, the cluster this effort found is evidence for absorption rather than for grouping. |
|
||||
@@ -0,0 +1,144 @@
|
||||
---
|
||||
effort: 005-domain-grouping
|
||||
updated: 2026-08-23
|
||||
---
|
||||
|
||||
# Which modules actually change together
|
||||
|
||||
## Method
|
||||
|
||||
Every commit in the code repository's main branch that touches the module catalogue, reduced
|
||||
to the set of modules it touched. Platform-namespace modules are excluded — their
|
||||
decomposition is settled by
|
||||
[ADR 0001](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md). Modules that no
|
||||
longer exist are excluded, because pre-rename names dominate the raw signal and describe a
|
||||
catalogue nobody works in.
|
||||
|
||||
Commits touching more than eight modules are excluded from pair counting: a sweep across
|
||||
forty modules says the sweep happened, not that the modules are related.
|
||||
|
||||
## Finding 1 — these modules overwhelmingly change alone
|
||||
|
||||
| Measure | Value |
|
||||
|---|---|
|
||||
| Non-platform modules today | 89 |
|
||||
| Commits touching at least one | 312 |
|
||||
| Commits touching two to eight together | 33 — **10%** |
|
||||
| Modules that never co-change with anything | **50 of 89** |
|
||||
|
||||
Nine commits in ten touch exactly one module. More than half the catalogue has never been
|
||||
edited alongside anything else.
|
||||
|
||||
This is the first thing a grouping proposal has to survive, and it is not what the premise
|
||||
predicts.
|
||||
|
||||
## Finding 2 — two clusters exist
|
||||
|
||||
Everything above two co-changes falls into one of two groups.
|
||||
|
||||
**The provisioned infrastructure providers.** The densest cluster: the relational stores with
|
||||
each other and with the cache, the broker, the object store and the image registry — a
|
||||
strongly connected set, six to two co-changes per pair.
|
||||
|
||||
**Reachability.** The reverse proxy with the resolver, the firewall and the VPN; the firewall
|
||||
with the intrusion filter. Three to two co-changes per pair.
|
||||
|
||||
Outside those: a torrent client with the VPN, twice. A login manager with a media player,
|
||||
twice. Nothing else reaches two.
|
||||
|
||||
## Finding 3 — the provider cluster is manifest churn, not cohesion
|
||||
|
||||
This is the result that matters, and it is only visible by reading the commits rather than
|
||||
counting them.
|
||||
|
||||
Every multi-provider commit is a **cross-cutting change to the machinery, applied N times**:
|
||||
|
||||
| Date | What the commit did |
|
||||
|---|---|
|
||||
| 2026-04-15 | modules own their tools — decentralised tool serving |
|
||||
| 2026-04-24 | filesystem-driven feature detection; manifest boilerplate removed |
|
||||
| 2026-04-27 | remove runtime dependencies that were only build dependencies |
|
||||
| 2026-05-01 | hooks move into their own directory with their own package |
|
||||
| 2026-05-05 | hooks renamed to a stage-and-feature convention; legacy install scripts removed |
|
||||
| 2026-08-06 | verify **by shape in the SDK**, not by copying a script into every module |
|
||||
| 2026-08-11 | bind declared volume paths that leaked an anonymous volume per deploy |
|
||||
| 2026-08-23 | scope services to the local network and mesh; respect the profile gate |
|
||||
|
||||
**Not one of them is a change to what a database is.** They are all "the manifest contract
|
||||
changed, therefore every manifest changed".
|
||||
|
||||
Two consequences follow.
|
||||
|
||||
Merging the providers into one module would not have prevented a single one of these commits.
|
||||
It would have made the same edit land in one file rather than six — which is a diff-size
|
||||
improvement, not a boundary.
|
||||
|
||||
And the history already shows the correct fix being applied, repeatedly and successfully:
|
||||
*verify by shape in the SDK, not by copying a script into every module*. When the same edit
|
||||
must be made in every provider, the answer that worked was **moving the concern into the
|
||||
machinery**, not merging the modules that carry it.
|
||||
|
||||
Co-change here measures coupling to the manifest contract. It does not measure domain
|
||||
cohesion, and using it as though it did would group the catalogue by which modules are most
|
||||
boilerplate-heavy.
|
||||
|
||||
## Finding 4 — reachability holds up
|
||||
|
||||
The same reading applied to the reachability cluster gives a different answer. Most of its
|
||||
twenty-five multi-module commits are the same machinery churn — but not all, and the
|
||||
remainder are genuine:
|
||||
|
||||
| Date | What the commit did |
|
||||
|---|---|
|
||||
| 2026-07-29 | let the mesh's proxy coexist with another listener on the same port |
|
||||
| 2026-08-03 | derive public split-DNS from node accessors so requests stop hairpinning |
|
||||
| 2026-08-06 | firewall mesh-only by default, public by declaration |
|
||||
|
||||
Each is one intent — *change how a node is reachable* — landing across the proxy, the
|
||||
resolver, the firewall and the VPN together. That is exactly the shape ADR 0022 describes, and
|
||||
it is the only place in the catalogue where the measurement finds it.
|
||||
|
||||
The 2026-08-23 scoping commit is the sharpest case: it spans the reachability cluster **and**
|
||||
two providers, because "which network is this exposed on" is a reachability question asked of
|
||||
a database.
|
||||
|
||||
## What this means for ADR 0022
|
||||
|
||||
The record's principle stands, and its scope needs narrowing. Grouping by domain is:
|
||||
|
||||
- **Justified by evidence** for reachability. One intent, several modules, repeatedly.
|
||||
- **Argued against by evidence** for the providers. The coupling is to the manifest contract,
|
||||
and the demonstrated fix is to move the concern into the machinery.
|
||||
- **Unsupported either way** for the fifty modules that co-change with nothing. Silence is not
|
||||
evidence of independence — many are simply rarely touched — but there is no measured basis
|
||||
for grouping them, and a proposal that groups them is drawn from intuition. It should say so.
|
||||
|
||||
The catalogue's shape is still wrong in the way the record describes. Measurement says
|
||||
grouping is the right fix in fewer places than the record implies, and that a second fix —
|
||||
moving cross-cutting concerns into the machinery — accounts for most of what looks like
|
||||
grouping pressure.
|
||||
|
||||
## Recommendation on the provisioning reference
|
||||
|
||||
Asked directly, and stated as an opinion because it is not yet decided.
|
||||
|
||||
**Do not group the provider modules.** Three reasons, in order of weight:
|
||||
|
||||
1. **The evidence for grouping them is the wrong evidence.** Finding 3.
|
||||
2. **A requirement names a provider module.** Grouping providers means a consumer names a
|
||||
resource type and something else chooses the implementation. That is not a folder move; it
|
||||
is implementation selection, a substantially larger design with its own failure modes, and
|
||||
nothing currently asks for it.
|
||||
3. **It would hide which implementation serves a requirement** — the one place the mesh most
|
||||
needs to be explicit, and precisely the indirection ADR 0022 warns grouping causes.
|
||||
|
||||
A provider module is already exactly one purpose: it provisions one resource type. That is a
|
||||
boundary, not an accident of installation.
|
||||
|
||||
## Still to do
|
||||
|
||||
- Draw the list for the cases where grouping is justified, and say plainly which entries rest
|
||||
on measurement and which on judgement.
|
||||
- Decide the ~50 silent modules: correctly sized, or unmeasured?
|
||||
- Test the reachability grouping against the migration cost — the modules in it are among the
|
||||
most-changed in the catalogue, so churn during a move is not hypothetical.
|
||||
@@ -0,0 +1,94 @@
|
||||
---
|
||||
status: active
|
||||
initiated: 2026-08-23
|
||||
touches:
|
||||
- 02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md
|
||||
- 02-DECISIONS/0009-modules-and-the-graph.md
|
||||
- 03-DESIGN/00-as-is/00-overview.md
|
||||
- 03-DESIGN/01-to-be/00-work-breakdown.md
|
||||
became: []
|
||||
---
|
||||
|
||||
# 006 — The mesh designed from nothing
|
||||
|
||||
## What is being investigated
|
||||
|
||||
What the mesh would look like if it were laid out today, with the requirements known and none
|
||||
of the accumulated shape — expressed as a **skeleton**: repositories at the root, modules
|
||||
inside them, and whatever turns out to be the right leaf unit below that.
|
||||
|
||||
The deliverables are [`skeleton.md`](skeleton.md) — tiers, repositories and the four design
|
||||
moves — and [`code-skeleton.md`](code-skeleton.md) — the tier test, what a module looks like on
|
||||
disk, and where today's catalogue lands.
|
||||
|
||||
## Why
|
||||
|
||||
Every structural decision so far has been a **correction**: eight contexts replacing thirty-three
|
||||
modules ([ADR 0001](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md)), domains
|
||||
replacing single-function modules
|
||||
([ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md)). A
|
||||
correction inherits the frame of the thing it corrects, and two of the mesh's oldest problems
|
||||
look unsolvable from inside that frame:
|
||||
|
||||
- **The bootstrap circularity.** The mesh needs a database, a bus, a registry and an identity
|
||||
provider. Those are modules the mesh installs. The mesh cannot install them before it exists.
|
||||
This has been worked around repeatedly and never designed away.
|
||||
- **Participation requires privilege.** Everything assumes root on a machine whose packages and
|
||||
services the mesh owns. A phone cannot participate on those terms, and neither can a machine
|
||||
someone else administers.
|
||||
|
||||
Designing from nothing is a way to find out which parts of the current shape are requirements
|
||||
and which are residue.
|
||||
|
||||
**And there is more residue than expected, from a knowable source.** This began as a
|
||||
**dotfiles repository** — the first two days of history adopt dotfiles, add per-node dotfile
|
||||
overrides, and introduce service symlinking with an ignore file. The flat one-directory-per-tool
|
||||
catalogue, linking rather than copying, adoption of already-configured machines, per-node
|
||||
overrides, and the desktop modules are all inherited from that, not chosen for a mesh. Recorded
|
||||
in [`03-DESIGN/00-as-is/10-module-catalogue.md`](../../03-DESIGN/00-as-is/10-module-catalogue.md).
|
||||
|
||||
That makes this effort's question sharper than "what would we do differently": much of what
|
||||
looks like design is a generalisation of *place files on my machines*, never revisited because
|
||||
it was never stated as an assumption.
|
||||
|
||||
## The requirements this is designed against
|
||||
|
||||
Stated by the operator, recorded here so the skeleton can be checked against them rather than
|
||||
against taste:
|
||||
|
||||
1. The mesh manages multiple computers — **full control**, through modules installed to nodes.
|
||||
2. Mesh state lives in a **database**: which modules on which nodes, logs, configuration.
|
||||
3. Configuration has **several touchpoints** — tool surface, web interface, others — all hosted
|
||||
by the mesh itself.
|
||||
4. **Connectivity** is core: every node reachable from every other over a shared overlay, some
|
||||
nodes publicly exposed, firewalls configured.
|
||||
5. The mesh **hosts applications** — and requires some of them itself. This is the circularity.
|
||||
6. **Arch Linux only for now**; ideally any device, including phones, on lighter terms.
|
||||
7. The end goal is to **operate an IT company** on it — development, design, deployment, full
|
||||
circle, self-hosted. Personal cloud infrastructure.
|
||||
8. **Agents make it self-improving and self-healing.**
|
||||
9. It is **end-to-end testable on one machine**
|
||||
([ADR 0016](../../02-DECISIONS/0016-the-lab.md)).
|
||||
|
||||
## Status
|
||||
|
||||
A first skeleton exists, with four design moves that the current shape does not have. It is
|
||||
`active` because two of them are unproven and one contradicts a record that is already
|
||||
accepted.
|
||||
|
||||
**Finding worth stating up front:** [ADR 0001](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md)
|
||||
names nine bounded contexts and **none of them owns connectivity** — no overlay, no resolution,
|
||||
no firewall, no ingress. Requirement 4 has no home in the accepted decomposition, while
|
||||
[research 005](../005-domain-grouping/analysis.md) found reachability to be the *only* part of
|
||||
the catalogue where modules genuinely change together under one intent. The skeleton adds it.
|
||||
|
||||
## Open questions
|
||||
|
||||
| Question | Why it is open |
|
||||
|---|---|
|
||||
| Does the record — the event log contexts integrate through — belong to the substrate or the control plane? | It is infrastructure by shape and domain by content. Placing it wrong reintroduces a circularity. |
|
||||
| One repository per tier, or per context? | Already open from ADR 0001 as "catalogue destination — one repository or many". The skeleton assumes per tier and does not settle it. |
|
||||
| ~~Does an unprivileged node earn a place in the inventory, or only a presence?~~ | **Answered 2026-08-25** by the operator: a node is a *managed machine inside the mesh*, not an unprivileged something — and a disconnected node is still a node, in a different situation. The question posed a class distinction; the answer is that there is none, and what varies is **state**. Recorded as [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md). |
|
||||
| ~~Does absorbing overlay, filtering, packages, supervision and the container runtime make the host too large?~~ | **Answered 2026-08-25** — [`host-size.md`](host-size.md). Measured: the absorption is smaller than the machinery that already applies state, and eight of ten adapters already carry no dependency. The risk is not size but direction, and it is two modules wide. The claim survives with its scope corrected — the host carries one concern, *apply declared state on this machine*, of which the six are instances. Recorded as [ADR 0005](../../02-DECISIONS/0005-the-node-host.md), designed in [`05-the-node-host.md`](../../03-DESIGN/01-to-be/05-the-node-host.md). |
|
||||
| ~~Four substrate services or five?~~ | **Answered conditionally**, which is the honest form — [`07-the-foundation.md`](../../03-DESIGN/01-to-be/07-the-foundation.md). The substrate is *what the control plane consumes and cannot grant itself*. The identity provider qualifies only if the control plane delegates authentication; if it authenticates natively it is an ordinary hosted service. The count follows from a decision not yet taken, and asserting four was asserting that decision. |
|
||||
| Does `feature` survive? | The skeleton splits it in two and argues the conflation is what makes the delivery pipeline hard to reason about. Unproven. |
|
||||
@@ -0,0 +1,272 @@
|
||||
---
|
||||
effort: 006-mesh-from-scratch
|
||||
updated: 2026-08-23
|
||||
---
|
||||
|
||||
# The code skeleton
|
||||
|
||||
[`skeleton.md`](skeleton.md) laid out tiers and repositories. This is the level below: what a
|
||||
module looks like on disk, and — the question that forced a correction — where a given piece of
|
||||
today's catalogue actually lands.
|
||||
|
||||
## The tier test
|
||||
|
||||
A decision procedure, so placement is answerable rather than argued. Ask in order; first match
|
||||
wins:
|
||||
|
||||
1. **Does it apply state on a machine?** → tier 0, inside the host.
|
||||
2. **Can the control plane *start* without it?** If *no* → tier 1, substrate. Note the verb:
|
||||
*start*, not *function fully*. A capability the control plane loses without something is not
|
||||
the same as a thing it cannot come up without — see "becoming self-hosting" below.
|
||||
3. **Does it decide what should be true across nodes?** → tier 2, a control-plane context.
|
||||
4. **Is it a way to talk to tier 2, holding no logic of its own?** → tier 3, a surface.
|
||||
5. **Otherwise** → tier 4, a workload.
|
||||
|
||||
## Worked example — where postgres ends up
|
||||
|
||||
The obvious answer is "twice": once as the mesh's own database in the substrate, once as a
|
||||
hosted database in the catalogue. That answer is wrong, and seeing why fixes something.
|
||||
|
||||
Run the test. *Can the control plane exist without a relational store?* No. **Postgres is
|
||||
tier 1.** It lives once:
|
||||
|
||||
```
|
||||
mesh-substrate/store/postgres/
|
||||
```
|
||||
|
||||
There is no second copy in the catalogue, because the thing that differs between the mesh's own
|
||||
database and a project's database is **not the module**. It is how that instance is brought up:
|
||||
|
||||
| | The mesh's own instance | A project's database |
|
||||
|---|---|---|
|
||||
| Brought up by | the host, from the pinned bundle, with no control plane present | the ordinary delivery and provisioning path |
|
||||
| Declared in | `mesh-substrate/bundle.yml` | the consuming module's `requires:` |
|
||||
| Exists because | the control plane cannot start without it | something asked for it |
|
||||
|
||||
Same module, two roles. **Tier is a property of the module** — what must exist before what — and
|
||||
**the bundle is a property of the mesh's own instance.**
|
||||
|
||||
This is also legal under the dependency rule, which is worth checking rather than assuming: a
|
||||
tier-4 workload that requires a database depends on tier 1, which points *downward*. The
|
||||
inverse — substrate reaching into the catalogue for a module — would not be, and is the shape
|
||||
the naive "twice" answer would have created.
|
||||
|
||||
The same reasoning places the rest of the substrate: the bus, the object store, the image
|
||||
registry. Each fails step 2, each lives once, each is pinned. That is the whole substrate: four
|
||||
services, and the deliberate absence of a fifth.
|
||||
|
||||
**The identity provider was raised as a boundary case and is settled: it is not substrate.**
|
||||
The mesh does not require one — tier 2 authenticates its own callers natively, and an identity
|
||||
provider is a service the mesh hosts like any other. The substrate is four services, not five.
|
||||
|
||||
The test earned its keep here by turning a vague unease into one answerable question — *does the
|
||||
control plane delegate authentication?* — rather than a debate about how important identity
|
||||
feels.
|
||||
|
||||
## Becoming self-hosting — the forge and the registries
|
||||
|
||||
Self-improvement means the mesh hosts the things it improves itself with: a forge, an image
|
||||
registry, a package registry. The obvious worry is that these duplicate — a `mesh-gitea`
|
||||
for the mesh and a `gitea` for everyone else. **They do not, and the reason is worth stating
|
||||
carefully, because it is the same reason the bootstrap keeps failing today.**
|
||||
|
||||
### They are not substrate
|
||||
|
||||
Run the test with the sharpened verb. *Can the control plane start without a forge?* **Yes.** It
|
||||
comes up, holds inventory, answers questions and manages nodes with the modules it already has.
|
||||
What it cannot do is **change itself**. That is a capability, not a precondition.
|
||||
|
||||
So: forge, image registry and package registry are **tier 4 workloads**. One module each, in the
|
||||
catalogue, exactly like the media server. There is no mesh-specific copy.
|
||||
|
||||
This is not a technicality. It buys a property worth having: **if the forge dies, the mesh keeps
|
||||
running.** Nodes stay managed, services stay up, only self-modification stops. Putting the forge
|
||||
in the substrate would make losing it fatal, for no gain.
|
||||
|
||||
### But delivery needs them — is that not an upward dependency?
|
||||
|
||||
It would be, stated naively, and that would break the one rule the whole skeleton rests on.
|
||||
|
||||
It is resolved the way the constitution already says to resolve it — **depend on abstractions,
|
||||
not on concrete dependencies**. Tier 2's delivery context does not require *the forge module*.
|
||||
It declares requirements:
|
||||
|
||||
| Delivery requires | Satisfied by |
|
||||
|---|---|
|
||||
| a source of record for module code | whichever module provides it |
|
||||
| somewhere to publish images | whichever module provides it |
|
||||
| somewhere to publish packages | whichever module provides it |
|
||||
| somewhere to put build artifacts | the substrate's object store |
|
||||
|
||||
Tier 2 defines the requirement; tier 4 provides the implementation; the binding is data. The
|
||||
dependency points **downward from the provider to the interface**, which is legal, and the
|
||||
control plane never names a concrete module.
|
||||
|
||||
The mechanism for this already exists and is the mesh's most valuable one: **provisioning**. A
|
||||
module declares what it provides; a consumer declares what it requires; the mesh binds them.
|
||||
The only new idea is that **the control plane is itself a consumer** — it has requirements, and
|
||||
they are satisfied the same way a workload's are.
|
||||
|
||||
That generalisation is significant enough to need its own study, and is not settled here.
|
||||
|
||||
### Self-hosting is a state the mesh reaches, not a precondition
|
||||
|
||||
This is the part today's mesh gets wrong, and it explains a recurring class of pain.
|
||||
|
||||
A first node comes up from **pinned external artifacts** — upstream images, by digest, carried
|
||||
in the bundle. It has to: the mesh's own registry does not exist yet, and cannot. The mesh at
|
||||
this point is running and manages nodes, and is not yet self-hosting.
|
||||
|
||||
Self-hosting is then **reached**: the forge is installed as an ordinary workload, the mesh's own
|
||||
source moves into it, the registries come up, and delivery's requirements are re-bound from
|
||||
external providers to internal ones. From that point the mesh builds and deploys itself.
|
||||
|
||||
Stated as a lifecycle:
|
||||
|
||||
```
|
||||
pinned external artifacts ─► mesh runs, manages nodes
|
||||
│
|
||||
│ forge + registries installed as workloads
|
||||
│ delivery's requirements re-bound
|
||||
▼
|
||||
mesh builds and deploys itself
|
||||
```
|
||||
|
||||
**Today's mesh assumes the second state from the first moment.** Its source, its packages and
|
||||
its images are all expected to be self-hosted before there is anything to host them — which is
|
||||
why raising a first node needs a script that exists solely to paper over the impossibility, and
|
||||
why that script is the least-exercised path in the system.
|
||||
|
||||
Making the transition explicit has a second benefit: it is reversible. A mesh whose forge is
|
||||
broken can re-bind delivery to external providers and keep improving itself while it repairs
|
||||
the forge. Today that escape hatch does not exist, because the dependency is not expressed
|
||||
anywhere it could be changed.
|
||||
|
||||
### So, concretely
|
||||
|
||||
One `gitea` module. One image-registry module. One package-registry module. Each a tier-4
|
||||
workload. The mesh's own instances are distinguished from any other instance **by what they are
|
||||
bound to, not by being different modules** — precisely the same answer as postgres, arrived at
|
||||
by the same test.
|
||||
|
||||
## The five fates of a module
|
||||
|
||||
Research 005 asked whether a catalogue module should be **grouped into a domain** or **leave
|
||||
the repository**. Working the tier test across the catalogue surfaces a third answer that
|
||||
neither option covers, and it is the most common one.
|
||||
|
||||
| Fate | Means | Examples from today |
|
||||
|---|---|---|
|
||||
| **Absorbed into the host** | It is not a module at all. It is part of what "managing a machine" means, and belongs in tier 0. | overlay membership, packet filtering, package management, service supervision, container runtime, filesystem management |
|
||||
| **Substrate** | The control plane cannot exist without it. Pinned, host-applied. | relational store, bus, object store, image registry |
|
||||
| **Control-plane context** | It decides something across nodes. | connectivity policy, inventory, delivery, provisioning, observability |
|
||||
| **Workload module** | The mesh hosts it. Grouped per [ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md). | media library, desktop session, collaboration tooling |
|
||||
| **Leaves the repository** | A standalone application, per [ADR 0015](../../02-DECISIONS/0015-applications-live-in-their-own-repository.md). | the applications identified in research 005 |
|
||||
|
||||
**The first fate is the finding.** Research 005 measured the reachability cluster — proxy,
|
||||
resolver, firewall, overlay — as the only place in the catalogue where modules genuinely change
|
||||
together under one intent. The skeleton explains *why*: they are not four modules that ought to
|
||||
be one domain module. They are four facets of one thing the host should own, currently
|
||||
expressed as modules because a module was the only unit available.
|
||||
|
||||
Under this skeleton the overlay module and the firewall module **stop existing**. The host holds
|
||||
membership and applies filtering; tier 2 decides the policy; the swappable backends stay
|
||||
modules. That is a different and better answer than grouping them, and it was not visible from
|
||||
inside the current frame.
|
||||
|
||||
It also partly answers research 005's other open question — the fifty modules that co-change
|
||||
with nothing. Several are host concerns rather than domains: package management, container
|
||||
runtime, filesystem tooling. Silence was the right signal after all; the wrong conclusion was
|
||||
that grouping was the only available fix.
|
||||
|
||||
## What a module looks like on disk
|
||||
|
||||
Per [`skeleton.md`](skeleton.md) Move 4, a module declares **parts** — independently selectable
|
||||
pieces of desired state — and produces **artifacts** — things built once per version.
|
||||
|
||||
```
|
||||
<module>/
|
||||
module.yml identity, what it provides, what it requires,
|
||||
which host profiles it can land on
|
||||
parts/
|
||||
service/ desired state: container, volumes, exposure
|
||||
provisioner/ how it grants its resource to consumers
|
||||
migrations/ its own persistent state
|
||||
tools/ capabilities it contributes
|
||||
artifacts/
|
||||
<name>/ source for something built and published
|
||||
```
|
||||
|
||||
A module with no artifacts consumes an upstream image and builds nothing. A module with no
|
||||
parts is not a module.
|
||||
|
||||
Worked through for the substrate's relational store:
|
||||
|
||||
```
|
||||
mesh-substrate/store/postgres/
|
||||
module.yml provides: database · profiles: [managed]
|
||||
parts/
|
||||
service/ the container, its volume, its network exposure
|
||||
provisioner/ grants a database and role to a consumer
|
||||
migrations/ none — it holds no state of its own
|
||||
artifacts/ none — upstream image, pinned by digest in bundle.yml
|
||||
```
|
||||
|
||||
## The tree, at file level
|
||||
|
||||
```
|
||||
mesh-host/ TIER 0
|
||||
cmd/host/
|
||||
internal/
|
||||
apply/ reconcile declared state
|
||||
inventory/ what this machine is and can do
|
||||
link/ outbound connection to the control plane
|
||||
overlay/ membership: address, keys, tunnel
|
||||
filter/ packet filtering from tier-2 policy
|
||||
packages/ package management
|
||||
services/ supervision
|
||||
containers/ container runtime
|
||||
store/ embedded local state
|
||||
profile/ managed · user · edge
|
||||
substrate.lock pinned tier-1 descriptor
|
||||
|
||||
mesh-substrate/ TIER 1
|
||||
bundle.yml the pinned set, by digest
|
||||
store/postgres/
|
||||
bus/<broker>/
|
||||
objects/<object-store>/
|
||||
images/<registry>/
|
||||
|
||||
mesh-control/ TIER 2
|
||||
record/ the event log contexts integrate through
|
||||
inventory/ nodes · modules · assignments · versions
|
||||
config/ settings · secrets · derivation
|
||||
connectivity/ addresses · resolution · exposure · filtering · certificates
|
||||
provisioning/ grants between modules
|
||||
delivery/ source → artifact → node
|
||||
observability/ health · logs · metrics
|
||||
identity/ agents · humans · services · authorisation
|
||||
work/ tasks · workflows · runs
|
||||
knowledge/ memory · documents · retrieval
|
||||
api/ the one interface surfaces speak to
|
||||
|
||||
mesh-surfaces/ TIER 3
|
||||
tools/ web/ cli/
|
||||
|
||||
mesh-catalog/ TIER 4
|
||||
<domain>/<module>/ layout as above
|
||||
|
||||
mesh-lab/ mesh-sdk/ hq/ (company-scoped — ADR 0019)
|
||||
```
|
||||
|
||||
## What this does not settle
|
||||
|
||||
- **The identity boundary case.** Four substrate services or five.
|
||||
- **Where the record lives.** Still the open question from
|
||||
[`00-overview.md`](00-overview.md), and the tier test does not resolve it: the record is
|
||||
needed by tier 2 and is *of* tier 2, which is exactly the shape that produces a circularity.
|
||||
- **Whether absorbing into the host makes the host too large.** Six internal concerns is already
|
||||
a lot for a binary whose whole argument is that it has no dependencies. The counter-argument
|
||||
is that each is small and none can be optional — but this is the skeleton's biggest unproven
|
||||
claim, and it should be tested by writing the host's interface before anything else.
|
||||
- **The migration.** Nothing here says how today becomes this.
|
||||
@@ -0,0 +1,126 @@
|
||||
# Is the host too large?
|
||||
|
||||
The skeleton absorbs overlay membership, packet filtering, package management, service
|
||||
supervision, the container runtime and filesystem management into tier 0, and
|
||||
[`00-overview.md`](00-overview.md) calls this *"the skeleton's biggest unproven claim. A binary
|
||||
whose whole argument is that it has no dependencies now carries six concerns."*
|
||||
|
||||
This is that claim, measured.
|
||||
|
||||
## Method
|
||||
|
||||
Against `origin/main` of the monorepo at 2026-08-25 — read through git refs rather than a
|
||||
checkout, which sits on a feature branch 1031 commits behind with uncommitted work.
|
||||
|
||||
For each module that implements one of the six concerns: how much code it is, and **what it
|
||||
depends on to do its job**. The second question turned out to be the one that matters.
|
||||
|
||||
## Finding 1 — by size, the concern is misplaced
|
||||
|
||||
| Absorbed into the host | Lines |
|
||||
|---|---|
|
||||
| `dnsmasq-app` | 706 |
|
||||
| `traefik` | 426 |
|
||||
| `ufw` | 385 |
|
||||
| `mesh-ca` | 353 |
|
||||
| `wireguard` | 313 |
|
||||
| `incus` | 285 |
|
||||
| `package-manager` | 118 |
|
||||
| `zfs` | 67 |
|
||||
| `fail2ban` | 60 |
|
||||
| `docker-app` | 42 |
|
||||
| **total** | **2 755** |
|
||||
|
||||
| Machinery that already applies state on a node | Lines |
|
||||
|---|---|
|
||||
| `hal/meshware` | 1 829 |
|
||||
| `hal/env-sync` | 626 |
|
||||
| `hal/config-sync` | 604 |
|
||||
| **total** | **3 059** |
|
||||
|
||||
**Everything being absorbed is smaller than the machinery that already exists to apply it.**
|
||||
The six concerns are not six subsystems; they are ten thin adapters averaging 275 lines, most
|
||||
of which is rendering a config file and running a command.
|
||||
|
||||
The host is not a new large thing. It already exists, spread across three core modules, and
|
||||
the absorption adds less code than those three already contain.
|
||||
|
||||
Size is therefore the wrong axis, and the open question asked about the wrong thing.
|
||||
|
||||
## Finding 2 — the real risk is direction, and it is narrow
|
||||
|
||||
What each adapter needs in order to decide what to write:
|
||||
|
||||
| Module | Gets its inputs from | Applier only? |
|
||||
|---|---|---|
|
||||
| `ufw` | `~/.hal/modules/*/module.yml` on the local disk | yes |
|
||||
| `dnsmasq-app` | `process.env.DNSMASQ_*`, derived centrally by `env-sync` | yes |
|
||||
| `fail2ban`, `mesh-ca`, `package-manager`, `docker-app`, `incus`, `zfs` | local state and declared env | yes |
|
||||
| `wireguard` | **direct `pg` connection** — `nodes`, `node_accessors`, `node_wg_keys`, `module_env` | **no** |
|
||||
| `traefik` | **direct `pg` connection** — `nodes`, `mesh_ca` | **no** |
|
||||
|
||||
**Eight of ten are already pure appliers.** They receive derived state and put it on the
|
||||
machine. Absorbing those into tier 0 moves no dependency at all — it moves code that already
|
||||
has none.
|
||||
|
||||
**Two reach upward.** `wireguard` and `traefik` open a connection to the control plane's
|
||||
database and compute their own configuration from it. Absorbing them *as they are* would put a
|
||||
Postgres client and knowledge of the mesh schema inside tier 0 — an upward dependency, which
|
||||
is precisely what
|
||||
[the tier rule](skeleton.md) forbids and what the entire bootstrap argument rests on.
|
||||
|
||||
So the danger in the absorption is real, and it is two modules wide rather than six concerns
|
||||
wide.
|
||||
|
||||
## Finding 3 — the split has already been happening, unnamed
|
||||
|
||||
`dnsmasq-app` is the same *kind* of module as `wireguard`: it needs every node's addresses and
|
||||
names. It does not query for them. Its own comments record why:
|
||||
|
||||
> *"the mesh DB already holds [this] in `node_accessors` and the WireGuard address, duplicated
|
||||
> by hand on all four nodes. Renaming the namespace then meant editing four override rows
|
||||
> nobody [knew about]."*
|
||||
|
||||
and
|
||||
|
||||
> *"now derived from `node_accessors`"*
|
||||
|
||||
That is one module having already made the move the skeleton proposes — deciding centrally,
|
||||
applying locally — for the ordinary reason that hand-duplicated state went wrong. Nobody named
|
||||
it as an architectural direction; it was reached by fixing a bug.
|
||||
|
||||
Eight of ten adapters are on the far side of that migration. Two are not.
|
||||
|
||||
## What this means for the open question
|
||||
|
||||
**The absorption is not a move. It is a split, and it is mostly already done.**
|
||||
|
||||
The question "does the host become too large" assumed six concerns would arrive whole. They do
|
||||
not. Each divides:
|
||||
|
||||
- **deciding** — what this node's overlay, names, exposure and filtering should be, which needs
|
||||
every other node and therefore belongs in tier 2;
|
||||
- **applying** — putting that on the machine, which needs root and locality and therefore
|
||||
belongs in tier 0.
|
||||
|
||||
Tier 0 absorbs the applying. That is ~2 755 lines today, most of it already dependency-free,
|
||||
against 3 059 lines of apply machinery the host needs regardless.
|
||||
|
||||
**The claim survives, with its scope corrected.** The host does not carry six concerns; it
|
||||
carries one — *apply declared state on this machine* — of which the six are instances. That is
|
||||
the skeleton's own tier-0 test (*"does it apply state on a machine?"*) applied to itself.
|
||||
|
||||
## What remains open
|
||||
|
||||
- **The two unsplit modules are the two hardest.** Overlay needs every node's key, address,
|
||||
site and endpoint reachability; the proxy needs certificates and every node's exposed names.
|
||||
These are where "derive centrally, apply locally" is most work, and neither has been done.
|
||||
The measurement says the design is right; it does not say the migration is cheap.
|
||||
- **Six concerns is still six vocabularies.** Absorbing them adds no dependencies but does add
|
||||
surface: the host must know what a WireGuard peer, an nftables rule, a package, a unit, a
|
||||
container and a dataset *are*. Nothing here measures that cost, and it is the residue of the
|
||||
original worry.
|
||||
- **What the host must carry versus what it must find.** The host manages `wg`, `nft`,
|
||||
`pacman`, `docker`; it does not contain them.
|
||||
[Issue 007](../../04-ISSUES/007-an-installed-package-is-not-a-capability/00-report.md) is
|
||||
exactly this question and is unresolved.
|
||||
@@ -0,0 +1,281 @@
|
||||
---
|
||||
effort: 006-mesh-from-scratch
|
||||
updated: 2026-08-23
|
||||
---
|
||||
|
||||
# The skeleton
|
||||
|
||||
Repositories at the root, modules inside them, parts at the leaf. Four tiers, and a dependency
|
||||
rule that only points downward.
|
||||
|
||||
## What a tier is
|
||||
|
||||
A tier answers one question: **what has to exist before this can exist?** It is not importance,
|
||||
and it is not a layer in the networking sense. It is bootstrap order, made explicit.
|
||||
|
||||
Walk a bare machine to a running mesh and the tiers fall out of the story:
|
||||
|
||||
```
|
||||
bare machine
|
||||
│ one command lands ONE binary. Nothing else exists. TIER 0 host
|
||||
│
|
||||
│ it reads a pinned file it already carries and raises a
|
||||
│ database, a bus, an object store and an image
|
||||
│ registry — locally, alone. TIER 1 substrate
|
||||
│
|
||||
│ on those, the mesh's brain starts: which nodes exist,
|
||||
│ what runs where, what is reachable. TIER 2 control plane
|
||||
│
|
||||
│ ways to talk to that brain. TIER 3 surfaces
|
||||
│
|
||||
└ everything the mesh then carries. TIER 4 workloads
|
||||
```
|
||||
|
||||
**Dependencies point only downward.** The substrate never references the control plane. That
|
||||
one constraint is the entire bootstrap answer, because it guarantees there is always a place to
|
||||
start.
|
||||
|
||||
Why it earns its keep here: **the current mesh violates this, and that is the circularity that
|
||||
keeps recurring.** The mesh database is a module; modules are installed by the delivery
|
||||
pipeline; the pipeline needs the database. No order works, so a first-node script exists to
|
||||
paper over it, and every later substrate change has to pretend the problem is not there.
|
||||
|
||||
A tier is not a repository and not a bounded context. Those are different cuts: a context says
|
||||
*who owns this concept*, a tier says *what must already be running*.
|
||||
|
||||
## The tree
|
||||
|
||||
```
|
||||
mesh-host/ TIER 0 — the only thing ever installed by hand
|
||||
apply/ reconcile declared state on this machine
|
||||
inventory/ what this node is, has, and is capable of
|
||||
link/ the single outbound connection to the control plane
|
||||
store/ embedded local state — authoritative while disconnected
|
||||
profile/ capability detection: managed · user · edge
|
||||
substrate.lock pinned tier-1 descriptor, appliable with no mesh present
|
||||
|
||||
mesh-substrate/ TIER 1 — declarations only, no logic of its own
|
||||
store/ relational state
|
||||
bus/ commands and events
|
||||
objects/ blobs and build artifacts
|
||||
images/ container images
|
||||
bundle.yml the pinned set tier 0 can raise alone
|
||||
|
||||
mesh-control/ TIER 2 — the control plane
|
||||
record/ the event log every context integrates through
|
||||
inventory/ nodes · modules · assignments · versions
|
||||
config/ settings · secrets · derivation onto nodes
|
||||
connectivity/ overlay · resolution · exposure · filtering · certificates
|
||||
provisioning/ resource grants between modules
|
||||
delivery/ source → artifact → node
|
||||
observability/ health · logs · metrics · alerts
|
||||
identity/ agents · humans · services · authorisation
|
||||
work/ tasks · workflows · runs
|
||||
knowledge/ memory · documents · retrieval
|
||||
api/ the one interface every surface speaks to
|
||||
|
||||
mesh-surfaces/ TIER 3 — thin; no logic lives here
|
||||
tools/ the agent-facing tool surface
|
||||
web/ the operator-facing interface
|
||||
cli/ the shell-facing interface
|
||||
|
||||
mesh-catalog/ TIER 4 — what the mesh hosts
|
||||
<domain>/ grouped per ADR 0022, list per research 005
|
||||
|
||||
mesh-lab/ the whole mesh, disposable, on one machine
|
||||
mesh-sdk/ contracts shared across tiers — types, not behaviour
|
||||
|
||||
hq/ company-scoped, not a mesh repository — ADR 0019
|
||||
```
|
||||
|
||||
## The dependency rule
|
||||
|
||||
**A tier may depend only on tiers below it.** Substrate never references the control plane.
|
||||
The control plane never reaches into a node except through the host. A surface holds no logic
|
||||
a second surface would have to reimplement.
|
||||
|
||||
This is the whole of the bootstrap answer, and per this repository's own rule it must say how
|
||||
it is checked: a dependency-direction lint in the build, failing on an upward import. A tier
|
||||
rule enforced by intention is the same as no tier rule — that is
|
||||
[ADR 0010](../../02-DECISIONS/0010-delivery.md) applied to architecture.
|
||||
|
||||
## Move 1 — the substrate is applied, not delivered
|
||||
|
||||
**The problem.** The mesh needs a database, a bus, an object store and an image registry.
|
||||
Today those are modules, and modules are installed by the delivery pipeline, which
|
||||
needs the database and the bus. The first node is therefore raised by a special script that
|
||||
exists only because of the circularity, and every later change to the substrate has to pretend
|
||||
the circularity is not there.
|
||||
|
||||
**The move.** The host can apply a declaration without anyone telling it to. The substrate is
|
||||
a **pinned bundle** the host carries: a fixed, versioned, self-contained descriptor of the
|
||||
four services and nothing else. Raising a first node is `host apply substrate.lock` — not a
|
||||
special path, just the ordinary one with no control plane on the other end.
|
||||
|
||||
The circularity disappears rather than being worked around: **the substrate is applied by tier
|
||||
0, the mesh is delivered by tier 2, and they are different mechanisms on purpose.**
|
||||
|
||||
The price is real and should be named: the substrate is upgraded by bumping a pin and
|
||||
re-applying, not by the pipeline. It gets less machinery than everything else — no per-node
|
||||
selection, no provisioning, no fan-out — and that is the point. Five services justify a
|
||||
simpler mechanism than a hundred.
|
||||
|
||||
## Move 2 — the host is one binary with capability profiles
|
||||
|
||||
**The problem.** Everything assumes root on a machine whose packages, services and network the
|
||||
mesh owns. A phone cannot offer that, and neither can a work laptop. The current answer would
|
||||
be a lightweight fork, which means two implementations and one of them rotting.
|
||||
|
||||
**The move.** One host binary, and a **profile** it detects rather than is told:
|
||||
|
||||
| Profile | Can | Typical |
|
||||
|---|---|---|
|
||||
| `managed` | packages, services, network, filesystem — the full surface | a machine the mesh owns |
|
||||
| `user` | user-level services and tools; no package or network management | a shared or administered machine |
|
||||
| `edge` | report presence, relay, expose a tool surface; hold nothing | a phone |
|
||||
|
||||
A module declares which profiles it can land on. Assignment to an incapable node fails at
|
||||
declaration time, not at deploy time — a phone is not a machine that fails to install a
|
||||
firewall; it is a node the firewall module cannot be assigned to.
|
||||
|
||||
This makes the phone case a **capability question rather than a platform question**, which is
|
||||
what keeps it from becoming a second implementation. It also removes the current unstated
|
||||
assumption that every node is equivalent — already false, and today handled by remembering.
|
||||
|
||||
## Move 3 — connectivity becomes a context
|
||||
|
||||
[ADR 0001](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md) names nine contexts
|
||||
and none of them owns the overlay, the resolver, the firewall or the ingress. `config` owns
|
||||
PKI, which is the closest thing, and it is not close.
|
||||
|
||||
Meanwhile [research 005](../005-domain-grouping/analysis.md) measured the whole catalogue and
|
||||
found that reachability is the **only** place where modules genuinely change together under one
|
||||
intent — the proxy with the resolver, the firewall with the overlay, repeatedly, because *how a
|
||||
node is reachable* is one question asked in four places.
|
||||
|
||||
So the evidence and the gap point the same way. `connectivity` owns:
|
||||
|
||||
- the overlay every node joins, and the addresses on it
|
||||
- name resolution, internal and public
|
||||
- exposure — which services answer from outside, on which names
|
||||
- filtering — what may reach a node at all
|
||||
- certificates for both name spaces
|
||||
|
||||
This is an addition to an accepted record, so it is a decision, not a drafting choice. It
|
||||
belongs in a new record that extends ADR 0001 the way
|
||||
[ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md) does —
|
||||
not written here.
|
||||
|
||||
### Where the networking actually lives
|
||||
|
||||
A context is an authority, not a running thing, so naming one does not say who brings the
|
||||
overlay up. That splits three ways, and the split is the design:
|
||||
|
||||
| Concern | Tier | Why there |
|
||||
|---|---|---|
|
||||
| **Overlay membership** — this node joins, holds an address, keeps the tunnel up | **0, in the host** | Everything cross-node needs it *before* the substrate is reachable from elsewhere. A module cannot provide it, because installing a module is itself a cross-node operation. |
|
||||
| **Policy** — who holds which address, what resolves, what is exposed, what is filtered | **2, `connectivity`** | Bookkeeping and authority. It decides; it runs nothing. |
|
||||
| **Machinery** — resolver, reverse proxy, firewall backend, certificate issuance | **4, modules** | Swappable, and not every node needs them. A node without a reverse proxy is still a node. |
|
||||
|
||||
There is a second circularity hiding here, and it has to be closed explicitly: **the host's link
|
||||
to the control plane does not run over the overlay.** If it did, the overlay would have to be up
|
||||
before the host could be told how to join it. The link is ordinary outbound internet to a public
|
||||
endpoint; the overlay carries node-to-node traffic only.
|
||||
|
||||
Joining is therefore: host lands → links out with a join token → control plane returns an
|
||||
address and keys → host raises membership → the substrate on other nodes becomes reachable.
|
||||
|
||||
This also makes the `edge` profile honest rather than special-cased. A phone can hold overlay
|
||||
membership in userspace without privilege, and cannot run the machinery. That is the profile
|
||||
distinction doing its job.
|
||||
|
||||
## Move 4 — `feature` splits in two
|
||||
|
||||
The invitation was to check whether the concept survives. It does not, in one piece.
|
||||
|
||||
Today a **feature** means both *a thing built once* and *a thing selected per node*, and the
|
||||
delivery pipeline is hard to reason about precisely because those have different cardinality
|
||||
and one word ([ADR 0010](../../02-DECISIONS/0010-delivery.md)
|
||||
is the pipeline half of the same confusion).
|
||||
|
||||
Split it:
|
||||
|
||||
| Concept | Is | Cardinality |
|
||||
|---|---|---|
|
||||
| **artifact** | something built and published — an image, a bundle, a package | once per module version |
|
||||
| **part** | an independently selectable piece of a module's desired state | chosen per node, per assignment |
|
||||
|
||||
A module declares desired state in parts, and produces artifacts. An assignment names the node
|
||||
and the parts. Delivery builds artifacts once and applies parts per node — and the two words
|
||||
now carry the two cardinalities that the pipeline already has.
|
||||
|
||||
This keeps what features are genuinely for — per-node opt-in of *some* of a module, which the
|
||||
work breakdown already calls for — and drops the conflation that makes the current model
|
||||
confusing.
|
||||
|
||||
## What the mesh is, versus what it hosts
|
||||
|
||||
Tiers 0–3 are the mesh. Tier 4 is everything it carries, and the boundary is stated by
|
||||
requirement rather than by taste: **a module is part of the mesh if removing it stops the mesh
|
||||
managing nodes.** A media server does not. A relational store does — which is why it sits in the
|
||||
substrate and not the catalogue, despite being, in every other respect, an application like any
|
||||
other. An identity provider, notably, does **not**: the control plane authenticates its own
|
||||
callers, so identity is a hosted service like the media server.
|
||||
|
||||
That test also settles the IT-company goal without a special category. Development, design and
|
||||
deployment tooling are **workloads** — tier 4, hosted, provisioned, delivered like anything
|
||||
else. The mesh does not grow a "company" feature; it hosts the tools a company runs on, and
|
||||
the fact that it runs its own development on them is dogfooding, not architecture.
|
||||
|
||||
## What agents are, structurally
|
||||
|
||||
Self-improvement and self-healing are not a tier. Agents are participants
|
||||
([ADR 0003](../../02-DECISIONS/0003-agents-are-persistent-employees.md)) that hold identity in
|
||||
tier 2, act through tier 3 like any other caller, and run as workloads in tier 4.
|
||||
|
||||
This matters for one reason: **an agent must not have a privileged path**. Anything an agent
|
||||
can do to the mesh, a person can do through the same surface, and anything it cannot express
|
||||
through the tool surface is a gap in the surface rather than a reason for a back door. Self-
|
||||
healing built on a private channel is unreviewable, and would be the one part of the mesh with
|
||||
no human checkpoint.
|
||||
|
||||
## How this is tested
|
||||
|
||||
The lab ([ADR 0016](../../02-DECISIONS/0016-the-lab.md)) raises the
|
||||
tree above on one machine: virtual machines as nodes, a real overlay between them, the real
|
||||
substrate bundle, the real control plane, the real delivery path.
|
||||
|
||||
The tier rule is what makes that affordable. A scenario needing only tiers 0 and 1 is one
|
||||
virtual machine and a pinned bundle — which is also, exactly, the bootstrap path. **The
|
||||
hardest thing to test becomes the cheapest scenario to run**, and the first-node path stops
|
||||
being the one thing nobody exercises until it breaks.
|
||||
|
||||
## What this skeleton does not answer
|
||||
|
||||
- Where the record lives. It is infrastructure by shape and domain by content, and putting it
|
||||
in the substrate risks recreating a circularity in the one place the design just removed one.
|
||||
- Whether tier 2's contexts are one repository or several. Open from ADR 0001 already.
|
||||
- Whether an `edge` node is in the inventory or merely present — which decides whether "node"
|
||||
is one concept or two.
|
||||
- The migration. Nothing here says how today's mesh becomes this, and the skeleton is worth
|
||||
little until that is costed.
|
||||
|
||||
## A naming near-miss, recorded
|
||||
|
||||
The tier-0 binary was first called `mesh-agent`, because "node agent" is the reflex everywhere
|
||||
else in the industry. That is wrong here, and wrong in the specific way
|
||||
[`how-we-build.md`](../../00-META/how-we-build.md) §4 exists to catch: **Agent** is a
|
||||
first-class concept in this mesh — a participant, some of whom are human, holding identity and
|
||||
memory ([ADR 0003](../../02-DECISIONS/0003-agents-are-persistent-employees.md)). One document
|
||||
carried both meanings.
|
||||
|
||||
It is the same failure as the anatomy naming in the current runtime: an evocative domain word
|
||||
pointing at infrastructure.
|
||||
|
||||
[ADR 0001](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md) supplies the fix in
|
||||
its own title — *the mesh brokers capabilities; nodes host; agents think.* Three verbs, three
|
||||
components: the control plane **brokers** (`mesh-control`), the tier-0 binary **hosts**
|
||||
(`mesh-host`), the participant **thinks** (`agents`, untouched).
|
||||
|
||||
`mesh-node` was the alternative and was rejected: *Node* is the aggregate in the inventory — the
|
||||
record of a machine — while the binary is what runs on it and does the hosting.
|
||||
@@ -0,0 +1,51 @@
|
||||
---
|
||||
status: active
|
||||
initiated: 2026-08-23
|
||||
touches:
|
||||
- 03-DESIGN/00-as-is/03-provisioning.md
|
||||
- 02-DECISIONS/0009-modules-and-the-graph.md
|
||||
- 01-RESEARCH/006-mesh-from-scratch/code-skeleton.md
|
||||
became: []
|
||||
---
|
||||
|
||||
# 007 — Provisioning as the mesh's core mechanism
|
||||
|
||||
## What is being investigated
|
||||
|
||||
Provisioning is the mechanism the whole mesh rests on: a module declares what it needs, and the
|
||||
mesh makes it exist, generates the credential, records the grant, and puts the values where the
|
||||
module will read them. [ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md)
|
||||
calls it the mesh's core concern rather than its plumbing.
|
||||
|
||||
[Research 006](../006-mesh-from-scratch/code-skeleton.md) then asks it to carry **more**: the
|
||||
control plane becomes a consumer with its own requirements — a source of record, an image
|
||||
registry, a package registry — satisfied by the same mechanism. That generalisation is only
|
||||
safe if the mechanism is sound, and the as-is record says it is not, in named ways.
|
||||
|
||||
## Why now
|
||||
|
||||
Four weaknesses are already documented in
|
||||
[`03-DESIGN/00-as-is/03-provisioning.md`](../../03-DESIGN/00-as-is/03-provisioning.md), each
|
||||
observed rather than theorised:
|
||||
|
||||
- **Rotation has no fan-out.** A shared credential can be rotated without telling the peers
|
||||
holding the old one. This has locked the mesh out of its own broker.
|
||||
- **A grant is not a check.** The record says a resource was provisioned. Nothing verifies it
|
||||
still exists, still has that credential, or is reachable from where the consumer runs.
|
||||
- **A frozen password outlives its generation.** A generated secret written once diverges from a
|
||||
persistent data directory initialised earlier, and presents as an authentication error.
|
||||
- **No requirements is indistinguishable from provisioning that did not run.** A module that
|
||||
declares nothing skips the stage, which is correct, and looks identical to failure.
|
||||
|
||||
Generalising a mechanism with these properties to the control plane's own dependencies would
|
||||
make each of them fatal rather than annoying.
|
||||
|
||||
## The questions
|
||||
|
||||
| Question | Why it matters |
|
||||
|---|---|
|
||||
| What does a **grant** mean, exactly — a record that a resource was created, or a claim about the world that is continuously reconciled? | The difference between the current model and one where "provisioned" is checkable. Almost every weakness above is a symptom of the first answer. |
|
||||
| How is a credential **rotated** with fan-out to every holder? | The mechanism grants easily and regrants not at all. This is the most damaging gap and it has taken the mesh down. |
|
||||
| Can the **control plane** hold requirements, and what satisfies them before anything is installed? | The generalisation research 006 needs. Ties directly to the self-hosting transition. |
|
||||
| What happens when a requirement **cannot** be satisfied — no provider, provider on an unreachable node, provider not yet installed? | Today this is silent or a stall. It should be a stated, visible state. |
|
||||
| Does a requirement belong to a **module** or to one of its **parts**? | Research 006 splits `feature` into artifact and part. A part-scoped requirement means a database is not provisioned where the part that needs it is not installed. |
|
||||
@@ -0,0 +1,101 @@
|
||||
---
|
||||
status: graduated
|
||||
initiated: 2026-08-23
|
||||
touches:
|
||||
- 03-DESIGN/00-as-is/04-delivery.md
|
||||
- 02-DECISIONS/0010-delivery.md
|
||||
- 02-DECISIONS/0010-delivery.md
|
||||
- 01-RESEARCH/006-mesh-from-scratch/code-skeleton.md
|
||||
became:
|
||||
- 02-DECISIONS/0010-delivery.md
|
||||
- 02-DECISIONS/0010-delivery.md
|
||||
---
|
||||
|
||||
# 008 — The coordinator: a change checked in becomes a deployed state
|
||||
|
||||
## What is being investigated
|
||||
|
||||
The mesh's own continuous delivery: a change is committed, and the mesh ends up in the state
|
||||
that change describes — across every node the change touches, with a verdict that says whether
|
||||
it worked.
|
||||
|
||||
The coordinator is what orchestrates that, and it is the mesh's most consequential machinery:
|
||||
everything reaches every node through it.
|
||||
|
||||
## Why now
|
||||
|
||||
The as-is record ([`03-DESIGN/00-as-is/04-delivery.md`](../../03-DESIGN/00-as-is/04-delivery.md))
|
||||
names problems that are structural rather than incidental:
|
||||
|
||||
- **A green pipeline proves transport, not effect.** The stages report that a message was
|
||||
dispatched and accepted, which is not the same as the thing running, correct, or present.
|
||||
This is the mesh's single most consistent failure shape.
|
||||
- **Detection is the most fragile input.** A merge that creates no pipeline, with nothing saying
|
||||
so, is the characteristic bad outcome — and it has happened for reasons unrelated to the
|
||||
change.
|
||||
- **The fan-out point is asymmetric.** The build node has already passed two silos when work
|
||||
fans out, and code that knew only about the first parked it forever while every other node
|
||||
deployed cleanly.
|
||||
- **There is no end-to-end coverage.** The harness has not built since 2026-06-04
|
||||
([`04-ISSUES/005`](../../04-ISSUES/005-pipeline-test-harness-unbuildable/00-report.md)).
|
||||
|
||||
Research 006 adds a requirement the current design does not have: the coordinator must work
|
||||
**before the mesh is self-hosting**, when source and artifacts come from outside, and keep
|
||||
working across the transition to self-hosted providers.
|
||||
|
||||
## What it became
|
||||
|
||||
*Closed 2026-08-28.* All six questions are answered, by two records, and the second exists
|
||||
because the first was honest about what it did not fix.
|
||||
|
||||
**Does the coordinator dispatch stages, or converge nodes on a declaration?** — *Converge.*
|
||||
[ADR 0010](../../02-DECISIONS/0010-delivery.md): a pipeline ends when the
|
||||
declaration is updated, and the host applies it and reads back — so the reporter is the applier.
|
||||
|
||||
**Does the three-silo split survive?** — *Yes, with the third redefined.* The cardinality
|
||||
observation holds; the third silo is not a stage any more.
|
||||
|
||||
**How does a change become a pipeline, reliably?** — *It does not become a pipeline at all.*
|
||||
[ADR 0010](../../02-DECISIONS/0010-delivery.md) applies 0058's
|
||||
move one level up: the control plane holds what source exists and what has been built, and builds
|
||||
the difference. **An event makes it fast; nothing makes it necessary.** The failures this effort
|
||||
catalogued — a truncated commit list, a broken path match — become latency rather than silence.
|
||||
|
||||
**What is a deployed state?** — *Two comparisons, not an event.* Does every node's reported state
|
||||
match what is declared, and is what is declared built from current source? A milestone can be
|
||||
claimed by something that did not check; a comparison cannot.
|
||||
|
||||
**What produces a verdict, and what is it about?** — *An artifact, and it gates eligibility.* The
|
||||
mesh must not converge onto something broken, so an artifact may be declared only once the lab
|
||||
has judged it fit. Sharper than the question expected: a verdict is a property an artifact has,
|
||||
not a report about a run.
|
||||
|
||||
**How does delivery work before self-hosting?** — *It mostly stops being a question.* A reconciler
|
||||
reads source and writes artifacts; where those live is a binding, external first and internal
|
||||
later. The transition looked hard because a pipeline's stages name their targets.
|
||||
|
||||
## What this effort was right about
|
||||
|
||||
Its first question — *what is a deployed state, and how does the mesh know it is in one* — was
|
||||
marked "everything follows from this", and everything did. Both records above are answers to it:
|
||||
0058 makes the applier the reporter, and 0063 makes currency a comparison. The effort put the
|
||||
load-bearing question first.
|
||||
|
||||
## What is NOT closed by this
|
||||
|
||||
[ADR 0010](../../02-DECISIONS/0010-delivery.md) names four costs
|
||||
and one of them is a real risk rather than a trade: **a reconciler that cannot reach its target
|
||||
retries forever, and without something that notices, the failure is silence** — which is the
|
||||
fault this effort exists to catalogue, reintroduced in a new place. That belongs to observability
|
||||
and it is not designed.
|
||||
|
||||
## The questions (all answered above)
|
||||
|
||||
| Question | Why it matters |
|
||||
|---|---|
|
||||
| What is a **deployed state**, and how does the mesh know it is in one? | Everything follows from this. If a stage reports transport, "deployed" is a claim nobody checked. A desired-state model with reconciliation gives a different answer from a job-completion model. |
|
||||
| Does the coordinator dispatch **stages**, or converge nodes on a **declaration**? | The current model is a state machine over stages. The alternative is that a node is told what should be true and reports what is. The second makes drift visible; the first cannot see it. |
|
||||
| How does a change **become** a pipeline, reliably? | Detection has failed for reasons unrelated to the change, silently. |
|
||||
| What produces a **verdict**, and what is it a verdict about? | Ties to the lab ([ADR 0016](../../02-DECISIONS/0016-the-lab.md)) and to a module carrying its own assertions. |
|
||||
| How does delivery work **before self-hosting**, and across the transition? | From research 006: source and artifacts start external and are re-bound to internal providers. The coordinator has to be indifferent to which. |
|
||||
| Does the **three-silo** split survive the artifact/part split? | [ADR 0010](../../02-DECISIONS/0010-delivery.md) is cardinality-driven, and research 006 renames the thing the cardinality is about. |
|
||||
@@ -0,0 +1,95 @@
|
||||
---
|
||||
status: active
|
||||
initiated: 2026-08-23
|
||||
touches:
|
||||
- 01-RESEARCH/006-mesh-from-scratch/code-skeleton.md
|
||||
- 03-DESIGN/01-to-be/01-end-to-end-testing.md
|
||||
- 02-DECISIONS/0016-the-lab.md
|
||||
- 03-DESIGN/00-as-is/00-overview.md
|
||||
became: []
|
||||
---
|
||||
|
||||
# 009 — Getting from the mesh that exists to the mesh that is designed
|
||||
|
||||
## What is being investigated
|
||||
|
||||
How a running mesh becomes the one in
|
||||
[research 006](../006-mesh-from-scratch/code-skeleton.md), without losing what it currently
|
||||
carries.
|
||||
|
||||
The proposed shape: **build tiers 0, 1 and 2, then replace the current setup in one move.**
|
||||
|
||||
## Why big-bang is the right instinct here
|
||||
|
||||
Recorded because incremental is the reflex answer and it is wrong in this case.
|
||||
|
||||
- **The two models are structurally incompatible.** The tier rule, the host absorbing what are
|
||||
now modules, the artifact/part split, provisioning generalised to the control plane's own
|
||||
requirements — none of these can half-apply. Running both models at once means the old one's
|
||||
assumptions keep constraining the new one, which is how a migration becomes permanent.
|
||||
- **Nothing external depends on it.** No users outside the operator, no service level to hold.
|
||||
- **The lab exists precisely for this** ([ADR 0016](../../02-DECISIONS/0016-the-lab.md)).
|
||||
A big-bang that has been rehearsed end to end, repeatedly, on identical machines is not the
|
||||
same risk as one performed for the first time on the real mesh. This is also why the lab is
|
||||
phase 0 rather than a verification step later: the new mesh is *developed* inside it, so by
|
||||
cutover the procedure has been run hundreds of times rather than rehearsed a few.
|
||||
- **Incremental would carry the rot forward.** The as-is layer documents silent failure paths,
|
||||
a dead test harness and unenforced rules. A gradual migration preserves them by definition.
|
||||
|
||||
## The distinction that lowers the risk
|
||||
|
||||
**Replace the control plane; do not move the workloads.**
|
||||
|
||||
The things that would hurt to lose — mail, media, source, databases, their data directories —
|
||||
are not the mesh. They are what the mesh manages. They sit in container volumes on nodes, and
|
||||
they do not need to move for the control plane above them to be replaced.
|
||||
|
||||
So the big-bang is: the old control plane stops managing these nodes, and the new one starts —
|
||||
with the workload data untouched, in place, and re-declared rather than migrated.
|
||||
|
||||
That reframing turns "replace the mesh" into "replace the part with no persistent state of its
|
||||
own", which is a materially smaller act than it first sounds.
|
||||
|
||||
## The tension this exposes
|
||||
|
||||
Taking over already-running workloads is **adoption**, and adoption was ruled out of scope —
|
||||
recorded as a legacy path in
|
||||
[`03-DESIGN/01-to-be/01-end-to-end-testing.md`](../../03-DESIGN/01-to-be/01-end-to-end-testing.md).
|
||||
The migration appears to need exactly the capability the design declared it would not have.
|
||||
|
||||
The way out, to be tested: the new mesh does not adopt anything. It **declares** the workloads
|
||||
from scratch and points them at data directories that already exist. Nothing inspects a running
|
||||
machine to learn what is there; the declarations are written from the as-is layer, which is what
|
||||
that layer is for. Data survives because it was never touched, not because it was adopted.
|
||||
|
||||
If that holds, adoption stays out of scope and the migration is ordinary declaration. If it does
|
||||
not, adoption needs a one-time, explicitly unsupported tool, and that should be a decision rather
|
||||
than a discovery.
|
||||
|
||||
## Sequencing
|
||||
|
||||
| Phase | What | Done when |
|
||||
|---|---|---|
|
||||
| **0** | **Build the lab's bootstrap scenario** ([ADR 0016](../../02-DECISIONS/0016-the-lab.md)) — virtual machines, a network, a way to place a binary, snapshot and reset. No forge, no coordinator, no pipeline. | A machine can be raised from nothing, reset, and raised again, repeatably. |
|
||||
| A | Build tier 0, **inside the lab**. The host's interface first — it carries the skeleton's biggest unproven claim. | A bare machine becomes a managed node with no mesh present. |
|
||||
| B | Build tier 1 and 2. The bootstrap scenario grows into the full one by addition — the same machines, with more placed inside them. | The lab raises a full mesh from nothing, repeatedly, from pinned external artifacts. |
|
||||
| C | Enough of tier 3 to operate it. | The mesh can be driven without direct database access. |
|
||||
| D | Declare the existing workloads against the new model. | The lab runs them, with copies of real data shapes. |
|
||||
| E | **Rehearse the cutover in the lab** against a mesh built to resemble the real one. | Repeatable, and repeatably reversible. |
|
||||
| F | Cut over. | The real nodes are managed by the new control plane. |
|
||||
| G | Reach self-hosting — re-bind delivery from external providers to the mesh's own forge and registries. | The mesh builds and deploys itself. |
|
||||
|
||||
Phase G is deliberately last. Per research 006, self-hosting is a state the mesh **reaches**;
|
||||
attempting the cutover and the self-hosting transition in the same move recreates exactly the
|
||||
circularity the skeleton removes — and would mean a failed cutover could take away the means to
|
||||
fix it.
|
||||
|
||||
## The questions
|
||||
|
||||
| Question | Why it matters |
|
||||
|---|---|
|
||||
| What state must **survive** the cutover, versus be re-created? | Workload data must. Provisioned credentials could be re-issued. The mesh's own inventory could be re-declared. Each answer changes the risk. |
|
||||
| What is the **way back**? | A cutover with no rollback is not a plan. If workload data is untouched, reverting may be as small as re-pointing the old control plane at it — to be verified, not assumed. |
|
||||
| How is the cutover **rehearsed** against something resembling the real mesh, without copying the real mesh into a repository? | The lab must be able to model the real topology's shape without carrying its identity. |
|
||||
| Does anything have to keep running **during** the cutover? | Mail and source are the obvious candidates. If yes, "big-bang" is really "big-bang with exceptions", and the exceptions should be named now. |
|
||||
| Is the forge inside or outside the cutover? | If the mesh's own forge goes down with the old control plane, the means of deploying a fix goes with it. This is the self-hosting circularity appearing as a migration risk. |
|
||||
@@ -0,0 +1,60 @@
|
||||
---
|
||||
status: active
|
||||
initiated: 2026-08-24
|
||||
touches:
|
||||
- 03-DESIGN/01-to-be/03-scenario-lifecycle.md
|
||||
- 02-DECISIONS/0016-the-lab.md
|
||||
- 02-DECISIONS/0016-the-lab.md
|
||||
became: []
|
||||
---
|
||||
|
||||
# 010 — What the lab's inner loop actually costs
|
||||
|
||||
## What is being investigated
|
||||
|
||||
[The lifecycle design](../../03-DESIGN/01-to-be/03-scenario-lifecycle.md) closes on an open
|
||||
question that is measurable rather than arguable:
|
||||
|
||||
> **What a snapshot costs.** Whole-scenario snapshots of several machines are the operation the
|
||||
> inner loop repeats most, so their cost sets the loop's speed. If restoring is slow, the loop
|
||||
> is slow, and everything above is theory.
|
||||
|
||||
Measured on a workstation, 2026-08-24. Numbers in [`measurements.md`](measurements.md).
|
||||
|
||||
## Why it matters
|
||||
|
||||
[ADR 0016](../../02-DECISIONS/0016-the-lab.md) makes the
|
||||
bootstrap scenario the inner development loop for tiers 0 and 1 — the argument being that
|
||||
raising a node from nothing stops being the least-exercised path and becomes the most-exercised
|
||||
one. **That argument is only true if raising and resetting are cheap.** A loop that costs
|
||||
minutes is a loop people avoid, and the least-exercised path stays least-exercised.
|
||||
|
||||
## Status
|
||||
|
||||
Measured, and the finding is a blocker rather than a data point.
|
||||
|
||||
**A snapshot on the current host is a full copy of the machine's disk.** 1.6 GB and ten seconds
|
||||
for one small virtual machine at best — and over two minutes when observed a second time. Cost
|
||||
scales with the number of machines and the size of their disks, not with what changed.
|
||||
|
||||
**With copy-on-write it is 0.13 seconds and costs the delta.** Verified, not assumed. The
|
||||
projected four-machine reset cycle falls from roughly ninety seconds to roughly fifteen, of
|
||||
which almost all is a boot that cannot be avoided.
|
||||
|
||||
The cause is not virtual machines and not incus. It is that the host offers incus exactly one
|
||||
storage driver, `dir`, which has no copy-on-write and therefore no cheap snapshot. The kernel
|
||||
supports btrfs; the userspace tool that would let incus use it is simply not installed.
|
||||
|
||||
So the question *"is the lab's inner loop fast enough"* currently answers itself the wrong way,
|
||||
for a reason that is one declared package away from being fixed — and declaring packages is
|
||||
something the mesh already does.
|
||||
|
||||
## Open questions
|
||||
|
||||
| Question | Why it matters |
|
||||
|---|---|
|
||||
| ~~How much does a copy-on-write pool actually improve it?~~ **Measured: snapshot 9.9 s → 0.13 s, restore 10.4 s → 0.80 s, three snapshots sharing 1.36 GB rather than costing 4.8 GB.** The projected four-machine cycle falls from ~90 s to ~15 s. | Answered. The inner-loop argument holds *with* copy-on-write and did not without it. |
|
||||
| How does the lab install its own prerequisites on a clean machine? | The lab needs a virtualisation daemon, copy-on-write tooling and a pool before it can do anything — and it cannot depend on the mesh for them, since it is where the mesh is built. |
|
||||
| Why was the second snapshot more than twelve times slower than the first? | If snapshot cost is unpredictable rather than merely high, that is worse — a loop with a variable multi-minute step is one nobody trusts. |
|
||||
| Does a scenario snapshot need the machines stopped? | Stateless snapshots of a running virtual machine capture the disk but not memory. Whether a mesh restored that way is coherent is not established. |
|
||||
| What is the cost at scenario scale — four machines rather than one? | Only single-machine numbers were taken. If the operation is serial, four machines is four times the wait. |
|
||||
@@ -0,0 +1,141 @@
|
||||
---
|
||||
effort: 010-lab-inner-loop-cost
|
||||
updated: 2026-08-24
|
||||
---
|
||||
|
||||
# Measurements
|
||||
|
||||
Taken 2026-08-24 on a workstation with hardware virtualisation available, an NVMe-backed ext4
|
||||
root, and 300 GB free. One virtual machine, 1 GiB memory, 2 CPUs, from a cached distribution
|
||||
image.
|
||||
|
||||
## The environment, before anything ran
|
||||
|
||||
| Fact | Value | Consequence |
|
||||
|---|---|---|
|
||||
| Hardware virtualisation | present | virtual machines run at native speed; the choice in [ADR 0016](../../02-DECISIONS/0016-the-lab.md) is not paying an emulation penalty |
|
||||
| Storage drivers the daemon offers | **`dir` only** | no copy-on-write, therefore no cheap snapshot |
|
||||
| Host filesystems | ext4 throughout | nothing copy-on-write to put a pool on |
|
||||
| btrfs kernel module | **available** | the kernel can do it |
|
||||
| `btrfs-progs` | **not installed** | which is the entire reason the driver is absent |
|
||||
|
||||
The last two rows are the finding. The daemon advertises only `dir` because the userspace tool
|
||||
for anything better is missing — not because the host cannot do better.
|
||||
|
||||
## Raising a machine
|
||||
|
||||
| Step | Time |
|
||||
|---|---|
|
||||
| launch call returns | **3.4 s** |
|
||||
| machine actually usable — a command executes on it | **14.3 s** |
|
||||
|
||||
The gap matters for the lifecycle design: `raise` returning is not the same as the scenario
|
||||
being ready, so the verb has to wait for the second number, not report the first. Reporting
|
||||
the first would be the mesh's own recurring failure — transport reported as effect.
|
||||
|
||||
## Snapshot and restore
|
||||
|
||||
| Operation | Time | Disk |
|
||||
|---|---|---|
|
||||
| snapshot, first | **9.9 s** | **+1.6 GB** |
|
||||
| snapshot, second | **> 120 s — did not complete** | — |
|
||||
| restore call returns | **10.4 s** | — |
|
||||
| machine usable again | **20.1 s** total | — |
|
||||
|
||||
Instance on disk before snapshotting: 1.5 GB. Snapshot directory afterwards: 1.6 GB. **A `dir`
|
||||
snapshot is a full copy** — the storage cost equals the instance, and nothing is shared.
|
||||
|
||||
Implied copy throughput on the first snapshot is roughly 160 MB/s, which is far below what the
|
||||
underlying NVMe can do and is consistent with a real, durable copy rather than a metadata
|
||||
operation.
|
||||
|
||||
**The second snapshot is the more troubling number.** It exceeded two minutes and was still
|
||||
running when the observation was cut off; only the first snapshot exists. Whatever the cause —
|
||||
page cache exhausted by the preceding restore, writeback contention — the practical
|
||||
consequence is that snapshot cost here is **not merely high, it is unpredictable**.
|
||||
|
||||
## What this projects to
|
||||
|
||||
A four-machine scenario, taking the optimistic single-machine numbers and assuming the
|
||||
operations are serial:
|
||||
|
||||
| | one machine | four machines |
|
||||
|---|---|---|
|
||||
| raise, to usable | 14 s | ~57 s |
|
||||
| snapshot | 10 s, 1.6 GB | ~40 s, 6.4 GB |
|
||||
| restore, to usable | 20 s | ~80 s |
|
||||
|
||||
A reset-and-rerun cycle is therefore **around a minute and a half at best**, and unbounded at
|
||||
worst, before any of the mesh's own work begins.
|
||||
|
||||
## The judgement
|
||||
|
||||
**This is too slow for an inner loop**, and the reason is not the design.
|
||||
|
||||
[ADR 0016](../../02-DECISIONS/0016-the-lab.md) argues that
|
||||
making the bootstrap path the inner development loop turns the least-exercised code in the
|
||||
system into the most-exercised. That argument holds only while resetting is cheap. At a minute
|
||||
and a half a cycle, with occasional multi-minute stalls, the loop is one a person works around
|
||||
— and the path stays under-exercised for exactly the reason it always was.
|
||||
|
||||
Nothing about virtual machines causes this. Hardware virtualisation is present and the machines
|
||||
boot in fourteen seconds. **The cost is entirely the storage driver**, and the driver is absent
|
||||
because one userspace package is not installed on the host.
|
||||
|
||||
The mesh already has the mechanism for that: a module declares a package, and a hook makes it a
|
||||
working capability — which is precisely what was just done for the virtualisation daemon
|
||||
itself, and what [`04-ISSUES/007`](../../04-ISSUES/007-an-installed-package-is-not-a-capability/00-report.md)
|
||||
is about.
|
||||
|
||||
## The fix, measured
|
||||
|
||||
The comparison was subsequently run. One package — `btrfs-progs`, no dependencies — installed by
|
||||
hand, the daemon restarted so it re-detected drivers, a copy-on-write pool created on a loop
|
||||
file, and the identical image launched onto it.
|
||||
|
||||
| Operation | `dir` | copy-on-write | |
|
||||
|---|---|---|---|
|
||||
| snapshot | 9.9 s, then **> 120 s** | **0.13 s** | ~76× faster, and *consistent* |
|
||||
| snapshot again | — | 0.12 s | |
|
||||
| snapshot a third time | — | 0.13 s | |
|
||||
| restore call | 10.4 s | **0.80 s** | ~13× faster |
|
||||
| restore, to usable | 20.1 s | **10.5 s** | the remainder is boot, which is irreducible |
|
||||
| three snapshots, storage | ~4.8 GB | **1.36 GB total, shared** | cost is the delta, not the disk |
|
||||
|
||||
**The fix is real, and larger than expected.** Snapshot goes from ten seconds to a tenth of a
|
||||
second, and — more importantly — from *wildly variable* to *flat*. Three consecutive snapshots
|
||||
took 0.13, 0.12 and 0.13 seconds. On `dir` the second snapshot never finished.
|
||||
|
||||
Storage stops scaling with the machine and starts scaling with what changed: three snapshots of
|
||||
a 1.5 GB instance occupied 1.36 GB in total, because they share.
|
||||
|
||||
### What it projects to
|
||||
|
||||
A four-machine reset-and-rerun cycle, the operation the inner loop repeats most:
|
||||
|
||||
| | `dir` | copy-on-write |
|
||||
|---|---|---|
|
||||
| snapshot the scenario | ~40 s, 6.4 GB | **~0.5 s**, delta-sized |
|
||||
| restore it | ~40 s + boot | **~3 s** + boot |
|
||||
| **cycle** | **~90 s, unbounded at worst** | **~15 s, dominated by boot** |
|
||||
|
||||
At fifteen seconds, dominated by a boot that cannot be avoided, the inner loop is viable and
|
||||
[ADR 0016](../../02-DECISIONS/0016-the-lab.md)'s argument holds.
|
||||
At ninety it did not.
|
||||
|
||||
### One honest counter-observation
|
||||
|
||||
Launching onto the fresh copy-on-write pool was **slower** — 20.2 s to usable against 14.3 s —
|
||||
because the image had to be unpacked into a pool that had never seen it. That cost is paid once
|
||||
per pool, not per scenario, and it is dwarfed by what snapshotting saves. But it is a real
|
||||
number and it went the other way.
|
||||
|
||||
### State this left behind
|
||||
|
||||
Recorded because hand-made state is exactly what the mesh's rules exist to prevent, and it must
|
||||
be declared properly rather than left as an artefact of a measurement:
|
||||
|
||||
- `btrfs-progs` installed by hand. Its installation regenerated the boot initramfs, a side
|
||||
effect worth knowing about.
|
||||
- The daemon restarted once, to re-detect drivers.
|
||||
- The test pool and instance were **removed**; the pool the lab actually needs does not exist.
|
||||
@@ -0,0 +1,203 @@
|
||||
---
|
||||
status: graduated
|
||||
initiated: 2026-08-25
|
||||
became:
|
||||
- 02-DECISIONS/0009-modules-and-the-graph.md
|
||||
- 02-DECISIONS/0008-a-context-owns-its-store.md
|
||||
- 03-DESIGN/01-to-be/06-the-controller.md
|
||||
- 03-DESIGN/01-to-be/07-the-foundation.md
|
||||
touches:
|
||||
- 02-DECISIONS/0009-modules-and-the-graph.md
|
||||
- 02-DECISIONS/0009-modules-and-the-graph.md
|
||||
- 02-DECISIONS/0004-a-node-and-how-it-joins.md
|
||||
- 03-DESIGN/00-as-is/02-modules-and-manifests.md
|
||||
- 03-DESIGN/00-as-is/10-module-catalogue.md
|
||||
- 04-ISSUES/007-an-installed-package-is-not-a-capability/00-report.md
|
||||
---
|
||||
|
||||
# 011 — The module graph
|
||||
|
||||
> **A third edge was found after this graduated.** This effort established *presence* and
|
||||
> *instantiation*, and both are **runtime** edges — they answer *what does this need in order to
|
||||
> run*. Delivery needs a different question answered — *what has to be rebuilt when this changes*
|
||||
> — and that is a **build** edge, fixed inside an artifact rather than negotiated when it runs.
|
||||
> Recorded by [ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md), which also
|
||||
> notes what this effort's three entities turn out to be good for
|
||||
> ([ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md)).
|
||||
|
||||
## What is being investigated
|
||||
|
||||
Whether the catalogue's missing structure is a **graph** — modules declaring what they need,
|
||||
what they offer, and what they exclude — and what that replaces.
|
||||
|
||||
**[`worked-provider.md`](worked-provider.md) works one module through completely**, and breaks
|
||||
the tidy version. A database is nine things, not one — and a small game asking the mesh for its
|
||||
own database shows there are **two kinds of edge**: *presence*, where the thing must exist, and
|
||||
*instantiation*, where a provider makes something for a consumer and hands back credentials.
|
||||
Instantiation implies presence and not the reverse. The current system already had this split
|
||||
and the design had collapsed it.
|
||||
|
||||
**[`features.md`](features.md) answers what happens to `feature`.** It is one word for four
|
||||
things spanning three tiers — artifacts built once per version, resources applied to a machine,
|
||||
actions run against something that is not this machine, and checks that are requirements in
|
||||
disguise. Measured: every one of the twenty-one handlers implements all six stages, so `configs`
|
||||
has a build stage with nothing to build and `npm` has a start stage with nothing to start. That
|
||||
emptiness is the conflation, and it is why a stage that did nothing and a stage that failed look
|
||||
alike. Nothing replaces it, because it was never one concept. **One property is worth keeping:
|
||||
content is detected, relationships are declared.**
|
||||
|
||||
**[`cases.md`](cases.md) enumerates what a module can be** — twenty kinds of thing the mesh has
|
||||
to install, run, own or know about — and extracts the axes a manifest must express. Two of those
|
||||
axes appear in no current thinking: **how many instances** a thing may have, and **whether two
|
||||
can coexist**.
|
||||
|
||||
**The design is in [`proposal.md`](proposal.md): one kind of edge.** A module provides names
|
||||
and requires names, and that single relation absorbs requiring a module, requiring a resource,
|
||||
and the interface-and-adapter idea. An abstract name is legitimate **only where providers are
|
||||
genuinely substitutable** — `terminal` passes, `database` does not, because a consumer speaking
|
||||
Postgres does not speak MongoDB. The adapter is what creates an interface; without one there is
|
||||
a **tag**, which describes and does not bind.
|
||||
A **node provides names too**, which makes capability checking stop being a separate mechanism
|
||||
and makes the host's own capability report an input to resolution rather than something a
|
||||
person reads.
|
||||
|
||||
[`analysis.md`](analysis.md) measured the current catalogue. Its value to the design is two
|
||||
lessons rather than its machinery — a field that means *depends on* should say so, and
|
||||
placement does not belong in a manifest — and the rest is recorded as as-is evidence.
|
||||
|
||||
**Measured, and the premise was wrong for the existing system: the graph is not missing there.**
|
||||
[`analysis.md`](analysis.md) — 126 manifests, 103 edges, no cycles, nothing dangling, and a
|
||||
resolver that topologically sorts them, already called by the tool loader, the installer and
|
||||
the delivery coordinator. What the effort assumed would need building is a thing to call.
|
||||
|
||||
What survives is narrower: three declarations that do not exist (`excludes`, a required node
|
||||
capability, an interface with adapters), and two defects worth fixing whatever else is
|
||||
concluded — `provider:` is a dependency edge that is not read as one, which makes the closure
|
||||
for a working mesh come out without a database; and the resolver continues past a cycle and
|
||||
past a missing dependency, contrary to ADR 0001.
|
||||
|
||||
[ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md) proposes
|
||||
grouping modules by domain. [Research 005](../005-domain-grouping/analysis.md) measured that
|
||||
proposal and found its evidence holds in exactly one place — reachability — which
|
||||
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md) has since absorbed
|
||||
into the host. The measured case for domain grouping has therefore been consumed by a decision
|
||||
taken for unrelated reasons, and what remains is fifty modules that co-change with nothing.
|
||||
|
||||
That leaves the original complaint unanswered: the catalogue records **what was installed**
|
||||
rather than **what anything is for**, and nothing in the system can see a relationship between
|
||||
two modules. Grouping asserts relationships. A graph records them.
|
||||
|
||||
## Why now
|
||||
|
||||
A proposal to split modules into *provisioning services* and *applications* was worked through
|
||||
and abandoned in favour of one concept with facets, for a reason worth keeping:
|
||||
|
||||
- The split cannot be filed consistently. A git forge is consumed as a service *and* operated
|
||||
through a web interface. An analytics service grants tracking identity *and* is a dashboard
|
||||
somebody reads. An identity provider grants authentication *and* has an admin console.
|
||||
- The operator's correction is the sharper form: **what runs on the machine is a supervised
|
||||
container, not something a user started.** That is a fact about *how a thing runs*, not about
|
||||
what kind of thing it is — so it is a facet, not a taxonomy.
|
||||
|
||||
Filing decisions that follow from nothing are the disease research 005 measured. A second
|
||||
taxonomy would reproduce it.
|
||||
|
||||
So [ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md) survives, and the question
|
||||
becomes what a module must be able to **declare**.
|
||||
|
||||
## The shape being investigated
|
||||
|
||||
Five declarations, of which two exist today.
|
||||
|
||||
| Declaration | Today | Notes |
|
||||
|---|---|---|
|
||||
| **requires a resource** — a database, a bucket | yes | provisioning, [ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md) |
|
||||
| **provides a resource** | yes | as above |
|
||||
| **requires another module** | **no** | the dependency edge — the graph's substance |
|
||||
| **excludes another module** | **no** | installing A makes B unavailable |
|
||||
| **requires a node capability** | **no** | a graphical session, a container runtime, an architecture |
|
||||
|
||||
And one structural idea on top: **an interface module carries no implementation.** Adapters
|
||||
provide it. An assistant interface with several model-provider adapters; a terminal interface
|
||||
with several terminal adapters. A dependent names the interface and never an implementation.
|
||||
|
||||
**Prior art to measure against, not invent past.** This is a package manager's model, and the
|
||||
platform's own package manager already has all of it: `depends` is the dependency edge,
|
||||
`conflicts` is exclusion, and `provides` is the interface — several packages provide one
|
||||
virtual name, and a dependent names the virtual one. That the design arrived at the same shape
|
||||
independently is evidence for it. It is also a warning: dependency resolution, version
|
||||
constraints, conflict handling and rollback are a long-solved and easily-botched problem, and
|
||||
the effort should establish what to **delegate** rather than reimplement.
|
||||
|
||||
## Capabilities, and what may be installed
|
||||
|
||||
The operator's formulation: *system specs are capabilities, and capabilities unlock installable
|
||||
modules — you cannot install a graphical application on a node with no display server.*
|
||||
|
||||
The question that follows is whether the mesh may install a capability. The effort's working
|
||||
position, to be tested:
|
||||
|
||||
- **Intrinsic capabilities** — hardware, architecture, network position — are facts about a
|
||||
machine. They are detected, never installed, and a module requiring one it does not have is
|
||||
not unresolved but **impossible** on that node.
|
||||
- **Provided capabilities** — a display server, a container runtime — are not a separate kind
|
||||
of thing at all. They are modules that provide a capability, and requiring one is an ordinary
|
||||
edge the graph resolves by installing it.
|
||||
|
||||
If that holds, "may the mesh install a capability" is not a policy question. It is dependency
|
||||
resolution, and the only genuinely new thing is detection.
|
||||
|
||||
Which is where [issue 007](../../04-ISSUES/007-an-installed-package-is-not-a-capability/00-report.md)
|
||||
bears directly: *an installed package is not a capability*. A provided capability is not real
|
||||
because a package is present — it is real when it is present, running and working, and the
|
||||
difference is exactly the class of fault this repository keeps recording.
|
||||
|
||||
## What it touches beyond the catalogue
|
||||
|
||||
The operator's assessment is that the machinery around a module is wanted and its integration
|
||||
is not: **scheduled tasks, hooks, migrations, configuration and environment settings are worth
|
||||
keeping; seeds are not; and the current integration is wrong enough to need a major refactor.**
|
||||
|
||||
That is a claim to test rather than adopt. Research 005 already found supporting evidence from
|
||||
a different direction — that the densest apparent coupling in the catalogue is manifest
|
||||
boilerplate churn, cross-cutting changes to the machinery applied N times — which is what an
|
||||
integration being wrong looks like from the outside.
|
||||
|
||||
## Open questions
|
||||
|
||||
Struck-through rows are answered, with where. The rest are live.
|
||||
|
||||
### Settled
|
||||
|
||||
| Question | Answer |
|
||||
|---|---|
|
||||
| ~~What does the graph **delete**?~~ | For the existing system: nothing, it is already there ([`analysis.md`](analysis.md)). For the design: the module/resource distinction, the interface as a kind of thing, capability checking as a separate mechanism, domain grouping, and — the first clear deletion — **grant kinds**, once a module may only be granted what it exclusively owns ([`worked-provider.md`](worked-provider.md)). |
|
||||
| ~~Is an interface a module, or a name?~~ | A **name**, and only where providers are genuinely substitutable. The adapter is what creates one; without an adapter there is a **tag**, which describes and does not bind ([`proposal.md`](proposal.md)). |
|
||||
| ~~Where do domain modules fit?~~ | They do not. There is core infrastructure — concrete modules named individually, not flavourable, nothing standing in front of them. |
|
||||
| ~~What happens to domain grouping?~~ | Superseded. Folders assert relationships; edges record them. What grouping was for is a tag and a query. [ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md) is `proposed` and should be superseded rather than narrowed. |
|
||||
| ~~Is there one kind of edge?~~ | **No — two.** *Presence*, where a thing must exist, and *instantiation*, where a provider makes something for a consumer and hands back credentials. Instantiation implies presence, not the reverse. |
|
||||
| ~~When two modules provide one name, who chooses?~~ | Neither the consumer naming a node nor the consumer not caring. The consumer declares the **scope of its own need** — shared across its instances, or one each — the mesh binds, and the binding is written down and sticky. Where it is written follows the scope. |
|
||||
| ~~Can several modules share one database?~~ | **No.** A module is granted only what it exclusively owns — no shared writes and no read role on another's store, because reading couples you to its layout just as firmly. |
|
||||
| ~~What about a dashboard reading a dozen stores?~~ | **The rule is about contexts, not processes.** The mesh's own board reading the mesh's own store is the mesh showing its own data — not a boundary crossing. Everything inside a context reads its store freely; what is forbidden is a *different* context reading it. An earlier answer here was wrong. |
|
||||
| ~~Can every registry consumer be served another way?~~ | **Largely dissolves.** Of eighteen direct consumers, the owner keeps its database, node appliers are already stopped by ADR 0005, and the bulk are **foreign tenants** — thirteen tables across three contexts — who need to move out rather than read differently. |
|
||||
|
||||
### Live
|
||||
|
||||
| Question | Why it is open |
|
||||
|---|---|
|
||||
| How many instances should a module have? | Not derivable and not global: one-per-mesh is right for the broker and wrong for a store a disconnected node needs. It is per-module and nothing in the schema says it. |
|
||||
| Can two of something coexist? | `excludes` covers part of it. Two terminals are fine, two things wanting one port are not, two brokers might be either. |
|
||||
| What does a provider hand back? | Credentials and an address for a store; a command for a terminal. Same relation, different shape crossing it. |
|
||||
| Is provisioning one mechanism or two? | The mesh's own registry is provisioned **before there is a mesh**, so provisioning is part of the bootstrap and part of what the carried bundle expresses. At bootstrap the store is local; afterwards it is on another node. Same operation, both sides of a tier boundary. |
|
||||
| Is a tool surface one relation with two audiences, or two? | 56 of 126 modules carry tools — more than carry a service — and what consumes them is an **agent**, not a module. |
|
||||
| ~~Do the remaining cross-context reads want an interface or events?~~ | **Derived, not chosen.** Neither is SQL — that only ever runs against your own store. ADR 0004 makes disconnection ordinary, so anything that must work while disconnected cannot use a request and needs a local copy: a subscription. Anything where a stale answer is worse than none cannot use a subscription. |
|
||||
| What does a consumer do about events it missed while disconnected? | Replay from a point, ask once for a full picture and resume, or rebuild. The question every projection has, and unanswered here. |
|
||||
| What happens to a grant when its consumer is removed? | Dropping is data loss; keeping is a leak. ADR 0005's removal rule does not obviously carry, because the thing lives inside another module's state. |
|
||||
| Is a declaration composed per node, from what that node reported? | Some configuration follows the hardware. Either the host fills a blank — deciding, against ADR 0005 — or the control plane composes from the node's inventory first. |
|
||||
| Would `excludes` and capability requirements actually be used? | Zero manifests declare either, which is equally consistent with *nobody needs them* and *nobody can express them*. |
|
||||
| What does an exclusion mean for something already installed? | Refuse the install, or surface the conflict and let it be decided. |
|
||||
| Are tiers a view of the graph, or a constraint on it? | If a tier is a computed level the word is a convenience. If *a tier may depend only on tiers below it* is to be enforced, it is a constraint and must be stated as one. |
|
||||
| Where does resolution happen — the mesh, or the platform's package manager? | The mesh must model mesh-level edges. Whether it also resolves operating-system packages decides whether a solver gets written. |
|
||||
| How far may the control plane be split? | A single board over several contexts works because they are contexts *inside* one control plane with one interface. If a context becomes its own deployable with its own interface, the board is coupled to N of them and the composition has nowhere to live that tier 3 permits. A constraint on splitting, worth knowing before splitting. |
|
||||
| What does the pipeline schedule, once features are gone? | It schedules features today. The four categories they split into have different lifecycles, so the unit of work differs for each and needs naming. |
|
||||
| Should placement leave the catalogue? | A provision pins itself to a named node in the manifest. Placement is an inventory decision, and having it in the catalogue means a second node cannot provide the mesh's store without editing its consumer. |
|
||||
@@ -0,0 +1,138 @@
|
||||
# The graph is not missing
|
||||
|
||||
Measured against `origin/main` of the code repository, 2026-08-26. Every manifest, read through
|
||||
git refs rather than a checkout.
|
||||
|
||||
The effort was opened to ask whether the catalogue's missing structure is a graph. It is not
|
||||
missing. **It exists, it is healthy, and three separate parts of the system already use it.**
|
||||
|
||||
That is the finding, and it changes what is worth asking.
|
||||
|
||||
## Finding 1 — the graph is already declared, and it is clean
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| manifests | 126 |
|
||||
| declare a dependency on another module | 64 |
|
||||
| declare a requirement on a provision | 11 |
|
||||
| **edges** | **103** |
|
||||
| declare neither | 62 — 49% |
|
||||
| **cycles** | **0** |
|
||||
| **dependencies declared but absent** | **0** |
|
||||
| deepest chain | 5 |
|
||||
|
||||
Half the catalogue is unconnected, which matches
|
||||
[research 005](../005-domain-grouping/analysis.md)'s finding that fifty modules co-change with
|
||||
nothing. The connected half is well formed: no cycles, nothing dangling.
|
||||
|
||||
## Finding 2 — it is already resolved, and already used
|
||||
|
||||
The platform SDK carries a dependency resolver that topologically sorts modules, and it is
|
||||
called from three places: the tool loader at startup, the installer when syncing modules onto a
|
||||
node, and the delivery coordinator when expanding what a change affects.
|
||||
|
||||
It also already does something the effort assumed would need designing: **a requirement on
|
||||
another module's provision is treated as an implicit edge to that module**, so a consumer does
|
||||
not have to declare the same relationship twice.
|
||||
|
||||
So *ordering by the graph* — which
|
||||
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md) says
|
||||
the control plane will do — is not a thing to build. It is a thing to call.
|
||||
|
||||
## Finding 3 — the most important edges in the mesh are invisible
|
||||
|
||||
The one place the graph is wrong, and it is wrong about the substrate.
|
||||
|
||||
A module that needs a database declares it like this:
|
||||
|
||||
```yaml
|
||||
provisions:
|
||||
- name: mesh-db
|
||||
provider: postgres # ← a dependency on the postgres module
|
||||
node: <a named node> # ← and where it must run
|
||||
```
|
||||
|
||||
`provider:` names a module. It is a dependency, declared, in the manifest — and it sits inside
|
||||
`provisions:`, which is what a module *offers*. The resolver reads `dependencies:` and
|
||||
`requires:`, so it never sees it.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| provider references that name a real module | 4 |
|
||||
| **invisible to the resolver** | **3, across 2 modules** |
|
||||
|
||||
Three edges is nothing, and they are the mesh's own database, the mesh's own broker, and the
|
||||
work engine's database. The most load-bearing relationships in the system are the ones the
|
||||
graph cannot see.
|
||||
|
||||
**The consequence is measurable.** Computing what a working mesh needs, from the declared
|
||||
graph:
|
||||
|
||||
```
|
||||
registry → sdk → mesh → meshware 4 modules, 4 levels
|
||||
```
|
||||
|
||||
No database. No broker. A closure that is arithmetically correct and obviously wrong, and wrong
|
||||
for exactly one reason: a field that means *depends on* is not read as one.
|
||||
|
||||
This is [04-ISSUES/003](../../04-ISSUES/003-firewall-scope-is-read-by-no-code/00-report.md)
|
||||
again, in a new place — not a key nothing reads, but a key read as something other than what it
|
||||
means.
|
||||
|
||||
## Finding 4 — the resolver continues past faults it should stop on
|
||||
|
||||
Two behaviours, both contrary to
|
||||
[ADR 0010](../../02-DECISIONS/0010-delivery.md):
|
||||
|
||||
- **A cycle warns and falls back to input order.** A cycle means no correct order exists; the
|
||||
resolver proceeds with an arbitrary one and logs a line.
|
||||
- **A dependency that does not exist warns and continues.** The validation is documented as
|
||||
*non-fatal, logged as warnings*.
|
||||
|
||||
Neither has fired in the current catalogue — there are no cycles and nothing dangling — which
|
||||
is why nobody has noticed. They are latent, and they are in the component that
|
||||
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md) makes
|
||||
responsible for the ordering a host will apply without question.
|
||||
|
||||
## Finding 5 — placement is decided in the catalogue
|
||||
|
||||
`node:` in a provision pins it to a named node, in the manifest. Two modules do this today,
|
||||
and they are the substrate ones.
|
||||
|
||||
Which node runs what is an inventory and placement decision — tier 2 by the skeleton's own
|
||||
test. Having it in a manifest means the catalogue decides placement, and a second node cannot
|
||||
provide the mesh's database without editing the module that consumes it.
|
||||
|
||||
## What this means for the effort
|
||||
|
||||
**The opening question — "what does the graph delete?" — has an answer: nothing, because the
|
||||
graph is already there.** The premise was wrong, and finding that out is the effort's first
|
||||
result rather than a setback.
|
||||
|
||||
The questions that survive are narrower and answerable:
|
||||
|
||||
| Missing declaration | Manifests using it today |
|
||||
|---|---|
|
||||
| `excludes` — installing A makes B unavailable | **0** |
|
||||
| a required node capability | **0** |
|
||||
| an interface, with adapters providing it | **0** |
|
||||
|
||||
Those are what a graph would *add*. What it would delete is a different and smaller list, and
|
||||
the honest version of it is: nothing yet.
|
||||
|
||||
**And two defects worth fixing regardless of what else this effort concludes:**
|
||||
|
||||
1. `provider:` is a dependency edge and is not read as one. Fixing it makes the closure correct
|
||||
— which is what [research 012](../012-the-minimum-viable-node/00-overview.md) needs in order
|
||||
to answer what a one-node mesh requires.
|
||||
2. The resolver continues past a cycle and past a missing dependency. Both should refuse.
|
||||
|
||||
## What was not measured
|
||||
|
||||
- **Whether the missing declarations would be used.** Zero manifests declare exclusions or
|
||||
capabilities, and that is equally consistent with *nobody needs them* and *nobody can express
|
||||
them*. Nothing here separates those.
|
||||
- **Whether the interface-and-adapter idea has a consumer.** It is a good shape, and it is
|
||||
argued for rather than measured.
|
||||
- **What the closure should be.** Finding 3 says the computed one is wrong. It does not say
|
||||
what the right one is; that needs the fix first.
|
||||
@@ -0,0 +1,128 @@
|
||||
# What a module can be
|
||||
|
||||
Every kind of thing the mesh has to install, run, own or know about, before deciding what a
|
||||
manifest says. Written to be argued with: a case here that turns out not to exist should be
|
||||
struck, and one that is missing is a hole in whatever schema follows.
|
||||
|
||||
The hard cases are at the end, and they are the point.
|
||||
|
||||
## The ordinary cases
|
||||
|
||||
**1 — A supervised service.** A container the mesh runs and keeps running. Nobody starts it;
|
||||
it is simply up. *A relational store, a message broker, an object store, a dashboard.*
|
||||
|
||||
**2 — A system package with configuration.** Not a container. Installed into the machine,
|
||||
configured through files, run by the service manager. *A firewall, a resolver, an overlay.*
|
||||
Note: [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) says applying
|
||||
these is the host's job — so what the module contributes is the *deciding*, not the doing.
|
||||
|
||||
**3 — An application a person launches.** Installed on a node, started by a human, running only
|
||||
while they use it. *An editor, a chat client, a file manager, a terminal.*
|
||||
|
||||
**4 — A command-line tool.** Installed, on the path, run when invoked. No service, no window.
|
||||
*A formatter, a query client, a backup utility.*
|
||||
|
||||
**5 — A library.** Never runs at all. Consumed at build time by other modules. *An SDK.*
|
||||
|
||||
**6 — A one-shot task.** Runs once, changes something, exits. *A schema migration, a data
|
||||
import, a seed.*
|
||||
|
||||
**7 — A scheduled task.** Runs repeatedly on a timer, exits each time. *A backup, a prune, a
|
||||
report.*
|
||||
|
||||
**8 — An adapter.** Exists to make several unlike things look alike behind one name. *A model
|
||||
provider behind an assistant interface.*
|
||||
|
||||
**9 — A standalone application in its own repository.** Same shape as any of the above; the
|
||||
difference is only where its source lives
|
||||
([ADR 0015](../../02-DECISIONS/0015-applications-live-in-their-own-repository.md)). Worth
|
||||
listing because a schema that assumes a monorepo path would exclude it.
|
||||
|
||||
## The cases that break a naive schema
|
||||
|
||||
**10 — Something that is a service *and* an application.** A git forge is consumed by other
|
||||
modules as a remote and a registry, *and* operated by a person through a web interface. A web
|
||||
analytics service grants a tracking identity *and* is a dashboard somebody reads. Neither is a
|
||||
service-or-application choice; both are true simultaneously.
|
||||
|
||||
**11 — Something that provides to others *and* consumes from others.** The store provides
|
||||
databases and needs a filesystem. The forge provides a registry and needs a database. Provider
|
||||
and consumer are not kinds of module; they are ends of edges.
|
||||
|
||||
**12 — Something the mesh installs that then becomes a node capability.** The container runtime
|
||||
is installed *by* the mesh, and once it works, the node **provides** `container-runtime` to
|
||||
everything else. So a module can change what its node provides. The node's provides-list is
|
||||
therefore partly derived from what is installed on it, not only detected from what was already
|
||||
there — and the two have to agree.
|
||||
|
||||
**13 — Something that must be adopted rather than installed.** The machine already has the
|
||||
package manager, the container runtime, possibly the version control system, each with
|
||||
configuration somebody chose. The module does not install it; it takes it over
|
||||
([research 012](../012-the-minimum-viable-node/00-overview.md)).
|
||||
|
||||
**14 — Something that is a set, not a thing.** *Core infrastructure* is not installable — it is
|
||||
a name for the concrete modules a working node needs. Whether that is a module whose only
|
||||
content is `requires`, or a query over the graph, or a pinned list outside the catalogue, is
|
||||
undecided and is one of the sharper questions here.
|
||||
|
||||
**15 — Something with one instance for the whole mesh.** There is one mesh database, not one per
|
||||
node. Assigning it to two nodes is not redundancy, it is two meshes. Contrast with a terminal,
|
||||
where per-node is the only sensible reading.
|
||||
|
||||
**16 — Something that may be installed several times over.** Two terminals coexist happily. Two
|
||||
things wanting port 443 do not. Two message brokers might be fine or might be a split brain,
|
||||
and nothing in *provides* and *requires* distinguishes those.
|
||||
|
||||
**17 — Something that is not software at all.** A firewall policy. A DNS record. A certificate.
|
||||
It owns no binary, runs nothing, and is entirely *desired state* — which is the one case that
|
||||
fits the host's declaration model exactly and fits an installable-package model not at all.
|
||||
|
||||
**18 — An agent.** The mesh's own premise is that agents are participants. An agent has an
|
||||
identity, a licence, a node it runs on, and work it does. Whether that is a module, a record in
|
||||
the control plane, or something else is not obvious, and getting it wrong shapes everything
|
||||
about how agents are assigned.
|
||||
|
||||
**19 — The host itself.** Tier 0 installs everything else and is installed by hand. It is not a
|
||||
module, and a schema that cannot say so has a bootstrap problem hiding in it.
|
||||
|
||||
**20 — Something the mesh depends on and does not control.** A domain registrar, an upstream
|
||||
resolver, a certificate authority, an electricity supply. Almost certainly not modules — but
|
||||
the mesh's health depends on them, and they are the reason a node can be perfectly configured
|
||||
and still not work.
|
||||
|
||||
## What varies across the cases
|
||||
|
||||
The list above matters less than this. These are the axes a manifest has to express, and each
|
||||
one is a question the schema must answer or deliberately refuse.
|
||||
|
||||
| Axis | Range | Sharpest case |
|
||||
|---|---|---|
|
||||
| **Does it run?** | supervised · launched by a person · once · on a timer · never | 5, 6, 7, 17 |
|
||||
| **Who starts it?** | the mesh · a human · nothing | 1 vs 3 |
|
||||
| **Where does it come from?** | container image · system package · our source · already on the machine | 12, 13 |
|
||||
| **Does it provide to other modules?** | a resource · an abstract name · nothing | 8, 11 |
|
||||
| **Does it hold state?** | yes, and it matters where · no | 1 vs 3 |
|
||||
| **How many instances?** | one per mesh · one per node · many per node | 15, 16 |
|
||||
| **Can two coexist?** | yes · no · only with different settings | 16 |
|
||||
| **Is it installed or adopted?** | installed · adopted · either, depending on the machine | 13 |
|
||||
| **Does installing it change what the node provides?** | yes · no | 12 |
|
||||
|
||||
**Two of these are not in any current thinking**, and both come from the hard cases:
|
||||
*how many instances* (15) and *can two coexist* (16). `excludes` covers part of the second and
|
||||
nothing covers the first.
|
||||
|
||||
## Questions the cases raise
|
||||
|
||||
- **Is "runs" a property or a kind?** The axes suggest a property — one schema, with a field
|
||||
saying how it runs, `never` included. The alternative is several kinds of module with
|
||||
different schemas, which is the taxonomy [research 011](00-overview.md) already rejected once
|
||||
for services and applications.
|
||||
- **Is an agent a module?** (18) If yes, the schema carries identity and licensing. If no, the
|
||||
mesh has two catalogues.
|
||||
- **Is core infrastructure a module?** (14) A module whose only content is `requires` is either
|
||||
elegant or a category pretending to be a thing — the same trap `database` was.
|
||||
- **What names one-per-mesh?** (15) Nothing in provides, requires or excludes says it, and
|
||||
getting it wrong means two of something that must be one.
|
||||
- **Where does a policy live?** (17) Pure desired state fits the host's declaration exactly.
|
||||
Whether it is a module at all, or something the control plane derives and no catalogue entry
|
||||
exists for, is open.
|
||||
@@ -0,0 +1,131 @@
|
||||
# What a feature is, and what it splits into
|
||||
|
||||
Measured against `origin/main` of the code repository, 2026-08-26.
|
||||
|
||||
The operator wants the feature concept gone.
|
||||
[Research 006](../006-mesh-from-scratch/00-overview.md) left it open — *"does `feature`
|
||||
survive? The skeleton splits it in two and argues the conflation is what makes the delivery
|
||||
pipeline hard to reason about."*
|
||||
|
||||
This is what it actually is, and what it turns into.
|
||||
|
||||
## What it is today
|
||||
|
||||
A **feature** is a kind of content a module can carry, detected from what its directory
|
||||
contains rather than declared. Twenty-one of them, each with a handler that owns its whole
|
||||
lifecycle:
|
||||
|
||||
```
|
||||
configs dist-assets events hooks migrations migration-artifact npm npm-install
|
||||
prerequisite-env prerequisite-packages prerequisite-provision provision-migrations
|
||||
provision-seeds seeds service systemd tools verifiers verify vhost detect
|
||||
```
|
||||
|
||||
Each moves through **six stages**: build, publish, install, configure, start, verify.
|
||||
|
||||
## Finding — every handler implements every stage
|
||||
|
||||
The structural evidence, and it is not a style problem.
|
||||
|
||||
| handler | stages it implements |
|
||||
|---|---|
|
||||
| `configs` | build · configure · install · start · verify |
|
||||
| `service` | build · configure · install · start · verify |
|
||||
| `systemd` | build · configure · install · start · verify |
|
||||
| `tools` | build · configure · install · start · verify |
|
||||
| `vhost` | build · configure · install · start · verify |
|
||||
| `migrations` | build · configure · install · start · verify |
|
||||
| `npm` | build · publish · install · configure · start |
|
||||
|
||||
`configs` writes files onto a node. It has nothing to build, and it has a build stage.
|
||||
`npm` publishes a package to a registry. It has nothing to start, and it has a start stage.
|
||||
|
||||
**One interface spans build-time and apply-time, so every kind of content must implement both
|
||||
halves and most of them do nothing in one.** That is the conflation, and the emptiness is what
|
||||
makes the pipeline hard to reason about: a stage that does nothing and a stage that failed to
|
||||
do anything look identical from outside.
|
||||
|
||||
## What it splits into
|
||||
|
||||
The twenty-one are not one kind of thing. They are four, and they belong to different tiers.
|
||||
|
||||
**Artifacts — built once per version, then published.** `npm`, `dist-assets`,
|
||||
`migration-artifact`. Nothing about a node is involved; the output is a thing that exists in a
|
||||
registry. **Tier 2, delivery.**
|
||||
|
||||
**Resources — desired state on a machine.** `configs`, `service`, `systemd`, `vhost`, `tools`.
|
||||
Applied, converged, idempotent — which is exactly what
|
||||
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md)
|
||||
already describes and what the host already does. **Tier 0.**
|
||||
|
||||
**Actions — run once, against something that is not this machine.** `migrations`, `seeds`,
|
||||
`provision-migrations`, `provision-seeds`, `hooks`. A migration runs against a database, and the
|
||||
database may be on another node entirely. Neither an artifact nor node state, which is why they
|
||||
sit awkwardly in a scheme built for both. **Tier 2, and the operator wants seeds gone.**
|
||||
|
||||
**Checks — assertions, not changes.** `prerequisite-env`, `prerequisite-packages`,
|
||||
`prerequisite-provision`, `verify`, `verifiers`, `detect`. The prerequisites are **requirements
|
||||
in disguise** — a module saying what must be true before it can be installed, which is precisely
|
||||
what an edge in the graph says. The verifiers are the read-back the host already performs.
|
||||
|
||||
## So the answer is: it splits, and the split is a tier boundary
|
||||
|
||||
**`feature` is one word for four things spanning three tiers.** That is why every handler
|
||||
implements every stage, why half of them are empty, and why the pipeline is hard to reason
|
||||
about.
|
||||
|
||||
Nothing replaces it, because it was never one concept:
|
||||
|
||||
| Was a feature | Becomes | Whose |
|
||||
|---|---|---|
|
||||
| npm, dist-assets, migration-artifact | an **artifact** | delivery |
|
||||
| configs, service, systemd, vhost, tools | a **resource** in a declaration | the host |
|
||||
| migrations, hooks | an **action** against something else | delivery |
|
||||
| seeds | *nothing* — the operator wants them gone | — |
|
||||
| prerequisite-* | an **edge** in the graph | the catalogue |
|
||||
| verify, verifiers | the host's **read-back** | the host |
|
||||
|
||||
## Tools are a facet, and their consumer is not a module
|
||||
|
||||
The most common content in the catalogue: 56 of 126 modules carry a tool surface, more than
|
||||
carry a service.
|
||||
|
||||
It survives the split, and it does not fit either half cleanly. A tool is not an artifact and
|
||||
not node state — it is a **contract the mesh publishes on a module's behalf**, and what consumes
|
||||
it is an **agent**, not another module.
|
||||
|
||||
That is a second audience, and the design has only described one. A module `provides` things
|
||||
other modules require; a module also `provides` things agents call. Same word, different
|
||||
consumer, different lifecycle — a tool appears when the module is assigned somewhere and
|
||||
disappears when it is not, and nothing in the graph edges says so.
|
||||
|
||||
Whether that is one relation with two audiences or two relations is undecided, and it is the
|
||||
kind of question that is cheap now and expensive later.
|
||||
|
||||
## What is lost, and should not be
|
||||
|
||||
**Detection.** Features are detected from the directory rather than declared, and the reason is
|
||||
good: *a declared list and the directory it describes drift, and the directory is the one that
|
||||
is true.* That property is worth keeping whatever the concept is called — a module that says it
|
||||
has migrations and has none, or has them and does not say so, is a fault nobody sees until it
|
||||
matters.
|
||||
|
||||
Under the split, detection still applies: what a module **contains** is read from what is there.
|
||||
What it **requires**, **provides** and **excludes** is declared, because none of that is visible
|
||||
in a directory.
|
||||
|
||||
That line is worth stating precisely, because it is the one the current design got right:
|
||||
**content is detected, relationships are declared.**
|
||||
|
||||
## Open
|
||||
|
||||
- **Is an action a resource?** A migration is not node state and not an artifact. It could be a
|
||||
resource type the host applies with the target being a database rather than a machine — which
|
||||
would collapse the third category into the second, at the cost of the host reaching something
|
||||
that is not the machine it is on. That cost looks too high, but it has not been argued.
|
||||
- **Where do hooks go?** They are arbitrary code a module runs at a stage — the escape hatch. A
|
||||
design with no escape hatch is either very good or has not met reality yet, and this one has
|
||||
not.
|
||||
- **What does the pipeline schedule, once features are gone?** It currently schedules features.
|
||||
If the four categories have different lifecycles, the unit of work is different for each, and
|
||||
what the coordinator orchestrates needs naming.
|
||||
@@ -0,0 +1,167 @@
|
||||
# One kind of edge
|
||||
|
||||
> **Superseded in part by [`worked-provider.md`](worked-provider.md).** Working postgres through
|
||||
> completely shows there are **two** kinds of edge, not one: *presence* — the thing exists and is
|
||||
> reachable, nothing created — and *instantiation* — the provider is asked to make something for
|
||||
> this consumer and hands back credentials. Instantiation implies presence; presence does not
|
||||
> imply instantiation. Everything else below stands; the claim in the title does not.
|
||||
|
||||
A design, not an account of what exists. [`analysis.md`](analysis.md) measured the current
|
||||
catalogue and its value here is two lessons rather than its machinery: a field that means
|
||||
*depends on* should say so, and placement does not belong in a manifest.
|
||||
|
||||
## The idea
|
||||
|
||||
**A module provides names. A module requires names. That is the only edge.**
|
||||
|
||||
Everything the effort listed as separate declarations turns out to be one relation with
|
||||
different kinds of name on either end.
|
||||
|
||||
```
|
||||
postgres provides postgres
|
||||
kitty provides kitty, terminal
|
||||
xterm provides xterm, terminal
|
||||
anthropic provides anthropic, ai-assistant
|
||||
|
||||
meshboard requires postgres, lavinmq
|
||||
vscode requires terminal, display-server
|
||||
```
|
||||
|
||||
A name is either **concrete** — a module's own name, so `requires: postgres` means that module
|
||||
and no other — or **abstract**, so `requires: terminal` means whatever provides it.
|
||||
|
||||
That single relation absorbs three things the effort had listed separately:
|
||||
|
||||
| Was going to be | Is |
|
||||
|---|---|
|
||||
| requires another module | requires a concrete name |
|
||||
| requires a resource | requires the module that provides it, by name |
|
||||
| an interface, with adapters providing it | an abstract name, legitimate only where an adapter makes providers substitutable |
|
||||
|
||||
The interface stops being a kind of module. It is a name with more than one provider *and a
|
||||
contract they all satisfy*, and nothing has to declare that it is one.
|
||||
|
||||
### An abstract name is only legitimate when providers are actually substitutable
|
||||
|
||||
The test, and it is a strict one: **can a consumer be switched from one provider to another
|
||||
without changing?**
|
||||
|
||||
`terminal` passes. Anything that runs a command in a terminal works, and a consumer never
|
||||
learns which one it got.
|
||||
|
||||
**`database` fails, and it is the example worth keeping.** A module connecting to Postgres does
|
||||
not connect to MongoDB, or to SQL Server, or to MariaDB. Different wire protocol, different
|
||||
dialect, different driver. A consumer that declared `requires: database` and was handed any of
|
||||
them would break — so the name promises something no provider can deliver, and the resolver
|
||||
would satisfy a requirement that is not satisfied.
|
||||
|
||||
That failure has a shape this repository already knows: something declared, accepted, and not
|
||||
true. It is worse here than usual because the graph would report success.
|
||||
|
||||
**The adapter is what creates an interface.** `ai-assistant` is a legitimate abstract name
|
||||
exactly when adapters exist to normalise the providers behind it. Without an adapter there is
|
||||
no interface — there is a category, and a category is not an edge.
|
||||
|
||||
So `postgres` is required by name, and a module that could genuinely work with several stores
|
||||
requires whichever it actually speaks to.
|
||||
|
||||
### Categories are catalogue metadata, not structure
|
||||
|
||||
*Database* is still a useful word — for finding things, for a person browsing what the mesh can
|
||||
host, for grouping in an interface. It is a **tag**.
|
||||
|
||||
Tags describe. Edges bind. Keeping them apart is what stops the catalogue acquiring a second
|
||||
kind of relationship that looks like a dependency and is not — which is what a folder named
|
||||
after a domain already was.
|
||||
|
||||
## The node provides names too
|
||||
|
||||
The move that makes capabilities stop being a separate system.
|
||||
|
||||
A node's profile is a set of provided names. `display-server`. `container-runtime`.
|
||||
`amd64`. A module requiring `display-server` is satisfied by **the node**, exactly as a module
|
||||
requiring `postgres` is satisfied by another module.
|
||||
|
||||
So there is one resolution rather than two: *is this name provided by anything available here?*
|
||||
A graphical application cannot be installed on a node with no display server for the same
|
||||
reason, and through the same code, that it cannot be installed without its libraries.
|
||||
|
||||
And it means the host's capability detection — which already reports what a machine can be
|
||||
asked to do, with a reason for each verdict — **is the node's provides-list**. It was built to
|
||||
be read by a person; it turns out to be an input to resolution.
|
||||
|
||||
## What a module declares
|
||||
|
||||
```yaml
|
||||
name: meshboard
|
||||
|
||||
provides: [meshboard]
|
||||
requires: [postgres, lavinmq, container-runtime] # by name: it speaks their protocols
|
||||
excludes: []
|
||||
tags: [observability] # for finding it, never for resolving it
|
||||
|
||||
runs: # what applying it means
|
||||
- container: ...
|
||||
constraints: # what must be true of a node, never which node
|
||||
- architecture: amd64
|
||||
```
|
||||
|
||||
`postgres` and `lavinmq` are concrete because this module speaks their wire protocols and would
|
||||
break against anything else. `container-runtime` is abstract and satisfied by the **node**.
|
||||
|
||||
**`excludes`** is the one genuinely new relation: naming something that cannot coexist with
|
||||
this. It is not derivable from requires and provides, and without it two modules that both
|
||||
provide `terminal` — or two that both want port 443 — look independent right up until
|
||||
installing the second breaks the first.
|
||||
|
||||
**`constraints` are not placement.** They say what must be true of a node, never which node.
|
||||
Which node runs what is an inventory decision, and the measurement found the current catalogue
|
||||
deciding it in the manifest — a module pinning its database to a named node, so that a second
|
||||
node cannot provide it without editing the module that consumes it. That is the mistake this
|
||||
separation exists to avoid.
|
||||
|
||||
## What it deletes
|
||||
|
||||
The question the effort opened with, answered for the design rather than for what exists.
|
||||
|
||||
- **The distinction between requiring a module and requiring a resource.** One relation.
|
||||
- **The interface as a kind of thing.** A name with several providers.
|
||||
- **Capability checking as a separate mechanism.** The node is a provider.
|
||||
- **The domain module** — a `networking` module that exists to gather a firewall, a resolver and
|
||||
a proxy under one name. It came from an older shape and does not fit: there is no such thing
|
||||
to install. There is **core infrastructure**, which is a set of concrete modules named
|
||||
individually — a firewall, a store, a resolver — with no flavour and no grouping module
|
||||
standing in front of them.
|
||||
- **Domain grouping as structure** ([ADR 0009](../../02-DECISIONS/0009-modules-and-the-graph.md)).
|
||||
Folders assert relationships; edges record them. What grouping was for — finding things,
|
||||
seeing what belongs together — is a **tag** and a *query* over the graph, neither of which
|
||||
anybody has to keep true by hand.
|
||||
- **Tiers as a separate concept**, possibly. A tier is a level in the graph, and levels are
|
||||
computed. Whether the coarse boundary is still worth naming is below.
|
||||
|
||||
## What it does not delete, and should not
|
||||
|
||||
**A resolver still has to exist**, and it will need version constraints, conflict handling and
|
||||
a story for when two modules provide one name and nothing says which to use. That is the
|
||||
long-solved and easily-botched part, and the design should say what it **delegates** to the
|
||||
platform's package manager rather than reimplementing.
|
||||
|
||||
## Open
|
||||
|
||||
- **When two modules provide one abstract name, who chooses?** A node with both `kitty` and
|
||||
`xterm` satisfies `terminal` twice. Whether the choice is a node setting, a mesh setting, or
|
||||
an explicit pin is the substance of the interface idea, and this proposal does not settle it.
|
||||
It is a smaller question than it was: the strict substitutability test means the consumer
|
||||
genuinely does not care which it gets, so the choice is about preference rather than
|
||||
correctness.
|
||||
- **Are tiers a view of the graph, or a boundary that survives it?** If tiers are levels, the
|
||||
concept is derived and the word is a convenience. If the tier rule — *a tier may depend only
|
||||
on tiers below it* — is meant to be enforceable, it is a constraint on the graph rather than
|
||||
a description of it, and it has to be stated as one.
|
||||
- **What does a provider hand back?** A consumer requiring `postgres` needs credentials and an
|
||||
address; one requiring `terminal` needs a command. The requirement is satisfied by the same
|
||||
relation in both cases, and what flows across it is not the same shape. Whether that lives in
|
||||
the name, beside it, or in what the provider returns is undecided.
|
||||
- **What does a node provide that is not a capability?** Its architecture is a provided name
|
||||
under this scheme, and so is its operating system. That may be elegant or may be one
|
||||
abstraction too far; nothing here tests it.
|
||||
@@ -0,0 +1,491 @@
|
||||
# A provider, all the way through
|
||||
|
||||
One module, worked out completely, because it is the case that breaks the tidy version. It
|
||||
looks like *a container that runs a database* and it is at least nine things.
|
||||
|
||||
Postgres is the example. **The shape is not specific to it** — see the end.
|
||||
|
||||
## What it carries
|
||||
|
||||
**1 — A supervised container.** An image, a version, a data volume, and configuration. Easy,
|
||||
and the only part the phrase "a docker service" describes.
|
||||
|
||||
**2 — Persistent state, and where it lives matters.** The volume is the database. Moving this
|
||||
module between nodes is not rescheduling; it is a migration. Almost nothing else in the
|
||||
catalogue has this property, and nothing in `provides` / `requires` expresses it.
|
||||
|
||||
**3 — Configuration that is partly the machine's.** Tuning follows the hardware — memory,
|
||||
storage. A declaration generated centrally cannot know those, and
|
||||
[research 012](../012-the-minimum-viable-node/00-overview.md) says the machine's own values win
|
||||
on conflict. So some of this module's configuration is *derived from the node it lands on*.
|
||||
|
||||
**4 — A tool surface.** It exposes capabilities agents can call — query, list, provision. That
|
||||
is not a resource on a machine and not an artifact; it is a contract the mesh publishes on the
|
||||
module's behalf.
|
||||
|
||||
**5 — A provisioner.** The part that matters, and the one below.
|
||||
|
||||
**6 — Its own bookkeeping.** The provisioner must remember what it granted to whom, or it
|
||||
cannot revoke, rotate or clean up. So a module that provides state to others *also* holds state
|
||||
about its providing — and that state is not the database's data.
|
||||
|
||||
**7 — An exposure decision, per node it runs on.** Reachable from the machine only, from the
|
||||
local network, or publicly. That is a property of *this assignment*, not of the module: the
|
||||
same module on two nodes may answer differently.
|
||||
|
||||
**8 — Credentials it generates.** Per consumer, and they have to reach the consumer. Which
|
||||
means a provisioning edge carries a payload, and the payload is a secret.
|
||||
|
||||
**9 — Health that is not "the container is up".** A container running and a database accepting
|
||||
connections are different facts, and the second is the one anything cares about. This is the
|
||||
host's read-back rule, at a distance.
|
||||
|
||||
## The provisioner is a second kind of edge
|
||||
|
||||
The tidy version of this effort said: *a module provides names, a module requires names, that is
|
||||
the only edge.* A small game wanting to store data shows it is not.
|
||||
|
||||
```
|
||||
my-cool-game requires postgres # I speak its protocol, it must exist
|
||||
my-cool-game requires a database FROM postgres, called my-cool-game
|
||||
```
|
||||
|
||||
The first is **presence**: the thing exists and is reachable. Nothing is created; nothing flows
|
||||
back. `vscode requires terminal` is this, and so is `requires container-runtime`.
|
||||
|
||||
The second is **instantiation**: the provider is asked to make something *for this consumer*,
|
||||
and hands back what the consumer needs to use it. A database, a user, a password, an address.
|
||||
|
||||
They differ in every way that matters:
|
||||
|
||||
| | presence | instantiation |
|
||||
|---|---|---|
|
||||
| creates something | no | yes, one per consumer |
|
||||
| carries a payload back | no | credentials, an address |
|
||||
| can be revoked | — | yes, and must be when the consumer goes |
|
||||
| provider holds state about it | no | yes — who was granted what |
|
||||
| satisfied by | anything providing the name | that provider, specifically |
|
||||
|
||||
**This is the mesh's actual power**, in the operator's words: a small game declares it wants a
|
||||
database and the mesh makes one. Nobody creates a user by hand, nobody pastes a connection
|
||||
string. That is worth being precise about rather than folding into a single relation because
|
||||
one relation is prettier.
|
||||
|
||||
## What that costs the design
|
||||
|
||||
**The proposal's "one kind of edge" is wrong**, and the current system already knew: it has
|
||||
`dependencies` for presence and `requires: provision:` for instantiation, with the resolver
|
||||
deriving a presence edge from every instantiation edge. [`analysis.md`](analysis.md) recorded
|
||||
that derivation as a convenience. It is not — it is the correct relationship between two
|
||||
genuinely different relations.
|
||||
|
||||
So: **two kinds of edge, one graph.** Instantiation implies presence. Presence does not imply
|
||||
instantiation.
|
||||
|
||||
## Which provider, when two nodes run one
|
||||
|
||||
Asked directly, because two nodes can each run a relational store and a consumer has to be
|
||||
served by one of them. Neither obvious answer is right.
|
||||
|
||||
**Not "the consumer names the node."** That is placement in the consumer's manifest — a small
|
||||
game edited because a database moved, which is the fault
|
||||
[`proposal.md`](proposal.md) separates constraints from placement to avoid.
|
||||
|
||||
**Not "the consumer does not care" either.** For presence it genuinely does not: a terminal is a
|
||||
terminal. For instantiation it cares permanently, because the data lands in exactly one store
|
||||
and the wrong choice is discovered long afterwards.
|
||||
|
||||
**What the consumer does know is the scope of its own need.** Not which node — how many of the
|
||||
thing it wants, relative to itself:
|
||||
|
||||
| Scope | Means | Example |
|
||||
|---|---|---|
|
||||
| **shared** | one instance serves every instance of this consumer | the mesh's own registry: every node reads the same rows |
|
||||
| **per instance** | each instance of this consumer gets its own | a local cache, a per-node queue |
|
||||
|
||||
That is a property of the consumer, expressible without naming anything. And it is the thing
|
||||
that actually decides: a shared need cannot be satisfied by a provider each node runs
|
||||
separately, and a per-instance need should not be satisfied by a shared one.
|
||||
|
||||
**Then the mesh binds, and the binding is written down.** Not recomputed: a resolver that
|
||||
re-derives which store serves a consumer will one day derive a different answer and relocate a
|
||||
database, so the binding is made once and changed only deliberately.
|
||||
|
||||
**Where it is written down follows the scope.** A shared grant belongs to the *module* and every
|
||||
assignment of it references the same one — which is the answer for one module installed on two
|
||||
nodes wanting one database between them. A per-instance grant belongs to the *assignment*. Same
|
||||
relation, two homes, and which home is not a detail: it is what makes two instances share
|
||||
something or not.
|
||||
|
||||
The pieces that follow, none of them settled here:
|
||||
|
||||
- **When several providers satisfy the scope**, something chooses — most plausibly locality,
|
||||
preferring a provider on the same node. That is a default, and it must be overridable, because
|
||||
the reason to override it is exactly the reason nobody anticipated it.
|
||||
- **A binding is a thing that can be wrong.** Once recorded it can be inspected, and a consumer
|
||||
bound to a store on a node that no longer exists is a question somebody can be asked rather
|
||||
than a failure at connect time.
|
||||
- **Moving a binding moves data.** Whatever the mechanism, changing it is a migration and not a
|
||||
configuration change, and a design that lets it look like the latter will lose something.
|
||||
|
||||
## Provisioning is early, not late
|
||||
|
||||
An assumption worth killing: that provisioning is something the control plane does for
|
||||
consumers once a mesh is running.
|
||||
|
||||
**The mesh's own registry is a provisioned database.** So is its virtual host on the broker.
|
||||
Neither exists until something creates them, and nothing in the mesh works until they do. The
|
||||
order is:
|
||||
|
||||
```
|
||||
1 the store runs from the bundle the host carries
|
||||
2 a database is created in it a provisioning step
|
||||
3 the mesh's own schema is applied a migration, against that database
|
||||
4 the control plane starts and only now is there a mesh
|
||||
5 everything else is provisioned the ordinary path
|
||||
```
|
||||
|
||||
Steps 2 and 3 happen **before there is a mesh to do them**. So provisioning is not a
|
||||
control-plane service that consumers use; it is part of the bootstrap, and part of what the
|
||||
carried bundle has to be able to express.
|
||||
|
||||
**Which strains what a declaration is.** [ADR 0005](../../02-DECISIONS/0005-the-node-host.md)
|
||||
has the host applying *declared state on this machine*. A database inside a running store is not
|
||||
a file or a unit — and at bootstrap it is, at least, local: the store is on the same machine as
|
||||
the host applying the bundle.
|
||||
|
||||
Later it is not. A consumer on one node provisioned from a store on another is the ordinary
|
||||
case, and reaching it is not the host's job.
|
||||
|
||||
**Resolved as two mechanisms, which is the answer rather than a compromise**
|
||||
([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)). The host
|
||||
runs bootstrap actions locally from the bundle; the control plane provisions across the mesh
|
||||
afterwards. Different actors, different scopes, different trust paths — so there is no single
|
||||
operation with a tier boundary running through it.
|
||||
|
||||
## Several modules, one database — and the case for refusing
|
||||
|
||||
Asked, then reconsidered by the operator: *maybe we should not allow it.*
|
||||
|
||||
The permissive version was a per-consumer **schema** inside a shared database — its own
|
||||
namespace, its own migrations, revocable by dropping the schema, with a cross-context join
|
||||
possible but deliberate.
|
||||
|
||||
**The stricter version is better, and it goes further than schemas.**
|
||||
|
||||
> **A module is only ever granted a resource it exclusively owns.**
|
||||
|
||||
No shared writes. And **no read-only role on another module's database either** — reading
|
||||
another context's tables couples you to its layout exactly as firmly as writing them does, and
|
||||
the coupling is harder to see because nothing breaks until the owner changes a column.
|
||||
|
||||
That is what [`how-we-build`](../../00-META/how-we-build.md) §4 already says: *contexts integrate
|
||||
through the record, never through a shared schema.* The permissive version kept the letter of it
|
||||
and left the temptation in place, and the path of least resistance wins eventually. A boundary
|
||||
that is merely inconvenient to cross is a boundary that gets crossed.
|
||||
|
||||
### What it costs
|
||||
|
||||
**Cross-module reporting.** Anything wanting to know what several modules hold can no longer
|
||||
join across them. It consumes their events, or calls their interface, and neither is as
|
||||
immediate as a query.
|
||||
|
||||
That cost is the point rather than a regrettable side effect — it is §4's whole argument, and
|
||||
the mesh already has both mechanisms: an event stream every context publishes to, and a tool
|
||||
surface every module exposes. What gets harder is the thing that was making work belonging to
|
||||
one context keep having to be implemented in another.
|
||||
|
||||
**One more connection per consumer.** A dozen modules means a dozen databases rather than a
|
||||
dozen schemas in one. For a relational store this is unremarkable; it is worth stating only so
|
||||
nobody discovers it as a surprise.
|
||||
|
||||
### What it deletes
|
||||
|
||||
The effort has been looking for what the design *removes* rather than adds, and this is the
|
||||
first clear instance:
|
||||
|
||||
- **Grant kinds.** There is one — an exclusive resource. No schema grants, no read roles, no
|
||||
scoping rules for who may see what inside a shared thing.
|
||||
- **The question of who owns which table**, and with it the guessing at revocation time.
|
||||
- **Cross-module migration ordering.** Two modules migrating one database need their migrations
|
||||
ordered against each other. Exclusive ownership means a module's migrations are ordered only
|
||||
against itself.
|
||||
- **A whole class of permission modelling** that a shared store would otherwise need.
|
||||
|
||||
### The rule is about contexts, not processes
|
||||
|
||||
An earlier version of this file argued that a dashboard reading a dozen stores was caught by the
|
||||
rule, because a dashboard is a surface and surfaces speak to an interface. **That was wrong, and
|
||||
it drew the line in the wrong place.**
|
||||
|
||||
The mesh's own board showing nodes, modules and deployments is not a separate context reaching
|
||||
across a boundary. It is the mesh showing its own data. Reading that store is not a violation
|
||||
however it is done, and requiring it to go through an interface to reach facts its own context
|
||||
owns would be ceremony.
|
||||
|
||||
**The rule is: a context is granted what it exclusively owns.** Everything inside that context —
|
||||
its service, its surface, its tools — reads it freely. What is forbidden is a *different*
|
||||
context reading it.
|
||||
|
||||
Which is exactly what the count below shows: the problem was never surfaces. It was three other
|
||||
contexts keeping their tables in the mesh's database.
|
||||
|
||||
**And one surface over several contexts is normal.** The board visualises the mesh, the work
|
||||
engine, the knowledge base and more, and the alternative — a separate web application per
|
||||
context — is worse for everyone who uses it. That is not a compromise with the rule; composing
|
||||
several sources into one view is what a surface *is*.
|
||||
|
||||
What it changes is only where it reads from: each context's **interface**, not each context's
|
||||
**store**. Most of that already exists — 56 of 126 modules carry a tool surface, more than carry
|
||||
a service.
|
||||
|
||||
**And the unified board is what keeps those interfaces honest.** If a view cannot be built from
|
||||
a context's interface, that interface is inadequate — discovered in the one place where it is
|
||||
cheap to notice, rather than the first time something else needs the same data and quietly
|
||||
reaches for the store instead.
|
||||
|
||||
**But "the board calls each context's interface" is not quite it either**, and the objection is
|
||||
right: if every context runs its own service with its own interface, the board is coupled to N
|
||||
of them instead of N schemas, something has to compose them, and composition is logic — which
|
||||
tier 3 says a surface does not hold. That moves the problem up a layer rather than solving it.
|
||||
|
||||
**The skeleton already answers this and the argument above talked past it.** `work` and
|
||||
`knowledge` are not separate services; they are **contexts inside the control plane**, alongside
|
||||
the record, inventory, delivery and the rest — and `api` is listed there as *the one interface
|
||||
every surface speaks to*.
|
||||
|
||||
So the board speaks to **one** interface. Behind it the contexts stay separate: separate stores,
|
||||
integrating through the record. But they are one tier, one repository, one deployable, and
|
||||
coupling *within* a tier is not what the tier rule forbids.
|
||||
|
||||
Which resolves the objection rather than deflecting it: the problem does move up a layer, and
|
||||
the layer it moves to already exists and has this as its job.
|
||||
|
||||
**The caveat is load-bearing.** This holds only while the contexts are not separate deployables.
|
||||
The moment one becomes its own service with its own interface, the board is back to N clients,
|
||||
something must compose them, and the composition has nowhere to live that tier 3 permits. That
|
||||
is a real constraint on how far the control plane may be split, and it is worth knowing now
|
||||
rather than discovering it by splitting.
|
||||
|
||||
If composing even one interface turns out too slow, the answer is a projection the board owns
|
||||
and keeps current from events — not access to somebody else's tables.
|
||||
|
||||
### Checked against the real consumers
|
||||
|
||||
The rule's survival turned on whether every reader of the mesh's own registry could be served
|
||||
some other way. **Eighteen consumers open a direct connection to it.** Four groups, and only one
|
||||
of them is work.
|
||||
|
||||
| Group | What happens under the rule |
|
||||
|---|---|
|
||||
| **The owner and its machinery** — the mesh module, the SDK, the environment and configuration synchronisers, secrets | Nothing. It owns the database. |
|
||||
| **Node appliers** — the overlay, the shell daemon, the resolver | **Already resolved.** [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) stops the host querying the mesh database, decided for tier reasons with nothing to do with this. |
|
||||
| **Foreign tenants** — the work engine (10 tables), the knowledge base (2), pipeline logs (1) | They need **their own database**. They are not reading the registry; they are storing their own data in it. |
|
||||
| **Genuine cross-context reads** — the work engine reads `nodes`; two others read a handful | The only ones needing an interface or events. |
|
||||
|
||||
**Thirteen foreign tables live in the mesh's registry database**, belonging to three separate
|
||||
contexts. That is [`how-we-build`](../../00-META/how-we-build.md) §4's shared schema, counted.
|
||||
|
||||
**So the question was the wrong shape.** The bulk of the problem is not readers needing a new
|
||||
route to data — it is **tenants needing to move out**. Tasks, agents and teams have nothing to do
|
||||
with nodes and modules; they are co-located by history. Give that context its own database and
|
||||
its dependency on the registry shrinks to a single table.
|
||||
|
||||
What remains is a handful of genuine cross-context reads, small enough to enumerate rather than
|
||||
estimate. **The rule holds.**
|
||||
|
||||
### Request or subscription, and what decides
|
||||
|
||||
The remaining cross-context reads need one or the other. **Neither is SQL** — under exclusive
|
||||
ownership a module runs SQL against its own database and nothing else, whatever transport a
|
||||
query might travel over. Both options are the mesh's own channel, and both ride the broker
|
||||
([ADR 0002](../../02-DECISIONS/0002-nodes-communicate-over-a-broker.md)), so the transport is
|
||||
not the distinction.
|
||||
|
||||
**The distinction is where the answer lives when you need it.**
|
||||
|
||||
| | request | subscription |
|
||||
|---|---|---|
|
||||
| you ask | at the moment you need to know | never — you are told |
|
||||
| the answer lives | on the other side | in your own store |
|
||||
| freshness | always current | as current as the last event you received |
|
||||
| when the other side is down | you cannot answer | you answer from your copy |
|
||||
| what you must handle | a round trip that can fail | events you missed while you were down |
|
||||
|
||||
**What decides is not taste.** [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)
|
||||
makes disconnection an ordinary situation rather than an exception. So:
|
||||
|
||||
> **Anything that must keep working while disconnected cannot use a request** — there is nobody
|
||||
> to ask. It needs a local copy, which means a subscription.
|
||||
|
||||
And the converse: anything that must be *correct at the instant of asking*, where a stale answer
|
||||
is worse than no answer, cannot use a subscription. A display can lag. A decision about whether
|
||||
a grant is still valid cannot.
|
||||
|
||||
That turns an apparently open question into a derived one. What remains genuinely open is
|
||||
narrower: **what a consumer does about the events it missed** while it was disconnected — replay
|
||||
from a point, ask once for a full picture and resume, or rebuild from scratch. That is the same
|
||||
question every projection has, and nothing in the record answers it yet.
|
||||
|
||||
## A migration belongs to the consumer and runs on the provider
|
||||
|
||||
A game defines migrations. They run against the database the store granted **it**. So the
|
||||
migration is:
|
||||
|
||||
- **owned** by the consumer — it is that module's schema, versioned with that module;
|
||||
- **hosted** by the provider — it runs inside something the consumer does not control;
|
||||
- **ordered** after the provisioning edge — there is nothing to migrate until the grant exists;
|
||||
- **scoped** to the grant — the consumer's migrations touch its database and no other.
|
||||
|
||||
Ownership crosses the edge, which nothing in *provides* and *requires* expresses. And it is the
|
||||
*action* category from [`features.md`](features.md) made concrete: not an artifact, not node
|
||||
state, and not something the host can apply, because the thing it changes is not the machine.
|
||||
|
||||
It also gives a consumer's own install an internal order — **provisioned, then migrated, then
|
||||
started** — that depends on an edge rather than on the module's contents.
|
||||
|
||||
## What still has no answer
|
||||
|
||||
**How many instances of postgres should exist?** One per mesh is wrong — a node that must work
|
||||
while disconnected cannot depend on a database elsewhere. One per node is wrong — the mesh's own
|
||||
registry is one thing, not one per node. So the answer is per-module, and nothing in the schema
|
||||
says it. This is [`cases.md`](cases.md) axis *how many instances*, and postgres is the case that
|
||||
proves it cannot be a global rule.
|
||||
|
||||
**What happens to a grant when the consumer is removed?** The game is uninstalled. Its database
|
||||
still exists, holding its data. Dropping it silently is data loss; keeping it forever is a leak.
|
||||
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md) says
|
||||
the host removes what it applied and no longer declares — but this is not on the host, it is
|
||||
inside another module's state, and the same reasoning does not obviously carry.
|
||||
|
||||
**Where does node-derived configuration come from?** (3) The control plane composes a
|
||||
declaration, and cannot know this machine's memory. Either the host fills in a blank the
|
||||
declaration leaves — which makes the host decide something, against
|
||||
[ADR 0005](../../02-DECISIONS/0005-the-node-host.md) — or the control
|
||||
plane reads the node's inventory first and composes with it. The second is consistent and means
|
||||
a declaration is composed *per node from what the node reported*, which is a stronger claim than
|
||||
anything recorded so far.
|
||||
|
||||
|
||||
## Providing is not a substrate thing
|
||||
|
||||
The four pinned services are the obvious providers, and they are not a category.
|
||||
|
||||
| Service | What a consumer asks it for |
|
||||
|---|---|
|
||||
| a relational store | a database, a user, credentials |
|
||||
| another relational store, different vendor | a database — **and not the same one** |
|
||||
| a message broker | a virtual host, a user, permissions |
|
||||
| an object store | a bucket and keys |
|
||||
| an image registry | a repository |
|
||||
| an identity provider | a client, a realm, a secret |
|
||||
| an analytics service | a site, and a tracking identity |
|
||||
| a low-code data platform | a base, and a token |
|
||||
| an application platform | a project, which is several of the above at once |
|
||||
| a mail server | a mailbox, an alias, credentials |
|
||||
|
||||
**Any hosted service can be a factory.** Providing is a facet a module may have, not a kind of
|
||||
module it is — which is the same conclusion the effort reached about services and applications,
|
||||
arriving from the other direction.
|
||||
|
||||
That kills the last reason to keep *provider* as a category. A module runs something, or grants
|
||||
something, or both, or neither.
|
||||
|
||||
### Two stores, and why `database` still is not a name
|
||||
|
||||
Two relational stores from different vendors both grant *a database*. They are the sharpest
|
||||
possible test of the substitutability rule from [`proposal.md`](proposal.md), and they fail it
|
||||
completely: different wire protocol, different dialect, different driver, different client
|
||||
library compiled into the consumer.
|
||||
|
||||
A consumer declaring `requires: database` and being handed either would break against one of
|
||||
them. So the name promises what no provider delivers — and now with two real providers in the
|
||||
catalogue rather than a thought experiment.
|
||||
|
||||
*Database* remains a **tag**. It is how a person finds both. It is not how a consumer names what
|
||||
it needs.
|
||||
|
||||
## The same shape, three more times
|
||||
|
||||
The message broker has all nine. So does the object store, and so does the image registry. They
|
||||
differ in what a consumer asks for — a database, a virtual host, a bucket, a repository — and in
|
||||
nothing structural.
|
||||
|
||||
**A substrate service is a service plus a factory.** That is the whole pattern, and there are
|
||||
four of them. It generalises past the substrate too: anything that grants something per consumer
|
||||
has this shape, and anything that does not is the simpler case.
|
||||
|
||||
But two things differ *between* them, and both matter more than the similarity.
|
||||
|
||||
### The broker cannot be managed over the broker
|
||||
|
||||
[ADR 0002](../../02-DECISIONS/0002-nodes-communicate-over-a-broker.md) makes the broker the
|
||||
channel every node takes work from, and
|
||||
[ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md) makes it the security
|
||||
boundary — everything a node applies arrives through it.
|
||||
|
||||
So the module providing the broker is also **the way modules are managed**. A declaration cannot
|
||||
be delivered to it over itself, and reconfiguring it is done through the thing being
|
||||
reconfigured. Nothing else in the catalogue has that property; the store is consumed by the
|
||||
control plane but is not how the control plane *reaches* anything.
|
||||
|
||||
This is exactly what the carried bundle exists for
|
||||
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)): the broker is raised from
|
||||
what the host carries, before there is a channel, because there is no other way to raise it.
|
||||
Recorded here because it is a constraint on *one module*, not a general rule, and a schema with
|
||||
no way to say so hides it.
|
||||
|
||||
### Two modules of identical shape want different instance counts
|
||||
|
||||
The broker is one per mesh — a single point of failure and a single point of trust, by decision
|
||||
rather than by accident. The store cannot be: a node that must keep working while disconnected
|
||||
([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)) cannot depend on a database
|
||||
somewhere else.
|
||||
|
||||
Same nine properties, opposite answers. Which settles something the cases file left open: **how
|
||||
many instances is not derivable from what a module is.** It is a decision per module, it has to
|
||||
be declared, and nothing in `provides`, `requires` or `excludes` says it.
|
||||
|
||||
### And revocation differs in consequence
|
||||
|
||||
Revoking a database leaves data behind until something drops it — a leak, and recoverable.
|
||||
Revoking a virtual host drops whatever had not been delivered — not recoverable, and silent.
|
||||
|
||||
The relation is the same and the blast radius is not, which is an argument for the provider
|
||||
deciding what revocation means rather than the mesh applying one rule to all of them.
|
||||
|
||||
|
||||
## The assignment is a third thing
|
||||
|
||||
Recorded because the operator tried the alternative and abandoned it: **modules were once
|
||||
node-agnostic**, and it did not survive contact.
|
||||
|
||||
The worked example says why. Several of a provider's nine properties are not properties of the
|
||||
module at all:
|
||||
|
||||
- **where its state lives** — a volume on a particular machine;
|
||||
- **how it is reached** — the same module on two nodes may answer locally, on the network, or
|
||||
publicly, and that is a per-assignment decision;
|
||||
- **configuration derived from the hardware** — tuning follows the memory and storage of the
|
||||
machine it landed on;
|
||||
- **whether this instance is the one** a given consumer is provisioned from.
|
||||
|
||||
None of those belong in the catalogue, because they differ per node. None belong in the node,
|
||||
because they are about this module. **They belong to the pairing**, and a design with only
|
||||
modules and nodes has nowhere to put them — which is what "node-agnostic" ran out of.
|
||||
|
||||
So there are three entities, not two:
|
||||
|
||||
> **a module** · **a node** · **an assignment**, which is a module on a node and carries its own
|
||||
> configuration
|
||||
|
||||
The current system already has this, arrived at the same way: environment values are stored per
|
||||
module *and per node*, so a module's settings differ between the machines running it.
|
||||
|
||||
**This does not put placement back in the manifest.** A module still says what must be true of a
|
||||
node and never which node ([`proposal.md`](proposal.md)). What changes is that the *result* of
|
||||
placing it is a thing with its own state, rather than a fact recorded on one of the two ends.
|
||||
|
||||
And it makes the composed declaration question from above answerable: a declaration is built
|
||||
from the module, the node's inventory, and the assignment between them. Three inputs, which is
|
||||
why two were never enough.
|
||||
@@ -0,0 +1,213 @@
|
||||
---
|
||||
status: active
|
||||
initiated: 2026-08-26
|
||||
touches:
|
||||
- 02-DECISIONS/0005-the-node-host.md
|
||||
- 02-DECISIONS/0011-managed-files-are-generated-never-edited.md
|
||||
- 03-DESIGN/01-to-be/05-the-node-host.md
|
||||
- 03-DESIGN/00-as-is/05-runtime-and-installation.md
|
||||
- 01-RESEARCH/011-the-module-graph/00-overview.md
|
||||
- 02-DECISIONS/0078-the-store-and-broker-are-modules.md
|
||||
- 02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md
|
||||
- 02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md
|
||||
---
|
||||
|
||||
# 012 — The minimum viable node, and adopting what is already there
|
||||
|
||||
## What is being investigated
|
||||
|
||||
Two questions that turn out to be one:
|
||||
|
||||
**What is the bare minimum to run a one-node mesh?** Not the tiers as asserted, but the actual
|
||||
closure — take the thing that must run, walk what it needs, and the set that comes back is the
|
||||
answer.
|
||||
|
||||
**And how does a machine that is already in use become that?** A candidate node is not empty. It
|
||||
has a package manager, probably a container runtime, possibly a git installation, each with
|
||||
configuration somebody chose. The mesh must **own** those, and owning is not the same as finding
|
||||
them present.
|
||||
|
||||
## Why
|
||||
|
||||
Building tier 0 reached a wall that looked like a packaging problem and is not.
|
||||
|
||||
The host can be told to run a container or install a package. Both need a file — an image, an
|
||||
archive — and the question was where the host gets it. That framing produced a bad trilemma:
|
||||
carry everything in the bundle, download at apply time, or have something push the files in
|
||||
first. Downloading fails on the first node, which cannot fetch the image registry from the image
|
||||
registry it is trying to start.
|
||||
|
||||
> **Qualified by [ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md).** The
|
||||
> reframing below still holds for what a *tailored installer* contains — the missing pieces for a
|
||||
> given machine. It does **not** have to hold for container images: the installer fetches those
|
||||
> by digest, because a real machine has a network and the sealed case is the lab.
|
||||
|
||||
**The reframing:** the machine is not offline. What matters is *when* the fetching happens. Move
|
||||
it from apply time to **build time** — build the installer on a machine that has a network,
|
||||
tailored to the target, and apply it on a target that then needs nothing. That is the same move
|
||||
the lab already made for its router image, and the same property the delivery design already
|
||||
claims: what ships is self-contained and a deploy touches no network.
|
||||
|
||||
Which makes the interesting question not *where do artifacts come from* but **what is missing
|
||||
from this particular machine**, and that needs both of the questions above answered.
|
||||
|
||||
## What tailoring implies
|
||||
|
||||
- **The binary stays generic; the payload is tailored.** One static host per architecture. What
|
||||
is machine-specific is the bundle it applies. Rebuilding the host per machine would buy
|
||||
nothing.
|
||||
- **Detection is the input, not a report.** What the host already reports about a machine —
|
||||
its profile and inventory — is what the difference is computed against. This is the first use
|
||||
of stage 1 by something other than a person reading it.
|
||||
|
||||
## Adoption
|
||||
|
||||
**Simply having a package installed is not enough.** If the mesh manages the container runtime,
|
||||
it decides that runtime's configuration; a runtime found already installed carries settings
|
||||
somebody chose, and those cannot be discovered by noticing the binary exists.
|
||||
|
||||
So a machine already in use is **adopted**: what is there is read, taken over, and thereafter
|
||||
generated.
|
||||
|
||||
**This was the original path.** [`00-as-is/05`](../../03-DESIGN/00-as-is/05-runtime-and-installation.md)
|
||||
records adoption of a pre-existing machine's configuration as the original mechanism, since made
|
||||
legacy and explicitly out of scope for the lab. It returns here for a different reason than it
|
||||
was dropped for, which is a thing to notice rather than to gloss.
|
||||
|
||||
**It creates a state that does not exist today.** [ADR 0011](../../02-DECISIONS/0011-managed-files-are-generated-never-edited.md)
|
||||
has managed files generated and never edited; adoption needs a one-time import before that rule
|
||||
starts applying. Three states, and the middle one is new:
|
||||
|
||||
> unmanaged → **adopted once** → generated
|
||||
|
||||
**And it crosses a boundary just drawn.** [ADR 0005](../../02-DECISIONS/0005-the-node-host.md)
|
||||
says the host never touches what it did not create — the rule that stops a converger deleting
|
||||
what the mesh never put there. Adoption is the deliberate act of taking ownership of exactly
|
||||
that. The rule needs a companion rather than an exception: *never, unless adoption made it the
|
||||
host's*, with adoption being explicit, recorded, and visible in what the host says it owns.
|
||||
|
||||
## Nothing is taken over without keeping what was there
|
||||
|
||||
**Before adoption touches a file, the original is kept.** Adoption happens on machines somebody
|
||||
is already using, and the configuration being taken over is configuration somebody chose. A
|
||||
one-way door on a working machine is not an installation, it is a risk nobody agreed to.
|
||||
|
||||
This is a *never* rule rather than a courtesy, and it earns that by the same incident the mesh's
|
||||
strongest rule already carries: the worst loss in this record came from a tool acting on a path
|
||||
it did not own. Adoption is that act, made deliberate — which makes the safeguard obligatory
|
||||
rather than optional.
|
||||
|
||||
What that requires, and what remains open: where the copy lives, whether it is recorded in what
|
||||
the node knows about itself so that adoption is *visibly* reversible, and whether the mesh keeps
|
||||
it forever or hands it back when it stops managing the thing.
|
||||
|
||||
## Adoption produces a briefing, not just a result
|
||||
|
||||
Proposed by the operator, and it answers a question this effort had open with two bad answers.
|
||||
|
||||
Adoption meets things a script cannot decide. A container runtime configured with one storage
|
||||
driver and a mesh wanting another. A package pinned to a version somebody chose for a reason.
|
||||
Local settings the mesh has no opinion about and no business discarding. Silently winning is
|
||||
wrong in both directions; refusing outright makes a machine in use unadoptable.
|
||||
|
||||
**So adoption has two outputs.** What it did — mechanical, recorded, in the node's state. And a
|
||||
**briefing**: what it found, what it took over, and what it could not resolve, written to be
|
||||
read by a person or an agent, which is the first thing a session on that node has to work with.
|
||||
|
||||
Conflicts are **flagged, not resolved**. That is the same principle the declaration parser
|
||||
already applies — name every problem at once, to somebody who can act on it — at a larger
|
||||
scale, and applied to a case where refusing wholesale would be worse than proceeding.
|
||||
|
||||
### On conflict, the machine's configuration is kept
|
||||
|
||||
Decided by the operator, after first deciding the opposite — recorded that way because the
|
||||
reasoning for each direction is the useful part.
|
||||
|
||||
Where the existing configuration and the mesh's disagree, **what is already on the machine
|
||||
stays**, the conflict is flagged, and it is reconciled afterwards. Adoption always **completes**
|
||||
— flags inform, they do not block — and *adopted with open questions* prevents nothing. The
|
||||
node is a node.
|
||||
|
||||
**What this buys.** Adoption becomes non-destructive by construction. The class of conflict that
|
||||
made the opposite rule dangerous — a storage driver against the filesystem it is actually on, a
|
||||
data directory pointing at a mount that exists — cannot arise, because nothing tied to the
|
||||
machine's physical reality is ever overwritten. A machine in use keeps working exactly as it
|
||||
did.
|
||||
|
||||
**What it exposes, which is the mirror of what it fixes.** The mesh's configuration is not only
|
||||
preference. Some of it is what a module needs in order to function at all. Keeping the machine's
|
||||
version there produces a module that is installed and does not work — *an installed package is
|
||||
not a capability*
|
||||
([04-ISSUES/007](../../04-ISSUES/007-an-installed-package-is-not-a-capability/00-report.md))
|
||||
arriving from a direction that issue did not anticipate. And a fleet where every node kept its
|
||||
own settings is a fleet where a module works on one node and fails on another with nothing in
|
||||
the mesh able to say why.
|
||||
|
||||
**The distinction that dissolves both rules.** Neither direction is right as a blanket, because
|
||||
the question is not *whose configuration wins*. It is whether the module **requires** the
|
||||
setting or merely **prefers** it — required contradictions cannot be kept without breaking the
|
||||
module, and preferences should always yield to what is already there.
|
||||
|
||||
That is a property of the module's own declaration rather than of the adoption algorithm, which
|
||||
makes it one more thing the graph would carry
|
||||
([research 011](../011-the-module-graph/00-overview.md)). Until modules can say which of their
|
||||
settings are load-bearing, adoption is choosing a default in the dark, and the default chosen
|
||||
here is the one that does not break the machine it is adopting.
|
||||
|
||||
### The briefing carries an outcome, and the outcome is derived
|
||||
|
||||
Proposed by the operator: the report states plainly whether adoption succeeded, partly
|
||||
succeeded or failed, and each line carries its own severity.
|
||||
|
||||
**Each line is marked, and the overall is the worst mark present.** Derived rather than stated
|
||||
alongside, because two fields written independently drift — and a briefing reading *full
|
||||
success* while carrying a failed line is exactly the fault this record keeps cataloguing. An
|
||||
outcome computed from its lines cannot disagree with them.
|
||||
|
||||
| Mark | Means |
|
||||
|---|---|
|
||||
| `ok` | done, and verified |
|
||||
| `kept` | a disagreement; the machine's value was kept and somebody should look |
|
||||
| `unknown` | could not be determined |
|
||||
| `failed` | could not be done — the node is not what was asked for |
|
||||
|
||||
**`unknown` is not a shade of success.** Adoption will meet configuration it cannot parse and
|
||||
state it cannot read, and folding those into *fine* is the same move as reporting an installed
|
||||
package as a capability. A thing nobody could determine is a thing nobody can rely on, and it
|
||||
gets its own mark for the same reason a capability detector reports *why*.
|
||||
|
||||
**And this opens something the earlier rule did not cover.** *Flags inform, they do not block*
|
||||
was decided about **conflicts** — where the mesh chose, deliberately, and the machine still
|
||||
works. A **failure** is different in kind: not *we chose* but *we could not*. Treating both the
|
||||
same makes a node where something the mesh needed never happened indistinguishable from one
|
||||
where a log level differed. Whether a failed line still lets adoption complete is therefore
|
||||
reopened by adding severity, and is not decided here.
|
||||
|
||||
## Adoption as a mode, for the migration
|
||||
|
||||
*2026-09-22.* The migration from the predecessor mesh gave the middle state a length. A machine
|
||||
running the predecessor is **adopted** when the mesh comes up on it — the predecessor's control
|
||||
stopped, the machine's firewall and files kept in force, the mesh opening what it needs through
|
||||
them — and stays adopted while its modules migrate one at a time, until the operator **converges**
|
||||
it. Measured on the control-node, and weighed against the alternatives, in
|
||||
[*migrating a node that is in use*](migrating-a-node-in-use.md); decided in
|
||||
[ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md).
|
||||
|
||||
## Open questions
|
||||
|
||||
| Question | Why it is open |
|
||||
|---|---|
|
||||
| What is the closure for a one-node mesh? | The skeleton asserts four pinned services. [Research 006](../006-mesh-from-scratch/00-overview.md) already asks whether it is four or five and does not answer. A graph gives a computed answer instead of an asserted one, which is [research 011](../011-the-module-graph/00-overview.md). |
|
||||
| Is "tier" the same thing as a graph level? | Tiers were named as a bootstrap order. If the closure is computed, tiers may be a derived view of the graph rather than a separate concept — or they may be a coarser boundary that survives for a different reason. |
|
||||
| ~~What happens when existing configuration contradicts what the mesh needs?~~ | **Answered** — the machine's configuration is kept, the conflict is flagged, and it is reconciled afterwards. |
|
||||
| ~~Do flags block, or only inform?~~ | **Answered** — they inform. Adoption always completes, and the node is a node. |
|
||||
| Can a module say which of its settings are load-bearing? | The question that dissolves the conflict rule rather than choosing a side. A setting the module *requires* cannot be kept from the machine without producing something installed and broken; a setting it merely *prefers* should always yield. Until a module can say which is which, adoption is defaulting in the dark. Belongs with the graph. |
|
||||
| Does a `failed` line still let adoption complete? | *Flags inform, they do not block* was decided about conflicts, where the mesh chose and the machine works. A failure is *we could not*, which is different in kind — and treating them alike hides the worse one behind the commoner one. |
|
||||
| How is a flagged conflict reconciled, and by whom? | The briefing hands it to a session. What that session is empowered to change, and whether the resolution is recorded so the next adoption does not re-raise it, is undecided. |
|
||||
| ~~How long is a machine adopted?~~ | **Decided** ([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)) — for as long as it is being migrated: a mode per node, recorded, ended by an explicit and previewed flip. |
|
||||
| Where does the kept original live, and for how long? | Whether it is recorded in the node's state so adoption is visibly reversible, and whether it is returned when the mesh stops managing the thing. |
|
||||
| What shape is a briefing? | Structured enough to be acted on, prose enough to be read. It is the first thing a session on a new node sees, which makes it an interface rather than a log. |
|
||||
| Does owning a package mean owning its version? | Owning configuration and owning the package are different scopes. The second means the mesh decides which version is installed, and that decision then has to survive the machine's own package manager updating it. |
|
||||
| How does a bundle stay true between building and applying? | It is built against a scan of the target. The machine can move between the scan and the apply, so the host has to verify rather than assume — and fail plainly when the bundle no longer fits. |
|
||||
| What cannot be precomputed at all? | Anything built from source on the target still needs a toolchain and a network at that moment. Tailoring moves that cost rather than removing it, and *minimal viable* has to be honest about what it cannot ship ahead. |
|
||||
| Does presence differencing understate the gap? | Knowing a package manager is installed does not say it is the version the mesh needs. A difference computed on presence alone is optimistic. |
|
||||
@@ -0,0 +1,154 @@
|
||||
# Migrating a node that is in use — adoption as a mode, not a moment
|
||||
|
||||
*2026-09-22. Measured on the machine that will be the control-node, which is running the
|
||||
predecessor mesh today. Nothing below names it; the counts are its own.*
|
||||
|
||||
## The question
|
||||
|
||||
The mesh replaces a predecessor mesh that is running, on the same machines, with the services
|
||||
people use. The control-node is decided: it is the machine that already carries the predecessor's
|
||||
broker and build pipeline. So the question is not *where* the mesh starts but **how a machine
|
||||
running the predecessor becomes a node of the mesh without its services noticing** — and then how
|
||||
each of the other machines follows.
|
||||
|
||||
## The operator's proposal
|
||||
|
||||
Proposed by the operator, 2026-09-22, and the shape this document tests:
|
||||
|
||||
1. **Stop the predecessor's control on a machine** — its daemons that write configuration: the
|
||||
network and firewall configuration above all, which decide what is reachable and what is
|
||||
blocked. Its services keep running; only the control over their configuration stops.
|
||||
2. **Bring the mesh up on that machine in adoption mode.** It takes custody of those files and
|
||||
keeps what it finds in force.
|
||||
3. **Migrate the modules one at a time**, data preserved, per the cutover procedure.
|
||||
4. **Move to the next machine and repeat** — adopted first, keeping its local configuration, then
|
||||
migrated.
|
||||
5. **When every machine is migrated, flip adoption mode**, and the mesh takes full control of the
|
||||
configuration it has been holding.
|
||||
|
||||
This is the research above made concrete. It keeps the conflict rule already decided here — *on
|
||||
conflict, what is on the machine stays* — and gives the middle state, *adopted*, a length: not a
|
||||
one-time import before generating starts, but a mode that lasts for as long as the machine is
|
||||
being migrated, ended by an explicit act.
|
||||
|
||||
## What the machine actually looks like
|
||||
|
||||
Measured, read-only:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Containers running | 60, all the predecessor's services and their stores |
|
||||
| The predecessor's control | user-level daemons, separate from the services; none of the 20 running system services is the predecessor's control |
|
||||
| Firewall | the predecessor's, active: 54 incoming rules and 52 forwarding rules, each served port allowed explicitly — how a default-deny firewall reads |
|
||||
| Files the predecessor's configuration sync writes | 12, of which 3 are system files (an ssh server drop-in, the package manager's configuration, one service's configuration); the rest are the operator's shell and agent files |
|
||||
| Per-service configuration | environment and composition files per service, written by the predecessor's service tooling |
|
||||
|
||||
**Stopping the predecessor's control stops nothing that serves.** The services are containers and
|
||||
system units that run without it; what stops is the rewriting of their configuration. Nothing
|
||||
changes on the machine until something else writes.
|
||||
|
||||
## What collides, measured rather than assumed
|
||||
|
||||
An earlier note assumed the mesh's foundation could not stand beside the predecessor because both
|
||||
want the store's and the broker's standard ports. **The measurement says otherwise.** The
|
||||
predecessor publishes its own store on a non-standard port and its broker on another; the standard
|
||||
ports the foundation binds for its store and its bus are free.
|
||||
|
||||
What does collide:
|
||||
|
||||
| The mesh wants | Held by | When |
|
||||
|---|---|---|
|
||||
| the registry's port | the predecessor's registry | at genesis — the foundation raises a registry |
|
||||
| the broker's management port, on loopback | the predecessor's broker | at genesis |
|
||||
| the private network's port | the predecessor's own tunnel | at genesis on a control-node that is the private network's hub; otherwise when the node is placed on it |
|
||||
| the web ports | the predecessor's reverse proxy | when the route proxy is assigned |
|
||||
| the resolver's port | a resolver the predecessor runs | when a resolver module is assigned |
|
||||
|
||||
On the control-node, which is the private network's hub, three of these fall at genesis and cannot
|
||||
be deferred; two only when a particular module is taken, the moment its predecessor stops anyway.
|
||||
Today the foundation's ports are **fixed**: written in the installer's bundle and in the
|
||||
catalogue's manifests, so a collision is found when a container fails to bind, not before — and a
|
||||
port changed at genesis would be changed back when the foundation is adopted as modules, since the
|
||||
applier recreates a container whose declared spec differs. The private network's address range
|
||||
must also stay clear of the range the predecessor's tunnel uses; on the machines measured they are
|
||||
distinct.
|
||||
|
||||
## What would break if the mesh came up as it is today
|
||||
|
||||
**The firewall.** Genesis loads a base ruleset — drop anything undeclared — in its own table
|
||||
([ADR 0088](../../02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md)). The
|
||||
predecessor's firewall is a different table. The kernel runs every base chain registered at the
|
||||
same hook, in priority order: an accept ends only its own chain and the packet goes on to the next,
|
||||
and a drop in any is final — whether the other firewall's chains are nftables or legacy iptables.
|
||||
So the
|
||||
mesh's base ruleset would drop everything the predecessor's firewall allows and the mesh has not
|
||||
declared — every web, mail and database port in the table above — the moment genesis ran.
|
||||
|
||||
**The files.** The host writes a declared file whatever it finds at the path, reporting it as
|
||||
updated. A file the predecessor left — the ssh server drop-in, the resolver's configuration — is
|
||||
replaced the first time a module declaring that path is assigned, before that module's service
|
||||
has moved.
|
||||
|
||||
**The container names.** The applier keys a container on its name. A catalogue module whose
|
||||
container carries the same name as the predecessor's service it replaces takes that container over
|
||||
the moment it is assigned: today, *assigning a module is migrating it*, never a preparation.
|
||||
|
||||
**The published ports.** The foundation publishes its ports on every interface, and a published
|
||||
container port reaches the container through the forwarded path, not the incoming one. A firewall
|
||||
that filters only incoming traffic never sees it. The base ruleset is what keeps the store
|
||||
unreachable from outside today — and it is the thing that cannot be loaded on this machine.
|
||||
|
||||
## The pipeline freezes while the control-node migrates
|
||||
|
||||
The predecessor's build pipeline and its coordinator run on the control-node. Stopping its control
|
||||
there stops the predecessor's updates for **every** machine it manages, until the migration is
|
||||
done. Their services keep running; they receive nothing new. That is the price of the proposal and
|
||||
it is worth stating, not a reason against it: the migration is the period in which the predecessor
|
||||
is being replaced, and it does not need to keep changing.
|
||||
|
||||
## What adoption mode has to mean
|
||||
|
||||
For the proposal to hold, *adopted* must be a **state the mesh records per node**, not an
|
||||
intention, and each thing the mesh would otherwise take must say what it does in that state:
|
||||
|
||||
- **What is found is kept until its module is taken.** *Found* is precise: present at a declared
|
||||
path or name with no record in the host's store. A found file or container is held, its original
|
||||
recorded, until the operator **takes** the module on that node — the cutover, done when the
|
||||
module's data has moved. Assigning prepares; taking migrates. Without the distinction the rule
|
||||
never fires: the host only ever sees what assigned modules declare.
|
||||
- **The firewall found on the machine stays in force.** The mesh loads no table on an adopted node
|
||||
that drops by default or holds an accept. What it needs open it declares as openings the host converges
|
||||
*through the found firewall*, on the incoming and the forwarded path, marked as the mesh's and
|
||||
re-checked on every reconcile so a reload or reboot does not lose them. An accept in a table of
|
||||
its own would not help: the found firewall's drop would still be final. What a table of its own
|
||||
*can* do is refuse, and a refusal is final too — so the mesh guards the store and the broker's
|
||||
management port from everyone but the private network and the machine itself, in a table that
|
||||
only refuses, ahead of the container runtime's redirect — which the found firewall does not do
|
||||
and cannot undo. The bus, the registry and the hub's port stay open to
|
||||
anywhere: a node enrols before it has a private-network address.
|
||||
- **The foundation's ports are the node's to give** — set at genesis, checked free, and kept as
|
||||
that node's settings, read everywhere they are used, so adopting the foundation as modules does
|
||||
not move them back.
|
||||
- **The flip is per node and previewed** from what is actually reachable — listening sockets and
|
||||
published ports, not the found firewall's allow list, which does not see what a container runtime
|
||||
forwards.
|
||||
|
||||
## Options weighed
|
||||
|
||||
| Option | Verdict |
|
||||
|---|---|
|
||||
| Cut the machine over in one go: stop the predecessor's store, broker, registry and proxy, raise the foundation in their place | Rejected. Every predecessor service goes down until it has migrated, and the predecessor's other machines lose their broker. The rollback is restarting the predecessor, which is a recovery, not a step. |
|
||||
| A separate machine as the control-node | Rejected. Contradicts the decision that the control-node is the machine that already carries the predecessor's broker and pipeline. |
|
||||
| Converge on joining, as the mesh does today | Rejected. The base ruleset closes every predecessor port at genesis, and found files are replaced before their services move. |
|
||||
| Make only the foundation's ports configurable and otherwise converge | Rejected as insufficient. It solves the bind collisions and none of the firewall or file ones. |
|
||||
| Treat assigning a module as migrating it | Rejected on review. The rule that keeps found files would never fire — the host only sees what assigned modules declare — and an assignment on an adopted node would be an outage rather than a preparation. |
|
||||
| **Adoption as a mode, per node, ended by an explicit flip** | The proposal. It is the conflict rule already decided here, given a duration. |
|
||||
|
||||
## What this leaves open
|
||||
|
||||
Answered by the decision record this feeds — [ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)
|
||||
— only for the migration's needs. The general questions above stay open: whether a module can say
|
||||
which settings are load-bearing, what shape a briefing takes, how a flagged conflict is reconciled.
|
||||
One is newly sharp: the mesh opening ports through a firewall it did not install needs to speak that
|
||||
firewall. There is one kind on the machines measured; a machine with another is not covered until
|
||||
someone writes for it.
|
||||
@@ -0,0 +1,54 @@
|
||||
---
|
||||
status: active
|
||||
initiated: 2026-09-23
|
||||
touches:
|
||||
- 02-DECISIONS/0075-two-stores-and-which-provides-what.md
|
||||
- 02-DECISIONS/0079-the-foundation-seats-are-named-after-their-servers.md
|
||||
- 02-DECISIONS/0071-genesis-builds-from-a-mesh-that-already-exists.md
|
||||
- 02-DECISIONS/0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md
|
||||
- 04-ISSUES/085-the-packages-port-given-at-genesis-is-not-a-setting/00-report.md
|
||||
- 04-ISSUES/090-the-forge-module-does-not-take-over-the-forge-genesis-raised/00-report.md
|
||||
---
|
||||
|
||||
# 013 — The forge and the registries: what a seat is for, and what it is not
|
||||
|
||||
**The question, asked during the first migration:** the mesh runs a forge that serves git and
|
||||
packages, an image registry, and a bootstrap forge genesis raises before any module exists. Should
|
||||
there be a mesh-scoped **git seat**, the way there is one for the store and the broker?
|
||||
|
||||
**The short answer: no, and the question points at a different hole.** A seat answers *is there
|
||||
exactly one of you*. Every symptom around the forge is about **an address and a credential nobody
|
||||
resolves**, which a seat does not answer. The mechanism that would answer it — a provision, with
|
||||
`serves` and a grant — already exists, is already used for the image store, and is already
|
||||
declared for packages. It simply has no consumer: the one module that needs it carries a literal
|
||||
instead.
|
||||
|
||||
See [the survey](the-survey.md) for what the code does today, with evidence.
|
||||
|
||||
## What was found
|
||||
|
||||
- **A seat does exactly two things**: it refuses a second claimant at resolution, and in one place
|
||||
it answers *where is the broker*. It is a resolution-time predicate; no host ever hears of it.
|
||||
- **Four mesh-scoped seats exist**, all named after a singular server the mesh runs **for its own
|
||||
working** — controller, store, broker, catalogue. A forge is an application the world made, not
|
||||
part of that set.
|
||||
- **Nothing in the mesh requires git.** There is no provision for it, and the forge's git service —
|
||||
over https and ssh — is declared nowhere. Only its npm half is a provision.
|
||||
- **`package-registry` is a fully built provision with zero consumers.** The builder, its only
|
||||
real consumer, bypasses it with a file naming the module, the address `127.0.0.1` and the port.
|
||||
- **The builder is told where source lives, per build, by a person.** The clone URL is an opaque
|
||||
string; each module remembers its own; no credential is ever attached; nothing polls a forge.
|
||||
- **A seat would forbid something the mesh should allow**: a second forge — one for the mesh's own
|
||||
source, one for something else — which ADR 0075 already argues for against a node-scoped claim.
|
||||
|
||||
## What this leaves open
|
||||
|
||||
- Should the forge's git service become a **provision** (`source-forge`), so the builder resolves
|
||||
the address and a credential the way it already resolves the image store? That is what would let
|
||||
a forge move, be renamed, or be replaced without editing every module's stored URL, and it is the
|
||||
unanswered half of [issue 085](../../04-ISSUES/085-the-packages-port-given-at-genesis-is-not-a-setting/00-report.md).
|
||||
- The obstacle is known and written down: such a requirement **may go unanswered during genesis**,
|
||||
and the mesh has no optional requirement. Deciding that is the real work.
|
||||
- Should the mesh learn when a source moves? It records the fact and never discovers it: nothing
|
||||
polls, and there is no receiver for a forge's push. `build --behind` answers a question only a
|
||||
person can currently make true.
|
||||
@@ -0,0 +1,130 @@
|
||||
# The survey — seats, the forge, the registries, as the code has them
|
||||
|
||||
*2026-09-23. Read from the code and the records, not from memory. Every claim here was checked
|
||||
against a file.*
|
||||
|
||||
## What a seat is
|
||||
|
||||
The glossary calls a seat *a named position at a scope with a capacity*, and a claim *a module
|
||||
taking a spot on it*. **Capacity does not exist in the code.** There is no capacity field and no
|
||||
bench; every claim is exclusive. The "shared seat" of the design is the word `provides` wearing
|
||||
that name.
|
||||
|
||||
Claiming does two things and no others:
|
||||
|
||||
- **It refuses a second claimant when a node's modules are resolved** — within one node at any
|
||||
scope, and against every other node for mesh and site scope.
|
||||
- **It answers one question, once:** the controller finds where the broker is by looking for the
|
||||
module claiming the broker's seat.
|
||||
|
||||
No host is ever told about a seat. It is a resolution-time predicate.
|
||||
|
||||
Two mechanical asymmetries matter. A mesh-scoped seat's exclusivity is defended only by nodes
|
||||
that *resolve*: a node whose modules fail to resolve contributes nothing, so a broken node does
|
||||
not hold its seat against a second claimant. And a site-scoped seat does nothing at all when
|
||||
either node has no site.
|
||||
|
||||
## The seats that exist
|
||||
|
||||
Twelve claims, twelve names. Four are mesh-scoped: the controller, the store, the broker, the
|
||||
catalogue. All four are things the mesh runs **for its own working**, and three of them are raised
|
||||
by genesis and adopted in place — which is the argument of the record that named them.
|
||||
|
||||
The node-scoped ones split in two: a **role on this machine** (the build machine, the packet
|
||||
filter, the intrusion prevention, the private network) and **a scarce machine resource** (the
|
||||
resolver's configuration file, the DNS port). The second kind is a claim for the reason a port is:
|
||||
there is one of it on the machine.
|
||||
|
||||
Two oddities worth stating:
|
||||
|
||||
- The image store claims its seat at **node** scope while providing its service at **mesh** scope.
|
||||
The record that discussed this left open whether it should hold that claim at all.
|
||||
- **Nothing claims the two public ports.** The proxy binds them and claims nothing, which is the
|
||||
hole the route handover fell into.
|
||||
|
||||
## The forge, and what it serves
|
||||
|
||||
The forge serves three things: git over https, git over ssh on its own port, and an npm registry.
|
||||
**Only the npm half is in the mesh's vocabulary** — the forge provides `package-registry` and says
|
||||
where it answers. Git is served and declared nowhere: no provision, no `serves`, nothing that can
|
||||
require it. The only trace is the public label in its route contribution, which the mesh is
|
||||
explicitly not meant to interpret.
|
||||
|
||||
The image store is a separate module and a separate provision, plain HTTP, trusted because it is
|
||||
reachable only over the private network.
|
||||
|
||||
A second module also provides `package-registry`. Two providers of one provision is not a refusal
|
||||
but an ambiguity, and the mesh already has the command that settles it: pin one.
|
||||
|
||||
## Where the builder's three addresses come from
|
||||
|
||||
| What it needs | How it finds it |
|
||||
|---|---|
|
||||
| the broker | a sealed secret of its own |
|
||||
| the image store | **a real provision binding**, resolved by the mesh |
|
||||
| the package registry | **a file in its own manifest**, naming the module, `127.0.0.1` and the port |
|
||||
| the source repository | nothing at all — a person types a clone URL per build |
|
||||
|
||||
The installer says this plainly in a comment: *the one binding nothing resolves: the builder dials
|
||||
the forge by a number it carries.* Genesis can override the port of that literal, and nothing
|
||||
else: not the forge's identity, not its address.
|
||||
|
||||
`package-registry` therefore has **zero consumers**. The builder does not require it. The
|
||||
provision is fully built — two providers, grants, a served description including the registry's
|
||||
path — and nothing asks for it.
|
||||
|
||||
## How a module's source is found
|
||||
|
||||
It is not found; it is supplied. The clone URL arrives as an argument, travels to the build machine
|
||||
and reaches `git clone` unexamined. No credential is attached, so a private forge works only if the
|
||||
build machine's own git configuration already authenticates. Each module remembers the URL it was
|
||||
built from, so there are as many forge addresses as modules, each frozen at whatever was typed.
|
||||
|
||||
At genesis there are four independent source URLs, one per flag, unrelated to each other.
|
||||
|
||||
If the forge moves or is renamed: every stored URL goes stale independently, each failing at its
|
||||
own clone, and there is no command to re-point them. Genesis keeps working, because it takes URLs
|
||||
as flags — which is the existing record's position: the forge is reached *by a name outside the
|
||||
mesh*, and failover is repointing that name.
|
||||
|
||||
## What genesis's forge is
|
||||
|
||||
A bare container run before any module exists, because the toolchain resolves the mesh's own
|
||||
packages by version from an npm registry. It holds **no seat, no manifest, no provision**. The
|
||||
mesh is told one thing about it — a port — and only when that port is not the default. **On an
|
||||
ordinary genesis the mesh never learns the bootstrap forge exists.**
|
||||
|
||||
Its successor module differs from it in five ways: container name, network, data directory, the
|
||||
address it is told to call itself, and — contradicting a comment that says they are pinned
|
||||
identically — **the image digest**. So assigning the module raises a second forge beside the first.
|
||||
|
||||
A seat cannot fix that. A seat is checked between manifests; the bootstrap forge has no manifest,
|
||||
so no seat can see it. The check that is actually wanted — same name, same data, same network — is
|
||||
between an installer constant and a manifest.
|
||||
|
||||
## Would a git seat be coherent?
|
||||
|
||||
In form, yes: add the claim and a second forge is refused. In effect it buys one refusal nobody
|
||||
has hit, and forbids an arrangement the mesh should allow — a second forge for something other
|
||||
than the mesh's own source.
|
||||
|
||||
Measured against the four mesh seats, it does not fit: those are singular servers the mesh runs
|
||||
for itself, and a forge is an application. Nothing would *use* it, because the only seat consumer
|
||||
answers "where is the broker", and a forge address is not one fact: a module's source is a
|
||||
repository **and a path and a ref**.
|
||||
|
||||
## What the code already supports and nobody uses
|
||||
|
||||
- **`package-registry` with no consumers.** The builder's literal file is a hand-rolled copy of
|
||||
the binding the mesh would have written for it.
|
||||
- **Pinning a provider** already expresses *this* forge, of several, without forbidding the second.
|
||||
- **Grants** on the forge are the credential mechanism that would hand the builder an account.
|
||||
Today the installer creates that account directly, because nothing requires the provision and so
|
||||
the grant has no consumer to write for.
|
||||
|
||||
## The reading
|
||||
|
||||
The question *should there be a git seat* is the wrong shape for the problem under it. Every
|
||||
symptom — the literal address, the port that would not follow a node, the takeover that is not a
|
||||
takeover — is about **an address and a credential nobody resolves**. A seat resolves neither. The
|
||||
provision mechanism does, and is already built.
|
||||
@@ -0,0 +1,108 @@
|
||||
---
|
||||
status: graduated
|
||||
became: 02-DECISIONS/0106-the-bus-is-nats.md
|
||||
initiated: 2026-09-23
|
||||
touches:
|
||||
- 02-DECISIONS/0002-nodes-communicate-over-a-broker.md
|
||||
- 02-DECISIONS/0033-the-substrate-is-a-store-and-a-broker.md
|
||||
- 02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md
|
||||
- 02-DECISIONS/0041-events-are-a-relationship.md
|
||||
- 02-DECISIONS/0042-the-shape-of-an-event-on-the-wire.md
|
||||
- 02-DECISIONS/0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md
|
||||
- 02-DECISIONS/0078-the-store-and-broker-are-modules.md
|
||||
- 02-DECISIONS/0083-one-push-leaves-the-mesh-consistent.md
|
||||
---
|
||||
|
||||
# 014 — The bus on NATS: replace the broker, and when
|
||||
|
||||
**The question, asked mid-migration:** the operator wants the mesh's bus — today an AMQP broker —
|
||||
replaced by NATS, with everything the broker does today. Do we finish the migration on AMQP and
|
||||
move to NATS after, or go to NATS directly?
|
||||
|
||||
**The short answer: decide NATS now, build it in the lab in parallel, and cut the mesh's bus over
|
||||
in one rehearsed rollout — after the migration's core is done and never underneath it.** "First
|
||||
everything on AMQP" is already the state and costs nothing more: the predecessor's broker was
|
||||
merged into the mesh's tonight, and every module converted from here on targets the sdk's broker
|
||||
contract, which names no protocol. The only code that speaks AMQP is the mesh's own, in three
|
||||
places, and it is swapped once.
|
||||
|
||||
## What was measured
|
||||
|
||||
**Where AMQP is spoken** (non-test files): the controller's `internal/link` (5 files, ~2.4k lines
|
||||
with tests) and the builder's main; the host's `internal/link` (~1.5k); the tool runtime's
|
||||
`broker-amqp.ts` (~600 with its main). **The sdk speaks none** — its `Broker` is `request`,
|
||||
`handle`, `publish`, `subscribe`, `close` ([ADR 0039](../../02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md)
|
||||
put the client in the runtime for exactly this). **Every module's tools and events go through the
|
||||
sdk**; one module (`amqp-ping`, a probe) talks AMQP on purpose.
|
||||
|
||||
**What the bus carries:** a control queue (`control`, `.upgrades`, `.catchup`), one queue per node
|
||||
(`node.<name>`), `builds`, the enrolment/report/alive/built flows, a topic exchange of events
|
||||
(`mesh.events.<module>.<event>`, dead-lettered to `mesh.events.dead`), an RPC exchange (`mesh.rpc`)
|
||||
and per-tool service queues (`serve.<module>.<tool>`), plus an MQTT exchange.
|
||||
|
||||
**What it relies on:** TLS for the bus a node enrols over; prefetch with reject/nack so the control
|
||||
queue **holds messages unacknowledged while the store restarts** and retries
|
||||
([ADR 0083](../../02-DECISIONS/0083-one-push-leaves-the-mesh-consistent.md)); publisher confirms and
|
||||
mandatory routing; a dead-letter exchange; **per-module accounts scoped by `emits`/`consumes`**
|
||||
([ADR 0043](../../02-DECISIONS/0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md)),
|
||||
minted through the broker's management API; the broker as a module holding the `mesh-broker` seat
|
||||
([ADR 0078](../../02-DECISIONS/0078-the-store-and-broker-are-modules.md)).
|
||||
|
||||
**The predecessor's world**, now on the same broker: ~50 AMQP connections per machine from four
|
||||
machines, 595 queues, four vhosts, six users. All of it AMQP, all of it retiring module by module.
|
||||
NATS speaks no AMQP: that world cannot move; it does not need to.
|
||||
|
||||
## How NATS answers each of those
|
||||
|
||||
| The mesh relies on | NATS | Note |
|
||||
|---|---|---|
|
||||
| topic routing keys | subjects with wildcards | same shape (`mesh.events.>`) |
|
||||
| RPC and tool invocation over an exchange + reply queue | request/reply, native | simpler than today |
|
||||
| competing consumers | queue groups | same |
|
||||
| durable control queue, hold-unacked-and-retry, catch-up | JetStream: streams, durable consumers, ack/nak with delay, replay | the 0083 guarantee moves to JetStream; core NATS alone is at-most-once and would not do |
|
||||
| dead-letter | max-deliver + advisories, or a stream fed from them | different mechanism, same effect |
|
||||
| TLS bus | TLS | same |
|
||||
| accounts scoped by emits/consumes, vhosts | accounts (isolation) with users and per-subject publish/subscribe permissions | stronger than today; the controller writes an auth config the host declares and the server reloads, instead of calling a management API |
|
||||
| management API | `nats` CLI and an HTTP monitoring endpoint | no vhost concept — accounts instead |
|
||||
| MQTT | built in | same |
|
||||
| a module seat `mesh-broker` | unchanged — the seat is the server, the module changes | ADR 0079 |
|
||||
| multi-node | clusters and leaf nodes | not needed now; a leaf per node is a later question |
|
||||
|
||||
Nothing the mesh needs is missing. The differences are in the shape of durability (JetStream must
|
||||
be declared, streams and consumers are objects) and of accounts (configuration, not API calls).
|
||||
|
||||
## The cost, honestly
|
||||
|
||||
The bus is the mesh's nervous system. Moving it means: a new `nats` module in the catalogue taking
|
||||
the `mesh-broker` seat; the controller's and the host's link packages rewritten; the tool runtime's
|
||||
client swapped behind the unchanged contract; enrolment, reports, builds and the guard's ports
|
||||
re-derived; the lab beds that prove the bus (store window, enrolment, upgrades) re-run on the new
|
||||
one; and eight decisions amended or superseded. Weeks, not days — and none of it can be done
|
||||
halfway on a live mesh: the controller, every host and every tool runtime move together.
|
||||
|
||||
## The sequencing question, answered
|
||||
|
||||
1. **NATS first, directly.** Stalls the migration for the length of the build; the mesh's bus
|
||||
changes under a half-migrated node; the predecessor's clients still need AMQP, so a second
|
||||
broker runs anyway, plus a bridge for whatever crosses. Rejected.
|
||||
2. **Finish on AMQP, NATS after.** Wastes nothing — no module written from here on speaks AMQP —
|
||||
but leaves the decision unmade while modules are written, and the runtimes idle. Adequate.
|
||||
3. **Decide NATS now; build it in the lab in parallel; cut the mesh's bus over in one rehearsed
|
||||
rollout after the core is migrated.** The predecessor's clients never notice: their broker is
|
||||
the one the mesh adopted, kept as a compatibility module with an end date — the day the last
|
||||
AMQP client is gone. **Recommended.**
|
||||
|
||||
## What this leaves open
|
||||
|
||||
- The **decision itself**, as a record: the bus is NATS; the AMQP broker becomes the predecessor's
|
||||
compatibility broker and retires with the last AMQP client. Written when the operator says so.
|
||||
- **JetStream's shape for the control plane**: one stream per concern (control, nodes, builds,
|
||||
events) or one with subjects; retention; what the store window guarantee looks like as ack-wait
|
||||
and nak-delay. Measured in the lab, not designed on paper.
|
||||
- **Accounts as configuration**: the controller writes users and permissions into a file the host
|
||||
declares, reloaded on change — which is the [ADR 0102](../../04-ISSUES/102-an-address-recorded-at-genesis-or-build-does-not-follow-the-nodes-ports/00-report.md)
|
||||
discipline applied from the start — or the JWT/operator model. The first is simpler and matches
|
||||
how the mesh already writes everything.
|
||||
- **The MCP bridge** the operator asked for the same day is written against the sdk's contract, so
|
||||
it moves with the bus and is not written twice.
|
||||
- Whether a **leaf node per machine** replaces the hub-and-spoke bus later — out of scope here.
|
||||
@@ -0,0 +1,142 @@
|
||||
---
|
||||
status: active
|
||||
initiated: 2026-09-24
|
||||
touches:
|
||||
- 02-DECISIONS/0028-the-substrate-supplies-the-control-plane-and-nothing-else.md
|
||||
- 02-DECISIONS/0033-the-substrate-is-a-store-and-a-broker.md
|
||||
- 02-DECISIONS/0048-a-provider-creates-the-credential-the-mesh-minted.md
|
||||
- 02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md
|
||||
- 02-DECISIONS/0078-the-store-and-broker-are-modules.md
|
||||
- 02-DECISIONS/0084-which-provider-serves-a-consumer.md
|
||||
- 03-DESIGN/00-as-is/03-provisioning.md
|
||||
- 03-DESIGN/01-to-be/07-the-foundation.md
|
||||
- 04-ISSUES/113-the-object-stores-images-were-withdrawn-upstream/00-report.md
|
||||
---
|
||||
|
||||
# 015 — The object store after MinIO: which S3 implementation, and how the data moves
|
||||
|
||||
**The question.** The mesh's object store is MinIO. Its community edition is archived upstream,
|
||||
its server and client images have been deleted from every public registry, and the pinned release
|
||||
is four and a half years old and will never be patched
|
||||
([issue 113](../../04-ISSUES/113-the-object-stores-images-were-withdrawn-upstream/00-report.md)).
|
||||
Which S3-compatible implementation replaces it, and what is the migration track for the data and
|
||||
the provisioning model that sit on top of it?
|
||||
|
||||
**Why now, and why not sooner.** Nothing is on fire: nodes that already hold the images keep
|
||||
running, and issue 113 establishes that the deploy path tolerates an unfetchable-but-present
|
||||
image by design. The forcing function is not an outage but a one-way door — **no node that does
|
||||
not already hold the images can ever provision the module again**, so the mesh's ability to stand
|
||||
a node up from its declarations is already broken for this module, and silently.
|
||||
|
||||
**The direction is not a departure from the design; it is the design.** The foundation document
|
||||
already states the commitment:
|
||||
|
||||
> The dependency is on the **protocol**, not the product: AMQP for the bus, S3 for the object
|
||||
> store, the OCI protocol for the registry. That is what keeps the naming safe rather than a
|
||||
> commitment that cannot be revisited.
|
||||
|
||||
The object store is also **not** a foundation service — ADR 0028 removed it, and it is an
|
||||
ordinary module required through the module graph by whatever wants one. (The "exception that is
|
||||
not a swap" in that passage is the relational store, whose provisioning model borrows PostgreSQL's
|
||||
own meaning of databases, roles and schemas. The object store carries no such coupling: a bucket
|
||||
is a bucket.) So this effort is an instantiation of an existing principle, not a redesign — which
|
||||
is the cheapest kind of decision to make and the strongest kind to cite.
|
||||
|
||||
## What the replacement has to carry, measured
|
||||
|
||||
Taken from the module's manifest, its composition, its tool surface, and a search for its
|
||||
consumers across the catalogue — not from assumption.
|
||||
|
||||
| Requirement | Evidence in the module today |
|
||||
|---|---|
|
||||
| S3 API | The protocol every consumer speaks; already the design's stated dependency. |
|
||||
| ~~OIDC login against the mesh's identity provider~~ | **Struck 2026-09-24. Not a requirement, and it never worked.** Six variables are wired and an entrypoint blocks on the provider, which reads as a live feature. The module's own hook comment records the end state as *"policy claim missing"* — a failing login. See [01](01-candidate-comparison.md). |
|
||||
| **Per-application access keys, each scoped to a bucket** | The real requirement. A "user" of the store is normally an application; the mesh already mints a credential per provisioned bucket. |
|
||||
| **One live consumer using it as opaque primary storage** | A file-sync application, since early 2023: objects named by internal id, metadata in its own database. Highest-risk consumer — a live copy drifts, and its bucket name must be preserved. |
|
||||
| Erasure-coded multi-node topology | Four server nodes with two data directories each, behind a load balancer. |
|
||||
| A single-node form | Declared as a flavour, for development and small nodes. |
|
||||
| Buckets as a typed provision | The module declares a provision type of `bucket` on a named network; the mesh mints the credential and the provider creates it (ADRs 0048, 0084). |
|
||||
| A tool surface | Bucket create/list/delete, object list/info/delete, presigned URL, and provisioning. |
|
||||
| A console | Published on its own subdomain through the reverse proxy, with an unlimited request-body middleware for uploads. |
|
||||
|
||||
**Consumers, counted:** one application module, one capture module that takes a private bucket per
|
||||
node, one workflow module's tools, and the delivery/rescue internals of the shared library. The
|
||||
surface is small — the cost is concentrated in the provisioning handler, the tool handlers and the
|
||||
OIDC story, not spread across the catalogue.
|
||||
|
||||
## Candidates
|
||||
|
||||
**Four candidates, not three.** The comparison was briefly narrowed to SeaweedFS on the strength of
|
||||
console single sign-on; that axis turned out not to be a requirement, and the incumbent's own
|
||||
maintained fork had been omitted altogether. Both errors, and why they happened, are recorded in
|
||||
[01 — the candidates measured](01-candidate-comparison.md), which carries the evidence and the
|
||||
requirement-by-requirement detail.
|
||||
|
||||
In short, and only in short:
|
||||
|
||||
- **The maintained fork of the incumbent** — the community edition was archived and its images
|
||||
deleted, but a fork publishes, tracks CVEs, and preserves the on-disk format, S3 API and
|
||||
environment surface. Costs **an image reference** where every other option costs a data
|
||||
migration, two rewrites and a maintenance window. Does not end the dependence on an abandoned
|
||||
codebase; buys time to choose deliberately.
|
||||
- **Garage** — its permission model *is* the requirement (per access key, per bucket), its admin
|
||||
API is the closest match to how the mesh provisions, and the highest-risk consumer is
|
||||
first-party documented against it. Remaining cost: no object versioning, no server-side
|
||||
encryption or object locking, partial lifecycle — **unmeasured against the ten buckets, and the
|
||||
one thing that could still disqualify it**.
|
||||
- **SeaweedFS** — longest field record and erasure coding. Its console sign-on is a paid feature,
|
||||
which is now beside the point. What weighs against it is narrower: its S3 surface is a gateway
|
||||
translating onto its own file-system API, with no first-party support for the opaque consumer.
|
||||
- **RustFS** — closest in shape to the incumbent, so the least porting. But it reached general
|
||||
availability eight days before this was written, and carries an open defect in the credential
|
||||
path. Two earlier claims about it are corrected in 01: it is **not** a drop-in that retains
|
||||
existing data.
|
||||
- **Ceph RGW** — remains rejected as disproportionate where the object store is an ordinary
|
||||
module rather than a platform.
|
||||
|
||||
**This is now two decisions, not one:** whether to repoint to the fork or migrate, and — if
|
||||
migrating — to which. Repointing does not foreclose migrating, which is the argument for taking it
|
||||
first. On the corrected requirement the migration ranking is Garage, then SeaweedFS, and not yet
|
||||
RustFS. Two measurements gate any graduation: **which S3 endpoints the consumers actually call**
|
||||
(Garage cannot be ranked fairly until counted), and **whether the fork can read the incumbent's
|
||||
on-disk format in place** — tested on a copy, because the migration between them is one-way. Both
|
||||
are in [01](01-candidate-comparison.md#what-is-still-unmeasured).
|
||||
|
||||
## The migration track, in outline
|
||||
|
||||
Data movement is the easy half, and deliberately reversible.
|
||||
|
||||
1. **Stand the replacement up beside the incumbent**, on its own ports, its own provision type and
|
||||
**its own data directory**. Nothing removed. The data directory matters: reusing one the
|
||||
incumbent already holds would put a fresh single-drive store on top of a live erasure set.
|
||||
2. **Copy bucket by bucket with a neutral tool.** `rclone` rather than the incumbent's own client
|
||||
— the client has been withdrawn upstream too, so building the migration on it would inherit
|
||||
the same dependency this effort exists to remove.
|
||||
3. **Verify per bucket** — object counts and checksums, not a transfer exit code.
|
||||
4. **Repoint consumers through the connection the module already publishes.** Consumers read an
|
||||
API URL from the module's declared connections rather than addressing the store directly, so
|
||||
the cutover surface is that value plus the provisioning and tool handlers.
|
||||
5. **Freeze writes, final incremental sync, flip**, and keep the incumbent read-only as the
|
||||
rollback until confidence is earned. For the opaque consumer this is **not optional and not
|
||||
instant**: it stores objects by internal id with metadata in its own database, so a copy taken
|
||||
while it runs will drift. It needs a maintenance window for the final sync, and the window is
|
||||
proportional to 82,496 objects rather than to 230 GiB.
|
||||
6. **Retire**, and only then remove the module.
|
||||
|
||||
The genuinely new work is not the copy. It is the **provisioning handler** and the **tool
|
||||
handlers**, both written against the incumbent's admin API. *The OIDC wiring was previously listed
|
||||
here and is struck: it is not a requirement and it never worked.*
|
||||
|
||||
## Open questions
|
||||
|
||||
- ~~How much of the OIDC requirement survives, and in which build?~~ **Answered, and it was the
|
||||
wrong question.** The console requirement does not exist, and the login it referred to never
|
||||
worked. What replaced it: which S3 endpoints consumers actually call, and whether the fork reads
|
||||
the incumbent's format in place.
|
||||
- Does the mesh's bucket provision translate to the candidate's identity model without weakening
|
||||
what ADR 0049 says about a consumer's identity fitting the tightest backend?
|
||||
- Should this effort also answer issue 113's general question — mirroring third-party images into
|
||||
the mesh's own registry — or is that a separate decision? Replacing one withdrawn product with
|
||||
another unmirrored upstream leaves the same one-way door in place, just further from the hinge.
|
||||
- Is the four-node erasure-coded topology still warranted, or was it inherited? Worth re-asking
|
||||
while the product is being chosen, rather than reproducing a shape by default.
|
||||
@@ -0,0 +1,176 @@
|
||||
# 015 / 01 — The candidates measured
|
||||
|
||||
*Rewritten 2026-09-24. An earlier version of this document ranked the candidates on whether they
|
||||
preserved single-sign-on to the object store's **console**. That was the wrong axis — it is not a
|
||||
requirement — and a fourth candidate was missing entirely. Both errors are recorded at the end,
|
||||
because how a comparison came to be ranked on the wrong thing is worth more than the ranking was.*
|
||||
|
||||
## The requirement, corrected
|
||||
|
||||
Taken from the operator and from the running system, not from the module's shape.
|
||||
|
||||
**A "user" of the object store is normally an application.** The requirement is therefore
|
||||
**per-application access keys, each scoped to its own bucket** — not per-human single sign-on. The
|
||||
mesh already works this way: it mints a credential for every provisioned bucket, and the consumer
|
||||
reads an endpoint from the module's declared connection rather than addressing the store directly.
|
||||
|
||||
**The console is not a requirement.** It was the axis the previous version ranked on, and it should
|
||||
not have been.
|
||||
|
||||
**The identity-provider login never worked.** The predecessor's module wires six OIDC variables and
|
||||
blocks startup until the provider answers, which reads like a working feature. It is not: the
|
||||
module's own hook comment records the end state as *"policy claim missing"* — a **failing** login,
|
||||
written up as progress because it proved the provider had registered. The identity provider emits no
|
||||
such claim, nothing in the module creates the mapper, and the configured scope alone would not carry
|
||||
a custom one. Two days of logs show no genuine login attempts, only internet scanners failing on an
|
||||
STS API version. **Nothing should be carried forward on the assumption this works**, and no
|
||||
candidate should be credited or penalised for matching it.
|
||||
|
||||
**One consumer is live, opaque, and holds real user files.** A file-sync application has used the
|
||||
store as its **primary storage** since early 2023: objects named by an internal id, with all
|
||||
metadata in its own database. Three consequences — a copy taken while it runs will drift, its bucket
|
||||
name must be preserved or its database references break, and it is the highest-risk consumer of the
|
||||
lot.
|
||||
|
||||
## What is actually stored, measured
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Logical | **230 GiB, 82,496 objects, 10 buckets** |
|
||||
| Raw on disk | **468 GiB** — eight drive directories at 59 GiB each |
|
||||
| Implied scheme | 468 ÷ 230 = **2.03×**, confirming erasure coding at half parity |
|
||||
| Headroom | ~1.3 TiB free on the filesystem holding it |
|
||||
|
||||
**All eight "drives" are directories on one filesystem on one machine.** The erasure coding is
|
||||
therefore not buying independent-drive redundancy; the real failure domain is the array underneath,
|
||||
which has its own. This single fact decides more of the comparison than any product feature: a
|
||||
scheme's redundancy model is close to irrelevant here, and what remains is its storage overhead.
|
||||
|
||||
At 230 GiB with 1.3 TiB free, **storage overhead is not a deciding cost either.** Replication at
|
||||
three copies would run ~690 GiB against the present 468 GiB — about **+222 GiB**, comfortably
|
||||
absorbed. Erasure coding at a wider stripe would *save* roughly 146 GiB. Both are rounding errors
|
||||
against the headroom, and neither should decide this.
|
||||
|
||||
## The candidates
|
||||
|
||||
Four, not three. The previous version omitted the first.
|
||||
|
||||
### The maintained fork of the incumbent
|
||||
|
||||
The community edition was archived upstream and its images deleted
|
||||
([issue 113](../../04-ISSUES/113-the-object-stores-images-were-withdrawn-upstream/00-report.md)),
|
||||
but **a fork is maintained and publishing** — `pgsty/minio`, from the Pigsty project. It restores
|
||||
the console stripped from the community
|
||||
build, rebuilt image and package distribution, tracks CVEs, and states that it preserves the on-disk
|
||||
format, the S3 API and the environment-variable surface. Verified by pulling it: it reports a
|
||||
current release, permissive-to-copyleft licensing unchanged from upstream, and identifies itself as
|
||||
a community fork. Adoption is real — the server image has been pulled three quarters of a million
|
||||
times.
|
||||
|
||||
**Why it reorders the comparison.** Every other candidate costs a data migration, a provisioning
|
||||
handler rewritten against a different admin API, a tool surface ported, and a maintenance window for
|
||||
the opaque consumer. The fork costs **an image reference**. It also closes the issue's one-way door:
|
||||
a node holding nothing can provision the module again, and patches resume.
|
||||
|
||||
**What it does not do** is end the dependence on a codebase its original authors abandoned. It is
|
||||
maintenance mode, largely one project's effort, with no new features intended. It buys time to
|
||||
choose deliberately rather than under pressure — which is worth a great deal, and is not the same as
|
||||
a decision.
|
||||
|
||||
### Garage
|
||||
|
||||
**The best fit for how the mesh provisions.** Its permission model is *per access key, per bucket,
|
||||
read/write/owner* — which is the requirement above stated verbatim rather than approximated. Its
|
||||
admin API is a first-class REST surface with tokens scopeable to exactly the two operations a bucket
|
||||
provision performs. The opaque consumer is **first-party documented** against it, for primary
|
||||
storage, including client-side encryption support.
|
||||
|
||||
Its previously-recorded penalties mostly dissolve under the corrected requirement: it has no console
|
||||
and no identity-provider integration, neither of which is wanted; and it replaces AWS-style ACLs and
|
||||
bucket policies with its own per-key-per-bucket model, which is the thing being asked for.
|
||||
|
||||
**What genuinely remains.** It replicates rather than erasure-codes — immaterial at this volume and
|
||||
on a single array, as above. It does **not implement the full span of S3 endpoints**: object
|
||||
versioning is absent, object locking and server-side encryption endpoints are absent, and lifecycle
|
||||
is partial. **Whether any of the ten buckets depends on those is unmeasured, and it is the one thing
|
||||
that could still disqualify it.**
|
||||
|
||||
### SeaweedFS
|
||||
|
||||
Longest field record of the group, permissive licence, erasure coding, and identity-provider
|
||||
integration on the S3 API through token exchange. **Its console sign-on is a paid feature** — the
|
||||
admin UI itself is open, its identity integration is not. That finding is what falsified the
|
||||
previous version's narrowing, and it is now largely beside the point, since the console is not a
|
||||
requirement.
|
||||
|
||||
What weighs against it here is narrower and more specific: its S3 surface is a **gateway
|
||||
translating onto its own file-system API**, with acknowledged divergence from AWS behaviour at the
|
||||
edges, and there is no first-party documentation for the opaque consumer. For a store already
|
||||
holding real user files in an opaque layout, first-party support is worth more than a feature list.
|
||||
It also carries more moving parts than a single-machine deployment needs.
|
||||
|
||||
### RustFS
|
||||
|
||||
Closest in shape to the incumbent — a similar admin API and client compatibility, so the existing
|
||||
handlers would port with least effort — under a permissive licence, with erasure coding and a
|
||||
console that does integrate an identity provider.
|
||||
|
||||
Two things were recorded about it earlier that were **wrong, and are corrected here**: it is *not* a
|
||||
binary-level drop-in that retains existing data (API compatibility and on-disk compatibility are
|
||||
separate paths, and the on-disk one is preview-scoped with documented encryption limits), and it
|
||||
therefore offers no shortcut around the migration. It also carries **an open defect in the exact
|
||||
area the mesh depends on** — an access key created by an identity-provider user reported denied on
|
||||
all S3 operations.
|
||||
|
||||
Decisively for now: **it reached general availability eight days before this was written.** For a
|
||||
component holding 230 GiB of real user files, field record is a feature, and it does not have one.
|
||||
|
||||
### Ceph RGW
|
||||
|
||||
Remains rejected, for the reason already recorded: disproportionate where the object store is an
|
||||
ordinary module rather than a platform.
|
||||
|
||||
## Where this leaves it
|
||||
|
||||
**The decision is no longer "which product replaces the incumbent".** It is two decisions, and they
|
||||
can be taken in either order but should not be confused:
|
||||
|
||||
1. **Repoint to the maintained fork, or migrate now?** Repointing is an image reference and it
|
||||
closes the issue. Migrating now costs a data copy, two rewrites and a maintenance window, and
|
||||
buys independence from an abandoned codebase sooner.
|
||||
2. **If migrating, which?** On the corrected requirement the ranking is **Garage first** — its
|
||||
permission model *is* the requirement, its provisioning API is the closest match, and the
|
||||
highest-risk consumer is first-party supported. SeaweedFS second, on field record, with a
|
||||
translation-layer caveat that matters more here than its feature list. RustFS not yet, on age.
|
||||
|
||||
Taking (1) does not foreclose (2), and that asymmetry is the argument for taking (1) first.
|
||||
|
||||
## What is still unmeasured
|
||||
|
||||
1. **Whether any of the ten buckets needs object versioning, server-side encryption or lifecycle.**
|
||||
This gates Garage specifically and nothing else here answers it.
|
||||
2. **Whether the fork's release can actually read the incumbent's on-disk format in place.** The
|
||||
format is claimed compatible across a multi-year gap; the migration between them is one-way, so
|
||||
this is tested on a copy or not at all.
|
||||
3. **Whether the opaque consumer's maintenance window is acceptable**, and how long it actually is
|
||||
at 82,496 objects.
|
||||
4. **Whether the eight-drive erasure-coded shape is warranted at all.** The evidence above says it
|
||||
is not buying what it appears to: eight directories, one array, one machine. It looks inherited.
|
||||
|
||||
**Nothing graduates to a decision before 1 and 2.**
|
||||
|
||||
## Two errors in the previous version of this document
|
||||
|
||||
Recorded because the shape of both survives anonymisation and neither is unique to this effort.
|
||||
|
||||
**It ranked on a requirement that did not exist.** Console single sign-on was treated as the axis
|
||||
because the module's configuration showed it wired up, and a wired-up configuration was read as a
|
||||
used feature. It was neither used nor working. *A configured feature is not an observed one*, and
|
||||
the evidence needed was the operator's answer and the logs — both cheap, neither consulted before
|
||||
the ranking was written.
|
||||
|
||||
**It omitted the incumbent's own fork.** The whole effort began because an upstream withdrew its
|
||||
images; whether anyone had continued that upstream was the first question to ask and it was not
|
||||
asked. The candidate list was assembled from a search for *alternatives*, which by construction
|
||||
returns things that are not the incumbent. *When a dependency dies, "who took it over" precedes
|
||||
"what replaces it".*
|
||||
@@ -0,0 +1,57 @@
|
||||
---
|
||||
status: graduated
|
||||
became:
|
||||
- 02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md
|
||||
- 03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md
|
||||
initiated: 2026-09-26
|
||||
touches:
|
||||
- 02-DECISIONS/0113-the-vault-makes-every-secret.md
|
||||
- 02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md
|
||||
- 02-DECISIONS/0048-a-provider-creates-the-credential-the-mesh-minted.md
|
||||
- 03-DESIGN/01-to-be/13-credentials-and-their-rotation.md
|
||||
- 03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md
|
||||
- 04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md
|
||||
---
|
||||
|
||||
# 016 — How a credential can be rotated
|
||||
|
||||
**What.** Which rotation mechanisms the mesh's providers can actually support, measured against
|
||||
their code rather than assumed. Every provider in the catalogue was read, found by listing every definition that provides something: how it names what it
|
||||
makes for a consumer, what its remove destroys, whether it re-applies a password, whether its
|
||||
backend can hold two secrets for one login or two logins on one resource, and how its own
|
||||
administrative credential is set. The consumer side was read too: when a module reads a secret, and
|
||||
what makes it read a new one.
|
||||
|
||||
**Why.** [ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md), as first drafted,
|
||||
chose *overlap*: add a second login beside the first, move every reader, then remove the old one,
|
||||
"through the adapter's existing create and remove", with "no consumer changes". A review showed that
|
||||
claim false. In most providers the consumer's data is named after its login, and remove drops the data
|
||||
with the login. Overlap as written would have deleted every consumer's database on its first
|
||||
rotation. The mechanism has to be chosen on what the providers do.
|
||||
|
||||
**What it touches.** Rotation in 0113 and [to-be 27](../../03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md),
|
||||
which [ADR 0114](../../02-DECISIONS/0114-a-shared-credential-rotates-over-two-credentials.md) decided on
|
||||
these findings. The identity budget in
|
||||
[ADR 0049](../../02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md), if a consumer
|
||||
gets two logins. The rotation already implemented, which [to-be 13](../../03-DESIGN/01-to-be/13-credentials-and-their-rotation.md)
|
||||
describes.
|
||||
|
||||
**Documents.**
|
||||
|
||||
- [01 — The providers](01-the-providers.md): the survey, one row per provider, and what it shows.
|
||||
- [02 — The readers](02-the-readers.md): how a secret reaches a running process, and what already
|
||||
recreates it.
|
||||
- [03 — The options](03-the-options.md): each rotation mechanism against those facts, and a
|
||||
recommendation.
|
||||
|
||||
**Finding, in one paragraph.** All nine credential providers already re-apply a consumer's password
|
||||
in place on every create, and the controller's `rotate` command relies on that. It is a working
|
||||
rotation with a stated window. Eight of the nine name the consumer's resource after its login, and five
|
||||
destroy the consumer's data when they remove the login. The harness, keyed by login, would do the same
|
||||
on any change of login. Only one backend holds two passwords on one login, and two more hold several
|
||||
tokens. Eight backends can grant two logins the same rights over one resource; the ninth can give one
|
||||
login a second token. So every provider can hold **two credentials** over one resource, but only after
|
||||
each adapter separates *the consumer's resource* from *the credential that reaches it*. In postgres
|
||||
that also means the resource belongs to a role no login owns. Administrative credentials are a
|
||||
different case. They have one party and a fixed name, and five backends take them only at first
|
||||
initialisation, so changing one needs the old and the new value at once.
|
||||
@@ -0,0 +1,103 @@
|
||||
# 01 — The providers
|
||||
|
||||
Read from the catalogue's main branch: each provider's provisioner adapter (`create`, `remove`),
|
||||
the client functions they call, and each definition's own credentials. The providers were found by
|
||||
listing every definition that provides something and has a provisioner, not from memory. A first pass
|
||||
of this survey worked from memory and missed one, mailu.
|
||||
|
||||
**The provisioner harness** in `mesh-sdk` calls `create` for a consumer when its contribution appears
|
||||
or changes (its login, password or values), and after the provisioner restarts. It calls `remove` for
|
||||
a login it applied earlier in the same process that is no longer contributed. Its record of what was
|
||||
applied is kept in memory and keyed by login. Two things follow:
|
||||
|
||||
- a consumer whose derived login changes is removed under the old login and created under the new one,
|
||||
in one pass;
|
||||
- a contribution that disappears while the provisioner is down is never removed, and is left behind.
|
||||
|
||||
## The credential providers
|
||||
|
||||
`login` is the consumer's derived identity, which the adapter receives as `as`
|
||||
([ADR 0049](../../02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md)).
|
||||
|
||||
| provider | the consumer's resource is named | remove destroys | create re-applies the password | two secrets on one login | two logins on one resource |
|
||||
|---|---|---|---|---|---|
|
||||
| postgres | a database named `login`, owned by the role `login` | the database and the role | yes, `ALTER ROLE … PASSWORD` when the role exists | no: a role has one password | yes, but only through a role that cannot log in owning the database, with each login working as it. Otherwise whatever one login creates is its own, and dropping that login means handing its objects over first. Not done today |
|
||||
| mssql | a database named `login`, with the login mapped into it | the database and the login | yes, `ALTER LOGIN … WITH PASSWORD` | no: a login has one password | yes, two logins mapped to users in `db_owner`. A user owning a schema cannot be dropped, and a login with an open session cannot. Not done today |
|
||||
| mongodb | a database named `login`, with a user holding `dbOwner` | the database and the user | yes, `updateUser` with the new password | no: a user has one credential | yes, two users with `dbOwner` on one database. Not done today |
|
||||
| redis | the key prefix `login:` on an ACL user named `login` | the user, **not** its keys | yes: `ACL SETUSER … reset … >password` replaces all of them | **yes**: an ACL user holds several passwords, added with `>` and removed with `<`. Today's `reset` discards all but the new one | yes, two users on one key prefix, once the prefix is not the login |
|
||||
| minio | a bucket derived from `login`, and a service account whose access key is `login` | the access key; the bucket **only if empty**. A bucket holding objects is left, and the failure logged | yes, by removing the access key and adding it again, which leaves a moment with no key | no, but an access key *is* the login: a second key is a second login | yes, two service accounts with one bucket policy. The access key is capped at 20 characters |
|
||||
| lavinmq | a virtual host named `login`, and a user named `login` with permissions on it | the virtual host, with any queued messages, and the user | yes, the user is written again with the password | no: a user has one password | yes, permissions for two users on one virtual host |
|
||||
| mosquitto | a client named `login`, with a role named for it on the topic prefix `login/#` | the client and its role | yes, the password is set when the client exists | no: a client has one password | yes, two clients holding one role, once the prefix is not the login. The MQTT client identifier is chosen by the consumer, not tied to the login; a duplicate one takes the older session over |
|
||||
| mailu | a mailbox `login@domain`, unless the consumer contributes its own account name | the mailbox with its mail, for a login-named one; a contributed name is left for an operator | yes, the password is set when the user exists | no for the password; a user can hold several authentication tokens, per the backend's documentation | **no**: a mail user *is* its mailbox |
|
||||
| gitea (npm) | a user named `login` on a team of an organisation that owns every package | the user; **packages survive**, because the organisation owns them | yes, the user's password is set on every run | no for the password; a user can hold several access tokens | yes, trivially: a second member of the same team |
|
||||
|
||||
## The other providers
|
||||
|
||||
| provider | answers with | credential |
|
||||
|---|---|---|
|
||||
| umami | a website, found by its public name | none. The site id it makes has no way back to the consumer today |
|
||||
| cloudflare-dns | a public name derived from `login` | none handed to the consumer; its own API token is an operator value |
|
||||
| showcase | a route | none |
|
||||
| mesh-vault | custody: it records and withdraws sealed values in a ledger | it holds secrets; it makes none today |
|
||||
|
||||
verdaccio provides the npm registry too, and has no provisioner.
|
||||
|
||||
## Each provider's own administrative credential
|
||||
|
||||
| provider | identity | how the backend takes it |
|
||||
|---|---|---|
|
||||
| postgres | a fixed superuser | from a file **only at first initialisation** |
|
||||
| mssql | `sa` | from the environment at first setup. The image documents no file form, and the definition records that as a declared exception |
|
||||
| mongodb | a fixed `root` | from a file **only at first initialisation**, when the data directory is empty |
|
||||
| mosquitto | a fixed admin client | seeded into the broker's dynamic-security file **once**; the seeding step skips when the file exists |
|
||||
| lavinmq | a fixed admin name | per its own bootstrap code, **only on a first boot** with an empty data directory. No resource in the definition runs that bootstrap; what sets it on a running mesh is outside the catalogue |
|
||||
| redis | the default user | from `requirepass` in a configuration the mesh renders, read when the server starts |
|
||||
| minio | a fixed root user | from a file, read when the server starts |
|
||||
|
||||
**In five of seven, a new administrative value takes effect only through a command run with the old
|
||||
one.** The credential file is mounted directly into both the server and the provisioner. So replacing
|
||||
it recreates the provisioner, which then holds only the new value while the backend still expects the
|
||||
old one, and the provisioner is locked out. That is worse than changing nothing.
|
||||
|
||||
Every provider module also has its own bus account, an own secret, read at start.
|
||||
|
||||
## What the tables show
|
||||
|
||||
1. **Every credential provider already rotates in place.** All nine re-apply the password on the
|
||||
same login each time `create` runs. The controller's `rotate` command relies on that: it replaces
|
||||
the credential in the inventory and sends both ends in one push. Its own comments state the window,
|
||||
between the provider applying and the consumer restarting, in which the consumer cannot
|
||||
authenticate.
|
||||
2. **Eight of nine name the consumer's resource after its login.** Only gitea separates them,
|
||||
because an organisation owns the packages. A second login therefore has no resource of its own to
|
||||
reach, and cannot share the first one's without the adapter granting it.
|
||||
3. **Five of nine destroy the consumer's data when they remove the login**: postgres, mssql and
|
||||
mongodb drop the database, lavinmq drops the virtual host with its queued messages, and mailu
|
||||
deletes the mailbox with its mail. minio drops only an empty bucket, and redis leaves the keys. In
|
||||
those five, *retire a login* and *delete the consumer's data* are one call. With the harness keyed
|
||||
by login, a changed login triggers it too.
|
||||
4. **One backend holds two passwords on one login** (redis). Two hold several tokens beside one
|
||||
password (gitea and mailu). A rotation built on two secrets per login would work for three
|
||||
providers out of nine.
|
||||
5. **Eight of nine can give two logins the same rights over one resource.** Group roles in postgres,
|
||||
database roles in mssql and mongodb, permissions in lavinmq, a shared role in mosquitto, a shared
|
||||
policy in minio, a shared key prefix in redis, a shared team in gitea. mailu cannot, because its
|
||||
user is its mailbox, but it can give one user a second token. So every provider can hold **two
|
||||
credentials** over one resource, though not every one as two logins. No adapter does either today.
|
||||
6. **Ownership is a trap in two backends.** In postgres whatever a login creates is that login's, so a
|
||||
second login cannot alter the first one's tables, and the first cannot be dropped while it owns
|
||||
them. The one-step way out deletes them. In mssql, a login cannot be dropped with a session open,
|
||||
nor its user while it owns a schema.
|
||||
7. **The administrative credentials have one party and a fixed name**, and five backends take them
|
||||
only at first initialisation. The provisioner needs the old and the new value at once to change
|
||||
them. Today nothing can give it both.
|
||||
8. **A consumer's identity is already the resource's name.** The login is derived from the
|
||||
assignment, which is a module on a node, so the current login and "the consumer" are the same
|
||||
string today. A second login would need a new name. The resource can keep the one it has.
|
||||
|
||||
## Seen on the way
|
||||
|
||||
The redis configuration names no ACL file, so a consumer's ACL user exists only in memory. A restart
|
||||
of the redis server erases every consumer's user. The provisioner does not create them again until it
|
||||
restarts itself, because its in-memory record says they are done. That is not a rotation finding, but
|
||||
it is a live fault, and it is recorded here so it is not lost.
|
||||
@@ -0,0 +1,46 @@
|
||||
# 02 — The readers
|
||||
|
||||
How a secret reaches a running process, and what makes the process take a new one.
|
||||
|
||||
## No module watches a secret
|
||||
|
||||
A search of every module's code in the catalogue found no file watching of any kind, and no
|
||||
re-reading of a secret while running. **Every reader reads a secret when it starts.** There is no
|
||||
consumer that takes a new value live, so every rotation that changes what a consumer presents ends in
|
||||
the consumer restarting.
|
||||
|
||||
## The host already recreates what read a changed file
|
||||
|
||||
The node host records, for every long-running container, the digest of each file it read when it
|
||||
was created: its env-files, and every file bind-mounted into it directly. When a digest changes, the
|
||||
host recreates the container, even though its spec is otherwise unchanged. This is the fix for
|
||||
[issue 103](../../04-ISSUES/103-a-container-is-not-recreated-when-a-file-it-reads-changes/00-report.md).
|
||||
It is on the host's main branch, while the issue is still recorded as located, not fixed.
|
||||
|
||||
Two cases are deliberately left out and need `restart-on` in the definition:
|
||||
|
||||
- a file read out of a **mounted directory**, because the host cannot know whether the service reads
|
||||
it once or watches it (a route proxy re-reads its routes live; a provisioner polls what it receives);
|
||||
- a **process** rather than a container.
|
||||
|
||||
## What the catalogue does with it
|
||||
|
||||
22 definitions declare a secret they receive. In 16 of them it reaches the service through a
|
||||
rendered file, usually an env-file. That case the host already covers. 14 declare `restart-on` for
|
||||
something. Whether each of the 22 is fully covered depends on how its secret travels: through an
|
||||
env-file or a direct mount, which the host covers, or through a directory or into a process, which
|
||||
needs `restart-on`. **That was not classified module by module.** It is the check to run before a
|
||||
rotation mechanism relies on it.
|
||||
|
||||
The count covers only the `secrets` field. The 49 modules with their own bus account, and 54 with any
|
||||
own secret, are readers too, and their bus accounts are rotated like any credential two parties hold.
|
||||
Their files are mounted directly, which the host covers, but the classification has to name them.
|
||||
|
||||
## What this means for rotation
|
||||
|
||||
- The *read at start* half of 0113's recipient model is already true, and mostly already handled by
|
||||
the host. The restart is derived from the files a container reads, not declared per secret.
|
||||
- Any mechanism, in place or overlapping, ends with the reader being recreated. What differs is
|
||||
whether the credential it held until then still works.
|
||||
- For a single-party secret, a module's own, the reader is also the only holder. There is nobody to
|
||||
overlap with, and delivering the new file recreates the reader.
|
||||
@@ -0,0 +1,79 @@
|
||||
# 03 — The options
|
||||
|
||||
Three mechanisms, weighed against [01](01-the-providers.md) and [02](02-the-readers.md).
|
||||
|
||||
## A. In place, as today
|
||||
|
||||
The vault makes a new value. Every applier re-applies it on the same login, which all nine
|
||||
providers already do. Every reader is recreated by the host.
|
||||
|
||||
- **Works with:** every provider, unchanged. It is what `rotate` does now.
|
||||
- **Costs:** a window per consumer, from the provider applying to the consumer being recreated. They
|
||||
are on different machines, and nothing orders them. A reader whose machine is unreachable from the
|
||||
mesh but still reaches its provider stays locked out until the mesh reaches it again.
|
||||
- **Admin credentials:** the natural form. The provider module is the only party, and it has to
|
||||
apply the new value with the old one anyway (finding 6).
|
||||
|
||||
## B. Two secrets on one login
|
||||
|
||||
The applier adds the new password beside the old one, readers move, and the old one is removed.
|
||||
|
||||
- **Works with:** redis natively, and gitea and mailu through tokens. **Not** with the other six, whose
|
||||
backends hold one password per login (finding 4).
|
||||
- **Verdict:** not a mechanism, a special case. Using it where it exists and something else
|
||||
elsewhere is the "this way or that way" the design is trying to remove.
|
||||
|
||||
## C. Two credentials per consumer, over one resource
|
||||
|
||||
The consumer has two credentials and uses one at a time. The applier ensures the other with the new
|
||||
value and gives it the same rights over the consumer's resource. Readers move to it, and then the old
|
||||
credential is retired, which removes the credential only, never the resource. **What a credential is,
|
||||
is the adapter's**: a second login for eight providers (finding 5), a second token on the same login
|
||||
for mailu. The mesh sees one mechanism.
|
||||
|
||||
- **Works with:** every provider, **after** each adapter changes:
|
||||
- the resource is named after the consumer, not the login. Today the two are the same string (finding
|
||||
8), so existing resources keep their names, and the current login stays one of the two;
|
||||
- the resource is owned by the resource, not by a login. In postgres that is a role no one logs in
|
||||
as, which each login works as, and ownership of an existing database moves to it once (finding 6);
|
||||
- both credentials get the same rights, over data and structure;
|
||||
- *retire a credential* and *remove the consumer* become two operations. Today they are one call, and
|
||||
in five providers that call destroys data (finding 3). The harness must key by consumer, so that a
|
||||
changed login is not a removal. This is the whole of the danger, and it has to be split, whatever
|
||||
else is chosen.
|
||||
- **Costs:**
|
||||
- every credential adapter changes;
|
||||
- the harness learns the alternation and a confirmation per step, and rotation state has to live
|
||||
somewhere that survives a restart, which the harness's memory does not;
|
||||
- the second login's name must fit the tightest backend. That is 20 characters for a minio access
|
||||
key ([ADR 0049](../../02-DECISIONS/0049-a-consumers-identity-fits-the-tightest-backend.md)), and
|
||||
a suffix spends part of it;
|
||||
- retiring an mssql login has to end its sessions first.
|
||||
- **Gains:** no window. A reader that cannot be reached keeps a working credential until it can.
|
||||
- **Does not apply to** single-party secrets: admin credentials and a module's own secrets. There is
|
||||
no second party to overlap with.
|
||||
|
||||
## Independent of the choice
|
||||
|
||||
- **Split remove, and key the harness by consumer.** Retiring a credential, or a login changing, must
|
||||
never be able to destroy a consumer's data. That holds under A too, because A's remove is the same
|
||||
call.
|
||||
- **Admin credentials are applied by their own provider**, using the old value, with the new one staged
|
||||
beside it. Five backends take the value only at first initialisation. Replacing the file first locks
|
||||
the provisioner out (finding 7).
|
||||
- **Classify the readers** ([02](02-the-readers.md)), bus-account readers included, before relying on
|
||||
derived restarts.
|
||||
|
||||
## Recommendation
|
||||
|
||||
- **Two-party credentials, consumer credentials and bus accounts: C**, because it is the only
|
||||
mechanism every provider supports, and it closes the window instead of shortening it. Its prerequisite, separating the resource from the login and
|
||||
retiring a login from removing a consumer, is worth doing on its own, because it removes a
|
||||
data-loss path that exists today.
|
||||
- **Single-party secrets (admin credentials, a module's own): A, staged.** In place, applied by the
|
||||
provider that holds them, with the new value beside the old until it has taken.
|
||||
- **Until the adapters are changed, A stays** as `rotate` implements it, with its window stated. It is
|
||||
not replaced by a mechanism the providers cannot yet carry.
|
||||
|
||||
This is two mechanisms, split by a property of the secret rather than by provider: whether it has one
|
||||
party or two. Every provider is treated the same way for the same kind of secret.
|
||||
@@ -0,0 +1,47 @@
|
||||
---
|
||||
status: active
|
||||
initiated: 2026-09-26
|
||||
touches:
|
||||
- 00-META/mission.md
|
||||
- 02-DECISIONS/0106-the-bus-is-nats.md
|
||||
- 02-DECISIONS/0010-delivery.md
|
||||
- 02-DECISIONS/0083-one-push-leaves-the-mesh-consistent.md
|
||||
- 03-DESIGN/01-to-be/06-the-controller.md
|
||||
- 03-DESIGN/01-to-be/09-the-node-lifecycle.md
|
||||
- 03-DESIGN/00-as-is/09-interfaces-and-observability.md
|
||||
---
|
||||
|
||||
# 017 — A mesh that heals itself
|
||||
|
||||
**What.** The behaviour the operator wants: a mesh that runs itself. It notices what is wrong,
|
||||
repairs what it can, and hands what it cannot repair to someone who can, with the reason. This effort
|
||||
writes that wish down as intended behaviour, designed for the bus the mesh is moving to
|
||||
([ADR 0106](../../02-DECISIONS/0106-the-bus-is-nats.md): NATS). It also records what can be done
|
||||
pragmatically before that move.
|
||||
|
||||
**Why.** The mission is *a mesh that controls itself* ([mission](../../00-META/mission.md)). The
|
||||
mesh can tell whether it is up. It cannot tell whether it is right. The as-is page on observability says so
|
||||
([as-is 09](../../03-DESIGN/00-as-is/09-interfaces-and-observability.md)). To-be 06 names an
|
||||
`observability` context in the controller and leaves its store undecided. Nothing routes a condition
|
||||
the mesh cannot fix to anyone. The cost is measurable: **46 of the 116 issue reports in this
|
||||
repository describe a failure that was silent.** A mesh that heals itself is, first, a mesh that stops
|
||||
failing silently.
|
||||
|
||||
**What it touches.** The controller's observability context, the node lifecycle's liveness, delivery
|
||||
([ADR 0010](../../02-DECISIONS/0010-delivery.md), [ADR 0083](../../02-DECISIONS/0083-one-push-leaves-the-mesh-consistent.md)),
|
||||
the provisioner harness, and rotation, which is proposed alongside to-be 27 as ADR 0114.
|
||||
|
||||
**Documents.**
|
||||
|
||||
- [01 — The intended behaviour](01-the-intended-behaviour.md): the wish, as principles and as how
|
||||
the mesh behaves once the bus is NATS.
|
||||
- [02 — Now, pragmatically](02-now-pragmatically.md): what is done before NATS, why it does not
|
||||
build anything the move would throw away, and what has been done already.
|
||||
|
||||
**Next.** Two measurements this effort owes before it can graduate:
|
||||
|
||||
1. **Every loop in the mesh**: what it converges, and whether it compares against observed state or
|
||||
against its own memory. Issue 120 found the provisioner harness trusting memory. The same pattern is
|
||||
expected elsewhere.
|
||||
2. **The 46 silent failures, classified**: a missing observation, a loop trusting memory, or a missing
|
||||
escalation. That shows which mechanism removes the most of them.
|
||||
@@ -0,0 +1,85 @@
|
||||
# 01 — The intended behaviour
|
||||
|
||||
The operator's wish, written as behaviour: what a person or an agent sees the mesh doing. This is a
|
||||
target to design toward, not a design. Every part of it is to be decided through a record before it
|
||||
is built.
|
||||
|
||||
## Principles
|
||||
|
||||
**1. Every loop compares what should be with what is, never with what it did.** Desired state is the
|
||||
mesh's: assignments, requirements, seats. Observed state is read from the thing itself: the container,
|
||||
the backend, the node. A loop that compares against its own memory of what it applied is blind to
|
||||
anything that changed behind its back. That is issue 120, and it is the pattern this whole effort is
|
||||
written against.
|
||||
|
||||
**2. Healing is the ordinary path run again, never a second path.** Repairing a lost login is
|
||||
provisioning it. Repairing a dead container is converging the node. Repairing a stale declaration is
|
||||
delivering it. A repair that needs its own code is a second way of doing something, which is exactly
|
||||
what the mesh is removing everywhere else.
|
||||
|
||||
**3. A repair never destroys.** Healing may recreate, re-provision, re-deliver and restart. It may
|
||||
never delete a consumer's data, retire a credential someone still uses, or pick a winner between two
|
||||
contradictory states. Where the only repair is destructive, it is escalated.
|
||||
|
||||
**4. Nothing fails silently.** Every condition the mesh cannot repair within its budget becomes
|
||||
visible. It is named, it says since when, why, and who can resolve it. It is visible until it is
|
||||
resolved, and resolved by observation, not by someone clicking it away.
|
||||
|
||||
**5. What the mesh cannot fix goes to an agent.** Per the mission, an agent may be human or not. A
|
||||
condition that needs judgement is handed to one, as work, with what the mesh knows. It is not handed
|
||||
over as a notification that someone may or may not read.
|
||||
|
||||
**6. Correctness, not only liveness.** A running process that authenticates with a dead credential,
|
||||
serves an old version, or routes nowhere is not healthy. What a provision's contract promises is what
|
||||
is checked: the credential authenticates, the route answers, the version is the declared one.
|
||||
|
||||
## The loop, everywhere
|
||||
|
||||
Every part of the mesh that owns something runs the same loop:
|
||||
|
||||
1. **know** what should be true: from assignments, requirements and seats;
|
||||
2. **observe** what is true: from the thing itself, on its own cadence;
|
||||
3. **repair** the difference by running the ordinary path again, within a budget of attempts and
|
||||
time;
|
||||
4. **raise** a *condition* when the budget is spent or the only repair is destructive;
|
||||
5. **clear** the condition when observation shows it resolved.
|
||||
|
||||
A **condition** is a durable fact about something the mesh owns, such as a node, an assignment, a
|
||||
provision, a seat or a rotation: what is wrong, since when, the evidence, what was tried, and who can
|
||||
resolve it. Conditions are the one thing a person or an agent looks at to know whether the mesh is
|
||||
right. `status` is the list of open conditions. When it is empty, the mesh is right, not just up.
|
||||
|
||||
## On NATS
|
||||
|
||||
[ADR 0106](../../02-DECISIONS/0106-the-bus-is-nats.md) moves the bus to NATS, and NATS makes most of
|
||||
this cheaper, because observation becomes something every component publishes rather than something
|
||||
a central process polls.
|
||||
|
||||
| the wish needs | on NATS |
|
||||
|---|---|
|
||||
| every component says it is alive | a heartbeat on a subject per node and assignment; silence past its interval is a condition, and nobody polls |
|
||||
| every component says what it observed | observations published on subjects (`mesh.observed.<node>.<assignment>`, for instance), consumed by whoever owns the comparison |
|
||||
| the last known state survives restarts | a JetStream key-value bucket of observed state per owner; the provisioner's "what I applied" and a rotation's step live there, not in memory |
|
||||
| conditions are durable and watchable | conditions as entries in a key-value bucket, watched by anyone who cares: a surface, an agent, the controller |
|
||||
| the bus itself is observed | the server's advisories (a consumer exceeding its deliveries, a slow consumer, a client disconnecting) and its monitoring endpoint become observations like any other |
|
||||
| a repair is retried, not lost | JetStream redelivery with delay, which is the same mechanism [ADR 0083](../../02-DECISIONS/0083-one-push-leaves-the-mesh-consistent.md)'s guarantee moves to |
|
||||
| work handed to an agent | a condition that needs judgement published as a task on a subject an agent's queue group consumes |
|
||||
|
||||
**Who compares.** Each owner compares its own: the host for its node's containers and files, a
|
||||
provisioner for its backend, the vault for rotations, the controller for delivery and seats. The
|
||||
controller's observability context does not repair anything. It holds conditions, their history,
|
||||
and the view across the mesh. It notices what no owner can see about itself: an owner gone silent.
|
||||
|
||||
## What stays human
|
||||
|
||||
Some repairs need the operator's key, and the mesh says so rather than pretending otherwise:
|
||||
re-raising the vault or the broker, and recovering a node's identity. These are conditions too, with
|
||||
the procedure named, and they are the only ones that can never clear themselves.
|
||||
|
||||
## Open
|
||||
|
||||
- The budgets: how many attempts, over how long, per kind of repair.
|
||||
- How a condition that needs judgement reaches an agent, and how the agent's action is recorded.
|
||||
- Where the observability context stores history (to-be 06 left it open; volume argues against the
|
||||
relational store).
|
||||
- Which correctness probe each provision's contract offers, and how often it runs.
|
||||
@@ -0,0 +1,46 @@
|
||||
# 02 — Now, pragmatically
|
||||
|
||||
The intended behaviour lands on NATS. The bus moves after the migration's core
|
||||
([ADR 0106](../../02-DECISIONS/0106-the-bus-is-nats.md)). Until then, work toward it is chosen by one
|
||||
test:
|
||||
|
||||
**Does it survive the move?** A change to what a loop compares against, or to what an adapter can
|
||||
tell about its backend, survives, because it is independent of the bus. A new AMQP queue for health
|
||||
reports, a poller written against the broker's management API, or a condition store built on the
|
||||
current broker does not survive, and is not built.
|
||||
|
||||
## Done
|
||||
|
||||
**The provisioner asks the backend, not memory** (issue 120). The SDK's harness gained an optional
|
||||
`holds` on the adapter, asked for every applied consumer every minute. A consumer the backend no
|
||||
longer holds is provisioned again. Being unable to ask is not treated as loss. The cache module
|
||||
implements it first, because its server keeps its users in memory and forgets them all on a restart.
|
||||
That was verified against a real server: a restart erases every consumer's user, and `holds` answers
|
||||
correctly for absent, present, wrong-password, disabled and deleted.
|
||||
Changes: mesh-sdk PR #7 (0.1.1) and mesh-catalog PR #84.
|
||||
|
||||
This is principle 1 applied to one loop. It survives the move unchanged. On NATS, the harness's
|
||||
record of what it applied moves from memory into a key-value bucket, and `holds` stays as it is.
|
||||
|
||||
## Next, in order of silent failures removed
|
||||
|
||||
1. **`holds` for the other credential providers.** Each backend can answer whether a login exists
|
||||
with the mesh's password without changing anything. Where a backend cannot check a password without
|
||||
logging in, logging in is the check.
|
||||
2. **The harness's other blind spot.** A consumer that goes away while its provisioner is down is never
|
||||
removed. The fix is the same principle in reverse: list what the backend holds, and compare it with
|
||||
what the mesh asks for. Removal stays subject to ADR 0114's rule that it never follows from a login
|
||||
changing.
|
||||
3. **`status` reports what owners already know.** The host knows which containers it recreated and
|
||||
why. A rotation knows who it waits on. Delivery knows what is outstanding. Surfacing those as
|
||||
conditions in the existing `status` needs no new transport. It is the shape the NATS condition store
|
||||
will hold.
|
||||
4. **The loop inventory and the classification** in [00](00-overview.md). They decide what comes after
|
||||
these three.
|
||||
|
||||
## Not now
|
||||
|
||||
- Heartbeats, observation subjects, key-value state, advisories: all NATS, all after the move.
|
||||
- Handing conditions to agents: designed with NATS, where a task on a subject is native.
|
||||
- Choosing the observability store: decided when there is something to store, which is after the
|
||||
move.
|
||||
@@ -0,0 +1,86 @@
|
||||
---
|
||||
status: graduated
|
||||
initiated: 2026-10-02
|
||||
touches:
|
||||
- 02-DECISIONS/0040-what-a-module-is.md
|
||||
- 02-DECISIONS/0011-managed-files-are-generated-never-edited.md
|
||||
- 02-DECISIONS/0132-a-seat-carries-the-tools-its-holder-must-serve.md
|
||||
- 02-DECISIONS/0150-a-modules-own-code-runs-as-supervised-processes-under-one-account.md
|
||||
- 02-DECISIONS/0152-the-operators-surface-is-a-module-the-console.md
|
||||
- 02-DECISIONS/0161-what-deserves-a-seat.md
|
||||
- 03-DESIGN/01-to-be/05-the-node-host.md
|
||||
- 03-DESIGN/01-to-be/29-a-node-has-operator-accounts.md
|
||||
- 03-DESIGN/01-to-be/33-the-tools-the-mesh-answers.md
|
||||
- 03-DESIGN/01-to-be/34-the-console.md
|
||||
- 03-DESIGN/00-as-is/10-module-catalogue.md
|
||||
- 04-ISSUES/160-a-machine-says-little-about-itself-and-only-when-asked/00-report.md
|
||||
- 04-ISSUES/168-a-setting-reaches-every-file-and-contribution/00-report.md
|
||||
became:
|
||||
- 03-DESIGN/01-to-be/37-the-operators-machine.md
|
||||
- 02-DECISIONS/0173-the-operators-machine-is-the-meshs-and-a-module-is-what-it-declares.md
|
||||
- 02-DECISIONS/0174-a-node-varies-a-module-through-settings-and-kept-regions-never-an-edit.md
|
||||
- 02-DECISIONS/0175-one-tool-runtime-per-node-serves-every-modules-tools-on-the-host-side.md
|
||||
- 02-DECISIONS/0176-the-login-shell-is-a-node-seat-and-execute-is-its-contract.md
|
||||
- 02-DECISIONS/0177-a-unit-may-be-user-scoped-and-the-service-manager-is-a-node-seat.md
|
||||
---
|
||||
|
||||
# 018 — The operator's machine as modules
|
||||
|
||||
**What.** The mesh owns the whole machine, not only the services on it. Everything a person
|
||||
configures on a node — the login manager, the display server, the window manager, the shell, the
|
||||
terminal, the launcher, the notifier, the audio setup, the boot images, the downloads folder, the
|
||||
agent at the terminal — is a module: a package, the files it owns under `/etc` and under the
|
||||
operator's home, the seat it holds, the tools it serves. One default configuration per module,
|
||||
varied per node only through settings rendered into the file or a kept operator region, never
|
||||
through an edit. The servers take the universal modules (shell, prompt, git, the agent); the
|
||||
workstations take those and the graphical stack, which a capability the machine reports gates.
|
||||
This effort writes that behaviour down, measures what the predecessor's desktop modules actually
|
||||
contain, and settles what the mesh must gain before the first of them can be written.
|
||||
|
||||
**Why.** The predecessor is retired on every node. What it still owned on the two workstations —
|
||||
about thirty modules' worth of dotfiles, user units and `/etc` files — is now owned by nothing:
|
||||
no generator regenerates them, and a fix to one of them is a hand edit that nothing records. The
|
||||
migration scoped these modules out as *the workstation's own environment*, and
|
||||
[to-be 29](../../03-DESIGN/01-to-be/29-a-node-has-operator-accounts.md) names them as the last
|
||||
thing the predecessor was keeping alive. To-be 29 covers one directory, `~/.ssh`, and draws a
|
||||
boundary inside it. The operator wants no boundary: the machine is the mesh's, as far as it makes
|
||||
sense to configure it. That is a wider scope than any design states, and it reaches three records
|
||||
that were written for services: what a module is, where a module's tools run, and what a managed
|
||||
file may be.
|
||||
|
||||
**What it touches.** The module definition ([ADR 0040](../../02-DECISIONS/0040-what-a-module-is.md)),
|
||||
seats and their contracts ([ADR 0132](../../02-DECISIONS/0132-a-seat-carries-the-tools-its-holder-must-serve.md)),
|
||||
where a module's tools run ([ADR 0150](../../02-DECISIONS/0150-a-modules-own-code-runs-as-supervised-processes-under-one-account.md),
|
||||
[ADR 0152](../../02-DECISIONS/0152-the-operators-surface-is-a-module-the-console.md),
|
||||
[to-be 33](../../03-DESIGN/01-to-be/33-the-tools-the-mesh-answers.md) §6), the host's vocabulary
|
||||
([to-be 05](../../03-DESIGN/01-to-be/05-the-node-host.md)), managed files and settings
|
||||
([ADR 0011](../../02-DECISIONS/0011-managed-files-are-generated-never-edited.md),
|
||||
[issue 168](../../04-ISSUES/168-a-setting-reaches-every-file-and-contribution/00-report.md)),
|
||||
and the catalogue's shape ([as-is 10](../../03-DESIGN/00-as-is/10-module-catalogue.md)).
|
||||
|
||||
**Documents.**
|
||||
|
||||
- [01 — The intended behaviour](01-the-intended-behaviour.md): the operator's wish, written as
|
||||
how the mesh behaves, in the mesh's own words.
|
||||
- [02 — What exists, and what is missing](02-what-exists-and-what-is-missing.md): the
|
||||
predecessor's desktop modules measured; which records already say what is wanted; the gaps.
|
||||
- [03 — One tool executor per node](03-one-tool-executor-per-node.md): where a module's tools
|
||||
run. The direction the operator set, the evidence for it, and what it supersedes.
|
||||
- [04 — The seats of the environment](04-the-seats-of-the-environment.md): the roles a machine
|
||||
has once, their candidate contracts, and what gates each.
|
||||
|
||||
**What it had to settle, and where each landed.** *(Graduated 2026-10-02.)*
|
||||
|
||||
1. A module is one *managed thing*, software or not, and every module may serve tools — or ADR
|
||||
0040 already says this and only its examples are narrow.
|
||||
2. One tool executor per node, host-side, module-agnostic; which records it supersedes and
|
||||
in what form the console continues.
|
||||
3. Per-node variation is a setting rendered into the file or a kept region, never an edit —
|
||||
ADR 0011 stands — and issue 168 is fixed before any environment module carries a setting.
|
||||
4. User-scoped units on the host's `service` shape, and a service-manager seat whose holder
|
||||
serves the tools about them.
|
||||
5. The operator account stated on every node; today no node record carries one.
|
||||
6. The seats of the environment and their verbs, one record per seat, slowly, because a
|
||||
seat's tools bind every future holder.
|
||||
7. Where the environment modules live: this catalogue, or one of their own as the media chain
|
||||
has; and whether a third-party organisation's tooling belongs in a public catalogue at all.
|
||||
@@ -0,0 +1,99 @@
|
||||
# 01 — The intended behaviour
|
||||
|
||||
*Written 2026-10-02 from the operator's words, in the mesh's words. What is wanted, before what
|
||||
exists. Where a sentence restates a record, the record is named; where it goes further, that is
|
||||
said.*
|
||||
|
||||
## The machine is the mesh's
|
||||
|
||||
**Everything configurable on a node is declared by a module.** Not only the services the mesh
|
||||
runs: the login manager, the display server, the window manager, the bar, the launcher, the
|
||||
notifier, the compositor, the lock screen, the terminal emulator, the clipboard, the shell and its
|
||||
prompt, the editor, the audio setup, the boot images, the package manager's configuration, the
|
||||
agent a person runs at a terminal, and the folders a person works in — a downloads folder that is
|
||||
tidied, backed up, distributed to other nodes and asked questions of. System folders and the
|
||||
operator's home alike. The operator is the only person on every node, so the mesh manages the
|
||||
person's machine, not a machine with a person on it.
|
||||
|
||||
This is [ADR 0040](../../02-DECISIONS/0040-what-a-module-is.md)'s definition applied without
|
||||
the service bias its examples carry. A module is one managed thing, named once, described
|
||||
completely by its manifest. It may have a package, files, a container, a unit, a binary, a seat it
|
||||
holds, and tools it serves — any one of these, or all, or two. There is **no kind of module**: zsh
|
||||
has a package, files, a seat claim and the tools that claim obliges it to serve; downloads has a
|
||||
folder, a process and tools; nftables has a package, files, a service, a seat and tools. The
|
||||
difference is what each declares, not what each is.
|
||||
|
||||
**The home has no boundary.** [To-be 29](../../03-DESIGN/01-to-be/29-a-node-has-operator-accounts.md)
|
||||
owns one directory under the home and draws a line inside it between the mesh's and the person's.
|
||||
Here the line is drawn only by what the modules declare: every file some module places is the
|
||||
mesh's; what no module declares is found and left alone, exactly as the adoption rules already
|
||||
say for a machine. The reach is bounded by sense, not by a rule — the mesh configures what can
|
||||
be configured, and a person's documents, projects and history are data under
|
||||
[ADR 0051](../../02-DECISIONS/0051-shared-data-is-the-operators.md), not configuration.
|
||||
|
||||
**A module names no node and no path.** The operator account is a node fact and the home is
|
||||
derived from it ([to-be 29](../../03-DESIGN/01-to-be/29-a-node-has-operator-accounts.md) §1–2,
|
||||
shipped in the controller; its record is proposed in an open change). A module places a file
|
||||
*under the home, owned by the account*, and the same manifest lands on a server and a laptop.
|
||||
|
||||
## One default, varied by settings, never by edits
|
||||
|
||||
**One module, one default configuration.** The window manager module ships the configuration
|
||||
that is right for every node. There are no flavors: the predecessor's one desktop module carried
|
||||
four, one per class of machine, and what differed between them is what settings are for.
|
||||
|
||||
**A node varies a module in exactly two ways.** A **setting**, declared by the module with its
|
||||
type, meaning and default (proposed alongside the container-runtime records), set for the mesh
|
||||
or for one node, and rendered into the file at composition — the value is in the file, not in an
|
||||
environment variable the file reads. Or a **kept region**: a block in a file the mesh writes
|
||||
*into*, where the operator's own lines survive every push
|
||||
([ADR 0102](../../02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md)). An
|
||||
edit to a managed file outside such a region is not a third way; it is overwritten, as
|
||||
[ADR 0011](../../02-DECISIONS/0011-managed-files-are-generated-never-edited.md) says, and the
|
||||
predecessor's habit of adopting disk drift back into its database is not carried over.
|
||||
|
||||
The predecessor's theming — some ninety environment variables substituted into templates at sync
|
||||
time, with tools to list and set them — is the same idea with the wrong rendering. The knobs
|
||||
become declared settings; the file carries the value.
|
||||
|
||||
## Roles a machine has once are seats, and seats carry tools
|
||||
|
||||
**A role a machine fills at most once is a node-scoped seat**, declared by a module
|
||||
([ADR 0121](../../02-DECISIONS/0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md),
|
||||
[ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md)): the login shell, the
|
||||
display session, the display server, the terminal emulator, the launcher, the notifier, the
|
||||
compositor, the lock screen, the service manager, the boot loader. Several modules may be able to
|
||||
hold one — zsh, fish and bash can all hold the login shell — and the assignment on each node says
|
||||
which does. Installing a shell is installing software; holding the seat is being *the* shell.
|
||||
|
||||
**A seat's contract is its tools** ([ADR 0132](../../02-DECISIONS/0132-a-seat-carries-the-tools-its-holder-must-serve.md)).
|
||||
Every holder of the login-shell seat serves `execute`, which takes one string, the command, and
|
||||
runs it on the node the seat is scoped to. Every holder of the boot seat serves "rebuild the boot
|
||||
images", so *"rebuild your boot images"* is a verb addressed to a machine, not a one-off step in
|
||||
a hook. Every holder of the service-manager seat answers for the units on the machine, system and
|
||||
user scope. A module may serve its own tools beside the seat's
|
||||
([ADR 0170](../../02-DECISIONS/0170-the-firewall-seat-serves-its-verbs.md) §2): show the rendered
|
||||
configuration, set a theme value, report status.
|
||||
|
||||
**Any tool may be called from any node.** The operator's statement, and the grant model it
|
||||
implies: the executor on each node may call everything, as the console already may. A verb that
|
||||
needs root on the machine is the module's concern — the tool escalates, the executor and the
|
||||
caller do not know.
|
||||
|
||||
## Servers and workstations differ by capability, not by catalogue
|
||||
|
||||
The same catalogue serves every node. A module declares what it needs — a graphical session, a
|
||||
display server, a container runtime — and the machine reports what it has, as the profile already
|
||||
reports eight capabilities today ([issue 160](../../04-ISSUES/160-a-machine-says-little-about-itself-and-only-when-asked/00-report.md)).
|
||||
Assignment refuses the wrong placement by name
|
||||
([ADR 0161](../../02-DECISIONS/0161-what-deserves-a-seat.md) §3). So every node takes the shell,
|
||||
the prompt, git and the agent; only a node with a graphical session can take the display server,
|
||||
and only a node holding the display server can take a window manager. Nothing in a module says
|
||||
"workstation".
|
||||
|
||||
## What the operator would say to the mesh
|
||||
|
||||
*Set the login shell on the build node to fish. Rebuild the laptop's boot images. Show me the
|
||||
window manager's effective configuration on the desktop and where each value comes from. Give
|
||||
the downloads folder on the laptop to the home server. Run `uptime` on every node.* Each of these
|
||||
is a seat verb or a module tool, addressed to a node, answered by whatever holds the role there.
|
||||
+91
@@ -0,0 +1,91 @@
|
||||
# 02 — What exists, and what is missing
|
||||
|
||||
*Measured 2026-10-02 on one installation: two workstations, two servers, all four converged to
|
||||
the mesh; the predecessor retired on the last workstation the day before. Numbers are from the
|
||||
machines and the repositories, not from memory.*
|
||||
|
||||
## 1. What the predecessor's desktop looks like
|
||||
|
||||
The predecessor's catalogue on the laptop held **34 modules**, of which **28** are the operator's
|
||||
environment rather than services. By what they declare:
|
||||
|
||||
| shape | count | examples |
|
||||
|---|---|---|
|
||||
| package only | 9 | browser, mail client, process monitor, media player, file manager, chat |
|
||||
| package + `/etc` files + system service | 5 | login manager, display server, power and thermal daemons, package manager configuration |
|
||||
| package + files under the home | 6 | shell and prompt, the agent at the terminal, scripts, the sync client, a music player |
|
||||
| files under the home + user units + hooks | 2 | the desktop environment, audio |
|
||||
| third-party organisation tooling | 6 | out of scope here |
|
||||
|
||||
**The desktop module alone** declares **88 files**, **4 flavors** (the window-manager stack, and
|
||||
one per class of machine), **2 user units** with a hook to enable them, 8 files under `/etc`, a
|
||||
wallpaper shipped as an asset, and reads **about 90 environment variables** as theme knobs,
|
||||
substituted into its templates at sync time and set through a theming tool. Its hook exists
|
||||
because *shipping a unit file does not run it*: one unit had been deployed for months and ran on
|
||||
one machine only, because somebody had enabled it there by hand.
|
||||
|
||||
**The shell module** ships `~/.zshrc`, the prompt configuration, an `~/.ssh/config` that the
|
||||
predecessor generated from its registry, and a `LOGIN_SHELL` variable applied with `chsh` by a
|
||||
hook. Two flavors: the prompt theme, and autocompletion.
|
||||
|
||||
**Other modules write into the desktop module's files.** The chat client places i3 and notifier
|
||||
snippets into `config.d` directories the desktop module owns, and its launch flags, window
|
||||
placement and notification colours are each a variable with a default.
|
||||
|
||||
**One-off steps live in hooks** across the set: enable user units, `chsh`, create a swap file,
|
||||
`mkinitcpio`, enable a vendor VPN service the package ships disabled. Every one is state the
|
||||
host could declare or a verb a seat could serve; none is today.
|
||||
|
||||
## 2. What the migration did with them
|
||||
|
||||
The migration's module to-do scoped the whole set out as *desktop / workstation ricing — the
|
||||
workstation's own environment* and *node/OS tooling — managed on the node, never catalogue*. The
|
||||
last workstation's runbook then split the same set three ways: **A**, system scope, which the
|
||||
host's vocabulary can express today (the login manager, the display server, the power daemons,
|
||||
the package manager, the container runtime); **B**, under a home or a user unit, waiting on
|
||||
to-be 29; **C**, package only, the operator's call. The migration log closes the workstation with
|
||||
*the operator's desktop awaiting its design*.
|
||||
|
||||
Two things followed from scoping them out. Nothing regenerates those files now, so a fix is a hand
|
||||
edit — the login manager's session script was fixed this way on the day of writing, and recorded
|
||||
in a repository nothing deploys from. And the one piece of this family written as a mesh module,
|
||||
the ssh client, was closed on hold in the catalogue until the controller carried the account fact.
|
||||
|
||||
## 3. What the records already give
|
||||
|
||||
| wanted | record | state |
|
||||
|---|---|---|
|
||||
| one module per managed thing; every module may have tools | [ADR 0040](../../02-DECISIONS/0040-what-a-module-is.md) | accepted; examples are services, and the shell is named as a *shared* seat |
|
||||
| a module declares its own node-scoped seat | [ADR 0121](../../02-DECISIONS/0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md), [ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md) | accepted |
|
||||
| a seat's contract is its tools; a holder may add its own | [ADR 0132](../../02-DECISIONS/0132-a-seat-carries-the-tools-its-holder-must-serve.md), [ADR 0170](../../02-DECISIONS/0170-the-firewall-seat-serves-its-verbs.md) | accepted; one node seat serves verbs live |
|
||||
| a capability the machine reports gates a holder | [ADR 0161](../../02-DECISIONS/0161-what-deserves-a-seat.md) §3 | accepted; the profile already reports `graphical-session` |
|
||||
| the account as a node fact; a file under the home owned by it | [to-be 29](../../03-DESIGN/01-to-be/29-a-node-has-operator-accounts.md) §1–2 | built in the controller; its record proposed in an open change |
|
||||
| inside a home: owned, written into, written by the module, found | proposed in the same change | proposed |
|
||||
| a setting declared with type, meaning, default and cost | proposed with the container-runtime records | proposed |
|
||||
| a managed file is derived; an edit is overwritten | [ADR 0011](../../02-DECISIONS/0011-managed-files-are-generated-never-edited.md) | accepted |
|
||||
| the mesh writes into a shared file, never over it | [ADR 0102](../../02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md) | accepted |
|
||||
| a module names no path; the host resolves the home | [ADR 0112](../../02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md) | accepted |
|
||||
| the `user` shape: a login shell is declared state | [to-be 05](../../03-DESIGN/01-to-be/05-the-node-host.md) | designed; used by no module |
|
||||
|
||||
## 4. What is missing
|
||||
|
||||
1. **The account is recorded nowhere.** The node record has the column; on all four nodes it
|
||||
is empty. Every home-scoped module is unassignable until the operator states it.
|
||||
2. **User-scoped units.** The host's `service` shape has no user scope. To-be 29 says it
|
||||
plainly: *a workstation's per-user daemons have no form the mesh can send.* The desktop
|
||||
module's two units, the audio masks, the power module's memory guard and the thermal
|
||||
daemon's profile switcher all need it.
|
||||
3. **One-off steps.** `mkinitcpio`, `chsh`, creating a swap file. Each is either declared
|
||||
state the host lacks a shape for, or a verb a seat should serve. An action in a declaration
|
||||
is refused over the link, and rightly.
|
||||
4. **Settings leak** ([issue 168](../../04-ISSUES/168-a-setting-reaches-every-file-and-contribution/00-report.md)):
|
||||
a setting reaches every mergeable file and every contribution of its module. Ninety theme
|
||||
knobs on that mechanism would reach ninety files. The proposed settings record says a setting
|
||||
names the file it lands in; that has to ship first.
|
||||
5. **Where tools run.** Every module that serves a tool today does so from its own container
|
||||
per node. See [03](03-one-tool-executor-per-node.md).
|
||||
6. **A seat's verbs are undecided for every seat but three.** To-be 33 leaves which verbs each
|
||||
seat serves as *a decision per seat, slowly*. The environment adds a dozen seats.
|
||||
7. **Catalogue placement.** The media chain left this catalogue for its own; whether the
|
||||
environment does the same, and whether a third-party organisation's tooling belongs in a
|
||||
public catalogue, are unasked.
|
||||
@@ -0,0 +1,88 @@
|
||||
# 03 — One tool executor per node
|
||||
|
||||
*The direction the operator set on 2026-10-02, the evidence it rests on, and what it supersedes.
|
||||
A direction, not yet a decision: the record is written when this effort graduates.*
|
||||
|
||||
## Where tools are served today
|
||||
|
||||
[To-be 33](../../03-DESIGN/01-to-be/33-the-tools-the-mesh-answers.md) names three families — a
|
||||
role's tools on the seat, a module's own tools on the module, the mesh's own verbs on the
|
||||
controller seat — and [ADR 0160](../../02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md)
|
||||
says a runtime serves the subjects its membership issues. What *runs* that runtime is
|
||||
[ADR 0150](../../02-DECISIONS/0150-a-modules-own-code-runs-as-supervised-processes-under-one-account.md):
|
||||
one supervised process per module, under the module's own account, carrying that module's
|
||||
compiled tools. Measured on the live mesh:
|
||||
|
||||
| who answers | how it runs | count |
|
||||
|---|---|---|
|
||||
| the mesh's own verbs | the controller binary, on its node | 17 verbs |
|
||||
| the store seat | the store's own runtime | 2 verbs |
|
||||
| the packet-filter seat | **a container per node**, built on the tool-runtime base image, with the network namespace and `NET_ADMIN`, on all four nodes | 3 verbs and 1 own tool |
|
||||
| every module's own tools | the module's container, one per node it runs on | 67 tools across the catalogue |
|
||||
| the console | a container per node, loopback MCP, `invokes: *` | serves none, calls all |
|
||||
| the host | — | serves nothing; answers no question about the machine |
|
||||
|
||||
**The packet-filter holder is the case to look at.** The module is a package, three files and a
|
||||
system service. To serve three verbs it also declares a built image and a container on every
|
||||
node whose only job is to answer them. Scaled to the environment — a shell, a prompt, a launcher,
|
||||
a notifier, a compositor, a login manager, a service manager, a boot loader, a downloads folder —
|
||||
that is one container per module per node for software that is itself not a container, and the
|
||||
operator's judgement is that tools should not run inside a container at all.
|
||||
|
||||
## The direction
|
||||
|
||||
**One tool executor per node, on the host side.** A process the host supervises, the way the
|
||||
launcher supervises the host ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)): not a
|
||||
container, one bus credential for the node, module-agnostic. It loads the tool code of every
|
||||
module assigned to the node and serves each module's tools and each held seat's verbs on the
|
||||
subjects the membership issues — nothing changes in what [ADR 0159](../../02-DECISIONS/0159-a-tool-call-names-the-machine-and-a-holder-serves-its-seats-verbs.md)
|
||||
and [ADR 0160](../../02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md)
|
||||
say about subjects, grants and memberships; what changes is that one process subscribes for the
|
||||
node instead of one per module.
|
||||
|
||||
- **A module brings its tools as a built artifact**, a bundle the pipeline produces, never an
|
||||
image. The executor knows bundles and subjects; it knows nothing of zsh or nftables.
|
||||
- **A tool is code the module wrote**, one function behind an MCP verb. `execute` on the shell
|
||||
seat is a function with a string argument. The executor does not declare, template or
|
||||
interpret tools; it runs them.
|
||||
- **Root is the module's concern.** A tool that must change the packet filter or rebuild boot
|
||||
images escalates itself. The executor does not run as root for everyone, and the caller does
|
||||
not know.
|
||||
- **Any node may call any tool on any node.** The executor's credential may call everything,
|
||||
as the console's already does; per-module grants on the calling side are not kept.
|
||||
- **The mesh's own verbs stay with the controller** ([ADR 0154](../../02-DECISIONS/0154-the-meshs-own-verbs-are-the-controller-seats-tools.md)),
|
||||
and a mesh-scoped seat's verbs run on the node that holds it
|
||||
([ADR 0121](../../02-DECISIONS/0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md)).
|
||||
No hub is added; the controller's node is already one.
|
||||
|
||||
**The console is the executor, renamed.** It already runs on every node with a credential that
|
||||
may call everything, and it already serves the mesh's tools to whoever is on the machine over
|
||||
MCP on loopback ([ADR 0152](../../02-DECISIONS/0152-the-operators-surface-is-a-module-the-console.md)).
|
||||
It moves out of its container into the host's process tree, gains the serving half, and takes a
|
||||
name that says what it is — *the node's tool runtime* or simply *node tools*; "console" names
|
||||
the operator's half only.
|
||||
|
||||
## What it supersedes, and what it keeps
|
||||
|
||||
| record | effect |
|
||||
|---|---|
|
||||
| [ADR 0047](../../02-DECISIONS/0047-a-module-runs-its-code-as-its-own-process-with-its-own-account.md), [ADR 0150](../../02-DECISIONS/0150-a-modules-own-code-runs-as-supervised-processes-under-one-account.md) | superseded *for tools*: one process per node runs every module's tool code, under one account. A module's long-running service — a daemon, a container — is untouched; the executor runs tools, not services. The record must say why one account for every module's tools is acceptable: every tool may be called from every node anyway, and root is taken by the tool, not granted to the process |
|
||||
| [ADR 0152](../../02-DECISIONS/0152-the-operators-surface-is-a-module-the-console.md) | kept in substance — a module, assigned per node, loopback MCP, the machine's login is the authority — changed in form: host-side, not a container; serves as well as calls; renamed |
|
||||
| [to-be 33](../../03-DESIGN/01-to-be/33-the-tools-the-mesh-answers.md) §6, [to-be 34](../../03-DESIGN/01-to-be/34-the-console.md) | amended the same way |
|
||||
| [ADR 0170](../../02-DECISIONS/0170-the-firewall-seat-serves-its-verbs.md) §3, *a container may ask for a capability* | moot for that holder: the verbs run on the host side and escalate as they need |
|
||||
| the container-runtime seat, proposed in an open change: *the holder runs as a supervised process and serves the verbs locally to the host and on the bus* | consistent — a supervised process serving verbs is what the executor is; the open question is whether that holder keeps its own process or serves through the executor like everyone else |
|
||||
| the tool-runtime base image | no longer the way tools reach a node; may remain the way a module's *service* is built |
|
||||
|
||||
## What stays open
|
||||
|
||||
- **The executor's language.** The host is a static Go binary and loads no plugins, so the
|
||||
executor is a sibling process, and its language decides the language of every tool bundle.
|
||||
One decision, taken once.
|
||||
- **How a bundle reaches the node.** An artifact of the module's build, delivered as the host
|
||||
delivers everything else; whether it is a file resource in the declaration or a thing the
|
||||
executor fetches by digest.
|
||||
- **Reload.** A push that adds or upgrades a module's bundle reaches a running executor as a
|
||||
reload, not a restart, or every tool on the node blinks on every push.
|
||||
- **The host's own questions.** [Issue 160](../../04-ISSUES/160-a-machine-says-little-about-itself-and-only-when-asked/00-report.md)
|
||||
wants a machine to say more about itself. With an executor on every node, "what is this
|
||||
machine made of" is a seat verb like any other, served there.
|
||||
@@ -0,0 +1,65 @@
|
||||
# 04 — The seats of the environment
|
||||
|
||||
*Candidates, not decisions. To-be 33 says which verbs a seat serves is a decision per seat,
|
||||
taken slowly, because a seat's tools bind every future holder. This document lists the roles the
|
||||
operator's machine has once, who could hold each, what gates it, and a first verb or two — so
|
||||
each record has a starting point.*
|
||||
|
||||
## The rule for what is a seat here
|
||||
|
||||
A role the machine fills **at most once** is a node-scoped seat, declared by the module family
|
||||
that fills it ([ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md)). A thing
|
||||
several of which coexist without contention — editors, browsers, media players — is not a seat;
|
||||
each is a module with its own tools, and nothing is singular about it. A seat is held by one
|
||||
assignment per node; other modules of the same family may be installed beside it without
|
||||
holding it ([ADR 0040](../../02-DECISIONS/0040-what-a-module-is.md) §1, read with the sharper
|
||||
distinction: *installed* is not *holding*).
|
||||
|
||||
## Candidate seats
|
||||
|
||||
| seat | holders | gated by | first verbs |
|
||||
|---|---|---|---|
|
||||
| **login shell** | zsh, fish, bash | nothing: universal | `execute(command)`; `show-config`; the holding itself sets the account's login shell through the host's `user` shape |
|
||||
| **service manager** | systemd | the `service-manager` capability the profile reports | units: list, status, start, stop, restart, enable, journal; **user scope** on each |
|
||||
| **boot** | grub, systemd-boot | a machine that boots itself (not a container host) | `rebuild-images`; `entries` |
|
||||
| **package manager** | pacman, apt | the `package-manager` capability | search, installed, upgrade, orphans; today a capability the host uses, not a seat anyone holds |
|
||||
| **display server** | xorg, wayland compositors that are their own server | the `graphical-session` capability | `displays`; `layout` |
|
||||
| **display session** | i3, sway | the display server seat held on the node; i3 needs x11, sway needs wayland | `reload`; `workspaces`; `windows`; `move` |
|
||||
| **terminal emulator** | xterm, alacritty, foot | display session | `open`; `font` |
|
||||
| **launcher** | rofi, dmenu | display session | `show`; `theme` |
|
||||
| **notifier** | dunst, mako | display session | `send`; `history`; `rule` |
|
||||
| **compositor** | picom | display server (x11 only) | `restart`; `effects` |
|
||||
| **lock screen** | i3lock, swaylock | display session | `lock` |
|
||||
| **bar** | i3status-rust, waybar | display session | `reload`; `blocks` |
|
||||
| **login manager** | lemurs, greetd | graphical session | `sessions`; `default-session` |
|
||||
| **audio** | pipewire, pulseaudio | the machine reports a sound device | `sinks`, `sources`, `default`, `volume`, `mute` |
|
||||
| **clipboard** | greenclip, cliphist | display session | `history`; `clear` |
|
||||
|
||||
Not seats, modules with their own tools: the editor, the browser, the mail client, the file
|
||||
manager, the media player, the chat client, the agent at the terminal, the downloads folder, the
|
||||
scripts folder, the sync client, the power and thermal daemons that are specific to one machine's
|
||||
hardware.
|
||||
|
||||
## What the table implies
|
||||
|
||||
**Capabilities come first.** `graphical-session`, `service-manager` and `package-manager` are
|
||||
reported today. *A display server is held* is not a capability but a seat being held, and a
|
||||
module that needs it declares a dependency on the seat, not on a capability: *i3 needs the
|
||||
display server seat held by xorg*. Whether a held seat can gate another's assignment is a
|
||||
question for the controller's resolver, and the first environment module after the shell will
|
||||
ask it.
|
||||
|
||||
**The service manager comes early.** Four of the predecessor's modules ship user units, and the
|
||||
executor itself is a unit. User scope on the host's `service` shape is a host change whichever
|
||||
module holds the seat; the seat's holder answers the questions about units, it does not apply
|
||||
them — the host does, as it does for every declared resource.
|
||||
|
||||
**The shell comes first.** Universal, no capability, one verb that is immediately useful on
|
||||
every node, and the `user` shape already makes the login shell declared state. It is the module
|
||||
that proves the pattern: a package, files under the home owned by the account, a seat claim,
|
||||
tools served by the executor, settings for the few things that vary per node, and a kept region
|
||||
for the operator's own lines.
|
||||
|
||||
**The login manager is the first system-scope one**, because it needs nothing new: a package,
|
||||
two files under `/etc`, a service — the same shape the ssh daemon module has today — and the
|
||||
session script it owns is the file that was hand-fixed the day this effort opened.
|
||||
@@ -0,0 +1,55 @@
|
||||
---
|
||||
status: active
|
||||
initiated: 2026-10-02
|
||||
touches: [lab, the lab module, the catalogue, assignments, settings, the controller's store]
|
||||
became: []
|
||||
---
|
||||
|
||||
# 019 — A warm twin of the running mesh
|
||||
|
||||
## What is being investigated
|
||||
|
||||
Whether the lab can keep a **warm twin of the mesh as it actually runs**: the same machines, carrying
|
||||
the same catalogue, the same assignments and the same settings as the live mesh, raised once and kept
|
||||
ready, so that a change can be tested against the mesh as it is rather than against a scenario
|
||||
written to resemble it. A run against the twin would go through the lab module like any other run:
|
||||
a branch per repository, the twin restored from its snapshot, the change applied, the beds run.
|
||||
|
||||
## Why
|
||||
|
||||
The lab's beds raise meshes from declarations written for the bed. They prove the mechanism. They
|
||||
do not prove that a change works on the mesh that runs, with its accumulated assignments, its
|
||||
operator settings, its adopted machines and its modules in their real combinations. The gap showed
|
||||
on 2026-10-02:
|
||||
|
||||
- a change to how a module's settings reach its files was correct in every bed, and would have put a
|
||||
setting into the container runtime's configuration on every machine running that module. Only the
|
||||
composed plan for a real machine showed it;
|
||||
- a firewall change composed cleanly and still left one machine's wired port unfiltered, because
|
||||
of a link that machine had and no bed did;
|
||||
- a recovery step was needed on every machine at once, after a change that every bed had passed.
|
||||
|
||||
The lab already has a warm mode, a snapshot of a raised scenario restored between attempts. What it
|
||||
does not have is a scenario that **is** the running mesh, kept current with it.
|
||||
|
||||
## What it touches
|
||||
|
||||
- **What a twin is made of.** The catalogue and the assignments are records; settings are records;
|
||||
secrets are sealed to machines and cannot be copied. Which of these can be carried to the lab as
|
||||
they are, which must be substituted, and how a twin says what it substituted.
|
||||
- **Data.** A twin with the real catalogue and no real data proves composition and delivery, not a
|
||||
migration. Whether a twin carries data, a sample of it, or none.
|
||||
- **Keeping it current.** A twin raised once goes stale with the first merge. Whether it is
|
||||
re-derived from the live records on each run, refreshed on a schedule, or rebuilt only when asked.
|
||||
- **Machines.** The live mesh has machines of different kinds: a server on the internet, machines
|
||||
behind a home router, a laptop that sleeps. Which of their properties a twin must reproduce for a
|
||||
test to mean anything (reachability, the private network, the found firewall).
|
||||
- **Cost.** The lab machine's memory and disk, and how long a twin takes to raise from cold.
|
||||
- **The lab module's tools.** A run against the twin rather than a named bed: one more tool, or an
|
||||
argument to the run tool.
|
||||
|
||||
## Starting point
|
||||
|
||||
The lab module (ADR 0172) runs beds through the mesh, and the lab's warm mode already snapshots and
|
||||
restores a raised scenario. The beds that raise a machine shaped like one live machine from the
|
||||
catalogue are the nearest existing thing, and the first to compare against.
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
status: graduated
|
||||
initiated: 2026-10-03
|
||||
touches: [the tool runtime, the catalogue's tool bundles, the controller's declaration composer, settings, own secrets, 03-DESIGN/01-to-be/38-building-the-operators-machine.md]
|
||||
became: [02-DECISIONS/0192-a-tools-bundle-declares-what-it-is-given-and-the-runtime-hands-it-to-that-bundle-alone.md, 03-DESIGN/01-to-be/38-building-the-operators-machine.md]
|
||||
---
|
||||
|
||||
# 020 — What a bundled tool is given
|
||||
|
||||
## What is being investigated
|
||||
|
||||
How a module's tools, once they are a bundle the node's runtime loads
|
||||
([ADR 0175](../../02-DECISIONS/0175-one-tool-runtime-per-node-serves-every-modules-tools-on-the-host-side.md),
|
||||
[ADR 0188](../../02-DECISIONS/0188-a-modules-own-code-is-bundles-in-any-language-and-a-tools-bundle-speaks-mcp-to-the-runtime.md)),
|
||||
learn the things their container used to be handed: where the module's configuration file is, where
|
||||
its token or password is, which port the service listens on, where a provision's address is written.
|
||||
A container is given these as an environment and mounts, composed by the mesh per module per machine.
|
||||
A bundle has no environment of its own: the runtime's process carries four words for every bundle it
|
||||
loads, and nothing per module ([design 38](../../03-DESIGN/01-to-be/38-building-the-operators-machine.md) WP4).
|
||||
|
||||
## Why
|
||||
|
||||
Two holders moved on 2026-10-03 — the packet filter and the intrusion prevention — and both could,
|
||||
because neither needs anything but a fixed path and root. Of the thirty-three modules whose tools
|
||||
still run as containers on the runtime's image, thirty-one are not like that: their environment
|
||||
names a configuration file, a credential file, a service address, a grants directory. Moving them
|
||||
one by one without a rule for this would give the mesh thirty-one answers to one question. The
|
||||
measurement and the options are in [01](01-what-the-containers-are-given.md).
|
||||
|
||||
## What it touches
|
||||
|
||||
The runtime (which hands a bundle what it is given), the composer (which resolves `${dir:…}` and
|
||||
`${port:…}` for a container today and would for a bundle), the manifest (where a bundle would say
|
||||
what it needs), and design 38, which records the gap and must say the rule once there is one.
|
||||
@@ -0,0 +1,86 @@
|
||||
# What the tool containers are given, measured
|
||||
|
||||
Counted 2026-10-03 in the catalogue, after the two holders moved.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| modules whose tools still run as a container on the runtime's image | 33 |
|
||||
| tool containers among them (two modules run two) | 36 |
|
||||
| modules whose container's environment carries only the bus credential | 1 (the intrusion prevention, now moved) |
|
||||
| modules whose container's environment carries more | 32 — 31 still containers |
|
||||
|
||||
## What "more" is
|
||||
|
||||
Every value a container is given is one of five shapes. The reference kinds the composer resolves
|
||||
in those values, over the 36 containers: a module directory (`${dir:…}`) in all 36, a mesh-chosen
|
||||
port (`${port:…}`) in 12, a seat and an access grant once each.
|
||||
|
||||
1. **A file the mesh already places on the host, mounted in.** The module's configuration as
|
||||
JSON (`…_CONFIG_FILE`), its own secret (`…_TOKEN_FILE`, `…_PASSWORD_FILE`, `MESH_BROKER_FILE`),
|
||||
a provision's address and secret written for it. Every one is a path under one of the module's
|
||||
directories — its mesh state, its state, its grants, what it has written — mounted at a path of
|
||||
the container's choosing and named to the tool through the environment. **The file is on the
|
||||
host already; only the name under which the tool finds it is the container's.**
|
||||
2. **The service's address, with the port the mesh chose:** `http://127.0.0.1:${port:3000}`. The
|
||||
port is the composer's; the rest is the manifest's constant.
|
||||
3. **A provision's address as a constant string** (a database's URL on the module's own network
|
||||
name), paired with a mounted secret file from shape 1.
|
||||
4. **A directory of grants** (`MESH_RECEIVES`): shape 1 again, a directory rather than a file.
|
||||
5. **Literals the image needs:** a time zone, a user id, a memory limit. These belong to the
|
||||
service's container where one exists; a tool bundle needs none of them.
|
||||
|
||||
So the whole of what a bundled tool needs is: the paths of its module's directories on this
|
||||
machine, the ports the mesh chose for its module here, and the constants its own manifest wrote.
|
||||
Nothing a container had that a bundle cannot have; the mesh composes all three for the container
|
||||
today, per module per machine.
|
||||
|
||||
## What the runtime already has for it
|
||||
|
||||
- The SDK's tool contributor is `(env) => tools`, and `collectTools(env)` takes the environment to
|
||||
hand each contributor. The runtime calls it without one, so every contributor reads the process's
|
||||
— the four words. The hook for a per-module environment exists and is unused.
|
||||
- A launched bundle ([ADR 0188](../../02-DECISIONS/0188-a-modules-own-code-is-bundles-in-any-language-and-a-tools-bundle-speaks-mcp-to-the-runtime.md))
|
||||
is spawned with the runtime's environment; the launch takes an environment argument.
|
||||
- The composer resolves `${dir:…}` and `${port:…}` for a container's `env` and `volumes`; the same
|
||||
resolution over a bundle's declaration is the same code.
|
||||
|
||||
## Options
|
||||
|
||||
**A. The bundle declares its environment on its artifact, and the mesh composes it as a
|
||||
container's.** The manifest's tools artifact gains `env`, resolved with the same references;
|
||||
values that were mount targets become the host-side paths directly (`${dir:mesh-state}/config.json`
|
||||
rather than `/run/config/config.json`). The controller composes one environment per bundle per
|
||||
machine into the runtime's declaration; the runtime hands it to the bundle's contributor and to a
|
||||
launched child, and to nothing else. *For:* the tool code does not change — it reads the same
|
||||
names; the conversion of the thirty-one is a mechanical move of the container's `env` with the
|
||||
mounts folded in; one rule, one place. *Against:* the runtime's process carries thirty-one
|
||||
environments in its declaration, and a bundle's environment is visible to the other bundles in the
|
||||
process unless the runtime keeps them apart, which it must — a tool that reads `process.env`
|
||||
instead of the environment it was handed would see its neighbours' paths.
|
||||
|
||||
**B. The runtime derives the environment from the module's placed manifest.** No new field: the
|
||||
runtime reads, for each module it serves, where that module's directories and ports are, and hands
|
||||
a conventional set of words. *For:* nothing to declare. *Against:* a convention the tool code must
|
||||
be rewritten to, thirty-one times; the runtime learns the composer's job; a module that names its
|
||||
file `config.json` and one that names it `settings.json` need different words anyway.
|
||||
|
||||
**C. Tools read their module's files through the bus** — ask the controller. *Against:* a tool
|
||||
that cannot start without the bus answering a question is a tool that fails in the one case the
|
||||
tools exist for, and a secret crossing the bus to reach a file already on the machine is a
|
||||
disclosure for nothing.
|
||||
|
||||
A is the one that keeps the tool code and the composer's vocabulary as they are, and names the one
|
||||
thing the runtime must add: an environment per bundle, kept apart. The thing to decide beside it:
|
||||
whether a bundle's environment may name a secret file at all, or whether secrets stay mounts in
|
||||
spirit — a path the tool reads, never a value in the environment — which is what every container
|
||||
does today and what A keeps if the rule says *paths, not values*.
|
||||
|
||||
## What a decision would have to say
|
||||
|
||||
- Where a bundle says what it is given (the artifact, option A), and that values are paths and
|
||||
constants, never a secret's content.
|
||||
- That the composer resolves it with the references it already has, per module per machine.
|
||||
- That the runtime hands each bundle its own environment and nothing of another's, and how that is
|
||||
checked: a test loading two bundles whose environments differ and asserting each sees only its own.
|
||||
- That the thirty-one move in one mechanical change after the rule lands, each proven by its tools
|
||||
answering from the runtime, and the registration gate then refuses the container shape for all.
|
||||
@@ -0,0 +1,48 @@
|
||||
---
|
||||
status: graduated
|
||||
initiated: 2026-10-03
|
||||
touches: [the console, 03-DESIGN/01-to-be/34-the-console.md, the tool runtime, seats, assignments]
|
||||
became: [02-DECISIONS/0195-the-meshs-tools-are-found-by-address-not-announced-whole.md, 03-DESIGN/01-to-be/34-the-console.md]
|
||||
---
|
||||
|
||||
# 021 — Finding a tool in the mesh
|
||||
|
||||
## What was investigated
|
||||
|
||||
How an agent finds the one tool it needs among everything the mesh answers, and how a call names
|
||||
exactly what it asks — a role the mesh holds once, a role every machine holds, or one assignment of a
|
||||
module on one machine — rather than receiving the whole catalogue and a name that can mean several
|
||||
things.
|
||||
|
||||
## Why
|
||||
|
||||
The operator's observation on 2026-10-03: *Claude should not see all tools at once; they should be
|
||||
discoverable — and `postgres.list_databases` is wrong, asking one machine's postgres is not asking
|
||||
another's.* Measured the same day from the console's own answer:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| tools announced to every session at its start | 228, in 110 KB |
|
||||
| names (module or seat prefixes) | 43 |
|
||||
| node seats' verbs, which require `node` | 22 |
|
||||
| modules with tools on more than one machine | 4 — fail2ban, nftables (every machine), postgres, mssql (two each) |
|
||||
| modules reported "not answering", most with no tools and several retired | 47 |
|
||||
|
||||
The two stateful modules on two machines are listed **once**, with `node` optional and *whichever
|
||||
answers* when it is left out — though their two instances hold different databases. Design 34 §3 says
|
||||
such a module is listed once per machine; the live console does not do that. The list is taken once
|
||||
per session, so a tool that arrives later is invisible until the client reconnects. And only Claude
|
||||
Code's own deferral of long tool lists keeps the 228 from the model's context; another MCP client
|
||||
would receive them whole.
|
||||
|
||||
## Options
|
||||
|
||||
1. **Keep the flat list; rely on the client to defer it.** Rejected: a property of one client, and it
|
||||
leaves the ambiguity and the stale list.
|
||||
2. **One flat tool per assignment** (`ace_postgres_list_databases`). Removes the ambiguity, multiplies
|
||||
the list, and runs into the API's tool-name limit (letters, digits, `_`, `-`, 64 characters).
|
||||
3. **A small fixed set of tools that walk the mesh's own structure**, with the full address as an
|
||||
argument: the mesh's seats; a machine's node seats and assignments; a search; a description; a
|
||||
call. Chosen — see [ADR 0195](../../02-DECISIONS/0195-the-meshs-tools-are-found-by-address-not-announced-whole.md).
|
||||
4. **MCP resources or prompts for discovery.** Clients support them unevenly, and an agent acts
|
||||
through tools; a resource it cannot be relied on to read is not a discovery path.
|
||||
@@ -0,0 +1,41 @@
|
||||
---
|
||||
status: graduated
|
||||
initiated: 2026-10-03
|
||||
touches: [the tool runtime, the per-module containers, the SDK, the bus grants, 03-DESIGN/01-to-be/38-building-the-operators-machine.md]
|
||||
became: [02-DECISIONS/0198-a-modules-long-running-code-is-launched-by-the-node-runtime-and-reaches-the-bus-through-it.md, 03-DESIGN/01-to-be/38-building-the-operators-machine.md]
|
||||
---
|
||||
|
||||
# 022 — Where a module's long-running code runs
|
||||
|
||||
## What was investigated
|
||||
|
||||
Twenty-three modules still run their own code in a container built on the runtime's image. Their tools
|
||||
can move as bundles ([ADR 0192](../../02-DECISIONS/0192-a-tools-bundle-declares-what-it-is-given-and-the-runtime-hands-it-to-that-bundle-alone.md),
|
||||
[ADR 0193](../../02-DECISIONS/0193-every-bundle-the-runtime-serves-is-launched-and-the-runtime-knows-no-language.md));
|
||||
the rest of what those containers run cannot yet. This asks where that code goes and how it reaches
|
||||
what its container handed it.
|
||||
|
||||
## What that code is, measured 2026-10-03
|
||||
|
||||
| | modules |
|
||||
|---|---|
|
||||
| subscribes to events on the bus | audit-logger (everything), mesh-catalog (two seat events), mesh-vault, records (`gitea.pull.merged`), and postgres, mongodb, mssql, redis, mosquitto logging their own lifecycle |
|
||||
| provisioners: read the grants the mesh delivered as files, act on the backend, emit | 12 |
|
||||
| a run-once preparation step | mesh-catalog |
|
||||
| a command-line client of the backend | psql, mosquitto_ctrl, git (packages on every machine's system); mongosh, sqlcmd (not in its repositories) |
|
||||
| a service reached by a container name | icecast, mailu-admin, minio, mongodb-server, mssql |
|
||||
| a main of its own | anthropic-consumer, openai-consumer, route-adapter |
|
||||
|
||||
A provisioner needs nothing a launched bundle lacks: files named by its words, its backend, and an emit
|
||||
that already travels through the runtime. The one thing missing is **a subscription** — events
|
||||
delivered to the module's code, acknowledged when it has handled them.
|
||||
|
||||
## Options
|
||||
|
||||
1. **The runtime launches it and is its bus**: the stdio channel gains a subscription; the runtime
|
||||
binds the module's durable consumer and delivers each event to the child, acknowledging when the
|
||||
child answers. One bus connection per machine; any language. Chosen.
|
||||
2. **A process per module with its own bus client and credential.** Every language's SDK would carry
|
||||
a transport and every module a credential on disk — what ADR 0188 rejected for tools, for the same
|
||||
reasons.
|
||||
3. **Keep the containers for this code.** Leaves ADR 0188's rule broken for 23 modules indefinitely.
|
||||
@@ -0,0 +1,37 @@
|
||||
---
|
||||
status: active
|
||||
initiated: 2026-10-03
|
||||
touches: [the seats, the seat protocol, the controller's ownership check, 03-DESIGN/01-to-be/26-the-seats.md]
|
||||
---
|
||||
|
||||
# 023 — A seat protocol that defines what its holder owns
|
||||
|
||||
## What is investigated
|
||||
|
||||
**A seat is a definition — a protocol — and a module occupies it by implementing that protocol.**
|
||||
Today the protocol is what the holder accepts, emits and serves (ADR 0118, 0129, 0132): its verbs, as MCP
|
||||
tool definitions. This asks whether the protocol should also name the **files and directories the
|
||||
holder owns**, so that occupying the seat means owning them: `node-resolver-config` owns
|
||||
`/etc/resolv.conf`, `node-hosts-file` owns `/etc/hosts`, the intrusion prevention owns its jail file.
|
||||
|
||||
The direction is the protocol's, not the holder's: the seat states what any holder must own; a module
|
||||
that wants the seat must declare those paths among its resources, or the controller refuses the claim
|
||||
as not implementing the seat. Two seats may not name one path.
|
||||
|
||||
## Why
|
||||
|
||||
Who owns a singular file is today answered by reading every manifest, and enforced only after the fact,
|
||||
when two modules on one machine both declare the same path. The question *which module owns
|
||||
`/etc/resolv.conf`?* came up on 2026-10-03 with no place to look it up. A seat that names the path answers
|
||||
it from the seat table, before any module is written, and makes "implements the seat" checkable.
|
||||
|
||||
## What it touches
|
||||
|
||||
- The seat definition and its table (ADR 0122) — a new part of the protocol.
|
||||
- The controller's ownership check (`checkResources`), which already refuses two modules owning one path.
|
||||
- Every node seat that is really about a file: `node-resolver-config`, `node-hosts-file`
|
||||
([ADR 0199](../../02-DECISIONS/0199-a-module-that-answers-names-declares-its-zone-and-a-nodes-hosts-file-is-one-modules.md)),
|
||||
`node-intrusion-prevention`, `node-packet-filter`.
|
||||
|
||||
Raised by the operator during the resolver work of ADRs 0194–0199 and parked there so that work was not
|
||||
widened by it.
|
||||
@@ -0,0 +1,152 @@
|
||||
---
|
||||
status: graduated
|
||||
initiated: 2026-10-04
|
||||
touches: [the bus, what a module declares, the tool runtime, the SDK, the bus grants, 03-DESIGN/01-to-be/25-the-bus-on-nats.md, 03-DESIGN/01-to-be/32-what-a-module-declares.md]
|
||||
became: [02-DECISIONS/0201-a-module-keeps-its-current-state-in-key-value-buckets-it-declares-and-reaches-through-the-runtime.md, 03-DESIGN/01-to-be/32-what-a-module-declares.md, 03-DESIGN/01-to-be/25-the-bus-on-nats.md]
|
||||
---
|
||||
|
||||
# 024 — State a module keeps on the bus
|
||||
|
||||
## What is investigated
|
||||
|
||||
A place on the bus where a module's own code keeps **current state** — not history — that every
|
||||
machine sees, including a machine that joins after the state was written: put, get, delete, list and
|
||||
watch, reached through the node's runtime the way a bundle already publishes, asks and subscribes
|
||||
([ADR 0198](../../02-DECISIONS/0198-a-modules-long-running-code-is-launched-by-the-node-runtime-and-reaches-the-bus-through-it.md)).
|
||||
On NATS that is a key-value bucket. The questions are what a module declares, who creates the
|
||||
bucket, what the grants are, what the runtime's verbs are, and what may never be stored.
|
||||
|
||||
## Why
|
||||
|
||||
The mesh carries two kinds of module traffic and a third is missing.
|
||||
|
||||
- **Events** land in the EVENTS stream: limits retention, seven days, ten thousand messages per
|
||||
subject, a durable consumer per consuming module that replays what it missed. Never a secret
|
||||
([design 32](../../03-DESIGN/01-to-be/32-what-a-module-declares.md) §10).
|
||||
- **Requests** are core request/reply — tool calls, a bundle's `mesh/ask` — and are kept nowhere.
|
||||
|
||||
Neither is *the current value of something*. Two cases from the first module that needs it, the
|
||||
operator's agent on a machine ([design 36](../../03-DESIGN/01-to-be/36-the-operators-agent-on-a-machine.md)):
|
||||
|
||||
1. **An MCP server registered for every machine.** Registering emits an event every machine's copy
|
||||
of the module consumes. A machine the module is assigned to *after* the registration has no
|
||||
durable consumer yet — the consumer is created at assignment — so it never hears of it. Wanted
|
||||
instead: one entry per server, for every machine or for one; every machine reads the whole current
|
||||
set when it starts and watches for changes; unregistering is a delete; any machine can list it.
|
||||
2. **Which licence a machine is bound to** ([design 39](../../03-DESIGN/01-to-be/39-the-anthropic-licence-manager.md)).
|
||||
As events, a machine that was off for a day replays every rotation since and asks for a token
|
||||
after each. It needs only the latest binding and its generation. The token itself stays on
|
||||
request/reply and is never stored.
|
||||
|
||||
The design already expects this. [Design 25](../../03-DESIGN/01-to-be/25-the-bus-on-nats.md) §1:
|
||||
"conditions and observed state in key-value buckets that anything may watch".
|
||||
[Research 017](../017-a-mesh-that-heals-itself/01-the-intended-behaviour.md) wants a provisioner's
|
||||
"what I applied" and a rotation's step kept in one rather than in memory. Nothing implements it.
|
||||
|
||||
## What exists, measured 2026-10-04
|
||||
|
||||
| | fact | where |
|
||||
|---|---|---|
|
||||
| streams | five kinds of mesh stream: CONTROL (work queue), NODES and ASSIGNMENTS (last per subject), EVENTS (limits: 7 days, 10 000 per subject), one work queue per seat that accepts | the controller's broker streams |
|
||||
| the state relationship | [design 32](../../03-DESIGN/01-to-be/32-what-a-module-declares.md) §4 already names *state* — 1:1, last per subject — and says it is "declared: the mesh's own". Two streams use it, both written by the controller. No module can declare it | design 32, the controller |
|
||||
| key-value buckets | none, anywhere | all four code repositories |
|
||||
| the runtime's bus verbs | `mesh/publish`, `mesh/ask`, `mesh/subscribe`; delivery back to the bundle is `mesh/event` | the runtime's launcher |
|
||||
| the runtime's principal | one bus user per machine carries every assigned module; its grant is the union of theirs. That one module's code does not act as another is the runtime's to keep: it publishes under the module's own name by construction | the controller's grant composition, the runtime's bus |
|
||||
| what a bundle is issued | a membership per assignment, last per subject, read directly by the runtime: where it serves, where it emits, what it reaches | ADR 0160 |
|
||||
| who creates bus objects | the controller only — mesh streams on every raise, a seat's stream at registration, a module's consumer at assignment. No module reaches the JetStream API | design 25 §3 |
|
||||
|
||||
### What a key-value bucket needs from a grant, against a real server
|
||||
|
||||
Measured against nats-server 2.10 with the Go client the runtime already uses, a bucket created by
|
||||
an unrestricted user and used by two users holding only the subjects below (`B` is the bucket):
|
||||
|
||||
| operation | subject published | writer | reader |
|
||||
|---|---|---|---|
|
||||
| bind to the bucket | `$JS.API.STREAM.INFO.KV_B` | yes | yes |
|
||||
| get | `$JS.API.DIRECT.GET.KV_B.>` | yes | yes |
|
||||
| put, delete | `$KV.B.>` | yes | **refused** |
|
||||
| list keys, watch | `$JS.API.CONSUMER.CREATE.KV_B.>` — an ordered, ephemeral consumer | yes | yes |
|
||||
| stop a watch cleanly | `$JS.API.CONSUMER.DELETE.KV_B.>` | yes | yes |
|
||||
| answers | its own inbox, which every principal already subscribes | — | — |
|
||||
|
||||
*Checked again once built, 2026-10-04:* the grants the controller composes for two machines' runtimes —
|
||||
one carrying the owner, one only a reader — were loaded into a server as composed, and each operation
|
||||
was run as each runtime's user. The owner's did all of them; the reader's read, listed and watched,
|
||||
and its put and delete were refused by the server.
|
||||
|
||||
Three things the measurement showed that reading the documentation would not have:
|
||||
|
||||
1. **A refused put is not an error to the caller; it is a timeout.** The server reports the
|
||||
permission violation asynchronously, on the connection, and the client waits out its deadline
|
||||
for an acknowledgement that never comes. So a runtime that relies on the grant alone tells a
|
||||
bundle "timed out" for "you may not write this" — it must refuse first, from what the module was
|
||||
issued, with the reason.
|
||||
2. **A watch's current values include deletions.** A key deleted earlier arrives among the initial
|
||||
values as a delete marker, before the end-of-current marker. A bundle asking "what is there now"
|
||||
must not be handed those.
|
||||
3. **Without the consumer-delete grant, stopping a watch hangs** until its deadline, and the
|
||||
ephemeral consumer lingers on the server until it times out by itself.
|
||||
|
||||
### Whether the events shape is enough instead
|
||||
|
||||
Honestly compared, because a new primitive is a cost:
|
||||
|
||||
- **EVENTS cannot be made last-per-subject for some subjects.** Retention is per stream, and
|
||||
JetStream refuses a second stream overlapping the first (verified and recorded in design 32 §3).
|
||||
A state subject inside `mesh.mod.*.event.>` keeps EVENTS' seven days: a licence binding unchanged
|
||||
for a week disappears.
|
||||
- **A separate last-per-subject stream per module** is possible — it is exactly what a key-value
|
||||
bucket *is* on the server: a stream with one message per subject, a rollup for purge, and direct
|
||||
reads. Building it by hand gives up the client's get, list, delete and watch, which are the
|
||||
operations both cases need, and would be the mesh writing NATS's own key-value layer again.
|
||||
- **Consumers are the wrong reader.** A durable consumer per reading module is created at
|
||||
assignment and replays from where it is; state wants "everything current, now, then changes",
|
||||
which an ordered ephemeral consumer from the last value per subject gives and a durable does not.
|
||||
|
||||
So key-value is not a convenience over events; it is the state relationship design 32 already
|
||||
names, opened to modules.
|
||||
|
||||
## Questions, and what this effort proposes
|
||||
|
||||
1. **What a manifest says.** `state` names the buckets a module owns, by local name — every
|
||||
instance of the module may write them and read them. `reads` names another module's bucket as
|
||||
`<module>.<name>`, read-only. Names only, never a bucket or subject (design 32 §1). A bucket's
|
||||
options — how many past values it keeps, how long a value lives — are the owner's to declare,
|
||||
the way a seat declares its own retention (design 32 §3).
|
||||
2. **Scope.** One bucket per module per name, mesh-wide. A key may carry a machine by the module's
|
||||
own convention (`all.<server>`, `<machine>.<server>`). A bucket per machine was considered and
|
||||
not proposed: "list every server for every machine" becomes a walk over buckets, and the grant
|
||||
could only narrow writes, which nothing asked for — every instance of the owner already writes.
|
||||
3. **Who creates the bucket.** The controller, from the catalogue, on every raise — a bucket exists
|
||||
from registration, like a seat's stream, so a reader can watch before the owner is assigned
|
||||
anywhere. Never a module.
|
||||
4. **The runtime's verbs.** `mesh/state.get`, `mesh/state.put`, `mesh/state.delete`,
|
||||
`mesh/state.keys`, `mesh/state.watch`, each naming the bucket as the module named it. A watch
|
||||
is answered once the current values are on their way, then each change is delivered to the
|
||||
bundle as a `mesh/state` request it answers — current values first (no deletions among them), an
|
||||
end-of-current marker, then changes. A child that restarts watches again, as it subscribes
|
||||
again. The runtime refuses, with the reason, a bucket the module was not issued, and a write to
|
||||
one it only reads.
|
||||
5. **Secrets.** None in a bucket, sealed or not: a bucket is a stream (design 32 §10). Sealed values
|
||||
are plain base64 and cannot be recognised, so the mechanical check is partial and said to be: the
|
||||
runtime refuses a value carrying a field whose name says it is a credential (`password`,
|
||||
`secret`, `token`, `authorization`, …), which catches the ordinary mistake and not a determined
|
||||
one. For the first consumer this has a concrete consequence: an MCP server registered with an
|
||||
authorisation header keeps that header out of the bucket.
|
||||
6. **History, lifetime, size.** One value per key unless the owner says more; no expiry unless it
|
||||
says one; a value at most 256 KiB and a bucket at most 64 MiB, the mesh's caps rather than a
|
||||
module's. **A bucket outlives its module's assignment** — what a module stored is data, and data
|
||||
outlives what declared it ([ADR 0030](../../02-DECISIONS/0030-data-outlives-the-mesh-that-declared-it.md));
|
||||
unassigning is not cleaning up. A bucket whose declaration is gone is reported, never removed.
|
||||
7. **Events or state.** State (above).
|
||||
|
||||
## The work, once decided
|
||||
|
||||
1. A decision record, then design 32 (*state* becomes a relationship a module declares) and design
|
||||
25 (key-value buckets are part of the bus) amended.
|
||||
2. The controller: the manifest's two words and their registration check; buckets asserted on every
|
||||
raise; the grants for owners' and readers' runtimes; the buckets issued in each membership.
|
||||
3. The runtime: the five verbs, the watch delivery, the refusals; tested against a real server.
|
||||
4. The SDK, TypeScript and Go: a small state surface over the verbs.
|
||||
5. Proved on a running mesh with one small module, then handed to the operator's agent, whose
|
||||
registered servers move from events to a bucket.
|
||||
@@ -0,0 +1,94 @@
|
||||
---
|
||||
status: graduated
|
||||
initiated: 2026-10-04
|
||||
touches:
|
||||
- 02-DECISIONS/0174-a-node-varies-a-module-through-settings-and-kept-regions-never-an-edit.md
|
||||
- 02-DECISIONS/0176-the-login-shell-is-a-node-seat-and-execute-is-its-contract.md
|
||||
- 02-DECISIONS/0177-a-unit-may-be-user-scoped-and-the-service-manager-is-a-node-seat.md
|
||||
- 02-DECISIONS/0126-a-module-declares-its-own-seats.md
|
||||
- 02-DECISIONS/0102-the-mesh-writes-into-a-shared-file-never-over-it.md
|
||||
- 02-DECISIONS/0182-inside-a-home-the-mesh-owns-what-it-places-and-holds-the-rest-as-found.md
|
||||
- 02-DECISIONS/0112-a-module-definition-names-no-node-mesh-or-path.md
|
||||
- 03-DESIGN/01-to-be/27-a-module-requires-the-mesh-resolves.md
|
||||
- 03-DESIGN/01-to-be/31-a-module-declares-its-fail2ban-jail.md
|
||||
- 03-DESIGN/01-to-be/37-the-operators-machine.md
|
||||
- 03-DESIGN/01-to-be/38-building-the-operators-machine.md
|
||||
- 04-ISSUES/168-a-setting-reaches-every-file-and-contribution/00-report.md
|
||||
became:
|
||||
- 02-DECISIONS/0203-the-accounts-environment-is-one-modules-and-every-module-contributes-to-it.md
|
||||
- 02-DECISIONS/0204-a-module-contributes-shell-code-to-the-login-shell-in-named-slots.md
|
||||
- 02-DECISIONS/0205-software-the-distribution-does-not-package-ships-as-a-pinned-archive-of-the-module.md
|
||||
- 03-DESIGN/01-to-be/41-the-shell-and-the-accounts-environment.md
|
||||
---
|
||||
|
||||
# 025 — How a module plugs into the operator's shell
|
||||
|
||||
## What is investigated
|
||||
|
||||
The shell module writes the mesh's part of the account's shell startup file. But the shell is not the
|
||||
only module that needs a line there. A prompt theme loads itself from it. A language version manager
|
||||
sets a variable and sources its loader. A toolchain puts its directory on `PATH`. A desktop module
|
||||
names the browser. Today all of these sit in one hand-written file, and the shell module as written
|
||||
carries some of them in its own block and loads others only "if a module placed them". Nothing says
|
||||
how they get placed.
|
||||
|
||||
This effort asks four things:
|
||||
|
||||
1. **How a module contributes to the shell**: what it declares, who composes it, and in what order it
|
||||
lands.
|
||||
2. **Where the environment lives.** Variables and `PATH` entries are facts about the account, not
|
||||
lines of one shell's syntax. They should reach every shell (interactive or not), the login shell's
|
||||
`execute` verb, and programs a graphical session starts.
|
||||
3. **Where the operator's own lines go,** so that assigning the shell module loses nothing the
|
||||
machine does today.
|
||||
4. **Which part of a file the mesh owns.** ADR 0174 calls the kept region the operator's; the host and
|
||||
to-be 38 implement the inverse (the mesh owns a marked block, and everything outside it is the
|
||||
operator's). The record this becomes says which.
|
||||
|
||||
## Why
|
||||
|
||||
Rolling out the shell module (to-be 38 WP5) was stopped on 2026-10-04 after a review of what assigning
|
||||
it would do. Measured in [01](01-what-the-shell-file-holds-today.md):
|
||||
|
||||
- Every machine carries the same predecessor-written startup file, so the module's block would be
|
||||
appended after its own older copy and everything would run twice.
|
||||
- The block drops lines the machines rely on today.
|
||||
- Nothing installs the prompt theme or the plugins the block loads.
|
||||
- The `execute` verb runs a non-interactive login shell, which never reads the file the block is
|
||||
written into.
|
||||
|
||||
The operator's direction: other modules must be able to plug themselves into the shell; the prompt
|
||||
becomes its own module; assigning the shell module must lose no functionality; and the environment,
|
||||
`PATH` above all, needs an answer of its own.
|
||||
|
||||
## What it touches
|
||||
|
||||
- The manifest. A contribution to the shell is either a new use of the existing `contributes` /
|
||||
`receives` pair or a new gathered field like `jails` (to-be 31).
|
||||
- The controller's composition, if the controller assembles the text.
|
||||
- The `login-shell` seat (ADR 0176): what a holder must do with what is contributed to it, and
|
||||
whether a module or the mesh declares the seat. Possibly a new seat for the environment, beside it
|
||||
and beside the service manager's (ADR 0177). Research 023 asks the related question of a seat
|
||||
naming the files its holder owns.
|
||||
- ADR 0174's wording of the kept region, and ADR 0182's classification of the paths under a home.
|
||||
- The zsh module, and the modules this makes possible: an environment module, the prompt, a version
|
||||
manager, a toolchain.
|
||||
|
||||
## Where it stands
|
||||
|
||||
The operator proposed a separate **environment module**: one module, holding a mesh seat of its own,
|
||||
that alone writes the account's environment. It writes a file that shells source and the service
|
||||
manager's user environment, from the variables and `PATH` entries every other module contributes to
|
||||
it. That is the starting position for the environment ([02](02-how-a-module-plugs-in.md) §1, option
|
||||
E6). It leaves the shell's contribution as shell code only (§2), addressed to the `login-shell` seat,
|
||||
which moves into the mesh's own seat set beside the new `node-environment` (§6).
|
||||
|
||||
Graduated on 2026-10-04 with one change from the starting positions: the controller, not the
|
||||
environment module's own code, renders the environment into the module's files, so that the result
|
||||
is in the declaration before a machine applies it ([ADR 0203](../../02-DECISIONS/0203-the-accounts-environment-is-one-modules-and-every-module-contributes-to-it.md),
|
||||
option 6b).
|
||||
|
||||
## Documents
|
||||
|
||||
- [01 — What the shell file holds today](01-what-the-shell-file-holds-today.md): evidence.
|
||||
- [02 — How a module plugs in](02-how-a-module-plugs-in.md): the options and the starting position.
|
||||
+103
@@ -0,0 +1,103 @@
|
||||
# 01 — What the shell file holds today
|
||||
|
||||
Measured 2026-10-04 on the four machines of one installation: two servers and two workstations. All
|
||||
four have the account's login shell set to zsh, zsh installed from the distribution, and a
|
||||
predecessor-written startup file. The predecessor is retired, so nothing manages these files any more.
|
||||
|
||||
## The startup file is the same everywhere
|
||||
|
||||
The account's `~/.zshrc` is **byte-identical on all four machines**: 102 lines, one checksum.
|
||||
`~/.zshrc.local`, which the last line of `~/.zshrc` sources, comes in **two variants**: one shared by
|
||||
both servers, and one shared by both workstations. So the "per-machine" part is really a
|
||||
per-*kind*-of-machine part.
|
||||
|
||||
The predecessor produced these from one module with two *flavors*: a prompt flavor and an
|
||||
autocomplete flavor, each of which swapped in a different local file. Its install hook also:
|
||||
|
||||
- cloned the prompt theme and three plugins from their upstream repositories into `~/.zsh/`;
|
||||
- installed fonts;
|
||||
- changed the login shell.
|
||||
|
||||
On the workstations the theme and plugins are still on disk, left over and now owned by nothing. The
|
||||
servers have none of them.
|
||||
|
||||
## What the 102 lines are
|
||||
|
||||
Sorted by who should own each line once the machine is modules:
|
||||
|
||||
| Lines today | What they are | Natural owner |
|
||||
|---|---|---|
|
||||
| `EDITOR`, `VISUAL`, `XDG_CONFIG_HOME`, `PATH` gaining `~/.local/bin` and two script directories | the account's environment | the shell's default, or the environment itself |
|
||||
| `PATH` gaining a toolchain's directory | environment, for one tool | the toolchain's module |
|
||||
| a version manager's directory variable plus sourcing its loader | environment *and* shell code | the version manager's module |
|
||||
| two variables naming the operator's own script library | environment, the operator's own | the operator |
|
||||
| a variable that turns off an agent's terminal-title handling | environment, for one tool | the agent's module |
|
||||
| the terminal title hook, keybindings, `dircolors`, the `ls`/`grep` aliases, `ll`/`la`/`l`, a container-run alias, two disk-usage functions, two port aliases | interactive shell behaviour | the shell's default |
|
||||
| the prompt's instant-prompt cache, the theme, the prompt's own configuration file | shell code, order-sensitive (instant prompt first) | the prompt module |
|
||||
| autosuggestions, syntax highlighting (and, unloaded, an autocomplete plugin on disk) | shell code, order-sensitive (syntax highlighting last) | a plugin module, or the prompt module |
|
||||
| sourcing `~/.zshrc.local` | the operator's hook | the operator |
|
||||
|
||||
The workstation variant of the local file adds:
|
||||
|
||||
- more environment: a desktop toolkit theme, a file manager's plugin list, `BROWSER`, `VISUAL`
|
||||
overridden to a graphical editor, a language toolchain's binary directory on `PATH`;
|
||||
- two pieces of shell code: one that pads the prompt to the bottom of the terminal under a display, and
|
||||
one that sources a function file another module places;
|
||||
- a hook sourcing a further per-node file.
|
||||
|
||||
The server variant holds only that last module-placed source line.
|
||||
|
||||
**Count:** a workstation runs 65 non-comment lines from the two files (53 shared, 12 local); a server
|
||||
runs 54. Of a workstation's 65:
|
||||
|
||||
- about a quarter (15) are environment;
|
||||
- about half are interactive defaults no other module cares about;
|
||||
- the remaining quarter is other modules' code and hooks (a prompt, plugins, a version manager, an
|
||||
agent's functions), loaded from the shell file only because there was nowhere else to put it.
|
||||
|
||||
## The shell module as written
|
||||
|
||||
The `zsh` module of to-be 38 WP5 (catalogue change, unmerged):
|
||||
|
||||
- writes one block, appended at the end of `~/.zshrc`, holding a subset of the shared file:
|
||||
- its environment lines, minus the toolchain directory, the version manager and the agent variable;
|
||||
- the title hook, keybindings and the most common aliases, minus the port aliases;
|
||||
- guarded `source` lines for the theme and two plugins *if present*;
|
||||
- the source of `~/.zshrc.local`.
|
||||
- assigned to any of the four machines, appends that block after the identical lines already there, so
|
||||
every line in it runs twice, `~/.zshrc.local` included.
|
||||
- on the servers, the guarded prompt lines find nothing; nothing installs the theme anywhere.
|
||||
|
||||
## Which startup file reaches what
|
||||
|
||||
zsh's startup order, and what each path through it reads:
|
||||
|
||||
| started as | reads |
|
||||
|---|---|
|
||||
| interactive login (a console, ssh with a terminal) | `.zshenv`, `.zprofile`, `.zshrc`, `.zlogin` |
|
||||
| interactive non-login (a new terminal window) | `.zshenv`, `.zshrc` |
|
||||
| non-interactive login: `zsh -lc …`, what the `execute` verb runs | `.zshenv`, `.zprofile`, `.zlogin`, **not** `.zshrc` |
|
||||
| non-interactive: a script, `ssh host command` | `.zshenv` only |
|
||||
|
||||
So an environment written into `.zshrc` reaches neither `execute` nor a script. The distribution's
|
||||
system-wide login profile, which zsh's system `zprofile` sources, only ever *appends* to `PATH` when an
|
||||
entry is missing. An entry the account's `.zshenv` puts first therefore survives a login.
|
||||
|
||||
A graphical session's programs (a launcher, a bar, a window manager's key bindings) are started from the
|
||||
display manager and the service manager, not from a shell, and read none of these files. The service
|
||||
manager's own place for the account's environment is `~/.config/environment.d/`. Today it holds nothing
|
||||
on any of the four machines, so a program launched from the window manager does not see `PATH` entries
|
||||
that a terminal does.
|
||||
|
||||
## What the mesh already has for "many modules, one file"
|
||||
|
||||
Measured over the catalogue's 69 module definitions:
|
||||
|
||||
| mechanism | used by | shape |
|
||||
|---|---|---|
|
||||
| `contributes` / `receives` | 28 contribute, 15 receive | A consumer contributes **facts** keyed by a requirement. The provider receives all of them as one file in the mesh's own format, and **renders them itself**. "The controller does not know what a reverse proxy is." |
|
||||
| `listens` / `filtering` | 40 declare listens, 1 composes | The controller derives the whole firewall rule set from every module's ports and writes it where the holder asks. |
|
||||
| `jails` / `jailing` | 3 declare, 1 composes | Each module supplies its jail **in the tool's own format**. The controller assembles them, sorted, into the one file the holder names. |
|
||||
| `into: block` on a file | 2 | One module's marked region inside a file something else owns. Text outside the region is kept byte for byte. Placement is at the end, or at the start. |
|
||||
|
||||
None of these is a contribution of shell code or of environment today.
|
||||
@@ -0,0 +1,197 @@
|
||||
# 02 — How a module plugs in
|
||||
|
||||
Six questions, taken one at a time: the environment, shell code, the operator's own lines, order,
|
||||
who renders, and what a contribution is addressed to. Each has the options weighed and a starting
|
||||
position. The positions were set with the operator on 2026-10-04 and are what this effort tests, not
|
||||
what it has decided.
|
||||
|
||||
## 1. The environment: variables and `PATH`
|
||||
|
||||
A variable or a `PATH` entry is a fact about the account. It holds whichever shell is the login shell,
|
||||
and it is wanted by:
|
||||
|
||||
- every shell, interactive or not;
|
||||
- the login shell's `execute`;
|
||||
- a graphical session's programs.
|
||||
|
||||
[01](01-what-the-shell-file-holds-today.md) measures that `.zshrc` reaches only the first kind, and
|
||||
only interactively.
|
||||
|
||||
| | option | for | against |
|
||||
|---|---|---|---|
|
||||
| E1 | Each module writes lines into the shell's rc file (today) | nothing new | misses `execute`, scripts and the graphical session; written in one shell's syntax, so a second shell module starts over |
|
||||
| E2 | A module contributes environment facts (a variable and its value; a `PATH` entry and its position) **to the login shell**. The holder renders them into its shell's always-read file (`~/.zshenv` for zsh) | reaches every zsh, `execute` included; a contributor names no path and no shell | the graphical session sees none of it; the environment is tied to which module holds the shell; every shell module reimplements the same rendering |
|
||||
| E3 | E2, and the service-manager holder (ADR 0177) renders the same facts a second time into `~/.config/environment.d/` | the graphical session sees the same `PATH` as the terminal | one fact set, two owners, two renderings that can disagree; the service manager's module gains a duty unrelated to managing services |
|
||||
| E4 | One composed file in `environment.d` syntax, sourced by the shell with export-all | one file, two readers | ties the shell to the service manager's syntax, which is close to POSIX assignments but not equal (its `${VAR:-default}` and quoting rules differ); a value with a space breaks one reader or the other |
|
||||
| E5 | Shells take the environment from the service manager's environment generator, which prints the merged `environment.d` | no file of the shell's at all | every shell depends on the service manager and starts a process on every start; the generator's output is unquoted, so a value with a space breaks it |
|
||||
| **E6** | **An environment module.** A module of its own (working name `node-env`) holds a mesh seat, `node-environment`, and is the only writer of the account's environment. Every module contributes its variables and `PATH` entries to that seat. The holder writes them in each reader's format: a POSIX file of `export` lines that shells source, and the service manager's `~/.config/environment.d/` | the environment no longer depends on which shell holds `login-shell`; one owner and one rendering per format, both from the same facts; the graphical session included without the service manager's module; a contributor addresses "the environment", never a shell; the `PATH` rules (order, de-duplication) live in one module's code, where a test can hold them | one more module and seat, assigned on every node beside the shell; the `login-shell` protocol gains a duty, to source the environment file, which must be written down and checked |
|
||||
|
||||
**Starting position: E6.** It was the operator's proposal on 2026-10-04, and it replaces this
|
||||
document's first position (E2, then E3).
|
||||
|
||||
- The facts are the contribution. Each format is rendered once, by the one module whose subject is the
|
||||
environment.
|
||||
- A shell module's part shrinks to one line in its always-read file: `.zshenv` for zsh, sourcing the
|
||||
environment module's POSIX file. A bash or fish module writes the same line in its own file, and no
|
||||
contributor changes when the login shell does.
|
||||
|
||||
Sketched, for a node with zsh, the environment module, and a toolchain:
|
||||
|
||||
```
|
||||
toolchain ──contributes PATH entry──▶ node-environment ◀──contributes EDITOR, ~/.local/bin── zsh
|
||||
│ (held by node-env)
|
||||
┌──────────────────┴──────────────────┐
|
||||
▼ ▼
|
||||
POSIX export file ~/.config/environment.d/
|
||||
▲ ▲
|
||||
sourced from ~/.zshenv read by the service manager
|
||||
(every zsh, execute too) (the graphical session)
|
||||
```
|
||||
|
||||
The shell module still contributes its own environment (`EDITOR`, `XDG_CONFIG_HOME`, `~/.local/bin` on
|
||||
`PATH`) as a contributor like any other; it does not write those lines itself. Once issue 168 closes,
|
||||
the values a person varies become settings of whichever module contributes them (ADR 0174).
|
||||
|
||||
## 2. Shell code: a prompt, plugins, a version manager's loader
|
||||
|
||||
This *is* one shell's syntax, and order matters: a prompt's instant-prompt cache must run first, and
|
||||
syntax highlighting last.
|
||||
|
||||
| | option | for | against |
|
||||
|---|---|---|---|
|
||||
| S1 | **A contribution of code for one shell** (the shell it is for, the code, a slot), gathered by the controller and placed inside the holder's block in slot order. The same shape as `jails`, which a module supplies in fail2ban's own format and the controller assembles | a contributor names no path; the order is declared and checkable; unassigning the contributor removes its code at the next composition; a node holding fish simply has no zsh code rendered, and the resolver can say so | the controller gains one more gathered field; code for a shell travels in the declaration (in the clear, so no secrets in it, as for any file) |
|
||||
| S2 | **A drop-in directory**: each module places its own `~/.zsh/rc.d/NN-name.zsh`, and the shell's block sources the directory | no controller change; each file is its module's own, removed when undeclared | every contributor hard-codes a path inside the shell module's territory, against ADR 0112's spirit; order is a naming convention nothing checks; nothing ties the file to the shell actually being zsh |
|
||||
| S3 | Contributions as facts the holder renders (`contributes`/`receives` proper) | one mechanism with question 1 | code is not a fact; the holder would only paste it, which is S1 with an extra file |
|
||||
|
||||
**Starting position: S1.** A contribution to the shell carries **only code**, for named shells, each
|
||||
piece in a slot. Variables and `PATH` entries never go here; they go to the environment (§1). So a
|
||||
module touching both makes two contributions:
|
||||
|
||||
- A prompt module contributes zsh code in the first slot, and its own configuration file is its own
|
||||
owned file (ADR 0182).
|
||||
- A version manager contributes its directory variable to the environment, and its loader as code
|
||||
for each shell it supports.
|
||||
- A toolchain contributes a `PATH` entry to the environment and nothing to the shell.
|
||||
|
||||
What has to be settled: what each contribution is *addressed to*. Section 6 covers that.
|
||||
|
||||
## 3. The operator's own lines: the "local override"
|
||||
|
||||
Assigning the shell module must lose nothing the machine does today. That has two halves.
|
||||
|
||||
**What is common is the module's default, not an override.** The startup file is identical on all four
|
||||
machines ([01](01-what-the-shell-file-holds-today.md)). A line every machine has is the shell module's
|
||||
default, or another module's contribution. It is not a local override that a person would keep in step
|
||||
on every machine by hand. Most of today's file therefore moves into the shell module's block and into
|
||||
the contributions above. Little of it stays the operator's.
|
||||
|
||||
**What is the operator's is everything outside the mesh's block.** The host already works this way:
|
||||
|
||||
- the mesh's region is the marked block;
|
||||
- text outside it is kept byte for byte, and checked unchanged;
|
||||
- the region is given back when the module goes.
|
||||
|
||||
| | option | for | against |
|
||||
|---|---|---|---|
|
||||
| O1 | The mesh's block at the **start** of the file; the operator's lines after it | the operator's lines run last and win, which is what an override means; already supported (`at: start`) | a file the operator later rewrites must keep the markers; the host refuses a broken pair rather than guess |
|
||||
| O2 | A named operator region *inside* a file the mesh writes whole (ADR 0174's wording) | the file is entirely the mesh's except one hole | the opposite of what the host implements; a file a person already owns becomes the mesh's |
|
||||
| O3 | Only `~/.zshrc.local`, sourced from the block; `~/.zshrc` the mesh's whole | one obvious place | takes over a file the person owns today; ADR 0182 classifies the shell's own file as *written into*, not owned |
|
||||
|
||||
**Starting position: O1.** `~/.zshrc.local` keeps working because the operator's own lines source it,
|
||||
not because the mesh's block does.
|
||||
|
||||
The record this effort becomes corrects ADR 0174's description of the kept region as a **progressive
|
||||
insight**: the decision stands (a node varies a module by settings or by the operator's own lines,
|
||||
never by an edit), and only its description of which side is marked changes.
|
||||
|
||||
**The one-off migration** is a person's act, listed in the module's documentation (ADR 0182):
|
||||
|
||||
- remove from today's file every line the block or a contribution now carries;
|
||||
- keep the rest below the block.
|
||||
|
||||
Until a prompt module and the other contributors exist, the lines they will carry stay among the
|
||||
operator's own. Nothing is lost at any step.
|
||||
|
||||
## 4. Order
|
||||
|
||||
Order matters only for code. The environment is set before any code runs, because zsh reads
|
||||
`.zshenv` first. `PATH` entries carry their own position (before or after the system's), which the
|
||||
environment module orders, not the shell.
|
||||
|
||||
| | option | for | against |
|
||||
|---|---|---|---|
|
||||
| R1 | Numbers (`10`, `50`, `90`) | familiar | every contributor guesses a number; collisions are silent |
|
||||
| R2 | **A few named slots**, `first` / `normal` / `last`, with the module name breaking ties | the prompt says `first` and highlighting says `last` because that is what they mean; the composed result is the same bytes every time | three slots may not be enough |
|
||||
|
||||
**Starting position: R2.** Inside the shell module's block, the order is:
|
||||
|
||||
1. the line sourcing the environment module's file (in `.zshenv`, so it runs for every zsh; the rest
|
||||
of this list is `.zshrc`);
|
||||
2. the `first` slot;
|
||||
3. the shell module's own defaults;
|
||||
4. the `normal` slot;
|
||||
5. the `last` slot.
|
||||
|
||||
The operator's lines come after the block, as option O1 says.
|
||||
|
||||
## 5. Who renders: the controller or the holder's code
|
||||
|
||||
There are two different renderings, and E6 lets them be answered differently.
|
||||
|
||||
**The environment** is facts rendered into two fixed formats by the one module whose subject they are.
|
||||
|
||||
- The environment module receives the gathered contributions (the `contributes` / `receives` shape:
|
||||
facts in the mesh's own format, rendered by the receiver).
|
||||
- Its own code writes the POSIX file and the `environment.d` file whenever what it receives changes.
|
||||
That is ADR 0182's third class, written by the module's own process, owned by the account,
|
||||
atomically.
|
||||
- The controller learns no shell and no service manager. The `PATH` rules (prepend or append,
|
||||
de-duplicate, keep the system's entries) are ordinary code with ordinary tests.
|
||||
- To settle: what runs that code when the received file changes. The candidates are a host action
|
||||
that restarts on the received file, or a subscription through the runtime (ADR 0198).
|
||||
|
||||
**Shell code** is not facts. It is text in the shell's own syntax, assembled in slot order, which is
|
||||
what the controller already does for fail2ban jails: sort the pieces and concatenate them into the
|
||||
holder's region. The controller assembles; it never interprets the code. This keeps the shell module
|
||||
bundle-free for its files, and keeps the composed result visible in the declaration before a machine
|
||||
applies it.
|
||||
|
||||
## 6. What a contribution is addressed to
|
||||
|
||||
Under E6 there are two addressees: the environment and the login shell.
|
||||
|
||||
| | option | for | against |
|
||||
|---|---|---|---|
|
||||
| A1 | **Seats**: environment facts to `node-environment`, shell code to `login-shell`. Each seat's protocol says what its holder does with what is contributed to it | a contributor depends on a role ("the environment", "the login shell"), never on zsh or on one module; works the same for any holder | `login-shell` today is declared by the zsh module itself (ADR 0126). A second shell module may only claim it, never declare it, and the seat exists only while zsh's definition is registered |
|
||||
| A2 | Requirements the modules provide (`contributes` keyed by them, as the reverse proxy is) | an existing mechanism | a contributor on a node without the provider fails to resolve, though a toolchain's `PATH` entry with no environment module is merely unwritten |
|
||||
|
||||
**Starting position: A1, both seats in the mesh's own seat set** beside the service manager.
|
||||
|
||||
- `node-environment` is new, and is the mesh's from the start.
|
||||
- `login-shell` moves there from the zsh module's definition. A shell is as universal a role as a
|
||||
service manager, and a protocol that now carries duties (render the shell code contributed to it,
|
||||
source the environment file) should not depend on one module's registration.
|
||||
|
||||
Research 023 (a seat's protocol naming what its holder owns) is the general form of this: the
|
||||
environment seat would own the two environment files, and the login-shell seat the shell's
|
||||
startup-file region. The two efforts should not decide it twice.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Whether a contribution may be conditional on a capability: the workstation-only environment (a
|
||||
browser, a toolkit theme) is a desktop module's contribution, which arrives only where that module is
|
||||
assigned. Measured, this may need nothing new.
|
||||
- What a node without the environment module does with environment contributions: refuse them at
|
||||
resolve, or leave them unwritten and say so. The position here is to say so; a missing `PATH` entry
|
||||
is a visible gap, not a broken machine.
|
||||
- Whether the operator's own variables (the script-library paths in [01](01-what-the-shell-file-holds-today.md))
|
||||
are the operator's lines below the shell block, or a kept region of the environment module's file.
|
||||
The first needs nothing new, but reaches only interactive zsh.
|
||||
- How the prompt module and a plugin module divide the plugins. Packaging decides it as much as
|
||||
ownership: the plugins come from upstream repositories, not distribution packages, on these machines.
|
||||
- Whether the `execute` verb should read the interactive file at all once the environment is in
|
||||
`.zshenv`. The position here is no: a non-interactive login shell plus the environment is what a
|
||||
command needs, and the prompt's code should not run for it.
|
||||
- How a contribution reaches a second shell assigned beside the holder, which to-be 38 WP5 names as the
|
||||
first follow-up record. Under A1 a non-holder renders nothing, so the question becomes whether a
|
||||
non-holding shell module may render contributions for interactive use.
|
||||
@@ -0,0 +1,74 @@
|
||||
---
|
||||
status: active
|
||||
initiated: 2026-10-04
|
||||
touches:
|
||||
- 02-DECISIONS/0176-the-login-shell-is-a-node-seat-and-execute-is-its-contract.md
|
||||
- 02-DECISIONS/0177-a-unit-may-be-user-scoped-and-the-service-manager-is-a-node-seat.md
|
||||
- 02-DECISIONS/0182-inside-a-home-the-mesh-owns-what-it-places-and-holds-the-rest-as-found.md
|
||||
- 02-DECISIONS/0203-the-accounts-environment-is-one-modules-and-every-module-contributes-to-it.md
|
||||
- 02-DECISIONS/0204-a-module-contributes-shell-code-to-the-login-shell-in-named-slots.md
|
||||
- 02-DECISIONS/0205-software-the-distribution-does-not-package-ships-as-a-pinned-archive-of-the-module.md
|
||||
- 03-DESIGN/01-to-be/37-the-operators-machine.md
|
||||
- 03-DESIGN/01-to-be/38-building-the-operators-machine.md
|
||||
- 04-ISSUES/168-a-setting-reaches-every-file-and-contribution/00-report.md
|
||||
became: []
|
||||
---
|
||||
|
||||
# 026 — The graphical session as modules
|
||||
|
||||
## What is investigated
|
||||
|
||||
The workstations' graphical session as modules of the mesh, at the same level as the shell
|
||||
([to-be 41](../../03-DESIGN/01-to-be/41-the-shell-and-the-accounts-environment.md)): a package, files
|
||||
under the account's home, a seat, and nothing that names a machine. The pieces are:
|
||||
|
||||
- the login manager;
|
||||
- how a session starts and what environment it gets;
|
||||
- the display server (X today, Wayland as a sibling);
|
||||
- the window manager (i3, and sway as its Wayland sibling);
|
||||
- the terminal emulator (xterm);
|
||||
- the session's companions: bar, compositor, launcher, notifier, lock and idle, clipboard,
|
||||
wallpaper, theming, fonts.
|
||||
|
||||
[To-be 38](../../03-DESIGN/01-to-be/38-building-the-operators-machine.md) names this WP7, and says
|
||||
each seat begins with a record naming its holders and verbs. [To-be 37](../../03-DESIGN/01-to-be/37-the-operators-machine.md)
|
||||
§4 leaves one question for the resolver: whether a held seat can gate another's assignment.
|
||||
|
||||
## Why
|
||||
|
||||
The operator asked for the graphical modules next, at the shell's level, and for one consistent
|
||||
experience across machines. Since the predecessor retired, nothing manages the workstations'
|
||||
desktops. Measured in [01](01-what-the-workstations-run.md):
|
||||
|
||||
- Two workstations carry one 983-line predecessor module's output, still byte-identical in its core.
|
||||
- One workstation also carries another machine's hardware fragments.
|
||||
- One runs a session that predates two fixes, with two notification daemons and two portals.
|
||||
- The session's environment is a hand-kept second copy of the account's, beside the one the mesh
|
||||
now writes.
|
||||
|
||||
## How it is approached
|
||||
|
||||
**Adopting is also improving** (the operator, 2026-10-04). A module is not a copy of what a machine
|
||||
does today. Making it is the moment to fix what is broken, drop what is dead, choose the better tool
|
||||
and remove the leftovers. Every module's design lists its improvements over today. **Every module
|
||||
also serves tools,** many of them, for reading, acting and diagnosing; a module that only places a
|
||||
package and a file is unfinished. The tools are catalogued in
|
||||
[026/05](../026-the-graphical-session-as-modules/05-the-tools-each-module-serves.md).
|
||||
|
||||
## What it touches
|
||||
|
||||
- **The seat table:** up to ten node seats.
|
||||
- **The resolver:** a seat held on a node gating another module's assignment.
|
||||
- **The contribution mechanism of ADR 0204:** whether it generalises beyond shells, or whether
|
||||
tools' own drop-in directories serve.
|
||||
- **The host's user-scoped units** (mesh-host #72, still open).
|
||||
- **Settings** for per-machine values (issue 168).
|
||||
- **ADR 0205's archive** for the two pieces the distribution does not package.
|
||||
|
||||
## Documents
|
||||
|
||||
- [01 — What the workstations run](01-what-the-workstations-run.md): evidence.
|
||||
- [02 — The questions and the options](02-the-questions-and-the-options.md)
|
||||
- [04 — Screensaver, displays and menus](04-screensaver-displays-and-menus.md): the lock and idle module, monitor layouts by the monitors' identity, rofi and dmenu behind one launcher seat, the clipboard, fonts
|
||||
- [05 — The tools each module serves](05-the-tools-each-module-serves.md): a first catalogue for the modules of 026 and 027
|
||||
- [03 — What the predecessor taught](03-what-the-predecessor-taught.md): its 128 modules and 3,395 commits, as patterns to keep and failures not to repeat; shared with research 027.
|
||||
@@ -0,0 +1,141 @@
|
||||
# 01 — What the workstations run
|
||||
|
||||
Measured 2026-10-04 on the two workstations of one installation, read-only: a laptop with a hybrid
|
||||
GPU and an internal panel, and a desktop with one GPU and two external monitors. Both run the same
|
||||
predecessor-generated desktop. File equality was checked by checksum across the two machines.
|
||||
|
||||
## How a session starts
|
||||
|
||||
The chain is the same on both:
|
||||
|
||||
1. The login manager (`lemurs`, built from the distribution's user repository, its package now in
|
||||
the official one) runs its X setup script on a virtual terminal.
|
||||
2. That script sources the login shell's profile files, then `~/.xprofile`, then the system's
|
||||
`xinitrc.d` drop-ins, then merges `~/.Xresources`.
|
||||
3. `~/.xprofile` reuses the systemd user manager's bus, then sources `~/.xinitrc`.
|
||||
4. `~/.xinitrc` sets up the session and ends with `exec i3`.
|
||||
|
||||
The login manager's own window-manager entry (`exec startx`) is never reached. Its configuration
|
||||
file uses a format two releases old, and an unmerged newer one sits beside it.
|
||||
|
||||
**What `~/.xinitrc` does**, in order:
|
||||
|
||||
1. Sources the system drop-ins, which import `DISPLAY` and `XAUTHORITY` into the user manager.
|
||||
2. Starts the keyring and exports its ssh socket.
|
||||
3. Exports the session's environment:
|
||||
- `PATH`, with nine entries, one of them a directory that no longer exists;
|
||||
- toolchain variables;
|
||||
- `XDG_CONFIG_HOME` and `XDG_DATA_DIRS` (with flatpak);
|
||||
- five GTK/Qt theme variables;
|
||||
- the desktop's identity (`XDG_CURRENT_DESKTOP`, `XDG_SESSION_DESKTOP`);
|
||||
- three of the operator's own variables.
|
||||
4. Imports an explicit allowlist of ten of those into the user manager and D-Bus activation. It is
|
||||
never `--all`, because:
|
||||
5. a predecessor file of **secrets as environment variables** (package-registry and API tokens) is
|
||||
sourced next.
|
||||
6. Sets the screensaver and display power timeouts, restores the wallpaper, and starts the lock
|
||||
watcher in a respawn loop. It is deliberately not a unit, because it needs the login session.
|
||||
7. `exec i3`.
|
||||
|
||||
**The account's environment, as of today, has three sources that disagree:**
|
||||
|
||||
- this file, for the session;
|
||||
- the mesh's `environment.sh`, for shells
|
||||
([ADR 0203](../../02-DECISIONS/0203-the-accounts-environment-is-one-modules-and-every-module-contributes-to-it.md));
|
||||
- `~/.config/environment.d/`, for the user manager. It holds the mesh's `50-mesh.conf`, and a
|
||||
predecessor file that **sets `PATH` outright** and sorts after it.
|
||||
|
||||
## The roles, and what fills them
|
||||
|
||||
| role | software | where configured |
|
||||
|---|---|---|
|
||||
| login manager | lemurs | `/etc/lemurs/*` (identical on both, and to the predecessor's source) |
|
||||
| session start and environment | the login manager's X setup, `~/.xprofile`, `~/.xinitrc`, `xinitrc.d`, the D-Bus import, `environment.d` | `~/.xprofile`, `~/.xinitrc`, `~/.config/environment.d/*` |
|
||||
| display server | Xorg (`xorg-server`, `xinit`, the X apps; vendor drivers per GPU) | **no** `xorg.conf.d`; monitors by `xrandr` scripts |
|
||||
| monitor layout | `xrandr` scripts (arandr), a hotplug rule on the laptop | `~/.screenlayout/`, a scripts folder, a window-manager fragment |
|
||||
| window manager | i3 4.25 | `~/.config/i3/config` and `config.d/*`, a reload watcher (user unit) |
|
||||
| bar | i3bar with i3status-rust | `~/.config/i3status-rust/*`, 14 themes, a bar watchdog (user unit) |
|
||||
| terminal | xterm (the only terminal installed) | `~/.Xresources.d/xterm`, the window manager's binding, the compositor's opacity rule |
|
||||
| compositor | picom | `~/.config/picom/picom.conf` |
|
||||
| launcher and menus | rofi | `~/.config/rofi/*`, launcher, power-menu and theme-picker scripts |
|
||||
| notifier | dunst (D-Bus activated) | `~/.config/dunst/dunstrc`, `dunstrc.d/*` |
|
||||
| lock, idle, display power | xss-lock and i3lock-color, `xset` | `~/.xinitrc`, a lock script |
|
||||
| clipboard | greenclip, xclip | `greenclip.toml` |
|
||||
| wallpaper | feh | `~/.fehbg` (points into the predecessor's tree) |
|
||||
| theming | Adwaita dark, qt5ct/qt6ct, the desktop portal (GTK backend pinned) | GTK `settings.ini`, `qt*ct.conf`, `portals.conf`, an appearance script, `.Xresources` cursor |
|
||||
| fonts | Hack and Meslo Nerd fonts in `~/.local/share/fonts` (not packaged), noto | `~/.Xresources.d/xft` (DPI fixed at 96) |
|
||||
| keyboard | nothing set; the default layout; vendor keys via triggerhappy on the laptop | window-manager bindings, `/etc/triggerhappy` |
|
||||
|
||||
**Packages:** every piece except two is in the distribution's official repositories, and the login
|
||||
manager now is too. The two exceptions are the lock screen's colour build (`i3lock-color`) and the
|
||||
clipboard manager (`rofi-greenclip`). The Nerd fonts exist as official packages, but both machines
|
||||
carry hand-copied files instead.
|
||||
|
||||
## Identical, different, and why
|
||||
|
||||
**Byte-identical on both machines:**
|
||||
|
||||
- the session files: `.xinitrc`, `.xprofile`, `.Xresources` and its drop-ins;
|
||||
- the i3 main configuration and two of its fragments;
|
||||
- the bar's top configuration and themes;
|
||||
- picom, rofi, the GTK and Qt settings, the portal configuration, the login manager.
|
||||
|
||||
**Different, by cause:**
|
||||
|
||||
| cause | what |
|
||||
|---|---|
|
||||
| hardware | the monitor layout script; the bar's battery block; the laptop's power and vendor-key units and udev rules |
|
||||
| misassignment | the desktop carries the **laptop's** hardware fragments: the vendor-key daemon and its triggers, the backlight rule, the brightness drop-in, a touchpad reset, and the laptop's monitor layouts, in an older version |
|
||||
| drift | the notifier's position and corner radius; a "temporary" window-manager fragment from a test; the bar watchdog disabled; a second Qt configuration tool; different font builds |
|
||||
| a stale session | the desktop's session began before two fixes, so it runs two notification daemons and two portals, and its user manager lacks the desktop's identity |
|
||||
|
||||
**Dead references:** the window manager starts a polkit agent that is installed on neither machine,
|
||||
so there is no polkit agent at all. `PATH` names a directory that does not exist.
|
||||
|
||||
**Per-machine values inside shared files:**
|
||||
|
||||
- the DPI;
|
||||
- absolute home paths, in the clipboard configuration and the flatpak data directories;
|
||||
- the laptop's panel name, inside a fragment both machines carry.
|
||||
|
||||
## User units the desktop needs
|
||||
|
||||
| unit | does | laptop | desktop |
|
||||
|---|---|---|---|
|
||||
| reload watcher | reloads the window manager and bar when their files change | on | on |
|
||||
| bar watchdog | restarts a dead bar | on | off |
|
||||
| clipboard daemon | from its package | via the window manager | unit **and** window manager |
|
||||
| vendor power profile, memory guard | laptop power | on | — |
|
||||
|
||||
None is managed. Applying them as the account needs the host's user scope
|
||||
([ADR 0177](../../02-DECISIONS/0177-a-unit-may-be-user-scoped-and-the-service-manager-is-a-node-seat.md)),
|
||||
which is still an open change.
|
||||
|
||||
## The predecessor's module
|
||||
|
||||
One manifest of 983 lines covers the window manager, bar, launcher, notifier, compositor, lock
|
||||
screen, session bootstrap, theming and scripts. It:
|
||||
|
||||
- has four *flavors*: i3, laptop (i3 plus the monitor wizard and hotplug), desktop (i3 plus
|
||||
nothing) and a laptop model (laptop plus vendor keys);
|
||||
- has about **105 theme variables** substituted into templates: border, gaps, fonts, workspace
|
||||
names, every colour of bar, launcher, notifier and lock screen, compositor opacity, cursor, idle
|
||||
times, Qt and GTK theme names;
|
||||
- enables the two user units from an install hook.
|
||||
|
||||
Separate modules held the login manager and the display server (one flavor, `xorg`, with a comment
|
||||
calling `wayland` "the intended sibling"). The shell module held no graphical part.
|
||||
|
||||
## Wayland and sway
|
||||
|
||||
**Nothing exists.** There is no compositor, no sway configuration, no Wayland session entry, and the
|
||||
login manager's Wayland directory is empty. What is installed is libraries:
|
||||
|
||||
- Wayland itself and the Qt Wayland plugins, which other packages pull in;
|
||||
- `xwayland`, explicitly installed and required by nothing;
|
||||
- on the desktop, an orphaned compositor library from another desktop environment, and that
|
||||
environment's portal backend, pulled in by a game launcher. The portal configuration pins
|
||||
against it.
|
||||
|
||||
Every piece a sway session needs is in the official repositories: the compositor, its lock screen,
|
||||
a terminal (`foot`), a bar (`waybar`), a notifier (`mako`) and `xwayland`.
|
||||
@@ -0,0 +1,123 @@
|
||||
# 02 — The questions and the options
|
||||
|
||||
Seven questions. Each has its options and a starting position, which is what this effort tests, not
|
||||
what it has decided.
|
||||
|
||||
## 1. How finely the desktop splits into modules
|
||||
|
||||
| | option | for | against |
|
||||
|---|---|---|---|
|
||||
| G1 | One desktop module, as the predecessor had | one assignment | flavors again, per machine; ADR 0174 refuses them, and the evidence shows a flavor landing on the wrong machine |
|
||||
| G2 | **One module per piece of software:** `lemurs`, `xorg`, `i3`, `i3status-rust`, `xterm`, `picom`, `rofi`, `dunst`, `xss-lock` with the lock screen, `greenclip`, `feh`, a theme module, a fonts module | each is what it declares; a machine gets exactly what is assigned; the same split already works for the shell and its plugins | about thirteen assignments per workstation |
|
||||
| G3 | G2, plus a named **set** the controller assigns as one (for example *the X desktop*) | G2's precision with G1's convenience | a set is a new controller concept |
|
||||
|
||||
**Starting position: G2.** Whether a set is worth a record is left until the thirteen assignments
|
||||
have been done by hand once.
|
||||
|
||||
## 2. The seats
|
||||
|
||||
Research 018 listed the candidates. ADR 0204 has since put the login shell in the mesh's own set,
|
||||
because a role with a protocol should not depend on one module's registration. The same reasoning
|
||||
applies here:
|
||||
|
||||
| seat | holders | protocol, first verbs |
|
||||
|---|---|---|
|
||||
| `node-login-manager` | lemurs, greetd | which sessions it offers, the default session |
|
||||
| `node-display-server` | xorg, sway | `displays`, `layout` |
|
||||
| `node-display-session` | i3, sway | `reload`, `workspaces`, `windows` |
|
||||
| `node-terminal-emulator` | xterm, foot, alacritty | which terminal `$TERMINAL` names; `open` |
|
||||
| `node-bar`, `node-compositor`, `node-launcher`, `node-notifier`, `node-lock-screen`, `node-clipboard` | the pieces above, and their Wayland counterparts | one verb or none each, until a use asks for one |
|
||||
|
||||
**A compositor that is its own server holds two seats.** Sway is both the display server and the
|
||||
display session. A module may claim several seats, so this needs nothing new.
|
||||
|
||||
**Starting position:** the first four seats are in the mesh's own set. The companion seats are
|
||||
added only as each holder is written; for those, a module without a seat is acceptable at first.
|
||||
|
||||
## 3. One module requiring another seat to be held
|
||||
|
||||
i3 needs an X server held on its node, and sway needs nothing below it. A terminal needs a session.
|
||||
To-be 37 left open how that is said.
|
||||
|
||||
| | option | for | against |
|
||||
|---|---|---|---|
|
||||
| R1 | A seat **delivers a provision** (`x11-display`, `wayland-display`) and a module requires it at node scope. The seat table has a `delivers` field already, and requirements already resolve | existing machinery; the refusal names the seat and its possible holders, which design 27 already lists | a node-scoped requirement that never crosses machines has to be stated as such |
|
||||
| R2 | A new field, *needs the seat X held* | reads plainly | a second way to say what R1 says |
|
||||
| R3 | Nothing; assign carefully | — | the mistake the evidence shows (a laptop's fragments on a desktop) is exactly an unchecked assignment |
|
||||
|
||||
**Starting position: R1.** `xorg` and `sway` each deliver what they serve. `i3`, `picom` and `xss-lock`
|
||||
require `x11-display`. `foot` requires a Wayland display, and xterm requires an X one, which a Wayland
|
||||
session gives through `xwayland`.
|
||||
|
||||
## 4. Who starts the session, and with what environment
|
||||
|
||||
Today `~/.xinitrc` is a hand-kept second environment and the session's whole start script.
|
||||
|
||||
| | option | for | against |
|
||||
|---|---|---|---|
|
||||
| S1 | The display server's module writes `~/.xinitrc` **into**: a mesh block at the start that sources the account's environment (`environment.sh`), merges the X resources, and runs the session's contributed start lines. The session holder's module contributes its `exec` line. The operator's lines stay after the block | one environment for shells, the session and the user manager; nothing to keep in step | the order inside `.xinitrc` becomes the slot order of a contribution (question 5) |
|
||||
| S2 | The login manager's module owns the session script under `/etc` | system scope; no home file | the environment is the account's, and the script is the same for every account |
|
||||
| S3 | Leave `.xinitrc` the operator's | nothing to build | the third environment stays |
|
||||
|
||||
**Starting position: S1.**
|
||||
|
||||
- The desktop's identity (`XDG_CURRENT_DESKTOP`) and the theme variables become **environment
|
||||
contributions** (ADR 0203) from `i3` and from the theme module. They then also reach the user
|
||||
manager through `environment.d`, which replaces most of today's allowlist import.
|
||||
- The secrets file stays out of the environment until research 027 settles how a secret reaches an
|
||||
account.
|
||||
|
||||
## 5. How other modules contribute to a holder's file
|
||||
|
||||
The terminal's settings are X resources. A bar, a launcher binding and a hardware module's key
|
||||
bindings are window-manager configuration. Autostarts are the session's. ADR 0204 built slot
|
||||
contributions for shells only.
|
||||
|
||||
| | option | for | against |
|
||||
|---|---|---|---|
|
||||
| C1 | **The tool's own drop-in directory**, where it has one: i3's `include`, dunst's `dunstrc.d`, X resources' `#include`, XDG autostart entries, `environment.d`. Each contributor owns its own file there | no mesh change; the tools already read these directories; unassigning removes the file | each contributor names a path in another tool's directory (ADR 0204 rejected this for shells, where no drop-in convention exists); ordering is by file name |
|
||||
| C2 | **ADR 0204's mechanism generalised:** `contributes` text *for a format* (`zsh`, `xresources`, `i3`, `xinitrc`) in a slot, placed by the holder's placeholder | one mechanism, checked by the controller, order declared | every format must be named in the controller; a bigger change to ADR 0204 |
|
||||
| C3 | C1 where the tool has a drop-in convention, C2 where it does not (`.xinitrc`, `.Xresources` order) | uses each tool's own grain | two mechanisms to learn |
|
||||
|
||||
**Starting position: C3**, with the boundary drawn by the tools. A tool that reads a directory gets
|
||||
drop-ins. A file without one gets slots. This means amending ADR 0204's "shell" to "a format", which
|
||||
is a progressive extension rather than a reversal.
|
||||
|
||||
## 6. What varies per machine
|
||||
|
||||
| what | today | option |
|
||||
|---|---|---|
|
||||
| monitor layout | per-machine `xrandr` scripts, monitor names baked in | a **setting** of `xorg` (issue 168), and a `layout` verb of the display server seat |
|
||||
| DPI, fonts' size | fixed in an X resource | a setting |
|
||||
| battery block, vendor keys, brightness, touchpad | a laptop model's flavor | **a hardware module** per machine model, contributing its window-manager fragment, bar block and udev rules. The desktop simply is not assigned it |
|
||||
| theme (the 105 variables) | template substitution | settings of each tool's module, after issue 168 closes (ADR 0174). Until then each module carries today's values as its default |
|
||||
|
||||
**Starting position:**
|
||||
|
||||
- Hardware modules for what follows the machine.
|
||||
- Defaults now, settings after issue 168, for what the operator varies.
|
||||
- The monitor layout waits for settings. Until then it is an operator-owned script the display
|
||||
server's block calls if present.
|
||||
|
||||
## 7. Wayland and sway
|
||||
|
||||
Nothing of a Wayland session exists, and every piece is officially packaged. "Wayland" is a protocol,
|
||||
not a piece of software, so it has no module of its own. Its parts are `sway` (server and session),
|
||||
`swaylock`, `foot`, `waybar`, `mako`, and `xwayland` for X clients.
|
||||
|
||||
**Starting position:**
|
||||
|
||||
- The seats and the requirements (questions 2 and 3) are designed so that sway fits from the first
|
||||
day.
|
||||
- The X stack is built first, because it is what runs.
|
||||
- `sway` and its companions are written after that, and proven on one workstation as a second
|
||||
session the login manager offers beside i3. That lets the operator try it without losing the
|
||||
working desktop.
|
||||
|
||||
## Prerequisites this effort cannot remove
|
||||
|
||||
- **User-scoped units** (mesh-host #72) for the reload watcher and the bar watchdog.
|
||||
- **Settings** (issue 168) for monitors and theme values.
|
||||
- **The two packages not in the official repositories:** the lock screen's colour build and the
|
||||
clipboard manager. Each is ADR 0205's case, a pinned archive, or a choice of an official
|
||||
alternative (`i3lock` without colours; `clipmenu`/`cliphist`).
|
||||
@@ -0,0 +1,51 @@
|
||||
# 03 — What the predecessor taught
|
||||
|
||||
A study on 2026-10-04 of the retired predecessor:
|
||||
|
||||
- its 128 module manifests, their hooks, its installer and its sync engine;
|
||||
- 3,395 commits of history;
|
||||
- what it left on four machines.
|
||||
|
||||
This document holds what bears on the graphical session and on the system layer
|
||||
([research 027](../027-the-system-layer-as-modules/00-overview.md)). The evidence is in the
|
||||
predecessor's history. A commit is cited here by what it fixed, not by its hash, because the
|
||||
repository is private.
|
||||
|
||||
## Keep: what worked
|
||||
|
||||
| pattern | where it shows | in the mesh |
|
||||
|---|---|---|
|
||||
| ownership marked inside the file: inside the markers is reconciled, outside is kept verbatim | a block marker in a shared file, after an engine that rewrote whole files | kept regions (ADR 0174, ADR 0204) |
|
||||
| two writers get two files and an `include`, the include first | the ssh client's configuration, after two writers fought over one file | ADR 0203's two files; research 026 C1 |
|
||||
| one writer per file, one authority per action | only the reload watcher restarts the window manager, after three mechanisms each did | ADR 0182 |
|
||||
| refuse to write when the source of truth is unreadable; never empty a block because a query found nothing | a block of names was emptied by a failed query | — keep |
|
||||
| an unresolved template variable fails the install | a literal unfilled path was installed green | [issue 231](../../04-ISSUES/231-a-misspelled-placeholder-is-written-out-as-text/00-report.md): the mesh still has this gap |
|
||||
| prune only what you can prove you placed | stale files from earlier deliveries | ADR 0189 |
|
||||
| ensuring never rotates a credential | a silent rotation caused a retry storm, a ban of the shared address and a lost registry | ADR 0114 |
|
||||
| vendor only the files you use; never clone and link | three files instead of 77 MB | ADR 0205 |
|
||||
| copy, never symlink | a recursive delete followed a link, and every reinstall failed | ADR 0012 |
|
||||
| verification says what it did not check | a verifier said *clean* while the secret was still on disk | — keep |
|
||||
| alert once per condition | 411 alerts hid a 28-hour outage | ADR 0090 |
|
||||
|
||||
## Do not repeat
|
||||
|
||||
| failure | what it did | the mesh instead | where the mesh is still exposed |
|
||||
|---|---|---|---|
|
||||
| **Flavors** | variant files and packages per machine type: a gate dropped, the first-seen variant won, packages never installed, the verifier ignored the gate. On the day of the study a desktop carried a laptop model's fragments | one module per piece, assignment per machine (ADR 0174, research 026 §1) | a setting that switches which whole file is rendered is a flavor under another name |
|
||||
| **The freeze** | existing values outranked new defaults; templated files were rendered once (*copy if absent*) | files are generated (ADR 0011) | a created-once file (ADR 0087) is a deliberate freeze, and a push must say *kept* |
|
||||
| **Adopting drift** | a *merge* strategy made a local edit the record forever; switching strategies clobbered a person's model choice | nothing is read back (ADR 0174) | an edit outside a kept region is overwritten **silently**. The predecessor's *why is this back* loop: the push should name what it overwrote |
|
||||
| **Environment templating** | `${VAR}` matched any name; unresolved names stayed literal; comments and destination paths were interpolated | namespaced placeholders; `$` forbidden in contributed values (ADR 0203) | issue 231 |
|
||||
| **Hooks with privilege** | install hooks ran `sudo`, `chsh`, `systemctl`, `git clone` and `curl`, and swallowed failures into a warning | the `user` shape, the service shape, archives (ADR 0176, 0177, 0205) | the agent module writes under `/etc` from its own tool through `sudo` (no keep-original, no give-back); the prompt's helper downloads itself unpinned; that the operator escalates without a prompt is assumed by three modules and declared by none (research 027) |
|
||||
| **Secrets in environment files** | `.env` files left world-readable; the decryption key beside what it decrypts; a deleted secret stayed in the file, so rotation was a no-op | the vault (ADR 0113, 0114) | a predecessor file of secrets is still sourced into the graphical session on two machines (research 027 Q2) |
|
||||
| **Symlinks into a home** | a system file linked into a person's home | ADR 0012 | on the control machine, a fail2ban action file is still a predecessor link into its home tree. Deleting that tree would silently break the repeat-offender jail. The mesh's fail2ban module must own it as a file first |
|
||||
| **Green while broken** | a recorded version frozen for four months; a verifier passing what it skipped | ADR 0134, 0145, 0184 | issue 230: a plan waiting for ever reads as healthy |
|
||||
|
||||
## What it means here
|
||||
|
||||
- **Research 026:** the desktop's 88 flavor-gated files and 92 theme variables are the flavor and
|
||||
templating failures in one module. Question 1 (one module per piece) and question 6 (hardware
|
||||
modules, settings later) are the answer, and nothing in the new modules may switch whole files on a
|
||||
setting.
|
||||
- **Research 027:** the hooks that installed the AUR helper, enabled the login manager and changed
|
||||
shells are what the `package`, `service` and `user` shapes replace. Every remaining `sudo` in a module's
|
||||
own code is a debt to be named, starting with the agent module.
|
||||
+129
@@ -0,0 +1,129 @@
|
||||
# 04 — Screensaver, displays and menus
|
||||
|
||||
Three areas the operator named on 2026-10-04, as their own modules. Each sharpens a row of
|
||||
[01](01-what-the-workstations-run.md) and a question of [02](02-the-questions-and-the-options.md).
|
||||
|
||||
## The screensaver: idle, lock and display power
|
||||
|
||||
**Measured on both workstations:**
|
||||
|
||||
- **Idle and lock** are three things wired by hand in the session's start script:
|
||||
- the X screensaver timeout (`xset s 1800`);
|
||||
- the display power timeouts (`xset dpms`);
|
||||
- `xss-lock` running the colour build of `i3lock` through a wrapper, in a respawn loop.
|
||||
- **A second screensaver,** xscreensaver, is installed and deliberately not started. Earlier it
|
||||
overrode the display power settings with its own, and locked nothing. Its configuration file is
|
||||
still in the home.
|
||||
- **The lock screen's 20-odd colours and formats** were predecessor theme variables.
|
||||
- **The colour build is not in the official repositories** (research 026/01).
|
||||
|
||||
**Starting position:**
|
||||
|
||||
- **One module for the lock screen,** holding `node-lock-screen`: the locker and its wrapper as the
|
||||
module's own files, the screensaver and display power timeouts, and `xss-lock`.
|
||||
- The timeouts and colours are its defaults, and settings later (issue 168).
|
||||
- The colour build ships as ADR 0205's pinned archive, or the module uses the official `i3lock`.
|
||||
That is the operator's choice, and the colours are the only difference.
|
||||
- xscreensaver is not a module; its package and file are removed.
|
||||
- `xss-lock` needs the logind session, so it stays a session-start line contributed into
|
||||
`.xinitrc`'s block (question 4), not a unit.
|
||||
|
||||
## Monitor layout (xrandr)
|
||||
|
||||
**Measured:**
|
||||
|
||||
- Each workstation has a layout script generated by `arandr`, with the monitor names baked in. One
|
||||
workstation also has several layouts for named places, a hotplug rule and a wizard.
|
||||
- **The desktop carried the laptop's layout scripts.**
|
||||
- No `xorg.conf.d`, and no layout tool beyond the scripts.
|
||||
|
||||
**Starting position: `autorandr`** (official repositories) inside the display server's module.
|
||||
|
||||
- `autorandr` saves a layout as a profile **keyed by the connected monitors' identities** (their EDID)
|
||||
and applies the matching one at login and on hotplug.
|
||||
- Profiles therefore need no machine's name. A profile can be shared mesh-wide and simply never
|
||||
matches on a machine without those monitors. That is exactly the "say it by what is there, never by
|
||||
a name" rule (ADR 0112).
|
||||
- The profiles are the operator's data, saved by the tool itself, so they are *found* (ADR 0182). A
|
||||
`layout` verb on `node-display-server` lists, saves and applies them.
|
||||
- The arandr scripts and the hotplug rule retire once a profile exists for each.
|
||||
|
||||
## Menus: rofi and dmenu
|
||||
|
||||
**Measured:**
|
||||
|
||||
- rofi is the launcher, the power menu, the theme picker and the clipboard menu.
|
||||
- The operator's scripts call `rofi -dmenu` in four places and **plain `dmenu` in two. dmenu is
|
||||
installed on neither workstation, so those two fail.**
|
||||
|
||||
**Starting position:**
|
||||
|
||||
- **`rofi` holds `node-launcher`**, and the seat's protocol includes a **dmenu-compatible command**:
|
||||
read choices on standard input, print the chosen one. Scripts call that command, not a program by
|
||||
name.
|
||||
- **`dmenu` is a module of its own** (official repositories), able to hold the same seat on a machine
|
||||
that wants it, for instance a Wayland session where `wofi` or `fuzzel` would hold it instead.
|
||||
- The rofi module carries its theme files, and the menus that belong to other modules arrive as those
|
||||
modules' scripts:
|
||||
- power menu → the session;
|
||||
- clipboard menu → the clipboard module;
|
||||
- theme picker → settings, once issue 168 closes.
|
||||
|
||||
## The clipboard: xclip and greenclip
|
||||
|
||||
**Measured:**
|
||||
|
||||
- **greenclip** keeps the clipboard's history, and rofi shows it on a key binding.
|
||||
- **greenclip is not in the official repositories.**
|
||||
- It is started two ways: the window manager's configuration starts it on both workstations, and on
|
||||
one a user unit is enabled as well.
|
||||
- Its configuration names an absolute home path.
|
||||
- **xclip** (official) is the command-line clipboard the operator's scripts use.
|
||||
|
||||
**Starting position:**
|
||||
|
||||
- **`xclip` is a module of its own,** a package and nothing else. It is the tool scripts depend on,
|
||||
and a module that needs it requires it.
|
||||
- **The clipboard manager holds `node-clipboard`:** its daemon, started once by the session (a session
|
||||
contribution, or a user unit once user-scoped units ship, never both), its configuration with no
|
||||
absolute path, and its menu binding contributed to the window manager.
|
||||
- **Which manager holds it is the operator's choice:**
|
||||
- greenclip, as today, shipped under ADR 0205;
|
||||
- or `clipmenu` (official), which feeds the same dmenu-compatible command as the launcher seat above,
|
||||
and needs no archive.
|
||||
- **On Wayland** the same seat is held by `cliphist` with `wl-clipboard`, both official.
|
||||
|
||||
## Fonts
|
||||
|
||||
**Measured:**
|
||||
|
||||
- The fonts the desktop uses are **hand-copied files** in the account's font directory, not packages:
|
||||
- a Nerd font for the window manager, the bar and the terminal;
|
||||
- a second one for the prompt;
|
||||
- on one workstation, the same four files twice, once under URL-encoded names;
|
||||
- on the other, a different build of the same font and three more copied from a theme's repository.
|
||||
- The system's default monospace is a different font (`Noto Sans Mono`), so anything that asks for
|
||||
`monospace` gets another face than the terminal.
|
||||
- The DPI is fixed in an X resource.
|
||||
- **Every Nerd font in use is in the official repositories** (Hack, Meslo, Iosevka, JetBrains Mono).
|
||||
|
||||
**Decided** (the operator left the choice open, except that it must not be today's Hack):
|
||||
|
||||
| role | face | why |
|
||||
|---|---|---|
|
||||
| monospace: terminal, window manager, bar, launcher, prompt | **JetBrains Mono Nerd Font** | built for long reading in a terminal, unambiguous `0O1lI`, optional ligatures; a version-3 Nerd font, so every icon the prompt and bar use is present |
|
||||
| interface: GTK, Qt, notifications | **Inter** | designed for screens, clear at small sizes |
|
||||
| icons missing from any face | Nerd Fonts Symbols | a fallback, so a font without icons still shows them |
|
||||
| emoji | Noto Color Emoji | |
|
||||
| serif and every other script | Noto | |
|
||||
|
||||
All five are official packages.
|
||||
|
||||
**Starting position:**
|
||||
|
||||
- **A `fonts` module:** those packages, and a fontconfig file it owns that maps `monospace`, `sans-serif`,
|
||||
`serif` and the emoji and symbol fallbacks to the chosen faces, so every program agrees.
|
||||
- The terminal, bar, launcher and prompt modules name the family, not a file.
|
||||
- The DPI becomes the display server's setting (issue 168).
|
||||
- The copied files are removed by the operator once the packages are in (ADR 0182).
|
||||
- Fonts are not a seat: several coexist. The module owns the one place where *the* default is said.
|
||||
@@ -0,0 +1,77 @@
|
||||
# 05 — The tools each module serves
|
||||
|
||||
A first catalogue for the modules of research 026 and 027, as the operator asked: "all kinds of useful
|
||||
tools for all these modules". Each tool is served by the node's runtime (ADR 0175), on the machine the
|
||||
module runs on. Through discovery (ADR 0195) it is reachable from any machine as
|
||||
`<machine>/<module>.<tool>`, or as `<machine>/<seat>.<verb>` where a seat defines it.
|
||||
|
||||
**Conventions:**
|
||||
|
||||
- **(r)** reads.
|
||||
- **(a)** acts on the machine, escalating where it must, as the packet filter does (to-be 38 WP4).
|
||||
- **(d)** is a desktop act that needs the operator's session.
|
||||
- A tool that changes something a module declares says so in its answer: the next push restores the
|
||||
declaration.
|
||||
- Every tool answers structured data, not prose (issue 229).
|
||||
- **Seat verbs** (marked *seat*) are the protocol every holder of that seat serves. The rest are the
|
||||
module's own.
|
||||
|
||||
## The graphical session (026)
|
||||
|
||||
| module | tools |
|
||||
|---|---|
|
||||
| `xorg` (*node-display-server*) | *seat* `displays` (r: outputs, modes, rates, connected monitors with their identity) · *seat* `layout` (r/a: list, save, apply an autorandr profile) · `set-mode` (a: one output's resolution, rate, rotation, scale) · `primary` (a) · `dpi` (r/a) · `input-devices` (r) · `input-set` (a: touchpad tap, natural scroll, pointer speed) · `keyboard` (r/a: layout and options) · `screenshot` (d: one screen or all, as a file) · `x-log` (r: the server's errors since start) |
|
||||
| `i3` (*node-display-session*) | *seat* `reload` (a) · *seat* `workspaces` (r) · *seat* `windows` (r: tree with classes, titles, workspaces) · `focus` (d: window or workspace) · `move` (d: window to workspace or output) · `layout-save` / `layout-restore` (d: a workspace's arrangement) · `exec` (d: start a program in the session) · `kill` (d) · `bindings` (r: every key binding and what it runs) · `config-check` (r: validate the composed configuration before a reload) · `marks` (r) · `scratchpad` (d) |
|
||||
| `sway` (*node-display-server*, *node-display-session*) | the same seat verbs over Wayland, plus `outputs` (r) and `idle-inhibitors` (r) |
|
||||
| `lemurs` (*node-login-manager*) | *seat* `sessions` (r: what the login screen offers) · *seat* `default-session` (r/a) · `logins` (r: who logged in when, from the journal) |
|
||||
| `xterm` (*node-terminal-emulator*) | *seat* `open` (d: a terminal, optionally running a command, in a directory) · `font` (r/a: face and size) · `colours` (r) |
|
||||
| `i3status-rust` (*node-bar*) | *seat* `reload` (a) · `blocks` (r: what the bar shows and each block's current value) · `block-run` (r: run one block once and answer its output) · `themes` (r) |
|
||||
| `picom` (*node-compositor*) | *seat* `restart` (a) · `rules` (r: opacity, shadow and blur rules in force) · `window-opacity` (d) · `toggle` (d: compositing off and on, for a game or a test) |
|
||||
| `rofi` (*node-launcher*) | *seat* `menu` (d: show a list, answer the chosen line: the dmenu-compatible command as a tool) · `applications` (r: the desktop entries it would offer) · `themes` (r) · `run` (d) |
|
||||
| `dmenu` (*node-launcher*) | *seat* `menu` (d) |
|
||||
| `dunst` (*node-notifier*) | *seat* `send` (d: title, body, urgency, actions) · *seat* `history` (r) · `pause` / `resume` (d: do not disturb) · `close-all` (d) · `rules` (r) · `count` (r: shown, waiting, history) |
|
||||
| lock module (*node-lock-screen*) | *seat* `lock` (d) · `idle` (r/a: screensaver and display power timeouts) · `inhibit` (d: keep the screen on for a while) · `locked` (r: is the session locked now, and since when) |
|
||||
| clipboard manager (*node-clipboard*) | *seat* `history` (r: entries, newest first, length-limited) · *seat* `copy` (d: put text on the clipboard) · `paste` (r: what the clipboard holds now) · `clear` (d) · `delete` (d: one entry) |
|
||||
| `xclip` | `copy` (d) · `paste` (r): the plain clipboard without a manager |
|
||||
| `feh` (wallpaper) | `set` (d: an image, per output) · `current` (r) |
|
||||
| `fonts` | `families` (r: installed faces) · `match` (r: what `monospace`, `sans-serif` and `emoji` resolve to) · `glyph` (r: which installed font has a given character) · `cache-rebuild` (a) |
|
||||
| theme module | `appearance` (r/a: dark or light, for GTK, Qt and the portal at once) · `cursor` (r/a) · `icons` (r) · `portal-check` (r: which portal backend answers which interface) |
|
||||
| `gnome-keyring` (*node-secret-service*) | *seat* `unlocked` (r) · `lock` (d) · `collections` (r: names and item counts, never secrets) · `ssh-keys` (r: what the agent holds, by fingerprint) |
|
||||
| desktop hardware module (laptop) | `brightness` (r/a: panel and keyboard) · `battery` (r: charge, health, cycles, limit) · `charge-limit` (r/a) · `gpu-mode` (r/a: integrated, hybrid, discrete) · *seat* `profile` (r/a: quiet, balanced, performance) · `thermals` (r: temperatures and fan speeds) · `power-draw` (r) |
|
||||
|
||||
## The system and the account (027)
|
||||
|
||||
| module | tools |
|
||||
|---|---|
|
||||
| `docker` (*node-container-runtime*, ADR 0166) | *seat* `list`, `inspect`, `logs`, `stats`, `start`, `stop`, `restart` (r/a) · `images` (r: with size and which container uses each) · `prune` (a: dangling images, stopped containers not held by the mesh, build cache, with a dry run first) · `disk-usage` (r) · `networks` (r) · `volumes` (r: with what mounts each and whether the mesh holds it) · `events` (r: the last hour) · `daemon-config` (r) |
|
||||
| `docker-compose` | `projects` (r: compose projects running and where their files are) · `up` / `down` / `restart` (a: one project, by directory) · `logs` (r) · `ps` (r) |
|
||||
| `sudo` | `rules` (r: what the account may run, without a prompt and with one) · `check` (r: does the escalation the mesh relies on work here) |
|
||||
| `pacman` | `search` (r) · `installed` (r: with version and explicitly or as a dependency) · `info` (r) · `owns` (r: which package owns a path) · `files` (r) · `updates` (r: what an upgrade would change) · `upgrade` (a: with the news first) · `orphans` (r) · `remove-orphans` (a) · `cache` (r/a: size, clean to the last N versions) · `history` (r: installs and upgrades from the log) · `mirrors` (r/a: rank and refresh) · `news` (r: distribution news since the last upgrade) |
|
||||
| AUR (package repository, 027 question 1) | `search` (r) · `build` (a: on the build machine, into the mesh's repository) · `outdated` (r) · `published` (r) |
|
||||
| `snapd`, `flatpak` | `list` (r) · `install` / `remove` (a) · `update` (a) · `runtimes` (r) · `disk-usage` (r) |
|
||||
| `time-sync` | `status` (r: synchronised, offset, server) · `servers` (r) · `sync-now` (a) |
|
||||
| `localization` | `get` (r: locale, time zone, keymap) · `time-zone` (r/a) · `locales` (r) |
|
||||
| `kernel` | `running` (r: version, command line, uptime) · `installed` (r) · `modules` (r: loaded, with what uses them) · `reboot-needed` (r: a newer kernel or library than the one running) · `microcode` (r) · `boot-entries` (r) · `initramfs-rebuild` (a) · `dmesg` (r: errors since boot) |
|
||||
| `logrotate` | `status` (r: last rotation per log) · `force` (a: one configuration) · `big-logs` (r: the largest logs on the machine) |
|
||||
| `avahi` | `browse` (r: services on the local network) · `resolve` (r) |
|
||||
| `cups` | `printers` (r) · `queue` (r) · `cancel` (a) · `print` (a: a file to a printer) · `default` (r/a) |
|
||||
| `bluetooth` | `devices` (r: paired, connected, battery where reported) · `connect` / `disconnect` (a) · `scan` (r) · `power` (r/a) |
|
||||
| `ssh-client` (owns `~/.ssh`) | `hosts` (r: every `Host` and where it came from: the mesh, a module, the operator) · `check` (r: modes, keys without a passphrase, keys unused for a year, stale `known_hosts` entries) · `authorized` (r: who may log in, by fingerprint and comment) · `revoke` (a: one authorized key, into the operator's region) · `known-host` (r/a: verify, refresh one host's key) · `test` (r: can this machine reach a host and authenticate, batch mode) |
|
||||
| `sshd` | `sessions` (r: who is logged in, from where) · `config-effective` (r: `sshd -T`) · `failed-logins` (r: since a time, with fail2ban's verdicts) |
|
||||
| scripts modules | `list` (r: each script with its one-line description) · `run` (a: one script by name with arguments, as the account, bounded like `execute`) · `which` (r: which module ships a command) |
|
||||
| `node-env` (*node-environment*) | `show` (r: every variable and `PATH` entry with the module that contributed it) · `diff` (r: what a shell actually has versus what the mesh composed) |
|
||||
| `zsh` (*node-login-shell*) | *seat* `execute` · `zsh_config` (r) · `history-search` (r: the account's history, by pattern) · `functions` (r: aliases and functions in force, with where each came from) · `startup-time` (r: how long an interactive shell takes to start, per slot) |
|
||||
| `memory-pressure` | `status` (r: memory, swap, compressed swap ratio, pressure stall) · `top` (r: the largest processes) · `oom-history` (r: what was killed, when) |
|
||||
| `zfs` | `pools` (r: health, capacity, fragmentation) · `datasets` (r) · `snapshots` (r/a: list, create, destroy by name) · `scrub` (r/a: status, start) · `errors` (r) · `arc` (r: cache statistics) |
|
||||
| `nfs-server`, `samba` | `exports` / `shares` (r) · `clients` (r: who has it mounted now) · `reload` (a) |
|
||||
| `nfs-client`, `smb-client` | `mounts` (r: each share, mounted or not, and since when) · `mount` / `unmount` (a) · `test` (r: is the server reachable, is the export offered) |
|
||||
| hosts-file holder (*node-hosts-file*, ADR 0199) | *seat* `entries`, `add`, `remove` |
|
||||
| `vnstat`, `lm_sensors` | `traffic` (r: per interface, day, month) · `sensors` (r) |
|
||||
| mail consumer (future effort) | `accounts` (r) · `search` (r) · `unread` (r) · `read` (r: one message) · `mark` (a) · `send` (a) |
|
||||
|
||||
## What this catalogue is for
|
||||
|
||||
It is a starting list, not a contract. A tool becomes a contract only when it is a seat's verb, and
|
||||
each seat's verbs are decided in that seat's record (ADR 0132). A module's own tools can grow freely.
|
||||
Every row above is a tool the operator would otherwise run by hand over ssh. That is the measure of
|
||||
whether one is worth writing.
|
||||
@@ -0,0 +1,64 @@
|
||||
---
|
||||
status: active
|
||||
initiated: 2026-10-04
|
||||
touches:
|
||||
- 02-DECISIONS/0165-container-runtime-is-what-a-machine-can-run-and-a-running-runtime-is-its-holders-health.md
|
||||
- 02-DECISIONS/0166-the-container-runtime-is-a-node-seat-and-the-host-creates-containers-through-its-holder.md
|
||||
- 02-DECISIONS/0203-the-accounts-environment-is-one-modules-and-every-module-contributes-to-it.md
|
||||
- 02-DECISIONS/0205-software-the-distribution-does-not-package-ships-as-a-pinned-archive-of-the-module.md
|
||||
- 03-DESIGN/01-to-be/37-the-operators-machine.md
|
||||
- 03-DESIGN/01-to-be/38-building-the-operators-machine.md
|
||||
became: []
|
||||
---
|
||||
|
||||
# 027 — The system layer as modules
|
||||
|
||||
## What is investigated
|
||||
|
||||
What runs on the machines below the operator's home and outside the mesh's own services, and which
|
||||
of it should be modules. That covers:
|
||||
|
||||
- the container runtime and its tools;
|
||||
- privilege (sudo);
|
||||
- the package manager and the software it cannot install;
|
||||
- time, locale, the kernel and boot;
|
||||
- log rotation;
|
||||
- the machine-specific daemons the workstations and servers carry: printing, bluetooth, VPN
|
||||
clients, virtualisation, storage, sharing.
|
||||
|
||||
## Why
|
||||
|
||||
The operator asked for the system level beside the graphical session. In particular:
|
||||
|
||||
- a `docker` module (decided in principle by the proposed ADRs 0165 and 0166, never built);
|
||||
- a `docker-compose` module for development work, assigned **only to the two workstations**.
|
||||
|
||||
Measured in [01](01-what-the-machines-run.md): on four machines, almost nothing at this level is
|
||||
owned by a module. The pieces differ by machine for no recorded reason. Three findings are security
|
||||
matters on their own.
|
||||
|
||||
## How it is approached
|
||||
|
||||
**Adopting is also improving** (the operator, 2026-10-04). A module is not a copy of what a machine
|
||||
does today. Making it is the moment to fix what is broken, drop what is dead, choose the better tool
|
||||
and remove the leftovers. Every module's design lists its improvements over today. **Every module
|
||||
also serves tools,** many of them, for reading, acting and diagnosing; a module that only places a
|
||||
package and a file is unfinished. The tools are catalogued in
|
||||
[026/05](../026-the-graphical-session-as-modules/05-the-tools-each-module-serves.md).
|
||||
|
||||
## What it touches
|
||||
|
||||
- **The container runtime seat** (ADRs 0165 and 0166, both proposed).
|
||||
- **The host's `package` shape**, which installs from the distribution's official repositories only,
|
||||
while the workstations carry 67 and 114 packages from elsewhere.
|
||||
- **How a secret reaches the account's environment.** ADR 0203 forbids it in the contributed
|
||||
environment, but a predecessor file supplies such secrets today.
|
||||
- **The facts the mesh assumes and never declares,** above all that the operator account escalates
|
||||
without a prompt.
|
||||
|
||||
## Documents
|
||||
|
||||
- [01 — What the machines run](01-what-the-machines-run.md): evidence.
|
||||
- [02 — Candidates and questions](02-candidates-and-questions.md)
|
||||
- [03 — The account's own tools](03-the-accounts-own-tools.md): `~/.ssh` as one module's, scripts on every machine, the keyring, the laptop's power management, mail as events
|
||||
- The predecessor's lessons, shared with research 026: [026/03](../026-the-graphical-session-as-modules/03-what-the-predecessor-taught.md)
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user