diff --git a/.claude/skills/hal-graduate/SKILL.md b/.claude/skills/hal-graduate/SKILL.md index d1f4589..2d09869 100644 --- a/.claude/skills/hal-graduate/SKILL.md +++ b/.claude/skills/hal-graduate/SKILL.md @@ -19,7 +19,7 @@ it; this skill adds only the scaffolding. not recovered. Record the rejected options. 3. **Write the design** under `02-DESIGN/01-to-be/` with `layer: to-be`, `status: designed`, empty `code:`, today's `updated:`, and `decisions:` pointing at the record. -4. **Close the effort** — the research `status.md` gets `status: graduated` and `became:` +4. **Close the effort** — the research `00-overview.md` gets `status: graduated` and `became:` pointing at both. 5. **Add the ledger line** to `DECISIONS.md` under today's heading. diff --git a/.claude/skills/hal-new-research/SKILL.md b/.claude/skills/hal-new-research/SKILL.md index 0423452..ddeae00 100644 --- a/.claude/skills/hal-new-research/SKILL.md +++ b/.claude/skills/hal-new-research/SKILL.md @@ -14,12 +14,12 @@ this skill only does the mechanical setup. 1. Find the next free number: list `01-RESEARCH/NNN-*`, take the highest plus one, zero-padded to three digits. 2. Create `01-RESEARCH/NNN-descriptive-name/` (kebab-case from the topic). -3. Create `status.md` in it with exactly this frontmatter, then a prose summary of **what** is +3. Create `00-overview.md` in it with exactly this frontmatter, then a prose summary of **what** is being investigated, **why**, and **what it touches**: ```yaml --- - status: ongoing + status: active initiated: YYYY-MM-DD touches: [] became: [] diff --git a/.claude/skills/hal-status/SKILL.md b/.claude/skills/hal-status/SKILL.md index 2005e98..833fddd 100644 --- a/.claude/skills/hal-status/SKILL.md +++ b/.claude/skills/hal-status/SKILL.md @@ -13,7 +13,7 @@ generated on demand and never written back to disk. | Section | Files | Frontmatter | |---|---|---| -| Research | `01-RESEARCH/NNN-*/status.md` | `status` (`ongoing` / `graduated` / `abandoned`), `initiated`, `touches`, `became` | +| Research | `01-RESEARCH/NNN-*/00-overview.md` | `status` (`active` / `graduated` / `abandoned`), `initiated`, `touches`, `became` | | Design | `02-DESIGN/**/*.md` (not READMEs) | `layer` (`as-is` / `to-be`), `status` (`designed` / `in-progress` / `implemented` / `abandoned`), `code`, `updated`, `decisions` | | Decisions | `adr/NNNN-*.md` | `status` (`proposed` / `accepted` / `superseded`), `date`, `deciders`, `reconstructed`, `superseded-by`, `extends` | | Issues | `04-ISSUES/NNN-*/00-report.md` | `status` (`open` / `diagnosing` / `located` / `resolved` / `wontfix`), `opened`, `located-in`, `fixed-by`, `amended-design` | diff --git a/00-GENESIS/README.md b/00-GENESIS/README.md index d4efe11..518d1f6 100644 --- a/00-GENESIS/README.md +++ b/00-GENESIS/README.md @@ -38,9 +38,16 @@ comparing it against the code rather than by anyone noticing: - It listed the mesh as spanning a fixed number of named machines, which is exactly the content this repository cannot carry. -One earlier note in this file has been withdrawn as **wrong**, and is recorded here rather than -deleted. It claimed the overview's "symlinks, not copies" principle contradicted the mesh's -hard rule against symlinks. It does not. The rule forbids *creating* a symlink by hand; the -installer creates and reconciles every link the mesh needs, deliberately -([ADR 0011](../adr/0011-the-installer-owns-linking.md)). The rule is about who may link, not -about whether the mesh links — a misreading common enough that the ADR now says so explicitly. +It also lists **"symlinks, not copies" as a key design principle**, and that is a genuine +contradiction rather than a stale detail. The mesh's stated intent is that it creates no +symlinks at all — the rule is not merely "only the installer may link", and a founding document +elevating linking to a principle points the opposite way from where this is going. + +What exists today is that the installer owns and reconciles every link +([ADR 0011](../adr/0011-the-installer-owns-linking.md)) — an as-is fact, recorded in +[`02-DESIGN/00-as-is/05-runtime-and-installation.md`](../02-DESIGN/00-as-is/05-runtime-and-installation.md). +Centralising who may link narrowed the incident class; it did not close it. The intent is to +remove the mechanism, recorded as [ADR 0018](../adr/0018-the-mesh-creates-no-symlinks.md). + +A founding document contradicting the direction of travel is precisely the failure this folder +exists to prevent. diff --git a/00-GENESIS/how-we-build.md b/00-GENESIS/how-we-build.md index 7d0f577..c5defbc 100644 --- a/00-GENESIS/how-we-build.md +++ b/00-GENESIS/how-we-build.md @@ -39,7 +39,7 @@ incident behind it is not written down, and the fix is to write it down, not to | **Never write to a production database directly** | No insert, update, delete or schema statement executed against production by hand. Schema changes go through numbered migrations; data changes go through application code or the module's own capabilities. Raw statements skip every side effect the proper path has — events, audit, cache invalidation, fan-out. | | **Every schema change is a migration** | Numbered, in the module's own language, compiled with it. Both a baseline for a fresh installation *and* an incremental migration for installations that already exist. If code references a column, the migration creating it must exist. [ADR 0006](../adr/0006-schema-changes-are-numbered-migrations.md) | | **Never bypass the pipeline** | No manual database edit, no manual restart as a workaround. Fix the cause and deploy. A workaround that works is a workaround that is never removed, and the next person cannot tell the node from its declaration. | -| **Never create a symlink** | The installer owns all linking and reconciles it. A hand-made link caused production data loss through container volume resolution — and the judgement needed to make a safe exception is exactly the judgement unavailable at the moment it matters. [ADR 0011](../adr/0011-the-installer-owns-linking.md) | +| **Never create a symlink** | A hand-made link caused production data loss through container volume resolution, and the judgement needed to make a safe exception is exactly the judgement unavailable at the moment it matters. Today the installer owns and reconciles the links the mesh still uses [ADR 0011](../adr/0011-the-installer-owns-linking.md); the intent is that the mesh creates none at all [ADR 0018](../adr/0018-the-mesh-creates-no-symlinks.md). Neither reading permits you to make one. | | **Never push directly to the main branch** | Branch, push, review, merge. Every merge is a human checkpoint, without exception. | | **One change per pull request, and never merge your own** | Unrelated improvements bundled together cannot be reviewed or reverted separately. Self-merging removes the checkpoint that is the entire point. | | **Never open a pull request unprompted** | A permissions list saying it is allowed is not a request. | diff --git a/00-GENESIS/process/01-research.md b/00-GENESIS/process/01-research.md index 9a5b182..bfb33ad 100644 --- a/00-GENESIS/process/01-research.md +++ b/00-GENESIS/process/01-research.md @@ -9,7 +9,7 @@ design. Also: an as-is document that raises a question nobody can answer. 1. Take the next free number: highest `01-RESEARCH/NNN-*` plus one, zero-padded to three digits. Never skip or reuse a number. -2. Create `01-RESEARCH/NNN-descriptive-name/` and a `status.md` in it carrying: +2. Create `01-RESEARCH/NNN-descriptive-name/` and a `00-overview.md` in it carrying: ```yaml --- @@ -22,7 +22,7 @@ design. Also: an as-is document that raises a question nobody can answer. Then a short prose summary: **what** is being investigated, **why**, and **what it touches**. 3. Do the research in further documents in the same folder — notes, option analyses, evidence, - draft designs. Anything goes. Keep the summary in `status.md` current as the effort changes + draft designs. Anything goes. Keep the summary in `00-overview.md` current as the effort changes shape. ## What makes research worth reading @@ -37,7 +37,7 @@ carries the whole lesson without naming anything. ## Do not -- Do not put status in prose. It lives in `status.md`'s frontmatter, and the prose must not restate it. +- Do not put status in prose. It lives in `00-overview.md`'s frontmatter, and the prose must not restate it. - Do not set `became:` while the effort is open — playbook 02 sets it at closure. - Do not write into `02-DESIGN` from an open effort. diff --git a/00-GENESIS/process/02-graduation.md b/00-GENESIS/process/02-graduation.md index c0aadf7..62013e8 100644 --- a/00-GENESIS/process/02-graduation.md +++ b/00-GENESIS/process/02-graduation.md @@ -25,7 +25,7 @@ --- ``` -4. **Close the effort.** Set the effort's `status.md` frontmatter to `status: graduated` and +4. **Close the effort.** Set the effort's `00-overview.md` frontmatter to `status: graduated` and `became:` pointing at the design document and the decision record. 5. **Add a ledger line.** Append the decision to [`DECISIONS.md`](../../DECISIONS.md) under today's heading, pointing at the record. diff --git a/01-RESEARCH/001-module-domain-decomposition/status.md b/01-RESEARCH/001-module-domain-decomposition/00-overview.md similarity index 99% rename from 01-RESEARCH/001-module-domain-decomposition/status.md rename to 01-RESEARCH/001-module-domain-decomposition/00-overview.md index ddae694..2ba8f4b 100644 --- a/01-RESEARCH/001-module-domain-decomposition/status.md +++ b/01-RESEARCH/001-module-domain-decomposition/00-overview.md @@ -1,5 +1,5 @@ --- -status: ongoing +status: active initiated: 2026-08-22 touches: [02-DESIGN/00-as-is/02-modules-and-manifests.md, 02-DESIGN/00-as-is/10-module-catalogue.md, 02-DESIGN/01-to-be/00-work-breakdown.md] became: [adr/0015-mesh-brokers-nodes-host-agents-think.md] diff --git a/01-RESEARCH/002-local-mesh/status.md b/01-RESEARCH/002-local-mesh/00-overview.md similarity index 100% rename from 01-RESEARCH/002-local-mesh/status.md rename to 01-RESEARCH/002-local-mesh/00-overview.md diff --git a/01-RESEARCH/003-service-supervision/status.md b/01-RESEARCH/003-service-supervision/00-overview.md similarity index 99% rename from 01-RESEARCH/003-service-supervision/status.md rename to 01-RESEARCH/003-service-supervision/00-overview.md index cf75b1b..395815e 100644 --- a/01-RESEARCH/003-service-supervision/status.md +++ b/01-RESEARCH/003-service-supervision/00-overview.md @@ -1,5 +1,5 @@ --- -status: ongoing +status: active initiated: 2026-08-22 touches: [02-DESIGN/00-as-is/05-runtime-and-installation.md] became: [] diff --git a/01-RESEARCH/004-lab-network/status.md b/01-RESEARCH/004-lab-network/00-overview.md similarity index 99% rename from 01-RESEARCH/004-lab-network/status.md rename to 01-RESEARCH/004-lab-network/00-overview.md index b6e6216..26d8169 100644 --- a/01-RESEARCH/004-lab-network/status.md +++ b/01-RESEARCH/004-lab-network/00-overview.md @@ -1,5 +1,5 @@ --- -status: ongoing +status: active initiated: 2026-08-22 touches: [02-DESIGN/00-as-is/01-mesh-and-transport.md, 02-DESIGN/01-to-be/01-end-to-end-testing.md] became: [adr/0016-a-lab-node-is-a-virtual-machine.md] diff --git a/01-RESEARCH/README.md b/01-RESEARCH/README.md index 193c66f..11adf98 100644 --- a/01-RESEARCH/README.md +++ b/01-RESEARCH/README.md @@ -4,12 +4,12 @@ Investigations that have not yet hardened into design. ## Structure -Each effort lives in `NNN-descriptive-name/` and **must** contain `status.md`, carrying its +Each effort lives in `NNN-descriptive-name/` and **must** contain `00-overview.md`, carrying its state in YAML frontmatter and a prose summary below it: ```yaml --- -status: ongoing | graduated | abandoned +status: active | graduated | abandoned initiated: YYYY-MM-DD touches: [] # design docs, subsystems or areas the effort bears on became: [] # required when status is terminal — what it turned into @@ -26,7 +26,7 @@ draft designs. | status | Meaning | |---|---| -| `ongoing` | Investigation in progress. | +| `active` | Investigation in progress. | | `graduated` | Checked against `00-GENESIS`, decided in `adr/`, and specified in `02-DESIGN` — see `became:`. | | `abandoned` | Stopped or superseded. Nothing is deleted. | diff --git a/02-DESIGN/00-as-is/05-runtime-and-installation.md b/02-DESIGN/00-as-is/05-runtime-and-installation.md index 08b67ac..5ad0c8c 100644 --- a/02-DESIGN/00-as-is/05-runtime-and-installation.md +++ b/02-DESIGN/00-as-is/05-runtime-and-installation.md @@ -62,6 +62,11 @@ missing link is created, a stale one repointed, and a real file found where a li adopted into the node's override area and replaced. Nothing else — not a hook, not a fix, not a person debugging — creates one. +That is the as-is. The intent is to remove linking altogether and derive a real file instead, +which the reconciliation machinery already makes possible +([ADR 0018](../../adr/0018-the-mesh-creates-no-symlinks.md), proposed). What is described above +is what runs today. + ## Supervision Services run under the host's init system via a templated unit, one instance per module. It is diff --git a/DECISIONS.md b/DECISIONS.md index bf52f14..f14a11b 100644 --- a/DECISIONS.md +++ b/DECISIONS.md @@ -28,8 +28,8 @@ no pointer is one small enough that this line is the whole record. | # | Decision | Decided | Where | |---|---|---|---| -| 6 | Phase 0 is a **development environment**, not a fixture rig — it is what the host-borrowing dev tooling becomes. | jochen | [002](01-RESEARCH/002-local-mesh/status.md) | -| 7 | The delivery trigger is a **real forge** inside the lab, not a synthetic event — the webhook relay is part of what is under test. | jochen | [002](01-RESEARCH/002-local-mesh/status.md) | +| 6 | Phase 0 is a **development environment**, not a fixture rig — it is what the host-borrowing dev tooling becomes. | jochen | [002](01-RESEARCH/002-local-mesh/00-overview.md) | +| 7 | The delivery trigger is a **real forge** inside the lab, not a synthetic event — the webhook relay is part of what is under test. | jochen | [002](01-RESEARCH/002-local-mesh/00-overview.md) | | 8 | ~~A lab node is a **system container**, promotable to a virtual machine.~~ | jochen | **Superseded by #10** | | 9 | Adoption is a legacy path and is out of scope. Bringing nodes into being is the lab runner's job instead. | jochen | [design](02-DESIGN/01-to-be/01-end-to-end-testing.md) | | 10 | A lab node is a **virtual machine** running the real install. Supersedes #8: the scale argument for system containers was invented rather than required, and a virtual machine dissolves the fidelity question instead of answering it. | jochen | [ADR 0016](adr/0016-a-lab-node-is-a-virtual-machine.md) | @@ -77,7 +77,8 @@ no pointer is one small enough that this line is the whole record. | 35 | **Status lives in frontmatter.** No central status files; cross-cutting views are generated on demand. This file is a ledger of decisions as taken — an index, and the home of decisions too small to warrant a record — and explicitly not a status board. | jochen | [`process/00-overview`](00-GENESIS/process/00-overview.md) | | 36 | The ADR index is **generated, not maintained**. The hand-written one had already drifted after a single addition. | jochen | [`adr/README`](adr/README.md) | | 37 | Modules outside the platform core are grouped **by domain, not by single function**, extending #1. The principle is settled; the domain list deliberately is not. | jochen | [ADR 0017](adr/0017-modules-outside-the-core-are-grouped-by-domain.md) | -| 38 | ~~The overview in the code repository contradicts the symlink rule.~~ Withdrawn as wrong: the rule forbids creating a link by hand; the installer links deliberately. | jochen | [ADR 0011](adr/0011-the-installer-owns-linking.md) | +| 38 | **The mesh creates no symlinks at all** — not by hand, and not by the installer. Centralising who may link narrowed the incident class without closing it, and a derived file is a copy by nature. The position is settled; the staleness mechanism is not. | jochen | [ADR 0018](adr/0018-the-mesh-creates-no-symlinks.md) | +| 39 | Research efforts follow the **`00-overview.md`** convention, with `active` / `graduated` / `abandoned` in frontmatter — the same shape papa-hq uses, so the two repositories read alike. | jochen | [`01-RESEARCH/README`](01-RESEARCH/README.md) | --- diff --git a/adr/0011-the-installer-owns-linking.md b/adr/0011-the-installer-owns-linking.md index 0f8f0a2..1fbad3d 100644 --- a/adr/0011-the-installer-owns-linking.md +++ b/adr/0011-the-installer-owns-linking.md @@ -46,9 +46,10 @@ is exactly the judgement that was not available at the moment it mattered. - Links become reconcilable state rather than incidental filesystem facts. - The rule is stated for humans and agents and is enforced by convention, not mechanism. A check does not exist. -- The rule is regularly misread as "the mesh does not use symlinks", which is false and makes - the design documentation look self-contradictory. It uses them; it centralises who may make - them. +- The rule as written governs the mechanism rather than removing it. A link made by the + installer resolves the same way as one made by hand, so the hazard is narrowed and not + closed. [ADR 0018](0018-the-mesh-creates-no-symlinks.md) proposes widening this to "nothing + links, the installer included"; until that is accepted, this record governs. ## References diff --git a/adr/0018-the-mesh-creates-no-symlinks.md b/adr/0018-the-mesh-creates-no-symlinks.md new file mode 100644 index 0000000..2fc0909 --- /dev/null +++ b/adr/0018-the-mesh-creates-no-symlinks.md @@ -0,0 +1,101 @@ +--- +status: proposed +date: 2026-08-23 +deciders: jochen +reconstructed: false +extends: 0011-the-installer-owns-linking.md +--- + +# 18. The mesh creates no symlinks — a derived file is a copy + +## Context + +[ADR 0011](0011-the-installer-owns-linking.md) responded to production data loss — a hand-made +link, resolved through a container engine's volume handling, pointing a mount somewhere it +should not have — by centralising linking in the installer and forbidding it everywhere else. + +That narrowed the incident class. It did not close it. The hazard is not *who* made the link; +it is that a path can resolve somewhere other than where it appears to. A link made by the +installer resolves exactly the same way as a link made by hand. The rule made the mechanism +rarer and better-governed while leaving the mechanism in place. + +Two things have changed since, and together they remove the argument that kept it. + +**The original case for linking was staleness.** A copy of a service definition goes stale +silently while the catalogue moves on, so a link was the cheap way to guarantee the running +node reads a current definition. That argument assumes the node's copy is unmanaged. + +**It is not.** [ADR 0004](0004-managed-files-are-generated-never-edited.md) established that +everything on a node's disk is derived from the mesh and regenerated when its inputs change, +and the installer already **reconciles** links rather than assuming them — repointing stale +ones, adopting real files it finds where a link belongs. Reconciling content is the same +operation as reconciling a pointer, plus a comparison. + +So the mesh already has the machinery that makes a copy safe, and is using a link to solve a +problem that machinery solves better. Worse, a link is conceptually the wrong shape: it makes +the node's runtime state a *pointer into source*, which is the one thing +[ADR 0003](0003-the-mesh-database-is-the-source-of-truth.md) and ADR 0004 exist to prevent. +State is derived onto nodes; it does not reach back. + +## Considered options + +1. **Keep ADR 0011 as the final position** — centralised linking, forbidden elsewhere. + Rejected as the status quo. It governs the mechanism rather than removing it, and the + failure it was written for remains reachable by any code path the installer trusts. +2. **Keep links but harden them** — canonicalise before mounting, refuse a link that escapes + an expected root. Rejected: it is a check bolted onto a hazard, and it has to be correct in + every consumer, including container engines the mesh does not control. +3. **Copy, reconciled by the installer, with staleness detected rather than assumed away.** + Proposed here. + +## Decision + +*Proposed — the position is settled; the migration is not designed. See "Open" below.* + +**The mesh creates no symlinks.** A file a node needs is placed on that node as a real file, +derived from the mesh and reconciled by the installer like every other managed file +([ADR 0004](0004-managed-files-are-generated-never-edited.md)). + +The prohibition in ADR 0011 stands and widens: it ceases to be "only the installer may link" +and becomes "nothing links, the installer included". + +When this is accepted, ADR 0011 becomes superseded rather than edited — its reasoning is why +the rule exists at all, and the incident behind it is the reason anyone believes either record. + +## Consequences + +- The path-resolution hazard is removed rather than governed. There is no link for a container + engine to resolve, so the class of failure that cost production data is closed by + construction. +- A node's runtime state stops pointing into source. What a node holds is derived output, which + is what the mesh's model already says it is everywhere else. +- **Staleness becomes a real problem that must be answered, not assumed away.** This is the + cost, and it is the whole cost: today a link cannot be stale, and a copy can. The answer has + to be detection — the installer comparing what is on disk against what the mesh says should + be — and it must be loud, because a silently stale definition is exactly the failure shape + this mesh keeps producing ([ADR 0008](0008-a-failed-step-fails-the-job.md)). +- Reconciliation gets more expensive: comparing content rather than checking a pointer's + target, on every module, on every node. +- Disk usage rises, trivially, and is not a consideration. +- Existing links must be converted. A node mid-migration holds both forms, so reconciliation + has to handle finding a link where a file now belongs — the mirror image of the adoption it + already does. + +## Open + +- **How staleness is detected.** Content hash, version marker, or regeneration on every + reconcile. This is the decision that makes or breaks the change and it is not taken here. +- **Whether anything must keep a link** for reasons outside the mesh's control. If something + does, that is a finding worth recording rather than an exception worth granting quietly. +- **Migration order.** Converting a node's links is a change to how its services resolve their + own definitions, which is not a change to make everywhere at once. + +Until those are answered this record stays `proposed`, and ADR 0011 remains the governing rule. + +## References + +- [ADR 0011](0011-the-installer-owns-linking.md) — the incident, and the rule this widens. +- [ADR 0004](0004-managed-files-are-generated-never-edited.md) — the machinery that makes a + copy safe. +- [`02-DESIGN/00-as-is/05-runtime-and-installation.md`](../02-DESIGN/00-as-is/05-runtime-and-installation.md) + — what the installer does today, including reconciliation and adoption. diff --git a/adr/README.md b/adr/README.md index 4cdd546..441d744 100644 --- a/adr/README.md +++ b/adr/README.md @@ -36,7 +36,7 @@ outranks *"the dependency rule is not followed"*. ## Reconstructed records -Records 0001–0014 were written on 2026-08-23, after the decisions they describe. Those +Records 0001–0014 were written on 2026-08-23, after the decisions they describe. Records 0015 onward were taken as records. Those decisions were taken in implementation rather than in a document; the records state what was decided and the evidence it was decided from, and each carries `reconstructed: true` and says so in its first lines.