From cb1954e7b5c9e090326961da9692cfbc8ebf8c75 Mon Sep 17 00:00:00 2001 From: jochen Date: Mon, 31 Aug 2026 17:25:27 +0200 Subject: [PATCH] Accept 0024, and rewrite the work breakdown around what is actually being done MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit **0024 accepted.** Model access was decided, built, and proven in the lab, and two design documents rest on it; only the status had never moved. The gate is green again. **The work breakdown rewritten.** It planned a decomposition of the existing system in place — extract contexts, declared features, shrink the shared library. That is not the work. A replacement is being built beside it, and only the old Phase 0 survived contact with reality, so the one document meant to say what happens next was describing a system being retired. Now ordered by what "modules move across one at a time until the old registry is off" actually requires: - Phase 0 is marked done against the twenty-two lab assertions, **and carries its own limitation**: every module exercised was written to test the mechanism, so the vocabulary was shaped by its own fixtures. - Phase 1 is the vocabulary gaps found by asking what real modules need — an object-store provision, a session as a licence consumer, a network shape with ordering, public certificate issuance. - Phase 2 is one module, then a week of running it, because the point of going first is to find what Phase 1 missed. - Phase 3 picks modules that each prove something the first did not; the mail system is last because it is the one that may send work back into the declaration language. - Phase 4 is switching the registry off, named as a phase so it is not mistaken for the goal. Keeps the rules of engagement unchanged — they were about how work is done, not what it is — with one addition: stop and ask before anything that touches a machine outside the lab. Adds a section on keeping the list true, since the document it replaces was wrong for weeks and nothing said so. A claim here is counted, not reasoned, and a phase is done when the lab says so. --- .../0024-model-access-is-a-provision.md | 2 +- 02-DECISIONS/README.md | 2 +- 03-DESIGN/01-to-be/00-work-breakdown.md | 275 +++++++++--------- 03-DESIGN/01-to-be/01-end-to-end-testing.md | 5 +- 03-DESIGN/01-to-be/README.md | 2 +- 5 files changed, 152 insertions(+), 134 deletions(-) diff --git a/02-DECISIONS/0024-model-access-is-a-provision.md b/02-DECISIONS/0024-model-access-is-a-provision.md index eb177ea..c40bbb7 100644 --- a/02-DECISIONS/0024-model-access-is-a-provision.md +++ b/02-DECISIONS/0024-model-access-is-a-provision.md @@ -1,6 +1,6 @@ --- topic: what runs on it -status: proposed +status: accepted date: 2026-08-30 deciders: jochen reconstructed: false diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index ae17da5..889c264 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -98,7 +98,7 @@ python3 00-META/checks/index.py fail if stale - **0009** — [Modules and the graph](0009-modules-and-the-graph.md) - **0010** — [Delivery](0010-delivery.md) -- **0024** — [Model access is a provision, and a licence is a thing with a name](0024-model-access-is-a-provision.md) *(proposed)* +- **0024** — [Model access is a provision, and a licence is a thing with a name](0024-model-access-is-a-provision.md) - **0026** — [The mesh has a session of its own, and it is the node session's mechanism](0026-the-mesh-has-a-session-of-its-own.md) - **0027** — [A provision names what the consumer is coupled to, not the role it plays](0027-a-provision-names-what-the-consumer-is-coupled-to.md) diff --git a/03-DESIGN/01-to-be/00-work-breakdown.md b/03-DESIGN/01-to-be/00-work-breakdown.md index 3812141..7d5c8dc 100644 --- a/03-DESIGN/01-to-be/00-work-breakdown.md +++ b/03-DESIGN/01-to-be/00-work-breakdown.md @@ -1,158 +1,173 @@ --- layer: to-be status: designed -code: [hal] -updated: 2026-08-29 -decisions: [02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md] +code: [] +updated: 2026-08-31 +decisions: + - 02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md + - 02-DECISIONS/0005-the-node-host.md + - 02-DECISIONS/0009-modules-and-the-graph.md + - 02-DECISIONS/0016-the-lab.md + - 02-DECISIONS/0028-the-substrate-supplies-the-control-plane-and-nothing-else.md --- -# Work breakdown — the decomposition +# Work breakdown — replacing what provisions the mesh -How [ADR 0001](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md) gets built, in what order, and where a human must look. +*Rewritten 2026-08-31. The previous version planned a decomposition of the existing system in +place: extract contexts, convert modules to declared features, shrink its shared library. That is +not what is being done — a replacement is being built beside it, and the old plan's Phase 0 was +the only part that survived contact with it. So the document that was supposed to say what happens +next had been describing work on a system being retired.* -Ordering is not preference. Each phase removes a constraint the next one needs gone. +## The goal, in one sentence ---- +**Modules move to the new mesh one at a time, until the old registry can be switched off.** + +Everything below is ordered by what that requires. Nothing here is a rewrite of the old system; +its modules are the input. + +## Phase 0 — a mesh that runs — **done** + +Not *the code exists*. Twenty-two assertions on real machines in the lab, each confirmed to fail +when the behaviour is removed ([ADR 0016](../../02-DECISIONS/0016-the-lab.md), +[ADR 0017](../../02-DECISIONS/0017-a-test-defends-a-decision.md)). + +| what is proven | | +|---|---| +| **a mesh comes into being** | a bare machine becomes one; others join with nothing but a token | +| **credentials** | delivered to both ends with the mesh holding neither; rotated so the old one stops working | +| **declarations survive reality** | a stopped machine is waited for; one that fell behind catches up unnamed; unassigning takes away exactly what it should; what the mesh says nothing about is left alone | +| **failure is legible** | a machine that cannot do what it was told is named, with why | +| **the mesh runs itself** | its own artifact store, and a builder that is a module the mesh assigns | +| **names and reachability** | internal names, wildcards under a machine, containers reaching other machines, certificates the mesh issued, filtering that matches exactly what was declared | +| **delivery** | a new commit reaches a machine already running the old one | +| **model access** | answered by a record, with a key the mesh cannot read | + +**What Phase 0 does not prove, and it is the important sentence in this document:** every module +exercised above was written to test the mechanism. **No module from the existing system has ever +run on this.** The vocabulary was shaped by the things used to test it — the same fault as a +fixture agreeing with the code it checks +([`04-ISSUES/005`](../../04-ISSUES/005-pipeline-test-harness-unbuildable/00-report.md)), at the +scale of a design. + +## Phase 1 — the vocabulary a real module needs + +Found by taking real modules and asking what they would require. Each is a gap in what can be +*expressed*, not a defect in what is built. + +| # | task | done when | +|---|---|---| +| 1.1 | An **object-store provision** — a module can ask for a bucket ([ADR 0028](../../02-DECISIONS/0028-the-substrate-supplies-the-control-plane-and-nothing-else.md)) | a module requiring it is refused where nothing provides it, and given credentials where something does | +| 1.2 | **A session as a consumer of a licence** | the two sessions on one machine hold different licences and each uses its own ([`14-model-access.md`](14-model-access.md), [ADR 0026](../../02-DECISIONS/0026-the-mesh-has-a-session-of-its-own.md)) | +| 1.3 | A **network** shape, and ordering within a module | a module of several containers reaches itself, and one that must start after another does | +| 1.4 | **Public certificate issuance** | a name reachable from outside is served with a certificate from a public authority, obtained against a **staging** endpoint unless told otherwise ([`04-ISSUES/004`](../../04-ISSUES/004-certificate-issuance-targets-production/00-report.md)) | + +**1.1 blocks the first module; 1.3 and 1.4 block later ones** and are listed now so they are not +met as surprises. 1.3 is what a mail system needs and nothing else so far does. + +**Checkpoint:** each is demonstrated in the lab before the module needing it is attempted. + +## Phase 2 — the first real module + +| # | task | done when | +|---|---|---| +| 2.1 | Port an **object store** module | it runs on the new mesh, serves a bucket to another module, and its credential rotates | +| 2.2 | Run it beside the existing one | both exist; nothing depends on the new one yet | +| 2.3 | Move one dependent onto it | something real reads and writes through the new mesh's copy | + +**Checkpoint, and it is a human one:** it runs for a week before anything else moves. The point of +going first is to find what Phase 1 missed, and a week is roughly how long that takes to show. + +## Phase 3 — the modules that prove the shape + +Each exercises something the first one does not. + +| # | task | proves | +|---|---|---| +| 3.1 | An **identity provider** | a module that is itself a provider — the provides/requires chain, with consumers requiring it | +| 3.2 | A **forge** | a port claim against the machine's own daemon, and a module wanting both a database and an object store | +| 3.3 | A **mail system** | several containers as one module, a private network between them, and names that are not one-per-node | + +**3.3 is the hardest thing in this document** and is deliberately last. If the declaration +language turns out to be insufficient, it says so here. + +## Phase 4 — switch the old registry off + +| # | task | done when | +|---|---|---| +| 4.1 | Move the remainder | nothing is assigned in the old system that is not assigned in the new one | +| 4.2 | Run in parallel, the old one authoritative for nothing | a change to any module goes through the new mesh only | +| 4.3 | Switch it off | it is stopped, and nothing notices | + +**4.3 is a day's work and the phases above it are not.** Naming it as a phase is what stops it +being mistaken for the goal. + +## Sequencing + +- **1 before 2.** Attempting a module without the vocabulary it needs produces a workaround, and a + workaround in a manifest is a design decision taken by whoever was in a hurry. +- **2 before 3, with the week.** Moving three modules before running one is how three modules + acquire the same defect. +- **3.3 last.** It is the only one that may send work back into the declaration language. +- **4 cannot start early, and there is no partial credit.** A registry still authoritative for one + module is still running. + +## How this list is kept true + +*This section exists because the document it replaces was wrong for weeks and nothing said so.* + +**A claim here is counted, not reasoned.** The review of 2026-08-31 found a bundle described as +carrying two images that carries three, a bootstrap described as needing six shapes that uses +four, and ten documents calling themselves `designed` while naming lab-proven code. Each was +produced by describing the system from its design instead of reading it. + +**A phase is done when the lab says so**, and the lab keeps a receipt of when it last ran and +against which commits. A phase marked done here whose assertions have not run is a claim about the +past. + +**What is not proven gets said.** Phase 0 is done and its limitation is written into it. A list +that records only progress becomes a list nobody believes. ## Rules of engagement -These exist so the work can run largely unattended without accumulating the kind of -damage this refactor is meant to remove. +Unchanged from the previous version: they were about how work is done rather than what the work +is. ### Autonomous by default -An agent may, without asking: - -- read anything, measure anything, query any database read-only -- create branches, write code and tests, open pull requests -- run the test suite and typechecks -- write and update `hq/` documents +Read anything, measure anything, query read-only. Create branches, write code and tests, run the +suites, and write or update documents here. ### Always stop and ask -- **destroying or overwriting data** — dropping a table, deleting a provision, rotating a - live credential, removing a module from a node +- **destroying or overwriting data** — dropping a table, deleting a provision, rotating a live + credential, removing a module from a node - **merging anything** — every merge is a human checkpoint, without exception -- **a decision the ADRs do not already answer** — record the question in the relevant - research effort rather than picking and moving on -- **any change to `hq/00-META`** — it is stable by nature +- **anything touching a machine outside the lab**, including a configuration change that restarts + something people are using +- **a decision the records do not already answer** — record the question rather than picking and + moving on +- **any change to [`00-META`](../../00-META/)** — it is stable by nature ### Definition of done for every task 1. tests written **and failing first**, then passing 2. typecheck clean in every package the change touches -3. the local mesh (Phase 0) comes up, and the behaviour is demonstrated in it -4. `hq/` updated if the task changed or answered anything documented -5. deployed, and **delivery verified on every node** — not "the pipeline was green" +3. the behaviour demonstrated **in the lab, on real machines** — not asserted +4. documents here updated if the task changed or answered anything recorded +5. delivered, and the **effect** verified — not that a pipeline was green -### Non-negotiables carried from the current system +### Non-negotiables -- **Never edit mesh-managed files on disk.** Use the owning tool. -- **Never write to production databases directly.** Migrations for schema, application - code for data. -- **Every schema change ships twice** — consolidated schema *and* an incremental - migration. -- **Expand, then contract.** Add the new shape, migrate, verify, and only then remove the - old one — never in a single step. -- **A green pipeline proves transport, not effect.** Verify the effect. - ---- - -## Phase 0 — A mesh that runs locally *(prerequisite)* - -Nothing else starts until this exists. Every fault this refactor addresses was found in -production because there was nowhere else to find it. - -| # | task | done when | -|---|---|---| -| 0.1 | Container image for a node runtime | a node process starts in a container and registers | -| 0.2 | Compose topology: broker, registry DB, object store, *n* nodes | `up` yields a mesh that elects a provider node and settles | -| 0.3 | Seed a minimal mesh: nodes, one module, one provision | a module deploys end-to-end with no external service | -| 0.4 | Run the pipeline inside it | a push-equivalent produces a cascade and a deployed artifact | -| 0.5 | Fixtures for the failure modes already known | credential rotation reaching a running session; a provider deploy rotating a shared credential; a migration that ships nothing — each reproducible on demand | - -**Checkpoint:** a human confirms the local mesh reproduces at least one bug from -2026-08-22 before any decomposition begins. - ---- - -## Phase 1 — Make the model expressible - -The decomposition is impossible while a feature is a singleton per module. - -| # | task | done when | -|---|---|---| -| 1.1 | Decision record — named features, per-node opt-in (next free number) | accepted | -| 1.2 | Manifest: declared `features:` with type + directory | a module declares two of one kind and both build | -| 1.3 | Selection: `always` / `optional` | a node installs a subset; artifacts stay selection-blind | -| 1.4 | `requires:` moves onto the feature | a schema feature's database is not provisioned where the feature is not installed | -| 1.5 | Assignment carries the opted-in feature set | opting a node in requires no rebuild | - -**Checkpoint:** one existing module converted to declared features, deployed, verified — -before any others follow. - ---- - -## Phase 2 — Draw the boundary the domain already has - -Cheapest first, and each one proves the extraction pattern before the expensive ones. - -| # | task | extracted from | risk | -|---|---|---|---| -| 2.1 | `hal/knowledge` — one store, review workflow ported | hippocampus + noxflow `knowledge_*` | low — additive | -| 2.2 | `hal/stream` — the record; notifications and messaging as views | axon, synapse, notifications, meetings, conversations | medium | -| 2.3 | `hal/agents` — identity, licence, runs, memory, thoughts | noxflow agents, `hal/thoughts` | **high** — touches credentials | -| 2.4 | `hal/work` — what remains of noxflow | noxflow tasks | medium | -| 2.5 | `hal/ai` — one module per provider, each providing *a model provider* | `hal/claude*` | medium | - -Each extraction is expand-then-contract: new context alongside, dual-write, verify, cut -over, remove. **Never a move commit.** - -**Checkpoint:** after 2.1, a human confirms the extraction pattern before 2.2 begins. -After 2.3, a human confirms credentials still reach every agent on every node. - ---- - -## Phase 3 — Reclaim the kernel - -Only possible once domains have modules to own their code. - -| # | task | done when | -|---|---|---| -| 3.1 | Move work-domain code out of `hal/sdk` | `workflow-engine.ts`, `task-commands.ts` live in `hal/work` | -| 3.2 | Move provider code out | `claude-credentials.ts` lives in `hal/ai` | -| 3.3 | Move delivery code out | feature handlers, artifact manager, build executor live in `hal/delivery` | -| 3.4 | Decide the residue | ADR: what `hal/sdk` keeps (open question 4) | - -**Measure:** `hal/sdk` line count, tracked per task. Today: **34,636** across **155** -files. - ---- - -## Phase 4 — Separate what the mesh runs from the mesh - -| # | task | done when | -|---|---|---| -| 4.1 | Decide the destination (open question 3) | ADR accepted | -| 4.2 | Cross-repository dependency resolution proven | a catalogue module builds against a published `@hal/*` | -| 4.3 | Move the 91 catalogue modules | this repository contains only mesh contexts | - -**Checkpoint:** move one application first and run it for a week before the rest follow. - ---- - -## Sequencing constraints - -- **0 before everything.** Unverifiable refactors are how this list got long. -- **1 before 2.** Extracting into contexts without per-node features recreates the module - count inside the new names. -- **2 before 3.** A domain can only own its shared code once the domain has a module. -- **2.3 after 2.1 and 2.2.** Agents touch credentials; do it once the pattern is proven on - cheaper contexts. -- **4 last.** It is the only phase that is pure movement, so it is the only one safe to - defer indefinitely. +- **Never edit mesh-managed files on disk.** Use the thing that owns the file. +- **Never write to a production database directly.** Migrations for schema, application code for + data. +- **Every schema change ships twice** — consolidated schema *and* an incremental migration. +- **Expand, then contract.** Add the new shape, migrate, verify, and only then remove the old one. +- **A green pipeline proves transport, not effect.** ## What "done" looks like -Eight contexts. `hal/sdk` holding only what is genuinely cross-cutting. A mesh that stands -up on a laptop. A module count that grows only when the domain does. +The old registry is off. Every module runs on the new mesh, declared rather than scripted. A +machine that fails says what it could not do. And the number of modules grows when the work does, +not when the platform needs somewhere to put something. diff --git a/03-DESIGN/01-to-be/01-end-to-end-testing.md b/03-DESIGN/01-to-be/01-end-to-end-testing.md index a7e0574..967c23a 100644 --- a/03-DESIGN/01-to-be/01-end-to-end-testing.md +++ b/03-DESIGN/01-to-be/01-end-to-end-testing.md @@ -336,7 +336,10 @@ credential rotation reaches every consumer, that delivery to an absent node is r pending rather than done, that a returning node catches up. These are fewer and change rarely, but they are where the known production faults get encoded so they stay fixed. -The known faults become mesh tests that fail today. That is the Phase 0 checkpoint. +The known faults become mesh tests that fail today. Phase 0 of +[`00-work-breakdown.md`](00-work-breakdown.md) is now complete on this basis — twenty-two +assertions on real machines — and what it does **not** cover is recorded there: no module +from the existing system has run against any of it yet. --- diff --git a/03-DESIGN/01-to-be/README.md b/03-DESIGN/01-to-be/README.md index 395486e..cab4a7b 100644 --- a/03-DESIGN/01-to-be/README.md +++ b/03-DESIGN/01-to-be/README.md @@ -9,7 +9,7 @@ document is written and this one's status becomes `implemented`. | Document | Covers | Rests on | |---|---|---| -| [`00-work-breakdown.md`](00-work-breakdown.md) | How the decomposition gets built, in what order, and where a human must look | [ADR 0001](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md) | +| [`00-work-breakdown.md`](00-work-breakdown.md) | How modules move across one at a time, until the old registry can be switched off | [ADR 0001](../../02-DECISIONS/0001-mesh-brokers-nodes-host-agents-think.md), [ADR 0016](../../02-DECISIONS/0016-the-lab.md) | | [`01-end-to-end-testing.md`](01-end-to-end-testing.md) | The lab: a real mesh a change can be run against before it reaches nodes | [ADR 0016](../../02-DECISIONS/0016-the-lab.md), [0029](../../02-DECISIONS/0016-the-lab.md) | | [`02-scenario-declaration.md`](02-scenario-declaration.md) | What a scenario declares — the underlay, and what to place on it | [ADR 0016](../../02-DECISIONS/0016-the-lab.md) | | [`03-scenario-lifecycle.md`](03-scenario-lifecycle.md) | What happens to a scenario — raise, snapshot, restore, move, destroy | [ADR 0016](../../02-DECISIONS/0016-the-lab.md) |