From 6738de67e30bf981ab1bd14d0c6b9df2303e5cce Mon Sep 17 00:00:00 2001 From: jochen Date: Thu, 12 Nov 2026 10:50:52 +0100 Subject: [PATCH 1/3] Design 39 bus-and-switch: C4, room and the switch read the weekly window alone --- 03-DESIGN/01-to-be/39-milestones/bus-and-switch.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/03-DESIGN/01-to-be/39-milestones/bus-and-switch.md b/03-DESIGN/01-to-be/39-milestones/bus-and-switch.md index 7e271802..4ab7625d 100644 --- a/03-DESIGN/01-to-be/39-milestones/bus-and-switch.md +++ b/03-DESIGN/01-to-be/39-milestones/bus-and-switch.md @@ -549,3 +549,7 @@ written; they are cited as "design 39 bus-and-switch B*n*". - **C3** (met live after C1: the manager, now in its own runtime, was still issued none of its logs): the two logs `usage` and `audit` are declared in the manifest's `logs` section, the form the runtime on main issues today; the states stay templates. The move of logs to templates is design 47 milestone G's (P7). +- **C4** (the operator, 2026-11-12: "the session limit has been deprecated by anthropic"; met live: both licences read + `sessionPct` empty, so neither had room and the switch could choose no target): a licence's room is 100 less its + weekly percentage alone; the session window takes no part in room, the switch threshold, the warning or the + conditions, wherever §24 to §33 name it. A reading with no weekly percentage still has unknown room (B35). From 658d33892c5e009f657407f69644b7c80ced25f1 Mon Sep 17 00:00:00 2001 From: jochen Date: Thu, 12 Nov 2026 11:36:48 +0100 Subject: [PATCH 2/3] Write design 20 and playbook 06 from what the manifest validator accepts The module-development documents described a manifest the mesh no longer reads: four languages where the builder builds two, a Go bundle with no system, a scheduled process the guide never said how to write, the old kept-data sections as the norm, and the controller as the one refusing. They now name each part of a module and its manifest form as the manifest validator, the builder and the node-engine read it, and the playbook lists where those parts still disagree, so an author writes the form every part takes. --- 00-META/glossary.md | 34 ++- 00-META/process/06-writing-a-module.md | 52 +++- 00-META/process/playbooks.html | 2 +- 03-DESIGN/01-to-be/20-writing-a-module.md | 316 +++++++++++++--------- 03-DESIGN/01-to-be/README.md | 2 +- 5 files changed, 254 insertions(+), 152 deletions(-) diff --git a/00-META/glossary.md b/00-META/glossary.md index 8cf80298..a454bc47 100644 --- a/00-META/glossary.md +++ b/00-META/glossary.md @@ -89,8 +89,8 @@ is listed under [Homonyms](#homonyms), with the qualified form each domain uses. **Purpose.** To say what one module is, in the one document every other domain reads. **Recorded in** the catalog, one manifest per module. **Upstream of** every other domain; its words -are a published vocabulary, fixed by the manifest's schema. **Decided by** ADR 0040, 0126, 0174, 0188, -0207, 0236, 0262, 0634, 0635. +are a published vocabulary, fixed by the manifest's schema and judged by the manifest validator. **Decided by** ADR +0040, 0126, 0174, 0188, 0207, 0236, 0262, 0578, 0595, 0634, 0635. **Uses:** module, seat, config value (Config), provision (Provisioning), data class (Data), health (Health and repair). @@ -100,9 +100,32 @@ and repair). and never a pipeline step's status, which is the registration's policy (ADR 0666 §7, §14). Say manifest in prose, not the file's name. *Not:* ~~module definition~~ (hq) -- **resource** — one thing a manifest declares a node must have: a file, a directory, a service, a - container, a package, a bundle, an archive. A module **declares** resources; the verb *declares* is +- **resource** — one thing a manifest declares a node must have, by its `type`: a `directory`, a `file`, a + `service`, a `package`, a `container`, a `user`, an `archive`, a `network`, an `access`, a `process` or an + `opening`; never an `action` (ADR 0005). Its fields are the node-engine's vocabulary (mesh-sdk's declaration + package), which refuses a field its type does not use. A module **declares** resources; the verb *declares* is this domain's and means what a manifest states. +- **process resource** — a resource of type `process`: a bundle of the module's own, named by `artifact`, fetched by + digest and run by the node-engine as a unit with the command `run`, in one of three modes: one that stays up, a + **run-once step** or a **scheduled run**. The way a module runs its own code without an image (ADR 0052, ADR 0053). + *Not:* ~~daemon resource~~ (hq) +- **run-once step** — a `process` or `container` resource with `run-once` true, run to completion by the node-engine + before every resource after it, which it gates; with `restart-on` it runs again when what it names changes (ADR + 0052, ADR 0099). Not a migration of the module's own state, which is `prepares` (ADR 0135). + *Not:* ~~init step~~ (hq) +- **scheduled run** — a `process` or `container` resource with `schedule`, a five-field cron, run again on each fire + and never left running; a scheduled container may hold its own module's containers still while it runs + (`while-stopped`, ADR 0053, ADR 0189). + *Not:* ~~cron job~~ (hq) +- **runtime of its own** — the runtime that serves the tools bundle of a module saying `runs-as`, as that module's + account and on a bus account of its own, never the machine's runtime the tool runner runs as the operator's + account; its readiness is the manifest's `runtime-health` (ADR 0259 §8, ADR 0366). + *Not:* ~~private runtime~~ (hq) +- **old section** — a manifest's `state`, `logs`, `streams` or `objects`: kept data in the form before templates. + The parts the bus module needs before it can answer keep them for good; any other module's use of one is refused + once no accepted module outside those parts uses it any more (ADR 0595 §11, ADR 0619 §1). New kept data is asked + for in `requires` by template. + *Not:* ~~legacy section~~ (hq) - **claim** — a module taking a spot on a seat: `claims: [{name, scope}]` in a manifest. A mesh-scoped exclusive claim is how the mesh says "there is one of me". A foundation seat is named after the role it guards: the `mesh-controller`, `postgres` and `nats` modules claim the `mesh-controller`, @@ -112,7 +135,7 @@ and repair). lines survive every send and are given back when the module goes (ADR 0174). The other way a node varies a module, besides a config value; a **managed** file's marked block is one (Provisioning). - **bundle** — the artifact a module's own code is built into — its tools, a seat's implementation, a - daemon — in any language the mesh has a toolchain for; never an image. A **tools bundle** speaks MCP to + daemon — in any language the mesh has a build module for (Go and TypeScript); never an image. A **tools bundle** speaks MCP to the tool runner. One module may declare several (ADR 0188). - **tool** — one capability a module serves through the tool runner, addressed `.`. A @@ -1446,6 +1469,7 @@ renaming one sense, the old sense moves to a *Not:* line and becomes mechanical. | step | an action run on a module's commit before it is delivered | **pipeline step**, always whole | Delivery | | | the step an acting part acts in (`take-in`, `build`, `compose`, …) | an acting rule's **step** | Delivery | | | a step a person starts, like the bus's | **planned step** | Core | +| | a resource the node-engine runs to completion before what follows it (`run-once`) | **run-once step** | Module | | | a step of a rollout | **tier** | Delivery | | tier | a level of the mesh | **layer** | Core | | | a step of a rollout | **tier** | Delivery | diff --git a/00-META/process/06-writing-a-module.md b/00-META/process/06-writing-a-module.md index cbb5b6a0..80e494f9 100644 --- a/00-META/process/06-writing-a-module.md +++ b/00-META/process/06-writing-a-module.md @@ -33,11 +33,22 @@ Three questions, answered from the machine: `package-manager`, `seat`. A capability is detected and refused against; it is not something a module can install. -3. **Write the resources in the order they must happen.** They are applied in the order written +3. **Choose each part's form, from the table in [design 20](../../03-DESIGN/01-to-be/20-writing-a-module.md) *What a + module is built of*.** Tools are a `bundle` the tool runner loads (`loads`, and every tool in `tools`); a service of + the module's own code is a `process` resource; a scheduled job says `schedule`, a one-time step `run-once`, a + migration of the module's own state `prepares`; a seat's implementation is a claim and the verbs it serves; an image + is an `image` or `upstream` artifact run by a `container`; a program of the operating system is a `package` and its + `service` resources. A bundle or a package is in Go or TypeScript, the languages the builder has build modules for. + +4. **Ask for kept data in `requires`, by template** (`state`, `events`, `work-queue`, `log`, `objects`, `ledger`), each + entry with its `name`, `storage`, `template` and `options`. The old sections `state`, `logs`, `streams` and + `objects` are for the parts the bus module needs before it can answer, and for no new entry anywhere else. + +5. **Write the resources in the order they must happen.** They are applied in the order written and orphans are removed in reverse, so a `network` is written before the containers that join it and removed after them. -4. **Put every secret in a file, never in `env`.** A declaration travels over the broker in plain +6. **Put every secret in a file, never in `env`.** A declaration travels over the broker in plain text: a password in `env` is a password the broker sees. **The mesh delivers parts; a module that needs them combined combines them.** @@ -67,21 +78,39 @@ Three questions, answered from the machine: Add `restart-on` naming the env file, or the container keeps the credential it started with through every rotation. -5. **Pin every image by digest.** A tag moves. The manifest in a repository names artifacts; the - manifest the mesh holds names digests, and they are not the same document. +7. **Pin every image by digest.** A tag moves. An `upstream` artifact's `from`, and a container's `image` written + directly, name `@sha256:`: the builder refuses an upstream reference with a tag alone. The manifest in a repository + names artifacts; the manifest the mesh holds names digests, and they are not the same document. -6. **Decide generate or accept.** A new module's credential is generated. **An adopted one keeps +8. **Decide generate or accept.** A new module's credential is generated. **An adopted one keeps the credential it already has** — `secret accept` — because minting a new password for a database that already exists locks the application out of its own data. -7. **Add a provisioner only if the software cannot read a file.** A proxy that watches a +9. **Add a provisioner only if the software cannot read a file.** A proxy that watches a directory needs nothing. PostgreSQL needs `CREATE ROLE`, an object store needs a bucket and a policy, an identity provider needs a realm and a client — those need a small program beside them. It reads what the mesh granted and reconciles; it does not decide anything. -8. **Prove it in the lab, against the real software.** Not that a container started — that the - thing works: the credential authenticates, a wrong one is refused, the containers reach each - other, the data survives a restart. +10. **Prove it in the lab, against the real software.** Not that a container started — that the + thing works: the credential authenticates, a wrong one is refused, the containers reach each + other, the data survives a restart. + +## Where the parts of the mesh read a manifest differently + +*Written 2026-11-12 from reading the manifest validator, the builder, the node-engine's vocabulary and the tool runner +on their main branches.* The manifest validator accepting a manifest does not yet mean every other part takes it. +Until each of these is closed in the part named, write the form in the right column: + +| the manifest validator accepts | but | so write | +|---|---|---| +| a `bundle` or a `package` in any language word | the builder builds only Go and TypeScript bundles and TypeScript packages, and refuses the rest by name at the build | a bundle in Go or TypeScript | +| an `upstream` artifact whose `from` names a tag alone | the builder refuses an upstream reference with no `@sha256:` | the reference with its digest | +| a `log` asked for in `requires` by template | the tool runner's log calls reach only a log of the old `logs` section; a template log is reached only by its stream's read and fill calls, so a module cannot append to it through the runtime | a template log only where the module's own bus account publishes it; a log its tools append to through the runtime waits for the tool runner's fix | +| any `type` and any field on a resource | the node-engine refuses an unknown type and a field the type does not use, when the module is composed for a machine: in the change's composition step only for the machines the change touches, and for a module assigned nowhere never | each resource's fields as the node-engine's vocabulary has them (mesh-sdk's declaration package) | +| `schedule` and `run-once` on a `process` unjudged | the node-engine refuses a process with both, or a scheduled one with `restart-on`, `replaces` or `witness`, or an invalid cron | a process in one mode | + +How it is checked: each row is closed by a change in the part it names, which deletes the row in the same pull +request; a reviewer of a manifest reads these rows. ## A module that calls or holds a seat, or shares code @@ -124,8 +153,9 @@ time and the scaffolding was not.** ## Rules -- **Read the node-engine's log before theorising.** A declaration that was sent and not applied says so - there and nowhere else. +- **Read the check's verdict, then the node-engine's log, before theorising.** The change's check names every + problem the manifest validator and the composition step found; a declaration that was sent and not applied says + so in the node-engine's journal (`/node-service-manager.journal`) and nowhere else. - **A failing test is kept, not skipped.** It is the reproduction. - **Never rotate during an adoption.** Rotation is a separate act, afterwards, deliberately. - **A data directory is never removed by the mesh** (ADR 0030), diff --git a/00-META/process/playbooks.html b/00-META/process/playbooks.html index eae9b706..8065ef95 100644 --- a/00-META/process/playbooks.html +++ b/00-META/process/playbooks.html @@ -468,7 +468,7 @@ const P = [ {n:"06",name:"Writing a module",kind:"side",c:"--side", trig:"Something that runs today must run on the mesh.", who:"Whoever ports or writes it.", - steps:["Read what runs: compose file, environment, where data sits.","Name what it provides and requires.","Declare capabilities for machine facts.","Write resources in the order they must happen.","Every secret in a file, never in env.","Pin every image by digest.","New: generate the credential; adopted: accept it.","Prove it against the real software: auth, refusal, restart."], + steps:["Read what runs: compose file, environment, where data sits.","Name what it provides and requires.","Declare capabilities for machine facts.","Choose each part's form (design 20).","Kept data in requires, by template.","Write resources in the order they must happen.","Every secret in a file, never in env.","Pin every image by digest, upstream too.","New: generate the credential; adopted: accept it.","Prove it against the real software: auth, refusal, restart."], prod:"A module declaration, optionally a provisioner, and its proof.", next:"07",bounds:"no time bound", prev:["A service running with no password","A new password locking out adopted data","A test that cannot fail"], diff --git a/03-DESIGN/01-to-be/20-writing-a-module.md b/03-DESIGN/01-to-be/20-writing-a-module.md index 4bbed995..5a9c0362 100644 --- a/03-DESIGN/01-to-be/20-writing-a-module.md +++ b/03-DESIGN/01-to-be/20-writing-a-module.md @@ -3,36 +3,64 @@ layer: to-be status: proposed code: - mesh-catalog modules/showcase - - mesh-controller internal/builder - - mesh-sdk src + - mesh-delivery modules/mesh-delivery-indexer/judge (the manifest validator) + - mesh-delivery modules/mesh-delivery-builder + - mesh-sdk go/catalogue (the manifest's shapes) + - mesh-sdk go/declaration (the node-engine's resource vocabulary) - mesh-contracts -updated: 2026-11-05 +updated: 2026-11-12 decisions: + - "ADR 0578" + - "ADR 0595" + - "ADR 0619" - "ADR 0634" - "ADR 0635" - "ADR 0625" - "ADR 0627" - "ADR 0573" + - "ADR 0605" - "ADR 0606" + - "ADR 0607" - "ADR 0609" - "ADR 0593" - - "ADR 0150" - - "ADR 0099" - - "ADR 0053" + - "ADR 0499" + - "ADR 0498" + - "ADR 0478" + - "ADR 0436" + - "ADR 0366" + - "ADR 0316" + - "ADR 0267" + - "ADR 0259" + - "ADR 0262" + - "ADR 0348" + - "ADR 0564" + - "ADR 0596" + - "ADR 0201" + - "ADR 0193" + - "ADR 0192" - "ADR 0189" + - "ADR 0175" + - "ADR 0152" + - "ADR 0150" + - "ADR 0142" + - "ADR 0135" + - "ADR 0099" - "ADR 0074" + - "ADR 0053" + - "ADR 0052" - "ADR 0040" - "ADR 0039" - - "ADR 0053" + - "ADR 0005" --- # Writing a module -A worked guide. One module, four capabilities, four languages, and the packages it publishes. +A worked guide: what a module is built of, the manifest form of each part, and the packages it publishes. -The reference for *what can be said* is [`18-building-a-module`](18-building-a-module.md); the -reference for *what the code and the mesh say to each other* is -[`19-the-module-protocol`](19-the-module-protocol.md). This is how you actually write one. +The reference for *what a manifest may say* is the manifest validator, the rules of the seat `mesh-catalog`'s holder +that judge every manifest (ADR 0578 §1), with [design 32](32-what-a-module-declares.md) for what the bus makes of it; +the reference for *how a build is modelled* is [design 18](18-building-a-module.md); the reference for *what the code +and the mesh say to each other* is [design 19](19-the-module-protocol.md). This is how you actually write one. ## First: what is in the SDK, exactly @@ -93,136 +121,144 @@ the build modules' regenerate-and-compare step, the catalog check of a client an contracts module that claims a seat; the home of each new exported name, a domain SDK's importers included, and a role made a seat or a seat left as one module's tools, by the review of the pull request that adds it (ADR 0634 §10). -## A module with four capabilities, in four languages +## Who reads a manifest, and what each one judges -A module is **one piece of software** (ADR 0040) and -may still be written in several languages — each artifact names its own, compiles alone and is -packed alone. +*Rewritten 2026-11-12 from the manifest validator's code on mesh-delivery's main (the operator, 2026-11-12: "first +look into the indexer's code that reads the manifest and update our module-development documentation accordingly").* -``` -showcase/ - module.json - events/ ← TypeScript: reacts to what the mesh does - tools/ ← Go: answers questions - provisioner/ ← Rust: grants instances of what it provides - ingest/ ← Python: a scheduled job -``` +A manifest is read by four parts, and each judges only what is its own: -### The manifest +| part | reads | judges | where a refusal is said | +|---|---|---|---| +| the **manifest validator**, the rules of the seat `mesh-catalog`'s holder (the indexer) | every manifest it takes in, and every manifest a change touches, at the change's head | the manifest strictly: every field known, at every depth; every rule a manifest is held to alone (a name's form, an artifact's words, a claim on a seat of the seat list, kept data, events, secrets, health, a container's `run-once`, `schedule` and `while-stopped`) and every rule between the manifests of one catalog | the change's check, naming each problem; and the indexer's verdict, which every other part takes (ADR 0578 §1, §2) | +| the **builder** and its build modules | `build` | which build module builds each artifact (`kind` and `language`) and whether an upstream reference is pinned by digest (ADR 0478, ADR 0498) | the build's refusal, naming the artifact | +| the **node-engine**, through its composer in the change's composition step and on each node | the resources of each module composed for one node | each resource's `type` and fields strictly, by the node-engine's vocabulary (mesh-sdk's declaration package): a field a type does not use is refused, never ignored, and an `action` is refused (ADR 0005) | the composition step of the change, for each machine the change touches; the node's verdict at apply | +| every other part (the assigner, the grants part, the tool runner, the provisioner) | the manifest, decoded leniently | nothing: it takes the indexer's verdict (ADR 0578 §3) | — | -```json -{ - "module": "showcase", - "build": { "artifacts": [ - { "name": "events", "kind": "bundle", "language": "typescript", - "entrypoints": ["index.js"] }, - { "name": "tools", "kind": "bundle", "language": "go", - "entrypoints": ["tools"] }, - { "name": "provisioner", "kind": "bundle", "language": "rust", - "entrypoints": ["provisioner"] }, - { "name": "ingest", "kind": "bundle", "language": "python", - "entrypoints": ["ingest.py"] } - ]}, - "resources": [ - { "id": "events", "type": "process", "name": "showcase-events", - "artifact": "events", "run": ["node", "index.js"] }, - { "id": "tools", "type": "process", "name": "showcase-tools", - "artifact": "tools", "run": ["./tools"] }, - { "id": "grants", "type": "process", "name": "showcase-grants", - "artifact": "provisioner", "run": ["./provisioner"] }, - { "id": "ingest", "type": "process", "name": "showcase-ingest", - "artifact": "ingest", "run": ["python", "ingest.py"], - "schedule": "0 3 * * *" } - ] -} -``` +**A field the running manifest validator does not know** makes the indexer set the manifest aside to wait for a newer +validator; the change's check, which runs the validator built from the change's head, refuses it (ADR 0578 §1, as +changed). So a field is added to the validator in the same change that first writes it. -**Four artifacts, four toolchains, four processes, one module.** Nothing here says how any of them -is hosted — that is the mesh's, and it is why these are `process` rather than four containers. +The manifest validator does not judge a resource's own fields: those are the node-engine's vocabulary, judged where a +module is composed for a machine. A module assigned to no machine is therefore judged by the validator alone. -### Step 1 — say what it is, in each language's own terms +## What a module is built of -Each part is an ordinary project in its language, depending on the mesh SDK for that language the -way it would depend on anything: +A module is **one piece of software** (ADR 0040), and may be built of several parts. Each part is an artifact in +`build.artifacts`, named locally, and most are run by a resource in `resources` that names the artifact by its +`artifact` field: the resolved manifest names the artifact's digest where the repository's manifest names the +artifact (a `source` and a `digest` for a bundle or an archive, an `image` for an image). -``` -events/package.json "@novox/mesh-sdk": "^1.2.0" -tools/go.mod require novox.example/mesh-sdk v1.2.0 -provisioner/Cargo.toml mesh-sdk = "1.2" -ingest/pyproject.toml dependencies = ["mesh-sdk~=1.2"] -``` +**The artifact kinds** the manifest validator accepts are five, and a builder builds each with one build module: -**Resolved from the mesh's own package registries**, which are the ordinary registries for each -ecosystem, hosted by the mesh. An author does nothing unusual: `npm install`, `go mod tidy`, -`cargo build`, `pip install` all work, and a developer's laptop resolves exactly what a build does. +| kind | its words | built by | +|---|---|---| +| `bundle` — the module's own code | `language`; for Go also `system` (one of the systems the mesh builds for), `from` (the package it compiles, relative to the module) and `binary`; for TypeScript `entrypoints` and, where it keeps imports, `external`; `loads`, `env` and `bus` for a tools bundle (below); `paths` for a language other than Go | the Go or the Node build module: a bundle in Go or TypeScript (ADR 0478); a bundle in another language is a build module of its own first | +| `package` — code other modules compile against | `language` (Go or TypeScript), `series`, `paths` (the files it is built from, ADR 0499), `generated-from` for a contracts module's | its language's build module, which asks the registry seat of its language to publish it (ADR 0605 §3, ADR 0606) | +| `image` — a container image built from a recipe | `from` (a Dockerfile), `target`, `compiles` (the Go package a recipe compiles, ADR 0267), `context` (a second repository its build reaches into) | the image build module | +| `upstream` — somebody else's image, copied | `from`, a reference pinned by its digest (`@sha256:`) | the image build module (ADR 0498 §4) | +| `archive` — files packed as they stand | `from` (a directory of the module), or an upstream release by `url`, `sha256` and `version`, with `file` or `within` (ADR 0316) | the Go build module | -### Step 2 — write each part against its capability +Every artifact may also say `needs` (an edge no other line says), and every kind but `upstream` its `series`. A `bundle` reached by nothing (no +`loads`, no `tools`, no resource naming it) is refused by name (issue 216); a resource naming a `package` is refused, +since a package is a build input and runs nowhere. -Each uses only the part of the protocol it needs. The event consumer never learns what a grant is. +**The parts, and the manifest form of each:** -``` -events/index.ts on("module.builder.built", …) → the events capability -tools/main.go tool("showcase_state", …) → the tools capability -provisioner/main.rs grant → credential → the provisioning capability -ingest/ingest.py emit("module.showcase.ingested", …) → events, emitting only -``` +| part | the manifest form | what runs it | +|---|---|---| +| **tools** | a `bundle` with `loads` naming what the runtime loads (for a Go bundle its `binary`), and the module's `tools` listing every tool it serves; `replaces` for the shell commands a tool replaces | the node's tool runner (ADR 0175); with `runs-as`, a **runtime of its own** under the module's account (below) | +| **a long-running service of its own code** | a `process` resource: `artifact` (a bundle), `name` (what the unit is called), `run` (the command, relative to the unpacked bundle), `user`, `env` and `env-file`, `restart-on`, `health`, `yields`, `replaces` (the ids it takes over from) | the node-engine, as a unit that stays up | +| **its own bus account for a process** | the process's `bus`: the `emits`, `state`, `queues`, `invokes` and `asks` it is granted, a subset of what the manifest declares; an own secret holding the account, owned by the process's `user` (ADR 0436 §1) | the grants part composes the account from it | +| **a scheduled job** | a `process` or `container` resource with `schedule`, a five-field cron; never with `run-once`; a process with no `restart-on`, `replaces` or `witness`; a scheduled container may name `while-stopped` (below) | the node-engine, again on each fire (ADR 0053) | +| **a one-time step** (a seed, a first-start step, a validator of a file) | a `process` or `container` resource with `run-once` true: a **run-once step**, which gates every resource after it; `restart-on` makes it run again when what it names changes (ADR 0052, ADR 0099) | the node-engine, to completion | +| **a migration of its own state before a new version runs** | `prepares` true, and a `container` or `process` that stays up running an artifact the module built: the mesh runs that program in its preparation mode (ADR 0135); never a run-once step written by hand for it | the node-engine, before the new version starts | +| **a seat's implementation** | `claims`, each `{name, scope, serves}` for a seat on the seat list (the verbs it serves; a kinded seat's `kind`); a seat the module defines for itself in `seats`; the verbs answered by its tools bundle or a process | as its tools or its process | +| **an image** | an `image` or `upstream` artifact, and a `container` resource naming it by `artifact`; or a container naming an image pinned by digest in `image` directly, for an image the module does not copy; a container stays up unless it says `run-once` or `schedule` | the node-engine, on the container runtime the module's `capabilities` name | +| **a host program from the operating system** | a `package` resource (`package`, `absent`) and a `service` resource per unit it ships (`unit`, `state`, `boot`, `scope`, `restart-on`, `reload-on`, `health`); never an `action` | the node's package manager and service manager | +| **files** | `file` and `directory` resources (a directory by `place` or `path`), an `archive` resource unpacking an `archive` artifact, template files in the module's `templates` directory | the node-engine and the provisioner client | -### Step 3 — the mesh does the rest +**A runtime of its own.** A module that says `runs-as` names an account it makes (a `user` resource of that name, +never root nor the operator's); its tools bundle is then served by a runtime of its own, as that account, on a bus +account of its own (ADR 0259 §8), whose readiness is the manifest's `runtime-health` (ADR 0366). Its secrets belong to +that account (`secrets-owner`), and its bus account's secret is the own secret `broker`. A tools bundle that holds an +account of its own instead says `bus` on its artifact, a Go bundle only, with the own secret `broker-` in a +directory the runtime's account may pass (ADR 0564). -You do not write a Dockerfile, a unit file, or a port number. The builder picks the toolchain from -the language, compiles each artifact alone, and publishes it. The controller assigns the machine -and the ports; the node-engine writes the units. +**What it keeps on the bus.** Kept data is asked for in `requires`, each entry in its long form: `name` (a local name, +lower-case letters, digits and hyphens), `storage` (`key-value`, `stream`, `object-store` or `ledger`), `template` +(`state`, `events`, `work-queue`, `log`, `objects`, `ledger`) and its `options`, or a bare stream's `configuration` +(ADR 0595 §1 to §3). The manifest validator holds each option to its bound. The **old sections** `state`, `logs`, +`streams` and `objects` are kept by the parts the bus module needs before it can answer, and by no new entry +anywhere else: those parts ask nothing by template, and any other module's use of an old section is refused once no +accepted module outside them uses that section any more (ADR 0619 §1, ADR 0595 §11). One local name is in one form. +A module reads another module's data by asking its tools or seat verbs, never by its bus state (ADR 0201, as changed +by ADR 0596). -### What it costs you to use four languages +**What it says and hears.** `emits` names its events locally; `consumes` names `.`, with `*` for one +name and `**` for the rest; `invokes` names each tool `.` or verb `seat:.` it calls, or `*`, +and an entry's long form `{verb, fields}` names the fields of the answer it relies on (ADR 0152); `uses` names the +seats it sends to; `needs-approval` marks a tool, verb or event that waits for the operator's approval. -**Honestly: four sets of dependencies to keep current, and four SDKs that must agree.** The mesh -makes it possible, not free. A module in one language is simpler, and the reason to use four is -that one of them genuinely suits a part better — not that you can. +**What someone may change.** Each config value a module reads is declared once under `config` (or its alias +`settings`, never both), with its type, its default where it is a preference, and whether it is trusted; a file takes +it by `takes-config` or a template file's `${config:}` (ADR 0262, ADR 0348). + +*How it is checked:* the manifest validator's tests (mesh-delivery modules/mesh-delivery-indexer/judge) refuse each +form above that a manifest gets wrong, by name; the builder's plan test refuses a kind or language no build module +builds; the node-engine's declaration tests refuse a field a resource type does not use; and the showcase module in +mesh-catalog declares one part of each kind its language allows and composes on a node. + +## The showcase, worked + +The catalog's `showcase` module is the worked example: **one TypeScript bundle, several entrypoints, every mode**. + +| part | artifact | resource | its words | +|---|---|---|---| +| the module's code | `code`, a `bundle` in TypeScript, `entrypoints` for the tools, the provisioner, the service, the run-once step and the report | — | — | +| a run-once step | `code` | `migrate`, a `process` | `run` the step's entrypoint, `run-once` true, `env-file` | +| the service | `code` | `server`, a `process` | `run` the service's entrypoint, `user` the module's account, `env-file`, `restart-on` its settings file | +| a scheduled report | `code` | `reporting`, a `process` | `run` the report's entrypoint, `schedule` a five-field cron | +| a container from somebody else's image | `helper`, an `upstream` pinned by digest | `tools`, a `container` | `network`, `volumes` mounting its broker secret, `args` | +| packed files | `files`, an `archive` from a directory | `packed`, an `archive` | `path`, `unknown` | +| its settings file | `templates`, the module's templates directory | `settings`, a `file` | `path`, `mode`, `template` | +| a seat it defines for itself | — | — | `seats`, and `claims` of it | + +Nothing in it says how any part is hosted beyond its resource type — that is the mesh's: the builder picks the +toolchain from the language, compiles each artifact alone and publishes it, the assigner places the module, and the +node-engine writes the units. **A module in several languages is several bundles**, each naming its own language and +compiled alone; each language is one more build module the mesh must have, and one more set of dependencies to keep +current. The reason to use two is that one genuinely suits a part better, not that you can. + +### Its code, in each language's own terms + +Each part is an ordinary project in its language, depending on the mesh's packages the way it would depend on +anything, resolved from the mesh's own package registries, which are the ordinary registries for each ecosystem +hosted by the mesh (the registry seats, ADR 0605): `npm install` and `go mod tidy` work, and a developer's laptop +resolves what a build does. **A Go module requires a mesh package at the zero version only**: the builder builds +against the newest version of the package series the manifest's `packages` accepts (ADR 0573 §2, §4, ADR 0606), so no +member of a change names another's version. + +Each part uses only the part of the protocol it needs: the event consumer never learns what a grant is, and a run-once +step emits only. ## Publishing a package is a capability A module may publish libraries as well as run code. **The SDK is not special; it is simply the first module that did this**, and its own consumers are the mesh's modules. -```json -{ - "module": "plex", - "build": { "artifacts": [ - { "name": "server", "kind": "upstream", "from": "plexinc/pms-docker@sha256:…" }, - { "name": "client-ts", "kind": "package", "language": "typescript", "from": "clients/typescript" }, - { "name": "client-rust", "kind": "package", "language": "rust", "from": "clients/rust" }, - { "name": "client-py", "kind": "package", "language": "python", "from": "clients/python" } - ]} -} -``` - -A `package` artifact is built and **published to the mesh's registry for that ecosystem**, under the -version its own project file declares. Another module then depends on it the ordinary way: - -``` -"@novox/plex-client": "^2.0.0" -``` +A `package` artifact, in Go or TypeScript, is built from the trunk and **published to the mesh's registry seat for its +language** under its next version, `.`, which the builder counts (ADR 0606 §1 to §3); a branch's build is a +tagged version, taken only by a build that asks its tag (ADR 0607). Another module then depends on it the ordinary +way, accepting a package series by its manifest's `packages`. **Why this belongs to modules rather than being a separate thing.** A client for a piece of software changes when that software changes, and the module that owns the software is the only thing that knows. Putting the Plex client anywhere else is the shared-library disease with extra steps. -### What this leaves open - -- **Which private registry.** That there *is* one is settled — - ADR 0014 says each module consumes its dependencies - from the private registry, the mesh's own shared library included. Which software serves it is - not: the catalog holds `verdaccio`, and a git host usually serves package registries too. -- **And the bootstrap does not have one.** ADR 0014 assumes a registry exists; on a fresh mesh - nothing has installed one when the first SDK is built. That is the same pivot as everything else - and it has not been designed. -- **Who may publish.** A builder pushing a package needs an account on that registry, which is a - credential in the bootstrap path and does not exist yet. -- **Versions and ranges.** Everything else the mesh delivers is pinned by digest, and a range is - resolved at build time from whatever the registry holds. A lock file records what was chosen, and - the builder records that as the build edge — but *a mesh that can rebuild a commit and get a - different library* is a real change from how everything else here works, and it should be a - decision rather than a consequence. +What design 20 once left open here is decided: the registries are the registry seats (ADR 0605), who publishes is the +build module of the package's language, and a build resolves a series, never a range, recorded as its build edge +(ADR 0606, ADR 0573). ## A file and the thing that reads it are guarded by a gate @@ -230,47 +266,59 @@ knows. Putting the Plex client anywhere else is the shared-library disease with The apply is not a transaction: every resource is attempted, every failure reported, and only a failed gate stops what follows it (ADR 0053). -So a push can half-happen, and the pairing that matters is a configuration file beside the +So a send can half-happen, and the pairing that matters is a configuration file beside the service that reads it. A module keeps that pair correct by declaring them in this order: the file; -a `run-once` container that validates it; the service, with `restart-on` naming the file. A file -the validator refuses reaches the disk and nothing else — the gate fails, the service after it is -left as it was, and the machine reports the push failed. The service never serves what the -validator refused. That is the grouping, made from what the manifest already has; no primitive +a run-once step (a `run-once` container or process) that validates it; the service, with `restart-on` naming the +file. A file the validating step refuses reaches the disk and nothing else — the gate fails, the service after it is +left as it was, and the machine reports the send failed. The service never serves what the +step refused. That is the grouping, made from what the manifest already has; no primitive withholds a file from the disk, and a service that reads its file live rather than at start is the one shape this does not protect, and should not be written. -*How it is checked:* the lab's coupled-pair spike declares exactly this pair, pushes a refused +*How it is checked:* the lab's coupled-pair spike declares exactly this pair, sends a refused file, and asserts the file on disk is the new one, the service serves the old one, and the machine -reports the push failed. +reports the send failed. -A `run-once` step may itself name what it reads under `restart-on`; for a step the word means +A run-once step may itself name what it reads under `restart-on`; for a run-once step the word means *run again* — a step that fetches a fact from a provider names the binding it reads, and runs again when the provider moved. The service that consumes what the step made names the step, so it is recreated with the new fact (ADR 0099). *How it is -checked:* the node-engine's unit tests run a step again when its named file changed and not otherwise, -and recreate a container naming a step after the step ran. +checked:* the node-engine's unit tests run a run-once step again when its named file changed and not otherwise, +and recreate a container naming a run-once step after it ran. -## A recurring step may hold its own module's containers still +## A scheduled run may hold its own module's containers still *Written 2026-10-02, from ADR 0189 and [issue 108](../../04-ISSUES/108-the-registry-has-no-garbage-collection-once-it-has-two-doors/00-report.md).* Some work cannot be done underneath a running service: an artifact store's collector rollouts the -storage and requires every writer stopped. A `run-once` step runs *beside* containers and a -scheduled one is the same container again, so until this a module had no way to say it — and the +storage and requires every writer stopped. A run-once step runs *beside* containers and a +scheduled run is the same container again, so until this a module had no way to say it — and the mesh inherited a store that has never collected anything, because the predecessor said it with a shell script and a script beside a module is not a resource in it. -A scheduled step may name `while-stopped`: resource ids of **its own module's** containers, which -the node-engine stops before the run and starts again after it, in the reverse order, **whatever the step +A scheduled container may name `while-stopped`: resource ids of **its own module's** containers, which +the node-engine stops before the run and starts again after it, in the reverse order, **whatever the run did**. Three boundaries, each refused where it can be seen earliest — its own module's containers -only, because a module that could quiesce a neighbour could stop the mesh; scheduled steps only, -because at apply the declaration is applied in order and a step already gates what follows, so a +only, because a module that could quiesce a neighbour could stop the mesh; scheduled runs only, +because at apply the declaration is applied in order and a run-once step already gates what follows, so a one-time offline job says *before* rather than *instead of*; and restoring that is not conditional on anything, because the only real risk of the field is a window that never closes. *How it is checked:* the node-engine's unit tests assert stop–run–start in that order, the restart after a -step that **failed**, the reverse order for several containers, and a service left down said -loudly. The controller refuses, from the definition alone, a window with no schedule, one on a -run-once step, one naming a container the module does not declare, and one naming itself. +run that **failed**, the reverse order for several containers, and a service left down said +loudly. The manifest validator refuses, from the manifest alone, a window with no schedule, one on a +run-once container, one naming a container the module does not declare, and one naming itself. + +## Acceptance + +- A manifest written by this design's tables is accepted by the manifest validator, built by the builder and composed + by the node-engine for a machine, with no form one of them refuses: checked by the showcase module's check on every + change to it. +- Every refusal names the module, the part and the field, where its author reads it: the change's check. + +## Milestones + +None of its own. This design is the guide every module's change follows; a change to what a manifest may say is a +decision and a change of the manifest validator, the builder or the node-engine first, and this design follows it. diff --git a/03-DESIGN/01-to-be/README.md b/03-DESIGN/01-to-be/README.md index ef467593..1e3eecd0 100644 --- a/03-DESIGN/01-to-be/README.md +++ b/03-DESIGN/01-to-be/README.md @@ -29,7 +29,7 @@ document is written and this one's status becomes `implemented`. | [`17-raising-a-mesh.md`](17-raising-a-mesh.md) | How a mesh comes into existence, and how a machine joins one that exists | ADR 0067, ADR 0006, ADR 0005 | | [`18-building-a-module.md`](18-building-a-module.md) | How a build is modelled, and why a recipe that is always a Dockerfile does not fit what a module is | ADR 0040, ADR 0039, ADR 0009 | | [`19-the-module-protocol.md`](19-the-module-protocol.md) | What a module's code and the mesh say to each other; an SDK is an implementation of it | ADR 0074, ADR 0042, ADR 0043 | -| [`20-writing-a-module.md`](20-writing-a-module.md) | A worked guide: one module, four capabilities, four languages, and the packages it publishes | ADR 0074, ADR 0040, ADR 0039 | +| [`20-writing-a-module.md`](20-writing-a-module.md) | A worked guide: what a module is built of, the manifest form the manifest validator accepts for each part, and the packages it publishes | ADR 0578, ADR 0595, ADR 0074, ADR 0040 | | [`21-the-installation-in-full.md`](21-the-installation-in-full.md) | Every step from a bare machine to a mesh that maintains itself, and what is not yet true | ADR 0067, ADR 0073, ADR 0014 | | [`22-the-work-ahead.md`](22-the-work-ahead.md) | Everything decided and not yet built, in dependency order, each phase ending at a run | ADR 0074, ADR 0075, ADR 0014 | | [`23-choosing-a-provider.md`](23-choosing-a-provider.md) | Which of several providers of a kind serves a consumer, and when a module carries its own instead | ADR 0084, ADR 0027 | From f74f9b19be29580a6c3a481591ec3f2ed43d5b32 Mon Sep 17 00:00:00 2001 From: jochen Date: Thu, 12 Nov 2026 11:54:16 +0100 Subject: [PATCH 3/3] =?UTF-8?q?Design=2047=20G=20=C2=A7716:=20the=20gate?= =?UTF-8?q?=20keeps=20a=20group's=20members=20and=20their=20bases=20from?= =?UTF-8?q?=20the=20announcement?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Before any take-in the gate held no group: its records open at the take-in, and from G it writes `merges` no more, so the forge bridge's §715 read of each member's base through `groups` had nothing to answer from. ADR 0666 §13 has the gate form the group; §716 keeps it in the states `deliveries` and `groups` the gate already holds (§613), heard on the seat git's pull-updated, pull-merged and pull-closed, and says who deletes each key and how `groups` answers. A-236 now points to it, and A-279 is carried over from the answers branch in the form §716 makes it, its Approve-entry form dropped. Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h --- .../G-21-provider-and-revert-ask.md | 48 ++++++++++++++++++- .../47-milestones/G-22-lead-answers-2.md | 10 +++- 2 files changed, 54 insertions(+), 4 deletions(-) diff --git a/03-DESIGN/01-to-be/47-milestones/G-21-provider-and-revert-ask.md b/03-DESIGN/01-to-be/47-milestones/G-21-provider-and-revert-ask.md index 367c3a0a..10f663be 100644 --- a/03-DESIGN/01-to-be/47-milestones/G-21-provider-and-revert-ask.md +++ b/03-DESIGN/01-to-be/47-milestones/G-21-provider-and-revert-ask.md @@ -32,8 +32,9 @@ of the assigner's ask when a revert is held (§300). §711 answers the indexer's give an edge of kind 5 (§47), between which entries, and what a catalog entry's `acts` holds. §712 to §715 answer the indexer's develop questions on rows IX5 to IX7 (2026-11-12), which A-213 sent to design lines: the validation's run before take-in, what holds a pending revision while it runs, and how a watched branch other than the trunk gets -its merges, its pending revision and its groups. Each is the design's -choice within the record it cites, tested against [the principles](../../../00-META/principles.md).* +its merges, its pending revision and its groups. §716 answers the lead's question of 2026-11-12 (A-279) that no +principle settles: where the gate keeps a group's members and each member's base before the take-in. Each is the +design's choice within the record it cites, tested against [the principles](../../../00-META/principles.md).* ## The provider a requirement points to @@ -250,3 +251,46 @@ choice within the record it cites, tested against [the principles](../../../00-M and 7). How it is checked: whole (forge bridge, indexer): two members landing on `beta` give one `forge.group.merged` with `base` `beta` and one `beta` source revision; a group whose `beta` member never lands raises `take-in-waits` at 30 minutes. + +- §716 **The gate keeps a group's members from their announcement, in its states `deliveries` and `groups`, heard on + the seat `git`'s pull events.** (ADR 0666 §13; ADR 0656 §7; ADR 0633 §4, §5; §42, §43, §45, §151, §156, §613, §715, + A-236) This completes §42 and §715: ADR 0666 §13 has the gate form a group and its group verb say every member, and + before G's delivery the gate kept a group from the announcement in `deliveries` and `groups`, which §613 already + moves to template states under the same names; §716 has G's gate go on writing them. The gate's manifest `consumes` + the seat `git`'s `pull-updated` beside the `pull-merged` and `pull-closed` it consumes, through its one durable + consumer, and `invokes` `seat:git.open-pulls` and `seat:git.pull`. Each event carries the pull request's + repository, number, head, `base` and head branch (`branch`). + - **What it writes, in today's forms.** On `pull-updated` the gate writes the delivery under its id + `@`, escaped as a key name, as today, with `repository`, `number`, `commit` (the + head), `base`, `branch`, `group` and `state` `open`; a newer head of the same pull request takes the older's place + in its group and the older key is deleted. Pull requests with one head branch form one group, keyed by that branch + escaped as a key name, its `members` the delivery ids, as today (§42): a group of one with no member in a record is + no group: its key is deleted and its member's `group` cleared. `pull-merged` sets the delivery's `state` to `merged`. A delivery copied from + before G (§613) in another state of the old table counts `open`. Only the gate writes either state. + - **Who deletes it.** The gate: a delivery on `pull-closed` of a pull request not merged; a delivery once the record + that carries it opens (§151's take-in, or §156's switch), its `base` copied onto the record's member (`RecordPull` + gains `base`) and, for a member with no delivery, the take-in's `branch`, said in its journal; a group key once no + member remains outside a record. A deletion is made in the old section too (§613). + - **How `groups` answers.** A group is answered from its records and its `groups` key together, each member once, + a record's first: a record's member with its copied `base` and `state` `merged`, a kept member with its delivery's + `base`, `head`, `state` and `open` while not merged. So before any take-in the verb names every member and its + base, and the forge bridge reads them as §715 has it (A-236). + - **When a member has landed.** For §715, a member has landed once `groups` says it `merged` and the bridge has + published its merged message. The bridge asks `groups` at each such message it publishes and at each + `pull-merged` it hears of a group it holds open (`bridge_groups`), so the end follows whichever comes last, never + a timer. + - **At its start**, after §613's copy, the gate reads `open-pulls` once and writes each open pull request it lacks + or keeps at another head. For each delivery it keeps `open` that the answer does not name, it reads `pull`: merged + sets `merged`, closed deletes it. A read that fails is said in its journal and made again at the next start. Until + then, a group whose end never comes is said by §45's 30-minute `take-in-waits`, and the indexer's + `indexer_take_in_pending` takes it in as gathered. + + `merges` stays as §156 has it, read by the switch alone. Why: the gate forms the group and its verb says every + member (ADR 0666 §13), from its own states heard on the seat `git`'s events. It keeps the two states and forms it + already holds, adding no state and no new tool (principles 1, 4 and 7). A missed event is put right from the forge's + own record at the start, and a group that stays open is said (principle 3). How it is checked: whole (gate), two + pull requests on one head branch, bases `main` and `beta`. `groups` names both with their bases before any + take-in; a newer head replaces its member; a `pull-closed` before the merge removes it. A `take-in.taken` of the + `main` member opens a record with `base` `main`, and the `beta` member stays in `groups`. A `pull-merged` missed + while the gate was down is set from `pull` at its start. Whole (forge bridge): the group's end is published once + both the member's merged message and its `pull-merged` have come, in either order. diff --git a/03-DESIGN/01-to-be/47-milestones/G-22-lead-answers-2.md b/03-DESIGN/01-to-be/47-milestones/G-22-lead-answers-2.md index 2853dcf1..2acb4e9a 100644 --- a/03-DESIGN/01-to-be/47-milestones/G-22-lead-answers-2.md +++ b/03-DESIGN/01-to-be/47-milestones/G-22-lead-answers-2.md @@ -222,8 +222,8 @@ the principle that settles it, cited as "design 47 G A-n". - **A-236** (§509, §714, §715, §11, A-227; principles 2 and 7) A `pushed` message's new head is `head`; the bridge keeps each watched branch's last head written in its own state `heads` (key `branch...`), `before` read from it and its start read comparing with it; `pushed` carries the same take-in inputs as - `merged`. For §715 the mesh-delivery seat's `groups` member gains `base` in the contract, which the gate fills and - the bridge reads. + `merged`. For §715 the mesh-delivery seat's `groups` member gains `base` in the contract, which the gate fills as + §716 has it (from the seat `git`'s pull events, before any take-in) and the bridge reads. - **A-237** (§712, §354, ADR 0666 §16, A-139; principle 7) A run's input is any size: the indexer keeps it in its state `inputs` with the state's `-objects` store holding a large value (mesh-sdk's `stateobjects`), and serves it through its `input` verb answered in chunks (`buscalls.SeatChunks`, a verb marked `chunks`); `RunInputBound` goes, and the @@ -353,3 +353,9 @@ the principle that settles it, cited as "design 47 G A-n". on the process's own connection; the verb `asks` stays as it answers. The ask is published by a new `go/process` method that publishes on a seat's accept subject (`mesh.seat..accept..`), on the a3g line, as `Emit` publishes an event. +- **A-279** (A-236, §715, §716; principles 1, 3 and 7; corrected four times, this form stands) A group member's `base` + is its pull request's base branch as the seat `git` says it in `pull-updated`, kept by the gate on the member's + delivery in its state `deliveries` from the announcement, and copied onto the record's member when the take-in + opens the record. Its earlier forms (the base on the `merges` entry the Approve makes) are dropped: from G no + Approve makes an entry, and the gate writes `merges` no more (§156). Design line §716 says it whole: the event, the + states and their keys, who deletes them, and how `groups` answers.