diff --git a/03-DESIGN/01-to-be/47-delivery-from-commit-to-delivered.md b/03-DESIGN/01-to-be/47-delivery-from-commit-to-delivered.md index 41f2695f..267d5a29 100644 --- a/03-DESIGN/01-to-be/47-delivery-from-commit-to-delivered.md +++ b/03-DESIGN/01-to-be/47-delivery-from-commit-to-delivered.md @@ -2,7 +2,7 @@ layer: to-be status: in-progress code: [mesh-controller, mesh-catalog modules/mesh-delivery-gate, mesh-catalog modules/gitea, mesh-delivery, mesh-host, mesh-sdk, mesh-vault, mesh-contracts, hq 00-META/checks] -updated: 2026-11-10 +updated: 2026-11-11 decisions: - "ADR 0667" - "ADR 0666" @@ -1801,8 +1801,8 @@ integration test, which starts the module whole against the real bus and store; `modules/mesh-delivery-pipeline` in mesh-delivery, has one of its own. The checker's tests go with it at G's delivery, its whole-module test, its try test and the examiner's table among them, and so do the indexer's try-candidate rows and mesh-delivery's `tryevent`; tests 21 to 24, 26, 31, 41, 46, 52 and 58, which the checker proves, hold until then, and -the runner's tests below restate what of them stays (22 in 73, 23 in 84, 46 in 84, 58 in 84). No repository ends the -milestone with more tests than it starts with (research 085, Direction, One milestone; the test budget). +the runner's tests below restate what of them stays (22 in 73, 23 in 84, 46 in 84, 58 in 84). +No module ends the milestone with more tests than it starts with, counted per catalog module (research 085, Direction, One milestone; the test budget, per module, the operator 2026-11-11). 63. **R1, no hand act delivers a change.** A change to the indexer, the grants part and the tool runner is delivered end to end, and the hand-act log of the seat `mesh-hand-acts` names no act for it (live; mesh-delivery, mesh-tools). @@ -1871,7 +1871,9 @@ The failure paths: not run`, never passed; two modules owning one path are refused (whole, runner and assigner). 85. **A forged step result**, from an account other than the step's seat holder's, or naming another commit or fingerprint, is refused and said; a `step-ended` on another node's subject is refused by the bus; a verb whose - definition names no `callers` admits only the holder roles and the operator (whole, runner and grants part). + definition names no `callers` admits only the holder roles and the operator; a change adding a new `invokes` of + `node-packet-filter.remove`, a verb whose callers are `invokers`, names that grant in its `pipeline-ended`'s + `widens` (whole, runner and grants part). 86. **An author's own review** is refused; a reviewer's passes the step (whole, runner). 87. **A step seat with no holder** ends its pipeline failed, naming the seat, within five minutes (whole, runner). 88. **A runner stopped mid-step** takes its pipelines up again from its kept state when it starts, a replaced holder @@ -1958,6 +1960,12 @@ ADR 0667's checks, of milestone G itself: to be back" and nothing is raised; with a placing machine present that never runs it, `delivery-waits` is raised after 60 minutes naming it and its node-engine's version; each want names the released version and its parts' digests; `indexer_edges` answers the module's edges from both ends (whole, indexer, assigner and node-engine, mesh-delivery and `mesh-host`). +107. **A finding is stated from its observer's truth** (design 47 G §383, §384, §388). A fixture observer's + `finding-written` and `finding-cleared` raise and clear a condition; restarted after its cause has gone, it clears + its finding at its start; restarted with a call unanswered for two hours, it says neither `condition-cleared` nor + `condition-raised` (whole, condition keeper and a fixture observer, mesh-catalog and mesh-sdk). Live: after G's + delivery, a node-engine restarted with the full seat list clears its `seat-list-unread`, read with + `mesh-conditions.conditions`. ## What retires diff --git a/03-DESIGN/01-to-be/47-milestones/G-1-pipelines-registration-and-steps.md b/03-DESIGN/01-to-be/47-milestones/G-1-pipelines-registration-and-steps.md index edbd1744..e6dcba40 100644 --- a/03-DESIGN/01-to-be/47-milestones/G-1-pipelines-registration-and-steps.md +++ b/03-DESIGN/01-to-be/47-milestones/G-1-pipelines-registration-and-steps.md @@ -3,7 +3,7 @@ layer: to-be status: in-progress design: 47 milestone: G -updated: 2026-11-10 +updated: 2026-11-11 decisions: - "ADR 0668" - "ADR 0667" @@ -209,8 +209,10 @@ check and deliver milestone G itself (ADR 0667 §2); each of them says so in a l `mesh.seat..event.step-ended.`, the node that ran it in the subject and each holder's account granted only its own node's subject, through a durable consumer of its own, so a step outlives a restart of either side (ADR 0666 §7, ADR 0363 §2). A build seat's holder restarted mid-step gets the step again from its work queue and finishes it - (ADR 0664 §1). A unit or integration test step says its test counts in `step-ended`, and `pipeline-ended` names any - repository over 50 integration or 500 unit tests, and any catalog module over them (ADR 0666 §8; the test budget). A step's outcome is one of `passed`, `failed`, `nothing to run`, + (ADR 0664 §1). A unit or integration test step says its catalog module's test counts in `step-ended`; `pipeline-ended` carries each + catalog module's counts against the test budget, which is counted per catalog module (at most 50 integration and + 500 unit tests each), and `pipeline-ended` names any catalog module over it (ADR 0666 §8; the test budget, per + module, the operator 2026-11-11). A step's outcome is one of `passed`, `failed`, `nothing to run`, `bypassed`, `carried` (with the pipeline it was carried from), `could not run` (naming the part not heard), `waiting on `, `accepted` (a failed `optional` step a person accepted, with who and why), `not asked` (an `approval` step after a failed `required` step), `superseded` and `cancelled`. diff --git a/03-DESIGN/01-to-be/47-milestones/G-2-pipelines-sizes-checks-and-tools.md b/03-DESIGN/01-to-be/47-milestones/G-2-pipelines-sizes-checks-and-tools.md index efb17334..ecd38119 100644 --- a/03-DESIGN/01-to-be/47-milestones/G-2-pipelines-sizes-checks-and-tools.md +++ b/03-DESIGN/01-to-be/47-milestones/G-2-pipelines-sizes-checks-and-tools.md @@ -230,6 +230,9 @@ and an estimated 2 mesh-sdk publishes a day, each rebuilding about 120 artifacts | the assigner | `written`: the last source revision and end count per module and watched branch (§352) | old `state` section | until the branch is unwatched | a few KiB | | the indexer and the assigner | `inputs`: a `run` ask's input by write time and sha256 (§354) | old `state` section | until the ask's `step-ended`, at most a day | 64 MiB; an estimated 300 asks a day of under 100 KiB each is under 30 MiB; at the cap the asker refuses the ask loudly, naming the store | | the runner and the assigner | `findings` (§355), read by their own read tools alone; the keeper hears the findings as events (§383) | old `state` section | an entry deleted when its finding clears | a few KiB | +| every observer (§387) | the stream `-findings`: `finding-written`, `finding-cleared`, `findings-stated` (§383) | old `streams` section for a foundation part, the module's `streams` otherwise | 7 days, its oldest dropped when full; the next statement covers a lost one (§384) | 16 MiB; an estimate of a few hundred findings a day of about 1 KiB is under 2 MiB a week | +| the condition keeper | `findings-views`: one key per observer and node, its last stated set, pending keys and open asks (§388) | the form of its `conditions` | deleted when the module leaves the catalog or the node (§365) | under 1 MiB; about 30 observers on four machines | +| the watchman, the keeper | `machine-away`, `machine-back` (§389); `findings-asked` (§390) | the seats' definitions | 90 days; 7 days | 16 MiB each | | the runner | `aged`: three step-log sequences (§330) | old `state` section | never deleted | bytes | | the teller | `widens-said` (§335) | the template `state` | until the branch is unwatched | a few KiB; the template's | | each build module | `steps` (§326) | the template `state` | one key per artifact, dropped by `steps` with `drop` | under 1 MiB; the template's | @@ -269,7 +272,11 @@ step keys and the builder's build log, against the checker's last week (test 99) from the next change on; mesh-delivery's `contracts` and `sdk` move under its `modules/` folder (ADR 0667 §3). - §89 **Readers first** (ADR 0667 §4): an edge of kind 4 each, which the delivery plan derives: the indexer's manifest validator for the seat definitions' new fields (`callers`, `acts`, `subject`, `keep`) before any definition using - them; on each node the node-engine that reads `form` and `catch-up` before any manifest writing them. + them; the manifest validator for the manifests' new words (`version`, §368; a `*` emitter in `consumes`, §386; the + finding stream an observer declares, §383, §387) and the grants part that makes a `*` entry's durable consumers + (§386), before the keeper's or any observer's manifest using them; on each node the node-engine that reads `form` + and `catch-up` before any manifest writing them. The running checker's validator makes G's own check, so the reading + of these words goes in the reader release of §340, whether or not that validator already takes one of them. - §90 **The stream maker first** (ADR 0667 §5): the grants part, which makes each kept seat stream and its durable consumers, before the parts that hear a kept stream. - §91 **The bus module first** (ADR 0667 §6): the bus module, with `-objects`, the sent store and the grant of each diff --git a/03-DESIGN/01-to-be/47-milestones/G-7-answers.md b/03-DESIGN/01-to-be/47-milestones/G-7-answers.md index ba512476..0714bc9f 100644 --- a/03-DESIGN/01-to-be/47-milestones/G-7-answers.md +++ b/03-DESIGN/01-to-be/47-milestones/G-7-answers.md @@ -125,9 +125,8 @@ record it follows from, and replaces the builder's own choice where one was made reads that no module name stands in the runner's code (§7). - §334 **A publication's pipelines and their key.** `package-published` carries the package's artifact reference (`:`, from the catalog entry), its version and digest, the watched branch and the source revision - its build was made from. The runner reads the dependents from that revision's catalog graph, the artifacts with a - kind-1 edge to the package, and starts each one's pipeline keyed by that branch, that source revision and the - dependent module, the version in the pipeline's state; a `take-in.taken` of the same revision and module is the + its build was made from. The runner asks `outdated` for that package on that branch and starts a pipeline for each + dependent `outdated` names behind (§53), keyed by that branch, that source revision and the dependent module, the version in the pipeline's state; a `take-in.taken` of the same revision and module is the same pipeline (*A second start writes nothing*). A pipeline the runner starts from `outdated` (its start, §375, or `runner_catch_up`) is keyed by the dependency's watched branch, the dependency's source revision `outdated` names, and the dependent module, so it is the same pipeline the event would have started. Why: a pipeline's key is branch, source revision and module (above), @@ -157,7 +156,8 @@ record it follows from, and replaces the builder's own choice where one was made close to their holders and the operator: `node-hostname`'s `entries`, `add` and `remove`; `node-intrusion-prevention`'s `status`, `banned`, `ban` and `unban`; `node-packet-filter`'s `rules`, `reload` and `remove`; `node-resolver`'s `routes`, `route` and `unroute`; `node-uplink`'s `resolvers` and `links`. Each of their - definitions in `mesh-connectivity-contracts` names `invokers` in this milestone. How it is checked: unit (definition reader), each + definitions in `mesh-connectivity-contracts` names `invokers`, `persons` and `sessions` in this milestone, since + sessions and persons call them today through the mesh MCP server. How it is checked: unit (definition reader), each form accepted and any other entry refused; the grants part's table test grants each form. - §337 **The bus module's restore points.** The verbs the holder of `node-backup` on the bus module's node calls to take and restore a module's kept data on the bus name `holder:node-backup@same-node`. Why: only that holder takes them @@ -175,7 +175,9 @@ record it follows from, and replaces the builder's own choice where one was made above. Why: one source of caller limits (ADR 0666 §20). How it is checked: the generator's test refuses an unknown entry. - §340 **The reader release first.** mesh-sdk's seat-definition reader released before G's check reads `callers` as text - or as a list; G's own reader refuses text. Why: readers first (ADR 0667 §4), and every definition is a list from G; it is merged ahead of G's check because the + or as a list; G's own reader refuses text. The same release carries mesh-sdk's manifest reader of the new words of + §89 (`version`, a `*` emitter in `consumes`, an observer's finding stream), which the running checker's validator + reads before G's manifests use them. Why: readers first (ADR 0667 §4), and every definition is a list from G; it is merged ahead of G's check because the running checker, which makes G's own check (ADR 0667 §2), must read G's definitions, the operator's choice of a reader first (2026-11-09). How it is checked: unit rows for both forms in the release, and for the refusal in G. @@ -303,17 +305,21 @@ record it follows from, and replaces the builder's own choice where one was made modules whose entry declares the finding events of §383, the node-engine among them, its holder read from the seat list. It joins the readers of *Every other reader of which modules exist*, reading every watched branch's newest catalog revision, so a module watched on any branch is a source. A condition of a module that is in no branch's - revision is cleared, said with the reason `module left the catalog`. The keeper also takes a first view of the - assignment half (through `meshwantsclient`, following `assignment-revision` of `mesh-wants`), and a module's findings - of a node where the newest assignment graph revision no longer places that module are cleared, said with the reason - `module left the node`. Before its first view the keeper keeps the - conditions it holds and applies no finding event, which waits on its durable consumer, and says `view-waits` after - five minutes (*A part with no view says it waits*). It reads no module's `findings` state (§383). Why: no part fixes + revision is cleared, said with the reason `module left the catalog`. A module's findings of a node are cleared, + said with the reason `module left the node`, when the node-engine seat's `running..` says the module no + longer runs there and the newest assignment graph revision, from the keeper's first view of the assignment half + (through `meshwantsclient`, following `assignment-revision` of `mesh-wants`), no longer places it there; never for a + module no revision placed on that node. For it, `running..` (§309) carries `stopped`, with no + fingerprint, once the node-engine's have no longer holds the module, on the same subject, so the stream kept last per + subject keeps it. Until its first view the keeper applies the events of the sources it keeps (§388) and says + `view-waits` after five minutes (*A part with no view says it waits*), and a missing assignment half holds only the + clears of `module left the node`. It reads no module's `findings` state (§383). Why: no part fixes a name (§7), and the keeper hears findings and reads no other module's state (ADR 0596 §2, §4), departing from ADR 0564 §2 by the operator's answer of 2026-11-11; the assignment half and `module left the node` are the design's - choice. How it is checked: whole (keeper), a fixture module declaring the finding events is heard with no code - change, and a fixture module's condition on a node is cleared with `module left the node` when an assignment graph - revision no longer places it there; the catalog's validation finds no `findings` in the keeper's manifest `reads`. + choice, and so is `stopped`. How it is checked: whole (keeper, node-engine), a fixture module declaring the finding + events is heard with no code change; a fixture module's condition on a node is cleared with `module left the node` + once the node-engine says `stopped` for it and the assignment graph no longer places it, and is not cleared on the + assignment revision alone; the catalog's validation finds no `findings` in the keeper's manifest `reads`. ### From round 1 of the review by architectural domain (2026-11-10) @@ -361,7 +367,8 @@ record it follows from, and replaces the builder's own choice where one was made (`go/vMAJOR.MINOR.PATCH`, design 51) is written by the builder only on the catalog's `version-released`, through the seat `git`'s verb `tag`, whose callers are the builder alone, so a superseded build's reserved version is never tagged. How it is checked: whole (Go and TypeScript build modules), a built package and program carry the reserved - version; whole (builder), the tag is written on `version-released` and never for a superseded build. + version; whole (builder), the tag is written on `version-released` and never for a superseded build; whole (forge + module), a `tag` from any caller but the builder's account is refused by the bus. - §371 **Only the first watched branch releases.** Only a build of the registration's first watched branch reserves and releases a version; a build of another watched branch reserves none, publishes no version and is delivered by its build, its fingerprint, as every want names one. Why: the operator, 2026-11-10. How it is checked: whole (builder), @@ -369,10 +376,14 @@ record it follows from, and replaces the builder's own choice where one was made - §372 **The catalog's versions start from what is published.** At the new indexer's first start, `versions` is filled once, as released, for each module from the highest `published` version over every artifact of the module that the builder's state `versions` held (for mesh-sdk the higher of Go's `v0.1.14` and npm's `0.1.18`, so `0.1.18`), read by - `builder_versions` as a kept call (ADR 0668 §1, §2), an unanswered call raising `call-unanswered` after five minutes; + `builder_versions` as a kept call (ADR 0668 §1, §2), which the indexer's manifest names in its `invokes` + (`mesh-delivery-builder.builder_versions`) until that tool goes, an unanswered call raising `call-unanswered` after five minutes; it is not filled again. Then the indexer says `versions-filled` on its seat, once, kept on its stream with no age limit, and the builder reserves nothing before it (§369). The builder keeps serving `builder_versions` until then: the - running builder serves it already, and the new one serves it too, so no ordering edge is needed. The builder's state + running builder serves it already, and the new one serves it too, so no ordering edge is needed. The new builder's + `builder_versions` also answers each reserved version with its artifacts' digests (§369), so a registry holder still + on its version before G, which asks `builder_versions`, stores a publish of a reserved version made between the + builder's delivery and its own (§381). The builder's state `versions`, the floor and its tool `builder_floor_again`, the event `numbered`, `builder_versions`, and `tag-asks` are removed in the first change after the first pipeline after G's delivery has delivered a change end to end, the state through `drop` on a person's word (ADR 0595 §4). Why: numbering reads only the catalog's released versions (§348), @@ -444,7 +455,7 @@ record it follows from, and replaces the builder's own choice where one was made again. A dependent's new version that a change orders after its dependency's new version by kind 2, 4 or 6 is wanted only once that dependency's version is delivered to every machine that places it and is present. The assigner judges it, since placement is its own state: from its placements, the node-engine seat's `running` events and the - watchman's presence of each node, judged again on each of them and on each placement change, and says + watchman's `machine-away` and `machine-back` of each node (§389), never its state, judged again on each of them and on each placement change, and says `version-delivered` (the module, the version, its parts' digests and the machines away) on its seat `mesh-wants` (`mesh.seat.mesh-wants.event.version-delivered`) once every present machine placing it runs it healthy; the catalog hears it for its entries (§374), and the assigner writes the dependent's wants on it. The dependent's wait is the @@ -492,89 +503,14 @@ record it follows from, and replaces the builder's own choice where one was made seat asks the seat `mesh-builds`' verb `reserved` (§369), in place of `builder_versions`, whether the module, the version and the artifact's digest are reserved, and refuses loudly otherwise (ADR 0605 §4); a version the catalog already holds released for that digest is answered as held and stores nothing. Why: ADR 0605 §4 has the holder - refuse a version the builder did not give, and the builder's list goes (§372); the design's choice, round 2 of the review by architectural domain, 2026-11-10. How it is checked: whole - (registry holder, builder), a publish of an unreserved version is refused, naming it. + refuse a version the builder did not give, and the builder's list goes (§372); the design's choice, round 2 of the review by architectural domain, 2026-11-10. + The order is a served-verb edge (kind 6) the delivery plan derives from the holder's new `invokes` of + `mesh-builds.reserved`: the builder runs its new version, which serves `reserved`, before each registry holder's new + version is delivered; until then the old holder asks `builder_versions`, which answers reserved versions (§372). How + it is checked: whole (registry holder, builder), a publish of an unreserved version is refused, naming it; whole + (builder, registry holder on its version before G), a publish made between the builder's delivery and the holder's + is stored; the delivery plan of G names the edge from the builder to each registry holder. - §382 **An older version released after a newer starts nothing.** When every artifact of a version lower than the module's highest released is published late (a retry of an older reservation), the catalog records it `released late`, says no `version-released` for it, and the runner starts no rebuild on its `package-published` (§53). Why: a rebuild against an older part would move a dependent back; the design's choice, round 2 of the review by architectural domain, 2026-11-10. How it is checked: test 102. - -### The operator's answers to milestone G's design questions (2026-11-11) - -*ADR 0596 §1, §2, §4, ADR 0564 §2, §12, §13, ADR 0668 §1, §3, ADR 0636 §1, ADR 0666 §16, ADR 0363 §2. The operator, -2026-11-11, through lead-47: the condition keeper reads no other module's state and hears each module's findings as -events, and a module states its findings from its own truth; the shared tool-runner account stays as it is in G.* - -- §383 **An observer says its findings as events; the keeper reads no observer's state.** Every module that writes - findings (those the keeper reads today and those G adds: the watchman, the self-check, the healers, the runner, the - assigner, the indexer, the builder, the forge bridge and the node-engine among them) is an observer, and says each - change of a finding as an event of its own, declared in its manifest's `streams` and published by its account alone - on `mesh.stream..event..`, the node it runs on in the subject (§260, ADR 0363 §2): - `finding-written` when a finding is raised or what it says changes (only then, ADR 0289 §1, §4), and - `finding-cleared` when it no longer observes it. `finding-written` carries `observer` (the module), `node`, `scope`, - `id` and `kind` (the key `..` of ADR 0564 §2), `severity`, `subject`, `evidence`, ADR 0253's words - (`headline`, `explanation`), `since` and `at`; `finding-cleared` carries `observer`, `node`, `scope`, `id`, `kind`, - `at` and `why`. mesh-sdk's finding writer (`go/views`' `FindingWriter`) says them for every part, beside the key it - writes in the part's own state `findings`, which stays the owner's and is read only by the owner's read tool. The - keeper hears them on one durable consumer of its own, its manifest's `consumes` naming them with the emitter `*` - (`*.finding-written`, `*.finding-cleared`, `*.findings-stated`, §386). The keeper takes a finding only from a source its catalog view names (§365), raises or changes a condition on - `finding-written` and clears it on `finding-cleared`, save while ADR 0564 §13 holds it; a condition's looks are - those of its last event, and the observer's read tool shows its newest. What exists today goes in the same change: - every `findings` entry of the keeper's manifest `reads`, its `consumes` of the assigner's and the indexer's - `views.findings` and its `invokes` of `assigner_read_findings` and `indexer_read_findings`; the assigner and the - indexer declare no `views.findings`, which no reader follows any more; the read tools stay for persons and sessions. - No ordering edge is needed: a source still on its old version says nothing, its conditions stand as they are, and - §384's bound names it. Why: only its owner reaches a module's state, and a part hears another's events (ADR 0596 §2, - §4), departing from ADR 0564 §2 by the operator's answer of 2026-11-11. The subjects, the fields and the writer are - the design's choice. - How it is checked: unit (mesh-sdk's writer), a finding written and one cleared each say their event with these - fields after the key; the manifest validator's rows of §386; whole (keeper), a fixture observer's `finding-written` raises a condition and its `finding-cleared` - clears it, and the same event from an account that is not the source's is refused by the bus; the catalog's - validation finds no `findings` in the keeper's `reads`. -- §384 **An observer states every finding it owns at its start and when the keeper asks; the keeper's view is - rebuilt from those events alone.** At each start, and on each ask of the keeper, an observer judges every finding it - owns anew from its own truth, the cause each kind watches (the seat list it holds, a call still unanswered, a wait - still standing), never from what a process remembers and never from its `findings` state, and says `findings-stated` - on `mesh.stream..event.findings-stated.`, carrying `observer`, `node`, `at`, `asked` (the ask it - answers, none at a start) and `findings`, every finding it observes now with `finding-written`'s fields; a finding - whose bound has not passed since the start (a wait of under five minutes) is not among them. The keeper sets its view - of that observer on that node to exactly that set: it raises what is new and clears each condition of that observer - and node the set does not name, with the reason `not stated at 's start` or `not stated when asked`; for an - observer whose catalog entry claims only seats held once for the mesh, the set replaces its findings of every node, - so a holder moved to another machine leaves none behind. The keeper asks with its seat's event `findings-asked` - (`mesh.seat.mesh-conditions.event.findings-asked`, carrying `ask`, `at`, `by` and, to ask one, `module`), declared in - the seat's definition and heard by every observer through mesh-sdk's writer, which declares it in the observer's - `consumes`. It asks after its first view (§365) at each of its starts, and on `resync` (ADR 0564 §12), which so reads - no state. An observer that states nothing within five minutes of an ask while its machine is present is the keeper's - own finding `findings-unstated`, naming the module and the node, cleared by its next `findings-stated`; its conditions - stand meanwhile, and an observer on a machine away is not named. A set larger than one bus message goes in the - observer's sent store (§59). The node-engine's `seat-list-unread` (ADR 0636 §1) is stated so: a node-engine started - with the full list states no `seat-list-unread`, and the keeper clears it at that start. Why: the node-engine's - seat-list waiter kept only in its process that it had written its finding, so a node-engine restarted with the full - list never cleared it, and two machines said they waited for the seat list for a day while both held it (issue 1966); - departing from ADR 0564 §2 and §12 by the operator's answer of 2026-11-11: the fix is NATS-native, from each - observer's own truth. The event, its fields, the bound, - the reasons and the replacement for an observer held once are the design's choice. How it is checked: whole (the - node-engine, and each observer through mesh-sdk's writer), a module restarted after raising a finding whose cause has - gone clears it at its start, the node-engine's `seat-list-unread` among them; whole (keeper), a `findings-asked` is - answered by a fixture observer's `findings-stated`, a condition the set does not name is cleared with its reason, and - an observer that states nothing raises `findings-unstated` after five minutes; unit (mesh-sdk's writer), a start - states only the findings observed now. -- §385 **One tool-runner account for every module of a machine stays as it is.** On a machine where one tool runner - serves every module's tools on one bus account, any tools bundle there may read any sent store (§59) that account is - granted, and call `merge-pull` or the runner's `run` as a session; G changes neither that account nor its grants, a - known limit. Why: the operator, 2026-11-11; modules running as the machine's account are issue 1966's (note 193), - whose fix carries its check. How it is checked: none in G; the limit is named on issue 1966. -- §386 **A `*` emitter in `consumes` hears one event from every module that declares it.** A `consumes` entry - `*.` takes the event `` from every module whose manifest declares it, and is the one generic form for - hearing an event that many modules say; the keeper's three finding events are the first (§383, §384). The manifest - validator takes `*` only as the whole emitter, and only before an event that a seat's definition or a contract - in mesh-sdk defines, with its fields (for the finding events, mesh-sdk's finding writer); it refuses `*` - before any other event, `*.>`, and `*` in part of an emitter's name. The grants part grants the subscription per - declaring module, the subject of each module whose manifest declares the event, derived from the manifests as every - grant is (§59), so a module that declares it later is heard with no change to the consumer's manifest, and a module - that does not declare it is not granted. Why: the keeper hears modules it finds in the catalog and its manifest names - none (§7, §365); the form, the validator's rule and the grant are the design's choice following the operator's - answer of 2026-11-11 (§383). How it is checked: unit (manifest validator), `*.finding-written` accepted, `*.>`, - `*x.finding-written` and `*` before an event no contract defines refused; the grants part's table test grants a - `*.finding-written` consumer the subject of each declaring module and of no other. diff --git a/03-DESIGN/01-to-be/47-milestones/G-8-findings-heard-as-events.md b/03-DESIGN/01-to-be/47-milestones/G-8-findings-heard-as-events.md new file mode 100644 index 00000000..3daebf3a --- /dev/null +++ b/03-DESIGN/01-to-be/47-milestones/G-8-findings-heard-as-events.md @@ -0,0 +1,167 @@ +--- +layer: to-be +status: in-progress +design: 47 +milestone: G +updated: 2026-11-11 +decisions: + - "ADR 0668" + - "ADR 0667" + - "ADR 0666" + - "ADR 0636" + - "ADR 0603" + - "ADR 0596" + - "ADR 0595" + - "ADR 0564" + - "ADR 0363" + - "ADR 0289" + - "ADR 0253" +--- + +# 47, milestone G: the operator's answers of 2026-11-11, findings heard as events + +One file of milestone G's part of [design 47](../47-delivery-from-commit-to-delivered.md); the part opens at +[G.md](G.md). Its design lines are numbered across every file of the part and cited as "design 47 G §n". The lines +§336, §355 and §365, which these answers also change, stay in [G-7-answers](G-7-answers.md). + +## The operator's answers to milestone G's design questions (2026-11-11) + +*ADR 0596 §1, §2, §4, ADR 0564 §2, §4, §12, §13, ADR 0668 §1, §3, ADR 0636 §1, ADR 0666 §16, ADR 0667 §4, ADR 0363 +§1, §2, §5. The operator, 2026-11-11, through lead-47: the condition keeper reads no other module's state and hears +each module's findings as events; a module states its findings from its own truth, NATS-native; the shared +tool-runner account stays as it is in G; no record is written for these answers, so each line that departs from a +record says so in its Why. Round 3 of the review by architectural domain (2026-11-11) completed these lines.* + +- §383 **An observer says its findings as events on a stream of its own; the keeper reads no observer's state.** Every + module that writes findings is an observer (§387 names them) and says each change of a finding on its own stream + `-findings` (ADR 0363 §1: under its name in capitals on the bus, its `events` `finding-written.*`, + `finding-cleared.*` and `findings-stated.*`, kept 7 days and 16 MiB, its oldest message dropped when full, since the + next statement of §384 covers a lost one), published on `mesh.stream..event..`, the node it runs + on in the subject (ADR 0363 §2; §260). `finding-written` is said when a finding is raised or what it says changes, + and only then (ADR 0289 §1, §4); `finding-cleared` when the observer no longer observes it. `finding-written` + carries `scope`, `id` and `kind` (the key `..` of ADR 0564 §2), `severity`, `subject`, `evidence`, + ADR 0253's words (`headline`, `explanation`), `since` and `at`; `finding-cleared` carries `scope`, `id`, `kind`, `at` + and `why`. The observer and the node are the subject's alone: the grants part grants each module's account publish + on its own stream's subjects of its own node's segment only, the keeper takes both from the subject, and it refuses + an event whose body names another observer or node, said in its journal. mesh-sdk's finding writer (`go/views`' + `FindingWriter`) says them for every part, after the key it writes in the part's own state `findings`, which stays + the owner's and is read only by the owner's read tool; a publish the bus does not acknowledge is sent again for one + minute, then written in the part's journal as an error naming the finding, and the next statement covers it. The + keeper takes a finding only from a source of its own list (§388) and raises or changes a condition on + `finding-written` and clears it on `finding-cleared`, save while ADR 0564 §13 holds it; a condition's looks are those + of its last event, and the observer's read tool shows its newest. What exists today goes in the same change: every + `findings` entry of the keeper's manifest `reads`, its `consumes` of the assigner's and the indexer's + `views.findings`, and its `invokes` of `assigner_read_findings` and `indexer_read_findings`; the assigner and the + indexer no longer declare `views.findings`, which no reader follows; the read tools stay for persons and sessions. + Why: only its owner reaches a module's state, and a part hears another's events (ADR 0596 §2, §4); this departs from + ADR 0564 §2, where the keeper watches each observer's bucket, by the operator's answer of 2026-11-11. The stream, its + age and cap, the subjects, the fields, the writer and its retry are the design's choice. How it is checked: unit + (mesh-sdk's writer), a finding written and one cleared each say their event with these fields after the key, and an + unacknowledged publish is sent again and then logged; whole (keeper, a fixture observer on an account of its own), + `finding-written` raises a condition and `finding-cleared` clears it, the same event published by another module's + account and by the same module's account on another node's subject are refused by the bus, and a body naming another + observer or node is refused by the keeper; the catalog's validation finds no `findings` in the keeper's `reads`. +- §384 **An observer states every finding it owns at its start and when the keeper asks; the keeper's view is + rebuilt from those events alone.** At each start, and on each ask of the keeper, an observer judges every finding it + owns anew from its own truth, the cause each kind watches (the seat list it holds, a call still unanswered, a wait + still standing), never from what one process remembers, and says `findings-stated` on + `mesh.stream..event.findings-stated.`, carrying `at`, `asked` (the ask it answers, none at a start), + `findings` (every finding it observes now, with `finding-written`'s fields) and `pending`. A wait's bound counts from + the wait's own start, which the observer keeps in its own state when the wait begins (the `since` of its key in + `findings`, written before the bound passes), never from the process's start, so a wait that began before a restart + is judged against its whole age. `pending` names, with their `since` and their bound, the keys the observer cannot + judge yet since its start (a kept call it has not yet made again, a wait it has not yet looked at). The keeper sets + its view of that observer on that node to that set: it raises what is new, keeps each pending key's condition open + and unchanged, saying nothing for it, and clears each other condition of that observer and node the set does not + name, with the reason `not stated at 's start` or `not stated when asked`; a pending key is cleared only if + neither a `finding-written` nor a later set names it within its bound after the set's `at`, with the reason `not + judged within its bound`. For an observer whose catalog entry claims only seats held once for the mesh, the set + replaces its findings of every node, and the keeper takes `finding-written` and `finding-cleared` of it only from + the node the seat list names as its holder, and only with an `at` later than that holder's last `findings-stated`, so + a holder that took over leaves no finding of the one before and an old holder's late event changes nothing. A set + larger than one bus message goes in the observer's sent store (§59). The node-engine's `seat-list-unread` (ADR 0636 + §1) is stated so: a node-engine started with the full list states no `seat-list-unread`, and the keeper clears it at + that start. Why: the node-engine's seat-list waiter kept only in its process that it had written its finding, so a + node-engine restarted with the full list never cleared it, and two machines said they waited for the seat list for a + day while both held it (issue 1966); this departs from ADR 0564 §2 by the operator's answer of 2026-11-11: the fix is + NATS-native, from each observer's own truth. The event, its fields, `pending`, the kept start, the reasons and the + rule for an observer held once are the design's choice. How it is checked: whole (the node-engine, and each observer + through mesh-sdk's writer), a module restarted after raising a finding whose cause has gone clears it at its start, + the node-engine's `seat-list-unread` among them, and an observer restarted with a call unanswered for two hours says + neither `condition-cleared` nor `condition-raised`; whole (keeper), a condition the set does not name is cleared with + its reason, a pending key past its bound is cleared, and in a take-over the new holder's set clears the old holder's + findings and an old holder's later `finding-written` raises nothing; unit (mesh-sdk's writer), a start states the + findings observed now and pends the rest. Test 107. +- §385 **One tool-runner account for every module of a machine stays as it is.** On a machine where one tool runner + serves every module's tools on one bus account, any tools bundle there may read any sent store (§59) that account is + granted, call `merge-pull` or the runner's `run` as a session, and say the finding events of §383 of any module that + account publishes for; G changes neither that account nor its grants, a known limit. Why: the operator, 2026-11-11; + modules running as the machine's account are issue 1966's (note 193), whose fix carries its check. How it is checked: + none in G; the limit is named on issue 1966. +- §386 **A `*` emitter in `consumes` hears one event from every module that declares it.** A `consumes` entry + `*.` takes the event `` from every module whose manifest declares it, and is the one generic form for + hearing an event that many modules say; the keeper's `*.finding-written`, `*.finding-cleared` and + `*.findings-stated` are the first. The manifest validator takes `*` only as the whole emitter, and only before an + event that a seat's definition or a contract in mesh-sdk defines, with its fields (for the finding events, mesh-sdk's + finding writer); it refuses `*` before any other event, `*.>`, and `*` in part of an emitter's name. The grants part, + from the manifests as it derives every grant (§59), makes on each declaring module's stream one durable consumer + for the consumer, named `_` as ADR 0363 §5 names one, with its grants to read and acknowledge it and + its membership's word to bind it; it drops that consumer when the module no longer declares the event, and makes one + for a module that declares it later, with no change to the consumer's manifest. A `*` entry never reaches a + restricted stream (one with a `grant`, ADR 0363 §4). Why: the keeper hears modules it finds in the catalog and its + manifest names none (§7, §365); this departs from ADR 0363 §5, where an entry with a wildcard emitter never reaches + a module's stream, by the operator's answer of 2026-11-11; the validator's rule and the per-module consumer are the + design's choice. How it is checked: unit (manifest validator), `*.finding-written` accepted, `*.>`, + `*x.finding-written` and `*` before an event no contract defines refused; the grants part's table test makes a + `*.finding-written` consumer one durable consumer on each declaring module's stream and none on a restricted + stream; whole (keeper, grants part), a fixture module that declares the events after the keeper started is heard, + and a keeper stopped while findings change catches up from its durable consumers at its start. +- §387 **G's change writes the finding stream into every observer's manifest.** In the same change G writes the stream + `-findings` of §383 into the manifest's `streams`, and `mesh-conditions.findings-asked` into its `consumes`, + for every module that declares a state `findings` on main: `artifact-store-tools`, `claude-code`, `distribution`, `gitea`, + `mesh-build-go`, `mesh-config`, `mesh-healers`, `mesh-knowledge`, `mesh-provisioner`, `mesh-self-check`, `mesh-vault`, `mesh-watcher`, + `minio` and `nfs-server` in mesh-catalog; `mesh-delivery-assigner`, `mesh-delivery-builder`, `mesh-delivery-checker` (until its + removal, ADR 0667 §12), `mesh-delivery-indexer`, `mesh-delivery-teller`, `mesh-delivery-view`, `mesh-delivery-watchman` and + `mesh-enrolment` in mesh-delivery; `mesh-host`, the node-engine; `node-tools`, the tool runner, in mesh-tools; and the modules that gain + findings in G, the runner (`mesh-delivery-pipeline`, §355) and the forge bridge (`mesh-delivery-forge-bridge`, for `branch-unprotected`, §17). The manifest + validator refuses a manifest declaring the state `findings` without that stream and that `consumes`. Why: an + observer whose stream is missing says nothing the keeper hears; the list is read from main on 2026-11-11 and the rule + is the design's choice. How it is checked: unit (manifest validator), a manifest with `findings` and no finding + stream refused; the catalog's validation at G's heads refuses none of these manifests. +- §388 **The keeper keeps its sources in its own state, and raises its own findings itself.** The keeper keeps its + list of sources (each module whose catalog entry declares the finding stream, with the nodes it heard and each + observer's last stated set and open asks) in its own state `findings-views`, in the form of its `conditions`, one key + per observer and node, deleted when the module leaves the catalog (§365) or the node (§365). At its start it applies + the events of those sources at once from its durable consumers, before its first view of the catalog half; the view + only adds and removes sources. A source new to the list is asked with `findings-asked` naming it once its durable + consumer exists (§386). The keeper's own findings (`view-waits`, `findings-unstated`, `call-unanswered`) it raises and + clears directly as the one writer of conditions, never through its own events. A missing assignment half holds only + the clears of `module left the node`. Why: a keeper that waits for its view would hear nothing after a restart while + the catalog is down; the state and its key are the design's choice. How it is checked: whole (keeper), started with + the catalog's holder stopped it raises and clears from its kept sources' events, raises `view-waits` after five + minutes itself, and clears nothing with `module left the node` until the assignment half answers. +- §389 **The watchman says a machine's presence as events.** The seat `mesh-watchdogs` gains the events `machine-away` + and `machine-back` (`mesh.seat.mesh-watchdogs.event.machine-away.`, `.machine-back.`, carrying the node, + `at` and since when it was last heard), said by the watchman when its presence key of a machine expires and when the + machine is heard again (ADR 0564 §4), declared in the seat's definition and kept 90 days, 16 MiB. The keeper + consumes them for `findings-unstated` (no machine away is named, and an ask is said again to a module of a machine + back), and the assigner for its judgement of `version-delivered` (§376); neither reads the watchman's state. Why: + ADR 0596 §2; the events and their fields are the design's choice. How it is checked: whole (watchman), a machine + whose presence expires says `machine-away` and one heard again `machine-back`; whole (keeper), an observer on a + machine away is not named in `findings-unstated`. +- §390 **The keeper's ask, its tools, and its hold.** The keeper asks with its seat's event `findings-asked` + (`mesh.seat.mesh-conditions.event.findings-asked`, carrying `ask`, `at`, `by` and, to ask one, `module`), declared + in the seat's definition, kept 7 days and 16 MiB; an observer hears it through mesh-sdk's writer on a consumer that + starts from the newest ask, so an old ask is never answered again. The keeper asks after its start (§388) and on + `resync` (ADR 0564 §12), which gains an optional `module` and so reads no state; its callers are the operator, the + healers and agent sessions, and each call is written in the hand-act log of the seat `mesh-hand-acts`. An observer + that states nothing within five minutes of an ask, its machine not away (§389), is the keeper's finding + `findings-unstated`, naming the module and the node, cleared by its next `findings-stated`; its conditions stand + meanwhile. `findings-unstated` is held under ADR 0564 §13's hold while its module is being delivered, and G's + delivery plan says it may show while the observers move. The keeper's tool `conditions_findings_views` answers each + observer and node of `findings-views`: its last stated set with its `at`, its pending keys and the open asks; its + callers are any person, agent sessions and the healers. Why: ADR 0564 §12 has every act of the keeper started by + hand, and the tools-first rule a read of all it keeps; the names, the callers and the bound are the design's choice. + How it is checked: whole (keeper), `findings-unstated` is cleared by a `resync` naming that one module, the call is + in the hand-act log, an observer restarted after an old ask answers only a new one, and the tool answers a fixture + observer's set. diff --git a/03-DESIGN/01-to-be/47-milestones/G.md b/03-DESIGN/01-to-be/47-milestones/G.md index eb95d6e1..de83d5c5 100644 --- a/03-DESIGN/01-to-be/47-milestones/G.md +++ b/03-DESIGN/01-to-be/47-milestones/G.md @@ -75,7 +75,7 @@ file on 2026-11-10, and is cited as "design 47 G §n". A sentence that holds a d stayed in the main file, and G builds what it says from G's delivery. A section named in italics is in this part, or, when it is not here, in the main file. -The part is seven files, each read whole, its design lines numbered across them: +The part is eight files, each read whole, its design lines numbered across them: - [G-1-pipelines-registration-and-steps](G-1-pipelines-registration-and-steps.md): pipelines per module: registration, watched branches, steps and policy, what holds a delivery, approval, groups, dependencies, a publish. - [G-2-pipelines-sizes-checks-and-tools](G-2-pipelines-sizes-checks-and-tools.md): pipelines per module: any size, the pipeline makes the check, who may say what, health, recovery, tools, kept data, how G itself is delivered, the choices under ADR 0666 and ADR 0667. @@ -83,4 +83,5 @@ The part is seven files, each read whole, its design lines numbered across them: - [G-4-revision-view-and-steps](G-4-revision-view-and-steps.md): the revision view, what ADR 0656 leaves to this design, the choices under ADR 0656, what each step says and their tools. - [G-5-acting-parts-check-and-kept-data](G-5-acting-parts-check-and-kept-data.md): the acting parts, the removed hold, the check on the change's own acting parts, after the merge, taken in, and kept data on the bus. - [G-6-verbs-and-retires](G-6-verbs-and-retires.md): the verbs G adds and what retires. -- [G-7-answers](G-7-answers.md): the answers to the builders', the reviewers' and the operator's questions (§322 on). +- [G-7-answers](G-7-answers.md): the answers to the builders', the reviewers' and the operator's questions (§322 to §382). +- [G-8-findings-heard-as-events](G-8-findings-heard-as-events.md): the operator's answers of 2026-11-11, findings heard as events (§383 on). diff --git a/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md b/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md index 44f758a2..0daf1a1c 100644 --- a/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md +++ b/03-DESIGN/01-to-be/51-the-registries-keep-what-is-named.md @@ -11,7 +11,7 @@ code: - mesh-catalog modules/docker (image pruning) - mesh-host internal/apply (every new image of a module pulled before any of its containers is replaced; a failed pull applied again on the status view's `settled` for the store's holder, and at the node-engine's start), internal/compose (each routed image reference written with its seat and that seat's holder) - mesh-delivery modules/mesh-grants (what a node-engine hears: the status view's `settled`) -updated: 2026-11-10 +updated: 2026-11-11 decisions: - "ADR 0667" - "ADR 0666" @@ -130,7 +130,8 @@ the verb. ▼ registry seat's holder ── answers the parts it lacks │ the build module sends those parts, in chunks, by `publish` - holder: hashes what it received, asks `builder_versions`, + holder: hashes what it received, asks `builder_versions` + (from milestone G of design 47: `mesh-builds`' `reserved`), stores in its own store, emits `published` or `refused` ▼ builder (durable consumer: entry, done key), indexer, status view, checker