From 34384b6c9cfd59644165e6cb6864adbd6754c451 Mon Sep 17 00:00:00 2001 From: jochen Date: Wed, 4 Nov 2026 00:10:19 +0100 Subject: [PATCH] =?UTF-8?q?Design=2025=20=C2=A7=2012:=20answer=20the=20fir?= =?UTF-8?q?st=20review=20of=20the=20milestone=20amendment=20(the=20bus=20m?= =?UTF-8?q?odule's=20first=20view=20against=20replays=20and=20an=20unanswe?= =?UTF-8?q?red=20tool,=20the=20foundation=20readers'=20bounds,=20families?= =?UTF-8?q?=20and=20tools,=20the=20tests=20they=20join,=20the=20open=20gra?= =?UTF-8?q?nt)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig --- 03-DESIGN/01-to-be/25-the-bus-on-nats.md | 42 ++++++++++++++++++------ 1 file changed, 32 insertions(+), 10 deletions(-) diff --git a/03-DESIGN/01-to-be/25-the-bus-on-nats.md b/03-DESIGN/01-to-be/25-the-bus-on-nats.md index 2e330753..8d0c4a4b 100644 --- a/03-DESIGN/01-to-be/25-the-bus-on-nats.md +++ b/03-DESIGN/01-to-be/25-the-bus-on-nats.md @@ -697,13 +697,20 @@ modules' names meet. - **The trigger is the judge's verdict** (ADR 0595 §4), heard as the indexer's event `manifest-accepted`, which carries the module's entries whole, so the bus module never reads the indexer's state and never polls. The indexer says it for every newly accepted manifest, with its entries empty when it asks for none, and the bus module reads an - empty one as "this module asks nothing now", which is what its orphans need. It hears the verdicts on a durable - consumer, so one given while it was down is heard when it is back. **Its first view** at start is the indexer's + empty one as "this module asks nothing now", which is what its orphans need. It goes, as every event of the + indexer, on the module events stream (ADR 0363) under the indexer's own subjects. A publish of it that fails is + published again, and one still failing writes the indexer's finding naming the module, cleared once said (today + the indexer only logs it). It hears the verdicts on a durable consumer, so one given while it was down is heard + when it is back. **Its first view** at start is the indexer's tool `indexer_bus_state` (every accepted manifest's bus-state entries, by module and digest), asked once; the - events keep it current after that, and a stream that keeps a verdict only for its age is never its source. It keeps + events keep it current after that, and a stream that keeps a verdict only for its age is never its source. A + verdict replayed from its consumer is applied only if newer than what the first view answered for that module (by + the manifest's digest and the indexer's revision), so a replay never takes an entry back. An `indexer_bus_state` + that does not answer makes the bus module's health unhealthy, naming the verb, and is asked again on a bounded + backoff (from 10 seconds, doubling, at most every 5 minutes); nothing is created or orphaned meanwhile. It keeps what it last heard per module in its own state `asked` (an old `state` section, its own kept data staying there, - ADR 0595 §10), from which `list`, `check` and `orphans` answer; a key goes when the module's accepted manifest asks - for nothing, or the first view no longer names the module. Then it runs `check` over what it holds, creating what + ADR 0595 §10), from which `list`, `check` and `orphans` answer; a key goes only on an empty `manifest-accepted` for + the module, never because a manifest the judge set aside to wait is missing from the first view. Then it runs `check` over what it holds, creating what is missing. The judge checks every entry once, at the merge check and at take-in (ADR 0578; ADR 0595 §8), and also each ledger type's operations: a set-entry or a remove names `into` and `key`, `fields` and an index's `on` name fields of the schema, and `digest-of` names a text field. @@ -740,13 +747,26 @@ vault as an own secret: JetStream create and update under modules' names only (s through the tool runner, so the tool runner never asks through itself: the assigner serves `placement` to the grants part, the tool runner and the indexer, `seat-holders` to the tool runner and `wants` to the grants part; the indexer serves `manifests` and `modules` to the grants part, the tool runner and the assigner (ADR 0615 §3). Each - reader declares these as verbs it invokes. The owner answers at once from what it holds, as soon as its process is + reader declares these as verbs it invokes. Serving a verb on a process's own subject, outside the tool runner, has + no manifest form today (the assigner serves its seat verbs as tools); the form is new in mesh-sdk's manifest types + and rides the milestone's one rebuild. Which part issues these subjects' grants first is not yet decided: the + grants part composes every grant from the manifests, and is itself one of these readers, so its own first view + cannot wait on a grant it has not composed (an open point, tracked on issue 1966). The owner answers at once from what it holds, as soon as its process is connected and before it has its own first view of any other part, with the revision of its state it answered at, so the indexer and the assigner, each asking the other at start, never wait on each other (§4). The reader then follows the owner's declared events, each carrying the revision of the state it follows, and never asks the verb - again on a timer; a gap in revisions makes it ask once more, said in its log (§5). A reader that starts while its - owner is down raises a condition naming the owner and the verb past its bound and keeps asking until it answers, - never reading the owner's state instead; once it had its first view, it keeps working on what it heard (§6). The + again on a timer; a gap in revisions makes it ask once more, said in its log (§5). The owners' events are new + families of their own, apart from the indexer's `manifest-refused` and `manifest-accepted` and the assigner's + `put`: the indexer's `indexer.views` and the assigner's `assigner.views`, one event per changed key with its + bucket, key and revision, kept seven days, declared in each owner's old `streams` section (its own kept data), + so a waiter on an existing event never wakes on them. A reader that starts while its owner is down raises a + condition naming the owner and the verb past its bound and keeps asking until it answers, never reading the + owner's state instead; once it had its first view, it keeps working on what it heard (§6). The bound is longer + than an owner's start (two minutes), so a normal start, readers and owners coming up together, notifies nobody; + the owner's answer clears the condition. An answer that is empty at revision 0 (an owner that holds nothing yet) + is a first view of nothing, never a reason for the grants part to drop a login it holds: it keeps the list in + force until an answer at a revision above 0. Each reader's tools say the owner, verb and revision its first view + holds (a read tool), and ask the first view again by hand (a re-read tool, recorded as a hand act). The order of owner before reader is kept by the module graph's served-verb edge for the 19 readers (ADR 0614 §2), and, for the four, whose verb edges lie on cycles the graph leaves out (ADR 0614 §3), by the owner's answer at once and the reader's wait (ADR 0615 §2). @@ -919,7 +939,9 @@ issue 1966. | The bus module creates, and the binding follows | live, by tool call: a module asking for a state, a log, objects and a ledger in `requires` receives each binding in its membership, written after `created`, never before; `show` gives each entry's template, settings and fill; a module calling `create` is refused | | The bus module runs whole | one integration test in the nats module's repository that starts the bus module whole against the real bus server: a verdict, then `created`, then the binding in the membership; a restart that catches up a verdict given while it was down; a bucket whose settings differ raises `drifted`, the seat's `apply` of a larger cap clears it with `drift-cleared`; a refused create raises its condition and hands no binding; `recreate` and `drop` without the operator's warrant are refused, and with it act after a stream snapshot; an orphan is listed, raises `orphaned`, and goes only by `drop`; each template at a small cap refuses, events drop the oldest, and `full` is raised at the raise mark and cleared only below the clear mark; `recreated` rewrites the binding and `dropped` removes it; at start the first view from `indexer_bus_state` creates what a verdict given while it was down asked; the older remove tools refuse a new-form entry, naming `drop`'s tool | | A module asking bus state waits for the bus module | the module graph's table test: a manifest with a ledger entry has a requirement edge to the module providing `bus-state`, one without has none (ADR 0614 §5) | -| A foundation part's first view | unit tests of each owner and reader (ADR 0615's How it is checked): the owner answers its verb at once from what it holds, with its revision, before its own first view; the reader asks once at start, then only on a gap in revisions; a reader whose owner is down raises its condition past the bound and never reads the owner's state; the four readers' manifests hold no `reads` of the indexer's or the assigner's state | +| A foundation part's first view | unit tests of each owner and reader (ADR 0615's How it is checked), as table cases, and these cases of the whole-module integration tests that exist today (mesh-delivery is over its integration budget, so no new test function): the indexer's (`cmd/mesh-delivery-indexer`, its one whole-module test, which the milestone's packages test merges into), the assigner's (`cmd/mesh-delivery-assigner`) and the grants part's (`cmd/mesh-grants`), and the tool runner's in mesh-tools: the owner answers its verb at once from what it holds, with its revision, before its own first view; the reader asks once at start, then only on a gap in revisions; a reader whose owner is down raises its condition only past the bound and never reads the owner's state, and the owner's answer clears it; an empty answer at revision 0 drops no login; an existing waiter on `manifest-accepted` does not wake on the new families; the read tool shows the revision each reader holds. Live, after the delivery: the indexer's count of `reads` entries on the indexer's and the assigner's state reads 0 | +| The bus module's first view | cases in the nats module's one integration test: a replayed verdict older than the first view is not applied; an `indexer_bus_state` that does not answer turns the health unhealthy naming it and is asked again on the backoff; a set-aside manifest missing from the first view keeps its `asked` key, an empty `manifest-accepted` removes it | +| `manifest-accepted` is never lost silently | a unit test of the indexer: a failed publish is published again, and one still failing writes the finding | | Until the identity seat, the caller is the one the tool runner proved | a unit test of the bus module's process: a request on its tool subject naming another caller in its body acts for the proved caller; `drop` without the warrant is refused | | Drift and orphans, live | live, made only by a manifest change: a manifest that shrinks a setting raises `drifted`; the conditions clear as above. Nothing is filled on the live bus, and no orphan is made there | | Overdue and stopped are loud | live, carried by the change of the first module that moves in step 5, never by a merge made for the test alone: with the bus module's process stopped, an accepted manifest asking for an entry raises the overdue condition within five minutes, from the watchman, and a later `created` clears it; the bus module's health turns unhealthy when its pass is more than three minutes old |