Merge pull request 'Designs 25, 51: the registry milestone's prepare points (a failed pull retried on the store holder's settled, the owners' read tools, verdicts as a first view, ADR 0622)' (#512) from docs/registry-prepare-points into main
This commit was merged in pull request #512.
This commit is contained in:
@@ -16,6 +16,7 @@ decisions:
|
||||
- "ADR 0619"
|
||||
- "ADR 0615"
|
||||
- "ADR 0621"
|
||||
- "ADR 0622"
|
||||
- "ADR 0614"
|
||||
- "ADR 0613"
|
||||
- "ADR 0612"
|
||||
@@ -752,13 +753,18 @@ vault as an own secret: JetStream create and update under modules' names only (s
|
||||
ADR 0621). The four
|
||||
foundation parts that read each other (the grants part, the tool runner, the indexer and the assigner) stop
|
||||
reading the indexer's and the assigner's bus state in the milestone of ADR 0612, with the other 19 readers of that
|
||||
state (ADR 0615 §1). **Each owner serves them as tools of its own module**, the indexer `manifests` and
|
||||
`modules`, the assigner `placement`, `seat-holders` and `wants`, and declares `runs-as`, so its tools bundle runs in
|
||||
state (ADR 0615 §1). **Each owner serves them as tools of its own module**, the indexer `manifests`,
|
||||
`modules` and `verdicts` (ADR 0622 §1), the assigner `placement`, `seat-holders` and `wants`, and declares `runs-as`, so its tools bundle runs in
|
||||
a runtime of its own on the module's own bus account, never in the node's tool runner, and the tool runner never
|
||||
asks through itself (ADR 0621 §1). No new manifest form is added: a process still answers no tool call or seat verb
|
||||
(ADR 0436 §3). **Each verb is granted to exactly its readers** (ADR 0621 §2, as ADR 0615 §3 lists them): `placement`
|
||||
to the grants part, the tool runner and the indexer; `seat-holders` to the tool runner; `wants` to the grants part;
|
||||
`manifests` and `modules` to the grants part, the tool runner and the assigner. A reader declares each in `invokes`
|
||||
`manifests` and `modules` to the grants part, the tool runner and the assigner; `verdicts` to the grants part
|
||||
alone, its `callers` list holding the grants part only (ADR 0622 §1). These six are the first-view tools.
|
||||
`verdicts` follows the first-view rules whole (ADR 0622 §2, under ADR 0615 §4 to §6): its answer carries the
|
||||
revision, the grants part then follows `manifest-accepted` and `manifest-refused` and asks once more on a gap in
|
||||
their revisions, it raises the condition naming the indexer and `verdicts` past the bound and never reads the
|
||||
indexer's state, and `grants_first_views` and `grants_first_view_again` cover it. A reader declares each in `invokes`
|
||||
as `<module>.<tool>` only where it is granted, which is the grant the grants part composes as for any tool, and
|
||||
each tool checks the proved caller against a `callers` list holding exactly its readers and refuses any other, the
|
||||
owner itself included. **The judge of the milestone is its own group's** (ADR 0621 §4): since no field is added,
|
||||
@@ -815,6 +821,21 @@ vault as an own secret: JetStream create and update under modules' names only (s
|
||||
part and the tool runner their first views while the bus module still waits on it. The order of owner before reader is kept by the module graph's served-verb edge for the 19 readers (ADR 0614 §2), and,
|
||||
for the four, whose verb edges lie on cycles the graph leaves out (ADR 0614 §3), by the owner's answer at once and
|
||||
the reader's wait (ADR 0615 §2).
|
||||
- **The other 19 readers ask the owners' read tools, one per bucket they read** (ADR 0615 §1, §2). Beside the six
|
||||
first-view tools above, each owner serves a read tool for every bucket of its state one of the 19 reads today,
|
||||
named `indexer_read_<bucket>` or `assigner_read_<bucket>` and marked as a read: fifteen. The indexer's nine:
|
||||
`indexer_read_manifests`, `indexer_read_modules`, `indexer_read_predefined`, `indexer_read_config_candidates`,
|
||||
`indexer_read_candidates`, `indexer_read_packages`, `indexer_read_graph`, `indexer_read_heard` and
|
||||
`indexer_read_taken`. The assigner's six: `assigner_read_placement`, `assigner_read_seat_holders`,
|
||||
`assigner_read_wants`, `assigner_read_findings`, `assigner_read_asks` and `assigner_read_differences`. These are
|
||||
the nine buckets of the indexer and the six of the assigner the 19 read (§ 12.6), `modules`, `wants` and
|
||||
`placement` among them for the status view; the indexer's tenth, `verdicts`, is read by the grants part alone,
|
||||
through its first-view tool. **They are ordinary module tools**: granted, as any tool is, to every
|
||||
reader that declares it in `invokes`, and callable by a session; the `callers` list of ADR 0621 §2 and ADR 0622 §1 covers only the six
|
||||
first-view tools. **The status view keeps all its tools**, their names, arguments and answers
|
||||
unchanged, its `indexer_*` and `assigner_*` tools included: as a reader it answers them by asking the owners' read
|
||||
tools instead of reading their buckets. Checked live after the delivery: a session's call of each status view tool
|
||||
answers, and `mesh_search` finds each owner's read tool by its bucket's words.
|
||||
- **Only the owner reaches it** (ADR 0596 §2). The membership grants the owner's account its own entries and no
|
||||
other module's: the new form has no readers. What another module needs of that data it asks through the owner's
|
||||
tools or seat verbs, and what the owner lets others hear is its declared events, subscribed to through
|
||||
@@ -969,12 +990,17 @@ this milestone go on their old sections too (§ 12.2; design 51 *The move*; desi
|
||||
|
||||
**Readers move after their owners** (ADR 0596 §3). Today 195 `reads` entries in 37 modules read another module's
|
||||
bus state (counted 2026-11-03 02:00). **A correction of fact**: the indexer's and the assigner's state, counted again
|
||||
from every manifest on main later that day, is read by 23 modules, 9 buckets of the indexer and 6 of the assigner,
|
||||
from every manifest on main later that day, is read by 23 modules, 10 buckets of the indexer (`verdicts`, read by the grants part, among them) and 6 of the assigner,
|
||||
not by the 19 the first plan counted; and 151 state entries were counted, not 158 (the template table of § 12.1 keeps
|
||||
the first count, which chose the templates). The owners first gain their events and first-view tools: the indexer
|
||||
one per bucket its readers read, the assigner the same. Then all 23 readers move in the milestone of ADR 0612, the
|
||||
four foundation readers included, by the first view of § 12.2 (ADR 0615 §1); their order is the module graph's, not
|
||||
a review reading an owner delivered (ADR 0615 §2). The judge still refuses a moved module's subscription to an event
|
||||
the first count, which chose the templates). The owners first gain their events, first-view tools and read tools: the
|
||||
indexer a read tool per bucket its readers read, the assigner the same (§ 12.2 lists them). Then all 23 readers move in the milestone of ADR 0612, the
|
||||
four foundation readers included, by the first view of § 12.2 (ADR 0615 §1), the grants part's of the indexer's
|
||||
`verdicts` by its first-view tool `verdicts` (ADR 0622) and then `manifest-accepted` and `manifest-refused`; their order is the module graph's, not
|
||||
a review reading an owner delivered (ADR 0615 §2). The status view's move keeps every tool it serves, each answered by
|
||||
asking the owners' read tools of § 12.2. **The node-engine's reads granted in code stay**: the grants part's code
|
||||
grants each node's node-engine, with no manifest declaring them (`CandidateReads` and the composed reads beside it),
|
||||
the indexer's `candidates` and `verdicts` and its own node's keys of the assigner's `wants`. They are not among the
|
||||
23 readers of ADR 0615 §1 and move in the next milestone, queued on issue 1966. The judge still refuses a moved module's subscription to an event
|
||||
its owner does not declare. Every other reader moves when its module is next touched. These moves are tracked by
|
||||
issue 1966.
|
||||
|
||||
@@ -986,10 +1012,10 @@ issue 1966.
|
||||
| The bus module creates, and the binding follows | live, by tool call: a module asking for a state, a log, objects and a ledger in `requires` receives each binding in its membership, written after `created`, never before; `show` gives each entry's template, settings and fill; a module calling `create` is refused |
|
||||
| The bus module runs whole | one integration test in the nats module's repository that starts the bus module whole against the real bus server: a verdict, then `created`, then the binding in the membership; a restart that catches up a verdict given while it was down; a bucket whose settings differ raises `drifted`, the seat's `apply` of a larger cap clears it with `drift-cleared`; a refused create raises its condition and hands no binding; `recreate` and `drop` without the operator's warrant are refused, and with it act after a stream snapshot; an orphan is listed, raises `orphaned`, and goes only by `drop`; each template at a small cap refuses, events drop the oldest, and `full` is raised at the raise mark and cleared only below the clear mark; `recreated` rewrites the binding and `dropped` removes it; at start the first view from `indexer_bus_state` creates what a verdict given while it was down asked; the older remove tools refuse a new-form entry, naming `drop`'s tool |
|
||||
| A module asking bus state waits for the bus module | the module graph's table test: a manifest with a ledger entry has a requirement edge to the module providing `bus-state`, one without has none (ADR 0614 §5) |
|
||||
| A foundation part's first view | unit tests of each owner and reader (ADR 0615's How it is checked), as table cases, and these cases of the whole-module integration tests that exist today (mesh-delivery is over its integration budget, so no new test function): the indexer's (`cmd/mesh-delivery-indexer`, its one whole-module test, which the milestone's packages test merges into), the assigner's (`cmd/mesh-delivery-assigner`) and the grants part's (`cmd/mesh-grants`), and the tool runner's in mesh-tools: the owner answers its verb at once from what it holds, with its revision, before its own first view; the reader asks once at start, then only on a gap in revisions; a reader whose owner is down raises its condition only past the bound and never reads the owner's state, and the owner's answer clears it; an empty answer at revision 0 drops no login; an owner's event carries the changed value whole, and a reader applies a put and a delete from the events alone and never asks the verb; a revision lower than the one held makes the reader ask its first view again; an existing waiter on `manifest-accepted` or on the assigner's `put` does not wake on the new families; the read tool shows the revision each reader holds, and the re-read tool asks once and records the hand act; in the
|
||||
| A foundation part's first view | unit tests of each owner and reader (ADR 0615's How it is checked), as table cases, and these cases of the whole-module integration tests that exist today (mesh-delivery is over its integration budget, so no new test function): the indexer's (`modules/mesh-delivery-indexer`, its one whole-module test, which the milestone's packages test merges into), the assigner's (`modules/mesh-delivery-assigner`) and the grants part's (`modules/mesh-grants`), and the tool runner's in mesh-tools: the owner answers its verb at once from what it holds, with its revision, before its own first view; the reader asks once at start, then only on a gap in revisions; a reader whose owner is down raises its condition only past the bound and never reads the owner's state, and the owner's answer clears it; an empty answer at revision 0 drops no login; an owner's event carries the changed value whole, and a reader applies a put and a delete from the events alone and never asks the verb; a revision lower than the one held makes the reader ask its first view again; an existing waiter on `manifest-accepted` or on the assigner's `put` does not wake on the new families; for the indexer's `verdicts` (ADR 0622), called by the grants part while the node's tool runner is stopped, it answers, and from any other account, the indexer's own included, it is refused; a missed revision on `manifest-refused` makes the grants part ask `verdicts` once more; the grants part and the indexer started at the same moment, `verdicts` answers within the bound with its revision; with the indexer down past the bound the grants part raises the condition naming the indexer and `verdicts` and never reads the indexer's state, and the answer clears it; the read tool shows the revision each reader holds, and the re-read tool asks once and records the hand act; in the
|
||||
| The owners' first views are their own tools, in a runtime of their own (ADR 0621) | cases of the indexer's and the assigner's whole-module integration tests: each first-view tool, called from a reader's account while the node's tool runner is stopped, answers (§1); each answers its granted readers and refuses every other account, the owner's own included (§2); the two runtimes started at the same moment, each asking the other's first view at start, both answer within the first-view bound with their revisions (§3); and the milestone's group passes its merge check with no change to the judge's known fields for these verbs (§4) |
|
||||
assigner's whole-module test, the assigner starts before any new-form bus state exists, with no binding of the bus
|
||||
module, and the indexer, the grants part and the tool runner each get their first view of it. Live, after the delivery: the indexer's count of `reads` entries on the indexer's and the assigner's state reads 0 |
|
||||
module, and the indexer, the grants part and the tool runner each get their first view of it. Live, after the delivery: the indexer's `indexer_old_sections`, which the milestone extends to count the `reads` entries on each owner's state, reads 0 declared `reads` entries on the indexer's state and 0 on the assigner's, and says the node-engine's reads granted in the grants part's code remain (§ 12.6); and each owner's read tool of § 12.2 answers what its bucket holds, each status view tool answers a session's call as before the move, and `mesh_search` finds each owner's read tool |
|
||||
| A foundation part's first view comes within its bound | live, after the delivery: each reader's read tool (`grants_first_views`, the mesh MCP server's `mesh_first_views`, `indexer_first_views`, `assigner_first_views`) shows, for the indexer's and the assigner's verbs, the time from its ask to the owner's answer under the two-minute bound, and no first-view condition raised by the delivery |
|
||||
| The bus module's first view | cases in the nats module's one integration test: a replayed verdict older than the first view is not applied; an `indexer_bus_state` that does not answer turns the health unhealthy naming it and is asked again on the backoff; a set-aside manifest missing from the first view keeps its `asked` key, an empty `manifest-accepted` removes it; a deleted module's entry, after its empty `manifest-accepted`, is listed by `orphans` and raises `orphaned`, and a set-aside module's entry is neither; `nats_first_view_again` asks at once and records the hand act |
|
||||
| `manifest-accepted` is never lost silently | unit tests of the indexer: a failed publish is published again, and one still failing writes the finding; `indexer_say_accepted` says it again for one module from the verdict held, and refuses a module with no accepted verdict |
|
||||
|
||||
@@ -9,7 +9,8 @@ code:
|
||||
- mesh-delivery modules/mesh-delivery-builder (versions, tag asks), the forge bridge, the indexer, the checker, the assigner
|
||||
- mesh-sdk (the bus helper for sending in parts)
|
||||
- mesh-catalog modules/docker (image pruning)
|
||||
- mesh-host internal/apply (every new image of a module pulled before any of its containers is replaced)
|
||||
- mesh-host internal/apply (every new image of a module pulled before any of its containers is replaced; a failed pull applied again on the status view's `settled` for the store's holder, and at the node-engine's start), internal/compose (each routed image reference written with its seat and that seat's holder)
|
||||
- mesh-delivery modules/mesh-grants (what a node-engine hears: the status view's `settled`)
|
||||
updated: 2026-11-04
|
||||
decisions:
|
||||
- "ADR 0605"
|
||||
@@ -194,8 +195,32 @@ the verb.
|
||||
composition, on the node-engine's verb `compose` naming the module (by hand), or when its liveness judge finds a
|
||||
declared resource missing or down; only a failure of the bus itself is tried again on a clock (a bounded backoff,
|
||||
at most four times before the have says failed). So another module's apply whose pull falls in that window stays
|
||||
failed, its running containers kept, until one of those: its have says failed naming the image, and `compose`
|
||||
naming the module is its recovery.
|
||||
failed, its running containers kept, until one of those: its have says failed naming the image.
|
||||
**So a pull that falls in the swap is tried again when the store's holder has settled, never on a timer.** Neither
|
||||
the condition keeper's `condition-cleared` (ADR 0564 §3) nor the status view's `delivered` comes in the swap: a
|
||||
finding of the holder held during its own delivery raises no condition and so no clear (ADR 0564 §13 (c)), and
|
||||
`delivered` waits for every have to equal its want, which the failed have itself blocks (design 47). The status
|
||||
view's `settled`, said once for a module on a node at one composition when its have reaches its want running and
|
||||
healthy (design 47), is blocked by nothing else. `distribution` claims a seat held once for the mesh and runs on
|
||||
one node, so the node-engine on every node hears `settled` for the store's holder, on whichever node the holder
|
||||
runs, and then applies once more each module on its own node whose have says failed naming an image of that store,
|
||||
said in its journal. **No `settled` is missed in a race**: the node-engine keeps in memory the last `settled` it heard
|
||||
for each holder since its own start, never reading the status view's state (ADR 0596). When it records a pull
|
||||
failed against a store, it compares the pull's start with that event's time: only when a `settled` for that
|
||||
holder was heard after this pull started does it apply again at once. That is the one retry, once per `settled`: a
|
||||
second failure after the same `settled`, or the at-once retry failing again, starts no further apply. It finds the holder by the seat the failed image came through, read from its
|
||||
declaration, never by a module's name: **the composition writes, beside each image reference it routes, the seat the
|
||||
reference came through and that seat's holder**, taken from the holdings the composition already has, so the retry
|
||||
matches `settled` by the holder its declaration names. A change of the seat's holder composes the declaration
|
||||
again, so the name it carries is the holder of now. The seat and the holder are not part of what a container is
|
||||
(they are not in its digest), so a declaration composed again for them replaces no container. **The seam that changes is the grants part's list of what a node-engine
|
||||
hears** (`NodeAgentHears`; the node-engine declares no `consumes` of its own): it gains one entry, the status
|
||||
view's `settled` family, which the node-engine filters for the holder of the seat its failed image names. **At its
|
||||
own start** the node-engine also applies once more every have that failed naming an image of a store, so a restart
|
||||
or an absence across the swap misses nothing. A module already being applied is not applied a second time; a retry
|
||||
whose pull fails again says failed again and starts no further apply. A `settled` of any other module starts
|
||||
nothing, and no clock starts an apply. `compose` naming the module stays, but only as the hand recovery of a pull
|
||||
still failed after that, recorded as a hand act; it is no step of the delivery.
|
||||
- **The package seats' read sides need no credential** for the mesh's own packages: the forge serves the mesh owner's
|
||||
packages to anonymous reads, so the provisions `npm-package-registry` and `go-package-registry` carry no
|
||||
credential (form none), and mesh-build-node, which held a pair for both writing and installing, requires none (the
|
||||
@@ -463,6 +488,21 @@ test).
|
||||
- **Node-engine, unit** (a case of its apply test, with the runner it already fakes): a module of two containers
|
||||
whose second new image cannot be pulled replaces neither container, says the image, and keeps both running; with
|
||||
both images pullable, both are pulled before the first container is set aside.
|
||||
- **Node-engine, unit** (cases of its apply test, with the runner, the bus and the clock it already fakes): a
|
||||
module whose pull failed, its have failed naming an image of a store, is not applied again with the clock moved
|
||||
past the bus backoff's four tries and no event heard; it is applied once more, and its new image pulled, on a
|
||||
`settled` for the holder of the seat its image came through, a `settled` said for the holder on another node
|
||||
included; a pull that fails after a `settled` for its holder heard since that pull started is applied again at
|
||||
once, while a `settled` heard before the pull started applies nothing at once, a second failure after the same
|
||||
`settled` applies nothing, and a race retry that fails again starts no further apply; a
|
||||
`settled` of another module applies nothing; at the
|
||||
node-engine's start it is applied once more with no event; a module being applied when `settled` arrives is not
|
||||
applied twice; a retry whose pull fails again says failed again and starts no further apply.
|
||||
- **Node-engine's composition, unit** (cases of its composition test): a declaration's routed image reference names
|
||||
the seat it came through and that seat's holder; a change of the seat's holder composes the declaration again with
|
||||
the new holder named and recreates no container.
|
||||
- **The grants part, unit** (a case of its composition test): a node-engine's subscriptions hold the status view's
|
||||
`settled` family beside the backup holder's word, and nothing more.
|
||||
- **Checker.** A changed build module invoking a verb no running holder serves is not run, and the verdict names the
|
||||
modules composed from their manifests instead, and the verb.
|
||||
- **Vault.** Reverting the milestone's merge mints the build modules' credentials again; the bootstrap's one-off
|
||||
@@ -474,7 +514,9 @@ test).
|
||||
shown by `builder_versions`; a node runs a module from a reference in the new form; a node-engine away across the
|
||||
milestone wakes and applies the new node-engine; the node-engine's apply journal for `distribution` on each node
|
||||
shows every new image pulled before its first container was replaced, and no node's pull of any module failed
|
||||
during the delivery (each node's haves, none failed naming an image); mesh-build-node installs from a lock file
|
||||
during the delivery (each node's haves, none failed naming an image), or, where one fell in the swap, its have
|
||||
turned healthy after the status view's `settled` for the store's holder, the node-engine's journal naming that
|
||||
`settled`, and no `compose` was called for it; mesh-build-node installs from a lock file
|
||||
naming the registry's public root with no credential. Go's `direct`: its removal is named on the milestone's issue,
|
||||
and the change that removes it adds to the Go build module's recipe test a case that refuses `direct` in the fetch.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user