Take done keys in from a durable pull consumer, a batch at a time (issue 1945, ADR 0544)
The take-in watched the builder's and the indexer's buckets, started a follower per module that retried on a 30 s timer, and at start took every catalogued module in alphabetically: on 2026-10-25 a restart took in about 2 modules a minute, about 190 followers queued behind 16 slots, and every take-in composed two or three times even with nothing new, so a new build waited 30 minutes and more for its credential. Now two durable consumers named mesh-controller-take-in read the indexer's modules bucket (every module's last trunk value when the consumer is made, then each write) and the builder's builds bucket (each write), at most 16 keys unacknowledged each. The bus keeps what is not taken in: the start's look is the consumers' own catch-up, and a restart costs only what was left unacknowledged. A batch reads the catalogue once, takes each module in once, and asks the composer twice: once for its modules' nodes, once for the bus's node with the grants. A module registered at its trunk whose compositions and grant exist, or that this process took in at the same trunk, composes nothing; a key delivered again is taken in whole. A wait for the builder's or indexer's next write is acknowledged, a wait on a read or a failure refused with a 30 s delay. The backlog is in status, and a backlog standing still for 5 minutes raises take-in-stalled. The controller's account is granted the two consumers' acks. The follower tests go with the followers: twelve tests become three. Research 072: the take-ins on a bounded pull consumer (milestone 4, done keys alone) and the start's look composing only what changed (milestone 2).
This commit is contained in:
@@ -111,6 +111,9 @@ type meshStatus struct {
|
||||
// why they could not be read.
|
||||
Pending []pendingStatus `json:"pending,omitempty"`
|
||||
PendingUnread string `json:"pendingUnread,omitempty"`
|
||||
// TakeIn is the take-in's backlog of done keys: keys not delivered yet and keys being taken in (novox/hq ADR 0544
|
||||
// §6). Absent where it cannot be read.
|
||||
TakeIn *takeInBacklog `json:"take-in,omitempty"`
|
||||
}
|
||||
|
||||
// pendingStatus is one pending assignment as status says it.
|
||||
@@ -266,6 +269,7 @@ func statusAsJSON(asked answers) ([]byte, error) {
|
||||
out.Failing = providerStandings(asked.conditions)
|
||||
out.Overflowing = asked.overflowing
|
||||
out.PendingUnread = asked.pendingUnread
|
||||
out.TakeIn = asked.takeIn
|
||||
for _, p := range asked.pending {
|
||||
out.Pending = append(out.Pending, pendingStatus{Node: p.Node, Module: p.Module, State: p.State,
|
||||
Build: p.Build, Repository: p.Repository, Path: p.Path, Since: p.Since, Settled: p.Settled, Note: p.Note})
|
||||
|
||||
Reference in New Issue
Block a user