Take new builds in from a durable queue on the bus, register on the fingerprint, and issue a node's bus accounts when its assignment stands (issue 1945, ADR 0543, ADR 0544)
#465
Merged
jschoubbenmerged 9 commits from fix/1945-take-in-consumer into main2026-10-25 18:06:47 +00:00
registeredAt compared commits, so a catalogue merge that moved only the commit
(2df05030 at 18:02, a85d9037 before it) took every catalogue module in again
and composed it: about 200 take-ins, some 25 minutes, on every mesh-catalog
merge, which held nextcloud for an hour. A module whose trunk fingerprint is
the one it is registered at now has the new commit recorded (built_from and
source_head, as a registration moves them) and composes nothing unless its
composition or grant is missing. Readers of the registered commit: the
fingerprints read the commit's build or, with none, the indexer's value at it,
which names the same fingerprint; status, modules, the board and the facts
read it as the source's commit, which it is; the upgrade path counts it as
built from that commit, which it is.
The stall watch counted only settles the bus accepted, so each refused
acknowledgement at a first start left work held for good: it now counts the
loop's every attempt. The acknowledgement and refusals at the stop are sent
past the stop, and the connection closes only after the readers and the loop
have ended.
Research 072: the take-ins on a bounded pull consumer.
A refused key stays unacknowledged during its delay, so one waiting key on a
quiet mesh raised take-in-stalled, cleared at the next ack and raised again.
The watch now counts the keys the loop settled (acknowledged or refused, as
the bus took them) and calls it a stall only with work to do, keys not
delivered yet or a batch handed and not settled, and nothing settled for 5
minutes: a hung loop stays loud. Its first look always says what it found, so
a stall raised before a restart is cleared once this controller moves. A
consumer not made raises take-in-not-consumed under its own id per bucket,
cleared when that consumer is made. Acks are synchronous, so only those the
bus took count; on stop the iterator is drained and what it held, and a batch
cut by the stop, are refused with no delay.
Research 072: the take-ins on a bounded pull consumer.
A refused key stays unacknowledged until its delay ends, so with 8 a consumer
a handful refused for long (a manifest word this controller does not read
yet, a node that keeps failing) stopped that consumer for good, the build that
reads the word among what it held back. Each consumer now leaves up to 1000
unacknowledged; the store's bound is the one loop's batch of 16. A refusal
waits 30 s, doubling each time up to 10 minutes.
The stall watch judged movement by deliveries, which rise with every
redelivery: it now counts the keys this process acknowledged. A consumer that
cannot be made raises take-in-stalled instead of a log line. On stop the
gathered batch and a key in hand are refused with no delay, so the next
controller has them at once. A trunk value with no fingerprint of the new form
is said once, as every wait for a write.
Research 072: the take-ins on a bounded pull consumer.
The flag that made the first watch the start's look was cleared after every
watch, even one that failed to open or ended before its Caught mark: the next
watch then handed every standing assignment as fresh, about 300 settles at once
with a catalogue read each, or dropped the keys the cut watch had handed. The
flag is now cleared by the start's look itself, and a watch cut before it opens
the next with nothing heard. Two rows of the existing table test cover it.
Research 072: the grants piece (memberships and credentials heard from placement).
Two consumers of 16 each allowed 32 take-ins on the store at once, twice
issue 1945's bound. A batch takes up to 16. Research 072: the take-ins on a
bounded pull consumer.
The take-in watched the builder's and the indexer's buckets, started a follower
per module that retried on a 30 s timer, and at start took every catalogued
module in alphabetically: on 2026-10-25 a restart took in about 2 modules a
minute, about 190 followers queued behind 16 slots, and every take-in composed
two or three times even with nothing new, so a new build waited 30 minutes and
more for its credential.
Now two durable consumers named mesh-controller-take-in read the indexer's
modules bucket (every module's last trunk value when the consumer is made, then
each write) and the builder's builds bucket (each write), at most 16 keys
unacknowledged each. The bus keeps what is not taken in: the start's look is the
consumers' own catch-up, and a restart costs only what was left unacknowledged.
A batch reads the catalogue once, takes each module in once, and asks the
composer twice: once for its modules' nodes, once for the bus's node with the
grants. A module registered at its trunk whose compositions and grant exist, or
that this process took in at the same trunk, composes nothing; a key delivered
again is taken in whole. A wait for the builder's or indexer's next write is
acknowledged, a wait on a read or a failure refused with a 30 s delay. The
backlog is in status, and a backlog standing still for 5 minutes raises
take-in-stalled. The controller's account is granted the two consumers' acks.
The follower tests go with the followers: twelve tests become three.
Research 072: the take-ins on a bounded pull consumer (milestone 4, done keys
alone) and the start's look composing only what changed (milestone 2).
The assign act and the assignments' watch both checked whether an account was
minted and then minted it, and minting overwrites the hash: interleaved, the
bus kept one caller's hash and the node was sealed the other's password, the
crash loop this branch fixes. Both now mint through one statement that inserts
the row only when none stands; the caller whose insert wins seals, the other
seals nothing. issueMissingProcessAccounts mints the same way.
A watch opened again after one ended handed the standing assignments as the
start's look, so one that came to stand in the gap got no bus composition or
grant when its account already stood: only the process's first watch is the
start's look now. The start's look reads the catalogue once and ends with one
composition of the bus's node and the grants; a controller not holding the
lease is told so and tries again rather than dropping the key.
Tests folded to stay within the budget: the assignment tests are three table
tests, the process account and mayIssue tests folded into one of them, and
the concurrent mint is a case of the inventory's rotation test.
Research 072: the grants piece (memberships and credentials heard from placement).
ADR 0539 clause 7 issues a module's credential and grant only when its build is done.
A module built long before and assigned to one more node never has a build done again:
on 2026-10-25 at 17:19 mesh-build-node and mesh-build-go were assigned to ace and composed
for it, the bus's user list never named ace's accounts, and both runtimes restarted every
5 s (one with no credential, one refused by the bus). The assign act composes the assigned
node alone, never the node holding the bus, and writes no grant.
So a write of the assignments bucket is the event: when a key comes to stand (new, or
ended or pending before), the lease holder issues the module's accounts on that node that
are not minted (never re-issued), composes the node holding the bus and writes the grants,
through the composer the done keys' take-ins use. A rewrite of a standing assignment does
nothing; the start's look settles every standing one, one after another. A step of
research 072's grants (memberships and credentials heard from placement).
jschoubben