44 lines
2.4 KiB
Markdown
44 lines
2.4 KiB
Markdown
---
|
|
status: open
|
|
opened: 2026-10-01
|
|
located-in: [mesh-controller internal/link/serve.go (act handles one message at a time; sourceMoved waits for every build the merge asks)]
|
|
fixed-by:
|
|
amended-design: []
|
|
---
|
|
|
|
# 184 — A merge announcement blocks the controller's receive loop
|
|
|
|
## What was observed
|
|
|
|
The controller restarted at 12:53:32Z on 2026-10-01 and heard, among its first messages, a merge
|
|
announcement for the catalogue that touched some forty modules. Until 13:17:16Z — twenty-four
|
|
minutes — it took nothing else in: build outcomes that the build machine had announced and that
|
|
the console's `builds` log showed as done sat unrecorded, so `builds` listed none of them and the
|
|
modules stayed at their old versions; the node heartbeats were dropped by the bus as a slow consumer
|
|
on `mesh.control.*.alive`, twice. When the merge handler returned, everything queued arrived at once
|
|
and was taken in within a second.
|
|
|
|
## Why this is here
|
|
|
|
The receive loop acts on one message at a time, which is the right discipline for a store the
|
|
controller must write in order. Acting on a merge means asking builds and waiting for each outcome,
|
|
minutes of work, and that wait happens inside the loop that would otherwise be hearing the outcomes
|
|
of everything else. The bus keeps the merge message alive while the work runs — the fix for the
|
|
earlier repeated-merge fault — so nothing is lost and nothing is redelivered, and nothing is heard
|
|
either. The design permits the controller to go deaf for as long as a merge takes to build, and no
|
|
status says so: the mesh reads as quiet, builds read as missing, and heartbeats read as a slow
|
|
machine.
|
|
|
|
## What a fix needs to decide
|
|
|
|
Whether a merge is work the loop dispatches and returns from — the builds asked, the outcomes taken
|
|
in by the same `Built` handler every other outcome uses, since the stream already delivers them —
|
|
or whether the loop runs more than one handler at a time with the store's ordering kept for the
|
|
kinds that need it. The first is smaller and keeps one ordering. Either way, a controller that is
|
|
busy should say so where `status` is read.
|
|
|
|
*How this would be checked:* a controller test where a merge announcement that asks a slow build
|
|
and a build outcome for another module arrive together, and the outcome is recorded before the
|
|
build finishes; live, the controller's log during the next catalogue merge shows registrations
|
|
interleaved with the merge's own.
|