Merge pull request 'Issue 271: a new consumer replayed the morning to the operator' (#132) from issues/271-a-new-consumer-replayed-the-morning-to-the-operator into main

2026-10-06 10:06:28 +00:00
parent 95c024b7c6 04664e6acc
commit 2ccfd1d73d
1 changed files with 79 additions and 0 deletions
@@ -0,0 +1,79 @@
 ---
 status: located
 opened: 2026-10-06
 located-in: [mesh-catalog modules/messenger]
 fixed-by: mesh-catalog PR #88
 amended-design:
 ---
 # 271. A new consumer replayed the morning to the operator
 ## Symptom
 The holder of the `operator-channel` seat
 ([to-be 45](../../03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md) §5,
 [ADR 0227](../../02-DECISIONS/0227-the-core-holds-nine-rules-each-checked-and-is-built-to-them-in-six-phases.md)) went live the
 same morning it merged. Its durable consumer on the events stream did not exist at first; the
 controller made it once the bus's objects were asserted on every send ([issue 208](../208-a-seats-worker-is-made-only-when-the-controller-starts/00-report.md)). The moment it was
 made, at 09:48 UTC, the stream handed it every condition event it held — 96 of them, raised and cleared
 between 08:32 and 09:49: holders of the machines' seats silent for four minutes, the machines' tool
 runtimes and cores behind, a probe failing, a merge not acted on, the bus refusing the controller, and
 the holder's own consumer missing. Every one had been raised and cleared hours before; none was open.
 The holder read each as if it had just happened. Within one second it sent twenty desktop
 notifications, the per-channel cap, and folded the rest into a message waiting to go out. Its own
 record of sends showed every one at the same second, `sent` or `folded`, each a raising or a clearing
 from the morning. The operator was flooded, and the holder was unassigned.
 ## Cause
 The holder took **arrival as news**. It had two defences against noise — deduplication by the
 condition's key, and a cap of twenty an hour — and neither is about time:
 - **A stream replays.** The events stream keeps what it is given so that a consumer that was away
  catches up, which is what the design asks of it. A consumer that is new, made again, or whose holder
  was away receives everything held, at once. Each event carries `at`, when it happened; the holder
  never read it.
 - **State was never asked for.** The holder believed the events it was handed and nothing else.
  The controller answers what is open now (`conditions`), and at 09:48 the answer was: nothing.
 - **A clearing of something never said was said**, when the raising had been folded by the cap.
 - **The cap was the only brake**, so the first twenty in any burst always went out, one each, and
  the cap itself announced the rest.
 The same class had happened to the controller's own consumer ([issue 248](../248-the-controllers-event-consumer-replayed-a-week-and-held-every-merge-behind-it/00-report.md): a week of merges replayed
 and acted on). There the consumer was made to start from the stream's end. Here the reader must be
 safe on its own, since a replay also comes from a redelivery or a consumer made again.
 ## Fix
 In the holder, mesh-catalog PR #88 (open, not merged):
 - **History is never news.** On start, every ten minutes, and at once when an old event arrives,
  the holder reads the open conditions from the controller and takes them as state. An event older
  than that reading is not acted on. An event more than ten minutes old by its own `at` is recorded,
  never said. What the holder held open that the controller no longer does ended meanwhile, and is
  ended where an edit notifies nobody. An urgent condition open and never said is said, all of them in
  one message.
 - **A clearing of something never said says nothing**, and a message waiting to go out is dropped
  when its condition clears first.
 - **Bursts are one message.** A channel holds what it is to say for thirty seconds and sends it
  together, as one digest when it is several. The desktop takes warnings at most once every fifteen
  minutes. The cap stays as a last line, and reaching it is said once.
 - **A grace minute after start** in which nothing is sent but that one urgent message.
 The controller could also make a consumer of condition events start from the stream's end, as it does
 for the consumer of issue 248. That is a follow-up, not the fix: the holder must survive any replay.
 ## How it is checked
 The holder's tests replay that morning's 96 event bodies, exactly as the controller recorded them.
 With the open set read first: no message. With the controller unreachable: no message. Replayed twice,
 and again an hour later as from a consumer made again: no message. The same morning with one urgent
 condition left open: exactly one message on each channel urgent goes to. Live, after the holder is
 assigned again: its record of sends holds nothing from before its start.
 ## Design
 To-be 45 §5 says "at most twenty messages an hour; the excess is folded into one message naming them
 all" and says nothing of a holder that starts late or of bursts. The fix goes further than that line,
 and §5 needs amending to say what the holder now does (playbook 02) before this resolves.