Merge pull request 'Issue 271: a new consumer replayed the morning to the operator' (#132) from issues/271-a-new-consumer-replayed-the-morning-to-the-operator into main
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
mesh/delivery held for a person: merged without a passing check: only a person decides that it goes on
This commit was merged in pull request #132.
This commit is contained in:
@@ -0,0 +1,79 @@
|
||||
---
|
||||
status: located
|
||||
opened: 2026-10-06
|
||||
located-in: [mesh-catalog modules/messenger]
|
||||
fixed-by: mesh-catalog PR #88
|
||||
amended-design:
|
||||
---
|
||||
|
||||
# 271. A new consumer replayed the morning to the operator
|
||||
|
||||
## Symptom
|
||||
|
||||
The holder of the `operator-channel` seat
|
||||
([to-be 45](../../03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md) §5,
|
||||
[ADR 0227](../../02-DECISIONS/0227-the-core-holds-nine-rules-each-checked-and-is-built-to-them-in-six-phases.md)) went live the
|
||||
same morning it merged. Its durable consumer on the events stream did not exist at first; the
|
||||
controller made it once the bus's objects were asserted on every send ([issue 208](../208-a-seats-worker-is-made-only-when-the-controller-starts/00-report.md)). The moment it was
|
||||
made, at 09:48 UTC, the stream handed it every condition event it held — 96 of them, raised and cleared
|
||||
between 08:32 and 09:49: holders of the machines' seats silent for four minutes, the machines' tool
|
||||
runtimes and cores behind, a probe failing, a merge not acted on, the bus refusing the controller, and
|
||||
the holder's own consumer missing. Every one had been raised and cleared hours before; none was open.
|
||||
|
||||
The holder read each as if it had just happened. Within one second it sent twenty desktop
|
||||
notifications, the per-channel cap, and folded the rest into a message waiting to go out. Its own
|
||||
record of sends showed every one at the same second, `sent` or `folded`, each a raising or a clearing
|
||||
from the morning. The operator was flooded, and the holder was unassigned.
|
||||
|
||||
## Cause
|
||||
|
||||
The holder took **arrival as news**. It had two defences against noise — deduplication by the
|
||||
condition's key, and a cap of twenty an hour — and neither is about time:
|
||||
|
||||
- **A stream replays.** The events stream keeps what it is given so that a consumer that was away
|
||||
catches up, which is what the design asks of it. A consumer that is new, made again, or whose holder
|
||||
was away receives everything held, at once. Each event carries `at`, when it happened; the holder
|
||||
never read it.
|
||||
- **State was never asked for.** The holder believed the events it was handed and nothing else.
|
||||
The controller answers what is open now (`conditions`), and at 09:48 the answer was: nothing.
|
||||
- **A clearing of something never said was said**, when the raising had been folded by the cap.
|
||||
- **The cap was the only brake**, so the first twenty in any burst always went out, one each, and
|
||||
the cap itself announced the rest.
|
||||
|
||||
The same class had happened to the controller's own consumer ([issue 248](../248-the-controllers-event-consumer-replayed-a-week-and-held-every-merge-behind-it/00-report.md): a week of merges replayed
|
||||
and acted on). There the consumer was made to start from the stream's end. Here the reader must be
|
||||
safe on its own, since a replay also comes from a redelivery or a consumer made again.
|
||||
|
||||
## Fix
|
||||
|
||||
In the holder, mesh-catalog PR #88 (open, not merged):
|
||||
|
||||
- **History is never news.** On start, every ten minutes, and at once when an old event arrives,
|
||||
the holder reads the open conditions from the controller and takes them as state. An event older
|
||||
than that reading is not acted on. An event more than ten minutes old by its own `at` is recorded,
|
||||
never said. What the holder held open that the controller no longer does ended meanwhile, and is
|
||||
ended where an edit notifies nobody. An urgent condition open and never said is said, all of them in
|
||||
one message.
|
||||
- **A clearing of something never said says nothing**, and a message waiting to go out is dropped
|
||||
when its condition clears first.
|
||||
- **Bursts are one message.** A channel holds what it is to say for thirty seconds and sends it
|
||||
together, as one digest when it is several. The desktop takes warnings at most once every fifteen
|
||||
minutes. The cap stays as a last line, and reaching it is said once.
|
||||
- **A grace minute after start** in which nothing is sent but that one urgent message.
|
||||
|
||||
The controller could also make a consumer of condition events start from the stream's end, as it does
|
||||
for the consumer of issue 248. That is a follow-up, not the fix: the holder must survive any replay.
|
||||
|
||||
## How it is checked
|
||||
|
||||
The holder's tests replay that morning's 96 event bodies, exactly as the controller recorded them.
|
||||
With the open set read first: no message. With the controller unreachable: no message. Replayed twice,
|
||||
and again an hour later as from a consumer made again: no message. The same morning with one urgent
|
||||
condition left open: exactly one message on each channel urgent goes to. Live, after the holder is
|
||||
assigned again: its record of sends holds nothing from before its start.
|
||||
|
||||
## Design
|
||||
|
||||
To-be 45 §5 says "at most twenty messages an hour; the excess is folded into one message naming them
|
||||
all" and says nothing of a holder that starts late or of bursts. The fix goes further than that line,
|
||||
and §5 needs amending to say what the holder now does (playbook 02) before this resolves.
|
||||
Reference in New Issue
Block a user