Keep an event alive while its bundle handles it, and hold one at a time (hq issue 387)
mesh/merge-gate pass: builds mesh-tools, node-tools → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request

The consumer's ack wait is 30 seconds and the runtime gives a handler two minutes,
so an event handled longer was offered again while it was handled; events waiting
in the client's buffer ran out their ack wait too, so a burst reached max-deliver
with nothing failed. Say in-progress every 10 seconds, as a worker's work does,
and pull one event at a time.
This commit is contained in:
jochen
2026-10-10 17:18:35 +02:00
parent c761c22c2a
commit aef908b57b
2 changed files with 89 additions and 2 deletions
+27 -2
View File
@@ -848,6 +848,12 @@ func ConsumerOf(node, module string) string {
// (novox/hq issue 330).
var NakDelay = 5 * time.Second
// EventHeartbeat is how often an event a bundle is still handling is said to be in progress, well inside the
// consumer's ack wait. The client holds one event at a time (PullMaxMessages(1)): events are handed over in
// order, one at a time, so an event waiting in the client's buffer would have its ack wait run out before it is
// handed over, and a burst would end in max-deliver with nothing failed (novox/hq issue 387).
var EventHeartbeat = 10 * time.Second
// ConsumeAs reads a module's durable consumer and hands each event to deliver (novox/hq ADR 0198):
// acknowledged when deliver returns nil, negatively acknowledged after NakDelay when it returns an
// error, terminated when it is not an event at all. The consumer is the controller's to create; this
@@ -873,12 +879,31 @@ func (c *Conn) ConsumeAs(module string, deliver func(Envelope) error) (func(), e
_ = msg.Term()
return
}
if err := deliver(env); err != nil {
// Kept alive while the bundle works on it, as a worker's work is (seat.go): the consumer's ack wait
// (30 seconds) is for a runtime that died, not a handler that is slow — the runtime gives a handler
// launch.EventTimeout. Without it an event handled longer than the ack wait was offered again while
// it was handled, and counted toward max-deliver (novox/hq issue 387).
done := make(chan struct{})
go func() {
tick := time.NewTicker(EventHeartbeat)
defer tick.Stop()
for {
select {
case <-done:
return
case <-tick.C:
_ = msg.InProgress()
}
}
}()
err := deliver(env)
close(done)
if err != nil {
_ = msg.NakWithDelay(NakDelay)
return
}
_ = msg.Ack()
})
}, jetstream.PullMaxMessages(1))
if err != nil {
return nil, fmt.Errorf("reading %s's consumer %s: %w", module, durable, err)
}