An idle build machine's empty fetch is asked again, not read as the end (hq ADR 0190)

A fetch on a context without a deadline waits the client's own while and reports the deadline
passed — the client's, not ours — and the loop read it as "stop": every idle build agent exited
clean every half minute and was restarted by its supervisor, a crash loop with nothing in the log
to say why. Only our own context ending ends the machine; an empty fetch, however it is reported,
is asked again.
This commit is contained in:
jochen
2026-10-03 11:41:23 +02:00
parent 78de54381b
commit 59166b1031
+8 -2
View File
@@ -193,9 +193,15 @@ func (m *natsMachine) Take(ctx context.Context, do func(context.Context, Build))
// machine with nothing to build, and is asked again.
fetched, err := sub.Fetch(1, nats.Context(ctx))
switch {
case errors.Is(err, context.Canceled), errors.Is(err, context.DeadlineExceeded):
case ctx.Err() != nil:
// Ours ended: the machine is being stopped.
return nil
case errors.Is(err, nats.ErrTimeout):
case errors.Is(err, context.Canceled), errors.Is(err, context.DeadlineExceeded), errors.Is(err, nats.ErrTimeout):
// **An empty queue, not the end.** A fetch on a context without a deadline waits the
// client's own while and then says the deadline passed — the client's, not ours. Read
// as "stop", every idle build machine exited clean every half minute and was started
// again by its supervisor, which looked like a crash loop with nothing in the log to
// say why (2026-10-03, the first build agents). Asked again.
continue
case err != nil:
if sub.IsValid() {