Issue 334 and ADR 0264: an ask a seat gave up on can only be dropped for now
mesh/merge-gate pass: the change touches no module of the mesh's graph
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered

The review refused a grant over every seat's queue, and a copy published
again left the original stuck in it.
This commit is contained in:
jochen
2026-10-08 21:08:12 +02:00
parent b0986e2ae0
commit 6fdaeb79ba
3 changed files with 66 additions and 11 deletions
@@ -84,11 +84,13 @@ Three things the mesh had decide the shape:
4. **The controller's verb `dead-letters`** lists what DEAD_LETTERS holds, newest first, or for one
consumer; shows one whole by its id; **delivers one again** to the consumer that gave it up; or **drops
one**. Delivering again and dropping each require `why`, are recorded in the hand-act log first, and are
a person's decision, not a repair (S15 does not count them). An event is delivered again on the
consumer's again subject; an ask to a seat back on its own subject, since a seat's queue has one worker.
A message from any other stream, or a record without its message, can only be dropped, and the verb says
why. Delivering again carries the message's id, so asking twice delivers once. It is served by the
serving controller on its own connection, not by a fresh process.
a person's decision, not a repair (S15 does not count them). Only an event is delivered again, on the
consumer's again subject, and only when that consumer is on the bus and filters its again subject: a
dead letter is never let go as delivered while nothing would receive it. An ask a seat's worker gave up
on, a message from any other stream, and a record without its message can only be dropped, and the verb
says why (asks: [issue 334](../04-ISSUES/334-an-ask-a-seats-worker-gave-up-on-can-only-be-dropped/00-report.md)).
Delivering again carries the message's id, so asking twice delivers once. It is served by the serving
controller on its own connection, not by a fresh process.
5. **The `max-deliveries` condition is open while DEAD_LETTERS holds a message for that consumer**, one
condition per consumer, and clears when its messages are delivered again or dropped, never because the
server stopped saying it. Its words say who gave up on how many messages, that what they asked for was
@@ -103,9 +105,10 @@ Three things the mesh had decide the shape:
- Design 25 §1, §3 and §10 hold. The stream §3 names `mesh.events.dead` is DEAD_LETTERS; its subjects carry
the stream and consumer.
- The controller may publish on `mesh.events.dead.>`, `mesh.again.>` and every seat's `accept` subjects
(to give an ask back to its seat), and acknowledge on DEAD_LETTER_NOTICES. It already reached the stream
API.
- The controller may publish on `mesh.events.dead.>` and `mesh.again.>`, and acknowledge on
DEAD_LETTER_NOTICES. It already reached the stream API. It gets no grant over the seats' queues.
- Taking the notices never stops the controller serving: when they cannot be taken, that is said as a
condition (`not-kept`) and tried again every minute.
- Every module's consumer changes once, when its filters gain its again subject. Its position is kept.
- A message given up on is now visible and costs space until somebody acts. A consumer that gives up on
many fills DEAD_LETTERS; full, further copies are refused and said, which is the loud failure the mesh
@@ -122,7 +125,9 @@ Three things the mesh had decide the shape:
its consumer, stream, subject, deliveries and the time it was given up. It fails on the commit before the
fix and passes on it.
- mesh-controller `internal/link/deadletters_test.go`: delivered again, an event reaches the consumer that
gave it up and not another module that handled it; it is no longer kept, and a second delivery finds
gave it up and not another module that handled it; one is refused, and kept, when its consumer is gone or
does not filter its again subject, and when it is not an event; the list's total is the held count however
few it lists; notices that cannot be taken are said while the controller serves on; it is no longer kept, and a second delivery finds
nothing; a dropped one is gone; a notice whose message is gone is kept as a record that cannot be
delivered again; a notice taken twice is kept once; the controller acts on its own event delivered again.
- mesh-controller `cmd/mesh-controller/deadletters_test.go`: the verb lists, refuses an act without why or
@@ -79,8 +79,9 @@ which [design 25](../../03-DESIGN/01-to-be/25-the-bus-on-nats.md) §3 and
holds it, with its consumer, stream, subject, deliveries and when it was given up. It keeps them with no
age until a person acts.
- **The controller's `dead-letters` verb** lists them, shows one whole, delivers one again to the consumer
that gave it up and to nobody else, or drops one. Delivering and dropping say why and are recorded in the
hand-act log.
that gave it up and to nobody else, or drops one. An ask a seat's worker gave up on can only be dropped
for now ([issue 334](../334-an-ask-a-seats-worker-gave-up-on-can-only-be-dropped/00-report.md)).
Delivering and dropping say why and are recorded in the hand-act log.
- **The `max-deliveries` condition** is open per consumer while DEAD_LETTERS holds its messages, and says
where to act: "deliver them again or drop them, from the mesh MCP server."
- The comments that said a failing event dead-letters (the controller's streams and derived consumers, the
@@ -0,0 +1,49 @@
---
status: open
opened: 2026-10-08
located-in: [mesh-controller internal/link (deadletters.go, AgainTo), mesh-controller internal/broker (nats.go, the controller's grants)]
fixed-by:
amended-design:
---
# 334. An ask a seat's worker gave up on can only be dropped
## Symptom
[ADR 0264](../../02-DECISIONS/0264-a-message-a-consumer-gives-up-on-is-kept-until-a-person-delivers-it-again-or-drops-it.md)
keeps every message a durable consumer gives up on in DEAD_LETTERS. The controller's `dead-letters` verb
delivers an **event** again to the one consumer that gave it up. An **ask** to a seat that its worker gave up
on (a seat's work queue, `SEAT_<seat>`, consumer `SEAT_<seat>_worker`, five deliveries) is kept and said
the same way, but the verb refuses to deliver it again: it can only be dropped, and its sender asked to say
it again.
Found in the review of the fix for issue 330 (2026-10-08). The first version published the ask again on its
own subject. The review refused it for two reasons:
- **The grant.** It needed the controller to publish on every seat's `accept` subjects
(`mesh.seat.*.accept.>`), wider than the seats it asks today, and wider than least authority allows.
- **The original stays.** On a work-queue stream, a message its consumer gave up on is never acknowledged.
It stays in the queue until the stream's age drops it, so publishing a copy leaves two.
## Why it is an issue
An ask given up on is said and kept, but its answer is "drop it and have the sender ask again". The sender
may be a person or a module that does not know its ask was given up. The mesh holds the message and cannot
act on it.
## How it is checked
When fixed: a test in mesh-controller where a seat's worker gives up on an ask, the verb delivers it again to
that seat's worker, the original is gone from the work queue, and the controller's grant names only the
seats the mesh declares.
## Core or not
**Core.** It is located in the controller, so it resolves with a replay in mesh-lab's replays register or a
`replay-none:` saying why.
## Suggested fix
Grant the controller each declared seat's `accept` subjects from the catalogue (as its build seats are
granted now), deliver the ask again on its own subject, and delete the original from the work queue by its
`Mesh-Dead-Sequence` once the copy is published.