A report that arrives while the store restarts is kept, not lost (issue 082) #42

2026-09-22T11:53:04Z

jschoubben commented

2026-09-22 11:53:04 +00:00

The control consumer acknowledged a node's apply report even when recording it failed. A push that adopts the foundation store restarts that store, and a report arriving in the window was lost; the node reports an apply once, so the mesh never heard from it again.
inventory.Unreachable says whether the store could not be asked right now — a network error, a connection that ended mid-conversation, a timeout, anything the driver marks safe to retry, or the server's 08 / 57P01–57P03 — and checks the server's own answer first, so a wrong password or a cancelled statement is an answer, not an outage. Enrolment.Heard marks such failures ErrTryAgain; handleReport pauses and hands the report back to the broker. One report holds the queue at most two minutes, then is let go with a line saying it was lost. Shutdown does not wait out the pause.

Tests: classification cases including a real refused connection and a real wrong password against the test store; the consumer hands back, lets go, forgets, and does not hold up shutdown. Full suite green. Reviewed independently (store stopped, killed and restarted under Heard); both gaps it found are closed here, the sibling losses in enrolment, builds and upgrades are hq issue 083. Proof: mesh-lab two-node bed green against 3fd0c37 — the anchor now catches up through the adoption push it stalled on twice in three runs. Companion MRs on multiple-fixes: mesh-catalog, mesh-lab, hq.

- The control consumer acknowledged a node's apply report even when recording it failed. A push that adopts the foundation store restarts that store, and a report arriving in the window was lost; the node reports an apply once, so the mesh never heard from it again. - `inventory.Unreachable` says whether the store could not be asked right now — a network error, a connection that ended mid-conversation, a timeout, anything the driver marks safe to retry, or the server's 08 / 57P01–57P03 — and checks the server's own answer first, so a wrong password or a cancelled statement is an answer, not an outage. `Enrolment.Heard` marks such failures `ErrTryAgain`; `handleReport` pauses and hands the report back to the broker. One report holds the queue at most two minutes, then is let go with a line saying it was lost. Shutdown does not wait out the pause. Tests: classification cases including a real refused connection and a real wrong password against the test store; the consumer hands back, lets go, forgets, and does not hold up shutdown. Full suite green. Reviewed independently (store stopped, killed and restarted under `Heard`); both gaps it found are closed here, the sibling losses in enrolment, builds and upgrades are hq issue 083. Proof: mesh-lab two-node bed green against 3fd0c37 — the anchor now catches up through the adoption push it stalled on twice in three runs. Companion MRs on `multiple-fixes`: mesh-catalog, mesh-lab, hq.

jschoubben added 2 commits 2026-09-22 11:53:04 +00:00

A report the store cannot take yet is handed back to the broker, not acknowledged and lost (novox/hq issue 082) 047830a6b5

Review of 082: a store killed mid-conversation is an outage and a wrong password is not; one report holds the queue at most two minutes, then is let go loudly; shutdown does not wait out the pause 3fd0c37d22

jschoubben merged commit b9cdb96a90 into main

2026-09-22 11:53:13 +00:00

jschoubben deleted branch multiple-fixes

2026-09-22 11:53:13 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: novox/mesh-controller#42