Nothing the control queue carries is lost while the store restarts (issue 083) #43

2026-09-22T12:42:49Z

jschoubben commented

2026-09-22 12:42:49 +00:00

Held, not slept on. A report, build result, upgrade announcement or catch-up request the store cannot take right now is held unacknowledged under its subject and retried on a ticker, while the loop keeps answering — enrolments above all. A newer message for the same subject sets the held one aside (an old report can never overwrite a newer one). Held at most two minutes, then let go with a LOST line; capped below the prefetch (64, headroom 8) so enrolments always get through; on shutdown nothing is settled, so the broker keeps it. Upgrades are held only when reading the store failed, not when a push did.
Enrolment claims, then spends. Migration 0028 adds claimed_by/claimed_until. The presenting key claims the token for a two-minute lease; every write overwrites; the token is spent once the node is complete; the broker password is replaced only after the spend (a failure there leaves the node on the token's secret). A store outage or a token held by another presenter is answered try_again, nothing spent.
Finishing after a spend takes proof. An answer lost after the spend may be finished by the same presenter only with a valid signature over the request by its identity key, inside the lease, and never on a broker redelivery. Without it a spent token is spent to everyone.

Tests: held/superseded/let-go/ceiling/shutdown for every handler; claim, lease, spend and proven re-entry against a real store; the replay refused with no proof, a forged proof, a proof for another request, and a redelivery; the proof's known answer. Full suite and race detector green. Proof: mesh-lab store-window bed green at 4f3b4e6 — a report held through the store's absence while a second machine is answered "try again", then both complete, with the mesh holding exactly the keys the machine generated. Reviewed independently three times; the second found a replay path in the first rework, now closed. Companion MRs on multiple-fixes: mesh-host, mesh-lab, hq.

- **Held, not slept on.** A report, build result, upgrade announcement or catch-up request the store cannot take right now is held unacknowledged under its subject and retried on a ticker, while the loop keeps answering — enrolments above all. A newer message for the same subject sets the held one aside (an old report can never overwrite a newer one). Held at most two minutes, then let go with a LOST line; capped below the prefetch (64, headroom 8) so enrolments always get through; on shutdown nothing is settled, so the broker keeps it. Upgrades are held only when reading the store failed, not when a push did. - **Enrolment claims, then spends.** Migration 0028 adds `claimed_by`/`claimed_until`. The presenting key claims the token for a two-minute lease; every write overwrites; the token is spent once the node is complete; the broker password is replaced only after the spend (a failure there leaves the node on the token's secret). A store outage or a token held by another presenter is answered `try_again`, nothing spent. - **Finishing after a spend takes proof.** An answer lost after the spend may be finished by the same presenter only with a valid signature over the request by its identity key, inside the lease, and never on a broker redelivery. Without it a spent token is spent to everyone. Tests: held/superseded/let-go/ceiling/shutdown for every handler; claim, lease, spend and proven re-entry against a real store; the replay refused with no proof, a forged proof, a proof for another request, and a redelivery; the proof's known answer. Full suite and race detector green. Proof: mesh-lab store-window bed green at 4f3b4e6 — a report held through the store's absence while a second machine is answered "try again", then both complete, with the mesh holding exactly the keys the machine generated. Reviewed independently three times; the second found a replay path in the first rework, now closed. Companion MRs on `multiple-fixes`: mesh-host, mesh-lab, hq.

jschoubben added 5 commits 2026-09-22 12:42:49 +00:00

Nothing the control queue carries is lost while the store restarts: an enrolment claims its token and spends it last, and is asked to try again; build results, upgrades and catch-ups are handed back, bounded (novox/hq issue 083) 1a41b88ed3

Review of 083: a message the store cannot take is held and retried on a ticker, not slept on, so enrolments are answered meanwhile; a newer one per subject supersedes; the password is replaced after the spend; the same presenter may finish after a lost answer; upgrades retry only on the store a3b7e830c8

Review of 083: finishing an enrolment whose token was spent takes proof of the key's private half, a live lease and a first delivery — a public key alone cannot replay a spent token; shutdown leaves held messages for the broker; identical builds supersede; what is held leaves room in the prefetch 4567fa666c

The enrolment proof's message has a known answer, repeated in the host's test 32b8af6a9b

Review of 083: an upgrade handled during shutdown is left for the broker; an enrolment cut short by shutdown is told to try again; a refused redelivery says what it probably is 4f3b4e6014

jschoubben merged commit 0a39b7df82 into main

2026-09-22 12:42:58 +00:00

jschoubben deleted branch multiple-fixes

2026-09-22 12:42:58 +00:00

Sign in to join this conversation.