From 2f0ce4ce19578bac5b3d332ec22430865881f77a Mon Sep 17 00:00:00 2001 From: jochen Date: Sat, 3 Oct 2026 11:05:40 +0200 Subject: [PATCH] Issue 207: a re-made worker replayed every ask the stream kept, and the mesh re-registered its past --- .../00-report.md | 49 +++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 04-ISSUES/207-a-re-made-worker-replayed-every-ask-the-stream-kept/00-report.md diff --git a/04-ISSUES/207-a-re-made-worker-replayed-every-ask-the-stream-kept/00-report.md b/04-ISSUES/207-a-re-made-worker-replayed-every-ask-the-stream-kept/00-report.md new file mode 100644 index 0000000..3062e14 --- /dev/null +++ b/04-ISSUES/207-a-re-made-worker-replayed-every-ask-the-stream-kept/00-report.md @@ -0,0 +1,49 @@ +--- +status: open +opened: 2026-10-03 +located-in: + - mesh-controller +fixed-by: +amended-design: +--- + +# 207 — A re-made worker replayed every ask the stream kept, and the mesh re-registered its past + +## What was observed + +2026-10-03, recovering from [issue 206](../206-a-seats-worker-changing-type-strands-the-holder-and-the-build-that-would-fix-it/00-report.md). +The build seat's worker, a push consumer the new controller could not change to pull, was deleted by hand +and the controller restarted. On start it recreated the worker in the shape it derives — pull — and with +the delivery policy a new consumer gets when nothing says otherwise: *every message the stream holds*. +The seat's stream keeps its history. The build machine then took, in order, every build ask since +1 October: + +``` +a build request arrived for …/mesh-catalog.git (build-1790856308080864567) + [built] baserow from 6afc1160 +``` + +Each outcome was heard and taken in as any build's is. In the minute before the build machine was taken +off the control node to stop it, nine modules were re-registered from the 1 October commit — baserow, +cloudflare-dns, gitlab, grafana, icecast, influxdb, jira, keycloak, letta — the catalogue's recorded +head moved back to that commit, so status listed almost every module as "behind", and the upgrade +policy re-sent the machine running five of them, which replaced their tools containers with the old +images. Recovery: the nine rebuilt from main by hand, the worker re-made by hand as pull delivering only +new asks, the build machine assigned again. + +## Why it matters beyond this instance + +A work queue that keeps its history is a replay waiting for a consumer that starts from the beginning, +and a new consumer starts there unless told not to. Nothing in the controller's consumer derivation says +where a seat's worker starts, so any re-creation — by hand, or by the reconciliation issue 206's fix +adds — can replay the mesh's whole build history into the catalogue and onto machines. And the taking-in +of an outcome trusts the outcome's commit absolutely: an outcome for a commit older than what the +catalogue holds is registered as if it were news, and rolls out. + +## Questions HQ must answer + +- Does a seat's work queue keep acknowledged asks at all? If it is a work queue, acknowledged work + should leave it ([design 25](../../03-DESIGN/01-to-be/25-the-bus-on-nats.md) §1 calls it one); if it + keeps history for the record, its worker must be derived to start at new messages, always. +- Should a build outcome for a commit the catalogue has already moved past be recorded and **not** + registered — a build of the past is a fact, not a change — and never sent to machines? -- 2.54.0