Design 47 H: section 10, finding 153
This commit is contained in:
@@ -669,3 +669,4 @@ Added by the lead at 21:30 CET, from the run of 19:50 to 21:30; a row met again
|
||||
| Row 149's two fixes: the builder's (each reservation read once, the fallback over the unbuilt only, in-progress before the slow pass) was approved and merged at 16:47 as a pipeline of eight movers, its pack for the build node following; the SDK's (the event hub pulls at most twenty messages per pull and sends in-progress every third of the ack wait for a handler still running) is built and in review, and reaches every module at its next build against the new SDK release (row 149); the builder and mesh-sdk. | H1 |
|
||||
| The SDK's event hub fix (row 150) merged at 16:51, and its release re-pins 126 modules into one pipeline, the tool runner among them. Every merge of the afternoon moved the tool runner to a newer pipeline before the older one could pass (rows 142 and 147), so the lead holds further merges from 16:52 until the queue runs through to the tool runner's pass; the one approved change still unmerged (row 146) waits on the operator's word. The builder's read-once pack goes onto the build node meanwhile, so the events backlog drains faster (rows 142, 146 and 150); the pipeline runner (`mesh-delivery-pipeline`), the builder and the runbook. | H1 |
|
||||
| The builder was built again against the SDK's hub fix (row 151) and placed on the build node at 16:58 (its read-once pack at 16:52 before it, with no change in the drain rate): the redelivery stopped (two redelivered, against 175 before; nineteen unacknowledged, against the batch of twenty), and the drain rose to about 1.2 messages a second, so the backlog of 2,167 at 17:05 falls under the bound in about half an hour. What remains is about one second of work per event. Each restart of the builder cut one publish in flight, which the loop asking a publish again asked again; the tool runner's release 0.1.40 waits in main.117 on thirteen publishes (rows 149, 150 and 151); the builder. Owed: the cost per event of the builder's published handling, measured and cut. | H1 (the fixes live); H2 (the cost per event) |
|
||||
| Once the builder's backlog had drained (17:19), the tool runner's release could not publish for another reason: the tool runner imports about twenty packages its own pipeline releases, each package's publish re-pins it (H-A-204), each re-pin keeps a revision of the same source, each such revision's pipeline takes it out of the running one (H-A-223), and each newer build withdraws the older before it publishes; its release ran from 0.1.41 to 0.1.48 in nine minutes with none published: a livelock for any module that imports many packages of one pipeline (rows 142, 147 and 152); the pipeline runner (`mesh-delivery-pipeline`) and the indexer (`mesh-delivery-indexer`). Settled by H-A-283 (a re-pin revision of the same source keeps a mover in its pipeline and asks its build again there; only a new source moves movers), in build in the runner. Owed: pins taken from the reservation and confirmed at publish, so a release re-pins once. | H1 (the runner); H2 (the pins) |
|
||||
|
||||
Reference in New Issue
Block a user