49 lines
2.1 KiB
Markdown
49 lines
2.1 KiB
Markdown
---
|
|
status: resolved
|
|
opened: 2026-10-04
|
|
located-in:
|
|
- mesh-controller
|
|
fixed-by:
|
|
- mesh-controller#249
|
|
amended-design:
|
|
---
|
|
|
|
# 219 — An older build that finishes later replaces a newer one
|
|
|
|
## What was observed
|
|
|
|
2026-10-03. Two merges to the runtime module came minutes apart. Each plan asked for every module
|
|
built on the runtime image to be rebuilt. One module's two builds were of the same source and
|
|
differed only in the runtime image they stood on:
|
|
|
|
| Build | Requested | Finished | Stood on |
|
|
|---|---|---|---|
|
|
| asked by the first plan | 21:33 | 22:04 | the runtime image before the fix |
|
|
| asked by the second plan | 21:49 | 21:56 | the runtime image with the fix |
|
|
|
|
The older request finished last, and its image became the module's current artifact. The next push
|
|
deployed it, and the module's runtime crash-looped on a fault the newer image had already fixed
|
|
([issue 218](../218-a-mesh-seat-is-answered-by-a-module-that-does-not-hold-it/00-report.md)).
|
|
|
|
## Why it matters beyond this instance
|
|
|
|
Which build is current should follow what it was built from, not which build machine was slowest.
|
|
Whenever two plans overlap, which happens on any busy evening, a fix can be silently reverted by a
|
|
build that started before it existed. Every passing check still passes.
|
|
|
|
## Where to look
|
|
|
|
How a finished build is recorded and how a module's current artifact is chosen. **How it is checked:**
|
|
a test in which an older request completes after a newer one for the same artifact, and the newer
|
|
stays current.
|
|
|
|
## Resolved
|
|
|
|
A build is ordered by when it was requested, read from the id the controller gives it, and a
|
|
module's registered manifest is replaced only by a build requested at or after the one it came from.
|
|
An older request finishing later is recorded and changes nothing; a plan settles only from builds it
|
|
asked for itself. **How it is checked:** store-backed tests replay the incident — the newer request
|
|
stays what the module is — and fail without the fix. Live since 2026-10-04: the rebuilds of every
|
|
runtime-image module and the three waves of module code moves since then each registered the build
|
|
they asked for.
|