75 lines
4.8 KiB
Markdown
75 lines
4.8 KiB
Markdown
---
|
|
status: located
|
|
opened: 2026-10-01
|
|
located-in: [mesh-controller internal/broker/derived.go (the holder worker consumer let many asks stand in flight), mesh-controller internal/link/builds_nats.go (a running build said nothing to the bus), mesh-controller cmd/mesh-controller/upgrades.go (a merge rebuilt its own modules and not what stood on them)]
|
|
fixed-by:
|
|
amended-design: []
|
|
---
|
|
|
|
# 186 — A release across repositories is an order in a person's head, and a build is a line in a queue nobody keeps
|
|
|
|
## What was observed
|
|
|
|
On 2026-10-01 one decision (ADR 0161) was built as three pull requests in three repositories that
|
|
must land in order: the controller first, so the seat exists and a report's profile is kept; the host
|
|
second, so every machine reports the capability; the catalogue last, so a claim names a seat that
|
|
exists and a holder is not refused on every machine. That order is written in a work-order file and
|
|
in the pull requests' descriptions. The mesh holds none of it. A merge is handled as a merge: build
|
|
the modules whose recorded source is that repository, record what came back. Nothing says what the
|
|
mesh should end up as, and nothing checks whether it got there.
|
|
|
|
The same day showed what a build is. The build machine takes asks from an in-memory queue; when it
|
|
was itself rebuilt in the middle of a wave of forty-three asks, the machine rolled, the new one
|
|
started with an empty queue, and forty asks were gone without a word — the wave read "one of
|
|
forty-three" for two hours. A merge announcement that asks forty builds waits for them inside the
|
|
controller's one receive loop ([issue 184](../184-a-merge-announcement-blocks-the-controllers-receive-loop/00-report.md)).
|
|
"Did everything I asked for succeed" was answered by counting lines in two containers' logs.
|
|
|
|
## Why this is here
|
|
|
|
Every single step is sound: a build is reproducible, a push composes a machine from the store as it
|
|
is, memberships are last-per-subject, so the mesh converges to the right state once every build is
|
|
recorded and every machine pushed. What is missing is the whole: the mesh has no durable account of
|
|
work it has asked for and no account of the state a release is aiming at, so the two faults that
|
|
matter most to an operator — work silently lost, and a dependency between repositories merged in
|
|
the wrong order — are detected by nobody. The rule that a manifest word ships one release ahead of
|
|
its use is a discipline a person keeps, and a person kept it by hand eleven times this week.
|
|
|
|
## What a decision would settle
|
|
|
|
- Whether a build ask is a durable message on the bus (a work queue the build machine takes from and
|
|
acknowledges, as the build outcome already is an event), so a restarted builder resumes rather
|
|
than forgets, and `builds` can list what is asked and not yet built.
|
|
- Whether a release across repositories is a thing the mesh records — a set of commits that belong
|
|
together with the order they land in — so that a merge out of order is refused or held rather than
|
|
built, and `status` can say what a release still waits for.
|
|
- What the smallest honest surface is in the meantime: at least `builds` listing the asked and the
|
|
running beside the built, so a person polling logs becomes a person reading one table.
|
|
|
|
## Located, 2026-10-01 evening
|
|
|
|
Two of the three faults turned out to be mechanism, and are fixed; the third stands as the decision
|
|
this report asks for.
|
|
|
|
- **The queue was durable; the delivery was not.** A build ask is a message in the seat's work-queue
|
|
stream and survives a builder restart. What lost forty-three asks twice was the worker consumer:
|
|
with the server's default of many deliveries in flight, every ask behind the one being built was
|
|
handed over at once, left unacknowledged for the length of the build, redelivered after the ack
|
|
wait, and dropped after the fifth time. The builder's log shows the survivors in redelivery order.
|
|
Fixed by mesh-controller PR 194: one in flight, and a running build says it is still working.
|
|
- **A merge rebuilt what it changed and not what stood on it.** The relation existed in the store
|
|
and only the explicit `build --on` read it. Fixed by mesh-controller PR 193: a merge takes every
|
|
module standing on what moved, in base order.
|
|
- **A release across repositories is still an order in a person's head.** That is the decision.
|
|
|
|
*How this would be checked:* a builder restarted between an ask and its build still builds it; a
|
|
merge of a dependent repository before its prerequisite is held and named; `builds` lists asked,
|
|
running and built.
|
|
|
|
## Decided, 2026-10-01
|
|
|
|
The third fault is answered by [ADR 0162](../../02-DECISIONS/0162-a-merge-produces-a-tiered-plan-the-mesh-keeps.md):
|
|
a release across repositories is a plan whose dependency edges cross repositories, sorted into
|
|
tiers and deployed tier by tier, read in `status`. The order a person kept is the order the tiers
|
|
give.
|