jschoubben
4e13280604
The AMQP transport is gone from the control plane and the hosts (mesh-controller #112, mesh-host #39). On the way: no build had ever recorded its bases, so every bases-first order walked an empty graph; the builder now reports what it was handed and the graph is read from builds (mesh-controller #113/#114). Issue 131 is resolved by the forge module's merge event, the control plane following it, and those edges.
108 lines
6.7 KiB
Markdown
108 lines
6.7 KiB
Markdown
---
|
|
status: resolved
|
|
opened: 2026-09-27
|
|
located-in: [mesh-catalog modules/gitea, mesh-controller cmd/mesh-controller, mesh-controller internal/builder]
|
|
fixed-by: mesh-catalog #124 — the forge module watches for merged pull requests and emits `pull.merged` with the merge commit and the clone address; mesh-controller #110/#111 — the control plane follows that event on the bus, marks every module built from that repository and branch as moved, and builds them bases first, stopping when a base fails; mesh-controller #113/#114 — a build records the bases it was handed and the graph is read from builds, without which "bases first" had no edges to order by.
|
|
amended-design: 03-DESIGN/01-to-be/28-building-the-bus.md
|
|
---
|
|
|
|
# 131 — Nothing tells the mesh a source moved, and it reports itself current anyway
|
|
|
|
## What was observed
|
|
|
|
Six changes were merged to the trunk of six repositories in one sitting. The build machine built
|
|
nothing. Its last build, minutes before the first merge, was still the one it reported; no build was
|
|
requested, refused or failed, because none was ever asked for.
|
|
|
|
Asked afterwards what was wrong, the mesh said:
|
|
|
|
> 4 machine(s), all doing what they were told, all heard from, running what the mesh would send them,
|
|
> and every module current with its source
|
|
|
|
Every one of the six had moved. The last clause was false for all of them, and it is the clause a
|
|
person reads to decide whether there is anything to do.
|
|
|
|
## Why it matters beyond this instance
|
|
|
|
**The mesh learns a source moved by being told, and there is no longer anything to tell it.** The
|
|
command exists — a person names the module and the commit — and so does the question the overview
|
|
answers. What is missing is whatever used to connect the two. One repository still carries a forge
|
|
webhook aimed at a port; the rest carry none, and the port belongs to a different service than the
|
|
one the arrangement implies. So the state is not "the trigger is broken" but "there is no trigger,
|
|
and nothing says so".
|
|
|
|
**A wrong answer is worse here than no answer.** "Every module current with its source" is
|
|
indistinguishable, to a reader, from a mesh that has genuinely caught up. The overview is built to be
|
|
the thing you check instead of checking by hand, so a confident false negative removes the habit that
|
|
would otherwise have caught it. Nothing in the mesh is at fault for being out of date — it is at
|
|
fault for saying it is not.
|
|
|
|
**It is also why "current with its source" cannot be a stored fact.** The mesh compares what it built
|
|
against what it was last told the source was, and calls that agreement. Two facts agreeing tells you
|
|
nothing when both come from the same place.
|
|
|
|
## The intended shape, which is decided and not built
|
|
|
|
The forge emits what happened to it — a pull request merged — and the build machine reacts by
|
|
building what that commit affects. That keeps the forge ignorant of the build system and the build
|
|
machine ignorant of the forge's internals, which is the same argument
|
|
[ADR 0126](../../02-DECISIONS/0126-a-module-declares-its-own-seats.md) makes for addressing an event
|
|
to its emitter: the merge is a fact about the forge, and what should be rebuilt because of it is not
|
|
the forge's business to know.
|
|
|
|
The forge's module already declares the event. The build machine declares that it consumes nothing.
|
|
|
|
## What the trigger cannot be
|
|
|
|
**Not one build per changed module.** The modules form a graph: several are built from one
|
|
repository, and some are the base another is compiled on — a runtime image, a compiler base, a
|
|
repository whose context a second module builds from. Firing a build for each changed module
|
|
independently would start work that cannot succeed yet and produce a failure per dependent, for one
|
|
cause.
|
|
|
|
Observed while catching the mesh up by hand on 2026-09-27: a compiler base had to move before
|
|
anything compiled against it could build, and when it failed, the right behaviour was for its
|
|
dependents to wait rather than each fail the same way. Fifteen modules shared the cause. A trigger
|
|
that reports it fifteen times has buried it.
|
|
|
|
So whatever reacts to the forge's event resolves what changed into an order, builds the bases first,
|
|
and holds a dependent while its base is unbuilt or failed. That is a larger thing than "rebuild what
|
|
the commit touched", and knowing it now is cheaper than discovering it from fifteen identical
|
|
failures.
|
|
|
|
## Open questions
|
|
|
|
- Is "the source moved" still a thing a person can assert by hand once the event path exists, or does
|
|
the hand-operated form become the thing that made this failure possible?
|
|
- Which commit does the build machine act on — the merge, or each commit it brought — and what does
|
|
it do when several arrive for one module at once?
|
|
- How does the overview stop being able to lie? Comparing what was built against what was recorded
|
|
will always agree. Whether the trunk has moved is a question only the forge can answer, so either
|
|
the overview asks it, or it stops claiming to know.
|
|
- Does this want to be the same mechanism as the build request on the bus
|
|
([ADR 0129](../../02-DECISIONS/0129-a-seat-carries-the-protocol-of-its-role.md)), or does it sit in
|
|
front of it?
|
|
|
|
## What was done (2026-09-28)
|
|
|
|
The shape above was built as described: the forge's module emits the merge, the control plane
|
|
consumes it, and nothing on either side knows the other's internals. The build is asked for the
|
|
merge commit, not each commit the merge brought — the trunk moved once, to one place. Several merges
|
|
for one module arriving in a row are followed in turn, each moving the recorded source to its own
|
|
commit, so the last one to arrive is the one the mesh ends up built from.
|
|
|
|
The hand-operated form stays. `module moved` is how a source is recorded without a forge — a module
|
|
built from a repository elsewhere, or a mesh whose forge module is down — and it is the same act the
|
|
event performs, so the two cannot disagree about what "moved" means.
|
|
|
|
**Bases first needed edges, and there were none.** The order this report asked for was written and
|
|
walked a graph that no build had ever recorded: a recipe reads its base from a build argument, so the
|
|
digest was never in the file the builder read edges from. A build now reports what it was handed, the
|
|
control plane records it by artifact path, and the order is read from each module's newest build.
|
|
|
|
**What still can lie.** The overview compares what was built against where it was last told the
|
|
source is; the forge's event is now what moves that mark, so it is right for as long as the forge
|
|
module was listening. A merge made while that module was down is a merge the mesh does not know of
|
|
until the module next polls — it announces what merged since it last looked, so the gap closes when
|
|
it comes back, and not before. The overview does not say so.
|