diff --git a/02-DECISIONS/0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md b/02-DECISIONS/0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md index 7c6f927..3223921 100644 --- a/02-DECISIONS/0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md +++ b/02-DECISIONS/0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md @@ -9,6 +9,8 @@ extends: 0110-a-seat-is-a-module-assignment-from-a-closed-set.md # 121. A system seat is named for its scope, and a module may define its own +> **The mechanism changed — 2026-10-02, by [ADR 0190](0190-a-seats-work-is-shared-by-its-holders-and-building-is-the-first-such-role.md).** The naming rule stands. The build role this record made mesh-scoped — *the mesh's single build machine* — is node-scoped now: `node-build-agent`, one holder per machine, every holder taking from one work queue. + ## Context [ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) made seats a closed set the diff --git a/02-DECISIONS/0162-a-merge-produces-a-tiered-plan-the-mesh-keeps.md b/02-DECISIONS/0162-a-merge-produces-a-tiered-plan-the-mesh-keeps.md index 04a58da..b5343e6 100644 --- a/02-DECISIONS/0162-a-merge-produces-a-tiered-plan-the-mesh-keeps.md +++ b/02-DECISIONS/0162-a-merge-produces-a-tiered-plan-the-mesh-keeps.md @@ -9,6 +9,8 @@ extends: 02-DECISIONS/0157-a-build-says-what-it-does-on-the-bus-as-it-happens.md # 162. A merge produces a tiered plan the mesh keeps, and a module's dependencies are one relation in the catalogue +> **Progressive insight — 2026-10-02.** The context below says a dependent is *built by whichever build machine is running — the only one there could be*. That was a fact of the day, not of the decision: since [ADR 0190](0190-a-seats-work-is-shared-by-its-holders-and-building-is-the-first-such-role.md) a tier's asks are taken by every machine holding the build seat. The plan and its tiers are unchanged. + ## Context A merge on the forge reaches the controller as an event, and the controller asks the build diff --git a/02-DECISIONS/0190-a-seats-work-is-shared-by-its-holders-and-building-is-the-first-such-role.md b/02-DECISIONS/0190-a-seats-work-is-shared-by-its-holders-and-building-is-the-first-such-role.md new file mode 100644 index 0000000..6ddfd73 --- /dev/null +++ b/02-DECISIONS/0190-a-seats-work-is-shared-by-its-holders-and-building-is-the-first-such-role.md @@ -0,0 +1,117 @@ +--- +topic: the mesh +status: accepted +date: 2026-10-02 +deciders: jochen +reconstructed: false +extends: 02-DECISIONS/0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md +--- + +# 190. A seat's work is shared by its holders, and building is the first such role + +## Context + +Work addressed to a role goes to the seat's `accept` subjects, on a per-seat work queue +([design 25](../03-DESIGN/01-to-be/25-the-bus-on-nats.md) §1, [ADR 0041](0041-events-are-a-relationship.md)). +The holder's worker on that queue is already a queue group — *"even though the seat guarantees one +holder … the day somebody allows two holders for throughput, every message is processed twice with +nothing reporting it"* — and design 25 already says what a build queue shared by several machines is: +*a seat's `accept` subjects, on a work queue with a queue group of holders*. The mechanism was drawn. +Two things stopped it being used. + +First, the build role is a **mesh-scoped** seat, `mesh-build-machine`, so there is one holder in the +whole mesh ([ADR 0121](0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md): +*the mesh's single build machine*). Second, the worker is a push consumer with **one delivery in +flight** — set so after 2026-10-01, when a push consumer handing out many at once left twenty-six of +forty-three asks undelivered ([issue 175](../04-ISSUES/175-an-announcement-behind-a-long-build-comes-back/00-report.md)) — +and one in flight on a shared consumer is one build at a time across every holder there could be. + +Measured on 2026-10-02: a change to code comments in the tool runtime rebuilt its thirty-five +dependent images, one after another, on one machine, for about half an hour, while three other +machines with a container runtime sat idle; the work the mesh wanted next waited behind it. The +operator's words: *this is our first occurrence of a mesh advantage* — and: *make sure the setup is +done generically, so if another module also requires mesh functionality it can re-use the pattern.* + +## Considered Options + +1. **A second build machine by configuration** — a concurrency setting on the one holder, or a second + holder admitted by hand. Rejected: a setting on one machine shares nothing, and a second holder + of a mesh-scoped seat contradicts what a mesh seat means. +2. **A build-specific dispatcher** — the controller choosing a machine per build and asking it by + name. Rejected: it reinvents the queue the bus already is, it makes the controller a scheduler, + and it is specific to building; the next role needing the same would build its own. +3. **A seat's work is shared by its holders, and the build role becomes node-scoped.** Chosen. It is + what the bus was drawn to do, it is one rule for every role rather than one for building, and + "the machines that are online and hold the seat" is exactly the set a queue group's members is. + +## Decision + +**1. Work asked of a seat is taken by whichever of its holders is idle.** Every holder of a seat with +`accepts` reads the seat's one work queue; a node-scoped seat held on several machines has several +holders, and an ask goes to one of them. The asker addresses the role — `mesh.seat..accept.` +— and never a machine. The outcome, the role's own event, says which machine did the work (`on`), as a +build's already does ([ADR 0157](0157-a-build-says-what-it-does-on-the-bus-as-it-happens.md)). + +**2. A holder takes one ask at a time, when it is idle, by pulling.** The worker is a pull consumer: +a holder fetches one ask, works it, acknowledges, fetches the next. The server never hands an ask +to a busy holder, so a slow machine never holds work an idle one could take — the fault issue 175 +found in push delivery is removed by the shape rather than by a limit, and the one-in-flight limit +that made the shared queue serial goes with it. A holder that dies mid-work leaves its ask to be +redelivered to another, as today. + +**3. Work that must run on one particular machine is not a work queue.** That is a node seat's verb +asked of that machine ([design 33](../03-DESIGN/01-to-be/33-the-tools-the-mesh-answers.md) §4), and +nothing here changes it. A role's work queue is for work whose result is the same whichever holder +does it: a build is, because what comes out is published by digest to the mesh's store. + +**4. This is one pattern, not one role's.** Any module that declares a node-scoped seat with +`accepts` gets decisions 1 and 2 with no further mechanism: the controller derives the queue and the +worker, the holders pull, the module's manifest says what every holding machine must have. The +build agent is the first; a module needing work done *somewhere on the mesh* — a scan, a +conversion, a fetch — declares a seat of its own the same way +([ADR 0126](0126-a-module-declares-its-own-seats.md)). + +**5. Building is the first such role.** The build role is `node-build-agent`, scope node, with the +same `build` ask and the same `started`, `built` and `log.` events as before. Its holder is the +`build-agent` module: the builder as it is — a container runtime, the artifact store and the package +registry resolved as provisions, a workspace, the bus credential — assignable to every machine that +has a container runtime. `mesh-build-machine` and the `builder` module are retired when the new +holder is assigned where the old one was. The tiered plan ([ADR 0162](0162-a-merge-produces-a-tiered-plan-the-mesh-keeps.md)) +is unchanged: a tier's asks go out together and are now worked together. + +## Consequences + +- A tier of thirty-five images is built by as many machines as hold the seat and are online. A + machine that is off builds nothing and blocks nothing. +- Every holding machine fetches base images from the store and pushes what it builds; the store is + reached as a provision, so this is what the provision was for. A machine with a slow link builds + slowly, and takes fewer asks for it, which is the point of pulling. +- A build's outcome carries which machine built it, so a build that fails on one machine and not + another is a fact the record shows, not a mystery. +- What got harder: a build's cache is per machine, so a cold machine pays the first pull of every + base it has never seen; the artifact store is now asked by several machines at once, and the + package registry likewise. Both are provisions and both are made for that. +- ADR 0121's *"the mesh's single build machine"* and ADR 0162's *"built by whichever build machine + is running — the only one there could be"* were true and are no longer; both records carry a note. + +## How it is checked + +| Rule | Checked by | +|---|---| +| Two holders of one seat each take one of two asks, and a third ask waits for the first to be idle | the controller's test over the work queue against a real bus: two machines bound to one worker, three asks | +| An ask is never delivered to a busy holder | the same test: the busy holder's ask count stays at one until it acknowledges | +| A holder that dies mid-work leaves its ask for another | the same test, one holder closed mid-ask | +| The asker names no machine | the controller's seat table: `node-build-agent` accepts `build` and the asking side publishes to the seat's accept subject, as the existing tests of `build` already require | +| Live | `builds` shows a tier's builds `on` more than one machine within one plan; `seats` shows `node-build-agent` held on every machine with a container runtime | + +## References + +- [Design 25](../03-DESIGN/01-to-be/25-the-bus-on-nats.md) §1 and §5 — the work queue and the queue + group of holders this uses as drawn +- [Design 18](../03-DESIGN/01-to-be/18-building-a-module.md) — building a module, amended for + where a build runs +- [ADR 0041](0041-events-are-a-relationship.md), [ADR 0121](0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md), + [ADR 0126](0126-a-module-declares-its-own-seats.md), [ADR 0157](0157-a-build-says-what-it-does-on-the-bus-as-it-happens.md), + [ADR 0162](0162-a-merge-produces-a-tiered-plan-the-mesh-keeps.md) +- [Issue 175](../04-ISSUES/175-an-announcement-behind-a-long-build-comes-back/00-report.md) — why the + worker had one in flight, and why pulling removes the cause rather than the symptom diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index bf75179..d4f9cbf 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -188,6 +188,7 @@ python3 00-META/checks/index.py fail if stale - **0185** — [A control plane behind its seat's row serves what it can](0185-a-control-plane-behind-its-seats-row-serves-what-it-can.md) - **0186** — [A ban list never holds a neighbour, and the mesh's own bans are its own wherever they hang](0186-a-ban-list-never-holds-a-neighbour.md) - **0187** — [A dead tracker is not the machine's failure](0187-a-dead-tracker-is-not-the-machines-failure.md) +- **0190** — [A seat's work is shared by its holders, and building is the first such role](0190-a-seats-work-is-shared-by-its-holders-and-building-is-the-first-such-role.md) ### Its tiers, from the bottom up diff --git a/03-DESIGN/01-to-be/18-building-a-module.md b/03-DESIGN/01-to-be/18-building-a-module.md index 1de14b0..cd1d272 100644 --- a/03-DESIGN/01-to-be/18-building-a-module.md +++ b/03-DESIGN/01-to-be/18-building-a-module.md @@ -4,9 +4,10 @@ status: proposed code: - mesh-controller cmd/mesh-builder - mesh-controller internal/builder - - mesh-catalog modules/builder -updated: 2026-10-01 + - mesh-catalog modules/build-agent +updated: 2026-10-02 decisions: + - 02-DECISIONS/0190-a-seats-work-is-shared-by-its-holders-and-building-is-the-first-such-role.md - 02-DECISIONS/0157-a-build-says-what-it-does-on-the-bus-as-it-happens.md - 02-DECISIONS/0150-a-modules-own-code-runs-as-supervised-processes-under-one-account.md - 02-DECISIONS/0142-the-mesh-delivers-its-own-components-as-binaries.md @@ -268,6 +269,20 @@ ships one and wrong for code the mesh built, which has no unit until the mesh wr **Tools, hooks and consumers are not further modes**, which is the test of whether three is the right number: they are loaded by a tool host, and a tool host is a process that stays up. +## Where a build runs + +**On whichever machine holding the build role is idle** ([ADR 0190](../../02-DECISIONS/0190-a-seats-work-is-shared-by-its-holders-and-building-is-the-first-such-role.md)). +The role is `node-build-agent`, a node seat; its holder is the `build-agent` module, assignable to +every machine with a container runtime. The controller asks the role, never a machine: a tier's asks go +onto the seat's one work queue together, and each holder pulls one at a time when it is idle, so a +tier of many images is built by as many machines as hold the seat and are online, and a machine that +is off builds nothing and blocks nothing. What a holding machine needs is what the builder always +needed — a container runtime, the artifact store and the package registry as provisions, a workspace, +the bus credential — said once in the module's manifest. The outcome names the machine that built it. + +This is the bus's shared-work pattern, not a build-specific one: any module declaring a node seat with +`accepts` has its work shared by its holders the same way. Building is the first use. + ## A build says what it does, as it happens *2026-10-01 — [ADR 0157](../../02-DECISIONS/0157-a-build-says-what-it-does-on-the-bus-as-it-happens.md).*