Merge pull request 'ADR 0190: a seat's work is shared by its holders, and building is the first such role; design 18 says where a build runs' (#306) from decision/0190-a-seats-work-is-shared-by-its-holders into main

2026-10-03 00:44:27 +00:00
parent 3f4782cfb4 76b563e0ce
commit d8a58dadcf
5 changed files with 139 additions and 2 deletions
@@ -9,6 +9,8 @@ extends: 0110-a-seat-is-a-module-assignment-from-a-closed-set.md

 # 121. A system seat is named for its scope, and a module may define its own

+> **The mechanism changed — 2026-10-02, by [ADR 0190](0190-a-seats-work-is-shared-by-its-holders-and-building-is-the-first-such-role.md).** The naming rule stands. The build role this record made mesh-scoped — *the mesh's single build machine* — is node-scoped now: `node-build-agent`, one holder per machine, every holder taking from one work queue.
+
 ## Context

 [ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md) made seats a closed set the
@@ -9,6 +9,8 @@ extends: 02-DECISIONS/0157-a-build-says-what-it-does-on-the-bus-as-it-happens.md

 # 162. A merge produces a tiered plan the mesh keeps, and a module's dependencies are one relation in the catalogue

+> **Progressive insight — 2026-10-02.** The context below says a dependent is *built by whichever build machine is running — the only one there could be*. That was a fact of the day, not of the decision: since [ADR 0190](0190-a-seats-work-is-shared-by-its-holders-and-building-is-the-first-such-role.md) a tier's asks are taken by every machine holding the build seat. The plan and its tiers are unchanged.
+
 ## Context

 A merge on the forge reaches the controller as an event, and the controller asks the build
@@ -0,0 +1,117 @@
+---
+topic: the mesh
+status: accepted
+date: 2026-10-02
+deciders: jochen
+reconstructed: false
+extends: 02-DECISIONS/0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md
+---
+
+# 190. A seat's work is shared by its holders, and building is the first such role
+
+## Context
+
+Work addressed to a role goes to the seat's `accept` subjects, on a per-seat work queue
+([design 25](../03-DESIGN/01-to-be/25-the-bus-on-nats.md) §1, [ADR 0041](0041-events-are-a-relationship.md)).
+The holder's worker on that queue is already a queue group — *"even though the seat guarantees one
+holder … the day somebody allows two holders for throughput, every message is processed twice with
+nothing reporting it"* — and design 25 already says what a build queue shared by several machines is:
+*a seat's `accept` subjects, on a work queue with a queue group of holders*. The mechanism was drawn.
+Two things stopped it being used.
+
+First, the build role is a **mesh-scoped** seat, `mesh-build-machine`, so there is one holder in the
+whole mesh ([ADR 0121](0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md):
+*the mesh's single build machine*). Second, the worker is a push consumer with **one delivery in
+flight** — set so after 2026-10-01, when a push consumer handing out many at once left twenty-six of
+forty-three asks undelivered ([issue 175](../04-ISSUES/175-an-announcement-behind-a-long-build-comes-back/00-report.md)) —
+and one in flight on a shared consumer is one build at a time across every holder there could be.
+
+Measured on 2026-10-02: a change to code comments in the tool runtime rebuilt its thirty-five
+dependent images, one after another, on one machine, for about half an hour, while three other
+machines with a container runtime sat idle; the work the mesh wanted next waited behind it. The
+operator's words: *this is our first occurrence of a mesh advantage* — and: *make sure the setup is
+done generically, so if another module also requires mesh functionality it can re-use the pattern.*
+
+## Considered Options
+
+1. **A second build machine by configuration** — a concurrency setting on the one holder, or a second
+   holder admitted by hand. Rejected: a setting on one machine shares nothing, and a second holder
+   of a mesh-scoped seat contradicts what a mesh seat means.
+2. **A build-specific dispatcher** — the controller choosing a machine per build and asking it by
+   name. Rejected: it reinvents the queue the bus already is, it makes the controller a scheduler,
+   and it is specific to building; the next role needing the same would build its own.
+3. **A seat's work is shared by its holders, and the build role becomes node-scoped.** Chosen. It is
+   what the bus was drawn to do, it is one rule for every role rather than one for building, and
+   "the machines that are online and hold the seat" is exactly the set a queue group's members is.
+
+## Decision
+
+**1. Work asked of a seat is taken by whichever of its holders is idle.** Every holder of a seat with
+`accepts` reads the seat's one work queue; a node-scoped seat held on several machines has several
+holders, and an ask goes to one of them. The asker addresses the role — `mesh.seat.<seat>.accept.<verb>`
+— and never a machine. The outcome, the role's own event, says which machine did the work (`on`), as a
+build's already does ([ADR 0157](0157-a-build-says-what-it-does-on-the-bus-as-it-happens.md)).
+
+**2. A holder takes one ask at a time, when it is idle, by pulling.** The worker is a pull consumer:
+a holder fetches one ask, works it, acknowledges, fetches the next. The server never hands an ask
+to a busy holder, so a slow machine never holds work an idle one could take — the fault issue 175
+found in push delivery is removed by the shape rather than by a limit, and the one-in-flight limit
+that made the shared queue serial goes with it. A holder that dies mid-work leaves its ask to be
+redelivered to another, as today.
+
+**3. Work that must run on one particular machine is not a work queue.** That is a node seat's verb
+asked of that machine ([design 33](../03-DESIGN/01-to-be/33-the-tools-the-mesh-answers.md) §4), and
+nothing here changes it. A role's work queue is for work whose result is the same whichever holder
+does it: a build is, because what comes out is published by digest to the mesh's store.
+
+**4. This is one pattern, not one role's.** Any module that declares a node-scoped seat with
+`accepts` gets decisions 1 and 2 with no further mechanism: the controller derives the queue and the
+worker, the holders pull, the module's manifest says what every holding machine must have. The
+build agent is the first; a module needing work done *somewhere on the mesh* — a scan, a
+conversion, a fetch — declares a seat of its own the same way
+([ADR 0126](0126-a-module-declares-its-own-seats.md)).
+
+**5. Building is the first such role.** The build role is `node-build-agent`, scope node, with the
+same `build` ask and the same `started`, `built` and `log.<id>` events as before. Its holder is the
+`build-agent` module: the builder as it is — a container runtime, the artifact store and the package
+registry resolved as provisions, a workspace, the bus credential — assignable to every machine that
+has a container runtime. `mesh-build-machine` and the `builder` module are retired when the new
+holder is assigned where the old one was. The tiered plan ([ADR 0162](0162-a-merge-produces-a-tiered-plan-the-mesh-keeps.md))
+is unchanged: a tier's asks go out together and are now worked together.
+
+## Consequences
+
+- A tier of thirty-five images is built by as many machines as hold the seat and are online. A
+  machine that is off builds nothing and blocks nothing.
+- Every holding machine fetches base images from the store and pushes what it builds; the store is
+  reached as a provision, so this is what the provision was for. A machine with a slow link builds
+  slowly, and takes fewer asks for it, which is the point of pulling.
+- A build's outcome carries which machine built it, so a build that fails on one machine and not
+  another is a fact the record shows, not a mystery.
+- What got harder: a build's cache is per machine, so a cold machine pays the first pull of every
+  base it has never seen; the artifact store is now asked by several machines at once, and the
+  package registry likewise. Both are provisions and both are made for that.
+- ADR 0121's *"the mesh's single build machine"* and ADR 0162's *"built by whichever build machine
+  is running — the only one there could be"* were true and are no longer; both records carry a note.
+
+## How it is checked
+
+| Rule | Checked by |
+|---|---|
+| Two holders of one seat each take one of two asks, and a third ask waits for the first to be idle | the controller's test over the work queue against a real bus: two machines bound to one worker, three asks |
+| An ask is never delivered to a busy holder | the same test: the busy holder's ask count stays at one until it acknowledges |
+| A holder that dies mid-work leaves its ask for another | the same test, one holder closed mid-ask |
+| The asker names no machine | the controller's seat table: `node-build-agent` accepts `build` and the asking side publishes to the seat's accept subject, as the existing tests of `build` already require |
+| Live | `builds` shows a tier's builds `on` more than one machine within one plan; `seats` shows `node-build-agent` held on every machine with a container runtime |
+
+## References
+
+- [Design 25](../03-DESIGN/01-to-be/25-the-bus-on-nats.md) §1 and §5 — the work queue and the queue
+  group of holders this uses as drawn
+- [Design 18](../03-DESIGN/01-to-be/18-building-a-module.md) — building a module, amended for
+  where a build runs
+- [ADR 0041](0041-events-are-a-relationship.md), [ADR 0121](0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md),
+  [ADR 0126](0126-a-module-declares-its-own-seats.md), [ADR 0157](0157-a-build-says-what-it-does-on-the-bus-as-it-happens.md),
+  [ADR 0162](0162-a-merge-produces-a-tiered-plan-the-mesh-keeps.md)
+- [Issue 175](../04-ISSUES/175-an-announcement-behind-a-long-build-comes-back/00-report.md) — why the
+  worker had one in flight, and why pulling removes the cause rather than the symptom
@@ -188,6 +188,7 @@ python3 00-META/checks/index.py            fail if stale
 - **0185** — [A control plane behind its seat's row serves what it can](0185-a-control-plane-behind-its-seats-row-serves-what-it-can.md)
 - **0186** — [A ban list never holds a neighbour, and the mesh's own bans are its own wherever they hang](0186-a-ban-list-never-holds-a-neighbour.md)
 - **0187** — [A dead tracker is not the machine's failure](0187-a-dead-tracker-is-not-the-machines-failure.md)
+- **0190** — [A seat's work is shared by its holders, and building is the first such role](0190-a-seats-work-is-shared-by-its-holders-and-building-is-the-first-such-role.md)

 ### Its tiers, from the bottom up

@@ -4,9 +4,10 @@ status: proposed
 code:
  - mesh-controller cmd/mesh-builder
  - mesh-controller internal/builder
-  - mesh-catalog modules/builder
-updated: 2026-10-01
+  - mesh-catalog modules/build-agent
+updated: 2026-10-02
 decisions:
+  - 02-DECISIONS/0190-a-seats-work-is-shared-by-its-holders-and-building-is-the-first-such-role.md
  - 02-DECISIONS/0157-a-build-says-what-it-does-on-the-bus-as-it-happens.md
  - 02-DECISIONS/0150-a-modules-own-code-runs-as-supervised-processes-under-one-account.md
  - 02-DECISIONS/0142-the-mesh-delivers-its-own-components-as-binaries.md
@@ -268,6 +269,20 @@ ships one and wrong for code the mesh built, which has no unit until the mesh wr
 **Tools, hooks and consumers are not further modes**, which is the test of whether three is the
 right number: they are loaded by a tool host, and a tool host is a process that stays up.

+## Where a build runs
+
+**On whichever machine holding the build role is idle** ([ADR 0190](../../02-DECISIONS/0190-a-seats-work-is-shared-by-its-holders-and-building-is-the-first-such-role.md)).
+The role is `node-build-agent`, a node seat; its holder is the `build-agent` module, assignable to
+every machine with a container runtime. The controller asks the role, never a machine: a tier's asks go
+onto the seat's one work queue together, and each holder pulls one at a time when it is idle, so a
+tier of many images is built by as many machines as hold the seat and are online, and a machine that
+is off builds nothing and blocks nothing. What a holding machine needs is what the builder always
+needed — a container runtime, the artifact store and the package registry as provisions, a workspace,
+the bus credential — said once in the module's manifest. The outcome names the machine that built it.
+
+This is the bus's shared-work pattern, not a build-specific one: any module declaring a node seat with
+`accepts` has its work shared by its holders the same way. Building is the first use.
+
 ## A build says what it does, as it happens

 *2026-10-01 — [ADR 0157](../../02-DECISIONS/0157-a-build-says-what-it-does-on-the-bus-as-it-happens.md).*