Merge pull request 'Issues 211, 212, 214, 216, 217 resolved; 215's fix recorded' (#342) from issues/211-217-resolved into main

04-ISSUES/211-a-bundle-is-built-before-the-toolchain-it-is-compiled-in/00-report.md

@@ -1,9 +1,10 @@
 ---
-status: located
+status: resolved
 opened: 2026-10-03
 located-in:
   - mesh-controller
 fixed-by:
+  - mesh-controller#246
 amended-design:
 ---
 
@@ -45,3 +46,7 @@ have to write — a bundle of language L depends on the module that publishes L'
 bundle lands in the tier after it. A test: a merge touching the toolchain module and a TypeScript
 bundle plans the bundle one tier later. Worked around on the day by building the bundle again once
 the toolchain was built.
+
+## Resolved
+
+Every mesh-tools plan since the fix ran in two rounds: the toolchain and runtime images first, what is built in or on them after. Before it, a merge moving both tiered them together. The planner's test proves the order: a merge moving a bundle and its toolchain plans two rounds.

04-ISSUES/211-a-bundle-is-built-before-the-toolchain-it-is-compiled-in/01-diagnosis.md

@@ -0,0 +1,11 @@
+# 211 — Diagnosis
+
+*2026-10-03.* The planner orders a merge's modules by `inventory.Dependencies`, whose edges come from a
+manifest's `build.on`, from what a build recorded it stood on, from the repositories it read, and from
+the build machine. A bundle names its toolchain by `language`; the builder takes the toolchain image
+(`ToolchainFor(language)`) from what the mesh holds and records nothing of it as stood on. So no edge
+ran from a bundle to the module publishing its toolchain, and a merge moving both (mesh-tools: the
+images and node-tools) tiered them together. **Fix (mesh-controller, branch
+`fix/issue-211-a-bundle-stands-on-its-toolchain`, commit c72f6ca):** `dependenciesOf` adds a `stands-on`
+edge from every bundle artifact to its toolchain's module, read from the manifest. Tested: TypeScript
+bundle → mesh-tools, Go bundle → mesh-tools-go, image → none; a merge moving both plans two tiers.

04-ISSUES/212-a-toolchain-rebuild-keeps-the-sdk-it-cached/00-report.md

@@ -1,10 +1,11 @@
 ---
-status: located
+status: resolved
 opened: 2026-10-03
 located-in:
   - mesh-tools
   - mesh-controller
 fixed-by:
+  - mesh-tools#44
 amended-design:
 ---
 
@@ -42,3 +43,7 @@ install from a lockfile that a release updates, or build the install layer witho
 the build should record which SDK version the image carries, so a bundle's record says what it was
 compiled against. A check: after an SDK release and a toolchain rebuild, a bundle built on it reports
 the released version.
+
+## Resolved
+
+The toolchain now installs the exact SDK version the mesh last published, passed in as a build argument from the SDK module's published package, and the planner orders the toolchain after the SDK. Proven 2026-10-04: the toolchain built with the published SDK, and the six TypeScript bundles built in it serve their tools and seat verbs.

04-ISSUES/214-a-plan-loses-track-of-the-controller-it-rebuilds/00-report.md

@@ -1,9 +1,10 @@
 ---
-status: open
+status: resolved
 opened: 2026-10-03
 located-in:
   - mesh-controller
 fixed-by:
+  - mesh-controller#246
 amended-design:
 ---
 
@@ -34,3 +35,7 @@ Owner mesh-controller (the planner). **Fix direction:** on start, and whenever a
 build, the plan settles an `asked` build against the build records — a build recorded as built from
 the plan's commit is that tier's outcome — so a plan resumes after the controller replaced itself.
 A test: a plan whose build outcome was recorded while no controller followed it resumes on start.
+
+## Resolved
+
+A plan settles an asked build from the build records, whoever heard the outcome. Proven 2026-10-03 and 2026-10-04: both controller merge plans since the fix finished all three rounds, including the round that replaced the controller, without being stopped by hand.

04-ISSUES/214-a-plan-loses-track-of-the-controller-it-rebuilds/01-diagnosis.md

@@ -0,0 +1,10 @@
+# 214 — Diagnosis
+
+*2026-10-03.* A plan learns a tier's outcome only through `planBuilt`, called when a controller takes
+in a build result off the bus. A merge to the controller's repository replaces the controller in tier
+0; the build that produced the new controller was recorded, but the plan state the new controller
+loaded still read `asked`, and no path ever revisited it. **Fix (branch
+`fix/issue-214-a-plan-settles-from-the-build-records`, commit d86baeb):** `advanceOnce` settles every
+still-asked module from its build records — a build recorded after the ask is that ask's outcome,
+built or failed — on every advance and on the 30-second ticker. Tested with a pure helper. Live
+proof: the next merge to mesh-controller passes tier 0 on its own.

04-ISSUES/215-a-module-built-at-a-commit-stops-following-its-branch/00-report.md

@@ -1,9 +1,10 @@
 ---
-status: open
+status: located
 opened: 2026-10-03
 located-in:
   - mesh-controller
 fixed-by:
+  - mesh-controller#246
 amended-design:
 ---
 

04-ISSUES/215-a-module-built-at-a-commit-stops-following-its-branch/01-diagnosis.md

@@ -0,0 +1,10 @@
+# 215 — Diagnosis
+
+*2026-10-03.* `takeIn` registers a build's `Ref` as the branch the module follows. unifi was once
+built with `ref=9c97a8a`, which became its followed ref. `sourceIs` matches a merge only to modules
+whose ref is empty or the merged base — so every merge into main left unifi out — and `askTier`
+re-asks `Source.Ref`, so every plan that rebuilt unifi built the same old commit again (its build
+records all read "at 9c97a8a"). **Fix (branch `fix/issue-215-a-commit-is-never-a-branch-to-follow`,
+commit 6784efa):** registration keeps the followed branch when a build names a commit; matching and
+re-asking read a recorded commit as the default branch, healing existing records; a merge names the
+modules of its repository it leaves out. Store-backed test fails without the fix.

04-ISSUES/216-a-bundle-nothing-loads-is-built-and-never-delivered/00-report.md

@@ -1,9 +1,10 @@
 ---
-status: open
+status: resolved
 opened: 2026-10-03
 located-in:
   - mesh-controller
 fixed-by:
+  - mesh-controller#246
 amended-design:
 ---
 
@@ -28,3 +29,7 @@ Owner mesh-controller (the catalogue's registration check). **Fix direction:** a
 loads, runs or unpacks — no `loads`, no `tools` list on its module, no resource naming it — is refused
 at registration, naming the field that would deliver it. A test: such a manifest is refused; adding
 `loads` admits it.
+
+## Resolved
+
+A bundle nothing would deliver is refused at registration, by name. Proven by the controller's tests; the catalogue's bundles all name what the runtime loads (mesh-catalog#243).

04-ISSUES/216-a-bundle-nothing-loads-is-built-and-never-delivered/01-diagnosis.md

@@ -0,0 +1,9 @@
+# 216 — Diagnosis
+
+*2026-10-03.* The composer delivers a bundle as an archive only when its `Loads` is non-empty, and
+`Loads` derives from the artifact's `loads` or, failing that, from the module's `tools` list. The
+seven modules had neither, so their bundles were recorded and never composed into any declaration;
+nothing checked it. **Fix (branch `fix/issue-216-a-bundle-nothing-delivers-is-refused`, commit
+cf2bb3b):** registration refuses a bundle that nothing loads, runs or unpacks — no `loads`, no `tools`,
+no resource naming it, and not the runtime — naming the field that would deliver it. The current
+catalogue passes the check.

04-ISSUES/217-a-refused-announcement-took-down-every-containers-runtime/00-report.md

@@ -0,0 +1,58 @@
+---
+status: resolved
+opened: 2026-10-03
+located-in:
+  - mesh-tools
+fixed-by:
+  - mesh-tools#42
+  - mesh-tools#43
+amended-design:
+---
+
+# 217 — A refused announcement took down every container's runtime, and the console with it
+
+## What was observed
+
+2026-10-03, rolling out [ADR 0197](../../02-DECISIONS/0197-every-tool-announces-itself-on-the-bus-in-the-nats-services-protocol.md).
+The tool runtimes announced themselves by subscribing `$SRV.<verb>.>`; the grants composed for them
+allowed only `$SRV.<verb>` and the service's own name and instance. The bus refused the wildcard:
+
+```
+Subscription Violation - User "novox.gitea", Subject "$SRV.PING.>"
+NatsError: 'Permissions Violation for Subscription to "$SRV.PING.>"'
+```
+
+The TypeScript runtime the per-module containers run treats a refused subscription as fatal, so on
+the one machine that had received the new images nine containers crash-looped — gitea's runtime,
+postgres's, the catalogue, the vault, mongodb, mssql, keycloak, mailu, nextcloud. Gitea's tools went
+with them, which closed the usual path for merging the fix.
+
+Then the console stopped answering the controller's verbs, though the controller held every
+subscription: the console builds the list that tells a seat's verb from a module's tool by asking the
+controller *and* the catalogue, and gives up on both when the catalogue does not answer — so
+`mesh-controller.status` was asked of a module subject nobody serves.
+
+## Why it matters beyond this instance
+
+Two properties, each worse than the mistake that exposed it:
+
+- **A runtime dies for an optional subscription.** Announcing is discovery; serving tools and running
+  provisioners is the work. A refusal of the first should never stop the second.
+- **The console's view of the mesh's own verbs depended on a module.** The controller's verbs are how
+  the operator repairs the mesh; they must not become unreachable because the catalogue is down.
+
+## Diagnosis
+
+Owner mesh-tools. The wildcard is fixed on branch `fix/announce-only-what-the-grants-allow`: both
+runtimes subscribe exactly what the grants allow. The console that discovers from what announces itself
+(ADR 0195, 0197, on main) asks the bus and the controller, not the catalogue, which removes the second
+property once it is deployed. **Still to do:** the TypeScript runtime treats a refused announcement
+subscription as a logged warning, not as fatal; a test against a bus with real grants proves the
+announcement subscriptions are allowed for every principal kind.
+
+Recovered on the day without the forge's API: the toolchain images built by the controller straight
+from the fix branch, every module image rebuilt on them, and the machine pushed.
+
+## Resolved
+
+A refused announcement is logged and the runtime serves on, and every runtime subscribes only the discovery subjects its grants allow. Proven 2026-10-04: after rolling out to every machine, no container restarts anywhere, and the discovery console lists no runtime as not answering. A refused tool subscription was made non-fatal the same way afterwards, under issue 218 (mesh-tools#46).

04-ISSUES/218-a-mesh-seat-is-answered-by-a-module-that-does-not-hold-it/00-report.md

@@ -0,0 +1,92 @@
+---
+status: resolved
+opened: 2026-10-03
+located-in:
+  - mesh-controller
+  - mesh-tools
+fixed-by:
+  - mesh-controller#248
+  - mesh-tools#45
+  - mesh-tools#46
+amended-design:
+---
+
+# 218 — A seat held once for the mesh is answered by a module on a machine that does not hold it
+
+## What was observed
+
+2026-10-03. Asked which databases the mesh's store holds, `mesh-store.databases` answered from the
+postgres on one machine with that machine's application databases; the controller's own database
+lives on the postgres of the other machine, which the controller's records name as the seat's one
+holder:
+
+```
+mesh-store  scope: mesh  delivers: postgres-database  holders: [ {node: <the control machine>, module: postgres} ]
+```
+
+The discovery console, which reads what answers on the bus ([ADR 0197](../../02-DECISIONS/0197-every-tool-announces-itself-on-the-bus-in-the-nats-services-protocol.md)),
+shows the same seat announced from **both** machines running postgres.
+
+## Why it matters beyond this instance
+
+A seat held once for the mesh promises one answerer: the role's holder. A module that implements a
+seat's verbs on every machine it runs on, and is let serve them on each, turns "the mesh's store" into
+"whichever postgres replied first" — a read against the wrong database that looks like a right one,
+and a write would be worse. Every mesh-scoped seat whose implementing module runs on more than one
+machine has this shape.
+
+## Where to look
+
+Whether the runtime serves a seat's verbs where its module merely *claims* the seat rather than where
+the mesh made it the holder ([ADR 0159](../../02-DECISIONS/0159-a-tool-call-names-the-machine-and-a-holder-serves-its-seats-verbs.md),
+[ADR 0160](../../02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md)):
+the membership the controller issues each assignment, and what the runtime admits from it. A check:
+a mesh-scoped seat's verbs are served by exactly the holder the records name, on every machine.
+
+## Root cause
+
+The controller composed each assignment's held seats from what its module *claims*, once per module
+and not once per machine. Every machine running postgres was therefore given the store seat's grants
+and issued its subjects, and each runtime served the seat's verbs because it serves what it is issued
+([ADR 0160](../../02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md)).
+The runtime behaved as designed. The fault was in what it was issued.
+
+The seat's verbs are not the module's tools. The store's `databases` and `query` are a separate
+implementation registered under the seat's name ([ADR 0159](../../02-DECISIONS/0159-a-tool-call-names-the-machine-and-a-holder-serves-its-seats-verbs.md)).
+Only that implementation should be withdrawn where the module does not hold the seat. postgres's own
+tools stay served on every machine it runs on.
+
+## Fix
+
+The controller now reads the recorded seat holdings when it composes grants and memberships. A seat
+held once for the mesh is issued only to the machine and module the records name as its holder. A
+seat held once per machine, and a mesh seat with no holder on record, are issued as before. Grants
+and memberships come from the same list, so they cannot disagree.
+
+**How it is checked.** A controller test asserts that a claimant on another machine keeps its node
+seats and loses the recorded mesh seat. Live, the discovery console's overview must show each
+mesh-scoped seat announced from exactly the holder the records name. Status moves to `resolved` once
+that holds after the fix is rolled out.
+
+## A second cause, and what the rollout broke (2026-10-04)
+
+With the grants corrected, calls to the store reached only the holder, yet the console still showed
+the seat announced from both machines. The module's runtime added every seat its start-up credential
+claims, even after the mesh issued a membership that left the seat out. Once a membership exists,
+it now alone decides which seat verbs a runtime serves (mesh-tools#45).
+
+The rollout then exposed a third fault. The module on the machine that does not hold the seat was
+still running an image built before #45, so it subscribed to the seat's subject. The corrected grants
+refused that subscription, and the refusal ended the process. Its runtime crash-looped until the
+module was rebuilt on the new runtime image. The database itself kept running. A refused tool
+subscription is now logged and costs only that subject (mesh-tools#46), as a refused announcement
+already did ([issue 217](../217-a-refused-announcement-took-down-every-containers-runtime/00-report.md)).
+
+The module was not rebuilt by the plan that rebuilt the runtime image. This is the ordering gap of
+[issue 211](../211-a-bundle-is-built-before-the-toolchain-it-is-compiled-in/00-report.md) seen
+from a container module.
+
+**Proven 2026-10-04.** The discovery console's overview shows the store seat announced from the
+recorded holder only. Repeated calls to the store are answered by that machine, and the answers
+include the controller's own database. The non-holder still answers its own module tools. No
+container restarts on any machine.

04-ISSUES/219-an-older-build-that-finishes-later-replaces-a-newer-one/00-report.md

@@ -0,0 +1,37 @@
+---
+status: diagnosing
+opened: 2026-10-04
+located-in:
+  - mesh-controller
+fixed-by:
+amended-design:
+---
+
+# 219 — An older build that finishes later replaces a newer one
+
+## What was observed
+
+2026-10-03. Two merges to the runtime module came minutes apart. Each plan asked for every module
+built on the runtime image to be rebuilt. One module's two builds were of the same source and
+differed only in the runtime image they stood on:
+
+| Build | Requested | Finished | Stood on |
+|---|---|---|---|
+| asked by the first plan | 21:33 | 22:04 | the runtime image before the fix |
+| asked by the second plan | 21:49 | 21:56 | the runtime image with the fix |
+
+The older request finished last, and its image became the module's current artifact. The next push
+deployed it, and the module's runtime crash-looped on a fault the newer image had already fixed
+([issue 218](../218-a-mesh-seat-is-answered-by-a-module-that-does-not-hold-it/00-report.md)).
+
+## Why it matters beyond this instance
+
+Which build is current should follow what it was built from, not which build machine was slowest.
+Whenever two plans overlap, which happens on any busy evening, a fix can be silently reverted by a
+build that started before it existed. Every passing check still passes.
+
+## Where to look
+
+How a finished build is recorded and how a module's current artifact is chosen. **How it is checked:**
+a test in which an older request completes after a newer one for the same artifact, and the newer
+stays current.

04-ISSUES/220-a-delivered-bundle-keeps-the-files-of-the-one-before/00-report.md

@@ -0,0 +1,30 @@
+---
+status: open
+opened: 2026-10-04
+located-in: []
+fixed-by:
+amended-design:
+---
+
+# 220 — A delivered bundle keeps the files of the one before
+
+## What was observed
+
+2026-10-04. A tools bundle was rebuilt as one self-contained file per entrypoint
+([ADR 0193](../../02-DECISIONS/0193-every-bundle-the-runtime-serves-is-launched-and-the-runtime-knows-no-language.md)),
+so it no longer carries a package directory. On the machine it was delivered to, its directory still
+held the package directory and a compiled file from the earlier delivery, dated hours before the
+new files. The new files were written over the old directory, and nothing removed what the new
+bundle no longer contains.
+
+## Why it matters beyond this instance
+
+A bundle on disk should be exactly the artifact that was built. Leftover files can be imported by
+code that should no longer find them. A fix that removes a file then works on a fresh machine and
+fails on every machine that ran an earlier version. It also makes "what runs here" impossible to
+read from the artifact.
+
+## Where to look
+
+How the host unpacks a bundle into its directory. **How it is checked:** deliver a bundle, then a
+version without one of its files, and the file is gone.

04-ISSUES/221-a-build-machine-learns-a-new-builder-only-from-a-push/00-report.md

@@ -0,0 +1,33 @@
+---
+status: open
+opened: 2026-10-04
+located-in: []
+fixed-by:
+amended-design:
+---
+
+# 221 — A build machine learns a new builder only from a push
+
+## What was observed
+
+2026-10-04. A controller merge changed how TypeScript bundles are built: one file per entrypoint
+instead of a package directory. Its plan finished, and six bundles were rebuilt right after. They
+came out in the old shape, because the build machines still ran the previous builder. They got the
+new one only from the next push. Rebuilt after that push, the same six came out right.
+
+The same order showed in the bus grants the same night. A push sent while the controller was still
+the previous build composed grants with the previous code. A second push was needed after the new
+controller had started.
+
+## Why it matters beyond this instance
+
+A merge to the controller is not in effect when its plan says done. Anything built or pushed in the
+gap uses the old code and looks current. Today only the operator knows to push first and build
+second, and even the operator forgot.
+
+## Where to look
+
+Whether a controller plan should end by delivering itself to the build machines and the control
+machine, or whether a build should refuse a builder older than the controller that asked for it.
+**How it is checked:** after a controller merge, a build asked right after its plan finishes runs
+the new builder.