From eef03f2b0885236de4550bfa5a0e9d1a1a5058a6 Mon Sep 17 00:00:00 2001 From: jochen Date: Thu, 1 Oct 2026 15:23:26 +0200 Subject: [PATCH] ADR 0160 built: both halves, and what the first roll-out taught; issues 183 (the controller's grant) and 184 (a merge blocks the receive loop) --- ...-and-a-runtime-serves-what-it-is-issued.md | 19 ++++++++ .../00-report.md | 39 +++++++++++++++++ .../00-report.md | 43 +++++++++++++++++++ 3 files changed, 101 insertions(+) create mode 100644 04-ISSUES/183-the-controller-could-not-publish-the-memberships-it-issued/00-report.md create mode 100644 04-ISSUES/184-a-merge-announcement-blocks-the-controllers-receive-loop/00-report.md diff --git a/02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md b/02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md index 1517a86..080644d 100644 --- a/02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md +++ b/02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md @@ -103,6 +103,25 @@ ask what to listen on. This record decides exactly that. | The console lists a stateful module on two machines once per machine, and composes no subject | the MCP conformance test | | Live: the store's databases asked of one named machine and through the seat, after a controller release and one push, with no module rebuilt | by hand | +## Built, 2026-10-01 + +> **Progressive insight — 2026-10-01.** The decision stands; these are the facts of its building. + +- The controller's half: mesh-controller 188 — the membership, its subject, the assignments stream + read directly, a module's account granted its own membership and nothing else of the stream, a + membership published after each push. +- The runtime's half: mesh-tools 23 — the one address derived, the membership read and followed, + exactly the issued subjects served and re-served, the derived shape with a log line until one is + issued, a seat's verbs implemented under the seat's name and never listed as the module's, the + listing carrying subjects and the console composing none. A claim may now name the verbs it + serves for its seat (mesh-controller 186), so a holder's own tools need not be the seat's. +- What the first roll-out taught: the controller's own grant did not name the assignments it issues, + so the first memberships were refused by the server and every runtime kept the derived shape — + which is exactly the fallback this record asked for, and exactly why nobody noticed + ([issue 183](../04-ISSUES/183-the-controller-could-not-publish-the-memberships-it-issued/00-report.md)). + The SDK's `invokeTool` still composes a subject; it reaches a membership through the runtime's + broker, which does, so the caller-side rule is met there and not yet in the SDK's own words. + ## References - [ADR 0159](0159-a-tool-call-names-the-machine-and-a-holder-serves-its-seats-verbs.md) — extended: the same facts, issued rather than derived diff --git a/04-ISSUES/183-the-controller-could-not-publish-the-memberships-it-issued/00-report.md b/04-ISSUES/183-the-controller-could-not-publish-the-memberships-it-issued/00-report.md new file mode 100644 index 0000000..86f453a --- /dev/null +++ b/04-ISSUES/183-the-controller-could-not-publish-the-memberships-it-issued/00-report.md @@ -0,0 +1,39 @@ +--- +status: resolved +opened: 2026-10-01 +located-in: [mesh-controller internal/broker/nats.go (the controller's own publish grant)] +fixed-by: mesh-controller PR 189 (fix/the-controller-may-publish-memberships) +amended-design: [] +--- + +# 183 — The controller could not publish the memberships it issued + +## What was observed + +The controller release that issues a membership per assignment ([ADR 0160](../../02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md)) +rolled onto the control node. After its first push the controller's log said, once: + +``` +nats: permissions violation: Permissions Violation for Publish to "mesh.assignment.." +``` + +No membership reached the assignments stream. Nothing else changed: every runtime kept serving the +shape it derives for itself, which is what the decision says happens while no membership is issued, +and so the mesh looked healthy while the whole new mechanism was inert. + +## Why this is here + +The controller composes every account's grant, its own included, and its own grant named the +control, node and JetStream subjects and not the assignments it alone issues. A grant composed by its +holder is checked by nothing but the server at publish time, and a refused publish is one log line +that nothing reads. The fallback that makes the roll-out safe is the same thing that makes this +failure silent. + +## Resolved, 2026-10-01 + +The controller's grant names `mesh.assignment.>`; the broker golden changed by that one line. A +grant composed by its holder arrives late — the broker node is pushed after the controller rolls — +so the roll-out is merge, push the broker node, then any push issues memberships. + +*How it is checked:* the broker golden carries the allow line; live, a runtime's log after the next +push says it was issued a membership rather than that none exists. diff --git a/04-ISSUES/184-a-merge-announcement-blocks-the-controllers-receive-loop/00-report.md b/04-ISSUES/184-a-merge-announcement-blocks-the-controllers-receive-loop/00-report.md new file mode 100644 index 0000000..422662d --- /dev/null +++ b/04-ISSUES/184-a-merge-announcement-blocks-the-controllers-receive-loop/00-report.md @@ -0,0 +1,43 @@ +--- +status: open +opened: 2026-10-01 +located-in: [mesh-controller internal/link/serve.go (act handles one message at a time; sourceMoved waits for every build the merge asks)] +fixed-by: +amended-design: [] +--- + +# 184 — A merge announcement blocks the controller's receive loop + +## What was observed + +The controller restarted at 12:53:32Z on 2026-10-01 and heard, among its first messages, a merge +announcement for the catalogue that touched some forty modules. Until 13:17:16Z — twenty-four +minutes — it took nothing else in: build outcomes that the build machine had announced and that +the console's `builds` log showed as done sat unrecorded, so `builds` listed none of them and the +modules stayed at their old versions; the node heartbeats were dropped by the bus as a slow consumer +on `mesh.control.*.alive`, twice. When the merge handler returned, everything queued arrived at once +and was taken in within a second. + +## Why this is here + +The receive loop acts on one message at a time, which is the right discipline for a store the +controller must write in order. Acting on a merge means asking builds and waiting for each outcome, +minutes of work, and that wait happens inside the loop that would otherwise be hearing the outcomes +of everything else. The bus keeps the merge message alive while the work runs — the fix for the +earlier repeated-merge fault — so nothing is lost and nothing is redelivered, and nothing is heard +either. The design permits the controller to go deaf for as long as a merge takes to build, and no +status says so: the mesh reads as quiet, builds read as missing, and heartbeats read as a slow +machine. + +## What a fix needs to decide + +Whether a merge is work the loop dispatches and returns from — the builds asked, the outcomes taken +in by the same `Built` handler every other outcome uses, since the stream already delivers them — +or whether the loop runs more than one handler at a time with the store's ordering kept for the +kinds that need it. The first is smaller and keeps one ordering. Either way, a controller that is +busy should say so where `status` is read. + +*How this would be checked:* a controller test where a merge announcement that asks a slow build +and a build outcome for another module arrive together, and the outcome is recorded before the +build finishes; live, the controller's log during the next catalogue merge shows registrations +interleaved with the merge's own.