From 2bceb5fbaa3016b9bd0007672718c6468c175945 Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 9 Oct 2026 18:09:45 +0200 Subject: [PATCH 1/2] Issue 353: the controller asked before the bus held its grant --- .../00-report.md | 57 +++++++++++++++++++ 1 file changed, 57 insertions(+) create mode 100644 04-ISSUES/353-the-controller-asked-before-the-bus-held-its-grant/00-report.md diff --git a/04-ISSUES/353-the-controller-asked-before-the-bus-held-its-grant/00-report.md b/04-ISSUES/353-the-controller-asked-before-the-bus-held-its-grant/00-report.md new file mode 100644 index 00000000..0bff1898 --- /dev/null +++ b/04-ISSUES/353-the-controller-asked-before-the-bus-held-its-grant/00-report.md @@ -0,0 +1,57 @@ +--- +status: located +opened: 2026-10-09 +located-in: [mesh-controller cmd/mesh-controller (asker.go, asker_wire.go), mesh-controller internal/broker (users.go, asksSeatOf)] +fixed-by: mesh-controller pull request 186 +amended-design: +replay-none: the whole of it is one process's record against one bus's user list; the controller's own test drives the asker with the bus's word held and withheld (TestNothingIsAskedWhileTheBusLacksTheControllersGrant), and a container runtime adds nothing to that +--- + +# 353. The controller asked before the bus held its grant + +## Symptom + +On 2026-10-09, local times on the control node, from its controller's journal (one process throughout): + +- 17:32 the router (the `operator-channel` holder) was unassigned, so that a merge gate of the catalogue had + nothing to send there; 17:52:54 a walk of another module pushed the control node and composed the bus's user + list **without** the controller's grant to ask: the grant is composed from a holder of the seat that is + *assigned*, and none was. +- 17:54:05 the controller started asking (the router was assigned again by then, so its check of the record + said a router was here), and the bus refused its publish of the ask, on the same connection, seven times + until 17:56:05. Each refusal was followed by "the operator could not be asked about machine.…: context + deadline exceeded". The self-check raised `bus.controller.refused`. +- 17:55:56 the push that put the router back on the control node sent it the user list **with** the grant; 17:56:05 the bus reloaded + it in place (the same second the watchdogs saw "jetstream not enabled", the reload's moment); 17:56:10 the + same connection asked successfully, and again at 18:03:53. The server applies a reloaded permission to a + standing connection: no reconnect was needed or made. +- The condition stays open for an hour after the last refusal (the bus-advisory signal's quiet time) and + clears by itself. + +## Why it is an issue + +- **The record said yes before the bus did.** The controller's grant to ask follows the router's assignment + ([ADR 0259](../../02-DECISIONS/0259-an-ask-is-a-message-on-the-bus-any-channel-answers-it-and-the-asker-acts-on-a-warrant.md) + §3: a caller-named verb is granted to the module that uses the seat), and reaches the bus only when the + machine holding the bus is next pushed. Between `assign` and `push`, and on any push of that machine made + while the router is unassigned, the controller believed its own record of the router and published into a + refusal. The composition is right; the asker trusted it before it was on the bus. +- **The failure was loud in the wrong place.** Seven bus refusals and a warning about the bus, when the fact + was "the control node has not been pushed since the router was assigned", which the controller knows (its + push already judges whether the bus's machine is behind on the user list, to send it first). + +## Acceptance test (the operator's view) + +While the bus has not been given the controller's grant to ask, the controller asks nothing and the condition +"Questions for you not delivered" says which push carries the grant; once that push lands, the questions go +out, and the bus never refuses the controller's ask. + +## How it is checked + +- `TestNothingIsAskedWhileTheBusLacksTheControllersGrant` (mesh-controller `cmd/mesh-controller/asker_test.go`): + nothing is asked while the bus's machine was not sent the user list composed now, it is said once, the + conditions that need the operator are raised as undelivered naming the push, and they are asked once the bus + holds the grant. +- Live: after the fix is rolled out, `assign` the router, read `mesh-controller.conditions` for + `asks-undelivered` naming the push, `push` the control node, and read that the asks go out with no + `bus.controller.refused` raised. From adbd57a1f935920db3ab56d94b5e9d50378ff3c4 Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 9 Oct 2026 19:35:09 +0200 Subject: [PATCH 2/2] Issue 353: the condition cleared at 18:11:44, about 15 minutes after the last refusal, not an hour after --- .../00-report.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/04-ISSUES/353-the-controller-asked-before-the-bus-held-its-grant/00-report.md b/04-ISSUES/353-the-controller-asked-before-the-bus-held-its-grant/00-report.md index 0bff1898..f1e81cc9 100644 --- a/04-ISSUES/353-the-controller-asked-before-the-bus-held-its-grant/00-report.md +++ b/04-ISSUES/353-the-controller-asked-before-the-bus-held-its-grant/00-report.md @@ -25,8 +25,10 @@ On 2026-10-09, local times on the control node, from its controller's journal (o it in place (the same second the watchdogs saw "jetstream not enabled", the reload's moment); 17:56:10 the same connection asked successfully, and again at 18:03:53. The server applies a reloaded permission to a standing connection: no reconnect was needed or made. -- The condition stays open for an hour after the last refusal (the bus-advisory signal's quiet time) and - clears by itself. +- `bus.controller.refused` was raised at 17:54:06 and cleared by itself at 18:11:44, about 15 minutes after + the last refusal at 17:56:05 (the controller's condition history). The condition's evidence lists the same + refusal read again every 30 seconds until 18:10:35; the journal shows no refusal after 17:56:05, so those + are the signal reading its record again, not new refusals. ## Why it is an issue