Merge pull request 'Issue 299: an optional verb makes a holder look silent' (#180) from issues/299-an-optional-verb-makes-a-holder-look-silent into main

2026-10-07 21:01:34 +00:00
parent dda850863f 6006754172
commit 2adbfed90f
2 changed files with 93 additions and 0 deletions
@@ -0,0 +1,52 @@
+---
+status: located
+opened: 2026-10-07
+located-in: [mesh-controller cmd/mesh-controller (probes.go, the self-check's D3)]
+fixed-by: mesh-controller PR #121
+amended-design:
+---
+
+# 299. An optional verb makes a holder look silent
+
+## Symptom
+
+From 19:25 UTC on 2026-10-07, the controller's conditions held four urgent conditions
+`seat.node-uplink.<machine>.silent`, one for every machine of the mesh:
+
+> node-uplink's holder on <machine> does not answer the bus: its verbs reach nothing there
+
+Each condition's evidence read "no answer for node-uplink from <machine> to the bus's discovery". The
+healer H3 acted on each at 19:25 UTC, asserting the bus's objects again, and the conditions stayed open.
+Nothing was wrong with the holders: every machine's network manager held the seat as before.
+
+## What changed
+
+mesh-controller #116, merged about 19:20 UTC, gave the `node-uplink` seat its first verbs, `resolvers`
+and `links`. Both are optional
+([ADR 0246](../../02-DECISIONS/0246-a-seats-new-verb-is-promised-before-it-is-required.md), step 1): the
+seat's holders, the `networkmanager` and `systemd-networkd` modules, do not serve them yet, and
+mesh-catalog #107 will. Until that afternoon the seat served no verb, and the self-check's D3 skipped
+it. From #116 on, D3 expected every holder of the seat to answer the bus's discovery for it, and none
+did.
+
+## Why it is a design issue
+
+ADR 0246 and [design 33](../../03-DESIGN/01-to-be/33-the-tools-the-mesh-answers.md) §7 say that between
+steps 1 and 3 "a holder that does not yet serve it still holds the seat". The claim check honours that.
+D3 did not: it asked whether a seat has verbs, never whether its holder must serve any. So step 1 of the
+rule raised a false urgent condition on every machine holding the seat. Every verb added to a held seat
+the way ADR 0246 says would do the same, if the seat had no required verb before.
+
+A false urgent condition is not harmless. It trains the reader to ignore the board, and H3 acts on it.
+
+## How it is checked
+
+- `TestAHolderOfASeatWhoseVerbsAreAllOptionalIsNotSilent` replays this case. A seat row read back from
+  the store with `node-uplink`'s verbs, all optional, and a holder on four machines that answers nothing:
+  no holder is expected to answer, and nothing is said silent. It fails on the commit before the fix.
+- `TestAHolderServingNoRequiredVerbIsStillSilent` keeps D3 alive. A seat with one required and one
+  optional verb, whose holder answers nothing, is still said silent.
+- `TestHoldingNeedsAnAnswer` covers the rule itself.
+
+The trail is in [01-diagnosis.md](01-diagnosis.md). This is a core issue: at resolution it names its
+replay, or says in `replay-none:` why none is possible (ADR 0237).
@@ -0,0 +1,41 @@
+# 299 — diagnosis
+
+## 2026-10-07: where the condition is raised
+
+The text "does not answer the bus" is raised in one place: D3 of the controller's self-check
+(`probeHolders`). D3 builds an expectation, seat to machines, and asks the bus's discovery who serves
+what. An expected holder that is not heard twice, and again at the next run, is `silent`.
+
+The expectation took every seat with a protocol and skipped only a seat with no verbs:
+`len(s.Serves) == 0`. It did not look at whether a verb is optional. The seat set the controller works
+from is read from the store, and the optional mark comes from the compiled seat (#114), so the verbs of
+`node-uplink` were optional at run time. D3 read them as verbs all the same.
+
+Discovery answers by seat, not by verb: an endpoint carries the seat and the machine in its metadata. A
+holder that serves none of the seat's verbs registers no endpoint for it, so discovery has nothing to
+say about it.
+
+Ruled out:
+
+- **A real outage.** The holders had not changed, and the conditions opened at the first D3 run after
+  #116 reached the controller, on every machine at once.
+- **A late discovery answer** (issue 277). D3 already asks twice and confirms at the next run. The
+  holders were never going to answer: they register no endpoint for the seat.
+
+## The fix
+
+Silence is judged on required verbs only. A seat is expected to have answering holders only if it has at
+least one verb that is not optional (`holdingNeedsAnAnswer`). A seat whose verbs are all optional asks
+nothing of its holders. Step 3 of ADR 0246 makes a verb required, and from then on D3 expects its holders
+to answer. The expectation and the judgement were pulled out of `probeHolders` into two pure functions,
+`holdersToHear` and `silentHolders`, so they can be tested without a bus.
+
+That makes D3 agree with ADR 0246 and design 33 §7: a holder that does not serve an optional verb still
+holds the seat, and is not told that it is silent.
+
+## What it does not cover
+
+D3 still judges by seat. A holder that answers for some verb of a seat counts as answering, even if it
+does not serve one of the required ones. The claim check catches a holder that does not declare a
+required verb, at registration and at handover. Catching a holder that declares one and then fails to
+serve it would need discovery to report verbs, not only seats.