Merge pull request 'Issue 298 and ADR 0246: a seat's new verb is promised before it is required' (#178) from issues/298-a-new-seat-verb-deadlocks into main

2026-10-07 19:21:52 +00:00
parent 3083283e17 91daabaa97
commit 0bd12fc777
8 changed files with 249 additions and 3 deletions
@@ -53,7 +53,8 @@ is listed under [Homonyms](#homonyms), with the qualified form each domain uses.
  provision**, and its holder is then the mesh's answer for it when several modules provide it
  ([ADR 0126](../02-DECISIONS/0126-a-module-declares-its-own-seats.md)). The set, with who holds each
  seat, is the overview of what a mesh has ([26 — The seats](../03-DESIGN/01-to-be/26-the-seats.md)).
-  A seat carries **verbs** — the tools every holder must serve.
+  A seat carries **verbs** — the tools every holder must serve. A verb added to a held seat is first
+  promised as optional, then served, then required ([ADR 0246](../02-DECISIONS/0246-a-seats-new-verb-is-promised-before-it-is-required.md)).
 - **operator** — the person who runs the mesh, and the one the mesh talks to.
  *Not:* ~~master~~ (hq)
 - **person** — any human, as against the mesh acting unattended: *a person's word* releases a held
@@ -144,6 +144,18 @@ Nobody opens a terminal on the machine. The verbs are *staged*: promised and rou
 of holding the seat, and never stored in the seat's row. A controller older than them reads the row and
 would refuse every holder of its time. A later change requires them once both holders serve them.

+> **Progressive insight — 2026-10-07.** The verbs did not ship *staged*. This rule said they were
+> "promised and routed, but not yet a condition of holding the seat, and never stored in the seat's row",
+> and the check for rule 8 in the table below said "a staged verb is never seeded into the row and comes
+> back from the binary". The controller that merged (mesh-controller #116) has no staged seat. It marks
+> both verbs optional (`Verb.Optional`). An optional verb is promised, so a claim serving it is accepted,
+> but it is not required, so a holder serving neither still holds the seat. It is seeded into the seat's
+> row like any verb, and it stays optional when read back, because the mark is taken from the seat as
+> compiled. Mesh-controller's `TestTheUplinkVerbsReadBackFromTheRowStayOptional` holds that. The
+> decision is unchanged: promised now, required once both holders serve them. The general rule for a verb
+> added to a held seat is [ADR 0246](0246-a-seats-new-verb-is-promised-before-it-is-required.md), from
+> [issue 298](../04-ISSUES/298-a-new-seat-verb-deadlocks-across-two-repositories/00-report.md).
+
 ## Consequences

 - **The VPN rewriting the laptop's file is said within about a minute,** naming FortiClient from its
@@ -0,0 +1,83 @@
+---
+topic: the mesh
+status: accepted
+date: 2026-10-07
+deciders: jochen
+reconstructed: false
+extends: 02-DECISIONS/0132-a-seat-carries-the-tools-its-holder-must-serve.md
+---
+
+# 246. A seat's new verb is promised before it is required
+
+## Context
+
+[ADR 0132](0132-a-seat-carries-the-tools-its-holder-must-serve.md) made serving a seat's verbs a
+condition of holding it. [Design 33](../03-DESIGN/01-to-be/33-the-tools-the-mesh-answers.md) §7 says
+those verbs change "additively within a version". The controller's claim check enforces two things: a
+holder serves every verb the seat promises, and a claim names no verb the seat does not promise.
+
+A mesh seat is defined in the controller, and its holders live in the catalogue. On 2026-10-07 three
+verbs were added this way: `checks` on `mesh-delivery`, `failed` on `node-service-manager`, and
+`resolvers` and `links` on `node-uplink`. Each time, neither repository could merge first
+([issue 298](../04-ISSUES/298-a-new-seat-verb-deadlocks-across-two-repositories/00-report.md)). The new
+controller refused the old holder for not serving the verb. The old controller refused the new holder for
+claiming a verb it did not promise. Every catalogue check between the two failed on a module nobody had
+touched. The controller was unblocked by marking the added verb optional (mesh-controller #117, kept
+optional in a stored seat row by #114).
+
+## Options
+
+1. **Drop the check that refuses an unpromised verb.** The catalogue could then land first. But the check
+   catches a misspelt verb that would otherwise be served to nobody. The controller's change would still
+   refuse every holder not yet rebuilt, so only one order opens.
+2. **A new seat version for every added verb.** §7 already allows a version for a change that would
+   break a caller. Running two versions side by side to add one verb costs a set of subjects, grants and
+   a retirement for nothing a caller would notice.
+3. **Land both repositories at once.** Nothing joins two checked pull requests into one merge, and the
+   mesh would have to run both builds at the same moment on every machine.
+4. **Promise first, require later.** The controller promises the verb as optional. Then the holder
+   serves it. Then the controller makes it required.
+
+## Decision
+
+Option 4. **A verb added to a seat whose holder lives in another repository lands in three steps, each
+in its own pull request:**
+
+1. **The controller promises it, optional.** A claim that serves it is accepted, and a holder that does
+   not serve it still holds the seat. The mark comes from the seat as compiled and survives a seat row
+   read back from the store.
+2. **Every holder serves it.** Each holder is checked against a controller that already promises the
+   verb.
+3. **The controller requires it.** The mark is removed, and serving the verb becomes a condition of
+   holding like every other verb of the seat.
+
+A verb on a new seat, or on a seat that nothing holds yet, is required from the start. Removing a verb
+or changing its arguments in a way that breaks a caller is still a new version (§7).
+
+## Consequences
+
+- Adding a verb to a held seat takes two controller changes and one in each holder's repository, where
+  design 33 implied one change. That is the price of §3 holding at every moment in between.
+- Between steps 1 and 3, a caller can be told that a holder does not answer the verb. The caller finds
+  this out from the holder, not from the seat's protocol. Discovery lists the verb, because it is
+  promised.
+- Nothing yet checks that step 3 happens. A verb left optional is a promise that no future holder has to
+  keep. Issue 298 is not resolved until a check covers it.
+
+## How it is checked
+
+- The controller's claim check refuses a claim that serves an unpromised verb, and a holder that misses a
+  required verb. It runs at registration, at handover, and in `module check`, which the catalogue's merge
+  check runs over every manifest with the controller the mesh runs.
+- Tests in mesh-controller:
+  - `TestTheDeliverySeatPromisesChecks` (#117);
+  - `TestFailedIsAnOptionalVerbOfTheServiceManager`, `TestAnOptionalVerbStaysOptionalInASetReadFromTheStore`
+    and `TestASeatsVerbGainsTheArgumentsTheBinaryNames` (#114);
+  - `TestTheUplinkSeatPromisesItsVerbsAndRequiresNoneYet` and
+    `TestTheUplinkVerbsReadBackFromTheRowStayOptional` (#116).
+
+## References
+
+- [Issue 298](../04-ISSUES/298-a-new-seat-verb-deadlocks-across-two-repositories/00-report.md)
+- [Design 33](../03-DESIGN/01-to-be/33-the-tools-the-mesh-answers.md) §3 and §7
+- mesh-controller #114, #116 and #117; mesh-catalog #105, #107 and #109
@@ -208,6 +208,7 @@ python3 00-META/checks/index.py            fail if stale
 - **0237** — [A change is judged against the mesh that runs, before it merges, on the build seat](0237-a-change-is-judged-against-the-mesh-that-runs-before-it-merges-on-the-build-seat.md)
 - **0238** — [A commit is the build at hand: one commit, one change plan, checked off the trunk and published only on it](0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md)
 - **0239** — [A delivery is owned by the mesh-delivery module and runs from commit to delivered](0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md)
+- **0246** — [A seat's new verb is promised before it is required](0246-a-seats-new-verb-is-promised-before-it-is-required.md)

 ### Its tiers, from the bottom up

@@ -2,8 +2,9 @@
 layer: to-be
 status: implemented
 code: [mesh-controller, mesh-tools]
-updated: 2026-10-02
+updated: 2026-10-07
 decisions:
+  - 02-DECISIONS/0246-a-seats-new-verb-is-promised-before-it-is-required.md
  - 02-DECISIONS/0170-the-firewall-seat-serves-its-verbs.md
  - 02-DECISIONS/0179-the-intrusion-seat-serves-its-verbs-and-every-door-declares-its-jail.md
  - 02-DECISIONS/0160-the-mesh-issues-an-assignments-subjects-and-a-runtime-serves-what-it-is-issued.md
@@ -131,6 +132,24 @@ A seat's tools are an interface and change like one. Additive within a version.
 break a caller takes the version token the subject already has room for, and the two versions run side
 by side until nothing is bound to the old one.

+*Amended 2026-10-07 ([ADR 0246](../../02-DECISIONS/0246-a-seats-new-verb-is-promised-before-it-is-required.md),
+[issue 298](../../04-ISSUES/298-a-new-seat-verb-deadlocks-across-two-repositories/00-report.md)):*
+**additive takes two steps when the seat and its holder are in different repositories.** That is the
+case for every mesh seat whose holder is a catalogue module. §3 makes every promised verb a condition of
+holding, and a claim may name no verb the seat does not promise. So a verb added to the seat in the
+controller refuses today's holder, and a holder that serves it is refused by today's controller. Neither
+change can land first. An added verb therefore lands in three steps, each its own pull request:
+
+1. **The controller promises the verb as optional.** A holder that serves it is accepted, and one that
+   does not yet serve it still holds the seat. The mark comes from the seat as compiled, so a seat row
+   read back from the store keeps it.
+2. **Every holder serves it**, checked against a controller that already promises it.
+3. **The controller requires it.** The mark is removed, and the verb is a condition of holding like the
+   rest.
+
+A verb on a seat that nothing holds yet is required from the start. A change that breaks a caller is
+still a new version.
+
 ## How it is checked

 - **A holder missing a verb cannot take the seat.** One test per condition of holding, as the existing
@@ -142,6 +161,16 @@ by side until nothing is bound to the old one.
  answer equals what the seats declare — if it needed a module up, it would not be a read.
 - **Two nodes holding one node-scoped seat derive two addresses.** Checked by the same test as the rest
  of the subject table.
+- **An added verb lands in steps, and each step is checked (§7).** The controller's claim check refuses
+  a claim that serves a verb the seat does not promise, and a holder that misses a required verb. It runs
+  at registration, at handover, and in `module check`, which the catalogue's merge check runs over every
+  manifest with the controller the mesh runs. So a step out of order fails the pull request that took
+  it. That an optional verb stays optional when read back from the store is held by mesh-controller's
+  `TestAnOptionalVerbStaysOptionalInASetReadFromTheStore` (#114), and that a seat's row gains a verb's
+  new arguments by `TestASeatsVerbGainsTheArgumentsTheBinaryNames` (#114). Each added verb has its own
+  test that it is promised and not yet required: `TestTheDeliverySeatPromisesChecks` (#117) and
+  `TestFailedIsAnOptionalVerbOfTheServiceManager` (#114). **Not yet checked:** that step 3 happens. A
+  verb left optional binds no future holder ([issue 298](../../04-ISSUES/298-a-new-seat-verb-deadlocks-across-two-repositories/01-diagnosis.md)).

 ## What is built, 2026-09-30

@@ -261,7 +261,10 @@ the machine itself, beside its liveness looks, and says it in the same statement
 - **It reads and never acts** (§7). The reconcile writes the file back as it always has.
 - **Asked further through the uplink seat.** `node-uplink` serves `resolvers` (the file, whether it is the
  mesh's, its writer) and `links` (each link, its default route, the resolvers its manager knows). They are the
-  same from every holder, and staged until both holders serve them.
+  same from every holder, and optional until both holders serve them: promised, so a holder may serve them,
+  and not yet required to hold the seat (mesh-controller #116; the three steps of an added verb are
+  [ADR 0246](../../02-DECISIONS/0246-a-seats-new-verb-is-promised-before-it-is-required.md), from
+  [issue 298](../../04-ISSUES/298-a-new-seat-verb-deadlocks-across-two-repositories/00-report.md)).

 ## Phases

@@ -0,0 +1,71 @@
+---
+status: located
+opened: 2026-10-07
+located-in: [mesh-controller internal/catalogue (seats.go, verbs.go), mesh-catalog (the seats' holders)]
+fixed-by: mesh-controller PR #117, mesh-controller PR #114
+amended-design: 03-DESIGN/01-to-be/33-the-tools-the-mesh-answers.md
+---
+
+# 298. A new seat verb deadlocks across two repositories
+
+## Symptom
+
+[Design 33](../../03-DESIGN/01-to-be/33-the-tools-the-mesh-answers.md) §7 says a seat's verbs are
+"additive within a version". §3 says a module may not hold a seat unless it serves every verb the seat
+declares. A mesh seat's verbs are defined in the controller, and the modules that hold it live in the
+catalogue. Under those two rules, adding a verb cannot land in one step, in either order:
+
+- **The controller first.** The new controller promises the verb. The holder in the catalogue does not
+  serve it yet, so the claim check refuses the holder ("claims … but does not serve …, which that
+  seat's protocol promises"). Every check of the catalogue run against that controller fails on a module
+  nobody touched.
+- **The catalogue first.** The new holder's claim says it serves the verb. The controller the mesh runs
+  does not promise it, so the same check refuses the claim ("says it serves …, which that seat's
+  protocol does not promise"). The catalogue's merge check runs every manifest through the controller
+  the mesh runs, so the pull request cannot pass.
+
+Each repository's change waits on the other's, and neither can merge first.
+
+## Evidence
+
+On 2026-10-07 this happened three times, once for each verb added to a mesh seat that day:
+
+| Seat | Verb | Controller | Catalogue |
+|---|---|---|---|
+| `mesh-delivery` | `checks` | mesh-controller #117 | mesh-catalog #109 |
+| `node-service-manager` | `failed` (until then the systemd module's own tool) | mesh-controller #114 | mesh-catalog #105 |
+| `node-uplink` | `resolvers`, `links` | mesh-controller #116 | mesh-catalog #107 |
+
+Each pair was unblocked the same way. The controller's `Verb` gained an `Optional` mark (#117): an
+optional verb is promised, so a claim that serves it is accepted, but it is not required, so a holder
+that does not serve it still holds the seat. `failed` was first written as required and refused today's
+holder. #114 made it optional and fixed a second hole. The mark was never stored, so a verb seeded into
+a seat row came back required when the row was read. The working set now takes the mark from the
+compiled seat. #116 adds the uplink verbs as optional from the start. ADR 0241 rule 8 and to-be 48 §10
+had called those two verbs *staged*, never stored in the seat's row. That was corrected on the day,
+as a progressive insight in ADR 0241 and in place in to-be 48: they are optional, stored in the row, and
+read back optional.
+
+## Why it is a design issue
+
+Design 33 says what may change (additively) but not how a change crosses the two repositories that
+share a seat. §3's condition of holding, enforced as designed, makes "additive" impossible whenever the
+seat's definition and its holder are in different repositories. That is the case for every mesh seat
+whose holder is a catalogue module. The rule that settles it is now in §7, by
+[ADR 0246](../../02-DECISIONS/0246-a-seats-new-verb-is-promised-before-it-is-required.md): the
+controller first promises the verb as optional, the holder serves it, and the controller requires it.
+
+## How it is checked
+
+- The controller's claim check (`internal/catalogue/seats.go`) refuses both an unpromised verb and a
+  missing required one. It runs at registration, at handover, and in `module check`, which the
+  catalogue's merge check runs over every manifest.
+- Tests in the controller:
+  - `TestTheDeliverySeatPromisesChecks` (#117);
+  - `TestFailedIsAnOptionalVerbOfTheServiceManager`, `TestAnOptionalVerbStaysOptionalInASetReadFromTheStore`
+    and `TestASeatsVerbGainsTheArgumentsTheBinaryNames` (#114);
+  - `TestTheUplinkSeatPromisesItsVerbsAndRequiresNoneYet` and
+    `TestTheUplinkVerbsReadBackFromTheRowStayOptional` (#116).
+
+The trail is in [01-diagnosis.md](01-diagnosis.md). This is a core issue: at resolution it names its
+replay, or says in `replay-none:` why none is possible (ADR 0237).
@@ -0,0 +1,46 @@
+# 298 — diagnosis
+
+## 2026-10-07: the two refusals
+
+Both refusals are in one function of the controller, the claim check in `internal/catalogue/seats.go`.
+After scope and provision, it checks two things about a claim:
+
+1. `unservedVerbs`: every verb the seat's protocol promises is among what the claimant serves, under
+   the claim's `serves` or among its own tools. This is design 33 §3 and ADR 0132.
+2. `unpromised`: everything the claim's `serves` names is a verb the protocol promises. A verb nobody
+   promised is served to nobody, so this catches a typo before a caller finds it as a timeout.
+
+The protocol of a mesh seat is compiled into the controller and seeded into the store. Its holders are
+catalogue modules. Check 1 refuses the old holder against the new controller, and check 2 refuses the new
+holder against the old controller. The catalogue's merge check runs `module check` over every manifest
+with the controller the mesh runs (`MESH_GATE`), so either order fails a check on one side.
+
+Ruled out:
+
+- **Dropping check 2.** Without it, a claim could name verbs that nothing promises, and the typo it
+  catches would come back. It would also only open one order (catalogue first). The controller's change
+  would still refuse every holder that is not yet rebuilt.
+- **One commit in both repositories.** The two repositories merge separately, each through its own
+  checked pull request (ADR 0238). Nothing joins two pull requests into one merge.
+- **A new seat version (§7's second sentence).** A version is for a change that would break a caller,
+  and running two versions side by side for an added verb costs more than the verb.
+
+## 2026-10-07: the mark, and the store
+
+`Verb.Optional` (mesh-controller #117) makes check 1 skip a verb that is marked, while check 2 still
+counts it as promised. With the mark, the controller lands first and the old holder still holds the seat.
+The holder that serves the verb then lands against a controller that promises it. Once every holder
+serves it, removing the mark makes it a condition of holding like any other verb.
+
+#114 found two holes in the store. First, re-seeding a seat row added a verb but never a new argument to
+an existing verb, so the journal window's arguments would never have reached the row the mesh MCP server
+reads. Second, the mark has `json:"-"` and is never stored, so a seeded verb came back required and the
+deadlock returned on any mesh whose seat rows already existed. The working set now takes the mark from
+the compiled seat (`optionalAsCompiled`), and a row's verb gains the arguments the binary names.
+
+## What is still not checked
+
+Nothing checks that the mark is ever removed. A verb that stays optional after every holder serves it is
+a promise that no future holder has to keep. The design now names the third step. A check that fails
+when a holder of the mesh already serves an optional verb, or one that fails after a date, would close
+this. Both are left to the controller, and are the reason this issue stays `located`.