From 127675e2da943d56aa733578cf97bf0c0c5cd020 Mon Sep 17 00:00:00 2001 From: jochen Date: Mon, 5 Oct 2026 09:59:17 +0200 Subject: [PATCH] Issue 243: a rebuilt licence store silences every node The manager's generation counter restarted with its store after issue 241, and nodes that wait for a higher number discarded every binding unseen. --- .../00-report.md | 44 +++++++++++++++++++ .../01-diagnosis.md | 33 ++++++++++++++ 2 files changed, 77 insertions(+) create mode 100644 04-ISSUES/243-a-rebuilt-licence-store-silences-every-node/00-report.md create mode 100644 04-ISSUES/243-a-rebuilt-licence-store-silences-every-node/01-diagnosis.md diff --git a/04-ISSUES/243-a-rebuilt-licence-store-silences-every-node/00-report.md b/04-ISSUES/243-a-rebuilt-licence-store-silences-every-node/00-report.md new file mode 100644 index 0000000..84e0d81 --- /dev/null +++ b/04-ISSUES/243-a-rebuilt-licence-store-silences-every-node/00-report.md @@ -0,0 +1,44 @@ +--- +status: located +opened: 2026-10-05 +located-in: [mesh-catalog modules/claude-code, mesh-catalog modules/claude-licence-manager] +fixed-by: +amended-design: +--- + +# 243 — A rebuilt licence store silences every node, and nothing says so + +## What was observed + +The licence manager's store lost its tables in the incident of +[issue 241](../241-one-unreadable-grants-file-dropped-every-database-on-the-control-node/00-report.md). When +they were made again, the manager adopted the remaining licence from a node's report, as +[ADR 0206](../../02-DECISIONS/0206-a-node-reports-the-anthropic-grant-it-holds-and-the-licence-manager-adopts-a-licence-by-refreshing-it.md) +intends. It bound every node again and rotated the licence on schedule. Its log showed nothing wrong. + +On the machines, the morning after: + +| machine | generation it last applied | generation the manager gave it | what it did | +|---|---|---|---| +| workstation A | 16 | lower ones, then 17 | ignored three bindings; applied the fourth | +| home server | 11 | 10 | ignored it, kept a token issued before the loss | +| workstation B | 12 | 9 | ignored it, kept a token issued before the loss | +| control node | 12 | 12 | in step, by chance | + +- **A login waited three minutes.** The operator logged in to a second account on workstation A. The + manager adopted the login and moved the machine to it at once (ADR 0209). The machine went on + reporting its login as waiting. The manager adopted the same login again three more times, a vendor + refresh each time, about a minute apart. Only when the count passed 16 did the machine take it. +- **Two machines would have lost the agent when their tokens expired.** Neither had taken the rotation + made since the loss. Each would have stopped working, with nothing anywhere saying why. + +## Why it matters + +The manager counted generations from one again, because its counter is a database sequence and was +made again with the tables. A machine applies a binding only when its generation is greater than the +one it applied last. So a store that is rebuilt, the very event a mesh must survive, silences every +machine that applied a higher number. And the manager cannot see it: it publishes, the publish +succeeds, and the machine discards the binding without a word. + +The generation guards against something that cannot happen. The bindings state keeps only the latest +value per machine, so an older binding cannot arrive after a newer one. diff --git a/04-ISSUES/243-a-rebuilt-licence-store-silences-every-node/01-diagnosis.md b/04-ISSUES/243-a-rebuilt-licence-store-silences-every-node/01-diagnosis.md new file mode 100644 index 0000000..4559625 --- /dev/null +++ b/04-ISSUES/243-a-rebuilt-licence-store-silences-every-node/01-diagnosis.md @@ -0,0 +1,33 @@ +# 243 — Diagnosis + +## 2026-10-05 + +**Trail.** + +1. The operator reported that a login on workstation A was "not going well". +2. The machine's log showed it reporting a login as waiting, four times, while it still held the old + licence's binding at generation 16. +3. The manager's log showed it adopting that login four times, and the store's tables missing from + 02:09 until it adopted the remaining licence again at 02:22. +4. The bindings the manager listed carried lower generations than the ones each machine reported: + 10 against 11 on one machine, 9 against 12 on another. +5. A refresh of the remaining licence, the manager's own verb, gave every binding a generation + above the old numbers. All three of its machines applied it within the second. + +**Located in two places.** + +- **The agent module, on each machine.** It applies a binding only when its generation is greater than + the one applied, a rule that assumes the counter never goes back. It now applies any binding that + differs from the one applied, in licence or generation. The state holds only the latest value per + machine, so nothing older can arrive. Checked by the module's tests: a lower generation after a + rebuild is applied, and an equal one is still skipped. +- **The licence manager.** It numbers bindings from a database sequence, which starts again with a + rebuilt store. Before it considers the machines' reports, it now moves the sequence past every + generation a machine reports having applied, and never moves it back. Without this, a binding could + be given exactly the number a machine already applied, and that one would still be skipped. Checked + by the manager's tests, and the statement was tried on a database: a fresh sequence asked to pass + 16 gives 17 next, and asking it to pass 5 afterwards leaves it where it was. + +**Ruled out.** The bus delivered every binding: each machine that took the fourth binding did so in the +same second it was published. The licences themselves were sound. The remaining licence refreshed +on every attempt, and the second account's login was adopted from its first report.