A consumer's max-deliveries condition has one watcher and counts what was given up (issue 440) #214

Merged
mesh-admin merged 2 commits from fix/440-one-key-one-watcher into main 2026-10-11 01:04:29 +00:00
Contributor

Fixes novox/hq issue 440. Builds on #212 (issue 402).

What was wrong

  1. watchAdvisories and watchDeadLetters could both observe bus.<stream>.<consumer>.max-deliveries, because a max-deliveries advisory without a token lands on the dead-letter row's key. In one Reconcile, the advisory's observation carried Happened and the row's did not. So every look added an observation and an evidence line through the row, and the two summaries overwrote each other. A test of five looks read 6 on main.
  2. Even on its own, the dead-letter row counted looks. A letter held for a day would read "observed 2880 times", the same misreading #212 fixed for advisories.
  3. Observe read Observation.Times on the raise only when Happened was set, but on an update always.

In the live code no token-less max-deliveries advisory is recorded. HearAdvisories drops the server's max-deliveries advisories (issue 330), and both places that record one set Token: not-kept, which is a different key. So point 1 can be reached today only through the watcher's inputs. Point 2 happened for every held dead letter.

Which watcher owns the key, and why

The dead-letter row owns it. Issue 330 decided that a consumer's max-deliveries condition is said from DEAD_LETTERS while it holds a message, and clears when that message is delivered again or dropped, never because the server stopped saying it. The row also says what to do (deliver again or drop). watchAdvisories always skips a max-deliveries advisory without a token, not only while letters are held, because raising that key with nothing held would break issue 330's clearing rule. The not-kept advisory, under its own key, is still said from watchAdvisories.

How the dead-letter row counts

  • The time is when DEAD_LETTERS stored the consumer's newest held letter: the stored time of its last message on its DEAD_LETTERS subject (raw.Time from GetLastMsg). It rises with the stream's sequence, so every letter newly kept is newer than anything the condition has seen.
  • The give-up time does not work. A notice that could not be kept is offered again a minute later (noticeRetry), so it can be kept after a later give-up. That letter would carry a give-up time older than the condition's last observation. Observe would then treat it as not fresh and not count it.
  • The count (Observations) is a running total of the letters given up on while the condition is open. It is at least the number held now and does not go down when a letter is delivered again or dropped, as #212 keeps. Times is the number held, and Observe takes max(Observations+1, Times) for each fresh look.
  • A consumer whose letters vanish between HeldDeadLetters and GetLastMsg is left out of that look, so it is not observed without a time and counted as a look.

What changes

  • link.NewestDeadLetters: for each consumer with held letters, the stored time of the newest one. It makes one request per such consumer per look.
  • The watchdog gathers it into signalFacts.deadLettersNewest. A read error goes into advisoriesErr, as for HeldDeadLetters. A consumer with no time found is dropped from that look.
  • watchDeadLetters sets Happened to that time and Times to the number held. Without a time it counts looks, as a source of the present does.
  • watchAdvisories skips a max-deliveries advisory without a token.
  • Observation.check() refuses Times without Happened, saying why.
  • The S9 row of the signals table test now uses a not-kept advisory, the only max-deliveries advisory S9 still says.

Review rounds

  1. The review of #212 found the issue: two watchers on one key, and Times read differently on the raise and the update. The first commit (f83fcdc) made the dead-letter row the only owner, gave it a time and a count, and made check() refuse Times without Happened. That commit took the time from the letter's give-up header.
  2. The first review of this pull request blocked it. Its link test failed 2 of 20 runs, because the letters were kept in random map order and GetLastMsg returns the newest kept, not the newest given up. The same ordering happens in production through the retry of noticeRetry. The second commit (cfbbad8) uses the stored time instead. It makes the test keep letters in a fixed order, including a later give-up kept first. It adds a test for the skip of a token-less advisory and leaves a vanished consumer out of the look. It also changes the comments to call the count a running total.
  3. The re-review approved cfbbad8. This description was rewritten to match it; no code changed.

What I checked

  • These tests failed before the change they cover:
    • TestAHeldDeadLetterIsCountedByWhatWasGivenUpNotByTheLooks: five looks with one held letter and one token-less advisory read observed 1, with one evidence line and the row's summary on every look. A second letter makes 2. On main it read observed 6.
    • TestTheNewestHeldDeadLetterIsWhenItWasKept: fails with the give-up time, and passes 50 of 50 runs (-count=50).
    • TestAMaxDeliveriesAdvisoryAloneRaisesNothing: fails with the skip removed.
    • TestTimesWithoutWhenItHappenedIsRefused: on main Observe accepted it.
  • TestAMessageThatCouldNotBeKeptIsStillSaidFromTheAdvisory keeps the not-kept key coming from the advisory.
  • #212's tests pass. go vet ./... is clean, and go test ./internal/conditions/ ./internal/link/ ./cmd/mesh-controller/ -count=1 passes.

What I did not verify

  • Not run: make check (the store-backed guards).
  • Not run on a live bus: the cost of GetLastMsg on the real DEAD_LETTERS, and an already-open max-deliveries condition after the roll-out. I expect such a condition to keep its count and stop growing, because the newest stored time is older than its last observation.
  • The gather's handling of a consumer whose letters vanish between the two reads has no test, because the gather needs a live bus.
Fixes novox/hq issue 440. Builds on #212 (issue 402). ## What was wrong 1. `watchAdvisories` and `watchDeadLetters` could both observe `bus.<stream>.<consumer>.max-deliveries`, because a max-deliveries advisory without a token lands on the dead-letter row's key. In one Reconcile, the advisory's observation carried `Happened` and the row's did not. So every look added an observation and an evidence line through the row, and the two summaries overwrote each other. A test of five looks read 6 on main. 2. Even on its own, the dead-letter row counted looks. A letter held for a day would read "observed 2880 times", the same misreading #212 fixed for advisories. 3. `Observe` read `Observation.Times` on the raise only when `Happened` was set, but on an update always. In the live code no token-less max-deliveries advisory is recorded. `HearAdvisories` drops the server's max-deliveries advisories (issue 330), and both places that record one set `Token: not-kept`, which is a different key. So point 1 can be reached today only through the watcher's inputs. Point 2 happened for every held dead letter. ## Which watcher owns the key, and why The dead-letter row owns it. Issue 330 decided that a consumer's max-deliveries condition is said from DEAD_LETTERS while it holds a message, and clears when that message is delivered again or dropped, never because the server stopped saying it. The row also says what to do (deliver again or drop). `watchAdvisories` always skips a max-deliveries advisory without a token, not only while letters are held, because raising that key with nothing held would break issue 330's clearing rule. The `not-kept` advisory, under its own key, is still said from `watchAdvisories`. ## How the dead-letter row counts - **The time** is when DEAD_LETTERS stored the consumer's newest held letter: the stored time of its last message on its DEAD_LETTERS subject (`raw.Time` from `GetLastMsg`). It rises with the stream's sequence, so every letter newly kept is newer than anything the condition has seen. - **The give-up time does not work.** A notice that could not be kept is offered again a minute later (`noticeRetry`), so it can be kept after a later give-up. That letter would carry a give-up time older than the condition's last observation. `Observe` would then treat it as not fresh and not count it. - **The count** (`Observations`) is a running total of the letters given up on while the condition is open. It is at least the number held now and does not go down when a letter is delivered again or dropped, as #212 keeps. `Times` is the number held, and `Observe` takes `max(Observations+1, Times)` for each fresh look. - A consumer whose letters vanish between `HeldDeadLetters` and `GetLastMsg` is left out of that look, so it is not observed without a time and counted as a look. ## What changes - `link.NewestDeadLetters`: for each consumer with held letters, the stored time of the newest one. It makes one request per such consumer per look. - The watchdog gathers it into `signalFacts.deadLettersNewest`. A read error goes into `advisoriesErr`, as for `HeldDeadLetters`. A consumer with no time found is dropped from that look. - `watchDeadLetters` sets `Happened` to that time and `Times` to the number held. Without a time it counts looks, as a source of the present does. - `watchAdvisories` skips a max-deliveries advisory without a token. - `Observation.check()` refuses `Times` without `Happened`, saying why. - The S9 row of the signals table test now uses a `not-kept` advisory, the only max-deliveries advisory S9 still says. ## Review rounds 1. **The review of #212** found the issue: two watchers on one key, and `Times` read differently on the raise and the update. The first commit (f83fcdc) made the dead-letter row the only owner, gave it a time and a count, and made `check()` refuse `Times` without `Happened`. That commit took the time from the letter's give-up header. 2. **The first review of this pull request** blocked it. Its link test failed 2 of 20 runs, because the letters were kept in random map order and `GetLastMsg` returns the newest kept, not the newest given up. The same ordering happens in production through the retry of `noticeRetry`. The second commit (cfbbad8) uses the stored time instead. It makes the test keep letters in a fixed order, including a later give-up kept first. It adds a test for the skip of a token-less advisory and leaves a vanished consumer out of the look. It also changes the comments to call the count a running total. 3. **The re-review** approved cfbbad8. This description was rewritten to match it; no code changed. ## What I checked - These tests failed before the change they cover: - `TestAHeldDeadLetterIsCountedByWhatWasGivenUpNotByTheLooks`: five looks with one held letter and one token-less advisory read observed 1, with one evidence line and the row's summary on every look. A second letter makes 2. On main it read observed 6. - `TestTheNewestHeldDeadLetterIsWhenItWasKept`: fails with the give-up time, and passes 50 of 50 runs (`-count=50`). - `TestAMaxDeliveriesAdvisoryAloneRaisesNothing`: fails with the skip removed. - `TestTimesWithoutWhenItHappenedIsRefused`: on main `Observe` accepted it. - `TestAMessageThatCouldNotBeKeptIsStillSaidFromTheAdvisory` keeps the `not-kept` key coming from the advisory. - #212's tests pass. `go vet ./...` is clean, and `go test ./internal/conditions/ ./internal/link/ ./cmd/mesh-controller/ -count=1` passes. ## What I did not verify - Not run: `make check` (the store-backed guards). - Not run on a live bus: the cost of `GetLastMsg` on the real DEAD_LETTERS, and an already-open max-deliveries condition after the roll-out. I expect such a condition to keep its count and stop growing, because the newest stored time is older than its last observation. - The gather's handling of a consumer whose letters vanish between the two reads has no test, because the gather needs a live bus.
mesh-admin added 1 commit 2026-10-11 00:30:29 +00:00
Give a consumer's max-deliveries key one watcher, counting what was given up (issue 440)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
f83fcdc15f
The dead-letter row and a max-deliveries advisory without a token both said
bus.<stream>.<consumer>.max-deliveries: each look added an observation and a
line of evidence through the row, and the two overwrote each other's words.
The row owns the key since issue 330, so the advisory watcher leaves it, and
the row now says when the newest held message was given up, so a letter held
for a day no longer reads as observed every 30 seconds. Times without Happened
is refused, since the raise ignored it and an update counted it.
Author
Contributor

Delivery novox/mesh-controller@cfbbad8209aa — delivered since 2026-10-11T01:17:45Z

Delivery plan — builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step

  • tier 0: mesh-controller
  • tier 1: build-agent
  • ace: receives build-agent
  • g14: receives build-agent
  • novox: receives mesh-controller, build-agent
  • shanks: receives build-agent
  • build-agent keeps data (ADR 0233): what it holds is backed up before it moves

Machines

  • mesh-controller on novox: passed — healthy 3 times over 3m39s
  • mesh-controller on the rest running it: sent
  • build-agent on ace: passed — healthy 3 times over 3m42s
  • build-agent on the rest running it: sent

Transitions

  • 2026-10-11 00:37 (new) → proposed (announced): novox/mesh-controller#214's head announced
  • 2026-10-11 00:44 proposed → checked (checked): the verdict names this commit
  • 2026-10-11 00:44 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail
  • 2026-10-11 01:06 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move
  • 2026-10-11 01:06 published → delivering (go): its walk started: let go by this owner in its group's order, by a person, or on the controller's own path — not a failed walk that carried a later commit for it, after which it waits for a walk of its own commit
  • 2026-10-11 01:17 delivering → delivered (done): its walk is done, carrying its own commit: every machine of its deploy plan passed, or was left as its policy says

The commit's note under refs/notes/mesh-plan keeps every transition: git log --notes=mesh-plan.

<!-- mesh-delivery:view --> **Delivery** `novox/mesh-controller@cfbbad8209aa` — **delivered** since 2026-10-11T01:17:45Z **Delivery plan** — builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step - tier 0: mesh-controller - tier 1: build-agent - ace: receives build-agent - g14: receives build-agent - novox: receives mesh-controller, build-agent - shanks: receives build-agent - build-agent keeps data (ADR 0233): what it holds is backed up before it moves **Machines** - mesh-controller on novox: passed — healthy 3 times over 3m39s - mesh-controller on the rest running it: sent - build-agent on ace: passed — healthy 3 times over 3m42s - build-agent on the rest running it: sent **Transitions** - 2026-10-11 00:37 (new) → proposed (announced): novox/mesh-controller#214's head announced - 2026-10-11 00:44 proposed → checked (checked): the verdict names this commit - 2026-10-11 00:44 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail - 2026-10-11 01:06 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move - 2026-10-11 01:06 published → delivering (go): its walk started: let go by this owner in its group's order, by a person, or on the controller's own path — not a failed walk that carried a later commit for it, after which it waits for a walk of its own commit - 2026-10-11 01:17 delivering → delivered (done): its walk is done, carrying its own commit: every machine of its deploy plan passed, or was left as its policy says The commit's note under `refs/notes/mesh-plan` keeps every transition: `git log --notes=mesh-plan`.
Author
Contributor

Merge check at f83fcdc1

  • mesh/merge-gate: PASS — every machine composes with the change as it did without (4 of 4 compose) (modules: build-agent, mesh-controller)
  • mesh/repo-check: PASS — its merge-check.sh passed

Delivery plan — builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step

  • tier 0: mesh-controller
  • tier 1: build-agent
  • ace: receives build-agent
  • g14: receives build-agent
  • novox: receives mesh-controller, build-agent
  • shanks: receives build-agent
  • build-agent keeps data (ADR 0233): what it holds is backed up before it moves
  • read by no module's build: cmd/mesh-controller/advisories_counted_test.go, cmd/mesh-controller/signals_test.go, internal/conditions/store_test.go, internal/link/deadletters_test.go

Run by the build seat on ace as build-1791678662009571874 (builds --log build-1791678662009571874).

--- the gate: build-agent, mesh-controller
mesh-controller: ok, 56 tool(s); WARNING: merge-window ask(s) for a setting and do(es) not say whether it is trusted, so it counts as trusted: set at the terminal alone; say "trusted" false where nothing trusts it (novox/hq issue 339); WARNING: controller stay(s) up and say(s) not how it is ready — judged by liveness alone, refused from 2026-11-18 (ADR 0240 rule 8)
long-running resources without health: 1
files asking for a setting without saying whether it is trusted: 1
1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here
judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-11T00:24:56Z says they run
composing every machine as the mesh is
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
composing every machine with the change
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose)
judged by controller development build against the facts of 2026-10-11T00:24:56Z

modules the change moves:
  - mesh-controller (changed)

machines:
  - a machine (f35): composes
  - holds mesh-dns-resolver (idt): composes
  - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes
  - a machine (yvbpsm): composes

a merge would rebuild 2 module(s) in 2 tier(s): build-agent, mesh-controller

notes:
  - cmd/mesh-controller/advisories_counted_test.go, cmd/mesh-controller/signals_test.go, internal/conditions/store_test.go and internal/link/deadletters_test.go read by no module's build, and rebuild nothing (issue 280)
--- its merge-check.sh (go toolchain)
ok  	github.com/novox/mesh-controller/cmd/mesh-builder	1.042s
ok  	github.com/novox/mesh-controller/cmd/mesh-controller	216.752s
ok  	github.com/novox/mesh-controller/examples/objectstore-provisioner	1.018s
ok  	github.com/novox/mesh-controller/examples/postgres-provisioner	1.026s
ok  	github.com/novox/mesh-controller/examples/redis-provisioner	1.017s
ok  	github.com/novox/mesh-controller/examples/route-proxy	1.136s
ok  	github.com/novox/mesh-controller/internal/artifacts	1.057s
ok  	github.com/novox/mesh-controller/internal/broker	2.700s
ok  	github.com/novox/mesh-controller/internal/builder	2.636s
ok  	github.com/novox/mesh-controller/internal/catalogue	4.836s
ok  	github.com/novox/mesh-controller/internal/conditions	2.539s
ok  	github.com/novox/mesh-controller/internal/envfile	1.057s
ok  	github.com/novox/mesh-controller/internal/facts	1.056s
ok  	github.com/novox/mesh-controller/internal/identity	4.405s
ok  	github.com/novox/mesh-controller/internal/inventory	80.835s
ok  	github.com/novox/mesh-controller/internal/lease	17.718s
ok  	github.com/novox/mesh-controller/internal/licences	3.981s
ok  	github.com/novox/mesh-controller/internal/licences/adapters	1.051s
ok  	github.com/novox/mesh-controller/internal/link	52.336s
ok  	github.com/novox/mesh-controller/internal/lint	2.266s
ok  	github.com/novox/mesh-controller/internal/outward	1.081s
ok  	github.com/novox/mesh-controller/internal/overlay	1.069s
ok  	github.com/novox/mesh-controller/internal/secrets	1.416s
ok  	github.com/novox/mesh-controller/internal/store	3.381s
ok  	github.com/novox/mesh-controller/internal/testbus	1.214s
ok  	github.com/novox/mesh-controller/internal/token	1.016s
**Merge check** at `f83fcdc1` - `mesh/merge-gate`: **PASS** — every machine composes with the change as it did without (4 of 4 compose) (modules: build-agent, mesh-controller) - `mesh/repo-check`: **PASS** — its merge-check.sh passed **Delivery plan** — builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step - tier 0: mesh-controller - tier 1: build-agent - ace: receives build-agent - g14: receives build-agent - novox: receives mesh-controller, build-agent - shanks: receives build-agent - build-agent keeps data (ADR 0233): what it holds is backed up before it moves - read by no module's build: cmd/mesh-controller/advisories_counted_test.go, cmd/mesh-controller/signals_test.go, internal/conditions/store_test.go, internal/link/deadletters_test.go Run by the build seat on ace as `build-1791678662009571874` (`builds --log build-1791678662009571874`). ``` --- the gate: build-agent, mesh-controller mesh-controller: ok, 56 tool(s); WARNING: merge-window ask(s) for a setting and do(es) not say whether it is trusted, so it counts as trusted: set at the terminal alone; say "trusted" false where nothing trusts it (novox/hq issue 339); WARNING: controller stay(s) up and say(s) not how it is ready — judged by liveness alone, refused from 2026-11-18 (ADR 0240 rule 8) long-running resources without health: 1 files asking for a setting without saying whether it is trusted: 1 1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-11T00:24:56Z says they run composing every machine as the mesh is the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued composing every machine with the change the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose) judged by controller development build against the facts of 2026-10-11T00:24:56Z modules the change moves: - mesh-controller (changed) machines: - a machine (f35): composes - holds mesh-dns-resolver (idt): composes - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes - a machine (yvbpsm): composes a merge would rebuild 2 module(s) in 2 tier(s): build-agent, mesh-controller notes: - cmd/mesh-controller/advisories_counted_test.go, cmd/mesh-controller/signals_test.go, internal/conditions/store_test.go and internal/link/deadletters_test.go read by no module's build, and rebuild nothing (issue 280) --- its merge-check.sh (go toolchain) ok github.com/novox/mesh-controller/cmd/mesh-builder 1.042s ok github.com/novox/mesh-controller/cmd/mesh-controller 216.752s ok github.com/novox/mesh-controller/examples/objectstore-provisioner 1.018s ok github.com/novox/mesh-controller/examples/postgres-provisioner 1.026s ok github.com/novox/mesh-controller/examples/redis-provisioner 1.017s ok github.com/novox/mesh-controller/examples/route-proxy 1.136s ok github.com/novox/mesh-controller/internal/artifacts 1.057s ok github.com/novox/mesh-controller/internal/broker 2.700s ok github.com/novox/mesh-controller/internal/builder 2.636s ok github.com/novox/mesh-controller/internal/catalogue 4.836s ok github.com/novox/mesh-controller/internal/conditions 2.539s ok github.com/novox/mesh-controller/internal/envfile 1.057s ok github.com/novox/mesh-controller/internal/facts 1.056s ok github.com/novox/mesh-controller/internal/identity 4.405s ok github.com/novox/mesh-controller/internal/inventory 80.835s ok github.com/novox/mesh-controller/internal/lease 17.718s ok github.com/novox/mesh-controller/internal/licences 3.981s ok github.com/novox/mesh-controller/internal/licences/adapters 1.051s ok github.com/novox/mesh-controller/internal/link 52.336s ok github.com/novox/mesh-controller/internal/lint 2.266s ok github.com/novox/mesh-controller/internal/outward 1.081s ok github.com/novox/mesh-controller/internal/overlay 1.069s ok github.com/novox/mesh-controller/internal/secrets 1.416s ok github.com/novox/mesh-controller/internal/store 3.381s ok github.com/novox/mesh-controller/internal/testbus 1.214s ok github.com/novox/mesh-controller/internal/token 1.016s ```
jschoubben added 1 commit 2026-10-11 00:37:00 +00:00
Count a dead letter by when it was kept, so one kept late still counts (issue 440)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
cfbbad8209
The give-up time is not newer for every letter kept: a notice that could
not be kept is offered again a minute later, so a later give-up can be kept
first and the one after it read as not fresh. The stored time rises with the
stream's sequence. A consumer whose letters vanish between the two reads is
left out of the look instead of counted as a look, and the skip of a
token-less max-deliveries advisory now has a test of its own.
Author
Contributor

Pushed cfbbad8, answering the review.

  • Blocker: the newest letter was the newest kept, read by its give-up time. NewestDeadLetters now returns the stored time of the consumer's last DEAD_LETTERS message (raw.Time), which rises with the stream's sequence. The give-up time is not newer for every letter kept: a notice that could not be kept is offered again a minute later (noticeRetry), so a later give-up can be kept first, and the letter kept after it read as not fresh and was not counted. The doc comment says this. The PR description's mention of the Mesh-Dead-Gave-Up header no longer holds.
  • The test is now TestTheNewestHeldDeadLetterIsWhenItWasKept. It keeps letters in a fixed order and includes a 21:04 give-up kept after a 21:05 one, which must read as newer. It fails with the give-up time and passes 50 of 50 runs (-count=50).
  • The skip of a token-less max-deliveries advisory now has a test: TestAMaxDeliveriesAdvisoryAloneRaisesNothing. It fails with the skip removed.
  • Nit: a consumer whose letters vanish between HeldDeadLetters and GetLastMsg is now left out of that look in the gather, so it is no longer observed without a time. This part has no test, because the gather needs a live bus.
  • The comments on Observations and on the dead-letter row now say the count is a running total of messages given up on while the condition is open, not the number held now.

Checked: go vet ./... is clean, and go test ./internal/conditions/ ./internal/link/ ./cmd/mesh-controller/ -count=1 passes.

Pushed cfbbad8, answering the review. - **Blocker: the newest letter was the newest kept, read by its give-up time.** `NewestDeadLetters` now returns the stored time of the consumer's last DEAD_LETTERS message (`raw.Time`), which rises with the stream's sequence. The give-up time is not newer for every letter kept: a notice that could not be kept is offered again a minute later (`noticeRetry`), so a later give-up can be kept first, and the letter kept after it read as not fresh and was not counted. The doc comment says this. The PR description's mention of the `Mesh-Dead-Gave-Up` header no longer holds. - The test is now `TestTheNewestHeldDeadLetterIsWhenItWasKept`. It keeps letters in a fixed order and includes a 21:04 give-up kept after a 21:05 one, which must read as newer. It fails with the give-up time and passes 50 of 50 runs (`-count=50`). - **The skip of a token-less max-deliveries advisory now has a test:** `TestAMaxDeliveriesAdvisoryAloneRaisesNothing`. It fails with the skip removed. - **Nit:** a consumer whose letters vanish between `HeldDeadLetters` and `GetLastMsg` is now left out of that look in the gather, so it is no longer observed without a time. This part has no test, because the gather needs a live bus. - The comments on `Observations` and on the dead-letter row now say the count is a running total of messages given up on while the condition is open, not the number held now. Checked: `go vet ./...` is clean, and `go test ./internal/conditions/ ./internal/link/ ./cmd/mesh-controller/ -count=1` passes.
Author
Contributor

Merge check at cfbbad82

  • mesh/merge-gate: PASS — every machine composes with the change as it did without (4 of 4 compose) (modules: build-agent, mesh-controller)
  • mesh/repo-check: PASS — its merge-check.sh passed

Delivery plan — builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step

  • tier 0: mesh-controller
  • tier 1: build-agent
  • ace: receives build-agent
  • g14: receives build-agent
  • novox: receives mesh-controller, build-agent
  • shanks: receives build-agent
  • build-agent keeps data (ADR 0233): what it holds is backed up before it moves

Run by the build seat on novox as build-1791679026887007693 (builds --log build-1791679026887007693).

--- the gate: build-agent, mesh-controller
mesh-controller: ok, 56 tool(s); WARNING: merge-window ask(s) for a setting and do(es) not say whether it is trusted, so it counts as trusted: set at the terminal alone; say "trusted" false where nothing trusts it (novox/hq issue 339); WARNING: controller stay(s) up and say(s) not how it is ready — judged by liveness alone, refused from 2026-11-18 (ADR 0240 rule 8)
long-running resources without health: 1
files asking for a setting without saying whether it is trusted: 1
1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here
judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-11T00:35:35Z says they run
composing every machine as the mesh is
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
composing every machine with the change
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose)
judged by controller development build against the facts of 2026-10-11T00:35:35Z

modules the change moves:
  - mesh-controller (changed)

machines:
  - a machine (f35): composes
  - holds mesh-dns-resolver (idt): composes
  - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes
  - a machine (yvbpsm): composes

a merge would rebuild 2 module(s) in 2 tier(s): build-agent, mesh-controller
--- its merge-check.sh (go toolchain)
ok  	github.com/novox/mesh-controller/cmd/mesh-builder	1.037s
ok  	github.com/novox/mesh-controller/cmd/mesh-controller	262.221s
ok  	github.com/novox/mesh-controller/examples/objectstore-provisioner	1.026s
ok  	github.com/novox/mesh-controller/examples/postgres-provisioner	1.028s
ok  	github.com/novox/mesh-controller/examples/redis-provisioner	1.026s
ok  	github.com/novox/mesh-controller/examples/route-proxy	1.132s
ok  	github.com/novox/mesh-controller/internal/artifacts	1.066s
ok  	github.com/novox/mesh-controller/internal/broker	2.904s
ok  	github.com/novox/mesh-controller/internal/builder	3.596s
ok  	github.com/novox/mesh-controller/internal/catalogue	5.579s
ok  	github.com/novox/mesh-controller/internal/conditions	2.645s
ok  	github.com/novox/mesh-controller/internal/envfile	1.029s
ok  	github.com/novox/mesh-controller/internal/facts	1.032s
ok  	github.com/novox/mesh-controller/internal/identity	5.138s
ok  	github.com/novox/mesh-controller/internal/inventory	104.036s
ok  	github.com/novox/mesh-controller/internal/lease	17.699s
ok  	github.com/novox/mesh-controller/internal/licences	4.617s
ok  	github.com/novox/mesh-controller/internal/licences/adapters	1.010s
ok  	github.com/novox/mesh-controller/internal/link	55.945s
ok  	github.com/novox/mesh-controller/internal/lint	2.370s
ok  	github.com/novox/mesh-controller/internal/outward	1.037s
ok  	github.com/novox/mesh-controller/internal/overlay	1.022s
ok  	github.com/novox/mesh-controller/internal/secrets	1.481s
ok  	github.com/novox/mesh-controller/internal/store	3.367s
ok  	github.com/novox/mesh-controller/internal/testbus	1.192s
ok  	github.com/novox/mesh-controller/internal/token	1.018s
**Merge check** at `cfbbad82` - `mesh/merge-gate`: **PASS** — every machine composes with the change as it did without (4 of 4 compose) (modules: build-agent, mesh-controller) - `mesh/repo-check`: **PASS** — its merge-check.sh passed **Delivery plan** — builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step - tier 0: mesh-controller - tier 1: build-agent - ace: receives build-agent - g14: receives build-agent - novox: receives mesh-controller, build-agent - shanks: receives build-agent - build-agent keeps data (ADR 0233): what it holds is backed up before it moves Run by the build seat on novox as `build-1791679026887007693` (`builds --log build-1791679026887007693`). ``` --- the gate: build-agent, mesh-controller mesh-controller: ok, 56 tool(s); WARNING: merge-window ask(s) for a setting and do(es) not say whether it is trusted, so it counts as trusted: set at the terminal alone; say "trusted" false where nothing trusts it (novox/hq issue 339); WARNING: controller stay(s) up and say(s) not how it is ready — judged by liveness alone, refused from 2026-11-18 (ADR 0240 rule 8) long-running resources without health: 1 files asking for a setting without saying whether it is trusted: 1 1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-11T00:35:35Z says they run composing every machine as the mesh is the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued composing every machine with the change the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose) judged by controller development build against the facts of 2026-10-11T00:35:35Z modules the change moves: - mesh-controller (changed) machines: - a machine (f35): composes - holds mesh-dns-resolver (idt): composes - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes - a machine (yvbpsm): composes a merge would rebuild 2 module(s) in 2 tier(s): build-agent, mesh-controller --- its merge-check.sh (go toolchain) ok github.com/novox/mesh-controller/cmd/mesh-builder 1.037s ok github.com/novox/mesh-controller/cmd/mesh-controller 262.221s ok github.com/novox/mesh-controller/examples/objectstore-provisioner 1.026s ok github.com/novox/mesh-controller/examples/postgres-provisioner 1.028s ok github.com/novox/mesh-controller/examples/redis-provisioner 1.026s ok github.com/novox/mesh-controller/examples/route-proxy 1.132s ok github.com/novox/mesh-controller/internal/artifacts 1.066s ok github.com/novox/mesh-controller/internal/broker 2.904s ok github.com/novox/mesh-controller/internal/builder 3.596s ok github.com/novox/mesh-controller/internal/catalogue 5.579s ok github.com/novox/mesh-controller/internal/conditions 2.645s ok github.com/novox/mesh-controller/internal/envfile 1.029s ok github.com/novox/mesh-controller/internal/facts 1.032s ok github.com/novox/mesh-controller/internal/identity 5.138s ok github.com/novox/mesh-controller/internal/inventory 104.036s ok github.com/novox/mesh-controller/internal/lease 17.699s ok github.com/novox/mesh-controller/internal/licences 4.617s ok github.com/novox/mesh-controller/internal/licences/adapters 1.010s ok github.com/novox/mesh-controller/internal/link 55.945s ok github.com/novox/mesh-controller/internal/lint 2.370s ok github.com/novox/mesh-controller/internal/outward 1.037s ok github.com/novox/mesh-controller/internal/overlay 1.022s ok github.com/novox/mesh-controller/internal/secrets 1.481s ok github.com/novox/mesh-controller/internal/store 3.367s ok github.com/novox/mesh-controller/internal/testbus 1.192s ok github.com/novox/mesh-controller/internal/token 1.018s ```
mesh-admin merged commit 47c7877e8d into main 2026-10-11 01:04:29 +00:00
mesh-admin deleted branch fix/440-one-key-one-watcher 2026-10-11 01:04:29 +00:00
This repo is archived. You cannot comment on pull requests.
No Reviewers
2 Participants
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: novox/mesh-controller#214