Fixes novox/hq issue 402 (a bus refusal said once reads as refused again every 30 seconds).
What was wrong
On 2026-10-10 the bus refused the controller one publish (mesh.seat.mesh-delivery.tool.times) at 18:52:59 UTC, before its grant reached the bus user list, and never again. Signal S9 (watchAdvisories) reads the one remembered advisory (link.AdvisoryLog, Count 1, Last 18:52:59) again on every look for the hour it is kept, and the condition keeper counted each look as an observation and kept a line of evidence for it. The condition bus.controller.refused said "observed 15 time(s), last 13s ago" at 19:00 UTC, with the same refusal in its evidence every 30 seconds, and was read as a refusal going on.
Captured from the live mesh (conditions history --key bus.controller.refused):
2026-10-10 20:52:59 raised bus.controller.refused — the bus refused the controller: nats: permissions violation: Permissions Violation for Publish to "mesh.seat.mesh-delivery.tool.times"
2026-10-10 21:40:26 cleared bus.controller.refused — S9 no longer observes it
The new test reproduces it on main: 15 looks at that one refusal gave observations 15 and 10 evidence lines 30 seconds apart.
What changes
conditions.Observation gains two optional fields, Happened and Times, for a source that reads a record of what happened rather than looking at the present. When Happened is set, the keeper takes LastObserved and the evidence time from it and observations from Times (never counting down). A look that brings nothing newer than the condition's last observation refreshes the words only and adds no observation or evidence.
S9's advisory observations set Happened to the advisory's Last and Times to its Count.
No new field in a stored condition. The meaning of observations and last-observed is unchanged for every source that does not set Happened.
Which conditions change, which do not
Change: the S9 advisory conditions (bus.controller.refused, bus.controller.slow-consumer, bus.<stream>.<consumer>.consumer-lost, and the max-deliveries not-kept condition). For these, "observed N time(s), last X ago" in conditions key=… now means how many times the bus said it and when it last did, and the evidence has one line per look that saw something new.
Do not change: every other source still counts its looks (S1 and the other watchdogs, probes, health, the dead-letters condition, which says what DEAD_LETTERS holds now). For these, a look that sees the fault is an observation, and words such as "N look(s) in a row" and "raised 6 times" (Count, untouched) depend on that.
Readers: the messenger (mesh-catalog modules/messenger, condition.go) decodes observations and last-observed but does not show them; the issue tracker does not read them. Neither misreads the change, because the fields keep their type and only S9 conditions get different numbers.
What I checked
TestARefusalSaidOnceIsCountedOnceHoweverOftenItIsLookedAt fails on main (observed 15 times, 10 evidence lines) and passes with the fix. It also covers three more refusals arriving between two looks: the count becomes 4, last-observed is the newest refusal, and one evidence line is added.
TestAConditionOfThePresentCountsItsLooks passes before and after: a present-tense condition still counts each look.
go vet and go test -count=1 pass for ./internal/conditions/ and ./cmd/mesh-controller/. I did not run make check (issue 431) and started no store; mesh/repo-check runs the store-backed suite.
What I did not do or verify
The issue also asks that a refusal followed by the grant arriving (the bus user list reloaded) clears the condition or says it was before the grant. That part is not in this pull request: the controller hears no signal that the bus reloaded its user list, so it would need one first. Issue 402 stays open for it, or it becomes a new issue.
Raised is still the time of the first look, not of the refusal, because the gate reads OpenAt from it; for S9 last-observed can therefore be a few seconds before raised.
A controller restarted mid-hour starts a fresh advisory log. Its first look clears the old condition, so the count does not carry across; I have not seen that on the live mesh.
Not verified on the live mesh: that needs a merge, a rollout and a refusal.
Fixes novox/hq issue 402 (a bus refusal said once reads as refused again every 30 seconds).
## What was wrong
On 2026-10-10 the bus refused the controller one publish (`mesh.seat.mesh-delivery.tool.times`) at 18:52:59 UTC, before its grant reached the bus user list, and never again. Signal S9 (`watchAdvisories`) reads the one remembered advisory (`link.AdvisoryLog`, Count 1, Last 18:52:59) again on every look for the hour it is kept, and the condition keeper counted each look as an observation and kept a line of evidence for it. The condition `bus.controller.refused` said "observed 15 time(s), last 13s ago" at 19:00 UTC, with the same refusal in its evidence every 30 seconds, and was read as a refusal going on.
Captured from the live mesh (`conditions history --key bus.controller.refused`):
```
2026-10-10 20:52:59 raised bus.controller.refused — the bus refused the controller: nats: permissions violation: Permissions Violation for Publish to "mesh.seat.mesh-delivery.tool.times"
2026-10-10 21:40:26 cleared bus.controller.refused — S9 no longer observes it
```
The new test reproduces it on main: 15 looks at that one refusal gave `observations` 15 and 10 evidence lines 30 seconds apart.
## What changes
- `conditions.Observation` gains two optional fields, `Happened` and `Times`, for a source that reads a record of what happened rather than looking at the present. When `Happened` is set, the keeper takes `LastObserved` and the evidence time from it and `observations` from `Times` (never counting down). A look that brings nothing newer than the condition's last observation refreshes the words only and adds no observation or evidence.
- S9's advisory observations set `Happened` to the advisory's `Last` and `Times` to its `Count`.
- No new field in a stored condition. The meaning of `observations` and `last-observed` is unchanged for every source that does not set `Happened`.
## Which conditions change, which do not
- Change: the S9 advisory conditions (`bus.controller.refused`, `bus.controller.slow-consumer`, `bus.<stream>.<consumer>.consumer-lost`, and the max-deliveries `not-kept` condition). For these, "observed N time(s), last X ago" in `conditions key=…` now means how many times the bus said it and when it last did, and the evidence has one line per look that saw something new.
- Do not change: every other source still counts its looks (S1 and the other watchdogs, probes, health, the dead-letters condition, which says what DEAD_LETTERS holds now). For these, a look that sees the fault is an observation, and words such as "N look(s) in a row" and "raised 6 times" (Count, untouched) depend on that.
- Readers: the messenger (mesh-catalog modules/messenger, condition.go) decodes `observations` and `last-observed` but does not show them; the issue tracker does not read them. Neither misreads the change, because the fields keep their type and only S9 conditions get different numbers.
## What I checked
- `TestARefusalSaidOnceIsCountedOnceHoweverOftenItIsLookedAt` fails on main (observed 15 times, 10 evidence lines) and passes with the fix. It also covers three more refusals arriving between two looks: the count becomes 4, `last-observed` is the newest refusal, and one evidence line is added.
- `TestAConditionOfThePresentCountsItsLooks` passes before and after: a present-tense condition still counts each look.
- `go vet` and `go test -count=1` pass for `./internal/conditions/` and `./cmd/mesh-controller/`. I did not run `make check` (issue 431) and started no store; `mesh/repo-check` runs the store-backed suite.
## What I did not do or verify
- The issue also asks that a refusal followed by the grant arriving (the bus user list reloaded) clears the condition or says it was before the grant. That part is not in this pull request: the controller hears no signal that the bus reloaded its user list, so it would need one first. Issue 402 stays open for it, or it becomes a new issue.
- `Raised` is still the time of the first look, not of the refusal, because the gate reads `OpenAt` from it; for S9 `last-observed` can therefore be a few seconds before `raised`.
- A controller restarted mid-hour starts a fresh advisory log. Its first look clears the old condition, so the count does not carry across; I have not seen that on the live mesh.
- Not verified on the live mesh: that needs a merge, a rollout and a refusal.
S9 reads the one remembered advisory again on every look for an hour, and
each look was kept as an observation and a line of evidence, so a single
refusal read as one repeated every 30 seconds. An observation may now say
when what it reads happened and how often; the keeper counts that, and a
look with nothing newer adds nothing. Sources that look at the present
keep counting their looks.
Deliverynovox/mesh-controller@c11222026a3b — delivered since 2026-10-11T00:27:30Z
Delivery plan — builds mesh-controller → novox; no bus step
tier 0: mesh-controller
novox: receives mesh-controller
Machines
mesh-controller on novox: passed — healthy 3 times over 3m39s
mesh-controller on the rest running it: sent
Transitions
2026-10-11 00:11 (new) → proposed (announced): novox/mesh-controller#212's head announced
2026-10-11 00:18 proposed → checked (checked): the verdict names this commit
2026-10-11 00:18 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail
2026-10-11 00:22 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move
2026-10-11 00:22 published → delivering (go): its walk started: let go by this owner in its group's order, by a person, or on the controller's own path — not a failed walk that carried a later commit for it, after which it waits for a walk of its own commit
2026-10-11 00:27 delivering → delivered (done): its walk is done, carrying its own commit: every machine of its deploy plan passed, or was left as its policy says
The commit's note under refs/notes/mesh-plan keeps every transition: git log --notes=mesh-plan.
<!-- mesh-delivery:view -->
**Delivery** `novox/mesh-controller@c11222026a3b` — **delivered** since 2026-10-11T00:27:30Z
**Delivery plan** — builds mesh-controller → novox; no bus step
- tier 0: mesh-controller
- novox: receives mesh-controller
**Machines**
- mesh-controller on novox: passed — healthy 3 times over 3m39s
- mesh-controller on the rest running it: sent
**Transitions**
- 2026-10-11 00:11 (new) → proposed (announced): novox/mesh-controller#212's head announced
- 2026-10-11 00:18 proposed → checked (checked): the verdict names this commit
- 2026-10-11 00:18 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail
- 2026-10-11 00:22 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move
- 2026-10-11 00:22 published → delivering (go): its walk started: let go by this owner in its group's order, by a person, or on the controller's own path — not a failed walk that carried a later commit for it, after which it waits for a walk of its own commit
- 2026-10-11 00:27 delivering → delivered (done): its walk is done, carrying its own commit: every machine of its deploy plan passed, or was left as its policy says
The commit's note under `refs/notes/mesh-plan` keeps every transition: `git log --notes=mesh-plan`.
mesh/merge-gate: PASS — every machine composes with the change as it did without (4 of 4 compose) (modules: mesh-controller)
mesh/repo-check: PASS — its merge-check.sh passed
Delivery plan — builds mesh-controller → novox; no bus step
tier 0: mesh-controller
novox: receives mesh-controller
read by no module's build: cmd/mesh-controller/advisories_counted_test.go
Run by the build seat on g14 as build-1791677499273367041 (builds --log build-1791677499273367041).
--- the gate: mesh-controller
mesh-controller: ok, 56 tool(s); WARNING: merge-window ask(s) for a setting and do(es) not say whether it is trusted, so it counts as trusted: set at the terminal alone; say "trusted" false where nothing trusts it (novox/hq issue 339); WARNING: controller stay(s) up and say(s) not how it is ready — judged by liveness alone, refused from 2026-11-18 (ADR 0240 rule 8)
long-running resources without health: 1
files asking for a setting without saying whether it is trusted: 1
1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here
judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-11T00:04:44Z says they run
composing every machine as the mesh is
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
composing every machine with the change
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose)
judged by controller development build against the facts of 2026-10-11T00:04:44Z
modules the change moves:
- mesh-controller (changed)
machines:
- a machine (f35): composes
- holds mesh-dns-resolver (idt): composes
- holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes
- a machine (yvbpsm): composes
a merge would rebuild 1 module(s) in 1 tier(s): mesh-controller
notes:
- cmd/mesh-controller/advisories_counted_test.go read by no module's build, and rebuild nothing (issue 280)
--- its merge-check.sh (go toolchain)
ok github.com/novox/mesh-controller/cmd/mesh-builder 1.037s
ok github.com/novox/mesh-controller/cmd/mesh-controller 222.837s
ok github.com/novox/mesh-controller/examples/objectstore-provisioner 1.009s
ok github.com/novox/mesh-controller/examples/postgres-provisioner 1.015s
ok github.com/novox/mesh-controller/examples/redis-provisioner 1.009s
ok github.com/novox/mesh-controller/examples/route-proxy 1.178s
ok github.com/novox/mesh-controller/internal/artifacts 1.041s
ok github.com/novox/mesh-controller/internal/broker 2.597s
ok github.com/novox/mesh-controller/internal/builder 2.423s
ok github.com/novox/mesh-controller/internal/catalogue 4.377s
ok github.com/novox/mesh-controller/internal/conditions 2.356s
ok github.com/novox/mesh-controller/internal/envfile 1.025s
ok github.com/novox/mesh-controller/internal/facts 1.030s
ok github.com/novox/mesh-controller/internal/identity 4.045s
ok github.com/novox/mesh-controller/internal/inventory 76.714s
ok github.com/novox/mesh-controller/internal/lease 17.695s
ok github.com/novox/mesh-controller/internal/licences 3.624s
ok github.com/novox/mesh-controller/internal/licences/adapters 1.007s
ok github.com/novox/mesh-controller/internal/link 51.689s
ok github.com/novox/mesh-controller/internal/lint 1.983s
ok github.com/novox/mesh-controller/internal/outward 1.033s
ok github.com/novox/mesh-controller/internal/overlay 1.025s
ok github.com/novox/mesh-controller/internal/secrets 1.410s
ok github.com/novox/mesh-controller/internal/store 3.144s
ok github.com/novox/mesh-controller/internal/testbus 1.158s
ok github.com/novox/mesh-controller/internal/token 1.011s
**Merge check** at `c1122202`
- `mesh/merge-gate`: **PASS** — every machine composes with the change as it did without (4 of 4 compose) (modules: mesh-controller)
- `mesh/repo-check`: **PASS** — its merge-check.sh passed
**Delivery plan** — builds mesh-controller → novox; no bus step
- tier 0: mesh-controller
- novox: receives mesh-controller
- read by no module's build: cmd/mesh-controller/advisories_counted_test.go
Run by the build seat on g14 as `build-1791677499273367041` (`builds --log build-1791677499273367041`).
```
--- the gate: mesh-controller
mesh-controller: ok, 56 tool(s); WARNING: merge-window ask(s) for a setting and do(es) not say whether it is trusted, so it counts as trusted: set at the terminal alone; say "trusted" false where nothing trusts it (novox/hq issue 339); WARNING: controller stay(s) up and say(s) not how it is ready — judged by liveness alone, refused from 2026-11-18 (ADR 0240 rule 8)
long-running resources without health: 1
files asking for a setting without saying whether it is trusted: 1
1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here
judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-11T00:04:44Z says they run
composing every machine as the mesh is
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
composing every machine with the change
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose)
judged by controller development build against the facts of 2026-10-11T00:04:44Z
modules the change moves:
- mesh-controller (changed)
machines:
- a machine (f35): composes
- holds mesh-dns-resolver (idt): composes
- holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes
- a machine (yvbpsm): composes
a merge would rebuild 1 module(s) in 1 tier(s): mesh-controller
notes:
- cmd/mesh-controller/advisories_counted_test.go read by no module's build, and rebuild nothing (issue 280)
--- its merge-check.sh (go toolchain)
ok github.com/novox/mesh-controller/cmd/mesh-builder 1.037s
ok github.com/novox/mesh-controller/cmd/mesh-controller 222.837s
ok github.com/novox/mesh-controller/examples/objectstore-provisioner 1.009s
ok github.com/novox/mesh-controller/examples/postgres-provisioner 1.015s
ok github.com/novox/mesh-controller/examples/redis-provisioner 1.009s
ok github.com/novox/mesh-controller/examples/route-proxy 1.178s
ok github.com/novox/mesh-controller/internal/artifacts 1.041s
ok github.com/novox/mesh-controller/internal/broker 2.597s
ok github.com/novox/mesh-controller/internal/builder 2.423s
ok github.com/novox/mesh-controller/internal/catalogue 4.377s
ok github.com/novox/mesh-controller/internal/conditions 2.356s
ok github.com/novox/mesh-controller/internal/envfile 1.025s
ok github.com/novox/mesh-controller/internal/facts 1.030s
ok github.com/novox/mesh-controller/internal/identity 4.045s
ok github.com/novox/mesh-controller/internal/inventory 76.714s
ok github.com/novox/mesh-controller/internal/lease 17.695s
ok github.com/novox/mesh-controller/internal/licences 3.624s
ok github.com/novox/mesh-controller/internal/licences/adapters 1.007s
ok github.com/novox/mesh-controller/internal/link 51.689s
ok github.com/novox/mesh-controller/internal/lint 1.983s
ok github.com/novox/mesh-controller/internal/outward 1.033s
ok github.com/novox/mesh-controller/internal/overlay 1.025s
ok github.com/novox/mesh-controller/internal/secrets 1.410s
ok github.com/novox/mesh-controller/internal/store 3.144s
ok github.com/novox/mesh-controller/internal/testbus 1.158s
ok github.com/novox/mesh-controller/internal/token 1.011s
```
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Fixes novox/hq issue 402 (a bus refusal said once reads as refused again every 30 seconds).
What was wrong
On 2026-10-10 the bus refused the controller one publish (
mesh.seat.mesh-delivery.tool.times) at 18:52:59 UTC, before its grant reached the bus user list, and never again. Signal S9 (watchAdvisories) reads the one remembered advisory (link.AdvisoryLog, Count 1, Last 18:52:59) again on every look for the hour it is kept, and the condition keeper counted each look as an observation and kept a line of evidence for it. The conditionbus.controller.refusedsaid "observed 15 time(s), last 13s ago" at 19:00 UTC, with the same refusal in its evidence every 30 seconds, and was read as a refusal going on.Captured from the live mesh (
conditions history --key bus.controller.refused):The new test reproduces it on main: 15 looks at that one refusal gave
observations15 and 10 evidence lines 30 seconds apart.What changes
conditions.Observationgains two optional fields,HappenedandTimes, for a source that reads a record of what happened rather than looking at the present. WhenHappenedis set, the keeper takesLastObservedand the evidence time from it andobservationsfromTimes(never counting down). A look that brings nothing newer than the condition's last observation refreshes the words only and adds no observation or evidence.Happenedto the advisory'sLastandTimesto itsCount.observationsandlast-observedis unchanged for every source that does not setHappened.Which conditions change, which do not
bus.controller.refused,bus.controller.slow-consumer,bus.<stream>.<consumer>.consumer-lost, and the max-deliveriesnot-keptcondition). For these, "observed N time(s), last X ago" inconditions key=…now means how many times the bus said it and when it last did, and the evidence has one line per look that saw something new.observationsandlast-observedbut does not show them; the issue tracker does not read them. Neither misreads the change, because the fields keep their type and only S9 conditions get different numbers.What I checked
TestARefusalSaidOnceIsCountedOnceHoweverOftenItIsLookedAtfails on main (observed 15 times, 10 evidence lines) and passes with the fix. It also covers three more refusals arriving between two looks: the count becomes 4,last-observedis the newest refusal, and one evidence line is added.TestAConditionOfThePresentCountsItsLookspasses before and after: a present-tense condition still counts each look.go vetandgo test -count=1pass for./internal/conditions/and./cmd/mesh-controller/. I did not runmake check(issue 431) and started no store;mesh/repo-checkruns the store-backed suite.What I did not do or verify
Raisedis still the time of the first look, not of the refusal, because the gate readsOpenAtfrom it; for S9last-observedcan therefore be a few seconds beforeraised.Delivery
novox/mesh-controller@c11222026a3b— delivered since 2026-10-11T00:27:30ZDelivery plan — builds mesh-controller → novox; no bus step
Machines
Transitions
The commit's note under
refs/notes/mesh-plankeeps every transition:git log --notes=mesh-plan.Merge check at
c1122202mesh/merge-gate: PASS — every machine composes with the change as it did without (4 of 4 compose) (modules: mesh-controller)mesh/repo-check: PASS — its merge-check.sh passedDelivery plan — builds mesh-controller → novox; no bus step
Run by the build seat on g14 as
build-1791677499273367041(builds --log build-1791677499273367041).