Fixes issue 420 (the conditions bucket is rewritten on every observation).
Why. Every look rewrote a condition's value in mesh-controller_conditions, because the value carried the count of observations, the last observed time and the newest evidence. Measured on 2026-10-10: 131 647 writes in four days for twelve open conditions (mesh.hand-acts.disk-load.healer-wanted alone 7 003). That breaks novox/hq ADR 0289 §1 (a bucket is written only when what it describes changed) and §4 (no idle-changing fields). The decision on the issue (2026-10-11, shanks session): the bucket holds what changes on a transition; the looks live in the controller's store and the verb and the events still carry them.
What changes
internal/conditions/looks.go: Looks (observations, last-observed, evidence) and a LookStore. BucketValue writes a condition without its looks, keys in one order.
Keeper.Observe compares the new bucket value with the stored bytes and writes only when they differ; a fresh look is kept in the LookStore after the bucket write (so a lost compare-and-set does not double-count). Clear forgets the looks. Every read (Open, Get, Read/ReadWith, ReadOne/ReadOneWith) and every event puts the looks back.
A condition written before this (its value still carries the looks) keeps its count: the first look counts on from the value and rewrites it once without them.
Looks that cannot be read or kept never stop a raise (a store going away is exactly what must still be raised): the keeper says it, does not write over the count, and the condition carries looks-unread with why, never a count of none.
internal/inventory: migration 0095condition_looks (key, observations, last_observed, evidence jsonb, epoch, kept_at); written under the lease (actingEpoch), forgotten on a clear. There is no consolidated schema in this repository; migrations are the schema.
cmd/mesh-controller/condition_looks.go: the store adapter. The serving controller and doctor run locally use it; the conditions verb, which runs as a process of its own, opens the store to read the looks (looksHere), or says them unread.
Writers table line says where the looks are kept.
Readers. The messenger reads the conditions verb answer and the condition events, not the bucket (checked in mesh-catalog main); both still carry observations, last-observed and evidence. No other reader of the bucket was found. The hq side: novox/hq branch fix/420-condition-looks-beside-the-bucket amends to-be 45 §2's field table.
Tests (each fails without the fix: replacing the compare with an unconditional write fails all three)
TestALookThatChangesNothingWritesNothingToTheBucket: five looks write the bucket once; the verb reads 5 observations and 5 evidence lines; the bucket's value holds no look field; a severity change writes once more and its event carries the looks; the clearing forgets them.
TestAConditionWrittenWithItsLooksMovesThemOutOnce: an old-shaped value with 7 003 observations is rewritten once and counts on to 7 006.
TestLooksThatCannotBeKeptAreSaidNotCountedAsNone: with the store away the condition is still raised and says looks-unread.
TestTheConditionsVerbReadsTheLooksTheBucketNoLongerHolds (store + real bus): four looks leave the bucket stream at sequence 1; conditions show and the list, as a process other than the serving controller, read 4 observations from the store.
TestAConditionsLooksAreKeptBesideTheBucket (store): kept under the lease only, replaced, listed, forgotten.
Two existing tests adjusted to the new rule: the race test now races a write that changes severity (an unchanged look no longer writes, so there is nothing to race); the bus test uses a wrong revision rather than revision-1, which is 0 now that the second look writes nothing.
Both store tests are listed in testdata/store-tests. sh merge-check.sh passed locally with a throwaway PostgreSQL 18.6, race on, every listed store test PASS.
Reviewers' checklist
A look that changes nothing makes no bucket write (bytes.Equal in Observe), and a lost compare-and-set does not count a look twice (looks kept after the write).
Every path that writes the bucket uses BucketValue (raise, observe, amend/tried/escalate, silence, end of silence).
The events and the verb still carry observations, last-observed and evidence; nothing reads them from the bucket.
A looks store away never blocks a raise and never reads as zero.
Migration 0095 is the next number on main; no other open PR here adds one (#247, #246, #203, #202, #188, #182 checked).
Overlap. Does not touch any file #247 changes (walk_said.go, walksaid.go, plans.go, delivery.go, release_plan.go). A controller rollout: merge only while mesh-controller.plans says no plan is mid-gate, and not while #247 is mid-gate.
By fix-420@g14. Not merged; waits for review.
Fixes issue 420 (the conditions bucket is rewritten on every observation).
**Why.** Every look rewrote a condition's value in `mesh-controller_conditions`, because the value carried the count of observations, the last observed time and the newest evidence. Measured on 2026-10-10: 131 647 writes in four days for twelve open conditions (`mesh.hand-acts.disk-load.healer-wanted` alone 7 003). That breaks novox/hq ADR 0289 §1 (a bucket is written only when what it describes changed) and §4 (no idle-changing fields). The decision on the issue (2026-10-11, shanks session): the bucket holds what changes on a transition; the looks live in the controller's store and the verb and the events still carry them.
**What changes**
- `internal/conditions/looks.go`: `Looks` (observations, last-observed, evidence) and a `LookStore`. `BucketValue` writes a condition without its looks, keys in one order.
- `Keeper.Observe` compares the new bucket value with the stored bytes and **writes only when they differ**; a fresh look is kept in the `LookStore` after the bucket write (so a lost compare-and-set does not double-count). Clear forgets the looks. Every read (`Open`, `Get`, `Read`/`ReadWith`, `ReadOne`/`ReadOneWith`) and every event puts the looks back.
- A condition written before this (its value still carries the looks) keeps its count: the first look counts on from the value and rewrites it once without them.
- Looks that cannot be read or kept never stop a raise (a store going away is exactly what must still be raised): the keeper says it, does not write over the count, and the condition carries `looks-unread` with why, never a count of none.
- `internal/inventory`: migration **0095** `condition_looks` (key, observations, last_observed, evidence jsonb, epoch, kept_at); written under the lease (`actingEpoch`), forgotten on a clear. There is no consolidated schema in this repository; migrations are the schema.
- `cmd/mesh-controller/condition_looks.go`: the store adapter. The serving controller and `doctor` run locally use it; the `conditions` verb, which runs as a process of its own, opens the store to read the looks (`looksHere`), or says them unread.
- Writers table line says where the looks are kept.
**Readers.** The messenger reads the `conditions` verb answer and the condition events, not the bucket (checked in mesh-catalog main); both still carry `observations`, `last-observed` and `evidence`. No other reader of the bucket was found. The hq side: novox/hq branch `fix/420-condition-looks-beside-the-bucket` amends to-be 45 §2's field table.
**Tests** (each fails without the fix: replacing the compare with an unconditional write fails all three)
- `TestALookThatChangesNothingWritesNothingToTheBucket`: five looks write the bucket once; the verb reads 5 observations and 5 evidence lines; the bucket's value holds no look field; a severity change writes once more and its event carries the looks; the clearing forgets them.
- `TestAConditionWrittenWithItsLooksMovesThemOutOnce`: an old-shaped value with 7 003 observations is rewritten once and counts on to 7 006.
- `TestLooksThatCannotBeKeptAreSaidNotCountedAsNone`: with the store away the condition is still raised and says `looks-unread`.
- `TestTheConditionsVerbReadsTheLooksTheBucketNoLongerHolds` (store + real bus): four looks leave the bucket stream at sequence 1; `conditions show` and the list, as a process other than the serving controller, read 4 observations from the store.
- `TestAConditionsLooksAreKeptBesideTheBucket` (store): kept under the lease only, replaced, listed, forgotten.
- Two existing tests adjusted to the new rule: the race test now races a write that changes severity (an unchanged look no longer writes, so there is nothing to race); the bus test uses a wrong revision rather than revision-1, which is 0 now that the second look writes nothing.
- Both store tests are listed in `testdata/store-tests`. `sh merge-check.sh` passed locally with a throwaway PostgreSQL 18.6, race on, every listed store test PASS.
**Reviewers' checklist**
- [ ] A look that changes nothing makes no bucket write (`bytes.Equal` in `Observe`), and a lost compare-and-set does not count a look twice (looks kept after the write).
- [ ] Every path that writes the bucket uses `BucketValue` (raise, observe, amend/tried/escalate, silence, end of silence).
- [ ] The events and the verb still carry observations, last-observed and evidence; nothing reads them from the bucket.
- [ ] A looks store away never blocks a raise and never reads as zero.
- [ ] Migration 0095 is the next number on main; no other open PR here adds one (#247, #246, #203, #202, #188, #182 checked).
**Overlap.** Does not touch any file #247 changes (`walk_said.go`, `walksaid.go`, `plans.go`, `delivery.go`, `release_plan.go`). A controller rollout: merge only while `mesh-controller.plans` says no plan is mid-gate, and not while #247 is mid-gate.
By fix-420@g14. Not merged; waits for review.
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
Every look rewrote the condition's value in the conditions bucket, because the
value carried the count of looks, the last look and the newest evidence: 131 647
writes in four days for twelve open conditions, against ADR 0289's rule that a
bucket is written only when what it describes changed. The looks now live in the
controller's store (migration 0095, condition_looks), the bucket holds what a
condition says and is written only when that changes, and the conditions verb and
the condition events put the looks back.
build-agent keeps data (ADR 0233): what it holds is backed up before it moves
route-proxy provides route: its consumers are sent again after it
route-proxy keeps data (ADR 0233): what it holds is backed up before it moves
grants step: if this changes what the bus's user list says (a new tool, subject or user), novox is sent that list alone, every build there kept, before build-agent, route-proxy is judged on its first machine
Machines
mesh-controller on novox: passed — healthy 3 times over 3m39s
mesh-controller on the rest running it: sent
build-agent on ace: passed — healthy 1 times over 1m33s
build-agent on the rest running it: sent
route-proxy on ace: passed — healthy 1 times over 1m39s
route-proxy on the rest running it: sent
Transitions
2026-10-12 10:12 (new) → proposed (announced): novox/mesh-controller#249's head announced
2026-10-12 10:18 proposed → checked (checked): the verdict names this commit
2026-10-12 10:18 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail
2026-10-12 10:29 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move
2026-10-12 10:29 published → delivering (go): its walk started: let go by this owner in its group's order, by a person, or on the controller's own path — not a failed walk that carried a later commit for it, after which it waits for a walk of its own commit
2026-10-12 10:41 delivering → delivered (done): its walk is done, carrying its own commit: every machine of its deploy plan passed, or was left as its policy says
The commit's note under refs/notes/mesh-plan keeps every transition: git log --notes=mesh-plan.
<!-- mesh-delivery:view -->
**Delivery** `novox/mesh-controller@dcdafd37f008` — **delivered** since 2026-10-12T10:41:33Z
**Delivery plan** — builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step
- tier 0: mesh-controller
- tier 1: build-agent
- tier 2: route-proxy
- ace: receives build-agent, route-proxy
- g14: receives build-agent
- novox: receives mesh-controller, build-agent, route-proxy
- shanks: receives build-agent
- build-agent keeps data (ADR 0233): what it holds is backed up before it moves
- route-proxy provides route: its consumers are sent again after it
- route-proxy keeps data (ADR 0233): what it holds is backed up before it moves
- grants step: if this changes what the bus's user list says (a new tool, subject or user), novox is sent that list alone, every build there kept, before build-agent, route-proxy is judged on its first machine
**Machines**
- mesh-controller on novox: passed — healthy 3 times over 3m39s
- mesh-controller on the rest running it: sent
- build-agent on ace: passed — healthy 1 times over 1m33s
- build-agent on the rest running it: sent
- route-proxy on ace: passed — healthy 1 times over 1m39s
- route-proxy on the rest running it: sent
**Transitions**
- 2026-10-12 10:12 (new) → proposed (announced): novox/mesh-controller#249's head announced
- 2026-10-12 10:18 proposed → checked (checked): the verdict names this commit
- 2026-10-12 10:18 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail
- 2026-10-12 10:29 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move
- 2026-10-12 10:29 published → delivering (go): its walk started: let go by this owner in its group's order, by a person, or on the controller's own path — not a failed walk that carried a later commit for it, after which it waits for a walk of its own commit
- 2026-10-12 10:41 delivering → delivered (done): its walk is done, carrying its own commit: every machine of its deploy plan passed, or was left as its policy says
The commit's note under `refs/notes/mesh-plan` keeps every transition: `git log --notes=mesh-plan`.
build-agent keeps data (ADR 0233): what it holds is backed up before it moves
route-proxy provides route: its consumers are sent again after it
route-proxy keeps data (ADR 0233): what it holds is backed up before it moves
grants step: if this changes what the bus's user list says (a new tool, subject or user), novox is sent that list alone, every build there kept, before build-agent, route-proxy is judged on its first machine
read by no module's build: cmd/mesh-controller/condition_looks_test.go, cmd/mesh-controller/replay327_test.go, internal/conditions/bus_test.go, internal/conditions/looks_test.go, internal/conditions/store_test.go, internal/inventory/looks_test.go, testdata/store-tests
Run by the build seat on ace as build-1791799936294115889 (builds --log build-1791799936294115889).
--- the gate: build-agent, mesh-controller, route-proxy
mesh-controller: ok, 57 tool(s); WARNING: merge-window ask(s) for a setting and do(es) not say whether it is trusted, so it counts as trusted: set at the terminal alone; say "trusted" false where nothing trusts it (novox/hq issue 339); WARNING: controller stay(s) up and say(s) not how it is ready — judged by liveness alone, refused from 2026-11-18 (ADR 0240 rule 8)
long-running resources without health: 1
files asking for a setting without saying whether it is trusted: 1
1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here
judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-12T10:08:43Z says they run
composing every machine as the mesh is
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
composing every machine with the change
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose)
judged by controller development build against the facts of 2026-10-12T10:08:43Z
modules the change moves:
- mesh-controller (changed)
machines:
- a machine (f35): composes
- holds mesh-dns-resolver (idt): composes
- holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes
- a machine (yvbpsm): composes
a merge would rebuild 3 module(s) in 3 tier(s): build-agent, mesh-controller, route-proxy
notes:
- cmd/mesh-controller/condition_looks_test.go, cmd/mesh-controller/replay327_test.go, internal/conditions/bus_test.go, internal/conditions/looks_test.go, internal/conditions/store_test.go, internal/inventory/looks_test.go and testdata/store-tests read by no module's build, and rebuild nothing (issue 280)
--- its merge-check.sh (go toolchain)
ok github.com/novox/mesh-controller/cmd/mesh-builder 1.038s
ok github.com/novox/mesh-controller/cmd/mesh-controller 225.308s
ok github.com/novox/mesh-controller/examples/objectstore-provisioner 1.025s
ok github.com/novox/mesh-controller/examples/postgres-provisioner 1.034s
ok github.com/novox/mesh-controller/examples/redis-provisioner 1.028s
ok github.com/novox/mesh-controller/examples/route-proxy 1.143s
ok github.com/novox/mesh-controller/internal/artifacts 1.071s
? github.com/novox/mesh-controller/internal/beside [no test files]
ok github.com/novox/mesh-controller/internal/broker 4.133s
ok github.com/novox/mesh-controller/internal/builder 16.061s
ok github.com/novox/mesh-controller/internal/catalogue 7.809s
ok github.com/novox/mesh-controller/internal/conditions 2.661s
ok github.com/novox/mesh-controller/internal/envfile 1.071s
ok github.com/novox/mesh-controller/internal/facts 1.069s
ok github.com/novox/mesh-controller/internal/identity 4.682s
ok github.com/novox/mesh-controller/internal/inventory 87.982s
ok github.com/novox/mesh-controller/internal/lease 17.735s
ok github.com/novox/mesh-controller/internal/licences 4.287s
ok github.com/novox/mesh-controller/internal/licences/adapters 1.087s
ok github.com/novox/mesh-controller/internal/link 59.428s
ok github.com/novox/mesh-controller/internal/lint 2.355s
ok github.com/novox/mesh-controller/internal/outward 1.090s
ok github.com/novox/mesh-controller/internal/overlay 1.081s
ok github.com/novox/mesh-controller/internal/secrets 1.486s
ok github.com/novox/mesh-controller/internal/store 3.502s
ok github.com/novox/mesh-controller/internal/testbus 1.257s
ok github.com/novox/mesh-controller/internal/token 1.020s
store tests, each run against the store:
./cmd/mesh-controller --- PASS: TestAStopWithoutAMergeWalksTheOthersAgain (0.39s)
./cmd/mesh-controller --- PASS: TestAStopWithoutAMergeEndsTheMergesThatContainIt (0.45s)
./cmd/mesh-controller --- PASS: TestAStopWalksTheOthersAgainInTheOpenBatch (0.37s)
./cmd/mesh-controller --- PASS: TestAStopNamesWhatStaysOnTheMachinesAndWhatStillCarriesIt (0.40s)
./cmd/mesh-controller --- PASS: TestAStopWithoutAMergeLeavesItOutOfABatch (0.38s)
./cmd/mesh-controller --- PASS: TestSavingPlansAndMovingMergesIsOneAct (0.36s)
./cmd/mesh-controller --- PASS: TestNodeShowAndStatusReadTheLastSendFromTheStore (0.41s)
./cmd/mesh-controller --- PASS: TestTheConditionsVerbReadsTheLooksTheBucketNoLongerHolds (0.28s)
./internal/inventory --- PASS: TestTheLastSendSaysWhatTheMachineAnswered (0.25s)
./internal/inventory --- PASS: TestAReportIsTheLastSendsAnswerOnlyWhenItIsAboutThatSend (0.24s)
./internal/inventory --- PASS: TestAnIssuedValueIsSealedToEveryHolderOnceAndNotAgain (0.25s)
./internal/inventory --- PASS: TestAnIssuedValueIsSealedAgainAfterAKeyChangeOrAnAcceptance (0.25s)
./internal/inventory --- PASS: TestAConditionsLooksAreKeptBesideTheBucket (0.25s)
**Merge check** at `dcdafd37`
- `mesh/merge-gate`: **PASS** — every machine composes with the change as it did without (4 of 4 compose) (modules: build-agent, mesh-controller, route-proxy)
- `mesh/repo-check`: **PASS** — its merge-check.sh passed
**Delivery plan** — builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step
- tier 0: mesh-controller
- tier 1: build-agent
- tier 2: route-proxy
- ace: receives build-agent, route-proxy
- g14: receives build-agent
- novox: receives mesh-controller, build-agent, route-proxy
- shanks: receives build-agent
- build-agent keeps data (ADR 0233): what it holds is backed up before it moves
- route-proxy provides route: its consumers are sent again after it
- route-proxy keeps data (ADR 0233): what it holds is backed up before it moves
- grants step: if this changes what the bus's user list says (a new tool, subject or user), novox is sent that list alone, every build there kept, before build-agent, route-proxy is judged on its first machine
- read by no module's build: cmd/mesh-controller/condition_looks_test.go, cmd/mesh-controller/replay327_test.go, internal/conditions/bus_test.go, internal/conditions/looks_test.go, internal/conditions/store_test.go, internal/inventory/looks_test.go, testdata/store-tests
Run by the build seat on ace as `build-1791799936294115889` (`builds --log build-1791799936294115889`).
```
--- the gate: build-agent, mesh-controller, route-proxy
mesh-controller: ok, 57 tool(s); WARNING: merge-window ask(s) for a setting and do(es) not say whether it is trusted, so it counts as trusted: set at the terminal alone; say "trusted" false where nothing trusts it (novox/hq issue 339); WARNING: controller stay(s) up and say(s) not how it is ready — judged by liveness alone, refused from 2026-11-18 (ADR 0240 rule 8)
long-running resources without health: 1
files asking for a setting without saying whether it is trusted: 1
1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here
judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-12T10:08:43Z says they run
composing every machine as the mesh is
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
composing every machine with the change
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose)
judged by controller development build against the facts of 2026-10-12T10:08:43Z
modules the change moves:
- mesh-controller (changed)
machines:
- a machine (f35): composes
- holds mesh-dns-resolver (idt): composes
- holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes
- a machine (yvbpsm): composes
a merge would rebuild 3 module(s) in 3 tier(s): build-agent, mesh-controller, route-proxy
notes:
- cmd/mesh-controller/condition_looks_test.go, cmd/mesh-controller/replay327_test.go, internal/conditions/bus_test.go, internal/conditions/looks_test.go, internal/conditions/store_test.go, internal/inventory/looks_test.go and testdata/store-tests read by no module's build, and rebuild nothing (issue 280)
--- its merge-check.sh (go toolchain)
ok github.com/novox/mesh-controller/cmd/mesh-builder 1.038s
ok github.com/novox/mesh-controller/cmd/mesh-controller 225.308s
ok github.com/novox/mesh-controller/examples/objectstore-provisioner 1.025s
ok github.com/novox/mesh-controller/examples/postgres-provisioner 1.034s
ok github.com/novox/mesh-controller/examples/redis-provisioner 1.028s
ok github.com/novox/mesh-controller/examples/route-proxy 1.143s
ok github.com/novox/mesh-controller/internal/artifacts 1.071s
? github.com/novox/mesh-controller/internal/beside [no test files]
ok github.com/novox/mesh-controller/internal/broker 4.133s
ok github.com/novox/mesh-controller/internal/builder 16.061s
ok github.com/novox/mesh-controller/internal/catalogue 7.809s
ok github.com/novox/mesh-controller/internal/conditions 2.661s
ok github.com/novox/mesh-controller/internal/envfile 1.071s
ok github.com/novox/mesh-controller/internal/facts 1.069s
ok github.com/novox/mesh-controller/internal/identity 4.682s
ok github.com/novox/mesh-controller/internal/inventory 87.982s
ok github.com/novox/mesh-controller/internal/lease 17.735s
ok github.com/novox/mesh-controller/internal/licences 4.287s
ok github.com/novox/mesh-controller/internal/licences/adapters 1.087s
ok github.com/novox/mesh-controller/internal/link 59.428s
ok github.com/novox/mesh-controller/internal/lint 2.355s
ok github.com/novox/mesh-controller/internal/outward 1.090s
ok github.com/novox/mesh-controller/internal/overlay 1.081s
ok github.com/novox/mesh-controller/internal/secrets 1.486s
ok github.com/novox/mesh-controller/internal/store 3.502s
ok github.com/novox/mesh-controller/internal/testbus 1.257s
ok github.com/novox/mesh-controller/internal/token 1.020s
store tests, each run against the store:
./cmd/mesh-controller --- PASS: TestAStopWithoutAMergeWalksTheOthersAgain (0.39s)
./cmd/mesh-controller --- PASS: TestAStopWithoutAMergeEndsTheMergesThatContainIt (0.45s)
./cmd/mesh-controller --- PASS: TestAStopWalksTheOthersAgainInTheOpenBatch (0.37s)
./cmd/mesh-controller --- PASS: TestAStopNamesWhatStaysOnTheMachinesAndWhatStillCarriesIt (0.40s)
./cmd/mesh-controller --- PASS: TestAStopWithoutAMergeLeavesItOutOfABatch (0.38s)
./cmd/mesh-controller --- PASS: TestSavingPlansAndMovingMergesIsOneAct (0.36s)
./cmd/mesh-controller --- PASS: TestNodeShowAndStatusReadTheLastSendFromTheStore (0.41s)
./cmd/mesh-controller --- PASS: TestTheConditionsVerbReadsTheLooksTheBucketNoLongerHolds (0.28s)
./internal/inventory --- PASS: TestTheLastSendSaysWhatTheMachineAnswered (0.25s)
./internal/inventory --- PASS: TestAReportIsTheLastSendsAnswerOnlyWhenItIsAboutThatSend (0.24s)
./internal/inventory --- PASS: TestAnIssuedValueIsSealedToEveryHolderOnceAndNotAgain (0.25s)
./internal/inventory --- PASS: TestAnIssuedValueIsSealedAgainAfterAKeyChangeOrAnAcceptance (0.25s)
./internal/inventory --- PASS: TestAConditionsLooksAreKeptBesideTheBucket (0.25s)
```
Independent review for issue 420 (the conditions bucket is rewritten on every observation). It follows the decision note on the issue (2026-10-11): the bucket holds what changes on a transition, the looks live in the controller's store, and the verb and the events still carry them.
Checklist
Schema rule. This repository has no consolidated schema, and I verified how a fresh install builds it. internal/inventory/inventory.go:20 embeds migrations/*.sql, and internal/store/migrate.go:201 (Migrate) applies every pending migration in order, each in a transaction under an advisory lock, recorded in a ledger. A fresh store therefore runs 0001 to 0095. No schema.sql or other non-migration .sql exists. Not blocking.
Migration 0095. The number is unique: main's highest is 0094, and main moved only by #246, which adds no migration. It is plain create table with no if not exists, like all 92 other migrations here. It runs once because of the ledger and the transaction, and a changed or duplicate file is refused (migrate_test.go). During rollout, old code never touches condition_looks. New code reading an old-shape bucket value keeps the value's looks (looks.goattachOne, the LastObserved non-zero branch) and rewrites the value once without them. TestAConditionWrittenWithItsLooksMovesThemOutOnce proves this: 7 003 goes to 7 006. On a rollback, old code would read observations 0 and count on from 1, which is acceptable.
Readers of the bucket. In mesh-catalog main, the only references to mesh-controller_conditions are the nats snapshot tests (modules/nats/snapshot/live_test.go, which backs up and restores the stream, independent of field shape) and a messenger test fixture. The messenger's condition.go reads last-observed and observations from the verb answer and the events, and both still carry them. mesh-tools, mesh-sdk, mesh-host (only the NATS permission line in an example lock) and mesh-lab do not read the bucket's fields. I found no reader that loses data.
Lease. KeepConditionLooks (internal/inventory/looks.go) records actingEpoch, following the same pattern as plans, so a process without the lease keeps nothing. A clear forgets the row (store.go, after the bucket delete). If that forget fails, a re-raise overwrites the row through keepLooks on the Create path, so no stale count survives. Evidence is capped at KeptEvidence = 10, and rows exist only for open conditions, so growth is bounded.
Tests. I ran sh merge-check.sh on this head against a throwaway PostgreSQL 18.6 with gcc present, so under -race. It passed: gofmt, vet, all 26 packages ok, and every listed store test PASS, including TestAConditionsLooksAreKeptBesideTheBucket and TestTheConditionsVerbReadsTheLooksTheBucketNoLongerHolds. Mutation: I replaced !bytes.Equal(body, entry.Value) in Observe with true ||. That fails TestALookThatChangesNothingWritesNothingToTheBucket, TestAConditionWrittenWithItsLooksMovesThemOutOnce and the store-backed verb test. The tests bite.
Checks at this head: mesh/merge-gate success (builds build-agent, mesh-controller, route-proxy for ace, g14, novox, shanks; every machine composes) and mesh/repo-check success. mesh-controller.plans shows no plan mid-gate at 12:25.
Non-blocking observations. These are follow-ups, not reasons to hold the merge.
An old-shape condition loses its count if the store is away on its first look after rollout (internal/conditions/store.go, Observe). In that case attach returns false and c keeps the value's looks. BucketValue strips them and the value differs, so the bucket is rewritten without them, but keepLooks is skipped because looked is false. The count then exists nowhere, and the next look starts from 1. The keeper says that the looks could not be read, but the loss itself goes unreported. A small fix: when !looked, skip the bucket write if the only difference is the looks fields, that is, compare BucketValue(c) with BucketValue of the value as read.
Concurrent looks at the same key can lose a count. Before this change, the compare-and-set on the bucket serialised two Observe calls on one key. Now an unchanged look writes no bucket, so two concurrent looks can both read N and both keep N+1. The condition_looks upsert has no revision guard. I think this is rare, because one watchdog owns one key, so it is not blocking, but the count can now undercount. If it matters, an observations = condition_looks.observations + 1-style increment or a version column would fix it.
The word "looks" is not in the glossary. The hq pull request (#321) proposes it. That is for the operator or a glossary pull request to settle.
Not merged by me.
Verdict: approve at dcdafd37f008690935a1560c79145025e4c3cd88
Independent review for issue 420 (the conditions bucket is rewritten on every observation). It follows the decision note on the issue (2026-10-11): the bucket holds what changes on a transition, the looks live in the controller's store, and the verb and the events still carry them.
Checklist
1. Schema rule. This repository has no consolidated schema, and I verified how a fresh install builds it. `internal/inventory/inventory.go:20` embeds `migrations/*.sql`, and `internal/store/migrate.go:201` (`Migrate`) applies every pending migration in order, each in a transaction under an advisory lock, recorded in a ledger. A fresh store therefore runs 0001 to 0095. No `schema.sql` or other non-migration `.sql` exists. Not blocking.
2. Migration 0095. The number is unique: main's highest is 0094, and main moved only by #246, which adds no migration. It is plain `create table` with no `if not exists`, like all 92 other migrations here. It runs once because of the ledger and the transaction, and a changed or duplicate file is refused (`migrate_test.go`). During rollout, old code never touches `condition_looks`. New code reading an old-shape bucket value keeps the value's looks (`looks.go` `attachOne`, the `LastObserved` non-zero branch) and rewrites the value once without them. `TestAConditionWrittenWithItsLooksMovesThemOutOnce` proves this: 7 003 goes to 7 006. On a rollback, old code would read observations 0 and count on from 1, which is acceptable.
3. Readers of the bucket. In mesh-catalog main, the only references to `mesh-controller_conditions` are the nats snapshot tests (`modules/nats/snapshot/live_test.go`, which backs up and restores the stream, independent of field shape) and a messenger test fixture. The messenger's `condition.go` reads `last-observed` and `observations` from the verb answer and the events, and both still carry them. mesh-tools, mesh-sdk, mesh-host (only the NATS permission line in an example lock) and mesh-lab do not read the bucket's fields. I found no reader that loses data.
4. Lease. `KeepConditionLooks` (`internal/inventory/looks.go`) records `actingEpoch`, following the same pattern as plans, so a process without the lease keeps nothing. A clear forgets the row (`store.go`, after the bucket delete). If that forget fails, a re-raise overwrites the row through `keepLooks` on the Create path, so no stale count survives. Evidence is capped at `KeptEvidence = 10`, and rows exist only for open conditions, so growth is bounded.
5. Tests. I ran `sh merge-check.sh` on this head against a throwaway PostgreSQL 18.6 with gcc present, so under `-race`. It passed: gofmt, vet, all 26 packages ok, and every listed store test PASS, including `TestAConditionsLooksAreKeptBesideTheBucket` and `TestTheConditionsVerbReadsTheLooksTheBucketNoLongerHolds`. Mutation: I replaced `!bytes.Equal(body, entry.Value)` in `Observe` with `true ||`. That fails `TestALookThatChangesNothingWritesNothingToTheBucket`, `TestAConditionWrittenWithItsLooksMovesThemOutOnce` and the store-backed verb test. The tests bite.
7. Checks at this head: mesh/merge-gate success (builds build-agent, mesh-controller, route-proxy for ace, g14, novox, shanks; every machine composes) and mesh/repo-check success. `mesh-controller.plans` shows no plan mid-gate at 12:25.
Non-blocking observations. These are follow-ups, not reasons to hold the merge.
- **An old-shape condition loses its count if the store is away on its first look after rollout** (`internal/conditions/store.go`, `Observe`). In that case `attach` returns false and `c` keeps the value's looks. `BucketValue` strips them and the value differs, so the bucket is rewritten without them, but `keepLooks` is skipped because `looked` is false. The count then exists nowhere, and the next look starts from 1. The keeper says that the looks could not be read, but the loss itself goes unreported. A small fix: when `!looked`, skip the bucket write if the only difference is the looks fields, that is, compare `BucketValue(c)` with `BucketValue` of the value as read.
- **Concurrent looks at the same key can lose a count.** Before this change, the compare-and-set on the bucket serialised two `Observe` calls on one key. Now an unchanged look writes no bucket, so two concurrent looks can both read N and both keep N+1. The `condition_looks` upsert has no revision guard. I think this is rare, because one watchdog owns one key, so it is not blocking, but the count can now undercount. If it matters, an `observations = condition_looks.observations + 1`-style increment or a version column would fix it.
- The word "looks" is not in the glossary. The hq pull request (#321) proposes it. That is for the operator or a glossary pull request to settle.
Not merged by me.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Fixes issue 420 (the conditions bucket is rewritten on every observation).
Why. Every look rewrote a condition's value in
mesh-controller_conditions, because the value carried the count of observations, the last observed time and the newest evidence. Measured on 2026-10-10: 131 647 writes in four days for twelve open conditions (mesh.hand-acts.disk-load.healer-wantedalone 7 003). That breaks novox/hq ADR 0289 §1 (a bucket is written only when what it describes changed) and §4 (no idle-changing fields). The decision on the issue (2026-10-11, shanks session): the bucket holds what changes on a transition; the looks live in the controller's store and the verb and the events still carry them.What changes
internal/conditions/looks.go:Looks(observations, last-observed, evidence) and aLookStore.BucketValuewrites a condition without its looks, keys in one order.Keeper.Observecompares the new bucket value with the stored bytes and writes only when they differ; a fresh look is kept in theLookStoreafter the bucket write (so a lost compare-and-set does not double-count). Clear forgets the looks. Every read (Open,Get,Read/ReadWith,ReadOne/ReadOneWith) and every event puts the looks back.looks-unreadwith why, never a count of none.internal/inventory: migration 0095condition_looks(key, observations, last_observed, evidence jsonb, epoch, kept_at); written under the lease (actingEpoch), forgotten on a clear. There is no consolidated schema in this repository; migrations are the schema.cmd/mesh-controller/condition_looks.go: the store adapter. The serving controller anddoctorrun locally use it; theconditionsverb, which runs as a process of its own, opens the store to read the looks (looksHere), or says them unread.Readers. The messenger reads the
conditionsverb answer and the condition events, not the bucket (checked in mesh-catalog main); both still carryobservations,last-observedandevidence. No other reader of the bucket was found. The hq side: novox/hq branchfix/420-condition-looks-beside-the-bucketamends to-be 45 §2's field table.Tests (each fails without the fix: replacing the compare with an unconditional write fails all three)
TestALookThatChangesNothingWritesNothingToTheBucket: five looks write the bucket once; the verb reads 5 observations and 5 evidence lines; the bucket's value holds no look field; a severity change writes once more and its event carries the looks; the clearing forgets them.TestAConditionWrittenWithItsLooksMovesThemOutOnce: an old-shaped value with 7 003 observations is rewritten once and counts on to 7 006.TestLooksThatCannotBeKeptAreSaidNotCountedAsNone: with the store away the condition is still raised and sayslooks-unread.TestTheConditionsVerbReadsTheLooksTheBucketNoLongerHolds(store + real bus): four looks leave the bucket stream at sequence 1;conditions showand the list, as a process other than the serving controller, read 4 observations from the store.TestAConditionsLooksAreKeptBesideTheBucket(store): kept under the lease only, replaced, listed, forgotten.testdata/store-tests.sh merge-check.shpassed locally with a throwaway PostgreSQL 18.6, race on, every listed store test PASS.Reviewers' checklist
bytes.EqualinObserve), and a lost compare-and-set does not count a look twice (looks kept after the write).BucketValue(raise, observe, amend/tried/escalate, silence, end of silence).Overlap. Does not touch any file #247 changes (
walk_said.go,walksaid.go,plans.go,delivery.go,release_plan.go). A controller rollout: merge only whilemesh-controller.planssays no plan is mid-gate, and not while #247 is mid-gate.By fix-420@g14. Not merged; waits for review.
Delivery
novox/mesh-controller@dcdafd37f008— delivered since 2026-10-12T10:41:33ZDelivery plan — builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step
Machines
Transitions
The commit's note under
refs/notes/mesh-plankeeps every transition:git log --notes=mesh-plan.Merge check at
dcdafd37mesh/merge-gate: PASS — every machine composes with the change as it did without (4 of 4 compose) (modules: build-agent, mesh-controller, route-proxy)mesh/repo-check: PASS — its merge-check.sh passedDelivery plan — builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step
Run by the build seat on ace as
build-1791799936294115889(builds --log build-1791799936294115889).Verdict: approve at
dcdafd37f0Independent review for issue 420 (the conditions bucket is rewritten on every observation). It follows the decision note on the issue (2026-10-11): the bucket holds what changes on a transition, the looks live in the controller's store, and the verb and the events still carry them.
Checklist
internal/inventory/inventory.go:20embedsmigrations/*.sql, andinternal/store/migrate.go:201(Migrate) applies every pending migration in order, each in a transaction under an advisory lock, recorded in a ledger. A fresh store therefore runs 0001 to 0095. Noschema.sqlor other non-migration.sqlexists. Not blocking.create tablewith noif not exists, like all 92 other migrations here. It runs once because of the ledger and the transaction, and a changed or duplicate file is refused (migrate_test.go). During rollout, old code never touchescondition_looks. New code reading an old-shape bucket value keeps the value's looks (looks.goattachOne, theLastObservednon-zero branch) and rewrites the value once without them.TestAConditionWrittenWithItsLooksMovesThemOutOnceproves this: 7 003 goes to 7 006. On a rollback, old code would read observations 0 and count on from 1, which is acceptable.mesh-controller_conditionsare the nats snapshot tests (modules/nats/snapshot/live_test.go, which backs up and restores the stream, independent of field shape) and a messenger test fixture. The messenger'scondition.goreadslast-observedandobservationsfrom the verb answer and the events, and both still carry them. mesh-tools, mesh-sdk, mesh-host (only the NATS permission line in an example lock) and mesh-lab do not read the bucket's fields. I found no reader that loses data.KeepConditionLooks(internal/inventory/looks.go) recordsactingEpoch, following the same pattern as plans, so a process without the lease keeps nothing. A clear forgets the row (store.go, after the bucket delete). If that forget fails, a re-raise overwrites the row throughkeepLookson the Create path, so no stale count survives. Evidence is capped atKeptEvidence = 10, and rows exist only for open conditions, so growth is bounded.sh merge-check.shon this head against a throwaway PostgreSQL 18.6 with gcc present, so under-race. It passed: gofmt, vet, all 26 packages ok, and every listed store test PASS, includingTestAConditionsLooksAreKeptBesideTheBucketandTestTheConditionsVerbReadsTheLooksTheBucketNoLongerHolds. Mutation: I replaced!bytes.Equal(body, entry.Value)inObservewithtrue ||. That failsTestALookThatChangesNothingWritesNothingToTheBucket,TestAConditionWrittenWithItsLooksMovesThemOutOnceand the store-backed verb test. The tests bite.mesh-controller.plansshows no plan mid-gate at 12:25.Non-blocking observations. These are follow-ups, not reasons to hold the merge.
internal/conditions/store.go,Observe). In that caseattachreturns false andckeeps the value's looks.BucketValuestrips them and the value differs, so the bucket is rewritten without them, butkeepLooksis skipped becauselookedis false. The count then exists nowhere, and the next look starts from 1. The keeper says that the looks could not be read, but the loss itself goes unreported. A small fix: when!looked, skip the bucket write if the only difference is the looks fields, that is, compareBucketValue(c)withBucketValueof the value as read.Observecalls on one key. Now an unchanged look writes no bucket, so two concurrent looks can both read N and both keep N+1. Thecondition_looksupsert has no revision guard. I think this is rare, because one watchdog owns one key, so it is not blocking, but the count can now undercount. If it matters, anobservations = condition_looks.observations + 1-style increment or a version column would fix it.Not merged by me.