Keep the declarations retired once their stream is gone, so the bus is not called unhealthy for a stream release 6 deleted (issue 1945) #462

2026-10-25T14:12:30Z

mesh-tools commented

2026-10-25 14:12:30 +00:00

At 14:52 on 2026-10-25 the operator was paged urgently: 'The bus is not healthy' and 'Part of the bus's storage is wrong' (bus.mesh.unhealthy, bus.NODES.stream-missing, bus.NODES..missing). The start deletes NODES once every node runs the release-6 node-engine (ADR 0539 clause 11), but 'retired' was judged from every node's mesh-host have against the registered fingerprint, which turns false again whenever a newer node-engine is registered or landing. Then expectedBusObjects named NODES again, H-bus and D7 raised it, and healer H3 made NODES anew, empty; the start deleted it again: flapping.

What changes:

composed_start.go:147-149: a NODES stream that is gone counts as retired, whatever the nodes run; release 6 writes no declaration, so a remade NODES would hold nothing. While it exists, the old test decides as before.
watchdogs.go:515-517: lostConsumers treats ErrStreamNotFound like ErrConsumerNotFound (it failed the whole look when the four NODES consumers went with their stream).
The conditions clear by themselves at the next probe run after delivery.

Tests: TestTheDeclarationsStayRetiredOnceTheirStreamIsGone (unit, seen failing first); TestTheDeclarationsStreamGoesOnceEveryNodeRunsTheNewEngine extended (a later node-engine change keeps it retired; expectedBusObjects names no NODES; assertOnSend does not remake it).

Tries: try-1792936878038885855 passed on 0a536d7b (1015 unit, 1681 integration).
Reviewed-by: reviewer-r6m@shanks at 0a536d7b6d
Pushed-by: lead-47@shanks

At 14:52 on 2026-10-25 the operator was paged urgently: 'The bus is not healthy' and 'Part of the bus's storage is wrong' (bus.mesh.unhealthy, bus.NODES.stream-missing, bus.NODES.<node>.missing). The start deletes NODES once every node runs the release-6 node-engine (ADR 0539 clause 11), but 'retired' was judged from every node's mesh-host have against the registered fingerprint, which turns false again whenever a newer node-engine is registered or landing. Then expectedBusObjects named NODES again, H-bus and D7 raised it, and healer H3 made NODES anew, empty; the start deleted it again: flapping. What changes: 1. composed_start.go:147-149: a NODES stream that is gone counts as retired, whatever the nodes run; release 6 writes no declaration, so a remade NODES would hold nothing. While it exists, the old test decides as before. 2. watchdogs.go:515-517: lostConsumers treats ErrStreamNotFound like ErrConsumerNotFound (it failed the whole look when the four NODES consumers went with their stream). The conditions clear by themselves at the next probe run after delivery. Tests: TestTheDeclarationsStayRetiredOnceTheirStreamIsGone (unit, seen failing first); TestTheDeclarationsStreamGoesOnceEveryNodeRunsTheNewEngine extended (a later node-engine change keeps it retired; expectedBusObjects names no NODES; assertOnSend does not remake it). Tries: try-1792936878038885855 passed on 0a536d7b (1015 unit, 1681 integration). Reviewed-by: reviewer-r6m@shanks at 0a536d7b6d794541be11600a11a04681c2032b4e Pushed-by: lead-47@shanks

mesh-tools added 1 commit 2026-10-25 14:12:31 +00:00

Keep the declarations retired once their stream is gone, so nothing expects, asserts or makes NODES again (issue 1945)

mesh/check pass 0a536d7b6d79 on 2bcd85d2df4c | unit 966 run, 172 kept, 2 skipped | integration 1622 run, 236 kept, 7 skipped | over bound: integration…

mesh/delivery merged: source.main moved to novox/mesh-controller@2814262ac0fa

Details

0a536d7b6d

A node-engine newer than the registered one, or a node asleep on an older
have, made the done test false after the start deleted NODES; the bus's
health check, probes D6 and D7 and healer H3 then expected the stream and
every node's consumer again, raised them urgent and made them anew. Release
6 sends no declaration, so a stream made again helps no node: a gone stream
is retired. The advisory watchdog also reads a consumer whose stream is gone
as gone instead of failing.

mesh-tools commented

2026-10-25 14:12:35 +00:00

Delivery novox/mesh-controller@0a536d7b6d79 — merged since 2026-10-25T14:25:13Z

Source — source.main moved to novox/mesh-controller@2814262ac0. Whether it reached the machines is mesh-delivery-view's, derived from source.main, the wants and the haves.

Transitions

2026-10-25 14:12 (new) → proposed (announced): novox/mesh-controller#462's head announced
2026-10-25 14:24 proposed → ready (accepted): a verdict names this commit and the trunk's head as its trunk commit, the gate passed or warned, and the repository's own check did not fail
2026-10-25 14:25 ready → merged (merged): on the trunk its modules follow: merged there

The commit's note under refs/notes/mesh-plan keeps every transition of this change up to its merge: git log --notes=mesh-plan.

**Delivery** `novox/mesh-controller@0a536d7b6d79` — **merged** since 2026-10-25T14:25:13Z **Source** — source.main moved to novox/mesh-controller@2814262ac0fa. Whether it reached the machines is `mesh-delivery-view`'s, derived from `source.main`, the wants and the haves. **Transitions** - 2026-10-25 14:12 (new) → proposed (announced): novox/mesh-controller#462's head announced - 2026-10-25 14:24 proposed → ready (accepted): a verdict names this commit and the trunk's head as its trunk commit, the gate passed or warned, and the repository's own check did not fail - 2026-10-25 14:25 ready → merged (merged): on the trunk its modules follow: merged there The commit's note under `refs/notes/mesh-plan` keeps every transition of this change up to its merge: `git log --notes=mesh-plan`.

mesh-tools merged commit 2814262ac0 into main

2026-10-25 14:25:12 +00:00

mesh-tools deleted branch fix/r6-nodes-stream-gone

2026-10-25 14:25:12 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: novox/mesh-controller#462