Issue 270: Phase 1 watches signals that come later; to-be 45 in progress
Building Phase 1 of to-be 45 found four of its rows depend on later phases and S9 hears only the controller's own connection; the decisions the build made that the design does not state are recorded for an amendment.
This commit is contained in:
+69
@@ -0,0 +1,69 @@
|
||||
---
|
||||
status: located
|
||||
opened: 2026-10-06
|
||||
located-in: [mesh-controller]
|
||||
fixed-by:
|
||||
amended-design:
|
||||
---
|
||||
|
||||
# 270. Phase 1 watches signals that come later, and hears only the controller's own bus faults
|
||||
|
||||
## Symptom
|
||||
|
||||
Found building Phase 1 of [to-be 45](../../03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md)
|
||||
on 2026-10-06. The phase table gives the controller "watchdogs for S1–S10 and S12–S14" and "`doctor`
|
||||
with D1–D10". Four of those cannot be built in Phase 1, because what they watch does not exist until a
|
||||
later phase, and one is built to half of what the signals table says:
|
||||
|
||||
| Row | What the design asks | Why it cannot be done in Phase 1 |
|
||||
|---|---|---|
|
||||
| S12 *the controller lease renewed* | a watchdog on the lease's renewal | the lease is Phase 2 (§6); there is nothing renewed |
|
||||
| S14 *facts snapshot exported* | a watchdog on a daily export | the facts snapshot is Phase 5 (§9); nothing exports one |
|
||||
| D5 *exactly one lease holder, no stale epoch* | a probe of the lease | Phase 2, as S12 |
|
||||
| S13 *stale refusals, naming the writer* | more than 5 refusals from one writer in 5 min | the node-engine refuses a stale declaration today, but no declaration carries its writer's epoch until Phase 2: the watchdog counts refusals per **machine** and cannot name the writer |
|
||||
| S9 *slow consumer, permission violation* | every principal's | the bus has one account and no system account, and the server says these of other principals only to a system account: the controller hears them for **its own connection** (the client library is told) and the account-wide JetStream advisories (maximum deliveries, consumer deleted) — not a module's slow consumer or a module refused a subject |
|
||||
|
||||
Built as each phase's dependency allows, the rows are in the compiled signals table and the probe
|
||||
registry marked deferred, each with its reason and phase; the test generated from the table refuses a
|
||||
deferred row that is watched or a watched row without a suppression. Nothing is silent about them —
|
||||
but the design says Phase 1 delivers them, and a reader of the design believes it.
|
||||
|
||||
## Decided while building, and not written in the design
|
||||
|
||||
1. **The condition history is its own bucket**, `mesh-controller_condition-history`, kept ninety days.
|
||||
A bucket has one age for every key; the open conditions must have none, or a condition open longer
|
||||
than the history would vanish while still true. The writers table names only
|
||||
`mesh-controller_conditions`.
|
||||
2. **The condition events carry the condition at the top level**, with `event`, `at`, `change`
|
||||
(raised, reopened, severity, resolver, silenced, silence-ended, cleared), `why`, `was`, `cleared`
|
||||
and `show` beside it — the shape the operator-channel's holder was written against. A silence and
|
||||
its ending are `condition-changed`; a reopening within ten minutes is `condition-raised` with change
|
||||
`reopened`. The self-check's heartbeat is the seat's event `doctor-heartbeat`.
|
||||
3. **A key's last token is the kind's short word** where §2's examples give one —
|
||||
`provider.<module>.<node>.<consumer>.failing` for `provider-failing`, `core.controller.deaf` — and
|
||||
the kind elsewhere. A key is opaque to every reader; the kind is the field.
|
||||
4. **A probe the registry did not have, DW: the watchdogs ran within three ticks.** Rule 6 has the
|
||||
self-check watched from a second machine; the watchdogs themselves had no watcher. The self-check
|
||||
watches them, and they raise S10 when the self-check stops.
|
||||
5. **A deleted consumer is said only for a consumer the mesh names** (the controller's, a machine's
|
||||
declaration consumer, `<node>_<module>`, a seat's worker), and only while the mesh expects it and it
|
||||
is missing. Every watch of a bucket and every read-back of a stream makes and deletes a consumer of
|
||||
its own, and the server advises each deletion — five a tick, found running the controller against
|
||||
a real bus.
|
||||
6. **Bounds the design leaves to the build**, provisional like the rest: S6, no build timeout is
|
||||
declared, so max(20 min, 3 × the p90 of measured builds), one hour while nothing is measured; S7,
|
||||
per verb (push, rotate and command 30 min; assign and unassign 15 min; doctor 3 min; others 10);
|
||||
S9, an advisory clears after an hour with no other; D6, a consumer more than 1000 messages from its
|
||||
stream's head is behind.
|
||||
7. **ADR 0224's standing moved into the store without importing what the old table held.** A
|
||||
provider says its failing word again every fifteen minutes; the table `provider_standing` is no
|
||||
longer read or written and is left in place, because dropping it is a removal of data and that is
|
||||
the operator's word to give.
|
||||
|
||||
## Open questions
|
||||
|
||||
1. Does Phase 1's table move S12, D5 to Phase 2 and S14 to Phase 5, and S13's naming of the writer to
|
||||
Phase 2? (The build assumes yes.)
|
||||
2. Does the bus gain a system account — or the controller read the server's monitoring endpoint — so
|
||||
S9 hears every principal's slow consumer and refused subject? Either changes the foundation's shape.
|
||||
3. Do decisions 1–6 above go into §1–§4 as written, through an amendment of to-be 45?
|
||||
Reference in New Issue
Block a user