Say when the mesh is wrong: conditions, watchdogs, the bus's advisories, doctor (hq to-be 45 Phase 1)

Every one of the 48 core failures of research 031 was found by a person
looking; the mesh's answers carried the fact for whoever asked and told
nobody.

- The condition store (to-be 45 §2): mesh-controller_conditions, one key
  per open condition, written by compare-and-set so a person's silence
  and the watchdogs never lose each other's word; every transition kept
  ninety days in mesh-controller_condition-history and said as the
  seat's events condition-raised / condition-changed / condition-cleared
  (the condition at the top level, with event, at, change, why, show),
  offered again while the bus is away. Raised and cleared by observation
  only; a clearing reopened within ten minutes is the same condition with
  its count up, its silence kept. Verbs: conditions, conditions show,
  conditions silence (a hand act, at most a week), conditions history.
- ADR 0224's provider standing is the first kind, provider-failing, held
  by the provider's events; the provider_standing table is no longer read
  or written (left in place: dropping it is the operator's word).
- status leads with the open conditions, urgent first, and says all well
  only with none open; conditions it cannot read are said and not well.
- The signals table compiled in, one watchdog loop over it every 30s: S1
  heartbeat (3 intervals, asleep machines excepted, control node urgent
  after 30 min), S2 report after a send, S3 plan tier, S4 event loop deaf,
  S5 merge not acted, S6 ask lost, S7 call hung, S8 provider silent, S9
  advisories, S10 self-check silent, S11 node tools silent, S13 stale
  refusals; S12, S14, S15 deferred with their reasons. A row that cannot
  see raises probe-failed and clears nothing. A test generated from the
  table suppresses each signal inside and past its bound.
- The bus's advisories (maximum deliveries, a mesh consumer deleted) and
  the controller's own slow consumer and refused subjects, said in the
  mesh's words.
- doctor: the probe registry D1-D10 (D5 deferred) and DW, every five
  minutes, each in thirty seconds; a probe that cannot run is never a
  pass. D1 validates with mesh-host's own validator. Every run ends with
  the doctor-heartbeat event mesh-watcher listens for.
- The controller is granted its new buckets, events, the two advisories
  and $SRV.INFO; the node tools their tools-alive heartbeat. The streams
  and consumers the controller asserts and the ones D6/D7 expect are one
  derivation.

internal/broker/controller_buckets.go

@@ -19,10 +19,18 @@ import (
 // like the streams, so a bus raised from nothing has them before the first call is served.
 
 // CallsBucket keeps every call of the mesh's own verbs and what came of it; HandActsBucket every act
-// a person did by hand, with why.
+// a person did by hand, with why; ConditionsBucket every condition open now (to-be 45 §2), one key
+// each, and ConditionHistoryBucket every transition of one — raised, changed, silenced, cleared —
+// for ninety days.
+//
+// **The history is a bucket of its own** because its keys expire and an open condition's must not:
+// a bucket has one age for every key, and a condition open longer than the history is kept would
+// otherwise vanish from the store while still true.
 var (
-	CallsBucket    = BucketName(ControllerSeat, "calls")
-	HandActsBucket = BucketName(ControllerSeat, "hand-acts")
+	CallsBucket            = BucketName(ControllerSeat, "calls")
+	HandActsBucket         = BucketName(ControllerSeat, "hand-acts")
+	ConditionsBucket       = BucketName(ControllerSeat, "conditions")
+	ConditionHistoryBucket = BucketName(ControllerSeat, "condition-history")
 )
 
 // The bounds to-be 45 §6 sets for calls: the last thousand, or fourteen days, whichever is fewer.
@@ -37,11 +45,19 @@ const (
 	// HandActsKeptFor is as long as a condition's history (to-be 45 §2): an act by hand is read
 	// back beside what it addressed.
 	HandActsKeptFor = 90 * 24 * time.Hour
+	// ConditionHistoryKeptFor is how long a condition's transitions are kept (to-be 45 §2).
+	ConditionHistoryKeptFor = 90 * 24 * time.Hour
 )
 
 // IsControllerBucket says a bucket is the controller's own, not a module's state nothing declares.
 func IsControllerBucket(bucket string) bool {
-	return bucket == CallsBucket || bucket == HandActsBucket
+	return bucket == CallsBucket || bucket == HandActsBucket || bucket == ConditionsBucket ||
+		bucket == ConditionHistoryBucket
+}
+
+// ControllerBuckets are the controller's own buckets, in the order they are asserted.
+func ControllerBuckets() []string {
+	return []string{CallsBucket, HandActsBucket, ConditionsBucket, ConditionHistoryBucket}
 }
 
 // ControllerBucketsAsserter is what raising the controller's buckets needs of a connection.
@@ -98,5 +114,30 @@ func (j *JetStream) EnsureControllerBuckets() error {
 	}); err != nil {
 		return fmt.Errorf("asserting bucket %s: %w", HandActsBucket, err)
 	}
+	// **No age on the open conditions.** A condition is removed when observation clears it and at no
+	// other moment: one that expired would be a fault the store forgot while it was still true.
+	if _, err := js.CreateOrUpdateKeyValue(ctx, jetstream.KeyValueConfig{
+		Bucket: ConditionsBucket,
+		Description: "every condition open now, one key each (novox/hq to-be 45 §2): written by the " +
+			"controller alone, raised and cleared by observation, read through `conditions`",
+		History:      1,
+		MaxValueSize: 64 << 10,
+		MaxBytes:     64 << 20,
+		Storage:      jetstream.FileStorage,
+	}); err != nil {
+		return fmt.Errorf("asserting bucket %s: %w", ConditionsBucket, err)
+	}
+	if _, err := js.CreateOrUpdateKeyValue(ctx, jetstream.KeyValueConfig{
+		Bucket: ConditionHistoryBucket,
+		Description: "every transition of a condition — raised, changed, silenced, cleared — kept ninety " +
+			"days (novox/hq to-be 45 §2): written by the controller alone, read through `conditions history`",
+		History:      1,
+		TTL:          ConditionHistoryKeptFor,
+		MaxValueSize: 64 << 10,
+		MaxBytes:     256 << 20,
+		Storage:      jetstream.FileStorage,
+	}); err != nil {
+		return fmt.Errorf("asserting bucket %s: %w", ConditionHistoryBucket, err)
+	}
 	return nil
 }