Heal what is known, under a brake, and say every repair (hq to-be 45 Phase 3)

Research 031 counted the repairs people made by hand: a push to unstick a
plan waiting on a report, a controller restarted to make an object again, a
plan closed, a consumer re-made from now. Each was the ordinary path taken
again by someone who noticed. The healer registry makes each a registered
response to one condition kind, with a budget, a settle and its event:

- H1 sent-not-reported: ask the machine's node-engine to report again
  (mesh.node.<n>.ask.report); if it does not report what it was sent, send
  it again, never moving a build a policy or a plan holds back
- H2 stalled: close a plan whose wait is superseded or finished
- H3 holder-silent / consumer-lost: the send's own assertion of the bus's
  objects (issue 208's note)
- H4 consumer-behind: consumer-reset, only for a consumer the stream table
  marks resettable (the controller's own events consumer)
- H5 is the identity provider's own repair (ADR 0224 §5), registered only

Success is the observation clearing the condition, never the healer; a spent
budget hands the condition to the operator, urgent, with what was tried, and
no healer touches it again. Every act is begun in the store before it is made
(migration 0070), kept in the condition's tried as "healer Hn" and said as
the seat event healer-acted; a heal is never a hand act. More than twelve acts
in an hour stop every healer until an hour after the last, said urgently.
Only the lease holder heals.

S15 is live: a cause repaired by hand twice in a fortnight raises
healer-wanted, naming the healer that was not enough where one exists. D6's
far-behind finding has its own kind, consumer-behind. Nodes are granted the
question; the controller's grant gains healer-acted (genesis lock in
mesh-host). `healers` lists the registry, the acts and the brake; status
counts the week's heals.

internal/broker/derived.go

@@ -54,6 +54,11 @@ type Consumer struct {
 	// that exists keeps where it is, whatever this says; only its making is decided here.
 	FromNow bool
 	Why     string
+	// Resettable says why this consumer may be re-made to deliver from now by the mesh itself, with
+	// nobody asked (novox/hq to-be 45 §7, healer H4): what a reset drops, something else catches up.
+	// Empty for every consumer where nothing would — a module's, whose events would be lost to it.
+	// Not a property of the consumer on the bus: nothing here is sent to the server.
+	Resettable string
 }
 
 // seatStreamName is the stream holding a seat's inbound work. Named after the seat rather than

internal/broker/nats.go

@@ -179,6 +179,10 @@ func (p Principal) Username() string {
 	return ""
 }
 
+// AskReportSubject is where the mesh asks one machine's node-engine to say again what it last applied
+// (novox/hq to-be 45 §6): its answer is an ordinary report, on its own report subject.
+func AskReportSubject(node string) string { return "mesh.node." + node + ".ask.report" }
+
 // inbox is a principal's own reply space. No user is ever granted a bare `_INBOX.>` (design 25
 // §4): with one account, inbox privacy is the permission list or it is nothing, so each user's
 // inbox is derived from its own identity and its permissions name that prefix and no other.
@@ -378,7 +382,11 @@ func PermissionsFor(p Principal) (Permissions, error) {
 		// next assertion moves it, and a permission that only allowed the new shape would refuse
 		// every node in the mesh for exactly as long as that took.
 		sub = []string{"mesh.node." + p.Node + ".declare",
-			"_DELIVER." + p.Node, "_DELIVER." + p.Node + ".>"}
+			"_DELIVER." + p.Node, "_DELIVER." + p.Node + ".>",
+			// And the mesh asking it to say again what it last applied (novox/hq to-be 45 §6, the
+			// `report` verb healer H1 asks): its own machine's, on core NATS and off any stream. It
+			// answers through its report, the one thing it already says — no reply to anybody's inbox.
+			AskReportSubject(p.Node)}
 
 	case KindModule:
 		// 1. Its own namespace: it publishes its events there and serves its tools there. Nothing

internal/broker/states_agreement_test.go

@@ -26,6 +26,8 @@ func TestTheFactsTheGrantPermitsAreTheFactsTheMeshStates(t *testing.T) {
 	states = append(states, conditions.HeartbeatEvent)
 	// And a value given by hand, replaced after its module's first good start (novox/hq ADR 0228).
 	states = append(states, link.KeySecretReplaced)
+	// And every act a healer takes (novox/hq to-be 45 §7).
+	states = append(states, link.KeyHealerActed)
 	for _, event := range states {
 		if !slices.Contains(broker.ControllerStates, event) {
 			t.Errorf("the mesh states %q and its account may not publish it", event)

internal/broker/streams.go

@@ -210,7 +210,10 @@ var ControllerStates = []string{"applied", "refused", "built-before",
 	// machine that is not the control node, so the controller going quiet is itself said.
 	"doctor-heartbeat",
 	// And a value given by hand, replaced after its module's first good start (novox/hq ADR 0228).
-	"secret-replaced"}
+	"secret-replaced",
+	// And every act a healer takes on a condition (novox/hq to-be 45 §7, Phase 3): a repair the mesh
+	// made by itself is said like one a person made, never quietly.
+	"healer-acted"}
 
 // BusAdvisories are what the bus server says about the mesh's own account that the controller
 // reads (novox/hq to-be 45 §3, S9): a durable consumer that handed a message over as often as it
@@ -304,6 +307,13 @@ func MeshConsumers() []Consumer {
 			// client and come back to be acted on again.
 			MaxAckPending: 1,
 			FromNow:       true,
+			// **The one consumer the mesh resets by itself** (healer H4, issue 248): it fell a week
+			// behind once and held every merge after it. What a reset drops is caught up elsewhere —
+			// a merge by the catch-up pass that reads the forge (issue 266), a build's outcome from the
+			// build records a plan settles from (issue 214), a provider's failing word by the provider
+			// saying it again every quarter of an hour (ADR 0224).
+			Resettable: "what it drops is caught up: merges by the catch-up pass (issue 266), build outcomes " +
+				"from the build records (issue 214), a provider's failing word said again (ADR 0224)",
 			Why: "the events the mesh's own controller reacts to, one at a time; after " +
 				"max-deliver it dead-letters, because an announcement it cannot act on will not " +
 				"become actionable",

internal/broker/testdata/composed.conf

@@ -24,7 +24,7 @@ accounts {
     jetstream: enabled
     users = [
       { user: "controller", password: "$2a$11$cccccccccccccccccccccc", permissions: {
-        publish: { allow: ["$JS.ACK.CONTROL.controller.>", "$JS.ACK.EVENTS.controller.>", "$JS.API.>", "$KV.SEAT_MESH_BUILD_MACHINE_cancelled.>", "$KV.SEAT_NODE_BUILD_AGENT_cancelled.>", "$KV.mesh-controller_calls.>", "$KV.mesh-controller_condition-history.>", "$KV.mesh-controller_conditions.>", "$KV.mesh-controller_hand-acts.>", "$KV.mesh-controller_lease.>", "$SRV.INFO", "_INBOX.enrol.>", "mesh.assignment.>", "mesh.mod.*.tool.>", "mesh.node.>", "mesh.seat.mesh-build-machine.accept.>", "mesh.seat.mesh-build-machine.tool.>", "mesh.seat.mesh-controller.event.applied", "mesh.seat.mesh-controller.event.built-before", "mesh.seat.mesh-controller.event.condition-changed", "mesh.seat.mesh-controller.event.condition-cleared", "mesh.seat.mesh-controller.event.condition-raised", "mesh.seat.mesh-controller.event.doctor-heartbeat", "mesh.seat.mesh-controller.event.refused", "mesh.seat.mesh-controller.event.secret-replaced", "mesh.seat.node-build-agent.accept.>", "mesh.seat.node-build-agent.tool.>", "mesh.seat.node-intrusion-prevention.tool.banned.*"] }
+        publish: { allow: ["$JS.ACK.CONTROL.controller.>", "$JS.ACK.EVENTS.controller.>", "$JS.API.>", "$KV.SEAT_MESH_BUILD_MACHINE_cancelled.>", "$KV.SEAT_NODE_BUILD_AGENT_cancelled.>", "$KV.mesh-controller_calls.>", "$KV.mesh-controller_condition-history.>", "$KV.mesh-controller_conditions.>", "$KV.mesh-controller_hand-acts.>", "$KV.mesh-controller_lease.>", "$SRV.INFO", "_INBOX.enrol.>", "mesh.assignment.>", "mesh.mod.*.tool.>", "mesh.node.>", "mesh.seat.mesh-build-machine.accept.>", "mesh.seat.mesh-build-machine.tool.>", "mesh.seat.mesh-controller.event.applied", "mesh.seat.mesh-controller.event.built-before", "mesh.seat.mesh-controller.event.condition-changed", "mesh.seat.mesh-controller.event.condition-cleared", "mesh.seat.mesh-controller.event.condition-raised", "mesh.seat.mesh-controller.event.doctor-heartbeat", "mesh.seat.mesh-controller.event.healer-acted", "mesh.seat.mesh-controller.event.refused", "mesh.seat.mesh-controller.event.secret-replaced", "mesh.seat.node-build-agent.accept.>", "mesh.seat.node-build-agent.tool.>", "mesh.seat.node-intrusion-prevention.tool.banned.*"] }
         subscribe: { allow: ["$JS.API.>", "$JS.EVENT.ADVISORY.CONSUMER.DELETED.>", "$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES.>", "$SRV.INFO", "$SRV.INFO.mesh-controller", "$SRV.INFO.mesh-controller.>", "$SRV.PING", "$SRV.PING.mesh-controller", "$SRV.PING.mesh-controller.>", "$SRV.STATS", "$SRV.STATS.mesh-controller", "$SRV.STATS.mesh-controller.>", "_DELIVER.controller", "_DELIVER.controller.>", "_INBOX.controller.>", "mesh.control.>", "mesh.mod.*.event.provisioner.failing", "mesh.mod.*.event.provisioner.recovered", "mesh.mod.gitea.event.pull.merged", "mesh.mod.mesh-catalog.event.catching-up", "mesh.mod.mesh-catalog.event.upgraded", "mesh.seat.mesh-build-machine.event.built", "mesh.seat.mesh-controller.tool.>", "mesh.seat.node-build-agent.event.built"] }
         allow_responses: { max: 1, ttl: "1m" }
       } }
@@ -34,7 +34,7 @@ accounts {
       } }
       { user: "node.one", password: "$2a$11$nnnnnnnnnnnnnnnnnnnnnn", permissions: {
         publish: { allow: ["$JS.ACK.NODES.one.>", "$JS.API.CONSUMER.INFO.NODES.one", "mesh.control.one.>"] }
-        subscribe: { allow: ["_DELIVER.one", "_DELIVER.one.>", "_INBOX.node.one.>", "mesh.node.one.declare"] }
+        subscribe: { allow: ["_DELIVER.one", "_DELIVER.one.>", "_INBOX.node.one.>", "mesh.node.one.ask.report", "mesh.node.one.declare"] }
       } }
       { user: "one.telegram", password: "$2a$11$tttttttttttttttttttttt", permissions: {
         publish: { allow: ["$JS.ACK.EVENTS.one_telegram.>", "$JS.ACK.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker.>", "$JS.API.CONSUMER.INFO.EVENTS.one_telegram", "$JS.API.CONSUMER.INFO.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.one_telegram", "$JS.API.CONSUMER.MSG.NEXT.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker", "$JS.API.DIRECT.GET.ASSIGNMENTS.mesh.assignment.one.telegram", "mesh.seat.telegram-sender.event.delivered", "mesh.seat.telegram-sender.event.failed"] }

internal/broker/writers.go

@@ -98,6 +98,10 @@ var WritersTable = []WriterRow{
 		Others: "read by id", Subjects: kvOf(CallsBucket), Writes: isController},
 	{State: "the hand-act log", Writer: "controller, through the verbs that act", KeptIn: "key-value " + HandActsBucket,
 		Others: "—", Subjects: kvOf(HandActsBucket), Writes: isController},
+	// What the healers did (novox/hq to-be 45 §7, Phase 3): the controller's alone, in its store — the
+	// budgets and the mesh-wide brake are counted from it, so a controller restarting cannot reset them.
+	{State: "the healers' acts and their brake", Writer: "controller (lease holder), each act begun before it is made",
+		KeptIn: "the controller's store", Others: "read through healers; each act said as healer-acted and in its condition's tried"},
 	{State: "stream definitions and bus permissions", Writer: "controller", KeptIn: "the bus", Others: "—",
 		// A stream's definition, and a durable consumer's by the API that names it so. Not every
 		// consumer create: a module watching its own bucket makes and deletes an ordered consumer on the

internal/broker/writers_test.go

@@ -21,6 +21,7 @@ var designRows = []string{
 	"conditions",
 	"calls and their outcomes",
 	"the hand-act log",
+	"the healers' acts and their brake",
 	"stream definitions and bus permissions",
 	"builds and their outcomes",
 	"a merge announced",

internal/catalogue/seats.go

@@ -87,7 +87,9 @@ var defaultSeats = append([]Seat{
 		Emits: []string{"applied", "refused", "built-before",
 			"condition-raised", "condition-changed", "condition-cleared", "doctor-heartbeat",
 			// A value given by hand, replaced after its module's first good start (novox/hq ADR 0228).
-			"secret-replaced"},
+			"secret-replaced",
+			// Every act a healer takes (novox/hq to-be 45 §7).
+			"healer-acted"},
 		Serves: ControllerVerbs},
 	// The store's first verbs (novox/hq ADR 0159): the smallest set that makes the store askable,
 	// served by whichever module holds the seat with tools of these names.

internal/catalogue/verbs.go

@@ -252,6 +252,12 @@ var ControllerVerbs = []Verb{
 			"why":      "with silence: why — required, and recorded in the hand-act log",
 			"cause":    "with silence: the cause in a word (the condition's kind when absent)",
 		}, nil, "history")},
+	// What the healers did (novox/hq to-be 45 §7).
+	{Name: "healers", Description: "The healers (novox/hq to-be 45 §7): each registered response to one kind " +
+		"of condition — its repair (the ordinary path again), its budget, what happens when it is spent — every act " +
+		"they took lately with its outcome, and the mesh-wide brake. A heal is never a hand act; whether it " +
+		"repaired anything is the condition's clearing to say. Each act is also in its condition's tried.",
+		Input: schema(map[string]string{"days": "how many days of acts back (default 7)"}, nil)},
 	{Name: "doctor", Description: "The self-check (novox/hq to-be 45 §4): the last run's verdict at once — " +
 		"each probe of the design's live invariants passed, failed or could not run, and how long ago. With " +
 		"run, a run now; with probes, the registry; with signals, every row of the signals table and the age " +

internal/conditions/condition.go

@@ -79,13 +79,21 @@ type Evidence struct {
 	Said string    `json:"said"`
 }
 
-// Attempt is one healer's try at a condition (to-be 45 §7; written from Phase 3).
+// Attempt is one healer's try at a condition (to-be 45 §7): when, what it did, what came of it, and
+// which healer — so a person reading `conditions show` sees the mesh repairing itself, by whom.
 type Attempt struct {
 	At      time.Time `json:"at"`
 	What    string    `json:"what"`
 	Outcome string    `json:"outcome"`
+	// By is the healer, as a person reads it: `healer H1`. Never a person: an act by hand is the
+	// hand-act log's, not a condition's attempt.
+	By string `json:"by"`
 }
 
+// KeptAttempts is how many attempts a condition keeps, newest last: a healer's budget is a few, and
+// a condition reopened again and again carries its tries forward only so far.
+const KeptAttempts = 10
+
 // Silence is a person saying they know: no messages until it ends (to-be 45 §2). Recorded as a hand
 // act; the condition stays open, and `status` still says it.
 type Silence struct {
@@ -139,6 +147,11 @@ func (c Condition) SilencedAt(now time.Time) bool {
 	return c.Silenced != nil && now.Before(c.Silenced.Until)
 }
 
+// Escalated says a healer tried and its budget is spent: the operator resolves it now (to-be 45 §2,
+// "budget spent ──► OPEN, resolver: operator, severity: urgent"). An observation does not lower its
+// severity again: the watchdog that sees it every half minute would otherwise undo the escalation.
+func (c Condition) Escalated() bool { return c.Resolver == ResolverOperator && len(c.Tried) > 0 }
+
 // Show is the verb that shows more about a condition, as a message carries it.
 func (c Condition) Show() string { return "mesh-controller.conditions key=" + c.Key }
 

internal/conditions/store.go

@@ -76,6 +76,9 @@ type clearing struct {
 	at       time.Time
 	count    int
 	silenced *Silence
+	// tried is what healers tried before it cleared: a reopening is the same fault, and what was
+	// tried on it is still what was tried.
+	tried []Attempt
 }
 
 // Options are what a Keeper is made with.
@@ -113,7 +116,8 @@ func NewKeeper(ctx context.Context, o Options) *Keeper {
 		if recent, err := k.history.Since(ctx, k.now().Add(-ReopenWithin)); err == nil {
 			for _, e := range recent {
 				if e.Change == ChangeCleared {
-					k.cleared[e.Key] = clearing{at: e.At, count: e.Condition.Count, silenced: e.Condition.Silenced}
+					k.cleared[e.Key] = clearing{at: e.At, count: e.Condition.Count, silenced: e.Condition.Silenced,
+						tried: e.Condition.Tried}
 				}
 			}
 		} else {
@@ -191,6 +195,7 @@ func (k *Keeper) Observe(ctx context.Context, o Observation) (Condition, error)
 				if before.silenced != nil && now.Before(before.silenced.Until) {
 					c.Silenced = before.silenced
 				}
+				c.Tried = before.tried
 			}
 			k.mu.Unlock()
 			if err := k.stamp(&c); err != nil {
@@ -216,7 +221,8 @@ func (k *Keeper) Observe(ctx context.Context, o Observation) (Condition, error)
 			return Condition{}, fmt.Errorf("the condition %s on the bus cannot be read: %w", key, err)
 		}
 		var changes []Event
-		if o.Severity != c.Severity {
+		// A healer's budget spent made it urgent; the watchdog seeing it again does not undo that.
+		if o.Severity != c.Severity && !(c.Escalated() && o.Severity != Urgent) {
 			changes = append(changes, Event{Change: ChangeSeverity, Was: string(c.Severity)})
 			c.Severity = o.Severity
 		}
@@ -224,7 +230,9 @@ func (k *Keeper) Observe(ctx context.Context, o Observation) (Condition, error)
 			changes = append(changes, Event{Change: ChangeResolver, Was: c.Resolver})
 			c.Resolver = r
 		}
-		c.Summary, c.Source, c.LastObserved = o.Summary, o.Source, now
+		// The kind as the source says it now: a source that gave the same key a kind of its own since
+		// (a probe's finding split out for a healer) is read by that kind from its next observation.
+		c.Kind, c.Summary, c.Source, c.LastObserved = o.Kind, o.Summary, o.Source, now
 		if o.Machine != "" {
 			c.Subject.Machine = o.Machine
 		}
@@ -282,7 +290,7 @@ func (k *Keeper) Clear(ctx context.Context, key, why string) (bool, error) {
 		}
 		now := k.now().UTC()
 		k.mu.Lock()
-		k.cleared[key] = clearing{at: now, count: c.Count, silenced: c.Silenced}
+		k.cleared[key] = clearing{at: now, count: c.Count, silenced: c.Silenced, tried: c.Tried}
 		k.mu.Unlock()
 		k.tell(Event{Condition: c, At: now, Change: ChangeCleared, Why: why, Cleared: &now})
 		return true, nil
@@ -290,6 +298,91 @@ func (k *Keeper) Clear(ctx context.Context, key, why string) (bool, error) {
 	return false, fmt.Errorf("the condition %s kept moving under this clearing; %d tries", key, tries)
 }
 
+// Tried records a healer's attempt on an open condition (to-be 45 §7) and makes the healer its
+// resolver: said as `condition-changed` when the resolver changes, kept in `tried` either way. **It
+// never clears the condition**: a repair that worked is seen by the observation that raised it, which
+// clears it — a healer marking its own work done would be a second opinion of the fact. False when no
+// condition is open under the key: it cleared meanwhile, and there is nothing to record against.
+func (k *Keeper) Tried(ctx context.Context, key string, a Attempt, resolver string) (Condition, bool, error) {
+	return k.amend(ctx, key, func(c *Condition, now time.Time) []Event {
+		c.Tried = appendAttempt(c.Tried, a, now)
+		var changes []Event
+		if resolver != "" && resolver != c.Resolver && !c.Escalated() {
+			changes = append(changes, Event{Change: ChangeResolver, Was: c.Resolver, Why: a.Outcome})
+			c.Resolver = resolver
+		}
+		return changes
+	})
+}
+
+// Escalate is a healer's budget spent, or a repair it may not make (to-be 45 §2, §7): the attempt that
+// says so is kept in `tried`, the resolver becomes the operator and the severity urgent, each said as
+// `condition-changed`. Observation still clears it when the fault goes; nothing else does.
+func (k *Keeper) Escalate(ctx context.Context, key string, a Attempt) (Condition, bool, error) {
+	return k.amend(ctx, key, func(c *Condition, now time.Time) []Event {
+		c.Tried = appendAttempt(c.Tried, a, now)
+		var changes []Event
+		if c.Severity != Urgent {
+			changes = append(changes, Event{Change: ChangeSeverity, Was: string(c.Severity), Why: a.Outcome})
+			c.Severity = Urgent
+		}
+		if c.Resolver != ResolverOperator {
+			changes = append(changes, Event{Change: ChangeResolver, Was: c.Resolver, Why: a.Outcome})
+			c.Resolver = ResolverOperator
+		}
+		return changes
+	})
+}
+
+// appendAttempt adds an attempt, stamped when it has no time, keeping the newest KeptAttempts.
+func appendAttempt(tried []Attempt, a Attempt, now time.Time) []Attempt {
+	if a.At.IsZero() {
+		a.At = now
+	}
+	tried = append(tried, a)
+	if len(tried) > KeptAttempts {
+		tried = tried[len(tried)-KeptAttempts:]
+	}
+	return tried
+}
+
+// amend changes one open condition by compare-and-set and says what changed. False when none is open.
+func (k *Keeper) amend(ctx context.Context, key string, change func(*Condition, time.Time) []Event) (Condition, bool, error) {
+	for i := 0; i < tries; i++ {
+		entry, found, err := k.store.Get(ctx, key)
+		if err != nil {
+			return Condition{}, false, fmt.Errorf("reading the condition %s: %w", key, err)
+		}
+		if !found {
+			return Condition{}, false, nil
+		}
+		var c Condition
+		if err := json.Unmarshal(entry.Value, &c); err != nil {
+			return Condition{}, false, fmt.Errorf("the condition %s on the bus cannot be read: %w", key, err)
+		}
+		now := k.now().UTC()
+		changes := change(&c, now)
+		if err := k.stamp(&c); err != nil {
+			return Condition{}, false, err
+		}
+		body, err := json.Marshal(c)
+		if err != nil {
+			return Condition{}, false, err
+		}
+		if err := k.store.Update(ctx, key, body, entry.Revision); errors.Is(err, ErrMoved) {
+			continue
+		} else if err != nil {
+			return Condition{}, false, fmt.Errorf("writing the condition %s: %w", key, err)
+		}
+		for _, e := range changes {
+			e.At, e.Condition = now, c
+			k.tell(e)
+		}
+		return c, true, nil
+	}
+	return Condition{}, false, fmt.Errorf("the condition %s kept moving under this write; %d tries", key, tries)
+}
+
 // Reconcile is one source's whole observation: every condition it observes is observed, and every
 // condition it raised before and no longer observes is cleared — the observation says it is
 // resolved. A source that could not observe must not call this: an empty observation clears all it

internal/conditions/tried_test.go

@@ -0,0 +1,105 @@
+package conditions
+
+import (
+	"testing"
+	"time"
+)
+
+// **A healer's attempt is said and kept, and never clears the condition** (to-be 45 §7): the resolver
+// becomes the healer, `tried` says what it did, and only an observation clears it.
+func TestAHealersAttemptIsKeptAndClearsNothing(t *testing.T) {
+	k, _, told, c := keeper(t)
+	ctx := t.Context()
+	if _, err := k.Observe(ctx, silent("ace")); err != nil {
+		t.Fatal(err)
+	}
+	held, open, err := k.Tried(ctx, "machine.ace.silent", Attempt{What: "asked ace to report", Outcome: "acted",
+		By: "healer H1"}, ResolverHealer("H1"))
+	if err != nil || !open {
+		t.Fatalf("tried: %v %v", open, err)
+	}
+	if held.Resolver != "healer:H1" || len(held.Tried) != 1 || held.Tried[0].By != "healer H1" || held.Tried[0].At.IsZero() {
+		t.Fatalf("after one attempt: %+v", held)
+	}
+	said := settled(t, told, 2)
+	if said[1].Event != EventChanged || said[1].Change != ChangeResolver || said[1].Was != ResolverSelf {
+		t.Errorf("the healer taking it is not said as a change of resolver: %+v", said[1])
+	}
+	// A second attempt by the same healer is kept and says nothing new.
+	if _, _, err := k.Tried(ctx, "machine.ace.silent", Attempt{What: "sent again", Outcome: "acted", By: "healer H1"},
+		ResolverHealer("H1")); err != nil {
+		t.Fatal(err)
+	}
+	if got, _, _ := k.Get(ctx, "machine.ace.silent"); len(got.Tried) != 2 {
+		t.Errorf("tried %+v", got.Tried)
+	}
+	time.Sleep(50 * time.Millisecond)
+	if n := len(told.Said()); n != 2 {
+		t.Errorf("a second attempt said %d events in all, want 2", n)
+	}
+	// Nothing open: nothing recorded, and no error.
+	if _, open, err := k.Tried(ctx, "machine.nobody.silent", Attempt{What: "x"}, ResolverHealer("H1")); open || err != nil {
+		t.Errorf("an attempt on nothing open: %v %v", open, err)
+	}
+	c.pass(time.Minute)
+}
+
+// **A spent budget is the operator's, urgent, and the watchdog seeing it again does not undo that.**
+func TestAnEscalationHoldsAgainstTheNextObservation(t *testing.T) {
+	k, _, told, _ := keeper(t)
+	ctx := t.Context()
+	if _, err := k.Observe(ctx, silent("ace")); err != nil {
+		t.Fatal(err)
+	}
+	if _, _, err := k.Tried(ctx, "machine.ace.silent", Attempt{What: "asked", Outcome: "acted", By: "healer H1"},
+		ResolverHealer("H1")); err != nil {
+		t.Fatal(err)
+	}
+	held, _, err := k.Escalate(ctx, "machine.ace.silent", Attempt{What: "budget spent", Outcome: "escalated", By: "healer H1"})
+	if err != nil {
+		t.Fatal(err)
+	}
+	if held.Severity != Urgent || held.Resolver != ResolverOperator || !held.Escalated() {
+		t.Fatalf("escalated: %+v", held)
+	}
+	said := settled(t, told, 4)
+	if said[2].Change != ChangeSeverity || said[3].Change != ChangeResolver || said[3].Was != "healer:H1" {
+		t.Errorf("the escalation is not said as severity then resolver: %+v %+v", said[2], said[3])
+	}
+	again, err := k.Observe(ctx, silent("ace")) // a warning, as the watchdog says it
+	if err != nil {
+		t.Fatal(err)
+	}
+	if again.Severity != Urgent || again.Resolver != ResolverOperator {
+		t.Errorf("the next observation undid the escalation: %+v", again)
+	}
+	// A healer trying later does not take it back from the operator.
+	after, _, _ := k.Tried(ctx, "machine.ace.silent", Attempt{What: "x", By: "healer H1"}, ResolverHealer("H1"))
+	if after.Resolver != ResolverOperator {
+		t.Errorf("a later attempt took the condition back from the operator: %s", after.Resolver)
+	}
+}
+
+// **What was tried is carried into a reopening**: the same fault again within ten minutes is the
+// same condition, and what the healers tried on it is still what they tried.
+func TestAReopeningCarriesWhatWasTried(t *testing.T) {
+	k, _, _, c := keeper(t)
+	ctx := t.Context()
+	if _, err := k.Observe(ctx, silent("ace")); err != nil {
+		t.Fatal(err)
+	}
+	if _, _, err := k.Tried(ctx, "machine.ace.silent", Attempt{What: "asked", By: "healer H1"}, ResolverHealer("H1")); err != nil {
+		t.Fatal(err)
+	}
+	if _, err := k.Clear(ctx, "machine.ace.silent", "heard again"); err != nil {
+		t.Fatal(err)
+	}
+	c.pass(5 * time.Minute)
+	again, err := k.Observe(ctx, silent("ace"))
+	if err != nil {
+		t.Fatal(err)
+	}
+	if again.Count != 2 || len(again.Tried) != 1 || again.Tried[0].What != "asked" {
+		t.Errorf("reopened without what was tried: %+v", again)
+	}
+}

internal/inventory/heals.go

@@ -0,0 +1,126 @@
+package inventory
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"strings"
+	"time"
+)
+
+// The healers' acts (novox/hq to-be 45 §7, Phase 3): see migration 0070. The controller is their
+// only writer, under its lease; `healers` and `conditions` read them.
+
+// The outcomes of a heal.
+const (
+	// HealActing is an act begun and not yet finished: counted against the budget and the brake from
+	// the moment it starts, so a controller that dies mid-act has still spent it.
+	HealActing = "acting"
+	// HealActed is an act that did what it does. Whether it repaired anything is the observation's to
+	// say, never the healer's: the condition clears, or it does not.
+	HealActed = "acted"
+	// HealFailed is an act that could not be done.
+	HealFailed = "failed"
+	// HealEscalated is a budget spent, said: the condition was handed to the operator.
+	HealEscalated = "escalated"
+)
+
+// HealsKeptFor is how long a heal is kept: as long as a condition's history.
+const HealsKeptFor = 90 * 24 * time.Hour
+
+// Heal is one act of a healer.
+type Heal struct {
+	ID           int64      `json:"id"`
+	Healer       string     `json:"healer"`
+	ConditionKey string     `json:"condition"`
+	Kind         string     `json:"kind"`
+	BudgetKey    string     `json:"budget-key"`
+	Act          string     `json:"act"`
+	Outcome      string     `json:"outcome"`
+	Said         string     `json:"said,omitempty"`
+	At           time.Time  `json:"at"`
+	Finished     *time.Time `json:"finished,omitempty"`
+	Epoch        uint64     `json:"epoch,omitempty"`
+}
+
+// BeginHeal writes an act about to be taken, under the lease: the act is counted from here. Refused,
+// and nothing written, by a process that may not act.
+func (i *Inventory) BeginHeal(ctx context.Context, h Heal) (Heal, error) {
+	epoch, err := i.actingEpoch(ctx)
+	if err != nil {
+		return h, fmt.Errorf("the heal %s of %s is not begun: %w", h.Healer, h.ConditionKey, err)
+	}
+	if strings.TrimSpace(h.Healer) == "" || strings.TrimSpace(h.ConditionKey) == "" || strings.TrimSpace(h.Act) == "" {
+		return h, errors.New("a heal names its healer, its condition and its act")
+	}
+	if h.BudgetKey == "" {
+		h.BudgetKey = h.ConditionKey
+	}
+	if h.Outcome == "" {
+		h.Outcome = HealActing
+	}
+	// At the runner's moment when it gives one, so its budgets and the brake are counted on one clock.
+	var at *time.Time
+	if !h.At.IsZero() {
+		at = &h.At
+	}
+	err = i.store.Pool().QueryRow(ctx,
+		`insert into heal (healer, condition_key, kind, budget_key, act, outcome, said, epoch, at)
+		 values ($1, $2, $3, $4, $5, $6, $7, $8, coalesce($9, now())) returning id, at`,
+		h.Healer, h.ConditionKey, h.Kind, h.BudgetKey, h.Act, h.Outcome, h.Said, epoch, at).Scan(&h.ID, &h.At)
+	if err != nil {
+		return h, err
+	}
+	if epoch != nil {
+		h.Epoch = uint64(*epoch)
+	}
+	return h, nil
+}
+
+// FinishHeal says what came of an act begun. Written whatever the lease says by now: what was done was
+// done, and its record must not depend on the doer still holding the lease a moment later.
+func (i *Inventory) FinishHeal(ctx context.Context, id int64, outcome, said string) error {
+	tag, err := i.store.Pool().Exec(ctx,
+		`update heal set outcome = $2, said = $3, finished = now() where id = $1`, id, outcome, said)
+	if err != nil {
+		return err
+	}
+	if tag.RowsAffected() != 1 {
+		return fmt.Errorf("no heal %d to finish", id)
+	}
+	return nil
+}
+
+// HealsSince is every heal from a moment, oldest first.
+func (i *Inventory) HealsSince(ctx context.Context, since time.Time) ([]Heal, error) {
+	rows, err := i.store.Pool().Query(ctx,
+		`select id, healer, condition_key, kind, budget_key, act, outcome, said, at, finished, epoch
+		   from heal where at >= $1 order by at, id`, since)
+	if err != nil {
+		return nil, err
+	}
+	defer rows.Close()
+	var out []Heal
+	for rows.Next() {
+		var h Heal
+		var epoch *int64
+		if err := rows.Scan(&h.ID, &h.Healer, &h.ConditionKey, &h.Kind, &h.BudgetKey, &h.Act, &h.Outcome, &h.Said,
+			&h.At, &h.Finished, &epoch); err != nil {
+			return nil, err
+		}
+		if epoch != nil {
+			h.Epoch = uint64(*epoch)
+		}
+		out = append(out, h)
+	}
+	return out, rows.Err()
+}
+
+// ForgetOldHeals removes what is older than HealsKeptFor, and says how many.
+func (i *Inventory) ForgetOldHeals(ctx context.Context) (int64, error) {
+	tag, err := i.store.Pool().Exec(ctx, `delete from heal where at < $1`, time.Now().Add(-HealsKeptFor))
+	if err != nil {
+		return 0, err
+	}
+	return tag.RowsAffected(), nil
+}

internal/inventory/heals_test.go

@@ -0,0 +1,43 @@
+package inventory
+
+import (
+	"context"
+	"errors"
+	"testing"
+	"time"
+)
+
+// **A heal is begun under the lease and kept with its epoch** (novox/hq to-be 45 §7): refused, and
+// nothing written, by a process that may not act; finished whatever the lease says by then; read back
+// in the order they were taken.
+func TestAHealIsBegunUnderTheLeaseAndFinishedAfter(t *testing.T) {
+	inv := ForTest(t)
+	ctx := t.Context()
+	inv.ActsUnder(func(context.Context) (uint64, error) { return 0, errors.New("the lease is not held") })
+	if _, err := inv.BeginHeal(ctx, Heal{Healer: "H1", ConditionKey: "machine.anchor.sent-not-reported", Act: "asked"}); err == nil {
+		t.Fatal("a heal was begun by a process that may not act")
+	}
+	inv.ActsUnder(func(context.Context) (uint64, error) { return 57, nil })
+	at := time.Now().Add(-time.Minute).UTC().Truncate(time.Microsecond)
+	first, err := inv.BeginHeal(ctx, Heal{Healer: "H1", ConditionKey: "machine.anchor.sent-not-reported",
+		Kind: "sent-not-reported", Act: "asked", At: at})
+	if err != nil || first.ID == 0 || first.Epoch != 57 || first.BudgetKey != first.ConditionKey ||
+		first.Outcome != HealActing || !first.At.Equal(at) {
+		t.Fatalf("begun: %+v %v", first, err)
+	}
+	if _, err := inv.BeginHeal(ctx, Heal{Healer: "H3"}); err == nil {
+		t.Fatal("a heal naming no condition and no act was begun")
+	}
+	inv.ActsUnder(func(context.Context) (uint64, error) { return 0, errors.New("lost meanwhile") })
+	if err := inv.FinishHeal(ctx, first.ID, HealActed, "it reported"); err != nil {
+		t.Fatalf("what was done could not be recorded once the lease was gone: %v", err)
+	}
+	if err := inv.FinishHeal(ctx, first.ID+1000, HealActed, ""); err == nil {
+		t.Fatal("a heal that was never begun was finished")
+	}
+	heals, err := inv.HealsSince(ctx, time.Now().Add(-time.Hour))
+	if err != nil || len(heals) != 1 || heals[0].Outcome != HealActed || heals[0].Said != "it reported" ||
+		heals[0].Finished == nil || heals[0].Epoch != 57 {
+		t.Fatalf("read back: %+v %v", heals, err)
+	}
+}

internal/inventory/migrations/0070-a-known-failure-heals-itself.sql

@@ -0,0 +1,34 @@
+-- A known failure heals itself, under a brake, and every repair is said (novox/hq to-be 45 §7,
+-- Phase 3, ADR 0227 rule 7).
+--
+-- Every act a healer takes is a row here, written before the act and finished after it: what the
+-- budgets and the mesh-wide brake count, and what `healers` reads back. In the store and not in a
+-- bucket on the bus, because the budget must outlive the controller — a controller restarting in a
+-- loop must not reset a healer's count each time — and because the bus's user list would have to
+-- grant a new bucket before the first heal could be counted.
+--
+-- Numbered 0070, past 0069, while the consumer-retirement feature (ADR 0230) is built beside this one:
+-- a migration it adds takes 0069 if it merges first; one merged after this must be numbered above
+-- 0070, because the store refuses to run a lower number than one already run.
+create table heal (
+    id            bigserial   primary key,
+    -- The healer's id in the registry: H1, H2, ...
+    healer        text        not null,
+    -- The condition it acted on, and the condition's kind.
+    condition_key text        not null,
+    kind          text        not null,
+    -- What its budget is counted against: the condition, a machine, a holder, a consumer.
+    budget_key    text        not null,
+    -- What it did, in the mesh's words.
+    act           text        not null,
+    -- 'acting' while the act runs; then 'acted', 'failed' or 'escalated' (a budget spent, said).
+    outcome       text        not null default 'acting',
+    said          text        not null default '',
+    at            timestamptz not null default now(),
+    finished      timestamptz,
+    -- The lease epoch it acted under (to-be 45 §6); null where none was claimed.
+    epoch         bigint
+);
+
+create index heal_at on heal (at);
+create index heal_budget on heal (healer, budget_key, at);

internal/link/events.go

@@ -66,6 +66,10 @@ const (
 	// KeySecretReplaced: a value given to the mesh by hand was replaced with one it made, after its
 	// module's first good start (novox/hq ADR 0228). Never the value.
 	KeySecretReplaced = "secret-replaced"
+	// KeyHealerActed: a healer acted on a condition — what it did and what came of it, or that its budget
+	// is spent and the condition is the operator's (novox/hq to-be 45 §7). Never a person's act: those
+	// are the hand-act log's.
+	KeyHealerActed = "healer-acted"
 )
 
 // Applied is what a machine now runs, as the mesh states it.