From 3904ef88bf3a84ece7fc9908a050886af43eb4dc Mon Sep 17 00:00:00 2001 From: jochen Date: Tue, 6 Oct 2026 14:25:04 +0200 Subject: [PATCH] ADR 0230: a consumer the mesh stops asking for is retired, and deleted only by a person The operator's decision replaces ADR 0229's hourly release: the mesh no longer finishes alone what its bound exists to question. To-be 45 and the module protocol amended; 0229 and 0224 point to where their mechanism moved. --- ...mer-is-a-problem-the-controller-reports.md | 7 + ...och-a-machine-is-sent-once-it-reads-one.md | 7 + ...is-retired-and-deleted-only-by-a-person.md | 194 ++++++++++++++++++ 02-DECISIONS/README.md | 1 + 03-DESIGN/01-to-be/19-the-module-protocol.md | 14 ++ .../45-a-core-that-cannot-fail-silently.md | 36 +++- 6 files changed, 252 insertions(+), 7 deletions(-) create mode 100644 02-DECISIONS/0230-a-consumer-the-mesh-stops-asking-for-is-retired-and-deleted-only-by-a-person.md diff --git a/02-DECISIONS/0224-a-provider-that-keeps-failing-a-consumer-is-a-problem-the-controller-reports.md b/02-DECISIONS/0224-a-provider-that-keeps-failing-a-consumer-is-a-problem-the-controller-reports.md index e0cf01d..e1290fe 100644 --- a/02-DECISIONS/0224-a-provider-that-keeps-failing-a-consumer-is-a-problem-the-controller-reports.md +++ b/02-DECISIONS/0224-a-provider-that-keeps-failing-a-consumer-is-a-problem-the-controller-reports.md @@ -60,6 +60,13 @@ has forgotten it. - **No secret travels.** The error is the provider's own text with the consumer's password removed, as its log line already is. +> **The mechanism changed — 2026-10-06, by [ADR 0230](0230-a-consumer-the-mesh-stops-asking-for-is-retired-and-deleted-only-by-a-person.md).** +> What stands: a consumer the mesh stopped asking for is announced recovered. What moved: a consumer the +> provider still holds is no longer withdrawn on the first pass that misses it; it is said recovered when +> it is *retired*, after five passes or a person's approval. A provider says what it retires, approves, +> re-enables and deletes on an event of its own, `provisioner.retirement`, permitted the same way as the +> two events here. + **2. Every provider may say it, whatever its manifest lists.** The permission to publish the two events is derived for every module that receives contributions; no manifest declares them. A provider whose manifest forgot them would otherwise have its announcement refused by the bus and fail as silently as diff --git a/02-DECISIONS/0229-the-cores-order-is-a-lease-the-store-remembers-and-an-epoch-a-machine-is-sent-once-it-reads-one.md b/02-DECISIONS/0229-the-cores-order-is-a-lease-the-store-remembers-and-an-epoch-a-machine-is-sent-once-it-reads-one.md index 2a58d1b..1e67d48 100644 --- a/02-DECISIONS/0229-the-cores-order-is-a-lease-the-store-remembers-and-an-epoch-a-machine-is-sent-once-it-reads-one.md +++ b/02-DECISIONS/0229-the-cores-order-is-a-lease-the-store-remembers-and-an-epoch-a-machine-is-sent-once-it-reads-one.md @@ -134,6 +134,13 @@ let go every hour, said. A consumer asked for again is kept. **The brake stops t intention**: an unassignment of many completes without a hand, a mistaken one costs at most one consumer an hour while the operator is told — and withdrawal destroys no data since issue 241. +> **Replaced in part — 2026-10-06, by [ADR 0230](0230-a-consumer-the-mesh-stops-asking-for-is-retired-and-deleted-only-by-a-person.md).** +> The paragraph above no longer stands; the operator decided against releasing one consumer an hour. +> A consumer the mesh stops asking for is now *retired* — disabled, reversibly, and marked to delete — +> once the same result holds for five passes; a set of more than three, or more than half of those held, +> waits for a person's `retire approve`; and only `cleanup delete` deletes. Every other decision in this +> record stands. The *how it is checked* row for the brake is replaced by 0230's. + **What Phase 1 decided while building** — [issue 270](../04-ISSUES/270-phase-1-watches-signals-that-come-later-and-hears-only-the-controllers-faults/00-report.md)'s decisions 1 to 6: the condition history as a bucket of its own, the events' shape, a key's last token, the probe DW, which deleted consumers are said, and the bounds the design left to the build — is taken diff --git a/02-DECISIONS/0230-a-consumer-the-mesh-stops-asking-for-is-retired-and-deleted-only-by-a-person.md b/02-DECISIONS/0230-a-consumer-the-mesh-stops-asking-for-is-retired-and-deleted-only-by-a-person.md new file mode 100644 index 0000000..6768866 --- /dev/null +++ b/02-DECISIONS/0230-a-consumer-the-mesh-stops-asking-for-is-retired-and-deleted-only-by-a-person.md @@ -0,0 +1,194 @@ +--- +topic: the mesh +status: accepted +date: 2026-10-06 +deciders: jochen +reconstructed: false +extends: 02-DECISIONS/0229-the-cores-order-is-a-lease-the-store-remembers-and-an-epoch-a-machine-is-sent-once-it-reads-one.md +--- + +# 230. A consumer the mesh stops asking for is retired, and deleted only by a person + +## Context + +**This is the operator's decision, taken on 2026-10-06**, the day the withdrawal brake of +[ADR 0229](0229-the-cores-order-is-a-lease-the-store-remembers-and-an-epoch-a-machine-is-sent-once-it-reads-one.md) +was built. The model below — the three states, the stable result, the threshold that hands the act to a +person, the cleanup verbs, the thirty-day reminder — is the operator's direction, recorded here as given. +What the record adds is the evidence it rests on and the facts the build had to settle. + +**What the brake did.** A provider's loop ([ADR 0224](0224-a-provider-that-keeps-failing-a-consumer-is-a-problem-the-controller-reports.md), +in the Go providers and the SDK's) that would withdraw more than one consumer in a pass, or more than +half of those it held, withdrew nothing and announced each kept consumer as failing; then, while the mesh +went on not asking, it let one go every hour, with no person involved. So a mistaken unassignment of +seven consumers still completed by itself in seven hours, and a person who did not read `status` in that +time lost all seven. + +**What withdrawal does today**, since [issue 241](../04-ISSUES/241-one-unreadable-grants-file-dropped-every-database-on-the-control-node/00-report.md): +nothing a provider withdraws is destroyed in the providers that issue names — the relational database +locks the role and keeps the database; the SQL Server provider disables the login; the document store +strips the user's roles; the object store revokes the key and keeps the bucket; the mail server disables +the mailbox; the forge prohibits the login; the analytics site is kept. But three things were still +wrong: + +- **The identity provider deleted the client.** Its withdrawal was a `DELETE` of the consumer's client, + outside issue 241's list; the secret, redirects and mappers went with it. +- **Nothing recorded that a withdrawal happened, or when.** A locked role is indistinguishable from one + an operator locked by hand. On the home server one consumer's login has been locked by an earlier + withdrawal, its database kept, with no record of when or why; on the control node six databases set + aside during issue 241's recovery sit under a dated name, which is the only record of them. +- **A restart forgot.** The loop withdrew only what *this process* had made. A consumer unassigned while + its provider was down was never withdrawn at all — its login stayed open, indefinitely. + +**And nothing ever deleted anything**, so withdrawn data accumulates with no surface that says it is +there. + +## Considered Options + +1. **Keep the hourly release.** Rejected by the operator: it lets the mesh finish, unattended, an act the + bound exists to question. Many changes at once means a person is at work, and that person can confirm + once. +2. **Delete after a grace period.** Rejected: a timer is the mesh acting alone again, only later; the + loss in issue 241 was data nobody had decided to lose. +3. **Withdraw at once, below the bound, as before.** Rejected: one pass is one read of one file, and + issue 241 was one bad read. A result has to hold before it is acted on. +4. **Three states — active, retired, deleted — where retiring is reversible and automatic only when + stable and small, and deleting is always a person's act through the controller, executed by the + provider.** Chosen. + +## Decision + +**1. A consumer is ACTIVE, RETIRED or DELETED.** + +- **Retired**: the mesh stopped asking for it. The provider **disables its access** — reversibly — and + **marks its login and data "to delete", with when and why**. Nothing is deleted. Asked for again, the + provider's ordinary create re-enables it at once, as it was, and clears the mark. +- **Deleted**: only through `cleanup delete`, after a person approved it. + +**2. What "disable" means, per backend** — each the smallest reversible switch that stops the consumer +reaching its data, keeping everything else: + +| Provider | Retired is | Kept | Mark | +|---|---|---|---| +| postgres | the role set `NOLOGIN`, its open sessions ended | the database, its owner and grants, every byte; `CONNECT` is not revoked and the database still accepts connections, so the nightly dump still backs it up | the role's comment, a JSON object naming the consumer's machine, when and why | +| keycloak, the identity provider | the client `enabled: false` — the server refuses its authorization and token requests | its secret, redirects, mappers and every other setting | the client's attributes `mesh.retired` and `mesh.retired-why` | +| the TypeScript providers | what their `remove` already does since issue 241 (a key revoked, a login disabled, roles stripped, a mailbox disabled), until each adds a `retire` of its own | as issue 241 lists | none yet: a TypeScript provider does not list or delete retired consumers until its adapter can | + +A database renamed aside — by postgres's `postgres_retire_database` tool or by hand, the +`_deleted_` form — is listed beside the retired consumers, retired since that date, so a +person sees it and can delete it. + +**3. Stable removals.** A provider retires a consumer only after it has seen **the same set** of +consumers no longer asked for in **five consecutive passes** that read the contributions file. A pass +that could not read it is not a result and starts the count again; so does a different set. At the loop's +pass interval of **five seconds**, retirement comes **twenty seconds** after the first pass that saw the +set — twenty to twenty-five seconds after the file changed. **Additions and changes to an existing +consumer — create, rotate — act on the first pass and are never delayed.** + +**4. Too many is a person.** A stable set of **more than three consumers, or of more than half of those +the provider holds where it holds more than one**, retires nothing. The provider **waits**: it says so in +its journal and announces it, the controller raises an **urgent** condition listing what would be +retired, and it waits for `retire approve ` or `retire reject +`. **An unassignment the controller itself made goes through the same threshold** — the +operator's words: *too many changes means a human is actively working on it*, so one confirmation. The +bound is the old brake's half, with three in place of one: one, two or three consumers of a larger +provider go without a hand; two of two do not. + +- **Approve** retires exactly the set waiting — the provider refuses any other set, so a person approves + what they were shown. +- **Reject** keeps the set active; the provider does not ask about it again while the set stays the same. + The rejection is held by the provider's process: a restart asks again, loudly. A rejected set can still + be approved later. +- **Settled**: a waiting or rejected set ends without retiring when the mesh asks for any of it again or + the set changes; a new stable set starts the count over. + +**5. The backend remembers, not the process.** Each provider lists from its own backend what the mesh +made, active and retired, with the mark. So a restart forgets nothing; a consumer the backend holds +active and the mesh no longer asks for is retired by the same rules after a restart as before one; and a +consumer **found disabled without the mark** — withdrawn before this record — is **adopted** as retired: +marked, said, announced, its clock starting then. What the mesh made is known by the mark, or, before +the mark existed, by its shape (postgres: a role valid until *infinity*, which only the +mesh's role statement sets, owning a database of its own name; keycloak: the +`mesh.provisioned` attribute it already had). Anything else is somebody else's and is never listed, +retired or deleted. + +**6. Cleanup is the controller's verbs, executed by the provider.** + +- `cleanup list` — every retired consumer per provider: its age, its size where the backend can say, + and why. +- `cleanup delete --why` — one. +- `cleanup delete --older-than --why` — lists what it would delete; deletes only with the + operator's explicit `--confirm`. +- **The provider deletes**, because it owns its backend: the controller asks the provider's tool on that + machine through the mesh, as any module's tool is asked, and never touches a backend itself. A provider + refuses to delete anything active or asked for, and anything not retired. +- **Every deletion is written to the hand-act log**, as is every approval and rejection — including one + made by asking a provider's tool directly rather than through the controller. + +**7. Every provider serves the same four tools**, the protocol between the verbs and the providers: +`provisioner_retirement` (what is held, waiting, rejected and retired), `provisioner_retire_approve`, +`provisioner_retire_reject` and `provisioner_delete`, each act requiring a why. And says one event, +`provisioner.retirement`, whose `change` is `waiting`, `settled`, `approved`, `rejected`, `retired`, +`reenabled`, `deleted` or `adopted`; the controller derives the permission to publish it for every module +that receives contributions, as ADR 0224 does the standing events. + +**8. Nothing is silent.** Retire, approve, reject, re-enable and delete are each announced and said in the +provider's journal; waiting is an urgent condition and a rejection a warning one, both naming the +provider and its machine; and a self-check probe raises a **low-severity `cleanup-waiting` condition** +for any provider holding something retired **more than thirty days**. + +**9. The withdrawal brake paragraph of ADR 0229 is replaced by this record.** The rest of 0229 — the +lease, the epoch, the report's order, one writer at composition, the plan by compare-and-set, the lint, +every consumed kind's contract — stands unchanged. + +## Consequences + +- **The first rollout surfaces what is already there.** On the home server the consumer locked by an + earlier withdrawal is adopted as retired; the control node's six set-aside databases are listed, and + raise `cleanup-waiting` thirty days after their date; any consumer a backend holds open that the mesh no + longer asks for becomes a retirement — waiting for a person if there are more than three. Nothing is + deleted by the rollout. +- **Retired data is still backed up and still takes space** until a person deletes it. That is the + point; the thirty-day condition is what keeps it from being forgotten. +- **A person who rejects must come back.** A rejected set is kept active and said as a warning until the + mesh asks for it again or someone approves it. +- **The TypeScript providers get the stable count and the threshold** from the SDK's loop on their next + build (the SDK's provisioner, every catalogue provider's range accepts it), but a set over the bound + there waits with no way to approve it until each provider passes its module name and an announcer to + the loop, and they cannot list or delete what they retired until their adapters can. Until then they + are covered for safety and not for cleanup, and the record says so rather than claiming otherwise. +- **The Go loop is still two identical copies**, in postgres and keycloak, + held together by a test — now over three files. Its home is the Go SDK ([ADR 0039](0039-what-the-sdk-holds-and-refuses.md)); + moving it there means a tagged SDK release before the catalogue can use it, a separate change. +- **The rollout order**: the controller first (it derives the permission to publish the new event and + hears it; an older controller makes the provider's announcement a refusal the provider logs), then the + catalogue's two Go providers, then the SDK's release and each TypeScript provider as it is rebuilt. + +## How it is checked + +| What | Checked by | +|---|---| +| a transient empty list for four passes retires nothing; five stable passes retire; an unreadable pass and a changed set restart the count; additions and changes are not delayed | the catalogue's shared `retirement_test.go` (identical in both Go providers) and the SDK's `retire.test.ts` | +| over the bound waits, is announced and said, again every fifteen minutes; approve takes only the exact set; reject keeps it and is not asked again; a set asked for again settles | the same tests | +| retired is disabled with data intact; asked for again it is enabled as it was | postgres's `live_test.go` and keycloak's `live_retire_test.go`, against throwaway servers | +| delete removes only that consumer, never an active or asked one | the same live tests, and `retirement_test.go` | +| a restart retires what the backend holds unasked and adopts what it finds disabled | `retirement_test.go`, the provider's inventory tests | +| the conditions, verbs, hand acts and the thirty-day probe | the controller's tests of the event, the verbs against a fake provider on a real bus, and the probe | +| the two Go copies agree | `harness_same_test.go` in each provider, over the three files | +| live, after rollout | `cleanup list` names the adopted and set-aside entries; `conditions` shows no `retire-waiting` on a mesh nobody is changing | + +## References + +- [ADR 0229](0229-the-cores-order-is-a-lease-the-store-remembers-and-an-epoch-a-machine-is-sent-once-it-reads-one.md) + — replaced in part: its withdrawal brake paragraph. Everything else in it stands. +- [ADR 0227](0227-the-core-holds-nine-rules-each-checked-and-is-built-to-them-in-six-phases.md) rule 4, + *nothing dropped silently* — which this keeps: a stable result over the bound stops and raises a + condition. +- [ADR 0224](0224-a-provider-that-keeps-failing-a-consumer-is-a-problem-the-controller-reports.md) + — the provider's events and their derived permission, which the retirement event follows. +- [Issue 241](../04-ISSUES/241-one-unreadable-grants-file-dropped-every-database-on-the-control-node/00-report.md) + — what withdrawal destroyed, and what it stopped destroying. +- [To-be 45](../03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md) §2, §4, §7 and Phase 2, and + [the module protocol](../03-DESIGN/01-to-be/19-the-module-protocol.md), amended alongside. +- mesh-catalog `modules/postgres` and `modules/keycloak` (`retirement.go`, `retire_pg.go`, `oidc.go`); + mesh-sdk `src/provisioner` 0.1.12; mesh-controller's `retire` and `cleanup` verbs and its probe. diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index dc037a7..f358464 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -201,6 +201,7 @@ python3 00-META/checks/index.py fail if stale - **0224** — [A provider that keeps failing a consumer is a problem the controller reports](0224-a-provider-that-keeps-failing-a-consumer-is-a-problem-the-controller-reports.md) - **0227** — [The core holds nine rules, each checked, and is built to them in six phases](0227-the-core-holds-nine-rules-each-checked-and-is-built-to-them-in-six-phases.md) - **0229** — [The core's order is a lease the store remembers, and an epoch a machine is sent once it reads one](0229-the-cores-order-is-a-lease-the-store-remembers-and-an-epoch-a-machine-is-sent-once-it-reads-one.md) +- **0230** — [A consumer the mesh stops asking for is retired, and deleted only by a person](0230-a-consumer-the-mesh-stops-asking-for-is-retired-and-deleted-only-by-a-person.md) - **0231** — [A healer acts on what observation raised, and only observation says it worked](0231-a-healer-acts-on-what-observation-raised-and-only-observation-says-it-worked.md) ### Its tiers, from the bottom up diff --git a/03-DESIGN/01-to-be/19-the-module-protocol.md b/03-DESIGN/01-to-be/19-the-module-protocol.md index 44b3337..c238bae 100644 --- a/03-DESIGN/01-to-be/19-the-module-protocol.md +++ b/03-DESIGN/01-to-be/19-the-module-protocol.md @@ -10,6 +10,7 @@ code: updated: 2026-10-06 decisions: - 02-DECISIONS/0224-a-provider-that-keeps-failing-a-consumer-is-a-problem-the-controller-reports.md + - 02-DECISIONS/0230-a-consumer-the-mesh-stops-asking-for-is-retired-and-deleted-only-by-a-person.md - 02-DECISIONS/0095-the-control-plane-is-the-way-to-ask-a-module.md - 02-DECISIONS/0106-the-bus-is-nats.md - 02-DECISIONS/0128-the-mesh-bus-is-required-not-ambient.md @@ -246,6 +247,19 @@ receives contributions may publish both, derived and never declared. The control failing word per provider, machine and consumer, and `status` names each one until it recovers ([ADR 0224](../../02-DECISIONS/0224-a-provider-that-keeps-failing-a-consumer-is-a-problem-the-controller-reports.md)). +### A consumer the mesh stops asking for is retired + +A consumer is active, retired or deleted +([ADR 0230](../../02-DECISIONS/0230-a-consumer-the-mesh-stops-asking-for-is-retired-and-deleted-only-by-a-person.md)). +The provider's loop retires one only after the same set has gone unasked in five consecutive passes +that read the contributions file; retiring disables its access reversibly and marks it, in the +provider's own backend, with when and why; asked for again, the ordinary create re-enables it. A set of +more than three, or of more than half of those held, waits for a person. Every provider serves four +tools for this — `provisioner_retirement`, `provisioner_retire_approve`, `provisioner_retire_reject`, +`provisioner_delete` — and says `provisioner.retirement`, permitted like the two events above. The +controller's `retire` and `cleanup` verbs ask those tools on the provider's machine; nothing else deletes +a consumer. + ### Checked, and it agrees (2026-09-16) This looked like the sharpest disagreement and was not one. The live wire is the contributions file diff --git a/03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md b/03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md index 6451218..4b022f1 100644 --- a/03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md +++ b/03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md @@ -1,12 +1,13 @@ --- layer: to-be status: in-progress -code: [mesh-controller, mesh-host, mesh-tools, mesh-catalog, mesh-lab] +code: [mesh-controller, mesh-host, mesh-tools, mesh-catalog, mesh-sdk, mesh-lab] updated: 2026-10-06 decisions: - 02-DECISIONS/0227-the-core-holds-nine-rules-each-checked-and-is-built-to-them-in-six-phases.md - 02-DECISIONS/0229-the-cores-order-is-a-lease-the-store-remembers-and-an-epoch-a-machine-is-sent-once-it-reads-one.md - 02-DECISIONS/0231-a-healer-acts-on-what-observation-raised-and-only-observation-says-it-worked.md + - 02-DECISIONS/0230-a-consumer-the-mesh-stops-asking-for-is-retired-and-deleted-only-by-a-person.md - 02-DECISIONS/0224-a-provider-that-keeps-failing-a-consumer-is-a-problem-the-controller-reports.md - 02-DECISIONS/0218-a-plan-sends-grants-before-code-rolls-out-one-machine-first-and-a-newer-merge-takes-over-an-older-plan.md - 02-DECISIONS/0141-the-host-delivers-its-own-successor.md @@ -45,8 +46,9 @@ Owning repositories: **mesh-controller** (the condition store, watchdogs, `docto calls, the hand-act log, the facts snapshot), **mesh-host** (the node-engine: the apply queue, report order, epoch refusal, the `report` verb, rollback witnessing), **mesh-tools** (the node tools and the console: their heartbeat, passing every argument, health answers), **mesh-catalog** (the -`operator-channel` seat's holder and channels, the watcher, the provider brake, the catalogue's merge -gate), **mesh-lab** (the replays and the induced-failure scenarios). +`operator-channel` seat's holder and channels, the watcher, the providers' retirement, the catalogue's +merge gate), **mesh-sdk** (the TypeScript providers' loop and its retirement), **mesh-lab** (the +replays and the induced-failure scenarios). --- @@ -68,6 +70,7 @@ Every kind of state the core keeps, and the one component that writes it. Anyone | builds and their outcomes | the build seat's holder | its own state | the controller asks | | a merge announced | **one** announcer per forge (the hook, or the poll when the hook is absent — never both) | the bus | — | | a provider's standing | the provider | the provider's events | the controller keeps the newest word as a condition | +| a consumer's retirement: active, retired (when, why), deleted | the provider | its own backend's mark; said on `provisioner.retirement` | the controller asks the provider's tools; a person approves, rejects and deletes through the controller's verbs | | the operator-channel's open messages | the seat's holder | its own key-value state | — | | the facts snapshot | controller | the artifact store, `facts/latest` | the build seat reads | @@ -145,6 +148,13 @@ outlive the history. with their expiry. The all-well sentence requires none open, silenced included. - **ADR 0224's provider standing** is the first kind: `provider-failing`, raised by the provider's event, cleared by its recovery, its silence after thirty minutes the row S8 below. +- **Retirement** ([ADR 0230](../../02-DECISIONS/0230-a-consumer-the-mesh-stops-asking-for-is-retired-and-deleted-only-by-a-person.md)) + raises three, keyed by the provider module and its machine: `retire-waiting` (urgent) while a stable + set too large to retire alone waits for `retire approve` or `retire reject`, listing it; `retire-rejected` + (warning) while a rejected set is kept active though the mesh no longer asks for it; and + `cleanup-waiting` (warning), from the probe D11, while anything retired is older than thirty days. The + first two clear on the provider's `retired`, `settled` or `approved` word, and all three when the + provider is no longer assigned there. ## 3. The signals table (rule 5) @@ -196,6 +206,7 @@ itself — an unanswered probe is never a pass. | D8 | no address the mesh owns is in a ban list | 238 | | D9 | `status` answers in full within ten seconds | 265 | | D10 | every machine runs the node-engine and node tools builds its plan says, or is inside a plan's window | version split | +| D11 | no provider holds a consumer retired more than thirty days: each provider assigned is asked `provisioner_retirement` on its machine (ADR 0230); one that does not serve it yet is named, not failed | ADR 0230 | | DW | the watchdogs of §3 ran within three of their intervals: the watchers are watched, and raise S10 when the self-check stops | rule 6 | | H-* | the health probes of §8, run for every core component on every machine | rule 8 | @@ -331,7 +342,9 @@ all healers together, stop every healer — `mesh.healers.braked`, urgent — un A heal is never a hand act. `healers` lists the registry, the acts and the brake. **The hand-act log.** Every verb that repairs by hand — a named `push` outside a plan, `plans close`, -`broker consumer-reset`, `conditions silence`, and `hand-act record` for an act done outside the mesh — +`broker consumer-reset`, `conditions silence`, `retire approve`, `retire reject` and `cleanup delete` +(ADR 0230; and an approval, rejection or deletion a provider reports that did not come through the +controller), and `hand-act record` for an act done outside the mesh — takes a required `--why` and writes an entry: who, which verb and arguments, why, when, the condition key it addresses if any, and a **cause** (the condition kind, or a word the person gives). `status` shows the week's count. A cause recorded twice within fourteen days raises `healer-wanted` (S15). @@ -396,7 +409,7 @@ outcome, run on every merge to mesh-controller, mesh-host and mesh-tools: | R3 | the node-engine self-updates during its report (230, 264) | the report arrives under the new build | | R4 | the bus's authorization reloads during a call (265) | the call's outcome is readable by id | | R5 | a consumer with several filter subjects under mixed traffic (266) | no announcement is skipped; S5 fires if one is | -| R6 | an unreadable contributions file (241) | refused by name; nothing withdrawn; a condition | +| R6 | an unreadable contributions file (241) | refused by name; nothing retired; a condition | | R7 | the controller rebuilds itself mid-plan (214) | the plan continues under the new epoch | | R8 | a broken controller, node-engine and node tools build | each rolled back with no hand; condition and message | | R9 | each signal of §3 suppressed | its condition within its bound, cleared on return | @@ -447,11 +460,11 @@ bound corrected in the table. | mesh-controller | the lease and epoch (§6); plans written by compare-and-set; a report kept by sequence; abandoned calls marked; S12, S13, D5; the writers table enforced at grant composition; contract tests for every consumed subject and the check listing them; the empty-on-error lint | | mesh-host | one apply queue; the report sequence kept on disk; epoch refusal reported; the `report` verb; the contract tests and lint | | mesh-tools | refusing an unreadable or unknown input by name; the lint | -| mesh-catalog | the withdrawal brake in the providers' loop: a reconcile that would withdraw more than one consumer, or a set fraction, stops and raises a condition | +| mesh-catalog, mesh-sdk | retirement in the providers' loop (ADR 0230, replacing ADR 0229's brake): a consumer no longer asked for is retired — disabled, marked, its data kept — only once the same result holds for five passes; a set of more than three, or more than half of those held, waits for a person and raises a condition; the four retirement tools on every provider | | mesh-lab | R1, R2, R6 | **Done when:** R1 ends with nothing from the stale epoch applied, R2 with one report of the newest -sequence, R6 with the withdrawal braked; every consumed subject has its contract test. +sequence, R6 with nothing retired; every consumed subject has its contract test. **Built 2026-10-06** ([ADR 0229](../../02-DECISIONS/0229-the-cores-order-is-a-lease-the-store-remembers-and-an-epoch-a-machine-is-sent-once-it-reads-one.md)): the controller's lease and epoch, plans by compare-and-set, accounts kept by order, S12, S13 by writer, @@ -461,6 +474,15 @@ providers' loop and the SDK's. **Not yet:** the replays R1, R2 and R6 in mesh-la and lint; the lint in mesh-host; the SDK's loop announcing a provider's standing, without which its brake is said in its journal only. +**Amended 2026-10-06** ([ADR 0230](../../02-DECISIONS/0230-a-consumer-the-mesh-stops-asking-for-is-retired-and-deleted-only-by-a-person.md), +the operator's decision): the brake's hourly release is replaced by retirement — three states (active, +retired, deleted), five stable passes before a consumer is retired, a person for more than three or more +than half, `retire approve|reject`, `cleanup list|delete`, `retirement` events and D11. Built on branches +in mesh-catalog (both Go providers), mesh-sdk (0.1.12) and mesh-controller, not yet merged. **Not yet:** +the TypeScript providers passing their module name and an announcer to the loop, without which a set +over the bound there waits with no way to approve it, and adapters that list and delete what they +retired. + ### Phase 3 — Healers | Repository | Delivers |