Merge pull request 'Issue 273 and ADR 0232: a binding to a consumer's data moves only by a person' (#137) from issues/273-a-rule-for-the-resolver-moved-a-machines-databases into main

02-DECISIONS/0232-a-binding-to-a-consumers-data-moves-only-by-a-person.md

@@ -0,0 +1,136 @@
+---
+topic: what runs on it
+status: accepted
+date: 2026-10-06
+deciders: jochen
+reconstructed: false
+extends: 02-DECISIONS/0110-a-seat-is-a-module-assignment-from-a-closed-set.md
+---
+
+# 232. A binding to a consumer's data moves only by a person
+
+## Context
+
+**A rule written for the resolver moved a machine's databases.** [Issue 258](../04-ISSUES/258-every-machine-bound-the-resolver-to-itself/00-report.md)
+found that a machine running its own provider of a mesh-wide provision bound its consumers to that
+provider, even when the mesh's seat for the provision was held elsewhere. Its fix made the seat's
+holder, or a pin naming another machine, win over the local provider. That is right for the mesh's
+resolver: every resolver gives the same answer, so a consumer moved between two loses nothing.
+
+The fix applied to every seat that delivers a provision. The store's seat delivers the relational
+database. On the home server, which runs its own store with five applications' databases in it, the
+fix's first push re-bound all five to the store on the control node, which holds the seat. That store
+did what a provider does for a new consumer: it made each one an empty database. The applications
+started, ran their first migrations, and served empty data for about twenty hours. Nothing warned
+([issue 273](../04-ISSUES/273-a-rule-for-the-resolver-moved-a-machines-databases/00-report.md)).
+Nothing was lost, only because the old store kept everything.
+
+The incident has two parts:
+
+1. **Issue 258's rule applies to two kinds of provision as if they were one.** It does not say which
+   kind it is for.
+2. **Nothing in the mesh knows where a consumer's data is.** A resolution answers one question:
+   *which provider would I choose now?* Every input to that answer can change under a consumer without
+   anybody meaning to move it: the seat's holder (a handover, ADR 0131), a pin added or removed, a
+   provider assigned or unassigned. Only the pair secrets in the store held a trace of the old
+   binding, and only because a new pair was minted beside it.
+
+## Considered Options
+
+1. **Revert issue 258's fix.** Rejected. It was correct for the resolver, and the resolver would
+   break again.
+2. **Make the store's seat not deliver its provision.** Rejected. That fixes one seat, and the next
+   seat that delivers a provision which keeps data repeats the incident.
+3. **Refuse any resolution that changes a binding.** Rejected. It would refuse the resolver's moves,
+   which are the point of 258, and a person moving a database on purpose would have no way to do it.
+4. **Know which provisions keep their consumers' data. For those, keep each consumer where it was
+   last sent, say every move the mesh would have made, and let only a pin move it.** Chosen.
+
+## Decision
+
+**1. An offer says whether it keeps its consumers' data.** The field is `keeps-consumer-data` on
+`provides`.
+
+- If unsaid, a provider that **grants** each consumer a credential of its own keeps that consumer's
+  data. The grant makes an account (a role and its database, a key and its bucket, a client), and
+  what the consumer writes under it stays with that provider.
+- A provider that grants nothing keeps nothing of anybody's. Examples: the resolver, a CA, the
+  artifact store, a route.
+- The property belongs to the provision's name, as brokering does (ADR 0009): if any provider says
+  the provision keeps data, it keeps data.
+
+**2. Issue 258's rule is for provisions that keep nothing.** For a provision that keeps data, the
+seat's holder on another machine does **not** overrule a provider beside the consumer. A pin naming
+another machine still does, because a pin is a person.
+
+**3. Where each such consumer was sent is recorded, and a resolution keeps it there.** One row per
+consumer and provision is written when a declaration carrying the binding is sent. The provider is
+recorded by name, so a provider's machine leaving the mesh does not erase where the data is.
+
+- **Recorded binding, different provider chosen, no pin naming the new one:** the recorded provider
+  keeps answering. The move is said.
+- **Recorded provider no longer provides the provision:** the machine's set is **refused**, naming the
+  pin that would confirm the move. It is never answered by the provider chosen instead. This is ADR
+  0009's stance, already applied to a pin naming a provider that is gone.
+- **New consumer with no record:** it binds as resolved, and is recorded when first sent.
+
+**4. Only a pin moves it.** The pin names the provider, for that machine and provision. The send
+that carries the move records the new provider and keeps where it was. The data is moved by a person
+before the push; the mesh does not move data.
+
+**5. Nothing is silent.** A kept move is printed by the push that composed it and raised at once as
+an **urgent** condition:
+
+> would move X's P from A to B — its data is on A; kept there. `pin …` to confirm a move (and move
+> the data first)
+
+A self-check probe ([to-be 45](../03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md) §4)
+checks every machine every run. It raises:
+
+- a kept move, as urgent;
+- a pinned move not yet sent, as a **warning**, so the data goes first;
+- any consumer about to be sent another provider than the one on record with no pin naming it, as
+  **urgent**. The resolver makes this impossible; the probe exists for the day it is not.
+
+## Consequences
+
+- **The first push after rollout records what each machine is bound to then.** A consumer moved
+  silently before the rollout and never moved back would be recorded where it was moved to. The
+  rollout therefore starts by checking that every binding to data points where the data is. On
+  2026-10-06 every one did: the five moved consumers had already been pinned back.
+- **A pin is per machine and provision, not per consumer.** Pinning one consumer of a machine
+  elsewhere pins all its consumers of that provision. That is how pins have always worked, and the
+  warning names every consumer that would move.
+- **Unassigning a store beside its consumers now refuses their machine** until a person pins them
+  elsewhere. Before this, they moved silently to whichever store answered.
+- **A retired consumer's binding stays on record** ([ADR 0230](0230-a-consumer-the-mesh-stops-asking-for-is-retired-and-deleted-only-by-a-person.md)).
+  Assigned again, the consumer returns to the provider that kept its retired data.
+- **No module definition changes.** Every provider in the catalogue that keeps data already grants
+  credentials, and none that keeps nothing does. A definition states the field only when the default
+  is wrong for it.
+
+## How it is checked
+
+| What | Checked by |
+|---|---|
+| the incident: a machine running its own store and its consumers, the store's seat held elsewhere — the consumers stay on their own store, and the resolver beside them still follows its seat | mesh-controller `internal/catalogue/bound_test.go`, and `cmd/mesh-controller/bindings_test.go` through the real stores; both fail without the fix |
+| a recorded consumer is kept when the seat's holder changes, and the move is said with its pin | `bound_test.go` |
+| a pin moves it; a gone provider, or a store beside it unassigned, is refused and never answered elsewhere | `bound_test.go`, `bindings_test.go` |
+| an offer's `keeps-consumer-data` is read and written, and unsaid follows the grant | `bound_test.go` |
+| a binding is recorded on send, a move keeps where it was, and a provider's machine leaving keeps the record | `internal/inventory/bindings_test.go` |
+| a push raises a kept move at once; the probe says kept (urgent), pinned and not yet sent (warning), and unasked (urgent) | `bindings_test.go` |
+| live, after rollout | the self-check passes its binding probe on every run, and `conditions` holds no `binding-kept` on a mesh nobody is changing |
+
+## References
+
+- [Issue 273](../04-ISSUES/273-a-rule-for-the-resolver-moved-a-machines-databases/00-report.md):
+  the incident.
+- [Issue 258](../04-ISSUES/258-every-machine-bound-the-resolver-to-itself/00-report.md): the rule
+  this record limits.
+- [ADR 0110](0110-a-seat-is-a-module-assignment-from-a-closed-set.md): a seat's holder answers. This
+  record extends it, because the holder now answers only where nothing is kept.
+- [ADR 0009](0009-modules-and-the-graph.md): refuse rather than guess.
+- [ADR 0230](0230-a-consumer-the-mesh-stops-asking-for-is-retired-and-deleted-only-by-a-person.md):
+  the provider keeps a consumer's data until a person deletes it.
+- mesh-controller PR #86: `catalogue/bound.go`, the `binding` table (migration 0071), and the
+  self-check's binding probe.

02-DECISIONS/README.md

@@ -330,6 +330,7 @@ python3 00-META/checks/index.py            fail if stale
 - **0220** — [What a machine asks needs its uplink held, and the retired resolver pieces go](0220-what-a-machine-asks-needs-its-uplink-held-and-the-retired-resolver-pieces-go.md)
 - **0225** — [A consumer's identity is bounded by the provision it requires, judged before merge, and never refuses its provider](0225-a-consumers-identity-is-bounded-by-the-provision-it-requires.md)
 - **0228** — [A value given by hand lives only until its module's first good start](0228-a-value-given-by-hand-lives-only-until-its-modules-first-good-start.md)
+- **0232** — [A binding to a consumer's data moves only by a person](0232-a-binding-to-a-consumers-data-moves-only-by-a-person.md)
 
 ### How it is built
 

04-ISSUES/258-every-machine-bound-the-resolver-to-itself/00-report.md

@@ -46,6 +46,14 @@ This is checked by three resolve tests in mesh-controller:
 
 The tests fail without the fix.
 
+**The fix broke something else** ([issue 273](../273-a-rule-for-the-resolver-moved-a-machines-databases/00-report.md)).
+It applied to every seat that delivers a provision, and the store's seat delivers the relational
+database. Its first push re-bound every database consumer on a machine running its own store to the
+store holding the seat. Each was made an empty database there, and nothing said so for twenty hours.
+[ADR 0232](../../02-DECISIONS/0232-a-binding-to-a-consumers-data-moves-only-by-a-person.md) limits
+this rule to provisions that keep nothing of their consumers'. For one that keeps data, only a pin
+moves a consumer away from a provider beside it.
+
 ## Verified
 
 2026-10-05, after the fix rolled out: all four machines' resolver configuration names the anchor's

04-ISSUES/273-a-rule-for-the-resolver-moved-a-machines-databases/00-report.md

@@ -0,0 +1,113 @@
+---
+status: located
+opened: 2026-10-06
+located-in: [mesh-controller]
+fixed-by: novox/mesh-controller PR #86
+amended-design:
+---
+
+# 273. A rule for the resolver moved a machine's databases
+
+## Symptom
+
+On 2026-10-06 the five applications on the home server were found serving empty data. Each is a
+consumer of the relational database provision and has its own database. The home server runs its
+own store, and that is where their data is: one application's hundreds of tables, another's
+hundred-and-seventy thousand rows, a workflow engine's workflows, an agent server's agents and
+messages, and a game's players.
+
+Their binding files named the store on the control node, which holds the mesh's store seat. That
+store had each application's database, made new and empty on 2026-10-05 at 19:16 UTC. The first
+application ran its first migration there at 19:17. All five then ran against the empty databases
+for about twenty hours.
+
+Nothing warned:
+
+- the push that moved them reported success;
+- `status` read them as applied and current;
+- no condition was raised.
+
+The real data stayed untouched on the home server's store. Pinning the home server to its own store
+for the provision, then pushing, put every application back on its data. **No data was lost.**
+
+The evidence is in the store's pair secrets: each of the five consumers holds one credential from
+the home server, minted in the days before, and a second from the control node. All five second
+credentials were minted in the same second, 19:16:21 UTC on 2026-10-05, two minutes after the
+controller carrying [issue 258](../258-every-machine-bound-the-resolver-to-itself/00-report.md)'s fix
+was merged and rolled out.
+
+## Cause
+
+Issue 258's fix made the holder of a provision's mesh seat on another machine, or a pin naming
+another machine, answer **before** a provider on the consumer's own machine. It was written for the
+resolver, which every holder answers alike. But the check ran for every provision a seat delivers,
+and the store's seat delivers the relational database. On a machine running its own store beside its
+consumers, every consumer was re-bound to the seat's holder. The holder did what a provider does for
+a consumer it has not seen: it made an empty database.
+
+Two gaps let this happen, and only the first was the change:
+
+1. **The rule did not know which provisions it was for.** Nothing in a module definition, or in the
+   resolver, separated a provision every provider answers alike from one whose provider keeps what
+   the consumer wrote.
+2. **Nothing knew where a consumer's data was.** A resolution chooses a provider from inputs that
+   change: seat holders, pins, assignments. A consumer bound to its data was re-bound like any other,
+   with nothing to compare against and nothing to say. Before this incident, a handover of the
+   store's seat, or a pin removed, could have done the same.
+
+## Other machines
+
+All four machines were checked on 2026-10-06, read-only. Each machine's planned binding files were
+compared with the pair secrets in the store and with where each database is. **No other machine's
+consumers were moved.**
+
+- Every other database consumer, and every consumer of the document store, object store, mail,
+  identity provider, package registry and broker topics, has one credential, from the provider it is
+  bound to now.
+- The control node's consumers were never at risk: the control node holds the seat itself.
+
+Three findings remain on the home server:
+
+- **A sixth consumer there** first asked for a database after the change, at 00:28 on 2026-10-06. It
+  was bound to the control node from its first send. Its data, made before the mesh bound it, is on
+  the home server, and it is bound there now through the pin.
+- **The control node's store still holds the six databases made for the home server's consumers.**
+  Four of them grew past an empty database's size while the applications ran against them. What was
+  written there in those twenty hours should be reviewed before they are retired
+  ([ADR 0230](../../02-DECISIONS/0230-a-consumer-the-mesh-stops-asking-for-is-retired-and-deleted-only-by-a-person.md)
+  makes that a person's act).
+- **The control-node pair credentials for those consumers are still on record**, though nothing is
+  sent them.
+
+## Fix
+
+The fix is mesh-controller PR #86, decided by
+[ADR 0232](../../02-DECISIONS/0232-a-binding-to-a-consumers-data-moves-only-by-a-person.md):
+
+- An offer says whether it keeps its consumers' data. If it does not say, a provider that grants
+  each consumer a credential does.
+- For such a provision, issue 258's seat rule no longer overrules a provider beside the consumer. A
+  pin still does.
+- Where each such consumer was sent is recorded. A resolution that would bind it elsewhere keeps it
+  where it is, and raises an urgent condition naming the move and the pin that would confirm it. If
+  the recorded provider is gone, the machine's set is refused; the consumer is never moved to another
+  provider.
+- A self-check probe raises:
+  - every kept move;
+  - every pinned move not yet sent, so its data is moved first;
+  - any move that slips past the resolver.
+
+## How it is checked
+
+The resolver's tests reproduce the incident: a machine running its own store and its consumers, with
+the store's seat held on another machine. Every consumer stays on its own store, and the resolver
+beside them still follows its seat. A second test runs the same incident through the controller's
+real stores. It checks:
+
+- the binding is recorded on send;
+- the probe is silent;
+- the store taken away is refused;
+- a pin moves the binding and is said as a warning;
+- once sent, it is quiet again.
+
+Both tests fail without the fix. The rest of the checks are listed in ADR 0232.