Building the bus: the decisions the work needed, and what it taught back #150
@@ -183,12 +183,140 @@ not an outage for its callers — it is latency.
|
||||
That difference is worth choosing on purpose. A dependency expressed as a provision must be
|
||||
ordered; the same dependency expressed as a seat need not be.
|
||||
|
||||
## 8. Open
|
||||
## 8. Versioning a protocol
|
||||
|
||||
**Protocol versioning.** A seat's protocol is a compatibility surface between modules that do not
|
||||
know each other, and nothing here says what happens when it changes under callers already bound to
|
||||
it. This is the first thing to answer and the one most likely to hurt in year two rather than week
|
||||
one.
|
||||
A seat's protocol is a compatibility surface between modules that do not know each other and are
|
||||
deployed at different times. Four ways it can change, and they are not equally dangerous:
|
||||
|
||||
| change | example | detectable |
|
||||
|---|---|---|
|
||||
| **additive** | a new `accepts` subject, a new optional field | nothing breaks |
|
||||
| **removal or rename** | `send` becomes `deliver` | yes, mechanically |
|
||||
| **shape** | an optional field becomes required | yes, if shapes are specified |
|
||||
| **semantic** | `send` starts meaning *queue for tomorrow* | **no** |
|
||||
|
||||
**Additive is free.** A seat may grow without a version, without re-registering a caller, and
|
||||
without ceremony. Most change is this.
|
||||
|
||||
**A breaking change is refused while anyone is bound.** Registration computes a compatibility
|
||||
fingerprint over the seat's protocol — its subjects and the shapes they carry. A registration that
|
||||
alters the fingerprint while callers are bound is refused, **and the refusal names them**. The
|
||||
mesh already holds the `uses` graph, so this is derived rather than declared, and it turns a
|
||||
runtime breakage into a registration-time conversation.
|
||||
|
||||
**When a break is genuinely needed, the version goes in the subject, not the name.** The seat stays
|
||||
one thing; `mesh.seat.<seat>.v2.<verb>` runs beside v1 and the holder serves both. A caller moves
|
||||
when it is ready. Versioning the *seat name* was considered and rejected: it forks the role, so
|
||||
"one holder" stops meaning one provider of the capability, and every document naming the seat has
|
||||
to be found and changed.
|
||||
|
||||
**Binding is recorded, not inferred.** A caller declares `uses: telegram-sender` with no version,
|
||||
and resolution binds it to the current one and records that — the same **pin** machinery
|
||||
[design 27](27-a-module-requires-the-mesh-resolves.md) already uses when resolution had a choice
|
||||
to make. Moving to v2 is a deliberate re-pin, so nothing drifts onto a new protocol because it
|
||||
happened to be newest.
|
||||
|
||||
**Retirement is reported, never automatic.** When the `uses` graph shows nothing bound to v1, the
|
||||
overview says it is retirable. The mesh does not remove it.
|
||||
|
||||
**And none of this catches a semantic change.** Same subject, same shape, new meaning: no
|
||||
fingerprint sees it, and no check proposed here would. The defences are review, and pushing
|
||||
meaning into shape wherever it can go — a required `channel` field is caught, a changed
|
||||
interpretation of an existing one is not. Saying so is better than implying the fingerprint is
|
||||
complete, because a team that believes it is complete stops reviewing for the case it misses.
|
||||
|
||||
## 9. Provisioning over the bus
|
||||
|
||||
Provisioning rides the bus, and the provider stops having an address.
|
||||
|
||||
| part of a provision | shape |
|
||||
|---|---|
|
||||
| the requirement resolving to a provider | the controller's, not the bus's |
|
||||
| the grant reaching the provisioner | request/reply to a **role** |
|
||||
| `holds` — the reconcile question, every minute | the same call, on a timer |
|
||||
| `provisioned` / `deprovisioned` | events |
|
||||
| the credential reaching the consumer | §10 — fetched, never carried |
|
||||
|
||||
A provisioner's interface is already three calls — create, remove, holds — which is exactly a
|
||||
`serves` protocol. So **a provision interface is a seat whose protocol is those three**, which is
|
||||
why [design 26](26-the-seats.md) already allows a seat to deliver a provision. The two concepts
|
||||
were converging before this document; here they meet.
|
||||
|
||||
What stays different, and must not be unified away: a provision has a **per-consumer resource and
|
||||
a sealed credential**, created and destroyed per consumer. A seat protocol has neither — it is a
|
||||
role you send to. Collapsing them would mean pretending a database is a subject.
|
||||
|
||||
### Where addresses survive
|
||||
|
||||
"Where is it?" is two different problems, and the bus solves one of them completely and the other
|
||||
not at all. Keeping them apart matters, because a reader who thinks the mesh no longer has
|
||||
addresses will believe a class of bug is fixed when it is untouched.
|
||||
|
||||
**Something that is on the bus: the address disappears.** A host reporting in used to need the
|
||||
controller's address — recorded somewhere, at some moment, and wrong as soon as anything moved.
|
||||
Now it publishes to `mesh.seat.mesh-controller.report` and the bus routes it to whoever holds the
|
||||
seat. Nothing anywhere records where the controller is, so nothing can record it *wrongly*. The
|
||||
same is true of the builder, the catalogue, the telegram sender. This class is not mitigated; it
|
||||
is gone, because the information is no longer stored.
|
||||
|
||||
**Something that is not on the bus: the address stays, exactly as before.** A module that requires
|
||||
a database does not reach postgres over NATS — it opens a postgres connection, because postgres
|
||||
speaks postgres and is not listening on any subject. Its credential contains a host and a port,
|
||||
and no amount of subject addressing changes that.
|
||||
|
||||
So the fix for that second class is unchanged and is not this document's:
|
||||
[ADR 0098](../../02-DECISIONS/0098-a-fact-a-provider-makes-at-first-start-is-fetched-from-it.md) —
|
||||
fetch the fact where it is used rather than storing a copy — which is what
|
||||
[issue 102](../../04-ISSUES/102-an-address-recorded-at-genesis-or-build-does-not-follow-the-nodes-ports/00-report.md)
|
||||
is actually about. The bus makes that class *smaller* by removing every mesh-internal address from
|
||||
it. It does not make it empty.
|
||||
|
||||
**And one address is irreducible: the bus's own.** A node has to know where the broker is before
|
||||
it can use subjects for anything, so that one cannot be a subject. It is the addressing equivalent
|
||||
of §10's bootstrap — the first thing cannot be found by the mechanism that finds everything else.
|
||||
|
||||
## 10. Secrets, and why they never enter a stream
|
||||
|
||||
Everything the vault does is request/reply to the `mesh-vault` role: mint, fetch, rotate. In that
|
||||
sense it is as much on the bus as anything else.
|
||||
|
||||
**The bus is not trusted with a secret, and does not need to be.** A secret is sealed to its
|
||||
recipient, so what crosses the bus is ciphertext only that recipient can open. The broker sees
|
||||
that a secret moved, and to whom — metadata, which is acceptable — and never a plaintext.
|
||||
|
||||
**But sealed is not enough on its own, because a stream persists.** A sealed secret written into
|
||||
a JetStream stream is a durable ciphertext sitting in the mesh's own storage, and the day a
|
||||
sealing key leaks, that stream is an archive rather than a moment. So:
|
||||
|
||||
- **A secret travels on core request/reply, never through a stream.** No persistence, no replay,
|
||||
nothing to exfiltrate later.
|
||||
- **A declaration names a secret; it does not carry one.** Declarations are the state shape, which
|
||||
*is* a stream — so the host fetches the secret from the vault at apply time, over the core path.
|
||||
That is [ADR 0098](../../02-DECISIONS/0098-a-fact-a-provider-makes-at-first-start-is-fetched-from-it.md)'s
|
||||
existing discipline — *fetched from it, not carried* — applied to the one payload where carrying
|
||||
it is worst.
|
||||
|
||||
**The bootstrap, which is circular and has a precedent.** The vault makes every secret
|
||||
([ADR 0113](../../02-DECISIONS/0113-the-vault-makes-every-secret.md)), including the bus's own
|
||||
passwords. The vault is a module, and a module needs a bus account, whose password the vault
|
||||
makes. Nothing can go first.
|
||||
|
||||
This is the shape [ADR 0067](../../02-DECISIONS/0067-genesis-is-a-pivot.md) already resolves for
|
||||
the control plane: **genesis is a pivot.** The controller mints the handful of foundation
|
||||
credentials itself, raises the store, the broker and the vault, and then the vault takes over and
|
||||
mints everything from there — the same move as raising a temporary control plane and reinstalling
|
||||
it as an ordinary module once the registry exists.
|
||||
|
||||
So there are exactly two things the normal path cannot make, both at genesis, both ending the
|
||||
moment the mesh can mint for itself: **the bus's own accounts** (§the bootstrap argument in
|
||||
[ADR 0117](../../02-DECISIONS/0117-the-bus-is-the-only-broker.md) — a provisioner is a module and
|
||||
needs an account before it can run) and **the vault's own credential**. Any third exception is a
|
||||
design failure, and naming these two is what makes a third one visible.
|
||||
|
||||
## 11. Open
|
||||
|
||||
**Semantic change has no mechanical defence** (§8). Recorded as open rather than solved, because
|
||||
it is the residue of a question the rest of §8 answers and the part a fingerprint cannot reach.
|
||||
|
||||
**Whether a module may declare a seat it does not itself claim** — the contract as one thing, the
|
||||
implementation as another, which is how two competing implementations would ever exist.
|
||||
@@ -197,7 +325,7 @@ implementation as another, which is how two competing implementations would ever
|
||||
an event's provenance is its meaning — but a consumer of `billing.order.placed` does depend on
|
||||
billing existing under that name.
|
||||
|
||||
## 9. How it is checked
|
||||
## 12. How it is checked
|
||||
|
||||
- **A manifest holds no subject.** A catalogue test: no manifest contains a string matching the
|
||||
subject grammar. The rule is worthless if it is followed by convention.
|
||||
|
||||
Reference in New Issue
Block a user