The bus is the mesh's centre, not a transport that replaced one
Two things. A paragraph from the superseded 0117 survived beside the 0119 correction that reversed it, so §5 said both that the amqp interface retires and that it does not. The stale one is gone. And the framing. §1 opened with "the bus carries five kinds of traffic today, and this design keeps the five", with a column mapping each to the queue it used to be — which describes the mesh's nervous system as a port of something that did a fraction of this. It now says what the bus is: a role addressable without knowing its holder, the mesh's own state, work that queues until somebody can do it, and permissions derived from what a module declared. Conditions, observation and a person's client land there too as they are built. Glossary gains `bus` and `the deprecated broker`, with a note on why not to say "compatibility broker" or name it after a protocol — the second invites exactly the backwards framing this commit removes.
This commit is contained in:
+16
-2
@@ -31,8 +31,22 @@ another — and a mesh you cannot name precisely is a mesh two people describe d
|
|||||||
- **store** — the one postgres server. It holds the controller's own context databases
|
- **store** — the one postgres server. It holds the controller's own context databases
|
||||||
(`inventory`, `identity`, `licences` — a context owns its store, [ADR 0008](../02-DECISIONS/0008-a-context-owns-its-store.md))
|
(`inventory`, `identity`, `licences` — a context owns its store, [ADR 0008](../02-DECISIONS/0008-a-context-owns-its-store.md))
|
||||||
and every module's own database. One server, many databases — never one shared "mesh database".
|
and every module's own database. One server, many databases — never one shared "mesh database".
|
||||||
- **broker** — the one lavinmq message bus. It carries the mesh bus on the `/` vhost and a vhost per
|
- **bus** — the mesh's own nervous system: NATS, one per mesh, carrying every link the mesh has —
|
||||||
consumer that requires `amqp`.
|
control, declarations, builds, events, tool calls
|
||||||
|
([ADR 0106](../02-DECISIONS/0106-the-bus-is-nats.md)). A module reaches it by requiring
|
||||||
|
`mesh-bus` ([ADR 0120](../02-DECISIONS/0120-the-mesh-bus-is-required-not-ambient.md)); one that
|
||||||
|
does not require it has no account on it. Held by the `mesh-broker` seat, which is named after
|
||||||
|
the *role* rather than the server, so the server can change without the seat doing so.
|
||||||
|
- **the deprecated broker** — the lavinmq module. It was the mesh's bus and is not any more. It
|
||||||
|
keeps running as an **ordinary provider** of the `amqp` provision, for modules that need a
|
||||||
|
message broker of their own the way something needs a database
|
||||||
|
([ADR 0119](../02-DECISIONS/0119-amqp-is-a-provision-not-the-bus.md)) — no seat, not foundation,
|
||||||
|
never raised at genesis, and a mesh that never installs it is complete.
|
||||||
|
|
||||||
|
Say *the deprecated broker*, not "the compatibility broker" (it serves the mesh's own modules,
|
||||||
|
not only the predecessor's) and not "the AMQP broker" (naming it after a protocol invites
|
||||||
|
describing the bus by contrast with it, which is backwards: the bus is the mesh's nervous
|
||||||
|
system and this is a module).
|
||||||
|
|
||||||
## What the mesh stores and serves
|
## What the mesh stores and serves
|
||||||
|
|
||||||
|
|||||||
@@ -30,19 +30,37 @@ Prose and diagrams only; no configuration is pasted.
|
|||||||
|
|
||||||
## 1. What the bus is for
|
## 1. What the bus is for
|
||||||
|
|
||||||
The bus carries five kinds of traffic today, and this design keeps the five, renaming nothing a
|
**The bus is where the mesh happens.** Not a transport the mesh sends things over — the place a
|
||||||
module can see:
|
module is reachable at all, where a role is addressed without knowing who holds it, where the
|
||||||
|
mesh's own state lives, and where what a module may say is decided by what it declared.
|
||||||
|
|
||||||
| Traffic | Today | Guarantee it needs |
|
| Traffic | Shape | Guarantee it needs |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| **control** — a node's report, its heartbeat, a build's outcome, an enrolment | queues `control`, `.upgrades`, `.catchup` | nothing lost while the store restarts; retried; in order per node |
|
| **control** — a node's report, a build's outcome, an enrolment | job | nothing lost while the store restarts; retried; in order per node |
|
||||||
| **declarations** — the controller tells a node what to be | queue `node.<name>` | the node gets the newest; a stale one is never applied |
|
| **heartbeat** — a node saying it is alive | fire and forget | none; a lost one is the next one |
|
||||||
| **builds** — the controller asks the build machine to build | queue `builds` | at least once, one builder at a time |
|
| **declarations** — the controller tells a node what to be | state | the node gets the newest; a stale one is never applied |
|
||||||
| **events** — a module says something happened | topic exchange `mesh.events`, keys `<module>.<event>` | delivered to every consumer that declared it; dead-lettered when it cannot be |
|
| **builds** — work for the build machine | job | at least once, one worker at a time |
|
||||||
| **tools** — one module or person asks another's tool a question | exchange `mesh.rpc`, per-tool service queues `serve.<module>.<tool>` | one answer, from one server, or a timeout |
|
| **events** — a module says something happened | 1:many | delivered to every consumer that declared it; dead-lettered when it cannot be |
|
||||||
|
| **tools** — a module or a person asks another's tool | request/reply | one answer, from one server, or a timeout |
|
||||||
|
| **work to a role** — a module submits to a capability without knowing who provides it | job | exactly one holder does it; it queues while nobody does |
|
||||||
|
|
||||||
The sdk's contract — `request`, `handle`, `publish`, `subscribe`, `close` — is the whole surface a
|
The last two rows are the ones worth dwelling on, because they are not messaging in the sense of
|
||||||
module sees, and it does not change ([ADR 0039](../../02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md)).
|
carrying bytes from A to B. **A role is addressable**, so a caller names the capability and never
|
||||||
|
the module or the node — and the implementation can be replaced under it without a caller
|
||||||
|
changing ([ADR 0118](../../02-DECISIONS/0118-a-module-declares-its-own-seats.md)). That is a
|
||||||
|
property of the mesh's architecture that happens to be expressed in subjects.
|
||||||
|
|
||||||
|
And more of the mesh lands here as it is built: conditions and observed state in key-value
|
||||||
|
buckets that anything may watch, the server's own advisories becoming observations like any other
|
||||||
|
([research 017](../../01-RESEARCH/017-a-mesh-that-heals-itself/00-overview.md)), and a person's
|
||||||
|
client speaking the bus directly rather than through a surface built over it (§7). None of that
|
||||||
|
is a message being moved; all of it is the bus being the mesh's centre.
|
||||||
|
|
||||||
|
**What a module sees of it is small and derived.** It declares what it emits, consumes, serves
|
||||||
|
and uses, and the subjects, streams, consumers and permissions all follow from that
|
||||||
|
([design 29](29-what-a-module-declares.md)). The sdk's contract — `request`, `handle`, `publish`,
|
||||||
|
`subscribe`, `close` — is the whole surface, and it does not change
|
||||||
|
([ADR 0039](../../02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md)).
|
||||||
|
|
||||||
## 2. Subjects
|
## 2. Subjects
|
||||||
|
|
||||||
@@ -212,11 +230,11 @@ same place `modules/gitea/token.ts` keeps its own state rather than asking the h
|
|||||||
The host's only job is what it already does for any directory resource: keep the file's content
|
The host's only job is what it already does for any directory resource: keep the file's content
|
||||||
current. Nothing is declared as `reload-on` or `restart-on` for this resource at all.
|
current. Nothing is declared as `reload-on` or `restart-on` for this resource at all.
|
||||||
|
|
||||||
Its guard is the same rule as the AMQP broker's: the monitoring port is refused from anything but
|
Its guard is the same rule as the deprecated broker's: the monitoring port is refused from anything but
|
||||||
the private network. It is raised at genesis like the store, adopted as a module in the same
|
the private network. It is raised at genesis like the store, adopted as a module in the same
|
||||||
phase.
|
phase.
|
||||||
|
|
||||||
**The AMQP broker is an ordinary module, not a compatibility layer.** Revision, second review
|
**The deprecated broker is an ordinary module, not a compatibility layer.** Revision, second review
|
||||||
([ADR 0119](../../02-DECISIONS/0119-amqp-is-a-provision-not-the-bus.md)): earlier text here, and
|
([ADR 0119](../../02-DECISIONS/0119-amqp-is-a-provision-not-the-bus.md)): earlier text here, and
|
||||||
ADR 0106 before it, called it `lavinmq-compat` — one purpose, the predecessor's clients, and a
|
ADR 0106 before it, called it `lavinmq-compat` — one purpose, the predecessor's clients, and a
|
||||||
retirement condition of no client connected for a period the operator sets. It is none of those.
|
retirement condition of no client connected for a period the operator sets. It is none of those.
|
||||||
@@ -231,17 +249,6 @@ that remains is about direction rather than software: *inter-module communicatio
|
|||||||
bus.* A module may hold a broker, a database or a cache for itself; it may not use one as a
|
bus.* A module may hold a broker, a database or a cache for itself; it may not use one as a
|
||||||
channel to another module.
|
channel to another module.
|
||||||
|
|
||||||
**And with one purpose, which it did not have when this was written.** Revision, second review
|
|
||||||
([ADR 0119](../../02-DECISIONS/0119-amqp-is-a-provision-not-the-bus.md) (superseding [ADR 0117](../../02-DECISIONS/0117-the-bus-is-the-only-broker.md))): two modules of the new mesh
|
|
||||||
required a broker *of their own* from it — a private vhost per consumer, the analog of
|
|
||||||
database-per-login, which is a different thing from the mesh's bus and was never examined here.
|
|
||||||
**There is one bus, and no module is handed a broker as a resource.** A module's messaging is
|
|
||||||
subjects on the bus under its own account, scoped by its `emits` and `consumes`; a module that
|
|
||||||
wants a queue of its own has a subject nothing else may publish to and a durable consumer, both
|
|
||||||
from its declaration. What it does not get is a server of its own. The `amqp` interface retires
|
|
||||||
with the compatibility broker instead of gaining a successor, and the two modules move to the bus
|
|
||||||
in step 4.
|
|
||||||
|
|
||||||
## 6. Joining: the enrolment handshake
|
## 6. Joining: the enrolment handshake
|
||||||
|
|
||||||
Unchanged in shape, changed in transport. A node that has a token connects to the bus over TLS
|
Unchanged in shape, changed in transport. A node that has a token connects to the bus over TLS
|
||||||
@@ -323,7 +330,7 @@ wrong until there is a second mesh. *Ends at: the genesis bed.*
|
|||||||
**Step 2 — adoption puts the broker in its seat.** A mesh already running does not get a foundation
|
**Step 2 — adoption puts the broker in its seat.** A mesh already running does not get a foundation
|
||||||
module by being raised again; it adopts one in place
|
module by being raised again; it adopts one in place
|
||||||
([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)). The
|
([ADR 0100](../../02-DECISIONS/0100-a-node-in-use-is-adopted-before-it-is-converged.md)). The
|
||||||
server is raised beside the AMQP broker on its own ports, carrying no mesh traffic yet, and the
|
server is raised beside the deprecated broker on its own ports, carrying no mesh traffic yet, and the
|
||||||
`nats` module is adopted onto it. The seat it claims is **`mesh-broker`**, unchanged — the
|
`nats` module is adopted onto it. The seat it claims is **`mesh-broker`**, unchanged — the
|
||||||
foundation seats are named after the server's role rather than the product
|
foundation seats are named after the server's role rather than the product
|
||||||
([ADR 0079](../../02-DECISIONS/0079-the-foundation-seats-are-named-after-their-servers.md)) for
|
([ADR 0079](../../02-DECISIONS/0079-the-foundation-seats-are-named-after-their-servers.md)) for
|
||||||
@@ -384,7 +391,7 @@ itself enforces and a plain client can therefore check:
|
|||||||
watcher-poll interval, without a restart.
|
watcher-poll interval, without a restart.
|
||||||
|
|
||||||
**Step 2 — the adoption bed**, a mesh already running that has never had this server:
|
**Step 2 — the adoption bed**, a mesh already running that has never had this server:
|
||||||
- the server is raised beside the AMQP broker on its own ports and the `nats` module is adopted
|
- the server is raised beside the deprecated broker on its own ports and the `nats` module is adopted
|
||||||
onto it in place, holding the data and the configuration it was raised with;
|
onto it in place, holding the data and the configuration it was raised with;
|
||||||
- the seat it claims is `mesh-broker`, and a second assignment of it anywhere in the mesh is
|
- the seat it claims is `mesh-broker`, and a second assignment of it anywhere in the mesh is
|
||||||
refused at resolution — *one per mesh*, as ADR 0079 requires;
|
refused at resolution — *one per mesh*, as ADR 0079 requires;
|
||||||
|
|||||||
@@ -148,7 +148,7 @@ paper is wrong until there is a second mesh to find out.
|
|||||||
server's role, not the product. **Already true of the controller and needed no change**: it
|
server's role, not the product. **Already true of the controller and needed no change**: it
|
||||||
resolves the broker by seat ("that is where the broker is, whatever else the topology says")
|
resolves the broker by seat ("that is where the broker is, whatever else the topology says")
|
||||||
and names no broker module anywhere in its source. What remains is naming `nats` instead of
|
and names no broker module anywhere in its source. What remains is naming `nats` instead of
|
||||||
the AMQP broker where a genesis module set is declared, which is scenario and installer
|
the deprecated broker where a genesis module set is declared, which is scenario and installer
|
||||||
configuration — carried with 1.6 rather than before it.
|
configuration — carried with 1.6 rather than before it.
|
||||||
- [ ] 1.6 the genesis-broker bed — **deferred**: beds are run once, at the end, rather than per
|
- [ ] 1.6 the genesis-broker bed — **deferred**: beds are run once, at the end, rather than per
|
||||||
step (novox/hq design 22's rule, and the operator's instruction). Every claim step 1 makes
|
step (novox/hq design 22's rule, and the operator's instruction). Every claim step 1 makes
|
||||||
@@ -188,7 +188,7 @@ a foundation module by being raised again; it adopts one in place
|
|||||||
true and now proved**: the refusal is generic to any mesh-scoped seat, and three tests pin
|
true and now proved**: the refusal is generic to any mesh-scoped seat, and three tests pin
|
||||||
what matters for this one — a second bus anywhere is refused naming the seat, a *different*
|
what matters for this one — a second bus anywhere is refused naming the seat, a *different*
|
||||||
bus implementation is refused for the same reason (which is what lets the bus be replaced
|
bus implementation is refused for the same reason (which is what lets the bus be replaced
|
||||||
at all), and the AMQP broker no longer contends for it, so both run on one mesh
|
at all), and the deprecated broker no longer contends for it, so both run on one mesh
|
||||||
- [ ] 2.4 the adoption bed — deferred with the other beds
|
- [ ] 2.4 the adoption bed — deferred with the other beds
|
||||||
|
|
||||||
> **Adoption here is not a no-op, and pretending it would be is the trap.** The host keeps an
|
> **Adoption here is not a no-op, and pretending it would be is the trap.** The host keeps an
|
||||||
@@ -344,13 +344,13 @@ reserves them for after the move, and a flow built ahead of its design would be
|
|||||||
|
|
||||||
**Why here.** It is the only step that moves a node's bus, and it moves every node's at once.
|
**Why here.** It is the only step that moves a node's bus, and it moves every node's at once.
|
||||||
|
|
||||||
- [ ] 5.1 the cutover bed: a mesh on AMQP with a predecessor stand-in on the compatibility broker
|
- [ ] 5.1 the cutover bed: a mesh on AMQP with a predecessor stand-in on the deprecated broker
|
||||||
moves its bus in one rollout, every node reporting on NATS afterwards, the stand-in's own
|
moves its bus in one rollout, every node reporting on NATS afterwards, the stand-in's own
|
||||||
client still connected throughout
|
client still connected throughout
|
||||||
- [ ] 5.2 the rollout: accounts composed, then the controller, every host and every runtime
|
- [ ] 5.2 the rollout: accounts composed, then the controller, every host and every runtime
|
||||||
together; every node confirmed heard before AMQP stops
|
together; every node confirmed heard before AMQP stops
|
||||||
- [ ] 5.3 the mesh's accounts removed from the compatibility broker, leaving the predecessor's users
|
- [ ] 5.3 the mesh's accounts removed from the deprecated broker, leaving the predecessor's users
|
||||||
- [ ] 5.4 the compatibility broker retires when its condition holds — no client connected for the
|
- [ ] 5.4 the deprecated broker retires when its condition holds — no client connected for the
|
||||||
period the operator sets
|
period the operator sets
|
||||||
|
|
||||||
**Done when.** Every node reports on NATS and the predecessor's clients never noticed.
|
**Done when.** Every node reports on NATS and the predecessor's clients never noticed.
|
||||||
|
|||||||
@@ -302,7 +302,7 @@ The mesh's own bus is **`mesh-bus`**, delivered by the `mesh-broker` seat and an
|
|||||||
controller — because the bus's accounts are configuration rather than something a provisioner
|
controller — because the bus's accounts are configuration rather than something a provisioner
|
||||||
creates, so there is no provisioner process in the path and nothing waiting on a bus account in
|
creates, so there is no provisioner process in the path and nothing waiting on a bus account in
|
||||||
order to make bus accounts. A module that runs a NATS server of its own and offers it as a
|
order to make bus accounts. A module that runs a NATS server of its own and offers it as a
|
||||||
backing service provides **`nats`**, exactly as the AMQP broker provides `amqp`
|
backing service provides **`nats`**, exactly as the deprecated broker provides `amqp`
|
||||||
([ADR 0119](../../02-DECISIONS/0119-amqp-is-a-provision-not-the-bus.md)).
|
([ADR 0119](../../02-DECISIONS/0119-amqp-is-a-provision-not-the-bus.md)).
|
||||||
|
|
||||||
They are never the same name. A manifest saying `nats` could otherwise mean either the mesh's
|
They are never the same name. A manifest saying `nats` could otherwise mean either the mesh's
|
||||||
|
|||||||
Reference in New Issue
Block a user