Files
hq/03-DESIGN/01-to-be/19-the-module-protocol.md
jschoubben ce6ae943b7 Merge main: renumber this branch's records around the trunk's
Both lines of work numbered from the same point, so four decision records and one design
document existed twice with different content. The trunk keeps its numbers and this branch
yields — the only rule that scales, because the trunk's are already cited by what merged
before them.

  0117 the bus is the only broker        -> 0125
  0118 a module declares its own seats   -> 0126
  0119 amqp is a provision, not the bus  -> 0127
  0120 the mesh bus is required          -> 0128
  0123 a seat carries its role's protocol -> 0129
  0124 the predecessor is ending          -> 0130
  design 29, what a module declares       -> design 32

Applied to the code repositories too, because a stale reference is worse when numbers
collide than when they dangle: the reader lands on a real record that decided something
else.

Two reconciliations the merge forced, both real:

**0110 was marked wholly superseded and was not.** Its successor says in as many words that
everything 0110 decided about what a seat *is* stands untouched — and two records that
landed on the trunk rest on exactly that part. So it is accepted again, extended rather than
replaced, with a note saying which of its claims moved and where.

**A seat's protocol becomes columns, not fields.** The trunk moved the seat set out of
compiled code into a table the controller owns. This branch had added what a role accepts,
emits and serves to the Go slice. The decision is unaffected and the mechanism is better for
it: giving a role a protocol is now a write rather than a rebuild, which is the trunk's own
argument applied to what this branch added.

One check still fails and it fails on main too: a record resting on ADR 0112 while that is
still 'proposed'. Left alone — it is not this merge's to answer.
2026-09-27 18:23:41 +02:00

14 KiB

layer, status, code, updated, decisions
layer status code updated decisions
to-be proposed
mesh-sdk src
mesh-tools src/broker-nats.ts (and broker-amqp.ts until the rollout)
mesh-controller internal/link
2026-09-26
02-DECISIONS/0095-the-control-plane-is-the-way-to-ask-a-module.md
02-DECISIONS/0106-the-bus-is-nats.md
02-DECISIONS/0128-the-mesh-bus-is-required-not-ambient.md
02-DECISIONS/0116-the-bus-is-built-in-five-steps.md
02-DECISIONS/0074-the-wire-is-specified-not-the-types.md
02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md
02-DECISIONS/0043-a-module-broker-account-is-scoped-by-emits-and-consumes.md
02-DECISIONS/0042-the-shape-of-an-event-on-the-wire.md
02-DECISIONS/0047-a-module-runs-its-code-as-its-own-process-with-its-own-account.md

The module protocol

What a module's code and the mesh say to each other. An SDK is an implementation of this in one language and nothing more (ADR 0074).

This is a specification, so it says what is required rather than how anything is arranged. Where it describes current behaviour that is not yet specified-and-conformed, it says so.

Rewritten onto NATS, 2026-09-26 (step 3 of ADR 0116). What ADR 0074 decided is untouched: a floor plus independent capabilities, an implementation legitimate when it claims less, identity from the sealed credential, at-least-once with dedup on x-event-id, and conformance as executable fixtures rather than prose. What changed is the transport beneath all of it — exchanges and queues became subjects and streams. The envelope keeps its shape (ADR 0042).

Statements here marked verified were checked against a running server while the runtime's client was written, not reasoned from documentation.

The shape of it

A floor every implementation needs, and three capabilities that are independent of each other. An SDK implements the floor plus whatever capabilities it claims; a module is refused at build time if it uses a capability its language's SDK does not implement.

part a module uses it to
connection reach the broker as itself
events emit, and react to what others emit
tools answer questions asked of it
provisioning give a consumer an instance of what it provides

The floor: connection

The credential

A module is given a sealed credential as a file, and told where by its declaration. The document:

field is required
url a tls:// URL for the bus, with the account's user and password yes
fingerprint sha256 of the certificate the bus must present yes for a scoped account
node the machine this account was issued for yes for a scoped account
module the module this account was issued for yes for a scoped account

A plain string rather than a document is a bootstrap URL — unscoped, for the moment before a mesh can issue anything. An implementation accepts both and must not treat the second as ordinary.

node and module are not decoration: every subject an implementation touches is derived from them. Its own namespace is mesh.mod.<module>, its consumer is <node>_<module>, its inbox is its own. So a credential without them is refused rather than guessed at — an implementation that fell back to an environment variable would let anything on the machine decide which module it is, which is what the identity rule below exists to prevent.

The credential itself is fetched, never carried in a declaration: a declaration is persisted as state and a sealed secret in a stream is an archive rather than a moment (design 29 §10).

Connecting

  • The connection pins the fingerprint. It does not trust a certificate authority, and it does not skip verification. A bus presenting a different certificate is refused, whatever else is true of it.
  • The certificate must also carry a name the bus is dialled by. Verified: the NATS client exposes no hook to replace hostname verification, so pinning no longer makes it redundant the way it did on AMQP — the pin happens before dialling and the library's own name check happens beside it. A certificate without a matching subject-alternative name is refused at connect, by a library error rather than by anything the mesh says.
  • An implementation creates nothing on the bus: not a stream, not a consumer, not a subject. Streams and durable consumers are the controller's alone (design 25 §3), and a module's account cannot reach the JetStream API to make one. An implementation binds the consumer the mesh created for it, and if it is absent that is a mesh that has not finished assigning the module, not something for the module to fix.

Identity

A module's node and module name come from its credential, never from its environment.

This is not a convenience. It is what makes what a module emits match what the mesh authorised: an environment variable can be set by anything on the machine, and a module that took its identity from one could emit events attributing them to another module. Where an environment variable and the credential disagree, the credential wins and the variable is overwritten.


Capability: events

The subjects

subject carries
mesh.mod.<module>.event.<key> an event that module emitted
mesh.seat.<seat>.event.<verb> an event the holder of that role emitted

Both are captured by the EVENTS stream. An event's source is enforced rather than claimed: a module's account may publish only into its own namespace, so x-source cannot disagree with where the message arrived from.

The event token is load-bearing. A module's namespace also carries its tool calls (mesh.mod.<module>.tool.<tool>), and a stream is defined by a subject filter — without the token the events stream would capture every tool invocation in the mesh, and a tool call must never be persisted.

The consumer

One durable consumer per module, named <node>_<module>, carrying one filter per pattern the module consumes. Durable because an event emitted while a module is restarting is exactly the one that must not be lost.

Created by the controller, bound by the implementation. A module declares what it reacts to and never how delivery works, so it does not name its consumer, does not choose its ack policy or delivery limit, and cannot misconfigure them.

Verified, and it is a trap: a durable name may not contain a dot, while the subject a consumer acknowledges on is $JS.ACK.<stream>.<consumer>.… — two names joined by one. An implementation that treats them as a single string reads correctly in a permission list and is refused as a consumer name. Left wrong, the symptom is every message redelivered forever while the permissions look right.

One consumer may carry filters wider than one handler's pattern, because a module subscribing twice gets one consumer with both. So an implementation still matches the key against its own patterns locally to decide which handlers run — and acknowledges a message no handler wanted, or it is redelivered until it expires.

The envelope

Headers ride as NATS headers; the body is JSON, and the body alone. Verified: the payload is the event's body, not the whole envelope re-encoded — an implementation that nested the envelope would pass every one of its own tests and agree with no other, which is the exact failure the conformance fixtures exist to catch. The key is recovered from the subject, not carried twice.

header is required
x-event-id a unique id, made by the emitter yes
x-source the module, context or node that emitted it yes
x-node the machine it was emitted from yes
x-time emit time, RFC-3339 yes
content-type always application/json yes
x-causation-id the event or command that caused this one no
x-schema a version of the body's shape no

An unknown x- header is ignored, never refused. An event is observed by parties that need not all understand every header, and an implementation that refused one would make adding a header a breaking change for everybody.

Delivery

At-least-once. Deduplication is on x-event-id, which only the emitter can produce — a consumer cannot tell a redelivery from a second event any other way.

On NATS the id does double duty: an implementation passes it as the publish's message id, so the server also refuses a duplicate inside its window. That narrows the window in which a consumer has to deduplicate; it does not remove the requirement, because the window is finite and a redelivery after it is still a redelivery.

What is true, checked (2026-09-16)

Go emits all five required headers; the SDK requires exactly those. x-causation-id and x-schema are optional — the SDK sets them when a handler has a causation or a schema, and reads them back; a bare event carrying neither is correct. So the envelope agrees across the two implementations. x-schema is available for versioning a body's shape and is set by whoever has a version to declare.


Capability: tools

A module's tools are its operator-facing surface.

  • A tool is served on mesh.mod.<module>.tool.<tool>, with a queue group — so several runtimes serving one tool compete for a call rather than each answering it.

  • A call is request and reply on core NATS, never a stream. A tool call is not persisted: a lost one is a timeout the caller already handles, and a stream of them would be the mesh's most voluminous and least valuable traffic competing for retention with the messages that matter.

  • The reply goes to the inbox the request carries. A responder may answer it because its account is granted allow_responses — one reply to the subject of a message it actually received, and nothing wider. That is what makes a per-account inbox prefix workable: no user is ever granted _INBOX.>, so without it a responder could not reach the caller at all.

  • A module may now call a tool, which on AMQP it could not. Verified: two modules on separate connections, one serving and one calling, with an answer returned and a throwing handler reaching the caller as an error rather than a timeout. Issue 049 recorded the old limit — a scoped account could not declare the reply queue a caller needs — and ADR 0095 routed every ask through the control plane because of it. That constraint is gone, and each account's own inbox prefix replaces it.

    ADR 0095 is not thereby reversed: the control plane remains a way to ask, and a person asking a module should still go through it. What changes is that "a module-to-module call, if one is wanted, is a later decision" is no longer a question about capability. It is a policy question, and the answer the mesh already has is uses: a module declares the seat it calls, and the permission follows the declaration.

  • A module declares nothing about being asked — serving a tool is being askable.


Capability: provisioning

A provider ships the provisioner that creates instances of what it offers (ADR 0040).

direction carries
grant, in the provision, who is asking — a module on a machine, not a machine — and what the consumer contributed
credential, out the fields the provision promises a consumer

Who is asking is one thing with two parts. A machine routinely runs several modules wanting the same provision, so a grant addressed to a node alone does not name a consumer, and withdrawing one would take another's away.

Checked, and it agrees (2026-09-16)

This looked like the sharpest disagreement and was not one. The live wire is the contributions file — as, secret, node, at, values — and it is the same on both sides. The types that disagreed (Grant, Interface in the SDK's contracts) were dead: exported, imported by nothing, describing fields the wire does not carry. They have been removed. The lesson kept: a type beside the wire that has drifted from it is worse than none, which is why the wire is specified and implementations are checked against it rather than trusted to still match a hand-kept shape.


How an implementation is checked

Per capability, against fixtures rather than prose — a specification nobody can run is a document two implementations drift from while both believe they conform.

Rule Checked by
The floor is the floor Every implementation reads the same credential fixture, and refuses one whose fingerprint does not match what the broker presents.
Identity comes from the credential A fixture sets an environment that disagrees with the credential; the emitted event carries the credential's.
The envelope is the envelope An emitted event is compared header by header against a fixture; a missing required header fails, an unknown x- header is accepted.
Delivery is at-least-once A fixture delivered twice is handled once.
A grant names a consumer A grant fixture is read by every implementation and yields the same module and the same node.
A partial SDK is legitimate An implementation claiming the floor and events passes those suites and is listed for them; a module using tools in that language is refused at build time with the reason.