Compare commits

..
Author SHA1 Message Date
mesh-admin 1513bbaac9 Merge pull request 'An older merge does not move a source' (#120) from fix/an-older-merge-does-not-move-a-source into main 2026-09-28 03:12:40 +00:00
jschoubben 0014984116 An older merge does not move a source
The forge announces what it finds merged, and an old merge surfacing late moved the recorded head
backwards and rebuilt everything built from that repository, once per old merge. A merge made
before the source was last seen is history; one that says nothing about when is taken as news.
The catalogue now carries when each source was last seen. The bus-records test follows #116:
a module's tools are every one under its own name.
2026-09-28 05:12:37 +02:00
mesh-admin d6e49dbd68 Merge pull request 'A base the registry already holds is not pulled from upstream again' (#119) from fix/a-mirrored-base-is-not-pulled-twice into main 2026-09-28 02:48:35 +00:00
jschoubben 3756bb3460 A base the registry already holds is not pulled from upstream again
A base is named by digest, and a digest the mesh's registry holds under the module's repository
is the same bytes whatever upstream would say. Asked on every build, the public hub's anonymous
pull limit was reached on the first merge that rebuilt a whole catalogue, and every module whose
base lives there failed on a copy it did not need.
2026-09-28 04:48:32 +02:00
mesh-admin 60be9c5360 Merge pull request 'A module hears what it consumes: its consumer is raised with the bus, and it pulls it' (#118) from fix/a-module-hears-what-it-consumes into main 2026-09-28 02:29:34 +00:00
jschoubben da31bcb11e A module hears what it consumes: its consumer is raised with the bus, and it pulls it
Every module moved onto the bus by the rollout was issued on the old one, so none had a consumer
waiting; and the grant named a push delivery a runtime's client never binds, while the pull it
does make — asking about its consumer, asking it for messages — was refused. The consumers a
module's declarations imply are now raised whenever the bus is, and the grant is the pull.
2026-09-28 04:29:32 +02:00
mesh-admin f03e7b33c9 Merge pull request 'The controller may ask any module's tool' (#117) from fix/the-controller-may-ask-a-tool into main 2026-09-28 02:21:26 +00:00
jschoubben 0baf727f36 The controller may ask any module's tool
The control plane is the way in for tool calls (novox/hq ADR 0095): a person or an agent asks
through it, so it alone may publish to every module's tool subject. The first ask on the new bus
was refused the publish.
2026-09-28 04:21:25 +02:00
mesh-admin 94dd49a968 Merge pull request 'A module serves every tool under its own name, and may answer' (#116) from fix/a-module-serves-its-own-namespace into main 2026-09-28 02:14:52 +00:00
jschoubben 83a298e7e0 A module serves every tool under its own name, and may answer
Every module that served a tool was refused the subscription on the new bus: the grant listed
tools from a manifest field no module fills, because the tools a module serves are what its code
answers and a second copy of that list would be a second source of truth. The grant is now the
module's own tool namespace; nothing else may subscribe it, a caller is still granted per tool by
name, and a module may answer what it was asked.
2026-09-28 04:14:47 +02:00
mesh-admin 220b79f5cd Merge pull request 'rollout check dials the bus the way the mesh does' (#115) from fix/the-check-dials-as-the-mesh-does into main 2026-09-28 02:05:35 +00:00
jschoubben e62201e227 rollout check dials the bus the way the mesh does
The probe connected bare, and a bus that requires TLS and a user refused it at the handshake —
so the check reported the standing server as absent. It now dials with the controller's own
credential and pin, which is the one fact the check is there to report.
2026-09-28 04:05:32 +02:00
mesh-admin 0ab9b86f0a Merge pull request 'An edge is recorded by path, like the artifact it points at' (#114) from fix/an-edge-is-recorded-by-path into main 2026-09-28 01:52:10 +00:00
jschoubben c7aabd3037 An edge is recorded by path, like the artifact it points at
The test pinned what a build stood on to the address the builder pulled from; the edge names
another module's artifact and is kept the way that artifact is (novox/hq 04-ISSUES/102).
2026-09-28 03:52:08 +02:00
mesh-admin c74d990cee Merge pull request 'A build records the bases it was handed, and the mesh reads its edges from builds' (#113) from feat/build-edges-are-recorded into main 2026-09-28 01:51:16 +00:00
jschoubben 35252af665 A build records the bases it was handed, and the mesh reads its edges from builds
Bases reach a recipe as build arguments, so the digest was never in the file the builder read
edges from: no build on the mesh recorded what it stood on, and 'build --on', the bases-first
order and the merge follow-up all walked a graph with no edges (novox/hq 04-ISSUES/131). The
builder now reports every base it resolved; the controller records them by artifact path and
reads the newest build's edges from the store, since a recorded manifest carries no build.on.
2026-09-28 03:51:14 +02:00
mesh-admin aab6ded41b Merge pull request 'One bus: the AMQP transport is gone from the controller' (#112) from feat/one-bus into main 2026-09-28 01:36:27 +00:00
jschoubben aecac5bda2 One bus: the AMQP transport is gone from the controller
The mesh runs on the seat's bus alone (novox/hq ADR 0131, design 28 task 5.5). The old
transport's consume loop, build request, tool ask, management API and account scoping are
deleted, and the bus switch with them; the controller connects to the broker seat and to
nothing else. The store-window tests keep their assertions on a bus-less fake, and the tests
that only made sense for the old transport's in-memory holding go with it.
2026-09-28 03:36:16 +02:00
mesh-admin 30362118a1 Merge pull request 'The controller follows the subject it decodes' (#111) from fix/the-controller-follows-what-it-decodes into main 2026-09-28 01:15:18 +00:00
jschoubben 81e76fa485 The controller follows the subject it decodes
The decoder named the forge's merge subject as the fourth thing followed and the
list was three long: every message that fell through to that switch panicked the
control plane (2026-09-28). The entry was written and lost between two attempts
at the same edit. A test now walks the list; the composed grants and the genesis
template carry the subject.
2026-09-28 03:13:57 +02:00
mesh-admin 12ed35e87d Merge pull request 'A merge on the forge builds what it moved, bases first' (#110) from feat/a-merge-on-the-forge-builds-what-it-moved into main 2026-09-28 00:59:04 +00:00
jschoubben 525f10b858 A merge on the forge builds what it moved, bases first
The controller follows the forge's merges (novox/hq 04-ISSUES/131). For each
module recorded as built from that repository and branch it records the move to
the merge commit and builds it — bases first, because a module built before the
module it stands on is built against the old one and reports success, and a base
that fails stops what stands on it. Nothing is pushed here: what a finished build
does to the machines running the module stays the upgrade's decision.

Two more things the same ordering gives: `build --behind` builds bases first, and
`build --on <module>` rebuilds everything that stands on a module — the rebuild a
changed base needs, which "behind" does not see because their sources did not
move.
2026-09-28 02:59:02 +02:00
mesh-admin 0547316cf2 Merge pull request 'rollout hand: a machine's membership, minted afresh and handed to an operator once' (#109) from feat/rollout-hand into main 2026-09-28 00:40:25 +00:00
jschoubben 9fe9b5349c Merge pull request 'A store row keeps its seat's protocol, and a holder may take work from its queue' (#108) from fix/store-seats-keep-their-protocol into main 2026-09-28 02:21:24 +02:00
jschoubben d1e488efaf Merge pull request 'A store row keeps its seat's protocol, and a holder may take work from its queue' (#108) from fix/store-seats-keep-their-protocol into main 2026-09-28 00:09:28 +00:00
jschoubben c5dc7e732a A store row keeps its seat's protocol, and a holder may take work from its queue
The seat table has name, scope, delivers and decision, and the protocol ADR 0129
gave a seat lives only in the compiled defaults; loading the rows dropped it, so
no role's work queue was ever raised and the first build submitted over the new
bus met "no response from stream". Until the table gains the columns, a row with
no protocol keeps the compiled one of its name. And the holder of a seat is
granted what taking work from its queue needs — asking about the worker consumer
it binds, and acknowledging on it — which the first machine to try was refused.

The control plane's own seat placeholders no longer include the old bus's port,
which the switch removed with the variable.
2026-09-28 02:08:56 +02:00
jschoubben 7efcccd013 Merge pull request 'The build machine takes work on the bus its credential names, and the work queue has a taker' (#107) from feat/the-build-machine-takes-work-on-nats into main 2026-09-27 23:59:56 +00:00
jschoubben 964285f08c The build machine takes work on the bus its credential names, and the work queue has a taker
Two halves of one gap the first build over the new bus met. The machine decided
its bus from a variable its container never received, so the credential the mesh
sealed to it went unread; a credential for the new bus names the bus by scheme and
carries user, password and fingerprint beside the address, and that is enough to
dial it, pinned. And the roles' work queues were raised with no holders, so the
consumer a machine binds to take work was never created: the holders are read
from the catalogue and the handover record, as the resolver reads them.
2026-09-28 01:59:20 +02:00
jschoubben 5698dda11f Merge pull request 'A principal may hear what its consumer delivers' (#106) from fix/a-principal-may-hear-its-consumer into main 2026-09-27 23:47:29 +00:00
jschoubben 6005a8471f A principal may hear what its consumer delivers
A push consumer delivers on _DELIVER.<its name>, and a client bound to it
subscribes exactly that. No principal was granted it, and the server refused
every one the first time it bound a consumer: the control plane, each machine,
and a module would have been next. Each kind is granted its own consumers'
delivery subjects and no other's. The line announcing the raised bus printed the
URL with the credential in it; the address alone now.
2026-09-28 01:46:16 +02:00
jschoubben e2ee0dfe98 Merge pull request 'The bus account has JetStream, and the control plane's client has its own inbox' (#105) from fix/the-bus-account-has-jetstream into main 2026-09-27 23:40:38 +00:00
jschoubben 70341cfbc7 The bus account has JetStream, and the control plane's client has its own inbox
Two refusals the first live connections met. A user in the MESH account was told
"JetStream not enabled for account" the first time it bound a consumer: with
accounts defined, JetStream is enabled per account, not only globally — the
account's setting, which the mesh owns, not the server's block, which it does not.
And the control plane's client used a random inbox prefix where it is granted
exactly _INBOX.<its user>.>, so the server's first answer could not reach it. The
prefix now follows from the user in the URL, for every principal that dials so.
2026-09-28 01:40:10 +02:00
jschoubben 77643aa3f4 Merge pull request 'The control plane pins the bus's certificate, and keeps its password out of errors' (#104) from fix/the-controller-pins-the-bus-certificate into main 2026-09-27 23:35:56 +00:00
jschoubben 1fd6194ff8 The control plane pins the bus's certificate, and keeps its password out of errors
The bus presents the mesh's own certificate, which names nothing a public verifier
accepts; the client verified by name and failed against a bus that was answering
("certificate is not valid for any names", 2026-09-28). It now pins the leaf's
fingerprint from MESH_BROKER_CERTIFICATE, as every host does. And a connection
error named the whole URL, password included — the address alone now.
2026-09-28 01:35:20 +02:00
jschoubben c37018fdd2 Merge pull request 'The control plane serves and pushes on the bus it is told to' (#103) from feat/the-controller-serves-on-nats into main 2026-09-27 23:30:52 +00:00
jschoubben 3907ea0db0 The control plane serves and pushes on the bus it is told to
The seams were there and nothing chose a side: serve, push, ask and build all
opened the old bus's connection and declared over its channel, whatever
MESH_BUS_NATS said. So the switch moved every host and left the control plane
unable to follow — "this control plane has no MESH_BROKER_AMQP" with the new bus
named and standing (2026-09-28). That was task 4.3 of design 28, still open.

One place now decides: connectLink reads the switch, refuses both buses named at
once, raises the new bus's streams and this controller's consumers when it is
handed the inventory, and opens the link over whichever bus it is on. Every
caller that sent a declaration or asked a tool through the old channel goes
through the server's bus instead, which the new transport has and the channel is
not. OverNats is that outbound: a declaration is a JetStream publish into the
node's own subject, an event is announced on the subject its name derives to, a
tool is request and reply on the module's tool subject.
2026-09-28 01:29:44 +02:00
jschoubben 40f5e9a41c Merge pull request 'A machine may bind its consumer' (#102) from fix/a-node-may-bind-its-consumer into main 2026-09-27 23:18:20 +00:00
jschoubben 2c2eb51878 Only CONSUMER.INFO was missing from a machine's grants; the rest was already there 2026-09-28 01:17:40 +02:00
jschoubben aa2d0b51ea Golden: a machine's user may bind its consumer, ack, and hear its inbox 2026-09-28 01:17:15 +02:00
jschoubben 64d154d9d7 A machine may bind its consumer and hear the answer
Binding to a consumer asks the server about it and hears the answer on the
client's inbox; hearing a declaration acknowledges it. A machine's user was granted
none of that and was refused the first time one dialled a permissioned server:
"this node cannot read its declarations". Its inbox is its own prefix, which the
host now sets.
2026-09-28 01:16:46 +02:00
jschoubben ffa390f916 Merge pull request 'The mint leaves the control plane's old-bus secret alone' (#101) from fix/mint-leaves-the-control-planes-old-secret-alone into main 2026-09-27 23:08:35 +00:00
jschoubben 4d62e6caf1 The mint leaves the control plane's old-bus secret alone
The control plane is a module too, and its broker secret is the old bus's
credential it is still using while the mint runs. Writing the new bus's blob there
cut the mesh off from its own old bus mid-move. Its new-bus credential is the
controller principal's bus secret; the module principal is skipped.
2026-09-28 01:08:11 +02:00
jschoubben 386ae676ca Merge pull request 'The control plane mounts the bus secret it reads' (#100) from fix/the-controller-mounts-its-bus-secret into main 2026-09-27 23:03:42 +00:00
jschoubben f8a9c3d6bc The control plane mounts the bus secret it reads
MESH_BUS_NATS_FILE named /run/secrets/bus and nothing put a file there: the
manifest binds each secret explicitly, and the switch added the secret and the
variable but not the bind. Found live — the control plane came up on the new bus
and could not read its own credential.
2026-09-28 01:03:18 +02:00
jschoubben 83671fae5f Merge pull request 'The network map resolves each machine with the seat holders on record' (#99) from fix/the-network-map-knows-the-holders into main 2026-09-27 22:56:55 +00:00
jschoubben 9b715524a2 The network map resolves each machine with the seat holders on record
Without them, a machine running the next holder of a seat beside the current one
resolves as two holders, is refused, and drops out of the map — and with it the
address every other machine composes for what it offers. Found live: the control
node vanished from the private network the moment the new bus was assigned beside
the old one, and nothing on any machine could be composed.
2026-09-28 00:55:39 +02:00
jschoubben e06fc1ed16 Merge pull request 'The control plane speaks the new bus' (#98) from switch/the-controller-speaks-nats into main 2026-09-27 22:38:27 +00:00
jschoubben 13d7c5c5dd Merge pull request 'The mint names the bus by bare host, and can mint again' (#97) from fix/mint-bare-host-and-again into main 2026-09-27 22:37:36 +00:00
jschoubben 7b02feaebb The mint names the bus by bare host, and can mint again
BareAddress adds a scheme where none was, so the host it yielded carried one and
every URL built from it carried two. Caught before a push: the host is now taken
with no scheme and no port, and every URL adds its own. `rollout mint --again`
mints every credential afresh for exactly this case — a mint that was wrong before
anything received it.
2026-09-28 00:37:10 +02:00
jschoubben bd10e2c695 Merge pull request 'The mint finds the bus on the hub' (#96) from fix/mint-finds-the-bus-on-the-hub into main 2026-09-27 22:34:41 +00:00
jschoubben 4b209d944d The mint finds the bus on the hub
The map of who is where on the private network lists the machines placed around
the hub, not the hub — and the control node is the hub, and runs the new bus.
Found on the first live run: refused for having no address. The address every
machine already dials the current bus at is the same machine, so that host with
the new port is what they are told.
2026-09-28 00:34:17 +02:00
jschoubben 84024cbdb6 The control plane speaks the new bus
One environment variable moves it (design 25): MESH_BUS_NATS, read from the `bus`
secret `rollout mint` sealed to its machine, and the two that named the old bus go,
because being told about both is refused at start. Registered and built ahead of
the push that flips it, so the flip and the bus it flips to arrive in one
declaration — the same push that starts the new bus and hands this machine its
membership. Deliberately not merged until every credential is minted.
2026-09-28 00:31:57 +02:00
jschoubben ad2eed2f71 Merge pull request 'The move mints every credential and tells each machine its membership' (#95) from feat/the-move-mints-and-delivers into main 2026-09-27 22:30:58 +00:00
jschoubben e8aa7ed9e7 The move mints every credential and tells each machine its membership
`rollout mint` gives every principal the new bus will have a credential it does
not yet have and puts each where its owner reads it: a machine's as a membership
— bus address, fingerprint, password, transport — sealed into its declaration
(migration 0041, the `bus-membership` resource the host reads after applying); a
module's as its broker secret, through the same delivery `module issue` uses; the
control plane's own as its `bus` secret. Idempotent, and worked out from where the
bus's module is assigned rather than from this process's environment, because this
process is still on the old bus when it runs and must be.

This is the half of design 28 task 5.2 the first live attempt found missing: a
credential was minted only at enrolment, at `module issue` and for a person, so no
machine already enrolled could ever be moved. `rollout check` was right to refuse;
now there is something to run first.
2026-09-28 00:16:24 +02:00
jschoubben 337aaea123 Merge pull request 'The first handover records the standing holder without re-judging it' (#93) from fix/record-the-standing-holder into main 2026-09-27 21:35:08 +00:00
jschoubben 4c41628b20 The first handover records the standing holder without re-judging it
On a mesh that predates the record, every handover has to begin by writing down who
already holds the seat — otherwise the next holder cannot be assigned beside it,
because two eligible claimants with nothing on record are refused. Found on the
live mesh minutes after 0040 moved the bus seat's row: the standing holder no
longer satisfies what the seat delivers, on purpose, and so could not be recorded,
and so nothing could stand beside it.

Recording who already holds is not making a new holder. Derivation never read
what the seat delivers, so the standing holder holds regardless; when nothing is
on record and the named assignment claims the seat at its scope, only that claim
is checked. Every change of holder is still judged in full.
2026-09-27 23:34:40 +02:00
jschoubben ae7fb520d7 Merge pull request 'AMQP is not a provision: the bus seat delivers the bus, and the word is refused' (#92) from feat/amqp-is-not-a-provision into main 2026-09-27 21:30:09 +00:00
jschoubben f325073982 AMQP is not a provision: the bus seat delivers the bus, and the word is refused
Two halves of novox/hq ADR 0131. Migration 0040 moves the mesh-broker row from
`amqp` to `mesh-bus`, so the seat's holder answers for the mesh's bus and not for
the wire protocol the old broker spoke — which is what let only the retiring
broker hold the seat that names the bus. Safe under the current holder: the
control plane composes its own address through the seat by name and the overview
derives holders by name; only registration and provision resolution read the
column. What must not happen in between is re-registering the current holder.

And the parser refuses a manifest that provides or requires `amqp`, each refusal
saying what to do instead: a module reaches the mesh's bus through the sdk and
depends on the seat, not on a protocol. A whole-catalogue test asserts nothing
beside this checkout names it; the three modules that did are removed there.

Two tests that used the old broker as a fixture now use the module that replaces
it or a manifest this package owns.
2026-09-27 23:29:00 +02:00
jschoubben 33c4e4be34 Merge pull request 'A seat is handed over as one act, and the holder is on record' (#91) from feat/seat-handover into main 2026-09-27 21:22:50 +00:00
jschoubben 585a6abbdd A seat is handed over as one act, and the holder is on record
`seat <name> --to <node>/<module>` makes one assignment the holder of a seat in
the same write that removes the previous one. The row is new (migration 0039);
without one, the resolver derives the holder as it always did — the sole eligible
assignment, two refused — so nothing changes for a mesh that never hands a seat
over. With one, the recorded assignment holds and any other whose module could
hold the seat is eligible and silent: not refused, not holding. That is what lets
the next holder run beside the current one until the switch (hq design 26, design
28 task 5.3, ADR 0131).

Why: the controller finds its own bus through a seat, and the day that seat was
left with nobody in it — because two eligible holders could not coexist and the
old one's claim was taken away — the control plane looped for two hours while
every service stayed up. A handover that is never empty in between is the fix,
not a workaround for it.

`CanHold` is the one judgement of whether a module may hold a seat — claims it at
its scope, provides what it delivers, against the store's row — shared by
registration and the handover so they cannot drift apart. The holding belongs to
the assignment and goes when it does, so a seat never points at nothing running.

Tests: the resolver with and without a record, on the same and another machine,
under a former name; the store's row replaced not added, refused for an
unassigned target, removed with its assignment; CanHold's four answers and that
they follow the store. Full suite green against a real NATS and store.
2026-09-27 23:22:20 +02:00
jschoubben 8d52a2cfb0 Merge pull request 'The move ends with the old broker going, not staying' (#90) from fix/rollout-retires-the-old-broker into main 2026-09-27 21:05:47 +00:00
jschoubben 1cfe6be9c4 The move ends with the old broker going, not staying
`rollout check` said the old broker stays running as an ordinary provider of
amqp, and this was not its retirement. That was ADR 0127, which ADR 0131 has
superseded: AMQP is not a provision, so once every machine reports on the new bus
nothing of the mesh speaks to the old broker and its module is unassigned. The
plan says so, as its last step.

The flag that made the "it stays" line conditional is gone with the line — there
is no case in which the broker is kept. The test that pinned the opposite now
pins this, and says which record changed under it. The stale citation of a
record numbered 0119 is corrected while here.
2026-09-27 23:04:59 +02:00
jschoubben 4e4481b6f2 Merge pull request 'The store owns the seat set, so only the control plane may judge a claim' (#89) from fix/the-store-owns-the-seat-set into main 2026-09-27 20:00:05 +00:00
jschoubben 63ca073938 The store owns the seat set, so only the control plane may judge a claim
A claim on a seat was checked against `SeatNamed` inside `ParseManifest`, and the
build machine parses manifests too. It has no store, so there it answered from the
set compiled into the binary — a copy of data the control plane owns (ADR 0122).

When the two disagreed, that copy won where it mattered. The store's row said the
bus seat answers for `amqp`; the binary's said `mesh-bus`; and a holder that
provides `amqp` was refused at build time for not providing `mesh-bus`. The seat
went unheld, the controller lost the address it composes through that seat, and the
control plane crash-looped on a bus that was healthy the whole time.

So the two checks that read the set — a seat's scope, and what its holder must
provide — move to CatalogueProblems, which runs only in the control plane and only
after UseSeats has replaced the set with the store's. The parser keeps what it can
judge from the manifest alone, the reserved-namespace rule included.

A test pins it: the same manifest, two different values in the store, and the answer
follows the store both times. It fails if the check moves back.
2026-09-27 21:59:40 +02:00
jschoubben b244a768a3 Merge pull request 'The Go base has to be 1.26: the NATS client requires it' (#88) from fix/go-126-base into main 2026-09-27 19:00:32 +00:00
jschoubben 81d52719f4 The Go base has to be 1.26: the NATS client requires it
`nats.go v1.54.0` declares `go >= 1.26`, so `go mod tidy` puts the directive at
1.26.0 and will not leave it at 1.25. The base image is pinned by digest at
1.25.14-alpine, so nothing in this repository compiles against it — caught when the
build machine's own build failed with "go.mod requires go >= 1.26.0 (running go
1.25.14)".

Moved to 1.26.8-alpine by digest, same flavour as before. The pin stays a digest:
the point of pinning is that the compiler does not change underneath a build, and
that is still true one version along.

Two other manifests carry the same pin for the same reason — they compile this
repository's code — and move with it in the catalogue.
2026-09-27 20:58:08 +02:00
jschoubben f6a93fe74c Merge pull request 'The bus on NATS: both transports behind seams, and the rollout switch' (#87) from feat/nats-genesis into main 2026-09-27 17:36:41 +00:00
jschoubben 497f8ea567 Merge main: the trunk renamed the seats and made them data
Both branches changed the seat set from the same starting point, so every number
collided and every `mesh-*` name existed twice. The trunk's numbers and names win:
this branch's records became 0129/0130 and its migrations 0037/0038, and the
hardcoded rename map gave way to the trunk's `seat_alias` table — a rename is a
row now (ADR 0122), not a recompile.

Three of my checks were wrong and the merge is what showed it:

A seat with an empty protocol is a marker, not an incomplete declaration. Most
node-scoped seats are markers — which module is this machine's packet filter —
and refusing one refused most of the set, the showcase module included. A
mistyped field name is already refused by the parser, so an empty protocol was
written as one deliberately.

A claim on a seat this manifest does not declare is not the parser's to judge. A
module may hold a seat another module declared; that is the whole reason ADR 0126
has callers name the seat and not its provider. Whether the seat exists is a fact
about the catalogue, so the refusal is at registration, where every declaration
is in view.

And a seat may share a name with the provision it delivers. `git`, the npm
registry and the artifact store still do, because renaming a delivering seat
cascades to every consumer requiring it, with a window where a holder stops
resolving mid-flight. The trunk deferred exactly those three on purpose.

Full suite green against a real NATS and store.
2026-09-27 18:50:18 +02:00
jschoubben dded086b54 rollout check: whether this mesh could move its bus, and what is missing
The rollout moves every node at once, so there is nothing to inspect afterwards and no
half to roll back — either the mesh was ready or it was not. That makes the readiness
question the valuable half: it costs nothing, it can be asked of a mesh that is serving as
many times as you like, and every answer is a thing somebody can go and fix.

It reads from records and dials once. Is a bus answering, does a machine hold the seat, has
that machine been sent the composed user list, does every machine have a credential for the
new bus, does every module that speaks. Each missing thing names its own next step, because
"not ready" that cannot be acted on is not an answer — and this is read at the point where
the next step is irreversible.

**A machine with no credential is the one that must stop it.** It keeps running and cannot
come back, and afterwards there is no bus to tell it anything over, so the remedy has to
happen first. The message says so.

A module that never reaches the bus is not counted as missing a credential. A third of the
catalogue never speaks, and listing those would bury the ones that matter.

`rollout --confirm` refuses and says why: the move is not being written before its check has
been run against a real mesh. And the plan it prints says the old broker stays — it remains
an ordinary provider of `amqp` for whatever else uses it, which on this installation is a
whole automation layer that has nothing to do with the mesh. This move is not its retirement,
and that is why it is survivable: what breaks if it goes wrong is the mesh's ability to
change things, not the services its modules serve.
2026-09-27 17:59:01 +02:00
jschoubben e7b5100324 Merge pull request 'The mesh owns the operator's ~/.ssh: account fact + home-scoped resources (to-be 29)' (#86) from feat/to-be-29-mesh-owns-ssh into main 2026-09-27 15:51:55 +00:00
jschoubben 8ceec32692 The mesh owns the operator's ~/.ssh: account fact + home-scoped resources (to-be 29)
A node carries its operator account (name + home; migration 0036, Node.Account,
SetAccount, 'node account' CLI). The account and its home are offered as
machine facts ${machine:account} / ${machine:account-home}, and machineInto
now resolves placeholders in a resource's path and owner (not just content), so
a module writes into a person's home naming what it cannot know. A RosterFile
gains Home: the file is placed under the account's home and chowned to it, its
template sees each node's Account, and a machine with no account gets none —
this is how the ssh Host blocks for every node reach a person's ~/.ssh. Roster
carries per-node accounts (Rendering.Accounts). Tested, including ssh-client
composed end-to-end. Not deployed.
2026-09-27 17:50:56 +02:00
jschoubben 5fcde512bc A build is announced under both names on the old bus, or merging breaks the live mesh
Found by asking what merging this would do to the mesh that is actually running — the
only place the question could have been asked, because the tests were green and both
buses were self-consistent.

Moving the build outcome to the role means a catalogue built from the current manifests
listens for the role's name. The catalogue *already running* listens for the module's,
because that is what it was told when it was installed. The two do not meet, so merging
as it stood would have stopped the live mesh's module graph being updated — silently,
since a binding that matches nothing is not an error.

A rename on a live bus needs the publisher and the subscriber to change together, and a
deployment cannot promise which arrives first. So the old bus announces under both names
and the order stops mattering. The module's own name retires with the bus, in step 5's
list; nothing has ever run on the bus being built, so there is no legacy name there and
this doubling has no counterpart.
2026-09-27 17:39:39 +02:00
jschoubben e5007a7daa The user list is composed before anything moves onto the bus
Found by reading the live mesh's own notes before touching it, which is where this
was heading next.

Composing the bus's user list was gated on the controller already being on the new
bus. That cannot work: the server needs its user list *before* anything moves onto
it. Step 2 of the whole change is exactly that — the server stands in the mesh
carrying nothing, on its own ports, while every node stays where it is. Under the
old gating that step was impossible: the module would come up, find no accounts
file, and its entrypoint would wait for one the controller had decided not to write.

So the only question is whether this machine runs the module that asked for the
file. A mesh that never moves has written a user list nothing reads, costing a few
hundred bytes on one node. The reverse cost a step that could not be taken.

Pinned by a test over the records of a mesh mid-change: everything running, nothing
on the new bus, and a user list that contains the controller — because a file
without it is a bus its own writer cannot connect to.
2026-09-27 17:33:52 +02:00
jschoubben cb77f35a27 A module may watch a role's events, and the catch-up turns out to be unnecessary
Moving the build outcome onto its role broke the one module that consumes it, and my own
agreement check passed anyway. The catalogue's subscription derived
`mesh.mod.mesh-build-machine.event.built` — a module namespace for a role's event, which no
such module owns — so it started, connected, and its graph stayed empty. The check compared
names, and the names agreed: the build machine does emit `built`. Only the subjects
disagreed, and a subscription that matches nothing is silence.

A consumed name is a module's event unless it names a role, and this package cannot tell by
looking — so whoever resolved the declaration says which, the way it already does for a seat
held or used. A module that watches a role gets the role's event subject and a consumer
filtered on it; watching grants subscribe and nothing else, because hearing what a role
announced is not taking part in it.

The check now compares the two halves that actually have to match — the subject a consumer
subscribes against the subject an emitter publishes — with a case pinning that it catches
this exact confusion. Comparing names was checking the easy half.

**And that answered the open question about catch-up: there is nothing to build.** The
mechanism exists because a queue on the old bus receives only what is published after it is
bound, so everything built before the catalogue existed was announced to nobody. A stream is
a log and a consumer is a position in it: a consumer created afterwards starts at the
beginning, so the builds are simply there. Asked of a real server, since the whole decision
rested on it — three builds published with nothing listening, then a consumer created, and
all three waiting for it.
2026-09-27 17:22:30 +02:00
jschoubben 71dbca6392 Merge pull request 'A module declares its fail2ban jail; mesh composes per node (to-be 31 mechanism)' (#85) from feat/a-module-declares-its-jail into main 2026-09-27 15:21:03 +00:00
jschoubben 47d412e13f A module declares its fail2ban jail; the mesh composes them per node (to-be 31)
The mechanism, mirroring Filtering: a module declares Jails (name, failregex,
jail stanza) naming no node/path (ADR 0112); the intrusion-prevention holder
declares Jailing (where composed jails go); the mesh gathers every assigned
module's jails into one jail.d file (a fixed id the fail2ban service restarts
on) plus a filter.d file per jail. A node not running a module has none of its
jails. Tested. Behaviour-neutral until a service module declares a jail — the
per-service content (postgres/mssql/mailu failregex+logpath) is authored next,
against how each container actually logs.
2026-09-27 17:20:50 +02:00
jschoubben 53e8f5bdd8 A person may be issued, listed and revoked
Design 25 §7's first item, which existed as a permission model and as nothing a person
could actually be given. There is a record now, and three commands.

Their authority is a list of tools and nothing else. Not a module: they hold no seat,
nothing is addressed to them, nothing is delivered to them, and they have no consumer to
acknowledge. What they have is permission to ask — which is why there is no scope and no
node in the record.

Stating what somebody may call replaces what was there rather than adding to it: a list
that could only grow is a permission nobody can take back. Forgetting somebody takes
their credential with them, because a person's row gone with their bus user left behind
is a credential that still works and that nothing derives — the worst of both, since it
keeps working and nobody can explain why.

The credential is printed once and the mesh keeps only a hash, the same contract a token
has. And it starts working at the next composition rather than immediately, because the
bus's users are a file — said out loud in both the issue and the revoke messages, since
"revoked" that still works for another minute is worth knowing about.

Four properties held by test, each a way of being wrong that would not announce itself:
a person may publish exactly the tool subjects they were given and nothing on control,
nodes or events; they cannot answer a request; changing the list removes what is no longer
named; and forgetting them revokes them.
2026-09-27 17:07:19 +02:00
jschoubben 6ecd631b3e Merge pull request 'A module can name the mesh's range: ${machine:mesh-range}' (#84) from feat/machine-names-the-mesh-range into main 2026-09-27 14:52:23 +00:00
jschoubben 19d2725c13 A module can name the mesh's range: ${machine:mesh-range} (novox/hq ADR 0112)
A module cannot know the private network's CIDR — it is a per-mesh value chosen
at genesis — but sometimes must name it: an intrusion filter that must never
ban a tunnel peer. Carry the overlay range on the Rendering and offer it as the
machine fact mesh-range, the same way a machine's own address is offered, so the
module names it rather than hardcoding a value (data is the mesh's). Absent when
the mesh has no range. Enables the fail2ban ignoreip fix.
2026-09-27 16:52:00 +02:00
jschoubben 8e2824201a Genesis can raise a mesh on the new bus, and the carried user list is checked against the composer
The mesh writes its own user list, and at genesis there is no mesh yet to write it. So
the installer carries the first one — the controller's own account at a well-known
bootstrap password, exactly as the store is reached at `postgres:bootstrap` and the old
bus at `guest:guest`, and rotated with them. From the controller's first composition
onward the file is the controller's.

That left a gap I would not have found by reading: the controller's own account is
created before there is a controller to mint one, so nothing recorded a hash for it, and
its first composition would have left the writer out of the file it was writing — a bus
nothing can connect to, produced by the thing connected to it. It now records a hash of
the credential it is actually using, and only if none is recorded, so a restart cannot
put the bootstrap password back over a rotated one.

The carried list and the derived one are two statements of one fact, so a test compares
them: every subject the controller derives must be in the template, and nothing wider.
It earned itself immediately — the composer was granting both a role's whole event
branch and the one event it actually follows, which is a wider way of saying the same
thing, and the wider one wins. Only the submitting half of a role is granted now; what
comes back is named exactly.

Getting this wrong is the worst kind of silent. A controller whose carried permissions
are narrower than the ones it derives comes up, connects, and is refused on the first
thing it tries, with an authorisation error naming a subject and not the template that
forgot it — and a mesh cannot be raised twice to find out.
2026-09-27 16:39:19 +02:00
jschoubben 75dab209d7 Merge pull request 'Seats keep their former names — a rename is one DB op (ADR 0122, phase 2)' (#83) from feat/seats-rename-is-a-db-op into main 2026-09-27 14:34:45 +00:00
jschoubben 6da9a5478b Seats keep their former names, so a rename breaks nothing (ADR 0122, phase 2)
Phase 1 made the set data; a rename still broke every reference to the old
name. This adds the stable identity: a seat's canonical name changes and its
old name becomes an alias that resolves to it forever. SeatNamed and the holder
and display matching resolve a name (former or current) to its seat, so a
manifest's claim, a held record, the git-seat lookup and the build machine's
embedded set all go on working unchanged after a rename. seat_alias table
(migration 0035), inventory Aliases/RenameSeat, openInventory loads them, and a
'seat rename <from> <to>' command does the whole thing — one operation, no
rebuild, no re-registration, no freeze. Behaviour-neutral until a seat is
renamed. Validated against postgres.
2026-09-27 16:32:22 +02:00
jschoubben ff51b329f6 Merge pull request 'Seats are data the controller owns, loaded from its store (ADR 0122, phase 1)' (#82) from feat/seats-are-data into main 2026-09-27 14:05:40 +00:00
jschoubben 2ec0fd218b Seats are data the controller owns, loaded from its store (ADR 0122, phase 1)
The seat set was a Go slice compiled into the controller and referenced by
name everywhere, so changing it meant a rebuild and a freeze-prone deploy. It
is now a table: catalogue keeps the shipped set as defaultSeats (the seed and
the fallback) and a loadable working set; inventory adds the seat table
(migration 0034), Seats to read it, and SeedSeats to fill it idempotently
without overwriting an operator's edit; migrate seeds it; openInventory loads
it, and an empty or unreadable table leaves the compiled defaults in force so
it can never brick the control plane's boot.

Behaviour-neutral: the seeded table equals the defaults. Phase 2 (reference by
a stable id so a rename touches no manifest or code, and the builder reads the
set from the mesh) follows.
2026-09-27 16:04:44 +02:00
jschoubben e4e960ec1c A build is work submitted to a role, on both buses
ADR 0121 carried through to working code. `Builders` is the asking side and
`BuildMachine` the taking side, each with an implementation per bus, and the builder
binary and the `build` command now go through them.

On the bus being built, one publish does what two did. The old bus answered the asker
through a reply queue and announced to an events exchange, because two audiences meant
two topologies. Here the outcome is the role's own event: the asker matches it by the
id its request carried, the controller records it, the catalogue places it in the graph.
So a build machine publishes once, needs a reply queue for nothing, and needs a grant
over nobody's inbox — which is what ruled out the alternatives.

The outcome carries the module name now. Only the manifest says what was built, and on
the old bus the separate announcement carried it; with one message for three readers it
belongs in the result. A failed build names none, because it produced no module version
and the catalogue would otherwise place something that was never made.

Checked against a real server: the whole round trip; a third party on the role's event
hearing the same outcome the asker did, which is the claim the decision rests on; work
leaving the queue once settled, so no second machine repeats it; work submitted with no
machine holding the role waiting instead of failing, and being done when one arrives;
and work a machine handed back coming round again.

One thing I got wrong twice now and have written down where it bit: binding to a
consumer must name that consumer's own filter subject, not the narrower subject the
caller cares about. The client compares the two and refuses anything that is not equal,
with "subject does not match consumer".
2026-09-27 16:01:53 +02:00
jschoubben cc69737934 The agreement check knows the mesh's own roles, and was passing vacuously without them
It read the roles modules declare and not the mesh's own, so the first consumer of a
role's event was skipped as "the emitter is not installed" — which is exactly the
silence the check exists to break. It passed, and it was checking nothing.

Now it is handed the mesh's own roles too, and there is a case pinning that a consumer
of a role event the role does not emit is caught. A check that cannot fail is worse
than no check, because it reads as evidence.
2026-09-27 15:37:21 +02:00
jschoubben 0c83ecf1b5 The mesh's own roles carry a protocol, and the build branch retires
ADR 0121, first half. The `mesh-*` seats said who does a job and nothing about what
may be said to them or by them, so the mesh had roles it could not describe. They
take the same three fields a module's seat has now, and the machinery that already
derives a work queue, a holder's worker and a permission set from a declared seat
does it for these too.

The build-machine role accepts a build and emits an outcome, so `mesh.build.request`,
`mesh.control.built` and the BUILDS stream are gone. A work queue shared by several
build machines is what a seat's `accepts` already is, and keeping a second mechanism
for it was two places a permission could be wrong.

The controller's own side of a seat is a named list rather than something derived: it
is not a module and declares no `uses`, so which roles the mesh itself submits work to
has to be stated — and stating it makes that question answerable.

Two things this caught:

**The followed event subjects were hard-coded and had just gone stale.** They were
written out while the catalogue still spelled its events as the old bus's routing keys,
so converting those (issue 127) turned the pair into a controller listening to a
subject nothing publishes — the same fault as the issue, from the other side. They
derive from the emitter and the event name now, through the same function the
permission uses, so the two cannot drift apart.

**A role's queue exists before its holder**, checked against a real server, and
asserting twice changes nothing. Work queues until somebody arrives to do it, so
assigning a build machine later flushes the backlog instead of having lost it.
2026-09-27 15:34:44 +02:00
jschoubben 05ff6065d0 Event names are checked now, per manifest and across the catalogue
Issue 127 stood because nothing compared the two halves. Every manifest was
well-formed on its own and every derivation correct on its own, and no
cross-module subscription in the mesh matched anything — a subscription that
matches nothing is not an error, it is silence.

Two checks, because the mistake is possible at two scales.

Per manifest: an event is a local name, and `module.` is refused with the name to
write instead. A module emitting under what reads as another module's name is
refused too, pointing at the seat, where a name outlives whoever holds it.

Across the catalogue: where a consumed event's emitter is present, it must emit
that event. It cannot demand a live emitter for everything — a module lives in its
own repository and may be installed long before the one whose events it wants — so
the rule is narrower and still catches this. It found two real dangling
subscriptions the moment it ran.

Wildcards were undecided and two manifests needed them: `*` is one name and `**`
is the rest, spelled the mesh's way and derived to `>` here and `#` on the old bus.
A manifest naming either would stop being true when the wire changed, which is the
whole reason names are local.

And the field documentation taught the old form, examples included — which is why
the drift was uniform across 37 manifests rather than scattered. Nobody was
guessing; everybody followed the comment.
2026-09-27 14:43:16 +02:00
jschoubben a7df0fc62f Merge pull request 'Name system seats by scope; let a module define its own (ADR 0121)' (#81) from feat/system-seats-named-by-scope into main 2026-09-27 12:32:27 +00:00
jschoubben 1c56210530 Name system seats by scope; let a module define its own (ADR 0121)
System seats are mesh-* (one, mesh-wide) or node-* (one per node). Renamed:
the-build-machine -> mesh-build-machine (+scope mesh), the-catalogue ->
mesh-catalog, the-dns-port -> node-dns-resolver, the-intrusion-prevention ->
node-intrusion-prevention, the-packet-filter -> node-packet-filter,
the-resolver-configuration -> node-resolver-config, the-uplink -> node-uplink.
Removed the-showcase from the set — it becomes the first module-defined seat.

A manifest may declare its own seats (DefinesSeats); a claim is a system seat,
a reserved mesh-*/node-* name the mesh does not define (refused), or a
module-defined seat valid only when the manifest declares it.

Deferred: the delivering registry seats (git, npm-package-registry,
the-artifact-store) and the-private-network (a scope + server/client model
change), per ADR 0121.
2026-09-27 14:30:56 +02:00
jschoubben d65c37caad The controller could not answer an enrolment, and a probe on an open server said it could
Found while reasoning about issue 127's replay question, in code committed earlier
today. The controller's permissions granted no inbox at all, so the answer to
every enrolment on the mesh would have been refused — "Permissions Violation for
Publish to _INBOX.enrol.anchor…" — while the controller logged that it had
enrolled the node.

**`allow_responses` does not cover it, and that is the trap.** It permits one reply
to the reply subject of a message the user received, and a message a JetStream
consumer delivers has had that field claimed for the consumer's own ack address
(design 25 §2). The address the controller actually answers is the one the request
carried in its *payload*, which the server does not recognise as a reply subject at
all. The two mechanisms look interchangeable and are not.

**My earlier verification could not have caught this.** The live enrolment tests run
against a server with no accounts and no permissions, so they exercise the subjects
and the round trip and nothing about authority. Composing the real configuration and
running a server on it is what found it.

Granted the enrolment inbox space and nothing wider: nothing but an enrolling node
ever subscribes under that prefix, each scoped to its own token's, so the controller
publishing there is the mesh answering enrolments and reaches nothing else. Confirmed
against the permissioned server both ways — the answer arrives, and a node's own
inbox is still refused.

Pinned as a rule that needs no server: whatever an enrolling node subscribes, the
controller must be able to publish to, and a node's, a module's and a person's inbox
must stay out of reach. That check is a subject-pattern match rather than a string
compare, so a grant that widened by a wildcard would not slip past it.

It also bears on 127's open question about who replays a build announcement: an
answer to a *module's* inbox would need `_INBOX.>`, which is exactly the blanket
grant design 25 §4 refuses. So the catch-up cannot become an inbox reply.
2026-09-27 14:13:52 +02:00
jschoubben eb72ec36ba 1.7 finished: minting, the file delivered, and a test flake I caused
**First, a correction: the previous commit went in on a false check.** Its message
says the suite passed; it did not. The check piped `go test` through a filter that
swallowed the failures and then printed "green" regardless. Two tests were failing
when 4de10e3 landed.

What was failing was my own doing. Purging the streams instead of deleting them
(4de10e3) left the *consumers* behind, because deleting a stream takes its
consumers with it and purging does not. A durable push consumer surviving between
tests keeps pushing to a delivery subject the previous test's subscription has gone
from: the messages count as delivered, go nowhere, and the next test waits out its
timeout for an announcement the server believes it already sent. Consumers are now
removed with the purge. Five consecutive clean runs.

`-p 1` stays, because two packages asserting and deleting the same fixed-name
objects on one bus is a real race — but its comment said the cause I had guessed
and not the one I found, so it now says the right thing.

**And delivery was not finished when I said it was.** Nothing filled
`Rendering.BusUsers`, so the composed file would never have reached a node.
`composeBusUsers` closes it: composed per push for the machine holding
`mesh-broker`, never kept, because the list is a function of the mesh's records and
a stored copy could disagree with them while both looked consistent. A user with no
credential is left out and named rather than written as a user without a password —
an ordinary situation with an obvious remedy — but a file with no users at all is
refused, because that bus would refuse every connection in the mesh.

**Minting, on both halves.** A node at enrolment and a module at `module issue`.
Three things differ from a management call and each is the point of the move: the
credential is minted into the mesh's records and becomes usable at the next
composition, so no server need be reachable; the password travels beside the address
rather than inside it, because a credential embedded in a URL leaks into every log
line that prints a connection; and a module's durable consumer is derived from what
it declared rather than named, so it cannot ask for delivery of something it did not
say it consumes.

A node reconnecting may be refused until that composition reaches the machine
running the bus. That is what the host's reconnect backoff is for and it is
survivable by design; waiting for the push would hold an enrolment open for as long
as a declaration takes to apply.

Tested that the switch is a switch: a node enrolling on one bus comes away with a
credential for that bus and none for the other, because one that held both could be
half-moved and nothing would say which half.
2026-09-27 03:19:41 +02:00
jschoubben 4de10e32e3 The bus's objects are raised on every start, and one switch says which bus
Two of 1.7's three remaining pieces.

**Raised on every start, not created once at genesis.** A stream somebody deleted,
a mesh raised from a restored backup, or a bus whose data directory was replaced
all have records and no objects — and a node whose consumer is missing hears
nothing while everything else about it looks correct.

The order is not a preference: a consumer on a stream that does not exist is
refused *naming the stream*, so somebody reading that refusal goes looking for a
deletion instead of a reversed pair of lines. Pinned by a test, along with the one
thing about seats that reads like an omission and is not — a seat's work queue is
asserted whether or not anybody holds it, because work queues until a holder
appears, so installing the module a week later flushes the backlog instead of
having lost it.

Against a real server: every object accepted, asserting twice changes nothing (a
start that failed the second time is a controller that cannot restart), a machine
joining an already-raised bus is accepted, each node's consumer is bound to its own
declaration subject and no other's, and CONTROL does not dead-letter — because the
store window's bound belongs to the controller and a server that gave up first
would discard the push the stream exists to protect.

**Which bus this mesh is on is one fact, read in one place.** Every seam the change
went behind ships both implementations; this is what the rollout flips. Being told
about both is refused at start rather than warned about: a mesh half on each is one
where a declaration goes out on one bus and the report comes back on the other, and
every component logs success while it happens — ADR 0074's failure arriving through
configuration instead of through code. The refusal names both variables and says
which to unset, because whoever reads it has to choose and the wrong choice is a
rollout half done.
2026-09-27 02:59:05 +02:00
jschoubben f8ab9f2dcf The mesh composes the accounts; the module owns its server
The delivery question, decided. The alternative was a manifest field enumerating
the server's ports, TLS paths and store directory so the controller could write a
whole configuration file. That is wrong: those are properties of the container the
module raises, they live in its image and its mounts, and the controller would
have to be kept in step with a Dockerfile it never sees. So the mesh writes only
what only the mesh knows — who may connect — and the module's own configuration
includes it.

`ComposeAccounts` is that file. A test says what must *not* be in it as plainly as
what must: no port, no tls block, no store_dir. Each of those in the mesh's file
is a value the controller would then own, and the module could no longer change
its own image without the mesh agreeing.

`bus-users` is where a module wants it written, and **asking is not enough to
receive it**: the file holds every user's password hash, so a module that could ask
for it could read every credential on the bus. The claim on `mesh-broker`
authorises it, checked from the manifest alone. A holder with nothing composed is
refused rather than given an empty file, for the reason a certificate is — a bus
with no user list refuses every connection in the mesh and looks like a machine
problem.

Six claims checked against a running server before any of this was committed to,
and two of them changed what got written:

**An absolute include path is resolved relative to the including file's
directory.** `include /etc/nats/accounts.conf` from /etc/nats-server/nats.conf
makes the server look for /etc/nats-server/etc/nats/accounts.conf and refuse to
start. So both files share one directory, and the module declares its own as a
file resource beside the mesh's.

**`verify: true` was refusing every connection in the mesh.** It makes the server
demand a *client* certificate, and nothing in the mesh presents one: a host pins
this server's exact certificate and authenticates with the password the mesh
minted, and so does a module's runtime. Every connection died at the TLS handshake
before any password was looked at, with an error — "client didn't provide a
certificate" — that reads as a fault in the client. Removed. TLS is still
required; verify only decides whether client certificates are checked.

The other four: a user in an included file authenticates, an unknown user is
refused so the include is the whole authority rather than an addition, a publish
outside a grant is refused, and rewriting the mesh's half alone makes a new user
appear — noticed by the module's own watcher, with no signal from outside, and
without dropping the connection the mesh already had. That last one is task 1.2's
payoff, collected.
2026-09-27 02:50:23 +02:00
jschoubben ccf50b1269 Merge pull request 'Facts carry the format as a template, so the control plane holds none (ADR 0120)' (#80) from feat/roster-facts-are-templates into main 2026-09-26 23:51:23 +00:00
jschoubben a6d89e3f73 Reconcile with hq 128: hosts template is the region form, node-names is shared
The merge commit took only the staged index; these reconciliation edits sat
unstaged in the working tree. Integrate the template mechanism with #79's
region write (hq 128): RosterFile gains Shared, FactsInto sets into:block for
a shared fact, /etc/hosts becomes the region form (no floor) and node-names is
marked shared. Without this the merge would have regressed /etc/hosts back to
a whole-file write, replacing the operator's own lines.
2026-09-27 01:50:11 +02:00
jschoubben ee1b8ffe24 1.7, second half: the user list read out of the mesh's records
The derivation had nothing feeding it. `BusRecords` reads what it needs — the
machines, what each runs, every manifest, and which machines hold a live token —
and turns it into the records the composer derives from.

**A module's authority comes from its manifest, not from its assignment.** The
assignment says where it runs; what it may say is what it declared. So the two are
read together and the manifest decides, which is also why a seat's protocol is
gathered across the whole catalogue rather than from one manifest: a seat is
declared by one module and held by another, and that is the whole reason a seat
exists.

Three things checked against a real store, each a user that would be wrong in a
way nothing reports:

- A module assigned to a machine becomes a user with exactly the authority it
  declared, including the protocol of a seat some *other* module declared — a
  module granted nothing on a seat it was assigned to send to would fail on its
  first publish with an authorisation error that says nothing about a seat.
- Only a machine holding a live token gets an enrolment user. One outliving its
  token is a right to join that nobody issued.
- A module assigned and absent from the catalogue is refused rather than composed
  with an empty permission list. The catalogue already refuses to forget an
  assigned module, so this is the second line — and it earns its place there,
  because relying on another package's invariant is how a rule ends up enforced by
  nothing.

People are left empty rather than guessed at: the account model is built and
`operator issue` is not, so there is nobody to derive yet.
2026-09-27 01:49:09 +02:00
jschoubben 0560c792d8 1.7, first half: the mesh can say who its bus users are, and hold their keys
Two pieces the composer has been waiting for since it was written.

**The credential has to outlive its own minting.** On the bus the mesh runs on
today an account is a management call: mint a password, hand it over, seal the
plaintext to whoever will use it, keep nothing — which works because the broker
remembers. Here the users are one file, rewritten whenever any of it changes, so
keeping nothing would mean the first person's access change silently blanking
every module's password. So a bus user's bcrypt hash is now recorded, keyed by the
username the file needs, and the plaintext comes back exactly once. Verified
against a real store that the hash verifies the password it was made from, that
the password itself is not in there, that minting again rotates rather than adds,
and that forgetting a node takes its host's and its modules' credentials with it.

**Permissions are not stored, and that is the point.** Only the credential is
kept. Authority is derived from what each module declares, every time the file is
written (ADR 0043) — a stored permission list would be a second account of a
user's authority, able to disagree with the records it came from, and both would
look internally consistent while they did.

`Users` derives the list: the controller always first and always present, one
user per node, one per module per node, one per live token, one per person. Two
users with one name is refused where both can be named, rather than left to be
whichever one the server happened to read. A user the mesh has never minted a
password for is *named* rather than dropped or written as a user anybody is:
that is an ordinary situation with an obvious remedy, and the caller decides
whether a partial file is worth writing.

What remains of 1.7: delivering the file to the node that runs the server, and
minting at enrolment and assignment — which is transport-coupled, because a node
on the old bus must not be handed a credential for the new one.
2026-09-27 01:44:53 +02:00
jschoubben 966c5ddad3 Merge remote-tracking branch 'origin/main' into feat/roster-facts-are-templates
# Conflicts:
#	internal/catalogue/facts.go
#	internal/catalogue/facts_test.go
2026-09-27 01:41:23 +02:00
jschoubben b36f822cb6 Facts carry the format as a template, so the control plane holds none (ADR 0120)
A roster fact used to be a name from a closed list, each formatted in Go
here — node-names as a hosts file, node-zones as a resolver's zones. Every
new consumer (ssh's known_hosts, an authorized_keys) meant another formatter
in the control plane, in the consumer's own configuration language.

Now a fact is a path and a Go template over the roster view (this node, the
suffix, and every served name vs the machines). The mesh owns the data; the
module owns the format. /etc/hosts is a template on the network module;
dnsmasq's zones move to dnsmasq. The controller renders and reads neither.

WireGuard stays a computed generator: the overlay is the substrate delivery
rides on, and its config is topology, not a roster projection.

Output is byte-for-byte unchanged, pinned by the hosts golden tests and the
resolver tests that compose the real dnsmasq manifest.
2026-09-27 01:33:20 +02:00
jschoubben 7180a273a2 The enrolment user is per token, and it has an inbox
Design 25 §6 says an enrolling node subscribes the inbox its own token derives.
It had none: `sub` was empty, so a node would publish its request and wait out
its timeout against a mesh that had answered — the handshake could not have
completed.

And there was one shared `enrolment` user, which cannot carry that inbox at all:
a permission belongs to a user, so an inbox per token means a user per token.
Named after the node, which **is** the token's id — a token is issued for a node
record, the mesh holds one live claim per record, and the node's name is the one
identifier both sides have before anything else is agreed. It is also exactly
what the other transport does, where the account is named after the node and the
secret is its password.

A nameless enrolment user is now refused rather than composed into
`_INBOX.enrol..>`: an empty subject token, and worse, one every nameless
enrolment user would share — which is one machine able to read the credentials
sealed to another.

Still to wire: something that composes one of these per live token. Nothing
composes enrolment users yet, on either bus — on the old one the account is made
imperatively through the broker's management API when a token is issued, and
here there is no management API, so issuing a token has to recompose the server's
configuration. That is the remaining half of enrolment on the new bus.
2026-09-27 01:31:34 +02:00
jschoubben e65b3950cc A node's declaration consumer, which only the mesh can make
A host's account reaches no part of the JetStream API — correctly, because the
controller is the only writer of consumer definitions — so the object a node
reads its declarations through has to be waiting before the host binds to it,
and nothing created one. Named after the node, because the node's own ack grant
is `$JS.ACK.NODES.<node>.>` and a consumer named anything else is one the host
cannot acknowledge a delivery from.

No max-deliver, and a five-minute ack wait: a declaration is settled only after
the node has applied it and reported, which is minutes on a machine pulling
images, and the stream holds exactly one message per node — so there is nothing
to dead-letter, only one message to redeliver for as long as that node is away.

Asserted on start as well as created at enrolment, for the reason the streams
are: a mesh raised from a restored backup has node records and no consumers, and
a node whose consumer is missing hears nothing while everything else about it
looks correct.

The test sets the consumer against the grant the node actually gets, because
each of the three ways of getting it wrong is silent: a wrong name cannot ack, a
wrong filter reads another node's declarations, and a pull consumer is one a host
has no authority to bind.
2026-09-27 01:24:46 +02:00
jschoubben a0e09695e5 Merge pull request 'The uplink seat (ADR 0117), and the mesh's names written into the hosts file, not over it (hq 128)' (#79) from feat/the-uplink-seat-and-the-hosts-region into main 2026-09-26 23:00:19 +00:00
jschoubben 88bef39952 The consume side on NATS, and the window held by the server
The other implementation behind the seam, so the store-window guarantee now has
both: one loop, one message at a time, the same window deciding. What differs is
where a held message lives, and that is the whole point of the move — the AMQP
side keeps an unacknowledged delivery in this process, bounded by the prefetch
and lost if the controller stops; this keeps eight bytes saying when the window
opened, and the message stays the server's.

Checked against a running server, seven claims that reasoning cannot answer: a
report is heard and leaves the work queue; one the store cannot take is naked
with a delay, stays in the stream, and is recorded when the store returns; one
about a superseded declaration is settled without being acted on; one the store
never takes is let go once the bound passes; a heartbeat is heard and nothing is
persisted; and the enrolment answer reaches the address the request carried in
its payload — the test design 25 §2 asks for, so the reason for that field
cannot quietly become folklore.

Three things the wiring forced into the open:

**The controller could not have consumed a module event.** Its permissions
granted no event subject to subscribe and no ack subject on the events stream,
so every announcement would have been redelivered for ever, refused by the list
it already had. Both narrow: each followed subject named, not `mesh.mod.*.>`.

**The controller's consumers are not derived.** It files no manifest, so its
authority cannot come from a declaration that does not exist; they sit beside the
mesh's own streams and are asserted the same way. No max-deliver on CONTROL —
the window's bound is the controller's, and a server that dead-lettered first
would discard the push the stream exists to protect.

**Channels, not callbacks.** The library would run a handler on its own
goroutine, and the window's bookkeeping is unlocked because the AMQP loop never
had two.
2026-09-27 00:53:33 +02:00
jschoubben 06cf3c04e5 The consume side behind a seam, and the window wiring into the loop
The outbound half went behind `Bus` and the transport stopped reaching its
callers; this is the other half, and the larger one. Every handler took
`amqp.Delivery`, so the serving loop could not move to another bus without
moving enrolment, reports, builds, upgrades and catch-up with it in one breath.

`Control` states one message in the mesh's words — took it, dropped it, or held
it for the store — and `Inbound` is where messages come from. The AMQP
implementation is today's loop moved rather than changed: same queues, same
prefetch, same holding, because the mesh is running on it and a bus nothing
speaks yet is no reason to alter the one every node is on.

The window (window.go) is now what decides, instead of the conditions that were
inlined in the loop. Two things that surfaced in the wiring:

**Supersession is asked before the store, not after.** A report about a
declaration the mesh has moved past would otherwise wait out a restarting store
to be written and then overwrite what the node is doing now.

**Half of a report is not about a declaration, and that half is never stale.**
What the machine *is* — the tunnel it took over, the ports its own bundle
holds, what an adopted node found, a node moving its overlay key — reaches the
mesh on a report and nowhere else. A rekey set aside as stale is a node whose
overlay key never moves, and no retry is coming, because the node said it once.
So staleness is asked only of a report that is purely an apply's account.

The one thing holding-in-memory can do that holding-in-the-server cannot is
named rather than hidden: `About` sets aside a held message when a newer one
about the same thing arrives, and the bus being built ignores it because the
digest answers the same question.
2026-09-27 00:44:16 +02:00
jschoubben 7a8a19b11b A person's account (step 4.4, the account half)
Design 25 §7. A person is not a module and holds no seat: nothing is
addressed to them, nothing is delivered to them, and they have no durable
consumer. What they have is permission to ask, as a list of tools or `*`
for an administrator.

Four properties the tests hold it to, each of which is a way of being
wrong that would not announce itself: a person reaches nothing but tools,
so one cannot claim a module said something; no ack subject, because
authority over a consumer that does not exist is authority nobody would
audit; no allow_responses, because a person who can answer a request is
impersonating a module on a bus where anyone may serve a tool; and two
people do not share an inbox.
2026-09-27 00:17:52 +02:00
jschoubben ce6ac057f6 gofmt the probe 2026-09-27 00:14:47 +02:00
jschoubben abce68fdf0 Verify that a reply address does not survive a stream
Design 25 §2 builds enrolment around carrying the reply subject in the
payload, because a JetStream consumer claims the transport Reply field for
its own ack address. The whole handshake rests on it, so it is checked:
the caller asked for _INBOX.LCr3M83q... and the consumer saw
$JS.ACK.PROBE.probe_consumer... The design was right, and the workaround
is necessary rather than defensive.

Worth having as a test rather than a note: if a future server version
stopped doing this, enrolment would keep working and the reason for the
payload field would quietly become folklore.
2026-09-27 00:14:32 +02:00
jschoubben cc019908c2 Asking a tool goes through the seam, and loses two problems
Step 3.4. On the new bus there is no reply queue to declare and no
correlation to check: each account is granted one inbox prefix and no
other, so an answer cannot reach the wrong asker. That settles a cost
build.go records having paid — on a shared reply exchange every asker saw
every result, which is why the correlation was checked rather than assumed.

And a tool nobody serves says so at once rather than after the whole wait.
The difference between "that module is down" and "that tool is slow" is the
first thing a person asking wants, and both tests are against a real server
because both are claims about what the server does, not about this code.

RequestBuild stays as it is, and is a different shape on the new bus rather
than the same one: a build takes minutes, so it is work submitted to a
queue with the outcome returning to a reply subject the request carries —
the pattern design 25 §2 already sets for anything crossing a stream. It
touches the builder too, so it goes with that conversion.
2026-09-27 00:11:30 +02:00
jschoubben d0a9abcb1c The store window as a decision, and the problem moving it to the server
introduces

Step 3.4, the consume side's hard part. The guarantee (ADR 0083) is that a
push the controller cannot record because its store is restarting is held
and retried — never dropped, never falsely acknowledged. Keeping the
delivery unacknowledged in memory becomes a nak with a delay: the server
holds it, the controller keeps no list of parked messages, and a controller
that restarts mid-window loses nothing it was holding.

That is a plain win and it introduces one problem. Holding in memory let
the controller drop an older report when a newer one for the same node
arrived, "because acting on it after the newer would undo the newer". A
naked message is the server's and comes back whatever happened meanwhile,
so the older report is redelivered after the newer was applied.

The answer was already in the message. A report carries `Declared`, the
digest of the declaration it is about, which exists because an earlier
attempt to order reports by time lost the race it invited. So supersession
stops being something the controller remembers and becomes something it
checks — the same shape as a node refusing a superseded declaration by
sequence (issue 107): ordering settled by what a message says, not by when
it arrived.

Pure, so the guarantee is testable without a bus, a store or a clock. Nine
tests, including that staleness is decided before the store is waited on —
a redelivery that lost its race must not hold a slot a current message
needs.
2026-09-27 00:05:52 +02:00
jochen 63ba2d178f review: hold the hosts region through composition, and say so in plan
A declaration-level test composes the shipped networking module with a
resolver and asserts /etc/hosts arrives as mesh-wireguard.fact-node-names
with into: block and region-only content, that the resolver's restart-on
still names it, and that a resource's at passes through untouched — a
composition step dropping into would otherwise go unnoticed. plan --show
marks files written into, so a region is not read as the whole file.
The rollout order is spelled out: every node's host, the controller's
own included, must be block-aware before this controller ships (hq 128).
2026-09-27 00:05:41 +02:00
jschoubben 6c12780abe Describe the bus on its own terms
Comments framed the new bus by what it replaces — a comparison in almost
every explanation, which reads as though NATS were a variant of the old
thing rather than the mesh's nervous system. Removed throughout, and
OverAMQP becomes OverCurrent: the seam's two sides are the bus the mesh
runs on today and the one being built, not two protocols.

What remains is the client library's own package name, which is its name.
2026-09-26 23:51:00 +02:00
jschoubben 92d87b0082 gofmt bus.go — import grouping 2026-09-26 23:47:26 +02:00
jschoubben 2fad32767e The controller's outbound link behind a seam, with both transports
Step 3.4, first half. Every one of these took an *amqp.Channel, so the
transport reached every caller and swapping it meant touching all of them.
The seam turned out to be small — the controller sends exactly two kinds of
message that expect no answer — which is the same measurement that said
this bus could be replaced at all.

Bus is stated in the mesh's words, not a transport's: PublishEvent and
PublishDeclaration. Two implementations, both shipping, because steps 1 to
4 leave every node on AMQP and the NATS one is selected at the rollout.
Both ship is also what makes them comparable: one conformance fixture holds
both to the same envelope, and the NATS one is checked against a real
server reading back from the stream rather than from the code that wrote it.

Still on *amqp.Channel: RequestBuild and Ask, which carry reply-queue
machinery, and the whole consume side — the control loop, enrolment, serve.
2026-09-26 23:47:15 +02:00
jochen 51163b9f14 The mesh's names are written into the hosts file, not over it (hq 128)
/etc/hosts is the machine's: the distribution's localhost lines, the
operator's own entries, and marked blocks other tools maintain there.
Writing node-names whole replaced all of it the moment the private
network was taken, and every later write by those tools was lost at
the next machine joining. The node-names fact is now emitted with
into: "block", so the host owns only its marked region and keeps the
rest byte for byte. The region holds only the mesh's names: no header
claiming the file, no localhost, no 127.0.1.1 line — the floor was
never the mesh's to write. How a fact is written is a property of the
fact in the closed table; node-zones stays a whole file the mesh owns.

Sequencing: a host older than the block mode refuses the whole
declaration on an unknown into, so every host must be upgraded before
this controller is rolled out.
2026-09-26 23:46:51 +02:00
jochen 6557aca750 A machine's uplink is a seat (hq ADR 0117)
the-uplink joins the closed set as a node seat delivering nothing. Its
holder is the module for the machine's own network manager, and keeps
that manager from contradicting the mesh — the resolver file left to
resolv-conf, mesh0 left alone — without ever declaring a link. Held
per machine, so a machine running two managers is refused at
assignment rather than found by its resolver being rewritten. The
count test moves to fifteen; one test holds the seat's shape.
2026-09-26 23:45:47 +02:00
jschoubben 1801f1178e Hold the Go emitter to the shared fixtures
Every required header set, each value in the pinned shape, and the subject
derived the same way. Read from the sdk's conformance directory by sibling
path, never copied.
2026-09-26 23:40:59 +02:00
jschoubben f21a84c510 Merge pull request 'The controller marks a deliberately-empty declaration owns_nothing (hq 127)' (#78) from feat/controller-marks-owns-nothing into main 2026-09-26 21:40:18 +00:00
jschoubben e14b02991e The controller marks a deliberately-empty declaration owns_nothing (hq 127)
The host refuses an empty body unless told the emptiness is meant
(mesh-host#29). When a node's declaration composes to no resources —
which #77 now sends rather than skips — Body() sets owns_nothing, so
the node applies it and drops what it last held. A declaration with
resources never carries the marker. One test.
2026-09-26 23:40:04 +02:00
jschoubben bf82805cfd A manifest holds no subject, checked across all 72
Design 29 §1's load-bearing rule: a module names its events, tools and
seats locally and the mesh derives the subject, so reorganising the subject
space leaves every manifest correct. It held by construction, and a rule
held by construction is one a later field breaks quietly.
2026-09-26 23:34:42 +02:00
jschoubben 08ebb8c999 Merge pull request 'An empty declaration is sent, so a node drops what it last held (hq 127)' (#77) from fix/127-an-empty-declaration-is-sent into main 2026-09-26 21:32:43 +00:00
jschoubben 0a8a592ef4 An empty declaration is sent, so a node drops what it last held (hq 127)
push skipped any node whose declaration composed to zero resources. A
node that HELD something before — the broker opening a placement gave
an adopted node, say — then kept it forever: the empty declaration that
would drop it was never sent, and the node's own heartbeat re-applied
the stale resource with no way for the mesh to say it is gone. Now the
empty declaration is sent; the host drops what the mesh owned and keeps
what it found. A node that never held anything applies it as a no-op.
Surfaced on ace: the foundation-opening fix (#74) removed its only
resource, and the correction could not reach it until this.
2026-09-26 23:32:30 +02:00
jschoubben 173c8c7c21 Rename the mesh's seats to mesh-*, keeping their interfaces
novox/hq ADR 0118: the prefix is the reservation rule, so a module
declaring any mesh-* name is refused and there is no reserved-names list to
drift. Ten seats renamed in the table, the manifests that claim them, the
controller's own shipped manifests, and the tests.

Not the migration 0118 expected: a holding is derived at resolution from
manifests and never stored, so nothing recorded points at an old name. A
kept rename table tells a manifest written against one what it became —
kept rather than retired, because a module lives in its own repository and
may be registered long after the catalogue stopped using it.

**A seat is not the interface it delivers.** The git seat became mesh-git
and the git provision did not; likewise the package registry. A blanket
replace renamed both, and the failure read "the package registry is served
on <nil>", which does not say "you renamed an interface". A test now pins
every seat against what it delivers, and that neither name is also the
other.
2026-09-26 23:07:09 +02:00
jschoubben 98d348d5b9 Merge pull request 'The mesh's interface takes over the found tunnel's MTU' (#76) from feat/controller-carries-tunnel-mtu into main 2026-09-26 20:41:00 +00:00
jschoubben 7fc5fd02fd The mesh's interface takes over the found tunnel's MTU
Carries MTU from the reported tunnel (mesh-host#28) through inventory,
the overlay graph's TakeOver, into the generated config's [Interface].
A tuned path keeps its MTU across the takeover instead of regressing to
1420 and hanging transfers no ping would reveal. Two emit tests; a
tunnel with no MTU writes no line.
2026-09-26 22:40:42 +02:00
jschoubben 50878a9d98 Merge pull request 'A taken tunnel brings its ListenPort, even on a node the hub cannot dial' (#75) from fix/a-taken-tunnel-brings-its-port into main 2026-09-26 20:34:10 +00:00
jschoubben aa74bd86ca Derive a seat's stream and a module's consumer, and wire JetStream
Task 3.9's other half and 1.4's missing client. The derivation is pure and
unit-tested; only "does the server accept this" needs one running, behind
MESH_TEST_NATS so the ordinary suite stays offline.

A seat's work queue is created at registration, not assignment, so work
queues until a holder appears — a stream created at assignment would make
"the holder is not here yet" mean "your messages are gone". Named after the
seat, because the holder can change and the queued work must not care.

A holder's worker uses a queue group even though the seat guarantees one
holder: the seat is authority, the queue group is delivery, and tying them
together means the day somebody allows two holders every message is
processed twice with nothing reporting it.

One consumer per module carrying every filter, because its ack permission is
derived from its name.

And a real bug the live server caught: a durable name may not contain a dot,
but an ack subject is $JS.ACK.<stream>.<consumer>, so the single string that
read correctly inside the permission was rejected as a consumer name. Split
in two, beside the permission that has to match. Unfixed, the symptom would
have been every message redelivered forever with a permission list that
looks right — which is the failure design 25 §4 warns about.
2026-09-26 22:28:42 +02:00
jschoubben 7232d6df4b A module may declare its own seats (novox/hq ADR 0118)
The manifest carries seats and uses; registration refuses a mesh-* name, a
duplicate declarer, an undeclared uses or claim, a seat with no protocol, a
scope mismatch, and a holder that does not answer what its seat promises.

The parser stops judging unknown claim names, because it cannot: another
module may declare that seat, and one manifest cannot tell. The test that
encoded the old rule is rewritten to assert the refusal at registration, and
a new one pins the case the parser could not have distinguished.

Tools are declared for the first time, under their own key — serves already
means a provision's facts.
2026-09-26 22:17:17 +02:00
jschoubben 112cbd294d Per-subject caps on EVENTS, and a comment corrected against the server
NATS refuses overlapping streams rather than double-storing, which is the
opposite of what the Overlaps comment claimed. The check still earns its
place — it names both streams at composition rather than one at apply — and
the refusal is what rules out a shared stream beside per-module ones.
2026-09-26 21:49:55 +02:00
jschoubben 553814b6eb The mesh bus seat is one per mesh, and the amqp broker does not contend
Step 2.3 of novox/hq ADR 0116. The refusal is the resolver's existing one;
these pin it for this seat, including that a different bus implementation is
refused for the same reason — the property that lets the bus be replaced.
2026-09-26 21:40:03 +02:00
jschoubben eeb0560fc2 The mesh-broker seat delivers mesh-bus (novox/hq ADR 0120) 2026-09-26 21:17:21 +02:00
jschoubben 1f36787d75 The mesh's four streams, asserted on every start
Task 1.4. The foundation set only — a seat's streams come at registration
and a module's consumers at assignment, neither of which has happened at
genesis (ADR 0118).

Asserted rather than created: a stream that was deleted, or a mesh raised
from a backup, must converge rather than run without the guarantee its
messages assume.

Two things the definitions have to get right, both tested:
- CONTROL names its subjects instead of taking mesh.control.>, because
  heartbeats live under that prefix and a stream of them competes for
  retention with the messages that matter
- EVENTS filters on the event token, which is why that token exists; a
  filter over a module's whole namespace would persist every tool call

Overlapping filters are refused where the set is written: NATS accepts two
streams matching one subject and stores the message twice under two
retentions, which nothing reports.

Adds nats.go as a dependency; it pulled golang.org/x/* forward. Full suite
green.
2026-09-26 21:02:18 +02:00
jschoubben c753f9d5c0 Compose the bus's accounts instead of calling a management API
Task 1.3 of novox/hq ADR 0116. On AMQP an account was an HTTP call; on NATS
it is text the controller composes and the server reloads (ADR 0106). Pure,
so the mesh's whole authority model is testable as strings.

NATS closes a gap management.go recorded rather than hid: LavinMQ has no
topic permissions, so an emitter was granted the events exchange whole and
ADR 0042's origin reservation was "stamped by the sdk, not enforced here".
Per-subject permissions make it the server's refusal.

Two things found by composing a real file rather than reading the design:

- a scoped inbox leaves a responder unable to reply, because the answer goes
  to the caller's inbox. allow_responses is the answer — one reply to the
  subject of a message actually received — and only principals that serve
  are granted it. Recorded in design 25 §4.
- composition must be deterministic: the module's entrypoint reloads on the
  file's digest, so an order-dependent composer would reload the whole bus
  on every controller restart. Covered by a test.

The golden fixture is the exact text `nats-server -t` accepts, so the syntax
is the server's rather than one we invented.
2026-09-26 20:58:49 +02:00
jschoubben fb87f9f7d6 The mesh-broker seat delivers nothing
The bus is the only broker (novox/hq ADR 0117): messaging is subjects on it,
scoped by a module's own emits/consumes, not a server handed out as a
provision. So the seat joins mesh-controller and the-catalogue in delivering
no interface. Full suite green.
2026-09-26 19:34:14 +02:00
131 changed files with 12568 additions and 2280 deletions
+9 -3
View File
@@ -86,13 +86,19 @@ proxy-image:
# The whole gate. Raises a database, runs everything against it, and takes it down again --
# including when the tests fail, which is why the teardown is not conditional.
#
# **One package at a time (-p 1), and it is not about speed.** The live tests reach one bus, and on
# it they assert, read and remove the mesh's own objects -- streams and consumers with fixed names,
# because those names are the mesh's and a test cannot choose others. Two packages doing that at once
# is one deleting a consumer the other is reading through, and the failure lands in whichever test
# was reading, as "no response from stream". That reads as a bug in the code under test.
check: fmt vet postgres
@go test ./... ; status=$$? ; $(MAKE) postgres-stop ; exit $$status
@go test -p 1 ./... ; status=$$? ; $(MAKE) postgres-stop ; exit $$status
# Without a database the live tests skip rather than fail, so this is the honest subset and not
# the gate.
# the gate. Serialised for the same reason check is: a bus may be configured even when a store is not.
test:
go test ./...
go test -p 1 ./...
vet:
go vet ./...
+55 -164
View File
@@ -17,22 +17,15 @@ package main
import (
"context"
"crypto/sha256"
"crypto/tls"
"crypto/x509"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"net/url"
"os"
"os/signal"
"strings"
"syscall"
"time"
amqp "github.com/rabbitmq/amqp091-go"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/builder"
"github.com/novox/mesh-controller/internal/link"
)
@@ -52,7 +45,6 @@ const usage = `mesh-builder — builds modules for the mesh
It consumes build requests and answers with what it made. Nothing is listened on and nothing
is dialled except the broker.
MESH_BROKER_AMQP where the broker is, with this builder's own credential
MESH_BROKER_FILE a file the mesh sealed to this machine holding the same
MESH_REGISTRY host:port to publish artifacts to, when the mesh has not said
MESH_BINDING a file the mesh wrote saying where the artifact store is
@@ -115,72 +107,48 @@ func run() error {
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
defer stop()
conn, err := dial(credential)
if err != nil {
// Not quoted back: the URL carries this builder's broker password.
return fmt.Errorf("cannot reach the broker: %w", err)
}
defer conn.Close()
channel, err := conn.Channel()
if err != nil {
return err
}
defer channel.Close()
if _, err := channel.QueueDeclare(link.BuildQueue, true, false, false, false, nil); err != nil {
return err
}
// One at a time. A build machine that took five requests at once would run five container
// builds against one runtime and finish all of them slower than it would have finished the
// first — and the queue is what shares work between machines, so nothing is lost by it.
if err := channel.Qos(1, 0, false); err != nil {
return err
}
// Not auto-acknowledged. A request acknowledged on arrival is a build that vanishes if this
// process dies mid-way, with nobody waiting on it ever hearing why.
requests, err := channel.ConsumeWithContext(ctx, link.BuildQueue, "mesh-builder",
false, false, false, false, nil)
machine, err := takeWorkFrom(credential, on)
if err != nil {
return err
}
defer machine.Close()
fmt.Fprintf(os.Stderr, "building for the mesh, publishing to %s\n", registry)
publisher := builder.Registry{Address: registry, Run: builder.Command}
for {
select {
case <-ctx.Done():
fmt.Println("stopping")
return nil
case delivery, ok := <-requests:
if !ok {
return fmt.Errorf("the broker closed the connection")
return machine.Take(ctx, func(ctx context.Context, work link.Build) {
answer(ctx, publisher, on, workspace, work)
})
}
answer(ctx, channel, publisher, on, workspace, delivery)
// takeWorkFrom opens this machine's link to whichever bus the mesh is on.
//
// **One place chooses**, as everywhere else the bus change went (novox/hq ADR 0116 step 5): a build
// machine told about both would take work from one and answer on the other, and every log line would
// say it was fine.
func takeWorkFrom(credential Credential, on string) (link.BuildMachine, error) {
// **The credential names the bus, and there is one** (novox/hq ADR 0131, design 28 task 5.5).
// A credential for the mesh's bus carries user, password and fingerprint beside the address,
// and that is enough to dial it, pinned.
if !credential.onTheNewBus() {
return nil, fmt.Errorf("the credential at hand names %q, which is not the mesh's bus", credential.URL)
}
js, err := broker.DialPinned(credential.natsURL(), credential.Fingerprint)
if err != nil {
return nil, err
}
return link.MachineOverNATS(js, on), nil
}
// answer does one build and says what happened, whichever way it went.
func answer(ctx context.Context, channel *amqp.Channel, publisher builder.Publisher,
on, workspace string, delivery amqp.Delivery) {
func answer(ctx context.Context, publisher builder.Publisher, on, workspace string, work link.Build) {
request := work.Request()
// **First thing, and to stdout.** A build request that arrives and produces no visible line
// until it either finishes or fails is indistinguishable from one that never arrived — which
// cost a long diagnosis against a running mesh, chasing "the handler never fired" when the
// truth was only that the handler said nothing until the end.
fmt.Fprintf(os.Stderr, "a build request arrived (%d bytes)\n", len(delivery.Body))
var request link.BuildRequest
if err := json.Unmarshal(delivery.Body, &request); err != nil {
// Unreadable. Acknowledged and dropped rather than requeued: a message this builder
// cannot parse will not become parseable by being delivered again, and requeueing it
// would put it in front of every real request for ever.
fmt.Fprintf(os.Stderr, "a request could not be read and was dropped: %v\n", err)
_ = delivery.Ack(false)
return
}
// **First thing, and to stdout.** A build request that arrives and produces no visible line until
// it either finishes or fails is indistinguishable from one that never arrived — which cost a long
// diagnosis against a running mesh, chasing "the handler never fired" when the truth was only that
// the handler said nothing until the end.
fmt.Fprintf(os.Stderr, "a build request arrived for %s\n", request.Repository)
result := link.BuildResult{
ID: request.ID, Repository: request.Repository, Path: request.Path,
@@ -199,8 +167,8 @@ func answer(ctx context.Context, channel *amqp.Channel, publisher builder.Publis
var built builder.Result
if err == nil {
// The package-registry credential is a build input, so it is resolved before the clone: a
// build that could not have resolved its dependencies is refused in front of the reason,
// not after a clone that then fails at npm ci.
// build that could not have resolved its dependencies is refused in front of the reason, not
// after a clone that then fails at npm ci.
built, err = builder.Build(ctx, builder.Command, publisher,
request.Repository, request.Path, request.Ref, workspace, request.Held, npmrc,
forgeFrom(),
@@ -230,67 +198,19 @@ func answer(ctx context.Context, channel *amqp.Channel, publisher builder.Publis
}
}
body, err := json.Marshal(result)
if err != nil {
fmt.Fprintf(os.Stderr, "cannot report a build: %v\n", err)
_ = delivery.Ack(false)
if err := work.Announce(ctx, result); err != nil {
// Said, not fatal: the build happened. A build reported as failed because announcing it
// failed is a lie about work that was done — and the request stays unsettled below only if
// nothing was said at all, so another machine can try.
fmt.Fprintf(os.Stderr, "cannot say what came of a build: %v\n", err)
return
}
// Always through the exchange, whether or not somebody is waiting.
//
// **Never the default exchange.** Permission there is granted per exchange rather than per
// queue, so a builder allowed to use it could publish into any node's queue — the privilege a
// build machine most obviously should not have. An asker binds its own reply queue to this
// key and filters by correlation; a control plane that records builds is bound to it too, so
// a result nobody asked for is still kept rather than reported into the void.
publishCtx, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel()
if err := channel.PublishWithContext(publishCtx, link.Exchange, link.KeyBuilt, false, false,
amqp.Publishing{
ContentType: "application/json",
CorrelationId: result.ID,
Body: body,
}); err != nil {
fmt.Fprintf(os.Stderr, "cannot answer a build request: %v\n", err)
// Settled only once the outcome is away, so a machine that dies before answering leaves the work
// for another rather than losing it.
if err := work.Done(); err != nil {
fmt.Fprintf(os.Stderr, "the outcome is away and the request could not be settled: %v\n", err)
}
// **And announced, which is a different act from answering.** The reply goes to whoever asked
// and is correlated to their request; this says to the whole mesh that a module now exists at
// a commit, and the catalogue places it in the module graph (novox/hq ADR 0072). A build
// nobody asked for still has to be announced, or the graph knows less than the registry does.
//
// Only on success: a failed build produced no module-version, and announcing one would put
// something in the graph that was never made.
if result.Failed == "" && result.Commit != "" {
announced := map[string]any{
"module": moduleOf(result.Manifest), "commit": result.Commit,
"repository": result.Repository, "path": result.Path, "ref": result.Ref,
"manifest": json.RawMessage(result.Manifest), "against": result.Against,
"made": result.Made,
}
if err := link.EmitEvent(publishCtx, channel, link.KeyModuleBuilt, "builder", on, announced); err != nil {
// Said, not fatal: the build happened and was answered. A module the catalogue has not
// heard of is a gap somebody can close; a build reported as failed because announcing
// it failed is a lie about work that was done.
fmt.Fprintf(os.Stderr, " built, but could not announce it: %v\n", err)
}
}
// Acknowledged only once the answer is away, so a builder that dies before answering leaves
// the request for another machine rather than losing it.
_ = delivery.Ack(false)
}
// moduleOf reads the module's name out of the manifest it just built, which is the only place it is
// authoritative — the request named a repository and a path, not a module.
func moduleOf(manifest json.RawMessage) string {
var named struct {
Module string `json:"module"`
}
if err := json.Unmarshal(manifest, &named); err != nil {
return ""
}
return named.Module
}
// packagesFrom is where a build resolves the mesh's own published packages — the SDK above all
@@ -499,13 +419,8 @@ func brokerFrom() (Credential, error) {
// broker is then verified against whatever this machine already trusts.
return Credential{URL: said}, nil
}
url := strings.TrimSpace(os.Getenv("MESH_BROKER_AMQP"))
if url == "" {
return Credential{}, fmt.Errorf(
"neither MESH_BROKER_FILE nor MESH_BROKER_AMQP: a builder with no broker has " +
"nothing to build")
}
return Credential{URL: url}, nil
"no MESH_BROKER_FILE: a build machine with no credential for the bus has nothing to build")
}
// Credential is what a build machine is given so it can reach the broker.
@@ -518,47 +433,23 @@ func brokerFrom() (Credential, error) {
// out of band — here, sealed with the credential — and the endpoint is verified once at connect.
type Credential struct {
URL string `json:"url"`
// Fingerprint is SHA-256 over the broker certificate's DER bytes, or empty to verify the
// ordinary way.
Fingerprint string `json:"fingerprint,omitempty"`
// User and Password ride beside the address on the bus being built (design 25): a credential
// embedded in a URL leaks into every log line that prints a connection, so the mesh seals them
// as two fields and this machine joins them once, here, to dial.
User string `json:"user,omitempty"`
Password string `json:"password,omitempty"`
}
// dial opens the connection, pinning the broker's certificate when there is one to pin.
func dial(held Credential) (*amqp.Connection, error) {
if held.Fingerprint == "" {
return amqp.Dial(held.URL)
}
return amqp.DialTLS(held.URL, pinning(held.Fingerprint))
}
// onTheNewBus is whether a credential is for the bus being built: its address says so, and the
// mesh only ever seals such a credential with the user and password beside it.
func (c Credential) onTheNewBus() bool { return strings.HasPrefix(strings.TrimSpace(c.URL), "nats://") }
// pinning is a TLS configuration that trusts exactly one certificate.
//
// InsecureSkipVerify with a VerifyPeerCertificate is **pinning, not skipping**: the standard chain
// check is replaced, not removed, and what replaces it is stricter — one certificate is accepted
// rather than every certificate a public authority would sign.
//
// Its own function so a test can drive it against a real handshake. A pin check that is only ever
// exercised through a broker is a pin check nothing tests.
func pinning(fingerprint string) *tls.Config {
return &tls.Config{
InsecureSkipVerify: true,
VerifyPeerCertificate: func(raw [][]byte, _ [][]*x509.Certificate) error {
if len(raw) == 0 {
return errors.New("the broker presented no certificate")
}
// The leaf, and in the same spelling the mesh writes it — `sha256:` and 64 hex
// characters. Comparing a bare digest against a written fingerprint never matches,
// and the failure is indistinguishable from being pointed at the wrong broker.
sum := sha256.Sum256(raw[0])
got := "sha256:" + hex.EncodeToString(sum[:])
if got != fingerprint {
return fmt.Errorf(
"this is not the broker this builder was told about\n expected %s\n "+
"got %s\nEither this mesh's broker was replaced, or this builder is "+
"being pointed at something else. Retrying will not help",
fingerprint, got)
}
return nil
},
// natsURL is the address with this machine's credential in it, for the one dial that needs it.
func (c Credential) natsURL() string {
rest := strings.TrimPrefix(strings.TrimSpace(c.URL), "nats://")
if c.User == "" {
return "nats://" + rest
}
return "nats://" + c.User + ":" + c.Password + "@" + rest
}
+4 -2
View File
@@ -15,6 +15,8 @@ import (
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/broker"
)
// Where a builder publishes.
@@ -193,9 +195,9 @@ func TestThePinIsComparedInTheSpellingTheMeshWritesIt(t *testing.T) {
}
}
// handshakeWith runs the builder's own pin check against an address.
// handshakeWith runs the pin check the builder dials with against an address.
func handshakeWith(address, pin string) error {
conn, err := tls.Dial("tcp", address, pinning(pin))
conn, err := tls.Dial("tcp", address, broker.PinnedToFingerprint(pin))
if err != nil {
return err
}
+7 -4
View File
@@ -17,8 +17,8 @@ import (
var aDigest = "sha256:" + strings.Repeat("e", 64)
// **A build is recorded by digest and path**, whatever address the builder pushed to — and only
// what the build made is rewritten: an image the module runs from elsewhere is left where it says.
// **A build is recorded by digest and path**, whatever address the builder pushed to — what it
// made and what it stood on both; an image the module runs from elsewhere is left where it says.
func TestABuildIsRecordedWithoutTheStoresAddress(t *testing.T) {
manifest, _ := json.Marshal(map[string]any{
"module": "gitea", "version": "1",
@@ -65,8 +65,11 @@ func TestABuildIsRecordedWithoutTheStoresAddress(t *testing.T) {
if strings.Contains(string(kept.Manifest), "anchor.internal:5100") {
t.Errorf("the recorded manifest still carries the store's address:\n%s", kept.Manifest)
}
if kept.Against[0] != "anchor.internal:5100/mesh-tools/runtime@"+aDigest {
t.Errorf("what the build stood on was rewritten: %v", kept.Against)
// What the build stood on is an edge to another module's artifact, and it is recorded the way
// that artifact is: by path in the store, so the edge still names the same thing when the
// store answers at another address.
if kept.Against[0] != catalogue.ArtifactStoreScheme+"mesh-tools/runtime@"+aDigest {
t.Errorf("what the build stood on was recorded by address: %v", kept.Against)
}
}
+2 -2
View File
@@ -37,13 +37,13 @@ func askCommand(ctx context.Context, args []string) error {
arguments = json.RawMessage(positionals[2])
}
server, err := link.Connect(nil, nil)
server, err := connectLink(ctx, nil, nil, nil)
if err != nil {
return err
}
defer server.Close()
answer, err := link.Ask(ctx, server.Channel(), module, tool, arguments, *wait)
answer, err := link.Ask(ctx, server.Bus(), module, tool, arguments, *wait)
if err != nil {
return err
}
+125 -77
View File
@@ -2,8 +2,6 @@ package main
import (
"context"
"crypto/rand"
"encoding/base64"
"encoding/json"
"errors"
"flag"
@@ -31,6 +29,51 @@ import (
// control plane may send a machine is bounded by the declaration language. This is the shape the
// builder module will take when it is given work over the broker; today a person runs it, and the
// mesh records the result the same way either way.
// buildOn rebuilds every module the mesh holds that stands on the named module's artifacts — the
// rebuild a changed base needs, which nothing else asks for: their sources did not move, and
// "behind" does not see a base that did (novox/hq 04-ISSUES/131). Bases first among them too.
func buildOn(ctx context.Context, base string, wait time.Duration) error {
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
held, err := open.inventory.Catalogued(ctx)
if err != nil {
return err
}
against, err := open.inventory.BuiltAgainst(ctx)
if err != nil {
return err
}
var on []inventory.Entry
for _, e := range held {
if standsOnModule(e, base, against) {
on = append(on, e)
}
}
if len(on) == 0 {
fmt.Printf("nothing the mesh holds stands on %s\n", base)
return nil
}
on = orderByBases(on, against)
fmt.Printf("%d module(s) stand on %s:\n", len(on), base)
var failed []string
for _, e := range on {
fmt.Printf("--- %s\n", e.Manifest.Module)
source := buildSource{Repository: e.Source.Repository, Seat: e.Source.Seat}
if err := buildOne(ctx, source, e.Source.Path, e.Source.Ref, wait); err != nil {
fmt.Printf(" %v\n", err)
failed = append(failed, e.Manifest.Module)
}
}
if len(failed) > 0 {
return fmt.Errorf("%d of %d could not be built: %s", len(failed), len(on), strings.Join(failed, ", "))
}
fmt.Printf("\n%d module(s) rebuilt on %s. `push --behind` sends them on\n", len(on), base)
return nil
}
func buildCommand(ctx context.Context, args []string) error {
set := flag.NewFlagSet("build", flag.ContinueOnError)
ref := set.String("ref", "", "the branch, tag or commit to build")
@@ -46,6 +89,7 @@ func buildCommand(ctx context.Context, args []string) error {
// retype each repository is asking them to be the loop. Naming a repository and asking which
// ones need building are different requests, so they are not combined.
behind := set.Bool("behind", false, "every module the mesh holds older than its source has")
on := set.String("on", "", "rebuild every module that stands on this module's artifacts — the rebuild a changed base needs")
// A repository on the mesh's own forge, named by its path there (novox/hq ADR 0111). Without it
// the repository is external, cloned exactly as given — see source.go.
self := set.Bool("self", false, "the repository is a path on the forge holding the git seat")
@@ -53,6 +97,13 @@ func buildCommand(ctx context.Context, args []string) error {
if err != nil {
return err
}
if *on != "" {
if len(positionals) != 0 || *behind || *self {
return errors.New("build --on <module> names a base and nothing else")
}
return buildOn(ctx, *on, *wait)
}
if *behind {
if len(positionals) != 0 || *self {
return errors.New("build <repository> or build --behind, not both: one names a " +
@@ -61,7 +112,7 @@ func buildCommand(ctx context.Context, args []string) error {
return buildBehind(ctx, *wait)
}
if len(positionals) != 1 {
return errors.New("build <repository> [--self] [--path P] [--ref R] [--wait D] [--dry-run]")
return errors.New("build <repository> [--self] [--path P] [--ref R] [--wait D] [--dry-run] | build --behind | build --on <module>")
}
source := buildSource{Repository: positionals[0]}
if *self {
@@ -81,8 +132,9 @@ func buildCommand(ctx context.Context, args []string) error {
//
// **By digest and path, never by where it was pushed** (novox/hq 04-ISSUES/102). The builder
// says `<registry>:<port>/<module>/<artifact>@sha256:…`; the mesh records the artifact-store
// reference and composes the store's address back in where a reference is used. `against` is kept
// as announced: it is what the build stood on as the builder saw it, and the catalogue's edge.
// reference and composes the store's address back in where a reference is used. `against` — what
// the build stood on, the catalogue's edge — is recorded the same way, so an edge names a module's
// artifact and not the machine it was pulled from.
func buildFrom(result link.BuildResult) inventory.Build {
kept := inventory.Build{
ID: result.ID, Repository: result.Repository, Ref: result.Ref,
@@ -92,7 +144,10 @@ func buildFrom(result link.BuildResult) inventory.Build {
// edges, and it is not always listening when a build happens — on a fresh mesh it cannot
// be, for exactly the modules it needs most. Keeping them is what makes a replay able to
// rebuild the graph rather than a list of names.
Path: result.Path, Against: result.Against,
Path: result.Path,
}
for _, ref := range result.Against {
kept.Against = append(kept.Against, catalogue.Recorded(ref))
}
var announced []inventory.Artifact
for _, made := range result.Made {
@@ -206,85 +261,45 @@ func builderCommand(ctx context.Context, args []string) error {
}
name := positionals[1]
management, err := broker.ManagementFromEnvironment()
// **The build machine's credential is a module's credential** (novox/hq ADR 0131, design 28
// task 5.5): minted into the mesh's records and sealed to the machine as the builder module's
// broker secret, usable at the next push — the same act `module issue` performs, and the same
// account the composed user list carries. Nothing is created on a server; the bus reads the list.
open, err := openStores(ctx)
if err != nil {
return err
}
// The same shape of secret a token carries: enough entropy that guessing is not a strategy,
// and safe to put in a URL because that is where it goes.
raw := make([]byte, 32)
if _, err := rand.Read(raw); err != nil {
return err
}
password := base64.RawURLEncoding.EncodeToString(raw)
if err := management.CreateBuilderAccount(ctx, name, password); err != nil {
return err
}
fmt.Printf("broker account %s created, scoped to the %s queue and the %s exchange\n\n",
name, link.BuildQueue, link.Exchange)
if *forNode != "" {
known, err := broker.FromEnvironment()
if err != nil {
return fmt.Errorf("cannot deliver a credential without knowing where the broker is: %w", err)
}
inv, err := openInventory(ctx)
defer open.Close()
inv := open.inventory
shelf, err := inv.Catalogue(ctx)
if err != nil {
return err
}
defer inv.Close()
brokerAddr, err := brokerReachableAt(ctx, inv, known, *forNode)
m, known := shelf[*module]
if !known {
return fmt.Errorf("%s is not in the catalogue; `module add` it first", *module)
}
fmt.Printf("build machine %s: ", name)
node := *forNode
if node == "" {
entries, err := inv.Catalogued(ctx)
if err != nil {
return err
}
// The URL and what verifies the broker, together. A mesh's broker presents a certificate
// of the mesh's own, which is in no public trust store — so a URL on its own reaches only
// a broker somebody else vouches for, and the connection fails at TLS with an error about
// an unknown authority rather than about a missing pin.
//
// **The same two facts a node's token carries** (novox/hq ADR 0004), delivered the same
// way: out of band relative to the broker, so what is trusted does not come from the thing
// being trusted.
held, err := json.Marshal(struct {
URL string `json:"url"`
Fingerprint string `json:"fingerprint,omitempty"`
}{
URL: fmt.Sprintf("amqps://%s:%s@%s/", name, password, brokerAddr),
Fingerprint: known.Fingerprint,
})
for _, e := range entries {
if e.Manifest.Module == *module && len(e.On) > 0 {
node = e.On[0]
}
}
}
if node == "" {
return fmt.Errorf("%s is assigned nowhere; `assign <machine> %s` first, or say --node", *module, *module)
}
address, err := broker.BusAddress()
if err != nil {
return err
}
if err := inv.AcceptSecretForModule(ctx, *forNode, *module, "broker", string(held)); err != nil {
return err
}
// Not printed. It is sealed to that machine and the mesh cannot read it back, which is
// the whole point — printing it here would put the one copy that matters on a terminal.
fmt.Printf(" sealed to %s, for the %s module. It arrives with the next push.\n",
*forNode, *module)
fmt.Printf(" run `push %s` to send it\n", *forNode)
return nil
}
// The whole line only when the address is known. A URL with a placeholder where the host
// should be is a URL somebody pastes and then debugs, and the placeholder is the last thing
// they look at.
if known, err := broker.FromEnvironment(); err == nil {
fmt.Printf(" MESH_BROKER_AMQP=amqps://%s:%s@%s/\n\n", name, password, known.Address)
} else {
fmt.Printf(" the password is %s\n\n", password)
fmt.Printf(" This control plane has no %s, so it cannot say where the broker is.\n"+
" Put the password in MESH_BROKER_AMQP on the build machine.\n\n",
broker.AddressVar)
}
// Shown once, like a token, and for the same reason: what is stored is the broker's own hash
// of it, and a control plane that could show it back would be a control plane that holds it.
fmt.Println("This is the only time it is shown.")
return nil
return issueOnTheNewBus(ctx, inv, m, node, address)
}
// buildBehind builds every module the mesh holds older than its source has.
@@ -329,6 +344,14 @@ func buildBehind(ctx context.Context, wait time.Duration) error {
}
fmt.Println()
// Bases first: a module built before the module it stands on is built against the old one
// and reports success (novox/hq 04-ISSUES/131).
against, err := inv.BuiltAgainst(ctx)
if err != nil {
return err
}
stale = orderByBases(stale, against)
var failed []string
for _, e := range stale {
fmt.Printf("--- %s\n", e.Manifest.Module)
@@ -368,7 +391,7 @@ func buildOne(ctx context.Context, source buildSource, path, ref string, wait ti
}
defer ident.Close()
server, err := link.Connect(nil, nil)
server, err := connectLink(ctx, nil, nil, nil)
if err != nil {
return err
}
@@ -395,7 +418,13 @@ func buildOne(ctx context.Context, source buildSource, path, ref string, wait ti
}
fmt.Println()
result, err := link.RequestBuild(ctx, server.Channel(), request, wait)
ask, err := askOver(server)
if err != nil {
return err
}
defer ask.Close()
result, err := ask.Submit(ctx, request, wait)
if err != nil {
return err
}
@@ -466,13 +495,19 @@ func buildAndShow(ctx context.Context, source buildSource, path, ref string, wai
return err
}
defer ident.Close()
server, err := link.Connect(nil, nil)
server, err := connectLink(ctx, nil, nil, nil)
if err != nil {
return err
}
defer server.Close()
result, err := link.RequestBuild(ctx, server.Channel(), link.BuildRequest{
ask, err := askOver(server)
if err != nil {
return err
}
defer ask.Close()
result, err := ask.Submit(ctx, link.BuildRequest{
ID: fmt.Sprintf("%s-%d", "build", time.Now().UnixNano()),
Repository: repository, Path: path, Ref: ref,
Held: heldBy(ctx),
@@ -557,3 +592,16 @@ func heldBy(ctx context.Context) map[string]string {
}
return routed
}
// askOver opens the way a build is asked for, on whichever bus the mesh is on.
//
// **One place chooses**, as everywhere else the bus change went (novox/hq ADR 0116 step 5). On the bus
// the mesh runs on today this needs the controller's own connection, so it is handed one; on the bus
// being built it dials, because a build request is a one-shot and holds nothing else.
func askOver(_ *link.Server) (link.Builders, error) {
address, err := broker.BusAddress()
if err != nil {
return nil, err
}
return link.BuildsOverNATS(address)
}
+5
View File
@@ -114,8 +114,12 @@ func run() error {
return planCommand(ctx, args[1:])
case "push":
return pushCommand(ctx, args[1:])
case "rollout":
return rolloutCommand(ctx, args[1:])
case "seats":
return seatsCommand(ctx, args[1:])
case "seat":
return seatCommand(ctx, args[1:])
case "status":
return statusCommand(ctx, args[1:])
case "version":
@@ -180,6 +184,7 @@ func usage() {
operator key show the operator key, and what it can recover
build <repository> [--ref R] have a build machine build it, and record what came out
build --behind build every module the mesh holds older than its source
build --on <module> rebuild every module that stands on this module's artifacts, bases first
builds [<module>] what has been built lately, and what came of it
builder issue <name> a broker account for a build machine, scoped to build work,
delivered as the builder module's broker secret (module add it first)
+94 -59
View File
@@ -2,8 +2,6 @@ package main
import (
"context"
"crypto/rand"
"encoding/base64"
"encoding/json"
"errors"
"flag"
@@ -257,66 +255,15 @@ func moduleCommand(ctx context.Context, args []string) error {
return err
}
management, err := broker.ManagementFromEnvironment()
// Which bus this mesh is on. A module gets a credential for exactly one, and the two are
// made in entirely different ways: on the bus the mesh runs on today an account is a
// management call, and on the bus being built it is a row the next composition writes into
// the server's user list (novox/hq design 25 §4).
busAddress, err := broker.BusAddress()
if err != nil {
return err
}
// The foundation owns the bus; make sure it exists before a module binds onto it.
if err := management.EnsureEventExchanges(ctx); err != nil {
return err
}
secret := make([]byte, 32)
if _, err := rand.Read(secret); err != nil {
return err
}
password := base64.RawURLEncoding.EncodeToString(secret)
account, err := management.CreateModuleAccount(ctx, *forNode, module, password, m.Emits, m.Consumes)
if err != nil {
return err
}
// A consumer's queue, with its dead-letter, is the foundation's to declare — its own account
// may not (ADR 0043). Made now, so it exists before the module binds onto it.
if len(m.Consumes) > 0 {
if err := management.EnsureModuleQueue(ctx, *forNode, module); err != nil {
return err
}
}
known, err := broker.FromEnvironment()
if err != nil {
return fmt.Errorf("cannot deliver a credential without knowing where the broker is: %w", err)
}
brokerAddr, err := brokerReachableAt(ctx, inv, known, *forNode)
if err != nil {
return err
}
// The URL and what verifies the broker, together — a mesh's broker presents its own
// certificate, in no public trust store, so a URL alone fails at TLS (as `builder issue`).
held, err := json.Marshal(struct {
URL string `json:"url"`
Fingerprint string `json:"fingerprint,omitempty"`
Node string `json:"node"`
Module string `json:"module"`
}{
URL: fmt.Sprintf("amqps://%s:%s@%s/", account, password, brokerAddr),
Fingerprint: known.Fingerprint,
// The node and module the account is for, so the runtime names its queue as the mesh
// scoped it (<node>.<module>.events) without a manifest having to interpolate a node.
Node: *forNode,
Module: module,
})
if err != nil {
return err
}
if err := inv.AcceptSecretForModule(ctx, *forNode, module, "broker", string(held)); err != nil {
return err
}
fmt.Printf("broker account %s created for %s, scoped to what it emits and consumes\n",
account, module)
fmt.Printf(" sealed to %s. It arrives with the next push — `push %s` to send it\n",
*forNode, *forNode)
return nil
return issueOnTheNewBus(ctx, inv, m, *forNode, busAddress)
default:
return fmt.Errorf("module has no %q; it has add, list, moved, forget and issue", args[0])
@@ -567,3 +514,91 @@ func mayIssue(m catalogue.Manifest) error {
}
return nil
}
// issueOnTheNewBus gives an assigned module its credential on the bus being built.
//
// **Three things differ from a management call, and each is the point of the move.** The credential
// is minted into the mesh's records and becomes usable at the next composition, so there is no
// server to be reachable for this to work. The password travels beside the address rather than inside
// it, because the runtime's contract already separates them and a credential embedded in a URL is one
// that leaks into every log line that prints a connection. And the module's durable consumer is
// derived from what it declared rather than declared by name, so a module cannot ask for delivery of
// something it did not say it consumes.
func issueOnTheNewBus(ctx context.Context, inv *inventory.Inventory, m catalogue.Manifest,
node, busAddress string) error {
user := broker.Principal{Kind: broker.KindModule, Node: node, Module: m.Module}.Username()
password, err := inv.MintBusPassword(ctx, inventory.BusUser{
Username: user, Kind: inventory.BusModule, Node: node, Module: m.Module,
})
if err != nil {
return err
}
// Where the module is told to find the bus, and what certificate it must present. The same pair
// a node is told, for the same reason: a mesh's bus presents its own certificate, in no public
// trust store, so an address alone fails at TLS.
known, err := broker.FromEnvironment()
if err != nil {
return fmt.Errorf("cannot deliver a credential without knowing where the bus is: %w", err)
}
reachable, err := brokerReachableAt(ctx, inv, known, node)
if err != nil {
return err
}
return issueWith(ctx, inv, m, node, busAddress, known, reachable, user, password)
}
// issueWith is the delivery half: the minted password sealed to the machine as the module's broker
// secret, and the module's consumer created where the bus can be reached. Split from the minting
// so the move can issue every module against a bus whose address it worked out itself
// (`rollout mint`, design 28 task 5.2) rather than the one in this process's environment.
func issueWith(ctx context.Context, inv *inventory.Inventory, m catalogue.Manifest,
node, busAddress string, known broker.Broker, reachable, user, password string) error {
held, err := json.Marshal(struct {
URL string `json:"url"`
Fingerprint string `json:"fingerprint,omitempty"`
Node string `json:"node"`
Module string `json:"module"`
User string `json:"user"`
Password string `json:"password"`
}{
URL: "nats://" + reachable, Fingerprint: known.Fingerprint,
Node: node, Module: m.Module, User: user, Password: password,
})
if err != nil {
return err
}
if err := inv.AcceptSecretForModule(ctx, node, m.Module, "broker", string(held)); err != nil {
return err
}
// And how it hears what it consumes. Derived from its declaration, and only when it declared
// something: a module that consumes nothing needs no consumer, and creating one would be a
// durable subscription nobody reads.
if consumer, needed := broker.ConsumerFor(broker.Principal{
Kind: broker.KindModule, Node: node, Module: m.Module,
Emits: m.Emits, Consumes: m.Consumes, Serves: m.Tools,
}); needed {
if busAddress == "" {
fmt.Printf(" %s consumes; its consumer is created when the bus is reachable (`push`, then "+
"`rollout mint` again is harmless)\n", m.Module)
} else {
js, err := broker.Dial(busAddress)
if err != nil {
return fmt.Errorf("the credential is minted and the mesh cannot reach the bus to create "+
"how %s hears what it consumes: %w", m.Module, err)
}
defer js.Close()
if err := js.EnsureConsumer(consumer); err != nil {
return err
}
}
}
fmt.Printf("bus user %s minted for %s, scoped to what it emits and consumes\n", user, m.Module)
fmt.Printf(" sealed to %s. It arrives with the next push — `push %s` to send it\n", node, node)
fmt.Printf(" and it works once the bus has been told: the user list is composed into the " +
"machine holding mesh-broker\n")
return nil
}
+10 -1
View File
@@ -537,6 +537,15 @@ func onTheNetwork(ctx context.Context, inv *inventory.Inventory,
if err != nil {
return nil, err
}
// **With the seat holders on record**, or a machine running the next holder of a seat beside
// the current one resolves as two holders, is refused, and drops out of the map — taking the
// address every other machine composes for what it offers (novox/hq ADR 0131). Found live:
// the control node vanished from the private network the moment the new bus was assigned
// beside the old one.
holdings, err := inv.Holdings(ctx)
if err != nil {
return nil, err
}
var out []inventory.Overlay
for _, p := range places {
if p.Address == "" {
@@ -549,7 +558,7 @@ func onTheNetwork(ctx context.Context, inv *inventory.Inventory,
caps, _ := inv.ProfileOf(ctx, p.Name)
got, err := catalogue.Resolve(shelf, assigned,
catalogue.Node{Name: p.Name, Site: p.Site, Capabilities: caps},
catalogue.World{Unchecked: true})
catalogue.World{Unchecked: true, Holdings: holdings})
if err != nil {
continue
}
+40 -11
View File
@@ -10,7 +10,6 @@ import (
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/token"
)
@@ -67,8 +66,14 @@ func nodeCommand(ctx context.Context, args []string) error {
// because the damage is already done by the time it prints.
return publicDomain(ctx, inv, args[1:])
case "account":
// The operator's login on this machine (novox/hq to-be 29): what a home-scoped file is
// owned by and which account `ssh <node>` uses. Reports with no argument; sets with one;
// an optional second argument is the home when it is not /home/<account>.
return nodeAccount(ctx, inv, args[1:])
default:
return fmt.Errorf("node has no %q; it has add, list, show and public-domain", args[0])
return fmt.Errorf("node has no %q; it has add, list, show, public-domain and account", args[0])
}
}
@@ -106,6 +111,39 @@ func modeOf(n inventory.Node) string {
}
// publicDomainUsage is the one description of the three forms, so a refusal and the help agree.
// nodeAccount reports or sets a node's operator account (novox/hq to-be 29). Read-shaped with no
// argument, like public-domain: `node account novox` answers, it does not change anything.
func nodeAccount(ctx context.Context, inv *inventory.Inventory, positionals []string) error {
if len(positionals) == 0 || len(positionals) > 3 {
return errors.New("node account <name> — what it is now; " +
"node account <name> <account> [home] — set it (home defaults to /home/<account>)")
}
node := positionals[0]
if len(positionals) == 1 {
who, err := inv.NodeByName(ctx, node)
if err != nil {
return err
}
if who.Account == "" {
fmt.Printf("%s has no operator account known\n", node)
fmt.Printf(" `node account %s <account>` sets it\n", node)
return nil
}
fmt.Printf("%s logs a person in as %s (home %s)\n", node, who.Account, who.Home())
return nil
}
home := ""
if len(positionals) == 3 {
home = positionals[2]
}
if err := inv.SetAccount(ctx, node, positionals[1], home); err != nil {
return err
}
fmt.Printf("%s logs a person in as %s\n", node, positionals[1])
fmt.Printf(" run `push %s` once ssh-client is assigned, to send its operator config\n", node)
return nil
}
const publicDomainUsage = "node public-domain <name> — what it is now; " +
"<name> <domain> to set it; <name> --clear to take it away"
@@ -219,15 +257,6 @@ func tokenCommand(ctx context.Context, args []string) error {
// chicken-and-egg entirely: the mesh runs the broker, so a joining node's credentials can
// exist before it does. The one-time secret IS the password, so a node's first connection is
// already authenticated and enrolment is what happens over it.
if management, err := broker.ManagementFromEnvironment(); err == nil {
if err := management.CreateNodeAccount(ctx, issued.Node.Name, issued.Secret); err != nil {
return err
}
fmt.Printf("broker account %s created, scoped to %s and the %s exchange\n\n",
issued.Node.Name, link.QueueFor(issued.Node.Name), link.Exchange)
} else if !errors.Is(err, broker.ErrNotConfigured) {
return err
}
made := token.Token{Node: issued.Node.Name, Signer: key.Public, Secret: issued.Secret,
Adopted: issued.Node.Adopted}
+142 -1
View File
@@ -2,12 +2,15 @@ package main
import (
"context"
"encoding/json"
"errors"
"flag"
"fmt"
"os"
"strings"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/secrets"
)
@@ -26,9 +29,25 @@ import (
// operator key make [--out <file>] make a keypair: private half to the file, public half printed
// operator key set <public> tell the mesh which key to seal to
// operator key show the public key, its fingerprint, and what it can recover
const operatorUsage = "operator key make [--out <file>] | operator key set <public> [--replace] | operator key show"
const operatorUsage = "operator key make [--out <file>] | operator key set <public> [--replace] | " +
"operator key show | operator issue <name> --invokes <tool,tool|*> | operator revoke <name> | " +
"operator list"
func operatorCommand(ctx context.Context, args []string) error {
if len(args) == 0 {
return errors.New(operatorUsage)
}
// The people who may reach the mesh's tools (design 25 §7). Beside the operator's key because
// both answer "who, other than a machine, may do something here" — and a person reading this
// command's usage is asking exactly that.
switch args[0] {
case "issue":
return personIssue(ctx, args[1:])
case "revoke":
return personRevoke(ctx, args[1:])
case "list":
return personList(ctx)
}
if len(args) < 2 || args[0] != "key" {
return errors.New(operatorUsage)
}
@@ -160,3 +179,125 @@ func readPrivateKey(path string) (string, error) {
}
return strings.TrimSpace(string(raw)), nil
}
// personIssue gives somebody a credential for the mesh's tools, and prints it once.
//
// **Printed, not stored.** The mesh keeps a hash and nothing else, so this is the only moment the
// credential exists anywhere but on the workstation that will use it — the same contract a token has,
// and for the same reason: a credential recoverable from the mesh's store has the store's blast
// radius.
func personIssue(ctx context.Context, args []string) error {
set := flag.NewFlagSet("operator issue", flag.ContinueOnError)
invokes := set.String("invokes", "", "the tools this person may call, comma-separated, or * for every one")
if err := set.Parse(args); err != nil {
return err
}
if set.NArg() != 1 {
return errors.New("operator issue <name> --invokes <tool,tool|*>")
}
name := set.Arg(0)
if *invokes == "" {
return errors.New(
"say what this person may call: --invokes mesh-catalog.catalog_tools,gitea.repo_create, " +
"or --invokes '*' for an administrator")
}
var tools []string
for _, t := range strings.Split(*invokes, ",") {
if t = strings.TrimSpace(t); t != "" {
tools = append(tools, t)
}
}
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
inv := open.inventory
if err := inv.RecordPerson(ctx, inventory.Person{Name: name, Invokes: tools}); err != nil {
return err
}
// Refused here rather than at the next composition, where it would stop the whole file being
// written for everybody. A name that cannot be part of a subject is one the server would read as
// a wider permission than anybody granted.
if _, err := broker.PermissionsFor(broker.Principal{
Kind: broker.KindPerson, Module: name, Invokes: tools, PasswordHash: "x",
}); err != nil {
return err
}
user := broker.Principal{Kind: broker.KindPerson, Module: name}.Username()
password, err := inv.MintBusPassword(ctx, inventory.BusUser{Username: user, Kind: inventory.BusPerson})
if err != nil {
return err
}
where, err := broker.FromEnvironment()
if err != nil && !errors.Is(err, broker.ErrNotConfigured) {
return err
}
held, err := json.Marshal(struct {
URL string `json:"url"`
Fingerprint string `json:"fingerprint,omitempty"`
User string `json:"user"`
Password string `json:"password"`
Person string `json:"person"`
Invokes []string `json:"invokes"`
}{
URL: "nats://" + where.Address, Fingerprint: where.Fingerprint,
User: user, Password: password, Person: name, Invokes: tools,
})
if err != nil {
return err
}
fmt.Printf("issued %s, who may call %s\n", name, strings.Join(tools, ", "))
fmt.Println(" this is the only time the credential is printed; the mesh keeps a hash")
fmt.Println(" it works once the bus has been told, which is the next push to the machine holding mesh-broker")
fmt.Println()
fmt.Println(string(held))
return nil
}
func personRevoke(ctx context.Context, args []string) error {
if len(args) != 1 {
return errors.New("operator revoke <name>")
}
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
if err := open.inventory.ForgetPerson(ctx, args[0]); err != nil {
return err
}
// **Revoked at the next composition, not now.** The bus's users are a file, so a credential stops
// working when the file no longer names it. Said plainly, because "revoked" that still works for
// another minute is worth knowing about.
fmt.Printf("%s is forgotten, and their credential stops working at the next composition — "+
"push the machine holding mesh-broker to make it so\n", args[0])
return nil
}
func personList(ctx context.Context) error {
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
people, err := open.inventory.People(ctx)
if err != nil {
return err
}
if len(people) == 0 {
fmt.Println("nobody but machines reaches this mesh")
return nil
}
for _, p := range people {
fmt.Printf("%-20s %s\n", p.Name, strings.Join(p.Invokes, ", "))
}
return nil
}
+108
View File
@@ -0,0 +1,108 @@
package main
import (
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// entry is a module as the catalogue holds it: built, so its manifest carries no `build` any more.
func entry(module string, _ ...string) inventory.Entry {
return inventory.Entry{Manifest: catalogue.Manifest{Module: module}}
}
// stoodOn is what each module's newest build recorded it was handed.
func stoodOn(edges map[string][]string) map[string][]string {
out := map[string][]string{}
for module, bases := range edges {
for _, b := range bases {
out[module] = append(out[module], catalogue.ArtifactStoreScheme+b+"/runtime@sha256:"+strings.Repeat("0", 64))
}
}
return out
}
// A module built before the module it stands on is built against the old one and reports success
// (novox/hq 04-ISSUES/131). So bases come first, however the set arrived.
func TestBasesAreBuiltBeforeWhatStandsOnThem(t *testing.T) {
in := []inventory.Entry{entry("app"), entry("runtime"), entry("other"), entry("base")}
edges := stoodOn(map[string][]string{"app": {"runtime"}, "runtime": {"base"}})
got := orderByBases(in, edges)
pos := map[string]int{}
for i, e := range got {
pos[e.Manifest.Module] = i
}
if !(pos["base"] < pos["runtime"] && pos["runtime"] < pos["app"]) {
t.Fatalf("bases not first: %v", pos)
}
if len(got) != 4 {
t.Fatalf("an entry was lost or doubled: %d", len(got))
}
// A base outside the set is not waited for: it is not being rebuilt.
got = orderByBases([]inventory.Entry{entry("app")}, stoodOn(map[string][]string{"app": {"elsewhere"}}))
if len(got) != 1 {
t.Fatalf("a dependency outside the set changed the set: %v", got)
}
}
// A module registered from its manifest and never built still names its bases there; once built,
// the recorded edge is what says so. Both are read, and a module never stands on itself.
func TestWhatStandsOnAModuleIsReadFromItsBuildOrItsManifest(t *testing.T) {
built := entry("gitea")
edges := stoodOn(map[string][]string{"gitea": {"mesh-tools"}})
if !standsOnModule(built, "mesh-tools", edges) {
t.Fatal("a recorded edge was not read")
}
if standsOnModule(built, "gitea", edges) || standsOnModule(built, "postgres", edges) {
t.Fatal("an edge was invented")
}
fresh := inventory.Entry{Manifest: catalogue.Manifest{Module: "plex", Build: &catalogue.Build{
On: []catalogue.BuildsOn{{Arg: "RUNTIME_BASE", Module: "mesh-tools", Artifact: "runtime"}},
}}}
if !standsOnModule(fresh, "mesh-tools", nil) {
t.Fatal("a manifest's own base was not read")
}
}
// A merge names a repository the way the forge does; a source is recorded the way a build was
// asked for. The two meet on owner/repo and branch, whichever form the record took.
func TestAMergeMatchesTheSourcesBuiltFromIt(t *testing.T) {
m := link.SourceMoved{Owner: "novox", Repo: "mesh-controller", Base: "main",
CloneURL: "http://forge.internal:20000/novox/mesh-controller.git"}
for _, s := range []inventory.Source{
{Repository: "http://forge.internal:20000/novox/mesh-controller.git", Ref: "main"},
{Repository: "novox/mesh-controller", Seat: "git", Ref: ""},
{Repository: "https://elsewhere.example/novox/mesh-controller", Ref: "main"},
} {
if !sourceIs(s, m) {
t.Errorf("%+v was not matched by the merge", s)
}
}
for _, s := range []inventory.Source{
{Repository: "novox/mesh-host", Seat: "git"},
{Repository: "http://forge.internal:20000/novox/mesh-controller.git", Ref: "release"},
} {
if sourceIs(s, m) {
t.Errorf("%+v was matched by a merge that is not its", s)
}
}
}
// A merge made before the source was last seen is history: it does not move the source, and a
// merge that says nothing about when it was made is taken as news.
func TestAMergeOlderThanTheLastLookIsHistory(t *testing.T) {
seen := time.Date(2026, 9, 28, 3, 0, 0, 0, time.UTC)
if !isHistory("2026-09-28T02:00:00Z", seen) {
t.Fatal("an older merge was taken as news")
}
if isHistory("2026-09-28T04:00:00Z", seen) {
t.Fatal("a newer merge was taken as history")
}
if isHistory("", seen) || isHistory("2026-09-28T02:00:00Z", time.Time{}) {
t.Fatal("a merge or a source with no time on it was refused")
}
}
+139 -5
View File
@@ -83,9 +83,17 @@ func planFor(ctx context.Context, open *stores, nodeName string) (catalogue.Reso
return catalogue.Resolution{}, nil, err
}
// The operator account this node logs a person in as, and where its home is (novox/hq to-be
// 29) — carried so a home-scoped file's owner and path resolve for this machine.
who, err := inv.NodeByName(ctx, nodeName)
if err != nil {
return catalogue.Resolution{}, nil, err
}
resolved, err := catalogue.Resolve(shelf, assigned,
catalogue.Node{Name: nodeName, Site: site, Capabilities: capabilities,
At: onNetwork[nodeName], PublicDomain: publicDomain}, world)
At: onNetwork[nodeName], PublicDomain: publicDomain,
Account: who.Account, AccountHome: who.AccountHome}, world)
if err != nil {
return catalogue.Resolution{}, nil, err
}
@@ -177,6 +185,15 @@ func theRestOfTheMesh(ctx context.Context, inv *inventory.Inventory,
// Every node, not only the placed ones. A machine that was never put on the private network
// still runs modules, still holds claims, and still offers whatever it offers.
// **Who holds each seat on record, before anything is resolved** (novox/hq ADR 0131). Both
// passes below need it: without it, the assignment standing beside a seat's holder — the next
// holder, waiting for the handover — is refused as a second holder, and its node's whole set
// with it.
holdings, err := inv.Holdings(ctx)
if err != nil {
return catalogue.World{}, err
}
nodes, err := inv.Nodes(ctx)
if err != nil {
return catalogue.World{}, err
@@ -219,7 +236,7 @@ func theRestOfTheMesh(ctx context.Context, inv *inventory.Inventory,
offered := map[string][]catalogue.Provider{}
var firstHeld []catalogue.Held
for _, o := range others {
got, err := catalogue.Resolve(shelf, o.assigned, o.node, catalogue.World{Unchecked: true})
got, err := catalogue.Resolve(shelf, o.assigned, o.node, catalogue.World{Unchecked: true, Holdings: holdings})
if err != nil {
// Their set does not resolve for some other reason. Not this node's problem to
// report, and nothing of theirs is running, so it offers nothing.
@@ -250,7 +267,7 @@ func theRestOfTheMesh(ctx context.Context, inv *inventory.Inventory,
// with several providers (novox/hq ADR 0110), so a node consuming one resolves only once the
// holder is known. Without them its set is refused here, and a refused node's own claims drop
// out of what the mesh holds — so a second holder of one of its seats would pass unrefused.
world := catalogue.World{Offered: offered, Held: firstHeld}
world := catalogue.World{Offered: offered, Held: firstHeld, Holdings: holdings}
var held []catalogue.Held
for _, o := range others {
got, err := catalogue.Resolve(shelf, o.assigned, o.node, world)
@@ -488,6 +505,13 @@ func renderingFor(ctx context.Context, open *stores, node string,
if err != nil {
return catalogue.Rendering{}, inventory.Node{}, err
}
// The private network's range, offered to a module as ${machine:mesh-range} — a module that must
// name the whole mesh (an intrusion filter that must never ban a tunnel peer) names it here
// rather than hardcoding a value it cannot know.
meshRange, err := overlayRange(ctx, inv)
if err != nil {
return catalogue.Rendering{}, inventory.Node{}, err
}
// The artifact store as this node reaches it now — the address every image and archive the
// mesh built is fetched through, composed here and recorded nowhere — with what the mesh has
@@ -513,6 +537,19 @@ func renderingFor(ctx context.Context, open *stores, node string,
return catalogue.Rendering{}, inventory.Node{}, err
}
// Each machine's operator account, so an ssh Host block can name the login for every node
// (novox/hq to-be 29). Keyed by the bare node name, which entriesFrom falls back to.
allNodes, err := inv.Nodes(ctx)
if err != nil {
return catalogue.Rendering{}, inventory.Node{}, err
}
accounts := map[string]string{}
for _, n := range allNodes {
if n.Account != "" {
accounts[n.Name] = n.Account
}
}
// And every routed name → the node that serves it (novox/hq ADR 0066). Alongside the
// `<node>.internal` names above, so a container — or an internal ACME validator — resolves a
// routed name to the proxy that serves it, mesh-wide. The mesh publishes the names it was told
@@ -593,12 +630,25 @@ func renderingFor(ctx context.Context, open *stores, node string,
if err != nil {
return catalogue.Rendering{}, inventory.Node{}, err
}
// The bus's user list, for the machine that runs the bus. Composed per push rather than kept,
// because it is a function of the mesh's records and a kept copy could disagree with them.
busUsers, err := composeBusUsers(ctx, inv, plan.Modules)
if err != nil {
return catalogue.Rendering{}, inventory.Node{}, err
}
memberships, err := inv.BusMemberships(ctx)
if err != nil {
return catalogue.Rendering{}, inventory.Node{}, err
}
return catalogue.Rendering{
BusMembership: memberships[node],
Settings: settings, Generators: gens, Grants: grants, Needed: needed, Ports: ports,
Certificate: certificate, Authority: authority, Mesh: private, Names: names,
Machines: machines,
Suffix: overlay.Suffix(), Foundation: foundation, Kept: kept, Adopted: record.Adopted,
Suffix: overlay.Suffix(), MeshRange: meshRange, Accounts: accounts, Foundation: foundation,
Kept: kept, Adopted: record.Adopted,
Given: given, Taken: taken, Seats: seats, ArtifactStore: artifactStore, Built: built,
BusUsers: busUsers,
}, record, nil
}
@@ -923,12 +973,26 @@ func planCommand(ctx context.Context, args []string) error {
if !ok {
continue
}
fmt.Printf("\n--- %v %v ---\n%s", r["id"], r["path"], content)
fmt.Printf("\n--- %s ---\n%s", shownAs(r), content)
}
}
return nil
}
// shownAs is the heading `plan --show` puts over a resource's content.
//
// **A file written into says so.** Its content is the mesh's part of a file that is otherwise the
// machine's — the keys of a JSON document (novox/hq ADR 0102), the region of a hosts file (issue
// 128). Shown under a bare path it reads as the whole file, and a person checking what a take
// replaces would see a hosts file of a dozen lines where the machine keeps thirty.
func shownAs(r map[string]any) string {
heading := fmt.Sprintf("%v %v", r["id"], r["path"])
if into, ok := r["into"].(string); ok && into != "" {
heading += fmt.Sprintf(" (written into, %s)", into)
}
return heading
}
// licencesFor is what this node can be answered with by record, and what it was put on.
//
// A mesh with no licences at all is the ordinary case and must not be an error: every existing
@@ -1144,6 +1208,76 @@ func portsOn(
return out, nil
}
// composeBusUsers is the bus's user list, for a push to the machine that runs the bus.
//
// Empty for every other machine, and for every machine while the mesh is on the bus it runs on
// today — where accounts are a management call and there is no file to write.
//
// **Composed on each push, never kept.** The list is a function of the mesh's records (who exists,
// what runs where, what each declares), and a stored copy would be a second account of who may reach
// the bus, able to disagree with the records while both looked internally consistent (ADR 0043).
//
// A user the mesh has never minted a password for is **left out and said**, not written as a user
// without one — the composer refuses that, because a user with no password is a user anybody is. That
// is an ordinary situation with an obvious remedy (`module issue`, or enrolling), so the push carries
// the rest rather than failing: a bus that is missing one module's user is a mesh where that module
// cannot connect, and a bus with no file at all is a mesh where nothing can.
func composeBusUsers(ctx context.Context, inv *inventory.Inventory,
onThisNode []catalogue.Manifest) (string, error) {
// **Not gated on which bus the controller is on, and that was a bug.** It read "compose this only
// once the mesh is on the new bus" — which cannot work, because the server needs its user list
// *before* anything moves onto it. Step 2 of the change is exactly that: the server stands in the
// mesh carrying nothing, on its own ports, while every node is still on the old bus (novox/hq
// ADR 0116). Under the old gating that step could not happen: the module would come up, find no
// accounts file, and its entrypoint would wait for one the controller had decided not to write.
//
// So the question is only whether this machine runs the module that asked for the file. A mesh
// that never moves has written a user list nothing reads, which costs a few hundred bytes on one
// node; the reverse cost a step that cannot be taken.
//
// Asked of what this push resolves to rather than of the seat's holder mesh-wide: the file is a
// resource of that module, so the question is whether it is here.
holdsTheBus := false
for _, m := range onThisNode {
if m.BusUsers != "" && m.ClaimsSeat("mesh-broker") {
holdsTheBus = true
}
}
if !holdsTheBus {
return "", nil
}
records, err := inv.BusRecords(ctx)
if err != nil {
return "", err
}
users, err := broker.Users(records)
if err != nil {
return "", err
}
kept, err := inv.BusUsers(ctx)
if err != nil {
return "", err
}
hashes := make(map[string]string, len(kept))
for name, u := range kept {
hashes[name] = u.PasswordHash
}
filled, missing := broker.WithPasswords(users, hashes)
if len(missing) > 0 {
fmt.Printf("the bus's user list leaves out %d user(s) the mesh has minted no credential "+
"for: %s. Each is a user that cannot connect until one is issued\n",
len(missing), strings.Join(missing, ", "))
}
if len(filled) == 0 {
return "", fmt.Errorf(
"this machine runs the bus and not one user has a credential, so the composed list " +
"would refuse every connection in the mesh")
}
return broker.ComposeAccounts(filled)
}
// foundationPortsFor is the broker port, kept only when a module resolved onto this node listens
// on it (novox/hq issue: the broker opening leaked onto every node). The foundation opening
// exists to WIDEN the broker's `from: mesh` port to from-anywhere, because a machine enrolling is
+14
View File
@@ -35,3 +35,17 @@ func TestListensLinesAreEmptyForAModuleWithNothingToListenOn(t *testing.T) {
t.Errorf("a module with no listens should print nothing, got %v", got)
}
}
// `plan --show` says when a file is written into rather than over (novox/hq issue 128), or the
// mesh's region of a hosts file reads as the whole file.
func TestAFileWrittenIntoIsShownAsSuch(t *testing.T) {
region := shownAs(map[string]any{
"id": "mesh-wireguard.fact-node-names", "path": "/etc/hosts", "into": "block"})
if region != "mesh-wireguard.fact-node-names /etc/hosts (written into, block)" {
t.Errorf("the region is shown as %q", region)
}
whole := shownAs(map[string]any{"id": "dnsmasq.fact-node-zones", "path": "/etc/mesh-resolver/nodes.conf"})
if strings.Contains(whole, "written into") {
t.Errorf("a whole file is shown as written into: %q", whole)
}
}
+167 -16
View File
@@ -38,6 +38,27 @@ func reportUnhostable(node string, plan catalogue.Resolution) {
// nothing in it was wrong, and no one edit was the one that should have been a new file.
// serve is the control plane running: one connection to the broker, one queue, one consumer.
// connectLink opens the controller's link over whichever bus this process is on (design 25: one
// variable moves it). The streams and this controller's consumers are raised first on the new bus,
// so nothing served here finds them missing.
func connectLink(ctx context.Context, inv *inventory.Inventory, enroller link.Enroller, listener link.Listener) (*link.Server, error) {
busAddress, err := broker.BusAddress()
if err != nil {
return nil, err
}
if inv != nil {
if err := raiseTheBus(ctx, inv, busAddress); err != nil {
return nil, err
}
}
js, err := broker.Dial(busAddress)
if err != nil {
return nil, fmt.Errorf("the mesh is on the bus at %s and this control plane cannot reach it: %w",
broker.BareAddress(busAddress), err)
}
return link.ConnectNats(js, enroller, listener), nil
}
func serve(ctx context.Context) error {
open, err := openStores(ctx)
if err != nil {
@@ -61,11 +82,6 @@ func serve(ctx context.Context) error {
}
fmt.Printf("signing as %s\n", key.Fingerprint()[:16])
management, err := broker.ManagementFromEnvironment()
if err != nil && !errors.Is(err, broker.ErrNotConfigured) {
return err
}
// Where the broker is and what to expect there, so a node can be told how to come back
// without a person and a new token.
known, err := broker.FromEnvironment()
@@ -77,12 +93,22 @@ func serve(ctx context.Context) error {
"reconnect. Set %s and %s.\n", broker.AddressVar, broker.CertificateVar)
}
work := link.Enrolment{Inventory: inv, Identity: ident, Management: management, Broker: known}
server, err := link.Connect(work, work)
// **Which bus this mesh is on, read once** (novox/hq ADR 0116 step 5). Both clients ship; both
// being live is refused, because a mesh half on each is one where a declaration goes out on one
// and the report comes back on the other, and every component logs success while it happens.
work := link.Enrolment{Inventory: inv, Identity: ident, Broker: known,
OnNATS: true}
server, err := connectLink(ctx, inv, work, work)
if err != nil {
return err
}
defer server.Close()
// The bus's own objects, asserted on every start. **Not created once at genesis**: a stream
// somebody deleted, a mesh raised from a restored backup, or a bus whose data directory was
// replaced all have records and no objects — and a node whose consumer is missing hears nothing
// while everything else about it looks correct.
// And build results nobody was waiting for. A build triggered any other way than `build`
// would otherwise be reported into the void, which is the same as not reporting it.
server.Records(builds{inv})
@@ -136,13 +162,13 @@ func declare(ctx context.Context, args []string) error {
return err
}
server, err := link.Connect(nil, nil)
server, err := connectLink(ctx, nil, nil, nil)
if err != nil {
return err
}
defer server.Close()
if err := link.Declare(ctx, server.Channel(), ident, node, raw, 15*time.Second); err != nil {
if err := link.Declare(ctx, server.Bus(), ident, node, raw, 15*time.Second); err != nil {
return err
}
fmt.Printf("sent %s a signed declaration (%d bytes)\n", node, len(raw))
@@ -249,7 +275,7 @@ func pushCommand(ctx context.Context, args []string) error {
return err
}
server, err := link.Connect(nil, nil)
server, err := connectLink(ctx, nil, nil, nil)
if err != nil {
return err
}
@@ -310,7 +336,7 @@ func pushCommand(ctx context.Context, args []string) error {
if err != nil {
return err
}
if err := link.Declare(ctx, server.Channel(), ident, s.node, body, 15*time.Second); err != nil {
if err := link.Declare(ctx, server.Bus(), ident, s.node, body, 15*time.Second); err != nil {
return err
}
// After it is away, not before. A digest recorded for something that failed to send would
@@ -393,7 +419,7 @@ func pushCommand(ctx context.Context, args []string) error {
return declarationWith(held, open, node, plan, settings, gens, Allocating)
},
func(s readyNode, body []byte) error {
if err := link.Declare(ctx, server.Channel(), ident, s.node, body,
if err := link.Declare(ctx, server.Bus(), ident, s.node, body,
15*time.Second); err != nil {
return err
}
@@ -502,8 +528,14 @@ func composeEach(names []string,
continue
}
if len(declared.Resources) == 0 {
fmt.Printf("%s is assigned nothing — skipped\n", name)
continue
// Sent, not skipped (novox/hq issue 127). A node whose declaration composes to
// nothing may have HELD something before — the broker opening a placement gave it,
// say — and skipping the empty declaration leaves that last resource in force
// forever, re-applied by the node's own heartbeat, with no way for the mesh to say
// it is gone. An empty declaration is the correction: the host drops what the mesh
// owned and keeps what it found (the adoption envelope still rides along). A node
// that never held anything applies it as the no-op it is.
fmt.Printf("%s owns nothing now — sent so it drops what it last held\n", name)
}
sending = append(sending, readyNode{name, declared})
}
@@ -600,7 +632,7 @@ func sendTo(ctx context.Context, open *stores, names []string) error {
len(refusals), strings.Join(refusals, "\n\n"))
}
server, err := link.Connect(nil, nil)
server, err := connectLink(ctx, nil, nil, nil)
if err != nil {
return err
}
@@ -611,7 +643,7 @@ func sendTo(ctx context.Context, open *stores, names []string) error {
if err != nil {
return err
}
if err := link.Declare(ctx, server.Channel(), ident, s.node, body, 15*time.Second); err != nil {
if err := link.Declare(ctx, server.Bus(), ident, s.node, body, 15*time.Second); err != nil {
return err
}
record, err := inv.NodeByName(ctx, s.node)
@@ -664,3 +696,122 @@ func wouldSend(ctx context.Context, open *stores,
}
return out, nil
}
// raiseTheBus asserts the streams and consumers the mesh's own traffic needs.
//
// **Every start, and it says what it did.** The objects are the mesh's, created by nothing else —
// the controller is their only writer (design 25 §3) — so a mesh that came up without them is one
// where nodes connect, authenticate, and hear nothing. Said rather than silent for the reason the
// first line of `serve` is said: a log that is quiet on success and loud on failure reads as broken
// when it is working.
func raiseTheBus(ctx context.Context, inv *inventory.Inventory, address string) error {
js, err := broker.Dial(address)
if err != nil {
return fmt.Errorf("the mesh is on the bus at %s and this control plane cannot reach it: %w",
broker.BareAddress(address), err)
}
defer js.Close()
// **Its own user, before anything else.** The controller's account is created by the installer at
// a bootstrap password, before there is a controller to mint one — so nothing recorded a hash for
// it, and the first composition would leave the writer out of the file it was writing. Recorded
// only if absent: a credential the mesh minted since is the one that counts.
// **Its own user, before anything else it does here.** The controller's account is created by the
// installer at a bootstrap password, before there is a controller to mint one — so nothing
// recorded a hash for it, and the first composition would leave the writer out of the file it was
// writing: a bus nothing can connect to, produced by the thing connected to it. Recorded only if
// absent, so a restart cannot put the bootstrap credential back over a rotated one.
if user, password, _ := broker.CredentialIn(address); user != "" && password != "" {
if err := inv.SeedBusUser(ctx, inventory.BusUser{
Username: user, Kind: inventory.BusController,
}, password); err != nil {
return fmt.Errorf("cannot record the credential this control plane is using: %w", err)
}
}
nodes, err := inv.Nodes(ctx)
if err != nil {
return err
}
names := make([]string, 0, len(nodes))
for _, n := range nodes {
names = append(names, n.Name)
}
if err := broker.Raise(js, names); err != nil {
return err
}
// The work queues of the mesh's own roles (novox/hq ADR 0121). The queue before the holder,
// deliberately: work queues until somebody arrives to do it, so assigning a build machine a week
// after something started asking for builds flushes the backlog instead of having lost it.
// With the seats' holders, so each role's work queue gets the consumer its holder takes
// work from. Passed as nil until the first live raise, which left the build machine bound to a
// consumer nothing had created (2026-09-28).
holders, err := seatHolders(ctx, inv)
if err != nil {
return err
}
if err := broker.RaiseSeats(js, inventory.MeshSeats(), holders); err != nil {
return err
}
// And how every module hears what it consumes. Derived from the same records the user list is
// composed from, so a module the mesh grants a consumer's subjects has that consumer waiting.
// Done on every raise, not only when a credential is issued: every module moved onto this bus
// by the rollout was issued on the old one, and came up with nothing to bind to (2026-09-28).
records, err := inv.BusRecords(ctx)
if err != nil {
return err
}
users, err := broker.Users(records)
if err != nil {
return err
}
hearing := 0
for _, p := range users {
consumer, needed := broker.ConsumerFor(p)
if !needed {
continue
}
if err := js.EnsureConsumer(consumer); err != nil {
return fmt.Errorf("how %s on %s hears what it consumes: %w", p.Module, p.Node, err)
}
hearing++
}
fmt.Printf("the bus at %s has its streams, %d machine(s) can hear a declaration, and %d module(s) "+
"can hear what they consume\n", broker.BareAddress(address), len(names), hearing)
return nil
}
// seatHolders is who holds each of the mesh's seats, by seat name: the record where a handover
// wrote one, and the assigned module claiming the seat otherwise — the same derivation the
// resolver makes, read from the catalogue rather than re-resolved.
func seatHolders(ctx context.Context, inv *inventory.Inventory) (map[string]broker.Holder, error) {
out := map[string]broker.Holder{}
entries, err := inv.Catalogued(ctx)
if err != nil {
return nil, err
}
for _, e := range entries {
if len(e.On) == 0 {
continue
}
for _, c := range e.Manifest.Claims {
seat, known := catalogue.SeatNamed(c.Name)
if !known {
continue
}
if _, taken := out[seat.Name]; !taken {
out[seat.Name] = broker.Holder{Node: e.On[0], Module: e.Manifest.Module}
}
}
}
recorded, err := inv.Holdings(ctx)
if err != nil {
return nil, err
}
for _, h := range recorded {
if seat, known := catalogue.SeatNamed(h.Claim); known {
out[seat.Name] = broker.Holder{Node: h.Node, Module: h.Module}
}
}
return out, nil
}
+6 -4
View File
@@ -39,12 +39,14 @@ func TestOneUnresolvableNodeStillLetsTheRestBeSent(t *testing.T) {
}
}
// And a machine assigned nothing is neither sent nor a refusal — it is nothing to say.
func TestAMachineAssignedNothingIsNotARefusal(t *testing.T) {
// A machine whose declaration composes to nothing is SENT the empty declaration, not skipped
// (novox/hq issue 127): it may have held something before, and only sending the empty
// declaration tells it to drop what the mesh owned. It is never a refusal.
func TestAnEmptyDeclarationIsSentSoTheNodeDropsWhatItHeld(t *testing.T) {
sending, refusals := composeEach([]string{"spare"},
func(string) (sendable, error) { return sendable{}, nil })
if len(sending) != 0 || len(refusals) != 0 {
t.Errorf("a machine assigned nothing was treated as something: %v / %v", sending, refusals)
if len(sending) != 1 || len(refusals) != 0 {
t.Errorf("an empty declaration must be sent, not skipped or refused: %v / %v", sending, refusals)
}
}
+455
View File
@@ -0,0 +1,455 @@
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"os"
"strings"
"time"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/secrets"
)
// Moving the mesh's own traffic to the bus being built (novox/hq ADR 0116 step 5).
//
// **The whole mesh moves at once, so there is nothing to inspect afterwards.** Every seam ships both
// transports and every one of them chooses by a single fact; this is the step that flips it. That
// shape is deliberate — steps 1 to 4 leave every node where it is, so the cost of being wrong stays
// bounded until here — and it means the useful work is almost all in the checking.
//
// So `rollout check` is the command that matters and the one that can be run any number of times
// against a mesh that is serving. It answers from records: what is missing, and what would happen.
// `rollout` itself refuses unless the check is clean.
//
// **The old broker goes with the move, and goes last** (novox/hq ADR 0131): AMQP is not a provision,
// so once every machine reports on the new bus its module is unassigned. Only the mesh's own traffic
// is what moves, which is why this is survivable at all: what breaks if it goes wrong is the mesh's
// ability to change things, not the services its modules are serving — measured on 2026-09-27, when
// a seat emptied mid-change and the control plane looped for two hours while every service stayed up.
const rolloutUsage = "rollout check | rollout mint [--again] | rollout hand <node> | rollout --confirm"
func rolloutCommand(ctx context.Context, args []string) error {
switch {
case len(args) == 1 && args[0] == "check":
return rolloutCheck(ctx)
case len(args) == 1 && args[0] == "mint":
return rolloutMint(ctx, false)
case len(args) == 2 && args[0] == "hand":
return rolloutHand(ctx, args[1])
case len(args) == 2 && args[0] == "mint" && args[1] == "--again":
// Every credential minted afresh, whether or not one exists — for a mint that was wrong
// before anything was pushed. Afterwards nothing that received the old one still works,
// which is fine exactly when nothing received it.
return rolloutMint(ctx, true)
case len(args) == 1 && args[0] == "--confirm":
return errors.New(
"the rollout itself is not built yet: `rollout check` answers whether it could run, and " +
"what is missing. Moving every node at once is the one step with nothing to inspect " +
"afterwards, so it is not being written before the check it depends on has been run " +
"against a real mesh")
default:
return errors.New(rolloutUsage)
}
}
// rolloutCheck says whether the mesh could move, and what would happen if it did.
func rolloutCheck(ctx context.Context) error {
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
inv := open.inventory
state, err := readinessOf(ctx, inv)
if err != nil {
return err
}
fmt.Println("the bus this mesh would move to")
if state.TheBus == "" {
fmt.Printf(" nothing names one (%s is unset)\n", broker.NATSVar)
} else {
standing := "not answering"
if state.ServerStanding {
standing = "answering"
}
fmt.Printf(" %s — %s\n", state.TheBus, standing)
}
fmt.Println()
fmt.Println("what would move")
for _, step := range broker.WhatMoves(state) {
fmt.Printf(" %s\n", step)
}
fmt.Println()
why := notReadyOf(state)
if len(why) == 0 {
fmt.Println("nothing is missing: this mesh could move its bus.")
fmt.Println()
fmt.Println("Read `what would move` above once more before running it. Every node moves at the")
fmt.Println("same moment and there is no half-moved state to look at afterwards.")
return nil
}
fmt.Printf("not ready — %d thing(s) to do first:\n", len(why))
for i, w := range why {
fmt.Printf(" %d. %s\n", i+1, w)
}
return nil
}
// readinessOf gathers what the mesh knows about its own ability to move.
//
// Reads and one dial, and nothing is written. Safe to run on a mesh that is serving, which is the
// point: the answer is only useful if it can be had without committing to anything.
func readinessOf(ctx context.Context, inv *inventory.Inventory) (broker.Readiness, error) {
state := broker.Readiness{
Credentialled: map[string]bool{},
ModuleCredentialled: map[string]bool{},
// The old broker keeps its other clients on this installation, and saying so is how the plan
// stops reading as a retirement.
}
address, _, err := broker.OnNATS()
if err != nil {
return state, err
}
state.TheBus = address
if address != "" {
// One dial, briefly. "Is it answering" is the one fact records cannot hold, and a mesh about
// to move onto a server that is not there should hear it here rather than afterwards.
//
// **Dialled the way the mesh dials it** — credential and pin — because a bare connect to a
// bus that requires TLS and a user fails at the handshake, and the check then reported a
// standing server as absent (seen live, 2026-09-28).
if js, err := broker.Dial(address, nats.Timeout(5*time.Second)); err == nil {
state.ServerStanding = true
js.Close()
}
}
nodes, err := inv.Nodes(ctx)
if err != nil {
return state, err
}
kept, err := inv.BusUsers(ctx)
if err != nil {
return state, err
}
shelf, err := inv.Catalogue(ctx)
if err != nil {
return state, err
}
for _, n := range nodes {
state.Nodes = append(state.Nodes, n.Name)
_, has := kept[broker.Principal{Kind: broker.KindNode, Node: n.Name}.Username()]
state.Credentialled[n.Name] = has
assigned, err := inv.Assigned(ctx, n.Name)
if err != nil {
return state, err
}
for _, module := range assigned {
m, known := shelf[module]
if !known {
continue
}
// The machine that holds the bus seat is the one that would be sent the user list.
if m.BusUsers != "" && m.ClaimsSeat("mesh-broker") {
state.Holder = n.Name
state.AccountsComposed = wasSentTheUserList(ctx, inv, n.Name)
}
// A module that never speaks needs no credential, so it is not counted as missing one.
if !speaksOnTheBus(m) {
continue
}
named := n.Name + "/" + module
state.Modules = append(state.Modules, named)
_, hasOne := kept[broker.Principal{
Kind: broker.KindModule, Node: n.Name, Module: module,
}.Username()]
state.ModuleCredentialled[named] = hasOne
}
}
return state, nil
}
// speaksOnTheBus says whether a module reaches the bus at all.
//
// A third of the catalogue never does (novox/hq ADR 0120), and counting those as missing a credential
// would bury the ones that matter under a list nobody can act on.
func speaksOnTheBus(m catalogue.Manifest) bool {
return len(m.Emits) > 0 || len(m.Consumes) > 0 || len(m.Tools) > 0 ||
len(m.DefinesSeats) > 0 || len(m.Uses) > 0 || len(m.Claims) > 0
}
// wasSentTheUserList says whether the machine holding the bus has had a declaration since the user
// list became part of one.
//
// Read from what the mesh recorded sending rather than asked of the machine: a machine that is away
// has still been sent it, and this question is about whether the mesh did its part.
func wasSentTheUserList(ctx context.Context, inv *inventory.Inventory, node string) bool {
digest, err := inv.Outstanding(ctx, node)
return err == nil && strings.TrimSpace(digest) != ""
}
// notReadyOf is the readiness reasoning, named here so a test can reach it without the command's
// printing. The reasoning itself is the broker package's, where it is pure.
func notReadyOf(state broker.Readiness) []string { return broker.NotReady(state) }
// rolloutMint gives every principal the new bus will have a credential it does not yet have, and
// puts each where its owner reads it (novox/hq design 28, task 5.2): a machine's as a membership
// sealed into its declaration, a module's as its broker secret, the control plane's own as its
// `bus` secret. Idempotent: what already has a hash is left alone, so running it again is harmless.
//
// **Before anything moves, and it is what makes moving possible.** A machine moved without a
// credential cannot come back, and afterwards there is no bus to tell it anything over — which is
// why `rollout check` refuses until this has run. The bus's address is worked out here, from where
// the module that provides it is assigned, rather than read from this process's environment: this
// process is still on the old bus when this runs, and must be.
func rolloutMint(ctx context.Context, again bool) error {
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
inv := open.inventory
known, err := broker.FromEnvironment()
if err != nil {
return fmt.Errorf("the bus's certificate is not known to this process, and every membership "+
"must carry its fingerprint: %w", err)
}
shelf, err := inv.Catalogue(ctx)
if err != nil {
return err
}
entries, err := inv.Catalogued(ctx)
if err != nil {
return err
}
var busNode, controllerNode string
for _, e := range entries {
switch {
case e.Manifest.ClaimsSeat("mesh-broker") && providesBus(e.Manifest) && len(e.On) > 0:
busNode = e.On[0]
case e.Manifest.Module == "mesh-controller" && len(e.On) > 0:
controllerNode = e.On[0]
}
}
if busNode == "" {
return errors.New("no assigned module provides mesh-bus and claims mesh-broker, so there is no " +
"bus to mint credentials for — register and assign it first")
}
onNetwork, err := whereEveryoneIs(ctx, inv, shelf)
if err != nil {
return err
}
busHost := onNetwork[busNode]
if busHost == "" {
// **The hub is not in that map.** The machine that took over the tunnel is where the current
// bus already answers, and every machine dials it at the address the mesh handed them — so
// when the new bus runs on the same machine, that address is the one to tell them, with the
// new port. Found live: the control node is the hub, and the map lists the machines placed
// around it.
// The host alone: no scheme (BareAddress adds one where none was, which is the wrong
// direction here — every URL built below adds its own) and no port.
_, _, host := broker.CredentialIn(known.Address)
if host == "" {
host = known.Address
}
if _, after, hasScheme := strings.Cut(host, "://"); hasScheme {
host = after
}
host = strings.TrimSpace(host)
if i := strings.LastIndex(host, ":"); i > 0 && !strings.Contains(host[i:], "]") {
host = host[:i]
}
if host == "" {
return fmt.Errorf("%s runs the new bus and has no address on the private network, and the "+
"current bus's address is unknown too, so no machine could be told where it is", busNode)
}
busHost = host
}
busAddress := busHost + ":4222"
records, err := inv.BusRecords(ctx)
if err != nil {
return err
}
users, err := broker.Users(records)
if err != nil {
return err
}
kept, err := inv.BusUsers(ctx)
if err != nil {
return err
}
hashes := make(map[string]string, len(kept))
for name, u := range kept {
hashes[name] = u.PasswordHash
}
_, missing := broker.WithPasswords(users, hashes)
wanted := map[string]bool{}
for _, m := range missing {
wanted[m] = true
}
var machines, modules, skipped int
for _, p := range users {
if !again && !wanted[p.Username()] {
continue
}
switch p.Kind {
case broker.KindController:
if controllerNode == "" {
return errors.New("the control plane is not assigned anywhere, so its credential has nowhere to go")
}
password, err := inv.MintBusPassword(ctx, inventory.BusUser{Username: p.Username(), Kind: inventory.BusController})
if err != nil {
return err
}
url := "nats://" + p.Username() + ":" + password + "@" + busAddress
if err := inv.AcceptSecretForModule(ctx, controllerNode, "mesh-controller", "bus", url); err != nil {
return fmt.Errorf("the control plane's credential is minted and could not be sealed to %s: %w", controllerNode, err)
}
fmt.Printf("control plane: credential minted, sealed to %s as its `bus` secret\n", controllerNode)
case broker.KindNode:
password, err := inv.MintBusPassword(ctx, inventory.BusUser{Username: p.Username(), Kind: inventory.BusNode, Node: p.Node})
if err != nil {
return err
}
membership, _ := json.Marshal(map[string]string{
"broker": busAddress, "fingerprint": known.Fingerprint, "password": password, "transport": "nats",
})
key, err := inv.SealingKeyOf(ctx, p.Node)
if err != nil {
return fmt.Errorf("%s has no sealing key, so its membership cannot be sealed to it: %w", p.Node, err)
}
sealed, err := secrets.Seal(key, membership)
if err != nil {
return err
}
if err := inv.PutBusMembership(ctx, p.Node, sealed); err != nil {
return err
}
machines++
case broker.KindModule:
if p.Module == "mesh-controller" {
// The control plane is a module too, and its `broker` secret is the old bus's
// credential it is still using while this runs. Writing the new bus's blob there
// cut the mesh off from its own old bus mid-move (2026-09-28). Its new-bus credential
// is the controller principal's `bus` secret above; nothing else is needed here.
skipped++
continue
}
m, inShelf := shelf[p.Module]
if !inShelf {
skipped++
continue
}
if _, reads := m.OwnSecrets["broker"]; !reads {
fmt.Printf(" %s on %s speaks on the bus but declares no `broker` secret to receive a credential in; skipped\n", p.Module, p.Node)
skipped++
continue
}
password, err := inv.MintBusPassword(ctx, inventory.BusUser{Username: p.Username(), Kind: inventory.BusModule, Node: p.Node, Module: p.Module})
if err != nil {
return err
}
if err := issueWith(ctx, inv, m, p.Node, "", known, busAddress, p.Username(), password); err != nil {
return err
}
modules++
default:
skipped++
}
}
fmt.Printf("minted for %d machine(s) and %d module runtime(s); %d skipped; the bus is at %s\n",
machines, modules, skipped, busAddress)
fmt.Println(" each machine's membership and each module's credential arrive with the next push of its machine;")
fmt.Println(" push the machine running the bus first, so the bus stands with its user list before anything dials it")
return nil
}
// providesBus is whether a manifest provides the mesh's bus.
func providesBus(m catalogue.Manifest) bool {
for _, o := range m.Provides {
if o.Name == "mesh-bus" {
return true
}
}
return false
}
// rolloutHand mints a machine its credential for the new bus afresh and prints its membership
// once, for an operator to carry by hand — the rescue for a machine that cannot be reached over
// any bus: rotated while it still held the old password, or reachable only by ssh. The plaintext
// exists on this terminal and then only where it is written; the store keeps the hash, and the
// sealed copy in the machine's declaration is replaced too, so the next push says the same.
func rolloutHand(ctx context.Context, node string) error {
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
inv := open.inventory
known, err := broker.FromEnvironment()
if err != nil {
return fmt.Errorf("the bus's certificate is not known to this process: %w", err)
}
busAddress, _, err := broker.OnNATS()
if err != nil {
return err
}
if busAddress == "" {
return errors.New("this control plane is not on the new bus, so there is no membership to hand out")
}
_, _, bare := broker.CredentialIn(busAddress)
if _, after, has := strings.Cut(bare, "://"); has {
bare = after
}
if _, err := inv.NodeByName(ctx, node); err != nil {
return err
}
p := broker.Principal{Kind: broker.KindNode, Node: node}
password, err := inv.MintBusPassword(ctx, inventory.BusUser{Username: p.Username(), Kind: inventory.BusNode, Node: node})
if err != nil {
return err
}
membership, _ := json.Marshal(map[string]string{
"broker": bare, "fingerprint": known.Fingerprint, "password": password, "transport": "nats",
})
key, err := inv.SealingKeyOf(ctx, node)
if err != nil {
return err
}
sealed, err := secrets.Seal(key, membership)
if err != nil {
return err
}
if err := inv.PutBusMembership(ctx, node, sealed); err != nil {
return err
}
// The one line of output is the membership itself, so it can be piped to the machine without
// being read on the way. Everything else goes to stderr.
fmt.Fprintf(os.Stderr, "%s's credential is minted afresh. Write this to %s on it and restart its host; "+
"then push the machine running the bus so the user list carries the new hash.\n",
node, catalogue.BusMembershipPath)
fmt.Println(string(membership))
return nil
}
+93
View File
@@ -0,0 +1,93 @@
package main
import (
"context"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
// Whether a mesh could move its bus, read from a real store.
//
// The readiness reasoning has its own tests; this is about the gathering — that the question is
// answered from what the mesh actually holds, on a store with machines and modules in it, without
// writing anything.
func TestReadinessIsGatheredFromWhatTheMeshHolds(t *testing.T) {
inv := inventory.ForTest(t)
ctx := context.Background()
// A mesh mid-change: two machines, the bus module on one of them, a module that speaks and a
// module that never does.
for _, m := range []catalogue.Manifest{
{Module: "nats", Version: "1", BusUsers: "/var/lib/nats-module/conf/accounts.conf",
Claims: []catalogue.Claim{{Name: "mesh-broker", Scope: catalogue.ScopeMesh}}},
{Module: "gitea", Version: "1", Tools: []string{"repo_create"}},
{Module: "wallpaper", Version: "1"},
} {
if err := inv.RegisterModule(ctx, m, inventory.Source{Repository: "/r"}); err != nil {
t.Fatal(err)
}
}
for _, n := range []string{"anchor", "laptop"} {
if _, err := inv.AddNode(ctx, n); err != nil {
t.Fatal(err)
}
}
for _, a := range [][2]string{{"anchor", "nats"}, {"anchor", "gitea"}, {"laptop", "wallpaper"}} {
if _, err := inv.Assign(ctx, a[0], a[1]); err != nil {
t.Fatal(err)
}
}
state, err := readinessOf(ctx, inv)
if err != nil {
t.Fatal(err)
}
if state.Holder != "anchor" {
t.Errorf("the machine holding the bus reads as %q", state.Holder)
}
if len(state.Nodes) != 2 {
t.Errorf("machines read as %v", state.Nodes)
}
// **A module that never speaks is not counted as missing a credential.** A third of the catalogue
// never reaches the bus, and listing those would bury the ones that matter.
for _, m := range state.Modules {
if strings.HasSuffix(m, "/wallpaper") {
t.Errorf("a module that never speaks was counted: %v", state.Modules)
}
}
if len(state.Modules) != 2 {
t.Errorf("modules that speak read as %v; expected the bus module and the one with a tool",
state.Modules)
}
// Nothing has been minted, so it is not ready — and it says so about each machine by name.
why := notReadyOf(state)
if len(why) == 0 {
t.Fatal("a mesh where nothing has a credential was reported ready to move")
}
said := strings.Join(why, "\n")
for _, name := range []string{"anchor", "laptop"} {
if !strings.Contains(said, name) {
t.Errorf("the refusal does not name %s: %s", name, said)
}
}
// Mint for one machine and it drops out of the complaint, which is how somebody works through it.
if _, err := inv.MintBusPassword(ctx, inventory.BusUser{
Username: "node.laptop", Kind: inventory.BusNode, Node: "laptop",
}); err != nil {
t.Fatal(err)
}
state, err = readinessOf(ctx, inv)
if err != nil {
t.Fatal(err)
}
if !state.Credentialled["laptop"] {
t.Error("a machine that was minted a credential still reads as having none")
}
}
+121 -4
View File
@@ -6,6 +6,7 @@ import (
"flag"
"fmt"
"os"
"slices"
"sort"
"strings"
"text/tabwriter"
@@ -43,15 +44,16 @@ type seatRow struct {
// overview would make the one thing the overview is for — what does this mesh have — quietly
// incomplete.
func seatsHeld(seats []catalogue.Seat, held []catalogue.Held) ([]seatRow, []catalogue.Held) {
defined := map[string]bool{}
rows := make([]seatRow, 0, len(seats))
for _, s := range seats {
defined[s.Name] = true
row := seatRow{Seat: s.Name, Scope: s.Scope, Delivers: s.Delivers, Decision: s.Decision,
Holders: []seatHolder{}}
seen := map[seatHolder]bool{}
for _, h := range held {
if h.Claim != s.Name || h.Scope != s.Scope {
// Resolve the held claim to a seat rather than comparing names, so a record naming a
// seat's former name groups under it after a rename (novox/hq ADR 0122).
hs, ok := catalogue.SeatNamed(h.Claim)
if !ok || hs.Name != s.Name || h.Scope != s.Scope {
continue
}
holder := seatHolder{Node: h.Node, Module: h.Module}
@@ -70,7 +72,8 @@ func seatsHeld(seats []catalogue.Seat, held []catalogue.Held) ([]seatRow, []cata
}
var outside []catalogue.Held
for _, h := range held {
if !defined[h.Claim] {
// Outside the set only if it resolves to no seat at all — a former name still resolves.
if _, ok := catalogue.SeatNamed(h.Claim); !ok {
outside = append(outside, h)
}
}
@@ -83,6 +86,120 @@ func seatsHeld(seats []catalogue.Seat, held []catalogue.Held) ([]seatRow, []cata
return rows, outside
}
// seatCommand changes the set — the whole point of it being data (novox/hq ADR 0122) — and, since
// ADR 0131, changes who holds a seat.
func seatCommand(ctx context.Context, args []string) error {
if len(args) == 3 && args[0] == "rename" {
from, to := args[1], args[2]
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
if err := open.inventory.RenameSeat(ctx, from, to); err != nil {
return err
}
fmt.Printf("%s is now %s — its former name still resolves, so nothing is rebuilt, "+
"re-registered or frozen (novox/hq ADR 0122)\n", from, to)
return nil
}
if len(args) == 3 && args[1] == "--to" {
return handOver(ctx, args[0], args[2])
}
return fmt.Errorf("seat rename <from> <to> | seat <name> --to <node>/<module>")
}
// handOver makes one assignment the holder of a seat, as one act, so the seat is never without a
// holder in between (novox/hq ADR 0131, design 28 task 5.3). The control plane finds its own bus
// through one of these seats; the day it was left empty mid-change is why this exists.
//
// Everything that could make the new holder wrong is refused here, before the row is written: the
// seat must exist, the assignment must exist, and the module must be able to hold the seat —
// claim it at its scope and provide what it delivers, judged against the store's row. What is
// **not** checked is whether the module is running yet: that is what `push` confirms afterwards,
// and refusing to record a handover to a module the node has not started would make the handover
// impossible to do before the switch instead of as the switch.
func handOver(ctx context.Context, seatName, to string) error {
nodeName, module, ok := strings.Cut(to, "/")
if !ok || nodeName == "" || module == "" {
return fmt.Errorf("the new holder is named <node>/<module>, not %q", to)
}
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
inv := open.inventory
seat, known := catalogue.SeatNamed(seatName)
if !known {
return fmt.Errorf("%q is not a seat this mesh defines — `seats` lists them", seatName)
}
assigned, err := inv.Assigned(ctx, nodeName)
if err != nil {
return err
}
if !slices.Contains(assigned, module) {
return fmt.Errorf("%s is not assigned to %s, so it cannot hold anything there — "+
"`assign %s %s` first", module, nodeName, nodeName, module)
}
entries, err := inv.Catalogued(ctx)
if err != nil {
return err
}
var m *catalogue.Manifest
for i := range entries {
if entries[i].Manifest.Module == module {
m = &entries[i].Manifest
}
}
if m == nil {
return fmt.Errorf("%s is assigned but not in the catalogue, which should not happen", module)
}
var was string
holdings, err := inv.Holdings(ctx)
if err != nil {
return err
}
for _, h := range holdings {
if hs, ok := catalogue.SeatNamed(h.Claim); ok && hs.Name == seat.Name {
was = h.Node
}
}
// **Recording who already holds the seat is not making a new holder, and is not judged like
// one.** On a mesh that predates the record, the first handover has to begin by writing down
// the standing holder — otherwise the next holder cannot be assigned beside it, because two
// eligible claimants with nothing on record are refused. That standing holder may no longer
// satisfy what the seat delivers (the row moved under it, on purpose, as ADR 0131's first step),
// and it holds regardless: derivation never read that column. So when nothing is on record and
// the named assignment is the one holding by derivation, only the claim itself is checked here.
// Every *change* of holder is judged in full.
claimsIt := false
for _, c := range m.Claims {
if cs, ok := catalogue.SeatNamed(c.Name); ok && cs.Name == seat.Name && c.At() == seat.Scope {
claimsIt = true
}
}
if was == "" && claimsIt {
fmt.Printf("nothing was on record for %s; recording %s on %s as its standing holder\n",
seat.Name, module, nodeName)
} else if err := catalogue.CanHold(*m, seat); err != nil {
return fmt.Errorf("%s cannot hold %s: %w", module, seat.Name, err)
}
if err := inv.HoldSeat(ctx, seat.Name, seat.Scope, nodeName, module); err != nil {
return err
}
fmt.Printf("%s is held by %s on %s\n", seat.Name, module, nodeName)
if was != "" && was != nodeName {
fmt.Printf(" `push %s` and `push %s` send both machines what changed\n", was, nodeName)
} else {
fmt.Printf(" `push %s` sends the machine what changed; every other machine that reads the "+
"seat is re-declared by `push --behind`\n", nodeName)
}
return nil
}
func seatsCommand(ctx context.Context, args []string) error {
set := flag.NewFlagSet("seats", flag.ContinueOnError)
asJSON := set.Bool("json", false, "the same, as JSON")
+4 -4
View File
@@ -35,13 +35,13 @@ func TestEverySeatIsListedIncludingTheOnesNobodyHolds(t *testing.T) {
func TestANodeSeatListsEveryMachineHoldingIt(t *testing.T) {
rows, _ := seatsHeld(catalogue.Seats(), []catalogue.Held{
{Claim: "the-packet-filter", Scope: catalogue.ScopeNode, Node: "node2", Module: "nftables"},
{Claim: "the-packet-filter", Scope: catalogue.ScopeNode, Node: "anchor", Module: "nftables"},
{Claim: "node-packet-filter", Scope: catalogue.ScopeNode, Node: "node2", Module: "nftables"},
{Claim: "node-packet-filter", Scope: catalogue.ScopeNode, Node: "anchor", Module: "nftables"},
// Resolved twice, reported once: a machine is one holder however many passes saw it.
{Claim: "the-packet-filter", Scope: catalogue.ScopeNode, Node: "anchor", Module: "nftables"},
{Claim: "node-packet-filter", Scope: catalogue.ScopeNode, Node: "anchor", Module: "nftables"},
})
for _, r := range rows {
if r.Seat != "the-packet-filter" {
if r.Seat != "node-packet-filter" {
continue
}
if len(r.Holders) != 2 || r.Holders[0].Node != "anchor" || r.Holders[1].Node != "node2" {
+6
View File
@@ -38,6 +38,12 @@ func (s sendable) Body() ([]byte, error) {
if s.Adoption != nil {
envelope["adoption"] = s.Adoption
}
// An empty declaration is deliberate here — the node owns nothing the mesh put there
// (novox/hq issue 127) — and the host refuses an empty body unless it is told the emptiness
// is meant, so a truncated or mis-composed body is never mistaken for "own nothing".
if len(s.Resources) == 0 {
envelope["owns_nothing"] = true
}
return json.Marshal(envelope)
}
+22
View File
@@ -355,3 +355,25 @@ func TestTheMachineSideOfAMappingIsMovedEverywhereTheNumberIsUsed(t *testing.T)
t.Fatalf("the consumer is told the forge answers on %v", told)
}
}
func TestAnEmptyDeclarationSaysOwnsNothing(t *testing.T) {
// The host refuses an empty body unless told the emptiness is meant (novox/hq issue 127).
body, err := sendable{}.Body()
if err != nil {
t.Fatal(err)
}
var env map[string]any
if err := json.Unmarshal(body, &env); err != nil {
t.Fatal(err)
}
if env["owns_nothing"] != true {
t.Fatalf("an empty declaration must mark owns_nothing; got %v", env)
}
// A declaration with resources does not carry the marker.
body, _ = sendable{Resources: []map[string]any{{"id": "x"}}}.Body()
var env2 map[string]any
_ = json.Unmarshal(body, &env2)
if _, present := env2["owns_nothing"]; present {
t.Fatalf("a non-empty declaration must not mark owns_nothing; got %v", env)
}
}
+21
View File
@@ -5,6 +5,7 @@ import (
"fmt"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/identity"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/licences"
@@ -72,6 +73,14 @@ func migrate(ctx context.Context) error {
}
fmt.Printf("provided %s\n", m.Module)
}
// The seats the mesh ships with, into the table that now holds the set (novox/hq ADR 0122).
// Idempotent: fills an empty table on first boot, adds a seat a release ships, and leaves an
// operator's changes in the table as they are.
added, err := inv.SeedSeats(ctx, catalogue.DefaultSeats())
if err != nil {
return err
}
fmt.Printf("seeded %d seat(s)\n", added)
return nil
}
@@ -85,6 +94,18 @@ func openInventory(ctx context.Context) (*inventory.Inventory, error) {
inv.Close()
return nil, err
}
// Load the seat set from the store, so the control plane reads the set as data rather than as
// the slice it was compiled with (novox/hq ADR 0122). A store not yet seeded — or one whose
// seat table a migration has not reached — returns nothing, and UseSeats leaves the compiled
// defaults in force: the set is never emptied by a read that found nothing, which would refuse
// every claim. So this can only ever replace the defaults with what the mesh actually holds.
if seats, err := inv.Seats(ctx); err == nil {
catalogue.UseSeats(seats)
}
// And the former names, so a reference to a seat's old name resolves after a rename (ADR 0122).
if aliases, err := inv.Aliases(ctx); err == nil {
catalogue.UseAliases(aliases)
}
return inv, nil
}
+164
View File
@@ -6,7 +6,9 @@ import (
"flag"
"fmt"
"strings"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
@@ -218,3 +220,165 @@ func notNow(err error) error {
}
return err
}
// SourceMoved is the forge announcing a merge: every module recorded as built from that
// repository and branch is marked as moved to the merge commit, and built — bases first, so a
// module that stands on another's artifact is built after it and not against the old one
// (novox/hq 04-ISSUES/131). Nothing is pushed here: what a finished build does to the machines
// running the module is the upgrade's decision, taken when the catalogue announces it.
func (f following) SourceMoved(ctx context.Context, m link.SourceMoved) error {
inv := f.open.inventory
entries, err := inv.Catalogued(ctx)
if err != nil {
return notNow(err)
}
var moved []inventory.Entry
for _, e := range entries {
if !sourceIs(e.Source, m) {
continue
}
if e.Source.BuiltFrom == m.Commit {
continue
}
// **A merge older than the last look at the source is history, not a move.** The forge
// announces what it finds merged, and an old merge surfacing late would otherwise move the
// recorded head backwards and rebuild everything built from that repository, once per old
// merge (2026-09-28).
if isHistory(m.MergedAt, e.Source.Seen) {
continue
}
if err := inv.SourceMoved(ctx, e.Manifest.Module, m.Commit); err != nil {
return notNow(err)
}
moved = append(moved, e)
}
if len(moved) == 0 {
fmt.Printf("%s/%s merged into %s (%.8s); nothing the mesh holds is built from it\n",
m.Owner, m.Repo, m.Base, m.Commit)
return nil
}
against, err := inv.BuiltAgainst(ctx)
if err != nil {
return notNow(err)
}
ordered := orderByBases(moved, against)
names := make([]string, 0, len(ordered))
for _, e := range ordered {
names = append(names, e.Manifest.Module)
}
fmt.Printf("%s/%s merged into %s (%.8s); building %s\n",
m.Owner, m.Repo, m.Base, m.Commit, strings.Join(names, ", "))
var failed []string
for _, e := range ordered {
source := buildSource{Repository: e.Source.Repository, Seat: e.Source.Seat}
if err := buildOne(ctx, source, e.Source.Path, e.Source.Ref, 20*time.Minute); err != nil {
fmt.Printf(" %s: %v\n", e.Manifest.Module, err)
failed = append(failed, e.Manifest.Module)
// A base that failed is a reason to stop: what stands on it would be built against
// the old one, and report success (novox/hq 04-ISSUES/131).
if standsOn(ordered, e.Manifest.Module, against) {
fmt.Printf(" stopping: %s is a base of what was still to build\n", e.Manifest.Module)
break
}
}
}
if len(failed) > 0 {
fmt.Printf("%d of %d not built: %s\n", len(failed), len(ordered), strings.Join(failed, ", "))
}
return nil
}
// sourceIs is whether a recorded source is the repository and branch a merge announced. A source on
// the git seat is recorded as its path on the forge; one elsewhere as the URL it was cloned from.
// An empty recorded ref is the repository's default branch, which is what a merge into the base
// branch of the forge's default means.
func sourceIs(s inventory.Source, m link.SourceMoved) bool {
want := strings.ToLower(m.Owner + "/" + m.Repo)
repo := strings.ToLower(strings.TrimSuffix(s.Repository, ".git"))
matches := repo == want || strings.HasSuffix(repo, "/"+want) ||
(m.CloneURL != "" && strings.EqualFold(strings.TrimSuffix(s.Repository, ".git"), strings.TrimSuffix(m.CloneURL, ".git")))
if !matches {
return false
}
return s.Ref == "" || s.Ref == m.Base
}
// orderByBases is the entries with every base before what stands on it: a module whose build stood
// on another's artifact comes after that module. Entries outside the set are not waited for — they
// are not being rebuilt. Stable for what has no order between it.
//
// `against` is what each module's newest build stood on (inventory.BuiltAgainst): the edges are
// derived from builds, not declared, because a recorded manifest no longer carries `build.on`.
func orderByBases(entries []inventory.Entry, against map[string][]string) []inventory.Entry {
inSet := map[string]bool{}
for _, e := range entries {
inSet[e.Manifest.Module] = true
}
var out []inventory.Entry
placed := map[string]bool{}
var place func(e inventory.Entry, seen map[string]bool)
place = func(e inventory.Entry, seen map[string]bool) {
name := e.Manifest.Module
if placed[name] || seen[name] {
return
}
seen[name] = true
for _, base := range entries {
if base.Manifest.Module != name && inSet[base.Manifest.Module] && standsOnModule(e, base.Manifest.Module, against) {
place(base, seen)
}
}
placed[name] = true
out = append(out, e)
}
for _, e := range entries {
place(e, map[string]bool{})
}
return out
}
// standsOn is whether anything in the set is built on the named module's artifacts.
func standsOn(entries []inventory.Entry, module string, against map[string][]string) bool {
for _, e := range entries {
if standsOnModule(e, module, against) {
return true
}
}
return false
}
// standsOnModule is whether an entry's build stood on the named module: by what its newest build
// recorded it was handed (`artifact-store://<module>/<artifact>@…`, the module's own artifact), or
// — for a module registered from a manifest and not yet built — by the base its manifest names.
func standsOnModule(e inventory.Entry, module string, against map[string][]string) bool {
if e.Manifest.Module == module {
return false
}
if e.Manifest.Build != nil {
for _, on := range e.Manifest.Build.On {
if on.Module == module {
return true
}
}
}
prefix := catalogue.ArtifactStoreScheme + module + "/"
for _, ref := range against[e.Manifest.Module] {
if strings.HasPrefix(ref, prefix) {
return true
}
}
return false
}
// isHistory is whether a merge made at mergedAt predates the last time the source was seen. A merge
// with no time on it is taken as news: refusing it would silence a forge that says less.
func isHistory(mergedAt string, seen time.Time) bool {
if mergedAt == "" || seen.IsZero() {
return false
}
at, err := time.Parse(time.RFC3339, mergedAt)
if err != nil {
return false
}
return at.Before(seen)
}
+10 -7
View File
@@ -1,19 +1,22 @@
module github.com/novox/mesh-controller
go 1.25.0
go 1.26.0
require (
github.com/jackc/pgx/v5 v5.10.0
github.com/rabbitmq/amqp091-go v1.14.0
golang.org/x/crypto v0.55.0
github.com/nats-io/nats.go v1.54.0
golang.org/x/crypto v0.57.0
)
require (
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
golang.org/x/net v0.57.0 // indirect
golang.org/x/sync v0.22.0 // indirect
golang.org/x/sys v0.47.0 // indirect
golang.org/x/text v0.41.0 // indirect
github.com/klauspost/compress v1.20.0 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
golang.org/x/net v0.58.0 // indirect
golang.org/x/sync v0.23.0 // indirect
golang.org/x/sys v0.48.0 // indirect
golang.org/x/text v0.42.0 // indirect
)
+18 -14
View File
@@ -9,27 +9,31 @@ github.com/jackc/pgx/v5 v5.10.0 h1:VhSvgU2jSli8o3AqIEOTJr7rZwAEUVo4E4XhR94Zfr0=
github.com/jackc/pgx/v5 v5.10.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
github.com/nats-io/nats.go v1.54.0/go.mod h1:y+DZoD1oBOYfZTU681eTUiUjI0vbqYGixNVFHcjHJ0k=
github.com/nats-io/nkeys v0.4.16 h1:rd5oAuLOb8mnAycB0xleuEBNS1pVVnN0fv/FF34Eypg=
github.com/nats-io/nkeys v0.4.16/go.mod h1:llLgWoI0o4z/Q57q2R1kHfmocyhGV6VG/U18Glg1Afs=
github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw=
github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/rabbitmq/amqp091-go v1.14.0 h1:RSaT7aOKt/OrkVUyswPDW29lnRz9psuGmfZFBmLqLek=
github.com/rabbitmq/amqp091-go v1.14.0/go.mod h1:Hy4jKW5kQART1u+JkDTF9YYOQUHXqMuhrgxOEeS7G4o=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
golang.org/x/crypto v0.55.0 h1:+KWHjbgOaAQ66dh/YlkZKHlz9ZUlq61AFirAR9ntP8M=
golang.org/x/crypto v0.55.0/go.mod h1:uq0V9dE/fzQuJtbnL+2EhWOE63vo164FY8xqEnV9xis=
golang.org/x/net v0.57.0 h1:K5+3DljvIuDG9/Jv9rvyMywYNFCQ9RSUY6OOTTkT+tE=
golang.org/x/net v0.57.0/go.mod h1:KpXc8iv+r3XplLAG/f7Jsf9RPszJzdR0f58q9vGOuEU=
golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek=
golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/text v0.41.0 h1:vz/seA0lnX87Othu2f/0L24RcgrXD9/YFTSuGjj3rH8=
golang.org/x/text v0.41.0/go.mod h1:jvf1O8ajNzZqhSrQBPbutR/EB83Cc0CFrezNQIwbb5M=
golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
golang.org/x/net v0.58.0 h1:ynWG7rqYi4ccpTEuPZ2QGWHktVEM9DMCj9yzDE0Q7To=
golang.org/x/net v0.58.0/go.mod h1:YwCddHnFlT7eLQqVprV19OnhLGtc5xOKgE0RyqgfWAU=
golang.org/x/sync v0.23.0 h1:KameEIfc1IkluZyXWLn39Wd4tURc6GbCiISGiZm2bQk=
golang.org/x/sync v0.23.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0=
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/text v0.42.0 h1:JbOZXgfeCPU9gacVtYliJqOhD+zhrEqK4LfdpmlUZqI=
golang.org/x/text v0.42.0/go.mod h1:ojzP1Z+2QtioaF8DTtO8K5q7JWVVYwZKenzujK0Zd0E=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
+125
View File
@@ -0,0 +1,125 @@
package broker
import (
"fmt"
"sort"
"strings"
)
// Do the emitters and the consumers of a catalogue agree?
//
// **The check that was missing** (novox/hq 04-ISSUES/127). Every manifest was individually
// well-formed and every derivation individually correct, and no cross-module subscription in the
// mesh matched anything: a consumer's declaration derived into a namespace nobody publishes to.
// Nothing failed, because a subscription that matches nothing is not an error — it is silence.
//
// The comparison has to be over the whole catalogue, because the two halves live in different
// manifests, and it cannot simply demand that every consumed event have a live emitter: a module
// may be installed long before the one whose events it wants. So the rule is narrower and still
// catches this: **where the emitter is present, it must emit what the consumer asked for.**
// AConsumer is one module's interest in another's events, as this check needs it.
type AConsumer struct {
Module string
Consumes []string
}
// AnEmitter is one module's events.
type AnEmitter struct {
Module string
Emits []string
}
// Disagreements are the consumed events whose emitter is in the catalogue and does not emit them.
//
// Returned as sentences rather than as structs: every one of them is read by a person deciding
// whether a manifest or a catalogue is wrong, and a pair of names without the reason is a puzzle.
func Disagreements(emitters []AnEmitter, consumers []AConsumer, seats []DeclaredSeat) []string {
emits := map[string]map[string]bool{}
for _, e := range emitters {
if emits[e.Module] == nil {
emits[e.Module] = map[string]bool{}
}
for _, name := range e.Emits {
emits[e.Module][name] = true
}
}
// A seat's events are published by its holder under the seat's name, so a consumer naming the
// seat is naming something real even though no module declares it as its own.
for _, s := range seats {
if len(s.Emits) == 0 {
continue
}
if emits[s.Name] == nil {
emits[s.Name] = map[string]bool{}
}
for _, name := range s.Emits {
emits[s.Name][name] = true
}
}
var out []string
for _, c := range consumers {
for _, pattern := range c.Consumes {
emitter, event, named := strings.Cut(pattern, ".")
// Every event from everyone, or every event from one module: both are deliberate and
// neither names a particular event to check.
if !named || emitter == "*" || emitter == catalogueTheRest || event == catalogueTheRest {
continue
}
known, present := emits[emitter]
if !present {
// Not installed here, which is ordinary: a module lives in its own repository and
// may be registered later. Nothing to compare, so nothing to say.
continue
}
if matchesAny(event, known) {
continue
}
out = append(out, fmt.Sprintf(
"%s consumes %q and %s emits %s — so that subscription would match nothing, and "+
"nothing would report it",
c.Module, pattern, emitter, listOf(known)))
}
}
sort.Strings(out)
return out
}
// matchesAny says whether one of an emitter's event names satisfies a consumer's pattern.
func matchesAny(pattern string, emitted map[string]bool) bool {
want := strings.Split(pattern, ".")
for name := range emitted {
if matches(want, strings.Split(name, ".")) {
return true
}
}
return false
}
func matches(pattern, name []string) bool {
for i, part := range pattern {
if part == catalogueTheRest {
return i < len(name)
}
if i >= len(name) {
return false
}
if part != "*" && part != name[i] {
return false
}
}
return len(pattern) == len(name)
}
func listOf(names map[string]bool) string {
if len(names) == 0 {
return "nothing"
}
out := make([]string, 0, len(names))
for n := range names {
out = append(out, n)
}
sort.Strings(out)
return strings.Join(out, ", ")
}
+236
View File
@@ -0,0 +1,236 @@
package broker
import (
"encoding/json"
"os"
"path/filepath"
"sort"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// **Do the catalogue's emitters and consumers agree?**
//
// This is the check whose absence let issue 127 stand: every manifest was individually well-formed,
// every derivation individually correct, and no cross-module subscription in the mesh matched
// anything. A subscription that matches nothing is not an error — it is silence — so nothing
// anywhere reported it.
//
// It compares what one manifest asks to hear against what another says it emits. It cannot demand
// that every consumed event have a live emitter, because a module lives in its own repository and
// may be registered long before the one whose events it wants. Where the emitter *is* here, it must
// emit what the consumer asked for.
func TestTheCataloguesEmittersAndConsumersAgree(t *testing.T) {
emitters, consumers, seats := theCataloguesEvents(t)
if bad := Disagreements(emitters, consumers, seats); len(bad) > 0 {
t.Fatalf("%d subscription(s) in the catalogue would match nothing:\n %s",
len(bad), strings.Join(bad, "\n "))
}
}
// And the check itself catches the thing it exists for, so it cannot pass by doing nothing.
func TestTheAgreementCheckCatchesASubscriptionThatMatchesNothing(t *testing.T) {
bad := Disagreements(
[]AnEmitter{{Module: "builder", Emits: []string{"built"}}},
[]AConsumer{{Module: "mesh-catalog", Consumes: []string{"builder.finished"}}},
nil)
if len(bad) != 1 {
t.Fatalf("a consumer asking for an event its emitter does not emit was not caught: %v", bad)
}
if !strings.Contains(bad[0], "builder.finished") || !strings.Contains(bad[0], "built") {
t.Fatalf("the report names neither what was asked for nor what is emitted: %s", bad[0])
}
// A module that is not here is not a disagreement: it may be registered later.
if bad := Disagreements(nil,
[]AConsumer{{Module: "plex", Consumes: []string{"sonarr.download.completed"}}}, nil); len(bad) != 0 {
t.Fatalf("a consumer whose emitter is not installed was reported: %v", bad)
}
// A wildcard over emitters is deliberate and names no particular event to check.
if bad := Disagreements([]AnEmitter{{Module: "sonarr", Emits: []string{"download.completed"}}},
[]AConsumer{{Module: "plex", Consumes: []string{"*.download.completed"}}}, nil); len(bad) != 0 {
t.Fatalf("a wildcard over emitters was reported: %v", bad)
}
// A consumer of a role's event whose role does not emit it is caught, which is what stops the
// catalogue check above from passing by knowing nothing about roles.
if bad := Disagreements(nil,
[]AConsumer{{Module: "mesh-catalog", Consumes: []string{"mesh-build-machine.finished"}}},
[]DeclaredSeat{{Name: "mesh-build-machine", Emits: []string{"built"}}}); len(bad) != 1 {
t.Fatalf("a consumer of a role event the role does not emit was not caught: %v", bad)
}
// An event published under a seat's name is real even though no module declares it as its own.
if bad := Disagreements(nil,
[]AConsumer{{Module: "watcher", Consumes: []string{"mesh-artifact-store.image.pushed"}}},
[]DeclaredSeat{{Name: "the-artifact-store", Emits: []string{"image.pushed"}}}); len(bad) != 0 {
t.Fatalf("an event a seat emits was reported as matching nothing: %v", bad)
}
}
func theCataloguesEvents(t *testing.T) ([]AnEmitter, []AConsumer, []DeclaredSeat) {
t.Helper()
root := filepath.Join("..", "..", "..", "mesh-catalog", "modules")
entries, err := os.ReadDir(root)
if err != nil {
t.Skipf("catalogue sibling not present: %v", err)
}
var emitters []AnEmitter
var consumers []AConsumer
// The mesh's own roles, which emit under the seat's name rather than any module's (novox/hq
// ADR 0121). Without these the check skips every consumer of a role's event as "the emitter is
// not installed" — which is how it passed vacuously the first time one existed.
var seats []DeclaredSeat
for _, own := range catalogue.SeatsWithAProtocol() {
seats = append(seats, DeclaredSeat{Name: own.Name, Accepts: own.Accepts, Emits: own.Emits})
}
for _, e := range entries {
if !e.IsDir() {
continue
}
raw, err := os.ReadFile(filepath.Join(root, e.Name(), "module.json"))
if err != nil {
continue
}
var m struct {
Module string `json:"module"`
Emits []string `json:"emits"`
Consumes []string `json:"consumes"`
Seats []struct {
Name string `json:"name"`
Emits []string `json:"emits"`
} `json:"seats"`
}
if err := json.Unmarshal(raw, &m); err != nil {
t.Fatalf("%s: %v", e.Name(), err)
}
if len(m.Emits) > 0 {
emitters = append(emitters, AnEmitter{Module: m.Module, Emits: m.Emits})
}
if len(m.Consumes) > 0 {
consumers = append(consumers, AConsumer{Module: m.Module, Consumes: m.Consumes})
}
for _, s := range m.Seats {
seats = append(seats, DeclaredSeat{Name: s.Name, Emits: s.Emits})
}
}
if len(emitters) == 0 {
t.Skip("no manifests found beside this checkout")
}
return emitters, consumers, seats
}
// **Do the derived subjects meet, not just the names?**
//
// The check above compares what a consumer asks for against what an emitter says it emits, by name. It
// passed while the catalogue's subscription pointed at `mesh.mod.mesh-build-machine.event.built` — a
// module namespace for a role's event, which no emitter owns. The names agreed; the subjects did not,
// and the graph stayed empty.
//
// So this compares the thing that actually has to match: the subject a consumer subscribes against the
// subject an emitter publishes. It is the last place the two halves can be held together, because
// after this the server is the only thing that knows and it says nothing — a subscription that matches
// nothing is silence.
func TestTheCataloguesDerivedSubjectsMeet(t *testing.T) {
emitters, consumers, seats := theCataloguesEvents(t)
// Every subject something publishes: a module's own events, and the events of every role.
published := map[string]bool{}
for _, e := range emitters {
for _, name := range e.Emits {
published["mesh.mod."+e.Module+".event."+name] = true
}
}
for _, s := range seats {
for _, name := range s.Emits {
published["mesh.seat."+s.Name+".event."+name] = true
}
}
byName := map[string]DeclaredSeat{}
for _, s := range seats {
byName[s.Name] = s
}
var lonely []string
for _, c := range consumers {
principal := Principal{Kind: KindModule, Node: "one", Module: c.Module, PasswordHash: "x"}
for _, want := range c.Consumes {
emitter, event, named := strings.Cut(want, ".")
if named {
if s, isASeat := byName[emitter]; isASeat {
principal.Watches = append(principal.Watches,
Seat{Name: s.Name, Emits: []string{event}})
continue
}
}
principal.Consumes = append(principal.Consumes, want)
}
perms, err := PermissionsFor(principal)
if err != nil {
t.Fatalf("%s: %v", c.Module, err)
}
for _, subject := range perms.Subscribe {
if !strings.Contains(subject, ".event.") {
continue
}
if reaches(subject, published) {
continue
}
// A wildcard over emitters reaches whatever arrives later, and an emitter that is not
// installed is ordinary — both are already excused by the check above, so only a subject
// that can never match anything gets here.
if strings.Contains(subject, "*") || strings.Contains(subject, ">") {
continue
}
lonely = append(lonely, c.Module+" subscribes "+subject+", which nothing publishes")
}
}
if len(lonely) > 0 {
sort.Strings(lonely)
t.Fatalf("%d subscription(s) derive to a subject no emitter owns:\n %s",
len(lonely), strings.Join(lonely, "\n "))
}
}
// And it catches the thing it exists for: a role's event read as a module's.
func TestTheDerivedSubjectCheckCatchesARolesEventReadAsAModules(t *testing.T) {
published := map[string]bool{"mesh.seat.mesh-build-machine.event.built": true}
// What the derivation produced before a consumed seat name was resolved as one.
if reaches("mesh.mod.mesh-build-machine.event.built", published) {
t.Fatal("a module namespace was treated as reaching a role's event, which is the bug")
}
// And the corrected one does reach it.
if !reaches("mesh.seat.mesh-build-machine.event.built", published) {
t.Fatal("the role's own subject does not reach the role's event")
}
}
// reaches says whether a subscribed subject admits any published one.
func reaches(subject string, published map[string]bool) bool {
for p := range published {
if admitsSubject(strings.Split(subject, "."), strings.Split(p, ".")) {
return true
}
}
return false
}
func admitsSubject(pattern, subject []string) bool {
for i, token := range pattern {
if token == ">" {
return i < len(subject)
}
if i >= len(subject) {
return false
}
if token != "*" && token != subject[i] {
return false
}
}
return len(pattern) == len(subject)
}
-67
View File
@@ -1,67 +0,0 @@
package broker_test
import (
"regexp"
"testing"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/link"
)
// The names are written twice, so a test keeps them agreeing.
//
// `link` imports `broker`, so `broker` cannot import `link` — the queue and exchange names
// therefore exist in both. A scoped account naming a queue nothing publishes to produces a
// builder that takes no work and says nothing about why, which is the worst kind of silence.
//
// An external test package, because it may import both without either importing the other.
func TestTheNamesTheBrokerScopesAreTheNamesTheLinkUses(t *testing.T) {
for _, agreed := range []struct {
what string
scoped string
actually string
}{
{"the build queue", broker.BuildQueueName, link.BuildQueue},
{"the exchange", broker.ExchangeName, link.Exchange},
{"a node's queue", broker.QueueFor("somewhere"), link.QueueFor("somewhere")},
} {
if agreed.scoped != agreed.actually {
t.Errorf("%s: the broker scopes %q and the link uses %q",
agreed.what, agreed.scoped, agreed.actually)
}
}
}
func TestABuilderMayWriteToAReplyQueueAndReadNoNodesDeclarations(t *testing.T) {
// The scoping, checked as patterns rather than by connecting: what it may write must include
// the queue an asker actually waits on, and what it may read must not include any node's.
//
// This exists because the first version scoped writes to `amq.gen-*` — the name one broker
// happens to generate — and the builder built, could not answer, and the connection simply
// closed saying only "not allowed to publish to exchange \'\'". Answering through the
// exchange is what removed the need for any of that.
write := regexp.MustCompile("^" + regexp.QuoteMeta(broker.ExchangeName) + "$")
if !write.MatchString(broker.ExchangeName) {
t.Error("a builder may not write to the exchange, so it can take work and never answer")
}
// And not the default exchange, where permission is per exchange rather than per queue — a
// builder allowed to use it could publish into any node's queue.
//
// Confirmed against a real broker as well, and worth recording how that nearly went wrong:
// an unconfirmed publish is asynchronous, so a refusal arrives as a channel close afterwards
// and a naive check reports success. With publisher confirms the broker's refusal is
// immediate. **A negative security assertion made against an asynchronous call is not an
// assertion.**
if write.MatchString("") {
t.Error("a builder may publish to the default exchange, and so into any node's queue")
}
read := regexp.MustCompile("^" + regexp.QuoteMeta(broker.BuildQueueName) + "$")
if !read.MatchString(broker.BuildQueueName) {
t.Error("a builder may not read the build queue")
}
if read.MatchString(link.QueueFor("someone-else")) {
// A build machine is not a node, and a node's queue carries its declarations.
t.Error("a builder may read another machine's declarations")
}
}
-13
View File
@@ -194,16 +194,3 @@ func TestTheAddressPortFollowsThePortTwin(t *testing.T) {
t.Fatalf("the address is %q; the node put the bus on 5679", b.Address)
}
}
func TestTheManagementPortFollowsThePortTwin(t *testing.T) {
t.Setenv(ManagementVar, "http://guest:guest@127.0.0.1:15672")
t.Setenv(ManagementVar+"_FILE", "")
t.Setenv(ManagementVar+"_PORT", "15673")
m, err := ManagementFromEnvironment()
if err != nil {
t.Fatal(err)
}
if m.base.Host != "127.0.0.1:15673" {
t.Fatalf("the management API is at %q; the node put it on 15673", m.base.Host)
}
}
+237
View File
@@ -0,0 +1,237 @@
package broker
import (
"fmt"
"sort"
"strings"
)
// Streams and consumers derived from what modules declare.
//
// The mesh's own four exist before any module does (streams.go). Everything here is the other
// half: a seat's stream comes into being when the module declaring it is **registered**, and a
// consumer when a module is **assigned** — which is why ADR 0116's task 1.4 had to be narrowed to
// the foundation set. Neither has happened at genesis.
//
// All of it is a pure function of declarations. The controller is still the only writer; this is
// only what it writes.
// A Consumer is a durable subscription the controller creates on a module's behalf. A module
// declares what it reacts to, never how delivery works, so it does not name these and cannot
// misconfigure them.
type Consumer struct {
Name string
Stream string
// Filters are the subjects this consumer receives. One consumer per module with several
// filters, rather than one per consumed event: its ack subject is derived from its name, and
// a module with five consumers would need five ack permissions to ack its own deliveries.
Filters []string
// Queue is the queue group, set for a seat's worker so that "exactly one holder" survives a
// seat later being relaxed to several. Authority and delivery are kept separate on purpose.
Queue string
// Push asks the server to deliver to a subject rather than wait to be pulled.
//
// For the mesh's own consumer, where the controller wants every message to arrive in the one
// loop it already runs: pulling would mean a second goroutine fetching batches and handing
// them over, and a loop that acts on one message at a time is the property the store window
// depends on. A queue group implies this, because a group has nothing to pull from.
Push bool
// AckWaitSeconds before an unacknowledged delivery is redelivered.
AckWaitSeconds int
// MaxDeliver before the message is dead-lettered; zero for the mesh's default.
MaxDeliver int
Why string
}
// seatStreamName is the stream holding a seat's inbound work. Named after the seat rather than
// the module holding it, because the holder can change and the queued work must not care — which
// is the whole reason a caller addresses a seat instead of a module.
func seatStreamName(seat string) string { return "SEAT_" + upperSnake(seat) }
// SeatStreams is one work queue per declared seat, created when the declaring module is
// registered rather than when it is assigned.
//
// **The stream exists before anyone holds the seat, and that is the point.** Work queues until a
// holder appears, so installing the telegram module a week after something started sending to it
// flushes the backlog instead of having lost it. A stream created at assignment would make "the
// holder is not here yet" mean "your messages are gone".
func SeatStreams(seats []DeclaredSeat) []Stream {
sorted := append([]DeclaredSeat(nil), seats...)
sort.Slice(sorted, func(i, j int) bool { return sorted[i].Name < sorted[j].Name })
var out []Stream
for _, s := range sorted {
if len(s.Accepts) == 0 {
// A seat that only emits and serves needs no stream: its events ride EVENTS and its
// tools are core request/reply, which is never persisted.
continue
}
retain := s.RetainSeconds
if retain == 0 {
retain = 7 * 24 * 60 * 60
}
out = append(out, Stream{
Name: seatStreamName(s.Name),
Subjects: []string{"mesh.seat." + s.Name + ".accept.>"},
Retention: RetentionWorkQueue,
MaxAge: retain,
Why: fmt.Sprintf("work submitted to the %s seat; one holder consumes it, and it "+
"queues while nobody does", s.Name),
})
}
return out
}
// A DeclaredSeat is a seat as the catalogue knows it. Mirrored here rather than imported so this
// package stays free of the catalogue's own types — the same reason the host mirrors the
// contracts instead of importing the sdk.
type DeclaredSeat struct {
Name string
Accepts []string
// Emits are the verbs the seat's holder publishes under the seat's own name. An event about a
// role belongs here rather than in the holder's namespace, because the name then outlives
// whoever fills it (novox/hq ADR 0121, 04-ISSUES/127).
Emits []string
// Serves are the verbs the holder answers, request and reply.
Serves []string
RetainSeconds int
}
// ConsumerFor is the durable consumer a module's declarations imply, or false when it subscribes
// to nothing and needs none.
//
// One per module, with every consumed subject as a filter, because its ack permission is derived
// from its name: a module with a consumer per event would need an ack permission per consumer,
// and the permission list would stop being derivable from the declaration.
func ConsumerFor(p Principal) (Consumer, bool) {
// A module that reacts to anything — a module's events or a role's (novox/hq ADR 0121). Watching
// a role was missing here, so the one module that does it got no consumer at all: it started,
// connected, and its graph stayed empty with nothing anywhere reporting why.
if p.Kind != KindModule || (len(p.Consumes) == 0 && len(p.Watches) == 0) {
return Consumer{}, false
}
perms, err := PermissionsFor(p)
if err != nil {
return Consumer{}, false
}
// Events, wherever they live: a module's own namespace, and the namespace of any role it watches
// (novox/hq ADR 0121). Tool subjects and inboxes are subscribed directly and are not a consumer's
// business, which is why this is a filter and not the whole list.
var filters []string
for _, s := range perms.Subscribe {
if strings.Contains(s, ".event.") {
filters = append(filters, s)
}
}
if len(filters) == 0 {
return Consumer{}, false
}
sort.Strings(filters)
return Consumer{
Name: consumerDurable(p),
Stream: consumerStream(p),
Filters: filters,
AckWaitSeconds: 30,
MaxDeliver: 5,
Why: "what " + p.Module + " declared it consumes; after max-deliver it dead-letters",
}, true
}
// HolderConsumerFor is the worker a seat's holder gets on that seat's work queue.
//
// **A queue group even though the seat guarantees one holder.** The seat is *authority* — who may
// be the telegram sender — and the queue group is *delivery*. Tie delivery to the seat and the
// day somebody allows two holders for throughput, every message is processed twice with nothing
// reporting it. Kept separate, relaxing one changes nothing about the other.
func HolderConsumerFor(node, module string, seat DeclaredSeat) (Consumer, bool) {
if len(seat.Accepts) == 0 {
return Consumer{}, false
}
return Consumer{
Name: "SEAT_" + upperSnake(seat.Name) + "_worker",
Stream: seatStreamName(seat.Name),
Filters: []string{"mesh.seat." + seat.Name + ".accept.>"},
Queue: "holders",
AckWaitSeconds: 60,
MaxDeliver: 5,
Why: fmt.Sprintf("%s on %s holds %s; it acknowledges after the work is done, so a "+
"crash mid-work redelivers rather than loses", module, node, seat.Name),
}, true
}
// NodeConsumer is the durable consumer a node reads its own declaration through.
//
// **Derived from a node existing, and created by the controller, because a host cannot create it.**
// A host's account may subscribe its own declaration subject and publish its own ack subject, and
// reaches no part of the JetStream API — which is correct (the controller is the only writer of
// consumer definitions, design 25 §3) and means the consumer must be waiting before the host binds
// to it. Named after the node, because the node's ack grant is `$JS.ACK.NODES.<node>.>` and a
// consumer named anything else is one the host cannot acknowledge a delivery from.
//
// **No max-deliver, and a long ack wait.** A declaration is settled only after the node has applied
// it and reported, which is minutes on a machine pulling images; and a declaration the mesh cannot
// get a node to accept is not one to dead-letter, because the stream keeps only the newest per node
// anyway — so there is exactly one message per node to redeliver, for as long as that node is away.
func NodeConsumer(node string) Consumer {
return Consumer{
Name: node,
Stream: "NODES",
Filters: []string{"mesh.node." + node + ".declare"},
Push: true,
AckWaitSeconds: 300,
Why: "how " + node + " hears what it should be; last-per-subject, so a node that was away " +
"gets exactly the current declaration and nothing older",
}
}
// AssertNodeConsumers brings every known node's declaration consumer into being.
//
// Asserted on start as well as created at enrolment, for the reason the streams are: a mesh raised
// from a restored backup, or one whose bus was recreated, has node records and no consumers, and a
// node whose consumer is missing hears nothing while everything else about it looks correct.
func AssertNodeConsumers(e Ensurer, nodes []string) error {
for _, n := range nodes {
if err := e.EnsureConsumer(NodeConsumer(n)); err != nil {
return fmt.Errorf("asserting how %s hears its declaration: %w", n, err)
}
}
return nil
}
// AllOverlaps reports subject filters claimed by more than one stream, across the mesh's own and
// every derived one.
//
// NATS refuses an overlapping stream rather than merging it (verified against nats-server 2.10:
// "subjects overlap with an existing stream"), so this is not a subtle divergence — it is a
// registration that fails. Catching it here names both streams, before a half-applied mesh does.
func AllOverlaps(seats []DeclaredSeat) []string {
all := append(MeshStreams(), SeatStreams(seats)...)
seen := map[string]string{}
var clashes []string
for _, s := range all {
for _, subject := range s.Subjects {
if first, ok := seen[subject]; ok {
clashes = append(clashes, fmt.Sprintf("%s and %s both claim %s", first, s.Name, subject))
continue
}
seen[subject] = s.Name
}
}
sort.Strings(clashes)
return clashes
}
// upperSnake makes a stream name from a seat name. NATS stream names may not contain a dot,
// a space or a wildcard, and a hyphen is legal but reads badly beside the mesh's own.
func upperSnake(s string) string {
out := []rune(s)
for i, r := range out {
switch {
case r >= 'a' && r <= 'z':
out[i] = r - 32
case r == '-' || r == '.':
out[i] = '_'
}
}
return string(out)
}
+155
View File
@@ -0,0 +1,155 @@
package broker
import (
"strings"
"testing"
)
func telegramSeat() DeclaredSeat {
return DeclaredSeat{Name: "telegram-sender", Accepts: []string{"send"}}
}
// The stream exists from registration, not assignment: work queues until a holder appears, so
// installing the module a week later flushes the backlog rather than having lost it.
func TestASeatGetsAWorkQueueOfItsOwn(t *testing.T) {
got := SeatStreams([]DeclaredSeat{telegramSeat()})
if len(got) != 1 {
t.Fatalf("expected one stream, got %d", len(got))
}
s := got[0]
if s.Retention != RetentionWorkQueue {
t.Fatalf("a seat's inbound queue retains as %q; one holder must take each message once", s.Retention)
}
if s.Subjects[0] != "mesh.seat.telegram-sender.accept.>" {
t.Fatalf("filters on %v", s.Subjects)
}
}
// A seat that only emits and serves needs no stream: its events ride EVENTS and its tools are
// core request/reply, which is never persisted.
func TestASeatThatAcceptsNothingGetsNoStream(t *testing.T) {
if got := SeatStreams([]DeclaredSeat{{Name: "announcer"}}); len(got) != 0 {
t.Fatalf("a seat with no inbound work got %d stream(s)", len(got))
}
}
// Retention belongs to whoever owns the namespace, and a seat owns its own.
func TestASeatsRetentionIsItsOwn(t *testing.T) {
s := SeatStreams([]DeclaredSeat{{Name: "slow", Accepts: []string{"work"}, RetainSeconds: 30 * 24 * 60 * 60}})
if s[0].MaxAge != 30*24*60*60 {
t.Fatalf("the seat's declared retention was not used: %d", s[0].MaxAge)
}
d := SeatStreams([]DeclaredSeat{telegramSeat()})
if d[0].MaxAge == 0 {
t.Fatal("a seat that declares no retention got an unbounded queue")
}
}
// NATS refuses an overlapping stream outright, so a clash here is a registration that fails.
func TestNoDerivedStreamOverlapsTheMeshsOwn(t *testing.T) {
seats := []DeclaredSeat{telegramSeat(), {Name: "licensing-master", Accepts: []string{"report"}}}
if c := AllOverlaps(seats); len(c) != 0 {
t.Fatalf("overlapping filters: %v", c)
}
}
// One consumer per module, with every consumed subject as a filter — because its ack permission
// is derived from its name, and a consumer per event would need an ack permission per consumer.
func TestAModuleGetsOneConsumerCarryingEveryFilter(t *testing.T) {
c, ok := ConsumerFor(Principal{Kind: KindModule, Node: "one", Module: "audit",
Consumes: []string{"shop.order.placed", "billing.invoice.sent"}, PasswordHash: "x"})
if !ok {
t.Fatal("a module that consumes got no consumer")
}
if len(c.Filters) != 2 {
t.Fatalf("expected both subjects as filters, got %v", c.Filters)
}
perms, _ := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "audit",
Consumes: []string{"shop.order.placed"}, PasswordHash: "x"})
ack := "$JS.ACK." + c.Stream + "." + c.Name + ".>"
found := false
for _, p := range perms.Publish {
if p == ack {
found = true
}
}
if !found {
t.Fatalf("the consumer is named %q but the ack permission is %v; a module could not ack "+
"its own deliveries", c.Name, perms.Publish)
}
}
// A module that subscribes to nothing needs no consumer, and creating one would leave an object
// nothing reads and everything has to maintain.
func TestAModuleThatConsumesNothingGetsNoConsumer(t *testing.T) {
if _, ok := ConsumerFor(Principal{Kind: KindModule, Node: "one", Module: "shop",
Emits: []string{"order.placed"}, PasswordHash: "x"}); ok {
t.Fatal("a pure emitter got a consumer")
}
}
// The seat is authority and the queue group is delivery. Tie them together and the day somebody
// allows two holders, every message is processed twice with nothing reporting it.
func TestAHoldersWorkerUsesAQueueGroupAnyway(t *testing.T) {
c, ok := HolderConsumerFor("one", "telegram", telegramSeat())
if !ok {
t.Fatal("the holder of a seat with inbound work got no worker")
}
if c.Queue == "" {
t.Fatal("the worker is not in a queue group, so a second holder would double-process")
}
if c.Stream != "SEAT_TELEGRAM_SENDER" {
t.Fatalf("the worker reads %q, not the seat's own stream", c.Stream)
}
if c.MaxDeliver == 0 {
t.Fatal("a failing worker would redeliver forever rather than dead-letter")
}
}
// The stream is named after the seat, not its holder: the holder can change and the queued work
// must not care.
func TestASeatsStreamIsNamedAfterTheSeat(t *testing.T) {
name := seatStreamName("telegram-sender")
if strings.Contains(name, "telegram-sender") {
t.Fatalf("%q keeps characters a stream name may not hold", name)
}
if name != "SEAT_TELEGRAM_SENDER" {
t.Fatalf("unexpected stream name %q", name)
}
}
// A node hears its declaration through a consumer only the controller can make.
//
// The three things that would each break it silently: a name other than the node's is one the host
// cannot acknowledge a delivery from, because its ack grant is derived from the node's name; a
// filter other than its own declaration subject is a node reading another's; and a pull consumer is
// one the host cannot bind a channel to without creating something, which it has no authority for.
func TestANodesDeclarationConsumerIsWhatItsOwnGrantAllows(t *testing.T) {
c := NodeConsumer("anchor")
if c.Name != "anchor" {
t.Fatalf("named %q, so the node cannot ack from it: its grant is $JS.ACK.NODES.anchor.>", c.Name)
}
if c.Stream != "NODES" {
t.Fatalf("on stream %q rather than the one declarations live in", c.Stream)
}
if len(c.Filters) != 1 || c.Filters[0] != "mesh.node.anchor.declare" {
t.Fatalf("filters %v, which is not this node's own declaration and nothing else", c.Filters)
}
if !c.Push {
t.Fatal("pulled, which a host cannot do: pulling needs the JetStream API and a host reaches none of it")
}
if c.MaxDeliver != 0 {
t.Fatalf("max-deliver %d: a declaration a node has not taken yet is not one to dead-letter, "+
"because the stream holds exactly one per node", c.MaxDeliver)
}
// And the grant the node actually gets has to match, or none of the above matters.
perms, err := PermissionsFor(Principal{Kind: KindNode, Node: "anchor"})
if err != nil {
t.Fatal(err)
}
// Without the ack grant every declaration a node receives is redelivered for ever; without the
// subscribe grant its consumer delivers to nobody.
has(t, perms.Publish, "$JS.ACK.NODES."+c.Name+".>")
has(t, perms.Subscribe, c.Filters[0])
}
+126
View File
@@ -0,0 +1,126 @@
package broker
import (
"encoding/json"
"os"
"path/filepath"
"regexp"
"sort"
"strings"
"testing"
"golang.org/x/crypto/bcrypt"
)
// **The first user list the installer carries must be the one the controller would compose.**
//
// At genesis there is no mesh to write the bus's user list, so the installer carries one: the
// controller's own account, at a bootstrap password, the way the store is reached at
// `postgres:bootstrap` (novox/hq design 25 §4, task 1.7). It is written by hand in a template and
// derived in code here, which is two statements of one fact — so this compares them.
//
// Getting it wrong is the worst kind of silent: a controller whose carried permissions are narrower
// than the ones it derives comes up, connects, and is refused on the first thing it tries, with an
// authorisation error that names a subject and not the template that forgot it. And a mesh cannot be
// raised twice to find out.
func TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose(t *testing.T) {
accounts := theCarriedAccounts(t)
want, err := PermissionsFor(Principal{Kind: KindController, PasswordHash: "x"})
if err != nil {
t.Fatal(err)
}
carriedPub := subjectsIn(accounts, "publish")
carriedSub := subjectsIn(accounts, "subscribe")
if diff := missing(want.Publish, carriedPub); len(diff) > 0 {
t.Errorf("the installer's user list does not let the controller publish %v — it would come up "+
"and be refused on the first thing it tried", diff)
}
if diff := missing(want.Subscribe, carriedSub); len(diff) > 0 {
t.Errorf("the installer's user list does not let the controller subscribe %v", diff)
}
// And nothing wider than what it derives, or genesis quietly grants a privilege the composition
// takes away again on the first push.
if diff := missing(carriedPub, want.Publish); len(diff) > 0 {
t.Errorf("the installer's user list lets the controller publish %v, which it does not derive", diff)
}
if diff := missing(carriedSub, want.Subscribe); len(diff) > 0 {
t.Errorf("the installer's user list lets the controller subscribe %v, which it does not derive", diff)
}
// The credential is the bootstrap one and the hash really is of it, because a hash of something
// else is a controller that cannot log in to the bus it was just given.
hash := regexp.MustCompile(`\$2[aby]?\$[0-9]+\$[A-Za-z0-9./]{53}`).FindString(accounts)
if hash == "" {
t.Fatal("the installer's user list carries no password hash")
}
if err := bcrypt.CompareHashAndPassword([]byte(hash), []byte("bootstrap")); err != nil {
t.Fatalf("the carried hash does not verify the bootstrap credential the template also carries: %v", err)
}
}
// theCarriedAccounts is the accounts file the installer's template writes at genesis.
func theCarriedAccounts(t *testing.T) string {
t.Helper()
path := filepath.Join("..", "..", "..", "mesh-host", "examples", "foundation-first-node-nats.lock")
raw, err := os.ReadFile(path)
if err != nil {
t.Skipf("the host's checkout is not beside this one: %v", err)
}
// The template is JSON with line comments, which is how every one of them is written.
var lines []string
for _, l := range strings.Split(string(raw), "\n") {
if !strings.HasPrefix(strings.TrimSpace(l), "//") {
lines = append(lines, l)
}
}
var bundle struct {
Resources []map[string]any `json:"resources"`
}
if err := json.Unmarshal([]byte(strings.Join(lines, "\n")), &bundle); err != nil {
t.Fatalf("the template is not readable: %v", err)
}
for _, r := range bundle.Resources {
if r["id"] == "bus-accounts" {
content, _ := r["content"].(string)
if content == "" {
t.Fatal("the template's accounts file is empty, so the bus would refuse every connection")
}
return content
}
}
t.Fatal("the template carries no accounts file, so a mesh raised from it has a bus nobody may use")
return ""
}
// subjectsIn reads one allow-list out of a composed accounts file.
func subjectsIn(accounts, which string) []string {
found := regexp.MustCompile(which + `: \{ allow: \[([^\]]*)\]`).FindStringSubmatch(accounts)
if len(found) != 2 {
return nil
}
var out []string
for _, part := range strings.Split(found[1], ",") {
if s := strings.Trim(strings.TrimSpace(part), `"`); s != "" {
out = append(out, s)
}
}
sort.Strings(out)
return out
}
// missing is what is in want and not in got.
func missing(want, got []string) []string {
have := map[string]bool{}
for _, g := range got {
have[g] = true
}
var out []string
for _, w := range want {
if !have[w] {
out = append(out, w)
}
}
return out
}
+213
View File
@@ -0,0 +1,213 @@
package broker
import (
"crypto/sha256"
"crypto/tls"
"crypto/x509"
"encoding/hex"
"errors"
"fmt"
"os"
"strings"
"time"
"github.com/nats-io/nats.go"
)
// The JetStream side of the controller: the one place the mesh's streams and consumers are
// actually created.
//
// Everything that decides *what* they are is pure and lives beside this (streams.go, derived.go).
// This is only the part that talks to a server, kept small on purpose: a bug in a subject filter
// should be findable in a unit test, and only a bug in "did the server accept it" should need one
// running.
// A JetStream is a connection to the bus, as the controller uses it.
type JetStream struct {
conn *nats.Conn
js nats.JetStreamContext
}
// Dial connects and returns the controller's JetStream handle.
func Dial(url string, opts ...nats.Option) (*JetStream, error) {
opts = append(opts, nats.Name("mesh-controller"), nats.Timeout(10*time.Second))
// **Pinned, not named.** The bus presents the mesh's own certificate, which names nothing a
// public verifier would accept (design 25 §4: a host pins the server's exact certificate and
// checks nothing else, and so does this). Without this, the first connection failed with
// "certificate is not valid for any names" against a bus that was answering (2026-09-28).
if path := strings.TrimSpace(os.Getenv(CertificateVar)); path != "" {
pinned, err := pinnedTo(path)
if err != nil {
return nil, err
}
opts = append(opts, nats.Secure(pinned))
}
// **Its own inbox, and nothing wider.** Every principal is granted `_INBOX.<its user>.>` and
// no other inbox; the client's default prefix is random, and the server refused the first
// subscription to it (2026-09-28). The user is in the URL, so the prefix follows from it.
if user, _, _ := CredentialIn(url); user != "" {
opts = append(opts, nats.CustomInboxPrefix("_INBOX."+user))
}
// The address in an error is the address alone. The URL carries this controller's password,
// and an error here is written on the assumption it will be logged.
where := BareAddress(url)
conn, err := nats.Connect(url, opts...)
if err != nil {
return nil, fmt.Errorf("connecting to the bus at %s: %w", where, err)
}
js, err := conn.JetStream()
if err != nil {
conn.Close()
return nil, fmt.Errorf("the bus at %s has no JetStream: %w", where, err)
}
return &JetStream{conn: conn, js: js}, nil
}
// pinnedTo is a TLS configuration that accepts exactly the certificate in the file and no other:
// the leaf's SHA-256, compared on every handshake, with the name and the chain deliberately not
// consulted — a self-signed certificate with no names is the ordinary case for a mesh's bus.
func pinnedTo(path string) (*tls.Config, error) {
want, err := FingerprintOf(path)
if err != nil {
return nil, err
}
return PinnedToFingerprint(want), nil
}
// DialPinned is Dial with the server's certificate pinned by a fingerprint the caller already holds
// — a module or a build machine that was handed one beside its credential, and has no file.
func DialPinned(url, fingerprint string, opts ...nats.Option) (*JetStream, error) {
if strings.TrimSpace(fingerprint) != "" {
opts = append(opts, nats.Secure(PinnedToFingerprint(fingerprint)))
}
return Dial(url, opts...)
}
// PinnedToFingerprint accepts exactly the certificate with this SHA-256 and no other.
func PinnedToFingerprint(want string) *tls.Config {
return &tls.Config{
InsecureSkipVerify: true, //nolint:gosec // replaced by the pin below, which is stricter
MinVersion: tls.VersionTLS12,
VerifyPeerCertificate: func(rawCerts [][]byte, _ [][]*x509.Certificate) error {
if len(rawCerts) == 0 {
return errors.New("the bus presented no certificate")
}
sum := sha256.Sum256(rawCerts[0])
got := "sha256:" + hex.EncodeToString(sum[:])
if got != want {
return fmt.Errorf("the bus presented a certificate this mesh does not know (%s…), expected %s…", got[:23], want[:23])
}
return nil
},
}
}
// Conn is the connection itself, for what the mesh keeps off JetStream on purpose — a heartbeat,
// a tool call — where a lost message is answered by the next one or by a timeout the caller
// already handles (design 25 §3).
func (j *JetStream) Conn() *nats.Conn { return j.conn }
// Context is the JetStream handle, for subscribing to what the consumers above define.
func (j *JetStream) Context() nats.JetStreamContext { return j.js }
func (j *JetStream) Close() {
if j.conn != nil {
j.conn.Close()
}
}
// EnsureStream creates the stream if it is absent and brings it to match if it is present.
//
// **Idempotent, because the controller asserts on every start** rather than creating once at
// genesis: a stream somebody deleted, or a mesh raised from a restored backup, has to converge
// rather than run without the guarantee its messages assume.
//
// An update, not a delete and recreate. Recreating would discard every message the stream holds
// and every consumer's position in it — which for CONTROL means the pushes being held through a
// store restart, exactly the guarantee the stream exists for.
func (j *JetStream) EnsureStream(s Stream) error {
want := &nats.StreamConfig{
Name: s.Name,
Subjects: s.Subjects,
Retention: retentionOf(s.Retention),
MaxAge: time.Duration(s.MaxAge) * time.Second,
MaxMsgsPerSubject: int64(s.MaxMsgsPerSubject),
Description: s.Why,
}
if s.Retention == RetentionLastPerSubject {
// Last-per-subject is a limits stream with one message kept per subject, not a
// retention policy of its own — the state shape, spelled the way the server spells it.
want.Retention = nats.LimitsPolicy
want.MaxMsgsPerSubject = 1
want.MaxAge = 0
}
switch _, err := j.js.StreamInfo(s.Name); {
case err == nil:
if _, err := j.js.UpdateStream(want); err != nil {
return fmt.Errorf("bringing stream %s to match: %w", s.Name, err)
}
return nil
case errors.Is(err, nats.ErrStreamNotFound):
if _, err := j.js.AddStream(want); err != nil {
return fmt.Errorf("creating stream %s: %w", s.Name, err)
}
return nil
default:
return fmt.Errorf("asking about stream %s: %w", s.Name, err)
}
}
// EnsureConsumer creates or updates one durable consumer.
//
// Explicit acknowledgement throughout: a consumer that acknowledges on delivery cannot redeliver
// work its holder died in the middle of, which is the whole difference between a queue and a
// firehose.
func (j *JetStream) EnsureConsumer(c Consumer) error {
want := &nats.ConsumerConfig{
Durable: c.Name,
AckPolicy: nats.AckExplicitPolicy,
AckWait: time.Duration(c.AckWaitSeconds) * time.Second,
MaxDeliver: c.MaxDeliver,
DeliverGroup: c.Queue,
DeliverSubject: "",
Description: c.Why,
}
switch len(c.Filters) {
case 0:
case 1:
want.FilterSubject = c.Filters[0]
default:
want.FilterSubjects = c.Filters
}
// A queue group needs a delivery subject: a pull consumer has no group, and declaring one
// without the other is refused by the server with a message that does not say which half is
// missing.
if c.Queue != "" || c.Push {
want.DeliverSubject = "_DELIVER." + c.Name
}
switch _, err := j.js.ConsumerInfo(c.Stream, c.Name); {
case err == nil:
if _, err := j.js.UpdateConsumer(c.Stream, want); err != nil {
return fmt.Errorf("bringing consumer %s on %s to match: %w", c.Name, c.Stream, err)
}
return nil
case errors.Is(err, nats.ErrConsumerNotFound):
if _, err := j.js.AddConsumer(c.Stream, want); err != nil {
return fmt.Errorf("creating consumer %s on %s: %w", c.Name, c.Stream, err)
}
return nil
default:
return fmt.Errorf("asking about consumer %s on %s: %w", c.Name, c.Stream, err)
}
}
func retentionOf(r Retention) nats.RetentionPolicy {
switch r {
case RetentionWorkQueue:
return nats.WorkQueuePolicy
default:
return nats.LimitsPolicy
}
}
+71
View File
@@ -0,0 +1,71 @@
package broker
import (
"os"
"testing"
)
// Against a real server, because the questions here are all "does the server accept this" —
// which a mock would answer by agreeing with whatever this file already believes.
//
// Skipped unless MESH_TEST_NATS names one, so the ordinary suite stays fast and offline:
//
// docker run -d --rm --name t -p 14222:4222 nats:2.10-alpine -js
// MESH_TEST_NATS=nats://127.0.0.1:14222 go test ./internal/broker/ -run TestAgainstARealServer
func TestAgainstARealServer(t *testing.T) {
url := os.Getenv("MESH_TEST_NATS")
if url == "" {
t.Skip("MESH_TEST_NATS unset")
}
js, err := Dial(url)
if err != nil {
t.Fatal(err)
}
defer js.Close()
t.Run("the mesh's own streams are accepted", func(t *testing.T) {
if err := AssertMeshStreams(js); err != nil {
t.Fatal(err)
}
})
t.Run("asserting again changes nothing and fails nothing", func(t *testing.T) {
if err := AssertMeshStreams(js); err != nil {
t.Fatalf("the second assertion failed, so the controller cannot restart: %v", err)
}
})
t.Run("a seat's work queue is accepted beside them", func(t *testing.T) {
seats := []DeclaredSeat{{Name: "telegram-sender", Accepts: []string{"send"}}}
for _, s := range SeatStreams(seats) {
if err := js.EnsureStream(s); err != nil {
t.Fatal(err)
}
}
if c := AllOverlaps(seats); len(c) != 0 {
t.Fatalf("overlaps the server would refuse: %v", c)
}
})
t.Run("a module's consumer is accepted and is idempotent", func(t *testing.T) {
c, ok := ConsumerFor(Principal{Kind: KindModule, Node: "one", Module: "audit",
Consumes: []string{"shop.order.placed", "billing.invoice.sent"}, PasswordHash: "x"})
if !ok {
t.Fatal("no consumer derived")
}
if err := js.EnsureConsumer(c); err != nil {
t.Fatal(err)
}
if err := js.EnsureConsumer(c); err != nil {
t.Fatalf("the second assertion failed: %v", err)
}
})
t.Run("a holder's worker is accepted with its queue group", func(t *testing.T) {
c, _ := HolderConsumerFor("one", "telegram",
DeclaredSeat{Name: "telegram-sender", Accepts: []string{"send"}})
if err := js.EnsureConsumer(c); err != nil {
t.Fatal(err)
}
})
}
-366
View File
@@ -1,366 +0,0 @@
package broker
import (
"bytes"
"context"
"encoding/json"
"fmt"
"github.com/novox/mesh-controller/internal/envfile"
"io"
"net/http"
"net/url"
"regexp"
"strings"
"time"
)
// The mesh runs the broker, so there is no chicken-and-egg in a node needing an account before it
// can connect: the account is created when the token is issued, and the one-time secret in that
// token IS the password. A node's first connection is already authenticated, and enrolment is
// what happens over it.
//
// novox/hq ADR 0004's *a node holds its own identity and nothing else* is why the account is per
// node rather than shared. A shared enrolment account would let any node consume another's queue,
// which is the shared-credential fault that record exists to remove, reappearing at the transport.
// ManagementVar holds the broker's management API, credentials included.
const ManagementVar = "MESH_BROKER_MANAGEMENT"
// safeName is what a node may be called at the broker.
//
// The name goes into a URL path and into permission patterns, which are regular expressions. A
// name carrying a `.` or a `*` would silently widen what that node may reach — so it is
// constrained here rather than escaped later, because an escape that is forgotten once is a node
// reading everybody's queues.
var safeName = regexp.MustCompile(`^[a-z0-9][a-z0-9-]{0,62}$`)
// Management is the broker's administrative interface.
type Management struct {
base *url.URL
client *http.Client
}
// ManagementFromEnvironment reads where the management API is, if it is configured.
//
// On the port MESH_BROKER_MANAGEMENT_PORT names when the node moved it (novox/hq 04-ISSUES/102);
// the URL's own port otherwise.
func ManagementFromEnvironment() (*Management, error) {
raw, err := envfile.Placed(ManagementVar)
if err != nil {
return nil, err
}
if raw == "" {
return nil, ErrNotConfigured
}
base, err := url.Parse(raw)
if err != nil || base.Host == "" {
// The value carries a password, so it is not quoted back.
return nil, fmt.Errorf("%s is not a usable URL", ManagementVar)
}
return &Management{base: base, client: &http.Client{Timeout: 15 * time.Second}}, nil
}
// QueueFor is the queue a node consumes from. One per node, named after it.
func QueueFor(node string) string { return "node." + node }
// ExchangeName is where nodes publish what they have to say. One exchange, and the control plane
// is the only consumer behind it (novox/hq ADR 0006 — one consumer, so two cannot silently split
// the traffic between them).
const ExchangeName = "mesh"
// BuildQueueName is where build work waits. Duplicated from `link` rather than imported, for the
// same reason QueueFor above is: this package must not depend on the one that uses it, and a
// constant that differed would be caught by the test that asserts they agree.
const BuildQueueName = "builds"
// The events bus (novox/hq ADR 0042): one topic exchange every event rides, a second for tool
// RPC kept apart, and a dead-letter home for a poison event. The foundation owns these — a module's
// account cannot declare them, only bind its own queue to the events one.
const (
EventsExchangeName = "mesh.events"
RPCExchangeName = "mesh.rpc"
DeadExchangeName = "mesh.events.dead"
)
// ModuleQueueFor is the durable queue a module consumes its events from — one per module per node
// (novox/hq ADR 0042), named so the account that may read it is exactly this module's.
func ModuleQueueFor(node, module string) string { return node + "." + module + ".events" }
// modulePermissions is a module's authority on the bus, derived from its manifest (novox/hq
// ADR 0043): what it consumes and what it emits, and nothing else. Pure, so the scope is tested as
// patterns without a broker — the way a builder's is.
//
// A note on the limit: the broker's write permission is per exchange, not per routing key (LavinMQ
// has no topic permissions), so an emitting module is granted the events exchange whole. ADR 0042's
// origin reservation — a module publishes only under `module.<self>.*` — is stamped by the sdk, not
// enforced here; that gap is the broker's, and is recorded rather than hidden. A pure consumer like
// the audit logger is unaffected: it is granted no write to the exchange at all.
func modulePermissions(node, module string, emits, consumes []string) (configure, write, read string) {
queue := regexp.QuoteMeta(ModuleQueueFor(node, module))
events := regexp.QuoteMeta(EventsExchangeName)
rpc := regexp.QuoteMeta(RPCExchangeName)
// A module serves each of its tools on its own queue, namespaced by the module (novox/hq
// ADR 0047) — serve.<module>.<tool> — so the account may declare, bind and read exactly its own,
// and no other module's.
serve := "serve\\." + regexp.QuoteMeta(module) + "\\..*"
// Declare its own events queue and its own tool serve queues.
configure = "^(" + queue + "|" + serve + ")$"
// Write to bind its queue and serve queues (binding is a write on the queue), and to the RPC
// exchange to publish replies (ADR 0047: replies ride mesh.rpc, never the default exchange, which
// would let it publish into any queue). To the events exchange only if it emits.
writes := []string{queue, serve, rpc}
if len(emits) > 0 {
writes = append(writes, events)
}
write = "^(" + strings.Join(writes, "|") + ")$"
// Read its own queue and serve queues to consume them, and the RPC exchange to bind its serve
// queues onto. The events exchange to bind onto only if it consumes.
reads := []string{queue, serve, rpc}
if len(consumes) > 0 {
reads = append(reads, events)
}
read = "^(" + strings.Join(reads, "|") + ")$"
return configure, write, read
}
// CreateModuleAccount gives an assigned module its own broker account, scoped by what it emits and
// consumes (novox/hq ADR 0043). The account name carries the node so the same module on two machines
// holds two accounts, each sealed to its own; the permissions carry the module so one module cannot
// read another's queue. Generic — the builder is one instance of this rule, not a separate kind.
func (m *Management) CreateModuleAccount(ctx context.Context, node, module, password string, emits, consumes []string) (string, error) {
if !safeName.MatchString(node) {
return "", fmt.Errorf("%q cannot be part of a broker account: it is a permission pattern", node)
}
if !safeName.MatchString(module) {
return "", fmt.Errorf("%q cannot be part of a broker account: it is a permission pattern", module)
}
account := node + "-" + module
if !safeName.MatchString(account) {
return "", fmt.Errorf("%q is not a usable broker account name", account)
}
if err := m.put(ctx, "/api/users/"+url.PathEscape(account),
map[string]string{"password": password, "tags": ""}); err != nil {
return "", fmt.Errorf("cannot create the broker account for %s on %s: %w", module, node, err)
}
configure, write, read := modulePermissions(node, module, emits, consumes)
if err := m.put(ctx, "/api/permissions/%2f/"+url.PathEscape(account), map[string]string{
"configure": configure, "write": write, "read": read,
}); err != nil {
return "", fmt.Errorf("cannot scope the broker account for %s on %s: %w", module, node, err)
}
return account, nil
}
// EnsureModuleQueue declares a consuming module's queue with its dead-letter exchange, idempotently.
// The foundation declares it because a scoped module account may not: the broker refuses a queue with
// a dead-letter exchange to a non-administrator (novox/hq ADR 0043), so a consumer passively checks
// the queue the mesh made rather than declaring its own.
func (m *Management) EnsureModuleQueue(ctx context.Context, node, module string) error {
queue := ModuleQueueFor(node, module)
if err := m.put(ctx, "/api/queues/%2f/"+url.PathEscape(queue), map[string]any{
"durable": true,
"arguments": map[string]any{"x-dead-letter-exchange": DeadExchangeName},
}); err != nil {
return fmt.Errorf("cannot declare the queue for %s on %s: %w", module, node, err)
}
return nil
}
// EnsureEventExchanges declares the bus's exchanges and the dead-letter home, idempotently. The
// foundation owns them (a module's account may not declare an exchange), and a dead-letter exchange
// with no queue behind it drops what it receives — so a durable queue bound to `#` retains a poison
// event for inspection, which is the whole reason the trail exists.
func (m *Management) EnsureEventExchanges(ctx context.Context) error {
for _, exchange := range []string{EventsExchangeName, RPCExchangeName, DeadExchangeName} {
if err := m.put(ctx, "/api/exchanges/%2f/"+url.PathEscape(exchange),
map[string]any{"type": "topic", "durable": true}); err != nil {
return fmt.Errorf("cannot declare the %s exchange: %w", exchange, err)
}
}
if err := m.put(ctx, "/api/queues/%2f/"+url.PathEscape(DeadExchangeName),
map[string]any{"durable": true}); err != nil {
return fmt.Errorf("cannot declare the dead-letter queue: %w", err)
}
if err := m.post(ctx, "/api/bindings/%2f/e/"+url.PathEscape(DeadExchangeName)+
"/q/"+url.PathEscape(DeadExchangeName), map[string]string{"routing_key": "#"}); err != nil {
return fmt.Errorf("cannot bind the dead-letter queue: %w", err)
}
return nil
}
// CreateNodeAccount gives a node its own broker account, with the token's secret as the password.
//
// Scoped so a node can reach its own queue and the one exchange, and nothing else. The patterns
// are anchored: a node called `laptop` must not be able to read `laptop-of-somebody-else`.
func (m *Management) CreateNodeAccount(ctx context.Context, node, password string) error {
if !safeName.MatchString(node) {
return fmt.Errorf(
"%q cannot be a broker account name: it becomes part of a permission pattern, so it "+
"is lower-case letters, digits and dashes", node)
}
if err := m.put(ctx, "/api/users/"+url.PathEscape(node),
map[string]string{"password": password, "tags": ""}); err != nil {
return fmt.Errorf("cannot create the broker account for %s: %w", node, err)
}
queue := regexp.QuoteMeta(QueueFor(node))
if err := m.put(ctx, "/api/permissions/%2f/"+url.PathEscape(node), map[string]string{
"configure": "^" + queue + "$",
"write": "^(" + regexp.QuoteMeta(ExchangeName) + "|" + queue + ")$",
"read": "^" + queue + "$",
}); err != nil {
return fmt.Errorf("cannot scope the broker account for %s: %w", node, err)
}
return nil
}
// CreateBuilderAccount scopes an account to taking build work and answering it.
//
// **A build machine is not a node**, and giving it a node's account would let it read another
// machine's declarations. What it needs is narrower and different: read the build queue, and
// write to the exchange and to whatever temporary queue an asker is waiting on.
//
// The reply queues are the reason `write` is not simply the exchange. `RequestBuild` declares an
// exclusive queue with a generated name and waits on it, so a builder that could not write to it
// could take work and never answer — which is the failure that looks like a builder that is not
// running.
func (m *Management) CreateBuilderAccount(ctx context.Context, name, password string) error {
if !safeName.MatchString(name) {
return fmt.Errorf(
"%q cannot be a broker account name: it becomes part of a permission pattern, so it "+
"is lower-case letters, digits and dashes", name)
}
if err := m.put(ctx, "/api/users/"+url.PathEscape(name),
map[string]string{"password": password, "tags": ""}); err != nil {
return fmt.Errorf("cannot create the broker account for %s: %w", name, err)
}
builds := regexp.QuoteMeta(BuildQueueName)
if err := m.put(ctx, "/api/permissions/%2f/"+url.PathEscape(name), map[string]string{
// It declares the build queue, because whichever builder starts first must be able to —
// and a queue nobody may declare is a queue that exists only if the control plane has
// already run, which makes the order they start in matter.
"configure": "^" + builds + "$",
// Two exchanges, and nothing else. **Not the default exchange**: permission there is
// granted per exchange rather than per queue, so a builder allowed to use it could
// publish into any node's queue — the privilege a build machine most obviously should
// not have. Answers go through the node exchange; announcing what was built goes through
// the events exchange, which is a different act with a different audience (novox/hq
// ADR 0072). A builder that could answer and not announce would leave the module graph
// knowing less than the registry does.
"write": "^(" + regexp.QuoteMeta(ExchangeName) + "|" + regexp.QuoteMeta(EventsExchangeName) + ")$",
// The build queue and nothing else. Not another machine's declarations.
"read": "^" + builds + "$",
}); err != nil {
return fmt.Errorf("cannot scope the broker account for %s: %w", name, err)
}
return nil
}
// RemoveNodeAccount withdraws a node's access.
func (m *Management) RemoveNodeAccount(ctx context.Context, node string) error {
if !safeName.MatchString(node) {
return fmt.Errorf("%q is not a broker account name", node)
}
return m.do(ctx, http.MethodDelete, "/api/users/"+url.PathEscape(node), nil)
}
// Accounts lists the broker's users, so a picture can be read from the system rather than assumed
// (novox/hq ADR 0018).
func (m *Management) Accounts(ctx context.Context) ([]string, error) {
body, err := m.get(ctx, "/api/users")
if err != nil {
return nil, err
}
var users []struct {
Name string `json:"name"`
}
if err := json.Unmarshal(body, &users); err != nil {
return nil, err
}
names := make([]string, 0, len(users))
for _, u := range users {
names = append(names, u.Name)
}
return names, nil
}
func (m *Management) put(ctx context.Context, path string, body any) error {
return m.do(ctx, http.MethodPut, path, body)
}
func (m *Management) post(ctx context.Context, path string, body any) error {
return m.do(ctx, http.MethodPost, path, body)
}
func (m *Management) get(ctx context.Context, path string) ([]byte, error) {
request, err := m.request(ctx, http.MethodGet, path, nil)
if err != nil {
return nil, err
}
response, err := m.client.Do(request)
if err != nil {
return nil, err
}
defer response.Body.Close()
if response.StatusCode >= 300 {
return nil, fmt.Errorf("the broker's management API answered %s to GET %s",
response.Status, path)
}
return io.ReadAll(io.LimitReader(response.Body, 1<<20))
}
func (m *Management) do(ctx context.Context, method, path string, body any) error {
request, err := m.request(ctx, method, path, body)
if err != nil {
return err
}
response, err := m.client.Do(request)
if err != nil {
return err
}
defer response.Body.Close()
if response.StatusCode >= 300 {
detail, _ := io.ReadAll(io.LimitReader(response.Body, 4096))
return fmt.Errorf("the broker's management API answered %s to %s %s: %s",
response.Status, method, path, strings.TrimSpace(string(detail)))
}
return nil
}
func (m *Management) request(ctx context.Context, method, path string, body any) (*http.Request, error) {
var payload io.Reader
if body != nil {
raw, err := json.Marshal(body)
if err != nil {
return nil, err
}
payload = bytes.NewReader(raw)
}
// Path joined by hand rather than through url.Parse: %2f is the default vhost and must reach
// the broker still encoded. Parsing would decode it to a slash and address a different route.
target := strings.TrimSuffix(m.base.String(), "/")
if user := m.base.User; user != nil {
target = strings.TrimSuffix(m.base.Scheme+"://"+m.base.Host, "/")
}
request, err := http.NewRequestWithContext(ctx, method, target+path, payload)
if err != nil {
return nil, err
}
if user := m.base.User; user != nil {
password, _ := user.Password()
request.SetBasicAuth(user.Username(), password)
}
if body != nil {
request.Header.Set("Content-Type", "application/json")
}
return request, nil
}
-96
View File
@@ -1,96 +0,0 @@
package broker_test
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"regexp"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/broker"
)
// The audit logger consumes everything and emits nothing. Its account must let it declare and read
// its own queue and read the events exchange to bind onto — and must not reach another module's
// queue, nor grant any write to the events exchange (novox/hq ADR 0043).
func TestAConsumerReadsItsOwnQueueAndTheEventsExchangeAndNoOthers(t *testing.T) {
// Rebuilt through CreateModuleAccount's own path by asking for the same scope it would apply.
// The queue this module reads:
mine := broker.ModuleQueueFor("anchor", "audit-logger")
other := broker.ModuleQueueFor("anchor", "plex")
read := scope(t, "read", "anchor", "audit-logger", nil, []string{"#"})
if !read.MatchString(mine) {
t.Error("the audit logger may not read its own queue, so it consumes nothing")
}
if !read.MatchString(broker.EventsExchangeName) {
t.Error("the audit logger may not read the events exchange, so it cannot bind onto it")
}
if read.MatchString(other) {
t.Error("the audit logger may read another module's queue")
}
// It emits nothing, so it is granted no write to the events exchange — only its own queue, to bind.
write := scope(t, "write", "anchor", "audit-logger", nil, []string{"#"})
if write.MatchString(broker.EventsExchangeName) {
t.Error("a pure consumer was granted write to the events exchange")
}
if !write.MatchString(mine) {
t.Error("the audit logger may not write to its own queue, so it cannot bind it")
}
}
// An emitter is granted the events exchange to write; a consumer is not.
func TestAnEmitterMayWriteTheEventsExchangeAndAConsumerMayNot(t *testing.T) {
emitter := scope(t, "write", "anchor", "umami", []string{"module.umami.site.created"}, nil)
if !emitter.MatchString(broker.EventsExchangeName) {
t.Error("an emitting module may not write the events exchange, so it cannot emit")
}
}
// scope reconstructs one of the three permission patterns CreateModuleAccount would apply, by
// reading it back from a captured request against a stub management API.
func scope(t *testing.T, which, node, module string, emits, consumes []string) *regexp.Regexp {
t.Helper()
pat := capturePermission(t, which, node, module, emits, consumes)
re, err := regexp.Compile(pat)
if err != nil {
t.Fatalf("the %s pattern does not compile: %v", which, err)
}
return re
}
// capturePermission runs CreateModuleAccount against a stub management API and returns the pattern
// it set for `which` ("configure"/"write"/"read"). The scope is tested where it is applied, not
// reconstructed by the test — so a change to the mapping cannot pass a test that hard-codes the old
// one.
func capturePermission(t *testing.T, which, node, module string, emits, consumes []string) string {
t.Helper()
var captured map[string]string
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if strings.HasPrefix(r.URL.Path, "/api/permissions/") {
_ = json.NewDecoder(r.Body).Decode(&captured)
}
w.WriteHeader(http.StatusNoContent)
}))
defer server.Close()
t.Setenv(broker.ManagementVar, server.URL)
m, err := broker.ManagementFromEnvironment()
if err != nil {
t.Fatalf("stub management not usable: %v", err)
}
if _, err := m.CreateModuleAccount(context.Background(), node, module, "pw", emits, consumes); err != nil {
t.Fatalf("CreateModuleAccount: %v", err)
}
if captured == nil {
t.Fatal("no permissions were set")
}
pattern, ok := captured[which]
if !ok {
t.Fatalf("no %s permission was set; got %v", which, captured)
}
return pattern
}
+589
View File
@@ -0,0 +1,589 @@
// Composing the bus's own configuration.
//
// An account is *composed*, never called for: the controller writes accounts, users and
// per-subject permissions into one file the host keeps current, and the server reloads it in
// place (novox/hq ADR 0106 — never through a management API; design 25 §4).
//
// Everything here is pure. Given the principals, it returns the file's text — so the whole of the
// mesh's authority model is testable as strings, with no server.
//
// **Permissions are per subject, so a module's own name is the server's to enforce.** ADR 0042
// reserves a module's origin — it publishes only under its own name — and here that is a refusal
// rather than something a library promises.
package broker
import (
"errors"
"fmt"
"regexp"
"sort"
"strings"
)
// A Kind is what a principal is, which decides the shape of its authority rather than its
// contents: a module's comes from its declaration, a host's from its node, and the controller's
// and the enrolment user's are fixed.
type Kind string
const (
KindModule Kind = "module"
KindNode Kind = "node"
KindController Kind = "controller"
KindEnrolment Kind = "enrolment"
// KindPerson is somebody reaching the mesh's tools from a workstation (design 25 §7). Its
// authority is a list of tools and nothing else — not control, not declarations, not builds,
// and no ability to answer anything, because a person asks.
KindPerson Kind = "person"
)
// Seat is a role on the bus as a principal relates to it: the subjects it accepts, and those it
// emits (novox/hq ADR 0118, design 29 §5).
type Seat struct {
Name string
Accepts []string
Emits []string
Serves []string
Versions []string // protocol versions served beside the current one; empty for v1 only
}
// A Principal is one user of the bus. Its permissions are derived from what it declares and
// nothing else (novox/hq ADR 0043), over the three namespaces of design 29 §2: its own, the seats
// it holds, and the seats it uses.
type Principal struct {
Kind Kind
Node string
Module string
Emits []string
Consumes []string
Serves []string
Holds []Seat
Uses []Seat
// Watches are seats whose events this principal consumes. Separate from Consumes because a
// role's event lives under the seat's namespace and not a module's, and this package cannot tell
// a seat's name from a module's by looking at it — whoever resolved the declaration can, and
// does (novox/hq ADR 0121).
//
// **Found by a consumer reading nothing.** The catalogue consumes the build machine's outcome;
// with that name read as a module's, its subscription pointed at `mesh.mod.mesh-build-machine.…`,
// a namespace no such module owns. Every service started and the graph stayed empty.
Watches []Seat
// Invokes are the tools a person may call, as `<module>.<tool>`; a single `*` is every tool,
// for an administrator. Only meaningful for KindPerson.
//
// **A list, not a role.** A person is not a module and holds no seat: nothing is addressed
// to them, nothing is delivered to them, and they have no durable consumer to acknowledge.
// What they have is permission to ask.
Invokes []string
// PasswordHash is the bcrypt hash the mesh minted. The plaintext is sealed to the principal
// and never appears here: this file is written to a node's disk and read by a server, and a
// secret that can be read from a configuration file is a secret with a wider blast radius
// than the one it protects (novox/hq design 29 §10).
PasswordHash string
}
// meshSeatsTheControllerUses are the roles the mesh's own flows submit work to. Named rather than
// derived from the seat set: the controller is not a module and declares no `uses`, so its side of a
// seat has to be stated, and a list is what makes "which roles does the mesh itself talk to" answerable.
var meshSeatsTheControllerUses = []string{"mesh-build-machine"}
// enrolmentPrefix is the space every enrolling node's user and inbox live under, so the one place the
// controller may answer an enrolment is derived from the same constant the user is named from.
const enrolmentPrefix = "enrol"
// safeSubject refuses anything that would change the meaning of a subject rather than sit inside
// one. A name carrying a dot would silently widen a permission by adding a token; a name carrying
// `>` or `*` would widen it to a wildcard, which is the whole authority model gone.
var safeSubject = regexp.MustCompile(`^[A-Za-z0-9_-]+$`)
// Username is how a principal is named to the server. The node is part of it, so the same module
// on two machines holds two users, each sealed to its own — the rule management.go already
// applies, kept.
func (p Principal) Username() string {
switch p.Kind {
case KindPerson:
return "person." + p.Module
case KindModule:
return p.Node + "." + p.Module
case KindNode:
return "node." + p.Node
case KindController:
return "controller"
case KindEnrolment:
// Per token, not one shared user. **The inbox is the reason**: with a single `enrolment`
// user every machine enrolling at once could read every other's answer, and an answer
// carries that node's credentials sealed to it. Design 25 §6 says the inbox a token
// derives, and a permission belongs to a user, so the user is per token.
//
// Named after the node, which **is** the token's id: a token is issued for a node record,
// the mesh holds one live claim per record, and the node's name is the one identifier both
// sides already have before anything else is agreed. It is also exactly what the other
// transport does, where the account is named after the node and the secret is its password.
return enrolmentPrefix + "." + p.Node
}
return ""
}
// inbox is a principal's own reply space. No user is ever granted a bare `_INBOX.>` (design 25
// §4): with one account, inbox privacy is the permission list or it is nothing, so each user's
// inbox is derived from its own identity and its permissions name that prefix and no other.
func (p Principal) inbox() string { return "_INBOX." + p.Username() + ".>" }
// Permissions is what a principal may publish and subscribe, and whether it may answer.
type Permissions struct {
Publish []string
Subscribe []string
// AllowResponses lets a principal reply to a request it received, on the reply subject that
// request carried, once.
//
// **This is what makes scoped inboxes possible at all**, and design 25 §4 did not say it. If
// every user's inbox is private to it, a module serving a tool cannot publish the answer —
// the answer goes to the *caller's* inbox, which the responder has no permission for. The two
// ways out are granting responders `_INBOX.>`, which is precisely the blanket grant §4
// refuses, or this: the server itself permits one reply to the subject of a message the user
// actually received, and nothing else. The authority is bounded by having been asked.
AllowResponses bool
}
// PermissionsFor derives a principal's authority. Pure, and the only place authority is decided:
// a permission that cannot be derived from a declaration is a permission nobody can explain.
func PermissionsFor(p Principal) (Permissions, error) {
for _, part := range []struct{ what, value string }{
{"node", p.Node}, {"module", p.Module},
} {
if part.value == "" {
continue
}
if !safeSubject.MatchString(part.value) {
return Permissions{}, fmt.Errorf(
"%q cannot be part of a subject: a permission is a subject pattern, and this would widen it", part.value)
}
}
var pub, sub []string
switch p.Kind {
case KindController:
// The controller owns the mesh's own traffic and the streams. It is the only writer of
// stream definitions (design 25 §3), so it alone reaches the JetStream API.
pub = []string{"mesh.control.>", "mesh.node.>", "$JS.API.>"}
// **And where its consumers deliver.** A push consumer delivers on `_DELIVER.<its name>`,
// and a client bound to it subscribes exactly that; the server refused it for every
// principal the first time one bound a consumer (2026-09-28). Each kind below is granted
// its own consumers' delivery subjects and no other's.
sub = []string{"mesh.control.>", "$JS.API.>", "_DELIVER." + ControllerName, "_DELIVER." + ControllerName + ".>"}
// Work the mesh's own flows submit to a role, and the outcomes they wait on (ADR 0121). A
// build is the one today: the controller asks, and reads the answer from the seat's event
// like the catalogue does — which is why no holder needs to publish into anybody's inbox.
for _, seat := range meshSeatsTheControllerUses {
pub = append(pub, "mesh.seat."+seat+".accept.>")
}
// Every module's tools: **the control plane is the way in** (novox/hq ADR 0095). A person
// or an agent asks through it and every question passes one process where an audit
// belongs — so it, alone among principals, may call any tool by name. The first `ask` on
// the new bus was refused the publish (2026-09-28).
pub = append(pub, "mesh.mod.*.tool.>")
// The two events it reacts to, and its ack subject on the stream they arrive from
// (streams.go). **Each named, not a pattern**: `mesh.mod.*.event.>` would make the
// controller a subscriber to every event in the mesh, and its permission list would stop
// saying what it is for. The ack grant below is scoped per stream because the controller's
// consumer name is the same on both and `$JS.ACK.CONTROL.controller.>` does not cover a
// delivery from EVENTS — a consumer that cannot ack has every message redelivered for
// ever, refused by the list it already has.
sub = append(sub, ControllerFollows...)
pub = append(pub, "$JS.ACK.EVENTS."+ControllerName+".>")
// **Where an enrolment's answer goes**, and `allow_responses` does not cover it. That
// permits one reply to the reply subject of a message the user received — and a message a
// JetStream consumer delivers has had that field claimed for the consumer's own ack address
// (design 25 §2), so the address the controller actually answers is the one the request
// carried in its payload, which is not a reply subject as the server understands it.
//
// Verified against a real server before this line existed: the answer was refused with
// "Permissions Violation for Publish to _INBOX.enrol.anchor…", and every enrolment on the
// mesh would have timed out while the controller logged success.
//
// **The enrolment inbox space, not a blanket `_INBOX.>`.** Design 25 §4 refuses that, and
// this is not it: nothing but an enrolling node ever subscribes under this prefix, each
// scoped to its own token's, so the controller publishing here is the mesh answering
// enrolments and can reach nothing else.
pub = append(pub, "_INBOX."+enrolmentPrefix+".>")
case KindPerson:
// Tools, and nothing else. Every subject a person may publish is a tool call; a person
// who could publish an event would be able to claim a module said something.
for _, t := range p.Invokes {
if t == "*" {
pub = append(pub, "mesh.mod.*.tool.>")
continue
}
module, tool, ok := strings.Cut(t, ".")
if !ok {
return Permissions{}, fmt.Errorf(
"%q does not name a tool: a person invokes <module>.<tool>, or * for every one", t)
}
pub = append(pub, "mesh.mod."+module+".tool."+tool)
}
case KindEnrolment:
// A leaked token is useless for anything but enrolling: it cannot read a declaration, hear
// an event, or subscribe any inbox but the one its own token derives (design 25 §6).
//
// **The inbox was missing and the handshake could not have completed without it.** An
// enrolling node publishes its request and waits on an address it states in the payload;
// with nothing to subscribe it waits out its timeout against a mesh that answered. Its own
// and no wider: `_INBOX.enrol.<node>.>`, so what is sealed to one machine cannot be read by
// another enrolling beside it.
if p.Node == "" {
// Refused rather than composed into `_INBOX.enrol..>`, which is a subject with an empty
// token in it — and worse, one every nameless enrolment user would share. A shared
// enrolment inbox is one machine able to read the credentials sealed to another.
return Permissions{}, errors.New(
"an enrolment user names no node, so its inbox would be shared with every other " +
"enrolment: a token is issued for a node record, and that record's name is " +
"the token's id")
}
pub = []string{"mesh.control.enrol"}
sub = []string{p.inbox()}
case KindNode:
// A host publishes its own node's control traffic and subscribes its own declaration —
// and nothing of any other node's.
// And binding to its consumer, which asks the server about it (CONSUMER.INFO) — the one
// thing the host does that nothing granted. Found the first time a machine dialled a
// permissioned server: "this node cannot read its declarations" (2026-09-28). The ack and
// the inbox are granted below with every principal's.
pub = []string{
"mesh.control." + p.Node + ".>",
"$JS.API.CONSUMER.INFO.NODES." + p.Node,
}
sub = []string{"mesh.node." + p.Node + ".declare", "_DELIVER." + p.Node}
case KindModule:
// 1. Its own namespace: it publishes its events there and serves its tools there. Nothing
// else may publish into it, so an event's source is a fact the server enforces rather
// than a claim in the body (design 29 §2).
own := "mesh.mod." + p.Module
for _, e := range p.Emits {
pub = append(pub, own+".event."+e)
}
// Every tool under its own name, not a list: the tools a module serves are what its code
// answers, and a second copy of that list in the manifest would be a second source of
// truth for the mesh to keep in step (2026-09-28: every module that served a tool was
// refused the subscription, because none had written the list twice). Nothing is given
// away — no other principal may subscribe this namespace, and a caller's authority is
// still granted per tool, by name, on the publish side.
sub = append(sub, own+".tool.>")
// 2. What it consumes, by the emitter's own subject — an event is addressed to its
// emitter, because the emitter's identity is the meaning (ADR 0118).
for _, c := range p.Consumes {
subject, err := consumedSubject(c)
if err != nil {
return Permissions{}, err
}
sub = append(sub, subject)
}
// 2b. Events of a role it watches, under the seat's own namespace. Subscribe only: watching a
// role is hearing what it announced, not taking part in it.
for _, w := range p.Watches {
for _, e := range w.Emits {
sub = append(sub, seatSubject(w, "event", e))
}
}
// 2c. Its own consumer, which it **pulls**: the runtime asks for the next message and is
// answered on its own inbox, so what it needs is to ask about the consumer and to ask it
// for messages — its own consumer's name, and no other's. Pulled rather than pushed
// because that is the one shape a runtime's client binds without creating anything; the
// controller and the hosts are pushed to. Named here rather than through ConsumerFor,
// which asks for these permissions to build the consumer and would ask forever. A
// subject for a consumer that turns out not to exist grants nothing anybody can use.
pub = append(pub,
"$JS.API.CONSUMER.INFO."+consumerStream(p)+"."+consumerDurable(p),
"$JS.API.CONSUMER.MSG.NEXT."+consumerStream(p)+"."+consumerDurable(p))
// 3. Seats it holds: full participation.
for _, s := range p.Holds {
// Taking work from the role's queue: the worker consumer it binds (asked about,
// delivered on, acknowledged), each on the seat's own stream. The first machine to
// take work over the new bus was refused the asking (2026-09-28).
worker := "SEAT_" + upperSnake(s.Name) + "_worker"
stream := seatStreamName(s.Name)
sub = append(sub, "_DELIVER."+worker)
pub = append(pub, "$JS.API.CONSUMER.INFO."+stream+"."+worker, "$JS.ACK."+stream+"."+worker+".>")
for _, a := range s.Accepts {
sub = append(sub, seatSubject(s, "accept", a))
}
for _, e := range s.Emits {
pub = append(pub, seatSubject(s, "event", e))
}
for _, t := range s.Serves {
sub = append(sub, seatSubject(s, "tool", t))
}
}
// 4. Seats it uses: publish only, and only the accepts half. A caller cannot subscribe a
// seat's inbound subject and watch other modules' traffic, nor publish its outbound
// events and lie about outcomes (design 29 §2).
for _, s := range p.Uses {
for _, a := range s.Accepts {
pub = append(pub, seatSubject(s, "accept", a))
}
for _, t := range s.Serves {
pub = append(pub, seatSubject(s, "tool", t))
}
}
}
if p.Kind == KindPerson {
// An inbox to hear answers in, and nothing else. No ack subject: a person has no durable
// consumer, because nothing is delivered to a person — they ask and are answered.
sub = append(sub, p.inbox())
}
if p.Kind == KindModule || p.Kind == KindNode || p.Kind == KindController {
// Its own reply space, and nothing wider.
sub = append(sub, p.inbox())
// Acking a JetStream delivery is a publish to that consumer's own ack address — a
// different subject from anything the consumer subscribes. Without it every message a
// module received would be redelivered forever, refused by the permission list it already
// has (design 25 §4). Scoped to this principal's own consumer name, so it can ack its own
// deliveries and no other's.
pub = append(pub, "$JS.ACK."+consumerStream(p)+"."+consumerDurable(p)+".>")
}
sort.Strings(pub)
sort.Strings(sub)
return Permissions{
Publish: pub,
Subscribe: sub,
// A module answers what it was asked — a tool call reaches it on its own namespace, so the
// authority is bounded by having been asked — and so does the controller. A node and a
// person are never asked anything, and are granted nothing here.
AllowResponses: p.Kind == KindModule || p.Kind == KindController,
}, nil
}
// seatSubject places a seat's verb under the kind of traffic it is.
//
// **The kind token is load-bearing, not decoration.** A stream is defined by a subject filter, so
// without it a stream over a seat or a module's namespace would capture that namespace's *tool*
// traffic too — and a tool call must never be persisted (design 25 §3: tools stay on core NATS).
// Found while defining the streams: the first draft of design 29 had one namespace per module
// with no kind, which reads well and cannot be filtered.
//
// A seat serving more than its current protocol version carries the version as a token
// (design 29 §8): the seat stays one role, and v1 and v2 run beside each other until nothing is
// bound to the old one.
func seatSubject(s Seat, kind, verb string) string {
return "mesh.seat." + s.Name + "." + kind + "." + verb
}
// consumerStream and consumerDurable are the two halves of a consumer's identity, and they are
// two functions because conflating them was a real bug.
//
// **A durable name may not contain a dot; an ack subject is built from two names that do.** The
// server acknowledges on `$JS.ACK.<stream>.<consumer>.…`, so a single string "EVENTS.one_audit"
// reads correctly inside the permission and is rejected as a consumer name — *nats: invalid
// consumer name*. Caught against a running server, and worth the comment because the shape of
// the failure if it had not been is the one design 25 §4 warns about: a consumer that cannot ack
// has every message redelivered forever, and its permission list looks right while it happens.
//
// They are derived here, beside the permission that must match them, because two places deriving
// the same name is how a module ends up unable to ack its own deliveries.
// consumedSubject is where a consumed event lands, from the local pattern a module declared.
//
// **The mesh's wildcards become this transport's** (design 29 §1): `*` is one name on both, and `**`
// — the rest — is `>` here. A module writes neither transport's spelling, so a manifest stays correct
// when the wire changes, which is the whole reason names are local.
//
// `**` on its own is every event from every module: the emitter is any, the event is anything. An
// audit logger wants exactly that and says so in one token.
func consumedSubject(pattern string) (string, error) {
if pattern == catalogueTheRest {
return "mesh.mod.*.event.>", nil
}
emitter, event, named := strings.Cut(pattern, ".")
if !named || emitter == "" || event == "" {
return "", fmt.Errorf(
"%q does not name an emitter and an event: a consumed event is <emitter>.<event>, or "+
"%q for every event", pattern, catalogueTheRest)
}
if emitter == catalogueTheRest {
return "", fmt.Errorf("%q stands for the rest of a name, so it cannot name the emitter", catalogueTheRest)
}
// Each name is checked before it becomes a subject: a name carrying a dot would add a token and
// silently widen the permission, which is the whole reason safeSubject exists.
var out []string
for _, part := range strings.Split(event, ".") {
switch part {
case catalogueTheRest:
out = append(out, ">")
case "*":
out = append(out, "*")
default:
if !safeSubject.MatchString(part) {
return "", fmt.Errorf("%q cannot be part of a subject: it would widen the permission", part)
}
out = append(out, part)
}
}
if emitter != "*" && !safeSubject.MatchString(emitter) {
return "", fmt.Errorf("%q cannot name an emitter: it would widen the permission", emitter)
}
return "mesh.mod." + emitter + ".event." + strings.Join(out, "."), nil
}
// catalogueTheRest is the mesh's wildcard for "the rest of a name", duplicated from the catalogue
// package for the one direction of dependency the build queue's name is duplicated for.
const catalogueTheRest = "**"
func consumerStream(p Principal) string {
switch p.Kind {
case KindModule:
return "EVENTS"
case KindNode:
return "NODES"
case KindController:
return "CONTROL"
}
return ""
}
func consumerDurable(p Principal) string {
switch p.Kind {
case KindModule:
return p.Node + "_" + p.Module
case KindNode:
return p.Node
case KindController:
return "controller"
}
return ""
}
// Server is everything the composed file needs that is not a principal.
type Server struct {
// ClientPort carries TLS itself. There is no plaintext port beside it: a bus reachable
// without TLS is one a module can reach without TLS by mistake.
ClientPort int
MonitoringPort int
TLSCert string
TLSKey string
TLSCA string
// StoreDir is a host directory bind, not a named volume — issue 115 is resolved and converted
// four modules away from named volumes; the bus's own data is not the place to bring one back.
StoreDir string
}
// Compose renders the server's whole configuration. The order is stable and the output is
// deterministic, because the file's digest is what the module's entrypoint watches to decide
// whether to reload: a composer that reordered a map on each run would signal a reload every time
// the controller restarted, for a file that had not changed.
func Compose(s Server, principals []Principal) (string, error) {
sorted := append([]Principal(nil), principals...)
sort.Slice(sorted, func(i, j int) bool { return sorted[i].Username() < sorted[j].Username() })
var b strings.Builder
b.WriteString("# Composed by the mesh controller. Do not edit: the next composition overwrites it.\n")
b.WriteString("# Accounts and permissions are derived from what each module declares and nothing\n")
b.WriteString("# else (novox/hq ADR 0043, design 29 §2).\n\n")
fmt.Fprintf(&b, "port: %d\n", s.ClientPort)
fmt.Fprintf(&b, "http: 127.0.0.1:%d\n\n", s.MonitoringPort)
// **No `verify`, and it said `verify: true` until this configuration was run.** That setting
// makes the server demand a *client* certificate, and nothing in the mesh presents one: a host
// pins this server's exact certificate and authenticates with the password the mesh minted
// (ADR 0004, design 25 §4), and so does a module's runtime. With it on, every connection in the
// mesh is refused at the TLS handshake before any password is looked at, and the error —
// "client didn't provide a certificate" — reads as a fault in the client.
//
// TLS is still required: the block is what requires it, and verify only decides whether client
// certificates are checked.
b.WriteString("tls {\n")
fmt.Fprintf(&b, " cert_file: %q\n", s.TLSCert)
fmt.Fprintf(&b, " key_file: %q\n", s.TLSKey)
fmt.Fprintf(&b, " ca_file: %q\n", s.TLSCA)
b.WriteString("}\n\n")
b.WriteString("jetstream {\n")
fmt.Fprintf(&b, " store_dir: %q\n", s.StoreDir)
b.WriteString("}\n\n")
accounts, err := ComposeAccounts(sorted)
if err != nil {
return "", err
}
b.WriteString(accounts)
return b.String(), nil
}
// ComposeAccounts is the accounts block alone — every user, and nothing about the server.
//
// **This is the only part of the configuration the mesh writes, and the split is deliberate.** A
// server's ports, its TLS paths and its store directory are properties of the container the module
// raises: they live in its image and its mounts, and they change when it does. The controller has no
// business knowing them, and a controller that did would have to be kept in step with a Dockerfile
// it never sees. What only the mesh knows is *who may connect*, so that is what it writes, and the
// module's own configuration includes it.
//
// Four things checked against a running server before this shape was committed to: a user in an
// included file authenticates; an unknown user is refused, so the include is the whole authority
// rather than an addition to something; a publish outside a user's grant is refused; and rewriting
// this file alone and signalling a reload makes a new user appear **without dropping the connection
// the mesh already has** — which is what makes every later account, permission or person change cost
// nothing (task 1.2's payoff).
func ComposeAccounts(principals []Principal) (string, error) {
sorted := append([]Principal(nil), principals...)
sort.Slice(sorted, func(i, j int) bool { return sorted[i].Username() < sorted[j].Username() })
var b strings.Builder
b.WriteString("# The mesh's users, composed by the controller. Do not edit: the next\n")
b.WriteString("# composition overwrites it. Permissions are derived from what each module\n")
b.WriteString("# declares and nothing else (novox/hq ADR 0043, design 29 §2).\n\n")
// One account for the mesh: accounts in NATS isolate subject spaces entirely, and the mesh is
// one space (design 25 §4). The cost of that — that permissions are the only isolation — is
// paid in the scoping of every inbox and every ack subject.
// JetStream is enabled per account once accounts exist at all: with only the global block set,
// a user in MESH is told "JetStream not enabled for account" the first time it binds a
// consumer, which is the first thing every host does (2026-09-28).
b.WriteString("accounts {\n MESH {\n jetstream: enabled\n users = [\n")
for _, p := range sorted {
perms, err := PermissionsFor(p)
if err != nil {
return "", err
}
if p.PasswordHash == "" {
return "", fmt.Errorf("%s has no password hash: a user without one is a user anybody is", p.Username())
}
fmt.Fprintf(&b, " { user: %q, password: %q, permissions: {\n", p.Username(), p.PasswordHash)
fmt.Fprintf(&b, " publish: { allow: [%s] }\n", quoted(perms.Publish))
fmt.Fprintf(&b, " subscribe: { allow: [%s] }\n", quoted(perms.Subscribe))
if perms.AllowResponses {
b.WriteString(" allow_responses: { max: 1, ttl: \"1m\" }\n")
}
b.WriteString(" } }\n")
}
b.WriteString(" ]\n }\n}\n")
return b.String(), nil
}
func quoted(values []string) string {
if len(values) == 0 {
return ""
}
out := make([]string, len(values))
for i, v := range values {
out[i] = fmt.Sprintf("%q", v)
}
return strings.Join(out, ", ")
}
+49
View File
@@ -0,0 +1,49 @@
package broker
import (
"flag"
"os"
"path/filepath"
"testing"
)
var update = flag.Bool("update", false, "rewrite the golden composition")
// The composed file is the mesh's whole authority model, so a change to it should be visible in a
// review rather than inferred from a diff of Go. The fixture is also the exact text checked
// against the real server's parser (`nats-server -t`), which is what says this syntax is the
// server's and not one we invented.
func TestTheComposedConfigMatchesTheGolden(t *testing.T) {
seat := Seat{Name: "telegram-sender", Accepts: []string{"send"}, Emits: []string{"delivered", "failed"}}
got, err := Compose(
Server{ClientPort: 4222, MonitoringPort: 8222, StoreDir: "/data",
TLSCert: "/tls/tls.crt", TLSKey: "/tls/tls.key", TLSCA: "/tls/ca.crt"},
[]Principal{
{Kind: KindController, PasswordHash: "$2a$11$cccccccccccccccccccccc"},
{Kind: KindEnrolment, Node: "one", PasswordHash: "$2a$11$eeeeeeeeeeeeeeeeeeeeee"},
{Kind: KindNode, Node: "one", PasswordHash: "$2a$11$nnnnnnnnnnnnnnnnnnnnnn"},
{Kind: KindModule, Node: "one", Module: "telegram", Holds: []Seat{seat},
Serves: []string{"status"}, PasswordHash: "$2a$11$tttttttttttttttttttttt"},
{Kind: KindModule, Node: "two", Module: "shop", Uses: []Seat{seat},
Emits: []string{"order.placed"}, PasswordHash: "$2a$11$ssssssssssssssssssssss"},
{Kind: KindModule, Node: "two", Module: "audit",
Consumes: []string{"shop.order.placed"}, PasswordHash: "$2a$11$aaaaaaaaaaaaaaaaaaaaaa"},
})
if err != nil {
t.Fatal(err)
}
golden := filepath.Join("testdata", "composed.conf")
if *update {
if err := os.WriteFile(golden, []byte(got), 0o644); err != nil {
t.Fatal(err)
}
return
}
want, err := os.ReadFile(golden)
if err != nil {
t.Fatal(err)
}
if got != string(want) {
t.Errorf("composition changed; re-run with -update and read the diff:\n%s", got)
}
}
+361
View File
@@ -0,0 +1,361 @@
package broker
import (
"slices"
"strings"
"testing"
)
func has(t *testing.T, subjects []string, want string) {
t.Helper()
for _, s := range subjects {
if s == want {
return
}
}
t.Fatalf("expected %q among %v", want, subjects)
}
func hasNot(t *testing.T, subjects []string, unwanted string) {
t.Helper()
for _, s := range subjects {
if s == unwanted {
t.Fatalf("did not expect %q among %v", unwanted, subjects)
}
}
}
// A module's authority comes from its declaration and nothing else (novox/hq ADR 0043).
func TestAModulePublishesOnlyWhatItEmits(t *testing.T) {
p := Principal{Kind: KindModule, Node: "one", Module: "billing",
Emits: []string{"order.placed"}, PasswordHash: "x"}
perms, err := PermissionsFor(p)
if err != nil {
t.Fatal(err)
}
has(t, perms.Publish, "mesh.mod.billing.event.order.placed")
hasNot(t, perms.Publish, "mesh.mod.billing.>")
hasNot(t, perms.Publish, "mesh.mod.shipping.event.order.placed")
}
// The gap AMQP left open — an emitter granted the events exchange whole — is closed by per-subject
// permissions. A module cannot publish under another module's name.
func TestAModuleCannotPublishUnderAnothersName(t *testing.T) {
perms, _ := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "billing",
Emits: []string{"order.placed"}, PasswordHash: "x"})
for _, p := range perms.Publish {
if strings.HasPrefix(p, "mesh.mod.") && !strings.HasPrefix(p, "mesh.mod.billing.") {
t.Fatalf("billing may publish %q, which is not its own namespace", p)
}
}
}
// A caller of a seat may publish what the seat accepts, and nothing else of it: not its outbound
// events, and not a subscription to its inbound queue (design 29 §2).
func TestUsingASeatIsPublishOnlyAndInboundOnly(t *testing.T) {
seat := Seat{Name: "telegram-sender", Accepts: []string{"send"}, Emits: []string{"delivered", "failed"}}
perms, _ := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "shop",
Uses: []Seat{seat}, PasswordHash: "x"})
has(t, perms.Publish, "mesh.seat.telegram-sender.accept.send")
hasNot(t, perms.Publish, "mesh.seat.telegram-sender.event.delivered")
hasNot(t, perms.Subscribe, "mesh.seat.telegram-sender.accept.send")
}
// The holder is the mirror image: it consumes what the seat accepts and publishes what it emits.
func TestHoldingASeatIsTheMirrorOfUsingIt(t *testing.T) {
seat := Seat{Name: "telegram-sender", Accepts: []string{"send"}, Emits: []string{"delivered", "failed"}}
perms, _ := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "telegram",
Holds: []Seat{seat}, PasswordHash: "x"})
has(t, perms.Subscribe, "mesh.seat.telegram-sender.accept.send")
has(t, perms.Publish, "mesh.seat.telegram-sender.event.delivered")
hasNot(t, perms.Publish, "mesh.seat.telegram-sender.accept.send")
}
// Without an ack permission a durable consumer never really consumes: every message it receives is
// redelivered forever, refused by the permission list it already has (design 25 §4).
func TestAModuleMayAckItsOwnDeliveriesAndNoOthers(t *testing.T) {
perms, _ := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "billing",
Consumes: []string{"shop.order.placed"}, PasswordHash: "x"})
has(t, perms.Publish, "$JS.ACK.EVENTS.one_billing.>")
hasNot(t, perms.Publish, "$JS.ACK.>")
hasNot(t, perms.Publish, "$JS.ACK.EVENTS.one_shop.>")
}
// With one account, inbox privacy is the permission list or it is nothing.
func TestAnInboxIsScopedToItsOwner(t *testing.T) {
perms, _ := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "billing", PasswordHash: "x"})
has(t, perms.Subscribe, "_INBOX.one.billing.>")
hasNot(t, perms.Subscribe, "_INBOX.>")
hasNot(t, perms.Subscribe, "_INBOX.one.shop.>")
}
// A responder answers on the caller's inbox, which it has no permission for. allow_responses is
// what makes a scoped inbox workable at all — the authority is bounded by having been asked. A
// module is asked on its own namespace and may answer; a node and a person are never asked.
func TestOnlyWhatCanBeAskedMayAnswer(t *testing.T) {
module, _ := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "audit",
Consumes: []string{"shop.order.placed"}, PasswordHash: "x"})
if !module.AllowResponses {
t.Fatal("a module cannot answer a tool call on its own namespace")
}
node, _ := PermissionsFor(Principal{Kind: KindNode, Node: "one", PasswordHash: "x"})
if node.AllowResponses {
t.Fatal("a node was granted the right to answer, and nothing asks a node anything")
}
}
// A module serves every tool under its own name, and no other module's.
func TestAModuleServesItsOwnNamespaceAndNoOthers(t *testing.T) {
p, _ := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "gitea", PasswordHash: "x"})
if !slices.Contains(p.Subscribe, "mesh.mod.gitea.tool.>") {
t.Fatalf("a module may not serve its own tools: %v", p.Subscribe)
}
for _, s := range p.Subscribe {
if strings.HasPrefix(s, "mesh.mod.") && !strings.HasPrefix(s, "mesh.mod.gitea.") {
t.Fatalf("a module may subscribe another's namespace: %s", s)
}
}
}
// A host reaches its own node's control traffic and its own declaration, and nothing of any
// other node's.
func TestAHostIsConfinedToItsOwnNode(t *testing.T) {
perms, _ := PermissionsFor(Principal{Kind: KindNode, Node: "one", PasswordHash: "x"})
has(t, perms.Publish, "mesh.control.one.>")
has(t, perms.Subscribe, "mesh.node.one.declare")
hasNot(t, perms.Subscribe, "mesh.node.two.declare")
hasNot(t, perms.Subscribe, "mesh.node.>")
}
// A leaked enrolment token is useless for anything but enrolling (design 25 §6).
func TestTheEnrolmentUserCanOnlyEnrol(t *testing.T) {
perms, err := PermissionsFor(Principal{Kind: KindEnrolment, Node: "anchor", PasswordHash: "x"})
if err != nil {
t.Fatal(err)
}
if len(perms.Publish) != 1 || perms.Publish[0] != "mesh.control.enrol" {
t.Fatalf("enrolment may publish %v", perms.Publish)
}
// Its own inbox and nothing else. **Nothing else** is the point: no declaration, no event, and
// no other machine's answer — and the inbox itself is needed, because a node that cannot
// subscribe one waits out its timeout against a mesh that answered.
if len(perms.Subscribe) != 1 || perms.Subscribe[0] != "_INBOX.enrol.anchor.>" {
t.Fatalf("enrolment may subscribe %v, which is not its own inbox alone", perms.Subscribe)
}
}
// An enrolment user that names no node is refused: its inbox would be an empty subject token, and
// one that every nameless enrolment user shared — which is one machine reading the credentials
// sealed to another.
func TestAnEnrolmentUserWithoutANodeIsRefused(t *testing.T) {
if _, err := PermissionsFor(Principal{Kind: KindEnrolment, PasswordHash: "x"}); err == nil {
t.Fatal("an enrolment user with no node was composed, so its inbox is shared")
}
}
// A name that would widen a permission is refused rather than quietly stretching one.
func TestANameThatWouldWidenAPermissionIsRefused(t *testing.T) {
for _, bad := range []string{"bill.ing", "billing.>", "*", "bil>ling"} {
if _, err := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: bad, PasswordHash: "x"}); err == nil {
t.Fatalf("%q was accepted as part of a subject", bad)
}
}
}
// The entrypoint reloads on the file's digest changing, so an unchanged mesh must compose an
// identical file — otherwise every controller restart signals a reload of the whole bus.
func TestComposingTwiceGivesTheSameBytes(t *testing.T) {
s := Server{ClientPort: 4222, MonitoringPort: 8222, StoreDir: "/data",
TLSCert: "/tls/tls.crt", TLSKey: "/tls/tls.key", TLSCA: "/tls/ca.crt"}
ps := []Principal{
{Kind: KindModule, Node: "two", Module: "shop", Emits: []string{"order.placed"}, PasswordHash: "b"},
{Kind: KindController, PasswordHash: "c"},
{Kind: KindModule, Node: "one", Module: "billing", Consumes: []string{"shop.order.placed"}, PasswordHash: "a"},
}
first, err := Compose(s, ps)
if err != nil {
t.Fatal(err)
}
shuffled := []Principal{ps[2], ps[0], ps[1]}
second, err := Compose(s, shuffled)
if err != nil {
t.Fatal(err)
}
if first != second {
t.Fatal("composition is order-dependent; every controller restart would reload the bus")
}
}
// A user without a password is a user anybody is.
func TestAUserWithoutAPasswordIsRefused(t *testing.T) {
_, err := Compose(Server{ClientPort: 4222}, []Principal{{Kind: KindController}})
if err == nil {
t.Fatal("composed a user with no password hash")
}
}
// A person reaches the mesh's tools from a workstation (design 25 §7). Their authority is a list
// of tools and nothing else.
func TestAPersonMayAskOnlyTheToolsTheyWereGiven(t *testing.T) {
perms, err := PermissionsFor(Principal{Kind: KindPerson, Module: "jo",
Invokes: []string{"shop.price", "telegram.status"}, PasswordHash: "x"})
if err != nil {
t.Fatal(err)
}
has(t, perms.Publish, "mesh.mod.shop.tool.price")
has(t, perms.Publish, "mesh.mod.telegram.tool.status")
hasNot(t, perms.Publish, "mesh.mod.shop.tool.refund")
hasNot(t, perms.Publish, "mesh.mod.*.tool.>")
}
// An administrator gets every tool, which is a different grant and looks like one.
func TestAnAdministratorMayAskAnyTool(t *testing.T) {
perms, _ := PermissionsFor(Principal{Kind: KindPerson, Module: "jo",
Invokes: []string{"*"}, PasswordHash: "x"})
has(t, perms.Publish, "mesh.mod.*.tool.>")
}
// **Nothing but tools.** A person who could publish an event would be able to claim a module
// said something; one who could publish control traffic would be a second controller.
func TestAPersonReachesNothingButTools(t *testing.T) {
perms, _ := PermissionsFor(Principal{Kind: KindPerson, Module: "jo",
Invokes: []string{"*"}, PasswordHash: "x"})
for _, p := range perms.Publish {
if !strings.Contains(p, ".tool.") {
t.Errorf("a person may publish %q, which is not a tool call", p)
}
}
for _, s := range perms.Subscribe {
if !strings.HasPrefix(s, "_INBOX.person.") {
t.Errorf("a person may subscribe %q; only their own inbox should be reachable", s)
}
}
}
// A person has no durable consumer, because nothing is delivered to a person — so no ack
// subject, and an ack permission would be authority over something that does not exist.
func TestAPersonHasNoAckSubject(t *testing.T) {
perms, _ := PermissionsFor(Principal{Kind: KindPerson, Module: "jo",
Invokes: []string{"*"}, PasswordHash: "x"})
for _, p := range perms.Publish {
if strings.HasPrefix(p, "$JS.ACK") {
t.Errorf("a person was granted %q, and has no consumer to acknowledge", p)
}
}
}
// A person asks and is answered; they never answer. allow_responses would let a person reply to
// a request — which, on a bus where anyone may serve a tool, is somebody impersonating a module.
func TestAPersonMayNotAnswer(t *testing.T) {
perms, _ := PermissionsFor(Principal{Kind: KindPerson, Module: "jo",
Invokes: []string{"*"}, PasswordHash: "x"})
if perms.AllowResponses {
t.Fatal("a person may answer a request, which is impersonating a module")
}
}
// Two people do not share an inbox, or one would read the other's answers.
func TestTwoPeopleDoNotShareAnInbox(t *testing.T) {
a, _ := PermissionsFor(Principal{Kind: KindPerson, Module: "jo", Invokes: []string{"*"}, PasswordHash: "x"})
b, _ := PermissionsFor(Principal{Kind: KindPerson, Module: "sam", Invokes: []string{"*"}, PasswordHash: "x"})
if a.Subscribe[0] == b.Subscribe[0] {
t.Fatalf("both read %s", a.Subscribe[0])
}
}
// A malformed grant is refused rather than widened into something that happens to parse.
func TestAToolGrantThatNamesNoToolIsRefused(t *testing.T) {
if _, err := PermissionsFor(Principal{Kind: KindPerson, Module: "jo",
Invokes: []string{"shop"}, PasswordHash: "x"}); err == nil {
t.Fatal("a grant naming a module but no tool was accepted")
}
}
// The controller can answer an enrolment, and reach no other inbox.
//
// **`allow_responses` does not cover this and that is the trap.** It permits one reply to the reply
// subject of a message the user received — and a message a JetStream consumer delivers has had that
// field claimed for the consumer's own ack address, so the address the controller actually answers is
// the one the request carried in its payload, which the server does not recognise as a reply subject
// at all.
//
// Found against a real server, after a live test on an *unpermissioned* one had passed: every
// enrolment on the mesh would have timed out while the controller logged success.
func TestTheControllerCanAnswerAnEnrolmentAndReachNoOtherInbox(t *testing.T) {
ctl, err := PermissionsFor(Principal{Kind: KindController, PasswordHash: "x"})
if err != nil {
t.Fatal(err)
}
enrolling, err := PermissionsFor(Principal{Kind: KindEnrolment, Node: "anchor", PasswordHash: "x"})
if err != nil {
t.Fatal(err)
}
// Whatever the enrolling node waits on, the controller must be able to publish to.
if len(enrolling.Subscribe) != 1 {
t.Fatalf("an enrolling node subscribes %v, and this test knows only how to check one",
enrolling.Subscribe)
}
waitsOn := enrolling.Subscribe[0]
if !covers(ctl.Publish, waitsOn) {
t.Fatalf("the controller may publish %v, none of which reaches %s — so every enrolment on "+
"the mesh times out while the controller logs success", ctl.Publish, waitsOn)
}
// And nothing wider. A node's own inbox and a module's are not the controller's to write into:
// that is the blanket grant design 25 §4 refuses.
for _, other := range []string{"_INBOX.node.anchor.x", "_INBOX.one.shop.x", "_INBOX.person.ada.x"} {
if covers(ctl.Publish, other) {
t.Errorf("the controller can publish to %s, which is an inbox privacy the permission "+
"list is the only thing protecting", other)
}
}
}
// covers says whether any granted subject pattern admits one concrete subject, with NATS's own
// wildcard meanings: `*` is one token, `>` is the rest.
func covers(granted []string, subject string) bool {
want := strings.Split(subject, ".")
for _, pattern := range granted {
if admits(strings.Split(pattern, "."), want) {
return true
}
}
return false
}
func admits(pattern, subject []string) bool {
for i, token := range pattern {
if token == ">" {
return i < len(subject)
}
if i >= len(subject) {
return false
}
if token != "*" && token != subject[i] {
return false
}
}
return len(pattern) == len(subject)
}
// A module pulls its own consumer — asks about it, asks it for messages — and no other module's.
func TestAModulePullsItsOwnConsumerAndNoOthers(t *testing.T) {
p, _ := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "audit",
Consumes: []string{"shop.order.placed"}, PasswordHash: "x"})
for _, want := range []string{"$JS.API.CONSUMER.INFO.EVENTS.one_audit", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.one_audit"} {
if !slices.Contains(p.Publish, want) {
t.Errorf("a module cannot bind its own consumer: %v lacks %s", p.Publish, want)
}
}
for _, s := range p.Publish {
if strings.Contains(s, "CONSUMER.") && !strings.HasSuffix(s, ".one_audit") {
t.Errorf("a module may reach another consumer: %s", s)
}
}
for _, s := range p.Subscribe {
if strings.HasPrefix(s, "_DELIVER.") {
t.Errorf("a module is granted a push delivery it never binds: %s", s)
}
}
}
+83
View File
@@ -0,0 +1,83 @@
package broker
import (
"fmt"
"strings"
"github.com/novox/mesh-controller/internal/envfile"
)
// Whether this mesh's own traffic is on the bus being built.
//
// **One switch, read in one place** (novox/hq ADR 0116 step 5). Every seam the bus change went
// behind ships both implementations, and until the rollout every one of them chooses the bus the
// mesh runs on today. This is what the rollout flips, and it is deliberately a single fact rather
// than a fact per component: a controller whose outbound is on one bus and whose inbound is on the
// other is a mesh that hears nothing, and no test of either half would catch it.
// NATSVar is where the controller finds the bus being built. Unset is the ordinary case and means
// the mesh runs on the bus it has always run on.
const NATSVar = "MESH_BUS_NATS"
// OnNATS is the address of the bus being built, and whether the mesh is on it.
//
// Read from the node's own settings rather than baked in, for the reason the broker's address is
// (novox/hq 04-ISSUES/102): an address recorded once does not follow a node's ports.
func OnNATS() (address string, on bool, err error) {
address, err = envfile.Placed(NATSVar)
if err != nil {
return "", false, err
}
address = strings.TrimSpace(address)
if address == "" {
return "", false, nil
}
return address, true, nil
}
// CredentialIn reads the user and password out of a bus address, and the address without them.
//
// The controller's own credential arrives in its address, the way the old bus's does. Split out so the
// controller can record a hash of what it is actually using: its user is created by the installer at a
// bootstrap password, before the controller exists to mint one, and a composition that left itself out
// would produce a bus the writer cannot connect to.
func CredentialIn(address string) (user, password, bare string) {
at := strings.LastIndex(address, "@")
if at < 0 {
return "", "", address
}
scheme := ""
rest := address[:at]
if i := strings.Index(rest, "://"); i >= 0 {
scheme, rest = rest[:i+3], rest[i+3:]
}
user, password, _ = strings.Cut(rest, ":")
return user, password, scheme + address[at+1:]
}
// BareAddress is a bus address with any credential stripped, for something that only needs to know
// whether a server is answering there.
func BareAddress(address string) string {
_, _, bare := CredentialIn(address)
if bare == "" {
return address
}
if strings.Contains(bare, "://") {
return bare
}
return "nats://" + bare
}
// BusAddress is where the mesh's bus is, with this process's credential, and refuses to be empty:
// there is one bus, and a control plane without it can hold records and answer nothing (novox/hq
// ADR 0131, design 28 task 5.5).
func BusAddress() (string, error) {
address, _, err := OnNATS()
if err != nil {
return "", err
}
if address == "" {
return "", fmt.Errorf("this control plane has no %s, so it cannot reach the mesh's bus", NATSVar)
}
return address, nil
}
+28
View File
@@ -0,0 +1,28 @@
package broker
import (
"testing"
)
// Which bus the mesh is on is one fact, and being told about both is refused.
//
// The controller's own credential arrives in its address, and has to be readable out of it — its user
// is created by the installer at a bootstrap password, before the controller exists to mint one.
func TestACredentialIsReadOutOfABusAddress(t *testing.T) {
for _, c := range []struct{ in, user, password, bare string }{
{"nats://controller:secret@127.0.0.1:4222", "controller", "secret", "nats://127.0.0.1:4222"},
{"controller:secret@127.0.0.1:4222", "controller", "secret", "127.0.0.1:4222"},
{"nats://127.0.0.1:4222", "", "", "nats://127.0.0.1:4222"},
{"127.0.0.1:4222", "", "", "127.0.0.1:4222"},
// A password containing an at-sign: split on the last one, or the address becomes part of the
// credential and the connection goes somewhere nobody named.
{"nats://controller:a@b@127.0.0.1:4222", "controller", "a@b", "nats://127.0.0.1:4222"},
} {
user, password, bare := CredentialIn(c.in)
if user != c.user || password != c.password || bare != c.bare {
t.Errorf("%q read as %q/%q at %q; wanted %q/%q at %q",
c.in, user, password, bare, c.user, c.password, c.bare)
}
}
}
+73
View File
@@ -0,0 +1,73 @@
package broker
import "fmt"
// Bringing the bus's own objects into being, in the one order that works.
//
// **Asserted on every start rather than created once at genesis.** A stream somebody deleted, a mesh
// raised from a restored backup, or a bus whose data directory was replaced all have records and no
// objects — and a node whose consumer is missing hears nothing while everything else about it looks
// correct. Idempotence is the whole requirement, and the parts are already idempotent; this is the
// order they have to be asked in.
// Raiser is everything asserting the bus's objects needs of a connection to it.
type Raiser interface {
Asserter
Ensurer
}
// Raise asserts the mesh's streams, the controller's own consumers, and one consumer per node.
//
// **The order is not a preference.** A consumer on a stream that does not exist is refused, and the
// refusal names the stream rather than the order — so somebody reading it goes looking for a deleted
// stream instead of a reversed pair of lines. Nodes last, because the one a node reads lives on a
// stream the mesh's own set defines.
func Raise(r Raiser, nodes []string) error {
if err := AssertMeshStreams(r); err != nil {
return err
}
if err := AssertMeshConsumers(r); err != nil {
return err
}
if err := AssertNodeConsumers(r, nodes); err != nil {
return err
}
return nil
}
// RaiseSeats asserts one work queue per declared seat, and the worker of whoever holds it.
//
// Separate from Raise because it is answered by a different question: the mesh's own objects exist
// because the mesh does, and a seat's exist because a module declaring one was registered. Kept
// beside it so the order is visible — a holder's worker needs the seat's stream, and a seat's stream
// needs nothing.
func RaiseSeats(r Raiser, seats []DeclaredSeat, holders map[string]Holder) error {
for _, s := range SeatStreams(seats) {
if err := r.EnsureStream(s); err != nil {
return fmt.Errorf("asserting the work queue for %s: %w", s.Name, err)
}
}
for _, s := range seats {
h, held := holders[s.Name]
if !held {
// **The stream exists and the consumer does not, on purpose.** Work queues until a
// holder appears, so installing the module a week after something started sending to it
// flushes the backlog instead of having lost it.
continue
}
c, needed := HolderConsumerFor(h.Node, h.Module, s)
if !needed {
continue
}
if err := r.EnsureConsumer(c); err != nil {
return fmt.Errorf("asserting how %s on %s works %s: %w", h.Module, h.Node, s.Name, err)
}
}
return nil
}
// Holder is which module on which machine holds a seat.
type Holder struct {
Node string
Module string
}
+196
View File
@@ -0,0 +1,196 @@
package broker
import (
"os"
"testing"
"github.com/nats-io/nats.go"
)
// Raising the bus's objects against a real server.
//
// The pure tests above say what is asked for and in what order. Only a server can say whether it
// accepts them — and two of these are claims about the server's own behaviour that nothing else
// could answer: that asserting twice changes nothing, and that a consumer really is bound to the one
// subject its node is allowed to read.
//
// docker run -d --rm --name t -p 14227:4222 nats:2.10-alpine -js
// MESH_TEST_NATS=nats://127.0.0.1:14227 go test ./internal/broker/ -run TestRaising
func aLiveBus(t *testing.T) *JetStream {
t.Helper()
url := os.Getenv("MESH_TEST_NATS")
if url == "" {
t.Skip("MESH_TEST_NATS unset")
}
js, err := Dial(url)
if err != nil {
t.Fatal(err)
}
t.Cleanup(js.Close)
// **Nothing is deleted here, deliberately.** These objects are the mesh's own and every live
// test in every package shares one server: a test that deleted a stream to get a clean slate
// took it out from under whatever was running beside it, and the failure landed in the other
// test as "stream not found" — which reads as a bug in the code under test. Raise is idempotent
// by requirement, so asserting against whatever is already there is both safe and the realistic
// case.
return js
}
// Every object the mesh's own traffic needs, accepted by a real server, and asserting again changes
// nothing — which is the whole requirement, because this runs on every start.
func TestRaisingTheBusIsAcceptedAndIdempotent(t *testing.T) {
js := aLiveBus(t)
if err := Raise(js, []string{"anchor", "laptop"}); err != nil {
t.Fatalf("a real server refused the mesh's own objects: %v", err)
}
// Twice, with nothing in between. A start that failed the second time is a controller that
// cannot restart.
if err := Raise(js, []string{"anchor", "laptop"}); err != nil {
t.Fatalf("asserting the bus's objects a second time failed, so a restart would: %v", err)
}
// And again with a machine that was not there before, which is what enrolling one is.
if err := Raise(js, []string{"anchor", "laptop", "workstation"}); err != nil {
t.Fatalf("a machine joining an already-raised bus was refused: %v", err)
}
for _, s := range MeshStreams() {
if _, err := js.Context().StreamInfo(s.Name); err != nil {
t.Errorf("stream %s is not there: %v", s.Name, err)
}
}
for _, c := range MeshConsumers() {
if _, err := js.Context().ConsumerInfo(c.Stream, c.Name); err != nil {
t.Errorf("the controller's consumer on %s is not there: %v", c.Stream, err)
}
}
for _, node := range []string{"anchor", "laptop", "workstation"} {
info, err := js.Context().ConsumerInfo("NODES", node)
if err != nil {
t.Errorf("%s has no way to hear its declaration: %v", node, err)
continue
}
// **Its own subject and no other node's.** A consumer filtered on anything wider is a node
// reading another machine's declaration, and its own ack grant would not cover it either.
if info.Config.FilterSubject != "mesh.node."+node+".declare" {
t.Errorf("%s's consumer reads %q", node, info.Config.FilterSubject)
}
if info.Config.AckPolicy != nats.AckExplicitPolicy {
t.Errorf("%s's consumer acknowledges on delivery, so a declaration it died applying is "+
"never sent again", node)
}
}
}
// The store window needs unlimited redelivery on CONTROL: the bound belongs to the controller, and a
// server that dead-lettered first would discard the push the stream exists to protect.
func TestTheControlConsumerDoesNotDeadLetterBeforeTheControllerGivesUp(t *testing.T) {
js := aLiveBus(t)
if err := Raise(js, nil); err != nil {
t.Fatal(err)
}
info, err := js.Context().ConsumerInfo("CONTROL", ControllerName)
if err != nil {
t.Fatal(err)
}
if info.Config.MaxDeliver > 0 {
t.Fatalf("max-deliver is %d: a push held through a store restart would be dead-lettered "+
"before the controller finished deciding about it", info.Config.MaxDeliver)
}
}
// A role's work queue exists before anybody holds it, against a real server.
//
// **The queue before the holder is the point** (novox/hq ADR 0121): work queues until somebody arrives
// to do it, so assigning a build machine a week after something started asking for builds flushes the
// backlog instead of having lost it. A stream created at assignment would make "the holder is not here
// yet" mean "your requests are gone".
func TestRaisingAMeshRolesWorkQueue(t *testing.T) {
js := aLiveBus(t)
seats := []DeclaredSeat{{Name: "mesh-build-machine", Accepts: []string{"build"},
Emits: []string{"built"}}}
t.Cleanup(func() { _ = js.Context().DeleteStream("SEAT_MESH_BUILD_MACHINE") })
if err := RaiseSeats(js, seats, nil); err != nil {
t.Fatalf("a real server refused a role's work queue: %v", err)
}
info, err := js.Context().StreamInfo("SEAT_MESH_BUILD_MACHINE")
if err != nil {
t.Fatalf("the role has no work queue: %v", err)
}
if info.Config.Retention != nats.WorkQueuePolicy {
t.Errorf("the queue retains as %v: work a holder took must leave it, or the next holder does "+
"it again", info.Config.Retention)
}
if len(info.Config.Subjects) != 1 || info.Config.Subjects[0] != "mesh.seat.mesh-build-machine.accept.>" {
t.Errorf("it carries %v rather than the role's own inbound subjects", info.Config.Subjects)
}
// Nobody holds it, so there is no worker — and asserting again changes nothing, because this runs
// on every start.
if err := RaiseSeats(js, seats, nil); err != nil {
t.Fatalf("asserting a role's queue a second time failed, so a restart would: %v", err)
}
// And once somebody holds it, the worker appears on that same queue.
if err := RaiseSeats(js, seats, map[string]Holder{
"mesh-build-machine": {Node: "anchor", Module: "builder"},
}); err != nil {
t.Fatal(err)
}
if _, err := js.Context().ConsumerInfo("SEAT_MESH_BUILD_MACHINE",
"SEAT_MESH_BUILD_MACHINE_worker"); err != nil {
t.Fatalf("the holder got no worker on the role's queue: %v", err)
}
}
// **A consumer created after the fact still sees what came before it**, which is why the mesh needs no
// catch-up at all on this bus (novox/hq 04-ISSUES/050).
//
// On the bus the mesh runs on today a queue receives only what is published after it is bound, so
// everything built before the catalogue existed was announced to nobody — and on a fresh mesh that is
// always the foundation, because those are the things the catalogue needed in order to exist. A whole
// mechanism was built for it: the catalogue asks, the controller re-publishes.
//
// A stream is a log and a consumer is a position in it. A consumer created later starts at the
// beginning by default, so the builds are simply there. Asked of a real server rather than assumed,
// because the whole decision about whether to keep that mechanism rests on it.
func TestAConsumerCreatedAfterwardsStillSeesWhatCameBefore(t *testing.T) {
js := aLiveBus(t)
if err := AssertMeshStreams(js); err != nil {
t.Fatal(err)
}
if err := js.Context().PurgeStream("EVENTS"); err != nil {
t.Fatal(err)
}
// Genesis: things are built before anything is listening.
built := []string{"base", "store", "mesh-catalog"}
for _, m := range built {
if _, err := js.Context().Publish("mesh.seat.mesh-build-machine.event.built",
[]byte(`{"module":"`+m+`"}`)); err != nil {
t.Fatal(err)
}
}
// Now the catalogue is installed and the controller creates its consumer.
c, ok := ConsumerFor(Principal{Kind: KindModule, Node: "one", Module: "mesh-catalog",
Watches: []Seat{{Name: "mesh-build-machine", Emits: []string{"built"}}}, PasswordHash: "x"})
if !ok {
t.Fatal("a module that watches a role got no consumer")
}
t.Cleanup(func() { _ = js.Context().DeleteConsumer(c.Stream, c.Name) })
if err := js.EnsureConsumer(c); err != nil {
t.Fatal(err)
}
info, err := js.Context().ConsumerInfo(c.Stream, c.Name)
if err != nil {
t.Fatal(err)
}
if info.NumPending != uint64(len(built)) {
t.Fatalf("a consumer created after %d builds has %d waiting for it — if this is 0 the mesh "+
"does need a catch-up after all, and the reasoning for deleting it is wrong",
len(built), info.NumPending)
}
}
+139
View File
@@ -0,0 +1,139 @@
package broker
import (
"fmt"
"sort"
"strings"
)
// Whether a mesh could move its bus, and what is missing if not.
//
// **Asked before anything moves, and answerable from records alone.** The rollout moves every node at
// once (novox/hq ADR 0116 step 5), so there is no partial state to inspect afterwards and no half to
// roll back: either the mesh was ready or it was not. That makes a readiness question the most
// valuable thing here — it costs nothing, it can be asked of a running mesh any number of times, and
// every answer is a thing somebody can go and fix.
//
// Deliberately pure. It is handed what the mesh knows and returns sentences; nothing here connects to
// anything, so it can be asked on a workstation about a mesh it has never reached.
// Readiness is what the mesh knows about its own ability to move.
type Readiness struct {
// TheBus is the address the mesh's own traffic would move to, empty when nothing names one.
TheBus string
// ServerStanding is whether a bus is reachable at that address, as somebody checked.
ServerStanding bool
// Holder is the node running the module that holds the bus seat, empty when nothing does.
Holder string
// AccountsComposed is whether that node has been sent the composed user list.
AccountsComposed bool
// Nodes is every machine the mesh knows.
Nodes []string
// Credentialled is which of them has a credential for the new bus.
Credentialled map[string]bool
// Modules is every assigned module, as `<node>/<module>`.
Modules []string
// ModuleCredentialled is which of those has one.
ModuleCredentialled map[string]bool
}
// NotReady is every reason this mesh cannot move its bus yet, in the order somebody would fix them.
//
// Empty means ready. **Each entry names one thing and what to do about it**, because a readiness check
// that says "not ready" is a check nobody can act on — and this is read at the point where the next
// step is irreversible.
func NotReady(r Readiness) []string {
var why []string
if strings.TrimSpace(r.TheBus) == "" {
why = append(why, "nothing names the bus to move to: set "+NATSVar+" on the control node "+
"to the address the new server answers on")
}
if !r.ServerStanding {
why = append(why, "no bus is answering at that address. Step 2 of the change raises it beside "+
"the one the mesh is on, carrying nothing — assign the module that holds "+
"mesh-broker and push the machine that runs it")
}
if r.Holder == "" {
why = append(why, "no machine holds mesh-broker, so nothing would compose the bus's user "+
"list. Assign the module that claims it")
} else if !r.AccountsComposed {
why = append(why, fmt.Sprintf(
"%s holds mesh-broker and has not been sent the composed user list, so the bus would "+
"refuse every connection. `push %s`", r.Holder, r.Holder))
}
// A node with no credential cannot come back after the move, and a node that cannot come back is
// a machine the mesh has lost until somebody goes to it.
var missing []string
for _, n := range r.Nodes {
if !r.Credentialled[n] {
missing = append(missing, n)
}
}
sort.Strings(missing)
if len(missing) > 0 {
why = append(why, fmt.Sprintf(
"%d machine(s) have no credential for the new bus and would not come back: %s. Each needs "+
"one minted before the move, not after — after, there is no bus to ask over",
len(missing), strings.Join(missing, ", ")))
}
// A module without one keeps running and stops being reachable, which is a smaller fault and still
// one somebody should choose rather than discover.
var quiet []string
for _, m := range r.Modules {
if !r.ModuleCredentialled[m] {
quiet = append(quiet, m)
}
}
sort.Strings(quiet)
if len(quiet) > 0 {
why = append(why, fmt.Sprintf(
"%d module(s) have no credential for the new bus: %s. Each keeps serving and stops "+
"answering tools and hearing events until it is issued one",
len(quiet), strings.Join(quiet, ", ")))
}
return why
}
// WhatMoves is what the rollout would do, in order, for somebody reading before they commit.
//
// **Written out rather than summarised.** This is the one step with nothing to inspect afterwards, so
// the last useful moment to disagree with it is while reading this.
func WhatMoves(r Readiness) []string {
out := []string{
fmt.Sprintf("compose the bus's user list and send it to %s", holderOr(r.Holder)),
fmt.Sprintf("move this control plane to %s, and confirm it is heard", busOr(r.TheBus)),
}
nodes := append([]string(nil), r.Nodes...)
sort.Strings(nodes)
for _, n := range nodes {
out = append(out, fmt.Sprintf("move %s, and confirm it reports", n))
}
if len(r.Modules) > 0 {
out = append(out, fmt.Sprintf("move %d module runtime(s), and confirm each answers",
len(r.Modules)))
}
// **The old broker goes, and it goes last** (novox/hq ADR 0131). AMQP is not a provision, so once
// every machine reports on the new bus nothing of the mesh is left speaking to it, and its module
// is unassigned. Said as a step so nobody reads the move as leaving a second bus behind.
out = append(out, "then unassign the old broker's module: AMQP is not a provision (ADR 0131), and "+
"once every machine reports on the new bus nothing of the mesh speaks to it")
return out
}
func holderOr(node string) string {
if node == "" {
return "whichever machine holds mesh-broker"
}
return node
}
func busOr(address string) string {
if address == "" {
return "the new bus"
}
return address
}
+98
View File
@@ -0,0 +1,98 @@
package broker
import (
"strings"
"testing"
)
// Whether a mesh could move its bus.
//
// Every case here is a way of moving that leaves something behind, and the one that matters most is a
// machine with no credential: after the move there is no bus to ask it over, so it is lost until
// somebody walks to it.
func aMeshReadyToMove() Readiness {
return Readiness{
TheBus: "nats://127.0.0.1:5671", ServerStanding: true,
Holder: "anchor", AccountsComposed: true,
Nodes: []string{"anchor", "laptop"},
Credentialled: map[string]bool{"anchor": true, "laptop": true},
Modules: []string{"anchor/gitea"},
ModuleCredentialled: map[string]bool{"anchor/gitea": true},
}
}
func TestAMeshWithEverythingInPlaceIsReady(t *testing.T) {
if why := NotReady(aMeshReadyToMove()); len(why) != 0 {
t.Fatalf("a mesh with everything in place was refused: %v", why)
}
}
// **A machine with no credential is the one that must stop this.** It keeps running and cannot come
// back, and there is no bus left to tell it anything over — so the remedy has to happen before, and
// the message says so.
func TestAMachineWithNoCredentialStopsTheMove(t *testing.T) {
r := aMeshReadyToMove()
r.Credentialled = map[string]bool{"anchor": true}
why := NotReady(r)
if len(why) == 0 {
t.Fatal("a machine that could not come back did not stop the move")
}
said := strings.Join(why, "\n")
if !strings.Contains(said, "laptop") {
t.Errorf("the refusal does not name the machine: %s", said)
}
if !strings.Contains(said, "before the move") {
t.Errorf("the refusal does not say the remedy comes first: %s", said)
}
}
// A bus nobody has raised, a seat nobody holds, and a user list nobody has been sent: each stops it,
// and each names its own next step, because "not ready" that cannot be acted on is not an answer.
func TestEachThingMissingNamesItsOwnRemedy(t *testing.T) {
for _, c := range []struct {
what string
break_ func(*Readiness)
says string
}{
{"no address", func(r *Readiness) { r.TheBus = "" }, NATSVar},
{"no server", func(r *Readiness) { r.ServerStanding = false }, "carrying nothing"},
{"no holder", func(r *Readiness) { r.Holder = "" }, "mesh-broker"},
{"no user list", func(r *Readiness) { r.AccountsComposed = false }, "push anchor"},
{"a module with none", func(r *Readiness) {
r.ModuleCredentialled = map[string]bool{}
}, "anchor/gitea"},
} {
r := aMeshReadyToMove()
c.break_(&r)
why := NotReady(r)
if len(why) == 0 {
t.Errorf("%s did not stop the move", c.what)
continue
}
if !strings.Contains(strings.Join(why, "\n"), c.says) {
t.Errorf("%s: the refusal does not mention %q: %v", c.what, c.says, why)
}
}
}
// What the move would do is written out rather than summarised, because this is the one step with
// nothing to inspect afterwards — so reading it is the last chance to disagree.
func TestWhatMovesNamesEveryMachineAndEndsWithTheOldBrokerGoing(t *testing.T) {
r := aMeshReadyToMove()
steps := strings.Join(WhatMoves(r), "\n")
for _, want := range []string{"anchor", "laptop", "user list", "module runtime"} {
if !strings.Contains(steps, want) {
t.Errorf("the plan does not mention %q:\n%s", want, steps)
}
}
// Said explicitly, and last: AMQP is not a provision (novox/hq ADR 0131), so the move ends with
// the old broker's module unassigned, not left behind as a second bus. An earlier version of this
// test pinned the opposite, under a record 0131 superseded.
lines := WhatMoves(r)
if last := lines[len(lines)-1]; !strings.Contains(last, "unassign the old broker") {
t.Errorf("the plan does not end with the old broker going:\n%s", steps)
}
}
+241
View File
@@ -0,0 +1,241 @@
package broker
import (
"fmt"
"sort"
)
// The mesh's own streams.
//
// **These four and no more** (novox/hq ADR 0116 task 1.4, as revised by ADR 0118). An earlier
// reading had the controller create *every* stream at genesis, from a fixed set. That is only the
// mesh's own half: a seat's streams are created when the module declaring it is registered, and a
// module's durable consumers when it is assigned — neither of which has happened at genesis. What
// is here is the foundation, which exists before any module does.
//
// The controller is the only writer of stream definitions (design 25 §3). A module declares
// nothing about them and cannot reach the JetStream API to make one.
// Retention is how a stream decides what to keep, which is the whole of what distinguishes the
// mesh's four relationships on the wire (design 29 §4).
type Retention string
const (
// RetentionWorkQueue: a message is removed once a consumer acknowledges it. Exactly one
// worker does the work, and a worker that dies has its message redelivered.
RetentionWorkQueue Retention = "workqueue"
// RetentionLastPerSubject: only the newest message on each subject survives. This is the
// state shape — a node that was away gets exactly the current declaration and nothing older.
RetentionLastPerSubject Retention = "last_per_subject"
// RetentionLimits: kept until it ages or the stream fills. Events, where a subscriber that
// was down catches up and nobody is obliged to act.
RetentionLimits Retention = "limits"
)
// A Stream is one of the mesh's own, as the controller asserts it.
type Stream struct {
Name string
Subjects []string
Retention Retention
// MaxAge in seconds, zero for unbounded. Per stream — JetStream has no per-subject age,
// which is why differing retention between modules would mean a stream each.
MaxAge int
// MaxMsgsPerSubject caps each subject independently, so one noisy emitter cannot push
// another's events out of a shared stream. Verified: with a cap of 3, ten messages on one
// subject and one on another leave four in the stream, not three.
MaxMsgsPerSubject int
// Why is carried into the assertion so an operator reading the server's own state finds the
// reason there, rather than only in a repository they may not have.
Why string
}
// MeshStreams is the foundation set, in the order a person reads it.
//
// **CONTROL names its subjects rather than taking `mesh.control.>`**, because heartbeats live
// under that prefix and must not be persisted: a lost heartbeat is the next heartbeat, and a
// stream of them is a stream of the least valuable messages the mesh sends, competing for the
// same retention as the ones that matter.
//
// **EVENTS filters on the `event` token**, which is the reason that token exists. A module's
// namespace carries both its events and its tool calls; a filter of `mesh.mod.*.>` would persist
// every tool invocation in the mesh, and a tool call must never be persisted (design 25 §3 keeps
// tools on core NATS, where a lost call is a timeout the caller already handles).
func MeshStreams() []Stream {
return []Stream{
{
Name: "CONTROL",
// A build's outcome is no longer here: it is the build-machine seat's own event, so one
// publish reaches whoever asked, the controller and the catalogue (novox/hq ADR 0121).
Subjects: []string{"mesh.control.*.report", "mesh.control.enrol"},
Retention: RetentionWorkQueue,
Why: "the store-window guarantee (ADR 0083): the controller naks with a delay while its " +
"store is away and the message is redelivered; nothing is dropped",
},
{
Name: "NODES",
Subjects: []string{"mesh.node.*.declare"},
Retention: RetentionLastPerSubject,
Why: "one declaration per node, always the newest; a node that sees sequence n refuses " +
"n-1 by construction (issue 107)",
},
{
Name: "EVENTS",
// A seat's own events ride here too: they are 1:many like any event, and the
// `event` token keeps them clear of both the seat's work queue (`accept`) and its
// tools (`tool`), which must not be persisted.
Subjects: []string{"mesh.mod.*.event.>", "mesh.seat.*.event.>"},
Retention: RetentionLimits,
MaxAge: 7 * 24 * 60 * 60,
MaxMsgsPerSubject: 10000,
Why: "a subscriber that was down catches up; tool traffic under the same prefix is " +
"excluded by the event token; per-subject caps keep a noisy emitter from " +
"evicting a quiet one without splitting the stream",
},
}
}
// An Asserter is the part of a JetStream connection stream assertion needs. Narrow on purpose: it
// keeps this testable without a server, and keeps the client library out of everything that only
// wants to know what the streams are.
type Asserter interface {
// EnsureStream creates the stream if absent and updates it to match if present. It must be
// idempotent: the controller asserts on every start, not only at genesis.
EnsureStream(s Stream) error
}
// AssertMeshStreams brings the foundation set into being, in order, and says which one failed
// rather than that something did.
//
// Asserted on every start rather than created once at genesis, because a stream that was deleted,
// or a mesh raised from a restored backup, must converge rather than run without the guarantee
// its messages assume. Idempotence is the whole requirement.
func AssertMeshStreams(a Asserter) error {
for _, s := range MeshStreams() {
if err := a.EnsureStream(s); err != nil {
return fmt.Errorf("asserting stream %s: %w", s.Name, err)
}
}
return nil
}
// Overlaps reports subject filters claimed by more than one stream.
//
// **Corrected against the server**: an earlier version of this comment said NATS accepts
// overlapping streams and stores the message twice. It does not — it refuses the second stream
// with "subjects overlap with an existing stream" (verified against nats-server 2.10). The check
// still earns its place, for a different reason: the server's refusal arrives when the controller
// is applying, naming one stream, at a moment when the mesh is half-configured. This one arrives
// where the set is written, names both, and cannot reach a running mesh.
//
// It also decides a design question. Because overlap is refused rather than merged, a shared
// EVENTS stream and a per-module stream cannot coexist — the module's would be refused — so
// "one stream for most, its own for a module that wants different retention" is not an option
// the server allows. It is all of one or all of the other.
func Overlaps() []string {
seen := map[string]string{}
var clashes []string
for _, s := range MeshStreams() {
for _, subject := range s.Subjects {
if first, ok := seen[subject]; ok {
clashes = append(clashes, fmt.Sprintf("%s and %s both claim %s", first, s.Name, subject))
continue
}
seen[subject] = s.Name
}
}
sort.Strings(clashes)
return clashes
}
// The mesh's own consumers.
//
// A seat's streams and a module's consumers are derived from declarations (derived.go). These two
// are not: **the controller is not a module and files no manifest**, so its authority and its
// subscriptions cannot come from a declaration that does not exist. They are named here, where the
// mesh's own streams are named, and narrowly — a controller subscribing `mesh.mod.*.event.>` would
// hear every event in the mesh, which it has no business doing and which would make its permission
// list stop explaining anything.
// ControllerName is the controller's durable consumer on each stream it reads, and the name its
// ack subject is derived from (nats.go: `$JS.ACK.<stream>.controller.>`).
const ControllerName = "controller"
// ControllerFollows are the events the controller reacts to: the catalogue saying a module's
// current version moved, and a catalogue that has just started saying it may have missed builds.
//
// **Derived the same way a module's subscription is**, from the emitter and the bare local event
// name, rather than written out. They were written out while the catalogue still spelled its events
// as the old bus's routing keys, and the moment those were converted (novox/hq 04-ISSUES/127) a
// hard-coded pair became a controller listening to a subject nothing publishes — the same fault, from
// the other side. Deriving them means the conversion could not leave these behind.
var ControllerFollows = []string{
moduleEventSubject("mesh-catalog", "upgraded"),
moduleEventSubject("mesh-catalog", "catching-up"),
// A build's outcome, which is the build-machine role's own event now (ADR 0121) rather than a
// message on the control branch. Same three audiences, one publish: whoever asked, this, and the
// catalogue.
seatEventSubject("mesh-build-machine", "built"),
// The forge's merges: what moved a source, so the mesh builds what that source produces
// without anybody telling it (novox/hq 04-ISSUES/131). Appended, because the index is a name.
moduleEventSubject("gitea", "pull.merged"),
}
// moduleEventSubject is where one module's event lands. The same derivation PermissionsFor uses, so
// what the controller subscribes and what the emitter is permitted to publish cannot drift apart.
func moduleEventSubject(module, event string) string {
return "mesh.mod." + module + ".event." + event
}
// seatEventSubject is where a role's own event lands, derived the same way a holder's permission is.
func seatEventSubject(seat, verb string) string {
return "mesh.seat." + seat + ".event." + verb
}
// MeshConsumers is what the controller consumes, in the order a person reads it.
//
// **Unlimited redelivery on CONTROL, deliberately.** The store window's bound is the controller's,
// not the server's (window.go): a message is held with a nak-and-delay until the controller either
// takes it or gives up and says so. A max-deliver here would dead-letter a push that was being
// held through a store restart — the exact message the stream exists to protect — some minutes
// before the controller had finished deciding about it.
func MeshConsumers() []Consumer {
return []Consumer{
{
Name: ControllerName,
Stream: "CONTROL",
Push: true,
AckWaitSeconds: 30,
Why: "the controller is the single consumer of what nodes say; explicit ack and no " +
"max-deliver, because the store window's bound is the controller's own",
},
{
Name: ControllerName,
Stream: "EVENTS",
Filters: ControllerFollows,
Push: true,
AckWaitSeconds: 30,
MaxDeliver: 5,
Why: "the two events the mesh's own controller reacts to; after max-deliver it " +
"dead-letters, because an announcement it cannot act on will not become actionable",
},
}
}
// Ensurer is the part of a JetStream connection consumer assertion needs, narrow for the reason
// Asserter is.
type Ensurer interface {
EnsureConsumer(c Consumer) error
}
// AssertMeshConsumers brings the controller's own consumers into being, and says which one failed.
//
// After the streams, necessarily: a consumer on a stream that does not exist is refused, and the
// refusal names the stream rather than the order.
func AssertMeshConsumers(e Ensurer) error {
for _, c := range MeshConsumers() {
if err := e.EnsureConsumer(c); err != nil {
return fmt.Errorf("asserting consumer %s on %s: %w", c.Name, c.Stream, err)
}
}
return nil
}
+235
View File
@@ -0,0 +1,235 @@
package broker
import (
"errors"
"strings"
"testing"
)
type recorder struct {
seen []Stream
fail string
}
func (r *recorder) EnsureStream(s Stream) error {
if s.Name == r.fail {
return errors.New("refused")
}
r.seen = append(r.seen, s)
return nil
}
// The controller asserts on every start, not only at genesis: a stream that was deleted, or a mesh
// raised from a backup, must converge rather than run without the guarantee its messages assume.
func TestAssertingTwiceIsTheSameAsOnce(t *testing.T) {
a, b := &recorder{}, &recorder{}
if err := AssertMeshStreams(a); err != nil {
t.Fatal(err)
}
if err := AssertMeshStreams(a); err != nil {
t.Fatal(err)
}
if err := AssertMeshStreams(b); err != nil {
t.Fatal(err)
}
if len(a.seen) != 2*len(b.seen) {
t.Fatalf("asserted %d then %d; assertion is not repeatable", len(a.seen), len(b.seen))
}
}
func TestAFailedAssertionNamesItsStream(t *testing.T) {
err := AssertMeshStreams(&recorder{fail: "NODES"})
if err == nil || !strings.Contains(err.Error(), "NODES") {
t.Fatalf("got %v, which does not say which stream failed", err)
}
}
// Two streams matching one subject is accepted by NATS and stores the message twice under two
// retentions. Nothing reports that, so it is refused where the set is written.
func TestNoTwoStreamsClaimTheSameSubject(t *testing.T) {
if clashes := Overlaps(); len(clashes) != 0 {
t.Fatalf("overlapping subject filters: %v", clashes)
}
}
// A heartbeat under mesh.control.> must not be persisted: a lost one is the next one, and a
// stream of them competes for retention with the messages that matter.
func TestHeartbeatsAreNotInTheControlStream(t *testing.T) {
for _, s := range MeshStreams() {
for _, subject := range s.Subjects {
if subject == "mesh.control.>" || strings.Contains(subject, "alive") {
t.Fatalf("stream %s claims %q, which captures heartbeats", s.Name, subject)
}
}
}
}
// The reason the kind token exists: a filter over a module's whole namespace would persist every
// tool call in the mesh.
func TestTheEventsStreamDoesNotCaptureToolCalls(t *testing.T) {
var events Stream
for _, s := range MeshStreams() {
if s.Name == "EVENTS" {
events = s
}
}
// Nothing a tool call rides may match any of the filters — a module's or a seat's.
for _, tool := range []string{
"mesh.mod.billing.tool.status",
"mesh.seat.telegram-sender.tool.status",
"mesh.seat.telegram-sender.accept.send", // work, not an event: its own stream
} {
for _, f := range events.Subjects {
if subjectMatches(f, tool) {
t.Fatalf("%q matches the events filter %q, so it would be persisted here", tool, f)
}
}
}
// And both kinds of event do match.
for _, event := range []string{
"mesh.mod.billing.event.order.placed",
"mesh.seat.telegram-sender.event.delivered",
} {
matched := false
for _, f := range events.Subjects {
if subjectMatches(f, event) {
matched = true
}
}
if !matched {
t.Fatalf("%q matches no events filter, so nothing would keep it", event)
}
}
}
// subjectMatches is NATS subject matching, enough for these filters: `*` is one token, `>` is the
// rest.
func subjectMatches(filter, subject string) bool {
f, s := strings.Split(filter, "."), strings.Split(subject, ".")
for i, tok := range f {
if tok == ">" {
return i <= len(s)
}
if i >= len(s) {
return false
}
if tok != "*" && tok != s[i] {
return false
}
}
return len(f) == len(s)
}
// Each relationship's retention is the thing that makes it what it is (design 29 §4).
func TestEachStreamCarriesTheRetentionItsShapeNeeds(t *testing.T) {
want := map[string]Retention{
"CONTROL": RetentionWorkQueue,
"NODES": RetentionLastPerSubject,
"EVENTS": RetentionLimits,
}
got := map[string]Retention{}
for _, s := range MeshStreams() {
got[s.Name] = s.Retention
if s.Why == "" {
t.Errorf("stream %s says no reason it exists", s.Name)
}
}
if len(got) != len(want) {
t.Fatalf("the foundation set is %v", got)
}
for name, r := range want {
if got[name] != r {
t.Errorf("%s retains as %q, expected %q", name, got[name], r)
}
}
}
// The order the bus's objects are asserted in, because getting it wrong is a refusal that names the
// wrong thing: a consumer on a stream that does not exist is refused naming the *stream*, so
// somebody reading it goes looking for a deletion instead of a reversed pair of lines.
func TestTheBusesObjectsAreAssertedStreamsBeforeConsumers(t *testing.T) {
r := &recording{}
if err := Raise(r, []string{"anchor", "laptop"}); err != nil {
t.Fatal(err)
}
// Every stream before every consumer.
firstConsumer := -1
for i, step := range r.steps {
if strings.HasPrefix(step, "consumer ") && firstConsumer < 0 {
firstConsumer = i
}
if strings.HasPrefix(step, "stream ") && firstConsumer >= 0 {
t.Fatalf("a stream was asserted after a consumer: %v", r.steps)
}
}
if firstConsumer < 0 {
t.Fatalf("no consumer was asserted: %v", r.steps)
}
// And every node got one, named after it — without which that node hears nothing while
// everything else about it looks correct.
for _, node := range []string{"anchor", "laptop"} {
if !containsStep(r.steps, "consumer NODES/"+node) {
t.Errorf("%s was given no way to hear its declaration: %v", node, r.steps)
}
}
// And the controller its own, on both streams it reads.
for _, want := range []string{"consumer CONTROL/controller", "consumer EVENTS/controller"} {
if !containsStep(r.steps, want) {
t.Errorf("the controller is missing %s: %v", want, r.steps)
}
}
}
// A seat's work queue is asserted whether or not anybody holds it; the holder's worker only when
// somebody does. **The stream without the consumer is the point**: work queues until a holder
// appears, so installing the module later flushes the backlog instead of having lost it.
func TestASeatsQueueExistsBeforeItsHolderDoes(t *testing.T) {
seats := []DeclaredSeat{{Name: "telegram-sender", Accepts: []string{"send"}}}
unheld := &recording{}
if err := RaiseSeats(unheld, seats, nil); err != nil {
t.Fatal(err)
}
if !containsStep(unheld.steps, "stream SEAT_TELEGRAM_SENDER") {
t.Fatalf("a declared seat got no work queue: %v", unheld.steps)
}
for _, step := range unheld.steps {
if strings.HasPrefix(step, "consumer ") {
t.Fatalf("a seat nobody holds got a worker: %v", unheld.steps)
}
}
held := &recording{}
if err := RaiseSeats(held, seats, map[string]Holder{
"telegram-sender": {Node: "anchor", Module: "telegram"},
}); err != nil {
t.Fatal(err)
}
if !containsStep(held.steps, "consumer SEAT_TELEGRAM_SENDER/SEAT_TELEGRAM_SENDER_worker") {
t.Fatalf("the seat's holder got no worker: %v", held.steps)
}
}
// recording is a connection to the bus that writes down what it was asked for.
type recording struct{ steps []string }
func (r *recording) EnsureStream(s Stream) error {
r.steps = append(r.steps, "stream "+s.Name)
return nil
}
func (r *recording) EnsureConsumer(c Consumer) error {
r.steps = append(r.steps, "consumer "+c.Stream+"/"+c.Name)
return nil
}
func containsStep(steps []string, want string) bool {
for _, s := range steps {
if s == want {
return true
}
}
return false
}
+56
View File
@@ -0,0 +1,56 @@
# Composed by the mesh controller. Do not edit: the next composition overwrites it.
# Accounts and permissions are derived from what each module declares and nothing
# else (novox/hq ADR 0043, design 29 §2).
port: 4222
http: 127.0.0.1:8222
tls {
cert_file: "/tls/tls.crt"
key_file: "/tls/tls.key"
ca_file: "/tls/ca.crt"
}
jetstream {
store_dir: "/data"
}
# The mesh's users, composed by the controller. Do not edit: the next
# composition overwrites it. Permissions are derived from what each module
# declares and nothing else (novox/hq ADR 0043, design 29 §2).
accounts {
MESH {
jetstream: enabled
users = [
{ user: "controller", password: "$2a$11$cccccccccccccccccccccc", permissions: {
publish: { allow: ["$JS.ACK.CONTROL.controller.>", "$JS.ACK.EVENTS.controller.>", "$JS.API.>", "_INBOX.enrol.>", "mesh.control.>", "mesh.mod.*.tool.>", "mesh.node.>", "mesh.seat.mesh-build-machine.accept.>"] }
subscribe: { allow: ["$JS.API.>", "_DELIVER.controller", "_DELIVER.controller.>", "_INBOX.controller.>", "mesh.control.>", "mesh.mod.gitea.event.pull.merged", "mesh.mod.mesh-catalog.event.catching-up", "mesh.mod.mesh-catalog.event.upgraded", "mesh.seat.mesh-build-machine.event.built"] }
allow_responses: { max: 1, ttl: "1m" }
} }
{ user: "enrol.one", password: "$2a$11$eeeeeeeeeeeeeeeeeeeeee", permissions: {
publish: { allow: ["mesh.control.enrol"] }
subscribe: { allow: ["_INBOX.enrol.one.>"] }
} }
{ user: "node.one", password: "$2a$11$nnnnnnnnnnnnnnnnnnnnnn", permissions: {
publish: { allow: ["$JS.ACK.NODES.one.>", "$JS.API.CONSUMER.INFO.NODES.one", "mesh.control.one.>"] }
subscribe: { allow: ["_DELIVER.one", "_INBOX.node.one.>", "mesh.node.one.declare"] }
} }
{ user: "one.telegram", password: "$2a$11$tttttttttttttttttttttt", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.one_telegram.>", "$JS.ACK.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker.>", "$JS.API.CONSUMER.INFO.EVENTS.one_telegram", "$JS.API.CONSUMER.INFO.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.one_telegram", "mesh.seat.telegram-sender.event.delivered", "mesh.seat.telegram-sender.event.failed"] }
subscribe: { allow: ["_DELIVER.SEAT_TELEGRAM_SENDER_worker", "_INBOX.one.telegram.>", "mesh.mod.telegram.tool.>", "mesh.seat.telegram-sender.accept.send"] }
allow_responses: { max: 1, ttl: "1m" }
} }
{ user: "two.audit", password: "$2a$11$aaaaaaaaaaaaaaaaaaaaaa", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.two_audit.>", "$JS.API.CONSUMER.INFO.EVENTS.two_audit", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_audit"] }
subscribe: { allow: ["_INBOX.two.audit.>", "mesh.mod.audit.tool.>", "mesh.mod.shop.event.order.placed"] }
allow_responses: { max: 1, ttl: "1m" }
} }
{ user: "two.shop", password: "$2a$11$ssssssssssssssssssssss", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.two_shop.>", "$JS.API.CONSUMER.INFO.EVENTS.two_shop", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_shop", "mesh.mod.shop.event.order.placed", "mesh.seat.telegram-sender.accept.send"] }
subscribe: { allow: ["_INBOX.two.shop.>", "mesh.mod.shop.tool.>"] }
allow_responses: { max: 1, ttl: "1m" }
} }
]
}
}
+127
View File
@@ -0,0 +1,127 @@
package broker
import (
"fmt"
"sort"
)
// Every user the composed file should contain, derived from what the mesh knows.
//
// **The list is derived, never kept.** A stored user list would be a second account of who may
// reach the bus, able to disagree with the records it came from — and the disagreement would be
// invisible, because both would look internally consistent. So this is a pure function of the
// mesh's records, run again every time the file is written.
//
// Records are mirrored into this package's own types rather than imported from the catalogue, for
// the reason DeclaredSeat is: composing authority is a different job from parsing a manifest, and
// this package stays free of the other's types so a change to a manifest field cannot quietly widen
// a permission.
// Declared is one module on one node, as composing its authority needs it.
type Declared struct {
Module string
Emits []string
Consumes []string
Serves []string
// Holds are the seats this module claims, with the protocol each seat declares. A seat the
// mesh defines for itself declares no protocol, so holding one grants nothing on the bus —
// which is right: those seats are about who does a job, not about who may say what.
Holds []Seat
// Uses are the seats this module sends to.
Uses []Seat
// Watches are the seats whose events it consumes.
Watches []Seat
}
// Records is what composing a user list needs to know about the mesh, and nothing more.
type Records struct {
// Nodes is every machine the mesh knows. Each gets a host user.
Nodes []string
// Assigned is the modules on each node, as they declare themselves.
Assigned map[string][]Declared
// Enrolling is every node with a live token — one enrolment user each, because the inbox an
// answer goes to is scoped to the token and a shared one is one machine reading another's
// sealed credentials (design 25 §6).
Enrolling []string
// People is each person's name against the tools they may invoke, `*` for an administrator.
People map[string][]string
}
// Users is every user the composed file should contain, in the order it will be written.
//
// The controller is always first and always present: a mesh whose own controller is not in the file
// is a mesh that cannot be told anything, and there is no state of the records in which that is
// correct.
func Users(r Records) ([]Principal, error) {
out := []Principal{{Kind: KindController}}
for _, node := range sortedCopy(r.Nodes) {
out = append(out, Principal{Kind: KindNode, Node: node})
for _, d := range r.Assigned[node] {
out = append(out, Principal{
Kind: KindModule, Node: node, Module: d.Module,
Emits: d.Emits, Consumes: d.Consumes, Serves: d.Serves,
Holds: d.Holds, Uses: d.Uses, Watches: d.Watches,
})
}
}
for _, node := range sortedCopy(r.Enrolling) {
out = append(out, Principal{Kind: KindEnrolment, Node: node})
}
for _, person := range sortedNames(r.People) {
out = append(out, Principal{Kind: KindPerson, Module: person, Invokes: r.People[person]})
}
// Refused here rather than discovered by the server. Two users with one name is a file the
// server reads as one of them, and which one depends on the order — so a module assigned to a
// node twice, or a person named after nothing, is a composition that must not be written.
seen := map[string]string{}
for _, p := range out {
name := p.Username()
if name == "" || name == "." {
return nil, fmt.Errorf("a %s user has no name, so nothing could authenticate as it", p.Kind)
}
if first, already := seen[name]; already {
return nil, fmt.Errorf(
"two users would be called %q (a %s and a %s): the server would read the file as "+
"one of them, and which one depends on the order", name, first, p.Kind)
}
seen[name] = string(p.Kind)
}
return out, nil
}
// WithPasswords fills each user's hash from what the mesh minted, and says which users have none.
//
// **Separated from Users because they fail differently.** A user missing from the records is a bug
// in deriving them; a user with no password is a step that has not happened yet — a module assigned
// but never given a credential, a node enrolled before this existed. The second is ordinary and its
// remedy is to mint one, so it is named rather than returned as an error, and the caller decides
// whether a partial composition is worth writing.
func WithPasswords(principals []Principal, hashes map[string]string) (filled []Principal, missing []string) {
for _, p := range principals {
hash, ok := hashes[p.Username()]
if !ok || hash == "" {
missing = append(missing, p.Username())
continue
}
p.PasswordHash = hash
filled = append(filled, p)
}
return filled, missing
}
func sortedCopy(in []string) []string {
out := append([]string(nil), in...)
sort.Strings(out)
return out
}
func sortedNames(in map[string][]string) []string {
out := make([]string, 0, len(in))
for k := range in {
out = append(out, k)
}
sort.Strings(out)
return out
}
+247
View File
@@ -0,0 +1,247 @@
package broker
import (
"strings"
"testing"
)
// Deriving the bus's user list from the mesh's records.
//
// Every test here is about a way the list could be wrong that the server would not tell anybody
// about: a user missing, a user named twice, a user with authority it did not declare.
func someRecords() Records {
return Records{
Nodes: []string{"two", "one"},
Assigned: map[string][]Declared{
"one": {{Module: "telegram", Serves: []string{"status"}}},
"two": {{Module: "shop", Emits: []string{"order.placed"}}},
},
Enrolling: []string{"three"},
People: map[string][]string{"ada": {"mesh-catalog.catalog_tools"}},
}
}
func namesOf(t *testing.T, r Records) []string {
t.Helper()
users, err := Users(r)
if err != nil {
t.Fatal(err)
}
out := make([]string, 0, len(users))
for _, u := range users {
out = append(out, u.Username())
}
return out
}
// The controller is always there. A mesh whose own controller is not in the file is a mesh that
// cannot be told anything, and there is no state of the records in which that is correct.
func TestTheControllerIsAlwaysInTheList(t *testing.T) {
for _, r := range []Records{{}, someRecords()} {
names := namesOf(t, r)
if len(names) == 0 || names[0] != "controller" {
t.Fatalf("the controller is not first in %v", names)
}
}
}
// One user per node, one per module per node, one per live token and one per person — and nothing
// else, because a user nobody derived is a user nobody can explain.
func TestEveryRecordBecomesExactlyOneUser(t *testing.T) {
names := namesOf(t, someRecords())
want := []string{
"controller",
"node.one", "one.telegram",
"node.two", "two.shop",
"enrol.three",
"person.ada",
}
if strings.Join(names, ",") != strings.Join(want, ",") {
t.Fatalf("derived %v\n want %v", names, want)
}
}
// Two users with one name is a file the server reads as one of them, and which one depends on the
// order. Refused here, where both can be named, rather than left to be whichever the server picked.
func TestTwoUsersWithOneNameAreRefused(t *testing.T) {
r := someRecords()
r.Assigned["one"] = append(r.Assigned["one"], Declared{Module: "telegram"})
_, err := Users(r)
if err == nil {
t.Fatal("a module assigned twice to one node composed two users with one name")
}
if !strings.Contains(err.Error(), "one.telegram") {
t.Fatalf("the refusal does not name the user: %v", err)
}
}
// A module's authority is what it declared and nothing more, carried through the derivation intact —
// because this is the step where a mistake would grant something no manifest asked for.
func TestAModulesAuthorityIsWhatItDeclared(t *testing.T) {
seat := Seat{Name: "telegram-sender", Accepts: []string{"send"}, Emits: []string{"delivered"}}
users, err := Users(Records{
Nodes: []string{"one"},
Assigned: map[string][]Declared{"one": {{
Module: "shop", Emits: []string{"order.placed"}, Uses: []Seat{seat},
}}},
})
if err != nil {
t.Fatal(err)
}
perms, err := PermissionsFor(users[len(users)-1])
if err != nil {
t.Fatal(err)
}
has(t, perms.Publish, "mesh.mod.shop.event.order.placed")
has(t, perms.Publish, "mesh.seat.telegram-sender.accept.send")
// A seat it uses, not one it holds: it may submit work and may not publish the seat's own
// events, or it could lie about outcomes on a role somebody else fills.
hasNot(t, perms.Publish, "mesh.seat.telegram-sender.event.delivered")
hasNot(t, perms.Subscribe, "mesh.seat.telegram-sender.accept.send")
}
// A user the mesh has never minted a password for is named rather than silently dropped or
// composed as a user anybody is. It is an ordinary situation — a module assigned a moment ago — and
// the remedy is to mint one, so the caller decides whether to write a partial file.
func TestAUserWithNoPasswordIsNamedRatherThanWritten(t *testing.T) {
users, err := Users(someRecords())
if err != nil {
t.Fatal(err)
}
filled, missing := WithPasswords(users, map[string]string{
"controller": "$2a$hash", "node.one": "$2a$hash",
})
if len(filled) != 2 {
t.Fatalf("composed %d users from two hashes", len(filled))
}
if len(missing) != len(users)-2 {
t.Fatalf("%d users are missing a password, of %d: %v", len(missing), len(users), missing)
}
for _, p := range filled {
if p.PasswordHash == "" {
t.Fatalf("%s was kept with no password, which is a user anybody is", p.Username())
}
}
}
// And the whole thing composes: records in, a file the server would read out.
func TestRecordsComposeIntoAFile(t *testing.T) {
users, err := Users(someRecords())
if err != nil {
t.Fatal(err)
}
hashes := map[string]string{}
for _, u := range users {
hashes[u.Username()] = "$2a$11$" + strings.Repeat("x", 22)
}
filled, missing := WithPasswords(users, hashes)
if len(missing) != 0 {
t.Fatalf("users with no password: %v", missing)
}
got, err := Compose(Server{ClientPort: 4222, MonitoringPort: 8222, StoreDir: "/data",
TLSCert: "/tls/tls.crt", TLSKey: "/tls/tls.key", TLSCA: "/tls/ca.crt"}, filled)
if err != nil {
t.Fatal(err)
}
for _, want := range []string{
`user: "controller"`, `user: "node.one"`, `user: "one.telegram"`,
`user: "enrol.three"`, `user: "person.ada"`,
`"_INBOX.enrol.three.>"`, `"mesh.mod.mesh-catalog.tool.catalog_tools"`,
} {
if !strings.Contains(got, want) {
t.Errorf("the composed file does not contain %s", want)
}
}
}
// The accounts block alone is what the mesh writes, and it holds nothing about the server.
//
// **The split is the whole design decision** (ComposeAccounts): ports, TLS paths and a store
// directory are properties of the container the module raises, and a controller that wrote them
// would have to be kept in step with a Dockerfile it never sees. So this test says what must not be
// in the file as plainly as what must.
func TestWhatTheMeshWritesIsUsersAndNothingAboutTheServer(t *testing.T) {
users, err := Users(someRecords())
if err != nil {
t.Fatal(err)
}
hashes := map[string]string{}
for _, u := range users {
hashes[u.Username()] = "$2a$11$" + strings.Repeat("x", 22)
}
filled, missing := WithPasswords(users, hashes)
if len(missing) != 0 {
t.Fatalf("users with no password: %v", missing)
}
got, err := ComposeAccounts(filled)
if err != nil {
t.Fatal(err)
}
for _, want := range []string{"accounts {", `user: "controller"`, `user: "one.telegram"`} {
if !strings.Contains(got, want) {
t.Errorf("the accounts file does not contain %s", want)
}
}
// None of the server's own settings. Each of these in the mesh's file is a value the controller
// would then own, and the module could no longer change its own image without the mesh agreeing.
// `jetstream {` is the server's block (its store, its limits); `jetstream: enabled` inside the
// account is the account's, and the mesh owns the account — a user in it is told "JetStream
// not enabled for account" without it (2026-09-28).
if !strings.Contains(got, "jetstream: enabled") {
t.Errorf("the account does not enable JetStream, so no user in it can bind a consumer")
}
for _, absent := range []string{"port:", "http:", "jetstream {", "tls {", "store_dir", "cert_file"} {
if strings.Contains(got, absent) {
t.Errorf("the accounts file contains %q, which belongs to the module that raises the "+
"server, not to the mesh", absent)
}
}
}
// A user with no password is refused here too, not only by the whole-file composition: this is the
// function the controller actually calls, and a user without a password is a user anybody is.
func TestTheAccountsFileRefusesAUserWithNoPassword(t *testing.T) {
if _, err := ComposeAccounts([]Principal{{Kind: KindController}}); err == nil {
t.Fatal("a user with no password hash was written")
}
}
// **A user list is composed for a bus the mesh has not moved onto yet**, and that is the whole of
// step 2 (novox/hq ADR 0116): the server stands in the mesh carrying nothing, on its own ports, while
// every node is still on the bus it was on.
//
// Pinned because the first version of the composing step got it backwards — it wrote the list only
// once the controller was already on the new bus, which is a step that cannot be taken: the module
// comes up, finds no accounts file, and waits for one the controller had decided not to write.
func TestAUserListIsComposedBeforeAnythingMovesOntoTheBus(t *testing.T) {
// Exactly the records of a mesh mid-change: everything running, nothing on the new bus.
users, err := Users(Records{
Nodes: []string{"anchor"},
Assigned: map[string][]Declared{"anchor": {{Module: "nats"}}},
})
if err != nil {
t.Fatal(err)
}
hashes := map[string]string{}
for _, u := range users {
hashes[u.Username()] = "$2a$11$" + strings.Repeat("x", 22)
}
filled, missing := WithPasswords(users, hashes)
if len(missing) != 0 {
t.Fatalf("users with no credential: %v", missing)
}
accounts, err := ComposeAccounts(filled)
if err != nil {
t.Fatal(err)
}
// The controller's own user above all: a file without it is a bus its writer cannot connect to,
// which is what the server would be left holding the moment it starts.
if !strings.Contains(accounts, `user: "controller"`) {
t.Fatalf("the composed list does not contain the controller:\n%s", accounts)
}
if !strings.Contains(accounts, `user: "node.anchor"`) {
t.Errorf("the composed list does not contain the machine running the bus")
}
}
+34 -15
View File
@@ -169,6 +169,8 @@ func Build(ctx context.Context, run Runner, publish Publisher,
}
var built []catalogue.Built
// stoodOn is every base the build was handed, as resolved — the edges the catalogue derives.
var stoodOn []string
if manifest.Build != nil {
// What this module said it stands on, answered with what this mesh actually holds. Done
// before anything is built, so a missing base is refused in front of the person who can
@@ -186,11 +188,12 @@ func Build(ctx context.Context, run Runner, publish Publisher,
}
return from, nil
}
args, err := standingOn(ctx, manifest, held, mirror)
args, bases, err := standingOn(ctx, manifest, held, mirror)
if err != nil {
say("bases", "UNMET: %v", err)
return Result{}, err
}
stoodOn = bases
if len(args) > 0 {
say("bases", "%d resolved from what the mesh holds", len(args)/2)
}
@@ -217,7 +220,7 @@ func Build(ctx context.Context, run Runner, publish Publisher,
}
say("done", "%s at %s — %d artifact(s) pinned", manifest.Module, short(commit), len(built))
return Result{Manifest: resolved, Commit: commit, Built: built,
Against: against(within, manifest)}, nil
Against: against(within, manifest, stoodOn)}, nil
}
// Log is where a build says what it is doing, step by step. Nil is silent — the tests pass none,
@@ -339,14 +342,27 @@ func describe(path string) string {
// allowed to name — a tag is something somebody else can move under you.
var pinnedImage = regexp.MustCompile(`[A-Za-z0-9][A-Za-z0-9._/:-]*@sha256:[0-9a-f]{64}`)
// against reads what this module's image artifacts are built on top of, out of the files that
// build them. Nothing is guessed: a reference that is not written down is not reported.
func against(within string, manifest catalogue.Manifest) []string {
// against is what this module's image artifacts are built on top of: every base the mesh resolved
// and handed the recipe as a build argument (`build.on`), and any image a recipe pins by digest
// itself. Nothing is guessed: a reference that was neither resolved nor written down is not
// reported.
//
// **The resolved bases are the edges.** A recipe reads its base from an argument (`FROM
// ${RUNTIME_BASE}`), so the digest is never in the file, and a derivation that read files alone
// recorded no edge for any module on the mesh — which is why nothing knew what a changed base
// meant to rebuild (novox/hq 04-ISSUES/131).
func against(within string, manifest catalogue.Manifest, resolved []string) []string {
if manifest.Build == nil {
return nil
}
seen := map[string]bool{}
var out []string
for _, r := range resolved {
if r != "" && !seen[r] {
seen[r] = true
out = append(out, r)
}
}
for _, a := range manifest.Build.Artifacts {
if a.Kind != catalogue.ArtifactImage || a.From == "" {
continue
@@ -704,40 +720,42 @@ var _ io.Writer = (*stringWriter)(nil)
// built cannot be built here yet, and the useful sentence names which module is missing — not the
// one a container runtime produces when a recipe's first line refers to an image nobody has.
//
// The order is fixed so two builds of one commit invoke the same command.
// The order is fixed so two builds of one commit invoke the same command. Returned alongside the
// arguments is every reference they resolved to, which is what the build stood on.
func standingOn(ctx context.Context, manifest catalogue.Manifest, held map[string]string,
mirror func(ctx context.Context, from, repository string) (string, error)) ([]string, error) {
mirror func(ctx context.Context, from, repository string) (string, error)) ([]string, []string, error) {
if manifest.Build == nil || len(manifest.Build.On) == 0 {
return nil, nil
return nil, nil, nil
}
on := append([]catalogue.BuildsOn{}, manifest.Build.On...)
sort.Slice(on, func(i, j int) bool { return on[i].Arg < on[j].Arg })
var args []string
var args, resolved []string
for _, base := range on {
if base.Image != "" {
// A vendor's image, declared (novox/hq 04-ISSUES/064, ADR 0097). Pinned, because a tag
// is what somebody else can move; copied into the mesh's registry, because a build
// that reaches a public registry on its own is a build that works sometimes.
if base.Arg == "" || base.Module != "" || base.Artifact != "" {
return nil, fmt.Errorf(
return nil, nil, fmt.Errorf(
"%s stands on the image %s, and a base is either a module's artifact or an "+
"image — never both — read from one build argument", manifest.Module, base.Image)
}
if !strings.Contains(base.Image, "@sha256:") {
return nil, fmt.Errorf(
return nil, nil, fmt.Errorf(
"%s stands on the image %q, which is not pinned by digest. A tag is what "+
"somebody else can move; name it as <image>@sha256:…", manifest.Module, base.Image)
}
reference, err := mirror(ctx, base.Image, manifest.Module+"/on-"+strings.ToLower(base.Arg))
if err != nil {
return nil, fmt.Errorf("%s stands on %s: %w", manifest.Module, base.Image, err)
return nil, nil, fmt.Errorf("%s stands on %s: %w", manifest.Module, base.Image, err)
}
args = append(args, "--build-arg", base.Arg+"="+reference)
resolved = append(resolved, reference)
continue
}
if base.Arg == "" || base.Module == "" || base.Artifact == "" {
return nil, fmt.Errorf(
return nil, nil, fmt.Errorf(
"%s says its build stands on something, and does not say all of what: a base "+
"needs the module, the artifact, and the build argument the recipe reads it "+
"from", manifest.Module)
@@ -745,14 +763,15 @@ func standingOn(ctx context.Context, manifest catalogue.Manifest, held map[strin
key := base.Module + "/" + base.Artifact
reference, has := held[key]
if !has {
return nil, fmt.Errorf(
return nil, nil, fmt.Errorf(
"%s builds on %s, and this mesh has not built it. Build %s first — every module "+
"in this toolchain stands on it, so it is the thing to have before anything "+
"else", manifest.Module, key, base.Module)
}
args = append(args, "--build-arg", base.Arg+"="+reference)
resolved = append(resolved, reference)
}
return args, nil
return args, resolved, nil
}
// compile runs a module's own code through its toolchain, and says where the result is.
+14
View File
@@ -180,6 +180,20 @@ func (r Registry) MirrorImage(ctx context.Context, from, repository string) (str
if err != nil {
return "", err
}
// **Already held is already mirrored.** A base is named by digest, and a digest this registry
// holds under the module's repository is the same bytes whatever upstream would say — so
// upstream is not asked. Asked every build, the public hub's anonymous pull limit was reached
// on the first merge that rebuilt a whole catalogue (2026-09-28), and every module whose base
// lives there failed on a copy it did not need.
if strings.HasPrefix(where.reference, "sha256:") {
held, err := r.has(ctx, "http://"+r.Address+"/v2/"+repository+"/manifests/"+where.reference)
if err != nil {
return "", fmt.Errorf("asking %s whether it holds %s: %w", r.Address, from, err)
}
if held {
return r.Address + "/" + repository + "@" + where.reference, nil
}
}
src := &source{client: r.client()}
digest, err := r.copyManifest(ctx, src, where, where.reference, repository)
if err != nil {
+30
View File
@@ -103,6 +103,12 @@ func (m *theMeshsRegistry) handler() http.Handler {
m.mu.Lock()
defer m.mu.Unlock()
switch {
case r.Method == http.MethodHead && strings.Contains(r.URL.Path, "/manifests/"):
if _, ok := m.manifests[r.URL.Path[strings.LastIndex(r.URL.Path, "/")+1:]]; ok {
w.WriteHeader(http.StatusOK)
} else {
w.WriteHeader(http.StatusNotFound)
}
case r.Method == http.MethodHead && strings.Contains(r.URL.Path, "/blobs/"):
if _, ok := m.blobs[r.URL.Path[strings.LastIndex(r.URL.Path, "/")+1:]]; ok {
w.WriteHeader(http.StatusOK)
@@ -219,3 +225,27 @@ func TestATagBeforeTheDigestIsNotPartOfTheRepository(t *testing.T) {
t.Fatalf("got %+v", got)
}
}
// A base this registry already holds by digest is not asked of upstream at all: the public hub
// limits anonymous pulls, and a catalogue rebuilt on one merge asked it once per module.
func TestABaseAlreadyHeldIsNotAskedOfUpstream(t *testing.T) {
src, indexDigest, _ := anUpstreamRegistry(t)
dst := &theMeshsRegistry{blobs: map[string][]byte{}, manifests: map[string][]byte{}}
dstServer := httptest.NewServer(dst.handler())
defer dstServer.Close()
address := strings.TrimPrefix(dstServer.URL, "http://")
r := Registry{Address: address, HTTP: src.Client()}
host := strings.TrimPrefix(src.URL, "http://")
if _, err := r.MirrorImage(context.Background(), host+"/library/thing:latest", "hello-web/server"); err != nil {
t.Fatal(err)
}
// Upstream gone: the pinned base is answered from what the mesh holds.
src.Close()
reference, err := r.MirrorImage(context.Background(), host+"/library/thing@"+indexDigest, "hello-web/server")
if err != nil {
t.Fatalf("a base the registry holds was asked of an upstream that is gone: %v", err)
}
if reference != address+"/hello-web/server@"+indexDigest {
t.Fatalf("pinned as %q", reference)
}
}
+34 -6
View File
@@ -22,7 +22,7 @@ func TestABaseTheMeshHasNotBuiltIsRefused(t *testing.T) {
On: []catalogue.BuildsOn{{Arg: "RUNTIME_BASE", Module: "mesh-tools", Artifact: "runtime"}},
},
}
_, err := standingOn(context.Background(), manifest, map[string]string{}, noMirror)
_, _, err := standingOn(context.Background(), manifest, map[string]string{}, noMirror)
if err == nil {
t.Fatal("a base nothing has built was accepted; the build would have failed on its first line")
}
@@ -42,7 +42,7 @@ func TestABaseTheMeshHoldsBecomesABuildArgument(t *testing.T) {
},
}
held := map[string]string{"mesh-tools/runtime": "127.0.0.1:5000/mesh-tools/runtime@sha256:" + strings.Repeat("a", 64)}
args, err := standingOn(context.Background(), manifest, held, noMirror)
args, _, err := standingOn(context.Background(), manifest, held, noMirror)
if err != nil {
t.Fatalf("a base this mesh holds was refused: %v", err)
}
@@ -54,7 +54,7 @@ func TestABaseTheMeshHoldsBecomesABuildArgument(t *testing.T) {
// A module naming no base asks for nothing, which is most modules.
func TestAModuleNamingNoBaseAddsNoArguments(t *testing.T) {
args, err := standingOn(context.Background(), catalogue.Manifest{Module: "hello-web", Build: &catalogue.Build{}}, nil, noMirror)
args, _, err := standingOn(context.Background(), catalogue.Manifest{Module: "hello-web", Build: &catalogue.Build{}}, nil, noMirror)
if err != nil || args != nil {
t.Fatalf("a module naming no base produced %v, %v", args, err)
}
@@ -66,7 +66,7 @@ func TestAnIncompleteBaseIsRefused(t *testing.T) {
Module: "postgres",
Build: &catalogue.Build{On: []catalogue.BuildsOn{{Module: "mesh-tools", Artifact: "runtime"}}},
}
if _, err := standingOn(context.Background(), manifest, map[string]string{"mesh-tools/runtime": "x"}, noMirror); err == nil {
if _, _, err := standingOn(context.Background(), manifest, map[string]string{"mesh-tools/runtime": "x"}, noMirror); err == nil {
t.Fatal("a base with no build argument was accepted; nothing would have read it")
}
}
@@ -86,7 +86,7 @@ func TestADeclaredVendorImageIsCopiedInAndHandedToTheRecipe(t *testing.T) {
},
}
var asked []string
args, err := standingOn(context.Background(), manifest, nil, func(_ context.Context, from, repository string) (string, error) {
args, _, err := standingOn(context.Background(), manifest, nil, func(_ context.Context, from, repository string) (string, error) {
asked = append(asked, from+" -> "+repository)
return "127.0.0.1:5000/" + repository + "@sha256:" + strings.Repeat("d", 64), nil
})
@@ -101,7 +101,7 @@ func TestADeclaredVendorImageIsCopiedInAndHandedToTheRecipe(t *testing.T) {
}
// Unpinned, it is refused: a tag is what somebody else can move.
manifest.Build.On[0].Image = "quay.io/minio/mc:latest"
if _, err := standingOn(context.Background(), manifest, nil, noMirror); err == nil || !strings.Contains(err.Error(), "not pinned") {
if _, _, err := standingOn(context.Background(), manifest, nil, noMirror); err == nil || !strings.Contains(err.Error(), "not pinned") {
t.Fatalf("an unpinned vendor image was accepted: %v", err)
}
}
@@ -151,3 +151,31 @@ func TestARecipeIsReadAsInstructions(t *testing.T) {
t.Fatalf("a heredoc line or a continued stage was read as a base: %v", bases)
}
}
// What a build was handed as its bases is what it stood on — recorded, so a changed base knows what
// to rebuild (novox/hq 04-ISSUES/131). A recipe reads the base from an argument, so nothing else
// could know.
func TestTheBasesABuildWasHandedAreWhatItStoodOn(t *testing.T) {
manifest := catalogue.Manifest{
Module: "gitea",
Build: &catalogue.Build{
On: []catalogue.BuildsOn{
{Arg: "RUNTIME_BASE", Module: "mesh-tools", Artifact: "runtime"},
{Arg: "BUILD_BASE", Module: "mesh-tools", Artifact: "build"},
},
Artifacts: []catalogue.Artifact{{Name: "runtime", Kind: catalogue.ArtifactImage, From: "Dockerfile"}},
},
}
held := map[string]string{
"mesh-tools/runtime": "127.0.0.1:5000/mesh-tools/runtime@sha256:" + strings.Repeat("a", 64),
"mesh-tools/build": "127.0.0.1:5000/mesh-tools/build@sha256:" + strings.Repeat("b", 64),
}
_, resolved, err := standingOn(context.Background(), manifest, held, noMirror)
if err != nil {
t.Fatal(err)
}
got := against(t.TempDir(), manifest, resolved)
if len(got) != 2 || got[0] != held["mesh-tools/build"] || got[1] != held["mesh-tools/runtime"] {
t.Fatalf("the bases the build was handed were not what it stood on: %v", got)
}
}
@@ -0,0 +1,48 @@
package catalogue
import (
"os"
"path/filepath"
"strings"
"testing"
)
// **The word does not come back through a manifest** (novox/hq ADR 0131). A module that wants
// messaging wants the mesh's bus, reached through the sdk and named by the `mesh-broker` seat. Naming
// the old wire protocol asks for the one server being retired, so both directions are refused at the
// parser — this is judged from the manifest alone, no store needed.
func TestAManifestProvidingAmqpIsRefused(t *testing.T) {
raw := []byte(`{"module":"old-broker","version":"1","provides":[{"name":"amqp","scope":"mesh"}]}`)
_, err := ParseManifest(raw)
if err == nil || !strings.Contains(err.Error(), `provides "amqp", which is not a provision`) {
t.Fatalf("a module providing amqp was not refused, or not for the reason: %v", err)
}
}
func TestAManifestRequiringAmqpIsRefused(t *testing.T) {
raw := []byte(`{"module":"forwarder","version":"1","requires":["amqp"]}`)
_, err := ParseManifest(raw)
if err == nil || !strings.Contains(err.Error(), `requires "amqp", which is not a provision`) {
t.Fatalf("a module requiring amqp was not refused, or not for the reason: %v", err)
}
}
// And the catalogue as checked out beside this repository names it nowhere — the three modules that
// did are removed under design 28 task 5.4, not converted.
func TestNoCatalogueManifestNamesAmqp(t *testing.T) {
modules, err := filepath.Glob("../../../mesh-catalog/modules/*/module.json")
if err != nil || len(modules) == 0 {
t.Skip("the catalogue is not checked out beside this repository")
}
for _, path := range modules {
raw, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
if strings.Contains(string(raw), `"amqp"`) {
t.Errorf("%s names amqp, which is not a provision (novox/hq ADR 0131)",
filepath.Base(filepath.Dir(path)))
}
}
}
+87
View File
@@ -0,0 +1,87 @@
package catalogue
import (
"strings"
"testing"
)
// The mesh's bus is one per mesh, read from the catalogue beside this checkout.
//
// **This is step 2's claim, and it is checked here rather than in a bed** (novox/hq ADR 0116):
// adoption puts the NATS server into the `mesh-broker` seat on a mesh that is already running,
// and the property that matters is that a second one anywhere is refused *when it is assigned*,
// not discovered later as two servers holding different halves of the mesh's traffic. A second
// bus is not a degraded mesh; it is two meshes that both believe they are the one.
func TestASecondMeshBusAnywhereIsRefusedByName(t *testing.T) {
nats := catalogueManifest(t, "nats")
if _, err := Resolve(shelf(nats), []string{"nats"}, workstation(), World{}); err != nil {
t.Fatalf("the bus alone does not resolve: %v", err)
}
elsewhere := World{Held: []Held{{Claim: "mesh-broker", Scope: ScopeMesh,
Node: "anchor", Module: "nats"}}}
other := workstation()
other.Name = "laptop"
_, err := Resolve(shelf(nats), []string{"nats"}, other, elsewhere)
if err == nil {
t.Fatal("a second bus was accepted on another machine")
}
if !strings.Contains(err.Error(), "mesh-broker") || !strings.Contains(err.Error(), "one per mesh") {
t.Fatalf("refused without naming the seat: %v", err)
}
}
// The seat is the server's role, not the product's name (novox/hq ADR 0079). A different
// implementation of the bus claims the same seat, and the mesh refuses it for the same reason —
// which is the property that lets the bus be replaced at all.
func TestTheSeatRefusesADifferentBusToo(t *testing.T) {
nats := catalogueManifest(t, "nats")
held := World{Held: []Held{{Claim: "mesh-broker", Scope: ScopeMesh,
Node: "anchor", Module: "some-other-broker"}}}
other := workstation()
other.Name = "laptop"
if _, err := Resolve(shelf(nats), []string{"nats"}, other, held); err == nil {
t.Fatal("the seat admitted a second holder because the module's name differed")
}
}
// The old broker is gone from the catalogue (novox/hq ADR 0131, design 28 task 5.4), so it is no
// longer a fixture here. That two eligible holders stand beside each other with one on record is
// pinned in holdings_test.go against manifests this package owns.
// **A seat and the interface it delivers are different names, and renaming one must not rename
// the other** (novox/hq ADR 0118). This nearly went wrong: the seats were renamed to the `mesh-*`
// prefix, and a blanket search-and-replace also renamed `npm-package-registry` and `git` where
// they are *provisions* — which a consumer requires and a provider offers. The tests failed with
// "the package registry is served on <nil>", which does not say "you renamed an interface".
func TestRenamingASeatDidNotRenameTheInterfaceItDelivers(t *testing.T) {
for _, pair := range []struct{ seat, delivers string }{
{"git", "git"},
{"npm-package-registry", "npm-package-registry"},
{"the-artifact-store", "artifact-store"},
{"mesh-store", "postgres-database"},
{"mesh-broker", "mesh-bus"},
} {
s, known := SeatNamed(pair.seat)
if !known {
t.Fatalf("%q is not a seat", pair.seat)
}
if s.Delivers != pair.delivers {
t.Errorf("the %s seat delivers %q, expected %q — renaming the seat moved the "+
"interface with it, and every consumer requiring it would stop resolving",
pair.seat, s.Delivers, pair.delivers)
}
// **Three of these deliberately share a name with what they deliver**, and that is not an
// incomplete rename. Renaming a seat that delivers a provision cascades to every consumer
// requiring it, with a mesh-wide window where a holder stops resolving mid-flight — so the
// trunk deferred exactly those three (novox/hq ADR 0121) while renaming the node-scoped ones.
// What this test is for is the other direction: that renaming a seat never moves the
// interface, which once produced "the package registry is served on <nil>".
}
}
// A manifest written against an old seat name is told what it became rather than refused as
// unknown. **That map is the controller's store now, not this package** (novox/hq ADR 0122): a
// rename is a row, so the courtesy survives a rename nobody recompiled for. Checked where the
// table is read, not here, where there is no longer a hardcoded list to check against.
+74 -2
View File
@@ -97,6 +97,29 @@ type Rendering struct {
// compose it a second time.
Suffix string
// BusUsers is the mesh's composed user list, for the module holding `mesh-broker`. Empty on
// every other node, and on this one until the controller has composed it.
//
// **Only the users, never the server's own settings**: those are the module's, in its image and
// its mounts (Manifest.BusUsers).
BusUsers string
// BusMembership is this machine's membership for the bus the mesh is moving to, sealed to it
// (design 28, task 5.2). Empty for a machine not being moved. Written as a file the host reads
// after the declaration has applied, so the bus it names is standing before the machine leaves
// the one it is on.
BusMembership string
// MeshRange is the private network's CIDR (the range node addresses are allocated from), for a
// module that must name the whole mesh rather than one machine — an intrusion filter that must
// never ban a tunnel peer, say. A per-mesh value the module cannot know, so it is carried here
// and offered as ${machine:mesh-range}, the same way one machine's address is.
MeshRange string
// Accounts is each machine's operator account, by the same internal name Names uses (novox/hq
// to-be 29). What an ssh Host block's `User` line is composed from; empty for a machine no
// operator account is known on.
Accounts map[string]string
// Kept is every operator-sealed secret in the mesh, for a module that `keeps` them. Nil when
// nothing on this node keeps them, or the mesh has no operator key.
Kept *KeptExport
@@ -220,9 +243,23 @@ func (r Resolution) Compose(with Rendering) (Composed, error) {
if err != nil {
return Composed{}, err
}
if with.BusMembership != "" {
// The machine's own, not any module's: how it reaches the mesh from now on. Sealed like a
// secret and placed where the host looks for exactly this (design 28, task 5.2).
resources = append(resources, map[string]any{
"id": BusMembershipID(), "type": "file", "path": BusMembershipPath,
"sealed": with.BusMembership, "mode": "0600",
})
}
return Composed{Resources: resources, Owner: owner}, nil
}
// BusMembershipID names the resource carrying a machine's membership for the new bus, and
// BusMembershipPath is where the host reads it — the same constant on both sides.
func BusMembershipID() string { return "bus-membership" }
const BusMembershipPath = "/var/lib/mesh/membership-next.json"
func (r Resolution) compose(with Rendering, owner map[string]string) ([]map[string]any, error) {
// Every manifest is placed first (novox/hq ADR 0112): the maps naming where its bindings,
// credentials and contributions land are resolved against this node's directories, so every
@@ -339,6 +376,12 @@ func (r Resolution) compose(with Rendering, owner map[string]string) ([]map[stri
"content": filtering, "mode": "0600",
})
}
// The node's fail2ban jails, composed from every module it runs (novox/hq to-be 31), written
// where the intrusion-prevention holder owns them. Like the rule set above: gathered from all
// modules, written by the one that holds the role.
if j := m.Jailing; j != nil {
first = append(first, jailsInto(r.Modules, j)...)
}
if c := m.Certificate; c != nil {
if with.Certificate == "" {
// Asked for and not issued. Refused rather than skipped: a module that serves TLS
@@ -359,6 +402,35 @@ func (r Resolution) compose(with Rendering, owner map[string]string) ([]map[stri
})
}
}
if m.BusUsers != "" {
// **The claim authorises it, not the field.** This file holds every user's password
// hash, so a module that could ask for it could read every credential on the bus.
// Checked from this manifest alone, which is the cheapest check there is: whether some
// other module also claims the seat is resolution's business elsewhere, and one holder
// mesh-wide is already guaranteed.
if !m.ClaimsSeat("mesh-broker") {
return nil, fmt.Errorf(
"%s asks for the mesh's user list and does not claim mesh-broker. That file "+
"holds every user's password hash, so the seat is what authorises it",
m.Module)
}
if with.BusUsers == "" {
// Asked for and not composed. Refused rather than skipped, for the reason a
// certificate is: a bus with no user list refuses every connection in the mesh, and
// an empty file would look like a configuration problem on the machine.
return nil, fmt.Errorf(
"%s holds mesh-broker and the mesh composed no user list, so the bus would "+
"refuse every connection", m.Module)
}
first = append(first, map[string]any{
"id": BusUsersID(), "type": "file", "path": m.BusUsers,
"content": with.BusUsers,
// Readable by the server and nothing else. Hashes rather than passwords, so this is
// not a set of working credentials — but a list of every user in the mesh is worth
// keeping to the one process that needs it.
"mode": "0600",
})
}
for _, name := range sortedKeys(m.OwnSecrets) {
sealed := with.Needed[m.Module][name]
if sealed == "" {
@@ -545,7 +617,7 @@ func (r Resolution) compose(with Rendering, owner map[string]string) ([]map[stri
// (novox/hq ADR 0112) — resolved once per module, named by ${dir:…} from any resource.
dirs := dirsFor(m, with)
// And the machine underneath, which no binding of its own can tell it.
thisMachine := machineFacts(r, with.Names)
thisMachine := machineFacts(r, with.Names, with.MeshRange)
// Which of this module's files carry a secret, for the rule that a container may not read
// one of them as its environment without saying so (ADR 0086, issue 041).
@@ -644,7 +716,7 @@ func (r Resolution) compose(with Rendering, owner map[string]string) ([]map[stri
// plane's; making a name resolve is the module's software. Emitted as ordinary files under
// this module's name, so they are applied, reported and removed exactly as anything else
// it declares.
given, err := FactsInto(m, r, with.Names, with.Machines, with.Suffix)
given, err := FactsInto(m, r, with.Names, with.Machines, with.Accounts, with.Suffix)
if err != nil {
return nil, err
}
+80
View File
@@ -0,0 +1,80 @@
package catalogue
import "testing"
// The mesh's user list reaches the module holding the bus, and nothing else.
//
// Three refusals and one delivery, because each of the refusals would be silent in a different way:
// a module that asked and was given it could read every credential on the bus; a bus given an empty
// file refuses every connection in the mesh and looks like a machine problem; and a bus that never
// asked gets nothing rather than a file it does not read.
func TestTheMeshsUserListGoesOnlyToTheModuleHoldingTheBus(t *testing.T) {
theBus := func() Manifest {
return Manifest{
Module: "nats", Version: "1",
Claims: []Claim{{Name: "mesh-broker", Scope: ScopeMesh}},
BusUsers: "/var/lib/nats-module/conf/accounts.conf",
Resources: []map[string]any{},
}
}
on := func(t *testing.T, m Manifest, with Rendering) ([]map[string]any, error) {
t.Helper()
return Resolution{Node: "anchor", Modules: []Manifest{m}}.Declaration(with)
}
t.Run("the holder is given it", func(t *testing.T) {
resources, err := on(t, theBus(), Rendering{BusUsers: "accounts { MESH { users = [] } }"})
if err != nil {
t.Fatal(err)
}
// Prefixed with the module it came from, like every resource: two modules may reasonably
// both call something "config", and without the prefix the second would silently replace
// the first.
var found map[string]any
for _, r := range resources {
if r["id"] == "nats."+BusUsersID() {
found = r
}
}
if found == nil {
t.Fatalf("the bus was given no user list: %+v", resources)
}
if found["path"] != "/var/lib/nats-module/conf/accounts.conf" {
t.Errorf("written to %v rather than where the module asked", found["path"])
}
if found["mode"] != "0600" {
t.Errorf("mode %v: a list of every user in the mesh belongs to the one process that "+
"needs it", found["mode"])
}
})
t.Run("a module that does not claim the seat is refused", func(t *testing.T) {
m := theBus()
m.Claims = nil
if _, err := on(t, m, Rendering{BusUsers: "accounts {}"}); err == nil {
t.Fatal("a module that claims nothing was handed every user's password hash")
}
})
t.Run("the holder with nothing composed is refused", func(t *testing.T) {
if _, err := on(t, theBus(), Rendering{}); err == nil {
t.Fatal("the bus was given an empty user list, so it would refuse every connection in " +
"the mesh and look like a machine problem")
}
})
t.Run("a module that did not ask gets nothing", func(t *testing.T) {
m := theBus()
m.BusUsers = ""
resources, err := on(t, m, Rendering{BusUsers: "accounts {}"})
if err != nil {
t.Fatal(err)
}
for _, r := range resources {
if r["id"] == "nats."+BusUsersID() {
t.Fatal("a module that asked for no user list was given one")
}
}
})
}
+161
View File
@@ -0,0 +1,161 @@
package catalogue
import (
"fmt"
"regexp"
"strings"
)
// What a module may call an event, and what a consumer may ask for.
//
// A module names an event **locally**: `order.placed`, not a subject and not a routing key
// (design 29 §1). A consumer names the emitter and the event: `billing.order.placed`. The mesh
// derives the subject from those, so reorganising the subject space leaves every manifest correct.
//
// **Nothing checked this until every manifest in the catalogue was wrong the same way**
// (novox/hq 04-ISSUES/127). All thirty-seven kept the old bus's routing key —
// `module.<module>.<verb>` — which the derivation read as "a module called `module`", so every
// cross-module subscription in the mesh pointed at a namespace nobody publishes to. Nothing failed:
// the services started and none of them reacted. The documentation on these fields taught the old
// form too, which is why the drift was uniform rather than scattered.
// eventName is one name in a local event: lower-case, and no wildcard.
var eventName = regexp.MustCompile(`^[a-z0-9][a-z0-9-]*$`)
// The wildcards a consumer may use, spelled the mesh's way and derived to whatever the transport
// spells them as.
//
// **A manifest holds no transport token**, which is the whole point of naming locally: the bus the
// mesh runs on today spells these `*` and `#`, and the one being built spells them `*` and `>`. A
// manifest that said either would be a manifest that stopped being true when the wire changed.
const (
// OneName stands for exactly one name.
OneName = "*"
// TheRest stands for one or more names, and may only come last.
TheRest = "**"
)
// EventProblems is what is wrong with a manifest's events.
//
// Refused at registration, because the alternative is a module that installs, starts, connects and
// reacts to nothing — and every log line says it is fine.
func EventProblems(m Manifest) []string {
var problems []string
for _, e := range m.Emits {
if was, stale := staleEventForm(e, m.Module); stale {
problems = append(problems, fmt.Sprintf(
"%s emits %q, which is the old bus's routing key. An event is named locally now, so "+
"write %q — the mesh derives the subject (novox/hq design 29 §1)",
m.Module, was, strings.TrimPrefix(was, "module."+m.Module+".")))
continue
}
if strings.HasPrefix(e, "module.") {
problems = append(problems, fmt.Sprintf(
"%s emits %q: `module.` is reserved, because it is how the old bus spelled a "+
"routing key and an event named that way derives into a namespace nobody owns",
m.Module, e))
continue
}
if err := localName(e); err != nil {
problems = append(problems, fmt.Sprintf("%s emits %q: %v", m.Module, e, err))
continue
}
// **Its own name, never another's.** The bus enforces that a namespace belongs to the module
// it is named for, so an event named for somebody else cannot be published at all. If the
// event is about a role rather than about this module, it belongs on the seat: a name that
// is stable across whoever fills it (04-ISSUES/127).
if first, _, split := strings.Cut(e, "."); split && isAModuleNameOtherThan(first, m.Module) {
problems = append(problems, fmt.Sprintf(
"%s emits %q, which reads as another module's event. A module publishes under its "+
"own name only. If this is about a role rather than about %s, declare it on that "+
"seat, where the name survives the holder changing",
m.Module, e, m.Module))
}
}
for _, c := range m.Consumes {
if strings.HasPrefix(c, "module.") {
problems = append(problems, fmt.Sprintf(
"%s consumes %q, which is the old bus's pattern. A consumed event names its emitter "+
"and the event: write %q", m.Module, c, strings.TrimPrefix(c, "module.")))
continue
}
if c == "#" {
problems = append(problems, fmt.Sprintf(
"%s consumes %q, which is the old bus's wildcard for everything. Write %q",
m.Module, c, TheRest))
continue
}
if err := consumePattern(c); err != nil {
problems = append(problems, fmt.Sprintf("%s consumes %q: %v", m.Module, c, err))
}
}
return problems
}
// staleEventForm says an emitted name is this module's own old routing key, and what it was.
func staleEventForm(event, module string) (string, bool) {
return event, module != "" && strings.HasPrefix(event, "module."+module+".")
}
// isAModuleNameOtherThan says a first token names some module of this mesh that is not this one.
//
// Only the mesh's own seats and the catalogue could answer this properly, and neither is reachable
// from a parser given one manifest. So this catches the case that actually happened — a name that
// is a *provision* the mesh defines, which is where "another module's event" comes from in practice
// — and the whole-catalogue check catches the rest.
func isAModuleNameOtherThan(first, module string) bool {
if first == module || first == "" {
return false
}
if _, isASeat := SeatNamed(first); isASeat {
return true
}
if _, isASeat := SeatDelivering(first); isASeat {
return true
}
return false
}
// localName checks one event name: dot-separated names, no wildcards, nothing else.
func localName(event string) error {
if event == "" {
return fmt.Errorf("an event needs a name")
}
for _, part := range strings.Split(event, ".") {
if part == OneName || part == TheRest {
return fmt.Errorf("an emitted event names one event, so it carries no wildcard")
}
if !eventName.MatchString(part) {
return fmt.Errorf("%q is not a usable name: lower-case letters, digits and dashes", part)
}
}
return nil
}
// consumePattern checks a consumed pattern: the emitter, then the event, with wildcards.
func consumePattern(pattern string) error {
if pattern == "" {
return fmt.Errorf("a consumed event needs an emitter and an event")
}
parts := strings.Split(pattern, ".")
for i, part := range parts {
switch {
case part == TheRest:
if i != len(parts)-1 {
return fmt.Errorf("%q stands for the rest of a name, so nothing may follow it", TheRest)
}
case part == OneName:
case !eventName.MatchString(part):
return fmt.Errorf("%q is not a usable name: lower-case letters, digits and dashes", part)
}
}
// `**` alone is every event from every module, which the audit logger wants and says plainly.
if len(parts) == 1 && parts[0] != TheRest {
return fmt.Errorf(
"%q names an emitter and no event. Write <emitter>.<event>, or %q for every event",
pattern, TheRest)
}
return nil
}
-184
View File
@@ -1,184 +0,0 @@
package catalogue
import (
"fmt"
"sort"
"strings"
)
// What only the mesh knows, written where a module asks for it.
//
// **The graph is the control plane's; using it is the module's.** The mesh knows which machines
// exist, what they are called, and where they are. Turning that into a name that resolves is
// somebody's software, and which software is a choice the mesh should not be making.
//
// This replaced three modules — names, a resolver's data, and the private network's own
// configuration — that existed only because computed output needed somewhere to live. They ran no
// software and could not be swapped for anything, which is the test of whether something is a
// module at all (novox/hq ADR 0040).
const (
// FactNodeNames is every machine's name and address, as a hosts file.
//
// Exact names only: `homer` and `homer.internal` resolve to homer. Anything *under* a machine
// is a wildcard, which a hosts file cannot express — that is FactNodeZones.
FactNodeNames = "node-names"
// FactNodeZones is every machine as a wildcard: `*.homer.internal` is homer.
//
// Written in the form a resolver reads. A machine's own name and everything under it are one
// fact — if homer is at an address, so is anything homer serves.
FactNodeZones = "node-zones"
)
// facts is every fact the mesh computes, and what writes it.
//
// **A closed list.** A module asking for a fact the mesh does not have is asking for a file nobody
// will write, and finding that out on a machine — as a daemon that starts, reads nothing, and
// answers no queries — is worse than being told where the manifest is.
// A fact is written from the names it is about. `every` is every name the mesh serves — machines
// and the names it was told to route; `machines` is only the machines. A fact takes the set it is
// true of, and the two must not be confused (novox/hq 04-ISSUES/111).
var facts = map[string]func(r Resolution, every, machines map[string]string, suffix string) string{
FactNodeNames: func(r Resolution, every, _ map[string]string, suffix string) string {
return nodeNames(r, every, suffix)
},
FactNodeZones: func(r Resolution, _, machines map[string]string, suffix string) string {
return nodeZones(r, machines, suffix)
},
}
// FactsInto renders the facts a module asked for, as files it will be given.
//
// The module owns everything after the file exists: loading it, restarting on it, what a resolver
// does with it. This only puts it there.
func FactsInto(m Manifest, r Resolution, addresses, machines map[string]string, suffix string) ([]map[string]any, error) {
if len(m.Facts) == 0 {
return nil, nil
}
names := make([]string, 0, len(m.Facts))
for name := range m.Facts {
names = append(names, name)
}
sort.Strings(names)
out := make([]map[string]any, 0, len(names))
for _, name := range names {
write, known := facts[name]
if !known {
return nil, fmt.Errorf(
"%s asks the mesh for %q, which it does not compute. It has %s",
m.Module, name, spokenFacts())
}
path := m.Facts[name]
if !strings.HasPrefix(path, "/") {
return nil, fmt.Errorf(
"%s asks for %q at %q, which is not an absolute path", m.Module, name, path)
}
out = append(out, map[string]any{
"id": "fact-" + name, "type": "file", "path": path, "mode": "0644",
"content": write(r, addresses, machines, suffix),
})
}
return out, nil
}
// spokenFacts lists them, so a refusal says what would have worked.
func spokenFacts() string {
names := make([]string, 0, len(facts))
for name := range facts {
names = append(names, name)
}
sort.Strings(names)
return strings.Join(names, ", ")
}
// nodeNames is every machine's name and address, as a hosts file.
//
// **A machine with no address is left out.** The mesh has a record for it — somebody added it —
// and does not yet know where it is, which is the ordinary state between adding a machine and it
// joining. Writing the name anyway would give a name that resolves to nothing, and a connection to
// that hangs; leaving it out fails at once and says the name is unknown.
func nodeNames(r Resolution, addresses map[string]string, suffix string) string {
var b strings.Builder
b.WriteString("# Generated by the mesh. Do not edit — this file is replaced whenever a machine\n")
b.WriteString("# joins or leaves, and an edit would survive until then and vanish.\n\n")
// The floor every Linux expects, and which removing would break things that have nothing to do
// with the mesh.
b.WriteString("127.0.0.1\tlocalhost\n")
b.WriteString("::1\t\tlocalhost ip6-localhost ip6-loopback\n")
if r.Node != "" {
fmt.Fprintf(&b, "127.0.1.1\t%s\n", r.Node)
}
b.WriteString("\n")
for _, name := range sortedNames(addresses) {
at := addresses[name]
internal, bare := meshName(name, suffix)
// Its mesh name resolves to its address on the private network rather than to loopback,
// so a service binding the name it was given stays reachable from everywhere else.
fmt.Fprintf(&b, "%s\t%s\t%s", at, internal, bare)
if bare == r.Node {
b.WriteString("\t# this machine")
}
b.WriteString("\n")
}
return b.String()
}
// nodeZones is every machine as a wildcard, in the form a resolver reads.
//
// `*.homer.internal` is homer, which is the whole rule: if homer is at an address, so is anything
// homer serves. A module wanting this runs the resolver; the mesh only says what is true.
//
// **And the suffix itself, as a local domain.** A resolver that forwards what it cannot answer
// would otherwise send a mesh name it does not know — a machine that left, a typo — to a public
// resolver, which is a leak of the mesh's names for no answer. `local=` keeps everything under the
// suffix here: answered from the lines below or refused. Written in this file rather than in the
// resolver's own configuration because the suffix is the mesh's choice (the operator may have
// picked another) and this file is the one place the mesh writes what it chose.
func nodeZones(_ Resolution, addresses map[string]string, suffix string) string {
var b strings.Builder
b.WriteString("# Generated by the mesh. Do not edit — this file is replaced whenever a machine\n")
b.WriteString("# joins or leaves, and an edit would survive until then and vanish.\n\n")
fmt.Fprintf(&b, "local=/%s/\n", strings.TrimPrefix(suffixOr(suffix), "."))
for _, name := range sortedNames(addresses) {
internal, _ := meshName(name, suffix)
fmt.Fprintf(&b, "address=/%s/%s\n", internal, addresses[name])
}
return b.String()
}
// meshName is a machine's internal name and its bare one, from either. The control plane keys
// the names it hands a resolution by the internal name (`homer.internal`), the same map a
// container gets as its hosts; a caller that keys by the bare name gets the same answer. The
// suffix is the one the control plane composed those names with, handed down rather than written
// here a second time — the alternative was `homer.internal.internal` on every machine.
func meshName(name, suffix string) (internal, bare string) {
dotted := "." + strings.TrimPrefix(suffixOr(suffix), ".")
if strings.HasSuffix(name, dotted) {
return name, strings.TrimSuffix(name, dotted)
}
return name + dotted, name
}
// suffixOr is the suffix given, or the one the mesh composes names with when none was handed down.
// The one place the default is written in this file, so a fact and a name cannot disagree about it.
func suffixOr(suffix string) string {
if suffix == "" {
return "internal"
}
return suffix
}
func sortedNames(addresses map[string]string) []string {
out := make([]string, 0, len(addresses))
for name, at := range addresses {
// See nodeNames: a machine the mesh cannot place is left out rather than named at nothing.
if at == "" {
continue
}
out = append(out, name)
}
sort.Strings(out)
return out
}
-188
View File
@@ -1,188 +0,0 @@
package catalogue
import (
"strings"
"testing"
)
// Keyed by the internal name, as the control plane hands them (issue 079).
var threeMachines = map[string]string{"homer.internal": "10.42.0.1", "marge.internal": "10.42.0.2", "bart.internal": ""}
// **`*.homer.internal` is homer. That is the whole rule.** And the suffix itself is local: a
// resolver that forwards what it cannot answer must not send a mesh name it does not know — a
// machine that left, a typo — to a public resolver (hal dnsmasq-app conversion, novox/hq
// 08-connectivity).
func TestEveryMachineIsAWildcardUnderItsOwnName(t *testing.T) {
out := nodeZones(Resolution{Node: "homer"}, threeMachines, "")
for _, want := range []string{
"local=/internal/",
"address=/homer.internal/10.42.0.1",
"address=/marge.internal/10.42.0.2",
} {
if !strings.Contains(out, want) {
t.Fatalf("missing %q:\n%s", want, out)
}
}
}
// A machine the mesh has a record for and cannot place is left out of both.
//
// **Not an oversight — the alternative is worse.** A name written with no address resolves to
// nothing, and a connection to that hangs. Leaving it out fails at once and says the name is
// unknown, which is a thing somebody can act on.
func TestAMachineWithNoAddressIsNotNamed(t *testing.T) {
for _, out := range []string{
nodeNames(Resolution{Node: "homer"}, threeMachines, ""),
nodeZones(Resolution{Node: "homer"}, threeMachines, ""),
} {
if strings.Contains(out, "bart") {
t.Fatalf("a machine with no address was named, so its name resolves to nothing:\n%s", out)
}
}
}
// A machine's own mesh name points at its address on the private network, not at loopback — or a
// service binding the name it was given is unreachable from everywhere else.
func TestAMachinesOwnNameIsItsMeshAddress(t *testing.T) {
out := nodeNames(Resolution{Node: "homer"}, threeMachines, "")
var line string
for _, l := range strings.Split(out, "\n") {
if strings.Contains(l, "homer.internal") {
line = l
}
}
if !strings.HasPrefix(line, "10.42.0.1") {
t.Fatalf("a machine's own mesh name is not its mesh address: %q", line)
}
// And the loopback floor is still there, or things with nothing to do with the mesh break.
if !strings.Contains(out, "127.0.0.1\tlocalhost") {
t.Fatalf("the loopback floor was removed:\n%s", out)
}
}
// A module says where it wants a fact, and is given a file.
func TestAModuleIsGivenTheFactsItAskedFor(t *testing.T) {
m := Manifest{Module: "dnsmasq", Facts: map[string]string{FactNodeZones: "/etc/mesh/zones.conf"}}
given, err := FactsInto(m, Resolution{Node: "homer"}, threeMachines, threeMachines, "")
if err != nil {
t.Fatal(err)
}
if len(given) != 1 {
t.Fatalf("expected one file, got %d", len(given))
}
if given[0]["path"] != "/etc/mesh/zones.conf" || given[0]["type"] != "file" {
t.Fatalf("not written where it was asked for: %v", given[0])
}
if !strings.Contains(given[0]["content"].(string), "homer.internal") {
t.Fatalf("the file does not hold the fact: %v", given[0]["content"])
}
}
// **Asking for a fact the mesh does not have is refused here, not on a machine.** A daemon that
// starts, reads a file nobody wrote, and answers no queries is a much worse way to find out.
func TestAskingForAFactTheMeshDoesNotHaveIsRefused(t *testing.T) {
m := Manifest{Module: "dnsmasq", Facts: map[string]string{"the-weather": "/etc/weather"}}
_, err := FactsInto(m, Resolution{}, nil, nil, "")
if err == nil {
t.Fatal("a module asked for something nobody computes and was given nothing, silently")
}
for _, known := range []string{FactNodeNames, FactNodeZones} {
if !strings.Contains(err.Error(), known) {
t.Fatalf("the refusal does not say what would have worked: %v", err)
}
}
}
// And a relative path is refused, or a module decides where the mesh writes on a machine.
func TestAFactMustBeAskedForAtAnAbsolutePath(t *testing.T) {
m := Manifest{Module: "dnsmasq", Facts: map[string]string{FactNodeNames: "etc/hosts"}}
if _, err := FactsInto(m, Resolution{}, nil, nil, ""); err == nil {
t.Fatal("a relative path was accepted")
}
}
// **The names the control plane hands a resolution are already internal names** — `homer.internal`,
// the same map every container gets as its hosts. Appending the suffix again wrote
// `homer.internal.internal` into every hosts file and every resolver's zones, and the large mesh
// bed's name test was the first to read it back. Either key gives the same files.
func TestNamesKeyedByInternalNameAreNotSuffixedTwice(t *testing.T) {
internal := map[string]string{"homer.internal": "10.42.0.1", "marge.internal": "10.42.0.2"}
bare := map[string]string{"homer": "10.42.0.1", "marge": "10.42.0.2"}
if a, b := nodeZones(Resolution{Node: "homer"}, internal, ""), nodeZones(Resolution{Node: "homer"}, bare, ""); a != b {
t.Fatalf("the zones differ by how the names were keyed:\n%s\n---\n%s", a, b)
}
if a, b := nodeNames(Resolution{Node: "homer"}, internal, ""), nodeNames(Resolution{Node: "homer"}, bare, ""); a != b {
t.Fatalf("the hosts differ by how the names were keyed:\n%s\n---\n%s", a, b)
}
zones := nodeZones(Resolution{Node: "homer"}, internal, "")
if strings.Contains(zones, "internal.internal") || !strings.Contains(zones, "address=/homer.internal/10.42.0.1") {
t.Fatalf("the zones carry a doubled suffix or miss the name:\n%s", zones)
}
hosts := nodeNames(Resolution{Node: "homer"}, internal, "")
if !strings.Contains(hosts, "10.42.0.1\thomer.internal\thomer\t# this machine") {
t.Fatalf("the hosts line for the machine itself is not name, bare name and the mark:\n%s", hosts)
}
}
// The suffix the control plane composed the names with is the one the facts write — an operator
// who chose another does not get `.internal` appended to it.
func TestTheFactsWriteTheSuffixTheNamesWereComposedWith(t *testing.T) {
names := map[string]string{"homer.lan": "10.42.0.1"}
zones := nodeZones(Resolution{Node: "homer"}, names, "lan")
if !strings.Contains(zones, "address=/homer.lan/10.42.0.1") || strings.Contains(zones, "internal") {
t.Fatalf("the zones do not carry the operator's suffix as given:\n%s", zones)
}
if !strings.Contains(zones, "local=/lan/") {
t.Fatalf("the local domain is not the operator's suffix, so its names would leak upstream:\n%s", zones)
}
hosts := nodeNames(Resolution{Node: "homer"}, names, "lan")
if !strings.Contains(hosts, "10.42.0.1\thomer.lan\thomer\t# this machine") {
t.Fatalf("the hosts line does not carry the operator's suffix as given:\n%s", hosts)
}
}
// novox/hq 04-ISSUES/111: the map the control plane hands a resolution holds every name the mesh
// serves — the machines, and the names it was told to route to whichever machine serves them. A
// container's hosts wants all of it. A resolver's zones want only the machines: told the mesh's
// suffix is its own, it answers authoritatively for everything under it and forwards nothing, so a
// routed name written there with the suffix appended is a name nobody will ever ask for, standing
// beside the machines and looking as real.
func TestTheResolverIsToldTheMachinesAndNotTheNamesTheMeshMerelyServes(t *testing.T) {
machines := map[string]string{"homer.internal": "10.42.0.1", "marge.internal": "10.42.0.2"}
every := map[string]string{
"homer.internal": "10.42.0.1", "marge.internal": "10.42.0.2",
"drive.example.test": "10.42.0.1", "git.example.test": "10.42.0.2",
}
m := Manifest{Module: "resolver", Facts: map[string]string{
FactNodeZones: "/etc/zones.conf", FactNodeNames: "/etc/hosts",
}}
given, err := FactsInto(m, Resolution{Node: "homer"}, every, machines, "")
if err != nil {
t.Fatal(err)
}
by := map[string]string{}
for _, f := range given {
by[f["path"].(string)] = f["content"].(string)
}
zones := by["/etc/zones.conf"]
for _, machine := range []string{"address=/homer.internal/10.42.0.1", "address=/marge.internal/10.42.0.2"} {
if !strings.Contains(zones, machine) {
t.Fatalf("the resolver was not told %q:\n%s", machine, zones)
}
}
for _, served := range []string{"drive.example.test", "git.example.test"} {
if strings.Contains(zones, served) {
t.Fatalf("the resolver was told %q, a name the mesh serves rather than a machine:\n%s", served, zones)
}
}
// And the hosts file is the other way about: every name, so a container reaching a routed name
// finds the machine serving it.
hosts := by["/etc/hosts"]
for _, name := range []string{"homer.internal", "drive.example.test", "git.example.test"} {
if !strings.Contains(hosts, name) {
t.Fatalf("a container would not resolve %q from its hosts:\n%s", name, hosts)
}
}
}
@@ -28,8 +28,10 @@ func TestTheStoreAndTheBrokerSayWhatTheMeshGuards(t *testing.T) {
if got := catalogueManifest(t, "postgres").Guards; !reflect.DeepEqual(got, []int{5432}) {
t.Errorf("postgres guards %v; the store's port must be refused from outside", got)
}
if got := catalogueManifest(t, "lavinmq").Guards; !reflect.DeepEqual(got, []int{15672}) {
t.Errorf("lavinmq guards %v; the management port must be refused from outside", got)
// The bus's monitoring port, not its client port: a node reaches the bus, nobody outside
// reads its state (novox/hq ADR 0131 — the broker that guarded 15672 has left the catalogue).
if got := catalogueManifest(t, "nats").Guards; !reflect.DeepEqual(got, []int{8222}) {
t.Errorf("nats guards %v; the monitoring port must be refused from outside", got)
}
}
+160
View File
@@ -0,0 +1,160 @@
package catalogue
import (
"strings"
"testing"
)
// **A seat's holder on record settles who holds it, and lets the next holder stand beside the
// current one** (novox/hq ADR 0131, design 28 task 5.3). Until the record existed, two assignments
// whose modules both claimed a seat were refused outright — which left no way to hand a seat over
// without a moment where nobody held it, and the control plane finds its own bus through one of
// these seats. That moment was the outage of 2026-09-27.
func busSeatDelivering(t *testing.T, delivers string) {
t.Helper()
was := Seats()
t.Cleanup(func() { UseSeats(was) })
UseSeats([]Seat{{Name: "mesh-broker", Scope: ScopeMesh, Delivers: delivers, Decision: "test"}})
}
func oldBroker() Manifest {
return Manifest{Module: "old-broker", Provides: []Offer{{Name: "mesh-bus", Scope: ScopeMesh}},
Claims: []Claim{{Name: "mesh-broker", Scope: ScopeMesh}}}
}
func newBroker() Manifest {
return Manifest{Module: "new-broker", Provides: []Offer{{Name: "mesh-bus", Scope: ScopeMesh}},
Claims: []Claim{{Name: "mesh-broker", Scope: ScopeMesh}}}
}
// Nothing on record: exactly the old rule. One claimant holds; two are refused.
func TestWithNoHolderOnRecordTheSoleClaimantHoldsAndTwoAreRefused(t *testing.T) {
busSeatDelivering(t, "mesh-bus")
node := Node{Name: "anchor"}
held, problems := checkClaims([]Manifest{oldBroker()}, node, nil, nil)
if len(problems) != 0 || len(held) != 1 || held[0].Module != "old-broker" {
t.Fatalf("a sole claimant did not hold the seat: held=%v problems=%v", held, problems)
}
_, problems = checkClaims([]Manifest{oldBroker(), newBroker()}, node, nil, nil)
if len(problems) != 1 || !strings.Contains(problems[0], "both claim") {
t.Fatalf("two claimants with nothing on record were not refused: %v", problems)
}
}
// With a holder on record, the other eligible assignment is silent: not refused, and not holding.
func TestTheHolderOnRecordHoldsAndTheOtherClaimantStandsBesideIt(t *testing.T) {
busSeatDelivering(t, "mesh-bus")
node := Node{Name: "anchor"}
record := []Held{{Claim: "mesh-broker", Scope: ScopeMesh, Node: "anchor", Module: "new-broker"}}
held, problems := checkClaims([]Manifest{oldBroker(), newBroker()}, node, nil, record)
if len(problems) != 0 {
t.Fatalf("the assignment beside the holder was refused: %v", problems)
}
if len(held) != 1 || held[0].Module != "new-broker" {
t.Fatalf("the holder on record is not the one holding: %v", held)
}
}
// The record names a node too: an eligible module on another machine holds nothing, and its
// machine's set still resolves.
func TestAHolderOnRecordElsewhereLeavesThisMachinesClaimantSilent(t *testing.T) {
busSeatDelivering(t, "mesh-bus")
record := []Held{{Claim: "mesh-broker", Scope: ScopeMesh, Node: "anchor", Module: "new-broker"}}
held, problems := checkClaims([]Manifest{oldBroker()}, Node{Name: "laptop"}, nil, record)
if len(problems) != 0 || len(held) != 0 {
t.Fatalf("a claimant elsewhere than the recorded holder was not simply silent: held=%v problems=%v",
held, problems)
}
}
// A record naming a seat's former name still applies to it after a rename (ADR 0122).
func TestAHolderRecordedUnderAFormerNameStillHolds(t *testing.T) {
busSeatDelivering(t, "mesh-bus")
wasAliases := aliases
t.Cleanup(func() { UseAliases(wasAliases) })
UseAliases(map[string]string{"the-broker": "mesh-broker"})
record := []Held{{Claim: "the-broker", Scope: ScopeMesh, Node: "anchor", Module: "new-broker"}}
held, problems := checkClaims([]Manifest{oldBroker(), newBroker()}, Node{Name: "anchor"}, nil, record)
if len(problems) != 0 || len(held) != 1 || held[0].Module != "new-broker" {
t.Fatalf("a record under the former name did not settle the seat: held=%v problems=%v", held, problems)
}
}
// CanHold is the one judgement registration and the handover share, against the store's row.
func TestCanHoldJudgesClaimScopeAndWhatTheSeatDelivers(t *testing.T) {
busSeatDelivering(t, "mesh-bus")
seat, _ := SeatNamed("mesh-broker")
if err := CanHold(newBroker(), seat); err != nil {
t.Fatalf("a module that claims the seat and provides what it delivers was refused: %v", err)
}
noClaim := Manifest{Module: "quiet", Provides: []Offer{{Name: "mesh-bus", Scope: ScopeMesh}}}
if err := CanHold(noClaim, seat); err == nil || !strings.Contains(err.Error(), "does not claim") {
t.Fatalf("a module that never claimed the seat was allowed to hold it: %v", err)
}
wrongScope := newBroker()
wrongScope.Claims[0].Scope = ScopeNode
if err := CanHold(wrongScope, seat); err == nil || !strings.Contains(err.Error(), "scope") {
t.Fatalf("a claim at the wrong scope was allowed: %v", err)
}
cannotAnswer := Manifest{Module: "amqp-only", Provides: []Offer{{Name: "amqp", Scope: ScopeMesh}},
Claims: []Claim{{Name: "mesh-broker", Scope: ScopeMesh}}}
if err := CanHold(cannotAnswer, seat); err == nil || !strings.Contains(err.Error(), `does not provide "mesh-bus"`) {
t.Fatalf("a holder that cannot answer for the seat was allowed: %v", err)
}
// And the judgement follows the store's row, not a compiled copy.
busSeatDelivering(t, "amqp")
seat, _ = SeatNamed("mesh-broker")
if err := CanHold(cannotAnswer, seat); err != nil {
t.Fatalf("with the row saying amqp, an amqp provider was refused: %v", err)
}
}
// A machine being moved is handed its membership for the new bus as a sealed file in its own
// declaration — the machine's, not any module's (design 28, task 5.2).
func TestAMembershipForTheNewBusIsComposedAsASealedFile(t *testing.T) {
r := Resolution{Node: "anchor"}
got, err := r.Compose(Rendering{BusMembership: "sealed-blob"})
if err != nil {
t.Fatal(err)
}
var found map[string]any
for _, res := range got.Resources {
if res["id"] == BusMembershipID() {
found = res
}
}
if found == nil {
t.Fatalf("no membership resource in %v", got.Resources)
}
if found["path"] != BusMembershipPath || found["sealed"] != "sealed-blob" || found["mode"] != "0600" {
t.Fatalf("the membership is not a sealed 0600 file where the host reads it: %v", found)
}
// And a machine not being moved is handed nothing.
got, _ = r.Compose(Rendering{})
for _, res := range got.Resources {
if res["id"] == BusMembershipID() {
t.Fatal("a machine with no membership on record was handed one")
}
}
}
// The store's seat rows have no protocol columns yet; loading them must not drop the protocol the
// bus is derived from, or no role's work queue is ever raised (found live, 2026-09-28).
func TestAStoreRowWithoutAProtocolKeepsTheCompiledOne(t *testing.T) {
was := Seats()
t.Cleanup(func() { UseSeats(was) })
UseSeats([]Seat{{Name: "mesh-build-machine", Scope: ScopeMesh, Decision: "row"}})
got, ok := SeatNamed("mesh-build-machine")
if !ok || len(got.Accepts) == 0 {
t.Fatalf("the build machine's seat lost what it accepts when loaded from the store: %+v", got)
}
if got.Decision != "row" {
t.Fatalf("the store's own columns were not kept: %+v", got)
}
}
+106
View File
@@ -0,0 +1,106 @@
package catalogue_test
import (
"reflect"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/overlay"
)
// novox/hq issue 128, held where the host will see it: the shipped networking module's names,
// through the whole composition, and not only through FactsInto.
//
// Composition prefixes a fact's id with the module that asked for it and passes everything else
// through; a step that dropped `into` on the way would send the region as a whole file, and the
// host would write the machine's hosts file over again with every unit test above still green.
// onTheNetwork stands in for the overlay's generator: the node is part of the private network,
// and what the generator writes is not what is under test here.
type onTheNetwork struct{}
func (onTheNetwork) Resources(string) ([]map[string]any, bool, error) {
return []map[string]any{{"id": "overlay-config", "type": "file",
"path": "/etc/wireguard/mesh0.conf", "mode": "0600", "content": "[Interface]\n"}}, true, nil
}
func TestTheHostsRegionArrivesAsTheHostWillReadIt(t *testing.T) {
shelf := provided(t)
// A resolver restarting on the names another module put on the machine, and one resource it
// only runs at start — neither of which composition has any business changing.
resolver, err := catalogue.ParseManifest([]byte(`{
"module": "resolver", "version": "1", "requires": ["mesh-addressing"],
"resources": [
{"id": "seed", "type": "file", "path": "/etc/resolver/seed", "mode": "0644",
"content": "seed\n", "at": "start"},
{"id": "daemon", "type": "service", "unit": "resolver.service", "state": "running",
"restart-on": ["seed", "mesh-wireguard.fact-node-names"]}
]}`))
if err != nil {
t.Fatal(err)
}
shelf[resolver.Module] = resolver
got, err := catalogue.Resolve(shelf, []string{overlay.Domain, "resolver"},
catalogue.Node{Name: "homer", At: "homer.internal"}, catalogue.World{})
if err != nil {
t.Fatal(err)
}
names := map[string]string{"homer.internal": "10.42.0.1", "marge.internal": "10.42.0.2"}
out, err := got.Declaration(catalogue.Rendering{
Names: names, Machines: names, Suffix: "internal",
Generators: map[string]catalogue.Generator{overlay.Name: onTheNetwork{}},
})
if err != nil {
t.Fatal(err)
}
ids := map[string]map[string]any{}
for _, r := range out {
ids[r["id"].(string)] = r
}
hosts := ids[overlay.Name+".fact-node-names"]
if hosts == nil {
t.Fatalf("no names reached the machine; the declaration has %v", keys(ids))
}
if hosts["path"] != "/etc/hosts" || hosts["into"] != "block" {
t.Fatalf("the hosts file is not written into as a region: %v", hosts)
}
content := hosts["content"].(string)
if !strings.Contains(content, "10.42.0.1\thomer.internal\thomer\t# this machine\n") {
t.Errorf("the region does not name the machine:\n%s", content)
}
for _, floor := range []string{"Generated by the mesh", "localhost", "127.0.1.1"} {
if strings.Contains(content, floor) {
t.Errorf("the region carries %q, which is the machine's:\n%s", floor, content)
}
}
// The resolver's reference to it still names a resource the host will be sent.
daemon := ids["resolver.daemon"]
if daemon == nil {
t.Fatalf("the resolver's service was not composed: %v", keys(ids))
}
for _, named := range daemon["restart-on"].([]any) {
if ids[named.(string)] == nil {
t.Errorf("the resolver restarts on %v, which is nothing the host is sent", named)
}
}
if !reflect.DeepEqual(daemon["restart-on"], []any{"resolver.seed", overlay.Name + ".fact-node-names"}) {
t.Errorf("restart-on is %v", daemon["restart-on"])
}
// And a resource's own `at` passes through as the manifest wrote it.
if seed := ids["resolver.seed"]; seed == nil || seed["at"] != "start" {
t.Errorf("a resource's at did not survive composition: %v", seed)
}
}
func keys(m map[string]map[string]any) []string {
out := make([]string, 0, len(m))
for k := range m {
out = append(out, k)
}
return out
}
+62
View File
@@ -0,0 +1,62 @@
package catalogue
import (
"fmt"
"sort"
"strings"
)
// A node's fail2ban jails, composed from the modules it runs (novox/hq to-be 31).
//
// **The same shape as the firewall.** Every module's `listens` become the node's rule set; every
// module's `jails` become the node's fail2ban config. A module that runs an authenticating service
// declares what a break-in on it looks like and how to ban it, naming no node and no path (ADR
// 0112); the intrusion-prevention holder — the one module with `jailing` — gathers them and writes
// them where it owns. A node not running a module has none of its jails.
// jailsInto composes every jail declared by the modules on a node into the files the holder writes:
// one jail file (all stanzas, so the fail2ban service restarts on a single resource) and one filter
// file per jail (its failregex, which fail2ban references by the jail's name).
//
// Owned by the holder, because the directory is: two modules writing into one fail2ban is the
// collision the holder model exists to prevent. Empty when nothing declares a jail — then the file
// is written empty rather than absent, so removing the last jail is an ordinary change the service
// restarts on rather than a file that vanishes.
func jailsInto(modules []Manifest, j *Jailing) []map[string]any {
type declared struct {
module string
jail Jail
}
var jails []declared
for _, m := range modules {
for _, jail := range m.Jails {
jails = append(jails, declared{m.Module, jail})
}
}
// A stable order the host applies as given (ADR 0005), and so the same set composes byte for
// byte every time rather than differing by map iteration.
sort.Slice(jails, func(a, b int) bool { return jails[a].jail.Name < jails[b].jail.Name })
var composed strings.Builder
composed.WriteString("# The mesh's jails, composed from the modules this node runs. Do not edit —\n")
composed.WriteString("# replaced whenever the node's modules change (novox/hq to-be 31).\n")
out := make([]map[string]any, 0, len(jails)+1)
for _, d := range jails {
fmt.Fprintf(&composed, "\n# from %s\n[%s]\nenabled = true\nfilter = %s\n%s\n",
d.module, d.jail.Name, d.jail.Name, strings.TrimRight(d.jail.Jail, "\n"))
// The filter is a file of its own, named as the jail's filter= references it.
out = append(out, map[string]any{
"id": "filter-" + d.jail.Name,
"type": "file", "path": strings.TrimRight(j.FilterInto, "/") + "/" + d.jail.Name + ".conf",
"mode": "0644",
"content": "# Generated by the mesh (from module " + d.module + "). Do not edit.\n" +
"[Definition]\nfailregex = " + d.jail.Failregex + "\n",
})
}
// The one jail file, first, with the fixed id the fail2ban service names in its restart-on.
return append([]map[string]any{{
"id": ComposedJailsID(), "type": "file", "path": j.Into, "mode": "0644",
"content": composed.String(),
}}, out...)
}
+46
View File
@@ -0,0 +1,46 @@
package catalogue
import (
"strings"
"testing"
)
// A node's fail2ban jails are composed from the modules it runs (novox/hq to-be 31): the holder
// (jailing) gathers every module's declared jail into one jail file and a filter file per jail.
func TestJailsAreComposedFromTheNodesModules(t *testing.T) {
modules := []Manifest{
{Module: "fail2ban", Jailing: &Jailing{Into: "/etc/fail2ban/jail.d/mesh-composed.conf", FilterInto: "/etc/fail2ban/filter.d"}},
{Module: "postgres", Jails: []Jail{{Name: "postgres-auth", Failregex: "auth failed from <HOST>", Jail: "port = 5432\nmaxretry = 5"}}},
}
files := jailsInto(modules, modules[0].Jailing)
by := map[string]map[string]any{}
for _, f := range files {
by[f["id"].(string)] = f
}
jail := by[ComposedJailsID()]
if jail == nil || jail["path"] != "/etc/fail2ban/jail.d/mesh-composed.conf" {
t.Fatalf("the composed jail file was not written: %v", jail)
}
body := jail["content"].(string)
if !strings.Contains(body, "[postgres-auth]") || !strings.Contains(body, "filter = postgres-auth") ||
!strings.Contains(body, "port = 5432") {
t.Fatalf("the postgres jail stanza was not composed in:\n%s", body)
}
filter := by["filter-postgres-auth"]
if filter == nil || filter["path"] != "/etc/fail2ban/filter.d/postgres-auth.conf" {
t.Fatalf("the jail's filter file was not written: %v", filter)
}
if !strings.Contains(filter["content"].(string), "failregex = auth failed from <HOST>") {
t.Fatalf("the failregex was not written: %v", filter["content"])
}
}
// A holder whose node runs no jail-declaring module still gets the file, empty — so removing the
// last jail is a change the service restarts on, not a file that vanishes.
func TestTheComposedJailFileIsWrittenEvenWhenEmpty(t *testing.T) {
files := jailsInto([]Manifest{{Module: "fail2ban"}}, &Jailing{Into: "/x", FilterInto: "/f"})
if len(files) != 1 || files[0]["id"] != ComposedJailsID() {
t.Fatalf("the empty composed jail file was not written alone: %v", files)
}
}
+39 -12
View File
@@ -57,7 +57,7 @@ func machineUsed(content string) []string {
// the hosts file and the resolver's wildcards are written from, so a file naming the machine's
// address and the file every other machine reaches it by cannot disagree. Absent, like `at`, when
// the machine is off the network or the mesh has not placed it.
func machineFacts(r Resolution, names map[string]string) map[string]string {
func machineFacts(r Resolution, names map[string]string, meshRange string) map[string]string {
out := map[string]string{"name": r.Node}
if r.At != "" {
out["at"] = r.At
@@ -65,32 +65,59 @@ func machineFacts(r Resolution, names map[string]string) map[string]string {
out["address"] = address
}
}
// The private network's whole range — a mesh-wide fact, not this machine's, but named here
// because a module cannot know it and sometimes must (an intrusion filter that must never ban a
// tunnel peer). Absent when the mesh has no range to give.
if meshRange != "" {
out["mesh-range"] = meshRange
}
// The operator's login on this machine and where its home is (novox/hq to-be 29), so a module
// that writes operator config names the account and its home rather than a value it cannot know.
// Absent when no operator account is known — a headless box a person never logs into.
if r.Account != "" {
out["account"] = r.Account
out["account-home"] = accountHomeOf(r.Account, r.AccountHome)
}
return out
}
// accountHomeOf is where an account's home is: what was stored, or the derived default — /root for
// root, /home/<account> otherwise. The one place the default is written, so a fact and the store
// cannot disagree about it.
func accountHomeOf(account, home string) string {
if home != "" {
return home
}
if account == "root" {
return "/root"
}
return "/home/" + account
}
// machineInto replaces a file's ${machine:…} placeholders with what the mesh knows about the
// machine the module was assigned to.
//
// A key the mesh does not hold is refused, for the same reason a binding's is: left alone, the
// literal would be written into a configuration file and read as a value.
func machineInto(resource map[string]any, facts map[string]string, module string) error {
if fmt.Sprint(resource["type"]) != "file" {
return nil
}
content, ok := resource["content"].(string)
// Content, and now the path and owner too: a module that writes into a person's home names it
// with ${machine:account-home} and ${machine:account}, which it cannot know until assigned
// (novox/hq to-be 29), the same reason its content names ${machine:address}.
for _, field := range []string{"path", "owner", "content"} {
s, ok := resource[field].(string)
if !ok {
return nil
continue
}
for _, key := range machineUsed(content) {
for _, key := range machineUsed(s) {
value, has := facts[key]
if !has {
return fmt.Errorf(
"%s has a file that says ${machine:%s}, and this machine says %s",
module, key, orNothing(namesOfFacts(facts)))
"%s has a %s that says ${machine:%s}, and this machine says %s",
module, field, key, orNothing(namesOfFacts(facts)))
}
s = strings.ReplaceAll(s, fmt.Sprintf("${machine:%s}", key), value)
resource[field] = s
}
resource["content"] = strings.ReplaceAll(
content, fmt.Sprintf("${machine:%s}", key), value)
content = resource["content"].(string)
}
return nil
}
@@ -103,3 +103,26 @@ func TestAModuleNamesTheAddressBehindItsMachinesName(t *testing.T) {
t.Fatalf("a machine off the network was given an address, or refused for another reason: %v", err)
}
}
// The mesh's private range is offered as ${machine:mesh-range}, so a module names it rather than
// hardcoding a value it cannot know (novox/hq ADR 0112) — the fail2ban ignoreip is the case.
func TestAModuleNamesTheMeshRange(t *testing.T) {
facts := machineFacts(Resolution{Node: "anchor", At: "anchor.internal"},
map[string]string{"anchor.internal": "10.10.0.1"}, "10.10.0.0/24")
if facts["mesh-range"] != "10.10.0.0/24" {
t.Fatalf("the mesh range is not a machine fact: %v", facts)
}
res := map[string]any{"type": "file", "id": "jail", "content": "ignoreip = 127.0.0.1/8 ${machine:mesh-range}\n"}
if err := machineInto(res, facts, "fail2ban"); err != nil {
t.Fatal(err)
}
if got := res["content"].(string); !strings.Contains(got, "10.10.0.0/24") || strings.Contains(got, "${machine:") {
t.Fatalf("the mesh range was not written in: %q", got)
}
// A mesh with no range gives no such fact, and a file that names it is refused rather than
// left with a literal placeholder in it.
none := machineFacts(Resolution{Node: "anchor"}, nil, "")
if _, has := none["mesh-range"]; has {
t.Fatal("a mesh with no range still offered one")
}
}
+162 -18
View File
@@ -176,15 +176,29 @@ type Manifest struct {
// Requires are names that must be provided by something assigned to the same node.
Requires []string `json:"requires,omitempty"`
// Emits are the event types this module publishes onto the broker — dotted topic keys, e.g.
// "module.umami.site.created". Declared so the mesh knows the event graph; events are
// provisioning's lighter sibling — 1:many and broadcast, no credential (novox/hq ADR 0041).
// Emits are the events this module publishes, named **locally**: `order.placed`, not a subject
// and not a routing key. The mesh derives where it lands (design 29 §1), so reorganising the
// subject space leaves this manifest correct. Events are provisioning's lighter sibling — 1:many
// and broadcast, no credential (novox/hq ADR 0041).
//
// A module publishes under its own name only. If the event is about a *role* rather than about
// this module, it belongs on that seat, where the name outlives whoever holds it.
//
// **This said "dotted topic keys, e.g. module.umami.site.created" until 04-ISSUES/127**, which
// is the old bus's routing key, and is why every manifest in the catalogue had the same mistake:
// nobody was guessing, everybody followed this comment.
Emits []string `json:"emits,omitempty"`
// Consumes are the event patterns this module subscribes to — topic patterns over module,
// mesh and node events alike, e.g. "node.*.joined" or "#" (the audit logger). The runtime
// wires the subscription; the module ships the handler. A Consumes for an event nothing on
// the mesh Emits is a dangling edge.
// Consumes are the events this module reacts to, each naming its emitter and the event:
// `billing.order.placed`. `*` stands for one name and `**` for the rest, so `*.download.completed`
// is that event from any module and `**` is every event in the mesh.
//
// Spelled the mesh's way rather than the wire's, for the reason Emits is: the bus the mesh runs
// on today spells these `*` and `#`, the one being built spells them `*` and `>`, and a manifest
// naming either would stop being true when the wire changed.
//
// The runtime wires the subscription; the module ships the handler. A Consumes for an event
// nothing on the mesh Emits is a dangling edge.
Consumes []string `json:"consumes,omitempty"`
// Claims are singular resources. Two modules claiming one thing within a scope cannot both
@@ -192,6 +206,34 @@ type Manifest struct {
// that every new module would force its predecessors to update.
Claims []Claim `json:"claims,omitempty"`
// DefinesSeats are the seats this module defines for itself, with their protocols
// (novox/hq ADR 0121, ADR 0129). The control plane defines the system seats — `mesh-*` and
// `node-*` — and a module may define its own, named outside that namespace, to coordinate its
// own instances: the mesh enforces one-holder-per-scope for it without knowing what it means. A
// module's declared seat is the only non-system name it may then claim; a claim to a name
// neither the mesh nor the module defines is refused.
//
// **Two lines of work built this at once**, one calling it `Seats` with a protocol and one
// `DefinesSeats` without. Same key in the file, so no manifest is affected: this is the trunk's
// name with the richer type, because what a role accepts, emits and serves is what lets the mesh
// check that a holder answers what its seat promises.
DefinesSeats []SeatDeclaration `json:"seats,omitempty"`
// Uses are seats this module sends to. It names the *seat*, never the module holding it, so
// the implementation can be replaced under it and no caller changes. A caller gets publish
// on that seat's inbound subjects and nothing else — not its outbound events, and not a
// subscription to the queue it writes to (design 29 §2).
Uses []string `json:"uses,omitempty"`
// Tools are the tools this module answers — request and reply, awaited.
//
// **New, and not `serves`**, which this manifest already uses for the facts a consumer needs
// in order to reach a provision. Two meanings under one key would be a footgun in the one
// file a module author reads most. Until now a module's tools were known only at runtime,
// from MESH_TOOL_MODULES in its image; declaring them is what lets the mesh check that a
// module claiming a seat answers what that seat's protocol promises (novox/hq ADR 0118).
Tools []string `json:"tools,omitempty"`
// Capabilities the machine must have. A different field from Requires because the remedy
// differs: a missing module can be assigned, and a missing capability means the wrong
// machine.
@@ -358,6 +400,14 @@ type Manifest struct {
// that could only see its own ports would write a rule set that closed everything else.
Filtering *Filtering `json:"filtering,omitempty"`
// Jails are the fail2ban jails this module declares for its own service (novox/hq to-be 31).
// Written into whichever node runs the module, the same way `listens` become that node's rules.
Jails []Jail `json:"jails,omitempty"`
// Jailing marks the module that composes the node's fail2ban jails — the intrusion-prevention
// holder. Like Filtering: one module per node gathers what every module declared and writes it.
Jailing *Jailing `json:"jailing,omitempty"`
// Guards are ports of this module's the mesh refuses on an adopted node except from the
// private network and from the machine itself (novox/hq ADR 0100) — the store's port and the
// broker's management port. The ports the software uses; the mesh guards where the machine
@@ -366,24 +416,29 @@ type Manifest struct {
// firewall does. Ignored on a converged node, whose derived filter already closes them.
Guards []int `json:"guards,omitempty"`
// Facts are things only the mesh knows, written where this module asks for them.
// Facts are things only the mesh knows, written where this module asks for them — in the
// module's own format.
//
// **The graph is the control plane's; how a machine uses it is the module's.** The mesh knows
// which machines exist, what they are called and where they are. Making a name resolve, or a
// peer reachable, is somebody's software — dnsmasq, a resolver, a VPN — and the mesh has no
// business shipping one, choosing which, or knowing its configuration language.
// **The graph is the control plane's; the format is the module's.** The mesh knows which
// machines exist, what they are called and where they are. Turning that into a name that
// resolves, a peer that is reachable, a host a client trusts, is somebody's software — dnsmasq,
// a resolver, a VPN, ssh — in its own configuration language, and the mesh has no business
// knowing it. So a module gives a path and a template; the mesh renders the roster through it
// and owns nothing of what the file says.
//
// So a module says *put the node names here* and owns everything after that. The same shape as
// `filtering`, generalised: a fact, and a path.
// This used to be a closed list of fact names, each formatted in Go in the control plane, so a
// new consumer meant a new formatter here in the consumer's language. Now the data is the mesh's
// and the format is the module's: the two built-in cases — the network module's `/etc/hosts` and
// dnsmasq's zones — render through the same template path any module uses, and no format lives
// in the control plane at all. See RosterFile for what a template sees.
//
// It replaces three modules that existed only because computed output needed somewhere to
// live — they ran no software, could not be swapped for anything, and appeared in the graph as
// modules while being a data channel wearing a costume.
//
// Keyed by fact name; the names are a closed list, because a module asking for one the mesh
// does not compute is asking for something nobody will write, and finding that out on a machine
// is worse than being told here.
Facts map[string]string `json:"facts,omitempty"`
// Keyed by a name the module chooses, which is the rendered file's id (`fact-<name>`) — what a
// `restart-on` names to restart when the roster changes.
Facts map[string]RosterFile `json:"facts,omitempty"`
// Certificate is where this module wants a certificate for its machine's name inside the
// mesh, and where the key that goes with it can be found.
@@ -403,6 +458,20 @@ type Manifest struct {
// A directory rather than one document for the same reason as above: each value is sealed
// separately and the mesh cannot open any of them to build a list.
Grants map[string]string `json:"grants,omitempty"`
// BusUsers is where this module wants the mesh's user list written, and it is only ever
// answered for the module holding `mesh-broker`.
//
// **The mesh writes who may connect; the module owns everything else about its server**
// (novox/hq design 25 §4, task 1.7). Ports, TLS paths and a store directory live in this
// module's image and its mounts and change when it does, so the module's own configuration
// carries them and includes this file. A controller that wrote the whole configuration would
// have to be kept in step with a Dockerfile it never sees.
//
// **Asking for it is not enough to receive it.** This file holds every user's password hash, so
// a module that could ask for it could read every credential on the bus — and the claim on
// `mesh-broker` is what authorises it, checked from this manifest alone.
BusUsers string `json:"bus-users,omitempty"`
}
// Build says how to produce this module's artifacts from its source.
@@ -602,6 +671,36 @@ func (l Listening) At() string {
return l.Protocol
}
// Jail is a fail2ban jail a module declares for its own service (novox/hq to-be 31).
//
// **The module names no node and no path** (ADR 0112): it says what a break-in on its service looks
// like — the failregex — and the jail's own keys (the port it watches, where it logs, how many
// tries, how long to ban). The mesh writes it into whichever node's fail2ban runs the module, the
// same way a module's `listens` become that node's firewall rules. A node not running the module
// has no such jail.
type Jail struct {
// Name is the jail and its filter, e.g. "postgres-auth". One holder of the name per node.
Name string `json:"name"`
// Failregex is what a failed authentication looks like in the service's log — the filter.
Failregex string `json:"failregex"`
// Jail is the body of the jail's stanza: the keys under [<name>] the module knows and the mesh
// does not — the port it watches, its logpath and backend, maxretry, bantime.
Jail string `json:"jail"`
}
// Jailing says a module composes the node's fail2ban jails — the intrusion-prevention holder. Like
// Filtering for the firewall: one module gathers what every other module declared and writes it
// where it owns. Into is the one jail file the stanzas are composed into (so the fail2ban service
// can restart on a single resource); FilterInto is the directory each jail's filter file goes in.
type Jailing struct {
Into string `json:"into"`
FilterInto string `json:"filter-into"`
}
// ComposedJailsID is the single jail file the mesh composes every declared jail into, so the
// fail2ban service names one resource in its restart-on and a jail added or removed reaches it.
func ComposedJailsID() string { return "composed-jails" }
// Filtering says where a module wants the computed rule set.
type Filtering struct {
// Into is the path to write it to. Whatever loads it is this module's own business — an
@@ -621,6 +720,21 @@ type Certificate struct {
// CertificateID and AuthorityID are the resource identities of what the mesh issued.
func CertificateID() string { return "certificate" }
// ClaimsSeat says whether this manifest claims one named seat.
func (m Manifest) ClaimsSeat(seat string) bool {
for _, c := range m.Claims {
if c.Name == seat {
return true
}
}
return false
}
// BusUsersID names the mesh's composed user list, so it is the same resource across every
// declaration and a change to it is an update rather than a second file beside the old one — which
// on a bus reading a directory would be two account lists, and the server would take both.
func BusUsersID() string { return "bus-users" }
func AuthorityID() string { return "certificate-authority" }
// FilteringID names the computed rule set, so it is the same resource across every declaration
@@ -913,6 +1027,28 @@ func ParseManifest(raw []byte) (Manifest, error) {
problems = append(problems, fmt.Sprintf(
"%q is not a usable slug: lower-case letters, digits, dashes and dots", m.Slug))
}
// **`amqp` is not a provision, and not a requirement** (novox/hq ADR 0131). A module that wants
// messaging wants the mesh's bus — it emits and consumes through the sdk, which the mesh hands the
// bus with the module's own credential — and the bus is whatever holds `mesh-broker`, spoken in
// whatever that holder speaks. Naming the old wire protocol asks for a specific server, and the
// only one that could answer is the one being retired. Refused here so the word cannot come back
// through a manifest.
for _, offer := range m.Provides {
if offer.Name == "amqp" {
problems = append(problems, fmt.Sprintf(
"%s provides %q, which is not a provision: the mesh's bus is whatever holds "+
"mesh-broker, and a module provides mesh-bus to be it (novox/hq ADR 0131)",
m.Module, offer.Name))
}
}
for _, r := range m.Requires {
if r == "amqp" {
problems = append(problems, fmt.Sprintf(
"%s requires %q, which is not a provision: a module reaches the mesh's bus through "+
"the sdk, and depends on the mesh-broker seat, not on a protocol (novox/hq ADR 0131)",
m.Module, r))
}
}
for _, offer := range m.Provides {
p := offer.Name
if !name.MatchString(p) {
@@ -950,6 +1086,10 @@ func ParseManifest(raw []byte) (Manifest, error) {
problems = append(problems, fmt.Sprintf("%s requires itself", m.Module))
}
}
// What it may call an event, and what it may ask to hear (events.go). Checked here because a
// module whose event names are wrong installs, starts, connects and reacts to nothing, with
// every log line saying it is fine (novox/hq 04-ISSUES/127).
problems = append(problems, EventProblems(m)...)
wellFormed := true
for _, c := range m.Claims {
if !name.MatchString(c.Name) {
@@ -970,6 +1110,10 @@ func ParseManifest(raw []byte) (Manifest, error) {
if wellFormed {
problems = append(problems, claimProblems(m)...)
}
// What one manifest can be judged on: a declaration's shape, its scope, and the reserved
// prefix. Whether a seat anybody names exists, and whether a holder answers for it, are
// facts about the catalogue and are checked at registration (CatalogueProblems).
problems = append(problems, declaredSeatProblems(m)...)
if m.Computed != "" && len(m.Resources) > 0 {
// One or the other. A module that both ships files and has them computed would leave
// nobody able to say where a given file came from.
+121
View File
@@ -0,0 +1,121 @@
package catalogue
import (
"encoding/json"
"os"
"path/filepath"
"regexp"
"strings"
"testing"
)
// **A manifest holds no subject** (novox/hq design 29 §1).
//
// A module names its events, tools and seats locally, and the mesh derives where they land. The
// property that buys: reorganise the subject space and every manifest in the catalogue is still
// correct. It holds today by construction — nothing reads a subject from a manifest — and a rule
// held by construction is one a later field breaks quietly, with the symptom appearing as a
// permission that does not match a subject rather than as a manifest that was wrong.
func TestNoManifestContainsASubject(t *testing.T) {
root := filepath.Join("..", "..", "..", "mesh-catalog", "modules")
entries, err := os.ReadDir(root)
if err != nil {
t.Skipf("catalogue sibling not present: %v", err)
}
// Anything in the mesh's own subject space, and anything shaped like a wire address.
subject := regexp.MustCompile(`^(mesh|\$JS)\.[a-zA-Z0-9_*>.-]+$`)
var found []string
var walk func(module string, path string, v any)
walk = func(module, path string, v any) {
switch t := v.(type) {
case string:
if subject.MatchString(t) {
found = append(found, module+" "+path+" = "+t)
}
case map[string]any:
for k, inner := range t {
walk(module, path+"."+k, inner)
}
case []any:
for _, inner := range t {
walk(module, path+"[]", inner)
}
}
}
checked := 0
for _, e := range entries {
if !e.IsDir() {
continue
}
raw, err := os.ReadFile(filepath.Join(root, e.Name(), "module.json"))
if err != nil {
continue
}
var m any
if err := json.Unmarshal(raw, &m); err != nil {
t.Errorf("%s: %v", e.Name(), err)
continue
}
checked++
walk(e.Name(), "", m)
}
if checked == 0 {
t.Skip("no manifests read")
}
if len(found) > 0 {
t.Errorf("a manifest names a subject, so reorganising the subject space would mean "+
"editing the catalogue:\n %s", strings.Join(found, "\n "))
}
t.Logf("%d manifests hold no subject", checked)
}
// **Every module's event names are what design 29 says, across the whole catalogue.**
//
// The rule above holds by construction and turned out to be weaker than it reads: a manifest holds
// no subject, and every manifest in the catalogue still held the old bus's routing key, which
// derives into a namespace nobody owns (novox/hq 04-ISSUES/127). Nothing failed — the services
// started and none of them reacted. This is the check that was missing.
func TestEveryManifestsEventNamesAreLocal(t *testing.T) {
manifests := theCatalogue(t)
var problems []string
for _, m := range manifests {
problems = append(problems, EventProblems(m)...)
}
if len(problems) > 0 {
t.Fatalf("the catalogue holds %d event name(s) the mesh would derive wrongly:\n %s",
len(problems), strings.Join(problems, "\n "))
}
}
// theCatalogue is every manifest beside this checkout, parsed the way registration parses one.
func theCatalogue(t *testing.T) []Manifest {
t.Helper()
root := filepath.Join("..", "..", "..", "mesh-catalog", "modules")
entries, err := os.ReadDir(root)
if err != nil {
t.Skipf("catalogue sibling not present: %v", err)
}
var out []Manifest
for _, e := range entries {
if !e.IsDir() {
continue
}
raw, err := os.ReadFile(filepath.Join(root, e.Name(), "module.json"))
if err != nil {
continue
}
var m Manifest
if err := json.Unmarshal(raw, &m); err != nil {
t.Fatalf("%s: %v", e.Name(), err)
}
out = append(out, m)
}
if len(out) == 0 {
t.Skip("no manifests found beside this checkout")
}
return out
}
+1 -1
View File
@@ -54,7 +54,7 @@ func TestTheShippedNetworkingModulesResolveOnTheirOwn(t *testing.T) {
// network is what gives a machine a name, so the provider asks for the node-names fact and
// there is nothing else to bring in. A module that ran nothing used to be here.
for _, m := range got.Modules {
if m.Module == overlay.Name && m.Facts["node-names"] == "" {
if m.Module == overlay.Name && m.Facts["node-names"].Path == "" {
t.Fatalf("the network's provider does not ask for the names: %+v", m.Facts)
}
}
+38 -2
View File
@@ -28,6 +28,10 @@ type Node struct {
// (novox/hq ADR 0066). A route contribution carries only a label — the subdomain — and the mesh
// joins <label>.<public-domain> to make the name it grants, interpreting neither half.
PublicDomain string
// Account is the operator's login on this machine, AccountHome where its home is (novox/hq
// to-be 29). What a home-scoped file is owned by and what ${machine:account} resolves to.
Account string
AccountHome string
}
// World is what the rest of the mesh already has.
@@ -37,6 +41,11 @@ type Node struct {
type World struct {
// Held is the claims already taken, for the scopes wider than one node.
Held []Held
// Holdings is every seat whose holder is **on record** (novox/hq ADR 0131): the one assignment
// that holds it, chosen by a handover. A seat absent here is held by derivation — the sole
// eligible assignment — as it always was. Present, it decides, and any other assignment whose
// module could hold the seat is eligible and silent rather than refused.
Holdings []Held
// Offered is what other nodes provide at mesh scope, and everything needed to use it.
Offered map[string][]Provider
// Pinned is which node this machine was told to get a provision from, by name. Only consulted
@@ -114,6 +123,11 @@ type Resolution struct {
// (novox/hq ADR 0066). Carried from the node so that composing <label>.<public-domain> for a
// route contribution needs no store lookup here — the join is a fact about this one machine.
PublicDomain string
// Account and AccountHome are the operator's login on this machine and where its home is
// (novox/hq to-be 29), carried from the node so a home-scoped file's owner and path resolve
// here without a store lookup.
Account string
AccountHome string
// Modules in the order they were resolved: assigned first, then what they pulled in.
Modules []Manifest
@@ -541,13 +555,14 @@ func Resolve(catalogue map[string]Manifest, assigned []string, node Node, world
}
resolution := Resolution{Node: node.Name, At: node.At, PublicDomain: node.PublicDomain,
Account: node.Account, AccountHome: node.AccountHome,
Because: because, Needs: needs, Unhostable: unhostable}
for _, n := range providersFirst(order, catalogue) {
resolution.Modules = append(resolution.Modules, catalogue[n])
}
problems = append(problems, checkCapabilities(resolution.Modules, node)...)
claims, claimProblems := checkClaims(resolution.Modules, node, elsewhere)
claims, claimProblems := checkClaims(resolution.Modules, node, elsewhere, world.Holdings)
problems = append(problems, claimProblems...)
problems = append(problems, checkResources(resolution.Modules)...)
resolution.Claims = claims
@@ -640,14 +655,35 @@ func checkCapabilities(modules []Manifest, node Node) []string {
//
// Within this node's own set, and against what is already held elsewhere for the wider scopes. A
// claim at mesh scope is the same idea as the mesh's one hub, said once instead of hard-coded.
func checkClaims(modules []Manifest, node Node, elsewhere []Held) ([]Held, []string) {
func checkClaims(modules []Manifest, node Node, elsewhere []Held, holdings []Held) ([]Held, []string) {
var problems []string
var held []Held
// onRecord is the recorded holder of a seat, if a handover ever named one.
onRecord := func(claim, scope string) (Held, bool) {
for _, h := range holdings {
hs, ok := SeatNamed(h.Claim)
cs, cok := SeatNamed(claim)
if ok && cok && hs.Name == cs.Name && h.Scope == scope {
return h, true
}
}
return Held{}, false
}
byScope := map[string]map[string]string{} // scope → claim → module
for _, m := range modules {
for _, c := range m.Claims {
scope := c.At()
// **A recorded holder settles it before any counting.** An assignment that could hold
// the seat but is not the one on record is eligible, and that is all: it is not a second
// holder, so it is not refused, and it does not hold (novox/hq ADR 0131). This is what
// lets the next holder stand beside the current one until the seat is handed over.
if rec, recorded := onRecord(c.Name, scope); recorded {
if rec.Node != node.Name || rec.Module != m.Module {
continue
}
}
if byScope[scope] == nil {
byScope[scope] = map[string]string{}
}
@@ -50,7 +50,7 @@ func TestTheResolverForwardsToFixedUpstreamsAndNeverReadsResolvConf(t *testing.T
"\nno-resolv\n", "\nserver=1.1.1.1\n", "\nserver=8.8.8.8\n",
"\nlisten-address=127.0.0.1\n", "\ninterface=mesh0\n", "\nbind-dynamic\n",
"\ndomain-needed\n", "\nbogus-priv\n",
"\nconf-file=" + m.Facts[FactNodeZones] + "\n",
"\nconf-file=" + m.Facts["node-zones"].Path + "\n",
} {
if !strings.Contains(config, want) {
t.Errorf("the resolver's configuration lacks %q:\n%s", strings.TrimSpace(want), config)
@@ -170,7 +170,7 @@ func TestTwoThingsDecidingWhatAMachineAsksAreRefused(t *testing.T) {
if err == nil {
t.Fatal("resolv-conf and resolved-split-dns were both assigned to one machine")
}
if !strings.Contains(err.Error(), "the-resolver-configuration") {
if !strings.Contains(err.Error(), "node-resolver-config") {
t.Fatalf("the refusal does not say what was claimed: %v", err)
}
}
+210
View File
@@ -0,0 +1,210 @@
package catalogue
import (
"bytes"
"fmt"
"sort"
"strings"
"text/template"
)
// What only the mesh knows, written where a module asks for it — in the module's own format.
//
// **The graph is the control plane's; the format is the module's.** The mesh knows which machines
// exist, what they are called and where they are. Turning that into a hosts file, a resolver's
// zones, an ssh known_hosts is somebody's configuration language, and the mesh has no business
// knowing it. So the mesh hands the roster to a template the module wrote and renders it; it never
// learns what the file means.
//
// This used to be a closed list of fact names, each with its format written in Go here — a hosts
// file, a resolver's zones. Every new consumer meant a new formatter in the control plane, in the
// consumer's configuration language. Now the data is the mesh's and the format is a template the
// module ships: the two built-in cases (the network module's `/etc/hosts`, dnsmasq's zones) render
// the same way any module's would, and the control plane holds no format at all.
//
// It replaced three modules that existed only because computed output needed somewhere to live —
// they ran no software, could not be swapped for anything, and appeared in the graph as modules
// while being a data channel wearing a costume (novox/hq ADR 0040).
// A RosterFile is a file the mesh renders from the roster of machines, in the format the module
// gives as a Go text/template. The template sees a rosterView: `.Node` (this machine's bare name),
// `.Suffix` (what its mesh name ends in), and two sets of `{Name, FQDN, Address}` — `.Names`, every
// name the mesh serves, and `.Machines`, only the nodes of the mesh. Which set a template ranges is
// how the hq issue 111 distinction is drawn: a container's hosts wants every name; a resolver told
// the suffix is its own wants only the machines.
type RosterFile struct {
// Path is where on the machine the rendered file goes. Absolute, or it is refused here rather
// than discovered as a daemon that reads nothing.
Path string `json:"path"`
// Template is the module's format, a Go text/template over the rosterView. It is the module's,
// not the mesh's: the mesh renders it and does not read it.
Template string `json:"template"`
// Shared is whether the file the fact goes to belongs to the machine rather than the mesh. When
// it does, the mesh owns only a marked region of it and keeps the rest byte for byte (novox/hq
// issue 128) — a hosts file is shared, since the distribution's `localhost`, the operator's own
// lines and other tools' blocks live there too; a resolver's zones file is not, the mesh owns it
// whole. A property of the fact, not of the path: the format determines whether the file is
// wholly the mesh's, not where a module happened to ask for it.
Shared bool `json:"shared,omitempty"`
// Home places the file under this node's operator-account home and chowns it to that account,
// rather than at an absolute system path (novox/hq to-be 29). Then Path is home-relative
// (`.ssh/config.d/mesh`), resolved against the account's home on the node it is composed for; a
// node with no operator account gets no such file. This is how the ssh-client config — every
// other node's Host block — is written into a person's home rather than into /etc.
Home bool `json:"home,omitempty"`
}
// rosterView is what a RosterFile's template sees. A closed shape — a template referencing a field
// the mesh does not compute fails to render here, not on a machine.
type rosterView struct {
Node string
Suffix string
Names []rosterEntry
Machines []rosterEntry
}
// rosterEntry is one machine as a template sees it: its bare name, its full mesh name, its address,
// and the operator account to log into it as (novox/hq to-be 29) — empty when none is known, so an
// ssh Host block template can omit the User line for a machine nobody has an account on.
type rosterEntry struct {
Name string
FQDN string
Address string
Account string
}
// FactsInto renders the roster files a module asked for, as files it will be given.
//
// The module owns everything after the file exists: loading it, restarting on it, what a resolver
// or a client does with it. This only puts it there. `every` is every name the mesh serves;
// `machines` is only the machines — the two must not be confused (novox/hq 04-ISSUES/111), so both
// are given and the template chooses.
func FactsInto(m Manifest, r Resolution, every, machines, accounts map[string]string, suffix string) ([]map[string]any, error) {
if len(m.Facts) == 0 {
return nil, nil
}
names := make([]string, 0, len(m.Facts))
for name := range m.Facts {
names = append(names, name)
}
sort.Strings(names)
view := rosterView{
Node: r.Node,
Suffix: strings.TrimPrefix(suffixOr(suffix), "."),
Names: entriesFrom(every, accounts, suffix),
Machines: entriesFrom(machines, accounts, suffix),
}
out := make([]map[string]any, 0, len(names))
for _, name := range names {
fact := m.Facts[name]
content, err := renderRoster(fact.Template, view)
if err != nil {
return nil, fmt.Errorf("%s cannot render %q: %w", m.Module, name, err)
}
// Where the file goes: under the operator's home and chowned to it (a home fact), or at the
// absolute system path it names. A home fact on a machine with no operator account cannot be
// placed, and is left out rather than written to nowhere (novox/hq to-be 29).
path := fact.Path
var owner string
if fact.Home {
if r.Account == "" {
continue
}
path = accountHomeOf(r.Account, r.AccountHome) + "/" + strings.TrimLeft(fact.Path, "/")
owner = r.Account
} else if !strings.HasPrefix(fact.Path, "/") {
return nil, fmt.Errorf(
"%s asks for %q at %q, which is not an absolute path", m.Module, name, fact.Path)
}
file := map[string]any{
"id": "fact-" + name, "type": "file", "path": path, "mode": "0644",
"content": content,
}
if owner != "" {
file["owner"] = owner
}
if fact.Shared {
// The host owns only the lines between `# BEGIN mesh <id>` and `# END mesh <id>` and
// keeps the rest of the file byte for byte; undeclared, the region goes and nothing else
// does (novox/hq issue 128). Every node on the private network receives this, so every
// node's host — the controller's own machine included — must be block-aware before a
// controller emitting it is rolled out: the order ADR 0102 set for `into: json`.
file["into"] = "block"
}
out = append(out, file)
}
return out, nil
}
// renderRoster runs a module's template over the roster. A template that will not parse, or reads
// a field the mesh does not have, is an error here — where the manifest is — rather than an empty
// file on a machine.
func renderRoster(tmpl string, view rosterView) (string, error) {
t, err := template.New("roster").Option("missingkey=error").Parse(tmpl)
if err != nil {
return "", err
}
var b bytes.Buffer
if err := t.Execute(&b, view); err != nil {
return "", err
}
return b.String(), nil
}
// entriesFrom is a name→address map as sorted roster entries.
//
// **A machine with no address is left out.** The mesh has a record for it — somebody added it —
// and does not yet know where it is, which is the ordinary state between adding a machine and it
// joining. Writing the name anyway would give a name that resolves to nothing, and a connection to
// that hangs; leaving it out fails at once and says the name is unknown.
func entriesFrom(addresses, accounts map[string]string, suffix string) []rosterEntry {
out := make([]rosterEntry, 0, len(addresses))
for _, name := range sortedNames(addresses) {
internal, bare := meshName(name, suffix)
// The account is looked up by whichever key the caller keys accounts on — the internal name
// or the bare one — so a template gets the right login however the maps were built.
account := accounts[name]
if account == "" {
account = accounts[bare]
}
out = append(out, rosterEntry{Name: bare, FQDN: internal, Address: addresses[name], Account: account})
}
return out
}
// meshName is a machine's internal name and its bare one, from either. The control plane keys
// the names it hands a resolution by the internal name (`homer.internal`), the same map a
// container gets as its hosts; a caller that keys by the bare name gets the same answer. The
// suffix is the one the control plane composed those names with, handed down rather than written
// here a second time — the alternative was `homer.internal.internal` on every machine.
func meshName(name, suffix string) (internal, bare string) {
dotted := "." + strings.TrimPrefix(suffixOr(suffix), ".")
if strings.HasSuffix(name, dotted) {
return name, strings.TrimSuffix(name, dotted)
}
return name + dotted, name
}
// suffixOr is the suffix given, or the one the mesh composes names with when none was handed down.
// The one place the default is written, so a fact and a name cannot disagree about it.
func suffixOr(suffix string) string {
if suffix == "" {
return "internal"
}
return suffix
}
func sortedNames(addresses map[string]string) []string {
out := make([]string, 0, len(addresses))
for name, at := range addresses {
// A machine the mesh cannot place is left out rather than named at nothing.
if at == "" {
continue
}
out = append(out, name)
}
sort.Strings(out)
return out
}
+260
View File
@@ -0,0 +1,260 @@
package catalogue
import (
"strings"
"testing"
)
// Keyed by the internal name, as the control plane hands them (issue 079). bart has no address —
// the ordinary state between adding a machine and it joining.
var threeMachines = map[string]string{"homer.internal": "10.42.0.1", "marge.internal": "10.42.0.2", "bart.internal": ""}
// A module says where it wants a roster file and in what format, and is given the rendered file.
func TestAModuleIsGivenTheFileItAskedFor(t *testing.T) {
m := Manifest{Module: "resolver", Facts: map[string]RosterFile{
"zones": {Path: "/etc/mesh/zones.conf", Template: "{{range .Machines}}address=/{{.FQDN}}/{{.Address}}\n{{end}}"},
}}
given, err := FactsInto(m, Resolution{Node: "homer"}, threeMachines, threeMachines, nil, "")
if err != nil {
t.Fatal(err)
}
if len(given) != 1 {
t.Fatalf("expected one file, got %d", len(given))
}
if given[0]["path"] != "/etc/mesh/zones.conf" || given[0]["type"] != "file" {
t.Fatalf("not written where it was asked for: %v", given[0])
}
// The id is fact-<name>, which is what a restart-on names when the roster changes.
if given[0]["id"] != "fact-zones" {
t.Fatalf("the file's id is not fact-<name>, so a restart-on cannot find it: %v", given[0]["id"])
}
if !strings.Contains(given[0]["content"].(string), "address=/homer.internal/10.42.0.1") {
t.Fatalf("the file does not hold the fact: %v", given[0]["content"])
}
}
// **A shared fact is written into a region of the machine's file, not over it** (novox/hq issue
// 128). A hosts file is the machine's — its localhost, the operator's lines, other tools' blocks —
// so the mesh owns only a marked region (`into: block`); a resolver's zones file is the mesh's
// whole, and carries no `into`.
func TestASharedFactIsWrittenIntoARegion(t *testing.T) {
roster := map[string]string{"homer.internal": "10.42.0.1"}
m := Manifest{Module: "net", Facts: map[string]RosterFile{
"node-names": {Path: "/etc/hosts", Template: "{{range .Names}}{{.FQDN}}\n{{end}}", Shared: true},
"node-zones": {Path: "/etc/zones", Template: "{{range .Machines}}{{.FQDN}}\n{{end}}"},
}}
given, err := FactsInto(m, Resolution{Node: "homer"}, roster, roster, nil, "")
if err != nil {
t.Fatal(err)
}
by := map[string]map[string]any{}
for _, f := range given {
by[f["path"].(string)] = f
}
if by["/etc/hosts"]["into"] != "block" {
t.Fatalf("a shared fact is not written into a region, so the mesh writes the file whole: %v", by["/etc/hosts"])
}
if _, has := by["/etc/zones"]["into"]; has {
t.Fatalf("an unshared fact was written into a region, so the mesh does not own its own file whole: %v", by["/etc/zones"])
}
}
// **The format is the module's — the mesh renders whatever template it gives.** The same roster
// through two templates is two entirely different files, and the control plane reads neither.
func TestTheFormatIsTheModulesOwn(t *testing.T) {
roster := map[string]string{"homer.internal": "10.42.0.1"}
hostsish := Manifest{Module: "a", Facts: map[string]RosterFile{
"f": {Path: "/f", Template: "{{range .Names}}{{.Address}}\t{{.Name}}\n{{end}}"}}}
sshish := Manifest{Module: "b", Facts: map[string]RosterFile{
"f": {Path: "/f", Template: "{{range .Names}}Host {{.Name}}\n HostName {{.FQDN}}\n{{end}}"}}}
h, err := FactsInto(hostsish, Resolution{Node: "homer"}, roster, roster, nil, "")
if err != nil {
t.Fatal(err)
}
s, err := FactsInto(sshish, Resolution{Node: "homer"}, roster, roster, nil, "")
if err != nil {
t.Fatal(err)
}
if h[0]["content"] != "10.42.0.1\thomer\n" {
t.Fatalf("the hosts-shaped template did not render its format: %q", h[0]["content"])
}
if s[0]["content"] != "Host homer\n HostName homer.internal\n" {
t.Fatalf("the ssh-shaped template did not render its format: %q", s[0]["content"])
}
}
// **A template that will not parse is refused here, not on a machine.** A daemon that starts, reads
// a file the mesh could not render, and answers nothing is a much worse way to find out.
func TestABrokenTemplateIsRefusedHere(t *testing.T) {
m := Manifest{Module: "resolver", Facts: map[string]RosterFile{
"zones": {Path: "/etc/zones", Template: "{{range .Machines}}oops"}}}
_, err := FactsInto(m, Resolution{}, nil, nil, nil, "")
if err == nil {
t.Fatal("a template that does not parse was accepted, so the machine gets an empty file")
}
if !strings.Contains(err.Error(), "resolver") || !strings.Contains(err.Error(), "zones") {
t.Fatalf("the refusal does not say whose template, or which: %v", err)
}
}
// A template reading something the mesh does not compute is refused, not rendered empty. The roster
// is a closed shape; asking it for the weather fails where the manifest is.
func TestATemplateReadingWhatTheMeshDoesNotHaveIsRefused(t *testing.T) {
m := Manifest{Module: "resolver", Facts: map[string]RosterFile{
"zones": {Path: "/etc/zones", Template: "{{.Weather}}"}}}
if _, err := FactsInto(m, Resolution{}, nil, nil, nil, ""); err == nil {
t.Fatal("a template read a field nobody computes and rendered anyway, silently")
}
}
// A relative path is refused, or a module decides where the mesh writes on a machine.
func TestAFactMustBeAskedForAtAnAbsolutePath(t *testing.T) {
m := Manifest{Module: "resolver", Facts: map[string]RosterFile{
"hosts": {Path: "etc/hosts", Template: "x"}}}
if _, err := FactsInto(m, Resolution{}, nil, nil, nil, ""); err == nil {
t.Fatal("a relative path was accepted")
}
}
// A machine the mesh has a record for and cannot place is left out of the roster.
//
// **Not an oversight — the alternative is worse.** A name written with no address resolves to
// nothing, and a connection to that hangs. Leaving it out fails at once and says the name is
// unknown, which is a thing somebody can act on.
func TestAMachineWithNoAddressIsNotInTheRoster(t *testing.T) {
m := Manifest{Module: "a", Facts: map[string]RosterFile{
"f": {Path: "/f", Template: "{{range .Machines}}{{.Name}}\n{{end}}"}}}
given, err := FactsInto(m, Resolution{Node: "homer"}, threeMachines, threeMachines, nil, "")
if err != nil {
t.Fatal(err)
}
if strings.Contains(given[0]["content"].(string), "bart") {
t.Fatalf("a machine with no address was in the roster, so its name resolves to nothing:\n%s", given[0]["content"])
}
}
// **The names the control plane hands a resolution are already internal names** — `homer.internal`,
// the same map every container gets as its hosts. A roster entry's FQDN is that name, not it with
// the suffix appended a second time; either key gives the same entries.
func TestNamesAreNotSuffixedTwice(t *testing.T) {
internal := map[string]string{"homer.internal": "10.42.0.1"}
bare := map[string]string{"homer": "10.42.0.1"}
tmpl := RosterFile{Path: "/f", Template: "{{range .Machines}}{{.FQDN}} {{.Name}}\n{{end}}"}
fromInternal, err := FactsInto(Manifest{Module: "a", Facts: map[string]RosterFile{"f": tmpl}}, Resolution{Node: "homer"}, internal, internal, nil, "")
if err != nil {
t.Fatal(err)
}
fromBare, err := FactsInto(Manifest{Module: "a", Facts: map[string]RosterFile{"f": tmpl}}, Resolution{Node: "homer"}, bare, bare, nil, "")
if err != nil {
t.Fatal(err)
}
if fromInternal[0]["content"] != fromBare[0]["content"] {
t.Fatalf("the roster differs by how the names were keyed:\n%q\n%q", fromInternal[0]["content"], fromBare[0]["content"])
}
got := fromInternal[0]["content"].(string)
if strings.Contains(got, "internal.internal") || !strings.Contains(got, "homer.internal homer") {
t.Fatalf("the entry carries a doubled suffix or the wrong bare name:\n%s", got)
}
}
// The suffix the control plane composed the names with is the one a template sees — an operator who
// chose another does not get `.internal`. `.Suffix` is the bare form, and FQDNs carry it.
func TestTheSuffixIsCarriedAsComposed(t *testing.T) {
names := map[string]string{"homer.lan": "10.42.0.1"}
m := Manifest{Module: "a", Facts: map[string]RosterFile{
"f": {Path: "/f", Template: "local=/{{.Suffix}}/\n{{range .Machines}}{{.FQDN}}\n{{end}}"}}}
given, err := FactsInto(m, Resolution{Node: "homer"}, names, names, nil, "lan")
if err != nil {
t.Fatal(err)
}
got := given[0]["content"].(string)
if !strings.Contains(got, "local=/lan/") || strings.Contains(got, "internal") {
t.Fatalf("the operator's suffix was not carried, so its names would be wrong:\n%s", got)
}
if !strings.Contains(got, "homer.lan") {
t.Fatalf("the FQDN does not carry the operator's suffix:\n%s", got)
}
}
// novox/hq 04-ISSUES/111: a template is given both sets and chooses. `.Names` is every name the mesh
// serves — the machines and the names it was told to route; `.Machines` is only the machines. A
// container's hosts wants every name so a routed name resolves to the machine serving it; a resolver
// told the suffix is its own wants only the machines, or a routed name written there with the suffix
// is a name nobody will ever ask for, standing beside the machines and looking as real.
func TestATemplateChoosesMachinesOrEveryName(t *testing.T) {
machines := map[string]string{"homer.internal": "10.42.0.1", "marge.internal": "10.42.0.2"}
every := map[string]string{
"homer.internal": "10.42.0.1", "marge.internal": "10.42.0.2",
"drive.example.test": "10.42.0.1", "git.example.test": "10.42.0.2",
}
m := Manifest{Module: "resolver", Facts: map[string]RosterFile{
"zones": {Path: "/etc/zones", Template: "{{range .Machines}}{{.FQDN}}\n{{end}}"},
"hosts": {Path: "/etc/hosts", Template: "{{range .Names}}{{.FQDN}}\n{{end}}"},
}}
given, err := FactsInto(m, Resolution{Node: "homer"}, every, machines, nil, "")
if err != nil {
t.Fatal(err)
}
by := map[string]string{}
for _, f := range given {
by[f["path"].(string)] = f["content"].(string)
}
zones := by["/etc/zones"]
for _, served := range []string{"drive.example.test", "git.example.test"} {
if strings.Contains(zones, served) {
t.Fatalf("a template over .Machines saw %q, a name the mesh serves rather than a machine:\n%s", served, zones)
}
}
if !strings.Contains(zones, "homer.internal") {
t.Fatalf("a template over .Machines did not see the machines:\n%s", zones)
}
hosts := by["/etc/hosts"]
for _, name := range []string{"homer.internal", "drive.example.test", "git.example.test"} {
if !strings.Contains(hosts, name) {
t.Fatalf("a template over .Names did not see %q, so a container would not resolve it:\n%s", name, hosts)
}
}
}
// A home fact is placed under the operator account's home and chowned to it, and its template sees
// each node's account (novox/hq to-be 29) — the ssh-client config is the case.
func TestAHomeFactIsPlacedUnderTheAccountsHomeAndOwnedByIt(t *testing.T) {
names := map[string]string{"homer.internal": "10.42.0.1", "marge.internal": "10.42.0.2"}
accounts := map[string]string{"homer": "jo", "marge": "jo"}
m := Manifest{Module: "ssh-client", Facts: map[string]RosterFile{
"ssh-config": {Path: ".ssh/config.d/mesh", Home: true,
Template: "{{range .Names}}Host {{.Name}}\n HostName {{.FQDN}}\n User {{.Account}}\n{{end}}"}}}
given, err := FactsInto(m, Resolution{Node: "homer", Account: "jo"}, names, names, accounts, "")
if err != nil {
t.Fatal(err)
}
f := given[0]
if f["path"] != "/home/jo/.ssh/config.d/mesh" {
t.Fatalf("the home fact was not placed under the account's home: %v", f["path"])
}
if f["owner"] != "jo" {
t.Fatalf("the home fact is not owned by the account: %v", f["owner"])
}
if !strings.Contains(f["content"].(string), "Host marge\n HostName marge.internal\n User jo") {
t.Fatalf("the config does not name the peer's account:\n%s", f["content"])
}
}
// A machine with no operator account gets no home fact — it cannot be placed, so it is left out
// rather than written to nowhere.
func TestAHomeFactIsSkippedWhereThereIsNoAccount(t *testing.T) {
names := map[string]string{"homer.internal": "10.42.0.1"}
m := Manifest{Module: "ssh-client", Facts: map[string]RosterFile{
"ssh-config": {Path: ".ssh/config", Home: true, Template: "x"}}}
given, err := FactsInto(m, Resolution{Node: "homer"}, names, names, nil, "")
if err != nil {
t.Fatal(err)
}
if len(given) != 0 {
t.Fatalf("a home fact was placed on a machine with no operator account: %v", given)
}
}
+1 -5
View File
@@ -13,7 +13,6 @@ func TestASeatPlaceholderAnswersWhereThisMachinePutTheHolder(t *testing.T) {
"type": "container", "id": "server", "name": "mesh-controller",
"env": map[string]any{
"MESH_STORE_INVENTORY_PORT": "${seat:mesh-store:5432}",
"MESH_BROKER_AMQP_PORT": "${seat:mesh-broker:5672}",
"MESH_BROKER_ADDRESS_PORT": "${seat:mesh-broker:5671}",
"MESH_STORE_INVENTORY_FILE": "/run/secrets/inventory",
},
@@ -27,7 +26,6 @@ func TestASeatPlaceholderAnswersWhereThisMachinePutTheHolder(t *testing.T) {
env := control["env"].(map[string]any)
for key, want := range map[string]string{
"MESH_STORE_INVENTORY_PORT": "6852",
"MESH_BROKER_AMQP_PORT": "5679",
"MESH_BROKER_ADDRESS_PORT": "5671",
"MESH_STORE_INVENTORY_FILE": "/run/secrets/inventory",
} {
@@ -146,7 +144,6 @@ func TestTheControlPlanesOwnAddressesFollowTheNodesPorts(t *testing.T) {
"MESH_STORE_INVENTORY_PORT": "6852",
"MESH_STORE_IDENTITY_PORT": "6852",
"MESH_STORE_LICENCES_PORT": "6852",
"MESH_BROKER_AMQP_PORT": "5679",
"MESH_BROKER_MANAGEMENT_PORT": "15673",
"MESH_BROKER_ADDRESS_PORT": "5671",
} {
@@ -164,7 +161,7 @@ func TestTheControlPlanesOwnAddressesFollowTheNodesPorts(t *testing.T) {
t.Fatal(err)
}
env, _ = fileNamed(out, "mesh-controller.server")["env"].(map[string]any)
if env["MESH_STORE_INVENTORY_PORT"] != "" || env["MESH_BROKER_AMQP_PORT"] != "" {
if env["MESH_STORE_INVENTORY_PORT"] != "" {
t.Errorf("with no settings, the control plane is told %v", env)
}
}
@@ -175,7 +172,6 @@ var SeatPorts = map[string]string{
"MESH_STORE_INVENTORY_PORT": "${seat:mesh-store:5432}",
"MESH_STORE_IDENTITY_PORT": "${seat:mesh-store:5432}",
"MESH_STORE_LICENCES_PORT": "${seat:mesh-store:5432}",
"MESH_BROKER_AMQP_PORT": "${seat:mesh-broker:5672}",
"MESH_BROKER_MANAGEMENT_PORT": "${seat:mesh-broker:15672}",
"MESH_BROKER_ADDRESS_PORT": "${seat:mesh-broker:5671}",
}
+208 -28
View File
@@ -27,40 +27,144 @@ type Seat struct {
// provision may only be held by a module providing it at the seat's scope, and its holder is
// what a requirement for that provision resolves to when several modules provide it.
Delivers string
// Accepts, Emits and Serves are the protocol of the role, as local verbs — the same three a
// module declares for a seat of its own (novox/hq ADR 0118), and empty for most of these: a seat
// is usually about who does a job and not about what may be said to them.
//
// **Named here so the mesh has no role it cannot describe** (ADR 0121). Without them a build
// machine had three audiences for one outcome and nothing derived a grant for any of them, and an
// event about a role had nowhere to live but the namespace of whichever module held that role
// today — which the bus refuses, because a namespace belongs to who it is named for.
Accepts []string
Emits []string
Serves []string
// Decision is the record that made it a seat.
Decision string
}
// seats is the whole set, in the order a person reads it: the mesh's own, then a node's.
var seats = []Seat{
// defaultSeats is the set the mesh ships with — the seed for the control plane's seat table and the
// fallback when it has none (novox/hq ADR 0122). It is the one place the closed set 0110 defines is
// written; the store's table is seeded from it and thereafter is the live, editable copy.
//
// In the order a person reads it: the mesh's own, then a node's.
var defaultSeats = []Seat{
{Name: "mesh-controller", Scope: ScopeMesh, Decision: "novox/hq ADR 0079"},
{Name: "mesh-store", Scope: ScopeMesh, Delivers: "postgres-database", Decision: "novox/hq ADR 0079"},
{Name: "mesh-broker", Scope: ScopeMesh, Delivers: "amqp", Decision: "novox/hq ADR 0079"},
// **Delivers the mesh's own bus, not `amqp`.** Those were the same word until
// ADR 0127 separated them: `amqp` is a backing service a module may require, and this seat is
// the mesh's own transport. ADR 0128 then made that connection something a module requires
// rather than receives ambiently — 23 of the catalogue's modules never speak, and an ambient
// connection would mint a credential for each.
{Name: "mesh-broker", Scope: ScopeMesh, Delivers: "mesh-bus", Decision: "novox/hq ADR 0079"},
{Name: "the-artifact-store", Scope: ScopeMesh, Delivers: "artifact-store", Decision: "novox/hq ADR 0075"},
{Name: "the-catalogue", Scope: ScopeMesh, Decision: "novox/hq ADR 0110"},
{Name: "mesh-catalog", Scope: ScopeMesh, Decision: "novox/hq ADR 0121"},
// Deferred renames (novox/hq ADR 0121): these deliver a provision, so renaming them is a
// delivering-seat migration with a mesh-wide cascade if a holder stops resolving mid-flight.
// They keep their names until that migration is done deliberately, apart from the node-* pass.
{Name: "npm-package-registry", Scope: ScopeMesh, Delivers: "npm-package-registry", Decision: "novox/hq ADR 0109"},
{Name: "git", Scope: ScopeMesh, Delivers: "git", Decision: "novox/hq ADR 0111"},
{Name: "the-build-machine", Scope: ScopeNode, Decision: "novox/hq ADR 0110"},
{Name: "the-dns-port", Scope: ScopeNode, Decision: "novox/hq ADR 0110"},
{Name: "the-intrusion-prevention", Scope: ScopeNode, Decision: "novox/hq ADR 0110"},
{Name: "the-packet-filter", Scope: ScopeNode, Decision: "novox/hq ADR 0110"},
// A build is work submitted to this role and its outcome is the role's own event (ADR 0129).
// One publish reaches whoever asked, the controller that records it, and the catalogue that
// places it in the graph — what the old bus's shared exchange did for free.
{Name: "mesh-build-machine", Scope: ScopeMesh,
Accepts: []string{"build"}, Emits: []string{"built"}, Decision: "novox/hq ADR 0121"},
{Name: "node-dns-resolver", Scope: ScopeNode, Decision: "novox/hq ADR 0121"},
{Name: "node-intrusion-prevention", Scope: ScopeNode, Decision: "novox/hq ADR 0121"},
{Name: "node-packet-filter", Scope: ScopeNode, Decision: "novox/hq ADR 0121"},
// Deferred (novox/hq ADR 0121): renaming to mesh-private-network is a scope + server/client
// model change, not a rename, so it stays until that is built.
{Name: "the-private-network", Scope: ScopeNode, Decision: "novox/hq ADR 0110"},
{Name: "the-resolver-configuration", Scope: ScopeNode, Decision: "novox/hq ADR 0110"},
{Name: "the-showcase", Scope: ScopeNode, Decision: "novox/hq ADR 0110"},
{Name: "node-resolver-config", Scope: ScopeNode, Decision: "novox/hq ADR 0121"},
// The program that manages the machine's own network. It delivers nothing: its holder only
// keeps the manager and the mesh from contradicting each other — the resolver file left to the
// mesh, the private network's interface left alone — and never declares a link, an address or
// a wireless network, because the link is the only channel a fix could arrive on. A seat
// rather than a condition in the resolver's module, so a machine running two managers is
// refused at assignment instead of found by the resolver being rewritten (novox/hq ADR 0117).
{Name: "node-uplink", Scope: ScopeNode, Decision: "novox/hq ADR 0117"},
}
// A system seat name is the control plane's namespace: `mesh-*` for a mesh-wide role, `node-*` for
// a per-node one (novox/hq ADR 0121). A claim to a system name the mesh does not define is refused;
// any other name is a module's own to define and claim. Some of the mesh's own seats predate this
// convention and are not yet renamed (git, npm-package-registry, the-artifact-store,
// the-private-network) — those are in the set, so they resolve by name, not by prefix.
func isSystemSeatName(name string) bool {
return strings.HasPrefix(name, "mesh-") || strings.HasPrefix(name, "node-")
}
// seats is the working set the lookups read. It starts as the compiled defaults and is replaced by
// what the control plane loaded from its store (novox/hq ADR 0122), so a change to the set is a
// change to data, not to this code.
var seats = defaultSeats
// DefaultSeats is the set the mesh ships with, for seeding the store's seat table.
func DefaultSeats() []Seat { return append([]Seat(nil), defaultSeats...) }
// UseSeats replaces the working set with the one the control plane read from its store.
//
// **Empty is ignored on purpose.** A store that has not been seeded yet — or one that could not be
// read — must leave the compiled defaults in force rather than emptying the set: an empty set would
// refuse every claim and could stop the control plane composing at all, which is a far worse failure
// than running on the set the binary shipped with. So the store can only ever *replace* the set with
// a non-empty one, never erase it.
func UseSeats(s []Seat) {
if len(s) == 0 {
return
}
// **The store's rows carry no protocol yet, and the protocol is what the bus is derived
// from.** ADR 0129 gives a seat what it accepts, emits and serves; ADR 0122 moved the set into
// a table that has name, scope, delivers and decision and nothing else, and the columns for
// the rest are not there yet. So a row replacing a compiled entry would silently drop the
// protocol, and the roles' work queues would never be raised — found live as "no response
// from stream" the first time a build was submitted over the new bus (2026-09-28). Until the
// table gains the columns, a row without a protocol keeps the compiled one of the same name.
byName := map[string]Seat{}
for _, d := range defaultSeats {
byName[d.Name] = d
}
merged := make([]Seat, 0, len(s))
for _, row := range s {
if len(row.Accepts)+len(row.Emits)+len(row.Serves) == 0 {
if d, known := byName[row.Name]; known {
row.Accepts, row.Emits, row.Serves = d.Accepts, d.Emits, d.Serves
}
}
merged = append(merged, row)
}
seats = merged
}
// aliases maps a seat's former names to its current canonical name (novox/hq ADR 0122). Loaded from
// the store alongside the set, so a reference to a name a seat used to have — a manifest's claim, a
// held record — still resolves to it after a rename, and nothing downstream has to change.
var aliases = map[string]string{}
// UseAliases replaces the former-name map with the one the control plane read from its store. Empty
// is fine and ordinary: a mesh whose seats have never been renamed has no aliases.
func UseAliases(m map[string]string) { aliases = m }
// Seats is every seat the mesh defines, in reading order.
func Seats() []Seat {
return append([]Seat(nil), seats...)
}
// SeatNamed is the seat a claim names, if the mesh defines one.
// SeatNamed is the seat a name refers to, whether that is its current name or one it used to have
// (novox/hq ADR 0122). A former name resolves to the seat's canonical row, so a rename breaks no
// reference to the old name.
func SeatNamed(name string) (Seat, bool) {
for _, s := range seats {
if s.Name == name {
return s, true
}
}
if canonical, aliased := aliases[name]; aliased {
for _, s := range seats {
if s.Name == canonical {
return s, true
}
}
}
return Seat{}, false
}
@@ -77,35 +181,93 @@ func SeatDelivering(provision string) (Seat, bool) {
return Seat{}, false
}
// claimProblems is what is wrong with a manifest's claims against the set.
// claimProblems is what is wrong with a manifest's claims and the seats it defines.
//
// Three refusals, each naming the seat: a seat the mesh does not define, a seat claimed at another
// scope, and a seat that delivers a provision claimed by a module that does not provide it — which
// would make the module the mesh's answer for something it cannot answer.
// A claim is one of three things (novox/hq ADR 0121): a **system seat** the control plane defines —
// checked for scope and, if it delivers a provision, that the claimant provides it; a **system name
// the mesh does not define** (`mesh-*`/`node-*`) — refused, because that namespace is the control
// plane's; or a **module-defined seat** — valid only when this manifest also declares it, since a
// module may coordinate its own instances through a seat of its own but may not invent one by
// claiming it. A module's own seat declaration may not sit in the system namespace or shadow a
// system seat.
func claimProblems(m Manifest) []string {
var problems []string
for _, c := range m.Claims {
seat, known := SeatNamed(c.Name)
if !known {
defined := map[string]SeatDeclaration{}
for _, d := range m.DefinesSeats {
if _, isSystem := SeatNamed(d.Name); isSystem || isSystemSeatName(d.Name) {
problems = append(problems, fmt.Sprintf(
"%s claims %q, which is not a seat this mesh defines (novox/hq ADR 0110) — "+
"the seats are: %s", m.Module, c.Name, seatNames()))
"%s defines a seat %q in the mesh's own namespace; a module's seat is named outside "+
"mesh-*/node-* (novox/hq ADR 0121)", m.Module, d.Name))
continue
}
if c.At() != seat.Scope {
problems = append(problems, fmt.Sprintf(
"%s claims %s at scope %q, and %s is a %s seat",
m.Module, c.Name, c.At(), c.Name, seat.Scope))
defined[d.Name] = d
}
if seat.Delivers != "" && !providesAt(m, seat.Delivers, seat.Scope) {
for _, c := range m.Claims {
if _, known := SeatNamed(c.Name); known {
// **A seat's scope and what it delivers are not judged here** (novox/hq ADR 0122).
// This function runs wherever a manifest is parsed, and one of those places is the
// build machine, which has no store: there, `SeatNamed` answers from the set the
// binary shipped with, so a build would be refused for disagreeing with a compiled
// copy of data the control plane owns. Exactly that happened — a holder of the bus
// seat was refused for not providing what a stale compiled row said the seat
// delivered, while the store's own row said otherwise.
//
// Both checks moved to CatalogueProblems, which only ever runs in the control plane,
// after UseSeats has replaced the set with the store's.
continue
}
if isSystemSeatName(c.Name) {
problems = append(problems, fmt.Sprintf(
"%s claims %s, whose holder answers for %q, and %s does not provide %q at %s scope",
m.Module, c.Name, seat.Delivers, m.Module, seat.Delivers, seat.Scope))
"%s claims %q, which is a seat in the mesh's own namespace (mesh-*/node-*) that it "+
"does not define (novox/hq ADR 0121) — the seats are: %s", m.Module, c.Name, seatNames()))
continue
}
d, ours := defined[c.Name]
if !ours {
// **A claim on a seat this manifest does not declare is not the parser's to judge.**
// A module may hold a seat another module declared — that is why ADR 0126 has callers
// name the seat and not its provider, so an implementation can be replaced without
// touching a caller. Whether the seat exists is a fact about the whole catalogue, so
// the refusal is at registration, where every declaration is in view
// (`CatalogueProblems`: "which no module declares and the mesh does not define").
continue
}
if c.At() != d.At() {
problems = append(problems, fmt.Sprintf(
"%s claims its own seat %s at scope %q, having declared it at %q",
m.Module, c.Name, c.At(), d.At()))
}
}
return problems
}
// CanHold is why a module could not hold a seat, or nothing: its definition must claim the seat at
// the seat's scope, and provide what the seat delivers, if it delivers anything. The seat is the
// store's row, so this is judged only where the store's set is loaded — at registration and in the
// handover command (novox/hq ADR 0131), never in the parser.
func CanHold(m Manifest, seat Seat) error {
var claimed *Claim
for i := range m.Claims {
if hs, ok := SeatNamed(m.Claims[i].Name); ok && hs.Name == seat.Name {
claimed = &m.Claims[i]
}
}
if claimed == nil {
return fmt.Errorf("%s does not claim %s", m.Module, seat.Name)
}
if claimed.At() != seat.Scope {
return fmt.Errorf("%s claims %s at scope %q, and %s is a %s seat",
m.Module, seat.Name, claimed.At(), seat.Name, seat.Scope)
}
if seat.Delivers != "" && !providesAt(m, seat.Delivers, seat.Scope) {
return fmt.Errorf("%s claims %s, whose holder answers for %q, and %s does not provide %q at %s scope",
m.Module, seat.Name, seat.Delivers, m.Module, seat.Delivers, seat.Scope)
}
return nil
}
func providesAt(m Manifest, provision, scope string) bool {
for _, o := range m.Provides {
if o.Name == provision && o.At() == scope {
@@ -136,7 +298,10 @@ func HolderAmong(provision string, providers []Provider, held []Held) (Provider,
return Provider{}, false
}
for _, h := range held {
if h.Claim != seat.Name || h.Scope != seat.Scope {
// Resolve the held claim to a seat rather than comparing names, so a record naming a seat's
// former name still matches it after a rename (novox/hq ADR 0122).
hs, ok := SeatNamed(h.Claim)
if !ok || hs.Name != seat.Name || h.Scope != seat.Scope {
continue
}
for _, p := range providers {
@@ -147,3 +312,18 @@ func HolderAmong(provision string, providers []Provider, held []Held) (Provider,
}
return Provider{}, false
}
// SeatsWithAProtocol are the mesh's own seats that say something about what may be said to them or by
// them, which is the set the bus derives streams, consumers and permissions from.
//
// Most of the set is not here, and that is the ordinary case: a seat saying only who does a job grants
// nothing on the bus and needs no queue.
func SeatsWithAProtocol() []Seat {
var out []Seat
for _, s := range seats {
if len(s.Accepts) > 0 || len(s.Emits) > 0 || len(s.Serves) > 0 {
out = append(out, s)
}
}
return out
}
+247
View File
@@ -0,0 +1,247 @@
package catalogue
import (
"fmt"
"sort"
"strings"
)
// Seats a module declares of its own (novox/hq ADR 0118).
//
// The set of seats a mesh has is **derived**: the mesh's own, in seats.go, plus those declared by
// every module it has registered. Still closed — a seat named nowhere is refused — but computed
// from the catalogue rather than written in the controller, which is what ADR 0110 actually
// needed and a hand-maintained table could not keep. Its own evidence: the enumeration done by
// hand while that record was written reported eleven claims where there were thirteen.
//
// **What can be checked from one manifest and what cannot.** A declaration's shape, its scope,
// and the reserved prefix are facts about the manifest in front of you. Whether a seat anybody
// names actually exists, whether two modules declared the same one, and whether a holder
// satisfies the protocol are facts about the *catalogue* — so they are checked at registration,
// by CatalogueProblems, which is the last moment the mesh can still say no.
// meshSeatPrefix is reserved to the mesh. The prefix *is* the reservation rule: no list of
// reserved names to maintain, no way for the mesh's own namespace to be colonised by a manifest,
// and nothing to keep in step when a mesh seat is added.
const meshSeatPrefix = "mesh-"
// A SeatDeclaration is a role a module offers on the bus: what may be sent to it, what it says,
// and what it answers. A caller declares that it uses the *seat*, never the module, so the
// implementation can be replaced under it.
type SeatDeclaration struct {
Name string `json:"name"`
Scope string `json:"scope,omitempty"`
// Accepts are the verbs others may submit work on. Each becomes a work-queue subject, and
// the holder is the only consumer — so exactly one worker does the job, by construction
// rather than by how carefully somebody wrote a subscribe call.
Accepts []string `json:"accepts,omitempty"`
// Emits are the verbs the holder publishes: 1:many, nobody obliged to act.
Emits []string `json:"emits,omitempty"`
// Serves are the verbs the holder answers: request and reply, awaited.
Serves []string `json:"serves,omitempty"`
// RetainSeconds is how long the inbound backlog survives with no holder, zero for the
// mesh's default. Retention belongs to whoever owns the namespace (design 29 §3) — a seat
// owns its own, which is why a seat is also the answer for a module that needs retention
// its events cannot have.
RetainSeconds int `json:"retain-seconds,omitempty"`
}
// At is this declaration's scope, with the default applied. Mesh by default, because a seat
// declared by a module is nearly always "there is one of these in the mesh" — a per-node worker
// is the deliberate case, and says so.
func (s SeatDeclaration) At() string {
if s.Scope == "" {
return ScopeMesh
}
return s.Scope
}
// verbs is everything the protocol names, for the checks that do not care which half.
func (s SeatDeclaration) verbs() []string {
out := append([]string{}, s.Accepts...)
out = append(out, s.Emits...)
return append(out, s.Serves...)
}
// declaredSeatProblems is what one manifest can be judged on alone.
func declaredSeatProblems(m Manifest) []string {
var problems []string
seen := map[string]bool{}
for _, s := range m.DefinesSeats {
switch {
case s.Name == "":
problems = append(problems, fmt.Sprintf("%s declares a seat with no name", m.Module))
continue
case !name.MatchString(s.Name):
problems = append(problems, fmt.Sprintf(
"%s declares a seat named %q, which is not a usable name", m.Module, s.Name))
continue
case strings.HasPrefix(s.Name, meshSeatPrefix):
// The mesh's own code dereferences its seats by name — the resolver *is* the thing
// that finds the store — so the prefix is not a convention, it is a namespace.
problems = append(problems, fmt.Sprintf(
"%s declares a seat named %q; %q is reserved to the mesh, which defines its own "+
"seats (novox/hq ADR 0118)", m.Module, s.Name, meshSeatPrefix+"*"))
continue
}
if seen[s.Name] {
problems = append(problems, fmt.Sprintf(
"%s declares the seat %q twice", m.Module, s.Name))
continue
}
seen[s.Name] = true
if _, isMesh := SeatNamed(s.Name); isMesh {
problems = append(problems, fmt.Sprintf(
"%s declares %q, which is a seat the mesh already defines", m.Module, s.Name))
}
switch s.At() {
case ScopeNode, ScopeSite, ScopeMesh:
default:
problems = append(problems, fmt.Sprintf(
"%s declares seat %s at scope %q; a seat is held per node, per site or per mesh",
m.Module, s.Name, s.Scope))
}
// A seat with an empty protocol is allowed, and is the mesh saying what a machine is:
// which module is this node's packet filter, or its showcase. Design 26 calls it a seat
// that delivers nothing, and that is most of the node-scoped ones. ADR 0126's "a declared
// seat carries a protocol" governs what a holder must satisfy, not that every seat offers
// something — a marker seat's protocol is satisfied by holding it. Nothing can reach this
// state by accident: a mistyped field name is refused by the parser above, so an empty
// protocol was written as one.
for _, v := range s.verbs() {
if !name.MatchString(v) {
problems = append(problems, fmt.Sprintf(
"%s declares %s.%s, which is not a usable verb", m.Module, s.Name, v))
}
}
}
for _, u := range m.Uses {
if !name.MatchString(u) {
problems = append(problems, fmt.Sprintf("%s uses %q, which is not a usable seat name", m.Module, u))
}
}
return problems
}
// A Shelf is every manifest the mesh has registered, by module name.
type Shelf map[string]Manifest
// CatalogueProblems are the rules no single manifest can be judged against.
//
// Run at registration, which is the last moment the mesh can still refuse: after it, a caller is
// bound to a seat and a refusal is an outage rather than a conversation.
func CatalogueProblems(shelf Shelf) []string {
var problems []string
// Who declares what, and who declared it first.
declaredBy := map[string]string{}
declared := map[string]SeatDeclaration{}
for _, module := range shelfOrder(shelf) {
for _, s := range shelf[module].DefinesSeats {
if s.Name == "" {
continue
}
if first, taken := declaredBy[s.Name]; taken {
// The second loses. A seat name meaning two different protocols is the failure
// nobody could diagnose afterwards — a caller would bind to whichever happened
// to register first, and the symptom would appear in the other module.
problems = append(problems, fmt.Sprintf(
"%s declares the seat %q, which %s already declares; a seat name means one "+
"protocol", module, s.Name, first))
continue
}
declaredBy[s.Name] = module
declared[s.Name] = s
}
}
exists := func(seat string) bool {
if _, isMesh := SeatNamed(seat); isMesh {
return true
}
_, ok := declaredBy[seat]
return ok
}
for _, module := range shelfOrder(shelf) {
m := shelf[module]
// A `uses` naming nothing is where ADR 0110's guarantee lands under a derived set: the
// same refusal, at the same moment, from a set nobody maintains by hand.
for _, u := range m.Uses {
if !exists(u) {
problems = append(problems, fmt.Sprintf(
"%s uses the seat %q, which no module declares and the mesh does not define",
module, u))
}
}
for _, c := range m.Claims {
if !exists(c.Name) {
problems = append(problems, fmt.Sprintf(
"%s claims the seat %q, which no module declares and the mesh does not define",
module, c.Name))
continue
}
s, isModuleSeat := declared[c.Name]
if !isModuleSeat {
// **A mesh seat is judged here and nowhere else** (novox/hq ADR 0122): the set is
// the store's, and this is the only place that runs with the store's set loaded.
// The parser cannot do it — it also runs on the build machine, against whatever
// set that binary was compiled with.
seat, _ := SeatNamed(c.Name)
if err := CanHold(m, seat); err != nil {
problems = append(problems, err.Error())
}
continue
}
if c.At() != s.At() {
problems = append(problems, fmt.Sprintf(
"%s claims %s at scope %q, and %s declares it at %s",
module, c.Name, c.At(), declaredBy[c.Name], s.At()))
}
// A holder that does not answer what the seat promises is a caller's timeout, found
// at assignment instead.
if missing := unserved(m, s); len(missing) > 0 {
problems = append(problems, fmt.Sprintf(
"%s claims %s but does not serve %s, which that seat's protocol promises",
module, c.Name, strings.Join(missing, ", ")))
}
}
}
sort.Strings(problems)
return problems
}
// unserved is what a seat's protocol promises and the claimant does not answer. Only the tools
// are checked: `accepts` and `emits` are wired by the runtime from the declaration, while a tool
// is code the module either has or has not written.
func unserved(m Manifest, s SeatDeclaration) []string {
has := map[string]bool{}
for _, t := range m.Tools {
has[t] = true
}
var missing []string
for _, t := range s.Serves {
if !has[t] {
missing = append(missing, t)
}
}
return missing
}
// shelfOrder is the catalogue in a stable order, so two runs report the same problems in the same
// sequence — a refusal that reorders itself is a refusal nobody can diff.
func shelfOrder(shelf Shelf) []string {
out := make([]string, 0, len(shelf))
for k := range shelf {
out = append(out, k)
}
sort.Strings(out)
return out
}
+146
View File
@@ -0,0 +1,146 @@
package catalogue
import (
"strings"
"testing"
)
func problemsFor(t *testing.T, shelf Shelf) string {
t.Helper()
return strings.Join(CatalogueProblems(shelf), "; ")
}
func telegram() Manifest {
return Manifest{Module: "telegram", Tools: []string{"status"}, DefinesSeats: []SeatDeclaration{{
Name: "telegram-sender", Scope: ScopeMesh,
Accepts: []string{"send"}, Emits: []string{"delivered", "failed"}, Serves: []string{"status"},
}}, Claims: []Claim{{Name: "telegram-sender", Scope: ScopeMesh}}}
}
// The whole point: a module contributes a capability without the mesh being changed.
func TestAModuleDeclaresItsOwnSeatAndHoldsIt(t *testing.T) {
shop := Manifest{Module: "shop", Uses: []string{"telegram-sender"}}
if got := problemsFor(t, Shelf{"telegram": telegram(), "shop": shop}); got != "" {
t.Fatalf("a declared seat and its caller were refused: %s", got)
}
}
// The prefix is the reservation rule, so there is no list to maintain and none to drift.
func TestAModuleCannotDeclareAMeshSeat(t *testing.T) {
for _, n := range []string{"mesh-broker", "mesh-anything", "mesh-store"} {
m := Manifest{Module: "impostor", DefinesSeats: []SeatDeclaration{{Name: n, Accepts: []string{"x"}}}}
got := strings.Join(declaredSeatProblems(m), "; ")
if !strings.Contains(got, "reserved to the mesh") {
t.Fatalf("%q was accepted as a module's seat: %q", n, got)
}
}
}
// A seat name meaning two protocols is the failure nobody could diagnose afterwards.
func TestTwoModulesCannotDeclareTheSameSeat(t *testing.T) {
other := Manifest{Module: "aardvark", DefinesSeats: []SeatDeclaration{{
Name: "telegram-sender", Scope: ScopeMesh, Accepts: []string{"something-else"}}}}
got := problemsFor(t, Shelf{"telegram": telegram(), "aardvark": other})
if !strings.Contains(got, "already declares") {
t.Fatalf("both declarations stood: %s", got)
}
// The first declarer keeps it; only the second is refused.
if strings.Count(got, "already declares") != 1 {
t.Fatalf("expected exactly one refusal: %s", got)
}
}
// Where ADR 0110's guarantee lands under a derived set: a typo is refused, not resolved to
// nothing at runtime.
func TestUsingASeatNobodyDeclaresIsRefused(t *testing.T) {
shop := Manifest{Module: "shop", Uses: []string{"telegram-sendr"}}
got := problemsFor(t, Shelf{"telegram": telegram(), "shop": shop})
if !strings.Contains(got, "telegram-sendr") || !strings.Contains(got, "no module declares") {
t.Fatalf("a misspelled seat was accepted: %s", got)
}
}
// A holder that does not answer what the seat promises is a caller's timeout, found here instead.
func TestAHolderMustServeWhatItsSeatPromises(t *testing.T) {
m := telegram()
m.Tools = nil // declares the seat, serves none of it
got := problemsFor(t, Shelf{"telegram": m})
if !strings.Contains(got, "does not serve status") {
t.Fatalf("a holder was accepted that answers nothing its seat promises: %s", got)
}
}
// A seat with no protocol is a marker: which module is this node's showcase, or its packet filter.
// Most node-scoped seats are markers, so refusing one would refuse the majority of the set.
func TestASeatWithoutAProtocolIsAMarkerNotAMistake(t *testing.T) {
m := Manifest{Module: "vague", DefinesSeats: []SeatDeclaration{{Name: "something", Scope: ScopeNode}}}
if got := strings.Join(declaredSeatProblems(m), "; "); got != "" {
t.Fatalf("a marker seat was refused: %s", got)
}
}
// A claim at the wrong scope is a different seat than the one declared.
func TestAClaimMustMatchTheDeclaredScope(t *testing.T) {
m := telegram()
m.Claims = []Claim{{Name: "telegram-sender", Scope: ScopeNode}}
got := problemsFor(t, Shelf{"telegram": m})
if !strings.Contains(got, "scope") {
t.Fatalf("a claim at the wrong scope was accepted: %s", got)
}
}
// The mesh's own seats still work, and are not shadowed by the derived half.
func TestTheMeshsOwnSeatsAreStillClaimable(t *testing.T) {
// It delivers the bus, so its holder provides the bus — the rule this check now enforces.
m := Manifest{Module: "nats",
Provides: []Offer{{Name: "mesh-bus", Scope: ScopeMesh}},
Claims: []Claim{{Name: "mesh-broker", Scope: ScopeMesh}}}
if got := problemsFor(t, Shelf{"nats": m}); got != "" {
t.Fatalf("a mesh seat was refused by the derived check: %s", got)
}
}
// A refusal that reorders itself between runs is a refusal nobody can diff.
func TestTheProblemsAreStable(t *testing.T) {
shelf := Shelf{"telegram": telegram(), "shop": {Module: "shop", Uses: []string{"nope"}},
"other": {Module: "other", Uses: []string{"also-nope"}}}
first, second := problemsFor(t, shelf), problemsFor(t, shelf)
if first != second {
t.Fatalf("unstable:\n%s\n%s", first, second)
}
}
// **A build machine has no store, so it may not judge a seat.** The set is data the control plane
// owns (novox/hq ADR 0122), and `ParseManifest` runs on the build machine too, against whatever set
// that binary was compiled with. When the two disagreed, a valid holder of the bus seat was refused
// mid-rollout — the compiled row said the seat delivered one provision, the store's row said
// another, and the build failed on the copy rather than the truth. The parser judges the manifest;
// the seat set judges the claim, where it is loaded.
func TestTheParserDoesNotJudgeWhatOnlyTheStoreKnows(t *testing.T) {
was := Seats()
t.Cleanup(func() { UseSeats(was) })
// A store whose bus seat delivers something this module does provide.
UseSeats([]Seat{{Name: "mesh-broker", Scope: ScopeMesh, Delivers: "mesh-bus", Decision: "test"}})
raw := []byte(`{"module":"a-bus","version":"1",` +
`"provides":[{"name":"mesh-bus","scope":"mesh"}],` +
`"claims":[{"name":"mesh-broker","scope":"mesh"}]}`)
m, err := ParseManifest(raw)
if err != nil {
t.Fatalf("the parser refused a claim only the seat set can judge: %v", err)
}
if got := CatalogueProblems(Shelf{m.Module: m}); len(got) != 0 {
t.Fatalf("a holder that provides what the store says the seat delivers was refused: %v", got)
}
// And with the store saying the seat delivers something else, registration is what refuses it.
UseSeats([]Seat{{Name: "mesh-broker", Scope: ScopeMesh, Delivers: "other-bus", Decision: "test"}})
if _, err := ParseManifest(raw); err != nil {
t.Fatalf("the parser judged it the second time: %v", err)
}
got := strings.Join(CatalogueProblems(Shelf{m.Module: m}), "; ")
if !strings.Contains(got, `does not provide "other-bus"`) {
t.Fatalf("registration did not refuse a holder that cannot answer for the seat: %q", got)
}
}
+136 -20
View File
@@ -50,44 +50,120 @@ func TestTheSeatsAreAClosedSetAndEachNamesItsDecision(t *testing.T) {
}
}
// novox/hq ADR 0117: a machine's uplink is a seat, held per machine, and delivers nothing.
//
// **Nothing, because nothing may be required of it.** A holder only keeps its network manager from
// contradicting the mesh; a requirement resolving to it would make the manager the mesh's answer
// for something, and the manager's link is the one thing the mesh must never be able to break.
func TestTheUplinkIsANodeSeatThatDeliversNothing(t *testing.T) {
seat, known := SeatNamed("node-uplink")
if !known {
t.Fatalf("the uplink is not a seat; the seats are: %s", seatNames())
}
if seat.Scope != ScopeNode || seat.Delivers != "" || seat.Decision != "novox/hq ADR 0117" {
t.Fatalf("the uplink is %+v, not a node seat delivering nothing by ADR 0117", seat)
}
// And a manager's module can hold it without providing anything.
raw := []byte(`{"module":"networkmanager","version":"1","claims":[{"name":"node-uplink","scope":"node"}]}`)
if _, err := ParseManifest(raw); err != nil {
t.Fatalf("a network manager's module could not hold the uplink: %v", err)
}
}
func claimed(claims string) []byte {
return []byte(`{"module":"thing","version":"1","provides":[{"name":"npm-package-registry","scope":"mesh"}],"claims":` + claims + `}`)
}
func TestAClaimOnASeatTheMeshDoesNotDefineIsRefused(t *testing.T) {
_, err := ParseManifest(claimed(`[{"name":"the-anything","scope":"node"}]`))
if err == nil {
t.Fatal("a module invented a seat by claiming it")
// **The refusal moved, it did not go** (novox/hq ADR 0118, superseding 0110). A module may now
// declare its own seats, so whether a claimed seat exists is a fact about the *catalogue* and not
// about the manifest in front of the parser: a claim on a seat another registered module declares
// is perfectly good, and the parser cannot tell the two cases apart. So the parser accepts it and
// registration refuses it — the same guarantee, at the same moment work would otherwise start,
// from a set nobody maintains by hand.
func TestAClaimOnASeatNobodyDeclaresIsRefusedAtRegistration(t *testing.T) {
m, err := ParseManifest(claimed(`[{"name":"the-anything","scope":"node"}]`))
if err != nil {
t.Fatalf("the parser judged a claim it cannot judge alone: %v", err)
}
if !strings.Contains(err.Error(), "the-anything") || !strings.Contains(err.Error(), "not a seat") {
t.Fatalf("the refusal does not say the seat is unknown: %v", err)
problems := CatalogueProblems(Shelf{m.Module: m})
if len(problems) == 0 {
t.Fatal("a module invented a seat by claiming it, and registration allowed it")
}
// And it says what the seats are, because "no" without the list sends somebody reading code.
if !strings.Contains(err.Error(), "the-packet-filter") {
t.Fatalf("the refusal does not list the seats: %v", err)
joined := strings.Join(problems, "; ")
if !strings.Contains(joined, "the-anything") || !strings.Contains(joined, "no module declares") {
t.Fatalf("the refusal does not say the seat is nobody's: %v", problems)
}
}
func TestASeatClaimedAtAnotherScopeIsRefused(t *testing.T) {
_, err := ParseManifest(claimed(`[{"name":"npm-package-registry","scope":"node"}]`))
if err == nil {
t.Fatal("a mesh seat was held per node")
// And the same claim is fine once something declares that seat, which is the case the parser
// could not have distinguished.
func TestAClaimOnASeatAnotherModuleDeclaresIsAccepted(t *testing.T) {
claimant, err := ParseManifest(claimed(`[{"name":"the-anything","scope":"node"}]`))
if err != nil {
t.Fatal(err)
}
if !strings.Contains(err.Error(), "mesh seat") {
t.Fatalf("the refusal does not say which scope the seat is: %v", err)
declarer := Manifest{Module: "someone", DefinesSeats: []SeatDeclaration{
{Name: "the-anything", Scope: ScopeNode, Accepts: []string{"work"}},
}}
if problems := CatalogueProblems(Shelf{claimant.Module: claimant, "someone": declarer}); len(problems) != 0 {
t.Fatalf("a claim on a declared seat was refused: %v", problems)
}
}
// A module may define its own seat and claim it — the mesh enforces exclusivity without knowing
// what it means (novox/hq ADR 0121). But it may not define one in the mesh's own namespace.
func TestAModuleDefinesAndClaimsItsOwnSeat(t *testing.T) {
ok := []byte(`{"module":"showcase","version":"1","seats":[{"name":"the-showcase","scope":"node"}],` +
`"claims":[{"name":"the-showcase","scope":"node"}]}`)
if _, err := ParseManifest(ok); err != nil {
t.Fatalf("a module could not define and claim its own seat: %v", err)
}
// Claiming a name nobody defines is still refused — but **at registration, not here**: with
// seats declared by modules, a claim on a seat *another* module declares is good, and the
// parser cannot tell that from an invented name. See
// TestAClaimOnASeatNobodyDeclaresIsRefusedAtRegistration.
claimant, err := ParseManifest(claimed(`[{"name":"the-anything","scope":"node"}]`))
if err != nil {
t.Fatalf("the parser judged a claim it cannot judge alone: %v", err)
}
if len(CatalogueProblems(Shelf{claimant.Module: claimant})) == 0 {
t.Fatal("a module claimed a seat nobody defines")
}
// A module may not carve its seat out of the mesh's own namespace.
bad := []byte(`{"module":"x","version":"1","seats":[{"name":"node-mine","scope":"node"}],` +
`"claims":[{"name":"node-mine","scope":"node"}]}`)
if _, err := ParseManifest(bad); err == nil || !strings.Contains(err.Error(), "own namespace") {
t.Fatalf("a module defined a seat in the mesh's namespace and was not refused: %v", err)
}
}
// **At registration, not in the parser** (novox/hq ADR 0122): a seat's scope is a property of the
// set, the set is the store's, and the parser also runs on a build machine that has no store.
func TestASeatClaimedAtAnotherScopeIsRefused(t *testing.T) {
m, err := ParseManifest(claimed(`[{"name":"npm-package-registry","scope":"node"}]`))
if err != nil {
t.Fatalf("the parser judged a scope it reads from data it may not have: %v", err)
}
got := strings.Join(CatalogueProblems(Shelf{m.Module: m}), "; ")
if !strings.Contains(got, "mesh seat") {
t.Fatalf("the refusal does not say which scope the seat is: %q", got)
}
}
func TestADeliveringSeatIsOnlyHeldByAModuleThatProvides(t *testing.T) {
// Holding it makes the module the mesh's answer for the provision. A module that cannot answer
// would be the answer anyway, and every consumer would be sent to it.
// And refused at registration, where the seat set is the store's: what a seat delivers is
// data, so a compiled copy of it may not be what refuses a build (novox/hq ADR 0122).
raw := []byte(`{"module":"thing","version":"1","claims":[{"name":"git","scope":"mesh"}]}`)
_, err := ParseManifest(raw)
if err == nil {
t.Fatal("a module holding the git seat need not provide git")
m, err := ParseManifest(raw)
if err != nil {
t.Fatalf("the parser judged what a seat delivers: %v", err)
}
if !strings.Contains(err.Error(), `does not provide "git"`) {
t.Fatalf("the refusal does not say what is missing: %v", err)
got := strings.Join(CatalogueProblems(Shelf{m.Module: m}), "; ")
if !strings.Contains(got, `does not provide "git"`) {
t.Fatalf("the refusal does not say what is missing: %q", got)
}
}
@@ -206,3 +282,43 @@ func TestTheHolderIsTheModuleNotTheMachine(t *testing.T) {
t.Fatalf("the holder was not told apart from a neighbour: %+v", holder)
}
}
// The working set is loaded from the store, and an empty load never erases it (novox/hq ADR 0122).
func TestUseSeatsReplacesTheSetButNeverEmptiesIt(t *testing.T) {
before := Seats()
defer UseSeats(DefaultSeats()) // restore for other tests, whatever this leaves it as
// An empty load (store not seeded, or unreadable) leaves the compiled defaults in force.
UseSeats(nil)
if len(Seats()) != len(before) {
t.Fatalf("an empty load changed the set from %d to %d seats", len(before), len(Seats()))
}
// A non-empty load replaces it — this is how a rename in the store reaches the lookups.
UseSeats([]Seat{{Name: "node-firewall", Scope: ScopeNode, Decision: "novox/hq ADR 0122"}})
if _, known := SeatNamed("node-firewall"); !known {
t.Fatal("the loaded set did not replace the working set")
}
if len(Seats()) != 1 {
t.Fatalf("the working set is %d seats, not the one that was loaded", len(Seats()))
}
}
// A former name resolves to the seat it was renamed from (novox/hq ADR 0122), so a manifest's claim
// and a held record naming the old name break nothing after a rename.
func TestAFormerNameResolvesAfterARename(t *testing.T) {
defer func() { UseSeats(DefaultSeats()); UseAliases(nil) }()
UseSeats([]Seat{{Name: "git", Scope: ScopeMesh, Delivers: "git", Decision: "novox/hq ADR 0121"}})
UseAliases(map[string]string{"git": "git"})
// The old name resolves to the renamed seat.
if s, ok := SeatNamed("git"); !ok || s.Name != "git" {
t.Fatalf("the former name did not resolve to the renamed seat: %+v ok=%v", s, ok)
}
// And a holder recorded under the old name is still found for the provision the seat delivers.
providers := []Provider{{Node: "anchor", At: "anchor.internal", Module: "gitea"}}
held := []Held{{Claim: "git", Scope: ScopeMesh, Node: "anchor", Module: "gitea"}}
holder, found := HolderAmong("git", providers, held)
if !found || holder.Module != "gitea" {
t.Fatalf("the holder recorded under the former name was not matched: %+v found=%v", holder, found)
}
}
+46
View File
@@ -0,0 +1,46 @@
package catalogue
import (
"strings"
"testing"
)
// The ssh-client module, composed as a machine receives it (novox/hq to-be 29): every other node's
// Host block written into a marked region of the operator's ~/.ssh/config, owned by the account,
// with ~/.ssh created 0700 — the operator's own config kept.
func TestSSHClientOwnsTheOperatorsSSHConfig(t *testing.T) {
shelf := shelf(catalogueManifest(t, "ssh-client"))
got, err := Resolve(shelf, []string{"ssh-client"},
Node{Name: "homer", At: "homer.internal", Account: "jo"}, World{})
if err != nil {
t.Fatal(err)
}
names := map[string]string{"homer.internal": "10.10.0.1", "marge.internal": "10.10.0.2"}
out, err := got.Declaration(Rendering{
Names: names, Machines: names, Accounts: map[string]string{"homer": "jo", "marge": "jo"},
Suffix: "internal",
})
if err != nil {
t.Fatal(err)
}
by := map[string]map[string]any{}
for _, r := range out {
by[r["id"].(string)] = r
}
dir := by["ssh-client.ssh-dir"]
if dir == nil || dir["path"] != "/home/jo/.ssh" || dir["owner"] != "jo" || dir["mode"] != "0700" {
t.Fatalf("~/.ssh is not created 0700 owned by the account: %v", dir)
}
cfg := by["ssh-client.fact-ssh-config"]
if cfg == nil || cfg["path"] != "/home/jo/.ssh/config" || cfg["owner"] != "jo" || cfg["into"] != "block" {
t.Fatalf("the ssh config is not written into the operator's ~/.ssh/config as a region: %v", cfg)
}
body := cfg["content"].(string)
if !strings.Contains(body, "Host marge marge.internal") || !strings.Contains(body, "User jo") {
t.Fatalf("the config does not name the peer node and its account:\n%s", body)
}
if strings.Contains(body, "Host homer ") {
t.Fatalf("the config names the machine itself, not only its peers:\n%s", body)
}
}
+35
View File
@@ -164,6 +164,41 @@ func (i *Inventory) Held(ctx context.Context) (map[string]string, error) {
return held, rows.Err()
}
// BuiltAgainst is what each module's newest successful build stood on, as recorded — the build
// edges (ADR 0009). A module whose last build recorded no bases is absent, which is also what a
// module standing on nothing looks like: an edge the mesh has not derived is not an edge.
func (i *Inventory) BuiltAgainst(ctx context.Context) (map[string][]string, error) {
rows, err := i.store.Pool().Query(ctx,
`select distinct on (module) module, built_against
from build
where module is not null and module <> '' and failed = ''
order by module, at desc`)
if err != nil {
return nil, err
}
defer rows.Close()
against := map[string][]string{}
for rows.Next() {
var module string
var raw []byte
if err := rows.Scan(&module, &raw); err != nil {
return nil, err
}
if len(raw) == 0 {
continue
}
var refs []string
if err := json.Unmarshal(raw, &refs); err != nil {
continue
}
if len(refs) > 0 {
against[module] = refs
}
}
return against, rows.Err()
}
// manifestOrNil keeps the difference between "declared nothing" and "predates this being kept".
//
// A build recorded before the mesh kept manifests has no manifest, and that is not the same as one
+178
View File
@@ -0,0 +1,178 @@
package inventory
import (
"context"
"fmt"
"strings"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
)
// What the bus's user list is derived from, read out of the mesh's records.
//
// The deriving itself is pure and lives in the broker package; this is the reading, and it is kept
// apart for the reason that package keeps its own types: a permission must be a function of what a
// module declared, and a query that decided anything would be a second place authority came from.
// BusRecords is every fact the composer needs about who may reach the bus.
//
// **A module's authority comes from the manifest, not from the assignment.** The assignment says
// *where* it runs; what it may say is in what it declared, so the two are read together and the
// manifest is the one that decides.
func (i *Inventory) BusRecords(ctx context.Context) (broker.Records, error) {
nodes, err := i.Nodes(ctx)
if err != nil {
return broker.Records{}, fmt.Errorf("cannot read the mesh's machines: %w", err)
}
declared, err := i.Catalogue(ctx)
if err != nil {
return broker.Records{}, fmt.Errorf("cannot read the catalogue: %w", err)
}
// Every seat any module declares, by name, so a module's claim can be resolved to the protocol
// that seat promises. **Across the whole catalogue, not one manifest**: a seat is declared by
// one module and held by another, which is the whole reason a seat exists (ADR 0118).
seats := map[string]catalogue.SeatDeclaration{}
for _, m := range declared {
for _, s := range m.DefinesSeats {
seats[s.Name] = s
}
}
// And the mesh's own, which carry protocol too (novox/hq ADR 0121). Added after the modules'
// rather than before, because a `mesh-*` name is the mesh's and registration refuses a module
// declaring one — so this cannot be shadowed, and if it ever were, the mesh's own would win.
for _, own := range catalogue.SeatsWithAProtocol() {
seats[own.Name] = catalogue.SeatDeclaration{
Name: own.Name, Scope: own.Scope,
Accepts: own.Accepts, Emits: own.Emits, Serves: own.Serves,
}
}
out := broker.Records{Assigned: map[string][]broker.Declared{}, People: map[string][]string{}}
for _, n := range nodes {
out.Nodes = append(out.Nodes, n.Name)
modules, err := i.Assigned(ctx, n.Name)
if err != nil {
return broker.Records{}, fmt.Errorf("cannot read what %s runs: %w", n.Name, err)
}
for _, module := range modules {
m, known := declared[module]
if !known {
// Assigned and not in the catalogue. Said rather than composed with no authority:
// a user with an empty permission list is a module that starts, connects, and is
// refused by the server on its first publish — an authorisation error that says
// nothing about a missing manifest.
//
// **The catalogue refuses to forget an assigned module, so this is the second line
// and not the first.** It earns its place there anyway: relying on another
// package's invariant is how a rule ends up enforced by nothing.
return broker.Records{}, fmt.Errorf(
"%s is assigned to %s and is not in the catalogue, so what it may say cannot "+
"be derived", module, n.Name)
}
out.Assigned[n.Name] = append(out.Assigned[n.Name], declaredFor(m, seats))
}
}
enrolling, err := i.NodesWithALiveToken(ctx)
if err != nil {
return broker.Records{}, err
}
out.Enrolling = enrolling
people, err := i.People(ctx)
if err != nil {
return broker.Records{}, err
}
for _, p := range people {
out.People[p.Name] = p.Invokes
}
return out, nil
}
// declaredFor is one module's manifest as the composer needs it: what it says about itself, and the
// protocol of every seat it holds or uses.
func declaredFor(m catalogue.Manifest, seats map[string]catalogue.SeatDeclaration) broker.Declared {
// A consumed name is a module's event unless it names a seat, and only somebody holding the seat
// set can tell (novox/hq ADR 0121). Split here, because the composer cannot look at a name and
// know — and a role's event read as a module's is a subscription to a namespace nobody owns.
var fromModules []string
var watches []broker.Seat
for _, c := range m.Consumes {
emitter, event, named := strings.Cut(c, ".")
if named {
if s, isASeat := seats[emitter]; isASeat {
watches = append(watches, broker.Seat{Name: s.Name, Emits: []string{event}})
continue
}
}
fromModules = append(fromModules, c)
}
d := broker.Declared{
Module: m.Module,
Emits: m.Emits,
Consumes: fromModules,
Watches: watches,
// The tools it answers, which is `tools` and not `serves`: the manifest's `serves` is the
// facts a consumer needs to reach a provision, a different meaning under a similar word.
Serves: m.Tools,
}
for _, c := range m.Claims {
// Every seat with a protocol, the mesh's own included. One that says only who does a job is
// not here and grants nothing, which is most of them.
if s, hasAProtocol := seats[c.Name]; hasAProtocol {
d.Holds = append(d.Holds, asSeat(s))
}
}
for _, name := range m.Uses {
if s, declaredSomewhere := seats[name]; declaredSomewhere {
d.Uses = append(d.Uses, asSeat(s))
}
}
return d
}
func asSeat(s catalogue.SeatDeclaration) broker.Seat {
return broker.Seat{Name: s.Name, Accepts: s.Accepts, Emits: s.Emits, Serves: s.Serves}
}
// MeshSeats are the mesh's own seats that carry a protocol, as the bus needs them: what to make a work
// queue for, and whose holder gets a worker on it (novox/hq ADR 0121).
func MeshSeats() []broker.DeclaredSeat {
var out []broker.DeclaredSeat
for _, s := range catalogue.SeatsWithAProtocol() {
out = append(out, broker.DeclaredSeat{
Name: s.Name, Accepts: s.Accepts, Emits: s.Emits, Serves: s.Serves,
})
}
return out
}
// NodesWithALiveToken is every machine holding a token that could still be presented — issued, not
// expired, not redeemed.
//
// **One enrolment user per such token** (design 25 §6): the inbox an answer goes to is scoped to the
// token, because an answer carries that machine's credentials sealed to it and a shared inbox is one
// machine able to read another's.
func (i *Inventory) NodesWithALiveToken(ctx context.Context) ([]string, error) {
rows, err := i.store.Pool().Query(ctx,
`select distinct n.name
from enrolment_token t join node n on n.id = t.node
where t.redeemed is null and t.expires > now()
order by n.name`)
if err != nil {
return nil, fmt.Errorf("cannot read which machines hold a live token: %w", err)
}
defer rows.Close()
var out []string
for rows.Next() {
var name string
if err := rows.Scan(&name); err != nil {
return nil, err
}
out = append(out, name)
}
return out, rows.Err()
}
+166
View File
@@ -0,0 +1,166 @@
package inventory
import (
"context"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
)
// Reading the bus's user list out of the mesh's records, against a real store.
//
// What each of these is about is a user that would be **missing or wrong in a way nothing reports**:
// the server reads whatever file it is given, and a module whose user is absent fails on its first
// publish with an authorisation error that says nothing about a missing assignment.
func aMeshWith(t *testing.T, manifests ...catalogue.Manifest) (*Inventory, context.Context) {
t.Helper()
inv := ForTest(t)
ctx := context.Background()
for _, m := range manifests {
if err := inv.RegisterModule(ctx, m, Source{Repository: "/r"}); err != nil {
t.Fatal(err)
}
}
return inv, ctx
}
func theSeatDeclarer() catalogue.Manifest {
return catalogue.Manifest{
Module: "telegram", Version: "1",
DefinesSeats: []catalogue.SeatDeclaration{{
Name: "telegram-sender", Accepts: []string{"send"}, Emits: []string{"delivered"},
}},
Claims: []catalogue.Claim{{Name: "telegram-sender", Scope: catalogue.ScopeMesh}},
}
}
// A module assigned to a machine becomes a user with the authority its manifest declared — and the
// protocol of a seat declared by a *different* module, which is the whole reason a seat exists.
func TestAnAssignedModuleBecomesAUserWithWhatItDeclared(t *testing.T) {
shop := catalogue.Manifest{
Module: "shop", Version: "1",
Emits: []string{"order.placed"}, Tools: []string{"price"},
Uses: []string{"telegram-sender"},
}
inv, ctx := aMeshWith(t, theSeatDeclarer(), shop)
if _, err := inv.AddNode(ctx, "one"); err != nil {
t.Fatal(err)
}
if _, err := inv.Assign(ctx, "one", "shop"); err != nil {
t.Fatal(err)
}
records, err := inv.BusRecords(ctx)
if err != nil {
t.Fatal(err)
}
on := records.Assigned["one"]
if len(on) != 1 || on[0].Module != "shop" {
t.Fatalf("the machine's modules read as %+v", on)
}
if len(on[0].Uses) != 1 || on[0].Uses[0].Accepts[0] != "send" {
t.Fatalf("the seat it uses carries no protocol: %+v — so it would be granted nothing on a "+
"seat it was assigned to send to", on[0].Uses)
}
if len(on[0].Serves) != 1 || on[0].Serves[0] != "price" {
t.Fatalf("its tools read as %v, and a module that cannot subscribe its own tool subject "+
"serves nothing", on[0].Serves)
}
// And it derives into a user the server would accept.
users, err := broker.Users(records)
if err != nil {
t.Fatal(err)
}
var found bool
for _, u := range users {
if u.Username() != "one.shop" {
continue
}
found = true
perms, err := broker.PermissionsFor(u)
if err != nil {
t.Fatal(err)
}
// Its tools are every one under its own name — the list in the manifest is a person's
// vocabulary for asking, not the module's permission to answer.
if !granted(perms.Publish, "mesh.mod.shop.event.order.placed") ||
!granted(perms.Publish, "mesh.seat.telegram-sender.accept.send") ||
!granted(perms.Subscribe, "mesh.mod.shop.tool.>") {
t.Fatalf("one.shop's authority is not what it declared: %+v", perms)
}
}
if !found {
t.Fatal("no user was derived for the assigned module")
}
}
// A machine holding a live token gets an enrolment user; one whose token is spent or expired does
// not. **An enrolment user outliving its token is a right to join that nobody issued.**
func TestOnlyAMachineWithALiveTokenHasAnEnrolmentUser(t *testing.T) {
inv, ctx := aMeshWith(t)
for _, name := range []string{"live", "expired", "none"} {
if _, err := inv.AddNode(ctx, name); err != nil {
t.Fatal(err)
}
}
if _, err := inv.IssueToken(ctx, "live", time.Hour); err != nil {
t.Fatal(err)
}
// Briefly, then waited out: a token with no lifetime is refused at issue, which is the right
// refusal and leaves this as the way to have an expired one.
if _, err := inv.IssueToken(ctx, "expired", 10*time.Millisecond); err != nil {
t.Fatal(err)
}
time.Sleep(50 * time.Millisecond)
records, err := inv.BusRecords(ctx)
if err != nil {
t.Fatal(err)
}
if strings.Join(records.Enrolling, ",") != "live" {
t.Fatalf("machines with a live token read as %v", records.Enrolling)
}
}
// A module assigned and absent from the catalogue is refused rather than composed with no authority.
//
// **The catalogue refuses to forget an assigned module, so this is the second line and not the
// first** — and it earns its place there: relying on another package's invariant is how a rule ends
// up enforced by nothing. Checked against the derivation directly, because the situation cannot be
// reached through the store.
func TestAnAssignmentWithNoManifestDerivesNoAuthority(t *testing.T) {
// What BusRecords would have produced had it composed a ghost: a module with nothing declared.
users, err := broker.Users(broker.Records{
Nodes: []string{"one"},
Assigned: map[string][]broker.Declared{"one": {{Module: "ghost"}}},
})
if err != nil {
t.Fatal(err)
}
perms, err := broker.PermissionsFor(users[len(users)-1])
if err != nil {
t.Fatal(err)
}
// Its inbox and its ack subject, and nothing it could say. That is a module which starts,
// connects, and is refused by the server on its first publish — an authorisation error that
// says nothing about a missing manifest, which is why BusRecords names it instead.
for _, p := range perms.Publish {
if strings.HasPrefix(p, "mesh.mod.ghost.event.") {
t.Fatalf("a module with no manifest was granted %s", p)
}
}
}
func granted(all []string, one string) bool {
for _, s := range all {
if s == one {
return true
}
}
return false
}
+264
View File
@@ -0,0 +1,264 @@
package inventory
import (
"context"
"crypto/rand"
"encoding/base64"
"errors"
"fmt"
"github.com/jackc/pgx/v5"
"golang.org/x/crypto/bcrypt"
)
// The bus's own users, as records.
//
// **Only the credential is kept here.** A user's *authority* is derived from what its module
// declares, every time the file is written (novox/hq ADR 0043) — a stored copy of a permission list
// would be a second account of a user's authority, able to disagree with the first, and the
// disagreement would be invisible until somebody compared a composed file with a manifest.
//
// What cannot be derived is the password, and on the bus being built it has to outlive its own
// minting: the whole user list is one file, rewritten whenever any of it changes, so a person's
// access change would blank every module's password if the mesh kept nothing (design 25 §4, and the
// migration beside this).
// BusUser is one user of the bus, as the mesh records it.
type BusUser struct {
Username string
Kind string
Node string
Module string
// PasswordHash is what the composed file carries. The plaintext is returned once, by Mint, and
// then exists only where it was sealed.
PasswordHash string
}
// The kinds of bus user the mesh records. The same words the composer uses, so a row and a
// principal do not need a translation table between them.
const (
BusController = "controller"
BusNode = "node"
BusModule = "module"
BusEnrolment = "enrolment"
BusPerson = "person"
)
// MintBusPassword makes a bus password and records its hash under a username, replacing whatever was
// there, and returns the plaintext **once**.
//
// **Once is the whole contract.** The caller seals it to whoever will use it — into an enrolment
// reply, into a module's sealed environment — and the mesh keeps only the hash, so a credential is
// never recoverable from the store. A caller that loses it must mint again, which is a rotation and
// is meant to feel like one.
func (i *Inventory) MintBusPassword(ctx context.Context, u BusUser) (string, error) {
if u.Username == "" || u.Kind == "" {
return "", errors.New("a bus user needs a username and a kind")
}
raw := make([]byte, 32)
if _, err := rand.Read(raw); err != nil {
return "", fmt.Errorf("cannot generate a bus password: %w", err)
}
password := base64.RawURLEncoding.EncodeToString(raw)
// The cost the server will pay on every connection. Left at the library's default rather than
// raised: a node reconnecting after a network blip pays it, and the mesh's own links reconnect
// far more often than a person logs in anywhere.
hash, err := bcrypt.GenerateFromPassword([]byte(password), bcrypt.DefaultCost)
if err != nil {
return "", fmt.Errorf("cannot hash a bus password: %w", err)
}
if _, err := i.store.Pool().Exec(ctx,
`insert into bus_user (username, kind, node, module, password_hash)
values ($1, $2, $3, $4, $5)
on conflict (username) do update
set kind = excluded.kind, node = excluded.node, module = excluded.module,
password_hash = excluded.password_hash, minted_at = now()`,
u.Username, u.Kind, u.Node, u.Module, string(hash)); err != nil {
return "", fmt.Errorf("cannot record the bus user %s: %w", u.Username, err)
}
return password, nil
}
// BusUsers is every user the composed file should contain, by username.
//
// Returned as a map because the composer asks by username: the principals are derived from records
// elsewhere, and this is only what each one's password is. A principal with no row here has no
// password, and the composer refuses it rather than writing a user anybody is.
func (i *Inventory) BusUsers(ctx context.Context) (map[string]BusUser, error) {
rows, err := i.store.Pool().Query(ctx,
`select username, kind, node, module, password_hash from bus_user order by username`)
if err != nil {
return nil, err
}
defer rows.Close()
out := map[string]BusUser{}
for rows.Next() {
var u BusUser
if err := rows.Scan(&u.Username, &u.Kind, &u.Node, &u.Module, &u.PasswordHash); err != nil {
return nil, err
}
out[u.Username] = u
}
return out, rows.Err()
}
// BusUserHash is one user's hash, or false when the mesh has never minted one for it.
func (i *Inventory) BusUserHash(ctx context.Context, username string) (string, bool, error) {
var hash string
err := i.store.Pool().QueryRow(ctx,
`select password_hash from bus_user where username = $1`, username).Scan(&hash)
if errors.Is(err, pgx.ErrNoRows) {
return "", false, nil
}
return hash, err == nil, err
}
// ForgetBusUser removes one user, so the next composition does not contain it.
//
// **Removal is what makes revocation real here.** On a bus with a management call, deleting an
// account ends its connections; here the credential stops working when the file no longer names it,
// which is the next composition — so forgetting the row and composing are one act, and a caller
// that does the first without the second has revoked nothing.
func (i *Inventory) ForgetBusUser(ctx context.Context, username string) error {
_, err := i.store.Pool().Exec(ctx, `delete from bus_user where username = $1`, username)
return err
}
// ForgetBusUsersOf removes every user belonging to one node — its host's, and every module assigned
// to it. What a forgotten node leaves behind on the bus is otherwise a set of credentials for a
// machine the mesh no longer knows.
func (i *Inventory) ForgetBusUsersOf(ctx context.Context, node string) error {
if node == "" {
return errors.New("forgetting the bus users of no node would forget every user that has none")
}
_, err := i.store.Pool().Exec(ctx, `delete from bus_user where node = $1`, node)
return err
}
// SeedBusUser records a hash of a credential the mesh did not mint, so a composition contains it.
//
// **Genesis is the reason this exists.** The controller's own user is created before the controller
// runs — by the installer, at a well-known bootstrap password, the way the store's and the old bus's
// are (`postgres:bootstrap`, `guest:guest`). Nothing minted it, so nothing recorded a hash for it, and
// the controller's first composition would leave itself out of the very file it was writing: a bus
// nothing can connect to, produced by the thing connected to it.
//
// Idempotent, and it does not overwrite. A credential the mesh *did* mint is the one that counts, so
// once there is a row this does nothing — otherwise a restart would put the bootstrap password back
// over a rotated one.
func (i *Inventory) SeedBusUser(ctx context.Context, u BusUser, password string) error {
if u.Username == "" || u.Kind == "" || password == "" {
return errors.New("a bus user needs a username, a kind and the credential it is using")
}
hash, err := bcrypt.GenerateFromPassword([]byte(password), bcrypt.DefaultCost)
if err != nil {
return fmt.Errorf("cannot hash a bus password: %w", err)
}
_, err = i.store.Pool().Exec(ctx,
`insert into bus_user (username, kind, node, module, password_hash)
values ($1, $2, $3, $4, $5)
on conflict (username) do nothing`,
u.Username, u.Kind, u.Node, u.Module, string(hash))
return err
}
// A person who may call the mesh's tools (novox/hq design 25 §7).
//
// **Their authority is a list of tools and nothing else.** Not a module: they hold no seat, nothing is
// addressed to them, nothing is delivered to them, and they have no consumer to acknowledge. What
// they have is permission to ask.
// Person is somebody who may reach the mesh's tools.
type Person struct {
Name string
// Invokes are the tools they may call, each `<module>.<tool>`, or the single entry `*` for an
// administrator.
Invokes []string
}
// RecordPerson adds somebody, or changes what they may call.
//
// Replacing rather than merging: what a person may call is stated in full, so a change that meant to
// remove a tool does remove it. A list that could only grow is a permission nobody can take back.
func (i *Inventory) RecordPerson(ctx context.Context, p Person) error {
if p.Name == "" {
return errors.New("a person needs a name: it becomes their user on the bus")
}
if len(p.Invokes) == 0 {
return fmt.Errorf(
"%s may call nothing, so there is no reason for them to reach the mesh. Name the tools, "+
"or `*` for an administrator", p.Name)
}
_, err := i.store.Pool().Exec(ctx,
`insert into person (name, invokes) values ($1, $2)
on conflict (name) do update set invokes = excluded.invokes`,
p.Name, p.Invokes)
return err
}
// People is everybody who may reach the mesh's tools.
func (i *Inventory) People(ctx context.Context) ([]Person, error) {
rows, err := i.store.Pool().Query(ctx, `select name, invokes from person order by name`)
if err != nil {
return nil, err
}
defer rows.Close()
var out []Person
for rows.Next() {
var p Person
if err := rows.Scan(&p.Name, &p.Invokes); err != nil {
return nil, err
}
out = append(out, p)
}
return out, rows.Err()
}
// ForgetPerson removes somebody and the credential they were given.
//
// **Both, or neither is a revocation.** A person's row gone and their bus user left behind is a
// credential that still works and that nothing derives, which is the worst of both: it keeps working
// and nobody can explain why.
func (i *Inventory) ForgetPerson(ctx context.Context, name string) error {
if name == "" {
return errors.New("forgetting nobody would forget everybody")
}
if _, err := i.store.Pool().Exec(ctx, `delete from person where name = $1`, name); err != nil {
return err
}
return i.ForgetBusUser(ctx, "person."+name)
}
// PutBusMembership records a machine's membership for the new bus, sealed to it (design 28, 5.2).
// Replaces any earlier one: a machine has one membership per bus, and re-minting is re-telling.
func (i *Inventory) PutBusMembership(ctx context.Context, nodeName, sealed string) error {
node, err := i.NodeByName(ctx, nodeName)
if err != nil {
return err
}
_, err = i.store.Pool().Exec(ctx,
`insert into bus_membership (node, sealed) values ($1, $2)
on conflict (node) do update set sealed = excluded.sealed, since = now()`, node.ID, sealed)
return err
}
// BusMemberships is every machine's sealed membership for the new bus, by node name.
func (i *Inventory) BusMemberships(ctx context.Context) (map[string]string, error) {
rows, err := i.store.Pool().Query(ctx,
`select n.name, b.sealed from bus_membership b join node n on n.id = b.node`)
if err != nil {
return nil, err
}
defer rows.Close()
out := map[string]string{}
for rows.Next() {
var name, sealed string
if err := rows.Scan(&name, &sealed); err != nil {
return nil, err
}
out[name] = sealed
}
return out, rows.Err()
}
+160
View File
@@ -0,0 +1,160 @@
package inventory
import (
"context"
"testing"
"golang.org/x/crypto/bcrypt"
)
// The bus's users as records — against a real store, because what is being checked is that the
// column exists, the upsert behaves, and a plaintext is returned exactly once.
func aBusUser(module string) BusUser {
return BusUser{Username: "one." + module, Kind: BusModule, Node: "one", Module: module}
}
// The plaintext comes back once and the store keeps only a hash that verifies against it. **A
// credential recoverable from the mesh's store is one whose blast radius is the store's**, so what
// is asserted is that the password is not in there.
func TestABusPasswordIsReturnedOnceAndOnlyItsHashIsKept(t *testing.T) {
inv := ForTest(t)
ctx := context.Background()
password, err := inv.MintBusPassword(ctx, aBusUser("shop"))
if err != nil {
t.Fatal(err)
}
if password == "" {
t.Fatal("no password came back, so nothing can be sealed to the module")
}
hash, known, err := inv.BusUserHash(ctx, "one.shop")
if err != nil || !known {
t.Fatalf("the user was not recorded: %v %v", known, err)
}
if hash == password {
t.Fatal("the store holds the password itself")
}
if err := bcrypt.CompareHashAndPassword([]byte(hash), []byte(password)); err != nil {
t.Fatalf("the recorded hash does not verify the password it was made from: %v", err)
}
}
// Minting again replaces what was there rather than failing or adding a second row: that is a
// rotation, and the old credential stops working at the next composition.
func TestMintingAgainRotatesRatherThanAddsAUser(t *testing.T) {
inv := ForTest(t)
ctx := context.Background()
first, err := inv.MintBusPassword(ctx, aBusUser("shop"))
if err != nil {
t.Fatal(err)
}
second, err := inv.MintBusPassword(ctx, aBusUser("shop"))
if err != nil {
t.Fatal(err)
}
if first == second {
t.Fatal("minting twice produced the same password")
}
users, err := inv.BusUsers(ctx)
if err != nil {
t.Fatal(err)
}
if len(users) != 1 {
t.Fatalf("%d users after two mints for one name", len(users))
}
hash := users["one.shop"].PasswordHash
if err := bcrypt.CompareHashAndPassword([]byte(hash), []byte(second)); err != nil {
t.Fatal("the kept hash is not the newest password's")
}
if bcrypt.CompareHashAndPassword([]byte(hash), []byte(first)) == nil {
t.Fatal("the previous password still verifies, so a rotation revoked nothing")
}
}
// Forgetting a node takes every credential that belonged to it — its host's and every module
// assigned to it. What a forgotten node leaves behind otherwise is a working set of credentials for
// a machine the mesh no longer knows.
func TestForgettingANodeTakesItsBusUsersWithIt(t *testing.T) {
inv := ForTest(t)
ctx := context.Background()
for _, u := range []BusUser{
{Username: "node.one", Kind: BusNode, Node: "one"},
aBusUser("shop"),
{Username: "node.two", Kind: BusNode, Node: "two"},
{Username: "controller", Kind: BusController},
} {
if _, err := inv.MintBusPassword(ctx, u); err != nil {
t.Fatal(err)
}
}
if err := inv.ForgetBusUsersOf(ctx, "one"); err != nil {
t.Fatal(err)
}
users, err := inv.BusUsers(ctx)
if err != nil {
t.Fatal(err)
}
if _, still := users["node.one"]; still {
t.Fatal("a forgotten node's host credential still works")
}
if _, still := users["one.shop"]; still {
t.Fatal("a module on a forgotten node still has a credential")
}
// And nothing else went with it: the controller has no node, and another machine's user is
// another machine's.
for _, kept := range []string{"node.two", "controller"} {
if _, ok := users[kept]; !ok {
t.Fatalf("%s was removed with another node's users", kept)
}
}
}
// Forgetting the users of no node would forget every user that has none — the controller and every
// person — so it is refused rather than run.
func TestForgettingTheUsersOfNoNodeIsRefused(t *testing.T) {
inv := ForTest(t)
if err := inv.ForgetBusUsersOf(context.Background(), ""); err == nil {
t.Fatal("forgetting the bus users of no node was allowed")
}
}
// The controller's own user is created by the installer, so the mesh has to be able to record a
// credential it did not mint — or the first composition leaves the writer out of the file it writes.
func TestACredentialTheMeshDidNotMintIsRecordedOnceAndNotOverwritten(t *testing.T) {
inv := ForTest(t)
ctx := context.Background()
if err := inv.SeedBusUser(ctx, BusUser{Username: "controller", Kind: BusController},
"bootstrap"); err != nil {
t.Fatal(err)
}
hash, known, err := inv.BusUserHash(ctx, "controller")
if err != nil || !known {
t.Fatalf("the credential was not recorded: %v %v", known, err)
}
if err := bcrypt.CompareHashAndPassword([]byte(hash), []byte("bootstrap")); err != nil {
t.Fatalf("what was recorded does not verify the credential given: %v", err)
}
// Minted since, then seeded again — which is what a restart does. The rotation must stand, or
// every restart would put the bootstrap password back over it.
minted, err := inv.MintBusPassword(ctx, BusUser{Username: "controller", Kind: BusController})
if err != nil {
t.Fatal(err)
}
if err := inv.SeedBusUser(ctx, BusUser{Username: "controller", Kind: BusController},
"bootstrap"); err != nil {
t.Fatal(err)
}
hash, _, err = inv.BusUserHash(ctx, "controller")
if err != nil {
t.Fatal(err)
}
if err := bcrypt.CompareHashAndPassword([]byte(hash), []byte(minted)); err != nil {
t.Fatal("a restart put the bootstrap credential back over a rotated one")
}
}
+10 -2
View File
@@ -8,6 +8,7 @@ import (
"sort"
"strconv"
"strings"
"time"
"github.com/jackc/pgx/v5"
"github.com/novox/mesh-controller/internal/catalogue"
@@ -38,6 +39,9 @@ type Source struct {
BuiltFrom string
// Head is the newest commit the source is known to have.
Head string
// Seen is when the source was last looked at — by a build, by hand, or by the forge saying it
// moved. What a late report of an older move is judged against.
Seen time.Time
}
// Current reports whether what the mesh holds is what the source last had.
@@ -936,12 +940,13 @@ func (i *Inventory) Catalogued(ctx context.Context) ([]Entry, error) {
`select m.name, m.manifest,
coalesce(m.source, ''), m.source_path, m.source_seat, coalesce(m.ref, ''),
coalesce(m.built_from, ''), coalesce(m.source_head, ''),
coalesce(m.source_seen, to_timestamp(0)),
coalesce(array_agg(n.name order by n.name) filter (where n.name is not null), '{}')
from module m
left join assignment a on a.module = m.name
left join node n on n.id = a.node
group by m.name, m.manifest, m.source, m.source_path, m.source_seat, m.ref, m.built_from,
m.source_head
m.source_head, m.source_seen
order by m.name`)
if err != nil {
return nil, err
@@ -955,9 +960,12 @@ func (i *Inventory) Catalogued(ctx context.Context) ([]Entry, error) {
var source Source
var on []string
if err := rows.Scan(&name, &raw, &source.Repository, &source.Path, &source.Seat, &source.Ref,
&source.BuiltFrom, &source.Head, &on); err != nil {
&source.BuiltFrom, &source.Head, &source.Seen, &on); err != nil {
return nil, err
}
if source.Seen.Unix() == 0 {
source.Seen = time.Time{}
}
var m catalogue.Manifest
if err := json.Unmarshal(raw, &m); err != nil {
return nil, err
+99
View File
@@ -0,0 +1,99 @@
package inventory
import (
"context"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// A seat's holder is a row, changed as one act (novox/hq ADR 0131). What these pin is the shape of
// that row's life: it needs an assignment to point at, it is replaced rather than added to, and it
// goes when the assignment does — so a seat never points at something that is not running anywhere.
func twoBrokersOnTwoNodes(t *testing.T) (*Inventory, context.Context) {
t.Helper()
old := catalogue.Manifest{Module: "old-broker", Version: "1",
Provides: []catalogue.Offer{{Name: "mesh-bus", Scope: catalogue.ScopeMesh}},
Claims: []catalogue.Claim{{Name: "mesh-broker", Scope: catalogue.ScopeMesh}}}
new := catalogue.Manifest{Module: "new-broker", Version: "1",
Provides: []catalogue.Offer{{Name: "mesh-bus", Scope: catalogue.ScopeMesh}},
Claims: []catalogue.Claim{{Name: "mesh-broker", Scope: catalogue.ScopeMesh}}}
inv, ctx := aMeshWith(t, old, new)
// The holding references the seat's row, which `migrate` seeds on a real mesh.
if _, err := inv.SeedSeats(ctx, catalogue.DefaultSeats()); err != nil {
t.Fatal(err)
}
for _, n := range []string{"anchor", "laptop"} {
if _, err := inv.AddNode(ctx, n); err != nil {
t.Fatal(err)
}
}
if _, err := inv.Assign(ctx, "anchor", "old-broker"); err != nil {
t.Fatal(err)
}
if _, err := inv.Assign(ctx, "laptop", "new-broker"); err != nil {
t.Fatal(err)
}
return inv, ctx
}
func TestAHandoverIsOneRowReplacedNotOneAdded(t *testing.T) {
inv, ctx := twoBrokersOnTwoNodes(t)
if err := inv.HoldSeat(ctx, "mesh-broker", catalogue.ScopeMesh, "anchor", "old-broker"); err != nil {
t.Fatal(err)
}
if err := inv.HoldSeat(ctx, "mesh-broker", catalogue.ScopeMesh, "laptop", "new-broker"); err != nil {
t.Fatal(err)
}
held, err := inv.Holdings(ctx)
if err != nil {
t.Fatal(err)
}
if len(held) != 1 || held[0].Node != "laptop" || held[0].Module != "new-broker" || held[0].Scope != catalogue.ScopeMesh {
t.Fatalf("after a handover the seat is not held by exactly the new holder: %+v", held)
}
}
func TestASeatCannotBeHandedToSomethingNotAssigned(t *testing.T) {
inv, ctx := twoBrokersOnTwoNodes(t)
// new-broker is assigned to laptop, not anchor.
if err := inv.HoldSeat(ctx, "mesh-broker", catalogue.ScopeMesh, "anchor", "new-broker"); err == nil {
t.Fatal("a seat was handed to a module not assigned where it was named")
}
}
func TestUnassigningTheHolderTakesTheHoldingWithIt(t *testing.T) {
inv, ctx := twoBrokersOnTwoNodes(t)
if err := inv.HoldSeat(ctx, "mesh-broker", catalogue.ScopeMesh, "laptop", "new-broker"); err != nil {
t.Fatal(err)
}
if err := inv.Unassign(ctx, "laptop", "new-broker"); err != nil {
t.Fatal(err)
}
held, err := inv.Holdings(ctx)
if err != nil {
t.Fatal(err)
}
if len(held) != 0 {
t.Fatalf("the holding outlived the assignment it pointed at: %+v", held)
}
}
func TestAMachinesMembershipIsOneRowReplacedAndGoesWithTheMachine(t *testing.T) {
inv, ctx := twoBrokersOnTwoNodes(t)
if err := inv.PutBusMembership(ctx, "anchor", "first"); err != nil {
t.Fatal(err)
}
if err := inv.PutBusMembership(ctx, "anchor", "second"); err != nil {
t.Fatal(err)
}
got, err := inv.BusMemberships(ctx)
if err != nil {
t.Fatal(err)
}
if got["anchor"] != "second" || len(got) != 1 {
t.Fatalf("a re-told membership did not replace the first: %v", got)
}
}
@@ -0,0 +1,17 @@
-- The seats are data the control plane owns, not a slice compiled into it (novox/hq ADR 0122).
--
-- Until this, the closed set 0110 defines lived only as a Go slice, referenced by name everywhere,
-- so renaming a seat or adding one meant a controller rebuild and a mesh-wide, freeze-prone deploy.
-- The set is now a table: one row per seat, seeded from the binary's defaults the first time the
-- control plane comes up, and thereafter the live copy the control plane reads and an operator can
-- change. A rename becomes an update here rather than a release.
--
-- The name is the key for now, because claims and held records still reference a seat by name; the
-- move to a stable id that a rename does not touch is the next step (ADR 0122). `delivers` is empty
-- for a seat that answers for no provision, matching the compiled default.
create table seat (
name text primary key,
scope text not null,
delivers text not null default '',
decided text not null
);
@@ -0,0 +1,12 @@
-- A seat keeps its former names, so a rename breaks nothing (novox/hq ADR 0122).
--
-- Phase 1 made the seat set data, but a rename still broke every reference to the old name — a
-- manifest's claim, a held record, the git-seat lookup — because they name the seat and the name
-- had changed. This is the stable identity ADR 0122 asked for, realised the simple way: a seat's
-- canonical name changes, and its old name becomes an alias that resolves to it forever. Nothing
-- downstream has to change — a manifest goes on claiming the old name, the build machine goes on
-- validating it — and a rename is one operation: set the new name, remember the old.
create table seat_alias (
alias text primary key, -- a former name of a seat
seat text not null -- the seat's current canonical name it resolves to
);
@@ -0,0 +1,13 @@
-- A node has an operator account: the human login on it (novox/hq to-be 29).
--
-- The mesh modelled the machine but not the person on it — `jochens` on novox, `ace` on ace,
-- `jochen` on shanks and g14. That name decides who a file under a home is owned by and which
-- account `ssh <node>` logs in as; it was silently lost when the predecessor's per-node `user:`
-- was not carried over, and `ssh ace` failed to `ace` because nothing here said so.
--
-- Empty rather than null and defaulted, because "no operator account known yet" is a real state
-- (a freshly enrolled machine, a headless box). The home is stored too rather than always assumed
-- to be /home/<account>, because root's is /root and a machine may put a home elsewhere; empty
-- means "derive it" (/root for root, /home/<account> otherwise), so the common case needs no entry.
alter table node add column account text not null default '';
alter table node add column account_home text not null default '';
@@ -0,0 +1,36 @@
-- Every bus user's password hash, because the file has to be written again.
--
-- novox/hq design 25 §4, task 1.7. On the bus the mesh runs on today an account is created by a
-- management call: the mesh mints a password, hands it over, seals the plaintext to whoever will
-- use it, and keeps nothing. That works because the broker remembers.
--
-- The bus being built has no management call — its users are a file the controller composes, and
-- **the whole file is written every time any of it changes**. So the first person's access change
-- would silently blank every module's password. The hash has to outlive its own minting, which is
-- state the mesh did not need before and does now.
--
-- Keyed by username, because the username is exactly what the composed file needs and what a
-- principal derives from its own identity. Nothing else about the user is here: **permissions are
-- not stored.** They are derived from what each module declares, every time the file is written
-- (ADR 0043) — a stored copy would be a second account of a user's authority, able to disagree
-- with the first, and the disagreement would be invisible until somebody compared a file with a
-- manifest.
--
-- The hash and not the password. A file on a node's disk holds the hash, and so does this: a
-- credential recoverable from the mesh's store is one whose blast radius is the store's.
create table bus_user (
username text primary key,
-- kind and what it names, so a user whose subject is gone can be found and removed: a module
-- unassigned, a node forgotten, a token spent. Recorded rather than parsed back out of the
-- username, because a name is for the server and a parser over it would be a second grammar.
kind text not null,
node text not null default '',
module text not null default '',
password_hash text not null,
minted_at timestamptz not null default now()
);
-- Finding every user of one kind, and every user belonging to one node — which is what removing a
-- node, or composing after an assignment, asks.
create index bus_user_kind on bus_user (kind);
create index bus_user_node on bus_user (node) where node <> '';
@@ -0,0 +1,19 @@
-- A person who may call the mesh's tools from a workstation.
--
-- novox/hq design 25 §7. Everything else that reaches the bus is a machine or a module running on
-- one; this is the exception the mesh has always had informally — somebody at a terminal — and never
-- recorded. Until now "the operator" meant whoever held the keys, which is a role and not a record,
-- so nothing could say who may call what.
--
-- **The authority is a list of tools and nothing else.** A person is not a module: they hold no seat,
-- nothing is addressed to them, nothing is delivered to them, and they have no consumer to
-- acknowledge. What they have is permission to ask. That is why there is no scope column and no node
-- column — a person is not on a machine.
create table person (
name text primary key,
-- The tools this person may invoke, each `<module>.<tool>`, or the single entry `*` for an
-- administrator. Stored as given: the permission is derived from it at every composition, so a
-- normalised form here would be a second opinion about authority (novox/hq ADR 0043).
invokes text[] not null default '{}',
created timestamptz not null default now()
);
@@ -0,0 +1,24 @@
-- A seat's holder is a recorded fact, not a derivation (novox/hq ADR 0131, design 26).
--
-- Until this, "which assignment holds the seat" was derived: the module that is assigned and
-- claims the seat holds it, and a second eligible assignment was refused at resolution. That has no
-- way to hand a seat from one holder to the next without a moment where nothing holds it — and the
-- control plane finds its own bus through one of these seats, so that moment was an outage
-- (2026-09-27). Now the holder is one row here, changed by `seat <name> --to <node>/<module>` as one
-- act, and other assignments whose module could hold the seat are simply eligible and silent.
--
-- No row means what it always meant: the sole eligible assignment holds the seat, and two eligible
-- ones are refused. So a mesh that has never handed a seat over behaves exactly as before, and the
-- row appears the first time somebody does.
--
-- The seat is referenced by name because claims still are (0034); the rename cascades here so a
-- handed-over seat survives being renamed. The holder is the assignment itself, so unassigning it
-- takes the holding with it and the seat falls back to derivation rather than pointing at nothing.
create table seat_holding (
seat text primary key references seat(name) on update cascade on delete cascade,
scope text not null,
node uuid not null,
module text not null,
since timestamptz not null default now(),
foreign key (node, module) references assignment(node, module) on delete cascade
);
@@ -0,0 +1,12 @@
-- The bus seat's holder answers for the mesh's bus, not for a wire protocol (novox/hq ADR 0131).
--
-- The row said `amqp`, which is the protocol the old broker spoke, and so only that broker could hold
-- the seat that names the mesh's bus — while the module that will carry the bus could not. The seat
-- delivers `mesh-bus`; whichever module provides that may hold it, and today that is one module.
--
-- Safe under the current holder: the control plane composes its own bus address through the seat by
-- name, and the overview derives holders by name. Only registration and provision-to-seat resolution
-- read this column. So the row changes, the current holder keeps holding by derivation, the next one
-- can register its claim, and the handover (0039) moves the seat when both are running. What must
-- not happen in between is re-registering the current holder — registration would now refuse it.
update seat set delivers = 'mesh-bus' where name = 'mesh-broker' and delivers = 'amqp';
@@ -0,0 +1,11 @@
-- A machine already enrolled is moved to the new bus by being told its membership for it
-- (novox/hq design 28, task 5.2). Until this, a membership — bus address, fingerprint, password,
-- transport — existed only in the enrolment reply, and nothing could hand one to a machine that
-- had already joined. The row is the membership sealed to that machine, composed into its
-- declaration as a file it reads after applying; the plaintext exists once, at minting, and then
-- only on the machine. One per node: the mesh moves to one bus.
create table bus_membership (
node uuid primary key references node(id) on delete cascade,
sealed text not null,
since timestamptz not null default now()
);
+60 -2
View File
@@ -55,6 +55,29 @@ type Node struct {
// AdoptedSince is when it last became so; zero for a converged node.
Adopted bool
AdoptedSince time.Time
// Account is the operator's login on this machine — `jochens` on novox, `ace` on ace (novox/hq
// to-be 29). Empty when none is known yet. AccountHome is where that account's home is; empty
// means derive it (/root for root, /home/<account> otherwise), so the common case needs no
// entry. What decides who a file under a home is owned by, and which account `ssh <node>` uses.
Account string
AccountHome string
}
// Home is the account's home directory, derived when not stored: /root for root, /home/<account>
// otherwise. Empty only when there is no account at all.
func (n Node) Home() string {
if n.AccountHome != "" {
return n.AccountHome
}
switch n.Account {
case "":
return ""
case "root":
return "/root"
default:
return "/home/" + n.Account
}
}
// Silent is how long since this node was last heard from, and whether it ever was.
@@ -112,12 +135,13 @@ func (i *Inventory) AddNodeAs(ctx context.Context, name string, adopted bool) (N
// nodeColumns and scanNode are the one reading of a node row, so every way of finding a node
// says whether it is adopted.
const nodeColumns = `id, name, created, last_seen, adopted, adopted_since`
const nodeColumns = `id, name, created, last_seen, adopted, adopted_since, account, account_home`
func scanNode(row pgx.Row) (Node, error) {
var n Node
var seen, since *time.Time
if err := row.Scan(&n.ID, &n.Name, &n.Created, &seen, &n.Adopted, &since); err != nil {
if err := row.Scan(&n.ID, &n.Name, &n.Created, &seen, &n.Adopted, &since,
&n.Account, &n.AccountHome); err != nil {
return Node{}, err
}
if seen != nil {
@@ -129,6 +153,21 @@ func scanNode(row pgx.Row) (Node, error) {
return n, nil
}
// SetAccount records the operator account on a node — its human login — and optionally where that
// account's home is (novox/hq to-be 29). An empty home means the mesh derives it. Clearing the
// account (empty name) is allowed: a machine may stop having a known operator.
func (i *Inventory) SetAccount(ctx context.Context, node, account, home string) error {
tag, err := i.store.Pool().Exec(ctx,
`update node set account = $1, account_home = $2 where name = $3`, account, home, node)
if err != nil {
return err
}
if tag.RowsAffected() == 0 {
return fmt.Errorf("%w: %s", ErrNoSuchNode, node)
}
return nil
}
// Nodes are every node record, oldest first.
func (i *Inventory) Nodes(ctx context.Context) ([]Node, error) {
rows, err := i.store.Pool().Query(ctx,
@@ -782,6 +821,25 @@ func (i *Inventory) RecordSent(ctx context.Context, node, digest string) error {
return err
}
// Outstanding is the digest of the declaration a machine was last sent, by its name, and empty
// for one that has never been sent anything.
//
// **By name rather than by id**, because the caller is the serving loop and what a node puts in a
// report is its name. Asked of one machine rather than read from Waiting's sweep, because it is
// asked per message: a report names the declaration it is about, and a report about one the mesh
// has already moved past is not acted on (design 25 §3).
func (i *Inventory) Outstanding(ctx context.Context, name string) (string, error) {
var sent string
err := i.store.Pool().QueryRow(ctx,
`select coalesce(sent, '') from node where name = $1`, name).Scan(&sent)
if errors.Is(err, pgx.ErrNoRows) {
// Not an error worth carrying up: a report from a machine the mesh has no record of has
// nothing to be stale against, and whatever is wrong with it is the listener's to say.
return "", nil
}
return sent, err
}
// Waiting is every machine whose declaration has changed since it was last sent one.
//
// The caller works out what each machine should be now, because only it can — resolution is the
+120
View File
@@ -0,0 +1,120 @@
package inventory
import (
"context"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/broker"
)
// Somebody who may call the mesh's tools, and what the bus makes of them.
// A person's authority is a list of tools, and it becomes exactly that on the bus — nothing on
// control, nothing on nodes, nothing they could publish as a module.
func TestAPersonMayCallToolsAndNothingElse(t *testing.T) {
inv := ForTest(t)
ctx := context.Background()
if err := inv.RecordPerson(ctx, Person{Name: "ada",
Invokes: []string{"mesh-catalog.catalog_tools"}}); err != nil {
t.Fatal(err)
}
records, err := inv.BusRecords(ctx)
if err != nil {
t.Fatal(err)
}
if got := records.People["ada"]; len(got) != 1 || got[0] != "mesh-catalog.catalog_tools" {
t.Fatalf("ada may call %v", got)
}
users, err := broker.Users(records)
if err != nil {
t.Fatal(err)
}
var found bool
for _, u := range users {
if u.Username() != "person.ada" {
continue
}
found = true
perms, err := broker.PermissionsFor(u)
if err != nil {
t.Fatal(err)
}
if len(perms.Publish) != 1 || perms.Publish[0] != "mesh.mod.mesh-catalog.tool.catalog_tools" {
t.Errorf("ada may publish %v, which should be the one tool and nothing else", perms.Publish)
}
for _, s := range perms.Publish {
if strings.HasPrefix(s, "mesh.control") || strings.HasPrefix(s, "mesh.node") ||
strings.Contains(s, ".event.") {
t.Errorf("a person may publish %s — an event would let them claim a module said "+
"something, and control is not theirs", s)
}
}
if perms.AllowResponses {
t.Error("a person may answer a request, which is impersonating a module on a bus where " +
"anyone may serve a tool")
}
}
if !found {
t.Fatal("no bus user was derived for a recorded person")
}
}
// Stating what somebody may call replaces what was there. A list that could only grow is a permission
// nobody can take back.
func TestChangingWhatAPersonMayCallRemovesWhatIsNotNamed(t *testing.T) {
inv := ForTest(t)
ctx := context.Background()
if err := inv.RecordPerson(ctx, Person{Name: "ada", Invokes: []string{"a.one", "b.two"}}); err != nil {
t.Fatal(err)
}
if err := inv.RecordPerson(ctx, Person{Name: "ada", Invokes: []string{"a.one"}}); err != nil {
t.Fatal(err)
}
people, err := inv.People(ctx)
if err != nil {
t.Fatal(err)
}
if len(people) != 1 || len(people[0].Invokes) != 1 || people[0].Invokes[0] != "a.one" {
t.Fatalf("ada may call %v; the removed tool is still there", people)
}
}
// Somebody who may call nothing is refused: there is no reason for them to reach the mesh, and an
// empty list is more likely a mistake than an intention.
func TestSomebodyWhoMayCallNothingIsRefused(t *testing.T) {
inv := ForTest(t)
if err := inv.RecordPerson(context.Background(), Person{Name: "ada"}); err == nil {
t.Fatal("somebody who may call nothing was recorded")
}
}
// Forgetting somebody takes their credential with them. **Both, or it is not a revocation**: a
// person's row gone and their bus user left behind is a credential that still works and that nothing
// derives.
func TestForgettingAPersonTakesTheirCredential(t *testing.T) {
inv := ForTest(t)
ctx := context.Background()
if err := inv.RecordPerson(ctx, Person{Name: "ada", Invokes: []string{"a.one"}}); err != nil {
t.Fatal(err)
}
if _, err := inv.MintBusPassword(ctx, BusUser{Username: "person.ada", Kind: BusPerson}); err != nil {
t.Fatal(err)
}
if err := inv.ForgetPerson(ctx, "ada"); err != nil {
t.Fatal(err)
}
if _, known, err := inv.BusUserHash(ctx, "person.ada"); err != nil || known {
t.Fatalf("a forgotten person's credential still works: %v %v", known, err)
}
records, err := inv.BusRecords(ctx)
if err != nil {
t.Fatal(err)
}
if _, still := records.People["ada"]; still {
t.Fatal("a forgotten person is still composed into the bus")
}
}
+149
View File
@@ -0,0 +1,149 @@
package inventory
import (
"context"
"fmt"
"github.com/novox/mesh-controller/internal/catalogue"
)
// The seats the mesh has, as data (novox/hq ADR 0122).
//
// The set the control plane reads is a table here, not a slice compiled into it. It is seeded from
// the binary's defaults the first time the mesh comes up (SeedSeats), and thereafter it is the live
// copy: a rename or an added seat is a write here, and the control plane loads it at startup rather
// than being rebuilt for it.
// Seats is every seat the mesh defines, read from the store.
func (i *Inventory) Seats(ctx context.Context) ([]catalogue.Seat, error) {
rows, err := i.store.Pool().Query(ctx,
`select name, scope, delivers, decided from seat order by name`)
if err != nil {
return nil, err
}
defer rows.Close()
var seats []catalogue.Seat
for rows.Next() {
var s catalogue.Seat
if err := rows.Scan(&s.Name, &s.Scope, &s.Delivers, &s.Decision); err != nil {
return nil, err
}
seats = append(seats, s)
}
return seats, rows.Err()
}
// SeedSeats writes the mesh's default set into the table where it is not already present.
//
// **Idempotent, and never overwriting.** Run every time the control plane migrates, it fills an
// empty table on first boot and adds a seat a new release ships — but it leaves a row already there
// exactly as it is, so an operator's rename in the table is not undone by the next deploy putting
// the old name back. What a release removes from the defaults is not deleted here either; retiring a
// seat is its own decision, not a silent consequence of it dropping out of the binary.
func (i *Inventory) SeedSeats(ctx context.Context, defaults []catalogue.Seat) (int, error) {
var added int
for _, s := range defaults {
tag, err := i.store.Pool().Exec(ctx,
`insert into seat (name, scope, delivers, decided) values ($1, $2, $3, $4)
on conflict (name) do nothing`,
s.Name, s.Scope, s.Delivers, s.Decision)
if err != nil {
return added, err
}
added += int(tag.RowsAffected())
}
return added, nil
}
// Aliases is every former seat name and the seat it now resolves to (novox/hq ADR 0122).
func (i *Inventory) Aliases(ctx context.Context) (map[string]string, error) {
rows, err := i.store.Pool().Query(ctx, `select alias, seat from seat_alias`)
if err != nil {
return nil, err
}
defer rows.Close()
aliases := map[string]string{}
for rows.Next() {
var alias, seat string
if err := rows.Scan(&alias, &seat); err != nil {
return nil, err
}
aliases[alias] = seat
}
return aliases, rows.Err()
}
// RenameSeat gives a seat a new name and keeps the old one as an alias (novox/hq ADR 0122).
//
// **This is the whole of a rename.** The seat's canonical name becomes `to`; `from` is remembered as
// an alias so every reference to it — a manifest's claim, a held record, the build machine's
// embedded set — goes on resolving to the same seat, unchanged. Nothing is rebuilt and nothing
// freezes. Any alias that pointed to `from` is repointed to `to`, so a chain of renames does not
// leave an older name resolving to a name that no longer exists.
func (i *Inventory) RenameSeat(ctx context.Context, from, to string) error {
if from == to {
return fmt.Errorf("a seat is renamed to a different name; %q is already its name", to)
}
tag, err := i.store.Pool().Exec(ctx, `update seat set name = $1 where name = $2`, to, from)
if err != nil {
return err
}
if tag.RowsAffected() == 0 {
return fmt.Errorf("no seat named %q to rename", from)
}
// The old name resolves to the new one; and any name that resolved to the old one now resolves
// to the new one, so no alias is left pointing at a name that is gone.
if _, err := i.store.Pool().Exec(ctx,
`insert into seat_alias (alias, seat) values ($1, $2)
on conflict (alias) do update set seat = excluded.seat`, from, to); err != nil {
return err
}
if _, err := i.store.Pool().Exec(ctx,
`update seat_alias set seat = $1 where seat = $2`, to, from); err != nil {
return err
}
return nil
}
// HoldSeat records that one assignment holds a seat, replacing whoever held it — as one write, so
// the seat is never without a holder in between (novox/hq ADR 0131, design 28 task 5.3). The
// assignment must exist; the store refuses otherwise, and that refusal is the right one: a seat
// cannot be handed to something that is not running anywhere.
func (i *Inventory) HoldSeat(ctx context.Context, seat, scope, nodeName, module string) error {
node, err := i.NodeByName(ctx, nodeName)
if err != nil {
return err
}
_, err = i.store.Pool().Exec(ctx,
`insert into seat_holding (seat, scope, node, module) values ($1, $2, $3, $4)
on conflict (seat) do update set scope = excluded.scope, node = excluded.node,
module = excluded.module, since = now()`,
seat, scope, node.ID, module)
if err != nil {
return fmt.Errorf("recording %s on %s as the holder of %s: %w", module, nodeName, seat, err)
}
return nil
}
// Holdings is every seat whose holder is on record, as the resolver reads it. A seat with no row here
// is held by derivation, exactly as before the table existed.
func (i *Inventory) Holdings(ctx context.Context) ([]catalogue.Held, error) {
rows, err := i.store.Pool().Query(ctx,
`select h.seat, h.scope, n.name, h.module, coalesce(n.site, '')
from seat_holding h join node n on n.id = h.node order by h.seat`)
if err != nil {
return nil, err
}
defer rows.Close()
var out []catalogue.Held
for rows.Next() {
var h catalogue.Held
if err := rows.Scan(&h.Claim, &h.Scope, &h.Node, &h.Module, &h.Site); err != nil {
return nil, err
}
out = append(out, h)
}
return out, rows.Err()
}

Some files were not shown because too many files have changed in this diff Show More