Compare commits

..
Author SHA1 Message Date
jochen ac9c2d57be The node's runtime reads the consumers of the modules it carries (hq ADR 0198)
A module's long-running code is a bundle the runtime launches, and the runtime is its bus: it binds
the module's own durable consumer — EVENTS, <node>_<module>, still the controller's to make from the
module's principal — and acknowledges what the module's code took. So the runtime principal is
granted, for each carried module that consumes, exactly what that module's own principal has for
its consumer: its info, its next message, its ack subject. Nothing is pushed to it; it pulls. ADR
0175's "consumes nothing" no longer holds. Memberships need nothing new: the consumer's name is
derived, as the module's own runtime derived it.
2026-10-03 22:30:55 +02:00
mesh-admin 473376259b Merge pull request 'The route proxy tells a backend the request was HTTPS, and for which name' (#245) from fix/the-route-proxy-says-the-request-was-https into main 2026-10-03 20:23:02 +00:00
jochen e1293fb0ad The route proxy tells a backend the request was HTTPS, and for which name
NewSingleHostReverseProxy sets only X-Forwarded-For, so a backend that writes its own addresses saw
the plain hop from the proxy: Gitea's Go import tag named an http clone URL and go get refused the
SDK's module path. The proxy now sets X-Forwarded-Proto, -Host and -For from the request it received,
and keeps the Host header as it was.
2026-10-03 22:22:51 +02:00
mesh-admin 82481099b7 Merge pull request 'The controller announces as the tool runtimes do, and answers $SRV.STATS (hq ADR 0197)' (#242) from fix/0197-the-controller-announces-as-the-runtimes-do into main 2026-10-03 20:18:48 +00:00
jochen b127f005c3 The controller announces as the tool runtimes do, and answers $SRV.STATS (hq ADR 0197)
Endpoints named <seat>__<verb> with the metadata the console identifies them by (kind, module,
tool, seat, scope); $SRV.STATS answered with its identity and endpoints, nothing counted. Grants:
STATS beside PING and INFO, and the tool runtime may answer under its own name, since it announces
everything it carries as one service — the bus lets it answer each request once.
2026-10-03 22:18:27 +02:00
jochen 58b4fcb8c8 A bundle stands on the toolchain it is compiled in (hq issue 211)
A manifest names its toolchain by language, not in build.on, so the planner did not know a bundle
depends on the module that publishes its toolchain and built the two in one tier: the bundle
against the old toolchain, recorded as built from the new commit. The edge is read from the
manifest, so it holds before any build recorded it, and a toolchain that moves rebuilds every
bundle compiled in it.
2026-10-03 22:18:20 +02:00
mesh-admin 796f6410c1 Merge pull request 'Discovery from what answers: grants, the controller announces its seat, JSON lists (hq ADR 0195, 0197)' (#241) from feat/node-and-module-lists-as-json into main 2026-10-03 20:11:29 +00:00
jochen e67c58cd98 Every serving principal may answer the services discovery for what it serves; the controller announces its seat (hq ADR 0197)
Grants: a principal that serves tools subscribes $SRV.PING/$SRV.INFO and those questions under
each name it serves — its own and no other's; the tool runtime and people may ask. The controller
answers discovery for the mesh-controller seat in NATS's services format, one endpoint per verb it
serves, with the seat's description and schema. module list --json says which modules declare tools,
so the console expects an announcement only from those.
2026-10-03 22:11:00 +02:00
jochen 85873b19e1 node list and module list answer --json, and the nodes and modules verbs use it (hq ADR 0195)
The console's discovery reads the machines and the modules; parsing a printed column breaks when it
is reworded. Both now answer JSON on --json, as status and seats do, and the seat verbs ask for it.
2026-10-03 21:55:35 +02:00
mesh-admin 2ebbb79937 Merge pull request 'A runtime compiled to a binary runs itself (hq ADR 0193)' (#240) from feat/0193-a-runtime-compiled-to-a-binary into main 2026-10-03 19:24:24 +00:00
jochen 9204190445 A runtime compiled to a binary runs itself (hq ADR 0193)
A compiled bundle records the binary it is (BinaryOf, shared by the builder and the composer), and
the node's runtime, when it is one, is run as ./<binary> from its own unpacked bundle rather than by
an interpreter and an entrypoint.
2026-10-03 21:24:12 +02:00
jschoubben 06ea2168d8 Merge pull request 'The roster is the machines: each node's internal domain covers its routes (hq ADR 0191)' (#238) from fix/the-mesh-publishes-the-names-it-composed into main 2026-10-03 19:12:23 +00:00
mesh-admin 2b149dd43e Merge pull request 'Beside every TypeScript entrypoint the builder writes an executable launcher; the runtime is told it (hq ADR 0193)' (#239) from feat/0193-a-launcher-beside-every-typescript-entrypoint into main 2026-10-03 19:08:22 +00:00
jochen 17dba2a34c Beside every TypeScript entrypoint the builder writes an executable launcher; the runtime is told it (hq ADR 0193)
The runtime knows no language: the build makes each served entrypoint executable. For a TypeScript
bundle that is <entry>.serve.mjs, which imports the entrypoint and serves what it registered over
MCP on stdio through the bundle's own SDK. The build records its launchers on the bundle, and the
composer names the launcher where a build wrote one and the entrypoint where it did not, so bundles
built before this keep serving until they are rebuilt.
2026-10-03 21:07:07 +02:00
jschoubben 11e4bc0ba1 The roster is the machines: each node's internal domain covers its routes (hq ADR 0191)
The roster published routed names — public ones first, then (in this PR's first take) internal ones
told apart by suffix. Neither is needed: a node has one internal domain and every route on it is a
name under it, answered by the resolver's per-node wildcard; a node's public domains are public
DNS's. routeNamesInTheMesh and NamesServed are removed, and a test pins .Names to the machines.
2026-10-03 16:10:16 +02:00
jschoubben e56f3aa1cb The roster publishes a route's internal name, never its public one (hq ADR 0191)
NamesServed read a route's public `name` and plan.go then filtered by suffix — telling the mesh's
names from public ones by their spelling, when the mesh composed both itself. It now publishes the
`internal-name` it composed under the serving node (ADR 0151); the suffix filter is gone.
2026-10-03 15:39:05 +02:00
mesh-admin f966693583 Merge pull request 'A file a tools bundle's words name restarts the runtime when it changes (hq ADR 0192)' (#237) from feat/0192-a-named-file-restarts-the-runtime into main 2026-10-03 13:33:57 +00:00
jochen 5acc763992 A file a tools bundle's words name restarts the runtime when it changes (hq ADR 0192)
The tool containers were restarted when their configuration file changed; the runtime now is
too, for every file a module's words name exactly — configuration and own secret alike.
2026-10-03 15:33:44 +02:00
mesh-admin 4f009fff83 Merge pull request 'A tools bundle is given its words, composed per machine; what they name is the account's to read (hq ADR 0192)' (#236) from feat/0192-a-bundles-env into main 2026-10-03 13:32:38 +00:00
jochen f27e31954f A tools bundle is given its words, composed per machine; what they name is the account's to read (hq ADR 0192)
build.artifacts[].env on a bundle: words and values written with ${dir:…} and ${port:…} only,
refused when a value carries any other reference (a secret's content, a binding) or names a word
the runtime sets for itself, and on any artifact that is not a bundle. Resolved per machine like a
container's environment and handed to the runtime as MESH_TOOL_ENV, module by module, in the unit
so a change restarts it. Every file and directory of the module a word names, or that holds one, is
owned by the account the runtime runs as where it says no owner, since a tool reads as that account.
2026-10-03 15:32:03 +02:00
jschoubben 62650cd48c Merge pull request 'The mesh answers only its own names privately; a public name resolves publicly (hq ADR 0191)' (#235) from fix/the-mesh-resolves-only-its-own-names into main 2026-10-03 13:16:14 +00:00
jschoubben 408f6dbad9 The mesh answers only its own names privately; a public name resolves publicly (hq ADR 0191)
Every routed public name was published into each machine's hosts region at its serving node's
private address. ace's resolver also answers its LAN, so a phone there got the control-node's
tunnel address for the mail server and could not connect. Routes have internal names under the
serving node (ADR 0151), so only names under the mesh suffix are published now.
2026-10-03 15:14:01 +02:00
mesh-admin eae0577567 Merge pull request 'Issues 203 and 206: an assignment issues its credential; the controller owns a worker's shape; the build seat's holder follows the controller' (#233) from fix/issues-203-206 into main 2026-10-03 09:49:54 +00:00
mesh-admin de26918c52 Merge pull request 'A declaration is numbered when it is composed, and a send is recorded even by a sender being replaced (hq issue 204)' (#232) from fix/issue-204 into main 2026-10-03 09:44:42 +00:00
mesh-admin e01d18e548 Merge pull request 'An idle build machine's empty fetch is asked again, not read as the end (hq ADR 0190)' (#234) from fix/an-empty-fetch-is-not-the-end into main 2026-10-03 09:41:52 +00:00
jochen 59166b1031 An idle build machine's empty fetch is asked again, not read as the end (hq ADR 0190)
A fetch on a context without a deadline waits the client's own while and reports the deadline
passed — the client's, not ours — and the loop read it as "stop": every idle build agent exited
clean every half minute and was restarted by its supervisor, a crash loop with nothing in the log
to say why. Only our own context ending ends the machine; an empty fetch, however it is reported,
is asked again.
2026-10-03 11:41:23 +02:00
jochen c294949f2a A worker of the wrong type on a history-keeping stream is re-made to deliver from now on, never from the start (hq issue 207)
Left for a hand, the hand re-made it with the server's default — everything the stream holds — and
on 2026-10-03 that replayed every build ask since 1 October into the catalogue. Re-made with
deliver-new instead: nothing acknowledged comes back; what was in flight is said and asked again.
2026-10-03 11:07:29 +02:00
jochen 28853a251b The build seat's holder follows the controller that defines its worker (hq issue 206)
A plan is ordered by artifacts and says nothing about what must be running before what (ADR 0162);
on 2026-10-03 that put the build machine in tier 0 and the controller in tier 1, and the new build
machine could not bind the worker the old controller had defined. One running order enters the
graph, named as its own edge: a module claiming the build seat follows the control plane, and the
built-by edge from the control plane to that holder yields to it — the controller is built by
whichever build machine is running, as the runtime image always was. The edge orders a plan and
never widens it, like built-by.
2026-10-03 04:04:41 +02:00
jochen 76a8b8df9e The controller owns a worker's shape, type included: one of the wrong type is re-made on a work queue (hq issue 206)
A holder built for a pull worker cannot bind a push one — `cannot pull subscribe to push based
consumer` — and on 2026-10-03 the build machine rolled before the controller that would have
redefined its worker, restarted on that for an hour, and nothing could build the controller that
would have ended it. The server cannot change a consumer's type in place, so the assertion re-makes
one of the wrong type: on a work queue nothing is lost, because what was acknowledged is gone from the
stream and what was not is delivered again from the start. On a stream that keeps its history it is
said and left, since a re-made consumer replays what this one acknowledged (issue 156), and that is a
person's call. Proven against a real bus: a push worker with one ask acknowledged and two pending is
re-made as pull, a pull subscription binds, and takes exactly the two.
2026-10-03 04:04:41 +02:00
jochen f86f6a74f0 A declaration is numbered when it is composed, and a send is recorded even by a sender being replaced (hq issue 204)
On 2026-10-02 a runtime assigned and applied on two machines was undone two seconds later by a
declaration that had the assignments of a minute earlier. Every path composes from the records at
compose time and holds the machines it sends — but the number went on at SEND time, after
composing, so a declaration composed before an assignment changed and sent after a newer one
carried the higher number, and the host, which rightly refuses a lower number, took the older
content as the mesh's newest word. The record of that send was never written either: it is written
after the declaration is away, on the sender's context, and the controller sending it was being
replaced in that very second — status read "applied, current" over a machine just told otherwise.

Now the number is taken before the composition reads anything, in every path, so what was composed
earlier is numbered lower however late it goes out and the host's refusal does what it is for; and
what was sent is written down on a context that outlives the sender, bounded, so a dying controller
still records what it told a machine. The `declare` command — a declaration a person sends by hand —
records its send too. Proven: compositions in one order and sends in the other keep the numbers in
composition order; a send is recorded after the sender's context is cancelled.
2026-10-03 04:02:36 +02:00
jochen a3e8a4185b An assignment issues its bus credential, a push refuses one nobody issued, and what reads a secret restarts on it (hq issue 203)
`assign` recorded a module and `push` sealed a random own secret where its bus credential belongs;
the process crash-looped until a person ran `module issue` and pushed again, and the only warning was
one line in a list printed on every push. Now assigning a module that declares a broker secret issues
the credential in the same act — kept when one exists, so re-assigning rotates nothing — and when the
bus cannot be reached from here the assignment says which verb to run. A push never seals a
placeholder in a credential's place: a module whose bus user is unminted is refused by name, with the
verb. The control plane's own user is the installer's, seeded at genesis, which the test now says.

And what reads one of a module's own secrets is restarted when it changes — composed for a container
or daemon that names the secret's path in its volumes, environment or env-files, so a manifest need
not say it: the build machine ran on an hour-old credential because its manifest restarted it on its
environment file alone (issue 206). A scheduled or run-once process is left alone; it reads afresh.
2026-10-03 04:01:17 +02:00
mesh-admin 78de54381b Merge pull request 'A seat's work is shared by its holders: node-build-agent, pulled one ask at a time (hq ADR 0190)' (#228) from feat/a-seats-work-is-shared-by-its-holders into main 2026-10-03 00:53:42 +00:00
jochen ff5ef0ab60 The controller asks the build role that has a holder, and hears both roles' outcomes (hq ADR 0190, the handover)
A controller that asked node-build-agent from its first run would queue every build where nothing
pulls, and the build that registers build-agent — the first holder — would be among them. So the
role is chosen at ask time from the catalogue: the current role when any assigned module claims it,
the retired one while only the builder does, the current one when neither. Outcomes are followed on
both seats, the controller may publish to both, and a build's log is read under whichever role did
it; a machine on the retired role is proven on the bus to take that role's asks. The switch order
is written where the role is named, and the retired half is marked for removal with the seat row.
2026-10-03 02:51:03 +02:00
jochen a5d6a1187c A build machine serves the seat its credential claims (hq ADR 0190, the handover)
After the build role moved to node-build-agent, nothing would hold it until build-agent is
registered — and registering build-agent needs a build outcome that only the running builder
could produce, bound as it was to the old seat by name. One binary, two roles: the seat a machine
serves is the first its credential claims, as the mesh writes the claims beside the credential it
issues (ADR 0159); the old builder keeps draining mesh-build-machine, a build-agent takes
node-build-agent, and what each says about a build goes out as that seat's events, so an outcome
is heard where the asker of that seat listens. A credential naming no claim serves the current role.
2026-10-03 02:47:49 +02:00
mesh-admin 06d0a4bfe8 Merge pull request 'A changed jail filter restarts fail2ban' (#231) from jschoubben/jail-filter-restart into main 2026-10-02 21:28:32 +00:00
jschoubben d293a0deaf A changed jail filter restarts fail2ban
fail2ban restarts when the composed jail file changes, and each filter is
a file of its own, so a module that changed only its failregex left the
running jail on the old pattern. The jail file now names each filter's
digest.
2026-10-02 23:28:25 +02:00
mesh-admin 11a44e1ec4 Merge pull request 'A resolved manifest keeps a built reference in the store's own form, never the address it was reached by (hq ADR 0155)' (#230) from fix/a-resolved-reference-is-kept-not-routed into main 2026-10-02 21:14:06 +00:00
jochen 28c7f05b39 A resolved manifest keeps a built reference in the store's own form, never the address it was reached by (hq ADR 0155)
The first bundle resolved on the mesh carried the store's host in bundles[0].source, and registration
refused node-tools as naming an installation — rightly. The build record already keeps the
store-relative form; the resolved manifest now keeps the same for bundles and archive resources,
and composition routes it through the store a machine reaches, as it already did for a kept one.
2026-10-02 23:13:22 +02:00
mesh-admin 93a0c6202e Merge pull request 'The bus is never public: the broker port is no longer a foundation opening (hq ADR 0169)' (#229) from jschoubben/the-bus-is-never-public into main 2026-10-02 20:52:28 +00:00
jschoubben 7d82751862 The bus is never public: the broker port is no longer a foundation opening
The controller widened the bus's from-mesh port to from-anywhere on the
broker's host so a machine could enrol before it had a tunnel. ADR 0169
has machines join through the tunnel and decides the bus is never public;
every live bus connection already comes from the mesh.
2026-10-02 22:52:22 +02:00
jochen 905f3363c9 Two machines holding the build role share one queue, and neither is handed an ask while busy (hq ADR 0190)
Against a real bus: three asks, two machines; each takes one, the third waits until one is free
and then goes to that one; a machine that stops leaves nothing taken twice. The redelivery of an
ask a dead machine held is the ack wait's, proven by the hand-back test beside this one.

And the order a live mesh switches over in, written where the role is named: queued builds first,
then this controller, then build-agent assigned where machines build, then the builder and the old
seat's stream forgotten.
2026-10-02 22:35:39 +02:00
jochen 9f9d9b3b25 A seat's holders pull one ask at a time from one shared worker (hq ADR 0190, issue 186)
The worker a holder bound was a push consumer in a queue group with one ask in flight: right for
one holder, and with two it would still be a queue of one — the server hands a pushed ask to
whichever subscriber it picks, busy or not, and the in-flight cap is per consumer, not per holder.
Now the worker is pulled: every machine holding the seat binds the same durable and fetches one
ask when it has finished the last, so an idle machine is the one that takes the next, the asks in
flight are bounded by the holders working, and nothing is delivered that nobody asked for — which
is also what ended the race issue 186 describes. A holder's grants trade the delivery subject for
MSG.NEXT on the worker; the ack grant and the heartbeat that keeps a long build alive stay.

Proven against a real bus: the build round trip, a backlog taken by a machine that arrives later,
and work handed back by one machine coming round again.
2026-10-02 22:34:44 +02:00
jochen bde4b61b3b The build role is the node-scoped seat node-build-agent, and its work is shared by every holder (hq ADR 0190)
One build machine built everything, in a queue of one, because the seat was mesh-scoped and a
mesh seat has one holder. ADR 0190 makes building a node role: node-build-agent, held on every
machine that builds, with the work asked of the role and taken by whichever holder is idle. The
work subject of a node-scoped seat carries no node — that token is for a seat's tools, asked of
one machine (design 33 §4) — so holders on several machines read one queue; a test now says so.

The retired mesh-build-machine row stays while the builder module's registered manifest claims
it: a claim to a seat the mesh no longer defines is refused, and the machine holding it would be
unresolvable until build-agent replaces it. Removed once no manifest claims it.

The installer's genesis template (in the host's repository) still grants the controller the old
seat's subjects; its test here says so until that template names node-build-agent.
2026-10-02 22:31:55 +02:00
mesh-admin d07018f3c5 Merge pull request 'WP3: a TypeScript bundle carries what it runs with, the runtime's credential belongs to its account, and the gate refuses spreading not standing (hq to-be 38)' (#226) from feat/wp3-node-tools into main 2026-10-02 19:57:21 +00:00
jochen 729a5f9e6c The gate refuses the old tool-container pattern spreading, not a rebuild of what already stands (hq to-be 38 WP2.4, amended by WP3)
Once the runtime module is registered, a module serving tools from a container built on the
runtime's image is refused at registration — as written, including every rebuild of the thirty-odd
modules already in that shape, from the day the runtime arrives until WP4 onward moves each one.
That would stop the catalogue's whole pipeline to make a point the record already makes. Now a
module already registered in that shape — judged from the manifest the catalogue holds and what
its newest build stood on, the same two things a new registration is judged by — is rebuilt as
before; a module new to the catalogue in that shape, or one that had moved to a bundle and comes
back, is refused naming the record.
2026-10-02 21:44:44 +02:00
jochen 773b561f5e The runtime's credential belongs to the account it runs as (hq to-be 38 WP3)
The runtime's process is composed `user: <account>` where the node has one, and its broker file was
root's at 0600: a credential the process could not read. Composed in the declaration rather than
said in the manifest, because a manifest cannot say ${machine:account} safely — a node with no
account has nothing to resolve it to — and there the runtime runs as root and the file stays root's.
2026-10-02 21:44:44 +02:00
jochen ca7e81e964 A TypeScript bundle carries what it runs with: the toolchain's runtime directory is copied into it (hq to-be 38 WP3, ADR 0188)
A bundle that compiled was not yet a bundle that ran. The compiler resolved `import "nats"` from
the toolchain image's own node_modules and the pack took only what the compiler wrote, so what a
machine unpacked could not find a single dependency — and Node would have read the bare `.js` as
CommonJS besides. No TypeScript bundle had run live to show it; the runtime's own is the first that
must. A toolchain now names a Dependencies directory in its image, copied whole into the output's
root after the compile by a second run in the same image: for TypeScript /app/runtime, which the
runtime's image puts a `"type": "module"` package.json and its pruned node_modules at. An older
image without it fails the build by name rather than packing a bundle that starts nowhere. The
SDK's and the runtime's dependencies, nothing module-specific yet: a skeleton, by ADR 0188 §5.
2026-10-02 21:44:44 +02:00
mesh-admin 08bb56f1d3 Merge pull request 'The controller composes one tool runtime per node: its principal, the bundles, its process, and the gate (hq ADR 0175, to-be 38 WP2)' (#223) from feat/the-operators-machine into main 2026-10-02 19:27:52 +00:00
mesh-admin 1a44d281c2 Merge pull request 'node show cites ADR 0180 for a removed front end (hq ADR 0186)' (#224) from fix/a-ban-list-never-holds-a-neighbour into main 2026-10-02 16:47:25 +00:00
jschoubben 1c8fe65601 node show cites ADR 0180 for a removed front end (hq ADR 0186)
Another session took 0175 while that record was in review; the line printed on every converged
machine was pointing at an unrelated decision.
2026-10-02 18:42:16 +02:00
jochen 8eb6c3e94a A tool container on the runtime's image is refused once the runtime is registered (hq ADR 0175, to-be 38 WP2.4)
A module declaring tools, a container, and a build on mesh-tools' runtime image is a container whose
purpose is serving tools — the pattern the node's tool runtime retires. Once node-tools is in the
catalogue, registering one is refused by name, with the record that says why; before, it is accepted
as it always was, so a mesh converts in the design's order and nothing is refused before there is
anything to move to. This is the mechanism that keeps the old pattern from returning by habit.

Judged from a repository manifest's own build.on, and for a built manifest — which carries no build
— from what its build stood on, now recorded beside the commit as part of a module's provenance.
2026-10-02 18:30:14 +02:00
jochen 9d4d847dc4 The machine runs one tool runtime, loading every delivered bundle (hq ADR 0175, to-be 38 WP2.3)
Where node-tools is in a node's set, the declaration ends with one process: the runtime module's own
bundle, run from its one entrypoint by its language's interpreter, told in MESH_TOOL_MODULES every
<module>=<file> the machine's bundles load, where its credential is (the module's own broker secret
as this node places it), and — on a machine with an operator account — who the operator is, running
as that account so a tool that needs root can escalate as the operator would. Restarted when any
bundle it loads or the credential changes. A machine with no account runs it as root without the two
operator words; a machine without the runtime is sent nothing new.

A bundle says which of its entrypoints the runtime LOADS (`loads`), because one bundle may carry a
daemon beside its tools and importing the daemon into the runtime would start it there; absent, a
module declaring tools has every entrypoint loaded. And the TypeScript toolchain is rooted at the
module, so an entrypoint lands at the path it is named by — the runtime loading bundles by their
declared paths is what made the compiler's common-directory default visible.
2026-10-02 18:26:54 +02:00
mesh-admin bea1a1c513 Merge pull request 'A control plane behind its seat's row serves what it can (hq ADR 0185)' (#222) from fix/a-service-asked-to-run-is-still-running into main 2026-10-02 16:21:55 +00:00
jochen b1df688c62 A module's tools bundle reaches the machine as an archive where the runtime runs (hq ADR 0175, to-be 38 WP2.2)
The resolved manifest now carries what the build compiled — each bundle's source, digest, language
and entrypoints — because a tools bundle is named by no resource of the module's own: the node's
runtime loads it, and until this the mesh held no trace of the one artifact that runtime needs. A
repository manifest that writes `bundles` beside its build is refused: the mesh derives it.

Where the runtime module is in a node's set, every assigned module's bundle with entrypoints is
composed as an archive under the mesh's own directory, routed through the artifact store like any
image or archive the mesh built. A bundle without entrypoints is run rather than loaded and is
delivered by the process that runs it. A node without the runtime is sent exactly what it was.
2026-10-02 18:19:10 +02:00
jschoubben 21d38c9b0e A control plane behind its seat's row serves what it can (hq ADR 0185)
One verb in the row that this binary cannot run aborted the start, and a stale push that put an
older control plane back took the whole mesh off the bus for ten minutes — recoverable only by a
person running the binary outside its service, because the push that repairs it is one of the verbs
that had stopped being served. Now the verbs it knows are served, the ones it does not answer the
reason, and the start names them once.
2026-10-02 18:16:24 +02:00
jochen 1f86ed4135 One runtime principal per node carries every assigned module's tools (hq ADR 0175, to-be 38 WP2.1)
Where the node-tools module is assigned, the machine's bus user list gains one principal of kind
node-tools in place of that module's own: it may subscribe every carried module's tool namespace
and every held seat's verbs on its node, read and follow every membership on its node, call any
tool anywhere, answer what it is asked — and consume nothing, because tools are what it runs.
Every other module keeps its own principal, so a module still serving tools from its container
holds its own credential until it moves.

Named exactly as the module it stands for, so `module issue` and `rollout mint` deliver its
credential through the path a module's already takes, into node-tools' own `broker` secret. The
runtime module's name is one constant in each of the broker and catalogue packages, held to one
string by the agreement test, because a rule turns on it.
2026-10-02 18:16:14 +02:00
mesh-admin 689dd060b0 Merge pull request 'A jail's pattern names &lt;HOST&gt; once, and is refused by name when it does not (hq ADR 0179)' (#221) from fix/a-jails-pattern-names-the-host-once into main 2026-10-02 15:32:14 +00:00
jschoubben 99c4c4ef04 A jail's pattern names <HOST> once, and is refused by name when it does not (hq ADR 0179)
fail2ban expands <HOST> into a named capture group, so a pattern naming it twice is a duplicate
group name: the daemon refuses its whole configuration and exits, and the machine keeps no bans at
all — for every jail, not the one at fault. Hit live on the control node the day the jails shipped.
A jail with no name, no pattern, or a name another of the module's jails took is refused too.
2026-10-02 17:25:34 +02:00
mesh-admin e5e1666f91 Merge pull request 'The intrusion-prevention seat serves status, banned, ban and unban; the proxy logs a refused name (hq ADR 0179)' (#219) from feat/the-intrusion-seat-serves-its-verbs into main 2026-10-02 15:15:41 +00:00
jschoubben bd58d1c3ef The intrusion-prevention seat serves status, banned, ban and unban; the proxy logs a refused name (hq ADR 0179)
Every holder of node-intrusion-prevention owes the four verbs, as the packet filter's holder owes
its three (ADR 0170). The proxy says in its log when a name this mesh does not serve is asked for,
with the asking address last, so its own jail can read it.
2026-10-02 17:02:49 +02:00
mesh-admin d134c89a7c Merge pull request 'The service manager is a node seat with the unit verbs, and a user shape names the account (hq ADR 0176, 0177)' (#218) from feat/the-operators-machine into main 2026-10-02 14:58:50 +00:00
jochen a9307d9f33 The controller seat gains a generic verb: command runs one line of the binary and answers what it printed
Beside the named verbs, `command` takes a command line as the controller's
own shell would — `node account g14 jochen`, `node show ace`, `module list` —
splits it as a shell does (quotes group, backslash escapes, nothing expanded)
and runs it in this binary like every other verb. The named verbs keep their
schemas; this is the whole binary, added because the operator decided any
node may call any tool (hq ADR 0175) and a verb per command was the only
thing keeping the rest behind a shell on the control node. ADR 0154 carries
the dated note. Tests: a plain line, quoted words, an empty line and an
unclosed quote refused.
2026-10-02 16:53:27 +02:00
jochen 5eb1c9c2b7 The service manager is a node seat with the unit verbs, and a user shape names the account (hq ADR 0176, 0177)
node-service-manager joins the mesh's own seats, node-scoped, serving eight
verbs — units, status, start, stop, restart, enable, disable, journal — each
with a schema that takes an optional scope, "system" or the operator
account's "user" manager. Sixteen seats now; the count test says so and names
the record.

${machine:account} resolves in a resource's `name` and `user` as it already
did in path, owner and content: the shell module makes the operator's account
its holder's login shell through the `user` shape, and the desktop's watchers
run as that account through a user-scoped unit (host change alongside).
Neither can name the person. A machine with no account refuses by name.
2026-10-02 16:48:16 +02:00
mesh-admin 37b8edeec6 Merge pull request 'node show says a found firewall that was uninstalled is removed (hq ADR 0175)' (#217) from feat/the-found-front-end-is-uninstalled into main 2026-10-02 14:29:02 +00:00
jschoubben 424406b2d6 node show says a found firewall that was uninstalled is removed (hq ADR 0175) 2026-10-02 16:27:39 +02:00
mesh-admin 90455c61f6 Merge pull request 'The example images build from the Go the manifest pins' (#216) from jschoubben/example-images-go-base into main 2026-10-02 13:56:22 +00:00
jschoubben 1f6793cf0c The example images build from the Go the manifest pins
Four example Dockerfiles named golang:1.25 while go.mod requires 1.26;
the control plane's image takes GO_BASE from the manifest and these did
not, so a build that could not fetch a newer toolchain failed at go mod
download (found running the lab through the mesh).
2026-10-02 15:52:25 +02:00
mesh-admin 6ef522fbda Merge pull request 'Cite hq ADR 0170, not 0169: the firewall seat's record was renumbered' (#215) from fix/adr-0170-cited into main 2026-10-02 12:53:13 +00:00
jschoubben 46f65c12a0 Cite hq ADR 0170, not 0169: the firewall seat's record was renumbered after a collision on hq main 2026-10-02 14:52:24 +02:00
mesh-admin 8f7c02d77a Merge pull request 'The virtualisation capability grants the lab its daemon's socket (hq ADR 0172)' (#214) from jschoubben/the-lab-is-a-module into main 2026-10-02 12:47:45 +00:00
jschoubben a637df01ea The virtualisation capability grants the lab its daemon's socket
The lab raises machines on the virtualisation daemon, and a module may
mount a machine's socket only through the capability that grants it
(novox/hq ADR 0172). Also brings the resolver's tests to the setting
dnsmasq's listen addresses now come from, and to a module left out
rather than refused.
2026-10-02 14:46:17 +02:00
mesh-admin 252eb786a7 Merge pull request 'A filter module's own filter file counts as declared for a mount (hq ADR 0169)' (#213) from feat/the-firewall-seat-serves-its-verbs into main 2026-10-02 12:05:09 +00:00
jschoubben 9d13b0593b A filter module's own filter file counts as declared for a mount (hq ADR 0169)
The nftables module's runtime mounts the file filtering.into names, to reload
the mesh's table; the mount check knew every other declaration of a path and
not this one, and the module's first build was refused for it.
2026-10-02 14:04:35 +02:00
mesh-admin 30d548a762 Merge pull request 'The node-packet-filter seat serves rules, reload and remove (hq ADR 0169)' (#212) from feat/the-firewall-seat-serves-its-verbs into main 2026-10-02 11:28:57 +00:00
jschoubben 550e4c6acb The node-packet-filter seat serves rules, reload and remove (hq ADR 0169)
What a person asks a machine's packet filter whatever filter answers: the
rules as enforced, reload the mesh's own, remove one rule set the mesh did
not write — named as the host reports it under ADR 0168. Every holder serves
all three; a running mesh widens its seat row at the next controller start.
2026-10-02 13:27:04 +02:00
mesh-admin 1587fd97f9 Merge pull request 'The mesh says what filters a converged machine: filters kept per node, shown, named by status, previewed with fates (hq ADR 0168)' (#211) from feat/one-thing-filters-a-converged-machine into main 2026-10-02 10:03:01 +00:00
jschoubben b5df244096 The mesh says what filters a converged machine: filters kept per node, shown by node show, named by status, and previewed with their fates (hq ADR 0168)
A host reports every table and chain that refuses traffic with its owner,
and a converged machine's found firewall's state. The controller keeps both
on the node's record (migration 0054), shows them on node show, names every
converged machine something other than the mesh filters in status — text
and JSON, and such a machine is not well — and the converge preview lists
what filters the machine with the fate of each: retired with the front end,
left as the runtime's, left as a ban, or left in force and not the mesh's.
What was invisible for eleven hours (issues 144, 145) is said by name.
2026-10-02 12:00:12 +02:00
mesh-admin db47bb68e9 Merge pull request 'The controller's tools can set an assignment's settings (hq issue 198)' (#210) from jschoubben/settings-verb into main 2026-10-02 09:42:08 +00:00
jschoubben db84142d38 The controller's tools can set an assignment's settings
Per-machine and mesh-wide settings could be set only from the
controller's command line. The settings verb runs settings set|clear,
passing the values inline, which the command now accepts as well as a
file (novox/hq issue 198).
2026-10-02 11:41:57 +02:00
mesh-admin c9204591bf Merge pull request 'The hub relays the mesh passing through it (hq issue 196)' (#209) from jschoubben/the-hub-relays-the-mesh into main 2026-10-02 09:17:17 +00:00
jschoubben e8e707e013 The hub relays the mesh passing through it
The forward chain judged a packet relayed from one machine of the mesh
to another by this machine's own published ports, so two machines behind
the hub reached each other only on ports the hub published for itself
(novox/hq issue 196). In and out on the tunnel is now accepted, and the
machine it is for filters it.
2026-10-02 11:17:09 +02:00
mesh-admin 0c003774ba Merge pull request 'A take acts on the preview it showed; a setting is judged where it is stored; a kept network and a minted secret are said (hq ADR 0163)' (#206) from feat/a-take-is-a-comparison-the-rest into main 2026-10-02 09:04:05 +00:00
jschoubben fbc3d320ea A take acts on the preview it showed; a setting is judged where it is stored; a kept network and a minted secret are said (hq ADR 0163)
take ends its preview with a digest and --yes names it, as the flip does; a
changed preview or an account older than the flip allows is refused. A module
the machine holds nothing for has nothing to compare, and --yes suffices. A
published port's reach is said as the machine reported it. Every secret the
module holds on the machine is listed with where it came from, and one the
mesh minted for a service whose data was found refuses unless --mint names it.

One judgement of a module's settings against its definition, in the catalogue:
settings set refuses what cannot compose or reaches nothing, naming node,
module, layer and key; Compose leaves out a module whose definition moved
under a stored setting, the envelope says so (left_out), plan and push say it
by name, and the machine is told everything else. A stray setting no longer
refuses the whole machine where it is read (issue 096).

The per-machine setting networks keeps a found network for a taken container,
on an adopted machine only; the container's declaration carries it and the
preview names it (rule 4).
2026-10-02 11:01:09 +02:00
mesh-admin cf495e315f Merge pull request 'The proxy is told its routes and the mesh on the bus, and serves internal names to the mesh only (hq issue 191, ADR 0167)' (#207) from jschoubben/an-internal-only-route into main 2026-10-02 07:42:45 +00:00
mesh-admin 86bc1fad2c Merge pull request 'The controller's tools can issue a module its bus account (hq issue 191)' (#208) from jschoubben/the-controller-issues-a-module-account into main 2026-10-02 00:22:23 +00:00
jschoubben c9eba5f3b8 The controller's tools can issue a module its bus account
A module's account was mintable only from the controller's command line,
so a rollout that gave a module one could not be finished through the
mesh's own tools (novox/hq issue 191). The issue verb runs module issue
for a module on a machine; the caller pushes the machine after.
2026-10-02 01:55:06 +02:00
jschoubben 24f024dd74 The proxy is told its routes and the mesh on the bus, and serves internal names to the mesh only
The proxy answered every routed name to any request carrying it, so an
internal-only route would have been public under its internal name. Each
membership now carries what its module receives, from the same
composition as its received file, and every machine's private-network
address, the list the packet filter's "from the mesh" is. The proxy
follows its membership, serves internal names only to those machines and
itself, and keeps the file until the bus has spoken (novox/hq ADR 0167,
issue 191).
2026-10-02 01:46:30 +02:00
jschoubben 9acb5f1292 route-proxy: serve a route that names only its internal host
Since ADR 0138 an endpoint that reaches only the private network gets an
internal-name and no name, and the proxy skipped it as naming nothing, so
every internal-only module was unreachable by name (novox/hq issue 191).
2026-10-01 23:31:57 +02:00
mesh-admin 20516e3fcb Merge pull request 'plans --what-if: the plan a merge would produce, read before merging' (#205) from feat/plans-what-if into main 2026-10-01 20:41:58 +00:00
jschoubben 0a40c046cf plans --what-if: the plan a merge would produce, read before merging
Given a repository and the files a branch changes (or the modules by name), plans answers with the tiers
the merge handler would produce — the modules those files touch, what packages their source, everything
reachable along the dependency relation — and what each tier does: built and sent to its machines, or
built and left because its policy records. Saved nowhere. On the seat as plans {repository, paths|modules}.
2026-10-01 22:38:49 +02:00
mesh-admin 8f51c66eb1 Merge pull request 'A plan rolls out every tier it built' (#204) from fix/a-plan-rolls-out-every-tier-it-built into main 2026-10-01 20:25:00 +00:00
jschoubben 477d3ac9a7 A plan rolls out every tier it built
A dependent rebuilt because its base moved, or a module that packages another repository's source,
keeps its source commit; the catalogue announces no move for it, and its machines kept the old image
until somebody pushed — forty-three modules after every runtime-image merge (hq issue 189). When a tier
is built, the plan now sends the machines of every module in it whose policy rolls out, once, moved
commit or not; a module whose policy records is built and left, as its policy says. The gate between
tiers waits as before for the ones a later tier is built by.
2026-10-01 22:21:42 +02:00
mesh-admin 6ff449e363 Merge pull request 'A plan sends what it waits for' (#203) from fix/a-plan-sends-what-it-waits-for into main 2026-10-01 19:58:04 +00:00
jschoubben 3ca1f5d26a A plan sends what it waits for
A tier whose module a later tier is built by waited for the machines running it to report after the
build — and relied on the catalogue's moved event to send them. A rebuild from the same source commit is
not a move the catalogue announces: the build machine rebuilt for a controller change kept its commit,
nothing sent it, and the plan waited on a report that would never come (2026-10-01, 19:45Z). The plan
now sends the machines running a gated module once, records when, and waits for the reports after that.
2026-10-01 21:54:40 +02:00
mesh-admin 74a1382164 Merge pull request 'A converged machine says its strays too' (#202) from fix/a-converged-machine-says-its-strays-too into main 2026-10-01 19:44:35 +00:00
jschoubben 40f169229d A converged machine says its strays too
node show printed what a machine reports only for an adopted one; a converged machine holds nothing
and can still run a container nobody asked for (hq ADR 0163), so its strays are said whatever its mode.
2026-10-01 21:41:23 +02:00
mesh-admin 9a99c422a1 Merge pull request 'A take is a comparison: the preview, its refusals, the strays, and the policy said at build (hq ADR 0163)' (#201) from feat/a-take-is-a-comparison into main 2026-10-01 19:28:35 +00:00
jschoubben 73a34cc0a7 A take is a comparison: the preview, its refusals, the strays, and the policy said at build (hq ADR 0163)
The host now reports, for every held thing, the facts a take compares; the controller keeps them, and
take puts them beside what the module declares — the found image and its age against the declared one,
the found networks and who else is on them, ports and mounts, a found file's difference from the
declared content — and refuses a downgrade without --downgrade and a differing file without --replace
<path>. Without --yes the comparison is printed and nothing is taken. node show lists the facts and
the strays the machine reports. build and the daemon's take-in say when a module's policy rolls the
result out at once. The own-secret refusal points at the provider form for a required secret.
2026-10-01 21:25:27 +02:00
mesh-admin 75932d6f3e Merge pull request 'The rest of the mesh is quiet about a machine it resolved without the excluded one's offers' (#200) from fix/the-rest-of-the-mesh-is-quiet-about-a-machine-it-excluded into main 2026-10-01 18:56:33 +00:00
jschoubben c430ca76f0 The rest of the mesh is quiet about a machine it resolved without the excluded one's offers
With one machine excluded, the others are resolved without its offers, and one that consumes them
cannot resolve by design — the view is partial, the machine is not dropped. The line that names a
dropped machine now speaks only for the whole-mesh view (seats), where a drop is a fault (hq 188).
2026-10-01 20:53:27 +02:00
mesh-admin ba6a3ea829 Merge pull request 'The rest of the mesh resolves each machine with its pins and names what it leaves out; a built-by edge never widens a plan; plans stop' (#199) from fix/a-machine-not-on-the-network-is-said into main 2026-10-01 16:25:09 +00:00
jschoubben dcf6278523 The rest of the mesh resolves each machine with its own pins, and says which machine it leaves out
The second pass of theRestOfTheMesh resolved every machine without its pins. Since a machine with two
providers of one provision is refused unless a pin names one (195/196), the control node was refused
there and vanished: every seat it holds read as unheld, the build machine refused what needs the git
seat, the roll-out was refused — and nothing said why (hq issue 188). Each machine is now resolved as
its plan resolves it, with its pins; a machine left out is named with the resolver's words.
2026-10-01 18:22:17 +02:00
jschoubben d94f9e7f9f A built-by edge orders and gates a plan; it never widens it — and plans stop
The first live plan took seventy-five modules along for a controller change: the builder packages the
controller's source, everything is built by the builder, so everything was reachable. A module built by
the build machine is not changed by a new build machine. Reachability now follows the code and build
edges only; built-by still orders a tier after the build machine and gates it on the machine's roll-out.
plans stop <id> ends a plan by hand: what was asked still builds and registers, nothing further is asked.
2026-10-01 18:16:47 +02:00
jschoubben 882687afd4 Merge remote-tracking branch 'origin/main' into fix/a-machine-not-on-the-network-is-said 2026-10-01 18:16:14 +02:00
jschoubben 884c088949 A machine the network filter drops is said, not skipped in silence (hq issue 188)
onTheNetwork resolves every machine unchecked and skipped one whose resolution refused. A machine
skipped there has no address, so its own plan fails on the first placeholder that needs one, in another
module's words, every seat held on it reads as unheld, and what is built from it cannot be built — four
symptoms, none naming the refusal (2026-10-01, the control node, forty minutes). The refusal is now said
where it happens, in the resolver's own words.
2026-10-01 18:14:34 +02:00
mesh-admin 76f27562a1 Merge pull request 'The plans verb: what the last merges produced and where each stands (hq ADR 0162)' (#198) from feat/the-plans-verb into main 2026-10-01 16:14:09 +00:00
jschoubben 89d43e0dad gofmt 2026-10-01 18:04:58 +02:00
jschoubben 09b636e628 The plans verb: what the last merges produced and where each stands (hq ADR 0162)
The mesh's seat answers plans — the recent plans with their tier, what each waits for and since when,
or one plan whole given its id — so the console reads a merge's progress where it reads everything
else, instead of a person reading the daemon's log.
2026-10-01 17:58:49 +02:00
mesh-admin d2ed6d50c9 Merge pull request 'A merge produces a tiered plan the mesh keeps (hq ADR 0162)' (#197) from feat/a-merge-produces-a-tiered-plan into main 2026-10-01 15:57:10 +00:00
jschoubben fdf338dbd1 A plan is kept, advanced and resumed from the store: the test 2026-10-01 17:56:06 +02:00
jschoubben a69a832248 A merge produces a tiered plan the mesh keeps (hq ADR 0162)
A module's dependencies are one relation in the catalogue — stands-on, packages, built-by, declared —
answered by one call. A merge takes what moved and everything reachable from it, sorts the set into
tiers (a code dependency in the same tier, a build dependency after its base is built, a runtime
dependency after the build machine is built and running; the build machine's own base comes first,
built by the one that runs), writes the plan to the store, asks the first tier and returns. Every
outcome advances the plan; a ticker advances what outcomes cannot; a controller replaced mid-plan
resumes it. status lists open plans and names one that has waited too long.
2026-10-01 17:53:55 +02:00
mesh-admin bdcbda801e Merge pull request 'The first pass does not refuse two providers beside the consumer (hotfix for #195)' (#196) from fix/first-pass-does-not-refuse-two-providers-beside into main 2026-10-01 15:34:23 +00:00
jschoubben 5a7ed56f61 The first pass does not refuse two providers beside the consumer
#195 refused two modules beside a consumer that both answer a bound
provision — in every pass. The first pass exists only to answer what a node
offers, and a refusal there makes the machine vanish from every other node's
world (resolve.go says so for its sibling case): with novox refused for its
own acme-ca, ace's plan lost the vault and the identity provider and read
"nothing in this mesh provides secret". Seen live within minutes of the
rollout. The second pass refuses it, where it is asked, as before.
2026-10-01 17:34:17 +02:00
mesh-admin d54ecb3bf2 Merge pull request 'A pin names the module as well as the node; a node that answers twice is refused' (#195) from feat/pin-names-the-provider into main 2026-10-01 15:26:02 +00:00
jschoubben cf84117638 A pin names the module as well as the node; a node that answers twice is refused
A provider is a (node, module) pair (design 23), and the pin — the one way a
consumer names its provider — named only the node. Two modules on one node
can both answer a provision (public-acme and step-ca both offer acme-ca on
novox), and then the resolver, given a pin naming that node, took the last
provider listed: a coin flip. The same ambiguity beside the consumer was
settled by a map walk — random per plan — which is how novox's own
route-proxy got its issuer (novox/hq #258).

- `pin <node> <provision> <from-node> <module>`: both halves, always. The
  console gains `pin` and `unpin`. The provider may be on the consumer's own
  node, since two modules beside it can both answer.
- The resolver refuses ambiguity instead of picking, across machines and
  beside the consumer alike, naming every candidate as node/module and the
  form of the pin that settles it. A plain capability that grants nothing
  and serves nothing (three shells beside an editor) is not a choice to put
  to anybody and stays as it was.
- provision_pin gains a nullable module (0050); records made before are
  completed where the node they name answers once, and left for a person
  where it answers twice (0051).
- The provider of something already satisfied is looked for among what was
  assigned, not only what the walk has reached — a consumer reached before
  the provider beside it no longer loses its binding.
- The start-time check that every declared verb is runnable samples each
  verb's required arguments from its schema instead of three guessed keys.

Live consequence: a node that has two providers of one bound provision
assigned (novox: acme-ca) resolves only once pinned —
`pin novox acme-ca novox public-acme`.
2026-10-01 17:23:31 +02:00
mesh-admin 8d4e940866 Merge pull request 'The build machine takes one ask at a time, and says so while it builds' (#194) from fix/the-build-machine-takes-one-ask-at-a-time into main 2026-10-01 14:19:18 +00:00
jschoubben 11b20b10ff The build machine takes one ask at a time, and says so while it builds
With the worker consumer's default of many deliveries in flight, every ask behind the one being
built was delivered at once, left unacknowledged for the length of the build, redelivered after the
ack wait and dropped after the fifth time: on 2026-10-01 twenty-six of forty-three builds asked in two
minutes were never built and the queue read as empty (hq issue 186). The holder's worker now has one
in flight, and a running build tells the bus it is still working, as the controller's long handlers
do, so a build longer than the ack wait is neither redelivered nor counted out.
2026-10-01 16:16:13 +02:00
mesh-admin 7e701c0db2 Merge pull request 'A merge of a base rebuilds what stands on it' (#193) from fix/a-merge-of-a-base-rebuilds-what-stands-on-it into main 2026-10-01 14:12:32 +00:00
jschoubben 853c63b181 A merge of a base rebuilds what stands on it
The controller knew which modules were built against which base artifacts and used it only when asked
(build --on). A merge that rebuilt the runtime image left forty-two modules on the old image until
somebody asked, twice, by hand (hq issue 186). The merge now takes every module standing on what moved,
through every layer, into the same rebuild, in base order — the same rebuild the flag does, asked by
the merge that made it necessary.
2026-10-01 16:09:26 +02:00
mesh-admin 84ac840ff4 Merge pull request 'The vault's seat, and a report that carries the machine's profile (hq ADR 0161)' (#192) from feat/the-vaults-seat-and-the-machines-profile into main 2026-10-01 14:02:10 +00:00
jschoubben e8e502343f The vault's seat, and a report that carries the machine's profile (hq ADR 0161)
mesh-vault joins the mesh's own set — mesh-scoped, delivering secret — because the controller seals
every minted credential with it, which is the test for a seat of the mesh's own; a second provider is
a second claimant, refused by name (issue 106). A report may carry the machine's profile, detected
again by the apply that reports, and the latest replaces the enrolled one: a machine that switched
its network manager is a machine whose uplink holder lacks a capability at its next push (issue 138).
2026-10-01 15:58:04 +02:00
mesh-admin 1edc44b25f Merge pull request 'The store seat promises two verbs, databases and query (ADR 0159)' (#186) from feat/the-store-seat-has-verbs into main 2026-10-01 13:45:29 +00:00
mesh-admin 5a1b37e477 Merge pull request 'A push issues the memberships of the machines it told' (#191) from fix/a-push-issues-memberships into main 2026-10-01 13:45:20 +00:00
jschoubben 4a6a4eadeb A push issues the memberships of the machines it told
sendTo — the path a roll-out, a rotation and a secret change take — issued memberships after its
sends (ADR 0160); the push command, which sends the same declarations through its own loop, did
not, so the one command operators run issued none. Said once in the same words after the sends.
2026-10-01 15:41:09 +02:00
mesh-admin 69f559ab4c Merge pull request 'A refused membership does not stop the controller' (#190) from fix/a-refused-membership-does-not-stop-the-controller into main 2026-10-01 13:31:27 +00:00
jschoubben 05b90f966a A refused membership does not stop the controller
A stream publish waits for its acknowledgement as long as its context lives, and the server never
acknowledges a publish it refuses. Issuing memberships after a push used the daemon's own context, so
the one refused membership of 2026-10-01 (hq issue 183) held the controller's receive loop for good:
no report, no build outcome, no merge was heard until a restart (hq issue 185). Issuing one
membership is now bounded to ten seconds, and a push says how many could not be issued and stands —
the machines keep the shape they derive until the next push.
2026-10-01 15:30:04 +02:00
mesh-admin 184913b620 Merge pull request 'The controller may publish the memberships it issues' (#189) from fix/the-controller-may-publish-memberships into main 2026-10-01 13:26:48 +00:00
jschoubben de7aed5016 The controller may publish the memberships it issues
The server refused every membership the controller published after a push (2026-10-01, Permissions
Violation for Publish to mesh.assignment.<node>.<module>): the controller's own grant named the
control, node and JetStream subjects and not the assignments it alone issues (hq ADR 0160). Broker
golden regenerated; one line differs.
2026-10-01 15:20:32 +02:00
jschoubben 18958154f0 A claim may name the verbs it serves for its seat (hq ADR 0160)
The store's databases is not postgres's postgres_list_databases, and a holder may serve both. A
claim's serves names the seat's verbs the module implements for the role; absent, the module's own
tools must list every verb the seat promises, which is how a module named like its seat says they
are one and the same. Registration refuses a claim naming a verb the seat never promised, and the
credential's claims carry the claim's own verbs to the runtime.
2026-10-01 15:18:34 +02:00
jschoubben a2b1f9e936 Merge remote-tracking branch 'origin/main' into feat/the-store-seat-has-verbs 2026-10-01 15:16:52 +02:00
mesh-admin c9a0b1f9f4 Merge pull request 'The mesh issues an assignment's subjects: a membership per module on a machine (ADR 0160)' (#188) from feat/the-mesh-issues-an-assignments-subjects into main 2026-10-01 12:45:45 +00:00
jschoubben f450303e8e A person invoking one tool holds it both ways it is addressed (the test, after #185) 2026-10-01 14:45:31 +02:00
jschoubben 603ad61142 The mesh issues an assignment's subjects: a membership per module on a machine (hq ADR 0160)
For every module on every machine the controller composes what that instance serves — its machine's
address always, the module's plain address in a queue when it is alone or its definition says its
instances are interchangeable — the verbs of the seats it holds at the seats' subjects, where its
events land, and what it may reach, resolved the same way for the modules it invokes. Published
beside the node's declaration on `mesh.assignment.<node>.<module>`, last per subject in a stream
that allows direct reads, and the account may read exactly its own. Composed from the same records
the bus's accounts are, so what a runtime serves and what its account may are one composition.
`instances: interchangeable` is the one fact a definition states for it.

The shape issued is the shape the mesh already had, so nothing moves when the membership arrives;
the runtime that reads it instead of deriving it is the next piece.
2026-10-01 14:43:26 +02:00
jschoubben e7cff3d38e The store seat promises two verbs, databases and query (hq ADR 0159)
Seeded additively into the seat's row at the controller's next start; a holder must list tools of
these names (design 33 §3), so this merges after the database engine's definition does.
2026-10-01 14:02:46 +02:00
mesh-admin 55c5c061ab Merge pull request 'A tool's grant covers the machine-addressed subject, and a credential names the seats its module claims (ADR 0159)' (#185) from feat/a-tool-call-names-the-machine into main 2026-10-01 12:01:29 +00:00
jschoubben ccae1ec303 gofmt 2026-10-01 13:58:41 +02:00
jschoubben 9fd5971212 A tool's grant covers the machine-addressed subject, and a credential names the seats its module claims (hq ADR 0159)
`invokes: [<module>.<tool>]` now grants `mesh.mod.<module>.tool.<tool>` and the same with the
machine as its last token, which is how a call reaches one machine's instance. The broker
credential the mesh writes carries `claims`: each seat the module claims, its scope, and the verbs
the seat promises, so the runtime serves them on the seat's subjects; the holder's grant, composed
from the holding, is what admits the subscription.
2026-10-01 13:58:23 +02:00
mesh-admin 05ccab2e4b Merge pull request 'The two seat commands appear in the usage text' (#94) from fix/seat-usage-lines into main 2026-10-01 11:17:51 +00:00
mesh-admin 05ae5e5040 Merge pull request 'A provider with one credential shares it with every consumer, remade for all at once (ADR 0158)' (#184) from feat/0158-a-provider-with-one-credential-shares-it into main 2026-10-01 10:27:08 +00:00
jschoubben 988250f37a A provider with one credential shares it with every consumer, remade for all at once (hq ADR 0158)
An offer may say `"credential": {"own": "<secret>"}`: the provider's own secret is the credential
every consumer of that provision receives, in the shape of a pair credential. The vault keeps one
value, sealed to the provider, to every consumer that holds the provision and to the operator, all
under one generation stamp; a consumer binding later, or `secret rotate` on the provider's secret,
makes a fresh value and seals it to every holder in one act, and the rotate command sends every
holding machine together. An accepted value is sealed to the consumers of the moment and never
remade: a consumer binding after it is refused with the way out (ADR 0113). The named own secret
must say how it is taken (issue 180), so the provider's start applies the file.

A need carries the shared secret's name from either side of the machine boundary; the plan mints a
consumer's copy from the provider's value. Registered manifests keep their bytes.
2026-10-01 12:26:44 +02:00
mesh-admin 6ca4ba68c8 Merge pull request 'A module's own secret rotates when its definition says the module reads it at start (hq 180, forge 231)' (#183) from feat/231-an-own-secret-rotates into main 2026-10-01 09:42:10 +00:00
jschoubben 1469f5ff82 A module's own secret rotates when its definition says the module reads it at start (hq 180)
`secret rotate <node> <module> <name>` makes the secret anew the way the first mint did, seals it
to the machine and the operator, and sends the machine, so the module starts again on the new value
— said in the log with who asked and when, never the value. Only for a secret whose definition says
`"taken": "at-start"`: an own secret is now a path, or {path, taken}, and a definition that says
nothing of how a secret is taken is refused with the word to write, because a credential rotated
under software that never reads it again is worse than one left alone (issue 179). `applied` is
refused by name until the staged form ADR 0114 decided is built; a value given to the mesh is
refused as ADR 0113 says. `rotate` is a verb on the controller's seat with two shapes — a pair
credential by provision, an own secret by machine, module and name — so the console can ask.
Registered manifests keep their bytes: a path alone is written back as a path.
2026-10-01 11:41:49 +02:00
mesh-admin 0e977399d4 Merge pull request 'The controller's own manifest places the mesh's files, now that the word is live (issue 174)' (#182) from feat/the-controllers-own-manifest-is-placed into main 2026-10-01 08:57:29 +00:00
jschoubben c462db105e The controller's own manifest places the mesh's files, now that the word is live (issue 174)
Held back one release (#177) because the controller that reads `place: "mesh"` could not be built
by the one that did not. It runs since 2026-10-01 00:06; the manifest names no host path now, and
resolves to exactly the paths it named (TestPlacedDirectoriesKeepTheirPaths over both).
2026-10-01 10:56:42 +02:00
mesh-admin 7273ca2f0f Merge pull request 'A routed name resolves to the node whose proxy serves it, never to a provider merely told it (hq 178, forge 227)' (#181) from fix/227-a-name-resolves-to-the-node-that-serves-it into main 2026-10-01 00:04:14 +00:00
jschoubben ad797d8742 A routed name resolves to the node whose proxy serves it, never to a provider merely told it (hq 178)
The names region attributed a composed name to whichever labelled contribution a map yielded last.
A dashboard contributes its label to its route and to the identity provider, which must know the
public name for a redirect; so on one plan grafana.<domain> pointed at the proxy's machine and on
the next at the identity provider's, and the whole region flipped with it (forge issue 227). And a
module routed several times contributed no name at all, because the single-value reading of its
contributions is empty for the many shape.

Now every node's resolution is read first and the names are attributed across them at once: the
terminus serves the name — the provider that is not itself published under a labelled name through
another — walked in order, so one mesh yields one region. Name-agnostic, structural, deterministic.
2026-10-01 02:03:50 +02:00
mesh-admin 5b39e95361 Merge pull request 'Two store-backed tests rotted because nothing runs the check (issue 177)' (#180) from fix/the-converged-declaration-guard into main 2026-09-30 23:37:40 +00:00
jschoubben 7e4a0ecf9a Two store-backed tests rotted because nothing runs the check (novox/hq issue 177)
The converged-declaration guard still expected `hosts` on a container after c978aa7 took it off
every container, and the adopted-anchor fixture reported no outward link after ADR 0140 made a
filter depend on one. Both failed under `make check` since 2026-09-28/30; the build does not run the
check, so the mesh never saw it. The guard is re-captured with the change named — a field an older
host never sees is the one change it permits — and the fixture reports a link as a real host does.
2026-10-01 01:37:18 +02:00
mesh-admin 7661f57833 Merge pull request 'A build is taken in where its outcome is heard, and the build tool answers at once (issue 176)' (#179) from fix/176-a-build-is-registered-where-it-is-heard into main 2026-09-30 23:27:29 +00:00
jschoubben 076e0ae259 A build is taken in where its outcome is heard, and the build tool answers at once (issue 176)
The console's `build` tool answered "no build machine answered within 0s", handed a forge path to
git as written, and a build heard afterwards was recorded and never registered: recording and
registration lived only in the waiting caller, and the tool did not wait.

Now one function takes a build's outcome in — records it, parses the manifest, refuses a definition
naming an installation, registers the module with its source as the seat and path the request
carried — and both the waiting command and the daemon that follows the role's `built` event call
it. `build --wait 0` asks and returns with the id; `builds --log <id>` follows it. The seat verb
says `--self` for a repository given without a scheme.
2026-10-01 01:27:04 +02:00
mesh-admin a96e2f0d78 Merge pull request 'A build says what it does on the bus, as it happens (ADR 0157)' (#178) from feat/a-build-says-what-it-does into main 2026-09-30 22:43:52 +00:00
jschoubben 17f7cb0d9c A build says what it does on the bus, as it happens (novox/hq ADR 0157)
The build-machine seat emits `started` and `log.<build id>` beside `built`. Every line the builder
speaks — each step, each command with its duration, and on failure the command's own output — goes
to stderr as before and onto the bus under the build's id, one subject per build, kept a week in
EVENTS with every other event. `builds --log <id>` reads it back from the stream with a consumer
that is gone when the reading is done, on the command line and as the controller's seat verb;
`builds` lists each build's id and `build` says the id it asked with.

Lines are core publishes with a sequence number, so a build is not slowed by an ack per line and a
gap is visible; `started` and `built` are awaited into the stream. The seat protocol widens
additively at the controller's next start; the holder's grant follows on the broker node's next
composition.
2026-10-01 00:43:07 +02:00
mesh-admin f6685ed22d Merge pull request 'An assignment places a module's directories and its accesses (hq 153)' (#176) from feat/153-an-assignment-places-directories-and-accesses into main 2026-09-30 22:03:33 +00:00
jschoubben 52c18f7a45 The path-preservation proof resolves access ids to their default paths too
An access named by id (issue 153) resolves to the path the definition still carries when the
assignment says nothing, and the proof compares that — the same rule as a placed directory.
2026-10-01 00:01:41 +02:00
jschoubben fa7415fcd0 Merge main into the branch: the assignment's placement sits beside the mesh's own place (issue 174) 2026-09-30 23:47:09 +02:00
mesh-admin f8947a806d Merge pull request 'The controller's own manifest names its paths for one more release' (#177) from fix/the-controllers-own-manifest-waits-a-release into main 2026-09-30 21:15:10 +00:00
jschoubben b98e503a61 The controller's own manifest names its paths for one more release
A manifest word ships one release after the code that reads it. The merged manifest already said
`place: "mesh"`, and the running controller, which does not know the word, refused its own build
result — so the controller that knows it could never be built. The literal paths come back here;
the conversion follows once this release runs.
2026-09-30 23:11:36 +02:00
jschoubben 5b832918df Merge pull request 'A setting reaches only what declares it, the mesh places its own files, registration refuses a name (issues 173, 174, ADR 0155)' (#175) from feat/the-mesh-places-its-own-files into main 2026-09-30 20:50:59 +00:00
jschoubben 17bbcc1596 An assignment places a module's directories and its accesses (hq 153)
A definition names no host path (ADR 0112); an adopted machine keeps its
data where the predecessor put it. Two settings, validated like endpoints:

  places:   {<directory id>: <path> | {path, owner}}
  accesses: {<access id>: <path>}

An access may now be declared by id (`{"id": "series", "mode": "read-write"}`)
and named in mounts, env and content as ${access:<id>}; the assignment
says where it is on this node, and an access nobody placed is refused by
name. A definition still carrying a path keeps it as the default the
assignment replaces. A placed directory takes the assignment's owner
where it says one. Resolved in composition, so the host receives paths
and owners exactly as before.
2026-09-30 22:42:50 +02:00
jschoubben 29be985c23 A served value or a contribution's may be the operator's: ${setting:…} fills there too, refused by name when unset
Issue 173's rule needs it: a mail provider serves its domain and an identity provider its issuer,
and neither is the definition's to state. Declared as ${setting:<key>}, filled from the layers after
the overrides; a key so asked for is not stray.
2026-09-30 22:34:43 +02:00
jschoubben 05d977666a A setting reaches only what declares it, the mesh places its own files, registration refuses a name
Three of novox/hq's group-4 leftovers, one branch.

Issue 173: a module's settings reached every route it contributed, every database it asked for and
every served fact its consumers read — a mail server's site name arrived at the proxy as a route
fact. A setting now overrides a key a contribution or served fact declares and adds none; a file
still merges any key, and a key nothing takes is named as stray instead of dropped silently.

Issue 174: the mesh's own files for a module — its bus credential, its merged config, its bindings —
were placed by the definition under /var/lib/mesh/<module>, 232 host paths in 50 definitions. A
directory may now say `place: "mesh"` and resolves to <root>/mesh/<module>; a directory beneath a
placed one may state its path as `${dir:<id>}/<rest>` and moves with it. The proof test resolves
both catalogues and compares: 48 definitions, no path moved. The controller's own manifest is
converted here; the catalogue in mesh-catalog.

ADR 0155: the installation check moves to registration. `module add` and a build's result both
refuse a definition that names an installation, in the check's words, with the way out; the build
stays recorded.
2026-09-30 22:29:28 +02:00
jschoubben e871991495 Merge pull request 'The catalogue-wide checks run against the sibling checkout by default' (#174) from fix/the-catalogue-checks-run-by-default into main 2026-09-30 20:10:44 +00:00
jschoubben f9e19814eb The catalogue-wide checks run against the checkout beside this one by default
A check that only ran when somebody remembered a variable was a check nobody ran (novox/hq issue
134). MESH_CATALOGUE still overrides; the checks skip only when no catalogue can be found.
2026-09-30 22:10:42 +02:00
jschoubben af31315a5f Merge pull request 'The controller takes one announcement at a time' (#173) from fix/one-announcement-at-a-time into main 2026-09-30 19:37:56 +00:00
jschoubben c585158836 The two seat commands appear in the usage text
Both existed and neither was listed: rename since ADR 0122, the handover since
ADR 0131. Found by asking the running binary for help and seeing only the list.
2026-09-27 23:45:59 +02:00
156 changed files with 11581 additions and 776 deletions
+4 -4
View File
@@ -59,7 +59,7 @@ PROVISIONER_IMAGE ?= mesh-provision-postgres:$(VERSION)
PROVISIONER_DEV_TAG ?= mesh-provision-postgres:development PROVISIONER_DEV_TAG ?= mesh-provision-postgres:development
provisioner-image: provisioner-image:
docker build -f examples/postgres-provisioner/Dockerfile \ docker build --build-arg GO_BASE=$(GO_BASE) -f examples/postgres-provisioner/Dockerfile \
-t $(PROVISIONER_IMAGE) -t $(PROVISIONER_DEV_TAG) . -t $(PROVISIONER_IMAGE) -t $(PROVISIONER_DEV_TAG) .
@echo @echo
@docker image inspect $(PROVISIONER_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes' @docker image inspect $(PROVISIONER_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
@@ -70,7 +70,7 @@ OBJECTSTORE_IMAGE ?= mesh-provision-objectstore:$(VERSION)
OBJECTSTORE_DEV_TAG ?= mesh-provision-objectstore:development OBJECTSTORE_DEV_TAG ?= mesh-provision-objectstore:development
objectstore-image: objectstore-image:
docker build -f examples/objectstore-provisioner/Dockerfile \ docker build --build-arg GO_BASE=$(GO_BASE) -f examples/objectstore-provisioner/Dockerfile \
-t $(OBJECTSTORE_IMAGE) -t $(OBJECTSTORE_DEV_TAG) . -t $(OBJECTSTORE_IMAGE) -t $(OBJECTSTORE_DEV_TAG) .
@echo @echo
@docker image inspect $(OBJECTSTORE_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes' @docker image inspect $(OBJECTSTORE_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
@@ -81,7 +81,7 @@ REDIS_PROVISIONER_IMAGE ?= mesh-provision-redis:$(VERSION)
REDIS_PROVISIONER_DEV_TAG ?= mesh-provision-redis:development REDIS_PROVISIONER_DEV_TAG ?= mesh-provision-redis:development
redis-provisioner-image: redis-provisioner-image:
docker build -f examples/redis-provisioner/Dockerfile \ docker build --build-arg GO_BASE=$(GO_BASE) -f examples/redis-provisioner/Dockerfile \
-t $(REDIS_PROVISIONER_IMAGE) -t $(REDIS_PROVISIONER_DEV_TAG) . -t $(REDIS_PROVISIONER_IMAGE) -t $(REDIS_PROVISIONER_DEV_TAG) .
@echo @echo
@docker image inspect $(REDIS_PROVISIONER_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes' @docker image inspect $(REDIS_PROVISIONER_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
@@ -91,7 +91,7 @@ PROXY_IMAGE ?= mesh-route-proxy:$(VERSION)
PROXY_DEV_TAG ?= mesh-route-proxy:development PROXY_DEV_TAG ?= mesh-route-proxy:development
proxy-image: proxy-image:
docker build -f examples/route-proxy/Dockerfile -t $(PROXY_IMAGE) -t $(PROXY_DEV_TAG) . docker build --build-arg GO_BASE=$(GO_BASE) -f examples/route-proxy/Dockerfile -t $(PROXY_IMAGE) -t $(PROXY_DEV_TAG) .
@echo @echo
@docker image inspect $(PROXY_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes' @docker image inspect $(PROXY_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
+44 -12
View File
@@ -137,7 +137,12 @@ func takeWorkFrom(credential Credential, on string) (link.BuildMachine, error) {
if err != nil { if err != nil {
return nil, err return nil, err
} }
return link.MachineOverNATS(js, on), nil // **The seat this machine serves is the one its credential claims** (novox/hq ADR 0190, the
// handover): the mesh issues a build machine's credential naming the seat its module claims,
// and one binary serves the old role as `builder` and the new as `build-agent` from that alone.
seat := link.BuildSeatClaimed(credential.seatsClaimed())
fmt.Fprintf(os.Stderr, "taking build work as a holder of %s\n", seat)
return link.MachineOverNATSOn(js, on, seat), nil
} }
// answer does one build and says what happened, whichever way it went. // answer does one build and says what happened, whichever way it went.
@@ -148,20 +153,34 @@ func answer(ctx context.Context, publisher builder.Publisher, on, workspace stri
// it either finishes or fails is indistinguishable from one that never arrived — which cost a long // it either finishes or fails is indistinguishable from one that never arrived — which cost a long
// diagnosis against a running mesh, chasing "the handler never fired" when the truth was only that // diagnosis against a running mesh, chasing "the handler never fired" when the truth was only that
// the handler said nothing until the end. // the handler said nothing until the end.
fmt.Fprintf(os.Stderr, "a build request arrived for %s\n", request.Repository) fmt.Fprintf(os.Stderr, "a build request arrived for %s (%s)\n", request.Repository, request.ID)
// **Everything a build says goes two ways**: to stderr, as always, and onto the bus as the
// role's own events under the build's id (novox/hq ADR 0157) — so whoever asked, and anybody
// watching, reads the same lines this container's log holds, live, and after the fact from the
// stream. Said first, before anything runs, so a build that hangs is one that visibly started.
say := func(step, message string) {
fmt.Fprintf(os.Stderr, " [%s] %s\n", step, message)
work.Say(step, message)
}
builder.Said = say
defer func() { builder.Said = nil }()
if err := work.Began(ctx); err != nil {
fmt.Fprintf(os.Stderr, "cannot say a build started: %v\n", err)
}
result := link.BuildResult{ result := link.BuildResult{
ID: request.ID, Repository: request.Repository, Path: request.Path, ID: request.ID, Repository: request.Repository, Path: request.Path,
Ref: request.Ref, On: on, Ref: request.Ref, On: on, Source: request.Source,
} }
fmt.Fprintf(os.Stderr, "building %s", request.Repository) what := "building " + request.Repository
if request.Path != "" { if request.Path != "" {
fmt.Fprintf(os.Stderr, " at %s", request.Path) what += " at " + request.Path
} }
if request.Ref != "" { if request.Ref != "" {
fmt.Fprintf(os.Stderr, " at %s", request.Ref) what += " on " + request.Ref
} }
fmt.Fprintln(os.Stderr) say("build", what)
npmrc, err := packagesFrom() npmrc, err := packagesFrom()
var built builder.Result var built builder.Result
@@ -171,15 +190,13 @@ func answer(ctx context.Context, publisher builder.Publisher, on, workspace stri
// after a clone that then fails at npm ci. // after a clone that then fails at npm ci.
built, err = builder.Build(ctx, builder.Command, publisher, built, err = builder.Build(ctx, builder.Command, publisher,
request.Repository, request.Path, request.Ref, workspace, request.Held, npmrc, request.Repository, request.Path, request.Ref, workspace, request.Held, npmrc,
forgeFrom(), func(step, message string) { forgeFrom(), say, request.Seats)
fmt.Fprintf(os.Stderr, " [%s] %s\n", step, message)
}, request.Seats)
} }
if err != nil { if err != nil {
// A failure is a result. A build that fails and says nothing is indistinguishable from a // A failure is a result. A build that fails and says nothing is indistinguishable from a
// builder that is not running, and those want completely different responses. // builder that is not running, and those want completely different responses.
result.Failed = err.Error() result.Failed = err.Error()
fmt.Fprintf(os.Stderr, " failed: %v\n", err) say("failed", err.Error())
} else { } else {
manifest, marshalErr := json.Marshal(built.Manifest) manifest, marshalErr := json.Marshal(built.Manifest)
if marshalErr != nil { if marshalErr != nil {
@@ -196,7 +213,7 @@ func answer(ctx context.Context, publisher builder.Publisher, on, workspace stri
for _, r := range built.Read { for _, r := range built.Read {
result.Read = append(result.Read, link.ReadRepository{Repository: r.Repository, Ref: r.Ref}) result.Read = append(result.Read, link.ReadRepository{Repository: r.Repository, Ref: r.Ref})
} }
fmt.Fprintf(os.Stderr, " built %s from %s\n", built.Manifest.Module, short(built.Commit)) say("built", built.Manifest.Module+" from "+short(built.Commit))
} }
} }
@@ -441,6 +458,21 @@ type Credential struct {
// as two fields and this machine joins them once, here, to dial. // as two fields and this machine joins them once, here, to dial.
User string `json:"user,omitempty"` User string `json:"user,omitempty"`
Password string `json:"password,omitempty"` Password string `json:"password,omitempty"`
// Claims are the seats the module this credential was issued for claims, as the mesh writes
// them beside the credential (novox/hq ADR 0159). The first is the build role this machine
// serves; a credential naming none is from before claims travelled in it.
Claims []struct {
Seat string `json:"seat"`
} `json:"claims,omitempty"`
}
// seatsClaimed is the seats the credential names, in order.
func (c Credential) seatsClaimed() []string {
out := make([]string, 0, len(c.Claims))
for _, claim := range c.Claims {
out = append(out, claim.Seat)
}
return out
} }
// onTheNewBus is whether a credential is for the bus being built: its address says so, and the // onTheNewBus is whether a credential is for the bus being built: its address says so, and the
+27
View File
@@ -0,0 +1,27 @@
package main
import (
"encoding/json"
"testing"
"github.com/novox/mesh-controller/internal/link"
)
// The seat a build machine serves comes from its credential (novox/hq ADR 0190 handover).
func TestTheCredentialSaysWhichBuildRoleThisMachineServes(t *testing.T) {
var held Credential
if err := json.Unmarshal([]byte(`{"url":"nats://bus:4222","user":"anchor.builder","password":"x",
"claims":[{"seat":"mesh-build-machine","scope":"mesh","serves":[]}]}`), &held); err != nil {
t.Fatal(err)
}
if got := link.BuildSeatClaimed(held.seatsClaimed()); got != "mesh-build-machine" {
t.Errorf("the old builder's credential serves %q", got)
}
var bare Credential
if err := json.Unmarshal([]byte(`{"url":"nats://bus:4222","user":"anchor.build-agent","password":"x"}`), &bare); err != nil {
t.Fatal(err)
}
if got := link.BuildSeatClaimed(bare.seatsClaimed()); got != link.TheBuildMachine {
t.Errorf("a credential without claims serves %q, want %s", got, link.TheBuildMachine)
}
}
+38
View File
@@ -3,6 +3,7 @@ package main
import ( import (
"context" "context"
"fmt" "fmt"
"github.com/novox/mesh-controller/internal/broker"
"sort" "sort"
"strings" "strings"
@@ -67,6 +68,13 @@ func assign(ctx context.Context, open *stores, node, module string) (string, err
for _, line := range settled { for _, line := range settled {
said += "\n " + line said += "\n " + line
} }
// Its bus credential, in the same act (novox/hq issue 203): an assignment pushed before its
// credential exists delivers a process that cannot authenticate and crash-loops until somebody
// runs a second verb and a second push. Issued here when the module speaks on the bus and has
// no credential yet; kept when it has one, so re-assigning rotates nothing.
if line := issueOnAssign(ctx, open, node, module); line != "" {
said += "\n " + line
}
plan, _, err := planFor(ctx, open, node) plan, _, err := planFor(ctx, open, node)
if err != nil { if err != nil {
// Kept, and still refused. Both halves are the answer, and the rest of the mesh is still // Kept, and still refused. Both halves are the answer, and the rest of the mesh is still
@@ -152,3 +160,33 @@ func blockedElsewhere(ctx context.Context, open *stores, except string) string {
out.WriteString("\nThis may or may not be what just changed — it is what is true now.") out.WriteString("\nThis may or may not be what just changed — it is what is true now.")
return out.String() return out.String()
} }
// issueOnAssign gives a newly assigned module its bus credential, the way `module issue` does, and
// says what it did in one line. Nothing for a module that declares no broker secret; nothing for one
// whose user is already minted (a credential is rotated on purpose, never by re-assigning); and when
// the bus cannot be reached from here, the line names the verb and the push that would refuse the
// module until it is run — never a silent placeholder (novox/hq issue 203).
func issueOnAssign(ctx context.Context, open *stores, node, module string) string {
inv := open.inventory
shelf, err := inv.Catalogue(ctx)
if err != nil {
return ""
}
m, known := shelf[module]
if !known || mayIssue(m) != nil {
return ""
}
user := broker.Principal{Kind: broker.KindModule, Node: node, Module: module}.Username()
if _, minted, err := inv.BusUserHash(ctx, user); err != nil || minted {
return ""
}
busAddress, err := broker.BusAddress()
if err == nil {
err = issueOnTheNewBus(ctx, inv, m, node, busAddress)
}
if err != nil {
return fmt.Sprintf("its bus credential is not issued (%v): `module issue %s --node %s` first — "+
"`push %s` refuses to send %s until it is", err, module, node, node, module)
}
return fmt.Sprintf("its bus credential is issued and sealed to %s, and arrives with the push", node)
}
+6
View File
@@ -175,6 +175,12 @@ func TestTheControlPlaneIsToldWhereTheNodePutTheStoreAndTheBroker(t *testing.T)
Guards: []int{15672}, Guards: []int{15672},
Resources: []map[string]any{{"id": "server", "type": "container", "name": "mesh-broker", Resources: []map[string]any{{"id": "server", "type": "container", "name": "mesh-broker",
"ports": []any{"5671:5671", "5672:5672", "127.0.0.1:15672:15672"}, "image": "mq@" + aDigest}}}) "ports": []any{"5671:5671", "5672:5672", "127.0.0.1:15672:15672"}, "image": "mq@" + aDigest}}})
// The control plane's own bus user is the installer's, seeded at genesis before the controller
// runs (SeedBusUser); without it a push now refuses the credential nobody issued (issue 203).
if err := open.inventory.SeedBusUser(ctx, inventory.BusUser{Username: "anchor.mesh-controller",
Kind: inventory.BusController, Node: "anchor", Module: "mesh-controller"}, "bootstrap"); err != nil {
t.Fatal(err)
}
if _, err := assign(ctx, open, "anchor", "mesh-controller"); err != nil { if _, err := assign(ctx, open, "anchor", "mesh-controller"); err != nil {
t.Fatal(err) t.Fatal(err)
} }
+116 -9
View File
@@ -91,11 +91,26 @@ func reportsReaching(t *testing.T, open *stores, reachable []link.Reach, held ..
if _, err := (link.Enrolment{Inventory: open.inventory}).Heard(ctx, link.Report{ if _, err := (link.Enrolment{Inventory: open.inventory}).Heard(ctx, link.Report{
Node: "anchor", Applied: []string{"hello-web.x"}, Declared: digestOf(body), Node: "anchor", Applied: []string{"hello-web.x"}, Declared: digestOf(body),
Firewall: "ufw", Held: held, Reachable: reachable, Firewall: "ufw", Held: held, Reachable: reachable,
// A machine says which of its links face outside on every apply (novox/hq ADR 0140), and a
// filter is not sent to one that has not. The anchor reports one, as a real host does; this
// fixture lacked it from 2026-09-28 and nothing ran the test (issue 177).
Outward: []string{"eth0"},
// And what filters it (ADR 0168): its front end, the runtime's own, and a chain a
// predecessor left in the runtime's user chain.
Filters: anchorFilters,
}); err != nil { }); err != nil {
t.Fatal(err) t.Fatal(err)
} }
} }
// anchorFilters is what the adopted anchor says filters it: ufw's chains, the runtime's, and a
// predecessor's chain the mesh did not write.
var anchorFilters = []link.Filter{
{Where: "table ip filter, chain ufw-reject-input", Owner: "found-firewall", Refuses: "reject"},
{Where: "table ip filter, chain DOCKER", Owner: "runtime", Refuses: `iifname != "docker0" oifname "docker0" drop`},
{Where: "table ip filter, chain DOCKER-USER", Owner: "other", Refuses: `iifname "eth0" tcp dport 6000 drop`},
}
var ( var (
heldContainer = link.Held{ID: "hello-web.server", Module: "hello-web", Kind: "container", heldContainer = link.Held{ID: "hello-web.server", Module: "hello-web", Kind: "container",
Target: "hello-web", Since: time.Now()} Target: "hello-web", Since: time.Now()}
@@ -105,10 +120,10 @@ var (
func TestTakingAModuleNotOnTheNodeIsRefused(t *testing.T) { func TestTakingAModuleNotOnTheNodeIsRefused(t *testing.T) {
open, _ := anAdoptedAnchor(t) open, _ := anAdoptedAnchor(t)
if _, err := take(t.Context(), open, "anchor", "nftables"); !errors.Is(err, inventory.ErrNotAssigned) { if _, err := take(t.Context(), open, "anchor", "nftables", takeOptions{Yes: true}); !errors.Is(err, inventory.ErrNotAssigned) {
t.Fatalf("taking an unassigned module gave %v", err) t.Fatalf("taking an unassigned module gave %v", err)
} }
if _, err := take(t.Context(), open, "laptop", "network"); !errors.Is(err, inventory.ErrNotAdopted) { if _, err := take(t.Context(), open, "laptop", "network", takeOptions{Yes: true}); !errors.Is(err, inventory.ErrNotAdopted) {
t.Fatalf("taking on a converged node gave %v", err) t.Fatalf("taking on a converged node gave %v", err)
} }
} }
@@ -139,7 +154,24 @@ func TestTheFlipIsRefusedWhileAFoundContainerIsHeld(t *testing.T) {
func TestTakingNamesWhatItReplaces(t *testing.T) { func TestTakingNamesWhatItReplaces(t *testing.T) {
open, _ := anAdoptedAnchor(t) open, _ := anAdoptedAnchor(t)
reportsHolding(t, open, heldContainer, heldFile) reportsHolding(t, open, heldContainer, heldFile)
said, err := take(t.Context(), open, "anchor", "hello-web") ctx := t.Context()
// The machine holds something for the module, so the take acts on the preview the operator
// saw and names its digest (novox/hq ADR 0163).
preview, err := take(ctx, open, "anchor", "hello-web", takeOptions{})
if err != nil {
t.Fatal(err)
}
saw := takeDigestIn(t, preview)
if !strings.Contains(preview, "nothing taken; `take anchor hello-web --yes "+saw+"`") {
t.Fatalf("the preview does not say how to act on it:\n%s", preview)
}
if taken, _ := open.inventory.Taken(ctx, "anchor"); len(taken) != 0 {
t.Fatal("the preview took something")
}
if _, err := take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true}); err == nil || !strings.Contains(err.Error(), "name its digest") {
t.Fatalf("--yes without the digest was not refused: %v", err)
}
said, err := take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true, Digest: saw})
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -149,10 +181,71 @@ func TestTakingNamesWhatItReplaces(t *testing.T) {
} }
} }
// takeDigestIn is the digest a take's preview printed.
func takeDigestIn(t *testing.T, preview string) string {
t.Helper()
for _, line := range strings.Split(preview, "\n") {
if fields := strings.Fields(line); len(fields) == 2 && fields[0] == "preview" {
return fields[1]
}
}
t.Fatalf("the preview printed no digest:\n%s", preview)
return ""
}
// A take acts on the preview the operator saw, and on an account of the machine that is still the
// machine: a changed preview and a stale account refuse (novox/hq ADR 0163, rule 1).
func TestATakeIsRefusedOnAChangedPreviewOrAStaleAccount(t *testing.T) {
open, _ := anAdoptedAnchor(t)
ctx := t.Context()
reportsHolding(t, open, heldContainer, heldFile)
preview, err := take(ctx, open, "anchor", "hello-web", takeOptions{})
if err != nil {
t.Fatal(err)
}
saw := takeDigestIn(t, preview)
// The machine reports again, and what it holds has changed: the found container now carries
// facts the preview never showed.
changed := heldContainer
changed.Facts = map[string]any{"image": "hello:2", "declared_image": "registry.example/hello"}
reportsHolding(t, open, changed, heldFile)
_, err = take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true, Digest: saw})
if err == nil || !strings.Contains(err.Error(), "has changed since preview "+saw) {
t.Fatalf("a changed preview was acted on: %v", err)
}
if taken, _ := open.inventory.Taken(ctx, "anchor"); len(taken) != 0 {
t.Fatal("a refused take took something")
}
// And an account older than the flip allows.
preview, err = take(ctx, open, "anchor", "hello-web", takeOptions{})
if err != nil {
t.Fatal(err)
}
saw = takeDigestIn(t, preview)
saved := reportFreshFor
reportFreshFor = -time.Second
defer func() { reportFreshFor = saved }()
_, err = take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true, Digest: saw})
if err == nil || !strings.Contains(err.Error(), "a take acts only on an account newer than") {
t.Fatalf("a stale account was acted on: %v", err)
}
reportFreshFor = saved
if _, err := take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true, Digest: saw}); err != nil {
t.Fatal(err)
}
// A module the machine holds nothing for has nothing to compare: --yes alone suffices.
if _, err := take(ctx, open, "anchor", "notes", takeOptions{Yes: true}); err == nil {
// notes holds a file, so this one needs the digest too.
t.Fatal("notes holds a found file and was taken without a digest")
}
}
func TestConvergingPreviewsThenChangesAndAdoptingKeepsWhatWasTaken(t *testing.T) { func TestConvergingPreviewsThenChangesAndAdoptingKeepsWhatWasTaken(t *testing.T) {
open, sent := anAdoptedAnchor(t) open, sent := anAdoptedAnchor(t)
ctx := t.Context() ctx := t.Context()
if _, err := take(ctx, open, "anchor", "hello-web"); err != nil { if _, err := take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true}); err != nil {
t.Fatal(err) t.Fatal(err)
} }
reportsHolding(t, open, heldFile) reportsHolding(t, open, heldFile)
@@ -182,6 +275,20 @@ func TestConvergingPreviewsThenChangesAndAdoptingKeepsWhatWasTaken(t *testing.T)
if strings.Contains(preview, "15672") { if strings.Contains(preview, "15672") {
t.Errorf("a loopback listener is in the preview:\n%s", preview) t.Errorf("a loopback listener is in the preview:\n%s", preview)
} }
// What filters the machine now, and the fate of each (novox/hq ADR 0168): the predecessor's
// chain is named as not the mesh's and left, so the reader knows before the flip.
for _, want := range []string{
"table ip filter, chain DOCKER-USER",
"NOT THE MESH'S; left in force",
`iifname "eth0" tcp dport 6000 drop`,
"table ip filter, chain ufw-reject-input",
"the found firewall's; retired with it",
"the container runtime's own; left",
} {
if !strings.Contains(preview, want) {
t.Errorf("the preview does not say %q:\n%s", want, preview)
}
}
for _, line := range strings.Split(preview, "\n") { for _, line := range strings.Split(preview, "\n") {
if strings.Contains(line, "5000") && !strings.Contains(line, "WILL CLOSE") { if strings.Contains(line, "5000") && !strings.Contains(line, "WILL CLOSE") {
t.Errorf("an undeclared published port is not said to close: %s", line) t.Errorf("an undeclared published port is not said to close: %s", line)
@@ -326,7 +433,7 @@ func digestIn(t *testing.T, preview string) string {
func TestTheFlipActsOnlyOnThePreviewTheOperatorSaw(t *testing.T) { func TestTheFlipActsOnlyOnThePreviewTheOperatorSaw(t *testing.T) {
open, sent := anAdoptedAnchor(t) open, sent := anAdoptedAnchor(t)
ctx := t.Context() ctx := t.Context()
if _, err := take(ctx, open, "anchor", "hello-web"); err != nil { if _, err := take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true}); err != nil {
t.Fatal(err) t.Fatal(err)
} }
reportsHolding(t, open, heldFile) reportsHolding(t, open, heldFile)
@@ -392,7 +499,7 @@ func TestTheFlipActsOnlyOnThePreviewTheOperatorSaw(t *testing.T) {
func TestTheFlipHoldsTheNodeWhileItSends(t *testing.T) { func TestTheFlipHoldsTheNodeWhileItSends(t *testing.T) {
open, _ := anAdoptedAnchor(t) open, _ := anAdoptedAnchor(t)
ctx := t.Context() ctx := t.Context()
if _, err := take(ctx, open, "anchor", "hello-web"); err != nil { if _, err := take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true}); err != nil {
t.Fatal(err) t.Fatal(err)
} }
reportsHolding(t, open, heldFile) reportsHolding(t, open, heldFile)
@@ -437,7 +544,7 @@ func TestTheFlipHoldsTheNodeWhileItSends(t *testing.T) {
func TestThePreviewNamesEveryHeldKind(t *testing.T) { func TestThePreviewNamesEveryHeldKind(t *testing.T) {
open, _ := anAdoptedAnchor(t) open, _ := anAdoptedAnchor(t)
ctx := t.Context() ctx := t.Context()
if _, err := take(ctx, open, "anchor", "hello-web"); err != nil { if _, err := take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true}); err != nil {
t.Fatal(err) t.Fatal(err)
} }
since := time.Now() since := time.Now()
@@ -480,7 +587,7 @@ func TestThePreviewNamesEveryHeldKind(t *testing.T) {
func TestTheFlipIsRefusedOnAnAccountNamingNothingReachable(t *testing.T) { func TestTheFlipIsRefusedOnAnAccountNamingNothingReachable(t *testing.T) {
open, sent := anAdoptedAnchor(t) open, sent := anAdoptedAnchor(t)
ctx := t.Context() ctx := t.Context()
if _, err := take(ctx, open, "anchor", "hello-web"); err != nil { if _, err := take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true}); err != nil {
t.Fatal(err) t.Fatal(err)
} }
// Only a loopback listener: nothing off the machine, which is the same silence. // Only a loopback listener: nothing off the machine, which is the same silence.
@@ -508,7 +615,7 @@ func TestTheFlipIsRefusedOnAnAccountNamingNothingReachable(t *testing.T) {
func TestAssigningWaitsForWhateverIsConvergingTheNode(t *testing.T) { func TestAssigningWaitsForWhateverIsConvergingTheNode(t *testing.T) {
open, _ := anAdoptedAnchor(t) open, _ := anAdoptedAnchor(t)
ctx := t.Context() ctx := t.Context()
if _, err := take(ctx, open, "anchor", "hello-web"); err != nil { if _, err := take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true}); err != nil {
t.Fatal(err) t.Fatal(err)
} }
reportsHolding(t, open, heldFile) reportsHolding(t, open, heldFile)
+484 -19
View File
@@ -24,6 +24,15 @@ import (
func showMode(ctx context.Context, inv *inventory.Inventory, node inventory.Node) error { func showMode(ctx context.Context, inv *inventory.Inventory, node inventory.Node) error {
if !node.Adopted { if !node.Adopted {
fmt.Printf(" mode converged\n") fmt.Printf(" mode converged\n")
// A converged machine holds nothing, and can still run what nobody asked for
// (novox/hq ADR 0163): what it reports as strays is said whatever its mode.
if said, err := inv.AdoptionOf(ctx, node.Name); err == nil && len(said.Strays) > 0 {
showStrays(said.Strays)
}
// And what filters it, truthfully (novox/hq ADR 0168): the mesh alone, or not.
if filtering, err := inv.FilteringOf(ctx, node.Name); err == nil {
showFiltering(filtering, false)
}
return nil return nil
} }
fmt.Printf(" mode adopted since %s\n", fmt.Printf(" mode adopted since %s\n",
@@ -62,11 +71,81 @@ func showMode(ctx context.Context, inv *inventory.Inventory, node inventory.Node
if h.Kept != "" { if h.Kept != "" {
fmt.Printf(" %-17s original kept at %s\n", "", h.Kept) fmt.Printf(" %-17s original kept at %s\n", "", h.Kept)
} }
for _, f := range comparisonLines(h) {
fmt.Printf(" %-17s %s\n", "", f)
}
}
showStrays(said.Strays)
if filtering, err := inv.FilteringOf(ctx, node.Name); err == nil {
showFiltering(filtering, true)
} }
fmt.Printf(" as of %s\n", said.At.Local().Format(time.DateTime)) fmt.Printf(" as of %s\n", said.At.Local().Format(time.DateTime))
return nil return nil
} }
// showFiltering says what filters a machine, with owners (novox/hq ADR 0168), and for a converged
// machine the state of the firewall it was found with. A machine that has not said is not said to
// be filtered by anything.
func showFiltering(f inventory.Filtering, adopted bool) {
if len(f.Filters) == 0 && f.FoundFirewall == nil {
return
}
if fw := f.FoundFirewall; fw != nil && !adopted {
switch {
case fw.Active:
fmt.Printf(" found firewall %s is ACTIVE on this converged machine; the next apply retires it again\n", fw.Kind)
case fw.RetiredBy == "removed":
fmt.Printf(" found firewall %s, removed: the mesh's filter is what filters this machine (novox/hq ADR 0180)\n", fw.Kind)
case fw.RetiredBy == inventory.FilterMesh || fw.RetiredBy == "mesh":
fmt.Printf(" found firewall %s, retired by the mesh; its configuration stays on disk\n", fw.Kind)
case fw.RetiredBy != "":
fmt.Printf(" found firewall %s, found inactive — not by the mesh\n", fw.Kind)
default:
fmt.Printf(" found firewall %s, inactive\n", fw.Kind)
}
}
if len(f.Filters) == 0 {
return
}
if f.Alone() {
fmt.Printf(" filtered by the mesh alone (%s)\n", filterSummary(f.Filters))
return
}
fmt.Printf(" filtered by NOT the mesh alone: %d rule set(s) the mesh did not write refuse traffic here\n", len(f.Others()))
for _, x := range f.Filters {
if x.Owner == inventory.FilterOther || x.Owner == inventory.FilterFoundFirewall {
fmt.Printf(" %-17s %s — %s: %s\n", "", x.Where, x.Owner, x.Refuses)
}
}
fmt.Printf(" %-17s and its own: %s\n", "", filterSummary(f.Filters))
}
// filterSummary counts a machine's filters by owner: "mesh 2, runtime 3, ban 1".
func filterSummary(filters []inventory.Filter) string {
counts := map[string]int{}
for _, x := range filters {
counts[x.Owner]++
}
var parts []string
for _, owner := range []string{inventory.FilterMesh, inventory.FilterRuntime, inventory.FilterBan, inventory.FilterFoundFirewall, inventory.FilterOther} {
if n := counts[owner]; n > 0 {
parts = append(parts, fmt.Sprintf("%s %d", owner, n))
}
}
return strings.Join(parts, ", ")
}
// showStrays says what a machine runs that the mesh neither wrote nor holds (ADR 0163).
func showStrays(strays []inventory.Stray) {
if len(strays) == 0 {
return
}
fmt.Printf(" strays %d container(s) the mesh neither wrote nor holds:\n", len(strays))
for _, s := range strays {
fmt.Printf(" %-17s %s (%s)\n", "", s.Name, s.Detail)
}
}
// showTunnel is the node show lines about the tunnel an adopted node found and carried (novox/hq // showTunnel is the node show lines about the tunnel an adopted node found and carried (novox/hq
// ADR 0105): what it presented at enrolment, and what it last said about taking it over. // ADR 0105): what it presented at enrolment, and what it last said about taking it over.
func showTunnel(ctx context.Context, inv *inventory.Inventory, name string) error { func showTunnel(ctx context.Context, inv *inventory.Inventory, name string) error {
@@ -136,7 +215,28 @@ const DefaultFilter = "nftables"
// take is a module's cutover on an adopted node: the operator's act, done when that module's data // take is a module's cutover on an adopted node: the operator's act, done when that module's data
// has moved. From the next push its resources converge there like any other, replacing what the // has moved. From the next push its resources converge there like any other, replacing what the
// node found and holds for it. // node found and holds for it.
func take(ctx context.Context, open *stores, node, module string) (string, error) { //
// **Previewed, and the preview is a comparison** (novox/hq ADR 0163): for every held thing the
// module would replace, what runs beside what the module declares, and the difference; the
// module's secrets on the machine and where each came from; its settings on the machine. Without
// --yes the comparison is printed and nothing changes. `--yes <digest>` cuts over exactly what was
// previewed, the way the flip is confirmed: the preview ends with a digest of what it said, and a
// take naming an older one, or acting on an account of the machine older than the flip allows, is
// refused. A module the machine holds nothing for has nothing to compare, and `--yes` suffices.
// takeOptions is what a take was told about the differences it may pass (novox/hq ADR 0163).
type takeOptions struct {
Yes bool
// Digest is the preview's, named with --yes; required whenever the machine holds something
// for the module.
Digest string
Downgrade bool
Replace map[string]bool
// Mint names the secrets the service shall take a new value for, although the mesh minted
// one and the service already has its own (rule 2).
Mint map[string]bool
}
func take(ctx context.Context, open *stores, node, module string, opts takeOptions) (string, error) {
inv := open.inventory inv := open.inventory
assigned, err := inv.Assigned(ctx, node) assigned, err := inv.Assigned(ctx, node)
if err != nil { if err != nil {
@@ -150,27 +250,337 @@ func take(ctx context.Context, open *stores, node, module string) (string, error
} }
} }
} }
// The comparison first (novox/hq ADR 0163): every held thing the module would replace, beside
// what the module declares, and the differences that refuse unless named.
c, err := comparisonFor(ctx, open, node, module)
if err != nil {
return "", err
}
preview, refusals, saw := comparisonOf(module, c, opts)
if len(refusals) > 0 {
return "", fmt.Errorf("taking %s on %s is refused:\n %s\n%s", module, node,
strings.Join(refusals, "\n "), preview)
}
holds := len(heldOf(c.reported, module)) > 0
if holds {
preview += "\n preview " + saw
}
if !opts.Yes {
if !holds {
return preview + fmt.Sprintf("\nnothing taken; `take %s %s --yes` declares it as the mesh's own", node, module), nil
}
return preview + fmt.Sprintf("\nnothing taken; `take %s %s --yes %s` cuts it over as previewed", node, module, saw), nil
}
if holds {
// The take acts on the preview the operator saw, and on an account of the machine that
// is still the machine: the same two refusals the flip makes.
if age := time.Since(c.reported.At); age > reportFreshFor {
return preview, fmt.Errorf("%s last said what it holds %s ago, and a take acts only on "+
"an account newer than %s: run `push %s --wait 2m`, then preview again",
node, age.Round(time.Second), reportFreshFor, node)
}
if opts.Digest == "" {
return preview, fmt.Errorf("taking %s on %s acts on the preview you saw: name its digest, "+
"`take %s %s --yes %s`, once you have read it", module, node, node, module, saw)
}
if opts.Digest != saw {
return preview, fmt.Errorf("what taking %s on %s would replace has changed since preview %s "+
"(it is now %s): read the preview above, and run `take %s %s --yes %s` if it is "+
"what you want", module, node, opts.Digest, saw, node, module, saw)
}
}
if err := inv.Take(ctx, node, module); err != nil { if err := inv.Take(ctx, node, module); err != nil {
return "", err return "", err
} }
said := fmt.Sprintf("%s is taken on %s", module, node) said := fmt.Sprintf("%s is taken on %s", module, node)
reported, err := inv.AdoptionOf(ctx, node) if holds {
if err != nil { said += "; the next push replaces what the node found and holds for it:\n" + preview
return "", err
}
var replaces []string
for _, h := range reported.Held {
if h.Module == module {
replaces = append(replaces, " "+heldLine(h))
}
}
if len(replaces) > 0 {
said += "; the next push replaces what the node found and holds for it:\n" +
strings.Join(replaces, "\n")
} }
return said + fmt.Sprintf("\n run `push %s` to cut it over", node), nil return said + fmt.Sprintf("\n run `push %s` to cut it over", node), nil
} }
// comparison is everything a take puts beside what the module declares: the machine's account of
// what it holds and what is reachable on it, the module's secrets on the machine, its settings
// there, and which found networks a setting keeps for each of its containers (by held id).
type comparison struct {
reported inventory.Adoption
secrets []inventory.SecretState
layers []catalogue.Layer
keeps map[string][]string
// settingsRefused is why the module's settings cannot compose with its definition, when
// they cannot — the module would be left out of the declaration (rule 6).
settingsRefused string
}
func comparisonFor(ctx context.Context, open *stores, node, module string) (comparison, error) {
inv := open.inventory
var c comparison
var err error
if c.reported, err = inv.AdoptionOf(ctx, node); err != nil {
return c, err
}
if c.secrets, err = inv.SecretsOf(ctx, node, module); err != nil {
return c, err
}
if c.layers, err = inv.SettingsFor(ctx, node, module); err != nil {
return c, err
}
shelf, err := inv.Catalogue(ctx)
if err != nil {
return c, err
}
if m, known := shelf[module]; known && len(c.layers) > 0 {
if err := catalogue.JudgeSettings(m, c.layers, true); err != nil {
c.settingsRefused = err.Error()
}
if kept, err := catalogue.KeptNetworks(m, c.layers, true); err == nil && len(kept) > 0 {
c.keeps = map[string][]string{}
for id, networks := range kept {
c.keeps[module+"."+id] = networks
}
}
}
return c, nil
}
// heldOf is what a node holds for one module.
func heldOf(reported inventory.Adoption, module string) []inventory.Held {
var out []inventory.Held
for _, h := range reported.Held {
if h.Module == module {
out = append(out, h)
}
}
return out
}
// comparisonOf is a take's preview: for every held thing of the module, what runs beside what the
// module declares; its secrets and its settings on the machine; and the refusals the differences
// earn unless the take named them (novox/hq ADR 0163): an image older than the one running, a
// declared file that differs from the found one, a secret the mesh minted for a service whose data
// was found. A narrowed port and a shared network are said and not refused. The digest is of what
// the preview says, so anything in it changing changes the digest.
func comparisonOf(module string, c comparison, opts takeOptions) (preview string, refusals []string, digest string) {
var b strings.Builder
held := heldOf(c.reported, module)
foundData := false
for _, h := range held {
if h.Kind == "container" || h.Kind == "directory" {
foundData = true
}
fmt.Fprintf(&b, " %s", heldLine(h))
if h.Kept != "" {
fmt.Fprintf(&b, ", original kept at %s", h.Kept)
}
b.WriteString("\n")
for _, line := range comparisonLinesWith(h, c.keeps[h.ID], c.reported) {
fmt.Fprintf(&b, " %s\n", line)
}
f := factsOf(h)
if f.downgrade && !opts.Downgrade {
refusals = append(refusals, fmt.Sprintf("%s: the module's image (%s, made %s) is older than the one running (%s, made %s) — "+
"a service that migrated its data forward may not start on it; `--downgrade` to take it anyway",
h.Target, f.declaredImage, day(f.declaredCreated), f.image, day(f.imageCreated)))
}
if f.differs && !opts.Replace[h.Target] && !opts.Replace["*"] {
refusals = append(refusals, fmt.Sprintf("%s: the module's content differs from the file found; the lines above "+
"marked - are lost by taking it; `--replace %s` to replace it anyway, or declare the file partially",
h.Target, h.Target))
}
}
// The module's secrets on the machine (rule 2 and 3): a service whose data was found already
// has a value for each, so one the mesh minted and nobody accepted refuses unless --mint says
// the service shall take a new one.
for _, sec := range c.secrets {
name := sec.Name
if sec.Local != "" {
name += " (" + sec.Local + ")"
}
what := "own secret"
accept := fmt.Sprintf("`secret accept <node> %s %s`", module, sec.Name)
if !sec.Own() {
what = "secret from " + sec.Provider
accept = fmt.Sprintf("`secret accept <node> %s %s --provider %s`", module, sec.Name, sec.Provider)
if sec.Local != "" {
accept = strings.TrimSuffix(accept, "`") + " --local " + sec.Local + "`"
}
}
switch {
case sec.Origin == inventory.OriginAccepted:
fmt.Fprintf(&b, " %s %s: accepted from a person, carried in as it is\n", what, name)
case opts.Mint[sec.Name]:
fmt.Fprintf(&b, " %s %s: minted by the mesh; the service takes the new value, as --mint said\n", what, name)
case foundData:
fmt.Fprintf(&b, " %s %s: MINTED by the mesh and not accepted — the running service already has one\n", what, name)
refusals = append(refusals, fmt.Sprintf("%s: the mesh minted a value and the service whose data was found "+
"already uses its own; %s carries the existing value in, or `--mint %s` says the service shall take "+
"the new one", name, accept, sec.Name))
default:
fmt.Fprintf(&b, " %s %s: minted by the mesh\n", what, name)
}
}
// And its settings on this machine, composed against its definition (rule 1, rule 6).
for _, layer := range c.layers {
keys := make([]string, 0, len(layer.Values))
for k := range layer.Values {
keys = append(keys, k)
}
sort.Strings(keys)
fmt.Fprintf(&b, " settings from %s: %s\n", layer.From, strings.Join(keys, ", "))
}
if c.settingsRefused != "" {
fmt.Fprintf(&b, " SETTINGS DO NOT COMPOSE with the module's definition, so the push leaves it out: %s\n", c.settingsRefused)
}
preview = strings.TrimRight(b.String(), "\n")
sum := sha256.Sum256([]byte(preview))
return preview, refusals, hex.EncodeToString(sum[:])[:12]
}
// facts is a held thing's facts as the preview reads them.
type facts struct {
image, imageCreated, declaredImage, declaredCreated string
downgrade, differs bool
networks map[string][]string
mounts, ports, declaredPorts, declaredVolumes []string
difference []string
}
func factsOf(h inventory.Held) facts {
var f facts
if h.Facts == nil {
return f
}
str := func(k string) string { s, _ := h.Facts[k].(string); return s }
list := func(k string) []string {
var out []string
if raw, ok := h.Facts[k].([]any); ok {
for _, x := range raw {
if s, ok := x.(string); ok {
out = append(out, s)
}
}
}
return out
}
f.image, f.imageCreated = str("image"), str("image_created")
f.declaredImage, f.declaredCreated = str("declared_image"), str("declared_image_created")
f.downgrade, _ = h.Facts["downgrade"].(bool)
f.differs, _ = h.Facts["differs"].(bool)
f.mounts, f.ports = list("mounts"), list("ports")
f.declaredPorts, f.declaredVolumes, f.difference = list("declared_ports"), list("declared_volumes"), list("difference")
if raw, ok := h.Facts["networks"].(map[string]any); ok {
f.networks = map[string][]string{}
for name, members := range raw {
var out []string
if ms, ok := members.([]any); ok {
for _, m := range ms {
if s, ok := m.(string); ok {
out = append(out, s)
}
}
}
f.networks[name] = out
}
}
return f
}
// comparisonLines says a held thing's facts the way a person weighs them.
func comparisonLines(h inventory.Held) []string {
return comparisonLinesWith(h, nil, inventory.Adoption{})
}
// comparisonLinesWith is comparisonLines knowing which found networks this machine's setting keeps
// for the container (rule 4) and what the machine reports reachable, so a published port's reach
// is said beside the port (rule 1).
func comparisonLinesWith(h inventory.Held, keeps []string, reported inventory.Adoption) []string {
f := factsOf(h)
var out []string
if f.image != "" || f.declaredImage != "" {
line := fmt.Sprintf("runs %s", orNone(f.image))
if f.imageCreated != "" {
line += " (made " + day(f.imageCreated) + ")"
}
line += "; the module declares " + orNone(f.declaredImage)
switch {
case f.declaredCreated != "":
line += " (made " + day(f.declaredCreated) + ")"
case f.declaredImage != "":
line += " (not on the machine yet, so its age is unknown)"
}
if f.downgrade {
line += " — DOWNGRADE"
}
out = append(out, line)
}
names := make([]string, 0, len(f.networks))
for n := range f.networks {
names = append(names, n)
}
sort.Strings(names)
for _, n := range names {
members := f.networks[n]
if len(members) == 0 {
continue
}
if slices.Contains(keeps, n) {
out = append(out, fmt.Sprintf("on the network %s with %s — kept by this machine's setting, so they still reach it by name once taken",
n, strings.Join(members, ", ")))
continue
}
out = append(out, fmt.Sprintf("on the network %s with %s, which may reach it by name and will not once it moves to the module's own network"+
" (`settings set %s --node <node>` with {%q: {<container>: [%q]}} keeps it)",
n, strings.Join(members, ", "), h.Module, catalogue.NetworksSetting, n))
}
for _, n := range keeps {
if _, found := f.networks[n]; !found {
out = append(out, fmt.Sprintf("keeps the network %s by this machine's setting, which the found container is not on", n))
}
}
if len(f.ports) > 0 || len(f.declaredPorts) > 0 {
out = append(out, fmt.Sprintf("publishes %s; the module declares %s",
orNone(strings.Join(f.ports, " ")), orNone(strings.Join(f.declaredPorts, " "))))
// How far each published port reaches now, as the machine reported it: the listener the
// runtime publishes for this container. The found firewall's and the guard's rules are
// not read; what they let through is said as what was reported reachable.
var reach []string
for _, r := range reported.Reachable {
if r.By == h.Target && r.Published {
reach = append(reach, fmt.Sprintf("%s:%d (%s, container port %d)", r.Address, r.Port, r.Protocol, r.ContainerPort))
}
}
switch {
case len(reach) > 0:
line := "reachable now at " + strings.Join(reach, ", ")
if reported.Firewall != "" && reported.Firewall != "none" {
line += ", behind the found firewall (" + reported.Firewall + "), whose rules are not read"
}
out = append(out, line)
case len(f.ports) > 0 && len(reported.Reachable) > 0:
out = append(out, "not reported reachable on the machine")
}
}
if len(f.mounts) > 0 || len(f.declaredVolumes) > 0 {
out = append(out, fmt.Sprintf("mounts %s; the module declares %s",
orNone(strings.Join(f.mounts, " ")), orNone(strings.Join(f.declaredVolumes, " "))))
}
if f.differs {
out = append(out, "the declared content differs from the file found (- lost, + new):")
for _, d := range f.difference {
out = append(out, " "+d)
}
}
return out
}
// day is a timestamp as a person reads it in a preview: its date.
func day(stamp string) string {
if len(stamp) >= 10 {
return stamp[:10]
}
return stamp
}
// reportFreshFor is how old a node's account of itself may be for the flip to act on it. A // reportFreshFor is how old a node's account of itself may be for the flip to act on it. A
// variable so a test can age a report without waiting. // variable so a test can age a report without waiting.
var reportFreshFor = 15 * time.Minute var reportFreshFor = 15 * time.Minute
@@ -310,7 +720,11 @@ func converge(ctx context.Context, open *stores, node string, yes bool, digest s
} }
derived := derivedFilter{rules: rules, foundation: with.Foundation, mesh: with.Mesh, derived := derivedFilter{rules: rules, foundation: with.Foundation, mesh: with.Mesh,
outward: plan.PublicDomain != "", outwardLinks: with.OutwardLinks} outward: plan.PublicDomain != "", outwardLinks: with.OutwardLinks}
preview, saw := previewOf(node, reported, derived, plan, taken, filter, runs[filter]) filtering, err := inv.FilteringOf(ctx, node)
if err != nil {
return "", err
}
preview, saw := previewOf(node, reported, filtering, derived, plan, taken, filter, runs[filter])
preview += "\n\n preview " + saw preview += "\n\n preview " + saw
if !yes { if !yes {
return preview + fmt.Sprintf("\n\nNothing has changed. Run `converge %s --yes %s` to do "+ return preview + fmt.Sprintf("\n\nNothing has changed. Run `converge %s --yes %s` to do "+
@@ -380,7 +794,7 @@ func converge(ctx context.Context, open *stores, node string, yes bool, digest s
// previewOf is what converging a node will change, before it changes it, and a short digest of // previewOf is what converging a node will change, before it changes it, and a short digest of
// what it said: every reachable thing and its fate, the modules the flip takes and the filter. The // what it said: every reachable thing and its fate, the modules the flip takes and the filter. The
// digest is what the flip is asked to act on, so it changes whenever any of those would. // digest is what the flip is asked to act on, so it changes whenever any of those would.
func previewOf(node string, reported inventory.Adoption, derived derivedFilter, func previewOf(node string, reported inventory.Adoption, filtering inventory.Filtering, derived derivedFilter,
plan catalogue.Resolution, taken []string, filter string, filterAssigned bool) (string, string) { plan catalogue.Resolution, taken []string, filter string, filterAssigned bool) (string, string) {
var said []string var said []string
var b strings.Builder var b strings.Builder
@@ -472,6 +886,30 @@ func previewOf(node string, reported inventory.Adoption, derived derivedFilter,
fmt.Fprintf(&b, " the found firewall (%s) is disabled, never flushed: its configuration stays on disk\n", fw) fmt.Fprintf(&b, " the found firewall (%s) is disabled, never flushed: its configuration stays on disk\n", fw)
} }
said = append(said, fmt.Sprintf("filter %s assigned=%t firewall=%s", filter, filterAssigned, fw)) said = append(said, fmt.Sprintf("filter %s assigned=%t firewall=%s", filter, filterAssigned, fw))
// What filters the machine now, and the fate of each (novox/hq ADR 0168): the found firewall
// retired, the runtime's own and bans left, and what the mesh did not write left and named —
// so the reader knows before the flip that the machine will not be filtered by the mesh alone.
if len(filtering.Filters) > 0 {
b.WriteString("\n what filters the machine now, and what the flip does to each:\n")
for _, x := range filtering.Filters {
fate := "left: " + x.Owner + "'s"
switch x.Owner {
case inventory.FilterMesh:
fate = "the mesh's guard; replaced by its filter"
case inventory.FilterFoundFirewall:
fate = "the found firewall's; retired with it"
case inventory.FilterRuntime:
fate = "the container runtime's own; left"
case inventory.FilterBan:
fate = "a ban list; left"
case inventory.FilterOther:
fate = "NOT THE MESH'S; left in force — the machine is not filtered by the mesh alone until you remove it"
}
fmt.Fprintf(&b, " %-50s %s\n", x.Where, fate)
fmt.Fprintf(&b, " %-50s %s\n", "", x.Refuses)
said = append(said, "filter "+x.Owner+" "+x.Where)
}
}
// Sorted: the same account, reported in another order, is the same preview. // Sorted: the same account, reported in another order, is the same preview.
sort.Strings(said) sort.Strings(said)
sum := sha256.Sum256([]byte(strings.Join(said, "\n"))) sum := sha256.Sum256([]byte(strings.Join(said, "\n")))
@@ -596,11 +1034,38 @@ func adopt(ctx context.Context, open *stores, node string) (string, error) {
// takeCommand, convergeCommand and adoptCommand are the command line's adapters to the acts above. // takeCommand, convergeCommand and adoptCommand are the command line's adapters to the acts above.
func takeCommand(ctx context.Context, args []string) error { func takeCommand(ctx context.Context, args []string) error {
if len(args) != 2 { set := flag.NewFlagSet("take", flag.ContinueOnError)
return errors.New("take <node> <module>") yes := set.Bool("yes", false, "cut over as previewed, naming the digest the preview printed after it; "+
"without it the comparison is printed and nothing is taken")
downgrade := set.Bool("downgrade", false, "take it although the module's image is older than the one running")
var replace, mint stringList
set.Var(&replace, "replace", "a found file's path whose content the module may replace although it differs (repeatable; * for every one)")
set.Var(&mint, "mint", "a secret the service shall take the mesh's minted value for, although it already has its own (repeatable)")
positionals, err := parseAround(set, args)
if err != nil {
return err
} }
return runAct(ctx, func(open *stores) (string, error) { return take(ctx, open, args[0], args[1]) }) if len(positionals) < 2 || len(positionals) > 3 || (len(positionals) == 3 && !*yes) {
return errors.New("take <node> <module> [--yes <digest>] [--downgrade] [--replace <path>]... [--mint <secret>]...")
} }
opts := takeOptions{Yes: *yes, Downgrade: *downgrade, Replace: map[string]bool{}, Mint: map[string]bool{}}
if len(positionals) == 3 {
opts.Digest = positionals[2]
}
for _, r := range replace {
opts.Replace[r] = true
}
for _, m := range mint {
opts.Mint[m] = true
}
return runAct(ctx, func(open *stores) (string, error) { return take(ctx, open, positionals[0], positionals[1], opts) })
}
// stringList is a repeatable flag.
type stringList []string
func (l *stringList) String() string { return strings.Join(*l, ",") }
func (l *stringList) Set(v string) error { *l = append(*l, v); return nil }
func convergeCommand(ctx context.Context, args []string) error { func convergeCommand(ctx context.Context, args []string) error {
set := flag.NewFlagSet("converge", flag.ContinueOnError) set := flag.NewFlagSet("converge", flag.ContinueOnError)
+3 -3
View File
@@ -102,7 +102,7 @@ func commands(who Authenticator) http.Handler {
})) }))
// Adoption (novox/hq ADR 0100): the same acts as `take`, `converge` and `adopt`. // Adoption (novox/hq ADR 0100): the same acts as `take`, `converge` and `adopt`.
mux.HandleFunc("POST /take", acting(who, true, func(ctx context.Context, open *stores, in request) (string, error) { mux.HandleFunc("POST /take", acting(who, true, func(ctx context.Context, open *stores, in request) (string, error) {
return take(ctx, open, in.Node, in.Module) return take(ctx, open, in.Node, in.Module, takeOptions{Yes: in.Yes, Digest: in.Digest})
})) }))
mux.HandleFunc("POST /converge", acting(who, false, func(ctx context.Context, open *stores, in request) (string, error) { mux.HandleFunc("POST /converge", acting(who, false, func(ctx context.Context, open *stores, in request) (string, error) {
return converge(ctx, open, in.Node, in.Yes, in.Digest, in.Filter) return converge(ctx, open, in.Node, in.Yes, in.Digest, in.Filter)
@@ -124,8 +124,8 @@ func commands(who Authenticator) http.Handler {
type request struct { type request struct {
Node string `json:"node"` Node string `json:"node"`
Module string `json:"module"` Module string `json:"module"`
// Yes, Digest and Filter are converge's: do it rather than preview it, the digest of the // Yes, Digest and Filter are converge's and take's: do it rather than preview it, the digest
// preview it acts on, and which module loads the mesh's filter. // of the preview it acts on, and (converge) which module loads the mesh's filter.
Yes bool `json:"yes,omitempty"` Yes bool `json:"yes,omitempty"`
Digest string `json:"digest,omitempty"` Digest string `json:"digest,omitempty"`
Filter string `json:"filter,omitempty"` Filter string `json:"filter,omitempty"`
+198 -39
View File
@@ -10,6 +10,8 @@ import (
"strings" "strings"
"time" "time"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker" "github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue" "github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
@@ -175,10 +177,14 @@ func buildFrom(result link.BuildResult) inventory.Build {
func buildsCommand(ctx context.Context, args []string) error { func buildsCommand(ctx context.Context, args []string) error {
set := flag.NewFlagSet("builds", flag.ContinueOnError) set := flag.NewFlagSet("builds", flag.ContinueOnError)
limit := set.Int("n", 20, "how many to show") limit := set.Int("n", 20, "how many to show")
logOf := set.String("log", "", "a build's id: print what the build machine said, line by line")
positionals, err := parseAround(set, args) positionals, err := parseAround(set, args)
if err != nil { if err != nil {
return err return err
} }
if *logOf != "" {
return buildLog(ctx, *logOf)
}
module := "" module := ""
if len(positionals) == 1 { if len(positionals) == 1 {
module = positionals[0] module = positionals[0]
@@ -219,8 +225,8 @@ func buildsCommand(ctx context.Context, args []string) error {
if !b.Worked() { if !b.Worked() {
outcome = "failed" outcome = "failed"
} }
fmt.Printf("%-18s %-14s %-10s %s\n", fmt.Printf("%-18s %-14s %-10s %s %s\n",
what, outcome, b.On, b.At.Local().Format("2006-01-02 15:04")) what, outcome, b.On, b.At.Local().Format("2006-01-02 15:04"), b.ID)
fmt.Printf(" %s", b.Repository) fmt.Printf(" %s", b.Repository)
if b.Ref != "" { if b.Ref != "" {
fmt.Printf(" at %s", b.Ref) fmt.Printf(" at %s", b.Ref)
@@ -414,6 +420,8 @@ func buildOne(ctx context.Context, source buildSource, path, ref string, wait ti
if source.Seat != "" { if source.Seat != "" {
fmt.Printf(" (%s)", repository) fmt.Printf(" (%s)", repository)
} }
// The id is how a person follows this build while it runs: `builds --log <id>`.
fmt.Printf(" as %s", request.ID)
if path != "" { if path != "" {
fmt.Printf(" at %s", path) fmt.Printf(" at %s", path)
} }
@@ -422,70 +430,114 @@ func buildOne(ctx context.Context, source buildSource, path, ref string, wait ti
} }
fmt.Println() fmt.Println()
ask, err := askOver(server) seat := buildSeatHeld(ctx)
ask, err := askOverOn(seat)
if err != nil { if err != nil {
return err return err
} }
defer ask.Close() defer ask.Close()
fmt.Printf(" of %s\n", seat)
if wait == 0 {
// Asked and not waited for (novox/hq issue 176): the outcome is the role's event, and the
// controller takes it in — records the build, registers the module — whether or not anybody
// is still here. A tool call cannot hold a connection for the minutes a build takes; it
// follows the build by its id instead.
if err := ask.Ask(ctx, request); err != nil {
return err
}
fmt.Printf("asked, not waited for: `builds --log %s` follows it as it runs, and `builds` "+
"shows what came of it; the module is registered when the outcome comes\n", request.ID)
return nil
}
result, err := ask.Submit(ctx, request, wait) result, err := ask.Submit(ctx, request, wait)
if err != nil { if err != nil {
return err return err
} }
// Kept before it is judged. A failed build that leaves no trace is indistinguishable from one
// nobody asked for, and the difference is the whole of whether somebody should be looking at
// something.
open, err := openStores(ctx) open, err := openStores(ctx)
if err != nil { if err != nil {
return err return err
} }
defer open.Close() defer open.Close()
inv := open.inventory manifest, kept, err := takeIn(ctx, open.inventory, result)
kept := buildFrom(result) if err != nil {
if err := inv.RecordBuild(ctx, kept); err != nil {
return err return err
} }
if result.Failed != "" {
// The builder's own words. Wrapping them in something about the control plane would put
// two explanations between a person and a build log.
return fmt.Errorf("%s could not build %s:\n%s", result.On, result.Repository, result.Failed)
}
// Said as recorded: what each artifact is, not where this builder happened to push it. // Said as recorded: what each artifact is, not where this builder happened to push it.
for _, made := range kept.Made { for _, made := range kept.Made {
fmt.Printf(" %-12s %s %s\n", made.Name, made.Kind, made.Reference) fmt.Printf(" %-12s %s %s\n", made.Name, made.Kind, made.Reference)
} }
fmt.Printf("\n%s %s, built on %s from %s\n",
// Parsed with the same parser a hand-written manifest goes through. A second path would be a manifest.Module, manifest.Version, result.On, short(result.Commit))
// second thing to disagree about what a manifest is. The manifest as recorded, so the catalogue saysWhenThePolicyActs(ctx, open.inventory, manifest.Module)
// holds references by digest and path and every declaration composes the store's address in. fmt.Printf(" run `assign <node> %s` to put it somewhere\n", manifest.Module)
manifest, err := catalogue.ParseManifest(kept.Manifest) return nil
if err != nil {
return fmt.Errorf("%s built %s and what came back is not a manifest: %w",
result.On, result.Repository, err)
} }
// Recorded with where it came from, so "is this current?" is answerable without building it // saysWhenThePolicyActs tells whoever built a module that its upgrade policy will send the
// again (novox/hq ADR 0009). **For a source on a seat, as the path and the seat, never the URL // result on at once (novox/hq issue 126, ADR 0163): a person choreographing a data move must
// just cloned** (ADR 0111): the URL is where the forge runs today, and recording it would put // know which module will not wait for them.
// the forge's address back into every module built from it. The build log above keeps the URL, func saysWhenThePolicyActs(ctx context.Context, inv *inventory.Inventory, module string) {
// because that is what was cloned. if u, err := inv.UpgradeOf(ctx, module); err == nil && u.RollOut {
how := "one machine at a time"
if u.Together {
how = "every machine at once"
}
fmt.Printf(" %s rolls out on build: the machines running it are sent this now, %s — "+
"`upgrade %s record` first if something must move before it does\n", module, how, module)
}
}
// takeIn is what the mesh does with a build's outcome, whoever hears it: the waiting command and
// the daemon that follows the role's events both come here (novox/hq issue 176), so a build's
// result reaches the catalogue whether or not the asker was still listening.
//
// Kept before it is judged. A failed build that leaves no trace is indistinguishable from one
// nobody asked for, and the difference is the whole of whether somebody should be looking at
// something. Then parsed with the same parser a hand-written manifest goes through — a second path
// would be a second thing to disagree about what a manifest is — and registered with where it came
// from: **for a source on a seat, as the path and the seat, never the URL just cloned** (ADR 0111),
// which the request carried and the outcome echoes. A definition naming an installation is refused
// here, where it would enter the catalogue; the build stays recorded and the refusal says which.
//
// Idempotent: the same outcome taken in twice registers the same module twice, which is one row
// written with the same values.
func takeIn(ctx context.Context, inv *inventory.Inventory, result link.BuildResult) (
catalogue.Manifest, inventory.Build, error) {
kept := buildFrom(result)
if err := inv.RecordBuild(ctx, kept); err != nil {
return catalogue.Manifest{}, kept, err
}
if result.Failed != "" {
// The builder's own words. Wrapping them in something about the control plane would put
// two explanations between a person and a build log.
return catalogue.Manifest{}, kept, fmt.Errorf("%s could not build %s:\n%s",
result.On, result.Repository, result.Failed)
}
manifest, err := catalogue.ParseManifest(kept.Manifest)
if err != nil {
return catalogue.Manifest{}, kept, fmt.Errorf("%s built %s and what came back is not a manifest: %w",
result.On, result.Repository, err)
}
recorded := inventory.Source{ recorded := inventory.Source{
Repository: result.Repository, Path: result.Path, Ref: result.Ref, Repository: result.Repository, Path: result.Path, Ref: result.Ref,
BuiltFrom: result.Commit, Head: result.Commit, BuiltFrom: result.Commit, Head: result.Commit,
// What it stood on, so registration can judge a built manifest's base (to-be 38 WP2.4).
Against: kept.Against,
} }
if source.Seat != "" { if result.Source != nil && result.Source.Seat != "" {
recorded.Repository, recorded.Seat = source.Repository, source.Seat recorded.Repository, recorded.Seat = result.Source.Repository, result.Source.Seat
}
if err := namesNoInstallation(manifest); err != nil {
return manifest, kept, fmt.Errorf("%s built %s (%s), and the mesh does not register it: %w",
result.On, result.Repository, short(result.Commit), err)
} }
if err := inv.RegisterModule(ctx, manifest, recorded); err != nil { if err := inv.RegisterModule(ctx, manifest, recorded); err != nil {
return err return manifest, kept, err
} }
fmt.Printf("\n%s %s, built on %s from %s\n", return manifest, kept, nil
manifest.Module, manifest.Version, result.On, short(result.Commit))
fmt.Printf(" run `assign <node> %s` to put it somewhere\n", manifest.Module)
return nil
} }
// buildAndShow builds and prints the manifest without recording anything. // buildAndShow builds and prints the manifest without recording anything.
@@ -505,7 +557,7 @@ func buildAndShow(ctx context.Context, source buildSource, path, ref string, wai
} }
defer server.Close() defer server.Close()
ask, err := askOver(server) ask, err := askOverOn(buildSeatHeld(ctx))
if err != nil { if err != nil {
return err return err
} }
@@ -548,6 +600,8 @@ type answers struct {
// pair that answers "has it caught up", which waiting alone cannot (the sent digest is // pair that answers "has it caught up", which waiting alone cannot (the sent digest is
// recorded at send, not at apply). // recorded at send, not at apply).
reported []inventory.Reported reported []inventory.Reported
// plans is what the last merges produced and where each stands (novox/hq ADR 0162).
plans []inventory.Plan
// refused is why a machine cannot be worked out at all, by name. A different thing from every // refused is why a machine cannot be worked out at all, by name. A different thing from every
// other answer here: those are about a machine that was told something, and this is about one // other answer here: those are about a machine that was told something, and this is about one
// that cannot be told anything — it never reaches waiting, because nothing was computed for it // that cannot be told anything — it never reaches waiting, because nothing was computed for it
@@ -557,6 +611,10 @@ type answers struct {
// a consequence of the refusals above: a node that does not resolve is not on the network, and // a consequence of the refusals above: a node that does not resolve is not on the network, and
// a mesh whose hub is that node has no hub. // a mesh whose hub is that node has no hub.
network string network string
// filtered is every converged machine that is not filtered by the mesh alone (novox/hq ADR
// 0168): what filters it beyond the mesh's own, the runtime's plumbing and bans, by name — a
// predecessor's chain, a found firewall in force again. Such a machine is not "all well".
filtered map[string]inventory.Filtering
// untaken is, per machine, each assigned module whose resources the machine is holding as it // untaken is, per machine, each assigned module whose resources the machine is holding as it
// found them, and how many — a module that was assigned, sent, and is running none of what it // found them, and how many — a module that was assigned, sent, and is running none of what it
// declares because nothing has taken it (novox/hq ADR 0100, 04-ISSUES/125). // declares because nothing has taken it (novox/hq ADR 0100, 04-ISSUES/125).
@@ -612,10 +670,111 @@ func heldBy(ctx context.Context) map[string]string {
// **One place chooses**, as everywhere else the bus change went (novox/hq ADR 0116 step 5). On the bus // **One place chooses**, as everywhere else the bus change went (novox/hq ADR 0116 step 5). On the bus
// the mesh runs on today this needs the controller's own connection, so it is handed one; on the bus // the mesh runs on today this needs the controller's own connection, so it is handed one; on the bus
// being built it dials, because a build request is a one-shot and holds nothing else. // being built it dials, because a build request is a one-shot and holds nothing else.
func askOver(_ *link.Server) (link.Builders, error) { func askOverOn(seat string) (link.Builders, error) {
address, err := broker.BusAddress() address, err := broker.BusAddress()
if err != nil { if err != nil {
return nil, err return nil, err
} }
return link.BuildsOverNATS(address) return link.BuildsOverNATSOn(address, seat)
}
// buildSeatHeld is the build role to ask: the one some assigned module claims (novox/hq ADR 0190,
// the handover). Read from the catalogue at ask time, because the answer changes exactly once, the
// moment the first build-agent is assigned — and a controller that asked the new role before then
// would queue work nothing takes, while the outcome that registers build-agent itself has to come
// from the old builder. When the catalogue cannot be read the current role is asked, said aloud.
func buildSeatHeld(ctx context.Context) string {
open, err := openStores(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "could not read what is assigned, so the build is asked of %s: %v\n",
link.TheBuildMachine, err)
return link.TheBuildMachine
}
defer open.Close()
entries, err := open.inventory.Catalogued(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "could not read the catalogue, so the build is asked of %s: %v\n",
link.TheBuildMachine, err)
return link.TheBuildMachine
}
return buildSeatAmong(entries)
}
// buildSeatAmong is the rule, over what the catalogue holds: the current build role when any
// assigned module claims it; else the retired role while an assigned module still claims that; else
// the current role, which is where every ask goes once the handover is done.
func buildSeatAmong(entries []inventory.Entry) string {
heldBefore := false
for _, e := range entries {
if len(e.On) == 0 {
continue
}
if e.Manifest.ClaimsSeat(link.TheBuildMachine) {
return link.TheBuildMachine
}
if e.Manifest.ClaimsSeat(link.TheBuildMachineBefore) {
heldBefore = true
}
}
if heldBefore {
return link.TheBuildMachineBefore
}
return link.TheBuildMachine
}
// buildLog prints everything a build machine said about one build, read back from the bus.
//
// **From the stream, not from a record** (novox/hq ADR 0157). A build's lines are the role's own
// events under the build's id, retained with every other event; the mesh keeps no second copy. Read
// with a consumer of its own that is gone when this returns, so nothing accumulates in the server
// for the reading, and filtered by subject, so one build's lines are all that travel.
func buildLog(ctx context.Context, id string) error {
address, err := broker.BusAddress()
if err != nil {
return err
}
js, err := broker.Dial(address)
if err != nil {
return fmt.Errorf("cannot reach the bus to read a build's log: %w", err)
}
defer js.Close()
// Under whichever build role did it: a build asked of the retired role during the handover
// (ADR 0190) said its lines as that role's events, and a reader should not have to know which.
lines := link.BuildLogOf("*", id)
sub, err := js.Context().PullSubscribe(lines, "",
nats.BindStream(broker.EventsStream), nats.DeliverAll(), nats.AckNone())
if err != nil {
return fmt.Errorf("cannot read %s from the bus: %w", lines, err)
}
defer func() { _ = sub.Unsubscribe() }()
printed := 0
for {
batch, err := sub.Fetch(200, nats.MaxWait(2*time.Second))
if err != nil && !errors.Is(err, nats.ErrTimeout) && !errors.Is(err, context.DeadlineExceeded) {
return fmt.Errorf("reading a build's log: %w", err)
}
for _, msg := range batch {
var line link.BuildLine
if err := json.Unmarshal(msg.Data, &line); err != nil {
fmt.Printf(" ? %s\n", string(msg.Data))
continue
}
at := line.At
if t, err := time.Parse(time.RFC3339Nano, line.At); err == nil {
at = t.Local().Format("15:04:05")
}
fmt.Printf("%s %4d [%s] %s\n", at, line.Seq, line.Step, line.Message)
printed++
}
if len(batch) < 200 {
break
}
}
if printed == 0 {
fmt.Printf("nothing on the bus for build %s: no build by that id in the last week, or a build "+
"machine older than this that said nothing while building\n", id)
}
return nil
} }
+43
View File
@@ -0,0 +1,43 @@
package main
import (
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
func claiming(module, seat string, on ...string) inventory.Entry {
return inventory.Entry{
Manifest: catalogue.Manifest{Module: module, Claims: []catalogue.Claim{{Name: seat}}},
On: on,
}
}
// The controller asks the build role that has a holder (novox/hq ADR 0190 handover): the retired
// one while only the builder is assigned, the current one from the first build-agent on, and the
// current one when nothing holds either — where every ask goes once the handover is done.
func TestTheControllerAsksTheBuildRoleThatHasAHolder(t *testing.T) {
onlyTheBuilder := []inventory.Entry{
claiming("builder", link.TheBuildMachineBefore, "anchor"),
claiming("build-agent", link.TheBuildMachine), // registered, assigned nowhere yet
}
if got := buildSeatAmong(onlyTheBuilder); got != link.TheBuildMachineBefore {
t.Errorf("with only the builder assigned, asked %q", got)
}
bothHeld := []inventory.Entry{
claiming("builder", link.TheBuildMachineBefore, "anchor"),
claiming("build-agent", link.TheBuildMachine, "home-server"),
}
if got := buildSeatAmong(bothHeld); got != link.TheBuildMachine {
t.Errorf("with a build-agent assigned anywhere, asked %q", got)
}
neither := []inventory.Entry{claiming("builder", link.TheBuildMachineBefore)}
if got := buildSeatAmong(neither); got != link.TheBuildMachine {
t.Errorf("with no holder of either, asked %q, want the current role", got)
}
if got := buildSeatAmong(nil); got != link.TheBuildMachine {
t.Errorf("an empty catalogue asks %q", got)
}
}
+65
View File
@@ -0,0 +1,65 @@
package main
import (
"encoding/json"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/link"
)
// A build's outcome is taken in the same way whoever hears it (novox/hq issue 176): recorded, and
// the module registered with its source as the seat and path when the request said so — never the
// URL. A definition naming an installation is recorded and not registered; a failure is recorded
// and said.
func TestABuildHeardIsRecordedAndRegistered(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
manifest, _ := json.Marshal(map[string]any{"module": "shop", "version": "3"})
m, _, err := takeIn(ctx, open.inventory, link.BuildResult{
ID: "b-1", Repository: "http://forge.internal:20000/novox/shop.git", Path: "modules/shop",
Ref: "main", On: "anchor", Commit: "abcdef0123", Manifest: manifest,
Source: &link.SourceOnSeat{Seat: "git", Repository: "novox/shop"},
})
if err != nil {
t.Fatal(err)
}
if m.Module != "shop" {
t.Fatalf("registered %q", m.Module)
}
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
t.Fatal(err)
}
if _, held := shelf["shop"]; !held {
t.Fatal("the module a heard build produced is not in the catalogue")
}
src, err := open.inventory.SourceOf(ctx, "shop")
if err != nil || src.Seat != "git" || src.Repository != "novox/shop" || src.BuiltFrom != "abcdef0123" {
t.Fatalf("the source is the seat and the path, never the URL: %+v %v", src, err)
}
builds, err := open.inventory.Builds(ctx, "shop", 5)
if err != nil || len(builds) != 1 || builds[0].ID != "b-1" {
t.Fatalf("the build is not recorded once: %v %v", builds, err)
}
named, _ := json.Marshal(map[string]any{"module": "idp", "version": "1", "resources": []any{
map[string]any{"id": "server", "type": "container", "image": "x@sha256:aa",
"env": map[string]any{"KC_HOSTNAME": "https://login.mesh-one.be"}}}})
_, _, err = takeIn(ctx, open.inventory, link.BuildResult{
ID: "b-2", Repository: "/r", On: "anchor", Commit: "0123456789", Manifest: named})
if err == nil || !strings.Contains(err.Error(), "does not register it") {
t.Fatalf("a definition naming an installation was taken in: %v", err)
}
if shelf, _ := open.inventory.Catalogue(ctx); shelf["idp"].Module != "" {
t.Fatal("the refused module was registered anyway")
}
if builds, _ := open.inventory.Builds(ctx, "idp", 5); len(builds) != 1 {
t.Fatalf("the refused build was not recorded: %v", builds)
}
_, _, err = takeIn(ctx, open.inventory, link.BuildResult{ID: "b-3", Repository: "/r", On: "anchor", Failed: "no compiler"})
if err == nil || !strings.Contains(err.Error(), "no compiler") {
t.Fatalf("a failure is said in the builder's words: %v", err)
}
}
+2 -2
View File
@@ -51,8 +51,8 @@ func moduleCheck(paths []string, out io.Writer) error {
failed++ failed++
continue continue
} }
// A definition names no installation (novox/hq ADR 0112, ADR 0155): judged here and in the // A definition names no installation (novox/hq ADR 0112, ADR 0155): judged here, in the
// catalogue-wide test, not yet at registration, while the declared exceptions shrink. // catalogue-wide test, and at registration, which refuses in the same words.
if named := catalogue.InstallationProblems(m); len(named) > 0 { if named := catalogue.InstallationProblems(m); len(named) > 0 {
for _, p := range named { for _, p := range named {
fmt.Fprintf(out, "%s: %s\n", path, p) fmt.Fprintf(out, "%s: %s\n", path, p)
+25
View File
@@ -5,6 +5,8 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"strings" "strings"
"github.com/novox/mesh-controller/internal/catalogue"
"testing" "testing"
) )
@@ -67,3 +69,26 @@ func TestModuleCheckPassesTheCatalogue(t *testing.T) {
t.Fatalf("the catalogue does not pass its own check: %v\n%s", err, out.String()) t.Fatalf("the catalogue does not pass its own check: %v\n%s", err, out.String())
} }
} }
func TestRegistrationRefusesADefinitionNamingAnInstallation(t *testing.T) {
// novox/hq ADR 0155: the check moves to registration once the catalogue passes it. Both
// ways in — `module add` and a build's result — go through this, and a name declared on
// purpose passes with its reason.
named := catalogue.Manifest{Module: "idp", Resources: []map[string]any{
{"id": "server", "type": "container", "image": "x@sha256:aa",
"env": map[string]any{"KC_HOSTNAME": "https://login.mesh-one.be"}},
}}
err := namesNoInstallation(named)
if err == nil || !strings.Contains(err.Error(), "login.mesh-one.be") ||
!strings.Contains(err.Error(), catalogue.NamesOnPurpose) {
t.Fatalf("a definition naming an installation is refused with the name and the way out; got %v", err)
}
meant := catalogue.Manifest{Module: "site", Resources: []map[string]any{
{"id": "server", "type": "container", "image": "registry.mesh-one.be/org/site@sha256:cc",
catalogue.NamesOnPurpose: map[string]any{
"registry.mesh-one.be": "built outside the mesh until its repository is a build source here"}},
}}
if err := namesNoInstallation(meant); err != nil {
t.Fatalf("a name declared on purpose passes; got %v", err)
}
}
@@ -0,0 +1,90 @@
package main
import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
// A fresh assignment is pushed before its credential exists (novox/hq issue 203): `assign` recorded
// the module, `push` sealed a random own secret where the bus credential belongs, and the process
// crash-looped until a person ran `module issue` and pushed again. Now assigning a module that speaks
// on the bus issues its credential in the same act — or, when the bus cannot be reached from here,
// says which verb to run — and a push never seals a placeholder in a credential's place.
func aTalker() catalogue.Manifest {
return catalogue.Manifest{Module: "talker", Version: "1",
OwnSecrets: catalogue.OwnSecrets{"broker": {Path: "/var/lib/mesh/talker/broker"}},
Resources: []map[string]any{
{"id": "state", "type": "directory", "path": "/var/lib/mesh/talker", "mode": "0700"},
}}
}
func TestAssigningAModuleThatSpeaksOnTheBusNamesItsCredential(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, aTalker())
// No bus is known to this process, so the credential cannot be issued here: the assignment
// stands and says exactly what must happen before a push — never silently.
said, err := assign(ctx, open, "laptop", "talker")
if err != nil {
t.Fatal(err)
}
if !strings.Contains(said, "module issue talker --node laptop") {
t.Fatalf("an assignment whose credential could not be issued does not name the verb:\n%s", said)
}
// And the push refuses to send it, naming the same verb, rather than sealing a placeholder.
plan, settings, err := planFor(ctx, open, "laptop")
if err != nil {
t.Fatal(err)
}
_, err = declarationFor(ctx, open, "laptop", plan, settings)
if err == nil {
t.Fatal("a push sealed a placeholder where talker's bus credential belongs")
}
if !strings.Contains(err.Error(), "module issue talker --node laptop") || !strings.Contains(err.Error(), "issue 203") {
t.Fatalf("the refusal does not say what to run: %v", err)
}
// Once the user is minted, the push goes on to the credential the mesh sealed, and re-assigning
// does not mint again: a credential rotates on purpose, never by habit.
if _, err := open.inventory.MintBusPassword(ctx, inventory.BusUser{
Username: "laptop.talker", Kind: inventory.BusModule, Node: "laptop", Module: "talker"}); err != nil {
t.Fatal(err)
}
hash, _, err := open.inventory.BusUserHash(ctx, "laptop.talker")
if err != nil {
t.Fatal(err)
}
said, err = assign(ctx, open, "laptop", "talker")
if err != nil {
t.Fatal(err)
}
if strings.Contains(said, "module issue") {
t.Fatalf("a module with a minted credential was told to issue one:\n%s", said)
}
again, _, err := open.inventory.BusUserHash(ctx, "laptop.talker")
if err != nil {
t.Fatal(err)
}
if again != hash {
t.Fatal("re-assigning rotated the credential")
}
}
// A module that declares no broker secret is left alone: nothing to issue, nothing said.
func TestAssigningAModuleThatDoesNotSpeakSaysNothingOfCredentials(t *testing.T) {
open := aMesh(t)
register(t, open, helloWeb())
said, err := assign(t.Context(), open, "laptop", "hello-web")
if err != nil {
t.Fatal(err)
}
if strings.Contains(said, "credential") {
t.Fatalf("a module without a broker secret was told about credentials:\n%s", said)
}
}
+52
View File
@@ -0,0 +1,52 @@
package main
import (
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
// A merge that rebuilds a base rebuilds what stands on it, through every layer, and nothing else
// (novox/hq issue 186): the runtime image moving means every module built on it moves too, and a
// module built on one of those moves as well.
func TestAMergeOfABaseTakesWhatStandsOnItAlong(t *testing.T) {
entry := func(name string) inventory.Entry {
return inventory.Entry{Manifest: catalogue.Manifest{Module: name}}
}
entries := []inventory.Entry{entry("mesh-tools"), entry("shop"), entry("shop-plugin"), entry("postgres"), entry("unrelated")}
against := map[string][]string{
"shop": {catalogue.ArtifactStoreScheme + "mesh-tools/runtime@sha256:a"},
"shop-plugin": {catalogue.ArtifactStoreScheme + "shop/runtime@sha256:b"},
"postgres": {catalogue.ArtifactStoreScheme + "mesh-tools/runtime@sha256:a"},
"unrelated": {catalogue.ArtifactStoreScheme + "alpine/base@sha256:c"},
}
got := dependentsOf([]inventory.Entry{entry("mesh-tools")}, entries, against)
var names []string
for _, e := range got {
names = append(names, e.Manifest.Module)
}
want := map[string]bool{"shop": true, "shop-plugin": true, "postgres": true}
if len(names) != len(want) {
t.Fatalf("rebuilt %v; wanted exactly the three that stand on the runtime, directly or through shop", names)
}
for _, n := range names {
if !want[n] {
t.Fatalf("%s was rebuilt and stands on nothing that moved (%v)", n, names)
}
}
// The dependents come in base order when the merge orders them: the runtime, then shop, then
// the plugin that stands on shop.
ordered := orderByBases(append([]inventory.Entry{entry("mesh-tools")}, got...), against)
pos := map[string]int{}
for i, e := range ordered {
pos[e.Manifest.Module] = i
}
if !(pos["mesh-tools"] < pos["shop"] && pos["shop"] < pos["shop-plugin"]) {
t.Fatalf("not in base order: %v", ordered)
}
// Nothing moved: nothing follows.
if more := dependentsOf(nil, entries, against); len(more) != 0 {
t.Fatalf("with nothing moved, %d module(s) were rebuilt", len(more))
}
}
@@ -1,33 +0,0 @@
package main
// The broker opening belongs only on the node that listens on it (novox/hq: it leaked onto
// every enrolled node's declaration, opening a from-anywhere hole for a port nothing there
// serves). foundationPortsFor is the scope.
import (
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
func TestTheBrokerHostGetsTheFoundationOpening(t *testing.T) {
broker := catalogue.Manifest{Module: "lavinmq", Listens: []catalogue.Listening{
{Port: 5671, Protocol: "tcp", From: "mesh"},
{Port: 5672, Protocol: "tcp", From: "mesh"},
}}
got := foundationPortsFor(5671, []catalogue.Manifest{broker})
if len(got) != 1 || got[0] != 5671 {
t.Fatalf("the node that listens on the broker port keeps it; got %v", got)
}
}
func TestANodeThatOnlyDialsTheBrokerGetsNoOpening(t *testing.T) {
// ace's set: things that reach the broker as a client, none listening on 5671.
ace := []catalogue.Manifest{
{Module: "plex", Listens: []catalogue.Listening{{Port: 32400, Protocol: "tcp", From: "anywhere"}}},
{Module: "postgres", Listens: []catalogue.Listening{{Port: 5432, Protocol: "tcp", From: "mesh"}}},
}
if got := foundationPortsFor(5671, ace); got != nil {
t.Fatalf("a node that only dials out opens nothing for the broker; got %v", got)
}
}
+93
View File
@@ -0,0 +1,93 @@
package main
import (
"context"
"testing"
"github.com/novox/mesh-controller/internal/inventory"
)
// A declaration composed earlier is numbered lower than one composed later, whatever order the two
// are sent in (novox/hq issue 204). The number used to be taken at send time, after composing, so a
// declaration composed before an assignment changed and sent after a newer one carried the higher
// number — and the machine, which refuses a lower number, took the older content as the mesh's
// newest word. Taken before the composition reads anything, the order of numbers is the order of
// compositions, and the host's refusal does what it is for.
func TestADeclarationComposedEarlierIsNumberedLowerWhateverOrderItIsSent(t *testing.T) {
allot := numbered()
var composed []string
compose := func(stamp string) func(string) (sendable, error) {
return func(node string) (sendable, error) {
composed = append(composed, stamp)
return sendable{Resources: []map[string]any{{"id": node + "." + stamp}}}, nil
}
}
// Composed first — before an assignment changed — and sent last.
stale, _ := composeEach([]string{"anchor"}, allot, compose("before"))
// Composed after the change, sent first.
fresh, _ := composeEach([]string{"anchor"}, allot, compose("after"))
if stale[0].declared.Sequence != 1 || fresh[0].declared.Sequence != 2 {
t.Fatalf("the numbers do not follow the compositions: before=%d after=%d",
stale[0].declared.Sequence, fresh[0].declared.Sequence)
}
// Sent in the other order, the numbers do not change — so the machine that has applied the
// fresh one (2) refuses the stale one (1) when it arrives late.
if !(stale[0].declared.Sequence < fresh[0].declared.Sequence) {
t.Fatal("a declaration composed earlier must carry the lower number, however late it is sent")
}
if len(composed) != 2 || composed[0] != "before" {
t.Fatalf("compositions happened in an unexpected order: %v", composed)
}
}
// The number is taken before the first read of the composition, not after it: an allotter that
// fails leaves nothing composed for that machine, and the others are still composed.
func TestTheNumberIsTakenBeforeComposingAndItsFailureIsARefusal(t *testing.T) {
calls := 0
allot := func(node string) (int64, error) {
if node == "anchor" {
return 0, context.DeadlineExceeded
}
return 7, nil
}
sending, refusals := composeEach([]string{"anchor", "laptop"}, allot, func(node string) (sendable, error) {
calls++
if node == "anchor" {
t.Fatal("anchor was composed although its number could not be taken")
}
return sendable{}, nil
})
if calls != 1 || len(sending) != 1 || sending[0].node != "laptop" || sending[0].declared.Sequence != 7 {
t.Fatalf("laptop should be composed with its number and anchor refused: %v / %v", sending, refusals)
}
if len(refusals) != 1 {
t.Fatalf("anchor's failed number should be a refusal naming it: %v", refusals)
}
}
// What was sent is written down even when the sender's context is already cancelled (issue 204): a
// controller replaced mid-send had told the machine and never recorded it, so status read "applied,
// current" over a machine that had just been sent something else.
func TestASendIsRecordedEvenWhenTheSenderIsBeingCancelled(t *testing.T) {
inv := inventory.ForTest(t)
ctx, cancel := context.WithCancel(t.Context())
if _, err := inv.AddNode(ctx, "anchor"); err != nil {
t.Fatal(err)
}
cancel() // the sender is going away: its context is cancelled between the send and the record
body := []byte(`{"declaration":1,"resources":[]}`)
digest, err := recordSent(ctx, inv, "anchor", body)
if err != nil {
// NodeByName on the cancelled context may itself refuse; the record must still be possible
// through the detached context, so look the node up again on a live one.
t.Fatalf("recording a send after cancellation failed: %v", err)
}
outstanding, err := inv.Outstanding(t.Context(), "anchor")
if err != nil {
t.Fatal(err)
}
if outstanding != digest || digest != digestOf(body) {
t.Fatalf("the send was not recorded: outstanding %q, sent %q", outstanding, digest)
}
}
+34 -3
View File
@@ -72,6 +72,8 @@ func run() error {
return askCommand(ctx, args[1:]) return askCommand(ctx, args[1:])
case "builds": case "builds":
return buildsCommand(ctx, args[1:]) return buildsCommand(ctx, args[1:])
case "plans":
return plansCommand(ctx, args[1:])
case "pin": case "pin":
return pinCommand(ctx, args[1:], true) return pinCommand(ctx, args[1:], true)
case "unpin": case "unpin":
@@ -172,11 +174,14 @@ func usage() {
upgrade <name> record ...record that they are behind, and send nothing upgrade <name> record ...record that they are behind, and send nothing
status [--json] what is wrong, what is quiet, and what is out of date status [--json] what is wrong, what is quiet, and what is out of date
seats [--json] every seat this mesh defines, what it delivers, and who holds it seats [--json] every seat this mesh defines, what it delivers, and who holds it
seat rename <from> <to> rename a seat; its former name still resolves (ADR 0122)
seat <name> --to <node>/<module> hand a seat to that assignment as one act; never empty in between (ADR 0131)
board [--listen ADDR] the same three questions, as a page that holds nothing board [--listen ADDR] the same three questions, as a page that holds nothing
api --issuer URL [--listen A] assign and unassign over http, for a surface that is not here api --issuer URL [--listen A] assign and unassign over http, for a surface that is not here
assign <node> <module> put a module on a node assign <node> <module> put a module on a node
unassign <node> <module> take it off unassign <node> <module> take it off
take <node> <module> cut a module over on an adopted node, once its data has moved take <node> <module> preview a module's cutover on an adopted node: what runs beside
what it declares; --yes <digest> cuts it over as previewed
converge <node> [--yes <digest>] [--filter nftables] preview, then make, an adopted node converged converge <node> [--yes <digest>] [--filter nftables] preview, then make, an adopted node converged
adopt <node> return a converged node to adopted; what was taken stays taken adopt <node> return a converged node to adopted; what was taken stays taken
settings set <module> <file> what a module's config should say, for the whole mesh settings set <module> <file> what a module's config should say, for the whole mesh
@@ -201,7 +206,8 @@ func usage() {
licence refresh <name> mint a new access token and seal it to every holder licence refresh <name> mint a new access token and seal it to every holder
rotate <provision> [--consumer <n>] a new credential for every holder, both ends at once rotate <provision> [--consumer <n>] a new credential for every holder, both ends at once
ask <module> <tool> [json] call one of a module's tools over the broker, and print its answer ask <module> <tool> [json] call one of a module's tools over the broker, and print its answer
pin <node> <provision> <from> which node this one gets a provision from pin <node> <provision> <from-node> <module>
which provider this one gets a provision from: the module, and its node
unpin <node> <provision> put that question back unpin <node> <provision> put that question back
plan <node> [--files|--json] what that node would run, and why plan <node> [--files|--json] what that node would run, and why
push [<node>] [--behind] send a node everything it should be, or only those that need it push [<node>] [--behind] send a node everything it should be, or only those that need it
@@ -241,6 +247,31 @@ func parseAround(set *flag.FlagSet, args []string) ([]string, error) {
} }
} }
// Built is the daemon hearing a build's outcome on the bus — its own asking, an announcement's, or
// a tool's that did not wait (novox/hq issue 176) — and taking it in: recorded, and the module
// registered, the same as the waiting command does. Said either way, so the daemon's log tells what
// became of a build nobody was watching.
func (b builds) Built(ctx context.Context, result link.BuildResult) error { func (b builds) Built(ctx context.Context, result link.BuildResult) error {
return b.inv.RecordBuild(ctx, buildFrom(result)) manifest, _, err := takeIn(ctx, b.inv, result)
switch {
case err != nil && result.Failed != "":
fmt.Printf("%s: %v\n", result.ID, err)
if result.Module != "" {
planBuilt(ctx, b.open, result.Module, result.Commit, result.Failed)
} else {
planFailedBuild(ctx, b.open, result)
}
return nil
case err != nil:
fmt.Printf("%s: heard and recorded, and not registered: %v\n", result.ID, err)
if manifest.Module != "" {
planBuilt(ctx, b.open, manifest.Module, result.Commit, err.Error())
}
return nil
}
fmt.Printf("%s: %s %s registered, built on %s from %s\n",
result.ID, manifest.Module, manifest.Version, result.On, short(result.Commit))
saysWhenThePolicyActs(ctx, b.inv, manifest.Module)
planBuilt(ctx, b.open, manifest.Module, result.Commit, "")
return nil
} }
+141 -16
View File
@@ -113,6 +113,9 @@ func moduleCommand(ctx context.Context, args []string) error {
if err != nil { if err != nil {
return err return err
} }
if err := namesNoInstallation(m); err != nil {
return err
}
if err := inv.RegisterModule(ctx, m, from); err != nil { if err := inv.RegisterModule(ctx, m, from); err != nil {
return err return err
} }
@@ -144,6 +147,40 @@ func moduleCommand(ctx context.Context, args []string) error {
if err != nil { if err != nil {
return err return err
} }
// **The same list, for something other than a person** (novox/hq ADR 0195): what each module
// is, where it runs, whether it is current, and what it says of itself.
if len(args) > 1 && args[1] == "--json" {
type listed struct {
Module string `json:"module"`
Version string `json:"version"`
Built string `json:"built,omitempty"`
Head string `json:"head,omitempty"`
Current bool `json:"current"`
Provided bool `json:"provided,omitempty"`
// Tools says whether the module answers tools anywhere it runs: a list of its own,
// a bundle the runtime serves, or a seat's verbs it claims (novox/hq ADR 0197) —
// what the console checks the bus's answers against.
Tools bool `json:"tools"`
On []string `json:"on"`
Provides []string `json:"provides,omitempty"`
Requires []string `json:"requires,omitempty"`
Claims []string `json:"claims,omitempty"`
Capabilities []string `json:"capabilities,omitempty"`
}
out := make([]listed, 0, len(entries))
for _, e := range entries {
m := e.Manifest
l := listed{Module: m.Module, Version: m.Version, Built: e.Source.BuiltFrom, Head: e.Source.Head,
Current: e.Provided || e.Source.Repository == "" || e.Source.Current(), Provided: e.Provided,
On: append([]string{}, e.On...), Provides: m.Offers(), Requires: m.Requires,
Capabilities: m.Capabilities, Tools: declaresTools(m)}
for _, c := range m.Claims {
l.Claims = append(l.Claims, c.At()+"/"+c.Name)
}
out = append(out, l)
}
return printJSON(out)
}
if len(entries) == 0 { if len(entries) == 0 {
fmt.Println("this mesh knows about no modules yet") fmt.Println("this mesh knows about no modules yet")
return nil return nil
@@ -349,10 +386,14 @@ func settingsCommand(ctx context.Context, args []string) error {
switch args[0] { switch args[0] {
case "set": case "set":
if len(positionals) != 2 { if len(positionals) != 2 {
return errors.New("settings set <module> <settings.json> [--node <node>]") return errors.New("settings set <module> <settings.json | {…}> [--node <node>]")
} }
raw, err := os.ReadFile(positionals[1]) // A file, or the values themselves when they begin with `{` — which is how the mesh's own
if err != nil { // `settings` tool passes them, having no file to hand over (novox/hq issue 198).
var raw []byte
if strings.HasPrefix(strings.TrimSpace(positionals[1]), "{") {
raw = []byte(positionals[1])
} else if raw, err = os.ReadFile(positionals[1]); err != nil {
return err return err
} }
var values map[string]any var values map[string]any
@@ -408,8 +449,8 @@ func describeOffers(offers []catalogue.Offer) string {
// database should not change where an existing machine gets its data the day a second one // database should not change where an existing machine gets its data the day a second one
// arrives. // arrives.
func pinCommand(ctx context.Context, args []string, setting bool) error { func pinCommand(ctx context.Context, args []string, setting bool) error {
if setting && len(args) != 3 { if setting && len(args) != 4 {
return errors.New("pin <node> <provision> <from-node>") return errors.New("pin <node> <provision> <from-node> <module>")
} }
if !setting && len(args) != 2 { if !setting && len(args) != 2 {
return errors.New("unpin <node> <provision>") return errors.New("unpin <node> <provision>")
@@ -428,17 +469,12 @@ func pinCommand(ctx context.Context, args []string, setting bool) error {
fmt.Printf("%s is no longer told where to get %s from\n", args[0], args[1]) fmt.Printf("%s is no longer told where to get %s from\n", args[0], args[1])
return nil return nil
} }
if args[0] == args[2] { // The provider's node may be this same machine: two modules beside the consumer can both
// Allowed by nothing here, and worth saying rather than resolving into a confusing // answer a provision, and then the module is the whole question (novox/hq #258).
// refusal later: a node providing something to itself is a node-scoped provision, and if err := inv.PinProvision(ctx, args[0], args[1], args[2], args[3]); err != nil {
// this field is for the other kind.
return fmt.Errorf("%s cannot get %s from itself; that would be a provision this machine "+
"provides, which does not need saying", args[0], args[1])
}
if err := inv.PinProvision(ctx, args[0], args[1], args[2]); err != nil {
return err return err
} }
fmt.Printf("%s gets %s from %s\n", args[0], args[1], args[2]) fmt.Printf("%s gets %s from %s/%s\n", args[0], args[1], args[2], args[3])
fmt.Printf(" run `push %s` to send it\n", args[0]) fmt.Printf(" run `push %s` to send it\n", args[0])
return nil return nil
} }
@@ -555,7 +591,7 @@ func issueOnTheNewBus(ctx context.Context, inv *inventory.Inventory, m catalogue
user := broker.Principal{Kind: broker.KindModule, Node: node, Module: m.Module}.Username() user := broker.Principal{Kind: broker.KindModule, Node: node, Module: m.Module}.Username()
password, err := inv.MintBusPassword(ctx, inventory.BusUser{ password, err := inv.MintBusPassword(ctx, inventory.BusUser{
Username: user, Kind: inventory.BusModule, Node: node, Module: m.Module, Username: user, Kind: busKindOf(m.Module), Node: node, Module: m.Module,
}) })
if err != nil { if err != nil {
return err return err
@@ -575,12 +611,30 @@ func issueOnTheNewBus(ctx context.Context, inv *inventory.Inventory, m catalogue
return issueWith(ctx, inv, m, node, busAddress, known, reachable, user, password) return issueWith(ctx, inv, m, node, busAddress, known, reachable, user, password)
} }
// busKindOf is what a module's bus user is recorded as: the node's tool runtime where the module is
// the runtime (novox/hq ADR 0175), a module otherwise. The username is the same either way — the
// runtime is issued through this same path — and the kind is what a reader of the records sees.
func busKindOf(module string) string {
if module == catalogue.RuntimeModule {
return inventory.BusNodeTools
}
return inventory.BusModule
}
// issueWith is the delivery half: the minted password sealed to the machine as the module's broker // issueWith is the delivery half: the minted password sealed to the machine as the module's broker
// secret, and the module's consumer created where the bus can be reached. Split from the minting // secret, and the module's consumer created where the bus can be reached. Split from the minting
// so the move can issue every module against a bus whose address it worked out itself // so the move can issue every module against a bus whose address it worked out itself
// (`rollout mint`, design 28 task 5.2) rather than the one in this process's environment. // (`rollout mint`, design 28 task 5.2) rather than the one in this process's environment.
func issueWith(ctx context.Context, inv *inventory.Inventory, m catalogue.Manifest, func issueWith(ctx context.Context, inv *inventory.Inventory, m catalogue.Manifest,
node, busAddress string, known broker.Broker, reachable, user, password string) error { node, busAddress string, known broker.Broker, reachable, user, password string) error {
// The seats this module claims, with the verbs each promises (novox/hq ADR 0159): the runtime
// serves a claimed seat's verbs with its tools of the same name, and the bus admits only the
// holder's subscription — so the runtime tries each claim and the grant decides. Written here
// because this file is the one thing the mesh writes that the runtime reads before it speaks.
claims, err := claimsFor(ctx, inv, m)
if err != nil {
return err
}
held, err := json.Marshal(struct { held, err := json.Marshal(struct {
URL string `json:"url"` URL string `json:"url"`
Fingerprint string `json:"fingerprint,omitempty"` Fingerprint string `json:"fingerprint,omitempty"`
@@ -588,9 +642,10 @@ func issueWith(ctx context.Context, inv *inventory.Inventory, m catalogue.Manife
Module string `json:"module"` Module string `json:"module"`
User string `json:"user"` User string `json:"user"`
Password string `json:"password"` Password string `json:"password"`
Claims []seatClaimed `json:"claims,omitempty"`
}{ }{
URL: "nats://" + reachable, Fingerprint: known.Fingerprint, URL: "nats://" + reachable, Fingerprint: known.Fingerprint,
Node: node, Module: m.Module, User: user, Password: password, Node: node, Module: m.Module, User: user, Password: password, Claims: claims,
}) })
if err != nil { if err != nil {
return err return err
@@ -662,3 +717,73 @@ func whereItComesFrom(repository, ref, commit, path string, self bool) (inventor
} }
return from, nil return from, nil
} }
// namesNoInstallation is the mesh refusing a definition that names an installation, at the moment
// it would enter the catalogue (novox/hq ADR 0112, ADR 0155). `module check` says the same thing
// earlier, where the author is; this is the last moment the mesh can still say no, and a
// definition that got past the check — written elsewhere, or checked by nobody — is refused here
// in the same words. A name meant on purpose is declared with its reason and passes.
func namesNoInstallation(m catalogue.Manifest) error {
named := catalogue.InstallationProblems(m)
if len(named) == 0 {
return nil
}
return fmt.Errorf("%s names an installation, and a definition names none — declare a name meant "+
"on purpose under %s with its reason, or take it out:\n - %s",
m.Module, catalogue.NamesOnPurpose, strings.Join(named, "\n - "))
}
// seatClaimed is one seat a module claims, as its runtime needs it: the name, the scope (a
// node-scoped seat's verb carries the machine, design 33 §4) and the verbs the seat promises.
type seatClaimed struct {
Seat string `json:"seat"`
Scope string `json:"scope"`
Serves []string `json:"serves,omitempty"`
}
// claimsFor joins a module's claims with the seats' protocols from the mesh's records.
func claimsFor(ctx context.Context, inv *inventory.Inventory, m catalogue.Manifest) ([]seatClaimed, error) {
if len(m.Claims) == 0 {
return nil, nil
}
seats, err := inv.Seats(ctx)
if err != nil {
return nil, err
}
byName := map[string]catalogue.Seat{}
for _, s := range seats {
byName[s.Name] = s
}
var out []seatClaimed
for _, c := range m.Claims {
claimed := seatClaimed{Seat: c.Name, Scope: c.At()}
if s, known := byName[c.Name]; known {
claimed.Scope = s.Scope
// The verbs the runtime serves for the seat: the claim's own when it names them
// (ADR 0160), else every verb the seat promises, which its tools then answer.
claimed.Serves = c.ServesFor(catalogue.Manifest{Tools: catalogue.VerbNames(s.Serves)})
}
out = append(out, claimed)
}
return out, nil
}
// declaresTools is whether a module answers tools wherever it runs (novox/hq ADR 0197): it names
// tools of its own, its build delivers a bundle the node's runtime serves, or it claims a seat
// whose verbs it serves. A module with none is never expected to announce anything.
func declaresTools(m catalogue.Manifest) bool {
if len(m.Tools) > 0 {
return true
}
for _, b := range m.Bundles {
if len(b.Loads) > 0 {
return true
}
}
for _, c := range m.Claims {
if len(c.Serves) > 0 {
return true
}
}
return false
}
+2 -2
View File
@@ -11,7 +11,7 @@ import (
// A module that declares none is refused before the account exists, so the bus never carries an // A module that declares none is refused before the account exists, so the bus never carries an
// account nothing reads (novox/hq 04-ISSUES/078). // account nothing reads (novox/hq 04-ISSUES/078).
func TestAModuleWithNoBrokerSecretCannotBeIssued(t *testing.T) { func TestAModuleWithNoBrokerSecretCannotBeIssued(t *testing.T) {
err := mayIssue(catalogue.Manifest{Module: "step-ca", OwnSecrets: map[string]string{"password": "/run/password"}}) err := mayIssue(catalogue.Manifest{Module: "step-ca", OwnSecrets: catalogue.OwnSecrets{"password": {Path: "/run/password"}}})
if err == nil { if err == nil {
t.Fatal("a module with no broker own secret was issued an account") t.Fatal("a module with no broker own secret was issued an account")
} }
@@ -20,7 +20,7 @@ func TestAModuleWithNoBrokerSecretCannotBeIssued(t *testing.T) {
t.Errorf("the refusal does not say %q: %v", want, err) t.Errorf("the refusal does not say %q: %v", want, err)
} }
} }
if err := mayIssue(catalogue.Manifest{Module: "redis", OwnSecrets: map[string]string{"broker": "/run/broker"}}); err != nil { if err := mayIssue(catalogue.Manifest{Module: "redis", OwnSecrets: catalogue.OwnSecrets{"broker": {Path: "/run/broker"}}}); err != nil {
t.Errorf("a module declaring its broker secret was refused: %v", err) t.Errorf("a module declaring its broker secret was refused: %v", err)
} }
} }
+3
View File
@@ -567,6 +567,9 @@ func onTheNetwork(ctx context.Context, inv *inventory.Inventory,
catalogue.Node{Name: p.Name, Site: p.Site, Capabilities: caps}, catalogue.Node{Name: p.Name, Site: p.Site, Capabilities: caps},
catalogue.World{Unchecked: true, Holdings: holdings}) catalogue.World{Unchecked: true, Holdings: holdings})
if err != nil { if err != nil {
// Said, not skipped in silence: a machine dropped here loses its address, and every
// plan that names it fails in another module's words (novox/hq issue 188).
fmt.Fprintf(os.Stderr, "%s is not counted as on the network: it does not resolve: %v\n", p.Name, err)
continue continue
} }
for _, m := range got.Modules { for _, m := range got.Modules {
+26
View File
@@ -3,6 +3,7 @@ package main
import ( import (
"context" "context"
"os" "os"
"reflect"
"strings" "strings"
"testing" "testing"
@@ -303,3 +304,28 @@ func TestTheResolverIsToldEveryMachineOnTheNetworkAndToldAgainWhenOneLeaves(t *t
t.Fatalf("a machine that left the network is still a wildcard, or the one that stayed is not:\n%s", after) t.Fatalf("a machine that left the network is still a wildcard, or the one that stayed is not:\n%s", after)
} }
} }
// The roster is the machines and nothing else (novox/hq ADR 0191): each node's internal domain covers
// every route on it, and a node's public domains are public DNS's. A routed name in `.Names` was a
// private answer for a public name, handed by a resolver serving a LAN to a phone that could not use it.
func TestTheRosterNamesOnlyTheMachines(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
gens, err := generators(ctx, open)
if err != nil {
t.Fatal(err)
}
for _, node := range []string{"anchor", "laptop"} {
plan, settings, err := planFor(ctx, open, node)
if err != nil {
t.Fatal(err)
}
with, _, err := renderingFor(ctx, open, node, plan, settings, gens, Reading)
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(with.Names, with.Machines) {
t.Fatalf("%s's roster names more than the machines:\n names %v\n machines %v", node, with.Names, with.Machines)
}
}
}
+26
View File
@@ -2,6 +2,7 @@ package main
import ( import (
"context" "context"
"encoding/json"
"errors" "errors"
"flag" "flag"
"fmt" "fmt"
@@ -44,6 +45,21 @@ func nodeCommand(ctx context.Context, args []string) error {
if err != nil { if err != nil {
return err return err
} }
// **The same list, for something other than a person** — the console's discovery reads it
// (novox/hq ADR 0195), and a reader that parses a printed column breaks when it is reworded.
if len(args) > 1 && args[1] == "--json" {
type listed struct {
Name string `json:"name"`
Heard string `json:"heard"`
Mode string `json:"mode"`
ID string `json:"id"`
}
out := make([]listed, 0, len(nodes))
for _, n := range nodes {
out = append(out, listed{Name: n.Name, Heard: heardFrom(n), Mode: modeOf(n), ID: n.ID})
}
return printJSON(out)
}
if len(nodes) == 0 { if len(nodes) == 0 {
// Said rather than printed as nothing: an empty list and a failed read must never // Said rather than printed as nothing: an empty list and a failed read must never
// look the same, and this command answering "none" is only honest because getting // look the same, and this command answering "none" is only honest because getting
@@ -500,3 +516,13 @@ func orNotReported(s string) string {
} }
return s return s
} }
// printJSON prints a value as indented JSON, the shape every `--json` answers in.
func printJSON(v any) error {
body, err := json.MarshalIndent(v, "", " ")
if err != nil {
return err
}
fmt.Println(string(body))
return nil
}
+2 -1
View File
@@ -1,6 +1,7 @@
package main package main
import ( import (
"reflect"
"strings" "strings"
"testing" "testing"
"time" "time"
@@ -191,7 +192,7 @@ func TestWhatAHandedOverModuleRecordsAboutItsSource(t *testing.T) {
t.Fatalf("the source records as %+v", from) t.Fatalf("the source records as %+v", from)
} }
// A manifest with no provenance at all is legitimate: fixing something in a hurry. // A manifest with no provenance at all is legitimate: fixing something in a hurry.
if from, err := whereItComesFrom("", "", "", "", false); err != nil || from != (inventory.Source{}) { if from, err := whereItComesFrom("", "", "", "", false); err != nil || !reflect.DeepEqual(from, inventory.Source{}) {
t.Fatalf("a manifest handed over with no provenance was refused: %+v, %v", from, err) t.Fatalf("a manifest handed over with no provenance was refused: %+v, %v", from, err)
} }
for _, c := range []struct { for _, c := range []struct {
+121 -152
View File
@@ -7,6 +7,7 @@ import (
"errors" "errors"
"flag" "flag"
"fmt" "fmt"
"os"
"sort" "sort"
"strings" "strings"
@@ -15,8 +16,6 @@ import (
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/licences" "github.com/novox/mesh-controller/internal/licences"
"github.com/novox/mesh-controller/internal/overlay" "github.com/novox/mesh-controller/internal/overlay"
"net"
"strconv"
) )
// working out what one machine should be. // working out what one machine should be.
@@ -163,7 +162,14 @@ func planFor(ctx context.Context, open *stores, nodeName string) (catalogue.Reso
} }
continue continue
} }
secret, err := inv.SecretFor(ctx, n.Name, nodeName, n.For, n.From, n.Local) var secret inventory.Secret
var err error
if n.SharedOwn != "" {
// The provider's one credential, sealed to this consumer too (novox/hq ADR 0158).
secret, err = inv.SharedSecretFor(ctx, n.Name, nodeName, n.For, n.From, providerModuleOf(resolved, open, ctx, n), n.Local, n.SharedOwn)
} else {
secret, err = inv.SecretFor(ctx, n.Name, nodeName, n.For, n.From, n.Local)
}
if err != nil { if err != nil {
// Said rather than skipped. A machine that resolves cleanly and receives no // Said rather than skipped. A machine that resolves cleanly and receives no
// credential is one that will fail to authenticate at some later, less obvious // credential is one that will fail to authenticate at some later, less obvious
@@ -178,8 +184,12 @@ func planFor(ctx context.Context, open *stores, nodeName string) (catalogue.Reso
// Settings for everything that resolved, including modules nobody assigned directly: a // Settings for everything that resolved, including modules nobody assigned directly: a
// requirement pulled in by something else is still configurable, and finding out that it is // requirement pulled in by something else is still configurable, and finding out that it is
// not only when you try would be an arbitrary line nobody could predict. // not only when you try would be an arbitrary line nobody could predict.
//
// A setting that reaches nothing, or cannot compose with the definition it was stored for,
// no longer refuses the machine here: it is judged where it is stored, and a definition that
// moved under it costs that module its place in the declaration, said by name (novox/hq ADR
// 0163, rule 6 — see Compose).
settings := catalogue.SettingsBy{} settings := catalogue.SettingsBy{}
var stray []string
for _, m := range resolved.Modules { for _, m := range resolved.Modules {
layers, err := inv.SettingsFor(ctx, nodeName, m.Module) layers, err := inv.SettingsFor(ctx, nodeName, m.Module)
if err != nil { if err != nil {
@@ -189,18 +199,6 @@ func planFor(ctx context.Context, open *stores, nodeName string) (catalogue.Reso
continue continue
} }
settings[m.Module] = layers settings[m.Module] = layers
stray = append(stray, catalogue.UnusedSettings(m, layers)...)
}
if len(stray) > 0 {
// Somebody set something that reaches no file. Said here rather than discovered by the
// machine not behaving differently, which is the slowest way there is.
//
// Marked like a set that will not compose, and for the same reason: it is a standing fact
// about this node's own configuration, not a question the mesh could not answer. A gatherer
// passes over it as it always did — one node's stray setting must not stop every other node
// being described (novox/hq 04-ISSUES/152).
return catalogue.Resolution{}, nil, notResolvable{fmt.Errorf(
"these settings reach nothing:\n - %s", strings.Join(stray, "\n - "))}
} }
return resolved, settings, nil return resolved, settings, nil
} }
@@ -274,8 +272,9 @@ func theRestOfTheMesh(ctx context.Context, inv *inventory.Inventory,
for _, o := range others { for _, o := range others {
got, err := catalogue.Resolve(shelf, o.assigned, o.node, catalogue.World{Unchecked: true, Holdings: holdings}) got, err := catalogue.Resolve(shelf, o.assigned, o.node, catalogue.World{Unchecked: true, Holdings: holdings})
if err != nil { if err != nil {
// Their set does not resolve for some other reason. Not this node's problem to // Said, not skipped: a machine dropped here offers nothing and holds nothing as far
// report, and nothing of theirs is running, so it offers nothing. // as every other machine's plan can tell (novox/hq issue 188).
fmt.Fprintf(os.Stderr, "%s is left out of the rest of the mesh: it does not resolve: %v\n", o.node.Name, err)
continue continue
} }
firstHeld = append(firstHeld, got.Claims...) firstHeld = append(firstHeld, got.Claims...)
@@ -306,8 +305,22 @@ func theRestOfTheMesh(ctx context.Context, inv *inventory.Inventory,
world := catalogue.World{Offered: offered, Held: firstHeld, Holdings: holdings} world := catalogue.World{Offered: offered, Held: firstHeld, Holdings: holdings}
var held []catalogue.Held var held []catalogue.Held
for _, o := range others { for _, o := range others {
got, err := catalogue.Resolve(shelf, o.assigned, o.node, world) // Each machine is resolved with its own pins, as its plan is: a machine that needs one to
// settle two providers would otherwise be refused here and vanish from the mesh — every
// seat it holds unheld, every build that needs one refused (2026-10-01, the control node;
// novox/hq issue 188).
theirs := world
if pins, err := inv.PinsFor(ctx, o.node.Name); err == nil {
theirs.Pinned = pins
}
got, err := catalogue.Resolve(shelf, o.assigned, o.node, theirs)
if err != nil { if err != nil {
// Said only for the whole-mesh view. With one machine excluded, the others are
// resolved without its offers, and one that consumes them cannot resolve here by
// design — that is not the machine being dropped, it is the view being partial.
if exclude == "" {
fmt.Fprintf(os.Stderr, "%s is left out of the rest of the mesh: it does not resolve: %v\n", o.node.Name, err)
}
continue continue
} }
held = append(held, got.Claims...) held = append(held, got.Claims...)
@@ -381,7 +394,33 @@ func declarationWith(ctx context.Context, open *stores, node string,
if err != nil { if err != nil {
return sendable{}, err return sendable{}, err
} }
return sendable{Resources: composed.Resources, Adoption: adoption}, nil return sendable{Resources: composed.Resources, Adoption: adoption,
Received: composed.Received, Mesh: with.Mesh,
LeftOut: sortedKeysOf(composed.LeftOut), leftOutWhy: composed.LeftOut}, nil
}
// sortedKeysOf is a map's keys, sorted — so what a declaration says it left out does not move
// for a reordering nobody made.
func sortedKeysOf(m map[string]string) []string {
if len(m) == 0 {
return nil
}
out := make([]string, 0, len(m))
for k := range m {
out = append(out, k)
}
sort.Strings(out)
return out
}
// reportLeftOut says which of a machine's modules its declaration leaves out and why (novox/hq ADR
// 0163, rule 6), one line each: the machine is told everything else, and is told it was left out.
func reportLeftOut(node string, declared sendable) {
for _, m := range declared.LeftOut {
fmt.Printf("%s: %s left out — a setting stored for it cannot compose with its definition; "+
"what the machine holds for it is kept and its containers are untouched. %s\n",
node, m, declared.leftOutWhy[m])
}
} }
// renderingFor is everything a node's declaration is composed with, and the node's record. // renderingFor is everything a node's declaration is composed with, and the node's record.
@@ -421,7 +460,10 @@ func renderingFor(ctx context.Context, open *stores, node string,
for _, m := range plan.Modules { for _, m := range plan.Modules {
g, err := catalogue.GivenPorts(m, settings[m.Module]) g, err := catalogue.GivenPorts(m, settings[m.Module])
if err != nil { if err != nil {
return catalogue.Rendering{}, inventory.Node{}, err // A given port its definition no longer publishes: the module is left out of the
// declaration, by name, when it is composed (novox/hq ADR 0163, rule 6) — never the
// machine refused here for it.
continue
} }
if g != nil { if g != nil {
given[m.Module] = g given[m.Module] = g
@@ -491,6 +533,23 @@ func renderingFor(ctx context.Context, open *stores, node string,
var sealed string var sealed string
var err error var err error
if choosing == Allocating { if choosing == Allocating {
// **The broker credential is never invented here** (novox/hq issue 203). Every other
// own secret is the mesh's to make — a password nobody else knows — but this one
// is an account on the bus, minted by `module issue` and sealed by it; a push that
// made a random one would deliver a file the process cannot read and report the
// machine applied. Refused by name, with the verb.
if name == "broker" {
user := broker.Principal{Kind: broker.KindModule, Node: node, Module: m.Module}.Username()
if _, minted, err := inv.BusUserHash(ctx, user); err != nil {
return catalogue.Rendering{}, inventory.Node{}, err
} else if !minted {
return catalogue.Rendering{}, inventory.Node{}, fmt.Errorf(
"%s on %s has no bus credential: nothing was issued for %s, and a push "+
"would seal a placeholder its process cannot read (novox/hq issue 203). "+
"`module issue %s --node %s`, then push again",
m.Module, node, user, m.Module, node)
}
}
sealed, err = inv.SecretForModule(ctx, node, m.Module, name) sealed, err = inv.SecretForModule(ctx, node, m.Module, name)
} else { } else {
var held bool var held bool
@@ -586,43 +645,24 @@ func renderingFor(ctx context.Context, open *stores, node string,
} }
} }
// And every routed name → the node that serves it (novox/hq ADR 0066). Alongside the // **The roster is the machines and nothing else** (novox/hq ADR 0191). Each node has one internal
// `<node>.internal` names above, so a container — or an internal ACME validator — resolves a // domain, `<node>.internal`, and every route on it is a name under that domain (ADR 0151), which
// routed name to the proxy that serves it, mesh-wide. The mesh publishes the names it was told // the resolver answers with one wildcard per machine — so no route needs a line of its own. A
// to serve and knows nothing about what they mean. // node's public domains are the operator's and public DNS answers them; the mesh gives no private
// Kept apart from the machines, because a fact about the machines must not be handed the names // answer for any of them. The roster once carried every routed name, public ones included, and a
// the mesh merely serves (novox/hq 04-ISSUES/111). // resolver that also serves a LAN handed a phone a tunnel address for the mail server.
// `.Names` and `.Machines` stay two fields so a module's template keeps rendering (issue 111).
machines := make(map[string]string, len(names)) machines := make(map[string]string, len(names))
for name, at := range names { for name, at := range names {
machines[name] = at machines[name] = at
} }
routes, err := routeNamesInTheMesh(ctx, open)
if err != nil {
return catalogue.Rendering{}, inventory.Node{}, err
}
for name, at := range routes {
names[name] = at
}
// The ports the mesh itself needs open, which no module declares. Read from the broker this // **The bus is never public** (novox/hq ADR 0169). It was a foundation port — widened from the
// control plane was told about rather than written down twice: the address a node is handed in // broker's own `from: mesh` to from-anywhere on the broker's host, so a machine could enrol
// its token and the port its machine must accept on are the same fact. // before it had an address on the private network. A machine joins through the tunnel now, and
// // every link to the bus crosses it, so its reach is what the `nats` module declares: the mesh.
// **Only on the node that listens on it** (novox/hq issue: the broker opening leaked onto // Nothing the mesh itself needs is opened beyond what a module declares.
// every node). The opening exists to WIDEN the broker's port to from-anywhere — a machine
// enrolling is not on the mesh yet, so the broker's own `from: mesh` listen would refuse its
// first dial. That widening belongs on the broker's host and nowhere else: a node that only
// dials out needs no incoming rule, and an opening for a port nothing here listens on is a
// from-anywhere hole for a dead port. So the foundation port is kept only when a module
// resolved onto THIS node actually listens on it.
var foundation []int var foundation []int
if b, err := broker.FromEnvironment(); err == nil {
if _, port, err := net.SplitHostPort(b.Address); err == nil {
if n, err := strconv.Atoi(port); err == nil {
foundation = foundationPortsFor(n, plan.Modules)
}
}
}
// And, for a module that keeps them, every operator-sealed secret in the mesh — the vault's // And, for a module that keeps them, every operator-sealed secret in the mesh — the vault's
// copy, outside the store (novox/hq ADR 0085, amended). Read only; nothing here mints. The // copy, outside the store (novox/hq ADR 0085, amended). Read only; nothing here mints. The
@@ -694,94 +734,6 @@ func renderingFor(ctx context.Context, open *stores, node string,
}, record, nil }, record, nil
} }
// routeNamesInTheMesh is every routed name and the address of the node that serves it (novox/hq
// ADR 0066).
//
// **Mesh-wide, so any container resolves any routed name to its proxy** — including an internal
// ACME validator, which cannot complete a challenge for a name it cannot reach. A routed name is
// composed on the consumer's node (from its label and that node's public domain) and served by the
// node answering the consumer's route requirement; this gathers both.
//
// It reads route names off resolutions rather than a table because there is no table: a route is a
// contribution, computed from what each node runs. Name-agnostic — a contribution counts as a
// routed name only because it carried a label the mesh composed, never because the mesh knows what
// "route" means. A node that does not resolve is skipped, so one machine's broken set does not cost
// the rest their names.
//
// **A node that could not be READ is a different matter and is raised.** Skipping one states, to
// every machine at once, that its names do not exist — and since the roster is part of every
// container's identity, that withdraws them and replaces every container (novox/hq 04-ISSUES/152,
// 151). So every failure here says which machine and which read, because the alternative is a
// mesh-wide refusal with nothing named in it.
func routeNamesInTheMesh(ctx context.Context, open *stores) (map[string]string, error) {
inv := open.inventory
places, err := inv.Overlays(ctx)
if err != nil {
return nil, fmt.Errorf("where the machines are cannot be read: %w", err)
}
address := map[string]string{}
for _, p := range places {
if strings.TrimSpace(p.Address) != "" {
address[p.Name] = p.Address
}
}
nodes, err := inv.Nodes(ctx)
if err != nil {
return nil, fmt.Errorf("which machines the mesh has cannot be read: %w", err)
}
out := map[string]string{}
for _, n := range nodes {
plan, settings, err := planFor(ctx, open, n.Name)
switch {
case unresolvable(err):
// Their set does not compose, so they serve no names. Passed over, so one machine's
// broken set does not cost the rest theirs.
continue
case err != nil:
// The mesh could not be asked. Returning the roster without this machine's names would
// state that they do not exist — to every machine, and indistinguishably from the
// operator having withdrawn them (novox/hq 04-ISSUES/152).
return nil, fmt.Errorf("the names %s serves cannot be read: %w", n.Name, err)
}
for _, m := range plan.Modules {
for to := range m.Contributes {
values, asks, err := plan.ContributionsFrom(to, m.Module, settings)
if err != nil {
return nil, err
}
if !asks {
continue
}
// A routed name, and only that: a contribution the mesh composed a name for from a
// label it was given. A grant that happens to carry a `name` of its own — a database
// name — carries no label and is left alone.
if _, labelled := values["label"]; !labelled {
continue
}
name, _ := values["name"].(string)
if name == "" {
continue
}
// The node that serves it: whoever answers this consumer's route requirement, or
// this same node when the proxy is beside the consumer.
serving := n.Name
for _, need := range plan.Needs {
if need.Name == to && need.For == m.Module {
serving = need.From
break
}
}
if at := address[serving]; at != "" {
out[strings.ToLower(name)] = at
}
}
}
}
return out, nil
}
// certificateFor is what the mesh certifies about one machine's internal name. // certificateFor is what the mesh certifies about one machine's internal name.
// //
// It reaches across two contexts and reads neither one's store from the other: `inventory` knows // It reaches across two contexts and reads neither one's store from the other: `inventory` knows
@@ -994,6 +946,20 @@ func planCommand(ctx context.Context, args []string) error {
return nil return nil
} }
// Which modules a push would leave out, and why — said before the plan, since the plan is of
// what the machine would be told (novox/hq ADR 0163, rule 6). Judged, never composed: `plan`
// without --json allocates nothing.
if record, err := open.inventory.NodeByName(ctx, args[0]); err == nil {
left := plan.LeftOut(settings, record.Adopted)
reportLeftOut(args[0], sendable{LeftOut: sortedKeysOf(left), leftOutWhy: left})
}
// And a setting that reaches nothing — refused where it is stored, and said here for one
// stored before its definition moved from under it.
for _, m := range plan.Modules {
for _, stray := range catalogue.UnusedSettings(m, settings[m.Module]) {
fmt.Printf("%s: a setting reaches nothing — %s\n", args[0], stray)
}
}
fmt.Printf("%s would run:\n", args[0]) fmt.Printf("%s would run:\n", args[0])
for _, m := range plan.Modules { for _, m := range plan.Modules {
fmt.Printf(" %-20s %s\n", m.Module, plan.Because[m.Module]) fmt.Printf(" %-20s %s\n", m.Module, plan.Because[m.Module])
@@ -1340,19 +1306,22 @@ func composeBusUsers(ctx context.Context, inv *inventory.Inventory,
return broker.ComposeAccounts(filled) return broker.ComposeAccounts(filled)
} }
// foundationPortsFor is the broker port, kept only when a module resolved onto this node listens // providerModuleOf is which module answers a need on the providing node: the one in this node's
// on it (novox/hq issue: the broker opening leaked onto every node). The foundation opening // own set when the provider is here, else the one the catalogue says offers it.
// exists to WIDEN the broker's `from: mesh` port to from-anywhere, because a machine enrolling is func providerModuleOf(resolved catalogue.Resolution, open *stores, ctx context.Context, n catalogue.Needed) string {
// not on the mesh yet and its first dial would be refused. That widening belongs on the broker's for _, m := range resolved.Modules {
// host alone: a node that only dials out needs no incoming rule, and an opening for a port if _, shared := m.SharedCredentialOf(n.Name); shared {
// nothing here listens on is a from-anywhere hole for a dead port. return m.Module
func foundationPortsFor(brokerPort int, modules []catalogue.Manifest) []int {
for _, m := range modules {
for _, l := range m.Listens {
if l.Port == brokerPort {
return []int{brokerPort}
} }
} }
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
return ""
} }
return nil for name, m := range shelf {
if _, shared := m.SharedCredentialOf(n.Name); shared {
return name
}
}
return ""
} }
+162 -41
View File
@@ -4,6 +4,7 @@ import (
"context" "context"
"crypto/sha256" "crypto/sha256"
"encoding/hex" "encoding/hex"
"encoding/json"
"errors" "errors"
"flag" "flag"
"fmt" "fmt"
@@ -112,7 +113,10 @@ func serve(ctx context.Context) error {
// while everything else about it looks correct. // while everything else about it looks correct.
// And build results nobody was waiting for. A build triggered any other way than `build` // And build results nobody was waiting for. A build triggered any other way than `build`
// would otherwise be reported into the void, which is the same as not reporting it. // would otherwise be reported into the void, which is the same as not reporting it.
server.Records(builds{inv}) server.Records(builds{inv, open})
// Open plans move on a timer as well as on outcomes (novox/hq ADR 0162): a tier waiting for
// machines to report moves when they have, and a plan left by a replaced controller resumes.
go planTicker(ctx, open)
// And what the catalogue decided a build meant. The builder's own result is already handled // And what the catalogue decided a build meant. The builder's own result is already handled
// above; this is the other half — the control plane is the only one of the three that knows // above; this is the other half — the control plane is the only one of the three that knows
// which machines run the thing, so it is the one that acts (novox/hq ADR 0072). // which machines run the thing, so it is the one that acts (novox/hq ADR 0072).
@@ -127,10 +131,17 @@ func serve(ctx context.Context) error {
// And the mesh's own verbs, as the seat this control plane holds (novox/hq ADR 0154). Served // And the mesh's own verbs, as the seat this control plane holds (novox/hq ADR 0154). Served
// from the store's row, so what the seat declares is what is answered. // from the store's row, so what the seat declares is what is answered.
handlers, err := seatToolHandlers() handlers, behind, err := seatToolHandlers()
if err != nil { if err != nil {
return err return err
} }
if len(behind) > 0 {
// Said once, loudly, and then served anyway (novox/hq ADR 0185): the mesh keeps answering
// while whatever put an older control plane here is undone.
fmt.Printf("this control plane is behind the %s row: it cannot run %s. "+
"Those answer the reason when called; everything else is served as usual\n",
catalogue.ControllerSeatName, strings.Join(behind, ", "))
}
bus, isNATS := server.Bus().(link.OverNATS) bus, isNATS := server.Bus().(link.OverNATS)
if !isNATS { if !isNATS {
return errors.New("the mesh's verbs are served over the bus, and this control plane is not on it") return errors.New("the mesh's verbs are served over the bus, and this control plane is not on it")
@@ -140,6 +151,12 @@ func serve(ctx context.Context) error {
return err return err
} }
defer stopServing() defer stopServing()
// And says so on the bus (novox/hq ADR 0197): what it serves, as the NATS services protocol asks.
stopAnnouncing, err := bus.Announce(seatAnnouncement(handlers), log.New(os.Stdout, "", log.LstdFlags))
if err != nil {
return err
}
defer stopAnnouncing()
return server.Serve(ctx) return server.Serve(ctx)
} }
@@ -194,6 +211,12 @@ func declare(ctx context.Context, args []string) error {
if err := link.Declare(ctx, server.Bus(), ident, node, raw, 15*time.Second); err != nil { if err := link.Declare(ctx, server.Bus(), ident, node, raw, 15*time.Second); err != nil {
return err return err
} }
// Written down like every other send (novox/hq issue 204): a declaration a person sent by hand
// is still what the machine was last told, and status must not read it as current for the one
// the mesh would compose.
if _, err := recordSent(ctx, inv, node, raw); err != nil {
return err
}
fmt.Printf("sent %s a signed declaration (%d bytes)\n", node, len(raw)) fmt.Printf("sent %s a signed declaration (%d bytes)\n", node, len(raw))
return nil return nil
} }
@@ -337,7 +360,7 @@ func pushCommand(ctx context.Context, args []string) error {
if err != nil { if err != nil {
return err return err
} }
sending, refusals := composeEach(asked, func(node string) (sendable, error) { sending, refusals := composeEach(asked, allotting(held, inv), func(node string) (sendable, error) {
plan, settings, err := planFor(held, open, node) plan, settings, err := planFor(held, open, node)
if err != nil { if err != nil {
return sendable{}, err return sendable{}, err
@@ -349,18 +372,18 @@ func pushCommand(ctx context.Context, args []string) error {
// The private network is in here with everything else. It used to be composed separately // The private network is in here with everything else. It used to be composed separately
// and prepended, which meant every machine with an address was on it and no machine could // and prepended, which meant every machine with an address was on it and no machine could
// be kept off. It is a module now, so it arrives the way a module does. // be kept off. It is a module now, so it arrives the way a module does.
return declarationWith(held, open, node, plan, settings, gens, Allocating) declared, err := declarationWith(held, open, node, plan, settings, gens, Allocating)
if err == nil {
reportLeftOut(node, declared)
}
return declared, err
}) })
sentDigest := map[string]string{} sentDigest := map[string]string{}
defer release() defer release()
for _, s := range sending { for _, s := range sending {
// Numbered under the hold, one higher than the last, before the body exists — the number is // The number is inside the signed bytes, so a replayed older declaration cannot borrow a
// inside the signed bytes, so a replayed older declaration cannot borrow a newer one's // newer one's (novox/hq 04-ISSUES/107); it was taken when the composition began (issue 204).
// (novox/hq 04-ISSUES/107).
if err := number(ctx, inv, &s); err != nil {
return err
}
body, err := s.declared.Body() body, err := s.declared.Body()
if err != nil { if err != nil {
return err return err
@@ -370,19 +393,20 @@ func pushCommand(ctx context.Context, args []string) error {
} }
// After it is away, not before. A digest recorded for something that failed to send would // After it is away, not before. A digest recorded for something that failed to send would
// make the machine look current for a declaration it never received. // make the machine look current for a declaration it never received.
record, err := inv.NodeByName(ctx, s.node) digest, err := recordSent(ctx, inv, s.node, body)
if err != nil { if err != nil {
return err return err
} }
digest := digestOf(body)
if err := inv.RecordSent(ctx, record.ID, digest); err != nil {
return err
}
sentDigest[s.node] = digest sentDigest[s.node] = digest
fmt.Printf("sent %s %d resource(s)\n", s.node, len(s.declared.Resources)) fmt.Printf("sent %s %d resource(s)\n", s.node, len(s.declared.Resources))
} }
release() release()
fmt.Printf("\n%d node(s) told\n", len(sending)) fmt.Printf("\n%d node(s) told\n", len(sending))
// And each machine's memberships, as every other send does (ADR 0160): a push is the one most
// operators run, and on 2026-10-01 it was the one path that issued none.
if err := issueMemberships(ctx, open, server, sending); err != nil {
return err
}
// **A named push leaves the mesh consistent, not just the machine it named** (novox/hq // **A named push leaves the mesh consistent, not just the machine it named** (novox/hq
// issue 057, ADR 0083). Assigning a cross-node consumer mints a provision, and the PROVIDER's // issue 057, ADR 0083). Assigning a cross-node consumer mints a provision, and the PROVIDER's
@@ -445,18 +469,18 @@ func pushCommand(ctx context.Context, args []string) error {
return sendable{}, err return sendable{}, err
} }
reportUnhostable(node, plan) reportUnhostable(node, plan)
return declarationWith(held, open, node, plan, settings, gens, Allocating) declared, err := declarationWith(held, open, node, plan, settings, gens, Allocating)
if err == nil {
reportLeftOut(node, declared)
}
return declared, err
}, },
func(s readyNode, body []byte) error { func(s readyNode, body []byte) error {
if err := link.Declare(ctx, server.Bus(), ident, s.node, body, if err := link.Declare(ctx, server.Bus(), ident, s.node, body,
15*time.Second); err != nil { 15*time.Second); err != nil {
return err return err
} }
record, err := inv.NodeByName(ctx, s.node) if _, err := recordSent(ctx, inv, s.node, body); err != nil {
if err != nil {
return err
}
if err := inv.RecordSent(ctx, record.ID, digestOf(body)); err != nil {
return err return err
} }
fmt.Printf("sent %s %d resource(s)\n", s.node, len(s.declared.Resources)) fmt.Printf("sent %s %d resource(s)\n", s.node, len(s.declared.Resources))
@@ -545,17 +569,31 @@ type readyNode struct {
// //
// The all-or-nothing rule is kept where it means something — sendTo, which rotates a credential // The all-or-nothing rule is kept where it means something — sendTo, which rotates a credential
// across two machines that must agree — and dropped here, where it never did. // across two machines that must agree — and dropped here, where it never did.
func composeEach(names []string, func composeEach(names []string, allot func(node string) (int64, error),
compose func(node string) (sendable, error)) ([]readyNode, []string) { compose func(node string) (sendable, error)) ([]readyNode, []string) {
var sending []readyNode var sending []readyNode
var refusals []string var refusals []string
for _, name := range names { for _, name := range names {
// **Numbered before it is composed, not before it is sent** (novox/hq issue 204). The
// number says where this declaration stands against every other the mesh composed for the
// machine, and the host refuses one lower than the last it applied. Taken at send time, as
// it was, a declaration composed a minute ago — before an assignment changed — went out with
// a number higher than one composed after the change and sent before it, and the machine
// took the older content as the newer word: on 2026-10-02 a runtime assigned and applied on
// two machines was undone two seconds later by exactly that. Taken here, before the first
// read, what was composed earlier is numbered lower whatever order the sends happen in.
seq, err := allot(name)
if err != nil {
refusals = append(refusals, fmt.Sprintf("%s:\n%v", name, err))
continue
}
declared, err := compose(name) declared, err := compose(name)
if err != nil { if err != nil {
refusals = append(refusals, fmt.Sprintf("%s:\n%v", name, err)) refusals = append(refusals, fmt.Sprintf("%s:\n%v", name, err))
continue continue
} }
declared.Sequence = seq
if len(declared.Resources) == 0 { if len(declared.Resources) == 0 {
// Sent, not skipped (novox/hq issue 127). A node whose declaration composes to // Sent, not skipped (novox/hq issue 127). A node whose declaration composes to
// nothing may have HELD something before — the broker opening a placement gave it, // nothing may have HELD something before — the broker opening a placement gave it,
@@ -583,13 +621,10 @@ func sendRound(ctx context.Context, open *stores, names []string,
return nil, err return nil, err
} }
defer release() defer release()
sending, refused := composeEach(names, func(node string) (sendable, error) { sending, refused := composeEach(names, allotting(held, open.inventory), func(node string) (sendable, error) {
return compose(held, node) return compose(held, node)
}) })
for _, s := range sending { for _, s := range sending {
if err := number(ctx, open.inventory, &s); err != nil {
return refused, err
}
body, err := s.declared.Body() body, err := s.declared.Body()
if err != nil { if err != nil {
return refused, err return refused, err
@@ -646,6 +681,12 @@ func sendTo(ctx context.Context, open *stores, names []string) error {
var sending []readyNode var sending []readyNode
var refusals []string var refusals []string
for _, name := range names { for _, name := range names {
// Numbered before composing, for the reason composeEach gives (novox/hq issue 204).
seq, err := allot(ctx, inv, name)
if err != nil {
refusals = append(refusals, fmt.Sprintf("%s:\n%v", name, err))
continue
}
plan, settings, err := planFor(ctx, open, name) plan, settings, err := planFor(ctx, open, name)
if err != nil { if err != nil {
refusals = append(refusals, fmt.Sprintf("%s:\n%v", name, err)) refusals = append(refusals, fmt.Sprintf("%s:\n%v", name, err))
@@ -657,6 +698,8 @@ func sendTo(ctx context.Context, open *stores, names []string) error {
refusals = append(refusals, fmt.Sprintf("%s:\n%v", name, err)) refusals = append(refusals, fmt.Sprintf("%s:\n%v", name, err))
continue continue
} }
declared.Sequence = seq
reportLeftOut(name, declared)
sending = append(sending, readyNode{name, declared}) sending = append(sending, readyNode{name, declared})
} }
if len(refusals) > 0 { if len(refusals) > 0 {
@@ -671,9 +714,6 @@ func sendTo(ctx context.Context, open *stores, names []string) error {
defer server.Close() defer server.Close()
for _, s := range sending { for _, s := range sending {
if err := number(ctx, inv, &s); err != nil {
return err
}
body, err := s.declared.Body() body, err := s.declared.Body()
if err != nil { if err != nil {
return err return err
@@ -681,15 +721,73 @@ func sendTo(ctx context.Context, open *stores, names []string) error {
if err := link.Declare(ctx, server.Bus(), ident, s.node, body, 15*time.Second); err != nil { if err := link.Declare(ctx, server.Bus(), ident, s.node, body, 15*time.Second); err != nil {
return err return err
} }
record, err := inv.NodeByName(ctx, s.node) if _, err := recordSent(ctx, inv, s.node, body); err != nil {
if err != nil {
return err
}
if err := inv.RecordSent(ctx, record.ID, digestOf(body)); err != nil {
return err return err
} }
fmt.Printf(" sent %s %d resource(s)\n", s.node, len(s.declared.Resources)) fmt.Printf(" sent %s %d resource(s)\n", s.node, len(s.declared.Resources))
} }
// And every assignment on those machines its membership (novox/hq ADR 0160): composed from the
// same records the bus's accounts are, so what a runtime serves and what its account may are one
// composition. Issued after the declaration, because the runtime it is for arrives with it.
return issueMemberships(ctx, open, server, sending)
}
// issueMemberships publishes the membership of every module on the machines just sent.
//
// Each carries what its module receives and the private network's addresses, from the same
// composition as the declaration it was sent (novox/hq ADR 0167): a provider reads what it is
// given on the bus, and the file written beside it says the same thing.
func issueMemberships(ctx context.Context, open *stores, server *link.Server, sent []readyNode) error {
records, err := open.inventory.BusRecords(ctx)
if err != nil {
return err
}
where := broker.PlacementsOf(records, records.Interchangeable)
bus, ok := server.Bus().(link.OverNATS)
if !ok {
return nil
}
// The declarations are sent and recorded by now; a membership that cannot be issued is said
// and does not unsay them. Every runtime without one serves the shape it derives (ADR 0160), so
// the push stands, the first failure is named once, and the next push tries again.
issued, failed := 0, 0
var first error
for _, s := range sent {
node := s.node
for _, d := range records.Assigned[node] {
membership := broker.MembershipFor(node, d, where)
membership.Mesh = s.declared.Mesh
for requirement, given := range s.declared.Received[d.Module] {
raw, err := json.Marshal(given)
if err != nil {
return err
}
if membership.Receives == nil {
membership.Receives = map[string]json.RawMessage{}
}
membership.Receives[requirement] = raw
}
body, err := json.Marshal(membership)
if err != nil {
return err
}
if err := bus.PublishMembership(ctx, node, d.Module, body); err != nil {
if first == nil {
first = err
}
failed++
continue
}
issued++
}
}
if issued > 0 {
fmt.Printf(" issued %d membership(s)\n", issued)
}
if failed > 0 {
fmt.Printf(" %d membership(s) could not be issued; the first: %v — the machines keep what "+
"they derive until the next push\n", failed, first)
}
return nil return nil
} }
@@ -864,15 +962,38 @@ func seatHolders(ctx context.Context, inv *inventory.Inventory) (map[string]brok
} }
// number gives one send the next sequence for its node (novox/hq 04-ISSUES/107). // number gives one send the next sequence for its node (novox/hq 04-ISSUES/107).
func number(ctx context.Context, inv *inventory.Inventory, s *readyNode) error { // allotting is allot over one inventory, in the shape composeEach takes.
record, err := inv.NodeByName(ctx, s.node) func allotting(ctx context.Context, inv *inventory.Inventory) func(node string) (int64, error) {
return func(node string) (int64, error) { return allot(ctx, inv, node) }
}
// allot takes the next sequence for a machine — the number its next declaration carries.
func allot(ctx context.Context, inv *inventory.Inventory, node string) (int64, error) {
record, err := inv.NodeByName(ctx, node)
if err != nil { if err != nil {
return err return 0, err
} }
seq, err := inv.NextSequence(ctx, record.ID) return inv.NextSequence(ctx, record.ID)
}
// recordSent writes down what a machine was just sent, and returns the digest.
//
// **On a context that outlives the caller's** (novox/hq issue 204). The record is written after the
// declaration is away, so a send that failed is never recorded as current — and a controller being
// replaced mid-send had its context cancelled between the two, so the machine was told and the mesh
// never wrote it down: status read "applied, current" over a machine that had just been sent
// something else. What was sent was sent; the record of it must not depend on the sender living
// another second. Bounded, so a store that is away does not hold a dying process open for ever.
func recordSent(ctx context.Context, inv *inventory.Inventory, node string, body []byte) (string, error) {
kept, cancel := context.WithTimeout(context.WithoutCancel(ctx), 10*time.Second)
defer cancel()
record, err := inv.NodeByName(kept, node)
if err != nil { if err != nil {
return err return "", err
} }
s.declared.Sequence = seq digest := digestOf(body)
return nil if err := inv.RecordSent(kept, record.ID, digest); err != nil {
return "", err
}
return digest, nil
} }
+8 -2
View File
@@ -17,7 +17,7 @@ import (
// the wrong machine no longer refuses the whole node), applied one level up. // the wrong machine no longer refuses the whole node), applied one level up.
func TestOneUnresolvableNodeStillLetsTheRestBeSent(t *testing.T) { func TestOneUnresolvableNodeStillLetsTheRestBeSent(t *testing.T) {
sending, refusals := composeEach( sending, refusals := composeEach(
[]string{"anchor", "home-server", "laptop"}, []string{"anchor", "home-server", "laptop"}, numbered(),
func(node string) (sendable, error) { func(node string) (sendable, error) {
if node == "anchor" { if node == "anchor" {
return sendable{}, errors.New(`nothing provides "acme-ca", wanted by route-proxy`) return sendable{}, errors.New(`nothing provides "acme-ca", wanted by route-proxy`)
@@ -43,7 +43,7 @@ func TestOneUnresolvableNodeStillLetsTheRestBeSent(t *testing.T) {
// (novox/hq issue 127): it may have held something before, and only sending the empty // (novox/hq issue 127): it may have held something before, and only sending the empty
// declaration tells it to drop what the mesh owned. It is never a refusal. // declaration tells it to drop what the mesh owned. It is never a refusal.
func TestAnEmptyDeclarationIsSentSoTheNodeDropsWhatItHeld(t *testing.T) { func TestAnEmptyDeclarationIsSentSoTheNodeDropsWhatItHeld(t *testing.T) {
sending, refusals := composeEach([]string{"spare"}, sending, refusals := composeEach([]string{"spare"}, numbered(),
func(string) (sendable, error) { return sendable{}, nil }) func(string) (sendable, error) { return sendable{}, nil })
if len(sending) != 1 || len(refusals) != 0 { if len(sending) != 1 || len(refusals) != 0 {
t.Errorf("an empty declaration must be sent, not skipped or refused: %v / %v", sending, refusals) t.Errorf("an empty declaration must be sent, not skipped or refused: %v / %v", sending, refusals)
@@ -74,3 +74,9 @@ func TestASkippedMachineIsStillAnError(t *testing.T) {
} }
} }
} }
// numbered is an allotter for tests: one higher per call, as the inventory's is per machine.
func numbered() func(string) (int64, error) {
var n int64
return func(string) (int64, error) { n++; return n, nil }
}
+35 -1
View File
@@ -3,6 +3,7 @@ package main
import ( import (
"encoding/json" "encoding/json"
"fmt" "fmt"
"github.com/novox/mesh-controller/internal/inventory"
"sort" "sort"
"time" "time"
) )
@@ -39,6 +40,9 @@ type meshStatus struct {
// whose is older is still working — and Waiting cannot tell those apart, because the sent // whose is older is still working — and Waiting cannot tell those apart, because the sent
// digest is recorded at send, not at apply. // digest is recorded at send, not at apply.
Reported []machineReported `json:"reported"` Reported []machineReported `json:"reported"`
// Plans is what the last merges produced and where each stands (novox/hq ADR 0162): the
// open ones first, each saying its tier, what it waits for, and whether it has waited too long.
Plans []planStatus `json:"plans"`
// Unresolved is every machine that cannot be worked out at all, with what the mesh said when // Unresolved is every machine that cannot be worked out at all, with what the mesh said when
// it tried. **A machine here is in none of the lists above**: nothing was computed for it, so // it tried. **A machine here is in none of the lists above**: nothing was computed for it, so
// there is nothing to compare it against and nothing it can be behind — which is why a // there is nothing to compare it against and nothing it can be behind — which is why a
@@ -63,6 +67,21 @@ type meshStatus struct {
// **A document without this said an outage was a well mesh.** Read from what each machine // **A document without this said an outage was a well mesh.** Read from what each machine
// reported, so it is the machine's account and not the mesh's take-time listing. // reported, so it is the machine's account and not the mesh's take-time listing.
Untaken []machineUntaken `json:"untaken,omitempty"` Untaken []machineUntaken `json:"untaken,omitempty"`
// Filtered is every converged machine that is not filtered by the mesh alone (novox/hq ADR
// 0168), one entry per rule set the mesh did not write — the found firewall in force again,
// or a chain nobody speaks for. Absent when every converged machine is filtered by the mesh
// alone. A document without this called a machine well while a predecessor's chain refused
// what the mesh declared open.
Filtered []machineFiltered `json:"filtered,omitempty"`
}
// machineFiltered is one rule set on a converged machine that the mesh did not write and that
// refuses traffic: where it is, whose the host reads it as, and what it refuses.
type machineFiltered struct {
Node string `json:"node"`
Where string `json:"where"`
Owner string `json:"owner"`
Refuses string `json:"refuses"`
} }
// machineUntaken is one module a machine is holding rather than running, and how many resources of // machineUntaken is one module a machine is holding rather than running, and how many resources of
@@ -152,7 +171,7 @@ func statusAsJSON(asked answers) ([]byte, error) {
out := meshStatus{Machines: len(nodes), Wrong: []machineDoing{}, out := meshStatus{Machines: len(nodes), Wrong: []machineDoing{},
Quiet: []machineQuiet{}, Behind: []moduleBehind{}, Waiting: []machineWaiting{}, Quiet: []machineQuiet{}, Behind: []moduleBehind{}, Waiting: []machineWaiting{},
Reported: []machineReported{}, Unresolved: []machineUnresolved{}, Reported: []machineReported{}, Unresolved: []machineUnresolved{},
Network: asked.network, Adopted: adoptedNodes(nodes)} Network: asked.network, Adopted: adoptedNodes(nodes), Plans: planStatuses(asked.plans, time.Now())}
// In a stated order, so two readings of an unchanged mesh are the same document. // In a stated order, so two readings of an unchanged mesh are the same document.
untakenNodes := make([]string, 0, len(asked.untaken)) untakenNodes := make([]string, 0, len(asked.untaken))
for name := range asked.untaken { for name := range asked.untaken {
@@ -170,6 +189,21 @@ func statusAsJSON(asked answers) ([]byte, error) {
machineUntaken{Node: name, Module: m, Held: asked.untaken[name][m]}) machineUntaken{Node: name, Module: m, Held: asked.untaken[name][m]})
} }
} }
filteredNodes := make([]string, 0, len(asked.filtered))
for name := range asked.filtered {
filteredNodes = append(filteredNodes, name)
}
sort.Strings(filteredNodes)
for _, name := range filteredNodes {
f := asked.filtered[name]
if fw := f.FoundFirewall; fw != nil && fw.Active {
out.Filtered = append(out.Filtered, machineFiltered{Node: name, Where: "the found firewall",
Owner: inventory.FilterFoundFirewall, Refuses: fw.Kind + " is in force again"})
}
for _, x := range f.Others() {
out.Filtered = append(out.Filtered, machineFiltered{Node: name, Where: x.Where, Owner: x.Owner, Refuses: x.Refuses})
}
}
for name := range asked.refused { for name := range asked.refused {
out.Unresolved = append(out.Unresolved, machineUnresolved{ out.Unresolved = append(out.Unresolved, machineUnresolved{
Node: name, Problem: asked.refused[name]}) Node: name, Problem: asked.refused[name]})
+43
View File
@@ -167,3 +167,46 @@ func TestAMachineFailingTheSameWayIsSaidToBeStuck(t *testing.T) {
t.Fatalf("one failure is not stuck: %v", once) t.Fatalf("one failure is not stuck: %v", once)
} }
} }
// A converged machine something other than the mesh filters is named, per rule set, and is not
// well (novox/hq ADR 0168); one filtered by the mesh alone is not in the list.
func TestAMachineNotFilteredByTheMeshAloneIsNamedAndNotWell(t *testing.T) {
alone := inventory.Filtering{Filters: []inventory.Filter{
{Where: "table inet mesh, chain input", Owner: inventory.FilterMesh, Refuses: "policy drop"},
{Where: "table ip filter, chain DOCKER", Owner: inventory.FilterRuntime, Refuses: "drop"},
{Where: "table ip filter, chain f2b-sshd", Owner: inventory.FilterBan, Refuses: "ip saddr 192.0.2.1 reject"},
}}
if !alone.Alone() {
t.Fatal("the mesh's own, the runtime's and a ban are not the mesh alone")
}
notAlone := inventory.Filtering{
Filters: append(alone.Filters, inventory.Filter{Where: "chain HAL-MESH-ONLY (iptables-legacy)",
Owner: inventory.FilterOther, Refuses: `-A HAL-MESH-ONLY -m comment --comment "not public" -j DROP`}),
FoundFirewall: &inventory.FoundFirewall{Kind: "ufw", Active: true},
}
asked := answers{nodes: []inventory.Node{{Name: "home-server"}, {Name: "laptop"}},
filtered: map[string]inventory.Filtering{"home-server": notAlone}}
if asked.well() {
t.Fatal("a machine not filtered by the mesh alone reads as well")
}
body, err := statusAsJSON(asked)
if err != nil {
t.Fatal(err)
}
var parsed struct {
Filtered []map[string]string `json:"filtered"`
}
if err := json.Unmarshal(body, &parsed); err != nil {
t.Fatal(err)
}
if len(parsed.Filtered) != 2 {
t.Fatalf("filtered: %v", parsed.Filtered)
}
if parsed.Filtered[0]["node"] != "home-server" || parsed.Filtered[0]["owner"] != inventory.FilterFoundFirewall ||
parsed.Filtered[1]["where"] != "chain HAL-MESH-ONLY (iptables-legacy)" || parsed.Filtered[1]["owner"] != inventory.FilterOther {
t.Fatalf("filtered: %v", parsed.Filtered)
}
if body, _ := statusAsJSON(answers{nodes: asked.nodes}); strings.Contains(string(body), `"filtered"`) {
t.Fatal("a mesh filtered by itself alone carries a filtered list")
}
}
+757
View File
@@ -0,0 +1,757 @@
package main
import (
"context"
"flag"
"fmt"
"sort"
"strings"
"time"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// A merge produces a tiered plan the mesh keeps (novox/hq ADR 0162).
//
// The handler that hears the merge computes the plan from the catalogue's one dependency relation,
// writes it to the store, asks the first tier and returns — the receive loop is never held by a
// build. Every outcome taken in advances the plan it belongs to; a ticker advances what outcomes
// alone cannot (a tier waiting for machines to report); a controller replaced mid-plan finds the
// plan where it left it.
// planWaitBound is how long a plan may wait on one thing before `status` names it red.
const planWaitBound = 30 * time.Minute
// tiersOf sorts a set of modules into tiers along the ordering edges among them: tier 0 depends
// on nothing else in the set, tier 1 only on tier 0, and so on. An edge to a module outside the set says
// nothing about the order inside it. A cycle — which the catalogue should never produce — puts
// what remains in one last tier rather than losing it, and is said by the caller.
func tiersOf(set []string, edges []inventory.Edge) [][]string {
in := map[string]bool{}
for _, m := range set {
in[m] = true
}
deps := map[string]map[string]bool{}
for _, m := range set {
deps[m] = map[string]bool{}
}
// The build seat's holders follow the controller that defines their worker (EdgeWorkerOf,
// novox/hq issue 206), so the built-by edge from that controller to such a holder yields: the
// controller is built by whichever build machine is running, as the runtime image always was.
worker := map[string]map[string]bool{}
for _, e := range edges {
if e.Kind == inventory.EdgeWorkerOf && in[e.From] && in[e.To] {
if worker[e.To] == nil {
worker[e.To] = map[string]bool{}
}
worker[e.To][e.From] = true
}
}
for _, e := range edges {
// A code dependency — B packages A's source — rebuilds B with A, in the same tier: B's
// build needs nothing of A's first. The other kinds order: stands-on and declared after
// the base is built, built-by after the build machine is built and running — except for
// what the build machine itself stands on, and for the controller whose worker the build
// machine binds. The runtime image is built by the builder and the builder is built on the
// runtime image; the image comes first, built by the builder that is running.
if !in[e.From] || !in[e.To] || e.From == e.To || e.Kind == inventory.EdgePackages {
continue
}
if e.Kind == inventory.EdgeBuiltBy && (isBaseOf(e.From, e.To, edges, in) || worker[e.From][e.To]) {
continue
}
deps[e.From][e.To] = true
}
placed := map[string]bool{}
var tiers [][]string
for len(placed) < len(set) {
var tier []string
for _, m := range set {
if placed[m] {
continue
}
free := true
for d := range deps[m] {
if !placed[d] {
free = false
break
}
}
if free {
tier = append(tier, m)
}
}
if len(tier) == 0 {
// A cycle: everything left, together, and the caller says so.
for _, m := range set {
if !placed[m] {
tier = append(tier, m)
}
}
}
sort.Strings(tier)
for _, m := range tier {
placed[m] = true
}
tiers = append(tiers, tier)
}
return tiers
}
// isBaseOf says whether `to` stands on `base`, directly or through other bases in the set, along
// the build edges alone.
func isBaseOf(base, to string, edges []inventory.Edge, in map[string]bool) bool {
seen := map[string]bool{}
var walk func(string) bool
walk = func(m string) bool {
if m == base {
return true
}
if seen[m] {
return false
}
seen[m] = true
for _, e := range edges {
if e.From == m && in[e.To] && (e.Kind == inventory.EdgeStandsOn || e.Kind == inventory.EdgeDeclared) && walk(e.To) {
return true
}
}
return false
}
return walk(to)
}
// reachableFrom is the moved modules plus everything that depends on them, through every layer:
// what a merge rebuilds. Along the code and build edges only: a module *built by* the build machine
// is not changed by a new build machine, so a built-by edge orders and gates a plan and never
// widens it — the first plan of 2026-10-01 took the whole catalogue along for a controller change.
func reachableFrom(moved []string, edges []inventory.Edge) []string {
in := map[string]bool{}
for _, m := range moved {
in[m] = true
}
for grew := true; grew; {
grew = false
for _, e := range edges {
// Built-by and worker-of order a plan; neither widens it. A new build machine changes
// nothing it builds, and a new controller changes nothing about the holder it orders —
// what packages the controller's source is already a code edge.
if e.Kind == inventory.EdgeBuiltBy || e.Kind == inventory.EdgeWorkerOf {
continue
}
if in[e.To] && !in[e.From] {
in[e.From] = true
grew = true
}
}
}
out := make([]string, 0, len(in))
for m := range in {
out = append(out, m)
}
sort.Strings(out)
return out
}
// hasCycle says whether the tiers' last tier holds modules that still depend on each other.
func hasCycle(tiers [][]string, edges []inventory.Edge) bool {
if len(tiers) == 0 {
return false
}
last := map[string]bool{}
for _, m := range tiers[len(tiers)-1] {
last[m] = true
}
for _, e := range edges {
if last[e.From] && last[e.To] {
return true
}
}
return false
}
// planFor is the plan a merge produces: the moved modules and everything reachable from them,
// tiered, with the merge it answers.
func planOfMerge(m link.SourceMoved, moved []string, edges []inventory.Edge) inventory.Plan {
set := reachableFrom(moved, edges)
tiers := tiersOf(set, edges)
modules := map[string]*inventory.PlanModule{}
for _, name := range set {
modules[name] = &inventory.PlanModule{}
}
return inventory.Plan{
ID: fmt.Sprintf("plan-%d", time.Now().UnixNano()),
Repository: m.Owner + "/" + m.Repo,
Commit: m.Commit,
Created: time.Now().UTC(),
State: inventory.PlanBuilding,
Tiers: tiers,
Modules: modules,
}
}
// gates is what the next tier needs running from this one: a module of the tier that a later
// tier is built by — the runtime dependency — and whose policy rolls it out, must be applied by
// the machines running it before the next tier is asked. A base an image stands on need only be
// built; a source another module packages need not even be that.
func gates(p inventory.Plan, edges []inventory.Edge, rollsOut func(string) bool) []string {
if p.Tier >= len(p.Tiers) {
return nil
}
inTier := map[string]bool{}
all := map[string]bool{}
for _, tier := range p.Tiers {
for _, m := range tier {
all[m] = true
}
}
for _, m := range p.Tiers[p.Tier] {
inTier[m] = true
}
later := map[string]bool{}
for _, tier := range p.Tiers[p.Tier+1:] {
for _, m := range tier {
later[m] = true
}
}
seen := map[string]bool{}
var out []string
for _, e := range edges {
if later[e.From] && inTier[e.To] && e.Kind == inventory.EdgeBuiltBy && !seen[e.To] && rollsOut(e.To) &&
!isBaseOf(e.From, e.To, edges, all) {
seen[e.To] = true
out = append(out, e.To)
}
}
sort.Strings(out)
return out
}
// applied says whether every machine running the module has reported since the module was built.
func applied(module string, builtAt time.Time, running []string, reports []inventory.Reported) (bool, []string) {
at := map[string]*time.Time{}
for _, r := range reports {
at[r.Node] = r.At
}
var waiting []string
for _, n := range running {
if t := at[n]; t == nil || t.Before(builtAt) {
waiting = append(waiting, n)
}
}
return len(waiting) == 0, waiting
}
// askTier asks the build machine for every module of the tier, and marks each asked. A module
// the catalogue no longer holds, or whose ask could not be made, is a failure of the plan: a tier
// half asked is a tier that will never complete.
func askTier(ctx context.Context, inv *inventory.Inventory, p *inventory.Plan) error {
entries, err := inv.Catalogued(ctx)
if err != nil {
return err
}
byName := map[string]inventory.Entry{}
for _, e := range entries {
byName[e.Manifest.Module] = e
}
now := time.Now().UTC()
for _, name := range p.Tiers[p.Tier] {
state := p.Modules[name]
if state == nil {
state = &inventory.PlanModule{}
p.Modules[name] = state
}
e, known := byName[name]
if !known {
state.State = "failed"
state.Why = "no longer in the catalogue"
p.State = inventory.PlanFailed
p.Note = name + " is no longer in the catalogue"
continue
}
source := buildSource{Repository: e.Source.Repository, Seat: e.Source.Seat}
fmt.Printf(" tier %d: ", p.Tier)
if err := buildOne(ctx, source, e.Source.Path, e.Source.Ref, 0); err != nil {
state.State = "failed"
state.Why = err.Error()
p.State = inventory.PlanFailed
p.Note = fmt.Sprintf("%s could not be asked for: %v", name, err)
continue
}
state.State = "asked"
state.AskedAt = &now
}
return nil
}
// planBuilt marks a module built (or failed) in every open plan whose current tier holds it, and
// advances what that completes. Called from the daemon's take-in of every outcome.
func planBuilt(ctx context.Context, open *stores, module, commit, failed string) {
inv := open.inventory
plans, err := inv.OpenPlans(ctx)
if err != nil {
fmt.Printf("plans: cannot read them: %v\n", err)
return
}
now := time.Now().UTC()
for i := range plans {
p := &plans[i]
if p.Tier >= len(p.Tiers) {
continue
}
inTier := false
for _, m := range p.Tiers[p.Tier] {
if m == module {
inTier = true
}
}
if !inTier {
continue
}
state := p.Modules[module]
if state == nil {
state = &inventory.PlanModule{}
p.Modules[module] = state
}
if failed != "" {
state.State = "failed"
state.Why = failed
p.State = inventory.PlanFailed
p.Note = fmt.Sprintf("%s failed to build in tier %d", module, p.Tier)
} else {
state.State = "built"
state.BuiltAt = &now
state.Commit = commit
}
if err := inv.SavePlan(ctx, *p); err != nil {
fmt.Printf("%s: cannot keep the plan: %v\n", p.ID, err)
continue
}
if p.State == inventory.PlanFailed {
fmt.Printf("%s: %s; the tiers after it are not asked\n", p.ID, p.Note)
}
}
advancePlans(ctx, open)
}
// advancePlans moves every open plan as far as the facts allow: a tier whose modules are all built
// and whose gates are applied gives way to the next; the last tier done is the plan done. Called
// after every outcome and on a timer, so a plan waiting on a machine's report moves when it comes.
func advancePlans(ctx context.Context, open *stores) {
inv := open.inventory
plans, err := inv.OpenPlans(ctx)
if err != nil {
fmt.Printf("plans: cannot read them: %v\n", err)
return
}
if len(plans) == 0 {
return
}
edges, err := inv.Dependencies(ctx)
if err != nil {
fmt.Printf("plans: cannot read the dependencies: %v\n", err)
return
}
rollsOut := func(module string) bool {
u, err := inv.UpgradeOf(ctx, module)
return err == nil && u.RollOut
}
for i := range plans {
p := &plans[i]
for p.Open() {
moved, err := advanceOnce(ctx, open, p, edges, rollsOut)
if err != nil {
fmt.Printf("%s: %v\n", p.ID, err)
break
}
if err := inv.SavePlan(ctx, *p); err != nil {
fmt.Printf("%s: cannot keep the plan: %v\n", p.ID, err)
break
}
if !moved {
break
}
}
}
}
// advanceOnce takes one step of one plan and says whether anything changed.
func advanceOnce(ctx context.Context, open *stores, p *inventory.Plan,
edges []inventory.Edge, rollsOut func(string) bool) (bool, error) {
inv := open.inventory
if p.Tier >= len(p.Tiers) {
p.State = inventory.PlanDone
fmt.Printf("%s: done — %s at %s, %d tier(s)\n", p.ID, p.Repository, short(p.Commit), len(p.Tiers))
return true, nil
}
tier := p.Tiers[p.Tier]
// Not yet asked: ask.
unasked := 0
for _, m := range tier {
if s := p.Modules[m]; s == nil || s.State == "" {
unasked++
}
}
if unasked == len(tier) {
if err := askTier(ctx, inv, p); err != nil {
return false, err
}
return true, nil
}
// Asked: wait for every build.
var latest time.Time
for _, m := range tier {
s := p.Modules[m]
if s == nil || s.State != "built" {
return false, nil
}
if s.BuiltAt != nil && s.BuiltAt.After(latest) {
latest = *s.BuiltAt
}
}
// Built: send every module of the tier whose policy rolls out, once, to the machines running
// it — whether or not its source commit moved. A dependent rebuilt because its base moved, or
// a module that packages another repository's source, keeps its commit; the catalogue announces
// no move for it and its machines would keep the old image until somebody pushed (novox/hq
// issue 189). A module whose policy records is built and left, as its policy says.
for _, m := range tier {
state := p.Modules[m]
if state == nil || state.SentAt != nil || !rollsOut(m) {
continue
}
running, err := inv.Running(ctx, m)
if err != nil {
return false, err
}
now := time.Now().UTC()
state.SentAt = &now
if len(running) == 0 {
continue
}
if err := sendTo(ctx, open, running); err != nil {
return false, fmt.Errorf("sending %s to %s after tier %d: %w", m, strings.Join(running, ", "), p.Tier, err)
}
fmt.Printf("%s: tier %d built; sent %s to %s\n", p.ID, p.Tier, m, strings.Join(running, ", "))
return true, nil
}
// And wait for what the next tier needs running.
needed := gates(*p, edges, rollsOut)
if len(needed) > 0 {
reports, err := inv.LastReports(ctx)
if err != nil {
return false, err
}
var waiting []string
for _, m := range needed {
running, err := inv.Running(ctx, m)
if err != nil {
return false, err
}
state := p.Modules[m]
if state == nil {
state = &inventory.PlanModule{}
p.Modules[m] = state
}
// The plan sends what it waits for. A rebuild from the same source commit is not a
// move the catalogue announces — the build machine rebuilt for a controller change
// is one — so the roll-out that opens this gate is the plan's to make, once, and
// the reports that open it are the ones after the send.
since := latest
if state.BuiltAt != nil {
since = *state.BuiltAt
}
if state.SentAt != nil && state.SentAt.After(since) {
since = *state.SentAt
}
if ok, on := applied(m, since, running, reports); !ok {
waiting = append(waiting, fmt.Sprintf("%s on %s", m, strings.Join(on, ", ")))
}
}
if len(waiting) > 0 {
note := "tier " + fmt.Sprint(p.Tier) + " built; waiting for " + strings.Join(waiting, "; ") + " to be applied"
changed := p.State != inventory.PlanRolling || p.Note != note
p.State = inventory.PlanRolling
p.Note = note
return changed, nil
}
}
p.Tier++
p.State = inventory.PlanBuilding
p.Note = ""
if p.Tier < len(p.Tiers) {
fmt.Printf("%s: tier %d done; asking tier %d: %s\n", p.ID, p.Tier-1, p.Tier, strings.Join(p.Tiers[p.Tier], ", "))
}
return true, nil
}
// planTicker advances open plans on a timer, for the steps outcomes alone cannot take.
func planTicker(ctx context.Context, open *stores) {
advancePlans(ctx, open)
tick := time.NewTicker(30 * time.Second)
defer tick.Stop()
for {
select {
case <-ctx.Done():
return
case <-tick.C:
advancePlans(ctx, open)
}
}
}
// planLine is one plan as `status` says it.
func planLine(p inventory.Plan, now time.Time) string {
where := fmt.Sprintf("tier %d of %d", min(p.Tier+1, len(p.Tiers)), len(p.Tiers))
switch p.State {
case inventory.PlanDone:
return fmt.Sprintf("%s %s done, %d tier(s)", p.Repository, short(p.Commit), len(p.Tiers))
case inventory.PlanFailed:
return fmt.Sprintf("%s %s FAILED at %s: %s", p.Repository, short(p.Commit), where, p.Note)
}
since := now.Sub(p.Updated).Round(time.Minute)
late := ""
if since > planWaitBound {
late = " — LATE"
}
what := "building"
if p.State == inventory.PlanRolling {
what = p.Note
}
return fmt.Sprintf("%s %s %s, %s for %s%s", p.Repository, short(p.Commit), where, what, since, late)
}
// planFailedBuild marks the module a failed build was for when the result names no module: by the
// repository and path the plan's modules were asked at.
func planFailedBuild(ctx context.Context, open *stores, result link.BuildResult) {
entries, err := open.inventory.Catalogued(ctx)
if err != nil {
return
}
for _, e := range entries {
if repositoryMatches(e.Source.Repository, result.Repository) && e.Source.Path == result.Path {
planBuilt(ctx, open, e.Manifest.Module, result.Commit, result.Failed)
return
}
}
}
func repositoryMatches(a, b string) bool {
trim := func(s string) string { return strings.ToLower(strings.TrimSuffix(s, ".git")) }
return trim(a) == trim(b) || strings.HasSuffix(trim(a), "/"+trim(b)) || strings.HasSuffix(trim(b), "/"+trim(a))
}
// planStatus is one plan as `status --json` says it.
type planStatus struct {
ID string `json:"id"`
Repository string `json:"repository"`
Commit string `json:"commit"`
State string `json:"state"`
Tier int `json:"tier"`
Tiers int `json:"tiers"`
Waiting string `json:"waiting,omitempty"`
Since time.Time `json:"since"`
Late bool `json:"late"`
}
func planStatuses(plans []inventory.Plan, now time.Time) []planStatus {
out := make([]planStatus, 0, len(plans))
for _, p := range plans {
ps := planStatus{ID: p.ID, Repository: p.Repository, Commit: p.Commit, State: p.State,
Tier: p.Tier, Tiers: len(p.Tiers), Since: p.Updated}
if p.Open() {
ps.Waiting = p.Note
if ps.Waiting == "" {
ps.Waiting = "builds of tier " + fmt.Sprint(p.Tier)
}
ps.Late = now.Sub(p.Updated) > planWaitBound
}
out = append(out, ps)
}
return out
}
// openPlans is the open plans among the recent ones, and how many have waited past the bound.
func openPlans(plans []inventory.Plan) ([]inventory.Plan, int) {
var open []inventory.Plan
late := 0
for _, p := range plans {
if p.Open() {
open = append(open, p)
if time.Since(p.Updated) > planWaitBound {
late++
}
}
}
return open, late
}
// plansCommand says what the last merges produced and where each stands; given an id, one plan
// tier by tier with every module's state.
func plansCommand(ctx context.Context, args []string) error {
set := flag.NewFlagSet("plans", flag.ContinueOnError)
limit := set.Int("n", 10, "how many to show")
whatIf := set.String("what-if", "", "owner/repository: the plan a merge there would produce, saving nothing — with --paths or --modules")
paths := set.String("paths", "", "the files the merge would change, comma-separated, from the repository's root")
modules := set.String("modules", "", "or the modules it would change, comma-separated")
positionals, err := parseAround(set, args)
if err != nil {
return err
}
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
inv := open.inventory
now := time.Now()
if len(positionals) == 1 {
p, err := inv.PlanByID(ctx, positionals[0])
if err != nil {
return err
}
fmt.Printf("%s — %s\n", p.ID, planLine(p, now))
for i, tier := range p.Tiers {
marker := " "
if i == p.Tier && p.Open() {
marker = ">"
}
fmt.Printf("%s tier %d\n", marker, i)
for _, m := range tier {
s := p.Modules[m]
state := "not yet asked"
if s != nil && s.State != "" {
state = s.State
if s.Commit != "" {
state += " from " + short(s.Commit)
}
if s.Why != "" {
state += ": " + s.Why
}
}
fmt.Printf(" %-22s %s\n", m, state)
}
}
return nil
}
if *whatIf != "" {
return planWhatIf(ctx, inv, *whatIf, splitList(*paths), splitList(*modules))
}
if len(positionals) == 2 && positionals[0] == "stop" {
p, err := inv.PlanByID(ctx, positionals[1])
if err != nil {
return err
}
if !p.Open() {
return fmt.Errorf("%s is already %s", p.ID, p.State)
}
p.State = inventory.PlanFailed
p.Note = "stopped by hand at tier " + fmt.Sprint(p.Tier)
if err := inv.SavePlan(ctx, p); err != nil {
return err
}
fmt.Printf("%s stopped at tier %d of %d; what was asked still builds and registers, nothing further is asked\n",
p.ID, p.Tier, len(p.Tiers))
return nil
}
plans, err := inv.RecentPlans(ctx, *limit)
if err != nil {
return err
}
if len(plans) == 0 {
fmt.Println("no merge has produced a plan yet")
return nil
}
for _, p := range plans {
fmt.Printf("%-28s %s\n", p.ID, planLine(p, now))
}
return nil
}
// planWhatIf is the plan a merge would produce, computed the way the merge handler computes one
// and saved nowhere: the modules the repository's changed files touch (or the modules named), what
// packages their source, everything reachable from them, in tiers. For reading before merging.
func planWhatIf(ctx context.Context, inv *inventory.Inventory, repository string, paths, modules []string) error {
owner, repo, found := strings.Cut(repository, "/")
if !found {
return fmt.Errorf("--what-if takes owner/repository, not %q", repository)
}
m := link.SourceMoved{Owner: owner, Repo: repo, Base: "main", Commit: "what-if", Paths: paths}
entries, err := inv.Catalogued(ctx)
if err != nil {
return err
}
read, err := inv.ReadRepositories(ctx)
if err != nil {
return err
}
var from, packaging []inventory.Entry
named := map[string]bool{}
for _, name := range modules {
named[name] = true
}
for _, e := range entries {
switch {
case named[e.Manifest.Module]:
from = append(from, e)
case len(named) == 0 && sourceIs(e.Source, m):
from = append(from, e)
case readsFrom(read[e.Manifest.Module], m):
packaging = append(packaging, e)
}
}
if len(named) == 0 {
from = whatTheMergeTouched(from, entries, m)
}
moved := append(append([]inventory.Entry{}, from...), packaging...)
if len(moved) == 0 {
fmt.Printf("a merge of %s changing %s would build nothing the mesh holds\n", repository,
orNone(strings.Join(append(paths, modules...), ", ")))
return nil
}
edges, err := inv.Dependencies(ctx)
if err != nil {
return err
}
var names []string
for _, e := range moved {
names = append(names, e.Manifest.Module)
}
p := planOfMerge(m, names, edges)
fmt.Printf("a merge of %s would build %d module(s) in %d tier(s):\n", repository, len(p.Modules), len(p.Tiers))
rolls := map[string]string{}
for i, tier := range p.Tiers {
fmt.Printf(" tier %d\n", i)
for _, name := range tier {
how := "built; its policy records, so nothing is sent"
if u, err := inv.UpgradeOf(ctx, name); err == nil && u.RollOut {
running, _ := inv.Running(ctx, name)
how = "built, then sent to " + orNone(strings.Join(running, ", "))
rolls[name] = how
}
fmt.Printf(" %-22s %s\n", name, how)
}
}
if hasCycle(p.Tiers, edges) {
fmt.Println(" the last tier depends on itself and would be built together, in no order")
}
if len(packaging) > 0 {
var also []string
for _, e := range packaging {
also = append(also, e.Manifest.Module)
}
fmt.Printf(" %s package source from %s, so they are rebuilt without their own source moving\n",
strings.Join(also, ", "), repository)
}
return nil
}
func splitList(s string) []string {
var out []string
for _, part := range strings.Split(s, ",") {
if part = strings.TrimSpace(part); part != "" {
out = append(out, part)
}
}
return out
}
+128
View File
@@ -0,0 +1,128 @@
package main
import (
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// A merge produces a tiered plan (novox/hq ADR 0162): what moved and everything reachable from it,
// sorted so a tier depends only on earlier ones — with the three kinds of dependency told apart.
func TestAMergeIsPlannedInTiersAlongTheThreeKindsOfDependency(t *testing.T) {
edges := []inventory.Edge{
// build dependencies: images on the runtime, a plugin on one of them
{From: "shop", To: "mesh-tools", Kind: inventory.EdgeStandsOn},
{From: "postgres", To: "mesh-tools", Kind: inventory.EdgeStandsOn},
{From: "shop-plugin", To: "shop", Kind: inventory.EdgeDeclared},
// a code dependency: the proxy packages the controller's source — same tier
{From: "route-proxy", To: "mesh-controller", Kind: inventory.EdgePackages},
{From: "builder", To: "mesh-controller", Kind: inventory.EdgePackages},
// runtime dependencies: everything source-built is built by the builder
{From: "shop", To: "builder", Kind: inventory.EdgeBuiltBy},
{From: "postgres", To: "builder", Kind: inventory.EdgeBuiltBy},
{From: "shop-plugin", To: "builder", Kind: inventory.EdgeBuiltBy},
{From: "route-proxy", To: "builder", Kind: inventory.EdgeBuiltBy},
{From: "mesh-controller", To: "builder", Kind: inventory.EdgeBuiltBy},
{From: "mesh-tools", To: "builder", Kind: inventory.EdgeBuiltBy},
{From: "builder", To: "mesh-tools", Kind: inventory.EdgeStandsOn},
{From: "unrelated", To: "alpine", Kind: inventory.EdgeStandsOn},
}
// The runtime image moved: everything on it, and what is built by what is on it.
set := reachableFrom([]string{"mesh-tools"}, edges)
// What stands on the runtime, and the builder that stands on it; not the controller, which the
// builder merely builds, nor the proxy that packages the controller.
want := []string{"builder", "mesh-tools", "postgres", "shop", "shop-plugin"}
if len(set) != len(want) {
t.Fatalf("reachable from the runtime: %v, want %v", set, want)
}
tiers := tiersOf(set, edges)
pos := map[string]int{}
for i, tier := range tiers {
for _, m := range tier {
pos[m] = i
}
}
if pos["mesh-tools"] != 0 || pos["builder"] != 1 {
t.Fatalf("the runtime then the builder: %v", tiers)
}
if !(pos["shop"] > pos["builder"] && pos["postgres"] > pos["builder"]) {
t.Fatalf("what the builder builds comes after the builder: %v", tiers)
}
if pos["shop-plugin"] <= pos["shop"] {
t.Fatalf("a plugin after what it is declared on: %v", tiers)
}
if hasCycle(tiers, edges) {
t.Fatalf("no cycle here: %v", tiers)
}
// The controller alone moved: the proxy with it, nothing else.
small := reachableFrom([]string{"mesh-controller"}, edges)
if len(small) != 3 {
t.Fatalf("a controller merge rebuilds the controller and what packages it: %v", small)
}
// The builder packages the controller's source (same tier by that edge) and the controller is
// built by the builder (next tier by that one): the builder first, then the controller and the
// proxy together — a code dependency in one tier, a runtime dependency across tiers.
smallTiers := tiersOf(small, edges)
if len(smallTiers) != 2 || smallTiers[0][0] != "builder" || len(smallTiers[1]) != 2 {
t.Fatalf("the builder, then the controller and the proxy together: %v", smallTiers)
}
// The builder alone moved: the builder, and nothing it builds.
if only := reachableFrom([]string{"builder"}, edges); len(only) != 1 {
t.Fatalf("a build machine change rebuilds the build machine alone: %v", only)
}
// Only a runtime dependency gates on deployment, and only when the module rolls out.
p := planOfMerge(link.SourceMoved{Owner: "novox", Repo: "mesh-tools", Commit: "abc"}, []string{"mesh-tools"}, edges)
p.Tier = pos["builder"]
rollsOut := func(m string) bool { return m == "builder" }
if g := gates(p, edges, rollsOut); len(g) != 1 || g[0] != "builder" {
t.Fatalf("the builder gates the tier after it: %v", g)
}
p.Tier = 0
if g := gates(p, edges, rollsOut); len(g) != 0 {
t.Fatalf("the runtime image is a build dependency and gates nothing: %v", g)
}
if g := gates(p, edges, func(string) bool { return false }); len(g) != 0 {
t.Fatalf("a module that only records its upgrade gates nothing: %v", g)
}
}
// A gate is open once every machine running the module has reported after it was built.
func TestAGateOpensWhenTheMachinesHaveReportedSinceTheBuild(t *testing.T) {
built := time.Date(2026, 10, 1, 15, 0, 0, 0, time.UTC)
before, after := built.Add(-time.Minute), built.Add(time.Minute)
reports := []inventory.Reported{{Node: "anchor", At: &after}, {Node: "home-server", At: &before}}
ok, waiting := applied("builder", built, []string{"anchor", "home-server"}, reports)
if ok || len(waiting) != 1 || waiting[0] != "home-server" {
t.Fatalf("one machine has not reported since the build: ok=%v waiting=%v", ok, waiting)
}
if ok, _ := applied("builder", built, []string{"anchor"}, reports); !ok {
t.Fatal("the machine that reported after the build holds the gate open")
}
if ok, _ := applied("builder", built, nil, reports); !ok {
t.Fatal("a module running nowhere gates nothing")
}
}
// A cycle is not lost: what remains is one last tier, and the caller says so.
func TestACycleIsOneLastTierAndSaidSo(t *testing.T) {
edges := []inventory.Edge{{From: "a", To: "b", Kind: inventory.EdgeStandsOn}, {From: "b", To: "a", Kind: inventory.EdgeStandsOn}}
tiers := tiersOf([]string{"a", "b"}, edges)
if len(tiers) != 1 || len(tiers[0]) != 2 || !hasCycle(tiers, edges) {
t.Fatalf("a cycle should be one tier of two, said: %v", tiers)
}
}
// novox/hq 04-ISSUES/211: a merge moving the toolchain and a bundle compiled in it builds the
// bundle a tier after the toolchain, not beside it.
func TestABundleIsPlannedAfterTheToolchainItIsCompiledIn(t *testing.T) {
edges := []inventory.Edge{{From: "node-tools", To: "mesh-tools", Kind: inventory.EdgeStandsOn}}
p := planOfMerge(link.SourceMoved{Owner: "novox", Repo: "mesh-tools", Commit: "abc"},
[]string{"mesh-tools", "node-tools"}, edges)
if len(p.Tiers) != 2 || p.Tiers[0][0] != "mesh-tools" || p.Tiers[1][0] != "node-tools" {
t.Fatalf("the toolchain, then the bundle: %v", p.Tiers)
}
}
+4 -2
View File
@@ -346,7 +346,9 @@ func rolloutMint(ctx context.Context, again bool) error {
} }
machines++ machines++
case broker.KindModule: case broker.KindModule, broker.KindNodeTools:
// The runtime is minted and delivered exactly as a module is (novox/hq ADR 0175): it is
// issued as the module it stands for, to that module's `broker` secret.
if p.Module == "mesh-controller" { if p.Module == "mesh-controller" {
// The control plane is a module too, and its `broker` secret is the old bus's // The control plane is a module too, and its `broker` secret is the old bus's
// credential it is still using while this runs. Writing the new bus's blob there // credential it is still using while this runs. Writing the new bus's blob there
@@ -365,7 +367,7 @@ func rolloutMint(ctx context.Context, again bool) error {
skipped++ skipped++
continue continue
} }
password, err := inv.MintBusPassword(ctx, inventory.BusUser{Username: p.Username(), Kind: inventory.BusModule, Node: p.Node, Module: p.Module}) password, err := inv.MintBusPassword(ctx, inventory.BusUser{Username: p.Username(), Kind: busKindOf(p.Module), Node: p.Node, Module: p.Module})
if err != nil { if err != nil {
return err return err
} }
+1 -54
View File
@@ -2,13 +2,12 @@ package main
import ( import (
"context" "context"
"strings"
"testing" "testing"
) )
// A node's own set failing to compose, and the mesh being unable to answer at all, are different // A node's own set failing to compose, and the mesh being unable to answer at all, are different
// things, and only the first may be passed over when something is gathered across every machine // things, and only the first may be passed over when something is gathered across every machine
// (novox/hq 04-ISSUES/152). These pin that distinction where the three gatherers rely on it. // (novox/hq 04-ISSUES/152). These pin that distinction where the gatherers rely on it.
func TestASetThatDoesNotComposeIsMarkedAsTheNodesOwnProblem(t *testing.T) { func TestASetThatDoesNotComposeIsMarkedAsTheNodesOwnProblem(t *testing.T) {
open := aMesh(t) open := aMesh(t)
@@ -45,55 +44,3 @@ func TestAStoreThatCannotBeReadIsNotANodeThatDoesNotCompose(t *testing.T) {
t.Fatalf("a question the mesh could not answer was read as a node that runs nothing: %v", err) t.Fatalf("a question the mesh could not answer was read as a node that runs nothing: %v", err)
} }
} }
func TestOneIncoherentNodeDoesNotCostTheRestTheirNames(t *testing.T) {
open := aMesh(t)
one, two := rivals()
register(t, open, one)
register(t, open, two)
for _, m := range []string{one.Module, two.Module} {
if _, err := open.inventory.Assign(t.Context(), "laptop", m); err != nil {
t.Fatal(err)
}
}
// laptop cannot compose. That is laptop's problem and nobody else's: the roster is still
// answerable, and anchor keeps whatever it serves.
if _, err := routeNamesInTheMesh(t.Context(), open); err != nil {
t.Fatalf("one node's broken set cost the whole mesh its roster: %v", err)
}
}
func TestARosterIsNeverReturnedWithNamesItCouldNotRead(t *testing.T) {
open := aMesh(t)
stopped, cancel := context.WithCancel(t.Context())
cancel()
names, err := routeNamesInTheMesh(stopped, open)
if err == nil {
t.Fatalf("a roster was composed from a store that could not be read: %v", names)
}
// The failure must be raised, not turned into an absence. A roster missing a machine's names
// is indistinguishable, on every machine that receives it, from the operator withdrawing them —
// and because the roster is part of every container's identity, it replaces all of them.
if names != nil {
t.Fatalf("a partial roster was returned beside the error: %v", names)
}
}
// Kept so the reason survives the next person reading it: the message the gatherer raises must say
// which machine could not be read, or the operator is left with a mesh-wide failure and no name.
func TestTheRaisedFailureNamesTheMachineItCouldNotRead(t *testing.T) {
open := aMesh(t)
stopped, cancel := context.WithCancel(t.Context())
cancel()
_, err := routeNamesInTheMesh(stopped, open)
if err == nil {
t.Fatal("no failure was raised")
}
if !strings.Contains(err.Error(), "cannot be read") {
t.Fatalf("the failure does not say the mesh could not be read: %v", err)
}
}
+242 -10
View File
@@ -6,8 +6,10 @@ import (
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
"github.com/nats-io/nats.go/micro"
"os" "os"
"os/exec" "os/exec"
"sort"
"strings" "strings"
"github.com/novox/mesh-controller/internal/catalogue" "github.com/novox/mesh-controller/internal/catalogue"
@@ -48,24 +50,60 @@ func argvFor(verb string, args map[string]any) ([]string, error) {
return nil return nil
} }
switch verb { switch verb {
case "command":
// The generic verb: the command line as given, split as a shell would split it, with
// nothing added — the named verbs add flags a caller cannot reach; this one is the whole
// binary and says so in its description (novox/hq ADR 0154, 0175).
if err := need("command"); err != nil {
return nil, err
}
argv, err := splitCommandLine(str("command"))
if err != nil {
return nil, err
}
if len(argv) == 0 {
return nil, errors.New("command names no command")
}
return argv, nil
case "status": case "status":
return []string{"status", "--json"}, nil return []string{"status", "--json"}, nil
case "nodes": case "nodes":
return []string{"node", "list"}, nil return []string{"node", "list", "--json"}, nil
case "node": case "node":
if err := need("node"); err != nil { if err := need("node"); err != nil {
return nil, err return nil, err
} }
return []string{"node", "show", str("node")}, nil return []string{"node", "show", str("node")}, nil
case "modules": case "modules":
return []string{"module", "list"}, nil return []string{"module", "list", "--json"}, nil
case "seats": case "seats":
return []string{"seats", "--json"}, nil return []string{"seats", "--json"}, nil
case "builds": case "builds":
if id := str("log"); id != "" {
return []string{"builds", "--log", id}, nil
}
if m := str("module"); m != "" { if m := str("module"); m != "" {
return []string{"builds", m}, nil return []string{"builds", m}, nil
} }
return []string{"builds"}, nil return []string{"builds"}, nil
case "plans":
if r := str("repository"); r != "" {
argv := []string{"plans", "--what-if", r}
if p := str("paths"); p != "" {
argv = append(argv, "--paths", p)
}
if m := str("modules"); m != "" {
argv = append(argv, "--modules", m)
}
return argv, nil
}
if id := str("stop"); id != "" {
return []string{"plans", "stop", id}, nil
}
if id := str("id"); id != "" {
return []string{"plans", id}, nil
}
return []string{"plans"}, nil
case "plan": case "plan":
if err := need("node"); err != nil { if err := need("node"); err != nil {
return nil, err return nil, err
@@ -76,6 +114,16 @@ func argvFor(verb string, args map[string]any) ([]string, error) {
return nil, err return nil, err
} }
return []string{verb, str("node"), str("module")}, nil return []string{verb, str("node"), str("module")}, nil
case "pin":
if err := need("node", "provision", "from", "module"); err != nil {
return nil, err
}
return []string{"pin", str("node"), str("provision"), str("from"), str("module")}, nil
case "unpin":
if err := need("node", "provision"); err != nil {
return nil, err
}
return []string{"unpin", str("node"), str("provision")}, nil
case "push": case "push":
// Sent and not waited for: the asker reads `status` for what the machine did, which is // Sent and not waited for: the asker reads `status` for what the machine did, which is
// what a person at a shell does too. A tool call that blocked for a push's whole apply would // what a person at a shell does too. A tool call that blocked for a push's whole apply would
@@ -84,11 +132,58 @@ func argvFor(verb string, args map[string]any) ([]string, error) {
return []string{"push", n, "--wait", "0"}, nil return []string{"push", n, "--wait", "0"}, nil
} }
return []string{"push", "--behind", "--wait", "0"}, nil return []string{"push", "--behind", "--wait", "0"}, nil
case "rotate":
if p := str("provision"); p != "" {
argv := []string{"rotate", p}
if c := str("consumer"); c != "" {
argv = append(argv, "--consumer", c)
}
return argv, nil
}
if str("node") != "" && str("module") != "" && str("secret") != "" {
return []string{"secret", "rotate", str("node"), str("module"), str("secret")}, nil
}
// Half of either shape: the command says its usage, which names both shapes, and that is
// the answer the caller needs.
return []string{"rotate"}, nil
case "settings":
// `settings set|clear` at a shell (novox/hq issue 198). The values travel as an argument
// because a tool has no file to hand the command; the command reads either.
if err := need("module"); err != nil {
return nil, err
}
argv := []string{"settings", "set", str("module")}
switch {
case str("clear") == "true":
argv = []string{"settings", "clear", str("module")}
case str("values") != "":
argv = append(argv, str("values"))
}
// Neither values nor clear: the command says its usage, which names both, and that is the
// answer the caller needs — the same as `rotate` given half of either shape.
if n := str("node"); n != "" {
argv = append(argv, "--node", n)
}
return argv, nil
case "issue":
// The same act as `module issue` at a shell (novox/hq design 25 §4): the account is minted
// into the mesh's records and delivered at the machine's next push, which is the caller's to
// ask for — so the mesh is never pushed as a side effect of a credential.
if err := need("node", "module"); err != nil {
return nil, err
}
return []string{"module", "issue", str("module"), "--node", str("node")}, nil
case "build": case "build":
if err := need("repository"); err != nil { if err := need("repository"); err != nil {
return nil, err return nil, err
} }
// Not waited for: a tool call cannot hold a connection for the minutes a build takes; the
// daemon takes the outcome in when it comes and the id follows the build (issue 176). A
// repository given without a scheme is a path on the forge holding the git seat.
argv := []string{"build", str("repository"), "--wait", "0"} argv := []string{"build", str("repository"), "--wait", "0"}
if !strings.Contains(str("repository"), "://") && !strings.HasPrefix(str("repository"), "git@") {
argv = append(argv, "--self")
}
if p := str("path"); p != "" { if p := str("path"); p != "" {
argv = append(argv, "--path", p) argv = append(argv, "--path", p)
} }
@@ -137,13 +232,15 @@ func runVerb(ctx context.Context, argv []string) (verbAnswer, error) {
} }
// seatToolHandlers are the handlers for every verb the mesh-controller seat declares, from the // seatToolHandlers are the handlers for every verb the mesh-controller seat declares, from the
// store's row, so a verb the row does not carry is not served and a verb it carries that this binary // store's row, so a verb the row does not carry is not served. A verb it carries that this binary
// cannot run is said at start rather than at the first call. // cannot run is named at start and answers the reason when called — never a refusal to serve, which
func seatToolHandlers() (map[string]link.ToolHandler, error) { // would take the whole control plane down for one word (novox/hq ADR 0185).
func seatToolHandlers() (map[string]link.ToolHandler, []string, error) {
seat, known := catalogue.SeatNamed(catalogue.ControllerSeatName) seat, known := catalogue.SeatNamed(catalogue.ControllerSeatName)
if !known { if !known {
return nil, fmt.Errorf("this mesh defines no %s seat", catalogue.ControllerSeatName) return nil, nil, fmt.Errorf("this mesh defines no %s seat", catalogue.ControllerSeatName)
} }
var behind []string
handlers := map[string]link.ToolHandler{} handlers := map[string]link.ToolHandler{}
for _, v := range seat.Serves { for _, v := range seat.Serves {
verb := v.Name verb := v.Name
@@ -153,9 +250,28 @@ func seatToolHandlers() (map[string]link.ToolHandler, error) {
} }
continue continue
} }
if _, err := argvFor(verb, map[string]any{"node": "x", "module": "x", "repository": "x"}); err != nil { if _, err := argvFor(verb, sampleArguments(v)); err != nil {
return nil, fmt.Errorf("the %s seat's row declares %q, which this control plane cannot run: %w", // **A row ahead of this binary is not a reason to go silent.**
catalogue.ControllerSeatName, verb, err) //
// The row is the store's and a control plane follows it (novox/hq ADR 0154), so a verb
// this build does not know means the row was widened by a newer one — the ordinary
// state of a roll-out, and of a push that put an older control plane back. Refusing to
// serve at all made that transient fatal: on 2026-10-02 one unknown verb took the whole
// mesh off the bus for ten minutes, and the way back was a human running the binary by
// hand, because the thing that would have repaired it is the thing that was down
// (novox/hq 04-ISSUES/201, ADR 0185).
//
// So the verbs this binary knows are served, and this one answers the reason instead of
// nothing: a caller gets a sentence naming the fault, and everything else keeps working
// — including the push that replaces this binary with the one whose verb it is.
behind = append(behind, verb)
reason := err
handlers[verb] = func(context.Context, json.RawMessage) (any, error) {
return nil, fmt.Errorf("%s is in this mesh's %s row and the control plane running "+
"here cannot run it: %w. It is a verb of a newer build; this one is behind",
verb, catalogue.ControllerSeatName, reason)
}
continue
} }
handlers[verb] = func(ctx context.Context, raw json.RawMessage) (any, error) { handlers[verb] = func(ctx context.Context, raw json.RawMessage) (any, error) {
args := map[string]any{} args := map[string]any{}
@@ -171,7 +287,7 @@ func seatToolHandlers() (map[string]link.ToolHandler, error) {
return runVerb(ctx, argv) return runVerb(ctx, argv)
} }
} }
return handlers, nil return handlers, behind, nil
} }
// seatTools is what `tools` answers: every seat with a protocol, and the tools each serves, from the // seatTools is what `tools` answers: every seat with a protocol, and the tools each serves, from the
@@ -192,3 +308,119 @@ func seatTools() map[string]any {
} }
return map[string]any{"seats": seats} return map[string]any{"seats": seats}
} }
// sampleArguments is one of every argument a verb's schema requires, so the check at start proves the
// verb runnable rather than that it happens to want the arguments the check guessed.
func sampleArguments(v catalogue.Verb) map[string]any {
sample := map[string]any{"node": "x", "module": "x", "repository": "x"}
switch required := v.Input["required"].(type) {
case []string:
for _, k := range required {
sample[k] = "x"
}
case []any:
for _, k := range required {
if name, ok := k.(string); ok {
sample[name] = "x"
}
}
}
return sample
}
// splitCommandLine splits a command line into words the way a POSIX shell does for the simple
// cases a controller command needs: spaces separate, single or double quotes group, a backslash
// escapes the next character inside double quotes or outside any. No expansion of anything.
func splitCommandLine(line string) ([]string, error) {
var words []string
var cur strings.Builder
inWord := false
quote := rune(0)
runes := []rune(line)
for i := 0; i < len(runes); i++ {
r := runes[i]
switch {
case quote == '\'':
if r == '\'' {
quote = 0
} else {
cur.WriteRune(r)
}
case quote == '"':
if r == '"' {
quote = 0
} else if r == '\\' && i+1 < len(runes) {
i++
cur.WriteRune(runes[i])
} else {
cur.WriteRune(r)
}
case r == '\'' || r == '"':
quote = r
inWord = true
case r == '\\' && i+1 < len(runes):
i++
cur.WriteRune(runes[i])
inWord = true
case r == ' ' || r == '\t' || r == '\n':
if inWord {
words = append(words, cur.String())
cur.Reset()
inWord = false
}
default:
cur.WriteRune(r)
inWord = true
}
}
if quote != 0 {
return nil, fmt.Errorf("command has an unclosed %c quote", quote)
}
if inWord {
words = append(words, cur.String())
}
return words, nil
}
// seatAnnouncement is what the controller says it serves on the bus (novox/hq ADR 0197): the
// mesh-controller seat, one endpoint per verb it answers, each with the seat's own description and
// argument schema — the same facts `tools` answers from the records, as NATS's services format.
func seatAnnouncement(handlers map[string]link.ToolHandler) micro.Info {
about := map[string]catalogue.Verb{}
for _, s := range catalogue.SeatsWithAProtocol() {
if s.Name == catalogue.ControllerSeatName {
for _, v := range s.Serves {
about[v.Name] = v
}
}
}
verbs := make([]string, 0, len(handlers))
for verb := range handlers {
verbs = append(verbs, verb)
}
sort.Strings(verbs)
var endpoints []micro.EndpointInfo
for _, verb := range verbs {
schema, _ := json.Marshal(about[verb].Input)
// The same shape every tool runtime announces in (node-tools' announce package): the name is
// `<seat>__<verb>`, as the protocol's characters allow; the metadata is what identifies it.
endpoints = append(endpoints, micro.EndpointInfo{
Name: catalogue.ControllerSeatName + "__" + verb,
Subject: link.SeatToolSubject(catalogue.ControllerSeatName, verb),
QueueGroup: "seat." + catalogue.ControllerSeatName,
Metadata: map[string]string{
"kind": "seat", "module": catalogue.ControllerSeatName, "tool": verb,
"seat": catalogue.ControllerSeatName, "scope": "mesh", "interchangeable": "false",
"description": about[verb].Description, "schema": string(schema),
},
})
}
return micro.Info{
ServiceIdentity: micro.ServiceIdentity{
Name: catalogue.ControllerSeatName, ID: "controller", Version: "0.1.0",
Metadata: map[string]string{"seat": catalogue.ControllerSeatName, "scope": "mesh"},
},
Description: "the mesh's own verbs, answered by the holder of the mesh-controller seat",
Endpoints: endpoints,
}
}
+204 -3
View File
@@ -1,6 +1,9 @@
package main package main
import ( import (
"context"
"fmt"
"github.com/novox/mesh-controller/internal/link"
"strings" "strings"
"testing" "testing"
@@ -30,6 +33,81 @@ func TestEveryDeclaredVerbHasACommandLine(t *testing.T) {
} }
} }
// `builds` given a build's id reads that build's log from the bus rather than listing builds
// (novox/hq ADR 0157).
func TestBuildsWithAnIdReadsThatBuildsLog(t *testing.T) {
argv, err := argvFor("builds", map[string]any{"log": "build-17"})
if err != nil {
t.Fatal(err)
}
if strings.Join(argv, " ") != "builds --log build-17" {
t.Fatalf("builds with a log id became %q", strings.Join(argv, " "))
}
}
// The build tool takes a repository as a URL or as its path on the forge holding the git seat, and
// says which it was given, so the command reads the path as a seat source rather than handing it to
// git as written (novox/hq issue 176). And it never waits: the id follows the build.
func TestTheBuildToolTellsAForgePathFromAURL(t *testing.T) {
argv, _ := argvFor("build", map[string]any{"repository": "novox/mesh-catalog", "path": "modules/x"})
if line := strings.Join(argv, " "); !strings.Contains(line, "--self") || !strings.Contains(line, "--wait 0") {
t.Fatalf("a forge path is a seat source, not waited for; got %q", line)
}
argv, _ = argvFor("build", map[string]any{"repository": "https://example.tld/o/r.git"})
if line := strings.Join(argv, " "); strings.Contains(line, "--self") {
t.Fatalf("a URL is cloned as given; got %q", line)
}
}
// `rotate` is one verb with two shapes (ADR 0114, issue 180): a pair credential by provision, or a
// module's own secret by machine, module and name.
func TestRotateTakesAProvisionOrAnOwnSecret(t *testing.T) {
argv, _ := argvFor("rotate", map[string]any{"provision": "postgres-database", "consumer": "ace"})
if strings.Join(argv, " ") != "rotate postgres-database --consumer ace" {
t.Fatalf("a pair credential: %v", argv)
}
argv, _ = argvFor("rotate", map[string]any{"node": "ace", "module": "nodered", "secret": "api-token"})
if strings.Join(argv, " ") != "secret rotate ace nodered api-token" {
t.Fatalf("an own secret: %v", argv)
}
argv, _ = argvFor("rotate", map[string]any{"node": "ace"})
if strings.Join(argv, " ") != "rotate" {
t.Fatalf("half an own secret falls to the command's usage: %v", argv)
}
}
// `settings` is `settings set|clear` at a shell, with the values passed inline (novox/hq issue 198).
func TestSettingsSetsOrClearsALayer(t *testing.T) {
argv, err := argvFor("settings", map[string]any{"module": "dnsmasq", "values": `{"a":1}`, "node": "ace"})
if err != nil || strings.Join(argv, " ") != `settings set dnsmasq {"a":1} --node ace` {
t.Fatalf("set on a machine: %v %v", argv, err)
}
argv, _ = argvFor("settings", map[string]any{"module": "dnsmasq", "clear": "true"})
if strings.Join(argv, " ") != "settings clear dnsmasq" {
t.Fatalf("clear for the mesh: %v", argv)
}
argv, _ = argvFor("settings", map[string]any{"module": "dnsmasq"})
if strings.Join(argv, " ") != "settings set dnsmasq" {
t.Fatalf("a set with no values falls to the command's usage: %v", argv)
}
}
// `issue` is `module issue` at a shell: the module and the machine, and nothing that would push. A
// module's bus account was mintable only from the controller's command line, so an agent working
// through the tools could not finish a rollout that gave a module one (novox/hq issue 191).
func TestIssueGivesAModuleItsAccountOnAMachine(t *testing.T) {
argv, err := argvFor("issue", map[string]any{"node": "ace", "module": "route-proxy"})
if err != nil {
t.Fatal(err)
}
if strings.Join(argv, " ") != "module issue route-proxy --node ace" {
t.Fatalf("issue runs %v", argv)
}
if _, err := argvFor("issue", map[string]any{"module": "route-proxy"}); err == nil {
t.Error("an account was issued without saying which machine reads it")
}
}
// A required argument missing is refused in the verb's own words, before anything runs. // A required argument missing is refused in the verb's own words, before anything runs.
func TestAVerbMissingWhatItNeedsIsRefused(t *testing.T) { func TestAVerbMissingWhatItNeedsIsRefused(t *testing.T) {
if _, err := argvFor("node", map[string]any{}); err == nil || !strings.Contains(err.Error(), `node needs "node"`) { if _, err := argvFor("node", map[string]any{}); err == nil || !strings.Contains(err.Error(), `node needs "node"`) {
@@ -40,24 +118,28 @@ func TestAVerbMissingWhatItNeedsIsRefused(t *testing.T) {
} }
} }
// A push and a build are sent, not waited for: the asker reads status for what happened. // A push and a build are sent, not waited for: the asker reads status, or the build's log by its
// id, for what happened. A repository given as a forge path is said to be one (issue 176).
func TestActsDoNotBlockTheCall(t *testing.T) { func TestActsDoNotBlockTheCall(t *testing.T) {
argv, _ := argvFor("push", map[string]any{"node": "one"}) argv, _ := argvFor("push", map[string]any{"node": "one"})
if strings.Join(argv, " ") != "push one --wait 0" { if strings.Join(argv, " ") != "push one --wait 0" {
t.Fatalf("push waits: %v", argv) t.Fatalf("push waits: %v", argv)
} }
argv, _ = argvFor("build", map[string]any{"repository": "novox/x", "path": "modules/x"}) argv, _ = argvFor("build", map[string]any{"repository": "novox/x", "path": "modules/x"})
if strings.Join(argv, " ") != "build novox/x --wait 0 --path modules/x" { if strings.Join(argv, " ") != "build novox/x --wait 0 --self --path modules/x" {
t.Fatalf("build: %v", argv) t.Fatalf("build: %v", argv)
} }
} }
// What `tools` answers is the seats' records, with each verb's schema. // What `tools` answers is the seats' records, with each verb's schema.
func TestToolsAnswersTheSeatsRecords(t *testing.T) { func TestToolsAnswersTheSeatsRecords(t *testing.T) {
handlers, err := seatToolHandlers() handlers, behind, err := seatToolHandlers()
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
if len(behind) != 0 {
t.Fatalf("this build cannot run %v of its own seat's verbs", behind)
}
if len(handlers) != len(catalogue.ControllerVerbs) { if len(handlers) != len(catalogue.ControllerVerbs) {
t.Fatalf("%d handlers for %d verbs", len(handlers), len(catalogue.ControllerVerbs)) t.Fatalf("%d handlers for %d verbs", len(handlers), len(catalogue.ControllerVerbs))
} }
@@ -95,3 +177,122 @@ func TestAJSONVerbsAnswerIsItsStandardOutput(t *testing.T) {
t.Fatalf("stderr and stdout are both what the command said: %s", answer.Output) t.Fatalf("stderr and stdout are both what the command said: %s", answer.Output)
} }
} }
// `command` is the generic verb: the command line as given, split as a shell would, nothing added —
// so an operator's `node account g14 jochen` is one call through the console rather than a shell on
// the control node (novox/hq ADR 0154, ADR 0175).
func TestCommandRunsTheLineAsGiven(t *testing.T) {
argv, err := argvFor("command", map[string]any{"command": "node account g14 jochen"})
if err != nil || strings.Join(argv, " ") != "node account g14 jochen" {
t.Fatalf("a plain line: %v %v", argv, err)
}
argv, err = argvFor("command", map[string]any{"command": `settings set dnsmasq '{"a": "b c"}' --node ace`})
if err != nil || len(argv) != 6 || argv[3] != `{"a": "b c"}` {
t.Fatalf("a quoted word stays one word: %q %v", argv, err)
}
argv, err = argvFor("command", map[string]any{"command": `node add "the box" --adopted`})
if err != nil || len(argv) != 4 || argv[2] != "the box" {
t.Fatalf("double quotes group: %q %v", argv, err)
}
if _, err := argvFor("command", map[string]any{"command": " "}); err == nil {
t.Fatal("an empty line was accepted")
}
if _, err := argvFor("command", map[string]any{"command": `node "unclosed`}); err == nil {
t.Fatal("an unclosed quote was accepted")
}
}
// A verb in the row that this binary cannot run does not take the control plane off the bus: the
// rest are served, the unknown one answers the reason, and the start-up names it (novox/hq ADR
// 0185). One unknown word cost the mesh ten minutes of silence on 2026-10-02, recoverable only by
// a person running the binary by hand — the push that would have repaired it needs the control
// plane that was down.
func TestARowAheadOfThisBuildIsServedAnyway(t *testing.T) {
seat, known := catalogue.SeatNamed(catalogue.ControllerSeatName)
if !known {
t.Fatal("no controller seat")
}
// The row as a newer control plane would have written it: every verb this build knows, and one
// it does not.
widened := seat
widened.Serves = append(append([]catalogue.Verb{}, seat.Serves...),
catalogue.Verb{Name: "teleport", Description: "a verb from a build that does not exist yet"})
rows := catalogue.DefaultSeats()
for i := range rows {
if rows[i].Name == catalogue.ControllerSeatName {
rows[i] = widened
}
}
catalogue.UseSeats(rows)
t.Cleanup(func() { catalogue.UseSeats(catalogue.DefaultSeats()) })
handlers, behind, err := seatToolHandlers()
if err != nil {
t.Fatalf("a row with one unknown verb refused to serve at all: %v", err)
}
if len(behind) != 1 || behind[0] != "teleport" {
t.Fatalf("the verbs this build cannot run were reported as %v", behind)
}
if len(handlers) != len(widened.Serves) {
t.Fatalf("%d handlers for %d verbs in the row", len(handlers), len(widened.Serves))
}
for _, known := range []string{"status", "nodes", "push"} {
if handlers[known] == nil {
t.Errorf("%s is not served although this build knows it", known)
}
}
_, err = handlers["teleport"](context.Background(), nil)
if err == nil {
t.Fatal("the unknown verb answered as though it had run")
}
for _, want := range []string{"teleport", "cannot run it", "behind"} {
if !strings.Contains(err.Error(), want) {
t.Errorf("the answer does not say %q: %v", want, err)
}
}
}
// novox/hq ADR 0195: the console's discovery reads the machines and the modules; they answer as JSON,
// as status and seats do, so nothing parses a printed column.
func TestTheNodesAndModulesVerbsAnswerAsJSON(t *testing.T) {
for verb, want := range map[string]string{"nodes": "[node list --json]", "modules": "[module list --json]"} {
argv, err := argvFor(verb, map[string]any{})
if err != nil {
t.Fatal(err)
}
if fmt.Sprint(argv) != want {
t.Errorf("%s runs %v, want %s", verb, argv, want)
}
}
}
// novox/hq ADR 0197: the controller announces exactly the verbs it serves, each on the subject and
// queue it serves it on, with the seat's own description and schema, in NATS's services format.
func TestTheControllerAnnouncesTheVerbsItServes(t *testing.T) {
handlers, _, err := seatToolHandlers()
if err != nil {
t.Fatal(err)
}
info := seatAnnouncement(handlers)
if info.Name != catalogue.ControllerSeatName || info.ID == "" || info.Version == "" {
t.Fatalf("the service is not named for the seat: %+v", info.ServiceIdentity)
}
if len(info.Endpoints) != len(handlers) {
t.Fatalf("%d endpoints announced for %d verbs served", len(info.Endpoints), len(handlers))
}
for _, e := range info.Endpoints {
verb := e.Metadata["tool"]
if _, served := handlers[verb]; !served || e.Name != catalogue.ControllerSeatName+"__"+verb {
t.Errorf("%s (%s) is announced and not served under that name", e.Name, verb)
}
if e.Metadata["kind"] != "seat" || e.Metadata["seat"] != catalogue.ControllerSeatName {
t.Errorf("%s is not announced as the seat's verb: %v", e.Name, e.Metadata)
}
if e.Subject != link.SeatToolSubject(catalogue.ControllerSeatName, verb) || e.QueueGroup != "seat."+catalogue.ControllerSeatName {
t.Errorf("%s is announced on %s/%s, not where it is served", e.Name, e.Subject, e.QueueGroup)
}
if e.Metadata["description"] == "" || e.Metadata["schema"] == "" || e.Metadata["scope"] != "mesh" {
t.Errorf("%s is announced without its description, schema or scope: %v", e.Name, e.Metadata)
}
}
}
+61 -1
View File
@@ -10,6 +10,7 @@ import (
"io" "io"
"os" "os"
"strings" "strings"
"time"
"github.com/novox/mesh-controller/internal/catalogue" "github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
@@ -38,6 +39,8 @@ func secretCommand(ctx context.Context, args []string) error {
} }
switch args[0] { switch args[0] {
case "accept": case "accept":
case "rotate":
return secretRotate(ctx, args[1:])
case "recover": case "recover":
return secretRecover(ctx, args[1:]) return secretRecover(ctx, args[1:])
case "export": case "export":
@@ -101,7 +104,8 @@ func secretCommand(ctx context.Context, args []string) error {
return nil return nil
} }
const secretUsage = "secret accept <node> <module> <name> [--from <file>] [--provider <node> [--local <name>]]\n" + const secretUsage = "secret rotate <node> <module> <name>\n" +
"secret accept <node> <module> <name> [--from <file>] [--provider <node> [--local <name>]]\n" +
"secret recover <node> <module> <name> --key <operator-key> [--out <file>] [--from-export <file>] [--provider <node>]\n" + "secret recover <node> <module> <name> --key <operator-key> [--out <file>] [--from-export <file>] [--provider <node>]\n" +
"secret export [--out <file>]" "secret export [--out <file>]"
@@ -359,3 +363,59 @@ func valueFor(node, module, name, from string) (string, error) {
return line, nil return line, nil
} }
} }
// secretRotate makes a module's own secret anew and sends the machine, so the module starts again on
// the new value (novox/hq ADR 0114, issue 180). A pair credential rotates with `rotate <provision>`;
// this is the secret with one party. Said in the log with who asked and when, never the value.
func secretRotate(ctx context.Context, args []string) error {
rest, _ := split(args)
if len(rest) != 3 {
return errors.New(secretUsage)
}
node, module, name := rest[0], rest[1], rest[2]
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
if err := open.inventory.RotateModuleSecret(ctx, node, module, name); err != nil {
var refused inventory.ErrNotRotatable
if errors.As(err, &refused) {
return fmt.Errorf("not rotated: %s", refused.Why)
}
return err
}
fmt.Printf("rotated %q of %s on %s at %s, asked by %s; the value is sealed and not shown\n",
name, module, node, time.Now().UTC().Format(time.RFC3339), whoAsked())
// A shared credential (ADR 0158) has as many holders as the provision has consumers, and all
// of them are sent in one act, so no machine is left reading a value the provider no longer takes.
machines, err := open.inventory.SharedHolders(ctx, node, module, name)
if err != nil {
return err
}
if len(machines) == 0 {
machines = []string{node}
}
if len(machines) == 1 {
fmt.Printf("sending %s, so %s starts again on the new value:\n", node, module)
} else {
fmt.Printf("shared with every consumer; sending %s together:\n", strings.Join(machines, ", "))
}
if err := sendTo(ctx, open, machines); err != nil {
return fmt.Errorf("%w\n\nThe new value is sealed and not yet delivered; what runs keeps the old "+
"one until the machines next apply. Fix the cause and run `push --behind`", err)
}
return nil
}
// whoAsked names the caller for the log: the account the command runs as, which for a tool call
// through the console is the mesh's own.
func whoAsked() string {
if u := os.Getenv("SUDO_USER"); u != "" {
return u
}
if u := os.Getenv("USER"); u != "" {
return u
}
return "the mesh"
}
+17
View File
@@ -25,6 +25,20 @@ type sendable struct {
// Adoption is nil for a converged node, and then the body is byte for byte what it was before // Adoption is nil for a converged node, and then the body is byte for byte what it was before
// adoption existed: an older host parses the envelope strictly and would refuse the key. // adoption existed: an older host parses the envelope strictly and would refuse the key.
Adoption *adoptionEnvelope Adoption *adoptionEnvelope
// Received and Mesh are not sent in the declaration. They are what this machine's memberships
// are issued with on the bus (novox/hq ADR 0167): each module's received contributions, from
// the same composition as its received files, and every machine's private-network address.
Received map[string]map[string][]catalogue.Contribution
Mesh []string
// LeftOut is every module of the machine's set left out of this declaration because a stored
// setting cannot compose with its definition (novox/hq ADR 0163, rule 6), sorted. The host
// keeps that module's held things and touches none of its containers; a machine is told
// everything or nothing about what it IS told, and what it is not told is said. Absent from
// the body when empty, so a declaration that leaves nothing out is byte for byte what it was.
LeftOut []string
// leftOutWhy is why each was, for push and plan to say; never on the wire.
leftOutWhy map[string]string
} }
// adoptionEnvelope is what an adopted node is told about its mode. Taken is every module taken on // adoptionEnvelope is what an adopted node is told about its mode. Taken is every module taken on
@@ -45,6 +59,9 @@ func (s sendable) Body() ([]byte, error) {
if s.Sequence > 0 { if s.Sequence > 0 {
envelope["sequence"] = s.Sequence envelope["sequence"] = s.Sequence
} }
if len(s.LeftOut) > 0 {
envelope["left_out"] = s.LeftOut
}
// An empty declaration is deliberate here — the node owns nothing the mesh put there // An empty declaration is deliberate here — the node owns nothing the mesh put there
// (novox/hq issue 127) — and the host refuses an empty body unless it is told the emptiness // (novox/hq issue 127) — and the host refuses an empty body unless it is told the emptiness
// is meant, so a truncated or mis-composed body is never mistaken for "own nothing". // is meant, so a truncated or mis-composed body is never mistaken for "own nothing".
+65 -1
View File
@@ -47,7 +47,12 @@ func composed(t *testing.T, open *stores, node string) sendable {
// aMesh's laptop with the private network taken off it, so nothing in the declaration is random: // aMesh's laptop with the private network taken off it, so nothing in the declaration is random:
// what changes this string is a change to what a converged machine is sent, which is the thing an // what changes this string is a change to what a converged machine is sent, which is the thing an
// older host would refuse. // older host would refuse.
const convergedBefore = `{"declaration":1,"resources":[{"content":"hello","id":"hello-web.page","path":"/var/lib/hello-web/index.html","type":"file"},{"hosts":["anchor.internal:10.77.0.1"],"id":"hello-web.server","image":"registry.example/hello@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","name":"hello-web","type":"container"},{"id":"hello-web.served","path":"/var/lib/hello-web","type":"directory"}]}` //
// Re-captured 2026-10-01 (novox/hq issue 177): c978aa7 took `hosts` off every container — a
// machine's own resolver knows the mesh's names now — and left this string carrying it, so the
// guard failed for a day and nothing ran it. A field an older host never sees is the one change
// this guard permits; a field it would refuse is the one it exists to catch.
const convergedBefore = `{"declaration":1,"resources":[{"content":"hello","id":"hello-web.page","path":"/var/lib/hello-web/index.html","type":"file"},{"id":"hello-web.server","image":"registry.example/hello@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","name":"hello-web","type":"container"},{"id":"hello-web.served","path":"/var/lib/hello-web","type":"directory"}]}`
func TestAConvergedDeclarationIsByteForByteWhatItWas(t *testing.T) { func TestAConvergedDeclarationIsByteForByteWhatItWas(t *testing.T) {
open := aMesh(t) open := aMesh(t)
@@ -377,3 +382,62 @@ func TestAnEmptyDeclarationSaysOwnsNothing(t *testing.T) {
t.Fatalf("a non-empty declaration must not mark owns_nothing; got %v", env) t.Fatalf("a non-empty declaration must not mark owns_nothing; got %v", env)
} }
} }
// A setting is judged where it is stored, and an impossible one costs a module, not a machine
// (novox/hq ADR 0163, rule 6): stored while it composed, a setting whose definition then moved from
// under it leaves that module out of the declaration — said in the envelope, so the host keeps the
// module's things — and the machine is told everything else.
func TestADefinitionMovingUnderAStoredSettingLeavesThatModuleOutNotTheMachine(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
web := helloWeb()
web.Resources[1]["ports"] = []any{"8080"}
register(t, open, web)
register(t, open, catalogue.Manifest{Module: "notes", Version: "1",
Resources: []map[string]any{{"id": "conf", "type": "file", "path": "/etc/notes.conf", "content": "x"}}})
for _, m := range []string{"hello-web", "notes"} {
if _, err := assign(ctx, open, "laptop", m); err != nil {
t.Fatal(err)
}
}
// Judged where it is stored: a port the module does not publish is refused by name.
err := open.inventory.SetSettings(ctx, "laptop", "hello-web",
map[string]any{catalogue.PortsSetting: map[string]any{"9999": 10000}})
if err == nil || !strings.Contains(err.Error(), "hello-web on laptop") || !strings.Contains(err.Error(), "9999") {
t.Fatalf("an impossible setting was stored: %v", err)
}
if err := open.inventory.SetSettings(ctx, "laptop", "hello-web",
map[string]any{catalogue.PortsSetting: map[string]any{"8080": 10000}}); err != nil {
t.Fatal(err)
}
if declared := composed(t, open, "laptop"); len(declared.LeftOut) != 0 {
t.Fatalf("a setting that composes left a module out: %v", declared.LeftOut)
}
// The definition moves: the container publishes another port now.
web.Version = "2"
web.Resources[1]["ports"] = []any{"9090"}
register(t, open, web)
declared := composed(t, open, "laptop")
if len(declared.LeftOut) != 1 || declared.LeftOut[0] != "hello-web" {
t.Fatalf("hello-web is not left out: %v", declared.LeftOut)
}
if !strings.Contains(declared.leftOutWhy["hello-web"], "no container of its publishes 8080") {
t.Fatalf("why it was left out is not said: %v", declared.leftOutWhy)
}
if hasID(declared.Resources, "hello-web.server") || !hasID(declared.Resources, "notes.conf") {
t.Fatalf("the machine was not told everything else: %v", declared.Resources)
}
body, err := declared.Body()
if err != nil {
t.Fatal(err)
}
var env map[string]any
if err := json.Unmarshal(body, &env); err != nil {
t.Fatal(err)
}
left, _ := env["left_out"].([]any)
if len(left) != 1 || left[0] != "hello-web" {
t.Fatalf("the envelope does not say what was left out: %v", env)
}
}
+75 -2
View File
@@ -138,6 +138,18 @@ func printStatus(asked answers) error {
len(quiet), strings.Join(said, "\n ")) len(quiet), strings.Join(said, "\n "))
} }
if open, late := openPlans(asked.plans); len(open) > 0 {
fmt.Printf("%d plan(s) open", len(open))
if late > 0 {
fmt.Printf(", %d waiting past %s", late, planWaitBound)
}
fmt.Println(":")
for _, p := range open {
fmt.Printf(" %s\n", planLine(p, time.Now()))
}
fmt.Println()
}
if len(behind) > 0 { if len(behind) > 0 {
var names []string var names []string
for m := range behind { for m := range behind {
@@ -215,6 +227,28 @@ func printStatus(asked answers) error {
" not readable from a commit; that needs a version the host reports as ordered\n\n") " not readable from a commit; that needs a version the host reports as ordered\n\n")
} }
if len(asked.filtered) > 0 {
// A converged machine is filtered by the mesh alone, and the mesh says truthfully which
// (novox/hq ADR 0168). One that is not — a predecessor's chain still refusing, a found
// firewall in force again — is said here, and is not well.
machines := make([]string, 0, len(asked.filtered))
for name := range asked.filtered {
machines = append(machines, name)
}
sort.Strings(machines)
fmt.Printf("%d converged machine(s) are not filtered by the mesh alone:\n", len(machines))
for _, name := range machines {
f := asked.filtered[name]
if fw := f.FoundFirewall; fw != nil && fw.Active {
fmt.Printf(" %-12s the found firewall (%s) is in force again; the next apply retires it\n", name, fw.Kind)
}
for _, x := range f.Others() {
fmt.Printf(" %-12s %s (%s): %s\n", name, x.Where, x.Owner, x.Refuses)
}
}
fmt.Printf("\n the mesh wrote none of these and removes none; `node show <node>` lists every filter with its owner\n\n")
}
if len(asked.untaken) > 0 { if len(asked.untaken) > 0 {
// **Before the adopted line, and it breaks "all well".** An adopted machine is a state // **Before the adopted line, and it breaks "all well".** An adopted machine is a state
// somebody chose and can leave alone; a module assigned to one and never taken is work // somebody chose and can leave alone; a module assigned to one and never taken is work
@@ -289,7 +323,10 @@ func firstLine(s string) string {
// A type of its own rather than a method on the enrolment, because they are unrelated things // A type of its own rather than a method on the enrolment, because they are unrelated things
// arriving on one queue and an implementation of one should not have to say anything about the // arriving on one queue and an implementation of one should not have to say anything about the
// other. // other.
type builds struct{ inv *inventory.Inventory } type builds struct {
inv *inventory.Inventory
open *stores
}
// theThreeQuestions reads what anything answering "is the mesh alright" needs. // theThreeQuestions reads what anything answering "is the mesh alright" needs.
// //
@@ -342,6 +379,17 @@ func theThreeQuestions(ctx context.Context, open *stores) (answers, error) {
if err != nil { if err != nil {
return answers{}, err return answers{}, err
} }
// And which converged machines something other than the mesh filters (novox/hq ADR 0168), as
// each last reported — the account that was missing when a predecessor's chain refused what the
// mesh declared open for eleven hours (04-ISSUES/144, 145).
out.filtered, err = filteredMachines(ctx, inv, out.nodes)
if err != nil {
return answers{}, err
}
out.plans, err = inv.RecentPlans(ctx, 5)
if err != nil {
return answers{}, err
}
// And which machines are not running what the mesh would send them. The same question as a // And which machines are not running what the mesh would send them. The same question as a
// module being behind its source, one level down: that one says the catalogue is out of date, // module being behind its source, one level down: that one says the catalogue is out of date,
@@ -387,6 +435,30 @@ func theThreeQuestions(ctx context.Context, open *stores) (answers, error) {
// //
// A machine that reports no holds contributes nothing, so a converged mesh answers an empty map and // A machine that reports no holds contributes nothing, so a converged mesh answers an empty map and
// the caller prints nothing. // the caller prints nothing.
// filteredMachines is every converged machine not filtered by the mesh alone, with what it last said
// filters it (novox/hq ADR 0168). An adopted machine keeps its found firewall by design and is not
// counted; a machine that has not said is not said to be filtered by anything.
func filteredMachines(ctx context.Context, inv *inventory.Inventory, nodes []inventory.Node) (
map[string]inventory.Filtering, error) {
out := map[string]inventory.Filtering{}
for _, n := range nodes {
if n.Adopted {
continue
}
f, err := inv.FilteringOf(ctx, n.Name)
if err != nil {
return nil, fmt.Errorf("what filters %s cannot be read: %w", n.Name, err)
}
if len(f.Filters) == 0 && f.FoundFirewall == nil {
continue
}
if !f.Alone() {
out[n.Name] = f
}
}
return out, nil
}
func untakenModules(ctx context.Context, inv *inventory.Inventory, nodes []inventory.Node) ( func untakenModules(ctx context.Context, inv *inventory.Inventory, nodes []inventory.Node) (
map[string]map[string]int, error) { map[string]map[string]int, error) {
@@ -423,7 +495,8 @@ func untakenModules(ctx context.Context, inv *inventory.Inventory, nodes []inven
// read as success for the whole of the edge cut-over outage (novox/hq 04-ISSUES/125). // read as success for the whole of the edge cut-over outage (novox/hq 04-ISSUES/125).
func (a answers) well() bool { func (a answers) well() bool {
return len(a.wrong) == 0 && len(a.quiet) == 0 && len(a.behind) == 0 && return len(a.wrong) == 0 && len(a.quiet) == 0 && len(a.behind) == 0 &&
len(a.waiting) == 0 && len(a.refused) == 0 && a.network == "" && len(a.untaken) == 0 len(a.waiting) == 0 && len(a.refused) == 0 && a.network == "" && len(a.untaken) == 0 &&
len(a.filtered) == 0
} }
// hostSplit is which machines report which host version, for every version more than one machine // hostSplit is which machines report which host version, for every version more than one machine
+143
View File
@@ -0,0 +1,143 @@
package main
import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
// What the forge's take compares, as a machine would report it.
func aForgeComparison() comparison {
return comparison{reported: inventory.Adoption{
Firewall: "ufw",
Held: []inventory.Held{
{ID: "forge.server", Module: "forge", Kind: "container", Target: "forge", Facts: map[string]any{
"image": "forge:1.27.3", "image_created": "2026-09-17T10:00:00Z",
"declared_image": "forge:1.22.6", "declared_image_created": "2026-08-20T10:00:00Z", "downgrade": true,
"networks": map[string]any{"predecessor_default": []any{"office", "db"}},
"ports": []any{"3000/tcp>0.0.0.0:3000"}, "declared_ports": []any{"3000:3000"},
}},
{ID: "forge.config", Module: "forge", Kind: "file", Target: "/etc/forge/app.ini", Kept: "/var/lib/mesh/kept/app.ini",
Facts: map[string]any{"differs": true, "difference": []any{"- private scope: local", "+ upstream: public"}}},
{ID: "other.server", Module: "other", Kind: "container", Target: "other"},
},
Reachable: []inventory.Reach{
{Protocol: "tcp", Address: "0.0.0.0", Port: 3000, By: "forge", Published: true, ContainerPort: 3000},
{Protocol: "tcp", Address: "0.0.0.0", Port: 22, By: "sshd"},
},
}}
}
// A take is a comparison (novox/hq ADR 0163): the preview puts what runs beside what the module
// declares, and an older image or a differing file refuses unless named.
func TestATakePreviewsTheComparisonAndRefusesWhatIsNotNamed(t *testing.T) {
c := aForgeComparison()
preview, refusals, saw := comparisonOf("forge", c, takeOptions{})
for _, want := range []string{"runs forge:1.27.3 (made 2026-09-17)", "declares forge:1.22.6 (made 2026-08-20)", "DOWNGRADE",
"on the network predecessor_default with office, db", "will not once it moves to the module's own network",
"publishes 3000/tcp>0.0.0.0:3000; the module declares 3000:3000",
// How far the port reaches now, as the machine reported it (rule 1).
"reachable now at 0.0.0.0:3000 (tcp, container port 3000), behind the found firewall (ufw)",
"- private scope: local", "original kept at /var/lib/mesh/kept/app.ini"} {
if !strings.Contains(preview, want) {
t.Errorf("the preview lacks %q:\n%s", want, preview)
}
}
if strings.Contains(preview, "other") {
t.Errorf("another module's held things are in the preview:\n%s", preview)
}
if len(refusals) != 2 || !strings.Contains(refusals[0], "--downgrade") || !strings.Contains(refusals[1], "--replace /etc/forge/app.ini") {
t.Fatalf("the downgrade and the differing file refuse, each naming its override: %v", refusals)
}
if len(saw) != 12 {
t.Fatalf("the preview's digest is %q", saw)
}
// Named, they pass.
if _, refusals, _ := comparisonOf("forge", c, takeOptions{Downgrade: true, Replace: map[string]bool{"/etc/forge/app.ini": true}}); len(refusals) != 0 {
t.Fatalf("named differences still refused: %v", refusals)
}
if _, refusals, _ := comparisonOf("forge", c, takeOptions{Downgrade: true, Replace: map[string]bool{"*": true}}); len(refusals) != 0 {
t.Fatalf("replace * did not cover the file: %v", refusals)
}
// A held thing with no facts yet — a host older than this — refuses nothing and says what it can.
if preview, refusals, _ := comparisonOf("other", c, takeOptions{}); len(refusals) != 0 || !strings.Contains(preview, "container other") {
t.Fatalf("a factless hold: %q %v", preview, refusals)
}
// The digest is of what the preview says: a fact changing changes it.
c.reported.Held[0].Facts["image"] = "forge:1.27.4"
if _, _, again := comparisonOf("forge", c, takeOptions{}); again == saw {
t.Fatal("the found image changed and the digest did not")
}
}
// A secret the mesh minted for a service whose data was found refuses: the running service already
// has a value (rule 2). Accepted, it is carried in; `--mint` says the service shall take the new one.
func TestAMintedSecretForFoundDataRefusesUnlessAcceptedOrMinted(t *testing.T) {
c := aForgeComparison()
c.secrets = []inventory.SecretState{
{Name: "admin", Origin: inventory.OriginMade},
{Name: "postgres-database", Origin: inventory.OriginMade, Provider: "anchor"},
{Name: "broker", Origin: inventory.OriginAccepted},
}
preview, refusals, _ := comparisonOf("forge", c, takeOptions{Downgrade: true, Replace: map[string]bool{"*": true}})
for _, want := range []string{
"own secret admin: MINTED by the mesh and not accepted",
"secret from anchor postgres-database: MINTED by the mesh and not accepted",
"own secret broker: accepted from a person, carried in as it is",
} {
if !strings.Contains(preview, want) {
t.Errorf("the preview lacks %q:\n%s", want, preview)
}
}
if len(refusals) != 2 {
t.Fatalf("two minted secrets refuse: %v", refusals)
}
if !strings.Contains(refusals[0], "`secret accept <node> forge admin`") || !strings.Contains(refusals[0], "`--mint admin`") {
t.Errorf("the own secret's refusal names accepting it and minting it: %s", refusals[0])
}
if !strings.Contains(refusals[1], "`secret accept <node> forge postgres-database --provider anchor`") {
t.Errorf("the required secret's refusal names its provider: %s", refusals[1])
}
preview, refusals, _ = comparisonOf("forge", c, takeOptions{Downgrade: true, Replace: map[string]bool{"*": true},
Mint: map[string]bool{"admin": true, "postgres-database": true}})
if len(refusals) != 0 || !strings.Contains(preview, "admin: minted by the mesh; the service takes the new value, as --mint said") {
t.Fatalf("--mint did not pass the minted secrets: %v\n%s", refusals, preview)
}
// With no found data — only a file held — the service has no value of its own, and a minted
// secret is simply said.
c.reported.Held = c.reported.Held[1:2]
if _, refusals, _ := comparisonOf("forge", c, takeOptions{Replace: map[string]bool{"*": true}}); len(refusals) != 0 {
t.Fatalf("a minted secret refused with no data found: %v", refusals)
}
}
// A found network a per-machine setting keeps is named in the preview (rule 4), and the module's
// settings are said with where each came from, composed or not (rules 1 and 6).
func TestTheKeptNetworkAndTheSettingsAreInThePreview(t *testing.T) {
c := aForgeComparison()
c.keeps = map[string][]string{"forge.server": {"predecessor_default"}}
c.layers = []catalogue.Layer{
{From: catalogue.MeshWideLayer, Values: map[string]any{"site": "x"}},
{From: "anchor", Values: map[string]any{catalogue.NetworksSetting: map[string]any{"server": []any{"predecessor_default"}}}},
}
preview, _, _ := comparisonOf("forge", c, takeOptions{Downgrade: true, Replace: map[string]bool{"*": true}})
for _, want := range []string{
"on the network predecessor_default with office, db — kept by this machine's setting, so they still reach it by name once taken",
"settings from the mesh: site",
"settings from anchor: networks",
} {
if !strings.Contains(preview, want) {
t.Errorf("the preview lacks %q:\n%s", want, preview)
}
}
if strings.Contains(preview, "will not once it moves") {
t.Errorf("a kept network is still said to be lost:\n%s", preview)
}
c.settingsRefused = "forge: ports is a { port: machine-port } map"
preview, _, _ = comparisonOf("forge", c, takeOptions{Downgrade: true, Replace: map[string]bool{"*": true}})
if !strings.Contains(preview, "SETTINGS DO NOT COMPOSE with the module's definition, so the push leaves it out: forge: ports") {
t.Errorf("settings that cannot compose are not said:\n%s", preview)
}
}
+59 -22
View File
@@ -295,17 +295,31 @@ func (f following) SourceMoved(ctx context.Context, m link.SourceMoved) error {
"built from\n", m.Owner, m.Repo, m.Base, m.Commit) "built from\n", m.Owner, m.Repo, m.Base, m.Commit)
return nil return nil
} }
against, err := inv.BuiltAgainst(ctx) // A merge produces a plan the mesh keeps (novox/hq ADR 0162): what moved and everything that
// depends on it, along the catalogue's one dependency relation, sorted into tiers. The plan is
// written before any build is asked; the first tier is asked; this returns. Outcomes advance it.
edges, err := inv.Dependencies(ctx)
if err != nil { if err != nil {
return notNow(err) return notNow(err)
} }
ordered := orderByBases(moved, against) var movedNames []string
names := make([]string, 0, len(ordered)) for _, e := range moved {
for _, e := range ordered { movedNames = append(movedNames, e.Manifest.Module)
names = append(names, e.Manifest.Module)
} }
fmt.Printf("%s/%s merged into %s (%.8s); building %s\n", plan := planOfMerge(m, movedNames, edges)
m.Owner, m.Repo, m.Base, m.Commit, strings.Join(names, ", ")) if hasCycle(plan.Tiers, edges) {
fmt.Printf(" the last tier depends on itself: %s — built together, in no order\n",
strings.Join(plan.Tiers[len(plan.Tiers)-1], ", "))
}
if err := inv.SavePlan(ctx, plan); err != nil {
return notNow(err)
}
var tiers []string
for i, t := range plan.Tiers {
tiers = append(tiers, fmt.Sprintf("%d: %s", i, strings.Join(t, ", ")))
}
fmt.Printf("%s/%s merged into %s (%.8s); plan %s, %d module(s) in %d tier(s)\n %s\n",
m.Owner, m.Repo, m.Base, m.Commit, plan.ID, len(plan.Modules), len(plan.Tiers), strings.Join(tiers, "\n "))
if len(packaging) > 0 { if len(packaging) > 0 {
var also []string var also []string
for _, e := range packaging { for _, e := range packaging {
@@ -314,22 +328,11 @@ func (f following) SourceMoved(ctx context.Context, m link.SourceMoved) error {
fmt.Printf(" %s package source from it, so they are rebuilt and their own source record "+ fmt.Printf(" %s package source from it, so they are rebuilt and their own source record "+
"is left where it is\n", strings.Join(also, ", ")) "is left where it is\n", strings.Join(also, ", "))
} }
var failed []string if err := askTier(ctx, inv, &plan); err != nil {
for _, e := range ordered { return notNow(err)
source := buildSource{Repository: e.Source.Repository, Seat: e.Source.Seat}
if err := buildOne(ctx, source, e.Source.Path, e.Source.Ref, 20*time.Minute); err != nil {
fmt.Printf(" %s: %v\n", e.Manifest.Module, err)
failed = append(failed, e.Manifest.Module)
// A base that failed is a reason to stop: what stands on it would be built against
// the old one, and report success (novox/hq 04-ISSUES/131).
if standsOn(ordered, e.Manifest.Module, against) {
fmt.Printf(" stopping: %s is a base of what was still to build\n", e.Manifest.Module)
break
} }
} if err := inv.SavePlan(ctx, plan); err != nil {
} return notNow(err)
if len(failed) > 0 {
fmt.Printf("%d of %d not built: %s\n", len(failed), len(ordered), strings.Join(failed, ", "))
} }
return nil return nil
} }
@@ -522,3 +525,37 @@ func isHistory(mergedAt string, seen time.Time) bool {
} }
return at.Before(seen) return at.Before(seen)
} }
// dependentsOf is every catalogued module that stands on one of the moved modules, directly or
// through another dependent, and is not itself among them — in the catalogue's order, so the
// answer is the same each time. A module standing on nothing that moved is left alone: a merge
// rebuilds what it changed and what is built on top of that, not the catalogue.
func dependentsOf(moved, entries []inventory.Entry, against map[string][]string) []inventory.Entry {
bases := map[string]bool{}
for _, e := range moved {
bases[e.Manifest.Module] = true
}
var out []inventory.Entry
taken := map[string]bool{}
for grew := true; grew; {
grew = false
for _, e := range entries {
name := e.Manifest.Module
if bases[name] || taken[name] {
continue
}
for base := range bases {
if standsOnModule(e, base, against) {
taken[name] = true
out = append(out, e)
grew = true
break
}
}
}
for _, e := range out {
bases[e.Manifest.Module] = true
}
}
return out
}
+45
View File
@@ -0,0 +1,45 @@
package main
import (
"testing"
"github.com/novox/mesh-controller/internal/inventory"
)
// The holder of the build seat follows the controller that defines its worker (novox/hq issue 206).
// On 2026-10-03 a plan put the build machine in tier 0 and the controller in tier 1; the new build
// machine could not bind the worker the old controller had defined, and nothing could build the
// controller that would have redefined it. The built-by edge from the controller to its build
// machine yields to that order: the controller is built by whichever build machine is running.
func TestTheBuildSeatsHolderFollowsTheControllerThatDefinesItsWorker(t *testing.T) {
edges := []inventory.Edge{
{From: "build-agent", To: "mesh-controller", Kind: inventory.EdgePackages},
{From: "build-agent", To: "mesh-controller", Kind: inventory.EdgeWorkerOf},
{From: "mesh-controller", To: "build-agent", Kind: inventory.EdgeBuiltBy},
{From: "route-proxy", To: "mesh-controller", Kind: inventory.EdgePackages},
{From: "route-proxy", To: "build-agent", Kind: inventory.EdgeBuiltBy},
}
set := reachableFrom([]string{"mesh-controller"}, edges)
if len(set) != 3 {
t.Fatalf("the controller, what packages it, and nothing more: %v", set)
}
tiers := tiersOf(set, edges)
pos := map[string]int{}
for i, tier := range tiers {
for _, m := range tier {
pos[m] = i
}
}
if pos["mesh-controller"] != 0 {
t.Fatalf("the controller first, built by the build machine that is running: %v", tiers)
}
if pos["build-agent"] <= pos["mesh-controller"] {
t.Fatalf("the build machine after the controller that defines its worker: %v", tiers)
}
if pos["route-proxy"] <= pos["build-agent"] {
t.Fatalf("what the build machine builds comes after it: %v", tiers)
}
if hasCycle(tiers, edges) {
t.Fatalf("no cycle here: %v", tiers)
}
}
+4 -1
View File
@@ -10,7 +10,10 @@
# The client is copied from the vendor's own image rather than installed from a distribution: # The client is copied from the vendor's own image rather than installed from a distribution:
# `apk add mc` on Alpine installs Midnight Commander, which is a different program with the same # `apk add mc` on Alpine installs Midnight Commander, which is a different program with the same
# name, and the failure would be a provisioner that starts cleanly and cannot do anything. # name, and the failure would be a provisioner that starts cleanly and cannot do anything.
FROM golang:1.25-alpine AS build # The Go it builds with is the one the manifest pins (build.on GO_BASE), passed by the Makefile and the
# build machine alike; the default only serves a hand build, and matches go.mod.
ARG GO_BASE=golang:1.26-alpine
FROM ${GO_BASE} AS build
WORKDIR /src WORKDIR /src
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
+4 -1
View File
@@ -3,7 +3,10 @@
# Built here so a machine can be given it by the mesh rather than by somebody putting a binary on # Built here so a machine can be given it by the mesh rather than by somebody putting a binary on
# it. Static and FROM scratch for the same reason the control plane's image is: it is fetched by # it. Static and FROM scratch for the same reason the control plane's image is: it is fetched by
# digest and run on a machine, and everything in it is something a person would have to audit. # digest and run on a machine, and everything in it is something a person would have to audit.
FROM golang:1.25-alpine AS build # The Go it builds with is the one the manifest pins (build.on GO_BASE), passed by the Makefile and the
# build machine alike; the default only serves a hand build, and matches go.mod.
ARG GO_BASE=golang:1.26-alpine
FROM ${GO_BASE} AS build
WORKDIR /src WORKDIR /src
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
+4 -1
View File
@@ -2,7 +2,10 @@
# #
# FROM scratch, like the postgres one and unlike the bucket one: it speaks the store's own wire # FROM scratch, like the postgres one and unlike the bucket one: it speaks the store's own wire
# protocol directly and needs no client in the image. # protocol directly and needs no client in the image.
FROM golang:1.25-alpine AS build # The Go it builds with is the one the manifest pins (build.on GO_BASE), passed by the Makefile and the
# build machine alike; the default only serves a hand build, and matches go.mod.
ARG GO_BASE=golang:1.26-alpine
FROM ${GO_BASE} AS build
WORKDIR /src WORKDIR /src
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
+4 -1
View File
@@ -2,7 +2,10 @@
# #
# Static and FROM scratch like the control plane's image, and for the same reason: it is fetched # Static and FROM scratch like the control plane's image, and for the same reason: it is fetched
# by digest and run on a machine, so everything in it is something a person would have to audit. # by digest and run on a machine, so everything in it is something a person would have to audit.
FROM golang:1.25-alpine AS build # The Go it builds with is the one the manifest pins (build.on GO_BASE), passed by the Makefile and the
# build machine alike; the default only serves a hand build, and matches go.mod.
ARG GO_BASE=golang:1.26-alpine
FROM ${GO_BASE} AS build
WORKDIR /src WORKDIR /src
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
+132
View File
@@ -0,0 +1,132 @@
package main
import (
"encoding/json"
"fmt"
"log"
"os"
"strings"
"sync/atomic"
"time"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker"
)
// What the mesh issued this proxy, read on the bus (novox/hq ADR 0160, ADR 0167).
//
// **The proxy is told, not left to work it out.** Its membership carries the routes it is given —
// the same contributions its file is written from — and every machine's address on the private
// network, which is who may be served an internal name. Read once at connect and followed, so a
// route added or a machine joining reaches a running proxy without a restart.
// credential is the bus account the mesh delivered as this module's own secret named broker.
type credential struct {
URL string `json:"url"`
Fingerprint string `json:"fingerprint"`
Node string `json:"node"`
Module string `json:"module"`
User string `json:"user"`
Password string `json:"password"`
}
// followMembership connects with the credential in path and applies every membership the mesh
// issues this proxy. It retries the first connection for as long as it takes: a proxy that started
// before the bus keeps serving the file, and takes the bus when it answers.
func followMembership(path string, held *table, fromBus *atomic.Bool) {
for {
err := followOnce(path, held, fromBus)
if err == nil {
return
}
log.Printf("cannot follow this proxy's membership, serving the file meanwhile: %v", err)
time.Sleep(30 * time.Second)
}
}
func followOnce(path string, held *table, fromBus *atomic.Bool) error {
raw, err := os.ReadFile(path)
if err != nil {
return err
}
var cred credential
if err := json.Unmarshal(raw, &cred); err != nil {
return fmt.Errorf("the broker credential is not one: %w", err)
}
if cred.Node == "" || cred.Module == "" {
return fmt.Errorf("the broker credential names no node or module, so it has no membership")
}
opts := []nats.Option{
nats.Name(cred.Node + "." + cred.Module),
nats.UserInfo(cred.User, cred.Password),
// Its own inbox, and nothing wider: every principal is granted `_INBOX.<its user>.>` alone.
nats.CustomInboxPrefix("_INBOX." + cred.User),
// The bus being restarted is an upgrade, not a reason to stop following.
nats.MaxReconnects(-1),
}
if strings.TrimSpace(cred.Fingerprint) != "" {
opts = append(opts, nats.Secure(broker.PinnedToFingerprint(cred.Fingerprint)))
}
conn, err := nats.Connect(cred.URL, opts...)
if err != nil {
return fmt.Errorf("connecting to the bus at %s: %w", broker.BareAddress(cred.URL), err)
}
subject := broker.MembershipSubject(cred.Node, cred.Module)
apply := func(body []byte) {
var issued broker.Membership
if err := json.Unmarshal(body, &issued); err != nil {
log.Printf("a membership arrived that is not one: %v", err)
return
}
if took := applyMembership(issued, held); took && !fromBus.Swap(true) {
log.Printf("routes now come from this proxy's membership on %s", subject)
}
}
// Followed first, read second: an issue landing between the two is applied, not missed.
if _, err := conn.Subscribe(subject, func(m *nats.Msg) { apply(m.Data) }); err != nil {
conn.Close()
return fmt.Errorf("cannot follow %s: %w", subject, err)
}
// The subject-addressed direct get: the one request this account may make of the stream.
got, err := conn.Request("$JS.API.DIRECT.GET."+broker.AssignmentsStream+"."+subject, nil, 5*time.Second)
switch {
case err != nil:
log.Printf("cannot read the membership issued on %s yet (%v); following it", subject, err)
case got.Header.Get("Status") != "" || len(got.Data) == 0:
log.Printf("no membership issued on %s yet; serving the file until one is", subject)
default:
apply(got.Data)
}
return nil
}
// applyMembership serves what a membership says, and says whether it said anything about routes.
//
// A membership with no routes in it is one from a controller older than ADR 0167, and the file stays
// the source rather than every route being withdrawn because a field was absent.
func applyMembership(issued broker.Membership, held *table) bool {
raw, carries := issued.Receives["route"]
if !carries {
return false
}
var contributions []contribution
if err := json.Unmarshal(raw, &contributions); err != nil {
log.Printf("the routes in this proxy's membership are not contributions, keeping what is served: %v", err)
return false
}
inside, err := sourcesOf(issued.Mesh)
if err != nil {
log.Printf("the mesh in this proxy's membership is unreadable, keeping what is served: %v", err)
return false
}
routes, public := routesOf(contributions)
held.set(routes, public)
held.setInside(inside)
log.Printf("serving %d route(s) from the membership, internal names to %d machine(s): %s",
len(routes), len(inside), strings.Join(held.names(), ", "))
return true
}
+29
View File
@@ -0,0 +1,29 @@
package main
import (
"crypto/tls"
"net/http"
"net/http/httptest"
"net/url"
"testing"
)
// A backend behind the proxy learns the client used TLS and which name it asked for, so the addresses
// it writes into its own pages are the ones a client can use (2026-10-03: a forge's Go import tag
// named an http clone URL, and Go refused the module path).
func TestABackendIsToldTheRequestWasHTTPSAndForWhichName(t *testing.T) {
var proto, host, fwdHost, fwdFor string
backend := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
proto, host, fwdHost, fwdFor = r.Header.Get("X-Forwarded-Proto"), r.Host, r.Header.Get("X-Forwarded-Host"), r.Header.Get("X-Forwarded-For")
}))
defer backend.Close()
where, _ := url.Parse(backend.URL)
req := httptest.NewRequest(http.MethodGet, "https://git.example.org/novox/mesh-sdk/go?go-get=1", nil)
req.TLS = &tls.ConnectionState{}
req.Host = "git.example.org"
req.RemoteAddr = "192.0.2.7:51000"
towards(where).ServeHTTP(httptest.NewRecorder(), req)
if proto != "https" || fwdHost != "git.example.org" || host != "git.example.org" || fwdFor != "192.0.2.7" {
t.Errorf("the backend was told proto=%q host=%q forwarded-host=%q for=%q", proto, host, fwdHost, fwdFor)
}
}
+194 -29
View File
@@ -54,12 +54,14 @@ import (
"net" "net"
"net/http" "net/http"
"net/http/httputil" "net/http/httputil"
"net/netip"
"net/url" "net/url"
"os" "os"
"path/filepath" "path/filepath"
"sort" "sort"
"strings" "strings"
"sync" "sync"
"sync/atomic"
"time" "time"
"golang.org/x/crypto/acme" "golang.org/x/crypto/acme"
@@ -196,6 +198,63 @@ type table struct {
// pass ACME's own validation (it has no public DNS to prove it against), so asking for it is // pass ACME's own validation (it has no public DNS to prove it against), so asking for it is
// not merely pointless but the failing order onlyWhatTheMeshSaid exists to prevent. // not merely pointless but the failing order onlyWhatTheMeshSaid exists to prevent.
public map[string]bool public map[string]bool
// inside is where a request must come from to be served a name that is only internal: every
// machine's address on the private network, as the mesh issued it in this proxy's membership
// (novox/hq ADR 0167). Empty until it is issued, and then only the machine itself is inside.
inside sources
}
// sources is who may be served an internal name: the private network's addresses as the mesh
// issued them. The machine itself is always inside — anything on a machine may call anything on it
// (novox/hq ADR 0144) — so loopback needs no entry.
type sources []netip.Prefix
// sourcesOf reads the addresses the mesh issued, each a single address or a range. One that does
// not parse is an error, not an entry skipped: the proxy would otherwise serve internal names to
// fewer machines than the mesh said, and say nothing.
func sourcesOf(mesh []string) (sources, error) {
var out sources
for _, entry := range mesh {
entry = strings.TrimSpace(entry)
if prefix, err := netip.ParsePrefix(entry); err == nil {
out = append(out, prefix.Masked())
continue
}
addr, err := netip.ParseAddr(entry)
if err != nil {
return nil, fmt.Errorf("%q is not an address on the private network", entry)
}
addr = addr.Unmap()
out = append(out, netip.PrefixFrom(addr, addr.BitLen()))
}
return out, nil
}
// holds says whether a request from this remote address came from the mesh or the machine itself.
//
// **By source, which the mesh's guard deliberately is not** — it names interfaces, because a source
// address can be claimed by whoever sends the packet. The proxy cannot see the interface a request
// arrived on, and here the claim does not carry: a connection needs its replies, and replies to a
// mesh address leave by the tunnel, never back to the claimant.
func (s sources) holds(remote string) bool {
host := remote
if h, _, err := net.SplitHostPort(remote); err == nil {
host = h
}
addr, err := netip.ParseAddr(host)
if err != nil {
return false
}
addr = addr.Unmap()
if addr.IsLoopback() {
return true
}
for _, prefix := range s {
if prefix.Contains(addr) {
return true
}
}
return false
} }
func (t *table) set(routes map[string][]rule, public map[string]bool) { func (t *table) set(routes map[string][]rule, public map[string]bool) {
@@ -214,7 +273,7 @@ func (t *table) set(routes map[string][]rule, public map[string]bool) {
log.Printf("route %s points at %q, which is not a URL: %v", host, r.target, err) log.Printf("route %s points at %q, which is not a URL: %v", host, r.target, err)
continue continue
} }
r.to = httputil.NewSingleHostReverseProxy(where) r.to = towards(where)
if r.insecure { if r.insecure {
r.to.Transport = &http.Transport{TLSClientConfig: &tls.Config{InsecureSkipVerify: true}} r.to.Transport = &http.Transport{TLSClientConfig: &tls.Config{InsecureSkipVerify: true}}
} }
@@ -314,6 +373,41 @@ func bareHost(host string) string {
return strings.ToLower(host) return strings.ToLower(host)
} }
// hiddenFrom says whether this host must look unrouted to a request from this address: it is
// only an internal name, and the request did not come from the private network.
//
// **The proxy is the only way in to a routed endpoint, so it is what makes `internal` true**
// (novox/hq ADR 0138, issue 191). It answers public names on the same listeners, so a request from
// anywhere can carry any Host header; a name being internal keeps nobody out unless this check does.
// Answered exactly as a name that was never routed, so an outsider learns nothing from asking.
func (t *table) hiddenFrom(host, remote string) bool {
if !t.eligibleForInternalACME(host) {
return false
}
t.mu.RLock()
defer t.mu.RUnlock()
return !t.inside.holds(remote)
}
// setInside replaces who the mesh is, as the membership said.
func (t *table) setInside(inside sources) {
t.mu.Lock()
t.inside = inside
t.mu.Unlock()
}
// namesSeenFrom is what this proxy says it serves to a request from this address — every routed
// name, less the internal-only ones when the request came from outside.
func (t *table) namesSeenFrom(remote string) []string {
out := []string{}
for _, name := range t.names() {
if !t.hiddenFrom(name, remote) {
out = append(out, name)
}
}
return out
}
func (t *table) names() []string { func (t *table) names() []string {
t.mu.RLock() t.mu.RLock()
defer t.mu.RUnlock() defer t.mu.RUnlock()
@@ -343,7 +437,20 @@ func run() error {
} }
held := newTable() held := newTable()
// **The bus first, the file until it has spoken** (novox/hq ADR 0167). The membership carries
// the routes and who the mesh is; the file carries the routes alone, so while the proxy reads
// it an internal name is served to this machine and to nobody else — refused, never opened.
fromBus := &atomic.Bool{}
if credential := strings.TrimSpace(os.Getenv("MESH_BROKER_FILE")); credential != "" {
go followMembership(credential, held, fromBus)
} else {
log.Printf("MESH_BROKER_FILE is not set: routes come from %s alone, and a name that is only "+
"internal is served to this machine alone", path)
}
read := func() { read := func() {
if fromBus.Load() {
return
}
routes, public, err := routesFrom(path) routes, public, err := routesFrom(path)
if err != nil { if err != nil {
// Kept serving what it had. A file being rewritten is momentarily unreadable, and // Kept serving what it had. A file being rewritten is momentarily unreadable, and
@@ -422,19 +529,7 @@ func run() error {
}() }()
tlsConfig := publicManager.TLSConfig() tlsConfig := publicManager.TLSConfig()
if internalManager != nil { tlsConfig.GetCertificate = certificateFor(held, tlsConfig.GetCertificate, internalManager)
// Dispatched by which authority may certify this name at all — the same question
// eligibleForInternalACME already answers, asked once more at handshake time rather than
// only when an order is placed, since a cached certificate is served here on every request
// and never goes through HostPolicy again.
fromPublic, fromInternal := tlsConfig.GetCertificate, internalManager.TLSConfig().GetCertificate
tlsConfig.GetCertificate = func(hello *tls.ClientHelloInfo) (*tls.Certificate, error) {
if held.eligibleForInternalACME(hello.ServerName) {
return fromInternal(hello)
}
return fromPublic(hello)
}
}
server := &http.Server{ server := &http.Server{
Addr: secure, Addr: secure,
@@ -592,6 +687,33 @@ func forThisAuthority(cache, directory string, root []byte) string {
return filepath.Join(cache, hex.EncodeToString(sum[:])[:16]) return filepath.Join(cache, hex.EncodeToString(sum[:])[:16])
} }
// certificateFor picks the certificate a handshake is answered with.
//
// Dispatched by which authority may certify this name at all — the same question
// eligibleForInternalACME already answers, asked once more at handshake time rather than only when
// an order is placed, since a cached certificate is served here on every request and never goes
// through HostPolicy again. And refused, exactly as an unrouted name is, to a client outside the
// private network asking for a name that is only internal: the certificate would name it.
func certificateFor(held *table, fromPublic func(*tls.ClientHelloInfo) (*tls.Certificate, error),
internalManager *autocert.Manager) func(*tls.ClientHelloInfo) (*tls.Certificate, error) {
var fromInternal func(*tls.ClientHelloInfo) (*tls.Certificate, error)
if internalManager != nil {
fromInternal = internalManager.TLSConfig().GetCertificate
}
return func(hello *tls.ClientHelloInfo) (*tls.Certificate, error) {
if held.eligibleForInternalACME(hello.ServerName) {
if hello.Conn != nil && held.hiddenFrom(hello.ServerName, hello.Conn.RemoteAddr().String()) {
return nil, fmt.Errorf("no public route for %q in this mesh, so no certificate is asked for",
hello.ServerName)
}
if fromInternal != nil {
return fromInternal(hello)
}
}
return fromPublic(hello)
}
}
// newTable is an empty routing table. // newTable is an empty routing table.
func newTable() *table { func newTable() *table {
return &table{to: map[string][]rule{}} return &table{to: map[string][]rule{}}
@@ -600,8 +722,9 @@ func newTable() *table {
// handler is the proxy itself, separated so it can be driven by a test without a listener. // handler is the proxy itself, separated so it can be driven by a test without a listener.
func handler(held *table) http.Handler { func handler(held *table) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
hidden := held.hiddenFrom(r.Host, r.RemoteAddr)
matched, known := held.find(r.Host, r.URL.Path) matched, known := held.find(r.Host, r.URL.Path)
if !known { if hidden || !known {
// **Named, not a bare 404.** A route that was withdrawn and a name that never existed // **Named, not a bare 404.** A route that was withdrawn and a name that never existed
// are different things, and a proxy that says only "not found" makes an operator go // are different things, and a proxy that says only "not found" makes an operator go
// and read the mesh to tell them apart. What it is serving is the answer to both. // and read the mesh to tell them apart. What it is serving is the answer to both.
@@ -609,15 +732,20 @@ func handler(held *table) http.Handler {
// And since a host may now be routed only on some paths, those are a third thing: // And since a host may now be routed only on some paths, those are a third thing:
// saying "no route for this name" while listing that very name as served is a // saying "no route for this name" while listing that very name as served is a
// contradiction an operator would have to disbelieve the proxy to get past. // contradiction an operator would have to disbelieve the proxy to get past.
// **Said in the log as well as to the client.** A name this mesh does not serve, asked
// for from outside, is what a scanner does, and the machine's intrusion prevention reads
// this proxy's log for exactly that line (novox/hq ADR 0179): the address last, as the
// jail's filter expects it.
log.Printf("refused: no route for %q, asked from %s", r.Host, r.RemoteAddr)
w.Header().Set("Content-Type", "text/plain; charset=utf-8") w.Header().Set("Content-Type", "text/plain; charset=utf-8")
w.WriteHeader(http.StatusNotFound) w.WriteHeader(http.StatusNotFound)
if held.routed(r.Host) { if !hidden && held.routed(r.Host) {
fmt.Fprintf(w, "%s is served here, but no route covers %q.\n", fmt.Fprintf(w, "%s is served here, but no route covers %q.\n",
bareHost(r.Host), r.URL.Path) bareHost(r.Host), r.URL.Path)
return return
} }
fmt.Fprintf(w, "no route for %q in this mesh.\nserving: %s\n", fmt.Fprintf(w, "no route for %q in this mesh.\nserving: %s\n",
r.Host, strings.Join(held.names(), ", ")) r.Host, strings.Join(held.namesSeenFrom(r.RemoteAddr), ", "))
return return
} }
@@ -716,6 +844,12 @@ func boolByte(b bool) byte {
// routesFrom reads what the mesh wrote and turns it into host → the rules for that host, and // routesFrom reads what the mesh wrote and turns it into host → the rules for that host, and
// which of those hosts is a public name — the second is `name`, ACME-eligible; a host reached // which of those hosts is a public name — the second is `name`, ACME-eligible; a host reached
// only through `internal-name` never appears there. // only through `internal-name` never appears there.
//
// **A route may carry either name, or both** (novox/hq ADR 0138). How far an endpoint reaches
// decides which names the mesh composes, so an endpoint that reaches only the private network
// arrives with an `internal-name` and no `name`. That is a whole route, not a malformed one: it is
// served under its internal name and certified by the internal authority. Only a route with
// neither name has nothing to be served under (novox/hq issue 191).
func routesFrom(path string) (map[string][]rule, map[string]bool, error) { func routesFrom(path string) (map[string][]rule, map[string]bool, error) {
raw, err := os.ReadFile(path) raw, err := os.ReadFile(path)
if err != nil { if err != nil {
@@ -725,17 +859,29 @@ func routesFrom(path string) (map[string][]rule, map[string]bool, error) {
if err := json.Unmarshal(raw, &said); err != nil { if err := json.Unmarshal(raw, &said); err != nil {
return nil, nil, err return nil, nil, err
} }
routes, public := routesOf(said.Given)
return routes, public, nil
}
// routesOf turns what the mesh gave into host → the rules for that host, and which hosts are public
// names — the same whether the contributions came in the file or in the membership.
func routesOf(contributions []contribution) (map[string][]rule, map[string]bool) {
out := map[string][]rule{} out := map[string][]rule{}
public := map[string]bool{} public := map[string]bool{}
for _, c := range said.Given { for _, c := range contributions {
name, _ := c.Values["name"].(string) name, _ := c.Values["name"].(string)
if name == "" { name = strings.TrimSpace(name)
internal, _ := c.Values["internal-name"].(string)
internal = strings.TrimSpace(internal)
if name == "" && internal == "" {
log.Printf("%s on %s asked for a route and named nothing; skipped", c.From, c.Node) log.Printf("%s on %s asked for a route and named nothing; skipped", c.From, c.Node)
continue continue
} }
host := strings.ToLower(name) // What the route is called in a log line: its public name when it has one.
public[host] = true called := name
if called == "" {
called = internal
}
made := rule{path: asPath(c.Values["path"])} made := rule{path: asPath(c.Values["path"])}
if p, ok := asWhole(c.Values["priority"]); ok { if p, ok := asWhole(c.Values["priority"]); ok {
@@ -752,7 +898,7 @@ func routesFrom(path string) (map[string][]rule, map[string]bool, error) {
if looksLikeACredential(named) { if looksLikeACredential(named) {
log.Printf("%s on %s declared route %q with a credential in the declaration rather "+ log.Printf("%s on %s declared route %q with a credential in the declaration rather "+
"than the name of a secret; the whole route is refused (novox/hq ADR 0108)", "than the name of a secret; the whole route is refused (novox/hq ADR 0108)",
c.From, c.Node, name) c.From, c.Node, called)
continue continue
} }
users, err := usersFrom(named) users, err := usersFrom(named)
@@ -770,7 +916,7 @@ func routesFrom(path string) (map[string][]rule, map[string]bool, error) {
port, ok := asPort(c.Values["port"]) port, ok := asPort(c.Values["port"])
if !ok { if !ok {
log.Printf("%s on %s asked for route %q and gave no usable port; skipped", log.Printf("%s on %s asked for route %q and gave no usable port; skipped",
c.From, c.Node, name) c.From, c.Node, called)
continue continue
} }
// Where the mesh says that machine is. Empty means it is this one — a workload beside // Where the mesh says that machine is. Empty means it is this one — a workload beside
@@ -791,7 +937,7 @@ func routesFrom(path string) (map[string][]rule, map[string]bool, error) {
} }
if scheme != "http" && scheme != "https" { if scheme != "http" && scheme != "https" {
log.Printf("%s on %s asked for route %q with scheme %q, which is neither http "+ log.Printf("%s on %s asked for route %q with scheme %q, which is neither http "+
"nor https; skipped", c.From, c.Node, name, scheme) "nor https; skipped", c.From, c.Node, called, scheme)
continue continue
} }
made.insecure, _ = c.Values["insecure"].(bool) made.insecure, _ = c.Values["insecure"].(bool)
@@ -802,7 +948,7 @@ func routesFrom(path string) (map[string][]rule, map[string]bool, error) {
bytes, whole := asWhole(asked) bytes, whole := asWhole(asked)
if !whole || bytes <= 0 { if !whole || bytes <= 0 {
log.Printf("%s on %s asked for route %q with a max-request-body of %v, which is "+ log.Printf("%s on %s asked for route %q with a max-request-body of %v, which is "+
"not a whole positive number of bytes; skipped", c.From, c.Node, name, asked) "not a whole positive number of bytes; skipped", c.From, c.Node, called, asked)
continue continue
} }
made.maxRequestBody = int64(bytes) made.maxRequestBody = int64(bytes)
@@ -810,19 +956,24 @@ func routesFrom(path string) (map[string][]rule, map[string]bool, error) {
made.target = fmt.Sprintf("%s://%s:%d", scheme, at, port) made.target = fmt.Sprintf("%s://%s:%d", scheme, at, port)
} }
if name != "" {
host := strings.ToLower(name)
out[host] = append(out[host], made) out[host] = append(out[host], made)
public[host] = true
}
// The internal-network alias, the same rule under a second host — a predecessor proxy // The internal-network name, the same rule under a second host — a predecessor proxy
// answered both for one route, as a convenience (reaching a service over the VPN without a // answered both for one route, as a convenience (reaching a service over the VPN without a
// public TLS round trip), not as an access boundary; composing it here restores exactly // public TLS round trip), not as an access boundary; composing it here restores exactly
// that, nothing more. Absent whenever the node composed no internal name (novox/hq ADR // that, nothing more. Absent whenever the node composed no internal name (novox/hq ADR
// 0056's internalDomain half) — the same "nothing to join a label to" case the public name // 0056's internalDomain half) — the same "nothing to join a label to" case the public name
// already has. // already has. And the only name, when the endpoint reaches no further than the private
if internal, _ := c.Values["internal-name"].(string); strings.TrimSpace(internal) != "" { // network.
if internal != "" {
out[strings.ToLower(internal)] = append(out[strings.ToLower(internal)], made) out[strings.ToLower(internal)] = append(out[strings.ToLower(internal)], made)
} }
} }
return out, public, nil return out, public
} }
// asWhole is any whole number the mesh wrote, whatever its magnitude. // asWhole is any whole number the mesh wrote, whatever its magnitude.
@@ -909,3 +1060,17 @@ func asPort(v any) (int, bool) {
} }
return 0, false return 0, false
} }
// towards proxies to one backend and tells it what the client asked: **X-Forwarded-Proto, -Host and
// -For**, set from the request this proxy received. A backend that builds its own addresses — a forge
// writing its clone URL into a page, a login redirect — otherwise sees the plain HTTP hop from this
// proxy and writes `http://`, though every client reached it over TLS: Go refused the forge's module
// path for exactly that on 2026-10-03, its import tag naming an http clone URL.
// The standard library's NewSingleHostReverseProxy sets only X-Forwarded-For.
func towards(where *url.URL) *httputil.ReverseProxy {
return &httputil.ReverseProxy{Rewrite: func(pr *httputil.ProxyRequest) {
pr.SetURL(where)
pr.Out.Host = pr.In.Host
pr.SetXForwarded()
}}
}
+206
View File
@@ -0,0 +1,206 @@
package main
import (
"crypto/tls"
"encoding/json"
"fmt"
"io"
"net"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/broker"
)
// behind is a workload the proxy can send to, and a table routing one public name and one
// internal-only name to it, with the mesh's machines as the membership would issue them.
func behind(t *testing.T, mesh ...string) *table {
t.Helper()
workload := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
io.WriteString(w, "the workload")
}))
t.Cleanup(workload.Close)
at, _ := url.Parse(workload.URL)
host, port, _ := net.SplitHostPort(at.Host)
routes, public, err := routesFrom(write(t, fmt.Sprintf(`{"given":[
{"from":"app","node":"anchor","at":%q,
"values":{"name":"app.example","internal-name":"app.anchor.internal","port":%s}},
{"from":"admin","node":"anchor","at":%q,
"values":{"internal-name":"admin.anchor.internal","port":%s}}
]}`, host, port, host, port)))
if err != nil {
t.Fatal(err)
}
held := newTable()
inside, err := sourcesOf(mesh)
if err != nil {
t.Fatal(err)
}
held.setInside(inside)
held.set(routes, public)
return held
}
// askFrom is what the proxy answers a request for host coming from remote.
func askFrom(held *table, host, remote string) (int, string) {
r := httptest.NewRequest(http.MethodGet, "http://"+host+"/", nil)
r.RemoteAddr = remote
w := httptest.NewRecorder()
handler(held).ServeHTTP(w, r)
return w.Code, w.Body.String()
}
// **An internal-only name is served to the private network and to nobody else** (novox/hq ADR
// 0138, issue 191). The proxy answers public names on the same listeners, so without this a name
// being internal kept nobody out: a request from the internet only had to carry it.
func TestAnInternalOnlyNameIsServedOnlyInsideThePrivateNetwork(t *testing.T) {
held := behind(t, "10.10.0.1", "10.10.0.7")
if code, body := askFrom(held, "admin.anchor.internal", "10.10.0.7:51000"); code != http.StatusOK ||
body != "the workload" {
t.Errorf("a request from the private network was not served: %d %q", code, body)
}
if code, body := askFrom(held, "admin.anchor.internal", "127.0.0.1:51000"); code != http.StatusOK {
t.Errorf("a request from the machine itself was not served: %d %q", code, body)
}
code, body := askFrom(held, "admin.anchor.internal", "203.0.113.9:51000")
if code != http.StatusNotFound {
t.Fatalf("a request from outside the private network reached an internal-only name: %d %q",
code, body)
}
// Answered as a name never routed, and the list of what is served does not name it either —
// otherwise the refusal would tell an outsider exactly what to ask for from inside.
if strings.Contains(strings.SplitN(body, "\n", 2)[1], "admin.anchor.internal") {
t.Errorf("the refusal names the internal-only route to an outsider: %q", body)
}
if !strings.Contains(body, "app.example") {
t.Errorf("the refusal stopped listing the public names: %q", body)
}
}
// The internal name of a route that also has a public one is internal too: served inside, and to
// an outsider only under the public name. Nothing is lost — the outsider has the public name — and a
// name stays one thing whichever route it came from.
func TestAnInternalAliasOfAPublicRouteIsServedInsideOnly(t *testing.T) {
held := behind(t, "10.10.0.1", "10.10.0.7")
if code, body := askFrom(held, "app.anchor.internal", "10.10.0.7:51000"); code != http.StatusOK {
t.Errorf("the internal alias stopped answering the private network: %d %q", code, body)
}
if code, _ := askFrom(held, "app.anchor.internal", "203.0.113.9:51000"); code != http.StatusNotFound {
t.Errorf("the internal alias was served to an outsider: %d", code)
}
if code, _ := askFrom(held, "app.example", "203.0.113.9:51000"); code != http.StatusOK {
t.Errorf("the public name was refused to an outsider: %d", code)
}
}
// Before a membership has said who the mesh is, only the machine itself is inside — refused to
// everyone else, never served to everyone.
func TestUntilTheMeshIsIssuedAnInternalOnlyNameIsServedToTheMachineAlone(t *testing.T) {
held := behind(t)
if code, _ := askFrom(held, "admin.anchor.internal", "10.10.0.7:51000"); code != http.StatusNotFound {
t.Errorf("an internal-only name was served with no private network said: %d", code)
}
if code, _ := askFrom(held, "admin.anchor.internal", "[::1]:51000"); code != http.StatusOK {
t.Errorf("an internal-only name was refused to the machine itself: %d", code)
}
}
type from struct {
net.Conn
remote net.Addr
}
func (c from) RemoteAddr() net.Addr { return c.remote }
// The handshake refuses an internal-only name to an outsider too: the certificate would name it,
// and serving it would answer the question the routing refuses to.
func TestTheHandshakeRefusesAnInternalOnlyNameToAnOutsider(t *testing.T) {
held := behind(t, "10.10.0.1", "10.10.0.7")
served := &tls.Certificate{}
pick := certificateFor(held, func(*tls.ClientHelloInfo) (*tls.Certificate, error) { return served, nil }, nil)
hello := func(name, remote string) *tls.ClientHelloInfo {
addr, _ := net.ResolveTCPAddr("tcp", remote)
return &tls.ClientHelloInfo{ServerName: name, Conn: from{remote: addr}}
}
if _, err := pick(hello("admin.anchor.internal", "203.0.113.9:443")); err == nil {
t.Error("an outsider was handed a certificate for an internal-only name")
}
if got, err := pick(hello("admin.anchor.internal", "10.10.0.7:443")); err != nil || got != served {
t.Errorf("a client on the private network was refused: %v", err)
}
if got, err := pick(hello("app.example", "203.0.113.9:443")); err != nil || got != served {
t.Errorf("a public name was refused to an outsider: %v", err)
}
}
// The mesh is issued as machines' addresses; a range is read as well. One that does not parse is
// refused rather than skipped, so a typo never quietly narrows or widens who is inside.
func TestTheMeshIsReadAsAddressesAndRanges(t *testing.T) {
if _, err := sourcesOf([]string{"10.10.0.1", "not-an-address"}); err == nil {
t.Error("an entry that is not an address was accepted")
}
inside, err := sourcesOf([]string{"10.10.0.1", "fd00::1", "10.20.0.0/24"})
if err != nil {
t.Fatal(err)
}
for remote, want := range map[string]bool{
"10.10.0.1:1": true,
"[::ffff:10.10.0.1]:1": true,
"[fd00::1]:1": true,
"10.20.0.200:1": true,
"10.10.0.2:1": false,
"192.168.1.10:1": false,
"not-an-address": false,
} {
if inside.holds(remote) != want {
t.Errorf("%s inside the mesh: got %v, want %v", remote, !want, want)
}
}
}
// What the mesh issues is what is served: the routes in the membership, internal names to the
// machines it names (novox/hq ADR 0167).
func TestAMembershipIsServedAsIssued(t *testing.T) {
held := newTable()
took := applyMembership(broker.Membership{
Receives: map[string]json.RawMessage{"route": json.RawMessage(`[
{"from":"admin","node":"anchor","at":"anchor.internal",
"values":{"internal-name":"admin.anchor.internal","port":8080}}]`)},
Mesh: []string{"10.10.0.7"},
}, held)
if !took {
t.Fatal("a membership carrying routes was not applied")
}
if code, _ := askFrom(held, "admin.anchor.internal", "10.10.0.7:1"); code == http.StatusNotFound {
t.Error("a machine the membership names was refused the internal-only route")
}
if code, _ := askFrom(held, "admin.anchor.internal", "10.10.0.9:1"); code != http.StatusNotFound {
t.Errorf("a machine the membership does not name was served the internal-only route: %d", code)
}
}
// A membership that says nothing about routes is one from a controller that does not issue them,
// and changes nothing: the file stays the source rather than every route being withdrawn.
func TestAMembershipWithoutRoutesLeavesTheFileServing(t *testing.T) {
held := behind(t, "10.10.0.7")
before := held.names()
if applyMembership(broker.Membership{Mesh: []string{"10.10.0.7"}}, held) {
t.Error("a membership without routes was taken as the source of routes")
}
if got := held.names(); strings.Join(got, ",") != strings.Join(before, ",") {
t.Errorf("a membership without routes changed what is served: %v, was %v", got, before)
}
if applyMembership(broker.Membership{
Receives: map[string]json.RawMessage{"route": json.RawMessage(`[]`)},
Mesh: []string{"not-an-address"},
}, held) {
t.Error("a membership whose mesh cannot be read was applied")
}
}
+44
View File
@@ -90,6 +90,50 @@ func TestARouteWithAnInternalNameIsReachableUnderBoth(t *testing.T) {
} }
} }
// A route whose endpoint reaches only the private network carries an internal name and no public
// one (novox/hq ADR 0138), and is served under that name rather than skipped as naming nothing —
// skipping it left every internal-only module unreachable by name (novox/hq issue 191).
func TestARouteWithOnlyAnInternalNameIsServed(t *testing.T) {
routes, public, err := routesFrom(write(t, `{"given":[
{"from":"app","node":"anchor","at":"anchor.internal",
"values":{"internal-name":"App.Anchor.Internal","port":8443,"scheme":"https","insecure":true}}
]}`))
if err != nil {
t.Fatal(err)
}
if targetOf(routes, "app.anchor.internal") != "https://anchor.internal:8443" {
t.Fatalf("the internal-only route is not served: %v", routes)
}
if len(routes) != 1 {
t.Errorf("an internal-only route made hosts it never named: %v", routes)
}
if len(public) != 0 {
t.Errorf("an internal-only route made a name eligible for a public certificate: %v", public)
}
held := newTable()
held.set(routes, public)
if err := onlyInternalNamesTheMeshSaid(held)(context.Background(), "app.anchor.internal"); err != nil {
t.Errorf("the internal authority refused the internal-only route's name: %v", err)
}
if err := onlyWhatTheMeshSaid(held)(context.Background(), "app.anchor.internal"); err == nil {
t.Error("a public certificate was ordered for an internal-only name")
}
}
// A route with neither name has nothing to be served under, and is still skipped.
func TestARouteWithNeitherNameIsSkipped(t *testing.T) {
routes, public, err := routesFrom(write(t, `{"given":[
{"from":"app","node":"anchor","at":"anchor.internal","values":{"internal-name":" ","port":8080}}
]}`))
if err != nil {
t.Fatal(err)
}
if len(routes) != 0 || len(public) != 0 {
t.Errorf("a route that named nothing was served: %v %v", routes, public)
}
}
// A route with no internal-name composed gets no second host — the ordinary case, unchanged. // A route with no internal-name composed gets no second host — the ordinary case, unchanged.
func TestARouteWithNoInternalNameGetsNoAlias(t *testing.T) { func TestARouteWithNoInternalNameGetsNoAlias(t *testing.T) {
routes, _, err := routesFrom(write(t, `{"given":[ routes, _, err := routesFrom(write(t, `{"given":[
@@ -234,3 +234,13 @@ func admitsSubject(pattern, subject []string) bool {
} }
return len(pattern) == len(subject) return len(pattern) == len(subject)
} }
// The two packages name the runtime module separately — the broker's types stay free of the
// catalogue's on purpose — so this is what holds them to one string. A rename that reached only one
// side would compose a runtime principal for a module nobody assigns, silently, and leave the one
// that is assigned with a module's own grants.
func TestTheBrokerAndTheCatalogueAgreeOnTheRuntimeModule(t *testing.T) {
if RuntimeModule != catalogue.RuntimeModule {
t.Fatalf("the broker calls the runtime %q and the catalogue %q", RuntimeModule, catalogue.RuntimeModule)
}
}
+16 -8
View File
@@ -144,12 +144,18 @@ func ConsumerFor(p Principal) (Consumer, bool) {
}, true }, true
} }
// HolderConsumerFor is the worker a seat's holder gets on that seat's work queue. // HolderConsumerFor is the worker a seat's holders share on that seat's work queue.
// //
// **A queue group even though the seat guarantees one holder.** The seat is *authority* — who may // **One worker for every holder, and each holder pulls one ask when it is idle** (novox/hq ADR
// be the telegram sender — and the queue group is *delivery*. Tie delivery to the seat and the // 0190). The seat is *authority* — who may be the telegram sender — and the worker is *delivery*,
// day somebody allows two holders for throughput, every message is processed twice with nothing // kept separate so that relaxing one changes nothing about the other: a node-scoped seat has a
// reporting it. Kept separate, relaxing one changes nothing about the other. // holder per machine, and all of them take from this one consumer, so the work is shared without
// any holder knowing about the others. Pulled rather than pushed because a push consumer hands the
// next ask to whichever subscriber the server picks, busy or not, and a pulled one is asked for by
// a holder that has just become free. Which is also what ends the race issue 186 describes — asks
// delivered behind the one being worked, expiring unacknowledged and dropped after the fifth
// redelivery: nothing is delivered that nobody asked for. A long build keeps its own ask alive
// (stillWorking); the ack wait is for a holder that died.
func HolderConsumerFor(node, module string, seat DeclaredSeat) (Consumer, bool) { func HolderConsumerFor(node, module string, seat DeclaredSeat) (Consumer, bool) {
if len(seat.Accepts) == 0 { if len(seat.Accepts) == 0 {
return Consumer{}, false return Consumer{}, false
@@ -158,11 +164,13 @@ func HolderConsumerFor(node, module string, seat DeclaredSeat) (Consumer, bool)
Name: "SEAT_" + upperSnake(seat.Name) + "_worker", Name: "SEAT_" + upperSnake(seat.Name) + "_worker",
Stream: seatStreamName(seat.Name), Stream: seatStreamName(seat.Name),
Filters: []string{"mesh.seat." + seat.Name + ".accept.>"}, Filters: []string{"mesh.seat." + seat.Name + ".accept.>"},
Queue: "holders",
AckWaitSeconds: 60, AckWaitSeconds: 60,
MaxDeliver: 5, MaxDeliver: 5,
Why: fmt.Sprintf("%s on %s holds %s; it acknowledges after the work is done, so a "+ // As many in flight as there are holders working, which pulling bounds by itself: a holder
"crash mid-work redelivers rather than loses", module, node, seat.Name), // fetches one and fetches again only after it acknowledged. The server's default stands.
Why: fmt.Sprintf("%s on %s holds %s; every holder pulls one ask at a time from this worker "+
"and acknowledges after the work is done, so a crash mid-work redelivers rather than "+
"loses and an idle holder is the one that takes the next ask", module, node, seat.Name),
}, true }, true
} }
+31 -5
View File
@@ -88,15 +88,20 @@ func TestAModuleThatConsumesNothingGetsNoConsumer(t *testing.T) {
} }
} }
// The seat is authority and the queue group is delivery. Tie them together and the day somebody // The seat is authority and the worker is delivery (novox/hq ADR 0190): one worker per seat, shared
// allows two holders, every message is processed twice with nothing reporting it. // by every holder and pulled from, so a second holder takes the next ask rather than a copy of the
func TestAHoldersWorkerUsesAQueueGroupAnyway(t *testing.T) { // same one — which is what a queue group used to guard, and what pulling one durable gives outright.
func TestAHoldersWorkerIsOneSharedByItsHolders(t *testing.T) {
c, ok := HolderConsumerFor("one", "telegram", telegramSeat()) c, ok := HolderConsumerFor("one", "telegram", telegramSeat())
if !ok { if !ok {
t.Fatal("the holder of a seat with inbound work got no worker") t.Fatal("the holder of a seat with inbound work got no worker")
} }
if c.Queue == "" { two, _ := HolderConsumerFor("two", "telegram", telegramSeat())
t.Fatal("the worker is not in a queue group, so a second holder would double-process") if c.Name != two.Name || c.Stream != two.Stream {
t.Fatal("two holders got two workers, so each would process every ask")
}
if c.Push || c.Queue != "" {
t.Fatal("the worker is pushed, so the server would hand an ask to a busy holder")
} }
if c.Stream != "SEAT_TELEGRAM_SENDER" { if c.Stream != "SEAT_TELEGRAM_SENDER" {
t.Fatalf("the worker reads %q, not the seat's own stream", c.Stream) t.Fatalf("the worker reads %q, not the seat's own stream", c.Stream)
@@ -153,3 +158,24 @@ func TestANodesDeclarationConsumerIsWhatItsOwnGrantAllows(t *testing.T) {
has(t, perms.Publish, "$JS.ACK.NODES."+c.Name+".>") has(t, perms.Publish, "$JS.ACK.NODES."+c.Name+".>")
has(t, perms.Subscribe, c.Filters[0]) has(t, perms.Subscribe, c.Filters[0])
} }
// Every holder of a seat shares one worker and pulls from it (novox/hq ADR 0190): no queue group
// and no delivery subject, because a push consumer hands the next ask to whichever subscriber the
// server picks, busy or not; and no cap of one in flight, because pulling bounds the asks in flight
// by the holders that are free — which is what ended the race of issue 186, where asks delivered
// behind the one being worked expired and were dropped.
func TestAHoldersWorkerIsPulledByEveryHolder(t *testing.T) {
c, found := HolderConsumerFor("anchor", "build-agent", DeclaredSeat{Name: "node-build-agent", Accepts: []string{"build"}})
if !found {
t.Fatal("a seat that accepts work has no worker")
}
if c.Queue != "" || c.Push {
t.Fatalf("the worker is pushed (queue %q, push %v); a holder pulls when it is free", c.Queue, c.Push)
}
if c.MaxAckPending != 0 {
t.Fatalf("the worker caps asks in flight at %d; pulling bounds them by the holders working", c.MaxAckPending)
}
if c.Name != "SEAT_NODE_BUILD_AGENT_worker" || c.Stream != "SEAT_NODE_BUILD_AGENT" {
t.Fatalf("the worker is %s on %s; one per seat, shared by its holders", c.Name, c.Stream)
}
}
+13
View File
@@ -91,3 +91,16 @@ func TestADeclaredInvokeReachesTheComposedUser(t *testing.T) {
} }
has(t, perms.Publish, "mesh.mod.*.tool.>") has(t, perms.Publish, "mesh.mod.*.tool.>")
} }
// A module's tool is addressed two ways (novox/hq ADR 0159): to whichever instance answers, and to
// the instance on one machine. A grant for the tool covers both and nothing wider.
func TestInvokingAToolMayAddressTheMachineToo(t *testing.T) {
got, err := invokedSubjects([]string{"postgres.postgres_query"})
if err != nil {
t.Fatal(err)
}
want := []string{"mesh.mod.postgres.tool.postgres_query", "mesh.mod.postgres.tool.postgres_query.*"}
if len(got) != 2 || got[0] != want[0] || got[1] != want[1] {
t.Fatalf("the grant is %v, want %v", got, want)
}
}
+46
View File
@@ -144,6 +144,7 @@ func (j *JetStream) EnsureStream(s Stream) error {
MaxMsgsPerSubject: int64(s.MaxMsgsPerSubject), MaxMsgsPerSubject: int64(s.MaxMsgsPerSubject),
Description: s.Why, Description: s.Why,
} }
want.AllowDirect = s.Direct
if s.Retention == RetentionLastPerSubject { if s.Retention == RetentionLastPerSubject {
// Last-per-subject is a limits stream with one message kept per subject, not a // Last-per-subject is a limits stream with one message kept per subject, not a
// retention policy of its own — the state shape, spelled the way the server spells it. // retention policy of its own — the state shape, spelled the way the server spells it.
@@ -213,6 +214,51 @@ func (j *JetStream) EnsureConsumer(c Consumer) error {
switch have, err := j.js.ConsumerInfo(c.Stream, c.Name); { switch have, err := j.js.ConsumerInfo(c.Stream, c.Name); {
case err == nil: case err == nil:
// **The controller owns the worker's shape, type included** (novox/hq issue 206). A holder
// built for a pull worker cannot bind a push one — `cannot pull subscribe to push based
// consumer` — and on 2026-10-03 the build machine rolled before the controller that would
// have redefined its worker, restarted on that for an hour, and nothing could build the
// controller that would have ended it. The server cannot change a consumer's type in place,
// so one of the wrong type is re-made: on a work queue nothing is lost, because what was
// acknowledged is gone from the stream and what was not is delivered again from the start.
// On any other stream a re-made consumer would replay what this one acknowledged (issue
// 156), so there it is said and left, and the person re-makes it knowing the cost.
if havePush, wantPush := have.Config.DeliverSubject != "", want.DeliverSubject != ""; havePush != wantPush {
shape := func(push bool) string {
if push {
return "push"
}
return "pull"
}
info, err := j.js.StreamInfo(c.Stream)
if err != nil {
return fmt.Errorf("asking about stream %s to re-make consumer %s: %w", c.Stream, c.Name, err)
}
if info.Config.Retention != nats.WorkQueuePolicy {
// **A stream that keeps its history is re-made from now on, never from the start.**
// Left for a hand, the hand re-makes it with the server's default — everything the
// stream holds — which on 2026-10-03 replayed every build ask since 1 October and
// re-registered nine modules from the past (novox/hq issue 207). What this consumer
// had not yet acknowledged is lost with it, and said: on a history stream that is
// the smaller cost, and the asks in flight are visible to whoever asked.
j.note("consumer %s on %s changes from %s to %s delivery on a stream that keeps its history: "+
"re-made to deliver from now on, so nothing this one acknowledged comes back (novox/hq issue "+
"207); %d ask(s) it had not acknowledged are not carried over and must be asked again",
c.Name, c.Stream, shape(havePush), shape(wantPush), have.NumPending+uint64(have.NumAckPending))
want.DeliverPolicy = nats.DeliverNewPolicy
} else {
j.note("consumer %s on %s changes from %s to %s delivery: re-made where it left off, nothing "+
"acknowledged comes back and nothing pending is lost (novox/hq issue 206); a holder bound to "+
"the old shape binds again", c.Name, c.Stream, shape(havePush), shape(wantPush))
}
if err := j.js.DeleteConsumer(c.Stream, c.Name); err != nil {
return fmt.Errorf("re-making consumer %s on %s as %s: %w", c.Name, c.Stream, shape(wantPush), err)
}
if _, err := j.js.AddConsumer(c.Stream, want); err != nil {
return fmt.Errorf("re-making consumer %s on %s as %s: %w", c.Name, c.Stream, shape(wantPush), err)
}
return nil
}
// Where an existing consumer starts is its history, not something an assertion may move: // Where an existing consumer starts is its history, not something an assertion may move:
// the server refuses a changed deliver policy outright. Carried across, so asserting twice // the server refuses a changed deliver policy outright. Carried across, so asserting twice
// is the no-op a restart depends on. // is the no-op a restart depends on.
+143
View File
@@ -0,0 +1,143 @@
package broker
import (
"encoding/json"
"sort"
"strings"
)
// What the mesh issues an assignment to serve and to reach (novox/hq ADR 0160).
//
// A module's code names its tools and its events; **where they land is the mesh's to decide**, and
// it decided it twice — once in the runtime, once here, by one rule compiled into both. Now the
// controller composes a membership for every module on every machine and publishes it to a subject
// only that assignment reads; the runtime serves exactly what the membership says, and the account's
// grant is the same composition read the other way. The shape issued today is the shape the mesh
// already had, so nothing moves when a membership first arrives; only who decides it moves.
// Membership is one assignment's subjects: what this instance of a module on this machine serves,
// and what it may reach.
type Membership struct {
Node string `json:"node"`
Module string `json:"module"`
// Serves is every address a tool of this instance answers on. `{tool}` stands for the tool's
// own name, which the module knows and the mesh does not need to: the mesh issues the address,
// the runtime fills the name. An address with a queue is shared with the module's other
// instances, and the bus hands each call to one of them; an address without is this instance's.
Serves []Served `json:"serves"`
// Seats is every verb of a seat this instance holds, at the subject the seat's callers use.
Seats []SeatServed `json:"seats,omitempty"`
// Emits is where an event of this module lands; `{event}` stands for the event's name.
Emits string `json:"emits"`
// Reaches is each tool this module may call, `<module>.<tool>`, to the subjects that reach it:
// the first is whichever instance answers, when the mesh issued one; the rest name a machine.
Reaches map[string][]string `json:"reaches,omitempty"`
// Tools is where this instance answers what it serves — the runtime's one verb of its own.
Tools string `json:"tools"`
// Receives is what this assignment is given for each requirement it receives, by requirement:
// the contributions of every module that asked for it, as the catalogue composed them (novox/hq
// ADR 0167). The same list its received file is written from, so the two cannot disagree; a
// requirement nobody contributed to is an empty list, never absent. Kept as JSON because the
// catalogue owns the shape of a contribution and the bus only carries it.
Receives map[string]json.RawMessage `json:"receives,omitempty"`
// Mesh is every machine's address on the private network — what a rule saying "from the mesh"
// resolves to in the packet filter, issued here from the same list (novox/hq ADR 0167). A
// module that must tell the mesh from the world, the route proxy serving an internal name, reads
// it here rather than keeping a definition of its own.
Mesh []string `json:"mesh,omitempty"`
}
// Served is one address a tool is answered on.
type Served struct {
Subject string `json:"subject"`
Queue string `json:"queue,omitempty"`
}
// SeatServed is one verb of a held seat, where its callers ask.
type SeatServed struct {
Seat string `json:"seat"`
Verb string `json:"verb"`
Subject string `json:"subject"`
}
// MembershipSubject is the one address a runtime derives for itself: where its own membership is
// published, from the two names its credential carries. Everything else is in the membership.
func MembershipSubject(node, module string) string {
return "mesh.assignment." + node + "." + module
}
// Placements is where every module runs, for deciding which instance answers for the module.
type Placements struct {
// Nodes is each module's machines.
Nodes map[string][]string
// Interchangeable is each module whose definition says its instances are the same anywhere,
// so the module's plain subject is issued to all of them in one queue.
Interchangeable map[string]bool
}
// AnswersForTheModule says whether an instance of a module on one machine is issued the module's
// plain subject: when it is the only instance, or when the definition says instances are
// interchangeable. A stateful module on two machines gets only its machines' subjects, so a call
// that names none reaches nothing rather than the wrong store.
func (p Placements) AnswersForTheModule(module string) bool {
return len(p.Nodes[module]) <= 1 || p.Interchangeable[module]
}
// MembershipFor composes one assignment's membership from what it declared and where everything
// runs. The subjects are the ones PermissionsFor grants, derived here once more only until the
// grant itself is read from the membership — which is the next step, not this one.
func MembershipFor(node string, d Declared, where Placements) Membership {
own := "mesh.mod." + d.Module
m := Membership{
Node: node, Module: d.Module,
Emits: own + ".event.{event}",
Tools: own + ".tool.tools",
}
// This machine's address always; the module's when this instance answers for the module.
m.Serves = append(m.Serves, Served{Subject: own + ".tool.{tool}." + node})
if where.AnswersForTheModule(d.Module) {
m.Serves = append(m.Serves, Served{Subject: own + ".tool.{tool}", Queue: "serve." + d.Module})
}
for _, s := range d.Holds {
for _, verb := range s.Serves {
m.Seats = append(m.Seats, SeatServed{Seat: s.Name, Verb: verb, Subject: seatToolSubject(s, verb, node)})
}
}
if len(d.Invokes) > 0 {
m.Reaches = map[string][]string{}
for _, t := range d.Invokes {
if t == "*" || strings.HasPrefix(t, "seat:") {
continue // every tool, or a role's: addressed by name, not resolved per instance
}
module, tool, ok := strings.Cut(t, ".")
if !ok {
continue
}
var reach []string
if where.AnswersForTheModule(module) {
reach = append(reach, "mesh.mod."+module+".tool."+tool)
}
nodes := append([]string{}, where.Nodes[module]...)
sort.Strings(nodes)
for _, n := range nodes {
reach = append(reach, "mesh.mod."+module+".tool."+tool+"."+n)
}
m.Reaches[t] = reach
}
}
return m
}
// PlacementsOf reads where everything runs from the records the bus's accounts are composed from.
func PlacementsOf(r Records, interchangeable map[string]bool) Placements {
p := Placements{Nodes: map[string][]string{}, Interchangeable: interchangeable}
for node, declared := range r.Assigned {
for _, d := range declared {
p.Nodes[d.Module] = append(p.Nodes[d.Module], node)
}
}
for _, nodes := range p.Nodes {
sort.Strings(nodes)
}
return p
}
+109
View File
@@ -0,0 +1,109 @@
package broker
import (
"reflect"
"testing"
)
// The mesh issues an assignment's subjects (novox/hq ADR 0160): a module alone on one machine
// answers for the module and for its machine; a stateful module on two machines answers only for
// each machine; one that says its instances are interchangeable answers for the module everywhere;
// a holder serves its seat's verbs; and what a module may reach is resolved the same way.
func TestAMembershipIsIssuedFromWhereEverythingRuns(t *testing.T) {
records := Records{Assigned: map[string][]Declared{
"anchor": {
{Module: "postgres", Serves: []string{"query"}, Holds: []Seat{{Name: "mesh-store", Scope: "mesh", Serves: []string{"databases", "query"}}}},
{Module: "catalog", Invokes: []string{"postgres.query", "search.find"}},
},
"home-server": {
{Module: "postgres"},
{Module: "search"},
{Module: "dashboard", Invokes: []string{"postgres.query"}},
},
"laptop": {{Module: "search"}},
}, Interchangeable: map[string]bool{"search": true}}
where := PlacementsOf(records, records.Interchangeable)
pg := MembershipFor("anchor", records.Assigned["anchor"][0], where)
if !reflect.DeepEqual(pg.Serves, []Served{{Subject: "mesh.mod.postgres.tool.{tool}.anchor"}}) {
t.Fatalf("a stateful module on two machines answers only for its machine: %+v", pg.Serves)
}
if len(pg.Seats) != 2 || pg.Seats[0].Subject != "mesh.seat.mesh-store.tool.databases" {
t.Fatalf("the holder serves the seat's verbs at the seat's subjects: %+v", pg.Seats)
}
if pg.Emits != "mesh.mod.postgres.event.{event}" || pg.Tools != "mesh.mod.postgres.tool.tools" {
t.Fatalf("events and the tools verb: %+v", pg)
}
search := MembershipFor("laptop", records.Assigned["laptop"][0], where)
if !reflect.DeepEqual(search.Serves, []Served{
{Subject: "mesh.mod.search.tool.{tool}.laptop"},
{Subject: "mesh.mod.search.tool.{tool}", Queue: "serve.search"},
}) {
t.Fatalf("an interchangeable module answers for the module in the queue too: %+v", search.Serves)
}
dashboard := MembershipFor("home-server", records.Assigned["home-server"][2], where)
if !reflect.DeepEqual(dashboard.Serves, []Served{
{Subject: "mesh.mod.dashboard.tool.{tool}.home-server"},
{Subject: "mesh.mod.dashboard.tool.{tool}", Queue: "serve.dashboard"},
}) {
t.Fatalf("a module alone on one machine answers for the module: %+v", dashboard.Serves)
}
if !reflect.DeepEqual(dashboard.Reaches["postgres.query"],
[]string{"mesh.mod.postgres.tool.query.anchor", "mesh.mod.postgres.tool.query.home-server"}) {
t.Fatalf("reaching a stateful module names each machine and no plain subject: %v", dashboard.Reaches)
}
catalog := MembershipFor("anchor", records.Assigned["anchor"][1], where)
if !reflect.DeepEqual(catalog.Reaches["search.find"],
[]string{"mesh.mod.search.tool.find", "mesh.mod.search.tool.find.home-server", "mesh.mod.search.tool.find.laptop"}) {
t.Fatalf("reaching an interchangeable module offers the plain subject first: %v", catalog.Reaches)
}
if MembershipSubject("anchor", "postgres") != "mesh.assignment.anchor.postgres" {
t.Fatal("the one subject a runtime derives for itself")
}
}
func TestAnAccountMayReadItsOwnMembershipAndNoOthers(t *testing.T) {
perms, err := PermissionsFor(Principal{Kind: KindModule, Node: "anchor", Module: "postgres", PasswordHash: "x"})
if err != nil {
t.Fatal(err)
}
has(t, perms.Subscribe, "mesh.assignment.anchor.postgres")
has(t, perms.Publish, "$JS.API.DIRECT.GET.ASSIGNMENTS.mesh.assignment.anchor.postgres")
hasNot(t, perms.Subscribe, "mesh.assignment.>")
}
// The runtime arriving on a machine changes nothing about what each module is issued (to-be 38 WP2):
// the memberships are composed as before and the runtime reads several of them. What the machine's
// user list gains is one runtime principal, and loses nothing but the runtime module's own.
func TestTheRuntimeArrivingLeavesEveryMembershipAsItWas(t *testing.T) {
filter := Seat{Name: "node-packet-filter", Scope: "node", Serves: []string{"rules", "reload"}}
three := []Declared{
{Module: "nftables", Holds: []Seat{filter}, Serves: []string{"firewall_rules"}},
{Module: "zsh", Serves: []string{"execute"}},
{Module: "systemd", Serves: []string{"units"}},
}
before := Records{Nodes: []string{"anchor"}, Assigned: map[string][]Declared{"anchor": three}}
after := Records{Nodes: []string{"anchor"}, Assigned: map[string][]Declared{
"anchor": append(append([]Declared{}, three...), Declared{Module: RuntimeModule}),
}}
for _, d := range three {
was := MembershipFor("anchor", d, PlacementsOf(before, nil))
is := MembershipFor("anchor", d, PlacementsOf(after, nil))
if !reflect.DeepEqual(was, is) {
t.Errorf("%s's membership changed when the runtime arrived:\n%+v\n%+v", d.Module, was, is)
}
}
users, err := Users(after)
if err != nil {
t.Fatal(err)
}
kinds := map[Kind]int{}
for _, p := range users {
kinds[p.Kind]++
}
if kinds[KindNodeTools] != 1 || kinds[KindModule] != 3 || kinds[KindNode] != 1 || kinds[KindController] != 1 {
t.Errorf("the machine's users are %v; one runtime, the three modules, the host and the controller", kinds)
}
}
+177 -13
View File
@@ -34,8 +34,20 @@ const (
// authority is a list of tools and nothing else — not control, not declarations, not builds, // authority is a list of tools and nothing else — not control, not declarations, not builds,
// and no ability to answer anything, because a person asks. // and no ability to answer anything, because a person asks.
KindPerson Kind = "person" KindPerson Kind = "person"
// KindNodeTools is a machine's tool runtime (novox/hq ADR 0175, to-be 38): one process per
// node, on the host side, serving every assigned module's tools and every held seat's verbs.
// Its authority is the union of what the modules it carries would each have had for their
// tools — and nothing of what they consume, because tools are what it runs, not reactions.
KindNodeTools Kind = "node-tools"
) )
// RuntimeModule is the module that IS the node's tool runtime (novox/hq ADR 0175). Where it is
// assigned, the mesh composes one runtime principal for the machine in place of that module's own,
// and the per-module containers that served tools until then stop being the way tools reach a node.
// Mirrored in the catalogue package, which the agreement test holds to the same string; one
// constant, so a rename is one edit and the two packages cannot drift.
const RuntimeModule = "node-tools"
// Seat is a role on the bus as a principal relates to it: the subjects it accepts, and those it // Seat is a role on the bus as a principal relates to it: the subjects it accepts, and those it
// emits (novox/hq ADR 0118, design 29 §5). // emits (novox/hq ADR 0118, design 29 §5).
type Seat struct { type Seat struct {
@@ -74,6 +86,13 @@ type Principal struct {
// a namespace no such module owns. Every service started and the graph stayed empty. // a namespace no such module owns. Every service started and the graph stayed empty.
Watches []Seat Watches []Seat
// Carries are the modules whose tools this principal serves, for a KindNodeTools principal
// (novox/hq ADR 0175): every module assigned to its node, as each declares itself. Its
// serving authority is the union of theirs — each module's own tool namespace and each held
// seat's verbs on this node — derived from the same declarations the modules' own principals
// are, so the runtime can serve nothing a module could not have served for itself.
Carries []Declared
// Invokes are the tools this principal may call, as `<module>.<tool>`; a single `*` is every // Invokes are the tools this principal may call, as `<module>.<tool>`; a single `*` is every
// tool. A person's whole authority (design 25 §7), and a module's only if its manifest says so // tool. A person's whole authority (design 25 §7), and a module's only if its manifest says so
// (novox/hq ADR 0152) — the console's does, and nothing else's. // (novox/hq ADR 0152) — the console's does, and nothing else's.
@@ -91,10 +110,13 @@ type Principal struct {
PasswordHash string PasswordHash string
} }
// meshSeatsTheControllerUses are the roles the mesh's own flows submit work to. Named rather than // seatsTheControllerAsks are the roles the mesh's own flows submit work to. Named rather than
// derived from the seat set: the controller is not a module and declares no `uses`, so its side of a // derived from the seat set: the controller is not a module and declares no `uses`, so its side of a
// seat has to be stated, and a list is what makes "which roles does the mesh itself talk to" answerable. // seat has to be stated, and a list is what makes "which roles does the mesh itself talk to" answerable.
var meshSeatsTheControllerUses = []string{"mesh-build-machine"} // Both build roles while the handover runs (novox/hq ADR 0190): the controller asks whichever has a
// holder, and the retired one has one until build-agent replaces the builder. The second entry
// goes with the retired seat row.
var seatsTheControllerAsks = []string{"node-build-agent", "mesh-build-machine"}
// enrolmentPrefix is the space every enrolling node's user and inbox live under, so the one place the // enrolmentPrefix is the space every enrolling node's user and inbox live under, so the one place the
// controller may answer an enrolment is derived from the same constant the user is named from. // controller may answer an enrolment is derived from the same constant the user is named from.
@@ -112,7 +134,10 @@ func (p Principal) Username() string {
switch p.Kind { switch p.Kind {
case KindPerson: case KindPerson:
return "person." + p.Module return "person." + p.Module
case KindModule: case KindModule, KindNodeTools:
// The runtime is named exactly as the module it stands for would have been: the mesh
// issues its credential through the same path a module's takes (`module issue`), and
// that path knows the node and the module, not the kind.
return p.Node + "." + p.Module return p.Node + "." + p.Module
case KindNode: case KindNode:
return "node." + p.Node return "node." + p.Node
@@ -173,8 +198,10 @@ func PermissionsFor(p Principal) (Permissions, error) {
switch p.Kind { switch p.Kind {
case KindController: case KindController:
// The controller owns the mesh's own traffic and the streams. It is the only writer of // The controller owns the mesh's own traffic and the streams. It is the only writer of
// stream definitions (design 25 §3), so it alone reaches the JetStream API. // stream definitions (design 25 §3), so it alone reaches the JetStream API — and it alone
pub = []string{"mesh.control.>", "mesh.node.>", "$JS.API.>"} // issues memberships (novox/hq ADR 0160), which it publishes into the assignments stream
// after each push; refused by the server on 2026-10-01 until this line named them.
pub = []string{"mesh.control.>", "mesh.node.>", "mesh.assignment.>", "$JS.API.>"}
// **And where its consumers deliver.** A push consumer delivers on `_DELIVER.<its name>`, // **And where its consumers deliver.** A push consumer delivers on `_DELIVER.<its name>`,
// and a client bound to it subscribes exactly that; the server refused it for every // and a client bound to it subscribes exactly that; the server refused it for every
// principal the first time one bound a consumer (2026-09-28). Each kind below is granted // principal the first time one bound a consumer (2026-09-28). Each kind below is granted
@@ -184,7 +211,9 @@ func PermissionsFor(p Principal) (Permissions, error) {
// Work the mesh's own flows submit to a role, and the outcomes they wait on (ADR 0121). A // Work the mesh's own flows submit to a role, and the outcomes they wait on (ADR 0121). A
// build is the one today: the controller asks, and reads the answer from the seat's event // build is the one today: the controller asks, and reads the answer from the seat's event
// like the catalogue does — which is why no holder needs to publish into anybody's inbox. // like the catalogue does — which is why no holder needs to publish into anybody's inbox.
for _, seat := range meshSeatsTheControllerUses { // A node-scoped seat's work subject carries no node (novox/hq ADR 0190): the ask goes to
// the role, and whichever machine holding it is idle takes it.
for _, seat := range seatsTheControllerAsks {
pub = append(pub, "mesh.seat."+seat+".accept.>") pub = append(pub, "mesh.seat."+seat+".accept.>")
} }
// **And what the mesh says it did** (novox/hq ADR 0134). The control plane states its own // **And what the mesh says it did** (novox/hq ADR 0134). The control plane states its own
@@ -207,6 +236,8 @@ func PermissionsFor(p Principal) (Permissions, error) {
// which this package mirrors rather than reads, and a verb the seat does not declare is a // which this package mirrors rather than reads, and a verb the seat does not declare is a
// subject nothing publishes. // subject nothing publishes.
sub = append(sub, "mesh.seat."+ControllerSeat+".tool.>") sub = append(sub, "mesh.seat."+ControllerSeat+".tool.>")
// And says so (novox/hq ADR 0197): it answers discovery for the seat it serves.
sub = append(sub, announcing(ControllerSeat)...)
// The two events it reacts to, and its ack subject on the stream they arrive from // The two events it reacts to, and its ack subject on the stream they arrive from
// (streams.go). **Each named, not a pattern**: `mesh.mod.*.event.>` would make the // (streams.go). **Each named, not a pattern**: `mesh.mod.*.event.>` would make the
@@ -242,6 +273,9 @@ func PermissionsFor(p Principal) (Permissions, error) {
return Permissions{}, err return Permissions{}, err
} }
pub = append(pub, invoked...) pub = append(pub, invoked...)
// And may ask what answers (novox/hq ADR 0197): a question every service answers about
// itself, its replies to the asker's own inbox.
pub = append(pub, discovering()...)
case KindEnrolment: case KindEnrolment:
// A leaked token is useless for anything but enrolling: it cannot read a declaration, hear // A leaked token is useless for anything but enrolling: it cannot read a declaration, hear
@@ -298,6 +332,19 @@ func PermissionsFor(p Principal) (Permissions, error) {
// away — no other principal may subscribe this namespace, and a caller's authority is // away — no other principal may subscribe this namespace, and a caller's authority is
// still granted per tool, by name, on the publish side. // still granted per tool, by name, on the publish side.
sub = append(sub, own+".tool.>") sub = append(sub, own+".tool.>")
// It says what it serves (novox/hq ADR 0197): discovery for its own name and every seat it
// holds a verb of, answered by the runtime that serves them.
announced := []string{p.Module}
for _, s := range p.Holds {
if len(s.Serves) > 0 {
announced = append(announced, s.Name)
}
}
sub = append(sub, announcing(announced...)...)
// Its own membership (ADR 0160): the one subject a runtime derives for itself, read
// directly from the stream and followed live. Nothing else's.
sub = append(sub, MembershipSubject(p.Node, p.Module))
pub = append(pub, "$JS.API.DIRECT.GET."+AssignmentsStream+"."+MembershipSubject(p.Node, p.Module))
// 1b. The tools it calls, if its manifest says it calls any (novox/hq ADR 0152). The same // 1b. The tools it calls, if its manifest says it calls any (novox/hq ADR 0152). The same
// grant a person gets and derived the same way, so "what may this module ask" is // grant a person gets and derived the same way, so "what may this module ask" is
@@ -340,13 +387,17 @@ func PermissionsFor(p Principal) (Permissions, error) {
// 3. Seats it holds: full participation. // 3. Seats it holds: full participation.
for _, s := range p.Holds { for _, s := range p.Holds {
// Taking work from the role's queue: the worker consumer it binds (asked about, // Taking work from the role's queue: the worker consumer every holder shares (asked
// delivered on, acknowledged), each on the seat's own stream. The first machine to // about, pulled from, acknowledged), on the seat's own stream (novox/hq ADR 0190). A
// take work over the new bus was refused the asking (2026-09-28). // holder pulls — asks the consumer for its next message, answered on its own inbox —
// so what it needs is MSG.NEXT on that worker and nothing delivered to it. The first
// machine to take work over the new bus was refused the asking (2026-09-28).
worker := "SEAT_" + upperSnake(s.Name) + "_worker" worker := "SEAT_" + upperSnake(s.Name) + "_worker"
stream := seatStreamName(s.Name) stream := seatStreamName(s.Name)
sub = append(sub, "_DELIVER."+worker, "_DELIVER."+worker+".>") pub = append(pub,
pub = append(pub, "$JS.API.CONSUMER.INFO."+stream+"."+worker, "$JS.ACK."+stream+"."+worker+".>") "$JS.API.CONSUMER.INFO."+stream+"."+worker,
"$JS.API.CONSUMER.MSG.NEXT."+stream+"."+worker,
"$JS.ACK."+stream+"."+worker+".>")
for _, a := range s.Accepts { for _, a := range s.Accepts {
sub = append(sub, seatSubject(s, "accept", a)) sub = append(sub, seatSubject(s, "accept", a))
} }
@@ -369,6 +420,75 @@ func PermissionsFor(p Principal) (Permissions, error) {
pub = append(pub, seatToolSubject(s, t, "*")) pub = append(pub, seatToolSubject(s, t, "*"))
} }
} }
case KindNodeTools:
// **One process serves what every module on the machine would have served for itself**
// (novox/hq ADR 0175). Each carried module's whole tool namespace — the same grant that
// module's own principal has, for the same reason: the tools a module serves are what its
// code answers, and a list here would be a second copy of it. Each held seat's verbs on
// this node, as the holder's own principal would be granted them.
var serves []string
for _, d := range p.Carries {
if !safeSubject.MatchString(d.Module) {
return Permissions{}, fmt.Errorf(
"%q cannot be part of a subject: a permission is a subject pattern, and this would widen it", d.Module)
}
serves = append(serves, d.Module)
for _, s := range d.Holds {
serves = append(serves, s.Name)
}
own := "mesh.mod." + d.Module
sub = append(sub, own+".tool.>")
// A tool that emits an event is the module's code and emits under the module's name
// (ADR 0042); the runtime carrying that code may publish what the module declared it
// emits, and nothing it did not.
for _, e := range d.Emits {
pub = append(pub, own+".event."+e)
}
for _, s := range d.Holds {
for _, t := range s.Serves {
sub = append(sub, seatToolSubject(s, t, p.Node))
}
}
}
// Every assigned module's membership on this node (ADR 0160): one per module, read
// directly from the stream and followed live. This node's and no other's — the one token
// that varies is the module, so the pattern is the machine's own assignments.
sub = append(sub, "mesh.assignment."+p.Node+".*")
pub = append(pub, "$JS.API.DIRECT.GET."+AssignmentsStream+".mesh.assignment."+p.Node+".*")
// And every tool on the mesh (ADR 0175, decision 5): any node may call any tool on any
// node, as the console already could — the runtime is the console's serving mode.
invoked, err := invokedSubjects([]string{"*"})
if err != nil {
return Permissions{}, err
}
pub = append(pub, invoked...)
// It says what it serves and may ask what answers (novox/hq ADR 0197): the runtime answers
// discovery for each module and seat it carries, and the console it is asks the bus.
// One service per runtime process, named for the runtime: the bus lets a principal answer each
// request once, so the runtime announces everything it carries under its own name.
sub = append(sub, announcing(append([]string{RuntimeModule}, serves...)...)...)
pub = append(pub, discovering()...)
// **And it consumes for the modules it carries** (novox/hq ADR 0198, which changes ADR 0175's
// "it consumes nothing"): a module's long-running code is a bundle this runtime launches, and
// the runtime is its bus — it reads the module's own durable consumer and acknowledges what
// the module's code took. Exactly the grants the module's own principal has for that consumer,
// on its name and no other's: asking about it, pulling from it, acknowledging it. The
// consumer is still the controller's to make, from the module's own principal.
for _, d := range p.Carries {
own := Principal{Kind: KindModule, Node: p.Node, Module: d.Module, Emits: d.Emits,
Consumes: d.Consumes, Serves: d.Serves, Holds: d.Holds, Uses: d.Uses, Watches: d.Watches}
if _, consumes := ConsumerFor(own); !consumes {
continue
}
stream, durable := consumerStream(own), consumerDurable(own)
pub = append(pub,
"$JS.API.CONSUMER.INFO."+stream+"."+durable,
"$JS.API.CONSUMER.MSG.NEXT."+stream+"."+durable,
"$JS.ACK."+stream+"."+durable+".>")
}
sub = unique(sub)
pub = unique(pub)
} }
if p.Kind == KindPerson { if p.Kind == KindPerson {
@@ -376,6 +496,11 @@ func PermissionsFor(p Principal) (Permissions, error) {
// consumer, because nothing is delivered to a person — they ask and are answered. // consumer, because nothing is delivered to a person — they ask and are answered.
sub = append(sub, p.inbox()) sub = append(sub, p.inbox())
} }
if p.Kind == KindNodeTools {
// Its reply space, so the answers to what its tools call come back to it. No ack subject
// for the same reason a person has none: nothing is delivered to it.
sub = append(sub, p.inbox())
}
if p.Kind == KindModule || p.Kind == KindNode || p.Kind == KindController { if p.Kind == KindModule || p.Kind == KindNode || p.Kind == KindController {
// Its own reply space, and nothing wider. // Its own reply space, and nothing wider.
@@ -397,7 +522,7 @@ func PermissionsFor(p Principal) (Permissions, error) {
// A module answers what it was asked — a tool call reaches it on its own namespace, so the // A module answers what it was asked — a tool call reaches it on its own namespace, so the
// authority is bounded by having been asked — and so does the controller. A node and a // authority is bounded by having been asked — and so does the controller. A node and a
// person are never asked anything, and are granted nothing here. // person are never asked anything, and are granted nothing here.
AllowResponses: p.Kind == KindModule || p.Kind == KindController, AllowResponses: p.Kind == KindModule || p.Kind == KindController || p.Kind == KindNodeTools,
}, nil }, nil
} }
@@ -619,6 +744,20 @@ func ComposeAccounts(principals []Principal) (string, error) {
return b.String(), nil return b.String(), nil
} }
// unique is a sorted list with each subject once. Two carried modules holding seats with the same
// verb, or the runtime module itself carried beside the others, would otherwise write a grant twice
// — harmless to the server, and noise in a file that is read as the mesh's authority model.
func unique(values []string) []string {
sort.Strings(values)
out := values[:0]
for i, v := range values {
if i == 0 || v != values[i-1] {
out = append(out, v)
}
}
return out
}
func quoted(values []string) string { func quoted(values []string) string {
if len(values) == 0 { if len(values) == 0 {
return "" return ""
@@ -660,7 +799,32 @@ func invokedSubjects(invokes []string) ([]string, error) {
return nil, fmt.Errorf( return nil, fmt.Errorf(
"%q does not name a tool: one invokes <module>.<tool>, seat:<seat>.<verb>, or * for every one", t) "%q does not name a tool: one invokes <module>.<tool>, seat:<seat>.<verb>, or * for every one", t)
} }
out = append(out, "mesh.mod."+module+".tool."+tool) // Both ways a module's tool is addressed (novox/hq ADR 0159): to whichever instance
// answers, and to the instance on one machine, which is the same subject with the machine
// as its last token.
out = append(out, "mesh.mod."+module+".tool."+tool, "mesh.mod."+module+".tool."+tool+".*")
} }
return out, nil return out, nil
} }
// announcing is what a principal that serves tools subscribes to answer the NATS services
// protocol's discovery (novox/hq ADR 0197): the questions asked of every service, and those asked of
// each name it serves — its own and no other's, so it cannot answer for a service it is not.
func announcing(names ...string) []string {
out := []string{"$SRV.PING", "$SRV.INFO", "$SRV.STATS"}
for _, n := range names {
if !safeSubject.MatchString(n) {
continue
}
for _, verb := range []string{"PING", "INFO", "STATS"} {
out = append(out, "$SRV."+verb+"."+n, "$SRV."+verb+"."+n+".>")
}
}
return out
}
// discovering is what a principal publishes to ask what answers (novox/hq ADR 0197): the services
// protocol's discovery requests, whose replies come to its own inbox.
func discovering() []string {
return []string{"$SRV.PING", "$SRV.PING.>", "$SRV.INFO", "$SRV.INFO.>"}
}
+121 -1
View File
@@ -71,6 +71,18 @@ func TestHoldingASeatIsTheMirrorOfUsingIt(t *testing.T) {
hasNot(t, perms.Publish, "mesh.seat.telegram-sender.accept.send") hasNot(t, perms.Publish, "mesh.seat.telegram-sender.accept.send")
} }
// A build machine may say everything about a build as it happens (novox/hq ADR 0157): that it
// started, and every line under the build's own id — the seat's `log.*` becomes a publish over
// one token, so a reader follows one build by subject and the holder can name no other subject.
func TestTheBuildMachineMaySayWhatItDoesUnderTheBuildsId(t *testing.T) {
seat := Seat{Name: "mesh-build-machine", Accepts: []string{"build"}, Emits: []string{"started", "built", "log.*"}}
perms, _ := PermissionsFor(Principal{Kind: KindModule, Node: "anchor", Module: "builder",
Holds: []Seat{seat}, PasswordHash: "x"})
has(t, perms.Publish, "mesh.seat.mesh-build-machine.event.started")
has(t, perms.Publish, "mesh.seat.mesh-build-machine.event.log.*")
hasNot(t, perms.Publish, "mesh.seat.mesh-build-machine.event.>")
}
// Without an ack permission a durable consumer never really consumes: every message it receives is // Without an ack permission a durable consumer never really consumes: every message it receives is
// redelivered forever, refused by the permission list it already has (design 25 §4). // redelivered forever, refused by the permission list it already has (design 25 §4).
func TestAModuleMayAckItsOwnDeliveriesAndNoOthers(t *testing.T) { func TestAModuleMayAckItsOwnDeliveriesAndNoOthers(t *testing.T) {
@@ -221,7 +233,9 @@ func TestAPersonReachesNothingButTools(t *testing.T) {
perms, _ := PermissionsFor(Principal{Kind: KindPerson, Module: "jo", perms, _ := PermissionsFor(Principal{Kind: KindPerson, Module: "jo",
Invokes: []string{"*"}, PasswordHash: "x"}) Invokes: []string{"*"}, PasswordHash: "x"})
for _, p := range perms.Publish { for _, p := range perms.Publish {
if !strings.Contains(p, ".tool.") { // A tool call, or asking what answers (novox/hq ADR 0197) — a question every service
// answers about itself, which claims nothing and controls nothing.
if !strings.Contains(p, ".tool.") && !strings.HasPrefix(p, "$SRV.") {
t.Errorf("a person may publish %q, which is not a tool call", p) t.Errorf("a person may publish %q, which is not a tool call", p)
} }
} }
@@ -359,3 +373,109 @@ func TestAModulePullsItsOwnConsumerAndNoOthers(t *testing.T) {
} }
} }
} }
// The runtime's authority is the union of what the modules it carries would have been granted for
// their tools (novox/hq ADR 0175): every carried module's tool namespace, every held seat's verbs
// on this node, every module's membership on this node, and a call to anything. Nothing it
// consumes, because it reacts to nothing.
func TestTheRuntimeServesTheUnionAndConsumesForItsModules(t *testing.T) {
filter := Seat{Name: "node-packet-filter", Scope: "node", Serves: []string{"rules", "reload"}}
p := Principal{Kind: KindNodeTools, Node: "anchor", Module: RuntimeModule, Carries: []Declared{
{Module: "nftables", Holds: []Seat{filter}, Serves: []string{"firewall_rules"}},
{Module: "zsh", Emits: []string{"shell.opened"}, Consumes: []string{"shop.order.placed"}},
{Module: RuntimeModule},
}}
perms, err := PermissionsFor(p)
if err != nil {
t.Fatal(err)
}
for _, want := range []string{
"mesh.mod.nftables.tool.>", "mesh.mod.zsh.tool.>", "mesh.mod." + RuntimeModule + ".tool.>",
"mesh.seat.node-packet-filter.tool.rules.anchor", "mesh.seat.node-packet-filter.tool.reload.anchor",
"mesh.assignment.anchor.*",
"_INBOX.anchor." + RuntimeModule + ".>",
} {
if !contains(perms.Subscribe, want) {
t.Errorf("the runtime may not subscribe %s: %v", want, perms.Subscribe)
}
}
for _, want := range []string{
"mesh.mod.*.tool.>", "mesh.seat.*.tool.>",
"$JS.API.DIRECT.GET.ASSIGNMENTS.mesh.assignment.anchor.*",
"mesh.mod.zsh.event.shell.opened",
} {
if !contains(perms.Publish, want) {
t.Errorf("the runtime may not publish %s: %v", want, perms.Publish)
}
}
// It reads the consumer of every carried module that consumes — that module's, by its name, as
// the module's own principal could (novox/hq ADR 0198) — and of no module that consumes nothing.
for _, want := range []string{
"$JS.API.CONSUMER.INFO.EVENTS.anchor_zsh",
"$JS.API.CONSUMER.MSG.NEXT.EVENTS.anchor_zsh",
"$JS.ACK.EVENTS.anchor_zsh.>",
} {
if !contains(perms.Publish, want) {
t.Errorf("the runtime may not read zsh's consumer: %s missing from %v", want, perms.Publish)
}
}
for _, s := range perms.Publish {
if (strings.HasPrefix(s, "$JS.ACK.") || strings.Contains(s, "CONSUMER")) && !strings.Contains(s, "anchor_zsh") {
t.Errorf("the runtime was granted a consumer no carried module of it consumes on: %s", s)
}
}
// It pulls; nothing is pushed to it, and it subscribes no event subject directly.
for _, s := range perms.Subscribe {
if strings.Contains(s, ".event.") || strings.HasPrefix(s, "_DELIVER.") {
t.Errorf("the runtime was granted a delivery: %s", s)
}
}
if !perms.AllowResponses {
t.Error("the runtime answers what it is asked, and may not reply")
}
if _, needed := ConsumerFor(p); needed {
t.Error("a consumer would be made for the runtime itself; it reads its modules' consumers, never one of its own")
}
// Each subject once in each list: the file is read as the mesh's authority model. One subject may
// stand in both — the runtime answers discovery on `$SRV.INFO` and, as the console, asks it
// (novox/hq ADR 0197) — because subscribing and publishing are two different grants.
for _, list := range [][]string{perms.Subscribe, perms.Publish} {
seen := map[string]bool{}
for _, s := range list {
if seen[s] {
t.Errorf("%s is granted twice", s)
}
seen[s] = true
}
}
}
func contains(list []string, want string) bool {
for _, s := range list {
if s == want {
return true
}
}
return false
}
// A node-scoped seat's work is shared (novox/hq ADR 0190): its holder on any machine subscribes the
// seat's one work subject, with no node in it, so holders on several machines read one queue. The
// node token belongs to a seat's tools, which are asked of one machine (design 33 §4), not to its work.
func TestANodeSeatsWorkSubjectCarriesNoNode(t *testing.T) {
seat := Seat{Name: "node-build-agent", Scope: "node", Accepts: []string{"build"}, Serves: []string{"status"}}
perms, err := PermissionsFor(Principal{Kind: KindModule, Node: "anchor", Module: "build-agent", Holds: []Seat{seat}})
if err != nil {
t.Fatal(err)
}
has(t, perms.Subscribe, "mesh.seat.node-build-agent.accept.build")
hasNot(t, perms.Subscribe, "mesh.seat.node-build-agent.accept.build.anchor")
// And its tools still carry the machine.
has(t, perms.Subscribe, "mesh.seat.node-build-agent.tool.status.anchor")
// The controller asks the role, not a machine.
controller, err := PermissionsFor(Principal{Kind: KindController})
if err != nil {
t.Fatal(err)
}
has(t, controller.Publish, "mesh.seat.node-build-agent.accept.>")
}
+24 -2
View File
@@ -47,8 +47,14 @@ type Stream struct {
// Why is carried into the assertion so an operator reading the server's own state finds the // Why is carried into the assertion so an operator reading the server's own state finds the
// reason there, rather than only in a repository they may not have. // reason there, rather than only in a repository they may not have.
Why string Why string
// Direct lets a client read a subject's last message without a consumer, which is how a
// runtime reads its own membership with no JetStream API beyond one request (ADR 0160).
Direct bool
} }
// AssignmentsStream holds every assignment's membership, the newest per subject.
const AssignmentsStream = "ASSIGNMENTS"
// MeshStreams is the foundation set, in the order a person reads it. // MeshStreams is the foundation set, in the order a person reads it.
// //
// **CONTROL names its subjects rather than taking `mesh.control.>`**, because heartbeats live // **CONTROL names its subjects rather than taking `mesh.control.>`**, because heartbeats live
@@ -79,7 +85,15 @@ func MeshStreams() []Stream {
"n-1 by construction (issue 107)", "n-1 by construction (issue 107)",
}, },
{ {
Name: "EVENTS", Name: AssignmentsStream,
Subjects: []string{"mesh.assignment.*.*"},
Retention: RetentionLastPerSubject,
Direct: true,
Why: "one membership per assignment, always the newest: what the mesh issued this module " +
"on this machine to serve and to reach (ADR 0160); read directly by the runtime it is for",
},
{
Name: EventsStream,
// A seat's own events ride here too: they are 1:many like any event, and the // A seat's own events ride here too: they are 1:many like any event, and the
// `event` token keeps them clear of both the seat's work queue (`accept`) and its // `event` token keeps them clear of both the seat's work queue (`accept`) and its
// tools (`tool`), which must not be persisted. // tools (`tool`), which must not be persisted.
@@ -203,10 +217,15 @@ var ControllerFollows = []string{
// A build's outcome, which is the build-machine role's own event now (ADR 0121) rather than a // A build's outcome, which is the build-machine role's own event now (ADR 0121) rather than a
// message on the control branch. Same three audiences, one publish: whoever asked, this, and the // message on the control branch. Same three audiences, one publish: whoever asked, this, and the
// catalogue. // catalogue.
seatEventSubject("mesh-build-machine", "built"), seatEventSubject("node-build-agent", "built"),
// The forge's merges: what moved a source, so the mesh builds what that source produces // The forge's merges: what moved a source, so the mesh builds what that source produces
// without anybody telling it (novox/hq 04-ISSUES/131). Appended, because the index is a name. // without anybody telling it (novox/hq 04-ISSUES/131). Appended, because the index is a name.
moduleEventSubject("gitea", "pull.merged"), moduleEventSubject("gitea", "pull.merged"),
// The retired build role's outcome too, while the handover runs (novox/hq ADR 0190): the one
// build machine keeps answering on its seat until build-agent replaces it, and the outcome that
// registers build-agent itself comes from there. Appended, for the same reason as above; goes
// with the retired seat row.
seatEventSubject("mesh-build-machine", "built"),
} }
// moduleEventSubject is where one module's event lands. The same derivation PermissionsFor uses, so // moduleEventSubject is where one module's event lands. The same derivation PermissionsFor uses, so
@@ -220,6 +239,9 @@ func seatEventSubject(seat, verb string) string {
return "mesh.seat." + seat + ".event." + verb return "mesh.seat." + seat + ".event." + verb
} }
// EventsStream holds every module's and every role's events, a build's log among them.
const EventsStream = "EVENTS"
// MeshConsumers is what the controller consumes, in the order a person reads it. // MeshConsumers is what the controller consumes, in the order a person reads it.
// //
// **Unlimited redelivery on CONTROL, deliberately.** The store window's bound is the controller's, // **Unlimited redelivery on CONTROL, deliberately.** The store window's bound is the controller's,
+1
View File
@@ -126,6 +126,7 @@ func TestEachStreamCarriesTheRetentionItsShapeNeeds(t *testing.T) {
"CONTROL": RetentionWorkQueue, "CONTROL": RetentionWorkQueue,
"NODES": RetentionLastPerSubject, "NODES": RetentionLastPerSubject,
"EVENTS": RetentionLimits, "EVENTS": RetentionLimits,
"ASSIGNMENTS": RetentionLastPerSubject,
} }
got := map[string]Retention{} got := map[string]Retention{}
for _, s := range MeshStreams() { for _, s := range MeshStreams() {
+8 -8
View File
@@ -24,8 +24,8 @@ accounts {
jetstream: enabled jetstream: enabled
users = [ users = [
{ user: "controller", password: "$2a$11$cccccccccccccccccccccc", permissions: { { user: "controller", password: "$2a$11$cccccccccccccccccccccc", permissions: {
publish: { allow: ["$JS.ACK.CONTROL.controller.>", "$JS.ACK.EVENTS.controller.>", "$JS.API.>", "_INBOX.enrol.>", "mesh.control.>", "mesh.mod.*.tool.>", "mesh.node.>", "mesh.seat.mesh-build-machine.accept.>", "mesh.seat.mesh-controller.event.applied", "mesh.seat.mesh-controller.event.built-before", "mesh.seat.mesh-controller.event.refused"] } publish: { allow: ["$JS.ACK.CONTROL.controller.>", "$JS.ACK.EVENTS.controller.>", "$JS.API.>", "_INBOX.enrol.>", "mesh.assignment.>", "mesh.control.>", "mesh.mod.*.tool.>", "mesh.node.>", "mesh.seat.mesh-build-machine.accept.>", "mesh.seat.mesh-controller.event.applied", "mesh.seat.mesh-controller.event.built-before", "mesh.seat.mesh-controller.event.refused", "mesh.seat.node-build-agent.accept.>"] }
subscribe: { allow: ["$JS.API.>", "_DELIVER.controller", "_DELIVER.controller.>", "_INBOX.controller.>", "mesh.control.>", "mesh.mod.gitea.event.pull.merged", "mesh.mod.mesh-catalog.event.catching-up", "mesh.mod.mesh-catalog.event.upgraded", "mesh.seat.mesh-build-machine.event.built", "mesh.seat.mesh-controller.tool.>"] } subscribe: { allow: ["$JS.API.>", "$SRV.INFO", "$SRV.INFO.mesh-controller", "$SRV.INFO.mesh-controller.>", "$SRV.PING", "$SRV.PING.mesh-controller", "$SRV.PING.mesh-controller.>", "$SRV.STATS", "$SRV.STATS.mesh-controller", "$SRV.STATS.mesh-controller.>", "_DELIVER.controller", "_DELIVER.controller.>", "_INBOX.controller.>", "mesh.control.>", "mesh.mod.gitea.event.pull.merged", "mesh.mod.mesh-catalog.event.catching-up", "mesh.mod.mesh-catalog.event.upgraded", "mesh.seat.mesh-build-machine.event.built", "mesh.seat.mesh-controller.tool.>", "mesh.seat.node-build-agent.event.built"] }
allow_responses: { max: 1, ttl: "1m" } allow_responses: { max: 1, ttl: "1m" }
} } } }
{ user: "enrol.one", password: "$2a$11$eeeeeeeeeeeeeeeeeeeeee", permissions: { { user: "enrol.one", password: "$2a$11$eeeeeeeeeeeeeeeeeeeeee", permissions: {
@@ -37,18 +37,18 @@ accounts {
subscribe: { allow: ["_DELIVER.one", "_DELIVER.one.>", "_INBOX.node.one.>", "mesh.node.one.declare"] } subscribe: { allow: ["_DELIVER.one", "_DELIVER.one.>", "_INBOX.node.one.>", "mesh.node.one.declare"] }
} } } }
{ user: "one.telegram", password: "$2a$11$tttttttttttttttttttttt", permissions: { { user: "one.telegram", password: "$2a$11$tttttttttttttttttttttt", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.one_telegram.>", "$JS.ACK.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker.>", "$JS.API.CONSUMER.INFO.EVENTS.one_telegram", "$JS.API.CONSUMER.INFO.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.one_telegram", "mesh.seat.telegram-sender.event.delivered", "mesh.seat.telegram-sender.event.failed"] } publish: { allow: ["$JS.ACK.EVENTS.one_telegram.>", "$JS.ACK.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker.>", "$JS.API.CONSUMER.INFO.EVENTS.one_telegram", "$JS.API.CONSUMER.INFO.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.one_telegram", "$JS.API.CONSUMER.MSG.NEXT.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker", "$JS.API.DIRECT.GET.ASSIGNMENTS.mesh.assignment.one.telegram", "mesh.seat.telegram-sender.event.delivered", "mesh.seat.telegram-sender.event.failed"] }
subscribe: { allow: ["_DELIVER.SEAT_TELEGRAM_SENDER_worker", "_DELIVER.SEAT_TELEGRAM_SENDER_worker.>", "_INBOX.one.telegram.>", "mesh.mod.telegram.tool.>", "mesh.seat.telegram-sender.accept.send"] } subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.telegram", "$SRV.INFO.telegram.>", "$SRV.PING", "$SRV.PING.telegram", "$SRV.PING.telegram.>", "$SRV.STATS", "$SRV.STATS.telegram", "$SRV.STATS.telegram.>", "_INBOX.one.telegram.>", "mesh.assignment.one.telegram", "mesh.mod.telegram.tool.>", "mesh.seat.telegram-sender.accept.send"] }
allow_responses: { max: 1, ttl: "1m" } allow_responses: { max: 1, ttl: "1m" }
} } } }
{ user: "two.audit", password: "$2a$11$aaaaaaaaaaaaaaaaaaaaaa", permissions: { { user: "two.audit", password: "$2a$11$aaaaaaaaaaaaaaaaaaaaaa", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.two_audit.>", "$JS.API.CONSUMER.INFO.EVENTS.two_audit", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_audit"] } publish: { allow: ["$JS.ACK.EVENTS.two_audit.>", "$JS.API.CONSUMER.INFO.EVENTS.two_audit", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_audit", "$JS.API.DIRECT.GET.ASSIGNMENTS.mesh.assignment.two.audit"] }
subscribe: { allow: ["_INBOX.two.audit.>", "mesh.mod.audit.tool.>", "mesh.mod.shop.event.order.placed"] } subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.audit", "$SRV.INFO.audit.>", "$SRV.PING", "$SRV.PING.audit", "$SRV.PING.audit.>", "$SRV.STATS", "$SRV.STATS.audit", "$SRV.STATS.audit.>", "_INBOX.two.audit.>", "mesh.assignment.two.audit", "mesh.mod.audit.tool.>", "mesh.mod.shop.event.order.placed"] }
allow_responses: { max: 1, ttl: "1m" } allow_responses: { max: 1, ttl: "1m" }
} } } }
{ user: "two.shop", password: "$2a$11$ssssssssssssssssssssss", permissions: { { user: "two.shop", password: "$2a$11$ssssssssssssssssssssss", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.two_shop.>", "$JS.API.CONSUMER.INFO.EVENTS.two_shop", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_shop", "mesh.mod.shop.event.order.placed", "mesh.seat.telegram-sender.accept.send"] } publish: { allow: ["$JS.ACK.EVENTS.two_shop.>", "$JS.API.CONSUMER.INFO.EVENTS.two_shop", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_shop", "$JS.API.DIRECT.GET.ASSIGNMENTS.mesh.assignment.two.shop", "mesh.mod.shop.event.order.placed", "mesh.seat.telegram-sender.accept.send"] }
subscribe: { allow: ["_INBOX.two.shop.>", "mesh.mod.shop.tool.>"] } subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.shop", "$SRV.INFO.shop.>", "$SRV.PING", "$SRV.PING.shop", "$SRV.PING.shop.>", "$SRV.STATS", "$SRV.STATS.shop", "$SRV.STATS.shop.>", "_INBOX.two.shop.>", "mesh.assignment.two.shop", "mesh.mod.shop.tool.>"] }
allow_responses: { max: 1, ttl: "1m" } allow_responses: { max: 1, ttl: "1m" }
} } } }
] ]
+23
View File
@@ -47,6 +47,9 @@ type Records struct {
Enrolling []string Enrolling []string
// People is each person's name against the tools they may invoke, `*` for an administrator. // People is each person's name against the tools they may invoke, `*` for an administrator.
People map[string][]string People map[string][]string
// Interchangeable is each module whose definition says its instances are the same anywhere
// (ADR 0160), which decides whether the module's plain subject is issued to every instance.
Interchangeable map[string]bool
} }
// Users is every user the composed file should contain, in the order it will be written. // Users is every user the composed file should contain, in the order it will be written.
@@ -59,13 +62,33 @@ func Users(r Records) ([]Principal, error) {
for _, node := range sortedCopy(r.Nodes) { for _, node := range sortedCopy(r.Nodes) {
out = append(out, Principal{Kind: KindNode, Node: node}) out = append(out, Principal{Kind: KindNode, Node: node})
// **Where the runtime is assigned, the machine gets one runtime principal in place of the
// runtime module's own** (novox/hq ADR 0175, to-be 38). It carries every module on the
// node: its serving grants are the union of theirs. Every other module keeps its own
// principal — a module still serving tools from its own container holds its own
// credential until it moves, and the two serve side by side in the meantime.
runtimeHere := false
for _, d := range r.Assigned[node] { for _, d := range r.Assigned[node] {
if d.Module == RuntimeModule {
runtimeHere = true
}
}
for _, d := range r.Assigned[node] {
if runtimeHere && d.Module == RuntimeModule {
continue
}
out = append(out, Principal{ out = append(out, Principal{
Kind: KindModule, Node: node, Module: d.Module, Kind: KindModule, Node: node, Module: d.Module,
Emits: d.Emits, Consumes: d.Consumes, Serves: d.Serves, Emits: d.Emits, Consumes: d.Consumes, Serves: d.Serves,
Holds: d.Holds, Uses: d.Uses, Watches: d.Watches, Invokes: d.Invokes, Holds: d.Holds, Uses: d.Uses, Watches: d.Watches, Invokes: d.Invokes,
}) })
} }
if runtimeHere {
out = append(out, Principal{
Kind: KindNodeTools, Node: node, Module: RuntimeModule,
Carries: append([]Declared(nil), r.Assigned[node]...),
})
}
} }
for _, node := range sortedCopy(r.Enrolling) { for _, node := range sortedCopy(r.Enrolling) {
out = append(out, Principal{Kind: KindEnrolment, Node: node}) out = append(out, Principal{Kind: KindEnrolment, Node: node})
+51
View File
@@ -245,3 +245,54 @@ func TestAUserListIsComposedBeforeAnythingMovesOntoTheBus(t *testing.T) {
t.Errorf("the composed list does not contain the machine running the bus") t.Errorf("the composed list does not contain the machine running the bus")
} }
} }
// Where the runtime module is assigned, the machine gets one runtime principal in place of the
// runtime module's own (novox/hq ADR 0175, to-be 38). Every other module keeps its own: a module
// still serving tools from its own container holds its own credential until it moves.
func TestTheRuntimeModuleBecomesTheMachinesRuntimePrincipal(t *testing.T) {
r := someRecords()
r.Assigned["one"] = append(r.Assigned["one"], Declared{Module: RuntimeModule})
users, err := Users(r)
if err != nil {
t.Fatal(err)
}
var runtime *Principal
for i := range users {
p := &users[i]
if p.Node == "one" && p.Module == RuntimeModule {
if p.Kind == KindModule {
t.Fatalf("%s on one was composed as an ordinary module beside the runtime", RuntimeModule)
}
runtime = p
}
}
if runtime == nil || runtime.Kind != KindNodeTools {
t.Fatalf("one runs %s and got no runtime principal: %v", RuntimeModule, namesOf(t, r))
}
if runtime.Username() != "one."+RuntimeModule {
t.Errorf("the runtime is named %q; `module issue` names it as the module it stands for", runtime.Username())
}
carried := map[string]bool{}
for _, d := range runtime.Carries {
carried[d.Module] = true
}
if !carried["telegram"] || !carried[RuntimeModule] {
t.Errorf("the runtime carries %v; it carries every module on its node", carried)
}
// And the other node, where the runtime is not assigned, is exactly as before.
for _, p := range users {
if p.Node == "two" && p.Kind == KindNodeTools {
t.Fatal("two runs no runtime and was given a runtime principal")
}
}
// A module serving its own tools beside the runtime keeps its own principal.
found := false
for _, p := range users {
if p.Kind == KindModule && p.Node == "one" && p.Module == "telegram" {
found = true
}
}
if !found {
t.Error("telegram lost its own principal when the runtime arrived on its node")
}
}
+167
View File
@@ -0,0 +1,167 @@
package broker
import (
"os"
"testing"
"time"
"github.com/nats-io/nats.go"
)
// A seat's worker that changed from push to pull delivery strands a holder built for the new shape
// (novox/hq issue 206): the server refuses a pull subscription on a push consumer, and the controller
// that would redefine it was the build that nobody could take. The controller owns the worker's
// shape, type included: on a work queue it re-makes one of the wrong type, losing nothing, and a
// pull subscription then binds and takes what was pending.
//
// docker run -d --rm --name t -p 14231:4222 nats:2.10-alpine -js
// MESH_TEST_NATS=nats://127.0.0.1:14231 go test ./internal/broker/ -run TestAWorker
func TestAWorkerOfTheWrongTypeIsRemadeOnAWorkQueueAndAPullThenBinds(t *testing.T) {
url := os.Getenv("MESH_TEST_NATS")
if url == "" {
t.Skip("MESH_TEST_NATS unset")
}
js, err := Dial(url)
if err != nil {
t.Fatal(err)
}
defer js.Close()
const stream, worker, filter = "SEAT_T_SHELF", "SEAT_T_SHELF_worker", "mesh.seat.t-shelf.accept.>"
_ = js.js.DeleteStream(stream)
if _, err := js.js.AddStream(&nats.StreamConfig{
Name: stream, Subjects: []string{filter}, Retention: nats.WorkQueuePolicy, Storage: nats.MemoryStorage,
}); err != nil {
t.Fatal(err)
}
defer func() { _ = js.js.DeleteStream(stream) }()
// The worker as the previous controller defined it: push, in a queue group.
if _, err := js.js.AddConsumer(stream, &nats.ConsumerConfig{
Durable: worker, AckPolicy: nats.AckExplicitPolicy, AckWait: 60 * time.Second, MaxDeliver: 5,
FilterSubject: filter, DeliverSubject: "_DELIVER." + worker, DeliverGroup: "holders",
}); err != nil {
t.Fatal(err)
}
for _, body := range []string{"one", "two", "three"} {
if _, err := js.js.Publish("mesh.seat.t-shelf.accept.build", []byte(body)); err != nil {
t.Fatal(err)
}
}
// The old holder took and acknowledged the first ask, then went away.
old, err := js.js.QueueSubscribeSync(filter, "holders", nats.Bind(stream, worker))
if err != nil {
t.Fatal(err)
}
m, err := old.NextMsg(twoSeconds)
if err != nil {
t.Fatal(err)
}
if string(m.Data) != "one" {
t.Fatalf("the first ask is %q", m.Data)
}
if err := m.AckSync(); err != nil {
t.Fatal(err)
}
if err := old.Unsubscribe(); err != nil {
t.Fatal(err)
}
// The new controller asserts the worker as the mesh derives it now: pull.
if err := js.EnsureConsumer(Consumer{
Name: worker, Stream: stream, Filters: []string{filter}, AckWaitSeconds: 60, MaxDeliver: 5,
Why: "the test's worker",
}); err != nil {
t.Fatal(err)
}
have, err := js.js.ConsumerInfo(stream, worker)
if err != nil {
t.Fatal(err)
}
if have.Config.DeliverSubject != "" || have.Config.DeliverGroup != "" {
t.Fatalf("the worker is still push: %+v", have.Config)
}
// A holder built for the new shape binds, and takes exactly what the old one left.
sub, err := js.js.PullSubscribe(filter, worker, nats.Bind(stream, worker), nats.ManualAck())
if err != nil {
t.Fatalf("a pull subscription does not bind the re-made worker: %v", err)
}
got, err := sub.Fetch(3, nats.MaxWait(twoSeconds))
if err != nil && len(got) == 0 {
t.Fatalf("nothing pending was delivered: %v", err)
}
var bodies []string
for _, g := range got {
bodies = append(bodies, string(g.Data))
_ = g.Ack()
}
if len(bodies) != 2 || bodies[0] != "two" || bodies[1] != "three" {
t.Fatalf("the pending asks after the acknowledged one, in order: %v", bodies)
}
// Asserted again, the pull worker is the no-op a restart depends on.
if err := js.EnsureConsumer(Consumer{
Name: worker, Stream: stream, Filters: []string{filter}, AckWaitSeconds: 60, MaxDeliver: 5,
}); err != nil {
t.Fatal(err)
}
}
// On a stream that keeps its history, a worker of the wrong type is re-made to deliver from now on:
// re-making it from the start would replay what it acknowledged (novox/hq issue 156), and leaving it
// for a hand re-made it exactly that way on 2026-10-03 (issue 207).
func TestAWorkerOfTheWrongTypeOnAHistoryStreamIsRemadeFromNowOn(t *testing.T) {
url := os.Getenv("MESH_TEST_NATS")
if url == "" {
t.Skip("MESH_TEST_NATS unset")
}
js, err := Dial(url)
if err != nil {
t.Fatal(err)
}
defer js.Close()
const stream, worker, filter = "EVENTS_T", "EVENTS_T_reader", "mesh.t.event.>"
_ = js.js.DeleteStream(stream)
if _, err := js.js.AddStream(&nats.StreamConfig{Name: stream, Subjects: []string{filter}, Storage: nats.MemoryStorage}); err != nil {
t.Fatal(err)
}
defer func() { _ = js.js.DeleteStream(stream) }()
if _, err := js.js.AddConsumer(stream, &nats.ConsumerConfig{
Durable: worker, AckPolicy: nats.AckExplicitPolicy, AckWait: 60 * time.Second,
FilterSubject: filter, DeliverSubject: "_DELIVER." + worker,
}); err != nil {
t.Fatal(err)
}
// History the old consumer would have acknowledged long ago, and must not come back.
for i := 0; i < 3; i++ {
if _, err := js.js.Publish("mesh.t.event.old", []byte("old")); err != nil {
t.Fatal(err)
}
}
if err := js.EnsureConsumer(Consumer{Name: worker, Stream: stream, Filters: []string{filter}, AckWaitSeconds: 60}); err != nil {
t.Fatal(err)
}
have, err := js.js.ConsumerInfo(stream, worker)
if err != nil {
t.Fatal(err)
}
if have.Config.DeliverSubject != "" {
t.Fatal("a history stream's consumer of the wrong type was left as it was")
}
if have.Config.DeliverPolicy != nats.DeliverNewPolicy || have.NumPending != 0 {
t.Fatalf("re-made consumer delivers %v with %d pending; it must deliver from now on with nothing of the past", have.Config.DeliverPolicy, have.NumPending)
}
// And what arrives from now on is delivered.
if _, err := js.js.Publish("mesh.t.event.new", []byte("new")); err != nil {
t.Fatal(err)
}
sub, err := js.js.PullSubscribe(filter, worker, nats.Bind(stream, worker))
if err != nil {
t.Fatal(err)
}
got, err := sub.Fetch(1, nats.MaxWait(3*time.Second))
if err != nil || len(got) != 1 || string(got[0].Data) != "new" {
t.Fatalf("the re-made consumer delivered %v, %v; want the one new message", got, err)
}
}
+97 -4
View File
@@ -587,6 +587,12 @@ func one(ctx context.Context, run Runner, publish Publisher,
if err != nil { if err != nil {
return catalogue.Built{}, fmt.Errorf("%s: compiling %s failed: %w", module, a.Name, err) return catalogue.Built{}, fmt.Errorf("%s: compiling %s failed: %w", module, a.Name, err)
} }
// **Every entrypoint the runtime may serve is executable** (novox/hq ADR 0193). The runtime
// knows no language; for one that runs through an interpreter the build writes the launcher.
launchers, err := writeLaunchers(compiled, chain, a)
if err != nil {
return catalogue.Built{}, fmt.Errorf("%s: writing %s's launchers failed: %w", module, a.Name, err)
}
say("bundle", "compiled, packing") say("bundle", "compiled, packing")
body, err := pack(compiled) body, err := pack(compiled)
if err != nil { if err != nil {
@@ -598,7 +604,7 @@ func one(ctx context.Context, run Runner, publish Publisher,
if err != nil { if err != nil {
return catalogue.Built{}, err return catalogue.Built{}, err
} }
return catalogue.Built{Name: a.Name, Kind: a.Kind, Reference: where, Digest: digest}, nil return catalogue.Built{Name: a.Name, Kind: a.Kind, Reference: where, Digest: digest, Launchers: launchers}, nil
case catalogue.ArtifactPackage: case catalogue.ArtifactPackage:
// Built and published on a public base, to the mesh's package registry, by version // Built and published on a public base, to the mesh's package registry, by version
@@ -719,6 +725,21 @@ func short(commit string) string {
return commit return commit
} }
// Said is where the lines Command speaks go, beside the build's own Log: what runs, how long it
// took, and that it failed. Nil prints them to stderr, as a build machine with nobody listening
// should. The machine sets it per build so every line reaches the bus too (novox/hq ADR 0157) —
// the step is "run", and the message is the line as it has always been printed.
var Said Log
func tell(step, format string, args ...any) {
message := fmt.Sprintf(format, args...)
if Said == nil {
fmt.Fprintf(os.Stderr, " %s\n", message)
return
}
Said(step, message)
}
// Command is a Runner that actually runs things. // Command is a Runner that actually runs things.
func Command(ctx context.Context, dir, name string, args ...string) (string, error) { func Command(ctx context.Context, dir, name string, args ...string) (string, error) {
// **Every command is echoed before it runs**, with where. On a build that hangs, the last line // **Every command is echoed before it runs**, with where. On a build that hangs, the last line
@@ -726,16 +747,23 @@ func Command(ctx context.Context, dir, name string, args ...string) (string, err
// nothing" and "git clone is waiting on a network that will not answer". Silent on success is // nothing" and "git clone is waiting on a network that will not answer". Silent on success is
// what made an empty workspace unreadable. // what made an empty workspace unreadable.
started := timeNow() started := timeNow()
fmt.Fprintf(os.Stderr, " $ (%s) %s %s\n", short(filepath.Base(dir)), name, strings.Join(args, " ")) tell("run", "$ (%s) %s %s", short(filepath.Base(dir)), name, strings.Join(args, " "))
cmd := exec.CommandContext(ctx, name, args...) cmd := exec.CommandContext(ctx, name, args...)
cmd.Dir = dir cmd.Dir = dir
out, err := cmd.CombinedOutput() out, err := cmd.CombinedOutput()
if err != nil { if err != nil {
fmt.Fprintf(os.Stderr, " ! %s %s failed after %s\n", name, args[0], since(started)) tell("run", "! %s %s failed after %s", name, args[0], since(started))
// The command's own output is part of what a reader needs — the compiler's error, the
// clone's refusal — and a line per output line keeps it readable on the bus.
for _, line := range strings.Split(strings.TrimSpace(string(out)), "\n") {
if line != "" {
tell("output", "%s", line)
}
}
return string(out), fmt.Errorf("%s %s: %w\n%s", return string(out), fmt.Errorf("%s %s: %w\n%s",
name, strings.Join(args, " "), err, strings.TrimSpace(string(out))) name, strings.Join(args, " "), err, strings.TrimSpace(string(out)))
} }
fmt.Fprintf(os.Stderr, " ✓ %s %s (%s)\n", name, firstArg(args), since(started)) tell("run", "✓ %s %s (%s)", name, firstArg(args), since(started))
return string(out), nil return string(out), nil
} }
@@ -946,6 +974,26 @@ func compile(ctx context.Context, run Runner, tree string, chain Toolchain,
if _, err := run(ctx, tree, "docker", invocation...); err != nil { if _, err := run(ctx, tree, "docker", invocation...); err != nil {
return "", err return "", err
} }
if chain.Dependencies != "" {
// **What the bundle runs with, from the image it was compiled in** (Toolchain.Dependencies).
// A second run in the same image rather than a shell wrapped around the compiler: the
// compile line stays a plain command a reader can run by hand, and the copy is one more
// plain command beside it. Refused by name when the image carries no such directory — an
// older toolchain image — because a bundle packed without its dependencies starts nowhere
// and says so three layers away from here.
copying := []string{
"run", "--rm",
"--volume", tree + ":" + within,
"--workdir", within,
base,
"sh", "-c",
`test -d "$1" || { echo "the toolchain image carries no $1: it predates the mesh shipping a bundle's dependencies, rebuild $2 first" >&2; exit 1; }; cp -a "$1/." "$3/"`,
"dependencies", chain.Dependencies, chain.Base, out,
}
if _, err := run(ctx, tree, "docker", copying...); err != nil {
return "", fmt.Errorf("copying the %s dependencies a bundle runs with: %w", chain.Language, err)
}
}
return filepath.Join(tree, out), nil return filepath.Join(tree, out), nil
} }
@@ -1123,6 +1171,9 @@ func readBy(manifest catalogue.Manifest) []catalogue.ArtifactContext {
// binaryName is what a compiled bundle's executable is called: what the artifact says, or the name of // binaryName is what a compiled bundle's executable is called: what the artifact says, or the name of
// the package it is built from, which is what a compiler would have chosen anyway. // the package it is built from, which is what a compiler would have chosen anyway.
func binaryName(a catalogue.Artifact) string { func binaryName(a catalogue.Artifact) string {
if name := catalogue.BinaryOf(a); name != "" {
return name
}
if name := strings.TrimSpace(a.Binary); name != "" { if name := strings.TrimSpace(a.Binary); name != "" {
return name return name
} }
@@ -1131,3 +1182,45 @@ func binaryName(a catalogue.Artifact) string {
} }
return a.Name return a.Name
} }
// launcherSuffix is what a TypeScript entrypoint's launcher is called beside it: index.js is
// started as index.serve.mjs (novox/hq ADR 0193). An ES module by its own extension, whatever the
// bundle's package.json says.
const launcherSuffix = ".serve.mjs"
// writeLaunchers writes, beside every entrypoint of a TypeScript bundle, an executable that
// imports the entrypoint and serves what it registered over MCP on stdio — through the bundle's
// own copy of the SDK, so registering and serving meet in one registry (novox/hq ADR 0193). Its
// answer is each entrypoint's launcher, by entrypoint, relative to the bundle's root; nothing for
// a language whose build is already executable.
func writeLaunchers(root string, chain Toolchain, a catalogue.Artifact) (map[string]string, error) {
if chain.Language != "typescript" {
return nil, nil
}
out := map[string]string{}
for _, entry := range a.Entrypoints {
if !strings.HasSuffix(entry, ".js") {
continue
}
launcher := strings.TrimSuffix(entry, ".js") + launcherSuffix
body := "#!/usr/bin/env node\n" +
"// Written by the mesh's builder (novox/hq ADR 0193): serve what " + entry + " registers,\n" +
"// over MCP on stdio, as the module the node's runtime names in MESH_SERVED_MODULE.\n" +
"import { serveRegisteredOverStdio } from \"@novox/mesh-sdk/stdio\";\n" +
"await import(\"./" + filepath.Base(entry) + "\");\n" +
"await serveRegisteredOverStdio();\n"
path := filepath.Join(root, filepath.FromSlash(launcher))
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return nil, err
}
if err := os.WriteFile(path, []byte(body), 0o755); err != nil {
return nil, err
}
// WriteFile honours the umask; the mode a machine unpacks is the one packed, so it is set.
if err := os.Chmod(path, 0o755); err != nil {
return nil, err
}
out[entry] = launcher
}
return out, nil
}
+40 -1
View File
@@ -82,6 +82,41 @@ func TestABundleIsCompiledAndPackedWithNoDockerfile(t *testing.T) {
if !strings.HasPrefix(digest, "sha256:") { if !strings.HasPrefix(digest, "sha256:") {
t.Fatalf("the bundle was not pinned: %v", got.Manifest.Resources[0]) t.Fatalf("the bundle was not pinned: %v", got.Manifest.Resources[0])
} }
// **And what it runs with, from the image it was compiled in** (novox/hq to-be 38 WP3). A
// second run in the same toolchain image copies the toolchain's runtime directory — the
// `"type": "module"` package.json and the pruned node_modules — into the output's root, and
// refuses by name when the image carries none rather than packing a bundle that starts nowhere.
var copied string
for _, line := range r.ran {
if strings.HasPrefix(line, "docker run") && strings.Contains(line, "/app/runtime") {
copied = line
}
}
if copied == "" {
t.Fatalf("the bundle's dependencies were not copied in after the compile:\n%s", strings.Join(r.ran, "\n"))
}
if !strings.Contains(copied, "mesh-tools/build@sha256:") || !strings.Contains(copied, "predates") ||
!strings.Contains(copied, Out("code")) {
t.Fatalf("the copy does not run in the same toolchain, refuse an older image by name, or land in the artifact's output: %s", copied)
}
if strings.Index(strings.Join(r.ran, "\n"), "--outDir") > strings.Index(strings.Join(r.ran, "\n"), "/app/runtime") {
t.Fatal("the dependencies were copied before the compile wrote its output")
}
}
// A language whose bundle carries its own dependencies copies nothing in: a Go binary is static.
func TestOnlyALanguageWithARuntimeDirectoryCopiesDependenciesIn(t *testing.T) {
ts, _ := ToolchainFor("typescript")
if ts.Dependencies != "/app/runtime" {
t.Fatalf("typescript bundles run with %q", ts.Dependencies)
}
for _, language := range []string{"go", "python"} {
chain, _ := ToolchainFor(language)
if chain.Dependencies != "" {
t.Fatalf("%s copies %q into every bundle, and its bundles carry their own", language, chain.Dependencies)
}
}
} }
// **Refused before anything is built, naming what to build first.** A base the mesh has not built // **Refused before anything is built, naming what to build first.** A base the mesh has not built
@@ -145,9 +180,13 @@ func TestTwoBundlesInOneModuleArePackedSeparately(t *testing.T) {
t.Fatalf("a module with two bundles did not build: %v", err) t.Fatalf("a module with two bundles did not build: %v", err)
} }
// Compiled into two different places. // Compiled into two different places. Only the compile lines: the copy of each bundle's
// dependencies names the same directory again, deliberately.
var outputs []string var outputs []string
for _, line := range r.ran { for _, line := range r.ran {
if !strings.Contains(line, "--outDir") {
continue
}
for _, part := range strings.Fields(line) { for _, part := range strings.Fields(line) {
if strings.HasPrefix(part, ".mesh-build/") { if strings.HasPrefix(part, ".mesh-build/") {
outputs = append(outputs, part) outputs = append(outputs, part)
+49
View File
@@ -0,0 +1,49 @@
package builder
import (
"os"
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// novox/hq ADR 0193: every entrypoint the runtime may serve is executable, and the runtime knows no
// language — so a TypeScript bundle carries a launcher beside each entrypoint.
func TestATypeScriptBundleCarriesAnExecutableLauncherBesideEachEntrypoint(t *testing.T) {
root := t.TempDir()
chain, err := ToolchainFor("typescript")
if err != nil {
t.Fatal(err)
}
got, err := writeLaunchers(root, chain, catalogue.Artifact{Name: "tools", Kind: catalogue.ArtifactBundle,
Language: "typescript", Entrypoints: []string{"tools/index.js", "index.js"}})
if err != nil {
t.Fatal(err)
}
if got["tools/index.js"] != "tools/index.serve.mjs" || got["index.js"] != "index.serve.mjs" {
t.Fatalf("launchers: %v", got)
}
path := filepath.Join(root, "tools", "index.serve.mjs")
info, err := os.Stat(path)
if err != nil {
t.Fatal(err)
}
if info.Mode().Perm() != 0o755 {
t.Errorf("the launcher is %v, not executable 0755", info.Mode().Perm())
}
body, _ := os.ReadFile(path)
for _, want := range []string{"#!/usr/bin/env node\n", `from "@novox/mesh-sdk/stdio"`, `await import("./index.js")`, "serveRegisteredOverStdio()"} {
if !strings.Contains(string(body), want) {
t.Errorf("the launcher lacks %q:\n%s", want, body)
}
}
// A compiled language's build is executable already: no launcher.
goChain, _ := ToolchainFor("go")
none, err := writeLaunchers(t.TempDir(), goChain, catalogue.Artifact{Name: "tools", Kind: catalogue.ArtifactBundle, Language: "go"})
if err != nil || len(none) != 0 {
t.Errorf("a Go bundle was given launchers: %v %v", none, err)
}
}
+25 -1
View File
@@ -57,6 +57,23 @@ type Toolchain struct {
// carrying its debug info. The mistake was believing a comment rather than reading the file it // carrying its debug info. The mistake was believing a comment rather than reading the file it
// produced (novox/hq 04-ISSUES/161). // produced (novox/hq 04-ISSUES/161).
LinkerFlags []string LinkerFlags []string
// Dependencies is a directory inside the toolchain image whose contents a bundle in this
// language runs with, copied whole into the compiled output's root after the compile.
//
// **A bundle that compiles is not yet a bundle that runs.** The compiler resolves `import
// "nats"` from the toolchain image's own node_modules and the pack takes only what the compiler
// wrote, so what a machine unpacked could not find a single dependency — and no TypeScript bundle
// had ever run live to show it (novox/hq to-be 38 WP3). For TypeScript the directory holds a
// `package.json` saying `"type": "module"` — Node reads a bare `.js` as CommonJS otherwise, so a
// bundle with its dependencies and without that line still fails to start — and the pruned,
// production-only node_modules the runtime itself ships with: the SDK's and the runtime's
// dependencies, and nothing module-specific yet (novox/hq ADR 0188 §5: a skeleton; a module's
// own npm dependencies are a later step). Empty for a language whose bundle carries its own —
// a Go binary is static, a Python bundle is installed with its dependencies.
//
// A toolchain image without the directory fails the build by name rather than packing a bundle
// that starts nowhere: the image predates this and must be rebuilt first.
Dependencies string
// SystemStamp is the variable this language's linker fills with the artifact's declared system, // SystemStamp is the variable this language's linker fills with the artifact's declared system,
// for a language whose binaries are pinned to one at link time (novox/hq ADR 0005). // for a language whose binaries are pinned to one at link time (novox/hq ADR 0005).
// //
@@ -107,14 +124,21 @@ var toolchains = []Toolchain{
// symlinks to a launcher that requires its library relatively — and the base image's own // symlinks to a launcher that requires its library relatively — and the base image's own
// assembly resolves them away, leaving a launcher whose relative require points nowhere. // assembly resolves them away, leaving a launcher whose relative require points nowhere.
// Every module's hand-written Dockerfile had to know this. Now none of them does. // Every module's hand-written Dockerfile had to know this. Now none of them does.
// **Rooted at the module, so an entrypoint lands where it is named.** Without a root the
// compiler takes the common directory of the files it is given: a module compiling only
// `tools/index.ts` had its output at `index.js`, and the entrypoint it declared —
// `tools/index.js`, "named as it will be found" — named a file the bundle did not
// contain. The runtime that loads bundles by their declared entrypoints (novox/hq ADR
// 0175) is what made this visible.
Compile: []string{ Compile: []string{
"node", "/app/node_modules/typescript/bin/tsc", "node", "/app/node_modules/typescript/bin/tsc",
"--module", "NodeNext", "--moduleResolution", "NodeNext", "--module", "NodeNext", "--moduleResolution", "NodeNext",
"--target", "ES2022", "--target", "ES2022", "--rootDir", ".",
}, },
OutputFlag: "--outDir", OutputFlag: "--outDir",
Unit: UnitSources, Unit: UnitSources,
SourceExt: ".ts", SourceExt: ".ts",
Dependencies: "/app/runtime",
}, },
{ {
Language: "go", Language: "go",
+5 -5
View File
@@ -25,7 +25,7 @@ func reachable() Node {
func onNetwork(nodes ...string) map[string][]Provider { func onNetwork(nodes ...string) map[string][]Provider {
out := make([]Provider, 0, len(nodes)) out := make([]Provider, 0, len(nodes))
for _, n := range nodes { for _, n := range nodes {
out = append(out, Provider{Node: n, At: n + ".internal"}) out = append(out, Provider{Node: n, At: n + ".internal", Module: "postgres"})
} }
return map[string][]Provider{"postgres-database": out} return map[string][]Provider{"postgres-database": out}
} }
@@ -99,7 +99,7 @@ func TestSayingWhichOneSettlesIt(t *testing.T) {
got, err := Resolve(brokeredShelf(), []string{"meshboard"}, reachable(), got, err := Resolve(brokeredShelf(), []string{"meshboard"}, reachable(),
World{ World{
Offered: onNetwork("anchor", "archive"), Offered: onNetwork("anchor", "archive"),
Pinned: map[string]string{"postgres-database": "archive"}, Pinned: map[string]Chosen{"postgres-database": {Node: "archive", Module: "postgres"}},
}) })
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
@@ -115,7 +115,7 @@ func TestBeingPointedAtAMachineThatDoesNotProvideItIsRefused(t *testing.T) {
_, err := Resolve(brokeredShelf(), []string{"meshboard"}, reachable(), _, err := Resolve(brokeredShelf(), []string{"meshboard"}, reachable(),
World{ World{
Offered: onNetwork("anchor", "archive"), Offered: onNetwork("anchor", "archive"),
Pinned: map[string]string{"postgres-database": "somewhere-else"}, Pinned: map[string]Chosen{"postgres-database": {Node: "somewhere-else", Module: "postgres"}},
}) })
if err == nil { if err == nil {
t.Fatal("a machine was silently given a different database from the one chosen") t.Fatal("a machine was silently given a different database from the one chosen")
@@ -131,12 +131,12 @@ func TestOneProviderDoesNotOverruleAChoice(t *testing.T) {
_, err := Resolve(brokeredShelf(), []string{"meshboard"}, reachable(), _, err := Resolve(brokeredShelf(), []string{"meshboard"}, reachable(),
World{ World{
Offered: onNetwork("anchor"), Offered: onNetwork("anchor"),
Pinned: map[string]string{"postgres-database": "archive"}, Pinned: map[string]Chosen{"postgres-database": {Node: "archive", Module: "postgres"}},
}) })
if err == nil { if err == nil {
t.Fatal("the only database was used although another was chosen") t.Fatal("the only database was used although another was chosen")
} }
if !strings.Contains(err.Error(), "only anchor provides it") { if !strings.Contains(err.Error(), "only anchor/postgres provides it") {
t.Fatalf("the refusal does not say what is available: %v", err) t.Fatalf("the refusal does not say what is available: %v", err)
} }
} }
+122 -1
View File
@@ -2,6 +2,7 @@ package catalogue
import ( import (
"fmt" "fmt"
"path"
"sort" "sort"
"strings" "strings"
) )
@@ -28,6 +29,9 @@ type Built struct {
Reference string Reference string
// Digest is "sha256:<hex>", for an archive. An image reference already ends in one. // Digest is "sha256:<hex>", for an archive. An image reference already ends in one.
Digest string Digest string
// Launchers are, for a bundle in an interpreted language, the executable the build wrote beside
// each entrypoint, by entrypoint (novox/hq ADR 0193): what the node's runtime starts to serve it.
Launchers map[string]string
} }
// Resolve fills a manifest's resources in from what was built. // Resolve fills a manifest's resources in from what was built.
@@ -67,6 +71,37 @@ func (m Manifest) Resolve(built []Built) (Manifest, error) {
out := m out := m
out.Build = nil out.Build = nil
out.Resources = nil out.Resources = nil
// What the build compiled, kept on the resolved manifest (novox/hq ADR 0175): a tools bundle is
// named by no resource of the module's own — the node's runtime loads it — so this is the only
// place the mesh would otherwise not have it. In artifact order, so two resolutions of one
// build compare equal.
out.Bundles = nil
if m.Build != nil {
for _, a := range m.Build.Artifacts {
if a.Kind != ArtifactBundle {
continue
}
made := by[a.Name]
// What the runtime loads: what the artifact said, else every entrypoint of a module
// that declares tools, else nothing (the field's own rule; see Artifact.Loads).
loads := append([]string(nil), a.Loads...)
if a.Loads == nil && len(m.Tools) > 0 {
loads = append([]string(nil), a.Entrypoints...)
}
// **Kept, never routed** (ADR 0155): the builder publishes to the store at the address
// it reached it by, and a manifest carrying that address names an installation —
// registration refused node-tools for exactly this on 2026-10-02. The build record
// already keeps the store-relative form; the resolved manifest keeps the same, and
// composition routes it through the store a machine reaches (Routed).
out.Bundles = append(out.Bundles, Bundle{
Name: a.Name, Source: Recorded(made.Reference), Digest: made.Digest,
Language: a.Language, Entrypoints: append([]string(nil), a.Entrypoints...),
Loads: loads, Env: copyWords(a.Env), Launchers: copyWords(made.Launchers),
Binary: BinaryOf(a),
})
}
sort.Slice(out.Bundles, func(i, j int) bool { return out.Bundles[i].Name < out.Bundles[j].Name })
}
for _, r := range m.Resources { for _, r := range m.Resources {
named, _ := r["artifact"].(string) named, _ := r["artifact"].(string)
if named == "" { if named == "" {
@@ -110,7 +145,8 @@ func (m Manifest) Resolve(built []Built) (Manifest, error) {
// The same on the wire: both are bytes fetched by digest and unpacked. They differ in // The same on the wire: both are bytes fetched by digest and unpacked. They differ in
// how they were made — one packed as it stood, the other compiled first — and a // how they were made — one packed as it stood, the other compiled first — and a
// machine has no reason to care which. // machine has no reason to care which.
filled["source"] = artifact.Reference // Kept, not routed, for the reason the bundles above are (ADR 0155).
filled["source"] = Recorded(artifact.Reference)
filled["digest"] = artifact.Digest filled["digest"] = artifact.Digest
// **And `${version}`, so a resource can name a place that is this build's alone** // **And `${version}`, so a resource can name a place that is this build's alone**
// (novox/hq ADR 0141, 04-ISSUES/142). A component is unpacked into a directory named // (novox/hq ADR 0141, 04-ISSUES/142). A component is unpacked into a directory named
@@ -172,6 +208,12 @@ func (b *Build) problems(module string) []string {
// A bundle's source is the module's own directory by definition, and what it needs to say // A bundle's source is the module's own directory by definition, and what it needs to say
// is which compiler — because the mesh chooses that, and cannot choose for a module that // is which compiler — because the mesh chooses that, and cannot choose for a module that
// has not said. // has not said.
if len(a.Env) > 0 && a.Kind != ArtifactBundle {
problems = append(problems, fmt.Sprintf(
"%s: %q is a %q and says what it is given (env). Only a bundle the node's runtime "+
"serves is given words (novox/hq ADR 0192); a container says its own environment",
module, a.Name, a.Kind))
}
if a.Kind == ArtifactBundle || a.Kind == ArtifactPackage { if a.Kind == ArtifactBundle || a.Kind == ArtifactPackage {
// **Except for a language that compiles to a binary, where it names which one** // **Except for a language that compiles to a binary, where it names which one**
// (novox/hq 04-ISSUES/142). A bundle in an interpreted language is the module's own // (novox/hq 04-ISSUES/142). A bundle in an interpreted language is the module's own
@@ -191,6 +233,21 @@ func (b *Build) problems(module string) []string {
"%s: %q is a bundle and says no language, so nothing can choose a compiler "+ "%s: %q is a bundle and says no language, so nothing can choose a compiler "+
"for it", module, a.Name)) "for it", module, a.Name))
} }
problems = append(problems, bundleEnvProblems(module, a)...)
// What the runtime loads is among what was compiled (ADR 0175): a name here that is
// not an entrypoint is a file the bundle does not contain, and the runtime would
// fail to import it on every machine rather than here.
for _, load := range a.Loads {
found := false
for _, e := range a.Entrypoints {
found = found || e == load
}
if !found {
problems = append(problems, fmt.Sprintf(
"%s: %q says the runtime loads %q, which is not among its entrypoints — "+
"what is loaded is compiled, so it is named there too", module, a.Name, load))
}
}
// **A system, for a language that compiles to a binary** (novox/hq ADR 0142). A binary // **A system, for a language that compiles to a binary** (novox/hq ADR 0142). A binary
// is pinned to one operating system at link time so a host refuses to touch a machine // is pinned to one operating system at link time so a host refuses to touch a machine
// it was not built for (novox/hq ADR 0005); an artifact that says nothing would be // it was not built for (novox/hq ADR 0005); an artifact that says nothing would be
@@ -315,3 +372,67 @@ func versionOf(digest string) string {
} }
return hex return hex
} }
// bundleEnvWords are the words the runtime sets for itself; a bundle that named one would be
// telling the runtime what it is, which is the mesh's to say (novox/hq ADR 0192).
var bundleEnvWords = map[string]bool{
RuntimeToolModules: true, RuntimeBrokerFile: true, RuntimeOperatorAccount: true,
RuntimeOperatorHome: true, RuntimeToolEnv: true,
}
// bundleEnvProblems says what is wrong with what a bundle says it is given (novox/hq ADR 0192):
// a value is a path or a constant written with the references a container's environment may use
// for a place or a port, and never a secret's content or another module's binding — a secret
// reaches a tool as a file whose path is named.
func bundleEnvProblems(module string, a Artifact) []string {
if len(a.Env) == 0 {
return nil
}
var problems []string
for _, word := range sortedKeys(a.Env) {
value := a.Env[word]
if bundleEnvWords[word] {
problems = append(problems, fmt.Sprintf(
"%s: %q gives itself %s, which the node's runtime sets for itself; a bundle is "+
"given its own words beside the runtime's, never in place of them (novox/hq ADR 0192)",
module, a.Name, word))
}
rest := ofPort.ReplaceAllString(dirRef.ReplaceAllString(value, ""), "")
if strings.Contains(rest, "${") {
problems = append(problems, fmt.Sprintf(
"%s: %q gives %s the value %q. A bundle's word is a path or a constant, written with "+
"${dir:…} and ${port:…} only; a secret reaches a tool as a file the mesh places, "+
"named by its path, never as its content (novox/hq ADR 0192)",
module, a.Name, word, value))
}
}
return problems
}
func copyWords(in map[string]string) map[string]string {
if len(in) == 0 {
return nil
}
out := make(map[string]string, len(in))
for k, v := range in {
out[k] = v
}
return out
}
// BinaryOf is what a bundle compiled to a binary is called once built: what the artifact names, else
// the package it is built from, else the artifact's own name (novox/hq 04-ISSUES/142). Empty for a
// language that does not compile to one. The builder writes the binary under this name, and the
// composer runs it by it, so both ask here.
func BinaryOf(a Artifact) string {
if !compilesToABinary(a.Language) {
return ""
}
if name := strings.TrimSpace(a.Binary); name != "" {
return name
}
if from := strings.Trim(a.From, "./"); from != "" {
return path.Base(from)
}
return a.Name
}
+18 -8
View File
@@ -11,11 +11,24 @@ import (
// //
// Not a fixture: the point is whether the manifests as written are accepted by the control plane that // Not a fixture: the point is whether the manifests as written are accepted by the control plane that
// will read them, and a copy of one manifest proves nothing about the other seventy-one. // will read them, and a copy of one manifest proves nothing about the other seventy-one.
func TestEveryCatalogueManifestParses(t *testing.T) { // catalogueRoot is the catalogue these checks run over: MESH_CATALOGUE when set, else the checkout
root := os.Getenv("MESH_CATALOGUE") // beside this one, the way the main layout has it. A check that only ran when somebody remembered a
if root == "" { // variable was a check nobody ran (novox/hq issue 134, 2026-09-30); it skips only when there is no
t.Skip("set MESH_CATALOGUE to a catalogue checkout to run this") // catalogue to be found at all.
func catalogueRoot(t *testing.T) string {
t.Helper()
if root := os.Getenv("MESH_CATALOGUE"); root != "" {
return root
} }
sibling := filepath.Join("..", "..", "..", "mesh-catalog")
if _, err := os.Stat(filepath.Join(sibling, "modules")); err != nil {
t.Skip("no catalogue beside this checkout and MESH_CATALOGUE unset")
}
return sibling
}
func TestEveryCatalogueManifestParses(t *testing.T) {
root := catalogueRoot(t)
found, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json")) found, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
if err != nil || len(found) == 0 { if err != nil || len(found) == 0 {
t.Fatalf("no manifests under %s: %v", root, err) t.Fatalf("no manifests under %s: %v", root, err)
@@ -51,10 +64,7 @@ func TestEveryCatalogueManifestParses(t *testing.T) {
// definition names a domain or a public address the mesh acts on, and every value that must for now // definition names a domain or a public address the mesh acts on, and every value that must for now
// carries its reason (novox/hq ADR 0155, issue 134). The list it prints is the one that shrinks. // carries its reason (novox/hq ADR 0155, issue 134). The list it prints is the one that shrinks.
func TestNoCatalogueManifestNamesAnInstallation(t *testing.T) { func TestNoCatalogueManifestNamesAnInstallation(t *testing.T) {
root := os.Getenv("MESH_CATALOGUE") root := catalogueRoot(t)
if root == "" {
t.Skip("set MESH_CATALOGUE to a catalogue checkout to run this")
}
found, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json")) found, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
if err != nil || len(found) == 0 { if err != nil || len(found) == 0 {
t.Fatalf("no manifests under %s: %v", root, err) t.Fatalf("no manifests under %s: %v", root, err)
+100
View File
@@ -0,0 +1,100 @@
package catalogue
import (
"sort"
)
// Chosen is the provider somebody named for a provision: the module, and the node it runs on. Both,
// always (novox/hq #258) — a provision comes from a module, and the same module on two machines is
// two answers, so neither half alone says which. Module is empty only on a record made before this
// was asked, and such a record is honoured exactly as long as it is unambiguous.
type Chosen struct {
Node string
Module string
}
func (c Chosen) String() string {
if c.Module == "" {
return c.Node
}
return c.Node + "/" + c.Module
}
// matches is whether this provider is the one chosen.
func (c Chosen) matches(p Provider) bool {
return p.Node == c.Node && (c.Module == "" || p.Module == c.Module)
}
// among is every offered provider the choice names — one, when the choice is whole.
func (c Chosen) among(where []Provider) []Provider {
var out []Provider
for _, p := range where {
if c.matches(p) {
out = append(out, p)
}
}
return out
}
// nameOf is how a refusal names a provider: the node and the module on it.
func nameOf(p Provider) string {
return Chosen{Node: p.Node, Module: p.Module}.String()
}
// providerNames is every provider named, sorted, for a refusal to list.
func providerNames(where []Provider) []string {
out := make([]string, 0, len(where))
for _, p := range where {
out = append(out, nameOf(p))
}
sort.Strings(out)
return out
}
// providersHere is which modules in this node's own set offer a provision, sorted.
func providersHere(catalogue map[string]Manifest, here func(string) bool, want string) []string {
var out []string
for name, m := range catalogue {
if !here(name) {
continue
}
for _, o := range m.Offers() {
if o == want {
out = append(out, name)
break
}
}
}
sort.Strings(out)
return out
}
// servedByOne is what one provider beside the consumer says a consumer needs to know, or nothing.
//
// Serving is *whether* a need is created at all when the provider is on this same machine (novox/hq
// 04-ISSUES/038's sibling): a need never created is a binding the consumer never gets. The manifest
// alone answers that; the values are settled later, with the node's settings.
func servedByOne(m Manifest, want string) map[string]any {
if _, ok := m.Serves[want]; ok {
return ServedOn(m, want, nil)
}
return nil
}
// sharedByOne is the own secret that provider names as its credential (ADR 0158), or "" when it
// gives each consumer its own.
func sharedByOne(m Manifest, want string) string {
if own, shared := m.SharedCredentialOf(want); shared {
return own
}
return ""
}
func oneOf(list []string, s string) bool {
for _, x := range list {
if x == s {
return true
}
}
return false
}
+21
View File
@@ -197,6 +197,27 @@ func TestARouteCanBeSetPerMesh(t *testing.T) {
} }
} }
func TestASettingReachesAContributionOnlyWhereItDeclaresTheKey(t *testing.T) {
// novox/hq 04-ISSUES/173: the mail module's site name, set so its environment file could read
// it, arrived in every route it contributed. A setting overrides a key the contribution
// declares and adds none — the provider reads the contribution as a contract.
got, _ := Resolve(shelf(proxy(), published("board", "board", 8080)), []string{"board"}, workstation(), World{})
out, err := got.Declaration(Rendering{Settings: SettingsBy{
"board": {{From: "the mesh", Values: map[string]any{"host": "dashboard", "sitename": "Board"}}},
}})
if err != nil {
t.Fatal(err)
}
given := received(t, out)
if given[0].Values["host"] != "dashboard" {
t.Fatalf("the setting did not override the route's host: %v", given[0].Values)
}
if _, leaked := given[0].Values["sitename"]; leaked {
t.Fatalf("a setting the route never declared reached the proxy: %v", given[0].Values)
}
}
func TestReceivingWhatYouDoNotProvideIsRefused(t *testing.T) { func TestReceivingWhatYouDoNotProvideIsRefused(t *testing.T) {
// It would create a file nobody ever writes to, on a machine where nothing asked for it. // It would create a file nobody ever writes to, on a machine where nothing asked for it.
_, err := ParseManifest([]byte(`{"module":"traefik","version":"1", _, err := ParseManifest([]byte(`{"module":"traefik","version":"1",
+245 -8
View File
@@ -241,15 +241,40 @@ func (r Resolution) Declaration(with Rendering) ([]map[string]any, error) {
// Owner is kept beside the resources because a resource id cannot be split back into its module: // Owner is kept beside the resources because a resource id cannot be split back into its module:
// a module's name may itself contain a dot. What the mesh adds of its own — an opening, the guard — // a module's name may itself contain a dot. What the mesh adds of its own — an opening, the guard —
// has no owner. // has no owner.
//
// Received is what each module on the machine is given for each requirement it receives — the same
// contributions its received file is written from, kept beside it so the mesh can also issue them
// on the bus in the module's membership (novox/hq ADR 0167). By module, then requirement.
type Composed struct { type Composed struct {
Resources []map[string]any Resources []map[string]any
Owner map[string]string Owner map[string]string
Received map[string]map[string][]Contribution
// LeftOut is every module of this machine's set that was left out of its declaration, and
// why (novox/hq ADR 0163, rule 6): a setting stored for it that its definition can no longer
// compose. Its held things are kept and its containers untouched — the machine is told so —
// and it is told everything else.
LeftOut map[string]string
}
// LeftOut is which of this machine's modules a declaration composed with these settings leaves
// out, and why (novox/hq ADR 0163, rule 6): each whose stored settings its definition can no longer
// compose. Empty when every module composes. The same judgement SetSettings makes before storing.
func (r Resolution) LeftOut(settings SettingsBy, adopted bool) map[string]string {
out := map[string]string{}
for _, m := range r.Modules {
if err := JudgeSettings(m, settings[m.Module], adopted); err != nil {
out[m.Module] = err.Error()
}
}
return out
} }
// Compose is Declaration with the owner of every resource said. // Compose is Declaration with the owner of every resource said.
func (r Resolution) Compose(with Rendering) (Composed, error) { func (r Resolution) Compose(with Rendering) (Composed, error) {
owner := map[string]string{} owner := map[string]string{}
resources, err := r.compose(with, owner) received := map[string]map[string][]Contribution{}
leftOut := map[string]string{}
resources, err := r.compose(with, owner, received, leftOut)
if err != nil { if err != nil {
return Composed{}, err return Composed{}, err
} }
@@ -261,7 +286,7 @@ func (r Resolution) Compose(with Rendering) (Composed, error) {
"sealed": with.BusMembership, "mode": "0600", "sealed": with.BusMembership, "mode": "0600",
}) })
} }
return Composed{Resources: resources, Owner: owner}, nil return Composed{Resources: resources, Owner: owner, Received: received, LeftOut: leftOut}, nil
} }
// BusMembershipID names the resource carrying a machine's membership for the new bus, and // BusMembershipID names the resource carrying a machine's membership for the new bus, and
@@ -270,7 +295,26 @@ func BusMembershipID() string { return "bus-membership" }
const BusMembershipPath = "/var/lib/mesh/membership-next.json" const BusMembershipPath = "/var/lib/mesh/membership-next.json"
func (r Resolution) compose(with Rendering, owner map[string]string) ([]map[string]any, error) { func (r Resolution) compose(with Rendering, owner map[string]string,
received map[string]map[string][]Contribution, leftOut map[string]string) ([]map[string]any, error) {
// **A setting is judged where it is stored, and an impossible one costs a module, not a
// machine** (novox/hq ADR 0163, rule 6). A definition that moved under a stored setting makes
// this module uncomposable; it is left out of the declaration — its held things kept, its
// containers untouched, the machine told so by name — and the machine is told everything else.
// Before placing, because a placement is a setting too.
left := r.LeftOut(with.Settings, with.Adopted)
kept := make([]Manifest, 0, len(r.Modules))
for _, m := range r.Modules {
if why, isLeft := left[m.Module]; isLeft {
if leftOut != nil {
leftOut[m.Module] = why
}
continue
}
kept = append(kept, m)
}
r.Modules = kept
// Every manifest is placed first (novox/hq ADR 0112): the maps naming where its bindings, // Every manifest is placed first (novox/hq ADR 0112): the maps naming where its bindings,
// credentials and contributions land are resolved against this node's directories, so every // credentials and contributions land are resolved against this node's directories, so every
// reader below — the binding files, the sealed secrets, the grant paths a contribution // reader below — the binding files, the sealed secrets, the grant paths a contribution
@@ -464,18 +508,41 @@ func (r Resolution) compose(with Rendering, owner map[string]string) ([]map[stri
return nil, fmt.Errorf( return nil, fmt.Errorf(
"%s needs a secret called %q and none was made for it", m.Module, name) "%s needs a secret called %q and none was made for it", m.Module, name)
} }
first = append(first, ownedBy(m.SecretsOwner, map[string]any{ // The runtime's credential belongs to the account the runtime runs as (novox/hq ADR 0175,
"id": NeedID(name), "type": "file", "path": m.OwnSecrets[name], "sealed": sealed, // to-be 38 WP3): its process is composed `user: <account>` where the node has one, and a
// root-owned 0600 file is one that process cannot read. Composed here rather than said in
// the manifest, because a manifest cannot say ${machine:account} safely — a node with no
// account has nothing to resolve it to, and then the runtime runs as root and the file
// stays root's.
owner := m.SecretsOwner
if m.Module == RuntimeModule && r.Account != "" {
owner = r.Account
}
first = append(first, ownedBy(owner, map[string]any{
"id": NeedID(name), "type": "file", "path": m.OwnSecrets[name].Path, "sealed": sealed,
})) }))
} }
// This module's tools bundles, where the machine runs the node's tool runtime (novox/hq
// ADR 0175, to-be 38 WP2). Mesh-computed like everything above it, and before the module's
// own resources for the same reason: the runtime's process names the files inside these
// and is restarted when one changes, so they are on the machine before it is.
if r.runtimeHere() {
first = append(first, bundleArchives(m)...)
}
// Operator-owned paths this module is granted use of (novox/hq ADR 0051). Written before // Operator-owned paths this module is granted use of (novox/hq ADR 0051). Written before
// the module's own resources, and so before the container that mounts them: the host must // the module's own resources, and so before the container that mounts them: the host must
// find each present — refusing clearly if the operator has not provided it — before it // find each present — refusing clearly if the operator has not provided it — before it
// starts anything that depends on it. The mesh creates, chowns and reconciles none of it; // starts anything that depends on it. The mesh creates, chowns and reconciles none of it;
// an `access` resource says only *this path must exist, and this module reaches it*. // an `access` resource says only *this path must exist, and this module reaches it*.
for _, a := range m.Accesses { // Where each is on THIS machine is the assignment's (novox/hq issue 153): placed by id
// where the operator said, the definition's default otherwise, refused where neither.
accesses, accessPaths, err := accessesFor(m, with.Settings[m.Module])
if err != nil {
return nil, err
}
for _, a := range accesses {
first = append(first, map[string]any{ first = append(first, map[string]any{
"id": AccessID(a.Path), "type": "access", "path": a.Path, "mode": a.At(), "id": AccessID(a.Path), "type": "access", "path": a.Path, "mode": a.Mode,
}) })
} }
for _, to := range m.SecretRequirements() { for _, to := range m.SecretRequirements() {
@@ -590,6 +657,12 @@ func (r Resolution) compose(with Rendering, owner map[string]string) ([]map[stri
return nil, err return nil, err
} }
first = append(first, file) first = append(first, file)
if received[m.Module] == nil {
received[m.Module] = map[string][]Contribution{}
}
// Empty rather than absent when nobody contributed, for the reason the file is
// written empty: "nothing asked" and "never told" want different responses.
received[m.Module][to] = append([]Contribution{}, given[to]...)
} }
if m.Keeps != "" && with.Kept != nil { if m.Keeps != "" && with.Kept != nil {
file, err := keptFile(m.Keeps, with.Kept) file, err := keptFile(m.Keeps, with.Kept)
@@ -670,6 +743,12 @@ func (r Resolution) compose(with Rendering, owner map[string]string) ([]map[stri
} }
// And where this node places the directories the module declared without a path // And where this node places the directories the module declared without a path
// (novox/hq ADR 0112) — resolved once per module, named by ${dir:…} from any resource. // (novox/hq ADR 0112) — resolved once per module, named by ${dir:…} from any resource.
// — and, on an adopted machine, where the assignment says they already are, with the
// owner the data already has (novox/hq issue 153). Malformed placements are refused here.
placed, err := Places(m, with.Settings[m.Module])
if err != nil {
return nil, err
}
dirs := dirsFor(m, with) dirs := dirsFor(m, with)
// And the machine underneath, which no binding of its own can tell it. // And the machine underneath, which no binding of its own can tell it.
thisMachine := machineFacts(r, with.Names, with.MeshRange) thisMachine := machineFacts(r, with.Names, with.MeshRange)
@@ -681,6 +760,10 @@ func (r Resolution) compose(with Rendering, owner map[string]string) ([]map[stri
// Which of this module's resources its preparation runs before, if it prepares anything. // Which of this module's resources its preparation runs before, if it prepares anything.
prepareBefore := preparationTarget(m) prepareBefore := preparationTarget(m)
// Which found networks this machine's setting keeps for each of its containers (novox/hq
// ADR 0163, rule 4); judged above, so an invalid one is not here.
keptNetworks, _ := KeptNetworks(m, with.Settings[m.Module], with.Adopted)
for _, unsettled := range resources { for _, unsettled := range resources {
resource, err := ApplySettings(unsettled, with.Settings[m.Module]) resource, err := ApplySettings(unsettled, with.Settings[m.Module])
if err != nil { if err != nil {
@@ -690,6 +773,16 @@ func (r Resolution) compose(with Rendering, owner map[string]string) ([]map[stri
for k, v := range resource { for k, v := range resource {
copied[k] = v copied[k] = v
} }
if networks, keeps := keptNetworks[fmt.Sprint(copied["id"])]; keeps {
// The container also joins the found network the setting names, so a neighbour
// that resolves it there keeps resolving it. Passed to the host as its own field,
// which it joins after the container is made.
joins := make([]any, 0, len(networks))
for _, n := range networks {
joins = append(joins, n)
}
copied["networks"] = joins
}
if err := refuseSecretsInEnvironment(copied, secretFiles, m.Module); err != nil { if err := refuseSecretsInEnvironment(copied, secretFiles, m.Module); err != nil {
return nil, err return nil, err
} }
@@ -710,6 +803,12 @@ func (r Resolution) compose(with Rendering, owner map[string]string) ([]map[stri
if err := dirInto(copied, dirs, m.Module); err != nil { if err := dirInto(copied, dirs, m.Module); err != nil {
return nil, err return nil, err
} }
// The operator's data the same way: ${access:…} becomes where this node keeps it,
// and a placed directory takes the owner the assignment said (issue 153).
if err := accessInto(copied, accessPaths, m.Module); err != nil {
return nil, err
}
ownerInto(copied, placed)
// **After settings, and that is the whole reason it is here.** A module's file // **After settings, and that is the whole reason it is here.** A module's file
// content is where a setting lands, so a placeholder may only exist once the setting // content is where a setting lands, so a placeholder may only exist once the setting
// has been put in — filling secrets first would look at content that is not yet what // has been put in — filling secrets first would look at content that is not yet what
@@ -773,6 +872,15 @@ func (r Resolution) compose(with Rendering, owner map[string]string) ([]map[stri
if renamed := reflectsRenamed(m.Module, resource["reload-on"]); renamed != nil { if renamed := reflectsRenamed(m.Module, resource["reload-on"]); renamed != nil {
copied["reload-on"] = renamed copied["reload-on"] = renamed
} }
// **What reads one of this module's own secrets is restarted when it changes** (novox/hq
// issue 203, issue 206). A credential is re-issued by the mesh, and a container that
// mounted the old file keeps the old one open: the build machine ran for an hour on a
// credential the mesh had replaced, because its manifest restarted it on its
// environment file and nobody had thought to name the credential too. Composed here so
// no manifest has to say it, for a container or a daemon that names the secret's path.
if reads := secretsReadBy(copied, m); len(reads) > 0 {
copied["restart-on"] = withRestartOn(copied["restart-on"], reads)
}
// **A version prepares its state before it runs** (novox/hq ADR 0135). Derived from the // **A version prepares its state before it runs** (novox/hq ADR 0135). Derived from the
// module's own resource rather than declared beside it: what prepares the state is the // module's own resource rather than declared beside it: what prepares the state is the
// module's own code, so what it is given has to be what that code is given — and a // module's own code, so what it is given has to be what that code is given — and a
@@ -803,6 +911,41 @@ func (r Resolution) compose(with Rendering, owner map[string]string) ([]map[stri
out = append(out, fact) out = append(out, fact)
} }
} }
// The node's tool runtime, last (novox/hq ADR 0175, to-be 38 WP2.3): one process loading every
// bundle delivered above and holding the credential sealed above, so both exist before it starts
// — the order written here is the order the machine applies.
if r.runtimeHere() {
process, err := r.runtimeProcess(with)
if err != nil {
return nil, err
}
owner[fmt.Sprint(process["id"])] = RuntimeModule
out = append(out, process)
// What each module's bundles are given is read as the account the runtime runs as.
words := map[string]map[string]string{}
for _, m := range r.Modules {
w, err := bundleWords(m, with)
if err != nil {
return nil, err
}
words[m.Module] = w
}
// And a file a module's words name is one the runtime is restarted for when it changes.
if named := givenTo(out, owner, words, r.Account); len(named) > 0 {
restarts, _ := process["restart-on"].([]any)
seen := map[string]bool{}
for _, id := range restarts {
seen[fmt.Sprint(id)] = true
}
for _, id := range named {
if !seen[id] {
restarts = append(restarts, id)
seen[id] = true
}
}
process["restart-on"] = restarts
}
}
if with.Adopted { if with.Adopted {
// First, before anything a module declares: what the mesh needs reachable, then its guard. // First, before anything a module declares: what the mesh needs reachable, then its guard.
// The order a machine applies is the order written here. // The order a machine applies is the order written here.
@@ -929,8 +1072,15 @@ func (r Resolution) filtersHere() string {
// computed for this machine, and each module's per-node exposure. The same answer whether the node // computed for this machine, and each module's per-node exposure. The same answer whether the node
// is adopted or converged — the one loads it as a filter, the other declares it as openings. // is adopted or converged — the one loads it as a filter, the other declares it as openings.
func (r Resolution) Rules(with Rendering) ([]Rule, error) { func (r Resolution) Rules(with Rendering) ([]Rule, error) {
// A module whose settings cannot compose is left out of the declaration (novox/hq ADR 0163,
// rule 6), and out of the filter with it: nothing of it is declared, so nothing of it is let
// through.
left := r.LeftOut(with.Settings, with.Adopted)
exposure := map[string]map[int]string{} exposure := map[string]map[int]string{}
for _, m := range r.Modules { for _, m := range r.Modules {
if _, isLeft := left[m.Module]; isLeft {
continue
}
e, err := Exposure(m, with.Settings[m.Module]) e, err := Exposure(m, with.Settings[m.Module])
if err != nil { if err != nil {
return nil, err return nil, err
@@ -1161,7 +1311,10 @@ func (r Resolution) contributions(settings SettingsBy, grants []Grant,
// module wrote another into its configuration. // module wrote another into its configuration.
func (r Resolution) composed(m Manifest, to string, raw map[string]any, layers []Layer, what string) ( func (r Resolution) composed(m Manifest, to string, raw map[string]any, layers []Layer, what string) (
map[string]any, error) { map[string]any, error) {
values, err := settle(raw, layers, nil, what) // Overridden, not merged: a setting changes a key the contribution declares and adds none.
// The provider reads the contribution as a contract, and a setting made for one of this
// module's files is no part of it (novox/hq 04-ISSUES/173).
values, err := overridden(raw, layers, what)
if err != nil { if err != nil {
return nil, fmt.Errorf("%s: %w", what, err) return nil, fmt.Errorf("%s: %w", what, err)
} }
@@ -1959,3 +2112,87 @@ func portOfEndpoint(values map[string]any, ports map[string]int) {
values["port"] = port values["port"] = port
} }
} }
// secretsReadBy is the file resources of this module's own secrets that a container or a daemon reads
// — named in its volumes, its environment or its env-files by the secret's placed path — as
// restart-on ids. Nothing for other shapes, and nothing for a scheduled or run-once process, which
// the host refuses a restart-on for (it runs again anyway, and reads the file afresh).
func secretsReadBy(resource map[string]any, m Manifest) []string {
kind := fmt.Sprint(resource["type"])
if kind != "container" && kind != "process" {
return nil
}
if resource["schedule"] != nil || resource["run-once"] == true {
return nil
}
var mentioned []string
for _, key := range []string{"volumes", "env", "env-file"} {
mentioned = append(mentioned, stringsIn(resource[key])...)
}
var out []string
for _, name := range sortedKeys(m.OwnSecrets) {
path := m.OwnSecrets[name].Path
if path == "" {
continue
}
for _, s := range mentioned {
// A volume is `source:destination[:mode]`; an env value or an env-file is the path itself.
if s == path || strings.HasPrefix(s, path+":") {
out = append(out, m.Module+"."+NeedID(name))
break
}
}
}
return out
}
// stringsIn is every string in a list or a map's values; nothing for anything else.
func stringsIn(v any) []string {
switch x := v.(type) {
case []any:
var out []string
for _, item := range x {
if s, ok := item.(string); ok {
out = append(out, s)
}
}
return out
case []string:
return x
case map[string]any:
var out []string
for _, k := range sortedKeys(x) {
if s, ok := x[k].(string); ok {
out = append(out, s)
}
}
return out
case map[string]string:
var out []string
for _, k := range sortedKeys(x) {
out = append(out, x[k])
}
return out
}
return nil
}
// withRestartOn is a resource's restart-on list with these ids added once each.
func withRestartOn(have any, add []string) []any {
var out []any
seen := map[string]bool{}
for _, id := range reflectsRenamed("", have) {
s := fmt.Sprint(id)
if !seen[s] {
seen[s] = true
out = append(out, s)
}
}
for _, id := range add {
if !seen[id] {
seen[id] = true
out = append(out, id)
}
}
return out
}
+63 -12
View File
@@ -20,6 +20,12 @@ import (
// declared with the path as the exception it is, and everything else in the module names it by // declared with the path as the exception it is, and everything else in the module names it by
// id — so moving it later is one line, not a search. // id — so moving it later is one line, not a search.
// //
// **The mesh's own files for a module are placed too** (novox/hq issue 174). What the mesh writes
// *for* a module — its sealed bus credential, its merged configuration, its bindings — is the
// mesh's plumbing, not the module's data, and sits under `<root>/mesh/<module>`. A directory
// saying `"place": "mesh"` is that place; the definition names the files beneath it by
// `${dir:<id>}` and states no path.
//
// **Resolved here, not on the machine.** The host receives concrete paths exactly as it always // **Resolved here, not on the machine.** The host receives concrete paths exactly as it always
// has; nothing new reaches it and it learns no field. Which also means a resolved path changing // has; nothing new reaches it and it learns no field. Which also means a resolved path changing
// is a spec change like any other — and the spec comparison must see it (novox/hq issue 126). // is a spec change like any other — and the spec comparison must see it (novox/hq issue 126).
@@ -27,6 +33,15 @@ import (
// defaultDataRoot is where module data lands when a node states no root of its own. // defaultDataRoot is where module data lands when a node states no root of its own.
const defaultDataRoot = "/var/lib" const defaultDataRoot = "/var/lib"
// The two places a pathless directory may name, beside its own id.
const (
// placeOwn is the assignment's own root, <root>/<module> — to-be 27's one directory per
// assignment, which every other placed thing of the module sits beneath.
placeOwn = "."
// placeMesh is where the mesh keeps what it writes for the module, <root>/mesh/<module>.
placeMesh = "mesh"
)
// dirRef is how a module names one of its placed directories: ${dir:<id>}. // dirRef is how a module names one of its placed directories: ${dir:<id>}.
var dirRef = regexp.MustCompile(`\$\{dir:([a-z0-9][a-z0-9-]*)\}`) var dirRef = regexp.MustCompile(`\$\{dir:([a-z0-9][a-z0-9-]*)\}`)
@@ -40,27 +55,54 @@ func dataRoot(with Rendering) string {
// dirsFor is every placed directory of a module, id → the path it resolves to on this node. // dirsFor is every placed directory of a module, id → the path it resolves to on this node.
// //
// A pathless directory saying `"place": "."` is the assignment's own root, <root>/<module> — // A pathless directory saying `"place": "."` is the assignment's own root, <root>/<module>; one
// to-be 27's one directory per assignment, which every other placed thing sits beneath. At most // saying `"place": "mesh"` is the mesh's directory for the module, <root>/mesh/<module>; one
// one makes sense; nothing enforces one, because two ids resolving to one path is a mistake the // saying neither is <root>/<module>/<id>. At most one of each place makes sense; nothing enforces
// module's own files make visible immediately. // one, because two ids resolving to one path is a mistake the module's own files make visible
// immediately.
//
// A stated path may itself begin with a placed reference — `${dir:mesh-state}/state` — and is
// filled after the directories it can name are resolved; one level, because a directory beneath
// a placed one is the whole of what an adopted layout needs (issue 174's `state` and `out`).
func dirsFor(m Manifest, with Rendering) map[string]string { func dirsFor(m Manifest, with Rendering) map[string]string {
dirs := map[string]string{} dirs := map[string]string{}
var beneath []map[string]any
// The assignment's placement wins over both (novox/hq issue 153). Refused elsewhere when
// malformed; here an invalid setting simply places nothing.
placed, _ := Places(m, with.Settings[m.Module])
for _, r := range m.Resources { for _, r := range m.Resources {
if fmt.Sprint(r["type"]) != "directory" { if fmt.Sprint(r["type"]) != "directory" {
continue continue
} }
id := fmt.Sprint(r["id"]) id := fmt.Sprint(r["id"])
if p, said := placed[id]; said {
dirs[id] = p.Path
continue
}
if path, stated := r["path"].(string); stated && path != "" { if path, stated := r["path"].(string); stated && path != "" {
if strings.HasPrefix(path, "${dir:") {
beneath = append(beneath, r)
continue
}
dirs[id] = strings.TrimRight(path, "/") dirs[id] = strings.TrimRight(path, "/")
continue continue
} }
if place, said := r["place"].(string); said && place == "." { switch place, _ := r["place"].(string); place {
case placeOwn:
dirs[id] = dataRoot(with) + "/" + m.Module dirs[id] = dataRoot(with) + "/" + m.Module
continue case placeMesh:
} dirs[id] = dataRoot(with) + "/mesh/" + m.Module
default:
dirs[id] = dataRoot(with) + "/" + m.Module + "/" + id dirs[id] = dataRoot(with) + "/" + m.Module + "/" + id
} }
}
for _, r := range beneath {
path := strings.TrimRight(r["path"].(string), "/")
// A reference to no directory is left as written and refused where the resource is
// placed (dirInto), with the message that names what exists.
filled, _ := dirFill(path, dirs, m.Module)
dirs[fmt.Sprint(r["id"])] = filled
}
return dirs return dirs
} }
@@ -119,9 +161,17 @@ func placedManifest(m Manifest, with Rendering) (Manifest, error) {
if m.Secrets, err = fillMap(m.Secrets); err != nil { if m.Secrets, err = fillMap(m.Secrets); err != nil {
return m, err return m, err
} }
if m.OwnSecrets, err = fillMap(m.OwnSecrets); err != nil { if len(m.OwnSecrets) > 0 {
own := make(OwnSecrets, len(m.OwnSecrets))
for name, s := range m.OwnSecrets {
filled, err := dirFill(s.Path, dirs, m.Module)
if err != nil {
return m, err return m, err
} }
own[name] = OwnSecret{Path: filled, Taken: s.Taken}
}
m.OwnSecrets = own
}
if m.Grants, err = fillMap(m.Grants); err != nil { if m.Grants, err = fillMap(m.Grants); err != nil {
return m, err return m, err
} }
@@ -244,10 +294,11 @@ func (m Manifest) unknownDirRefs() []string {
"%s states both path and place on %v — a stated path IS the placement", "%s states both path and place on %v — a stated path IS the placement",
m.Module, r["id"])) m.Module, r["id"]))
} }
if place != "." { if place != placeOwn && place != placeMesh {
problems = append(problems, fmt.Sprintf( problems = append(problems, fmt.Sprintf(
"%s says place %q on %v, and the only place is %q — the assignment's own root", "%s says place %q on %v, and the places are %q — the assignment's own root — and "+
m.Module, place, r["id"], ".")) "%q — where the mesh keeps what it writes for the module",
m.Module, place, r["id"], placeOwn, placeMesh))
} }
} }
seen := map[string]bool{} seen := map[string]bool{}
@@ -292,7 +343,7 @@ func (m Manifest) unknownDirRefs() []string {
} }
maps := map[string]map[string]string{ maps := map[string]map[string]string{
"receives": m.Receives, "binds": m.Binds, "secrets": m.Secrets, "receives": m.Receives, "binds": m.Binds, "secrets": m.Secrets,
"own-secrets": m.OwnSecrets, "grants": m.Grants, "own-secrets": m.OwnSecrets.Paths(), "grants": m.Grants,
} }
for field, entries := range maps { for field, entries := range maps {
for _, value := range entries { for _, value := range entries {
+77 -4
View File
@@ -164,7 +164,7 @@ func TestTheManifestsMapsArePlaced(t *testing.T) {
}, },
Binds: map[string]string{"route": "${dir:state}/route.json"}, Binds: map[string]string{"route": "${dir:state}/route.json"},
Secrets: map[string]string{"mongodb-database": "${dir:state}/database.secret"}, Secrets: map[string]string{"mongodb-database": "${dir:state}/database.secret"},
OwnSecrets: map[string]string{"admin-key": "${dir:state}/admin-key.secret"}, OwnSecrets: OwnSecrets{"admin-key": {Path: "${dir:state}/admin-key.secret"}},
Receives: map[string]string{"route": "${dir:state}/grants/mesh.json"}, Receives: map[string]string{"route": "${dir:state}/grants/mesh.json"},
} }
placed, err := placedManifest(m, Rendering{}) placed, err := placedManifest(m, Rendering{})
@@ -177,7 +177,7 @@ func TestTheManifestsMapsArePlaced(t *testing.T) {
if placed.Secrets["mongodb-database"] != "/var/lib/photos/database.secret" { if placed.Secrets["mongodb-database"] != "/var/lib/photos/database.secret" {
t.Fatalf("secrets are placed; got %v", placed.Secrets) t.Fatalf("secrets are placed; got %v", placed.Secrets)
} }
if placed.OwnSecrets["admin-key"] != "/var/lib/photos/admin-key.secret" { if placed.OwnSecrets["admin-key"].Path != "/var/lib/photos/admin-key.secret" {
t.Fatalf("own-secrets are placed; got %v", placed.OwnSecrets) t.Fatalf("own-secrets are placed; got %v", placed.OwnSecrets)
} }
if placed.Receives["route"] != "/var/lib/photos/grants/mesh.json" { if placed.Receives["route"] != "/var/lib/photos/grants/mesh.json" {
@@ -216,8 +216,48 @@ func TestPlaceIsValidatedAtTheManifest(t *testing.T) {
wrong := Manifest{Module: "x", Resources: []map[string]any{ wrong := Manifest{Module: "x", Resources: []map[string]any{
{"id": "d", "type": "directory", "place": "sub/dir"}, {"id": "d", "type": "directory", "place": "sub/dir"},
}} }}
if got := wrong.unknownDirRefs(); len(got) != 1 || !strings.Contains(got[0], `the only place is "."`) { if got := wrong.unknownDirRefs(); len(got) != 1 || !strings.Contains(got[0], `the places are "."`) {
t.Fatalf("a place that is not the root refuses; got %v", got) t.Fatalf("a place that is neither root refuses; got %v", got)
}
}
func TestTheMeshsDirectoryForAModuleIsAPlace(t *testing.T) {
// novox/hq issue 174. What the mesh writes for a module — its bus credential, its bindings —
// is the mesh's plumbing under <root>/mesh/<module>, and the definition names it by id.
m := Manifest{Module: "umami",
Resources: []map[string]any{
{"id": "mesh-state", "type": "directory", "place": "mesh"},
{"id": "state", "type": "directory", "place": "."},
{"id": "server", "type": "container", "image": "x@sha256:aa",
"volumes": []any{"${dir:mesh-state}/broker:/run/secrets/broker:ro"}},
},
OwnSecrets: OwnSecrets{"broker": {Path: "${dir:mesh-state}/broker"}},
Binds: map[string]string{"route": "${dir:state}/route.json"},
}
if got := m.unknownDirRefs(); len(got) != 0 {
t.Fatalf("place %q is a place; got %v", "mesh", got)
}
dirs := dirsFor(m, Rendering{})
if dirs["mesh-state"] != "/var/lib/mesh/umami" || dirs["state"] != "/var/lib/umami" {
t.Fatalf("the mesh's directory sits beside the module's, not in it; got %v", dirs)
}
dirs = dirsFor(m, Rendering{DataRoot: "/srv"})
if dirs["mesh-state"] != "/srv/mesh/umami" {
t.Fatalf("a node's root moves the mesh's files with the module's; got %v", dirs)
}
placed, err := placedManifest(m, Rendering{})
if err != nil {
t.Fatal(err)
}
if placed.OwnSecrets["broker"].Path != "/var/lib/mesh/umami/broker" {
t.Fatalf("own-secrets are placed under the mesh's directory; got %v", placed.OwnSecrets)
}
container := shallowCopy(m.Resources[2])
if err := dirInto(container, dirsFor(m, Rendering{}), m.Module); err != nil {
t.Fatal(err)
}
if container["volumes"].([]any)[0] != "/var/lib/mesh/umami/broker:/run/secrets/broker:ro" {
t.Fatalf("the mount's host side is placed; got %v", container["volumes"])
} }
} }
@@ -250,3 +290,36 @@ func shallowCopy(resource map[string]any) map[string]any {
} }
return copied return copied
} }
func TestADirectoryBeneathAPlacedOneIsPlacedWithIt(t *testing.T) {
// An adopted layout keeps a subdirectory the predecessor made under the mesh's directory
// (issue 174: a forge's runtime state, a manager's output). Stated as beneath the placed one,
// it moves with it — a node's root moves both, and the definition names no host path.
m := Manifest{Module: "gitea", Resources: []map[string]any{
{"id": "mesh-state", "type": "directory", "place": "mesh"},
{"id": "runtime-state", "type": "directory", "path": "${dir:mesh-state}/state"},
{"id": "server", "type": "container", "image": "x@sha256:aa",
"volumes": []any{"${dir:runtime-state}:/data"}},
}}
if got := m.unknownDirRefs(); len(got) != 0 {
t.Fatalf("a path beneath a placed directory is well formed; got %v", got)
}
dirs := dirsFor(m, Rendering{DataRoot: "/srv"})
if dirs["runtime-state"] != "/srv/mesh/gitea/state" {
t.Fatalf("the subdirectory follows the placed one; got %v", dirs)
}
sub := shallowCopy(m.Resources[1])
if err := dirInto(sub, dirs, m.Module); err != nil {
t.Fatal(err)
}
if sub["path"] != "/srv/mesh/gitea/state" {
t.Fatalf("the directory resource itself is resolved; got %v", sub["path"])
}
container := shallowCopy(m.Resources[2])
if err := dirInto(container, dirs, m.Module); err != nil {
t.Fatal(err)
}
if container["volumes"].([]any)[0] != "/srv/mesh/gitea/state:/data" {
t.Fatalf("a reference to the subdirectory resolves whole; got %v", container["volumes"])
}
}
+12
View File
@@ -444,6 +444,18 @@ func AsNftables(rules []Rule, mesh []string, outward bool, foundation []int,
b.WriteString("\t\t# this machine's own guests reaching outward: not a port opened to anybody\n") b.WriteString("\t\t# this machine's own guests reaching outward: not a port opened to anybody\n")
b.WriteString(fmt.Sprintf("\t\tiifname != { %s } accept\n", inward)) b.WriteString(fmt.Sprintf("\t\tiifname != { %s } accept\n", inward))
} }
// **The mesh passing through, not arriving.** A machine the mesh routes through — the hub, for
// every path between machines that are not co-located (novox/hq ADR 0007) — relays a packet that
// came in on the tunnel and leaves on it again, addressed to another machine of the mesh. That is
// no port of this machine's: the machine it is for filters it against its own rules. Without
// this, the chain below judged a relayed packet by this machine's own published ports, so two
// machines behind the hub reached each other only on ports the hub happened to publish for itself
// (novox/hq issue 196). In and out on the tunnel both: a packet off the tunnel for this machine's
// own containers leaves by a bridge, and still meets the rules below.
if tunnel != "" {
b.WriteString("\t\t# the mesh passing through to another of its machines, which filters it itself\n")
b.WriteString(fmt.Sprintf("\t\tiifname %q oifname %q accept\n", tunnel, tunnel))
}
if len(rules) > 0 { if len(rules) > 0 {
b.WriteString("\n") b.WriteString("\n")
+32 -1
View File
@@ -419,7 +419,7 @@ func TestWhatTheMeshComputesIsAppliedBeforeWhatTheModuleDeclared(t *testing.T) {
func TestAComputedModuleStillGetsWhatTheMeshMadeForIt(t *testing.T) { func TestAComputedModuleStillGetsWhatTheMeshMadeForIt(t *testing.T) {
r := Resolution{Modules: []Manifest{{ r := Resolution{Modules: []Manifest{{
Module: "networking", Computed: "mesh-network", Module: "networking", Computed: "mesh-network",
OwnSecrets: map[string]string{"key": "/var/lib/mesh/key"}, OwnSecrets: OwnSecrets{"key": {Path: "/var/lib/mesh/key"}},
}}} }}}
out, err := r.Declaration(Rendering{ out, err := r.Declaration(Rendering{
Needed: map[string]map[string]string{"networking": {"key": "sealed"}}, Needed: map[string]map[string]string{"networking": {"key": "sealed"}},
@@ -887,3 +887,34 @@ func TestAPublicPortNeedsNoGuestLine(t *testing.T) {
t.Fatalf("a public port was given a guest line it does not need:\n%s", nft) t.Fatalf("a public port was given a guest line it does not need:\n%s", nft)
} }
} }
// **The hub relays the mesh** (novox/hq ADR 0007, issue 196). Two machines that are not co-located
// reach each other through the hub, so the hub forwards a packet that arrives on the tunnel and
// leaves on it. The forward chain judged that packet by the hub's own published ports, and two
// machines behind the hub reached each other only on the ports the hub happened to publish.
//
// Measured: from one home machine to another through the hub, 17 of 55 ports answered, and they
// were exactly the hub's own; the SYN for the rest never left the hub.
func TestTheMeshPassingThroughIsRelayedNotJudgedAsThisMachines(t *testing.T) {
nft := AsNftables(nil, []string{"10.42.0.1", "10.42.0.2"}, false, nil, []string{"eth0"}, "mesh0")
relay := `iifname "mesh0" oifname "mesh0" accept`
if !strings.Contains(chainBody(t, nft, "forward"), relay) {
t.Errorf("the forward chain does not relay the mesh through this machine:\n%s", chainBody(t, nft, "forward"))
}
// Relaying is not receiving: nothing in the input chain opens because of it.
if strings.Contains(chainBody(t, nft, "input"), "oifname") {
t.Errorf("the input chain names an outgoing interface, which no packet for this machine has:\n%s",
chainBody(t, nft, "input"))
}
// And off the tunnel into this machine's own containers is still judged: the tunnel is not
// accepted wholesale, only in and out on it.
if strings.Contains(chainBody(t, nft, "forward"), `iifname "mesh0" accept`) {
t.Errorf("the forward chain accepts everything off the tunnel:\n%s", chainBody(t, nft, "forward"))
}
// A machine with no tunnel relays nothing, and names no interface it does not have.
alone := AsNftables(nil, nil, false, nil, []string{"eth0"}, "")
if strings.Contains(alone, "oifname") {
t.Errorf("a machine with no tunnel was given a relay rule:\n%s", alone)
}
}
+21
View File
@@ -107,6 +107,27 @@ func TestCanHoldJudgesClaimScopeAndWhatTheSeatDelivers(t *testing.T) {
if err := CanHold(cannotAnswer, seat); err == nil || !strings.Contains(err.Error(), `does not provide "mesh-bus"`) { if err := CanHold(cannotAnswer, seat); err == nil || !strings.Contains(err.Error(), `does not provide "mesh-bus"`) {
t.Fatalf("a holder that cannot answer for the seat was allowed: %v", err) t.Fatalf("a holder that cannot answer for the seat was allowed: %v", err)
} }
// A holder's own tools need not be the seat's verbs: the claim may name what it serves for the
// role (ADR 0160), and then only those count — and only the seat's verbs may be named.
promising := seat
promising.Serves = []Verb{{Name: "databases"}, {Name: "query"}}
engine := newBroker()
engine.Tools = []string{"engine_list_databases", "engine_query"}
if err := CanHold(engine, promising); err == nil || !strings.Contains(err.Error(), "does not serve databases, query") {
t.Fatalf("a holder whose tools are not the seat's verbs was allowed without saying what it serves: %v", err)
}
engine.Claims[0].Serves = []string{"databases", "query"}
if err := CanHold(engine, promising); err != nil {
t.Fatalf("a claim naming the seat's verbs was refused: %v", err)
}
engine.Claims[0].Serves = []string{"databases"}
if err := CanHold(engine, promising); err == nil || !strings.Contains(err.Error(), "does not serve query") {
t.Fatalf("a claim naming half the verbs was allowed: %v", err)
}
engine.Claims[0].Serves = []string{"databases", "query", "engine_query"}
if err := CanHold(engine, promising); err == nil || !strings.Contains(err.Error(), "does not promise") {
t.Fatalf("a claim naming a verb the seat never promised was allowed: %v", err)
}
// And the judgement follows the store's row, not a compiled copy. // And the judgement follows the store's row, not a compiled copy.
busSeatDelivering(t, "amqp") busSeatDelivering(t, "amqp")
seat, _ = SeatNamed("mesh-broker") seat, _ = SeatNamed("mesh-broker")
+12 -2
View File
@@ -1,6 +1,8 @@
package catalogue package catalogue
import ( import (
"crypto/sha256"
"encoding/hex"
"fmt" "fmt"
"sort" "sort"
"strings" "strings"
@@ -43,8 +45,16 @@ func jailsInto(modules []Manifest, j *Jailing) []map[string]any {
out := make([]map[string]any, 0, len(jails)+1) out := make([]map[string]any, 0, len(jails)+1)
for _, d := range jails { for _, d := range jails {
fmt.Fprintf(&composed, "\n# from %s\n[%s]\nenabled = true\nfilter = %s\n%s\n", // **The filter's digest rides in the jail file.** fail2ban is restarted when this file
d.module, d.jail.Name, d.jail.Name, strings.TrimRight(d.jail.Jail, "\n")) // changes, and the filter is a file of its own: a module that changed only what a failure
// looks like rewrote the filter on disk and left the running jail on the old pattern, with
// nothing said (novox/hq issue 191's rollout found it on gitea's sshd). Naming the filter's
// digest here makes a changed pattern a changed jail file, so the restart the service
// already takes on it covers the filter too.
sum := sha256.Sum256([]byte(d.jail.Failregex))
fmt.Fprintf(&composed, "\n# from %s, filter %s\n[%s]\nenabled = true\nfilter = %s\n%s\n",
d.module, hex.EncodeToString(sum[:])[:12], d.jail.Name, d.jail.Name,
strings.TrimRight(d.jail.Jail, "\n"))
// The filter is a file of its own, named as the jail's filter= references it. // The filter is a file of its own, named as the jail's filter= references it.
out = append(out, map[string]any{ out = append(out, map[string]any{
"id": "filter-" + d.jail.Name, "id": "filter-" + d.jail.Name,
+26
View File
@@ -44,3 +44,29 @@ func TestTheComposedJailFileIsWrittenEvenWhenEmpty(t *testing.T) {
t.Fatalf("the empty composed jail file was not written alone: %v", files) t.Fatalf("the empty composed jail file was not written alone: %v", files)
} }
} }
// A changed pattern restarts fail2ban (novox/hq issue 191's rollout): the service restarts when the
// composed jail file changes, and the filter is a file of its own, so the jail file names the
// filter's digest. Changing only the failregex must change the jail file; the same pattern must not.
func TestAChangedFilterChangesTheJailFile(t *testing.T) {
jailFile := func(failregex string) string {
modules := []Manifest{
{Module: "fail2ban", Jailing: &Jailing{Into: "/etc/fail2ban/jail.d/mesh.conf", FilterInto: "/etc/fail2ban/filter.d"}},
{Module: "gitea", Jails: []Jail{{Name: "gitea", Failregex: failregex, Jail: "port = 222"}}},
}
for _, f := range jailsInto(modules, modules[0].Jailing) {
if f["id"] == ComposedJailsID() {
return f["content"].(string)
}
}
t.Fatal("no composed jail file")
return ""
}
before := jailFile("web login failed from <HOST>")
if again := jailFile("web login failed from <HOST>"); again != before {
t.Errorf("the same pattern composed a different jail file, which would restart fail2ban for nothing")
}
if after := jailFile("web login failed from <HOST>\n Invalid user .* from <HOST>"); after == before {
t.Errorf("a changed pattern left the jail file as it was, so fail2ban keeps the old filter:\n%s", after)
}
}
+5 -2
View File
@@ -102,8 +102,11 @@ func accountHomeOf(account, home string) string {
func machineInto(resource map[string]any, facts map[string]string, module string) error { func machineInto(resource map[string]any, facts map[string]string, module string) error {
// Content, and now the path and owner too: a module that writes into a person's home names it // Content, and now the path and owner too: a module that writes into a person's home names it
// with ${machine:account-home} and ${machine:account}, which it cannot know until assigned // with ${machine:account-home} and ${machine:account}, which it cannot know until assigned
// (novox/hq to-be 29), the same reason its content names ${machine:address}. // (novox/hq to-be 29), the same reason its content names ${machine:address}. And the name a
for _, field := range []string{"path", "owner", "content"} { // `user` shape sets the login shell of, and the user a user-scoped unit or a process runs as:
// the shell module makes the operator's account its holder's login shell, and the desktop's
// watchers run as that account (novox/hq ADR 0176, ADR 0177) — neither can name the person.
for _, field := range []string{"path", "owner", "content", "name", "user"} {
s, ok := resource[field].(string) s, ok := resource[field].(string)
if !ok { if !ok {
continue continue
+304 -15
View File
@@ -52,6 +52,22 @@ type Claim struct {
Name string `json:"name"` Name string `json:"name"`
// Scope defaults to the node, which is where nearly everything singular is singular. // Scope defaults to the node, which is where nearly everything singular is singular.
Scope string `json:"scope,omitempty"` Scope string `json:"scope,omitempty"`
// Serves names the seat's verbs this module implements for the role, when its own tools are
// not the seat's (novox/hq ADR 0159, 0160): the store's `databases` is not postgres's
// `postgres_list_databases`, and a holder may well serve both. The runtime serves an
// implementation registered under the seat's name on the seat's subjects. Absent, the
// module's own `tools` must list every verb the seat promises, which is how a module named
// like its seat — the catalogue, the records — says they are one and the same.
Serves []string `json:"serves,omitempty"`
}
// ServesFor is what this claim offers a seat's protocol: the verbs it names, else the module's
// own tools.
func (c Claim) ServesFor(m Manifest) []string {
if len(c.Serves) > 0 {
return c.Serves
}
return m.Tools
} }
// At is this claim's scope, with the default applied. // At is this claim's scope, with the default applied.
@@ -91,8 +107,13 @@ const (
// If the path is absent when a machine applies, the host refuses clearly rather than creating it: // If the path is absent when a machine applies, the host refuses clearly rather than creating it:
// the mesh does not own it, so conjuring it would be a lie the host then acts on. // the mesh does not own it, so conjuring it would be a lie the host then acts on.
type Access struct { type Access struct {
// Path is the absolute path on the machine, as the operator provides it. // ID is the name the module gives this access, which the assignment places
Path string `json:"path"` // (`accesses: {<id>: <path>}`, novox/hq ADR 0112, issue 153) and the module's mounts name as
// ${access:<id>}. The shape a definition should use: it names no path of any machine.
ID string `json:"id,omitempty"`
// Path is the absolute path on the machine. A definition carrying one names an installation;
// tolerated as the default the assignment may replace, for accesses declared before ids.
Path string `json:"path,omitempty"`
// Mode is "read" or "read-write". Absent narrows to read. // Mode is "read" or "read-write". Absent narrows to read.
Mode string `json:"mode,omitempty"` Mode string `json:"mode,omitempty"`
} }
@@ -122,6 +143,38 @@ type Offer struct {
Name string `json:"name"` Name string `json:"name"`
// Scope defaults to the node, which is where most things must be to be usable. // Scope defaults to the node, which is where most things must be to be usable.
Scope string `json:"scope,omitempty"` Scope string `json:"scope,omitempty"`
// Credential, when set, says this provision's credential is one of the provider's own secrets,
// shared by every consumer (novox/hq ADR 0158): software that holds one password or one key
// cannot give each consumer a login of its own. The named secret must say how it is taken.
Credential *OfferCredential `json:"credential,omitempty"`
}
// OfferCredential names which of the provider's own secrets a provision's consumers receive.
type OfferCredential struct {
Own string `json:"own"`
}
// SharedCredentialOf is the own secret an offer of this module names as the provision's credential,
// and whether it names one.
func (m Manifest) SharedCredentialOf(provision string) (string, bool) {
for _, o := range m.Provides {
if o.Name == provision && o.Credential != nil && o.Credential.Own != "" {
return o.Credential.Own, true
}
}
return "", false
}
// ProvisionsSharing is every provision of this module whose credential is the named own secret.
func (m Manifest) ProvisionsSharing(own string) []string {
var out []string
for _, o := range m.Provides {
if o.Credential != nil && o.Credential.Own == own {
out = append(out, o.Name)
}
}
sort.Strings(out)
return out
} }
// At is this offer's scope, with the default applied. // At is this offer's scope, with the default applied.
@@ -142,24 +195,28 @@ func (o *Offer) UnmarshalJSON(raw []byte) error {
var full struct { var full struct {
Name string `json:"name"` Name string `json:"name"`
Scope string `json:"scope,omitempty"` Scope string `json:"scope,omitempty"`
Credential *OfferCredential `json:"credential,omitempty"`
} }
if err := json.Unmarshal(raw, &full); err != nil { dec := json.NewDecoder(bytes.NewReader(raw))
return fmt.Errorf("a provided name is either a string or {name, scope}: %w", err) dec.DisallowUnknownFields()
if err := dec.Decode(&full); err != nil {
return fmt.Errorf("a provided name is either a string or {name, scope, credential}: %w", err)
} }
o.Name, o.Scope = full.Name, full.Scope o.Name, o.Scope, o.Credential = full.Name, full.Scope, full.Credential
return nil return nil
} }
// MarshalJSON writes back the short form when there is nothing else to say, so a manifest that // MarshalJSON writes back the short form when there is nothing else to say, so a manifest that
// went through the mesh comes out looking like the one that went in. // went through the mesh comes out looking like the one that went in.
func (o Offer) MarshalJSON() ([]byte, error) { func (o Offer) MarshalJSON() ([]byte, error) {
if o.Scope == "" { if o.Scope == "" && o.Credential == nil {
return json.Marshal(o.Name) return json.Marshal(o.Name)
} }
return json.Marshal(struct { return json.Marshal(struct {
Name string `json:"name"` Name string `json:"name"`
Scope string `json:"scope"` Scope string `json:"scope,omitempty"`
}{o.Name, o.Scope}) Credential *OfferCredential `json:"credential,omitempty"`
}{o.Name, o.Scope, o.Credential})
} }
// Manifest is everything a module says about itself. // Manifest is everything a module says about itself.
@@ -252,6 +309,12 @@ type Manifest struct {
// module claiming a seat answers what that seat's protocol promises (novox/hq ADR 0118). // module claiming a seat answers what that seat's protocol promises (novox/hq ADR 0118).
Tools []string `json:"tools,omitempty"` Tools []string `json:"tools,omitempty"`
// Instances says whether this module's instances are the same anywhere — `interchangeable` —
// so a call that names no machine may be answered by any of them (novox/hq ADR 0160). A fact
// about the software, not about the bus: a stateless web tool says it; a database does not,
// and its instances are then each addressed by machine, never confused for one another.
Instances string `json:"instances,omitempty"`
// Invokes are the tools this module calls, each `<module>.<tool>` or a role's `seat:<seat>.<verb>`, // Invokes are the tools this module calls, each `<module>.<tool>` or a role's `seat:<seat>.<verb>`,
// or the single entry `*` for every tool on the mesh (novox/hq ADR 0152, ADR 0154). // or the single entry `*` for every tool on the mesh (novox/hq ADR 0152, ADR 0154).
// //
@@ -385,7 +448,16 @@ type Manifest struct {
// module running on three machines has three passwords and the mesh can read none of them. A // module running on three machines has three passwords and the mesh can read none of them. A
// manifest carrying one instead would put the same secret on every machine that ever runs the // manifest carrying one instead would put the same secret on every machine that ever runs the
// module, in a file anybody can read, for ever. // module, in a file anybody can read, for ever.
OwnSecrets map[string]string `json:"own-secrets,omitempty"` //
// **And how the module takes it** (novox/hq ADR 0114, issue 180): `"admin": "<path>"` says
// where and nothing else; `"admin": {"path": "<path>", "taken": "at-start"}` says the module
// reads the file when it starts, so the mesh may rotate it by making a new value and starting
// the module again; `"taken": "applied"` says the module's own code applies it to a backend
// that takes it only once, so a rotation must be staged beside the current value — the form the
// mesh does not build yet, and refuses by name. A secret that says neither is not rotated by
// the mesh: the one fault worse than an unrotated credential is a rotated one the software
// never saw.
OwnSecrets OwnSecrets `json:"own-secrets,omitempty"`
// SecretsOwner is who the files holding this module's secrets belong to on the machine — // SecretsOwner is who the files holding this module's secrets belong to on the machine —
// `uid:gid`, or a name — when its process is not root. // `uid:gid`, or a name — when its process is not root.
@@ -500,6 +572,44 @@ type Manifest struct {
// a module that could ask for it could read every credential on the bus — and the claim on // a module that could ask for it could read every credential on the bus — and the claim on
// `mesh-broker` is what authorises it, checked from this manifest alone. // `mesh-broker` is what authorises it, checked from this manifest alone.
BusUsers string `json:"bus-users,omitempty"` BusUsers string `json:"bus-users,omitempty"`
// Bundles are this module's compiled bundles as the build produced them: what each is called,
// where it is, what it hashes to, what language it is in and which files a tool runtime loads
// from it (novox/hq ADR 0175, to-be 38).
//
// **Derived, never written.** The manifest in a repository says `build.artifacts`; the manifest
// the mesh holds says what came out, the way a resource naming an artifact comes to name a
// digest. Kept here because a tools bundle is referenced by no resource of the module's own —
// the node's runtime loads it, and the runtime is composed by the mesh — so without this the
// resolved manifest would carry no trace of the one artifact the runtime needs. A repository
// manifest that writes this beside a build is refused: it would be stating the build's output
// by hand.
Bundles []Bundle `json:"bundles,omitempty"`
}
// Bundle is one compiled bundle after it exists, as the resolved manifest carries it.
type Bundle struct {
Name string `json:"name"`
// Source is where a machine fetches it, kept without the store's address like every reference
// the mesh records (artifacts.go); Digest is what it must hash to.
Source string `json:"source"`
Digest string `json:"digest"`
// Language is what it was compiled from, which is what says how it is run.
Language string `json:"language,omitempty"`
// Entrypoints are the compiled files it was built around, relative to its root.
Entrypoints []string `json:"entrypoints,omitempty"`
// Loads are the entrypoints a node's tool runtime imports from it: what the artifact said, or
// every entrypoint for a module declaring tools that said nothing. Empty for a bundle that is
// run rather than loaded.
Loads []string `json:"loads,omitempty"`
// Env is what the artifact said it is given (ADR 0192), as written; composed per machine.
Env map[string]string `json:"env,omitempty"`
// Launchers are the executables the build wrote beside its entrypoints, by entrypoint (novox/hq
// ADR 0193). A bundle built before them has none, and is served as it was built.
Launchers map[string]string `json:"launchers,omitempty"`
// Binary is the executable a bundle compiled to a binary is, at its root (novox/hq ADR 0193):
// what runs it, where an interpreted bundle names an interpreter and an entrypoint.
Binary string `json:"binary,omitempty"`
} }
// Build says how to produce this module's artifacts from its source. // Build says how to produce this module's artifacts from its source.
@@ -636,6 +746,21 @@ type Artifact struct {
// somebody adds a helper. An empty list is a bundle that is run rather than loaded — a // somebody adds a helper. An empty list is a bundle that is run rather than loaded — a
// provisioner or a step, named by whatever runs it. // provisioner or a step, named by whatever runs it.
Entrypoints []string `json:"entrypoints,omitempty"` Entrypoints []string `json:"entrypoints,omitempty"`
// Loads are the entrypoints of this bundle the node's tool runtime loads (novox/hq ADR 0175,
// to-be 38): the module's tool code, each file registering its tools as it is imported. A
// subset of Entrypoints, for a bundle that also carries things that are RUN — a daemon, a
// step, a report — and must not have them imported into the runtime.
//
// Absent means every entrypoint, for a module that declares `tools`: a bundle holding the
// module's tools and nothing else is the ordinary case and should not have to say the same
// list twice. A module declaring no tools has nothing the runtime loads, whatever it compiles.
Loads []string `json:"loads,omitempty"`
// Env is what a tools bundle is given on a machine (novox/hq ADR 0192): words and their values,
// paths and constants composed with ${dir:…} and ${port:…} exactly as a container's environment
// is, never a secret's content. The node's runtime hands it to this bundle and to no other.
Env map[string]string `json:"env,omitempty"`
} }
// Kinds an artifact may be. // Kinds an artifact may be.
@@ -1123,11 +1248,33 @@ func ParseManifest(raw []byte) (Manifest, error) {
m.Module, r)) m.Module, r))
} }
} }
if m.Instances != "" && m.Instances != InstancesInterchangeable {
problems = append(problems, fmt.Sprintf(
"%s says its instances are %q; the one word is %q, for a module that is the same on every machine",
m.Module, m.Instances, InstancesInterchangeable))
}
for _, offer := range m.Provides { for _, offer := range m.Provides {
p := offer.Name p := offer.Name
if !name.MatchString(p) { if !name.MatchString(p) {
problems = append(problems, fmt.Sprintf("%q is not a usable name to provide", p)) problems = append(problems, fmt.Sprintf("%q is not a usable name to provide", p))
} }
if offer.Credential != nil {
own, declared := m.OwnSecrets[offer.Credential.Own]
switch {
case offer.Credential.Own == "":
problems = append(problems, fmt.Sprintf(
"%s provides %q with a credential that names no own secret", m.Module, p))
case !declared:
problems = append(problems, fmt.Sprintf(
"%s provides %q with its own secret %q as the credential, and declares no such secret",
m.Module, p, offer.Credential.Own))
case own.Taken == "":
problems = append(problems, fmt.Sprintf(
"%s provides %q with its own secret %q as the credential every consumer receives, so "+
"the secret must say how the module takes it: \"taken\": \"at-start\" or \"applied\" (ADR 0158)",
m.Module, p, offer.Credential.Own))
}
}
if instead, generic := engineGeneric[p]; generic { if instead, generic := engineGeneric[p]; generic {
// A consumer is written against an engine, not a role (novox/hq ADR 0027). Providing // A consumer is written against an engine, not a role (novox/hq ADR 0027). Providing
// the role means a requirement for it matches any engine, resolves as satisfied, and // the role means a requirement for it matches any engine, resolves as satisfied, and
@@ -1239,6 +1386,15 @@ func ParseManifest(raw []byte) (Manifest, error) {
// //
// Refused here because the alternative is a build that never returns, on a mesh new enough // Refused here because the alternative is a build that never returns, on a mesh new enough
// that nobody is watching it yet. // that nobody is watching it yet.
if m.Build != nil && len(m.Bundles) > 0 {
// The output of a build, written beside the build that produces it (ADR 0175). A resource
// naming a digest beside an `artifact` would be the same mistake, and is caught the same way:
// what the mesh derives, a repository does not state.
problems = append(problems, fmt.Sprintf(
"%s writes `bundles` beside its build. The mesh derives that from what the build "+
"produced; a manifest states `build.artifacts` and nothing about what came out",
m.Module))
}
if m.Build != nil && len(m.Build.Artifacts) > 0 { if m.Build != nil && len(m.Build.Artifacts) > 0 {
for _, o := range m.Offers() { for _, o := range m.Offers() {
if o != ArtifactStoreProvision { if o != ArtifactStoreProvision {
@@ -1414,14 +1570,20 @@ func ParseManifest(raw []byte) (Manifest, error) {
} }
} }
} }
for name, where := range m.OwnSecrets { for name, own := range m.OwnSecrets {
if !placedOrAbsolute(where) { if !placedOrAbsolute(own.Path) {
problems = append(problems, fmt.Sprintf( problems = append(problems, fmt.Sprintf(
"%s needs %q at %q, which is neither an absolute path nor a placed one", m.Module, name, where)) "%s needs %q at %q, which is neither an absolute path nor a placed one", m.Module, name, own.Path))
} }
if name == "" { if name == "" {
problems = append(problems, m.Module+" needs a secret with no name") problems = append(problems, m.Module+" needs a secret with no name")
} }
if own.Taken != "" && own.Taken != TakenAtStart && own.Taken != TakenApplied {
problems = append(problems, fmt.Sprintf(
"%s says its secret %q is taken %q; a secret is taken %q (read when the module starts) "+
"or %q (applied by the module's own code to a backend that takes it once)",
m.Module, name, own.Taken, TakenAtStart, TakenApplied))
}
} }
localOf := map[string]string{} localOf := map[string]string{}
for _, to := range m.SecretRequirements() { for _, to := range m.SecretRequirements() {
@@ -1525,7 +1687,16 @@ func ParseManifest(raw []byte) (Manifest, error) {
} }
} }
for _, a := range m.Accesses { for _, a := range m.Accesses {
if !strings.HasPrefix(a.Path, "/") { if a.ID == "" && a.Path == "" {
problems = append(problems, fmt.Sprintf(
"%s declares an access with neither an id nor a path — an id, which the assignment places",
m.Module))
}
if a.ID != "" && !accessRef.MatchString("${access:"+a.ID+"}") {
problems = append(problems, fmt.Sprintf(
"%s accesses %q; an access id is lowercase letters, digits and dashes", m.Module, a.ID))
}
if a.Path != "" && !strings.HasPrefix(a.Path, "/") {
problems = append(problems, fmt.Sprintf( problems = append(problems, fmt.Sprintf(
"%s accesses %q, which is not an absolute path", m.Module, a.Path)) "%s accesses %q, which is not an absolute path", m.Module, a.Path))
} }
@@ -1557,6 +1728,8 @@ func ParseManifest(raw []byte) (Manifest, error) {
// the time it sees the mount it is being asked to create the directory, which it can do. // the time it sees the mount it is being asked to create the directory, which it can do.
problems = append(problems, m.undeclaredMounts()...) problems = append(problems, m.undeclaredMounts()...)
problems = append(problems, m.unknownDirRefs()...) problems = append(problems, m.unknownDirRefs()...)
problems = append(problems, m.unknownAccessRefs()...)
problems = append(problems, m.jailProblems()...)
for i, r := range m.Resources { for i, r := range m.Resources {
id, _ := r["id"].(string) id, _ := r["id"].(string)
@@ -1659,6 +1832,46 @@ func (m Manifest) MachineSide(port int) (at int, mayAssign bool) {
var facilitiesOf = map[string][]string{ var facilitiesOf = map[string][]string{
// Both spellings: /var/run is a link to /run on every machine the mesh runs on. // Both spellings: /var/run is a link to /run on every machine the mesh runs on.
"container-runtime": {"/var/run/docker.sock", "/run/docker.sock"}, "container-runtime": {"/var/run/docker.sock", "/run/docker.sock"},
// The virtualisation daemon's socket, for the lab (novox/hq ADR 0172): it raises machines there.
"virtualisation": {"/var/lib/incus/unix.socket"},
}
// jailProblems is every jail this module declares that the machine's intrusion prevention would
// refuse (novox/hq ADR 0179).
//
// **Because one bad pattern stops every jail, not its own.** fail2ban expands `<HOST>` into a named
// capture group, so a pattern naming it twice is a duplicate group name, and the daemon refuses the
// whole configuration and exits — the machine keeps no bans at all, for any jail, including the one
// watching its ssh. Caught live on the control node the day this was built, where a proxy's pattern
// matched two shapes of refusal in one line. A pattern matches one shape; several shapes are several
// patterns, one per line, as fail2ban's own filters are written.
func (m Manifest) jailProblems() []string {
var problems []string
seen := map[string]bool{}
for _, j := range m.Jails {
switch {
case strings.TrimSpace(j.Name) == "":
problems = append(problems, m.Module+" declares a jail with no name")
case seen[j.Name]:
problems = append(problems, m.Module+" declares two jails called "+strconv.Quote(j.Name))
}
seen[j.Name] = true
if strings.TrimSpace(j.Failregex) == "" {
problems = append(problems, m.Module+"'s jail "+strconv.Quote(j.Name)+" says nothing a failed attempt looks like")
}
for _, line := range strings.Split(j.Failregex, "\n") {
if strings.TrimSpace(line) == "" {
continue
}
if n := strings.Count(line, "<HOST>"); n > 1 {
problems = append(problems, fmt.Sprintf("%s's jail %s names <HOST> %d times in one pattern; "+
"fail2ban reads it as one capture group and refuses the whole configuration, so the machine "+
"keeps no bans at all — write one pattern per shape, each naming <HOST> once",
m.Module, strconv.Quote(j.Name), n))
}
}
}
return problems
} }
// undeclaredMounts is every bind-mount source no declaration covers — see the check above. // undeclaredMounts is every bind-mount source no declaration covers — see the check above.
@@ -1675,8 +1888,8 @@ func (m Manifest) undeclaredMounts() []string {
claim(fmt.Sprint(r["path"])) claim(fmt.Sprint(r["path"]))
} }
} }
for _, where := range m.OwnSecrets { for _, own := range m.OwnSecrets {
claim(where) claim(own.Path)
} }
for _, to := range m.SecretRequirements() { for _, to := range m.SecretRequirements() {
for _, f := range m.SecretFiles(to) { for _, f := range m.SecretFiles(to) {
@@ -1700,6 +1913,12 @@ func (m Manifest) undeclaredMounts() []string {
claim(p) claim(p)
} }
} }
// The file a filter module's rule set is written to is declared by `filtering.into`: the mesh
// writes it, the module loads it, and the module's runtime may read it back to reload the
// mesh's own table (novox/hq ADR 0170).
if m.Filtering != nil {
claim(m.Filtering.Into)
}
// Under a declared directory is declared: a module that says where its data lives has said so // Under a declared directory is declared: a module that says where its data lives has said so
// for what it puts inside. // for what it puts inside.
covers := func(path string) bool { covers := func(path string) bool {
@@ -1811,3 +2030,73 @@ func invokeProblems(m Manifest) []string {
} }
return problems return problems
} }
// How a module takes one of its own secrets (ADR 0114): read from the file when it starts, or
// applied by its own code to a backend that takes it once.
const (
TakenAtStart = "at-start"
TakenApplied = "applied"
)
// OwnSecret is where one of a module's own secrets lands, and how the module takes it.
type OwnSecret struct {
Path string
Taken string
}
// OwnSecrets is a module's own secrets by name. On the wire each is a path, or an object naming
// the path and how it is taken; written back the way it was read, so a manifest the mesh holds
// keeps its bytes.
type OwnSecrets map[string]OwnSecret
func (o *OwnSecrets) UnmarshalJSON(raw []byte) error {
var entries map[string]json.RawMessage
if err := json.Unmarshal(raw, &entries); err != nil {
return err
}
out := make(OwnSecrets, len(entries))
for name, body := range entries {
var path string
if err := json.Unmarshal(body, &path); err == nil {
out[name] = OwnSecret{Path: path}
continue
}
var long struct {
Path string `json:"path"`
Taken string `json:"taken,omitempty"`
}
dec := json.NewDecoder(bytes.NewReader(body))
dec.DisallowUnknownFields()
if err := dec.Decode(&long); err != nil {
return fmt.Errorf("own-secrets.%s: a path, or {\"path\", \"taken\"}: %w", name, err)
}
out[name] = OwnSecret{Path: long.Path, Taken: long.Taken}
}
*o = out
return nil
}
func (o OwnSecrets) MarshalJSON() ([]byte, error) {
entries := make(map[string]any, len(o))
for name, s := range o {
if s.Taken == "" {
entries[name] = s.Path
continue
}
entries[name] = map[string]string{"path": s.Path, "taken": s.Taken}
}
return json.Marshal(entries)
}
// Paths is each own secret's path by name — the shape every placement and file walk reads.
func (o OwnSecrets) Paths() map[string]string {
out := make(map[string]string, len(o))
for name, s := range o {
out[name] = s.Path
}
return out
}
// InstancesInterchangeable is the one value of a definition's `instances`: the module is the same
// on every machine, so any instance may answer for the module.
const InstancesInterchangeable = "interchangeable"
+10
View File
@@ -98,3 +98,13 @@ func TestAMountOfABoundFactIsAccepted(t *testing.T) {
t.Fatalf("a mount of the file the mesh writes a binding to was refused: %v", err) t.Fatalf("a mount of the file the mesh writes a binding to was refused: %v", err)
} }
} }
// The file a filter module's rule set is written to is declared by `filtering.into` (novox/hq ADR
// 0169): the module's runtime mounts it to reload the mesh's own table, and nothing else declares it.
func TestAMountOfTheFilterFileIsDeclaredByFilteringInto(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"nftables","filtering":{"into":"/etc/nftables.conf"},` +
`"resources":[` + strings.Replace(aContainerMounting, "%s", "/etc/nftables.conf", 1) + `]}`))
if err != nil {
t.Fatalf("a filter module mounting its own filter file was refused: %v", err)
}
}
+2 -2
View File
@@ -14,7 +14,7 @@ import (
func needy() Manifest { func needy() Manifest {
return Manifest{ return Manifest{
Module: "postgres", Version: "1", Module: "postgres", Version: "1",
OwnSecrets: map[string]string{"superuser": "/var/lib/mesh/postgres/superuser"}, OwnSecrets: OwnSecrets{"superuser": {Path: "/var/lib/mesh/postgres/superuser"}},
Resources: []map[string]any{ Resources: []map[string]any{
{"id": "store", "type": "container", "name": "mesh-postgres", "image": "postgres@sha256:x"}, {"id": "store", "type": "container", "name": "mesh-postgres", "image": "postgres@sha256:x"},
}, },
@@ -74,7 +74,7 @@ func TestANeedIsAnAbsolutePath(t *testing.T) {
func TestAModuleMayNeedSeveralThings(t *testing.T) { func TestAModuleMayNeedSeveralThings(t *testing.T) {
// A password and a token, say. Telling them apart is the module's business, not the mesh's. // A password and a token, say. Telling them apart is the module's business, not the mesh's.
m := needy() m := needy()
m.OwnSecrets["replication"] = "/var/lib/mesh/postgres/replication" m.OwnSecrets["replication"] = OwnSecret{Path: "/var/lib/mesh/postgres/replication"}
got, _ := Resolve(shelf(m), []string{"postgres"}, reachable(), World{}) got, _ := Resolve(shelf(m), []string{"postgres"}, reachable(), World{})
out, err := got.Declaration(Rendering{Needed: map[string]map[string]string{ out, err := got.Declaration(Rendering{Needed: map[string]map[string]string{
"postgres": {"superuser": "b25l", "replication": "dHdv"}, "postgres": {"superuser": "b25l", "replication": "dHdv"},
@@ -0,0 +1,60 @@
package catalogue
import (
"strings"
"testing"
)
// A `user` shape and a user-scoped unit name the operator account the way a home file does
// (novox/hq ADR 0176, ADR 0177): with ${machine:account}, resolved when the module is assigned.
func TestAUserShapeAndAUserScopedUnitNameTheAccount(t *testing.T) {
facts := map[string]string{"account": "ops", "account-home": "/home/ops"}
login := map[string]any{"type": "user", "id": "login", "name": "${machine:account}", "shell": "/usr/bin/zsh"}
if err := machineInto(login, facts, "zsh"); err != nil {
t.Fatal(err)
}
if login["name"] != "ops" {
t.Fatalf("the user shape did not learn the account: %v", login["name"])
}
watcher := map[string]any{"type": "service", "id": "watcher", "unit": "i3-reload-watcher.service",
"scope": "user", "user": "${machine:account}"}
if err := machineInto(watcher, facts, "i3"); err != nil {
t.Fatal(err)
}
if watcher["user"] != "ops" {
t.Fatalf("the user-scoped unit did not learn the account: %v", watcher["user"])
}
// A machine with no operator account refuses rather than writing the literal.
err := machineInto(map[string]any{"type": "user", "id": "login", "name": "${machine:account}"},
map[string]string{"address": "10.0.0.1"}, "zsh")
if err == nil || !strings.Contains(err.Error(), "${machine:account}") {
t.Fatalf("a user shape on a machine with no account was not refused by name: %v", err)
}
}
// The service manager is a seat of the mesh's own with the unit verbs as its contract (novox/hq
// ADR 0177): every verb described, with a schema, taking a scope.
func TestTheServiceManagerSeatServesTheUnitVerbs(t *testing.T) {
seat, ok := SeatNamed("node-service-manager")
if !ok {
t.Fatal("node-service-manager is not a seat the mesh defines")
}
if seat.Scope != ScopeNode {
t.Fatalf("the service manager is a role each machine has once, and the seat is %s-scoped", seat.Scope)
}
want := []string{"units", "status", "start", "stop", "restart", "enable", "disable", "journal"}
var got []string
for _, v := range seat.Serves {
got = append(got, v.Name)
if v.Description == "" || v.Input == nil {
t.Fatalf("%s is promised without a description or a schema", v.Name)
}
props, _ := v.Input["properties"].(map[string]any)
if _, has := props["scope"]; !has {
t.Fatalf("%s takes no scope, and a user unit could not be asked for", v.Name)
}
}
if strings.Join(got, ",") != strings.Join(want, ",") {
t.Fatalf("the seat serves %v, not %v", got, want)
}
}
@@ -0,0 +1,54 @@
package catalogue
import (
"encoding/json"
"strings"
"testing"
)
// An own secret says how the module takes it (novox/hq ADR 0114, issue 180): a path alone says
// nothing of it, an object says `at-start` or `applied`, and the bytes the mesh holds are the bytes
// it was given either way.
func TestAnOwnSecretSaysHowItIsTaken(t *testing.T) {
m, err := ParseManifest([]byte(`{"module":"idp","version":"1","own-secrets":{
"broker":"/var/lib/mesh/idp/broker",
"admin":{"path":"/var/lib/idp/admin.secret","taken":"applied"},
"session":{"path":"/var/lib/idp/session.secret","taken":"at-start"}}}`))
if err != nil {
t.Fatal(err)
}
if m.OwnSecrets["broker"] != (OwnSecret{Path: "/var/lib/mesh/idp/broker"}) {
t.Fatalf("a path alone is a path and nothing more: %+v", m.OwnSecrets["broker"])
}
if m.OwnSecrets["admin"].Taken != TakenApplied || m.OwnSecrets["session"].Taken != TakenAtStart {
t.Fatalf("the word was not kept: %+v", m.OwnSecrets)
}
// Written back the way it was read, so a registered manifest keeps its bytes.
out, err := json.Marshal(m.OwnSecrets)
if err != nil {
t.Fatal(err)
}
var again OwnSecrets
if err := json.Unmarshal(out, &again); err != nil {
t.Fatal(err)
}
if len(again) != 3 || again["admin"].Taken != TakenApplied || again["broker"].Taken != "" {
t.Fatalf("the round trip changed the secrets: %s", out)
}
if !strings.Contains(string(out), `"broker":"/var/lib/mesh/idp/broker"`) {
t.Fatalf("a path alone is written back as a path: %s", out)
}
}
func TestAnOwnSecretTakenSomeOtherWayIsRefused(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"idp","version":"1","own-secrets":{
"admin":{"path":"/var/lib/idp/admin.secret","taken":"sometimes"}}}`))
if err == nil || !strings.Contains(err.Error(), `taken "sometimes"`) {
t.Fatalf("an unknown word for how a secret is taken was accepted: %v", err)
}
_, err = ParseManifest([]byte(`{"module":"idp","version":"1","own-secrets":{
"admin":{"path":"/var/lib/idp/admin.secret","rotate":"yes"}}}`))
if err == nil {
t.Fatal("an unknown field on an own secret was accepted")
}
}
+115
View File
@@ -0,0 +1,115 @@
package catalogue
import (
"strings"
"testing"
)
// Two modules on one node both provide acme-ca — public-acme (Let's Encrypt) and step-ca (the
// mesh's own authority) on novox — and a route-proxy elsewhere must get the public one (novox/hq
// #258). A pin names the module as well as the node, so that it can say which.
func issuerShelf() map[string]Manifest {
return shelf(
Manifest{Module: "public-acme", Version: "1", Provides: FromAnywhere("acme-ca"),
Serves: map[string]map[string]any{"acme-ca": {"at": "acme-v02.api.letsencrypt.org"}}},
Manifest{Module: "step-ca", Version: "1", Provides: FromAnywhere("acme-ca"),
Serves: map[string]map[string]any{"acme-ca": {"at": "novox.internal"}}},
Manifest{Module: "route-proxy", Version: "1", Requires: []string{"acme-ca"}},
)
}
func twoIssuersOnOneNode() map[string][]Provider {
return map[string][]Provider{"acme-ca": {
{Node: "novox", At: "novox.internal", Module: "public-acme", Serves: map[string]any{"at": "acme-v02.api.letsencrypt.org"}},
{Node: "novox", At: "novox.internal", Module: "step-ca", Serves: map[string]any{"at": "novox.internal"}},
}}
}
func TestTwoProvidersOnOneNodeAreRefusedWithBothNamed(t *testing.T) {
// The refusal must name the pair, because a node alone cannot tell them apart.
_, err := Resolve(issuerShelf(), []string{"route-proxy"}, reachable(),
World{Offered: twoIssuersOnOneNode()})
if err == nil {
t.Fatal("two providers on one node were resolved by picking")
}
for _, want := range []string{"novox/public-acme", "novox/step-ca", "<node> <module>"} {
if !strings.Contains(err.Error(), want) {
t.Fatalf("the refusal does not say %q: %v", want, err)
}
}
}
func TestAPinNamesTheModule(t *testing.T) {
got, err := Resolve(issuerShelf(), []string{"route-proxy"}, reachable(),
World{Offered: twoIssuersOnOneNode(),
Pinned: map[string]Chosen{"acme-ca": {Node: "novox", Module: "public-acme"}}})
if err != nil {
t.Fatal(err)
}
if len(got.Needs) != 1 || got.Needs[0].From != "novox" || got.Needs[0].Serves["at"] != "acme-v02.api.letsencrypt.org" {
t.Fatalf("the named module was not the one taken: %+v", got.Needs)
}
}
func TestARecordNamingOnlyTheNodeIsRefusedWhenThatNodeAnswersTwice(t *testing.T) {
// A pin from before the module was asked for. It once took the last one listed — a coin flip.
_, err := Resolve(issuerShelf(), []string{"route-proxy"}, reachable(),
World{Offered: twoIssuersOnOneNode(), Pinned: map[string]Chosen{"acme-ca": {Node: "novox"}}})
if err == nil {
t.Fatal("a node that answers twice was resolved by picking")
}
if !strings.Contains(err.Error(), "provides it 2 times") || !strings.Contains(err.Error(), "pin workstation acme-ca novox <module>") {
t.Fatalf("the refusal does not ask for the module: %v", err)
}
}
func TestAPinNamingAModuleThatDoesNotProvideItIsRefused(t *testing.T) {
_, err := Resolve(issuerShelf(), []string{"route-proxy"}, reachable(),
World{Offered: twoIssuersOnOneNode(), Pinned: map[string]Chosen{"acme-ca": {Node: "novox", Module: "gitea"}}})
if err == nil || !strings.Contains(err.Error(), "novox/gitea does not provide it") {
t.Fatalf("a module that does not provide it was not refused by name: %v", err)
}
}
func TestTwoProvidersBesideTheConsumerAreRefusedUntilOneIsNamed(t *testing.T) {
// The same ambiguity on the consumer's own machine. This was settled by a map walk — random,
// per plan — which is how novox's own proxy got its issuer.
_, err := Resolve(issuerShelf(), []string{"route-proxy", "public-acme", "step-ca"}, reachable(), World{})
if err == nil {
t.Fatal("two providers beside the consumer were resolved by picking")
}
if !strings.Contains(err.Error(), "workstation provides \"acme-ca\" 2 times") || !strings.Contains(err.Error(), "public-acme, step-ca") {
t.Fatalf("the refusal does not list them: %v", err)
}
}
func TestAPinSettlesTwoProvidersBesideTheConsumer(t *testing.T) {
got, err := Resolve(issuerShelf(), []string{"route-proxy", "public-acme", "step-ca"}, reachable(),
World{Pinned: map[string]Chosen{"acme-ca": {Node: "workstation", Module: "public-acme"}}})
if err != nil {
t.Fatal(err)
}
var found bool
for _, n := range got.Needs {
if n.Name == "acme-ca" {
found = true
if n.Serves["at"] != "acme-v02.api.letsencrypt.org" {
t.Fatalf("the named module was not the one taken: %+v", n)
}
}
}
if !found {
t.Fatalf("no need for acme-ca was created: %+v", got.Needs)
}
}
func TestTheFirstPassDoesNotRefuseTwoProvidersBesideTheConsumer(t *testing.T) {
// The first pass answers only what a node offers. Refused there, the node vanishes from every
// other node's world — and the whole mesh loses its vault for an ambiguity one machine has to
// settle. The second pass is where it is refused, and the test above proves it is.
if _, err := Resolve(issuerShelf(), []string{"route-proxy", "public-acme", "step-ca"}, reachable(),
World{Unchecked: true}); err != nil {
t.Fatalf("the first pass refused what only the second may: %v", err)
}
}
+62 -3
View File
@@ -16,7 +16,9 @@ import (
// //
// Two checkouts: MESH_CATALOGUE_BEFORE, the catalogue as it was, and MESH_CATALOGUE, as it is now. // Two checkouts: MESH_CATALOGUE_BEFORE, the catalogue as it was, and MESH_CATALOGUE, as it is now.
// Every module in both is resolved with the controller's own rule (dirsFor, dirFill) and compared // Every module in both is resolved with the controller's own rule (dirsFor, dirFill) and compared
// whole — not only the directories, but every string a directory's id was written into. // whole — not only the directories, but every string a directory's id was written into. Both
// sides are resolved, so a manifest converted in two steps (issue 119, then issue 174) is judged
// against the paths it named, not the text it used to name them with.
func TestPlacedDirectoriesKeepTheirPaths(t *testing.T) { func TestPlacedDirectoriesKeepTheirPaths(t *testing.T) {
before, after := os.Getenv("MESH_CATALOGUE_BEFORE"), os.Getenv("MESH_CATALOGUE") before, after := os.Getenv("MESH_CATALOGUE_BEFORE"), os.Getenv("MESH_CATALOGUE")
if before == "" || after == "" { if before == "" || after == "" {
@@ -42,7 +44,11 @@ func TestPlacedDirectoriesKeepTheirPaths(t *testing.T) {
t.Errorf("%s: %v", module, err) t.Errorf("%s: %v", module, err)
continue continue
} }
dirs := dirsFor(m, Rendering{}) earlier, err := ParseManifest(old)
if err != nil {
t.Errorf("%s before: %v", module, err)
continue
}
var was, is any var was, is any
if err := json.Unmarshal(old, &was); err != nil { if err := json.Unmarshal(old, &was); err != nil {
t.Fatal(err) t.Fatal(err)
@@ -50,7 +56,15 @@ func TestPlacedDirectoriesKeepTheirPaths(t *testing.T) {
if err := json.Unmarshal(now, &is); err != nil { if err := json.Unmarshal(now, &is); err != nil {
t.Fatal(err) t.Fatal(err)
} }
resolved := resolvedTree(is, dirs, module, t) // Both sides resolved: the manifest before may itself already place some directories
// (issue 119's conversion), and what must not move is the path a machine sees.
was = resolvedTree(was, dirsFor(earlier, Rendering{}), module, t)
resolved := resolvedTree(is, dirsFor(m, Rendering{}), module, t)
// An access named by id resolves to the path the definition still carries as its default
// (issue 153) — the same rule as a placed directory: an assignment that says nothing moves
// nothing.
was = accessesResolved(was, earlier)
resolved = accessesResolved(resolved, m)
if !reflect.DeepEqual(was, resolved) { if !reflect.DeepEqual(was, resolved) {
wasJSON, _ := json.MarshalIndent(was, "", " ") wasJSON, _ := json.MarshalIndent(was, "", " ")
isJSON, _ := json.MarshalIndent(resolved, "", " ") isJSON, _ := json.MarshalIndent(resolved, "", " ")
@@ -119,3 +133,48 @@ func firstDifference(a, b string) string {
} }
return "(the difference is beyond the shorter document)" return "(the difference is beyond the shorter document)"
} }
// accessesResolved fills every ${access:<id>} with the default path the definition carries for that
// access, and drops the id, so a manifest that names its accesses is compared by the paths a machine
// with no placement receives.
func accessesResolved(node any, m Manifest) any {
defaults := map[string]string{}
for _, a := range m.Accesses {
if a.ID != "" && a.Path != "" {
defaults[a.ID] = a.Path
}
}
var walk func(any) any
walk = func(n any) any {
switch v := n.(type) {
case map[string]any:
out := map[string]any{}
for k, child := range v {
if k == "id" {
if _, isAccess := v["mode"]; isAccess && v["type"] == nil {
if _, hasPath := v["path"]; hasPath {
continue
}
}
}
out[k] = walk(child)
}
return out
case []any:
out := make([]any, len(v))
for i, child := range v {
out[i] = walk(child)
}
return out
case string:
return accessRef.ReplaceAllStringFunc(v, func(ref string) string {
if p, ok := defaults[accessRef.FindStringSubmatch(ref)[1]]; ok {
return p
}
return ref
})
}
return n
}
return walk(node)
}
+382
View File
@@ -0,0 +1,382 @@
package catalogue
import (
"fmt"
"regexp"
"sort"
"strings"
)
// Where a module's data is on THIS machine is the assignment's (novox/hq ADR 0112, issue 153).
//
// A definition names no host path. It declares the directories it owns by id, and the operator's
// shared data it needs by id too (an `access`, ADR 0051). A node has a default layout for the
// former — <root>/<module>/<id> — and nothing at all for the latter, because shared data is
// wherever the operator keeps it. An adopted machine keeps its data where the predecessor put it:
// a 40 TB library on its own pool, a configuration on a second disk. Both halves are said on the
// assignment, validated the way `endpoints` is — an id the module does not declare is refused,
// because a setting that reaches nothing is a mistake — and resolved here, so the host receives
// concrete paths exactly as it always has and learns no field.
//
// {"places": {"config": "/services/sonarr/config",
// "data": {"path": "/mnt/plex/data", "owner": "1000:1000"}},
// "accesses": {"series": "/storage/media/series",
// "downloads": "/storage/downloads"}}
//
// A placed directory is still the mesh's: created, chowned to the owner the assignment says (or
// the manifest's), removed when empty and no longer declared. A placed access is still the
// operator's: mounted, never created, chowned or removed.
// PlacesSetting is the settings key that places a module's declared directories, by id.
const PlacesSetting = "places"
// AccessesSetting is the settings key that says where a module's accesses are on this node, by id.
const AccessesSetting = "accesses"
// Placement is what an assignment says about one of a module's directories.
type Placement struct {
// Path is where the directory is on this machine. Absolute.
Path string
// Owner is "uid:gid" when the assignment overrides the manifest's — the predecessor's data is
// owned by whoever it ran as, and that is one machine's fact.
Owner string
}
var ownerShape = regexp.MustCompile(`^[0-9]+:[0-9]+$`)
// accessRef is how a module names one of its accesses: ${access:<id>}.
var accessRef = regexp.MustCompile(`\$\{access:([a-z0-9][a-z0-9-]*)\}`)
// Places reads where this node places the module's directories, by directory id.
//
// It refuses an id the module declares no directory for, a path that is not absolute, and an
// owner that is not uid:gid. A directory the assignment does not mention keeps the manifest's
// stated path or the node's default layout.
func Places(m Manifest, layers []Layer) (map[string]Placement, error) {
declared := map[string]bool{}
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) == "directory" {
declared[fmt.Sprint(r["id"])] = true
}
}
out := map[string]Placement{}
for _, layer := range layers {
raw, ok := layer.Values[PlacesSetting]
if !ok {
continue
}
blocks, ok := raw.(map[string]any)
if !ok {
return nil, fmt.Errorf("%s: %s is a { directory: path | { path, owner } } map, and %q set it to something else",
m.Module, PlacesSetting, layer.From)
}
for id, body := range blocks {
if !declared[id] {
return nil, fmt.Errorf(
"%s places the directory %q, which it does not declare — the setting reaches "+
"nothing. It declares %s", m.Module, id, orNothing(namesOfDirs(directoriesOf(m))))
}
p := out[id]
switch v := body.(type) {
case string:
p.Path = strings.TrimSpace(v)
case map[string]any:
if path, said := v["path"]; said {
text, _ := path.(string)
p.Path = strings.TrimSpace(text)
}
if owner, said := v["owner"]; said {
text, _ := owner.(string)
if !ownerShape.MatchString(strings.TrimSpace(text)) {
return nil, fmt.Errorf("%s places %q with owner %v; an owner is uid:gid, numeric",
m.Module, id, owner)
}
p.Owner = strings.TrimSpace(text)
}
default:
return nil, fmt.Errorf("%s places %q with %v; a placement is a path, or { path, owner }",
m.Module, id, body)
}
if p.Path == "" {
return nil, fmt.Errorf("%s places %q without a path", m.Module, id)
}
if !strings.HasPrefix(p.Path, "/") {
return nil, fmt.Errorf("%s places %q at %q, which is not an absolute path", m.Module, id, p.Path)
}
p.Path = strings.TrimRight(p.Path, "/")
out[id] = p
}
}
if len(out) == 0 {
return nil, nil
}
return out, nil
}
// AccessPlaces reads where this node keeps the operator's data the module accesses, by access id.
//
// It refuses an id the module declares no access under, and a path that is not absolute. An
// access declared by path alone cannot be placed — it has no name to place it by.
func AccessPlaces(m Manifest, layers []Layer) (map[string]string, error) {
declared := map[string]bool{}
for _, a := range m.Accesses {
if a.ID != "" {
declared[a.ID] = true
}
}
out := map[string]string{}
for _, layer := range layers {
raw, ok := layer.Values[AccessesSetting]
if !ok {
continue
}
blocks, ok := raw.(map[string]any)
if !ok {
return nil, fmt.Errorf("%s: %s is a { access: path } map, and %q set it to something else",
m.Module, AccessesSetting, layer.From)
}
for id, body := range blocks {
if !declared[id] {
return nil, fmt.Errorf(
"%s places the access %q, which it does not declare — the setting reaches "+
"nothing. It declares %s", m.Module, id, orNothing(namesOfAccesses(m)))
}
path, _ := body.(string)
path = strings.TrimSpace(path)
if !strings.HasPrefix(path, "/") {
return nil, fmt.Errorf("%s places the access %q at %v, which is not an absolute path",
m.Module, id, body)
}
out[id] = strings.TrimRight(path, "/")
}
}
if len(out) == 0 {
return nil, nil
}
return out, nil
}
// placedAccess is one access with the path it resolves to on this node.
type placedAccess struct {
ID string
Path string
Mode string
}
// accessesFor is every access of a module with its path on this node: the assignment's where it
// placed one, the definition's where it carries a default, and refused where neither says — an
// access that resolves to nowhere would reach the machine as a mount of nothing.
func accessesFor(m Manifest, layers []Layer) ([]placedAccess, map[string]string, error) {
placed, err := AccessPlaces(m, layers)
if err != nil {
return nil, nil, err
}
var out []placedAccess
byID := map[string]string{}
for _, a := range m.Accesses {
path := a.Path
if a.ID != "" {
if at, said := placed[a.ID]; said {
path = at
}
}
if path == "" {
return nil, nil, fmt.Errorf(
"%s accesses %q, and nothing says where that is on this node — the definition "+
"carries no path (it must not, novox/hq ADR 0112) and the assignment places "+
"none. Set %s: {%q: \"/where/it/is\"}",
m.Module, a.ID, AccessesSetting, a.ID)
}
out = append(out, placedAccess{ID: a.ID, Path: path, Mode: a.At()})
if a.ID != "" {
byID[a.ID] = path
}
}
return out, byID, nil
}
// accessFill resolves every ${access:…} in one string, or refuses a reference naming no access.
func accessFill(s string, accesses map[string]string, module string) (string, error) {
var missing error
out := accessRef.ReplaceAllStringFunc(s, func(ref string) string {
id := accessRef.FindStringSubmatch(ref)[1]
path, has := accesses[id]
if !has {
missing = fmt.Errorf(
"%s says ${access:%s}, and %s declares no access %q. It declares %s",
module, id, module, id, orNothing(namesOfAccessIDs(accesses)))
return ref
}
return path
})
return out, missing
}
// accessInto fills every ${access:…} a resource carries — in its path, its content, its mounts,
// its environment and its env-files — with the path this node resolved for it. The same walk as
// dirInto, for the same reason: a literal `${access:x}` reaching the machine would be mounted as
// a directory called that.
func accessInto(resource map[string]any, accesses map[string]string, module string) error {
if !mentionsAccess(resource) {
return nil
}
fill := func(s string) (string, error) { return accessFill(s, accesses, module) }
var err error
if path, ok := resource["path"].(string); ok {
if resource["path"], err = fill(path); err != nil {
return err
}
}
if content, ok := resource["content"].(string); ok {
if resource["content"], err = fill(content); err != nil {
return err
}
}
for _, field := range []string{"volumes", "env-file"} {
list, ok := resource[field].([]any)
if !ok {
continue
}
filled := make([]any, len(list))
for i, v := range list {
filled[i] = v
if s, ok := v.(string); ok {
if filled[i], err = fill(s); err != nil {
return err
}
}
}
resource[field] = filled
}
if env, ok := resource["env"].(map[string]any); ok {
filled := make(map[string]any, len(env))
for key, v := range env {
filled[key] = v
if s, ok := v.(string); ok {
if filled[key], err = fill(s); err != nil {
return err
}
}
}
resource["env"] = filled
}
return nil
}
func mentionsAccess(resource map[string]any) bool {
for _, field := range []string{"path", "content"} {
if s, ok := resource[field].(string); ok && accessRef.MatchString(s) {
return true
}
}
for _, field := range []string{"volumes", "env-file"} {
if list, ok := resource[field].([]any); ok {
for _, v := range list {
if s, ok := v.(string); ok && accessRef.MatchString(s) {
return true
}
}
}
}
if env, ok := resource["env"].(map[string]any); ok {
for _, v := range env {
if s, ok := v.(string); ok && accessRef.MatchString(s) {
return true
}
}
}
return false
}
// ownerInto gives a placed directory the owner the assignment said, where it said one. The
// manifest's owner is what the image expects on any machine; the assignment's is what this
// machine's data already is.
func ownerInto(resource map[string]any, placed map[string]Placement) {
if fmt.Sprint(resource["type"]) != "directory" {
return
}
if p, ok := placed[fmt.Sprint(resource["id"])]; ok && p.Owner != "" {
resource["owner"] = p.Owner
}
}
// unknownAccessRefs is every ${access:…} in the definition that names no access the definition
// declares by id — refused where the author is, as unknownDirRefs does for directories.
func (m Manifest) unknownAccessRefs() []string {
declared := map[string]bool{}
for _, a := range m.Accesses {
if a.ID != "" {
declared[a.ID] = true
}
}
seen := map[string]bool{}
var problems []string
refuse := func(s string, where any) {
for _, match := range accessRef.FindAllStringSubmatch(s, -1) {
id := match[1]
if declared[id] || seen[id] {
continue
}
seen[id] = true
problems = append(problems, fmt.Sprintf(
"%s says ${access:%s} in %v, and declares no access %q — a reference the mesh "+
"cannot place would reach the machine as a literal path",
m.Module, id, where, id))
}
}
for _, r := range m.Resources {
for _, field := range []string{"path", "content"} {
if s, ok := r[field].(string); ok {
refuse(s, r["id"])
}
}
for _, field := range []string{"volumes", "env-file"} {
if list, ok := r[field].([]any); ok {
for _, v := range list {
if s, ok := v.(string); ok {
refuse(s, r["id"])
}
}
}
}
if env, ok := r["env"].(map[string]any); ok {
for _, v := range env {
if s, ok := v.(string); ok {
refuse(s, r["id"])
}
}
}
}
sort.Strings(problems)
return problems
}
func directoriesOf(m Manifest) map[string]string {
dirs := map[string]string{}
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) == "directory" {
dirs[fmt.Sprint(r["id"])] = ""
}
}
return dirs
}
func namesOfAccesses(m Manifest) []string {
var names []string
for _, a := range m.Accesses {
if a.ID != "" {
names = append(names, fmt.Sprintf("%q", a.ID))
}
}
sort.Strings(names)
return names
}
func namesOfAccessIDs(accesses map[string]string) []string {
var names []string
for id := range accesses {
names = append(names, fmt.Sprintf("%q", id))
}
sort.Strings(names)
return names
}
+181
View File
@@ -0,0 +1,181 @@
package catalogue
import (
"strings"
"testing"
)
// The case novox/hq issue 153 records: an adopted machine keeps its data where the predecessor put
// it — a library on its own pool that must never move, a configuration on a second disk owned by
// whoever the predecessor ran as. A definition may name none of that (ADR 0112); the assignment
// says it, by the ids the definition declared, and the machine receives concrete paths as always.
func placeable() Manifest {
m := mod("arr", nil, nil, nil)
m.Resources = []map[string]any{
{"id": "state", "type": "directory", "place": ".", "mode": "0700"},
{"id": "config", "type": "directory", "mode": "0755", "owner": "1000:1000"},
{"id": "server", "type": "container", "name": "arr", "image": "arr:1",
"volumes": []any{"${dir:config}:/config", "${access:series}:/series", "${access:spool}:/downloads:ro"},
"env": map[string]any{"SPOOL": "${access:spool}"}},
}
m.Accesses = []Access{{ID: "series", Mode: AccessReadWrite}, {ID: "spool"}}
return m
}
func placedBy(values map[string]any) Rendering {
return Rendering{Settings: SettingsBy{"arr": {{From: "node anchor", Values: values}}}}
}
func TestAnAssignmentPlacesDirectoriesAndAccesses(t *testing.T) {
got, err := Resolve(shelf(placeable()), []string{"arr"}, workstation(), World{})
if err != nil {
t.Fatal(err)
}
out, err := got.Declaration(placedBy(map[string]any{
PlacesSetting: map[string]any{
"config": map[string]any{"path": "/services/arr/config/", "owner": "1001:2000"},
},
AccessesSetting: map[string]any{
"series": "/storage/media/series",
"spool": "/storage/downloads",
},
}))
if err != nil {
t.Fatal(err)
}
seen := map[string]map[string]any{}
for _, r := range out {
seen[r["id"].(string)] = r
}
config := seen["arr.config"]
if config["path"] != "/services/arr/config" || config["owner"] != "1001:2000" {
t.Fatalf("the placed directory is %v %v; want the assignment's path and owner", config["path"], config["owner"])
}
if seen["arr.state"]["path"] != "/var/lib/arr" {
t.Fatalf("an unplaced directory left the default layout: %v", seen["arr.state"]["path"])
}
var accesses []string
for _, r := range out {
if r["type"] == "access" {
accesses = append(accesses, r["path"].(string)+" "+r["mode"].(string))
}
}
if strings.Join(accesses, ",") != "/storage/media/series read-write,/storage/downloads read" {
t.Fatalf("the accesses reached the machine as %v", accesses)
}
server := seen["arr.server"]
mounts := server["volumes"].([]any)
if mounts[0] != "/services/arr/config:/config" || mounts[1] != "/storage/media/series:/series" ||
mounts[2] != "/storage/downloads:/downloads:ro" {
t.Fatalf("the mounts were not filled with the placed paths: %v", mounts)
}
if server["env"].(map[string]any)["SPOOL"] != "/storage/downloads" {
t.Fatalf("the environment was not filled: %v", server["env"])
}
}
// An access declared by id and placed by nobody resolves to nowhere, and that is refused with the
// setting to write — not mounted as the literal, not skipped. The refusal costs the module its
// place in the declaration, not the machine its declaration (novox/hq ADR 0163, rule 6).
func TestAnUnplacedAccessIsRefusedByName(t *testing.T) {
got, err := Resolve(shelf(placeable()), []string{"arr"}, workstation(), World{})
if err != nil {
t.Fatal(err)
}
with := placedBy(map[string]any{
AccessesSetting: map[string]any{"series": "/storage/media/series"},
})
composed, err := got.Compose(with)
if err != nil {
t.Fatal(err)
}
why := composed.LeftOut["arr"]
if why == "" || !strings.Contains(why, `"spool"`) || !strings.Contains(why, AccessesSetting) {
t.Fatalf("an access nobody placed was not refused by name: %v", composed.LeftOut)
}
if _, declared := byID(composed.Resources)["arr.server"]; declared {
t.Fatal("the module with the unplaced access was declared anyway")
}
}
// Validated like endpoints: an id the module does not declare reaches nothing, and the refusal
// says what it does declare; a relative path and a non-numeric owner are refused too.
func TestPlacementsAreValidated(t *testing.T) {
m := placeable()
layers := func(values map[string]any) []Layer { return placedBy(values).Settings["arr"] }
_, err := Places(m, layers(map[string]any{PlacesSetting: map[string]any{"data": "/mnt/data"}}))
if err == nil || !strings.Contains(err.Error(), "does not declare") || !strings.Contains(err.Error(), `"config"`) {
t.Fatalf("placing an undeclared directory was accepted: %v", err)
}
_, err = Places(m, layers(map[string]any{PlacesSetting: map[string]any{"config": "services/arr"}}))
if err == nil || !strings.Contains(err.Error(), "absolute") {
t.Fatalf("a relative placement was accepted: %v", err)
}
_, err = Places(m, layers(map[string]any{PlacesSetting: map[string]any{
"config": map[string]any{"path": "/services/arr", "owner": "media"}}}))
if err == nil || !strings.Contains(err.Error(), "uid:gid") {
t.Fatalf("a non-numeric owner was accepted: %v", err)
}
_, err = AccessPlaces(m, layers(map[string]any{AccessesSetting: map[string]any{"movies": "/storage/media/movies"}}))
if err == nil || !strings.Contains(err.Error(), "does not declare") || !strings.Contains(err.Error(), `"series"`) {
t.Fatalf("placing an undeclared access was accepted: %v", err)
}
_, err = AccessPlaces(m, layers(map[string]any{AccessesSetting: map[string]any{"series": "media/series"}}))
if err == nil || !strings.Contains(err.Error(), "absolute") {
t.Fatalf("a relative access was accepted: %v", err)
}
// And the two keys are never stray: they are validated here, not merged into a file.
if stray := UnusedSettings(m, layers(map[string]any{
PlacesSetting: map[string]any{"config": "/services/arr/config"},
AccessesSetting: map[string]any{"series": "/storage/media/series"},
})); len(stray) != 0 {
t.Fatalf("the placement keys were reported as unused: %v", stray)
}
}
// A definition that carries a path still works, as the default the assignment may replace — and
// the assignment's placement wins where both say.
func TestADefinitionsPathIsTheDefaultTheAssignmentReplaces(t *testing.T) {
m := placeable()
m.Accesses = []Access{{ID: "series", Path: "/services/media/series", Mode: AccessReadWrite}, {ID: "spool", Path: "/services/media/downloads"}}
got, err := Resolve(shelf(m), []string{"arr"}, workstation(), World{})
if err != nil {
t.Fatal(err)
}
out, err := got.Declaration(placedBy(map[string]any{
PlacesSetting: map[string]any{"config": "/services/arr/config"},
AccessesSetting: map[string]any{"series": "/storage/media/series"},
}))
if err != nil {
t.Fatal(err)
}
var paths []string
for _, r := range out {
if r["type"] == "access" {
paths = append(paths, r["path"].(string))
}
}
if strings.Join(paths, ",") != "/storage/media/series,/services/media/downloads" {
t.Fatalf("placed one, defaulted the other: got %v", paths)
}
}
// A reference to an access the definition does not declare is refused where the author is.
func TestAnUnknownAccessReferenceIsRefusedAtParse(t *testing.T) {
_, err := ParseManifest([]byte(`{
"module": "arr", "version": "1",
"accesses": [{"id": "series", "mode": "read-write"}],
"resources": [
{"id": "state", "type": "directory", "place": ".", "mode": "0700"},
{"id": "server", "type": "container", "name": "arr", "image": "arr:1",
"volumes": ["${access:movies}:/movies"]}
]}`))
if err == nil || !strings.Contains(err.Error(), "${access:movies}") {
t.Fatalf("a reference to an undeclared access was accepted: %v", err)
}
_, err = ParseManifest([]byte(`{"module": "arr", "version": "1", "accesses": [{"mode": "read"}]}`))
if err == nil || !strings.Contains(err.Error(), "neither an id nor a path") {
t.Fatalf("an access with no id and no path was accepted: %v", err)
}
}
@@ -0,0 +1,66 @@
package catalogue
import (
"encoding/json"
"reflect"
"testing"
)
// What a provider receives is composed once, and issued twice: as its received file, and in its
// membership on the bus (novox/hq ADR 0167). The two are the same list, so a proxy reading the bus
// and one reading the file serve the same routes — including the port the machine published, which
// is the same-node fix the file already carries.
func TestWhatAProviderReceivesIsTheSameOnTheBusAsInItsFile(t *testing.T) {
gitea := Manifest{
Module: "gitea", Version: "1",
Listens: []Listening{{Port: 3000, Protocol: "tcp", From: FromMesh}},
Contributes: map[string]map[string]any{"route": {"label": "git", "port": 3000}},
Resources: []map[string]any{{
"id": "server", "type": "container", "name": "gitea", "ports": []any{"3000"},
}},
}
r, err := Resolve(shelf(gitea, routeProxy(), stepCA()),
[]string{"gitea", "route-proxy", "step-ca"}, reachable(), World{})
if err != nil {
t.Fatal(err)
}
composed, err := r.Compose(Rendering{Ports: map[string]map[int]int{"gitea": {3000: 20000}}})
if err != nil {
t.Fatal(err)
}
file := fileNamed(composed.Resources, "route-proxy.received-route")
if file == nil {
t.Fatal("the proxy was given no routes file")
}
var written struct {
Given []Contribution `json:"given"`
}
if err := json.Unmarshal([]byte(file["content"].(string)), &written); err != nil {
t.Fatal(err)
}
issued, said := composed.Received["route-proxy"]["route"]
if !said {
t.Fatalf("nothing is issued for the proxy to receive on the bus: %v", composed.Received)
}
// Compared as JSON, which is what both are once they leave the controller.
a, _ := json.Marshal(written.Given)
b, _ := json.Marshal(issued)
var fromFile, fromBus any
_ = json.Unmarshal(a, &fromFile)
_ = json.Unmarshal(b, &fromBus)
if !reflect.DeepEqual(fromFile, fromBus) {
t.Errorf("the bus and the file disagree about the routes:\nfile %s\nbus %s", a, b)
}
if len(issued) != 1 {
t.Fatalf("expected one route on the bus, got %v", issued)
}
if port, ok := asPort(issued[0].Values["port"]); !ok || port != 20000 {
t.Errorf("the bus carries a port nothing listens on: %v", issued[0].Values["port"])
}
// A module that receives nothing is issued nothing to receive.
if _, any := composed.Received["gitea"]; any {
t.Errorf("a module that receives nothing was issued something: %v", composed.Received["gitea"])
}
}
+92 -39
View File
@@ -48,11 +48,12 @@ type World struct {
Holdings []Held Holdings []Held
// Offered is what other nodes provide at mesh scope, and everything needed to use it. // Offered is what other nodes provide at mesh scope, and everything needed to use it.
Offered map[string][]Provider Offered map[string][]Provider
// Pinned is which node this machine was told to get a provision from, by name. Only consulted // Pinned is which provider this machine was told to get a provision from, by name: a module and
// when more than one node could answer -- a choice recorded before it was needed should not // the node it runs on, both (novox/hq #258). Only consulted when more than one could answer -- a
// start meaning something the day a second provider appears, and one recorded and then made // choice recorded before it was needed should not start meaning something the day a second
// unnecessary should not quietly stop applying either. // provider appears, and one recorded and then made unnecessary should not quietly stop applying
Pinned map[string]string // either.
Pinned map[string]Chosen
// Licences is every provision answered by a **record rather than a node**, by provision name. // Licences is every provision answered by a **record rather than a node**, by provision name.
// //
// novox/hq ADR 0024: a hosted model is on nobody's machine and is reached over the public // novox/hq ADR 0024: a hosted model is on nobody's machine and is reached over the public
@@ -179,6 +180,10 @@ type Needed struct {
// for one requirement (ADR 0094); empty for the ordinary one. Part of what identifies the pair // for one requirement (ADR 0094); empty for the ordinary one. Part of what identifies the pair
// credential, so two secrets from one provider to one module are two secrets. // credential, so two secrets from one provider to one module are two secrets.
Local string Local string
// SharedOwn is set when the provision's credential is one of the provider's own secrets, shared
// by every consumer (novox/hq ADR 0158): the name of that secret in the provider's definition.
// The plan mints the pair's copy from the provider's value rather than a value of its own.
SharedOwn string
// Manager is set when this holder is a refreshable-grant licence's MANAGER, delivered the refresh // Manager is set when this holder is a refreshable-grant licence's MANAGER, delivered the refresh
// token rather than an access token (novox/hq ADR 0050). It changes one thing downstream: an empty // token rather than an access token (novox/hq ADR 0050). It changes one thing downstream: an empty
// Sealed is tolerated — the manager has not adopted a refresh token yet, which is a real waiting // Sealed is tolerated — the manager has not adopted a refresh token yet, which is a real waiting
@@ -256,6 +261,10 @@ func Resolve(catalogue map[string]Manifest, assigned []string, node Node, world
// still "choose one" after somebody has chosen one. That makes the remedy useless, and it is // still "choose one" after somebody has chosen one. That makes the remedy useless, and it is
// how this read when first used. // how this read when first used.
satisfied := map[string]bool{} satisfied := map[string]bool{}
// Which modules a person assigned here, hostable. The walk marks a module chosen only when it
// reaches it, and a consumer may be reached before the provider beside it — so the provider of
// something already satisfied is looked for among these as well as among the chosen.
assignedHere := map[string]bool{}
// Everything a person assigned goes in first, except what this machine cannot run. Those are // Everything a person assigned goes in first, except what this machine cannot run. Those are
// choices already made, and a requirement one of them answers is not a choice to put back to // choices already made, and a requirement one of them answers is not a choice to put back to
@@ -280,6 +289,7 @@ func Resolve(catalogue map[string]Manifest, assigned []string, node Node, world
for _, o := range m.Offers() { for _, o := range m.Offers() {
satisfied[o] = true satisfied[o] = true
} }
assignedHere[a] = true
} }
because[a] = "assigned" because[a] = "assigned"
queue = append(queue, a) queue = append(queue, a)
@@ -307,6 +317,51 @@ func Resolve(catalogue map[string]Manifest, assigned []string, node Node, world
// commonest arrangement of all — a service and its database on one node — the weakest // commonest arrangement of all — a service and its database on one node — the weakest
// handling, silently. // handling, silently.
if satisfied[want] && !isModule(catalogue, want) { if satisfied[want] && !isModule(catalogue, want) {
here := func(name string) bool { return chosen[name] || assignedHere[name] }
local := providersHere(catalogue, here, want)
// Which of them it matters to choose between. A plain capability — a shell, a display
// server — asks nothing of whoever answers it, and three shells beside an editor are
// not a choice to put to anybody. One that grants a credential, or serves a fact the
// consumer cannot guess, becomes a binding, and a binding is to one provider.
matter := local
if !brokered[want] {
matter = nil
for _, name := range local {
if _, ok := catalogue[name].Serves[want]; ok {
matter = append(matter, name)
}
}
}
var by Manifest
switch len(matter) {
case 0:
// Nothing to bind to; satisfied by its presence, as it was.
case 1:
by = catalogue[matter[0]]
default:
// Two modules on this machine answer it. Taking whichever a map walk met first
// was the rule until novox/hq #258 — random, per plan — and the same stance as
// across machines applies: ambiguity is refused, never resolved by picking.
//
// **Not in the first pass.** That pass exists only to answer *what does this node
// offer*, and refusing there makes the machine vanish rather than report a problem
// (the sibling case below says why): every other node then loses what this one
// provides — the vault, the identity provider — and refuses for a fault that is
// this node's to settle. The second pass refuses it properly, where it is asked.
if world.Unchecked {
continue
}
c, pinned := world.Pinned[want]
if !pinned || c.Node != node.Name || !oneOf(matter, c.Module) {
reported[want] = true
problems = append(problems, fmt.Sprintf(
"%s provides %q %d times, wanted by %s — say which with `pin %s %s %s <module>`: %s",
node.Name, want, len(matter), because[want], node.Name, want, node.Name,
strings.Join(matter, ", ")))
continue
}
by = catalogue[c.Module]
}
if brokered[want] { if brokered[want] {
// Answered here, and still a need: the provider is this node. // Answered here, and still a need: the provider is this node.
// //
@@ -322,8 +377,9 @@ func Resolve(catalogue map[string]Manifest, assigned []string, node Node, world
} }
needs = append(needs, Needed{ needs = append(needs, Needed{
Name: want, From: node.Name, At: at, Name: want, From: node.Name, At: at,
Serves: servedHere(catalogue, chosen, want), For: because[want]}) Serves: servedByOne(by, want), For: because[want],
} else if served := servedHere(catalogue, chosen, want); len(served) > 0 { SharedOwn: sharedByOne(by, want)})
} else if served := servedByOne(by, want); len(served) > 0 {
// Answered here with no credential to mint, but the provider serves facts the // Answered here with no credential to mint, but the provider serves facts the
// consumer cannot guess — a port, a model name — and so still needs a binding. // consumer cannot guess — a port, a model name — and so still needs a binding.
// **The reachability rule does not apply**: both ends are on this same machine, so // **The reachability rule does not apply**: both ends are on this same machine, so
@@ -353,11 +409,7 @@ func Resolve(catalogue map[string]Manifest, assigned []string, node Node, world
if brokered[want] { if brokered[want] {
reported[want] = true reported[want] = true
where := world.Offered[want] where := world.Offered[want]
names := make([]string, 0, len(where)) names := providerNames(where)
for _, p := range where {
names = append(names, p.Node)
}
sort.Strings(names)
take := func(p Provider) { take := func(p Provider) {
if node.At != "" && p.At == "" || node.At == "" && p.At != "" || node.At == "" && p.At == "" { if node.At != "" && p.At == "" || node.At == "" && p.At != "" || node.At == "" && p.At == "" {
// One of them is not on the private network, so there is no path between // One of them is not on the private network, so there is no path between
@@ -369,8 +421,12 @@ func Resolve(catalogue map[string]Manifest, assigned []string, node Node, world
node.Name, want, p.Node, meshNetwork)) node.Name, want, p.Node, meshNetwork))
return return
} }
shared := ""
if pm, known := catalogue[p.Module]; known {
shared, _ = pm.SharedCredentialOf(want)
}
needs = append(needs, Needed{Name: want, From: p.Node, At: p.At, needs = append(needs, Needed{Name: want, From: p.Node, At: p.At,
Serves: p.Serves, For: because[want]}) Serves: p.Serves, For: because[want], SharedOwn: shared})
} }
switch { switch {
case world.Unchecked: case world.Unchecked:
@@ -387,17 +443,17 @@ func Resolve(catalogue map[string]Manifest, assigned []string, node Node, world
"nothing in this mesh provides %q, wanted by %s %s", "nothing in this mesh provides %q, wanted by %s %s",
want, because[want], remedy)) want, because[want], remedy))
case len(where) == 1: case len(where) == 1:
if chosenNode, pinned := world.Pinned[want]; pinned && chosenNode != where[0].Node { if c, pinned := world.Pinned[want]; pinned && !c.matches(where[0]) {
// One provider, and it is not the one this machine was told to use. Silently // One provider, and it is not the one this machine was told to use. Silently
// using the other would be the mesh overruling a choice somebody made. // using the other would be the mesh overruling a choice somebody made.
problems = append(problems, fmt.Sprintf( problems = append(problems, fmt.Sprintf(
"%s was told to get %q from %s, and only %s provides it", "%s was told to get %q from %s, and only %s provides it",
node.Name, want, chosenNode, where[0].Node)) node.Name, want, c, nameOf(where[0])))
break break
} }
take(where[0]) take(where[0])
default: default:
chosenNode, pinned := world.Pinned[want] c, pinned := world.Pinned[want]
if !pinned { if !pinned {
// **The seat's holder answers, when a seat delivers this** (novox/hq ADR 0110). // **The seat's holder answers, when a seat delivers this** (novox/hq ADR 0110).
// Not a guess, which ADR 0009 refuses: the choice was made once, mesh-wide, by // Not a guess, which ADR 0009 refuses: the choice was made once, mesh-wide, by
@@ -409,27 +465,32 @@ func Resolve(catalogue map[string]Manifest, assigned []string, node Node, world
break break
} }
problems = append(problems, fmt.Sprintf( problems = append(problems, fmt.Sprintf(
"%d nodes provide %q, wanted by %s — say which with `pin %s %s <node>`: %s", "%d providers of %q, wanted by %s — say which with `pin %s %s <node> <module>`: %s",
len(where), want, because[want], node.Name, want, len(where), want, because[want], node.Name, want,
strings.Join(names, ", "))) strings.Join(names, ", ")))
break break
} }
var chosen *Provider matching := c.among(where)
for i, w := range where { switch len(matching) {
if w.Node == chosenNode { case 0:
chosen = &where[i] // Pointed at a provider that does not answer this. Refused rather than
}
}
if chosen == nil {
// Pointed at a machine that does not answer this. Refused rather than
// falling back to another: a fallback would quietly move somebody's data to // falling back to another: a fallback would quietly move somebody's data to
// a machine they did not choose, which is the whole reason this is asked. // a machine they did not choose, which is the whole reason this is asked.
problems = append(problems, fmt.Sprintf( problems = append(problems, fmt.Sprintf(
"%s was told to get %q from %s, and %s does not provide it — these do: %s", "%s was told to get %q from %s, and %s does not provide it — these do: %s",
node.Name, want, chosenNode, chosenNode, strings.Join(names, ", "))) node.Name, want, c, c, strings.Join(names, ", ")))
break case 1:
take(matching[0])
default:
// A record naming only the node, from before a pin named the module, and that
// node answers twice. This once took the last one listed (novox/hq #258): a
// coin flip, handed to whoever reads the certificate it chose.
problems = append(problems, fmt.Sprintf(
"%s was told to get %q from %s, and %s provides it %d times — say which with "+
"`pin %s %s %s <module>`: %s",
node.Name, want, c, c.Node, len(matching), node.Name, want, c.Node,
strings.Join(providerNames(matching), ", ")))
} }
take(*chosen)
} }
continue continue
} }
@@ -588,17 +649,6 @@ func Resolve(catalogue map[string]Manifest, assigned []string, node Node, world
// need that is never created is a binding the consumer never gets. It is right about that from the // need that is never created is a binding the consumer never gets. It is right about that from the
// manifest alone, which is why walking the catalogue mid-resolution is enough here and is not // manifest alone, which is why walking the catalogue mid-resolution is enough here and is not
// enough for the values. // enough for the values.
func servedHere(catalogue map[string]Manifest, chosen map[string]bool, want string) map[string]any {
for name, m := range catalogue {
if !chosen[name] {
continue
}
if _, ok := m.Serves[want]; ok {
return ServedOn(m, want, nil)
}
}
return nil
}
// isModule reports whether a name is a module in its own right rather than only something // isModule reports whether a name is a module in its own right rather than only something
// modules provide. // modules provide.
@@ -791,6 +841,9 @@ func checkResources(modules []Manifest) []string {
// which is what lets the stack in 04-ISSUES/036 co-resolve. // which is what lets the stack in 04-ISSUES/036 co-resolve.
for _, m := range modules { for _, m := range modules {
for _, a := range m.Accesses { for _, a := range m.Accesses {
if a.Path == "" {
continue // placed by the assignment; nothing to compare at registration
}
switch other := ownedPath[a.Path]; other { switch other := ownedPath[a.Path]; other {
case "": case "":
// Nobody owns it — the ordinary, correct case for shared data. // Nobody owns it — the ordinary, correct case for shared data.
@@ -0,0 +1,42 @@
package catalogue
import (
"strings"
"testing"
)
// A resolved manifest keeps a built artifact's reference in the store-relative form, never the
// address the builder reached the store by (novox/hq ADR 0155): on 2026-10-02 the first bundle
// resolved on the mesh carried the store's host in bundles[0].source and registration refused it
// as naming an installation. Archive resources are the same kind of reference and get the same.
func TestAResolvedReferenceIsKeptNotRouted(t *testing.T) {
m, err := ParseManifest([]byte(`{
"module": "sample", "version": "1",
"tools": ["one"],
"build": {"artifacts": [
{"name": "code", "kind": "bundle", "language": "typescript", "entrypoints": ["tools/index.js"]},
{"name": "files", "kind": "archive", "from": "files"}
]},
"resources": [{"id": "packed", "type": "archive", "path": "/opt/sample", "artifact": "files"}]
}`))
if err != nil {
t.Fatal(err)
}
digest := "sha256:" + strings.Repeat("ab", 32)
resolved, err := m.Resolve([]Built{
{Name: "code", Kind: ArtifactBundle, Reference: "http://store.example:5100/v2/sample/code/blobs/" + digest, Digest: digest},
{Name: "files", Kind: ArtifactArchive, Reference: "http://store.example:5100/v2/sample/files/blobs/" + digest, Digest: digest},
})
if err != nil {
t.Fatal(err)
}
if got, want := resolved.Bundles[0].Source, ArtifactStoreScheme+"sample/code/blobs/"+digest; got != want {
t.Errorf("bundle source %q, want the kept form %q", got, want)
}
if got, want := resolved.Resources[0]["source"], ArtifactStoreScheme+"sample/files/blobs/"+digest; got != want {
t.Errorf("archive source %q, want the kept form %q", got, want)
}
if problems := InstallationProblems(resolved); len(problems) != 0 {
t.Errorf("a resolved manifest names an installation: %v", problems)
}
}
+15 -5
View File
@@ -48,7 +48,9 @@ func TestTheResolverForwardsToFixedUpstreamsAndNeverReadsResolvConf(t *testing.T
} }
for _, want := range []string{ for _, want := range []string{
"\nno-resolv\n", "\nserver=1.1.1.1\n", "\nserver=8.8.8.8\n", "\nno-resolv\n", "\nserver=1.1.1.1\n", "\nserver=8.8.8.8\n",
"\nlisten-address=127.0.0.1\n", "\nlisten-address=${machine:address}\n", "\nbind-dynamic\n", // Loopback is the mesh-wide setting's default; a machine answering its own LAN adds its
// address there (novox/hq issue 198).
"\nlisten-address=${setting:listen-addresses}\n", "\nlisten-address=${machine:address}\n", "\nbind-dynamic\n",
"\ndomain-needed\n", "\nbogus-priv\n", "\ndomain-needed\n", "\nbogus-priv\n",
"\nconf-file=" + m.Facts["node-zones"].Path + "\n", "\nconf-file=" + m.Facts["node-zones"].Path + "\n",
} { } {
@@ -114,6 +116,7 @@ func TestTheResolverAndWhatAsksItComposeOnOneMachine(t *testing.T) {
// happen to be the same map, since nothing routed is part of it. // happen to be the same map, since nothing routed is part of it.
Names: twoMachines, Machines: twoMachines, Suffix: "internal", Names: twoMachines, Machines: twoMachines, Suffix: "internal",
Needed: map[string]map[string]string{"dnsmasq": {"broker": "sealed"}}, Needed: map[string]map[string]string{"dnsmasq": {"broker": "sealed"}},
Settings: SettingsBy{"dnsmasq": {{From: "the mesh", Values: map[string]any{"listen-addresses": "127.0.0.1"}}}},
}) })
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
@@ -207,9 +210,16 @@ func TestTheResolverOnAMachineOffTheNetworkIsRefused(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
_, err = got.Declaration(Rendering{Names: twoMachines, Suffix: "internal", // Left out of the declaration and said, rather than composed listening nowhere: a module that
Needed: map[string]map[string]string{"dnsmasq": {"broker": "sealed"}}}) // cannot compose on a machine is kept as it is there, with the reason (hq ADR 0163).
if err == nil || !strings.Contains(err.Error(), "${machine:address}") { composed, err := got.Compose(Rendering{Names: twoMachines, Suffix: "internal",
t.Fatalf("a machine off the network was composed a resolver, or refused for another reason: %v", err) Needed: map[string]map[string]string{"dnsmasq": {"broker": "sealed"}},
Settings: SettingsBy{"dnsmasq": {{From: "the mesh", Values: map[string]any{"listen-addresses": "127.0.0.1"}}}}})
if err == nil && !strings.Contains(composed.LeftOut["dnsmasq"], "${machine:address}") {
t.Fatalf("a machine off the network was composed a resolver, or left out for another reason: %v",
composed.LeftOut)
}
if err != nil && !strings.Contains(err.Error(), "${machine:address}") {
t.Fatalf("a machine off the network was refused for another reason: %v", err)
} }
} }
+4 -4
View File
@@ -28,10 +28,10 @@ import (
// A RosterFile is a file the mesh renders from the roster of machines, in the format the module // A RosterFile is a file the mesh renders from the roster of machines, in the format the module
// gives as a Go text/template. The template sees a rosterView: `.Node` (this machine's bare name), // gives as a Go text/template. The template sees a rosterView: `.Node` (this machine's bare name),
// `.Suffix` (what its mesh name ends in), and two sets of `{Name, FQDN, Address}` — `.Names`, every // `.Suffix` (what its mesh name ends in), and two sets of `{Name, FQDN, Address}` — `.Names` and
// name the mesh serves, and `.Machines`, only the nodes of the mesh. Which set a template ranges is // `.Machines`. Both are the nodes of the mesh: routed names were once in `.Names` too, and are not
// how the hq issue 111 distinction is drawn: a container's hosts wants every name; a resolver told // since every route became a name under its node's internal domain (novox/hq ADR 0191) — the hq
// the suffix is its own wants only the machines. // issue 111 distinction is kept as two fields so the templates that range either keep rendering.
type RosterFile struct { type RosterFile struct {
// Path is where on the machine the rendered file goes. Absolute, or it is refused here rather // Path is where on the machine the rendered file goes. Absolute, or it is refused here rather
// than discovered as a daemon that reads nothing. // than discovered as a daemon that reads nothing.
+365
View File
@@ -0,0 +1,365 @@
package catalogue
import (
"encoding/json"
"fmt"
"sort"
"strings"
)
// The node's tool runtime, as the catalogue knows it (novox/hq ADR 0175, to-be 38).
//
// **One module is the runtime.** Where it is assigned, one process per machine serves every assigned
// module's tools and every held seat's verbs, on the host side, from the bundles each module's build
// produced — and no module needs a container to reach the bus with its tools. The name is a constant
// rather than a manifest field because a rule turns on it: the composer places the runtime's process
// where this module is, and registration refuses the old pattern once this module exists.
// RuntimeModule is the module that is the node's tool runtime. Mirrored in the broker package,
// which composes a principal of its own for it; the agreement test there holds the two to one string.
const RuntimeModule = "node-tools"
// BundleRoot is where a machine keeps the tools bundles the mesh delivers to it: under the mesh's
// own directory, beside the daemons the host unpacks there, and never where a package manager also
// writes. One directory per module, one per bundle beneath it, at a path that does not move with
// the version — so the runtime's process names each entrypoint once and is restarted, not
// recomposed, when a bundle changes.
const BundleRoot = "/var/lib/mesh/bundles"
// BundleID names the archive resource that delivers one of a module's bundles; prefixed with the
// module like every resource of its own.
func BundleID(bundle string) string { return "bundle-" + bundle }
// BundlePath is where one module's bundle is unpacked on a machine.
func BundlePath(module, bundle string) string { return BundleRoot + "/" + module + "/" + bundle }
// runtimeHere says whether this node's set includes the runtime module, which is what decides
// whether anything about tools changes on the machine (to-be 38 WP2): until the runtime is assigned,
// a node is sent exactly what it was sent before, bundles included, because a bundle nothing loads
// is bytes nobody reads.
func (r Resolution) runtimeHere() bool {
for _, m := range r.Modules {
if m.Module == RuntimeModule {
return true
}
}
return false
}
// bundleArchives is one archive per tools bundle of a module — a bundle the runtime LOADS something
// from — as the host fetches and unpacks any artifact (novox/hq ADR 0175 §3: a module brings its
// tools as a bundle, delivered by the host like any artifact, never an image). A bundle it loads
// nothing from is run rather than loaded: a daemon, a step, the runtime itself — delivered by the
// process that runs it, and not again here.
//
// The source is the kept reference; the per-resource pass that follows routes it through the
// artifact store as this network reaches it now, as it does every image and archive the mesh built.
func bundleArchives(m Manifest) []map[string]any {
var out []map[string]any
for _, b := range m.Bundles {
if len(b.Loads) == 0 {
continue
}
out = append(out, map[string]any{
"id": BundleID(b.Name), "type": "archive",
"source": b.Source, "digest": b.Digest,
"path": BundlePath(m.Module, b.Name),
})
}
return out
}
// RuntimeProcessID names the one process the mesh composes for a machine's runtime; prefixed with
// the runtime module like a resource of its own, because that module is what the host sees it as.
func RuntimeProcessID() string { return "runtime" }
// RuntimeToolModules is the variable the runtime reads the modules it serves from: one
// `<module>=<entrypoint>` per file it loads, comma-separated — several entries may name one module.
// RuntimeBrokerFile is where it reads the node's credential; RuntimeOperatorAccount and
// RuntimeOperatorHome are the machine's operator account and home, handed to every tool's
// environment (to-be 38 WP1), and absent on a machine with no account.
const (
RuntimeToolModules = "MESH_TOOL_MODULES"
RuntimeBrokerFile = "MESH_BROKER_FILE"
RuntimeOperatorAccount = "MESH_OPERATOR_ACCOUNT"
RuntimeOperatorHome = "MESH_OPERATOR_HOME"
// RuntimeToolEnv is every served module's composed environment, as JSON (novox/hq ADR 0192):
// {"<module>": {"<word>": "<value>"}}. The runtime takes it at start, removes it from its own
// environment and hands each module's words to that module's bundles alone. In the unit, so a
// change to any module's words changes the process and restarts it.
RuntimeToolEnv = "MESH_TOOL_ENV"
)
// interpreterFor is how a bundle in a language is run: the program the host's unit starts, with the
// bundle's entrypoint after it. The one thing the composer takes from a language, and said here
// rather than in a manifest because the runtime's process is the mesh's to compose (to-be 38 WP3).
func interpreterFor(language string) (string, error) {
switch language {
case "typescript":
return "node", nil
}
return "", fmt.Errorf(
"%s is written in %q, and the mesh knows no interpreter to run a %q bundle with",
RuntimeModule, language, language)
}
// runtimeProcess is the one process a machine runs the node's tool runtime as (novox/hq ADR 0175,
// to-be 38 WP2.3): the runtime module's own bundle, run by its language's interpreter, told which
// modules it serves and from which files, where its credential is, and who the machine's operator
// is — and restarted when any bundle it loads or the credential it holds changes.
//
// Composed from the placed manifests, so the credential's path is where this node puts it. The
// runtime runs as the operator's account when the machine has one, which is what lets a tool that
// needs root escalate as the operator would (ADR 0175 §4); on a machine with no account it runs as
// root, and the two operator words are not set.
func (r Resolution) runtimeProcess(with Rendering) (map[string]any, error) {
var runtime *Manifest
for i := range r.Modules {
if r.Modules[i].Module == RuntimeModule {
runtime = &r.Modules[i]
}
}
if runtime == nil {
return nil, nil
}
if len(runtime.Bundles) != 1 {
return nil, fmt.Errorf(
"%s is assigned to %s and its build produced %d bundle(s); the runtime is one bundle "+
"the mesh runs, so the module declares exactly one (novox/hq to-be 38)",
RuntimeModule, r.Node, len(runtime.Bundles))
}
bundle := runtime.Bundles[0]
// What runs it (novox/hq ADR 0193): a runtime compiled to a binary runs itself, from its own
// unpacked bundle; an interpreted one is its language's interpreter and its one entrypoint.
var run []any
if bundle.Binary != "" {
run = []any{"./" + bundle.Binary}
} else {
if len(bundle.Entrypoints) != 1 {
return nil, fmt.Errorf(
"%s's bundle %q names %d entrypoint(s); the runtime is run from one, so the module "+
"declares exactly one (novox/hq to-be 38)", RuntimeModule, bundle.Name, len(bundle.Entrypoints))
}
interpreter, err := interpreterFor(bundle.Language)
if err != nil {
return nil, err
}
run = []any{interpreter, bundle.Entrypoints[0]}
}
credential, declared := runtime.OwnSecrets["broker"]
if !declared {
return nil, fmt.Errorf(
"%s declares no own secret named broker, and the node's credential is delivered there: "+
"a module that speaks on the bus declares \"own-secrets\": {\"broker\": <path>}",
RuntimeModule)
}
// What it serves, and from which files: every module on this machine that composes here, in
// name order, each bundle it loads from in the order the manifest gave. A module left out of
// the declaration — a filter on an adopted machine — is left out of this too, or the runtime
// would be told to load files that were never delivered.
var served []string
var restartOn []string
given := map[string]map[string]string{}
for _, m := range r.Modules {
if with.Adopted && m.Filtering != nil {
continue
}
words, err := bundleWords(m, with)
if err != nil {
return nil, err
}
if len(words) > 0 {
given[m.Module] = words
}
for _, b := range m.Bundles {
if len(b.Loads) == 0 {
continue
}
for _, load := range b.Loads {
// What the runtime starts: the launcher the build wrote beside the entrypoint, where
// it wrote one (ADR 0193); the entrypoint itself for a build from before them.
if launcher, has := b.Launchers[load]; has {
load = launcher
}
served = append(served, m.Module+"="+BundlePath(m.Module, b.Name)+"/"+load)
}
restartOn = append(restartOn, m.Module+"."+BundleID(b.Name))
}
}
sort.Strings(served)
restartOn = append(restartOn, RuntimeModule+"."+NeedID("broker"))
sort.Strings(restartOn)
env := map[string]string{
RuntimeToolModules: strings.Join(served, ","),
RuntimeBrokerFile: credential.Path,
}
if len(given) > 0 {
// Marshalled from maps, whose keys encoding/json sorts: the same words, the same unit.
body, err := json.Marshal(given)
if err != nil {
return nil, err
}
env[RuntimeToolEnv] = string(body)
}
process := map[string]any{
"id": RuntimeModule + "." + RuntimeProcessID(), "type": "process", "name": RuntimeModule,
"source": bundle.Source, "digest": bundle.Digest,
"run": run,
"env": env,
"restart-on": toAny(restartOn),
}
if r.Account != "" {
env[RuntimeOperatorAccount] = r.Account
env[RuntimeOperatorHome] = accountHomeOf(r.Account, r.AccountHome)
process["user"] = r.Account
}
// Routed through the artifact store as this network reaches it now, like everything the mesh
// built; refused with the same words when there is no store to route through.
if err := artifactsInto(process, RuntimeModule, with); err != nil {
return nil, err
}
return process, nil
}
func toAny(in []string) []any {
out := make([]any, 0, len(in))
for _, s := range in {
out = append(out, s)
}
return out
}
// RuntimeImageModule and RuntimeImageArtifact name the image every per-module tool container was
// built on: the tool runtime's own runtime image. With the runtime a module of its own, that image
// stays the way a module's SERVICE may be built and stops being the way tools reach a node (ADR 0175).
const (
RuntimeImageModule = "mesh-tools"
RuntimeImageArtifact = "runtime"
)
// ToolContainerOnTheRuntime says why a manifest is the pattern ADR 0175 retires — a module whose tools
// are served from a container built on the tool runtime's image — or nothing when it is not. Judged
// from the manifest's own `build.on` when it is a repository manifest, and from what its build stood
// on when it is a built one, because a resolved manifest carries no build. The gate itself is
// registration's (to-be 38 WP2.4): once the runtime module is in the catalogue, this is refused.
//
// Three things must hold, and each alone is fine: declaring tools (a bundle does that); a container
// (a module's service may well be one); building on the runtime's image (a service written against
// the SDK may). All three is a container whose purpose is tools, which the runtime now serves.
func ToolContainerOnTheRuntime(m Manifest, against []string) string {
if len(m.Tools) == 0 {
return ""
}
container := false
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) == "container" {
container = true
}
}
if !container {
return ""
}
onTheRuntime := false
if m.Build != nil {
for _, on := range m.Build.On {
if on.Module == RuntimeImageModule && on.Artifact == RuntimeImageArtifact {
onTheRuntime = true
}
}
}
for _, ref := range against {
path, kept := InArtifactStore(Recorded(ref))
if kept && strings.HasPrefix(path, RuntimeImageModule+"/"+RuntimeImageArtifact+"@") {
onTheRuntime = true
}
}
if !onTheRuntime {
return ""
}
return fmt.Sprintf(
"%s declares tools and a container built on %s's %s image — a container whose purpose is "+
"serving tools. The node's tool runtime (%s) serves every module's tools from its bundle "+
"now (novox/hq ADR 0175, to-be 38); declare the tools as a bundle and drop the container",
m.Module, RuntimeImageModule, RuntimeImageArtifact, RuntimeModule)
}
// bundleWords is what one module's tools bundles are given on this machine (novox/hq ADR 0192):
// each loaded bundle's env, its ${dir:…} resolved to where this machine places the module's
// directories and its ${port:…} to the port this machine gave it — the same resolution a
// container's environment gets. Two bundles of one module naming one word differently is refused:
// the runtime hands a module's words to all its bundles.
func bundleWords(m Manifest, with Rendering) (map[string]string, error) {
var out map[string]string
dirs := dirsFor(m, with)
for _, b := range m.Bundles {
if len(b.Loads) == 0 || len(b.Env) == 0 {
continue
}
for _, word := range sortedKeys(b.Env) {
value, err := dirFill(b.Env[word], dirs, m.Module)
if err != nil {
return nil, err
}
if value, err = portsFilledInto(value, m.Module+"'s bundle "+b.Name+" ("+word+")", m.Module, m.Listens, with); err != nil {
return nil, err
}
if out == nil {
out = map[string]string{}
}
if was, had := out[word]; had && was != value {
return nil, fmt.Errorf("%s's bundles give %s two values (%q, %q); a module's words are "+
"handed to all its bundles, so they agree (novox/hq ADR 0192)", m.Module, word, was, value)
}
out[word] = value
}
}
return out, nil
}
// givenTo makes what a bundle's words name readable by the account the runtime runs as (novox/hq
// ADR 0192): every file and directory of the module whose path a word names, or that holds one,
// is owned by the account — a tool reads its configuration and its secret as the account, and a
// root-owned 0600 file or a 0700 directory is one it cannot. Only where it says no owner already:
// a module that named one knew why. Nothing on a machine with no account, where the runtime is root.
//
// It answers the files a word names exactly: what a tool reads, whose change the runtime must be
// restarted for, as the container the tools came from was restarted when its configuration changed.
func givenTo(out []map[string]any, owner map[string]string, words map[string]map[string]string, account string) []string {
var named []string
if len(words) == 0 {
return nil
}
for _, resource := range out {
module := owner[fmt.Sprint(resource["id"])]
mine := words[module]
if len(mine) == 0 {
continue
}
kind := fmt.Sprint(resource["type"])
if kind != "file" && kind != "directory" {
continue
}
path, _ := resource["path"].(string)
if path == "" {
continue
}
for _, value := range mine {
if kind == "file" && value == path {
named = append(named, fmt.Sprint(resource["id"]))
}
}
if _, said := resource["owner"]; said || account == "" {
continue
}
for _, value := range mine {
if value == path || strings.HasPrefix(value, strings.TrimRight(path, "/")+"/") {
resource["owner"] = account
break
}
}
}
sort.Strings(named)
return named
}
+88
View File
@@ -0,0 +1,88 @@
package catalogue
import (
"strings"
"testing"
)
// The packet-filter manifest as it was the day the runtime was decided (novox/hq ADR 0175): tools,
// served from a container built on the tool runtime's image, with NET_ADMIN so the container could
// reach the filter. The exact pattern to-be 38 WP4 moves it off, and the one the gate refuses.
const thePacketFilterAsItWas = `{
"module": "nftables",
"version": "1",
"capabilities": ["firewall", "container-runtime"],
"claims": [{"name": "node-packet-filter", "scope": "node", "serves": ["rules", "reload", "remove"]}],
"filtering": {"into": "/etc/nftables.conf"},
"resources": [
{"id": "mesh-state", "type": "directory", "mode": "0700", "place": "mesh"},
{"id": "package", "type": "package", "package": "nftables"},
{"id": "unit", "type": "file", "path": "/etc/systemd/system/mesh-filter.service",
"content": "[Unit]\nDescription=The mesh's packet filter\n[Service]\nType=oneshot\nExecStart=nft -f /etc/nftables.conf\n", "mode": "0644"},
{"id": "load", "type": "service", "unit": "mesh-filter.service", "state": "running", "boot": "enabled",
"restart-on": ["unit"], "reload-on": ["filtering"]},
{"id": "runtime", "type": "container", "name": "mesh-nftables", "network": "host",
"capabilities": ["NET_ADMIN"],
"volumes": ["${dir:mesh-state}/broker:/run/secrets/broker:ro", "/etc/nftables.conf:/etc/nftables.conf:ro"],
"env": {"MESH_BROKER_FILE": "/run/secrets/broker", "MESH_FILTER_FILE": "/etc/nftables.conf"},
"artifact": "runtime"}
],
"tools": ["firewall_rules"],
"own-secrets": {"broker": "${dir:mesh-state}/broker"},
"build": {
"on": [
{"arg": "BUILD_BASE", "module": "mesh-tools", "artifact": "build"},
{"arg": "RUNTIME_BASE", "module": "mesh-tools", "artifact": "runtime"}
],
"artifacts": [{"name": "runtime", "kind": "image", "from": "Dockerfile"}]
}
}`
func TestAToolContainerOnTheRuntimeImageIsNamedForWhatItIs(t *testing.T) {
m, err := ParseManifest([]byte(thePacketFilterAsItWas))
if err != nil {
t.Fatal(err)
}
// From the repository: the manifest says what it builds on.
why := ToolContainerOnTheRuntime(m, nil)
if why == "" {
t.Fatal("the packet filter's tool container was not recognised from its build")
}
for _, word := range []string{"nftables", "mesh-tools", "runtime", "ADR 0175", "bundle"} {
if !strings.Contains(why, word) {
t.Errorf("the refusal does not say %q: %s", word, why)
}
}
// Built: the manifest carries no build, and what it stood on says the same.
built, err := m.Resolve([]Built{{Name: "runtime", Kind: ArtifactImage,
Reference: ArtifactStoreScheme + "nftables/runtime@" + digest}})
if err != nil {
t.Fatal(err)
}
stoodOn := []string{"anchor.internal:5100/mesh-tools/build@" + digest, "anchor.internal:5100/mesh-tools/runtime@" + digest}
if ToolContainerOnTheRuntime(built, stoodOn) == "" {
t.Error("the packet filter's tool container was not recognised from what its build stood on")
}
if ToolContainerOnTheRuntime(built, nil) != "" {
t.Error("a built manifest with no record of its base was judged to be on the runtime")
}
// Each of the three alone is an ordinary module.
bundle := m
bundle.Resources = m.Resources[:len(m.Resources)-1]
if ToolContainerOnTheRuntime(bundle, nil) != "" {
t.Error("a module with tools and no container is the pattern the runtime serves, and was refused")
}
service := m
service.Tools = nil
if ToolContainerOnTheRuntime(service, nil) != "" {
t.Error("a service built against the SDK, declaring no tools, was refused")
}
elsewhere := m
elsewhere.Build = &Build{On: []BuildsOn{{Arg: "NODE_BASE", Image: "node@" + digest}},
Artifacts: m.Build.Artifacts}
if ToolContainerOnTheRuntime(elsewhere, nil) != "" {
t.Error("a tool container on a public base was refused as though it were on the runtime's")
}
}

Some files were not shown because too many files have changed in this diff Show More