Author SHA1 Message Date
jochen 126f6e0c5d Merge main: hold a moving machine back, then deliver the rest grants first
Main sends every machine through deliver (memberships before declarations,
hq ADR 0218); this branch holds back a machine whose running module's data
would move (ADR 0217). Both stand: the held machines are filtered out before
delivery, in the named push and its cascade alike.
2026-10-05 18:47:52 +02:00
jochen 8f17db268a Renumber the migration to 0059: main took 0058 (hq ADR 0218) 2026-10-05 18:44:52 +02:00
mesh-admin f80b6cdbd1 Merge pull request 'Say a plan's first send in the machine's own time' (#56) from fix/a-plan-says-local-time into main 2026-10-05 16:35:25 +00:00
jochen 5dcf33db45 Say a plan's first send in the machine's own time, as every other line does
It is kept in UTC and was printed so: 16:32 beside log lines saying 18:32.
2026-10-05 18:35:13 +02:00
mesh-admin 6abce7e956 Merge pull request 'Judge each machine's report from its own send, so a first machine opens the gate (hq issue 256)' (#55) from fix/the-gate-reads-each-machine-from-its-own-send into main 2026-10-05 16:31:55 +00:00
jochen 0d2b304f08 Judge each machine's report from its own send, so a first machine opens the gate
With one machine first a module is sent twice, and the tier gate asked every
machine for a report after the second send: the first machine's report, made
between the two, read as stale and the plan waited for ever (hq issue 256).
Also round a plan's wait to the second, not the minute, so it is not 0s.
2026-10-05 18:29:59 +02:00
mesh-admin bdfbd0254f Merge pull request 'Delivery in order: grants before code, one machine first, a newer plan takes over (ADR 0218; hq issues 249, 252, 254)' (#54) from fix/delivery-in-order into main 2026-10-05 16:21:05 +00:00
jochen 16c5e78fa8 Stop a rollout whose first machine is silent, and choose one that is heard from (hq issue 249, ADR 0218)
A first machine that does not report within the bound now stops the
module's rollout, naming it. The first machine is the first by name heard
from lately; reports are judged by what the store says was last sent. A
plan that ends says what it built and never sent, and a failed send's
error is kept in the plan's note.
2026-10-05 18:17:52 +02:00
jochen ed90771382 Send the bus's machine before the grants, and only when its user list moved (hq issue 249)
Grants first could hold back the very declaration that lets the controller
issue them. The holder now goes first, then buckets and memberships, then
the rest; a membership that fails holds back only its own machine, and an
announcement whose send stopped at its grants is asked again. Whether the
holder must go first is read from a digest of the user list it was last
sent, not its whole declaration. Migration renumbered to 0058.
2026-10-05 18:17:52 +02:00
jochen 672d1f4ca1 Leave an announced move to the plan rolling the module out (hq issue 249)
The catalogue's upgrade announcement sent every machine one after another
without waiting for any to apply, beside the plan that now sends one
machine first. A module an open plan has not finished sending is the
plan's to roll out.
2026-10-05 18:17:52 +02:00
jochen 208901c6cc Let a newer plan supersede the older open plans of its repository (hq issue 254, ADR 0218)
A merge planned without looking at open plans, so two plans worked the
same modules and a stuck plan stayed open for ever. The newer plan folds
in what older plans of the same repository and branch had not built or
sent, and closes them as superseded. A person can close a stuck plan by
id with `plans close <id>`.
2026-10-05 18:17:52 +02:00
jochen 4ac5cfe3a7 Roll a plan's module out to one machine first (hq issue 249, ADR 0218)
A plan sent every machine running a module at once, ignoring the module's
upgrade policy. Unless the policy says together, the first machine by name
is sent, recorded in the plan, and the rest follow only once its report
after the send says it applied; a failed first machine stops the plan.
2026-10-05 18:17:52 +02:00
jochen 22660dc274 Issue grants before code, and the bus's machine first (hq issue 249)
A module's new state reached every machine before the permissions to use
it, which came only with a later push. Memberships and buckets now go
before declarations, the machine holding mesh-broker goes first when its
user list must change, and a grant that fails sends nothing and is an
error so the rollout is retried.
2026-10-05 18:17:52 +02:00
jochen d7f359c498 Read a new module's directory as its own, not as shared code (hq issue 252)
A merge adding a module the mesh has not registered rebuilt every module
built from the repository. A path under a directory known to hold modules
belongs to that module when its module.json is among the changed files.
2026-10-05 18:17:52 +02:00
mesh-admin ed25fd68e2 Merge pull request 'Hold every kept archive by a manifest so the store's collector keeps it (hq issue 253)' (#53) from fix/every-kept-archive-is-held into main 2026-10-05 16:15:15 +00:00
jochen 01c5ab2aab Hold every kept archive by a manifest so the store's collector keeps it (hq issue 253)
The store's garbage-collect marks only from manifests, and archives were
published as bare blobs, so the first real collection would delete every
archive the mesh keeps. PublishArchive now puts a deterministic OCI holder
manifest (empty config, one layer) beside each archive; the sweep holds every
kept archive before it lets anything go, which backfills existing bare blobs,
and lets go of an archive holder-first. A forgotten module no longer keeps its
five recent builds (ADR 0189). `collection [--json]` reports kept archives
held/unheld and what may be let go, so the dry run can be lifted on evidence.
2026-10-05 18:13:23 +02:00
mesh-admin bb3cd6437b Merge pull request 'Two tests that main broke: bus users after issue 195, and the event consumer made from now (issue 248)' (#52) from fix/the-bus-user-test-follows-issue-195 into main 2026-10-05 16:10:01 +00:00
jochen 0b07e68cb8 Publish the followed events once the controller's consumer exists
Made from now since issue 248, the consumer does not replay what was
published before it; the test raced the controller's start and published
first.
2026-10-05 18:04:40 +02:00
jochen 625d02862c Test the bus users as issue 195 made them: an account-reading module is one, another is not
The postgres-backed test still expected a module that declares no broker
secret to be a bus user, and failed on main since #270; it skips without a
database, so the change's own run did not see it.
2026-10-05 18:01:28 +02:00
mesh-admin e8478e208b Merge pull request 'Make the controller's event consumer from now, and give a stuck one a reset (hq issue 248)' (#51) from fix/a-consumer-on-a-history-stream-starts-from-now into main 2026-10-05 15:27:23 +00:00
jochen 5761737687 Name the issue the event consumer fix answers: 248 2026-10-05 17:15:49 +02:00
jochen 89ec48b9a9 Make the controller's event consumer from now, and give a stuck one a reset
A consumer made with the server's default replays everything a stream that
keeps history holds: the controller's EVENTS consumer, re-made that way,
replayed a week of merges and builds one at a time and held every new one
behind them. FromNow makes it start at the end; broker consumer-reset
re-makes a stuck one from now, refusing a work queue. hq issue 244.
2026-10-05 17:15:25 +02:00
jschoubben ac42d89391 No change to a machine takes effect unseen (hq ADR 0217)
Three guards, each silent when nothing is at stake:
- settings: show reads a layer; set says what it adds, changes and removes, refuses a removal
  without --replace, and keeps the layer it replaced (settings_history, migration 0057).
- push: plan --diff compares with what the machine was last sent, now kept as a summary that holds
  no file content; push with no machine needs --all.
- a running container whose mount would point at another directory holds that machine's push
  until --move names the module; the other machines go ahead.
2026-10-05 15:32:28 +02:00
jschoubben 869fb6d6bf Merge pull request 'The node-backup seat (hq ADR 0214, to-be 43)' (#49) from feat/node-backup into main 2026-10-05 10:12:34 +00:00
jschoubben d25b69178b The node-backup seat: a module contributes its backup, the mesh fills its directories
ADR 0214 / to-be 43: a node seat whose holder keeps nightly restore points of what every module on
the machine declares. A contribution of kind backup may name its module's own directories, filled
per module when placed. The catalogue check refuses a store provider that contributes no backup;
parsing does not, so the providers already running stay readable.
2026-10-05 11:42:40 +02:00
mesh-admin a7bb1e0b1c Merge pull request 'node-message-bus: the machine's D-Bus is a node seat (hq ADR 0215)' (#48) from feat/0215-node-message-bus into main 2026-10-05 09:34:14 +00:00
jochen 5eaed84271 node-message-bus: the machine's D-Bus is a node seat (hq ADR 0215) 2026-10-05 11:34:03 +02:00
jschoubben 326b1aec14 Merge pull request 'A dry-run build is taken in by nothing (hq issue 240)' (#47) from fix/a-dry-run-build-is-not-taken-in into main 2026-10-05 09:30:34 +00:00
jschoubben 134d039ff8 Take no dry run in: mark it on the request, echo it on the outcome, set it aside
A dry run of an unreviewed branch was heard by the daemon like any build, registered, and its
definition reached a machine (novox/hq issue 240). The mark now travels with the build and the
daemon records, registers and plans nothing for it.
2026-10-05 09:50:20 +02:00
jschoubben d90c6ab93a Merge pull request 'The mesh's one resolver: its seat, a provider's address, zones, and a node's hosts file (hq ADR 0194, 0196, 0199)' (#251) from feat/mesh-dns-resolver into main 2026-10-04 15:44:31 +00:00
mesh-admin 6b7d2ec49b Merge pull request 'Compose a bus user only for a module that can read an account (hq issue 195)' (#270) from fix/195-only-modules-that-read-an-account-are-bus-users into main 2026-10-04 15:34:06 +00:00
jochen 4ed1057df3 Compose a bus user only for a module that can read an account
Every assigned module was composed as a bus user, though only one declaring
a broker secret can ever be issued an account; the rest were named on every
status, plan and push as credentials never minted (137 now), burying the
real gaps. Their durable consumers are now derived from what the runtime
carries, so nothing they hear changes. The composed file is unchanged:
those users had no password and were already left out. Fixes hq issue 195.
2026-10-04 17:32:50 +02:00
jschoubben 1f4c67a01b The mesh's one resolver: its seat, a provider's address, zones, and a node's hosts file (hq ADR 0194, 0196, 0199)
- mesh-dns-resolver: a mesh seat delivering wildcard-resolution, so every node's resolver
  configuration resolves to its one holder; node-dns-resolver kept until nothing claims it.
- ${bound:<provision>:address}: the providing machine's private address, for the one consumer
  that cannot use a name — a machine's resolver configuration.
- zone: a module declares the zone it answers and the listen that answers it; the controller
  settles it per node, refuses duplicates and shadowing, and hands the resolver .Zones to forward.
- node-hosts-file: a node seat whose holder owns /etc/hosts, with entries/add/remove.
The resolver tests follow the catalogue: no runtime dns (containers copy the machine's resolvers),
live-restore held by resolv-conf, resolv.conf naming the resolver by address then a public one.
2026-10-04 17:32:08 +02:00
mesh-admin 5474ea2d41 Merge pull request 'A seat says what it receives, and its holder places every module's contribution (hq ADR 0212)' (#269) from feat/0212-contributions-to-a-seat into main 2026-10-04 14:48:13 +00:00
jochen b7912172af A seat says what it receives, and its holder places every module's contribution (hq ADR 0212)
Each contribution grain was a manifest field and a renderer of its own; a module now contributes
to any seat with a kind that seat receives, the holder places it with
${contribution:<seat>:<kind>}, and the contribution depends on the seat. node-hotkeys is the
first new seat to receive (triggers); the display session receives window-manager config.
2026-10-04 16:48:01 +02:00
mesh-admin b36babcd7d Merge pull request 'node-power: the power seat, and code for its moments (hq ADR 0211)' (#268) from feat/0211-node-power into main 2026-10-04 13:59:28 +00:00
jochen 49cf0aa562 node-power: the power seat, and code for its moments placed by its holder (hq ADR 0211)
A module that needs code after waking wrote into the service manager's sleep units; it now
contributes shell code for a named moment, which derives a dependency on node-power.
2026-10-04 15:59:17 +02:00
mesh-admin 5a4f73f761 Merge pull request 'A contribution depends on the seat that receives it; a collision is refused at assign (hq ADR 0210, issue 235)' (#267) from feat/0210-a-contribution-depends-on-its-seat into main 2026-10-04 13:45:46 +00:00
jochen 6af891e358 A contribution depends on the seat that receives it, and a collision is refused at assign (hq ADR 0210, issue 235)
The environment and shell contributions were written nowhere on a node without their holder;
they now derive a dependency on node-environment, node-login-shell or node-display-server, met
and refused as ADR 0207's are. Two modules declaring one package, path or unit made the node
unresolvable after the assignment was recorded; that is refused first now, because no later
assignment can complete it.
2026-10-04 15:45:35 +02:00
mesh-admin 568f55fd2e Merge pull request 'Refuse an unmet seat dependency the catalogue could meet (hq ADR 0207 §4)' (#266) from feat/0207-refuse-unmet-seat-dependencies into main 2026-10-04 11:07:52 +00:00
jochen 35314175f2 Refuse an unmet seat dependency the catalogue could meet (hq ADR 0207 §4)
status reported no unmet dependency on any node once systemd, pacman and docker
were assigned to all four (to-be 42), which is the condition ADR 0207 set for the
switch. A dependency no catalogue module could meet stays a report before and
after the switch, as assign already said it: there is no remedy to name.
2026-10-04 13:07:45 +02:00
mesh-admin ec2e6255a9 Merge pull request 'The unmet-dependency report names only the nodes an act touched (hq ADR 0207)' (#265) from fix/unheld-report-names-only-the-nodes-acted-on into main 2026-10-04 10:52:34 +00:00
jochen f3f34a170e Hold ssh-client to its new shape: an include region first, the mesh's hosts in config.d (mesh-catalog #266) 2026-10-04 12:52:03 +02:00
jochen e2622fd031 An act says the unmet seat dependencies of the node it acted on, not the mesh's (hq ADR 0207)
assign and unassign say only what they changed on their node; push <node>
lists that node's, push to many counts each and points at status. The
once-per-change log is the serving controller's alone: a one-shot command
starts with no memory, so it logged every node on every call.
2026-10-04 12:50:25 +02:00
mesh-admin 3ee32970ef Merge pull request 'Seat dependencies (hq ADR 0207), the graphical session's seats and display provisions (ADR 0208), groups from several modules' (#264) from feat/0207-a-module-depends-on-the-seats-that-apply-its-resources into main 2026-10-04 10:43:11 +00:00
jochen 11b654499b Several modules may add groups to one account; its shell and home stay one module's
The host only ever adds groups, so the container runtime's module can put the
operator in its group while the shell's module sets the same account's shell.
2026-10-04 12:42:00 +02:00
jochen d69e19103c The graphical session's seats, a display's machine reach, and the session's slots (hq ADR 0208)
Seed the eleven node seats with the verbs they start with. A provision may
have the machine's reach: a requirement for it resolves only to a provider
in the node's own set, is never pulled in, and is refused naming who could.
A shell contribution's for gains xinitrc and xresources, placed only by the
holder of node-display-server.
2026-10-04 12:38:53 +02:00
jochen 10f948e970 A module depends on the node seats that apply its resources (hq ADR 0207)
Seed node-package-manager and node-container-runtime. Derive each module's
dependencies from its declared service, package and container resources;
judge them over the node's whole set, exempting the foundation. Refuse at
assign (several modules may go on as one act) and at unassign of the last
holder; report at composition in status, behind one switch.
2026-10-04 12:34:11 +02:00
mesh-admin f421d4588c Merge pull request 'A grant secret belongs to whoever provisions (hq 225); the sweep skips what it will not address (hq 226); a container publishes only what it declares (hq 227)' (#263) from fix/a-grant-secret-is-read-by-the-account-that-provisions into main 2026-10-04 10:27:30 +00:00
jschoubben 74b0dab34c A container publishes only a port its module declares (hq issue 227)
The short form is a question the mesh answers: "80" means publish what the
software calls 80, and the mesh fills in the machine's half from the port it
assigned. It can only assign one for a port the module declared, so a number
appearing nowhere in listens gets no assignment and reaches the machine as
written — which is how the photo module asked for port 80 on the node whose
reverse proxy holds it.

Four modules publish 80 quite safely, because they declare 80. The difference
is the declaration, not the number. A catalogue-wide test now says so; it
names all three offenders against the catalogue as it was.
2026-10-04 12:25:27 +02:00
jschoubben 41b20b2782 A grant secret belongs to whoever provisions, and the sweep skips what it will not address
Issue 225. The mesh seals one credential per consumer beside the provider's
contributions file, and wrote it root-owned. That was right while a module's
own code ran in a container as root; ADR 0198 moved that code under the node's
runtime, as the node's account, and the secret stayed root's. On the control
machine two consumers went unprovisioned for three hours and the only sign
was a line reading 'secret not readable yet', 4330 times.

The same sentence is already written for a module's own secrets a few hundred
lines above — 'a root-owned 0600 file is one that process cannot read'. This
is that rule reaching the other kind of secret the mesh writes for a module.

Issue 226. The sweep met a reference recorded with the store's old address,
read 'I will not address this' as 'the store refuses everything', and
collected none of the 1681 it had found. Two changes: references from build
records are read through Recorded, where the provenance is known — not in
LetGo, which cannot tell one registry host from another and must stay strict
— and a reference the sweep will not address is now ErrNotOurs, skipped,
never a reason to stop. Only the store refusing ends a sweep.

make check: the two failures both fail on main as well — the resolver test
(hq 202/203) and the service-manager test, which reads this machine's own
shell environment.
2026-10-04 12:21:49 +02:00
mesh-admin 912e9f4e85 Merge pull request 'Assert every declared state's bucket on each push (hq ADR 0201)' (#262) from fix/buckets-on-push into main 2026-10-04 09:21:25 +00:00
jochen babd7b2f47 Assert every declared state's bucket on each push, before the memberships that name it (novox/hq ADR 0201)
The raise at start was the only place buckets were asserted, so a module
registered and assigned since had none until the control plane restarted —
found on the first module to declare state.
2026-10-04 11:13:34 +02:00
mesh-admin 892dfd1d08 Merge pull request 'Module state is hq ADR 0201 after all' (#261) from fix/module-state-is-0201 into main 2026-10-04 09:03:15 +00:00
jochen cfac579392 Module state is hq ADR 0201 after all: the derived-value record moved to 0202 on hq main 2026-10-04 11:02:42 +02:00
mesh-admin fe0d295490 Merge pull request 'Module state is hq ADR 0202 (0201 landed first for a provider's derivations)' (#258) from fix/adr-0202-module-state into main 2026-10-04 09:01:22 +00:00
mesh-admin d8a0238e02 Merge pull request 'The account's environment and the shell's contributions (hq ADR 0203, ADR 0204, to-be 41 WP2)' (#260) from feat/the-shell-and-its-environment into main 2026-10-04 08:49:35 +00:00
jochen dcf710a8d5 Merge remote-tracking branch 'origin/main' into feat/the-shell-and-its-environment 2026-10-04 10:31:03 +02:00
mesh-admin a2003ab616 Merge pull request 'while-stopped names the container as the machine knows it (hq ADR 0189)' (#259) from fix/while-stopped-names-the-composed-id into main 2026-10-04 02:18:44 +00:00
jschoubben 1363a2fe27 while-stopped names the container as the machine knows it (hq ADR 0189)
A module names its own resources locally; a declaration names them under the
module. restart-on and reload-on are rewritten for exactly that reason and
while-stopped was not, so the store's step said it held "store" still while
the machine's container is "distribution.store".

The host refuses a declaration naming a container it does not have — whole.
So novox took nothing at all, on every push, from 04:15 until this. The
machine was never damaged: refusing whole is what kept it serving.

Both sides' tests passed throughout. The controller's read manifests, the
host's read hand-written declarations with bare ids, and nothing composed one
and judged the result. That test now exists.
2026-10-04 04:18:22 +02:00
jochen f19a2254ac Compose the account's environment and the shell's code from every module (hq ADR 0203, 0204)
A module contributes environment variables, PATH entries and shell code in named slots;
the holder of the matching seat places them with ${environment:posix|systemd} and
${shell:<shell>:<slot>}. Rendered in module order with a naming line per contribution,
PATH entries added only when missing, machine facts resolved first. A variable two
modules set, or a placeholder outside its seat's holder, is refused at parse (the
catalogue check) and at composition. Filled after every other placeholder pass, so no
scanner ever reads a shell's own ${...}.
2026-10-04 04:03:50 +02:00
jochen 7d46e48b26 The account's environment and the login shell are the mesh's seats (hq ADR 0203, 0204)
node-environment says which module writes the account's environment; node-login-shell
replaces the module-declared login-shell, so a second shell claims it rather than
declaring a rival, and execute is the mesh's contract. login-shell is refused as a
module's seat name. Seeded into a live store by the existing additive seeding.
2026-10-04 04:03:50 +02:00
jochen 78915f9f7a Module state is hq ADR 0202: 0201 landed first for a provider's derivations 2026-10-04 03:44:50 +02:00
mesh-admin 17b8f14fe1 Merge pull request 'A module's state on the bus: buckets from the catalogue, grants, membership (hq ADR 0201)' (#257) from feat/module-state-on-the-bus into main 2026-10-04 01:43:36 +00:00
mesh-admin 42c394acc2 Merge pull request 'Group 8: a served value may name its consumer (hq ADR 0201), and the store keeps what the records name (hq ADR 0189)' (#227) from feat/the-store-keeps-what-the-records-name into main 2026-10-04 01:39:39 +00:00
jschoubben b9ad7a2948 Review before merge: refuse a silent disagreement, and bound the sweep
Three things found reading this back, each of which would have been quiet.

A consumer that keeps several holders of one provision (ADR 0094) gets a
login per holder, and a provider derives from the login — so it would make a
resource per holder while the consumer is told one value for the requirement.
That is issue 124's own failure one case to the side: authenticate, then be
refused on every object. Refused now, naming both ends.

The sweep runs inside somebody's build and was unbounded. At most two hundred
artifacts and sixty seconds, stopping at the first refusal because a store
that refuses one refuses all; the rest is offered again next build.

The citation and migration renumbers are in the commit before this one.
2026-10-04 03:27:32 +02:00
jochen cde22ff627 module check names a read of state its owner does not keep, and says what each module keeps and reads (novox/hq ADR 0201) 2026-10-04 02:50:02 +02:00
jschoubben 79993fb498 Rebased onto main: ADR 0188 renumbered to 0201, migration 0055 to 0056
The bundles refactor took ADR 0188 on main, so this work's record is 0201 and
every comment citing it moves with it. Main also took migration 0055 (an
older build never replaces a newer), so the store's collected-artifacts table
is 0056 — a number two migrations share is a schema nobody can trust.

make check passes except TestTheResolverIsToldEveryMachineOnTheNetworkAndToldAgainWhenOneLeaves,
which fails on main too and now for two stacked reasons (hq issues 203 and 202).
2026-10-04 02:45:06 +02:00
jochen aec55b7072 A module's state on the bus: buckets from the catalogue, grants, membership (novox/hq ADR 0201)
A manifest names the state it keeps (state) and reads (reads); the controller
asserts a key-value bucket per name on every raise, grants owners write and
readers read (measured against a running server), issues each assignment its
buckets in the membership, and reports buckets nothing declares without
removing them.
2026-10-04 02:40:49 +02:00
jschoubben c7884f5a72 The store keeps what the records name (hq ADR 0189)
The mesh names what may go from its own build records — a digest it did not
record making is never named, which is what keeps the sweep away from the
images genesis pushed. An artifact stays because a definition the mesh holds
names it, or because it belongs to one of the five most recent successful
builds of its module.

internal/artifacts asks the store to let go of one; internal/inventory
decides and remembers (migration 0055); the sweep runs after a build the mesh
recorded, which is when both the bytes and the keep set moved. Never fatal to
a build.

And the manifest side of while-stopped, refused from the definition alone:
no schedule, run-once, a container the module does not declare, itself.
2026-10-04 02:32:19 +02:00
jschoubben 580c4d66a7 A served value may name the consumer it is served to (hq ADR 0188)
${consumer:as} and ${consumer:as:dns} in a serves block are filled per
consumer at resolution, and the one filled value reaches both ends: the
consumer's binding and its ${bound:...} substitutions, and the provider's
contributions entry as `derived`. A fact or alphabet the mesh does not have
is refused at parse; a consumer whose own file already holds the derived
value is refused at resolution, naming the placeholder to write instead.
2026-10-04 02:32:19 +02:00
mesh-admin 63bfc5fda5 Merge pull request 'Refuse the tools-container shape for every module (to-be 38 WP4b's last step)' (#256) from feat/wp4b-the-gate-refuses-the-container-shape-for-all into main 2026-10-04 00:28:59 +00:00
jochen 02e3482eb5 Refuse the tools-container shape for every module (to-be 38 WP4b's last step)
While some thirty modules still stood in that shape, one already registered so was rebuilt without
complaint. Every module has moved since; the exception would only let one move back.
2026-10-04 02:28:54 +02:00
mesh-admin 294e83dab1 Merge pull request 'Run the controller as a Go bundle the host starts as a process (hq issue 213, 2 of 2)' (#253) from fix/issue-213-the-controller-is-a-process-manifest into main 2026-10-04 00:06:44 +00:00
jochen b5438bb331 Pin the image's Go base in the Dockerfile, which genesis builds as it stands (hq issue 223)
Genesis now raises a process-form controller as a container built from
this repository's Dockerfile with no build arguments (mesh-host
bootstrap, novox/hq issue 223); the manifest builds no image, so nothing
passes the base in. The default was a tag older than go.mod asks for.
It is now the digest the Makefile pins, and a test holds the two equal.
2026-10-04 01:49:05 +02:00
jochen c23be73d4d Run the controller as a Go bundle the host starts as a process (hq issue 213)
The controller is a Go program and was the one piece of the mesh's own Go
code still shipped and run as an image (novox/hq issue 213; ADR 0188 §1:
a module's own code is bundles; §3: a service bundle is a process).

The manifest now builds one Go bundle, `controller`, and runs it as the
process `mesh-controller` (`./mesh-controller serve`) under an account
the module declares. What the container gave it, replaced:

- host network: a process is on the host's network; nothing it reads
  names a container network
- user 65534: the account `mesh-controller`, which owns its secrets and
  its state directory
- the eight mounts: the env names the host paths the mesh already places
  (the store, broker and bus files under the state directory, the
  broker's certificate under /var/lib/mesh-broker-tls); the `broker`
  mount was read by nothing and is gone with the others
- `container-runtime` is no longer required on its machine

Its preparation is the same binary with `prepare`, as a run-once process,
and the process `replaces` the container `server`: the host keeps the
container answering until the process is running (mesh-host). Needs the
previous commit live in the running controller, and the host's
`replaces` on the controller's machine, before it is registered.

No image is built by the mesh any more. The Dockerfile stays for genesis
and the lab (`make image`, its Go base now pinned in the Makefile).
2026-10-04 01:45:25 +02:00
mesh-admin d2d171f2d2 Merge pull request 'Compose a module's Go service as a process the host runs (hq issue 213, 1 of 2)' (#252) from fix/issue-213-the-controller-is-a-process into main 2026-10-03 23:40:38 +00:00
mesh-admin 73fa64ea68 Merge pull request 'A TypeScript bundle installs its module's own packages before it is compiled (hq ADR 0198 §4)' (#255) from feat/a-bundle-installs-its-own-packages into main 2026-10-03 23:20:57 +00:00
jochen 1a13dbeb17 A TypeScript bundle installs its module's own packages before it is compiled
A bundle could import only what the toolchain image carried: the compiler and the bundler resolve an import from the module's directory and then the toolchain's node_modules, and nothing ever put anything in the first. So a module needing a database driver (pg, mongodb, mssql) could not be a bundle, and kept a container whose recipe installed it (hq ADR 0198 §4: the backend's own driver inside the bundle).

Now, when a module's package.json depends on anything beyond the SDK, the build installs its production dependencies into the module's directory, in the toolchain image, before the compile: npm ci from the lockfile when there is one, npm install from the ranges otherwise, the mesh's registry for the SDK's scope and the public one for the rest, install scripts off. esbuild then inlines them. A module depending only on the SDK runs exactly the commands it did before.

The SDK stays the toolchain's (hq issue 212): it is taken out of what is installed and any copy something pulls in is removed, so every import of it resolves past the module's node_modules to the one the toolchain carries; a module's own range never shadows it. npm's verified download cache is a named volume; nothing installed is kept between builds. Without a registry, a scoped package is refused rather than resolved on the public registry.
2026-10-04 01:17:49 +02:00
jochen e11caecdad Let two controllers overlap safely while one hands over to the other (hq issue 213)
The controller's machine moves it from the container to a process by
starting the process first and removing the container once the process
is up (mesh-host's `replaces`). For that moment two controllers share the
store and the bus. Checked what each does:

- the seat's verbs: a queue group per seat, each call answered once. Safe.
- the controller's consumers on CONTROL and EVENTS: push consumers with
  no delivery group, so the second bind is refused with "consumer is
  already bound" and serve exited. The process would restart for ever,
  the host would never see it up, and the container would never go. The
  second controller now stands by and binds when the first lets go
  (tested on a real bus; fails without the change).
- plans: read, changed and saved whole by the 30s timer, by build
  outcomes, by a merge and by `plans stop`. Two timers would each ask a
  tier the other had just asked. Working the plans now takes a
  session-level advisory lock on the inventory: the timer skips while
  another holds it, the other paths wait for it. Build asks happen only
  inside plan work and are covered by the same lock.
2026-10-04 01:11:26 +02:00
jochen 7bb9e55d0b Compose a module's Go service as a process the host runs (hq issue 213)
The controller is to be declared as a Go bundle run by a process instead of
an image (novox/hq issue 213, ADR 0188 §1, §3). The composer could not
express that honestly yet:

- a module declaring tools had every bundle served by the node's runtime,
  so the controller's own binary would have been launched a second time as
  an MCP child; a bundle one of the module's resources runs is now served
  only when it says `loads`
- a module's accounts went after the mesh-computed files, so secrets owned
  by the account a process runs as were refused on the first apply; a
  module's `user` resources now go first
- `prepares` derived its step only from a container; a process is now
  prepared by the same program with `prepare` as a run-once process
- a process may say what it `replaces` (a resource of its module it no
  longer declares), prefixed as the host records it, so the host keeps the
  old one running until the process is (needs mesh-host's `replaces`)

This lands before the controller's manifest uses any of it: the running
controller composes its own declaration, so the code that fills the new
shape must be live first.
2026-10-04 01:11:26 +02:00
mesh-admin 00037608ae Merge pull request 'The forge's tests compose its code as a bundle the node's runtime serves (hq ADR 0198, to-be 38 WP4c)' (#254) from feat/0198-waves-2-3-the-forges-code-is-a-bundle into main 2026-10-03 23:01:17 +00:00
jochen cea59428b1 The forge's tests compose its code as a bundle the node's runtime serves (hq ADR 0198)
gitea's own code moves out of its runtime container (mesh-catalog, to-be 38 WP4c waves 2-3), so the three tests that composed the forge from the catalogue beside this checkout resolve its build as the code bundle, compose it beside the node's runtime, and read the forge's address from the words the runtime hands the module rather than from a sidecar's env.
2026-10-04 00:50:06 +02:00
mesh-admin 5c832f2d19 Merge pull request 'Compose a process's environment as a container's' (#250) from feat/a-process-env-is-composed-like-a-containers into main 2026-10-03 22:29:11 +00:00
jochen cdebb7d1a5 Compose a process's environment as a container's
A module's own code moving out of its container (novox/hq to-be 38 WP4c)
becomes a process on the machine, and still has to be told what its
container was: the port this machine gave the module and where the
foundation's seats are. ${port:…} and ${seat:…} were filled only in a
file's content and a container's env, so in a process's env they reached
the machine as literals, and the modules that moved first (mesh-catalog
#245) wrote their run-once steps a 0600 env file instead. A process's env
now takes the same resolution and the same refusals; ${dir:…} and
${access:…} already did, and a bundle's env (ADR 0192) already resolves
${dir:…} and ${port:…}.
2026-10-04 00:27:42 +02:00
mesh-admin 6a803ea5b3 Merge pull request 'Issue 219: an older build request never replaces a newer one's artifact' (#249) from fix/issue-219-an-older-build-never-replaces-a-newer into main 2026-10-03 22:23:31 +00:00
jochen 9745c1ab31 An older build request never replaces a newer one's artifact
Builds of one module in flight together finish in any order, and the mesh
took whatever it heard last as what the module is: RegisterModule overwrote
the module's manifest unconditionally, and Held/BuiltAgainst/ReadRepositories
ordered builds by when they were recorded. A postgres build asked before the
mesh-tools runtime fix finished after the one asked after it, and the next
push deployed the stale image (novox/hq issue 219).

A build is now ordered by when it was asked, read from the build-<nanos> id
the controller writes: build.asked and module.built_asked (migration 0055).
A registration from an earlier request than the module's current one is
recorded and refused as superseded. A plan takes as its outcome only a build
asked at or after its own ask, so an earlier plan's leftover build cannot
settle a later plan. Ids of any other shape keep the old order.
2026-10-04 00:22:11 +02:00
mesh-admin c0c3c3fed4 Merge pull request 'A TypeScript bundle is one file per entrypoint, bundled in the toolchain (hq ADR 0193); a toolchain follows the SDK it stands on (hq issue 212)' (#247) from feat/a-typescript-bundle-is-one-file into main 2026-10-03 21:50:44 +00:00
mesh-admin c1449fffe9 Merge pull request 'Issue only the recorded holder a seat held once for the mesh (hq issue 218)' (#248) from fix/issue-218-only-the-holder-serves-a-mesh-seat into main 2026-10-03 21:30:50 +00:00
jochen f873c97db5 Issue only the recorded holder a seat held once for the mesh (novox/hq issue 218)
A module claiming a mesh-scoped seat was granted and issued the seat's subjects on every machine
it runs on, so the store's verbs answered from whichever postgres replied first. Where the mesh
records the seat's holder, only that (node, module) is now issued it; the module's own tools are
untouched everywhere.
2026-10-03 23:30:27 +02:00
jochen b0b3d87fe2 Run the toolchain's esbuild as itself: npm installs its native binary in place of the script 2026-10-03 23:26:08 +02:00
jochen ba189e6943 A TypeScript bundle is one file per entrypoint, bundled in the toolchain (hq ADR 0193); a toolchain follows the SDK it stands on (hq issue 212)
Every served bundle is its own process now, so each carries its own copy of what it imports: after
the compile and the launchers, the toolchain image's esbuild bundles every entrypoint in place and
every launcher under its own name into one ES module file, the SDK inlined, require provided to
inlined CommonJS, the launcher's shebang kept and its mode 0755. The toolchain's node_modules is
copied only for packages an artifact names external. An image without the bundler is refused by
name. Issue 212: build.on already passes a published package by its exact version and plans the
toolchain after it; tests say so.
2026-10-03 23:24:08 +02:00
mesh-admin cadf74a176 Merge pull request 'Tools pipeline: issues 214, 215, 216, Go tools bundles served, and the runtime consumes for its modules (ADR 0193, 0198)' (#246) from fix/tools-pipeline-issues-214-216-and-0198 into main 2026-10-03 21:16:13 +00:00
jochen 74efe8e2e7 Tests follow the grants and issue 203: a person may ask what answers; the resolver test mints its credential 2026-10-03 23:15:57 +02:00
jochen 68af9eff44 Merge remote-tracking branch 'origin/feat/0198-the-runtime-consumes-for-its-modules' into integrate 2026-10-03 23:12:21 +02:00
jochen 518eeb7941 integrate: go served 2026-10-03 23:12:21 +02:00
jochen bf2da878a0 Merge remote-tracking branch 'origin/fix/issue-216-a-bundle-nothing-delivers-is-refused' into integrate 2026-10-03 23:12:03 +02:00
jochen 50cf253a43 Merge remote-tracking branch 'origin/fix/issue-215-a-commit-is-never-a-branch-to-follow' into integrate 2026-10-03 23:12:03 +02:00
jochen 980a0dee93 integrate: 214 2026-10-03 23:12:03 +02:00
jochen 8b016cc62b A Go tools bundle is served by its binary (hq ADR 0193)
A bundle compiled to a binary has no entrypoints, and loads had to name one, so a Go bundle could
not be served. Its binary is what the runtime starts: loads names the binary, derived when the
module lists tools, and the runtime is told the binary's path, delivered like any tools bundle.
2026-10-03 22:27:01 +02:00
jochen cf2bb3b87d A bundle nothing would deliver is refused at registration (hq issue 216)
The composer delivers a bundle when the runtime loads from it, a resource names it, or it is the
runtime; one reached by none of them was built, recorded and pushed as success and was simply
absent. Seven modules' tools went missing that way. Refused at registration, naming the field that
would deliver it.
2026-10-03 22:25:34 +02:00
jochen 6784efae75 A commit is never a branch to follow (hq issue 215)
A build asked at a commit recorded that commit as the module's ref. Every merge after it failed to
match the module and its plan left it out without a word, and every plan that rebuilt it asked for
the same old commit again. Registration now keeps the branch the module followed (the default
branch for a new one); matching and re-asking read a recorded commit as the default branch, which
heals records already pinned this way; and a merge says which modules of its repository it leaves
out because they follow another branch.
2026-10-03 22:22:08 +02:00
jochen d86baebe9a A plan settles an asked build from the build records (hq issue 214)
A merge to the controller's own repository replaces the controller in its first tier; the build
that produced the new one was recorded, the plan never heard it, and it waited for ever with every
later plan behind it. The record is the fact: a build recorded after the ask is the tier's outcome,
whoever was listening when it came.
2026-10-03 22:20:33 +02:00
133 changed files with 12787 additions and 486 deletions
+8 -2
View File
@@ -1,5 +1,11 @@
ARG GO_BASE=golang:1.25-alpine
# The control plane's image.
# The Go it builds with, pinned here because genesis builds this file with no arguments (novox/hq
# issue 223) — the Makefile passes the same digest. A tag older than go.mod asks for is how
# `make image` broke once before (issue 146).
ARG GO_BASE=golang@sha256:8ac98ca534ac3f51e1f420a1dd2c15e74c75cfa0f23f3ad27eb5d7236c349a0c
# The control plane's image — for genesis and the lab only. The mesh runs the controller as a Go
# bundle the host starts as a process (module.json; novox/hq issue 213), and builds no image of it.
# Genesis builds this file and raises it as the container the process replaces on the first push
# (mesh-host internal/bootstrap, novox/hq issue 223).
#
# novox/hq ADR 0006: this image is pinned by digest in the bundle the host carries, fetched on a
# machine where no mesh exists yet, and run before there is anything to check it against. So it
+8 -4
View File
@@ -27,17 +27,21 @@ build:
IMAGE ?= mesh-controller:$(VERSION)
DEV_TAG ?= mesh-controller:development
# The base the module declares, read from the manifest rather than written here twice.
# The Go base the image is built on.
#
# **`make image` was broken and stayed broken**, because the Dockerfile's fallback base was a Go
# older than go.mod asks for: every build died at `go mod download` with "go.mod requires go >=
# 1.26.0", and the pipeline never saw it because the pipeline passes the declared base in. Anybody
# building the image by hand hit it and had to find the digest themselves (novox/hq 04-ISSUES/146,
# what it cost).
GO_BASE ?= $(shell python3 -c "import json;print(next(o['image'] for o in json.load(open('module.json'))['build']['on'] if o['arg']=='GO_BASE'))" 2>/dev/null)
#
# **Pinned here since the manifest stopped building an image** (novox/hq issue 213): the mesh builds
# the controller as a Go bundle with its own toolchain, and only `make image` — genesis and the lab —
# still needs a Go base. The digest is the one the manifest declared until then.
GO_BASE ?= golang@sha256:8ac98ca534ac3f51e1f420a1dd2c15e74c75cfa0f23f3ad27eb5d7236c349a0c
image:
@test -n "$(GO_BASE)" || { echo "module.json declares no GO_BASE; pass GO_BASE=<image> or fix the manifest"; exit 1; }
@test -n "$(GO_BASE)" || { echo "no GO_BASE; pass GO_BASE=<image>"; exit 1; }
docker build --build-arg GO_BASE=$(GO_BASE) --build-arg VERSION=$(VERSION) -t $(IMAGE) -t $(DEV_TAG) .
@echo
@docker image inspect $(IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
@@ -48,7 +52,7 @@ BUILDER_IMAGE ?= mesh-builder:$(VERSION)
BUILDER_DEV_TAG ?= mesh-builder:development
builder-image:
@test -n "$(GO_BASE)" || { echo "module.json declares no GO_BASE; pass GO_BASE=<image> or fix the manifest"; exit 1; }
@test -n "$(GO_BASE)" || { echo "no GO_BASE; pass GO_BASE=<image>"; exit 1; }
docker build --build-arg GO_BASE=$(GO_BASE) -f cmd/mesh-builder/Dockerfile -t $(BUILDER_IMAGE) -t $(BUILDER_DEV_TAG) .
@echo
@docker image inspect $(BUILDER_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
+7
View File
@@ -193,6 +193,13 @@ passes every check that only looks at the message.
## The image
**The mesh no longer runs the controller from it** (novox/hq issue 213). The module declares a Go
bundle, `controller`, which the host on the controller's machine unpacks and runs as the process
`mesh-controller` under the account of the same name (ADR 0188 §1, §3). The image stays for what
still runs a container of the controller: genesis, which raises the first controller from it and
installs the module from its manifest (mesh-host `internal/bootstrap`), and the lab. Neither is the
mesh's own build any more — `make image` builds it.
`FROM scratch`, holding one statically linked binary and nothing else — no shell, no package
manager, no libc, no CA certificates.
+1 -1
View File
@@ -171,7 +171,7 @@ func answer(ctx context.Context, publisher builder.Publisher, on, workspace stri
result := link.BuildResult{
ID: request.ID, Repository: request.Repository, Path: request.Path,
Ref: request.Ref, On: on, Source: request.Source,
Ref: request.Ref, On: on, Source: request.Source, DryRun: request.DryRun,
}
what := "building " + request.Repository
if request.Path != "" {
+147 -22
View File
@@ -4,6 +4,7 @@ import (
"context"
"fmt"
"github.com/novox/mesh-controller/internal/broker"
"slices"
"sort"
"strings"
@@ -39,7 +40,10 @@ import (
//
// It costs a resolution per machine. Assignment is a person typing a command, and being told which
// machines this just blocked is worth more than the milliseconds.
func assign(ctx context.Context, open *stores, node, module string) (string, error) {
func assign(ctx context.Context, open *stores, node string, modules ...string) (string, error) {
if len(modules) == 0 {
return "", fmt.Errorf("assign %s names no module", node)
}
// Held while it is recorded, so it cannot land between a converge's preview and its flip and
// be taken without ever having been previewed (novox/hq ADR 0100).
ctx, release, err := holdNodes(ctx, open, []string{node})
@@ -47,6 +51,16 @@ func assign(ctx context.Context, open *stores, node, module string) (string, err
return "", err
}
defer release()
// **The one assignment refused for what the node lacks** (novox/hq ADR 0207). Everything else
// an assignment leaves unresolved is kept, because assignment is not an ordering; a module whose
// resources are applied through a seat nothing on the node holds is refused, because that order
// — the service manager, the package manager and the runtime before anything that installs,
// runs or contains — is the mesh's to keep. Several modules in one act are judged together, so
// holders that depend on each other go on in one command.
shelf, before, err := seatDependenciesOnAssign(ctx, open, node, modules)
if err != nil {
return "", err
}
// **Before the new assignment can unsettle a seat somebody holds only by being alone**
// (novox/hq 04-ISSUES/170): what the mesh derived so far is written down, and then the
// assignment resolves against a record rather than against a coincidence.
@@ -54,65 +68,176 @@ func assign(ctx context.Context, open *stores, node, module string) (string, err
if err != nil {
return "", err
}
fresh, err := open.inventory.Assign(ctx, node, module)
if err != nil {
return "", err
var lines []string
var added []string
for _, module := range modules {
fresh, err := open.inventory.Assign(ctx, node, module)
if err != nil {
return strings.Join(lines, "\n"), err
}
if !fresh {
// Nothing changed, and saying "is assigned" would read as an action. One node runs one
// of each — the module's name is the assignment's identity (novox/hq ADR 0115).
lines = append(lines, fmt.Sprintf(
"%s already runs %s — one node runs one of each (ADR 0115); nothing changed", node, module))
continue
}
added = append(added, module)
lines = append(lines, fmt.Sprintf("%s is assigned %s", node, module))
}
if !fresh {
// Nothing changed, and saying "is assigned" would read as an action. One node runs one
// of each — the module's name is the assignment's identity (novox/hq ADR 0115).
return fmt.Sprintf("%s already runs %s — one node runs one of each (ADR 0115); nothing changed",
node, module), nil
if len(added) == 0 {
return strings.Join(lines, "\n"), nil
}
said := fmt.Sprintf("%s is assigned %s", node, module)
answer := strings.Join(lines, "\n")
for _, line := range settled {
said += "\n " + line
answer += "\n " + line
}
// What this act changed about this node's unmet seat dependencies, and nothing else (novox/hq
// ADR 0207): a dependency of a module just assigned, or one this assignment met. The rest of the
// node's list, and every other node's, is `status`'s.
for _, line := range unheldChange(shelf, node, before, append(append([]string(nil), before...), added...)) {
answer += "\n " + line
}
// Its bus credential, in the same act (novox/hq issue 203): an assignment pushed before its
// credential exists delivers a process that cannot authenticate and crash-loops until somebody
// runs a second verb and a second push. Issued here when the module speaks on the bus and has
// no credential yet; kept when it has one, so re-assigning rotates nothing.
if line := issueOnAssign(ctx, open, node, module); line != "" {
said += "\n " + line
for _, module := range added {
if line := issueOnAssign(ctx, open, node, module); line != "" {
answer += "\n " + line
}
}
plan, _, err := planFor(ctx, open, node)
if err != nil {
// Kept, and still refused. Both halves are the answer, and the rest of the mesh is still
// worth reporting: this machine's refusal is rarely the only consequence.
return said + blockedElsewhere(ctx, open, node), err
return answer + blockedElsewhere(ctx, open, node), err
}
// Kept, and cannot be hosted here. Said at once rather than discovered at push: a module whose
// capability the machine lacks is on the wrong machine, and the assignment records what a person
// meant while this line says it will not run until it moves. The rest of the node still pushes.
isAdded := map[string]bool{}
for _, m := range added {
isAdded[m] = true
}
for _, u := range plan.Unhostable {
if u.Module != module {
if !isAdded[u.Module] {
continue
}
for _, c := range u.Missing {
said += "\n but " + catalogue.WrongMachine(u.Module, c, node)
answer += "\n but " + catalogue.WrongMachine(u.Module, c, node)
}
}
return said + fmt.Sprintf("\n run `push %s` to send it", node) +
return answer + fmt.Sprintf("\n run `push %s` to send it", node) +
blockedElsewhere(ctx, open, node), nil
}
// unassign takes a module off a node. What it leaves behind is the host's business: a directory
// seatDependenciesOnAssign is the refusal ADR 0207 makes at assignment, or nothing, with the
// catalogue and the node's assignments it was judged against. Modules already assigned are not new
// and are not judged again.
func seatDependenciesOnAssign(ctx context.Context, open *stores, node string, modules []string) (
map[string]catalogue.Manifest, []string, error) {
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
return nil, nil, err
}
assigned, err := open.inventory.Assigned(ctx, node)
if err != nil {
return nil, nil, err
}
already := map[string]bool{}
for _, a := range assigned {
already[a] = true
}
var adding []string
for _, m := range modules {
if !already[m] {
adding = append(adding, m)
}
}
// The lines AssignRefusal says beside an assignment it lets through are said by unheldChange
// with everything else this act changed, so they are not said twice.
if _, err := catalogue.AssignRefusal(shelf, node, assigned, adding); err != nil {
return nil, nil, err
}
// Two modules declaring one package, path or unit is refused before anything is recorded
// (novox/hq ADR 0210, 04-ISSUES/235): kept, the node would not resolve until one came off again.
if err := catalogue.CollisionRefusal(shelf, node, assigned, adding); err != nil {
return nil, nil, err
}
return shelf, assigned, nil
}
// unassign takes modules off a node. What they leave behind is the host's business: a directory
// holding anything the mesh did not put there is kept (novox/hq ADR 0030).
//
// It reports the rest of the mesh for the same reason assign does, and more sharply: taking a
// module off one machine is the ordinary way to stop providing something to another, and nothing
// about the command's own output would ever have said so.
func unassign(ctx context.Context, open *stores, node, module string) (string, error) {
//
// **Refused when it takes away the last holder of a seat a module left on the node depends on**
// (novox/hq ADR 0207) — the other side of refusing that module's assignment without one. Several
// modules in one act are judged together, so a holder and its dependents come off in one command.
func unassign(ctx context.Context, open *stores, node string, modules ...string) (string, error) {
if len(modules) == 0 {
return "", fmt.Errorf("unassign %s names no module", node)
}
ctx, release, err := holdNodes(ctx, open, []string{node})
if err != nil {
return "", err
}
defer release()
if err := open.inventory.Unassign(ctx, node, module); err != nil {
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
return "", err
}
return fmt.Sprintf("%s no longer runs %s — run `push %s` to make it so",
node, module, node) + blockedElsewhere(ctx, open, node), nil
assigned, err := open.inventory.Assigned(ctx, node)
if err != nil {
return "", err
}
// Every one checked before any is taken off, so a refusal leaves the node as it was.
runs := map[string]bool{}
for _, a := range assigned {
runs[a] = true
}
for _, module := range modules {
if !runs[module] {
return "", fmt.Errorf("%s is not assigned to %s", module, node)
}
}
if err := catalogue.UnassignRefusal(shelf, node, assigned, modules); err != nil {
return "", err
}
for _, module := range modules {
if err := open.inventory.Unassign(ctx, node, module); err != nil {
return "", err
}
}
answer := fmt.Sprintf("%s no longer runs %s — run `push %s` to make it so",
node, strings.Join(modules, ", "), node)
var left []string
for _, a := range assigned {
if !slices.Contains(modules, a) {
left = append(left, a)
}
}
for _, line := range unheldChange(shelf, node, assigned, left) {
answer += "\n " + line
}
return answer + blockedElsewhere(ctx, open, node), nil
}
// splitModules is a surface's one `module` field as the modules it names: several, comma-separated,
// are one act (novox/hq ADR 0207), so the holders that depend on each other go on together from the
// command API and the controller seat's verbs as they do from the command line.
func splitModules(field string) []string {
var out []string
for _, m := range strings.Split(field, ",") {
if m = strings.TrimSpace(m); m != "" {
out = append(out, m)
}
}
return out
}
// blockedElsewhere is every OTHER machine that cannot be worked out as things now stand.
+6 -6
View File
@@ -145,7 +145,7 @@ func TestTheRegistryTrustAndEveryImageFollowThePortTheNodeGaveTheStore(t *testin
//
// Composed from the control plane's own manifest against a real inventory: the store's module is
// given 6852 on this node the way genesis or an operator gives it, and the control plane's
// container is told so beside the sealed connection genesis wrote.
// process is told so beside the sealed connection genesis wrote.
func TestTheControlPlaneIsToldWhereTheNodePutTheStoreAndTheBroker(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
@@ -157,8 +157,8 @@ func TestTheControlPlaneIsToldWhereTheNodePutTheStoreAndTheBroker(t *testing.T)
if err != nil {
t.Fatal(err)
}
control, err := withSeatPorts(m).Resolve([]catalogue.Built{{Name: "server", Kind: catalogue.ArtifactImage,
Reference: "registry.example/control@" + aDigest}})
control, err := withSeatPorts(m).Resolve([]catalogue.Built{{Name: "controller", Kind: catalogue.ArtifactBundle,
Reference: "https://registry.example/mesh-controller/controller.tar.gz", Digest: aDigest}})
if err != nil {
t.Fatal(err)
}
@@ -200,12 +200,12 @@ func TestTheControlPlaneIsToldWhereTheNodePutTheStoreAndTheBroker(t *testing.T)
var env map[string]any
for _, r := range composed(t, open, "anchor").Resources {
if r["id"] == "mesh-controller.server" {
if r["id"] == "mesh-controller.controller" {
env, _ = r["env"].(map[string]any)
}
}
if env == nil {
t.Fatal("the control plane's container is not in its own node's declaration")
t.Fatal("the control plane's process is not in its own node's declaration")
}
for key, want := range map[string]string{
"MESH_STORE_INVENTORY_PORT": "6852",
@@ -238,7 +238,7 @@ func withSeatPorts(m catalogue.Manifest) catalogue.Manifest {
out := m
out.Resources = nil
for _, r := range m.Resources {
if r["type"] != "container" {
if r["type"] != "container" && r["type"] != "process" {
out.Resources = append(out.Resources, r)
continue
}
+2 -2
View File
@@ -95,10 +95,10 @@ func commands(who Authenticator) http.Handler {
mux := http.NewServeMux()
mux.HandleFunc("POST /assign", acting(who, true, func(ctx context.Context, open *stores, in request) (string, error) {
return assign(ctx, open, in.Node, in.Module)
return assign(ctx, open, in.Node, splitModules(in.Module)...)
}))
mux.HandleFunc("POST /unassign", acting(who, true, func(ctx context.Context, open *stores, in request) (string, error) {
return unassign(ctx, open, in.Node, in.Module)
return unassign(ctx, open, in.Node, splitModules(in.Module)...)
}))
// Adoption (novox/hq ADR 0100): the same acts as `take`, `converge` and `adopt`.
mux.HandleFunc("POST /take", acting(who, true, func(ctx context.Context, open *stores, in request) (string, error) {
+33 -2
View File
@@ -148,6 +148,11 @@ func buildFrom(result link.BuildResult) inventory.Build {
// rebuild the graph rather than a list of names.
Path: result.Path,
}
// When it was asked, which is what orders it against another build of the same module
// (novox/hq 04-ISSUES/219) — not when it was heard.
if asked, ok := link.BuildAskedAt(result.ID); ok {
kept.Asked = asked
}
for _, ref := range result.Against {
kept.Against = append(kept.Against, catalogue.Recorded(ref))
}
@@ -409,7 +414,7 @@ func buildOne(ctx context.Context, source buildSource, path, ref string, wait ti
// Correlated by something the control plane makes, not by the module's name: two builds of one
// module can be in flight, and the second answer is not the first one's.
request := link.BuildRequest{
ID: fmt.Sprintf("%s-%d", "build", time.Now().UnixNano()),
ID: link.NewBuildID(time.Now()),
Repository: repository,
Path: path,
Ref: ref,
@@ -526,17 +531,37 @@ func takeIn(ctx context.Context, inv *inventory.Inventory, result link.BuildResu
BuiltFrom: result.Commit, Head: result.Commit,
// What it stood on, so registration can judge a built manifest's base (to-be 38 WP2.4).
Against: kept.Against,
// When it was asked, so an older request heard later does not replace a newer one
// (novox/hq 04-ISSUES/219).
Asked: kept.Asked,
}
if result.Source != nil && result.Source.Seat != "" {
recorded.Repository, recorded.Seat = result.Source.Repository, result.Source.Seat
}
// **A build at a commit does not change the branch a module follows** (novox/hq 04-ISSUES/215):
// the commit is built and recorded as what it was built from, and the module keeps following
// what it followed before — the repository's default branch for one new to the catalogue.
if followedBranch(result.Ref) == "" && result.Ref != "" {
recorded.Ref = ""
if was, err := inv.SourceOf(ctx, manifest.Module); err == nil {
recorded.Ref = followedBranch(was.Ref)
}
}
if err := namesNoInstallation(manifest); err != nil {
return manifest, kept, fmt.Errorf("%s built %s (%s), and the mesh does not register it: %w",
result.On, result.Repository, short(result.Commit), err)
}
if err := inv.RegisterModule(ctx, manifest, recorded); err != nil {
if errors.Is(err, inventory.ErrSuperseded) {
return manifest, kept, fmt.Errorf("%s built %s (%s), recorded and not registered: %w",
result.On, manifest.Module, short(result.Commit), err)
}
return manifest, kept, err
}
// The keep set just moved, and new bytes just landed (novox/hq ADR 0189). Asked here rather
// than on a timer of its own: this is the only moment either is true. Never fatal — the build
// worked and the module is registered.
collect(ctx, inv)
return manifest, kept, nil
}
@@ -564,9 +589,10 @@ func buildAndShow(ctx context.Context, source buildSource, path, ref string, wai
defer ask.Close()
result, err := ask.Submit(ctx, link.BuildRequest{
ID: fmt.Sprintf("%s-%d", "build", time.Now().UnixNano()),
ID: link.NewBuildID(time.Now()),
Repository: repository, Path: path, Ref: ref,
Held: heldBy(ctx), Seats: seatBases(ctx),
DryRun: true,
}, wait)
if err != nil {
return err
@@ -625,6 +651,11 @@ type answers struct {
// public name on the machine went dark. The holds were correct; they were recorded only in the
// machine's own state file, and the one visible symptom was a count that did not add up.
untaken map[string]map[string]int
// unheld is every module on a machine whose resources are applied through a seat nothing on
// that machine holds (novox/hq ADR 0207), with the modules that could hold it. Reported, not
// refused, until the switch — and while there is any, the mesh is not all well: the order the
// machines' modules are built in is the mesh's to keep, and this is where it says it is not kept.
unheld []catalogue.Unheld
}
// heldBy is every artifact this mesh has built, for a build that may need one as its base.
+87
View File
@@ -2,9 +2,12 @@ package main
import (
"encoding/json"
"errors"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
@@ -63,3 +66,87 @@ func TestABuildHeardIsRecordedAndRegistered(t *testing.T) {
t.Fatalf("a failure is said in the builder's words: %v", err)
}
}
// novox/hq 04-ISSUES/215: a build asked at a commit is recorded as built from that commit, and the
// module keeps following the branch it followed — a new one, the default branch.
func TestABuildAtACommitKeepsTheBranchTheModuleFollows(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
manifest, _ := json.Marshal(map[string]any{"module": "unifi", "version": "1"})
result := func(id, ref, commit string) link.BuildResult {
return link.BuildResult{ID: id, Repository: "http://forge.internal:20000/novox/mesh-catalog.git",
Path: "modules/unifi", Ref: ref, On: "anchor", Commit: commit, Manifest: manifest,
Source: &link.SourceOnSeat{Seat: "git", Repository: "novox/mesh-catalog"}}
}
if _, _, err := takeIn(ctx, open.inventory, result("b-1", "main", "1111111aaaa")); err != nil {
t.Fatal(err)
}
if _, _, err := takeIn(ctx, open.inventory, result("b-2", "9c97a8a", "9c97a8a1d2c3")); err != nil {
t.Fatal(err)
}
src, err := open.inventory.SourceOf(ctx, "unifi")
if err != nil {
t.Fatal(err)
}
if src.Ref != "main" || src.BuiltFrom != "9c97a8a1d2c3" {
t.Errorf("after a build at a commit the module follows %q, built from %q; want main, 9c97a8a1d2c3", src.Ref, src.BuiltFrom)
}
// One new to the catalogue, first built at a commit, follows the default branch.
other, _ := json.Marshal(map[string]any{"module": "letta", "version": "1"})
r := result("b-3", "deadbeef", "deadbeefcafe")
r.Manifest, r.Path = other, "modules/letta"
if _, _, err := takeIn(ctx, open.inventory, r); err != nil {
t.Fatal(err)
}
if src, _ := open.inventory.SourceOf(ctx, "letta"); src.Ref != "" {
t.Errorf("a module first built at a commit follows %q, want the default branch", src.Ref)
}
}
// novox/hq 04-ISSUES/219: an older request heard after a newer one is recorded and not registered,
// so a push sends what the newer request built.
func TestAnOlderBuildHeardLaterDoesNotReplaceTheNewer(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
older := time.Date(2026, 10, 3, 21, 33, 45, 0, time.UTC)
newer := time.Date(2026, 10, 3, 21, 51, 57, 0, time.UTC)
result := func(asked time.Time, image string) link.BuildResult {
manifest, _ := json.Marshal(map[string]any{"module": "postgres", "version": image})
return link.BuildResult{ID: link.NewBuildID(asked), Repository: "http://forge.internal:20000/novox/mesh-catalog.git",
Path: "modules/postgres", Ref: "main", On: "anchor", Commit: "efff5415", Manifest: manifest,
Source: &link.SourceOnSeat{Seat: "git", Repository: "novox/mesh-catalog"}}
}
if _, _, err := takeIn(ctx, open.inventory, result(newer, "4bcd5f73")); err != nil {
t.Fatal(err)
}
_, _, err := takeIn(ctx, open.inventory, result(older, "0ab07fa9"))
if !errors.Is(err, inventory.ErrSuperseded) {
t.Fatalf("the older request's outcome was taken in as current: %v", err)
}
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
t.Fatal(err)
}
if got := shelf["postgres"].Version; got != "4bcd5f73" {
t.Errorf("postgres is %q; want the newer request's 4bcd5f73", got)
}
if builds, _ := open.inventory.Builds(ctx, "postgres", 5); len(builds) != 2 {
t.Errorf("the late build was not recorded: %v", builds)
}
}
func TestABuildIDSaysWhenItWasAsked(t *testing.T) {
at := time.Date(2026, 10, 3, 21, 51, 57, 392539762, time.UTC)
if got, ok := link.BuildAskedAt(link.NewBuildID(at)); !ok || !got.Equal(at) {
t.Errorf("read back %v %v; want %v", got, ok, at)
}
if got, ok := link.BuildAskedAt("build-1791064317392539762"); !ok || got.Format(time.TimeOnly) != "21:51:57" {
t.Errorf("the incident's id reads as %v %v", got, ok)
}
for _, id := range []string{"b-1", "build-2", "build-", "build-x", ""} {
if _, ok := link.BuildAskedAt(id); ok {
t.Errorf("%q read as a request time", id)
}
}
}
+12
View File
@@ -7,6 +7,7 @@ import (
"os"
"path/filepath"
"sort"
"strings"
"github.com/novox/mesh-controller/internal/catalogue"
)
@@ -90,6 +91,17 @@ func moduleCheck(paths []string, out io.Writer) error {
if len(m.Invokes) > 0 {
fmt.Fprintf(out, ", invokes %s", joinInvokes(m.Invokes))
}
// The state it keeps and reads (novox/hq ADR 0201), so a reviewer sees what lands on the bus.
if len(m.State) > 0 {
kept := make([]string, 0, len(m.State))
for _, s := range m.State {
kept = append(kept, s.Name)
}
fmt.Fprintf(out, ", keeps state %s", strings.Join(kept, ", "))
}
if len(m.Reads) > 0 {
fmt.Fprintf(out, ", reads %s", strings.Join(m.Reads, ", "))
}
fmt.Fprintln(out)
}
if failed > 0 {
+175
View File
@@ -0,0 +1,175 @@
package main
import (
"context"
"errors"
"fmt"
"os"
"time"
"github.com/novox/mesh-controller/internal/artifacts"
"github.com/novox/mesh-controller/internal/inventory"
)
// Letting the artifact store go of what the mesh no longer keeps (novox/hq ADR 0189, issue 108).
//
// **Run where the records change.** A build is the moment new bytes landed in the store and the
// moment the keep set moved, so it is the moment to say what may go — and it needs no timer of
// its own. Reclaiming the bytes is the store's own nightly step; this only decides.
//
// Never fatal to a build. The build succeeded, the module is registered, and a store that could
// not be reached is a thing to say rather than a reason to undo any of that. The next build asks
// again, and the references it could not collect are still uncollected, so nothing is lost by
// having failed.
// collect asks the store to let go of everything the mesh made and no longer keeps, and records
// what it let go of. Says what it did and what it could not; returns nothing, because nothing
// upstream should branch on it.
func collect(ctx context.Context, inv *inventory.Inventory) {
references, err := inv.ToCollect(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "could not work out what the artifact store may let go of: %v\n", err)
return
}
kept, err := inv.KeptArchives(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "could not work out which archives the artifact store keeps: %v\n", err)
return
}
if len(references) == 0 && len(kept) == 0 {
return
}
shelf, err := inv.Catalogue(ctx)
if err != nil {
fmt.Fprintf(os.Stderr, "could not read the catalogue to find the artifact store: %v\n", err)
return
}
// As the mesh reaches it from the network. Empty means the store is not on the network — on a
// mesh being raised it is not yet, and there the store holds one build of anything and has
// nothing to collect.
address, err := artifactStoreAddress(ctx, inv, shelf, "")
if err != nil || address == "" {
if err != nil {
fmt.Fprintf(os.Stderr, "could not find the artifact store to collect from: %v\n", err)
}
return
}
// **Bounded, because this runs inside somebody's build.** The first sweep of a mesh that has
// never collected has the whole history to get through, and a person waiting on `build` should
// not pay for it. Two bounds, and what is left over is simply offered again next time —
// builds are frequent, and the point is that the store stops growing, not that it empties
// tonight.
within, stop := context.WithTimeout(ctx, sweepBudget)
defer stop()
store := artifacts.Store{Address: address}
// **Hold before letting go** (novox/hq issue 253). The store's collector keeps only what a
// manifest names, and archives were published as bare blobs, so every kept archive is first
// held by its manifest — which backfills the ones published before holders, a few at a time
// as builds come, and is two HEADs each once done. A kept archive that could not be held stops
// the sweep before it deletes anything: "everything kept is held" is the precondition the
// collector's safety rests on, and a store refusing a hold would refuse the deletes too.
wrote, missing, err := holdKept(within, store, kept)
if wrote > 0 {
fmt.Fprintf(os.Stderr, "the artifact store now holds %d more kept archive(s) by a manifest\n", wrote)
}
if missing > 0 {
fmt.Fprintf(os.Stderr, "%d archive(s) the mesh keeps are not in the artifact store at all; "+
"`collection` lists them\n", missing)
}
if err != nil {
fmt.Fprintf(os.Stderr, "not every kept archive could be held, so nothing was let go: %v\n", err)
return
}
var done []string
var left, skipped int
for i, reference := range references {
if i >= mostPerSweep || within.Err() != nil {
left = len(references) - i
break
}
err := store.LetGo(within, reference)
if err == nil || errors.Is(err, artifacts.Gone) {
// Gone is the outcome wanted, already true. Recorded so the next sweep does not ask
// again for ever.
done = append(done, reference)
continue
}
if errors.Is(err, artifacts.ErrNotOurs) {
// **A fact about this record, so this record is skipped** (novox/hq issue 226). Not
// marked collected — the mesh did not remove it and should not claim to — and not a
// reason to stop, because the store was never asked. One of these at the front of
// the oldest-first order ended every sweep until this.
skipped++
if skipped == 1 {
fmt.Fprintf(os.Stderr,
"the sweep will not address %s and went on: %v\n", reference, err)
}
continue
}
// **Stopped at the first refusal by the STORE, not pushed through.** A store that refuses
// one refuses all of them — deletion disabled, the store down, the network gone — so
// going on would be a hundred identical failures and a hundred identical log lines in
// front of whoever was building something.
fmt.Fprintf(os.Stderr, "the artifact store kept %s, so nothing more was asked of it: %v\n",
reference, err)
left = len(references) - i
break
}
if len(done) > 0 {
// Recorded outside `within`: the deletions happened, and losing the record of them because
// the sweep ran out of budget would mean asking about them again for ever.
if err := inv.MarkCollected(ctx, done); err != nil {
fmt.Fprintf(os.Stderr, "the store let go of %d artifact(s) and the record of it did not keep: %v\n",
len(done), err)
return
}
fmt.Fprintf(os.Stderr, "the artifact store let go of %d artifact(s) the mesh no longer keeps\n",
len(done))
}
if left > 0 {
fmt.Fprintf(os.Stderr, "%d more to collect; the next build asks again\n", left)
}
if skipped > 0 {
fmt.Fprintf(os.Stderr, "%d artifact(s) the sweep will not address were skipped\n", skipped)
}
}
// holdKept holds every kept archive by its manifest, stopping at the first refusal by the store.
// Answers how many holders it wrote and how many kept archives the store does not have.
//
// A missing archive is counted rather than fatal: there is nothing to hold, and that is a fact
// for an operator to read (`collection`), not a reason to stop collecting what is not kept. A
// reference the store cannot be asked about is skipped as the deletion loop skips one
// (novox/hq issue 226).
func holdKept(ctx context.Context, store artifacts.Store, kept []string) (wrote, missing int, err error) {
for _, reference := range kept {
if err := ctx.Err(); err != nil {
return wrote, missing, fmt.Errorf("ran out of time before %s: %w", reference, err)
}
did, err := store.Hold(ctx, reference)
switch {
case err == nil:
if did {
wrote++
}
case errors.Is(err, artifacts.Gone):
missing++
case errors.Is(err, artifacts.ErrNotOurs):
default:
return wrote, missing, fmt.Errorf("holding %s: %w", reference, err)
}
}
return wrote, missing, nil
}
// mostPerSweep is how many artifacts one sweep will ask about. Enough that a mesh building
// several times a day converges within days of this landing; small enough that no single build
// waits on the whole backlog.
const mostPerSweep = 200
// sweepBudget is the longest a sweep will keep a build waiting.
const sweepBudget = 60 * time.Second
+166
View File
@@ -0,0 +1,166 @@
package main
import (
"context"
"encoding/json"
"errors"
"flag"
"fmt"
"os"
"strings"
"github.com/novox/mesh-controller/internal/artifacts"
)
// What the artifact store keeps, whether each kept archive is held, and what the sweep may let go
// (novox/hq issue 253, ADR 0189).
//
// **The question to answer before the store's collector runs for real.** The collector deletes
// every blob no manifest names, and archives were published as bare blobs, so the nightly step
// runs `--dry-run` until every archive the mesh keeps is held by its manifest. The sweep holds
// them as builds come; this says how far that has got — "0 unheld" is the number that lets the
// dry run go.
//
// Reads and changes nothing: each kept archive is asked about with HEADs only. Reached over the
// console through the mesh-controller seat's `command` verb (`collection --json`), which needs no
// new verb in the seat's row.
type collectionReport struct {
// Store is the artifact store as this machine reached it; empty when it is not on the network.
Store string `json:"store"`
// KeptArchives is how many archives the mesh keeps, for either reason.
KeptArchives int `json:"kept_archives"`
// Held is how many of them the store holds by their manifest.
Held int `json:"held"`
// Unheld are the kept archives the collector would delete tonight if it ran for real.
Unheld []string `json:"unheld"`
// Missing are kept archives the store does not have at all.
Missing []string `json:"missing"`
// Unasked is how many could not be asked about, and why the asking stopped.
Unasked int `json:"unasked"`
Stopped string `json:"stopped,omitempty"`
// Eligible is what the sweep may let go of: made by the mesh, kept for no reason, not yet
// collected — split by kind.
Eligible int `json:"eligible"`
EligibleImages int `json:"eligible_images"`
EligibleArchives int `json:"eligible_archives"`
// SafeToCollect is whether every kept archive was asked about and every one is held.
SafeToCollect bool `json:"safe_to_collect"`
}
func collectionCommand(ctx context.Context, args []string) error {
set := flag.NewFlagSet("collection", flag.ContinueOnError)
asJSON := set.Bool("json", false, "answer as JSON")
positionals, err := parseAround(set, args)
if err != nil {
return err
}
if len(positionals) != 0 {
return errors.New("collection [--json]")
}
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
inv := open.inventory
kept, err := inv.KeptArchives(ctx)
if err != nil {
return err
}
eligible, err := inv.ToCollect(ctx)
if err != nil {
return err
}
report := collectionReport{KeptArchives: len(kept), Eligible: len(eligible), Unheld: []string{}, Missing: []string{}}
for _, reference := range eligible {
if strings.Contains(reference, "/blobs/") {
report.EligibleArchives++
} else {
report.EligibleImages++
}
}
shelf, err := inv.Catalogue(ctx)
if err != nil {
return err
}
report.Store, err = artifactStoreAddress(ctx, inv, shelf, "")
if err != nil {
return err
}
if report.Store == "" {
report.Unasked = len(kept)
report.Stopped = "this mesh has no artifact store on its network"
} else {
report.Unasked, report.Stopped = askHeld(ctx, artifacts.Store{Address: report.Store}, kept, &report)
}
report.SafeToCollect = report.Unasked == 0 && len(report.Unheld) == 0
if *asJSON {
encoder := json.NewEncoder(os.Stdout)
encoder.SetIndent("", " ")
return encoder.Encode(report)
}
printCollection(report)
return nil
}
// askHeld asks the store about each kept archive, stopping at the first answer that is not about
// the archive: a store that cannot be reached for one cannot be for the next, and a page of
// identical failures says less than one line.
func askHeld(ctx context.Context, store artifacts.Store, kept []string, report *collectionReport) (int, string) {
for i, reference := range kept {
held, err := store.Held(ctx, reference)
switch {
case err == nil && held:
report.Held++
case err == nil:
report.Unheld = append(report.Unheld, reference)
case errors.Is(err, artifacts.Gone):
report.Missing = append(report.Missing, reference)
case errors.Is(err, artifacts.ErrNotOurs):
// KeptArchives names only the mesh's own; counted as unasked if one ever is not.
report.Unasked++
default:
return report.Unasked + len(kept) - i, fmt.Sprintf("asking about %s: %v", reference, err)
}
}
return report.Unasked, ""
}
func printCollection(r collectionReport) {
store := r.Store
if store == "" {
store = "(not on the network)"
}
fmt.Printf("artifact store %s\n", store)
fmt.Printf("kept archives %d\n", r.KeptArchives)
fmt.Printf(" held %d\n", r.Held)
fmt.Printf(" unheld %d\n", len(r.Unheld))
fmt.Printf(" missing %d\n", len(r.Missing))
if r.Unasked > 0 {
fmt.Printf(" not asked %d (%s)\n", r.Unasked, r.Stopped)
}
fmt.Printf("eligible to let go %d (%d images, %d archives)\n", r.Eligible, r.EligibleImages, r.EligibleArchives)
if len(r.Unheld) > 0 {
fmt.Println("\nunheld — the store's collector would delete these; the next build's sweep holds them:")
for _, reference := range r.Unheld {
fmt.Printf(" %s\n", reference)
}
}
if len(r.Missing) > 0 {
fmt.Println("\nmissing — kept by the mesh, not in the store:")
for _, reference := range r.Missing {
fmt.Printf(" %s\n", reference)
}
}
fmt.Println()
if r.SafeToCollect {
fmt.Println("every kept archive is held: the store's collector may run for real")
} else {
fmt.Println("NOT every kept archive is known to be held: keep the store's collector on --dry-run")
}
}
+145
View File
@@ -0,0 +1,145 @@
package main
import (
"context"
"errors"
"reflect"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/link"
)
// recordingDelivery is a delivery that writes down what was done, in order, and fails where told.
type recordingDelivery struct {
did []string
grantErr error
declareErr error
}
func (r *recordingDelivery) grant(_ context.Context, sending []readyNode) error {
for _, s := range sending {
r.did = append(r.did, "grant "+s.node)
}
return r.grantErr
}
func (r *recordingDelivery) declare(_ context.Context, s readyNode, _ []byte) (string, error) {
if r.declareErr != nil {
return "", r.declareErr
}
r.did = append(r.did, "declare "+s.node)
return "digest-" + s.node, nil
}
func ready(names ...string) []readyNode {
var out []readyNode
for _, n := range names {
out = append(out, readyNode{node: n, declared: sendable{Resources: []map[string]any{{"id": "x"}}}})
}
return out
}
// novox/hq issue 249: the grants that come with a module's new declarations are issued before any
// machine is sent the code that uses them — except the machine holding the bus, whose declaration
// carries the controller's own right to issue them, and goes first.
func TestGrantsAreIssuedBeforeTheDeclarations(t *testing.T) {
d := &recordingDelivery{}
digests, err := deliver(t.Context(), d, "", ready("anchor", "laptop"))
if err != nil {
t.Fatal(err)
}
want := []string{"grant anchor", "grant laptop", "declare anchor", "declare laptop"}
if !reflect.DeepEqual(d.did, want) {
t.Fatalf("delivered in the order %v, wanted %v", d.did, want)
}
if digests["anchor"] != "digest-anchor" || digests["laptop"] != "digest-laptop" {
t.Fatalf("the digests sent were not answered: %v", digests)
}
held := &recordingDelivery{}
if _, err := deliver(t.Context(), held, "broker", ready("broker", "anchor")); err != nil {
t.Fatal(err)
}
want = []string{"declare broker", "grant broker", "grant anchor", "declare anchor"}
if !reflect.DeepEqual(held.did, want) {
t.Fatalf("with the bus's machine in the send: %v, wanted %v", held.did, want)
}
}
// A grant that cannot be issued holds back the machines it concerns and is an error the caller
// retries on — never "until the next push" — and the bus's own machine is sent regardless, so the
// grant that would let the controller issue memberships is never held behind them.
func TestAGrantThatFailsHoldsBackWhatItConcerns(t *testing.T) {
// A failure naming no machine (the buckets): everything but the bus's machine.
d := &recordingDelivery{grantErr: errors.New("the bus refused the bucket")}
_, err := deliver(t.Context(), d, "broker", ready("broker", "anchor", "laptop"))
if err == nil || !errors.Is(err, errGrants) || !strings.Contains(err.Error(), "the bus refused the bucket") ||
!strings.Contains(err.Error(), "anchor, laptop not sent") {
t.Fatalf("a failed grant was not said as the send's failure: %v", err)
}
if want := []string{"declare broker", "grant broker", "grant anchor", "grant laptop"}; !reflect.DeepEqual(d.did, want) {
t.Fatalf("delivered %v, wanted the bus's machine alone", d.did)
}
// A membership that failed for one machine: that machine alone.
one := &recordingDelivery{grantErr: &grantsRefused{nodes: map[string]error{"laptop": errors.New("no")}}}
_, err = deliver(t.Context(), one, "", ready("anchor", "laptop"))
if !errors.Is(err, errGrants) || !strings.Contains(err.Error(), "laptop not sent") {
t.Fatalf("one machine's refused membership was not said: %v", err)
}
if want := []string{"grant anchor", "grant laptop", "declare anchor"}; !reflect.DeepEqual(one.did, want) {
t.Fatalf("delivered %v, wanted anchor sent and laptop held back", one.did)
}
// The announced upgrade that hit it is asked again.
if !errors.Is(askAgainOnGrants(err), link.ErrTryAgain) {
t.Fatal("an announcement whose send stopped at its grants is not asked again")
}
if other := errors.New("laptop could not be resolved"); errors.Is(askAgainOnGrants(other), link.ErrTryAgain) {
t.Fatal("any failure is asked again, not only a grant's")
}
// Nothing to send is nothing granted either.
none := &recordingDelivery{grantErr: errors.New("never asked")}
if _, err := deliver(t.Context(), none, "", nil); err != nil || len(none.did) != 0 {
t.Fatalf("an empty send granted or failed: %v %v", none.did, err)
}
}
// Whether the bus's machine goes first is read from the user list alone, by its digest.
func TestTheBusMachineIsBehindByItsUserListAlone(t *testing.T) {
list := "users: [a, b]"
if userListBehind(list, digestOf([]byte(list))) {
t.Fatal("the list it was sent reads as behind")
}
if !userListBehind(list, digestOf([]byte("users: [a]"))) || !userListBehind(list, "") {
t.Fatal("a changed or never-sent list reads as current")
}
if userListBehind("", "") {
t.Fatal("a machine sent no list reads as behind")
}
}
// The machine holding the bus goes first: its declaration carries the user list the new grants are
// checked against. Among the machines it is moved to the front; not among them it is added only
// when it is behind.
func TestTheMachineHoldingTheBusIsSentFirst(t *testing.T) {
for _, c := range []struct {
what string
names []string
holder string
behind bool
want []string
}{
{"among them", []string{"ace", "g14", "novox"}, "novox", false, []string{"novox", "ace", "g14"}},
{"not among them, behind", []string{"ace", "g14"}, "novox", true, []string{"novox", "ace", "g14"}},
{"not among them, current", []string{"ace", "g14"}, "novox", false, []string{"ace", "g14"}},
{"nothing holds the bus", []string{"ace", "g14"}, "", true, []string{"ace", "g14"}},
{"only it", []string{"novox"}, "novox", false, []string{"novox"}},
} {
if got := brokerFirst(c.names, c.holder, c.behind); !reflect.DeepEqual(got, c.want) {
t.Errorf("%s: sent in the order %v, wanted %v", c.what, got, c.want)
}
}
}
+38
View File
@@ -0,0 +1,38 @@
package main
import (
"encoding/json"
"testing"
"github.com/novox/mesh-controller/internal/link"
)
// A dry run's outcome is looked at, never taken in (novox/hq issue 240). The daemon here holds no
// store at all, so anything that tried to record or register would fail rather than pass quietly.
func TestADryRunsOutcomeIsTakenInByNothing(t *testing.T) {
err := builds{}.Built(t.Context(), link.BuildResult{
ID: "build-1", Repository: "ssh://forge/app.git", Ref: "unreviewed", Module: "app", DryRun: true,
Manifest: json.RawMessage(`{"module":"app","version":"1"}`),
})
if err != nil {
t.Fatalf("a dry run's outcome was not simply set aside: %v", err)
}
}
// The mark survives the wire both ways: asked as a dry run, answered as one.
func TestTheDryRunMarkTravelsWithTheBuild(t *testing.T) {
raw, _ := json.Marshal(link.BuildRequest{ID: "build-1", Repository: "r", DryRun: true})
var asked link.BuildRequest
if err := json.Unmarshal(raw, &asked); err != nil || !asked.DryRun {
t.Fatalf("the request lost its dry-run mark: %s", raw)
}
raw, _ = json.Marshal(link.BuildResult{ID: "build-1", DryRun: true})
var answered link.BuildResult
if err := json.Unmarshal(raw, &answered); err != nil || !answered.DryRun {
t.Fatalf("the outcome lost its dry-run mark: %s", raw)
}
raw, _ = json.Marshal(link.BuildResult{ID: "build-2"})
if string(raw) != `{"id":"build-2","repository":"","on":""}` {
t.Fatalf("an ordinary outcome carries a dry-run mark: %s", raw)
}
}
+4 -4
View File
@@ -18,16 +18,16 @@ func TestASendRoundGivesItsHoldBackOnEveryWayOut(t *testing.T) {
plain := func(context.Context, string) (sendable, error) {
return sendable{Resources: []map[string]any{{"id": "x"}}}, nil
}
failing := func(readyNode, []byte) error { return errors.New("the broker went away") }
fine := func(readyNode, []byte) error { return nil }
failing := &recordingDelivery{declareErr: errors.New("the broker went away")}
fine := &recordingDelivery{}
for name, round := range map[string]func() error{
"a body that cannot be marshalled": func() error {
_, err := sendRound(ctx, open, []string{"anchor"}, unmarshallable, fine)
_, err := sendRound(ctx, open, []string{"anchor"}, unmarshallable, fine, "", nil)
return err
},
"a send that fails": func() error {
_, err := sendRound(ctx, open, []string{"anchor"}, plain, failing)
_, err := sendRound(ctx, open, []string{"anchor"}, plain, failing, "", nil)
return err
},
} {
+4 -4
View File
@@ -232,7 +232,7 @@ func licenceKey(ctx context.Context, args []string) error {
// and printing the value here would put the one copy that matters on a terminal.
fmt.Printf("sealed to %d holder(s). The mesh has discarded the key and cannot read it back\n",
sealed)
fmt.Printf(" run `push` to deliver it\n")
fmt.Printf(" run `push --behind` to deliver it\n")
return nil
}
@@ -340,7 +340,7 @@ func licenceSetGrant(ctx context.Context, args []string) error {
return err
}
fmt.Printf("%s now holds a refresh token for %s, sealed to that node's key and readable by it "+
"alone.\n the control plane stored the box without opening it; run `push` to deliver it\n",
"alone.\n the control plane stored the box without opening it; run `push --behind` to deliver it\n",
"the manager", name)
return nil
}
@@ -423,7 +423,7 @@ func licenceSubmitRefresh(ctx context.Context, args []string) error {
"manager node alone"
}
fmt.Printf("submitted a refresh for %s: a new access token sealed to %d holder(s), and %s.\n"+
" run `push` to deliver it\n", name, sealed, rotatedNote)
" run `push --behind` to deliver it\n", name, sealed, rotatedNote)
return nil
}
@@ -456,7 +456,7 @@ func licenceRefresh(ctx context.Context, args []string) error {
return err
}
fmt.Printf("refreshed %s: a new access token sealed to %d holder(s), and the refresh token left "+
"with its manager.\n run `push` to deliver it\n", name, sealed)
"with its manager.\n run `push --behind` to deliver it\n", name, sealed)
return nil
}
+25 -5
View File
@@ -8,6 +8,7 @@ package main
import (
"context"
"errors"
"flag"
"fmt"
"os"
@@ -72,6 +73,8 @@ func run() error {
return askCommand(ctx, args[1:])
case "builds":
return buildsCommand(ctx, args[1:])
case "collection":
return collectionCommand(ctx, args[1:])
case "plans":
return plansCommand(ctx, args[1:])
case "pin":
@@ -178,8 +181,8 @@ func usage() {
seat <name> --to <node>/<module> hand a seat to that assignment as one act; never empty in between (ADR 0131)
board [--listen ADDR] the same three questions, as a page that holds nothing
api --issuer URL [--listen A] assign and unassign over http, for a surface that is not here
assign <node> <module> put a module on a node
unassign <node> <module> take it off
assign <node> <module>... put modules on a node, judged together (ADR 0207)
unassign <node> <module>... take them off
take <node> <module> preview a module's cutover on an adopted node: what runs beside
what it declares; --yes <digest> cuts it over as previewed
converge <node> [--yes <digest>] [--filter nftables] preview, then make, an adopted node converged
@@ -199,6 +202,7 @@ func usage() {
build --behind build every module the mesh holds older than its source
build --on <module> rebuild every module that stands on this module's artifacts, bases first
builds [<module>] what has been built lately, and what came of it
collection [--json] kept archives held/unheld by a manifest, and what the sweep may let go
builder issue <name> a broker account for a build machine, scoped to build work,
delivered as the builder module's broker secret (module add it first)
licence add|list|use|key model access, under the name a person calls it
@@ -252,26 +256,42 @@ func parseAround(set *flag.FlagSet, args []string) ([]string, error) {
// registered, the same as the waiting command does. Said either way, so the daemon's log tells what
// became of a build nobody was watching.
func (b builds) Built(ctx context.Context, result link.BuildResult) error {
// **A dry run is looked at, never taken in** (novox/hq issue 240). On 2026-10-04 a dry run of an
// unmerged branch was heard here like any build, registered, and its definition reached a machine
// before anyone had reviewed it.
if result.DryRun {
fmt.Printf("%s: a dry run of %s on %s, not taken in\n", result.ID, result.Repository, result.Ref)
return nil
}
manifest, _, err := takeIn(ctx, b.inv, result)
// When it was asked, so a plan takes as its outcome only a build asked for it or after it
// (novox/hq 04-ISSUES/219). Zero when the id does not say.
asked, _ := link.BuildAskedAt(result.ID)
switch {
case err != nil && result.Failed != "":
fmt.Printf("%s: %v\n", result.ID, err)
if result.Module != "" {
planBuilt(ctx, b.open, result.Module, result.Commit, result.Failed)
planBuilt(ctx, b.open, result.Module, result.Commit, result.Failed, asked)
} else {
planFailedBuild(ctx, b.open, result)
}
return nil
case errors.Is(err, inventory.ErrSuperseded):
// Not a failure: the module is already at what a later request built. A plan that asked
// before that later request is answered by it; one that asked after it ignores this.
fmt.Printf("%s: %v\n", result.ID, err)
planBuilt(ctx, b.open, manifest.Module, result.Commit, "", asked)
return nil
case err != nil:
fmt.Printf("%s: heard and recorded, and not registered: %v\n", result.ID, err)
if manifest.Module != "" {
planBuilt(ctx, b.open, manifest.Module, result.Commit, err.Error())
planBuilt(ctx, b.open, manifest.Module, result.Commit, err.Error(), asked)
}
return nil
}
fmt.Printf("%s: %s %s registered, built on %s from %s\n",
result.ID, manifest.Module, manifest.Version, result.On, short(result.Commit))
saysWhenThePolicyActs(ctx, b.inv, manifest.Module)
planBuilt(ctx, b.open, manifest.Module, result.Commit, "")
planBuilt(ctx, b.open, manifest.Module, result.Commit, "", asked)
return nil
}
+87 -13
View File
@@ -8,7 +8,6 @@ import (
"fmt"
"net"
"os"
"sort"
"strings"
"github.com/novox/mesh-controller/internal/broker"
@@ -334,8 +333,10 @@ func moduleCommand(ctx context.Context, args []string) error {
}
func assignCommand(ctx context.Context, verb string, args []string) error {
if len(args) != 2 {
return fmt.Errorf("%s <node> <module>", verb)
// Several modules in one act (novox/hq ADR 0207): holders that depend on each other — the
// service manager and the package manager — can only go on, or come off, together.
if len(args) < 2 {
return fmt.Errorf("%s <node> <module> [<module>…]", verb)
}
open, err := openStores(ctx)
if err != nil {
@@ -349,7 +350,7 @@ func assignCommand(ctx context.Context, verb string, args []string) error {
if verb == "unassign" {
act = unassign
}
said, err := act(ctx, open, args[0], args[1])
said, err := act(ctx, open, args[0], args[1:]...)
if said != "" {
fmt.Println(said)
}
@@ -362,7 +363,8 @@ func assignCommand(ctx context.Context, verb string, args []string) error {
func settingsCommand(ctx context.Context, args []string) error {
if len(args) == 0 {
return errors.New("settings set <module> <file> [--node <node>], or settings clear <module> [--node <node>]")
return errors.New("settings show <module> [--node <node>] [--history], settings set <module> <file> " +
"[--node <node>] [--replace], or settings clear <module> [--node <node>]")
}
open, err := openStores(ctx)
if err != nil {
@@ -373,6 +375,11 @@ func settingsCommand(ctx context.Context, args []string) error {
set := flag.NewFlagSet("settings", flag.ContinueOnError)
node := set.String("node", "", "one machine, rather than the whole mesh")
// **What a set removes is refused unless meant** (novox/hq ADR 0217). A layer is replaced whole,
// and on 2026-10-05 setting one placement dropped a machine's whole layer for a module without a
// word (issue 246). Adding and changing keys needs nothing; removing one needs this.
replace := set.Bool("replace", false, "for set: remove the keys the new layer does not name")
history := set.Bool("history", false, "for show: the layers this one replaced, the latest first")
positionals, err := parseAround(set, args[1:])
if err != nil {
return err
@@ -400,17 +407,76 @@ func settingsCommand(ctx context.Context, args []string) error {
if err := json.Unmarshal(raw, &values); err != nil {
return fmt.Errorf("%s is not a settings file: %w", positionals[1], err)
}
before, _, err := inv.Layer(ctx, *node, positionals[0])
if err != nil {
return err
}
added, changed, removed := settingsChange(before, values)
if len(removed) > 0 && !*replace {
return fmt.Errorf("%s on %s: this layer would no longer set %s. A layer is replaced whole; "+
"read it with `settings show %s%s` and include what should stay, or add --replace if the "+
"removal is meant (novox/hq ADR 0217). Nothing was changed",
positionals[0], where, strings.Join(removed, ", "), positionals[0], nodeFlag(*node))
}
if err := inv.SetSettings(ctx, *node, positionals[0], values); err != nil {
return err
}
var keys []string
for k := range values {
keys = append(keys, k)
fmt.Printf("%s on %s:\n", positionals[0], where)
if len(added)+len(changed)+len(removed) == 0 {
fmt.Println(" nothing changed")
}
sort.Strings(keys)
fmt.Printf("%s on %s: %s\n", positionals[0], where, strings.Join(keys, ", "))
fmt.Println(" run `push` to send it")
for _, p := range added {
fmt.Printf(" + %s\n", p)
}
for _, p := range changed {
fmt.Printf(" ~ %s\n", p)
}
for _, p := range removed {
fmt.Printf(" - %s\n", p)
}
if before != nil {
fmt.Printf(" the layer it replaced is kept: settings show %s%s --history\n", positionals[0], nodeFlag(*node))
}
if *node != "" {
fmt.Printf(" run `plan %s --diff` to see what it changes, `push %s` to send it\n", *node, *node)
} else {
fmt.Println(" run `push --behind` to send it to the machines running it")
}
return nil
case "show":
if len(positionals) != 1 {
return errors.New("settings show <module> [--node <node>] [--history]")
}
if *history {
past, err := inv.SettingsHistory(ctx, *node, positionals[0])
if err != nil {
return err
}
if len(past) == 0 {
fmt.Printf("%s on %s: no layer has been replaced\n", positionals[0], where)
return nil
}
for _, p := range past {
shown, _ := json.MarshalIndent(p.Values, " ", " ")
fmt.Printf("%s on %s, until %s (%s):\n %s\n", positionals[0], where,
p.ReplacedAt.Local().Format("2006-01-02 15:04:05"), p.ReplacedBy, shown)
}
return nil
}
values, has, err := inv.Layer(ctx, *node, positionals[0])
if err != nil {
return err
}
if !has {
fmt.Printf("%s on %s: no layer — the module's definition says\n", positionals[0], where)
return nil
}
shown, err := json.MarshalIndent(values, "", " ")
if err != nil {
return err
}
fmt.Println(string(shown))
return nil
case "clear":
@@ -424,10 +490,18 @@ func settingsCommand(ctx context.Context, args []string) error {
return nil
default:
return fmt.Errorf("settings has no %q; it has set and clear", args[0])
return fmt.Errorf("settings has no %q; it has show, set and clear", args[0])
}
}
// nodeFlag is ` --node <node>` for a machine's layer, nothing for the whole mesh's.
func nodeFlag(node string) string {
if node == "" {
return ""
}
return " --node " + node
}
// describeOffers says what a module provides, and marks the ones answered from anywhere in the
// mesh — because "provides a database" and "provides a shell" are read the same way and mean
// entirely different things about where the answer has to be.
+14 -8
View File
@@ -254,11 +254,10 @@ func theResolver(t *testing.T) catalogue.Manifest {
return m
}
// The resolver is handed every machine on the private network as a wildcard, the same set and the
// same source as the hosts file, and is handed it again when a machine leaves — through the
// module's own manifest asking for the fact, with no module of the mesh's own in between (hal
// dnsmasq-app conversion, novox/hq 08-connectivity). The runtime on that machine is pointed at the
// machine's own address, where the resolver answers for its containers.
// The resolver is handed every machine on the private network as a wildcard, and is handed it again
// when a machine leaves — through the module's own manifest asking for the fact, with no module of the
// mesh's own in between (hal dnsmasq-app conversion, novox/hq 08-connectivity). It is the mesh's one
// resolver (ADR 0194), and the container runtime is given no resolver of its own (ADR 0196).
func TestTheResolverIsToldEveryMachineOnTheNetworkAndToldAgainWhenOneLeaves(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
@@ -266,6 +265,12 @@ func TestTheResolverIsToldEveryMachineOnTheNetworkAndToldAgainWhenOneLeaves(t *t
if _, err := assign(ctx, open, "anchor", "dnsmasq"); err != nil {
t.Fatal(err)
}
// Its bus credential, as assigning issues it where the bus is reachable (novox/hq issue 203):
// no bus is known to this test, so it is minted here, or composing refuses the placeholder.
if _, err := open.inventory.MintBusPassword(ctx, inventory.BusUser{
Username: "anchor.dnsmasq", Kind: inventory.BusModule, Node: "anchor", Module: "dnsmasq"}); err != nil {
t.Fatal(err)
}
zones := func() string {
t.Helper()
for _, r := range composed(t, open, "anchor").Resources {
@@ -287,11 +292,12 @@ func TestTheResolverIsToldEveryMachineOnTheNetworkAndToldAgainWhenOneLeaves(t *t
t.Errorf("the resolver's machines lack %q:\n%s", want, first)
}
}
// The container runtime is given no resolver of its own (novox/hq ADR 0196): it copies its
// machine's, which name the mesh's resolver first. A `dns` key would be a second account of where a
// container asks, read only when the runtime starts.
for _, r := range composed(t, open, "anchor").Resources {
if r["id"] == "dnsmasq.runtime-dns" {
if !strings.Contains(r["content"].(string), `"10.77.0.1"`) || r["into"] != "json" {
t.Errorf("the runtime is not pointed at this machine's own address, written into its file: %v", r)
}
t.Errorf("the resolver still writes the runtime's own dns: %v", r)
}
}
+33 -1
View File
@@ -386,8 +386,12 @@ func brokerCommand(ctx context.Context, args []string) error {
if len(args) > 0 && args[0] == "accounts" {
return busAccounts(ctx, args[1:])
}
if len(args) > 0 && args[0] == "consumer-reset" {
return consumerReset(args[1:])
}
if len(args) == 0 || args[0] != "show" {
return errors.New("broker show | broker certificate [--check] --into <directory> | broker accounts --into <file>")
return errors.New("broker show | broker certificate [--check] --into <directory> | broker accounts --into <file> | " +
"broker consumer-reset <stream> <consumer>")
}
known, err := broker.FromEnvironment()
if errors.Is(err, broker.ErrNotConfigured) {
@@ -407,6 +411,34 @@ func brokerCommand(ctx context.Context, args []string) error {
return nil
}
// consumerReset re-makes one consumer on a stream that keeps history to start from now (novox/hq issue
// 248): the way out of a consumer replaying a week of announcements, said rather than done by hand. A
// person's act — what was pending is dropped — so it is a command, and nothing calls it on its own.
func consumerReset(args []string) error {
if len(args) != 2 {
return errors.New("broker consumer-reset <stream> <consumer>, e.g. broker consumer-reset EVENTS controller")
}
address, err := broker.BusAddress()
if err != nil {
return err
}
js, err := broker.Dial(address)
if err != nil {
return fmt.Errorf("cannot reach the bus: %w", err)
}
defer js.Close()
before, after, err := js.ResetConsumer(args[0], args[1])
if err != nil {
return err
}
fmt.Printf("consumer %s on %s re-made to deliver from now\n", args[1], args[0])
fmt.Printf(" before: delivers %s, delivered to %d, acknowledged to %d, %d pending, %d unacknowledged\n",
before.DeliverPolicy, before.Delivered, before.AckFloor, before.Pending, before.AckPending)
fmt.Printf(" after: delivers %s, %d pending; what was pending is dropped. A holder bound to it may need its "+
"process restarted to bind again\n", after.DeliverPolicy, after.Pending)
return nil
}
// heardFrom says when a node was last heard from, in a form somebody can act on.
//
// "never" and "an hour ago" are different answers and are kept different. A node that has never
+32
View File
@@ -147,6 +147,14 @@ func TestAMergeRebuildsTheModulesItChanged(t *testing.T) {
{"a module the mesh does not hold", merge([]string{"modules/plex/index.ts"}, false), ""},
{"nothing said about the files", merge(nil, false), "gitea,keycloak"},
{"more files than were listed", merge([]string{"modules/gitea/index.ts"}, true), "gitea,keycloak"},
// novox/hq issue 252: a module the mesh has never registered is still a module, when the merge
// shows it is one — and a directory that may be shared code is still shared.
{"a new module beside a held one", merge([]string{"modules/gitea/x", "modules/newmod/module.json"}, false), "gitea"},
{"a new module's other files", merge([]string{"modules/newmod/index.ts", "modules/newmod/module.json"}, false), ""},
{"a module removed", merge([]string{"modules/gone/module.json"}, false), ""},
{"a directory with no manifest", merge([]string{"modules/lib/x.go"}, false), "gitea,keycloak"},
{"a file directly among the modules", merge([]string{"modules/README.md"}, false), "gitea,keycloak"},
{"the root's files still", merge([]string{"tsconfig.json"}, false), "gitea,keycloak"},
} {
if got := named(whatTheMergeTouched(candidates, known, c.m)); got != c.want {
t.Errorf("%s: rebuilt %q, wanted %q", c.what, got, c.want)
@@ -211,3 +219,27 @@ func TestWhatAHandedOverModuleRecordsAboutItsSource(t *testing.T) {
}
}
}
// novox/hq 04-ISSUES/215: a module once built at a commit still follows its branch — a merge into it
// matches the module, and a plan re-asks the branch, not the old commit.
func TestAModuleBuiltAtACommitStillFollowsItsBranch(t *testing.T) {
m := link.SourceMoved{Owner: "novox", Repo: "mesh-catalog", Base: "main"}
pinned := inventory.Source{Repository: "novox/mesh-catalog", Seat: "git", Ref: "9c97a8a"}
if !sourceIs(pinned, m) {
t.Error("a module whose record names a commit is left out of a merge into its branch")
}
full := inventory.Source{Repository: "novox/mesh-catalog", Seat: "git", Ref: "9c97a8a1d2c3b4a5f60718293a4b5c6d7e8f9012"}
if !sourceIs(full, m) {
t.Error("a full commit hash is read as a branch")
}
if got := followedBranch("9c97a8a"); got != "" {
t.Errorf("a plan would re-ask the old commit %q", got)
}
if got := followedBranch("release"); got != "release" {
t.Errorf("a branch is not followed as named: %q", got)
}
// A module that follows another branch is still not this merge's.
if sourceIs(inventory.Source{Repository: "novox/mesh-catalog", Seat: "git", Ref: "release"}, m) {
t.Error("a module following another branch was matched")
}
}
+201 -15
View File
@@ -8,8 +8,10 @@ import (
"flag"
"fmt"
"os"
"slices"
"sort"
"strings"
"sync"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
@@ -127,6 +129,7 @@ func planFor(ctx context.Context, open *stores, nodeName string) (catalogue.Reso
// a mesh-wide gatherer may pass over — see notResolvable.
return catalogue.Resolution{}, nil, notResolvable{err}
}
logUnheld(nodeName, resolved.Unheld)
// The credential for each thing this node takes from elsewhere. Made once and kept, so the
// password a provider is told to create is the one its consumer was given — and sealed to
@@ -395,7 +398,7 @@ func declarationWith(ctx context.Context, open *stores, node string,
return sendable{}, err
}
return sendable{Resources: composed.Resources, Adoption: adoption,
Received: composed.Received, Mesh: with.Mesh,
Received: composed.Received, Mesh: with.Mesh, BusUsers: with.BusUsers,
LeftOut: sortedKeysOf(composed.LeftOut), leftOutWhy: composed.LeftOut}, nil
}
@@ -657,6 +660,13 @@ func renderingFor(ctx context.Context, open *stores, node string,
machines[name] = at
}
// And every zone a module in the mesh answers itself (novox/hq ADR 0199), for the mesh's resolver
// to forward.
zones, err := zonesInTheMesh(ctx, open)
if err != nil {
return catalogue.Rendering{}, inventory.Node{}, err
}
// **The bus is never public** (novox/hq ADR 0169). It was a foundation port — widened from the
// broker's own `from: mesh` to from-anywhere on the broker's host, so a machine could enrol
// before it had an address on the private network. A machine joins through the tunnel now, and
@@ -726,14 +736,79 @@ func renderingFor(ctx context.Context, open *stores, node string,
BusMembership: memberships[node],
Settings: settings, Generators: gens, Grants: grants, Needed: needed, Ports: ports,
Certificate: certificate, Authority: authority, Mesh: private, Names: names,
Machines: machines,
Suffix: overlay.Suffix(), MeshRange: meshRange, TunnelInterface: overlay.Interface, Accounts: accounts, Foundation: foundation,
Machines: machines, Zones: zones,
Suffix: overlay.Suffix(), MeshRange: meshRange, TunnelInterface: overlay.Interface, Accounts: accounts, Foundation: foundation,
Kept: kept, Adopted: record.Adopted, OutwardLinks: outwardLinks,
Given: given, Taken: taken, Seats: seats, ArtifactStore: artifactStore, Built: built,
BusUsers: busUsers,
}, record, nil
}
// zonesInTheMesh is every zone a module in the mesh declares, where the mesh placed it (novox/hq ADR
// 0199): the zone settled from that node's settings, the node's private address, the port the
// answering listen is published on there.
//
// Read across every machine's resolution, as the roster once read routed names: a node whose set does
// not compose declares nothing and is passed over, so one broken machine does not cost the rest their
// zones; a store that cannot be read is raised, naming the machine, because returning the zones
// without it would withdraw them from the resolver as if the operator had (novox/hq 04-ISSUES/152).
// What the mesh refuses about the zones together — one declared twice, one shadowing the mesh's
// suffix or a node's public domain — is refused here, by name.
func zonesInTheMesh(ctx context.Context, open *stores) ([]catalogue.ZoneAt, error) {
inv := open.inventory
places, err := inv.Overlays(ctx)
if err != nil {
return nil, fmt.Errorf("where the machines are cannot be read: %w", err)
}
address := map[string]string{}
for _, p := range places {
if strings.TrimSpace(p.Address) != "" {
address[p.Name] = p.Address
}
}
nodes, err := inv.Nodes(ctx)
if err != nil {
return nil, fmt.Errorf("which machines the mesh has cannot be read: %w", err)
}
var zones []catalogue.ZoneAt
var public []string
for _, n := range nodes {
plan, _, err := planFor(ctx, open, n.Name)
switch {
case unresolvable(err):
continue
case err != nil:
return nil, fmt.Errorf("the zones %s answers cannot be read: %w", n.Name, err)
}
if plan.PublicDomain != "" {
public = append(public, plan.PublicDomain)
}
for _, m := range plan.Modules {
if m.Zone == nil {
continue
}
at := address[n.Name]
if at == "" {
// Not on the private network yet: nothing could reach its answerer.
continue
}
published, layers, err := portsGivenOn(ctx, inv, n.Name, m)
if err != nil {
return nil, fmt.Errorf("the zone %s declares on %s cannot be read: %w", m.Module, n.Name, err)
}
z, err := catalogue.ZoneOn(m, layers, published, n.Name, at)
if err != nil {
return nil, err
}
zones = append(zones, *z)
}
}
if problems := catalogue.ZonesProblems(zones, overlay.Suffix(), public); len(problems) > 0 {
return nil, fmt.Errorf("the mesh's zones cannot be forwarded:\n - %s", strings.Join(problems, "\n - "))
}
return zones, nil
}
// certificateFor is what the mesh certifies about one machine's internal name.
//
// It reaches across two contexts and reads neither one's store from the other: `inventory` knows
@@ -905,12 +980,15 @@ func planCommand(ctx context.Context, args []string) error {
// checking it against the host's own parser, most usefully, which is the only way to know
// that what the control plane emits is what the host accepts.
asJSON := set.Bool("json", false, "print the declaration this node would be sent")
// What a push would change, against what the machine was last sent (novox/hq ADR 0217): read
// before sending, so a change that removes, moves or recreates something is seen first.
asDiff := set.Bool("diff", false, "what a push would change on this node, against what it was last sent")
positionals, err := parseAround(set, args)
if err != nil {
return err
}
if len(positionals) != 1 {
return errors.New("plan <node> [--files] [--json]")
return errors.New("plan <node> [--files] [--json] [--diff]")
}
args = positionals
open, err := openStores(ctx)
@@ -927,6 +1005,17 @@ func planCommand(ctx context.Context, args []string) error {
fmt.Printf("%s is assigned nothing\n", args[0])
return nil
}
if *asDiff {
declared, err := declarationFor(ctx, open, args[0], plan, settings)
if err != nil {
return err
}
body, err := declared.Body()
if err != nil {
return err
}
return writePlanDiff(ctx, open.inventory, args[0], body)
}
if *asJSON {
declared, err := declarationFor(ctx, open, args[0], plan, settings)
if err != nil {
@@ -1266,6 +1355,20 @@ func composeBusUsers(ctx context.Context, inv *inventory.Inventory,
//
// Asked of what this push resolves to rather than of the seat's holder mesh-wide: the file is a
// resource of that module, so the question is whether it is here.
list, missing, err := busUserList(ctx, inv, onThisNode)
if len(missing) > 0 {
fmt.Printf("the bus's user list leaves out %d user(s) the mesh has minted no credential "+
"for: %s. Each is a user that cannot connect until one is issued\n",
len(missing), strings.Join(missing, ", "))
}
return list, err
}
// busUserList is composeBusUsers without saying anything: the list, and the users left out of it
// for want of a credential. Asked on every send to decide whether the machine holding the bus must
// go first (novox/hq issue 249), where saying the same missing users each time would bury them.
func busUserList(ctx context.Context, inv *inventory.Inventory,
onThisNode []catalogue.Manifest) (string, []string, error) {
holdsTheBus := false
for _, m := range onThisNode {
if m.BusUsers != "" && m.ClaimsSeat("mesh-broker") {
@@ -1273,37 +1376,33 @@ func composeBusUsers(ctx context.Context, inv *inventory.Inventory,
}
}
if !holdsTheBus {
return "", nil
return "", nil, nil
}
records, err := inv.BusRecords(ctx)
if err != nil {
return "", err
return "", nil, err
}
users, err := broker.Users(records)
if err != nil {
return "", err
return "", nil, err
}
kept, err := inv.BusUsers(ctx)
if err != nil {
return "", err
return "", nil, err
}
hashes := make(map[string]string, len(kept))
for name, u := range kept {
hashes[name] = u.PasswordHash
}
filled, missing := broker.WithPasswords(users, hashes)
if len(missing) > 0 {
fmt.Printf("the bus's user list leaves out %d user(s) the mesh has minted no credential "+
"for: %s. Each is a user that cannot connect until one is issued\n",
len(missing), strings.Join(missing, ", "))
}
if len(filled) == 0 {
return "", fmt.Errorf(
return "", missing, fmt.Errorf(
"this machine runs the bus and not one user has a credential, so the composed list " +
"would refuse every connection in the mesh")
}
return broker.ComposeAccounts(filled)
list, err := broker.ComposeAccounts(filled)
return list, missing, err
}
// providerModuleOf is which module answers a need on the providing node: the one in this node's
@@ -1325,3 +1424,90 @@ func providerModuleOf(resolved catalogue.Resolution, open *stores, ctx context.C
}
return ""
}
// unheldLogged is what was last logged about each node's unmet seat dependencies, so the log says
// each change once (novox/hq ADR 0207), on stderr so `status --json` stays a document.
//
// **The serving controller's log only.** planFor runs for every node on every push, assignment and
// status — `blockedElsewhere` alone resolves the whole mesh — and a one-shot command starts with an
// empty memory, so every node's report was "a change" and a push printed the whole mesh's list,
// burying the line about the node it acted on. A command says what concerns its own act instead
// (unheldChange, reportUnheldPushed); the full list is `status`'s.
var (
unheldLogged = map[string]string{}
unheldLoggedMu sync.Mutex
logUnheldChanges bool
)
// logUnheld logs a node's unmet seat dependencies when they differ from what was last logged for
// it, including when they become none — in the serving controller, and nowhere else.
func logUnheld(node string, unheld []catalogue.Unheld) {
if !logUnheldChanges {
return
}
lines := make([]string, 0, len(unheld))
for _, u := range unheld {
lines = append(lines, u.String())
}
now := strings.Join(lines, "\n")
unheldLoggedMu.Lock()
before, seen := unheldLogged[node]
unheldLogged[node] = now
unheldLoggedMu.Unlock()
if (seen && before == now) || (!seen && now == "") {
return
}
if now == "" {
fmt.Fprintf(os.Stderr, "%s: every seat its modules depend on is held (novox/hq ADR 0207)\n", node)
return
}
fmt.Fprintf(os.Stderr, "%s: %d unmet seat dependenc(ies), reported and not refused (novox/hq ADR 0207):\n %s\n",
node, len(lines), strings.Join(lines, "\n "))
}
// unheldChange is what an act on one node changed about its unmet seat dependencies, judged over
// its assignments before and after (novox/hq ADR 0207): each dependency now unmet that was not —
// which includes every one of a module just assigned — and each now met that was not. Nothing about
// any other node, and nothing that was already true before the act.
func unheldChange(shelf map[string]catalogue.Manifest, node string, before, after []string) []string {
judge := func(names []string) map[string]catalogue.Unheld {
var set []catalogue.Manifest
for _, n := range names {
if m, known := shelf[n]; known {
set = append(set, m)
}
}
out := map[string]catalogue.Unheld{}
for _, u := range catalogue.UnheldDependencies(shelf, node, set, nil) {
out[u.Module+" "+u.Seat] = u
}
return out
}
was, now := judge(before), judge(after)
var lines []string
for _, k := range sortedNames(now) {
if _, already := was[k]; !already {
lines = append(lines, "but "+now[k].String())
}
}
for _, k := range sortedNames(was) {
if _, still := now[k]; still {
continue
}
u := was[k]
if !slices.Contains(after, u.Module) {
continue // went with its module, which says nothing about the seat
}
lines = append(lines, fmt.Sprintf("and %s on %s now has %s held", u.Module, node, u.Seat))
}
return lines
}
func sortedNames[V any](m map[string]V) []string {
out := make([]string, 0, len(m))
for k := range m {
out = append(out, k)
}
sort.Strings(out)
return out
}
@@ -0,0 +1,47 @@
package main
import (
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
)
// novox/hq issue 213: for the moment a machine hands its controller over, the container and the
// process both run the plan timer on one store. Only the one holding the plans moves them; the other
// leaves them alone, and moves them once they are let go.
func TestAControllerLeavesThePlansToTheOneHoldingThem(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
now := time.Now().UTC()
// Every tier done: the next step is the plan's last, and needs nothing but the store.
plan := inventory.Plan{ID: "plan-213", Repository: "r", Commit: "abc", Created: now, Updated: now,
State: inventory.PlanRolling, Tier: 1, Tiers: [][]string{{"app"}},
Modules: map[string]*inventory.PlanModule{"app": {State: "built"}}}
if err := open.inventory.SavePlan(ctx, plan); err != nil {
t.Fatal(err)
}
// The other controller: its own connections to the same store, holding the plans.
other, err := inventory.Open(ctx)
if err != nil {
t.Fatal(err)
}
t.Cleanup(other.Close)
release, err := other.HoldPlans(ctx, false)
if err != nil {
t.Fatal(err)
}
t.Cleanup(release) // before the close above: a pool waits for a connection still held
advancePlans(ctx, open)
if p, err := open.inventory.PlanByID(ctx, "plan-213"); err != nil || !p.Open() {
t.Fatalf("a controller moved a plan another held: %+v %v", p, err)
}
release()
advancePlans(ctx, open)
if p, err := open.inventory.PlanByID(ctx, "plan-213"); err != nil || p.State != inventory.PlanDone {
t.Fatalf("the plan did not move once it was let go: %+v %v", p, err)
}
}
+429 -83
View File
@@ -8,6 +8,7 @@ import (
"errors"
"flag"
"fmt"
"io"
"log"
"os"
"sort"
@@ -62,6 +63,9 @@ func connectLink(ctx context.Context, inv *inventory.Inventory, enroller link.En
}
func serve(ctx context.Context) error {
// The one process whose log is read over time, so the one that says each change to a node's
// unmet seat dependencies once (novox/hq ADR 0207).
logUnheldChanges = true
open, err := openStores(ctx)
if err != nil {
return err
@@ -245,14 +249,31 @@ func pushCommand(ctx context.Context, args []string) error {
// (novox/hq ADR 0010). 0 waits for nothing, which is the old fire-and-forget.
wait := set.Duration("wait", 0,
"for a named node, how long to wait for it to report applying what it was sent (0: do not wait)")
// Every machine at once is said, not defaulted to (novox/hq ADR 0217): a bare `push` sent the
// whole mesh on 2026-10-05 when its usage was wanted.
all := set.Bool("all", false, "every machine")
// A running module's data moving is sent only when said, per module (novox/hq ADR 0217).
move := set.String("move", "", "modules whose data may move with this push, comma-separated")
positionals, err := parseAround(set, args)
if err != nil {
return err
}
args = positionals
if len(args) > 1 {
return errors.New("push [<node>] [--behind] — one node, or all of them")
return errors.New("push <node> | push --behind | push --all — one machine, the ones behind, or all of them")
}
if len(args) == 0 && !*behind && !*all {
return errors.New("push names a machine, or says --behind (the ones not running what they " +
"should) or --all (every machine) — a push to the whole mesh is not the default (novox/hq ADR 0217)")
}
if *all && (*behind || len(args) == 1) {
return errors.New("push --all is every machine; it takes no machine and no --behind")
}
movable := map[string]bool{}
for _, m := range splitModules(*move) {
movable[m] = true
}
var heldBack []string
if len(args) == 1 && *behind {
// Naming a machine and asking for the ones that need it are two different requests, and
// guessing which was meant would sometimes push to a machine somebody did not name.
@@ -354,12 +375,26 @@ func pushCommand(ctx context.Context, args []string) error {
asked = append(asked, n.Name)
}
// **The machine holding the bus first** (novox/hq issue 249): its declaration carries the bus's
// user list, and a module's new grants are refused by the bus until that list says them. Among
// the machines asked it goes first; not among them and behind, it is added — a named push whose
// module gained a state would otherwise send the code and leave the right to use it for the
// cascade below, after.
holder, holderBehind, err := brokerBehind(ctx, open, asked)
if err != nil {
return err
}
asked = brokerFirst(asked, holder, holderBehind)
// Held from composing to sending, so a converge on one of them cannot send between the two
// and be overtaken by what was composed before it (novox/hq ADR 0100).
held, release, err := holdNodes(ctx, open, asked)
if err != nil {
return err
}
// Each machine's unmet seat dependencies (novox/hq ADR 0207), said after the sends: in full
// for a machine named, as a count for each of many — the full list is `status`'s.
unheld := map[string][]catalogue.Unheld{}
sending, refusals := composeEach(asked, allotting(held, inv), func(node string) (sendable, error) {
plan, settings, err := planFor(held, open, node)
if err != nil {
@@ -369,6 +404,7 @@ func pushCommand(ctx context.Context, args []string) error {
// healthy modules beside it are still resolved and sent. Reported so it is not silently
// dropped — the remedy is to move it, and until then the rest of the node converges.
reportUnhostable(node, plan)
unheld[node] = plan.Unheld
// The private network is in here with everything else. It used to be composed separately
// and prepended, which meant every machine with an address was on it and no machine could
// be kept off. It is a module now, so it arrives the way a module does.
@@ -379,34 +415,22 @@ func pushCommand(ctx context.Context, args []string) error {
return declared, err
})
sentDigest := map[string]string{}
defer release()
for _, s := range sending {
// The number is inside the signed bytes, so a replayed older declaration cannot borrow a
// newer one's (novox/hq 04-ISSUES/107); it was taken when the composition began (issue 204).
body, err := s.declared.Body()
if err != nil {
return err
}
if err := link.Declare(ctx, server.Bus(), ident, s.node, body, 15*time.Second); err != nil {
return err
}
// After it is away, not before. A digest recorded for something that failed to send would
// make the machine look current for a declaration it never received.
digest, err := recordSent(ctx, inv, s.node, body)
if err != nil {
return err
}
sentDigest[s.node] = digest
fmt.Printf("sent %s %d resource(s)\n", s.node, len(s.declared.Resources))
// A machine whose declaration would move a running module's data is held, and only it (novox/hq
// ADR 0217); every other machine is delivered, its memberships first, then its declaration
// (novox/hq issue 249, ADR 0218).
bus := overTheBus{open: open, server: server, signer: ident}
toSend, err := holdingBack(ctx, inv, sending, movable, &heldBack)
if err != nil {
return err
}
sentDigest, err := deliver(ctx, bus, holder, toSend)
if err != nil {
return err
}
release()
fmt.Printf("\n%d node(s) told\n", len(sending))
// And each machine's memberships, as every other send does (ADR 0160): a push is the one most
// operators run, and on 2026-10-01 it was the one path that issued none.
if err := issueMemberships(ctx, open, server, sending); err != nil {
return err
}
reportUnheldPushed(os.Stdout, len(args) == 1, asked, unheld)
// **A named push leaves the mesh consistent, not just the machine it named** (novox/hq
// issue 057, ADR 0083). Assigning a cross-node consumer mints a provision, and the PROVIDER's
@@ -475,16 +499,9 @@ func pushCommand(ctx context.Context, args []string) error {
}
return declared, err
},
func(s readyNode, body []byte) error {
if err := link.Declare(ctx, server.Bus(), ident, s.node, body,
15*time.Second); err != nil {
return err
}
if _, err := recordSent(ctx, inv, s.node, body); err != nil {
return err
}
fmt.Printf("sent %s %d resource(s)\n", s.node, len(s.declared.Resources))
return nil
bus, holder, func(held context.Context, sending []readyNode) ([]readyNode, error) {
// The cascade is a push too, and held the same way (novox/hq ADR 0217).
return holdingBack(held, inv, sending, movable, &heldBack)
})
refusals = append(refusals, refused...)
if err != nil {
@@ -507,6 +524,11 @@ func pushCommand(ctx context.Context, args []string) error {
return err
}
}
if len(heldBack) > 0 {
sort.Strings(heldBack)
return fmt.Errorf("held %s: a running module's data would move — see above; nothing was sent "+
"there (novox/hq ADR 0217)", strings.Join(heldBack, ", "))
}
return couldNotBeResolved(refusals, len(sending))
}
@@ -612,10 +634,10 @@ func composeEach(names []string, allot func(node string) (int64, error),
// sendRound holds the named nodes, composes each and sends each that composed, and gives the hold
// back on every way out — a body that cannot be marshalled and a send that fails included
// (novox/hq ADR 0100). A node that cannot be composed is a refusal, not an error: the others are
// still sent.
// still sent. Their memberships go before their declarations, as every send's do (issue 249).
func sendRound(ctx context.Context, open *stores, names []string,
compose func(held context.Context, node string) (sendable, error),
send func(s readyNode, body []byte) error) ([]string, error) {
d delivery, holder string, keep func(context.Context, []readyNode) ([]readyNode, error)) ([]string, error) {
held, release, err := holdNodes(ctx, open, names)
if err != nil {
return nil, err
@@ -624,18 +646,276 @@ func sendRound(ctx context.Context, open *stores, names []string,
sending, refused := composeEach(names, allotting(held, open.inventory), func(node string) (sendable, error) {
return compose(held, node)
})
for _, s := range sending {
body, err := s.declared.Body()
if err != nil {
return refused, err
}
if err := send(s, body); err != nil {
if keep != nil {
if sending, err = keep(held, sending); err != nil {
return refused, err
}
}
if _, err := deliver(held, d, holder, sending); err != nil {
return refused, err
}
return refused, nil
}
// holdingBack leaves out each machine whose declaration would move a running module's data, says so,
// and names it among those held back (novox/hq ADR 0217); the rest go on to be delivered, grants first
// (ADR 0218). A declaration's body is the same bytes however often it is read.
func holdingBack(ctx context.Context, inv *inventory.Inventory, sending []readyNode, movable map[string]bool,
heldBack *[]string) ([]readyNode, error) {
var out []readyNode
for _, s := range sending {
body, err := s.declared.Body()
if err != nil {
return nil, err
}
moves, err := heldMoves(ctx, inv, s.node, body, movable)
if err != nil {
return nil, err
}
if len(moves) > 0 {
sayHeld(os.Stdout, s.node, moves)
*heldBack = append(*heldBack, s.node)
continue
}
out = append(out, s)
}
return out, nil
}
// delivery is the two acts of sending machines what they should be, apart, so the order between
// them is one function's and can be read and tested there (novox/hq issue 249).
type delivery interface {
// grant issues what the machines' modules may do — each module's state raised and its
// membership issued — for every machine about to be sent.
grant(ctx context.Context, sending []readyNode) error
// declare sends one machine its declaration and records it sent, answering the digest.
declare(ctx context.Context, s readyNode, body []byte) (string, error)
}
// errGrants marks a send that stopped because what the machines' modules may do could not be issued
// (novox/hq issue 249). Nothing about the machines is wrong; asked again, it is likely to work, so an
// announcement that hits it is held and asked again.
var errGrants = errors.New("what the machines' modules may do on the bus could not be issued, and code " +
"sent before its grants is refused there")
// grantsRefused is a grant that failed for some machines and not others: their memberships could not
// be issued, by machine, and only those machines are held back.
type grantsRefused struct{ nodes map[string]error }
func (g *grantsRefused) Error() string {
names := make([]string, 0, len(g.nodes))
for n := range g.nodes {
names = append(names, n)
}
sort.Strings(names)
return fmt.Sprintf("the memberships of %s could not be issued; the first: %v",
strings.Join(names, ", "), g.nodes[names[0]])
}
// deliver sends the machines their declarations: **the machine holding the bus, then the grants,
// then the rest** (novox/hq issue 249).
//
// A merge gave a module a new state; its bundle reached every machine within a minute, and the
// machines' permissions on the bus did not include the state until somebody pushed by hand: the code
// arrived before the right to use it. A module that read its new state on start failed its start; the
// one that was there retried for two minutes. The memberships were issued after the declarations —
// "because the runtime it is for arrives with it" — and a membership is retained last-per-subject on
// the bus (internal/link/bus.go), so issued first it waits for the runtime that arrives after it. A
// runtime still on the old code merely holds a grant it does not use yet.
//
// **The holder's declaration before the grants, though.** The bus's user list travels in it, and the
// controller's own right to publish memberships and raise buckets is in that list (the precedent of
// issue 183): grants first, and a grant the controller is not yet allowed to make would hold the very
// declaration that allows it — a lock only a hand on the broker could open. A runtime already running
// on that machine follows a membership issued after its declaration, as it always has.
//
// **A grant that cannot be issued holds back what it concerns, and says so as an error.** It used to
// be said and passed over — "the machines keep what they derive until the next push" — which reported
// a rollout done that had delivered code its machines could not run. A membership that failed holds
// back its own machine; a failure that names no machine (the buckets) holds back every machine but the
// holder, already sent. The error carries errGrants, so the caller's rollout is not marked sent and is
// tried again.
//
// The grants are issued, not waited on: a membership is a retained message the runtime reads when it
// comes, and the bus answers its publication; nothing here waits for a runtime to have read one.
func deliver(ctx context.Context, d delivery, holder string, sending []readyNode) (map[string]string, error) {
digests := map[string]string{}
if len(sending) == 0 {
return digests, nil
}
send := func(s readyNode) error {
// The number is inside the signed bytes, so a replayed older declaration cannot borrow a
// newer one's (novox/hq 04-ISSUES/107); it was taken when the composition began (issue 204).
body, err := s.declared.Body()
if err != nil {
return err
}
digest, err := d.declare(ctx, s, body)
if err != nil {
return err
}
digests[s.node] = digest
return nil
}
var rest []readyNode
for _, s := range sending {
if holder != "" && s.node == holder {
if err := send(s); err != nil {
return digests, err
}
continue
}
rest = append(rest, s)
}
held := map[string]error{}
if err := d.grant(ctx, sending); err != nil {
var some *grantsRefused
if !errors.As(err, &some) {
var names []string
for _, s := range rest {
names = append(names, s.node)
}
if len(names) == 0 {
return digests, fmt.Errorf("%w: %w", errGrants, err)
}
return digests, fmt.Errorf("%w; %s not sent: %w", errGrants, strings.Join(names, ", "), err)
}
held = some.nodes
}
var notSent []string
for _, s := range rest {
if _, refused := held[s.node]; refused {
notSent = append(notSent, s.node)
continue
}
if err := send(s); err != nil {
return digests, err
}
}
if len(notSent) > 0 {
return digests, fmt.Errorf("%w; %s not sent: %w", errGrants, strings.Join(notSent, ", "),
&grantsRefused{nodes: held})
}
if len(held) > 0 {
// Only the holder's own memberships failed, and it was sent before them.
return digests, fmt.Errorf("%w: %w", errGrants, &grantsRefused{nodes: held})
}
return digests, nil
}
// overTheBus is delivery as the mesh does it: memberships on the bus, declarations signed.
type overTheBus struct {
open *stores
server *link.Server
signer link.Signer
// indent is put before each "sent" line, for the callers whose output is nested.
indent string
}
func (b overTheBus) grant(ctx context.Context, sending []readyNode) error {
return issueMemberships(ctx, b.open, b.server, sending)
}
func (b overTheBus) declare(ctx context.Context, s readyNode, body []byte) (string, error) {
if err := link.Declare(ctx, b.server.Bus(), b.signer, s.node, body, 15*time.Second); err != nil {
return "", err
}
// After it is away, not before. A digest recorded for something that failed to send would make
// the machine look current for a declaration it never received.
digest, err := recordSent(ctx, b.open.inventory, s.node, body)
if err != nil {
return "", err
}
if s.declared.BusUsers != "" {
// And the user list it carried, so the next send reads whether it must go first from the
// list alone (novox/hq issue 249). On the same outliving context as the send's record.
kept, cancel := context.WithTimeout(context.WithoutCancel(ctx), 10*time.Second)
err := b.open.inventory.RecordSentBusUsers(kept, s.node, digestOf([]byte(s.declared.BusUsers)))
cancel()
if err != nil {
return "", err
}
}
fmt.Printf("%ssent %s %d resource(s)\n", b.indent, s.node, len(s.declared.Resources))
return digest, nil
}
// brokerFirst is the machines to send in the order a grant needs (novox/hq issue 249): the machine
// holding the bus first — its declaration carries the bus's user list (composeBusUsers), and a
// module's new permissions are refused by the bus until that list says them. Among the machines it
// is moved to the front; not among them, it is added only when it is behind.
func brokerFirst(names []string, holder string, behind bool) []string {
if holder == "" {
return names
}
present := false
rest := make([]string, 0, len(names))
for _, n := range names {
if n == holder {
present = true
continue
}
rest = append(rest, n)
}
if !present && !behind {
return names
}
return append([]string{holder}, rest...)
}
// brokerBehind is the machine holding the bus — the one whose declaration carries the user list —
// and, when it is not among the machines named, whether the user list it would be sent now differs
// from the one it was last sent (novox/hq issue 249).
//
// **The user list alone, not the whole declaration.** Read from the whole declaration, any change
// pending on that machine — an upgrade its policy records rather than rolls out — went with every
// send anywhere, and a module running there always put it in its first wave. A digest of the list
// last sent is kept for this (ADR 0043: the list is composed on each push, never kept itself).
func brokerBehind(ctx context.Context, open *stores, names []string) (string, bool, error) {
inv := open.inventory
holders, err := seatHolders(ctx, inv)
if err != nil {
return "", false, err
}
h, held := holders[theBrokerSeat]
if !held || h.Node == "" {
return "", false, nil
}
shelf, err := inv.Catalogue(ctx)
if err != nil {
return "", false, err
}
if m, known := shelf[h.Module]; !known || m.BusUsers == "" {
// A holder that is sent no user list carries no grant: nothing to send first.
return "", false, nil
}
for _, n := range names {
if n == h.Node {
return h.Node, false, nil
}
}
plan, _, err := planFor(ctx, open, h.Node)
if err != nil {
// It cannot be worked out: sending it would refuse the whole send, and `plan` says why.
return h.Node, false, nil
}
list, _, err := busUserList(ctx, inv, plan.Modules)
if err != nil {
return h.Node, false, nil
}
sent, err := inv.SentBusUsers(ctx, h.Node)
if err != nil {
return "", false, err
}
return h.Node, userListBehind(list, sent), nil
}
// userListBehind is whether the user list composed now is not the one last sent, by its digest. An
// empty list composed is never behind: there is nothing for it to carry.
func userListBehind(now, sentDigest string) bool {
return now != "" && digestOf([]byte(now)) != sentDigest
}
// couldNotBeResolved is what a push ends with when some machines could not be worked out.
//
// **After the rest have been sent, never instead of sending them.** It is still an error, because
@@ -658,23 +938,36 @@ func couldNotBeResolved(refusals []string, sent int) error {
// consumer and refused on the provider would leave one end holding a credential the other has
// never heard of — which is the state this whole mechanism exists to make impossible.
func sendTo(ctx context.Context, open *stores, names []string) error {
_, err := sendToEach(ctx, open, names)
return err
}
// sendToEach is sendTo, answering the machines it sent: those named, and before them the machine
// holding the bus when its user list must go first (novox/hq issue 249) — so a caller that waits for
// the machines it sent waits for that one too.
func sendToEach(ctx context.Context, open *stores, names []string) ([]string, error) {
inv := open.inventory
ident, err := openIdentity(ctx)
if err != nil {
return err
return nil, err
}
defer ident.Close()
gens, err := generators(ctx, open)
if err != nil {
return err
return nil, err
}
holder, behind, err := brokerBehind(ctx, open, names)
if err != nil {
return nil, err
}
names = brokerFirst(names, holder, behind)
// Held from composing to sending (novox/hq ADR 0100); a caller that holds them already —
// converge, which flips the node and then sends it — is not made to wait on itself.
ctx, release, err := holdNodes(ctx, open, names)
if err != nil {
return err
return nil, err
}
defer release()
@@ -703,33 +996,29 @@ func sendTo(ctx context.Context, open *stores, names []string) error {
sending = append(sending, readyNode{name, declared})
}
if len(refusals) > 0 {
return fmt.Errorf("nothing was sent. %d machine(s) could not be resolved:\n\n%s",
return nil, fmt.Errorf("nothing was sent. %d machine(s) could not be resolved:\n\n%s",
len(refusals), strings.Join(refusals, "\n\n"))
}
server, err := connectLink(ctx, nil, nil, nil)
if err != nil {
return err
return nil, err
}
defer server.Close()
for _, s := range sending {
body, err := s.declared.Body()
if err != nil {
return err
}
if err := link.Declare(ctx, server.Bus(), ident, s.node, body, 15*time.Second); err != nil {
return err
}
if _, err := recordSent(ctx, inv, s.node, body); err != nil {
return err
}
fmt.Printf(" sent %s %d resource(s)\n", s.node, len(s.declared.Resources))
}
// And every assignment on those machines its membership (novox/hq ADR 0160): composed from the
// same records the bus's accounts are, so what a runtime serves and what its account may are one
// composition. Issued after the declaration, because the runtime it is for arrives with it.
return issueMemberships(ctx, open, server, sending)
// composition. **Issued before the declarations** (novox/hq issue 249): the runtime the
// membership is for arrives with the declaration, and a membership waits for it on the bus; the
// code arriving first was refused its own state until somebody pushed.
if _, err := deliver(ctx, overTheBus{open: open, server: server, signer: ident, indent: " "}, holder, sending); err != nil {
return nil, err
}
sent := make([]string, 0, len(sending))
for _, s := range sending {
sent = append(sent, s.node)
}
return sent, nil
}
// issueMemberships publishes the membership of every module on the machines just sent.
@@ -747,11 +1036,26 @@ func issueMemberships(ctx context.Context, open *stores, server *link.Server, se
if !ok {
return nil
}
// The declarations are sent and recorded by now; a membership that cannot be issued is said
// and does not unsay them. Every runtime without one serves the shape it derives (ADR 0160), so
// the push stands, the first failure is named once, and the next push tries again.
issued, failed := 0, 0
var first error
// **Every declared state's bucket, before the memberships that name it** (novox/hq ADR 0201). The
// raise at start asserts them too, but a module registered and assigned since would otherwise have
// its bucket only after the control plane next restarts — found the first time a module declared
// state: its bundle asked for a bucket that did not exist. Idempotent and cheap.
//
// **A failure here is the send's failure** (novox/hq issue 249). It was said and the push stood,
// because the declarations were already away; they are sent after this now — all but the bus's
// own machine, sent before it (deliver) — and a module whose state does not exist is a module
// that fails its start, so they are not sent and the caller tries again rather than reporting the
// rollout done.
buckets, err := open.inventory.DeclaredBuckets(ctx)
if err != nil {
return fmt.Errorf("the modules' state could not be read, so no bucket was asserted: %w", err)
}
if _, err := broker.RaiseBuckets(broker.OnConn(bus.Conn), buckets); err != nil {
return fmt.Errorf("the modules' state could not be asserted on the bus: %w", err)
}
// Every membership is tried, and the first failure named once.
issued := 0
refused := map[string]error{}
for _, s := range sent {
node := s.node
for _, d := range records.Assigned[node] {
@@ -772,10 +1076,9 @@ func issueMemberships(ctx context.Context, open *stores, server *link.Server, se
return err
}
if err := bus.PublishMembership(ctx, node, d.Module, body); err != nil {
if first == nil {
first = err
if refused[node] == nil {
refused[node] = fmt.Errorf("%s: %w", d.Module, err)
}
failed++
continue
}
issued++
@@ -784,9 +1087,10 @@ func issueMemberships(ctx context.Context, open *stores, server *link.Server, se
if issued > 0 {
fmt.Printf(" issued %d membership(s)\n", issued)
}
if failed > 0 {
fmt.Printf(" %d membership(s) could not be issued; the first: %v — the machines keep what "+
"they derive until the next push\n", failed, first)
if len(refused) > 0 {
// Returned, never passed over (novox/hq issue 249): the declarations of the machines they are
// for are not sent, and the rollout that asked is tried again rather than waiting for a push.
return &grantsRefused{nodes: refused}
}
return nil
}
@@ -898,6 +1202,21 @@ func raiseTheBus(ctx context.Context, inv *inventory.Inventory, address string)
if err := broker.RaiseSeats(js, inventory.MeshSeats(), holders); err != nil {
return err
}
// Every module's state (novox/hq ADR 0201), from the catalogue: a bucket exists from
// registration, so a module reading one may watch it before its owner runs anywhere. One that
// nothing declares any more is said and kept — what it holds is data.
buckets, err := inv.DeclaredBuckets(ctx)
if err != nil {
return err
}
undeclared, err := broker.RaiseBuckets(js, buckets)
if err != nil {
return err
}
if len(undeclared) > 0 {
fmt.Printf("the bus holds state nothing declares any more, kept because it is data: %s — "+
"removing it is a person's act\n", strings.Join(undeclared, ", "))
}
// And how every module hears what it consumes. Derived from the same records the user list is
// composed from, so a module the mesh grants a consumer's subjects has that consumer waiting.
// Done on every raise, not only when a credential is issued: every module moved onto this bus
@@ -911,18 +1230,14 @@ func raiseTheBus(ctx context.Context, inv *inventory.Inventory, address string)
return err
}
hearing := 0
for _, p := range users {
consumer, needed := broker.ConsumerFor(p)
if !needed {
continue
}
if err := js.EnsureConsumer(consumer); err != nil {
return fmt.Errorf("how %s on %s hears what it consumes: %w", p.Module, p.Node, err)
for _, c := range broker.ConsumersOf(users) {
if err := js.EnsureConsumer(c.Consumer); err != nil {
return fmt.Errorf("how %s on %s hears what it consumes: %w", c.Module, c.Node, err)
}
hearing++
}
fmt.Printf("the bus at %s has its streams, %d machine(s) can hear a declaration, and %d module(s) "+
"can hear what they consume\n", broker.BareAddress(address), len(names), hearing)
fmt.Printf("the bus at %s has its streams, %d machine(s) can hear a declaration, %d module(s) "+
"can hear what they consume, and %d bucket(s) of state\n", broker.BareAddress(address), len(names), hearing, len(buckets))
return nil
}
@@ -995,5 +1310,36 @@ func recordSent(ctx context.Context, inv *inventory.Inventory, node string, body
if err := inv.RecordSent(kept, record.ID, digest); err != nil {
return "", err
}
// And what it was, summarised, so the next push can be compared with it (novox/hq ADR 0217).
// Never a reason to fail a send that is already away: a summary that cannot be kept means the
// next comparison has nothing to hold against, which is how every machine starts.
if summary, err := summarize(body); err == nil {
if raw, err := json.Marshal(summary); err == nil {
if err := inv.RecordSentSummary(kept, record.ID, raw); err != nil {
fmt.Fprintf(os.Stderr, "%s: what it was sent is not kept for comparison: %v\n", node, err)
}
}
}
return digest, nil
}
// reportUnheldPushed says what a push's machines lack of the seats their modules depend on
// (novox/hq ADR 0207): every line for a machine the push named, since that is the machine somebody
// is looking at, and one line per machine otherwise — a list per machine across the mesh is the
// hundred lines that buried the one that mattered. Nothing for a machine that lacks nothing.
func reportUnheldPushed(w io.Writer, named bool, asked []string, unheld map[string][]catalogue.Unheld) {
for _, node := range asked {
lines := unheld[node]
if len(lines) == 0 {
continue
}
if named {
fmt.Fprintf(w, "\n%s has %d unmet seat dependenc(ies) (novox/hq ADR 0207):\n", node, len(lines))
for _, u := range lines {
fmt.Fprintf(w, " %s\n", u)
}
continue
}
fmt.Fprintf(w, "%s: %d unmet seat dependenc(ies) — see `status`\n", node, len(lines))
}
}
+6
View File
@@ -3,6 +3,7 @@ package main
import (
"encoding/json"
"fmt"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"sort"
"time"
@@ -73,6 +74,10 @@ type meshStatus struct {
// alone. A document without this called a machine well while a predecessor's chain refused
// what the mesh declared open.
Filtered []machineFiltered `json:"filtered,omitempty"`
// Unheld is every module on a machine whose resources are applied through a seat nothing on
// that machine holds, with the modules that could hold it (novox/hq ADR 0207). Absent when every
// dependency is met. Reported, not refused, until the switch.
Unheld []catalogue.Unheld `json:"unheld,omitempty"`
}
// machineFiltered is one rule set on a converged machine that the mesh did not write and that
@@ -204,6 +209,7 @@ func statusAsJSON(asked answers) ([]byte, error) {
out.Filtered = append(out.Filtered, machineFiltered{Node: name, Where: x.Where, Owner: x.Owner, Refuses: x.Refuses})
}
}
out.Unheld = asked.unheld
for name := range asked.refused {
out.Unresolved = append(out.Unresolved, machineUnresolved{
Node: name, Problem: asked.refused[name]})
+401 -17
View File
@@ -2,6 +2,7 @@ package main
import (
"context"
"errors"
"flag"
"fmt"
"sort"
@@ -183,6 +184,7 @@ func planOfMerge(m link.SourceMoved, moved []string, edges []inventory.Edge) inv
return inventory.Plan{
ID: fmt.Sprintf("plan-%d", time.Now().UnixNano()),
Repository: m.Owner + "/" + m.Repo,
Branch: m.Base,
Commit: m.Commit,
Created: time.Now().UTC(),
State: inventory.PlanBuilding,
@@ -191,6 +193,57 @@ func planOfMerge(m link.SourceMoved, moved []string, edges []inventory.Edge) inv
}
}
// supersededBy is what a newer plan takes over from the open plans it supersedes (novox/hq issue
// 254, ADR 0218): the modules they had not finished, and those plans closed as superseded.
//
// **A merge looked at no plan but its own.** Two merges of one repository a few minutes apart were
// two open plans asking for the same modules, each sending machines what it built; and a plan that
// would never move again — waiting on a report that could not come, at 97b1b2b — stayed open for
// ever beside the newer ones, read as work in progress by everyone who looked. The newer merge is the
// newer intent for that repository and branch, so its plan takes over: every open plan of the same
// repository and branch **created before it** — by the time the plans were made, never by comparing
// commits, which have no order of their own — gives up the modules it had not built, and those are
// planned again in the newer plan beside what the newer merge moved.
//
// "Not built" is a module not yet asked, or asked and not answered; **and a module built and not
// yet sent to its machines**, where its policy rolls it out: closed, the older plan would never send
// it, and the catalogue announces no move for a rebuild (issue 189), so the newer plan builds and
// sends it. A build the older plan asked still finishes and registers as any build does — ordered by
// when it was asked (issue 219), so the newer plan's ask, made later, is the one that stands.
//
// A plan with no branch recorded is from before branches were kept, and is superseded by the next
// plan of its repository: what it had not built is folded in, so nothing is lost by it.
func supersededBy(newer inventory.Plan, open []inventory.Plan, rollsOut func(string) bool) ([]string, []inventory.Plan) {
folded := map[string]bool{}
var closed []inventory.Plan
for _, old := range open {
if old.ID == newer.ID || !old.Open() || !strings.EqualFold(old.Repository, newer.Repository) ||
(old.Branch != "" && old.Branch != newer.Branch) || !old.Created.Before(newer.Created) {
continue
}
var took []string
for name, s := range old.Modules {
if s == nil || s.State != "built" || (s.SentAt == nil && rollsOut(name)) {
folded[name] = true
took = append(took, name)
}
}
sort.Strings(took)
old.State = inventory.PlanSuperseded
old.Note = fmt.Sprintf("superseded at tier %d by %s (%s at %s)", old.Tier, newer.ID, newer.Repository, short(newer.Commit))
if len(took) > 0 {
old.Note += "; " + strings.Join(took, ", ") + " planned there again"
}
closed = append(closed, old)
}
out := make([]string, 0, len(folded))
for name := range folded {
out = append(out, name)
}
sort.Strings(out)
return out, closed
}
// gates is what the next tier needs running from this one: a module of the tier that a later
// tier is built by — the runtime dependency — and whose policy rolls it out, must be applied by
// the machines running it before the next tier is asked. A base an image stands on need only be
@@ -229,6 +282,22 @@ func gates(p inventory.Plan, edges []inventory.Edge, rollsOut func(string) bool)
}
// applied says whether every machine running the module has reported since the module was built.
// appliedEach is applied with a moment of its own for each machine: the reports that count are the ones
// after that machine was sent the build (novox/hq issue 256).
func appliedEach(module string, since func(node string) time.Time, running []string, reports []inventory.Reported) (bool, []string) {
at := map[string]*time.Time{}
for _, r := range reports {
at[r.Node] = r.At
}
var waiting []string
for _, n := range running {
if t := at[n]; t == nil || t.Before(since(n)) {
waiting = append(waiting, n)
}
}
return len(waiting) == 0, waiting
}
func applied(module string, builtAt time.Time, running []string, reports []inventory.Reported) (bool, []string) {
at := map[string]*time.Time{}
for _, r := range reports {
@@ -272,7 +341,8 @@ func askTier(ctx context.Context, inv *inventory.Inventory, p *inventory.Plan) e
}
source := buildSource{Repository: e.Source.Repository, Seat: e.Source.Seat}
fmt.Printf(" tier %d: ", p.Tier)
if err := buildOne(ctx, source, e.Source.Path, e.Source.Ref, 0); err != nil {
// The branch it follows, never a commit a build once named (novox/hq 04-ISSUES/215).
if err := buildOne(ctx, source, e.Source.Path, followedBranch(e.Source.Ref), 0); err != nil {
state.State = "failed"
state.Why = err.Error()
p.State = inventory.PlanFailed
@@ -287,8 +357,23 @@ func askTier(ctx context.Context, inv *inventory.Inventory, p *inventory.Plan) e
// planBuilt marks a module built (or failed) in every open plan whose current tier holds it, and
// advances what that completes. Called from the daemon's take-in of every outcome.
func planBuilt(ctx context.Context, open *stores, module, commit, failed string) {
//
// **Only a build asked at or after the plan's ask is its outcome** (novox/hq 04-ISSUES/219). Two
// plans a few minutes apart both ask for a module; the earlier plan's build, finishing late, is not
// the later plan's answer — it stood on the bases from before the later plan's merge, and taking it
// would send machines, and the next tier, what the later merge replaced. asked is zero when the
// build's request time is not known, and such an outcome is taken as before.
func planBuilt(ctx context.Context, open *stores, module, commit, failed string, asked time.Time) {
inv := open.inventory
// One controller works the plans at a time (novox/hq issue 213); an outcome waits its turn rather
// than write over what the holder is about to save. Not taken, it is still in the build records,
// which the holder settles the plan from (issue 214).
release, err := inv.HoldPlans(ctx, true)
if err != nil {
fmt.Printf("plans: %s's outcome is left to the build records: %v\n", module, err)
return
}
defer release()
plans, err := inv.OpenPlans(ctx)
if err != nil {
fmt.Printf("plans: cannot read them: %v\n", err)
@@ -314,11 +399,18 @@ func planBuilt(ctx context.Context, open *stores, module, commit, failed string)
state = &inventory.PlanModule{}
p.Modules[module] = state
}
if askedBefore(asked, state.AskedAt) {
continue
}
if failed != "" {
state.State = "failed"
state.Why = failed
p.State = inventory.PlanFailed
p.Note = fmt.Sprintf("%s failed to build in tier %d", module, p.Tier)
sayUnsent(p, func(m string) bool {
u, err := inv.UpgradeOf(ctx, m)
return err == nil && u.RollOut
})
} else {
state.State = "built"
state.BuiltAt = &now
@@ -332,13 +424,30 @@ func planBuilt(ctx context.Context, open *stores, module, commit, failed string)
fmt.Printf("%s: %s; the tiers after it are not asked\n", p.ID, p.Note)
}
}
advancePlans(ctx, open)
advanceHeld(ctx, open)
}
// advancePlans moves every open plan as far as the facts allow: a tier whose modules are all built
// and whose gates are applied gives way to the next; the last tier done is the plan done. Called
// after every outcome and on a timer, so a plan waiting on a machine's report moves when it comes.
//
// **One controller at a time** (novox/hq issue 213). A plan is read, changed and saved whole; two
// controllers — the old and the new while a machine hands its controller over — would each ask a
// tier the other had just asked. Taken without waiting: whoever holds the plans is moving them.
func advancePlans(ctx context.Context, open *stores) {
release, err := open.inventory.HoldPlans(ctx, false)
if err != nil {
if !errors.Is(err, inventory.ErrPlansBusy) {
fmt.Printf("plans: cannot hold them: %v\n", err)
}
return
}
defer release()
advanceHeld(ctx, open)
}
// advanceHeld is advancePlans for a caller already holding the plans.
func advanceHeld(ctx context.Context, open *stores) {
inv := open.inventory
plans, err := inv.OpenPlans(ctx)
if err != nil {
@@ -363,8 +472,17 @@ func advancePlans(ctx context.Context, open *stores) {
moved, err := advanceOnce(ctx, open, p, edges, rollsOut)
if err != nil {
fmt.Printf("%s: %v\n", p.ID, err)
// Kept in the plan, so `plans` says why it has not moved rather than the log alone;
// the state is left as it was and the step is tried again on the next tick.
p.Note = "tier " + fmt.Sprint(p.Tier) + ": " + err.Error() + " — tried again"
if err := inv.SavePlan(ctx, *p); err != nil {
fmt.Printf("%s: cannot keep the plan: %v\n", p.ID, err)
}
break
}
if p.State == inventory.PlanFailed {
sayUnsent(p, rollsOut)
}
if err := inv.SavePlan(ctx, *p); err != nil {
fmt.Printf("%s: cannot keep the plan: %v\n", p.ID, err)
break
@@ -399,6 +517,24 @@ func advanceOnce(ctx context.Context, open *stores, p *inventory.Plan,
}
return true, nil
}
// **Asked: settle from the build records first** (novox/hq 04-ISSUES/214). An outcome is taken
// in by whichever controller hears it, and a merge to the controller's own repository replaces
// the controller in its first tier: the build that produced the new one is recorded, and the
// plan never hears it. The record is the fact; a build recorded after the ask is that tier's
// outcome, whoever was listening.
recorded := map[string][]inventory.Build{}
for _, m := range tier {
if s := p.Modules[m]; s != nil && s.State == "asked" {
builds, err := inv.Builds(ctx, m, 5)
if err != nil {
return false, err
}
recorded[m] = builds
}
}
if settleFromRecords(p, tier, recorded) {
return true, nil
}
// Asked: wait for every build.
var latest time.Time
for _, m := range tier {
@@ -415,6 +551,15 @@ func advanceOnce(ctx context.Context, open *stores, p *inventory.Plan,
// a module that packages another repository's source, keeps its commit; the catalogue announces
// no move for it and its machines would keep the old image until somebody pushed (novox/hq
// issue 189). A module whose policy records is built and left, as its policy says.
//
// **One machine first, unless the module's policy says together** (novox/hq issue 249, ADR
// 0218). The plan sent every machine running the module at once, and the operator's policy —
// one at a time, stopping at the first that fails, which an announced upgrade honours — was not
// read here at all: a module whose new declarations broke it broke everywhere in the same
// minute. Now the first machine is sent, the plan records it and waits for that machine's report
// after the send to say it applied what it was sent; only then are the rest sent. A first machine
// that fails or refuses stops the module's rollout and the plan with it, the rest untouched.
var pending []string
for _, m := range tier {
state := p.Modules[m]
if state == nil || state.SentAt != nil || !rollsOut(m) {
@@ -424,17 +569,64 @@ func advanceOnce(ctx context.Context, open *stores, p *inventory.Plan,
if err != nil {
return false, err
}
policy, err := inv.UpgradeOf(ctx, m)
if err != nil {
return false, err
}
var reports []inventory.Reported
if !policy.Together {
// Read for the choice of the first machine as well as for its report.
if reports, err = inv.LastReports(ctx); err != nil {
return false, err
}
}
now := time.Now().UTC()
state.SentAt = &now
if len(running) == 0 {
step := nextRollout(*state, running, policy.Together, reports, now, planWaitBound)
switch {
case step.failed != "":
state.Why = step.failed
p.State = inventory.PlanFailed
p.Note = fmt.Sprintf("%s stopped at its first machine in tier %d: %s; %s left as it was",
m, p.Tier, step.failed, orNone(strings.Join(step.rest, ", ")))
fmt.Printf("%s: %s\n", p.ID, p.Note)
return true, nil
case step.waiting != "":
pending = append(pending, fmt.Sprintf("%s on %s, sent first at %s", m, step.waiting, state.FirstAt.Local().Format("15:04")))
continue
case len(step.send) == 0:
// No machine runs it: nothing to send, and nothing to wait for.
state.SentAt = &now
continue
}
if err := sendTo(ctx, open, running); err != nil {
return false, fmt.Errorf("sending %s to %s after tier %d: %w", m, strings.Join(running, ", "), p.Tier, err)
// What sendToEach answers, not what was asked: the machine holding the bus is sent before
// the first when its user list must change (issue 249), and the plan waits for it too.
sent, err := sendToEach(ctx, open, step.send)
if err != nil {
// Not marked sent, so the next step tries again (issue 249): a grant that could not be
// issued is a send that did not happen.
return false, fmt.Errorf("sending %s to %s after tier %d: %w", m, strings.Join(step.send, ", "), p.Tier, err)
}
fmt.Printf("%s: tier %d built; sent %s to %s\n", p.ID, p.Tier, m, strings.Join(running, ", "))
if step.first {
state.First = sent
state.FirstAt = &now
p.State = inventory.PlanRolling
p.Note = fmt.Sprintf("tier %d built; sent %s to %s first", p.Tier, m, strings.Join(sent, ", "))
fmt.Printf("%s: tier %d built; sent %s to %s first, the rest once it reports it applied\n",
p.ID, p.Tier, m, strings.Join(sent, ", "))
return true, nil
}
state.SentAt = &now
fmt.Printf("%s: tier %d built; sent %s to %s\n", p.ID, p.Tier, m, strings.Join(sent, ", "))
return true, nil
}
if len(pending) > 0 {
note := "tier " + fmt.Sprint(p.Tier) + " built; waiting for " + strings.Join(pending, "; ") +
" to report it applied before the rest are sent"
changed := p.State != inventory.PlanRolling || p.Note != note
p.State = inventory.PlanRolling
p.Note = note
return changed, nil
}
// And wait for what the next tier needs running.
needed := gates(*p, edges, rollsOut)
if len(needed) > 0 {
@@ -461,10 +653,24 @@ func advanceOnce(ctx context.Context, open *stores, p *inventory.Plan,
if state.BuiltAt != nil {
since = *state.BuiltAt
}
if state.SentAt != nil && state.SentAt.After(since) {
since = *state.SentAt
// **Each machine from its own send** (novox/hq issue 256). With one machine first (ADR 0218)
// a module is sent twice — the first machine, then the rest — and SentAt is the second.
// Asked of every machine, the first machine's report, made between the two sends, read as
// older than the build, and the gate waited for a report it already had, for ever.
sinceFor := func(node string) time.Time {
at := since
sent := state.SentAt
for _, n := range state.First {
if n == node {
sent = state.FirstAt
}
}
if sent != nil && sent.After(at) {
at = *sent
}
return at
}
if ok, on := applied(m, since, running, reports); !ok {
if ok, on := appliedEach(m, sinceFor, running, reports); !ok {
waiting = append(waiting, fmt.Sprintf("%s on %s", m, strings.Join(on, ", ")))
}
}
@@ -485,6 +691,110 @@ func advanceOnce(ctx context.Context, open *stores, p *inventory.Plan,
return true, nil
}
// rolloutStep is what a plan does next with one built module's machines (novox/hq issue 249).
type rolloutStep struct {
// send is the machines to send now; first, whether they are the first machine's send.
send []string
first bool
// waiting names the first machines whose report the rest wait for.
waiting string
// failed says how a first machine did not take it; rest is what is then left alone.
failed string
rest []string
}
// nextRollout is the next step of one module's rollout in a plan (novox/hq issue 249, ADR 0218).
//
// Together, every machine running it at once, as the policy says. Otherwise one machine first — the
// first by name among those that have reported within the bound, so a laptop that is away is not
// the one the rest wait on; the first by name when none has; the same choice on every controller and
// every resume — and the rest once each machine the first send reached reports, about the
// declaration it was last sent, that it applied it. Compared by the store's own record of what was
// sent (Reported.Current), never by this controller's clock against the machine's.
//
// **A first machine that fails, refuses, or does not report within the bound stops the rollout
// there** (ADR 0218 §2), naming the machine; the rest are not sent. A wait with no end is not a
// rollout: it held the plan open for ever, read as work in progress (issue 254).
func nextRollout(s inventory.PlanModule, running []string, together bool, reports []inventory.Reported,
now time.Time, bound time.Duration) rolloutStep {
if len(running) == 0 {
return rolloutStep{}
}
if together {
return rolloutStep{send: running}
}
byNode := map[string]inventory.Reported{}
for _, r := range reports {
byNode[r.Node] = r
}
if s.FirstAt == nil {
sorted := append([]string{}, running...)
sort.Strings(sorted)
for _, n := range sorted {
if r, said := byNode[n]; said && r.At != nil && now.Sub(*r.At) <= bound {
return rolloutStep{send: []string{n}, first: true}
}
}
return rolloutStep{send: sorted[:1], first: true}
}
sentFirst := map[string]bool{}
for _, n := range s.First {
sentFirst[n] = true
}
var rest []string
for _, n := range running {
if !sentFirst[n] {
rest = append(rest, n)
}
}
var waiting, failed []string
for _, n := range s.First {
r, said := byNode[n]
// Only a report about what it was last sent says anything about this build.
if !said || r.At == nil || !r.Current {
waiting = append(waiting, n)
continue
}
switch r.Outcome {
case inventory.OutcomeApplied:
case inventory.OutcomeFailed, inventory.OutcomeRefused:
failed = append(failed, n+" "+r.Outcome+" what it was sent")
default:
waiting = append(waiting, n)
}
}
if len(failed) > 0 {
return rolloutStep{failed: strings.Join(failed, "; "), rest: rest}
}
if len(waiting) > 0 {
if now.Sub(*s.FirstAt) > bound {
return rolloutStep{failed: fmt.Sprintf("%s did not report it applied within %s",
strings.Join(waiting, ", "), bound), rest: rest}
}
return rolloutStep{waiting: strings.Join(waiting, ", ")}
}
return rolloutStep{send: rest}
}
// sayUnsent adds to an ended plan's note the modules it built and never sent (novox/hq issue 249).
// An announced move of a module a plan held was left to that plan; a plan that ends without
// sending it — failed elsewhere, or closed by hand — would leave its machines behind with nothing
// saying so. A module whose rollout stopped at its first machine is not among them: that stop was
// the point. Said once.
func sayUnsent(p *inventory.Plan, rollsOut func(string) bool) {
var unsent []string
for name, s := range p.Modules {
if s != nil && s.State == "built" && s.SentAt == nil && s.FirstAt == nil && rollsOut(name) {
unsent = append(unsent, name)
}
}
if len(unsent) == 0 || strings.Contains(p.Note, "built and never sent") {
return
}
sort.Strings(unsent)
p.Note += "; built and never sent: " + strings.Join(unsent, ", ") + " — `push --behind` sends them"
}
// planTicker advances open plans on a timer, for the steps outcomes alone cannot take.
func planTicker(ctx context.Context, open *stores) {
advancePlans(ctx, open)
@@ -508,8 +818,10 @@ func planLine(p inventory.Plan, now time.Time) string {
return fmt.Sprintf("%s %s done, %d tier(s)", p.Repository, short(p.Commit), len(p.Tiers))
case inventory.PlanFailed:
return fmt.Sprintf("%s %s FAILED at %s: %s", p.Repository, short(p.Commit), where, p.Note)
case inventory.PlanSuperseded:
return fmt.Sprintf("%s %s %s", p.Repository, short(p.Commit), p.Note)
}
since := now.Sub(p.Updated).Round(time.Minute)
since := now.Sub(p.Updated).Round(time.Second)
late := ""
if since > planWaitBound {
late = " — LATE"
@@ -530,7 +842,8 @@ func planFailedBuild(ctx context.Context, open *stores, result link.BuildResult)
}
for _, e := range entries {
if repositoryMatches(e.Source.Repository, result.Repository) && e.Source.Path == result.Path {
planBuilt(ctx, open, e.Manifest.Module, result.Commit, result.Failed)
asked, _ := link.BuildAskedAt(result.ID)
planBuilt(ctx, open, e.Manifest.Module, result.Commit, result.Failed, asked)
return
}
}
@@ -637,7 +950,19 @@ func plansCommand(ctx context.Context, args []string) error {
if *whatIf != "" {
return planWhatIf(ctx, inv, *whatIf, splitList(*paths), splitList(*modules))
}
if len(positionals) == 2 && positionals[0] == "stop" {
// `stop`, or `close` (novox/hq issue 254): a person ending a plan that will not move again — one
// waiting on a report that cannot come — so it stops reading as work in progress. Marked failed
// with who ended it; what it asked still builds and registers.
if len(positionals) == 2 && (positionals[0] == "stop" || positionals[0] == "close") {
how := "stopped"
if positionals[0] == "close" {
how = "closed"
}
release, err := inv.HoldPlans(ctx, true)
if err != nil {
return err
}
defer release()
p, err := inv.PlanByID(ctx, positionals[1])
if err != nil {
return err
@@ -646,12 +971,16 @@ func plansCommand(ctx context.Context, args []string) error {
return fmt.Errorf("%s is already %s", p.ID, p.State)
}
p.State = inventory.PlanFailed
p.Note = "stopped by hand at tier " + fmt.Sprint(p.Tier)
p.Note = how + " by hand at tier " + fmt.Sprint(p.Tier)
sayUnsent(&p, func(m string) bool {
u, err := inv.UpgradeOf(ctx, m)
return err == nil && u.RollOut
})
if err := inv.SavePlan(ctx, p); err != nil {
return err
}
fmt.Printf("%s stopped at tier %d of %d; what was asked still builds and registers, nothing further is asked\n",
p.ID, p.Tier, len(p.Tiers))
fmt.Printf("%s %s at tier %d of %d; what was asked still builds and registers, nothing further is asked\n",
p.ID, how, p.Tier, len(p.Tiers))
return nil
}
plans, err := inv.RecentPlans(ctx, *limit)
@@ -726,7 +1055,13 @@ func planWhatIf(ctx context.Context, inv *inventory.Inventory, repository string
how := "built; its policy records, so nothing is sent"
if u, err := inv.UpgradeOf(ctx, name); err == nil && u.RollOut {
running, _ := inv.Running(ctx, name)
reports, _ := inv.LastReports(ctx)
how = "built, then sent to " + orNone(strings.Join(running, ", "))
// One machine first unless the policy says together (novox/hq issue 249).
if first := nextRollout(inventory.PlanModule{}, running, u.Together, reports, time.Now(), planWaitBound); first.first && len(running) > 1 {
how = fmt.Sprintf("built, then sent to %s first and to the rest once it has applied it",
first.send[0])
}
rolls[name] = how
}
fmt.Printf(" %-22s %s\n", name, how)
@@ -755,3 +1090,52 @@ func splitList(s string) []string {
}
return out
}
// settleFromRecords marks every module of the tier still `asked` built — or failed — from a build
// recorded after it was asked, and says whether it changed anything (novox/hq 04-ISSUES/214).
// Newest first, as Builds answers: the first record after the ask is the outcome of that ask.
func settleFromRecords(p *inventory.Plan, tier []string, recorded map[string][]inventory.Build) bool {
changed := false
for _, m := range tier {
s := p.Modules[m]
if s == nil || s.State != "asked" || s.AskedAt == nil {
continue
}
var outcome *inventory.Build
for i := range recorded[m] {
b := recorded[m][i]
if b.At.Before(*s.AskedAt) {
break
}
// Recorded after the ask and asked before it: an earlier ask's late outcome, not this
// one's (novox/hq 04-ISSUES/219).
if askedBefore(b.Asked, s.AskedAt) {
continue
}
outcome = &b
}
if outcome == nil {
continue
}
at := outcome.At
if outcome.Worked() {
s.State = "built"
s.BuiltAt = &at
s.Commit = outcome.Commit
} else {
s.State = "failed"
s.Why = outcome.Failed
p.State = inventory.PlanFailed
p.Note = fmt.Sprintf("%s failed to build in tier %d", m, p.Tier)
}
fmt.Printf("%s: %s settled from the build records as %s (%s)\n", p.ID, m, s.State, outcome.ID)
changed = true
}
return changed
}
// askedBefore is whether a build asked at asked was asked before a plan asked for its module — and
// so is not that plan's outcome (novox/hq 04-ISSUES/219). False when either time is not known.
func askedBefore(asked time.Time, planAsked *time.Time) bool {
return !asked.IsZero() && planAsked != nil && asked.Before(*planAsked)
}
+86
View File
@@ -126,3 +126,89 @@ func TestABundleIsPlannedAfterTheToolchainItIsCompiledIn(t *testing.T) {
t.Fatalf("the toolchain, then the bundle: %v", p.Tiers)
}
}
// novox/hq 04-ISSUES/214: a plan whose build outcome was recorded while no controller followed it —
// the controller rebuilding itself — settles from the build records instead of waiting for ever.
func TestAPlanSettlesAnAskedBuildFromTheRecords(t *testing.T) {
asked := time.Date(2026, 10, 3, 19, 20, 0, 0, time.UTC)
p := inventory.Plan{ID: "plan-1", Tiers: [][]string{{"mesh-controller", "builder"}, {"route-proxy"}},
Modules: map[string]*inventory.PlanModule{
"mesh-controller": {State: "asked", AskedAt: &asked},
"builder": {State: "asked", AskedAt: &asked},
}}
records := map[string][]inventory.Build{
// Newest first, as Builds answers: the build after the ask is the outcome.
"mesh-controller": {
{ID: "build-2", Commit: "2ebbb799", At: asked.Add(4 * time.Minute)},
{ID: "build-1", Commit: "06ea2168", At: asked.Add(-10 * time.Minute)},
},
// Only a build from before the ask: not this ask's outcome.
"builder": {{ID: "build-0", Commit: "06ea2168", At: asked.Add(-time.Hour)}},
}
if !settleFromRecords(&p, p.Tiers[0], records) {
t.Fatal("nothing settled, though the controller's build is recorded after the ask")
}
if s := p.Modules["mesh-controller"]; s.State != "built" || s.Commit != "2ebbb799" || s.BuiltAt == nil {
t.Errorf("the controller's ask is %+v, want built from 2ebbb799", s)
}
if s := p.Modules["builder"]; s.State != "asked" {
t.Errorf("an ask with no record after it was settled: %+v", s)
}
// novox/hq 04-ISSUES/219: a build recorded after the ask but asked before it — an earlier
// plan's late outcome — is not this ask's, built or failed.
r := inventory.Plan{ID: "plan-3", Tiers: [][]string{{"postgres"}},
Modules: map[string]*inventory.PlanModule{"postgres": {State: "asked", AskedAt: &asked}}}
late := map[string][]inventory.Build{"postgres": {
{ID: "build-old", Commit: "efff5415", Asked: asked.Add(-18 * time.Minute), At: asked.Add(12 * time.Minute)},
}}
if settleFromRecords(&r, r.Tiers[0], late) || r.Modules["postgres"].State != "asked" {
t.Errorf("an earlier ask's late outcome settled this ask: %+v", r.Modules["postgres"])
}
// Newest heard first: the earlier ask's late outcome, then this ask's own, heard before it.
late["postgres"] = append(late["postgres"], inventory.Build{ID: "build-mine", Commit: "4bcd5f73",
Asked: asked.Add(time.Second), At: asked.Add(5 * time.Minute)})
if !settleFromRecords(&r, r.Tiers[0], late) || r.Modules["postgres"].State != "built" ||
r.Modules["postgres"].Commit != "4bcd5f73" {
t.Errorf("this ask's own outcome, heard before the earlier ask's, did not settle it: %+v", r.Modules["postgres"])
}
// A failure recorded after the ask fails the plan, as hearing it would have.
q := inventory.Plan{ID: "plan-2", Tiers: [][]string{{"x"}},
Modules: map[string]*inventory.PlanModule{"x": {State: "asked", AskedAt: &asked}}}
settleFromRecords(&q, q.Tiers[0], map[string][]inventory.Build{"x": {{ID: "b", Failed: "no", At: asked.Add(time.Minute)}}})
if q.State != inventory.PlanFailed || q.Modules["x"].State != "failed" {
t.Errorf("a recorded failure did not fail the plan: %+v %+v", q, q.Modules["x"])
}
}
// The first machine's report, made between the send to it and the send to the rest, opens the gate for it:
// each machine is judged from its own send, not from the last one (novox/hq issue 256).
func TestTheGateJudgesEachMachineFromItsOwnSend(t *testing.T) {
built := time.Date(2026, 10, 5, 18, 22, 0, 0, time.UTC)
firstSent := built.Add(31 * time.Second)
firstReported := built.Add(43 * time.Second)
restSent := built.Add(58 * time.Second)
restReported := built.Add(74 * time.Second)
reports := []inventory.Reported{
{Node: "ace", At: &firstReported},
{Node: "g14", At: &restReported},
}
since := func(node string) time.Time {
if node == "ace" {
return firstSent
}
return restSent
}
if ok, waiting := appliedEach("build-agent", since, []string{"ace", "g14"}, reports); !ok {
t.Fatalf("the gate still waits on %v, though each reported after its own send", waiting)
}
// The old reading, every machine from the last send, is what held the plan.
if ok, _ := applied("build-agent", restSent, []string{"ace", "g14"}, reports); ok {
t.Fatal("the single-moment reading should hold the first machine back")
}
early := built.Add(10 * time.Second)
if ok, waiting := appliedEach("build-agent", since, []string{"ace"}, []inventory.Reported{{Node: "ace", At: &early}}); ok || waiting[0] != "ace" {
t.Fatal("a report from before the machine was sent opened the gate")
}
}
+152
View File
@@ -0,0 +1,152 @@
package main
import (
"reflect"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
)
// novox/hq issue 249, ADR 0218: a plan rolls a module out to one machine first and the rest only
// once that machine has reported it applied; a module whose policy says together goes everywhere at
// once, as before.
func TestAPlanSendsOneMachineFirstAndTheRestAfterItsReport(t *testing.T) {
running := []string{"novox", "ace", "g14"}
sentAt := time.Date(2026, 10, 5, 12, 0, 0, 0, time.UTC)
now := sentAt.Add(5 * time.Minute)
bound := 30 * time.Minute
after := sentAt.Add(time.Minute)
next := func(s inventory.PlanModule, running []string, together bool, reports []inventory.Reported) rolloutStep {
return nextRollout(s, running, together, reports, now, bound)
}
// Together: every machine at once.
if step := next(inventory.PlanModule{}, running, true, nil); !reflect.DeepEqual(step.send, running) || step.first {
t.Fatalf("a together policy did not send every machine at once: %+v", step)
}
// Otherwise the first by name, alone, when none has reported lately.
step := next(inventory.PlanModule{}, running, false, nil)
if !step.first || !reflect.DeepEqual(step.send, []string{"ace"}) {
t.Fatalf("the first send was %+v, wanted ace alone", step)
}
state := inventory.PlanModule{First: []string{"ace"}, FirstAt: &sentAt}
report := func(outcome string, current bool) []inventory.Reported {
return []inventory.Reported{{Node: "ace", At: &after, Outcome: outcome, Current: current},
{Node: "g14", At: &after, Outcome: inventory.OutcomeApplied, Current: true}}
}
// No report yet, or one about an older declaration than it was last sent: wait.
for what, reports := range map[string][]inventory.Reported{
"no report": nil,
"a report about older": report(inventory.OutcomeApplied, false),
} {
step := next(state, running, false, reports)
if len(step.send) != 0 || step.waiting != "ace" || step.failed != "" {
t.Errorf("%s: %+v, wanted to wait for ace", what, step)
}
}
// Applied what it was last sent: the rest, and only the rest.
step = next(state, running, false, report(inventory.OutcomeApplied, true))
if step.first || !reflect.DeepEqual(step.send, []string{"novox", "g14"}) {
t.Fatalf("after ace applied it the plan sent %+v, wanted novox and g14", step)
}
// Failed or refused: stop, the rest untouched.
for _, outcome := range []string{inventory.OutcomeFailed, inventory.OutcomeRefused} {
step := next(state, running, false, report(outcome, true))
if len(step.send) != 0 || !strings.Contains(step.failed, "ace "+outcome) ||
!reflect.DeepEqual(step.rest, []string{"novox", "g14"}) {
t.Errorf("a first machine that %s it: %+v", outcome, step)
}
}
// The machine holding the bus went with the first send: the rest wait for it too, and it is
// not sent again.
both := inventory.PlanModule{First: []string{"novox", "ace"}, FirstAt: &sentAt}
half := report(inventory.OutcomeApplied, true)
if step := next(both, running, false, half); step.waiting != "novox" {
t.Fatalf("the plan did not wait for the bus's machine sent first: %+v", step)
}
all := append(half, inventory.Reported{Node: "novox", At: &after, Outcome: inventory.OutcomeApplied, Current: true})
if step := next(both, running, false, all); !reflect.DeepEqual(step.send, []string{"g14"}) {
t.Fatalf("after both applied it the plan sent %+v, wanted g14 alone", step)
}
// One machine, or none: nothing is waited for that cannot come.
if step := next(inventory.PlanModule{}, nil, false, nil); len(step.send) != 0 || step.first {
t.Fatalf("a module nothing runs was sent: %+v", step)
}
if step := next(state, []string{"ace"}, false, report(inventory.OutcomeApplied, true)); len(step.send) != 0 || step.waiting != "" {
t.Fatalf("a module on one machine waited for more: %+v", step)
}
}
// ADR 0218 §2: a first machine that does not report within the bound stops the rollout there,
// naming the machine and the bound; the rest are left alone.
func TestAFirstMachineThatDoesNotReportStopsTheRollout(t *testing.T) {
sentAt := time.Date(2026, 10, 5, 12, 0, 0, 0, time.UTC)
state := inventory.PlanModule{First: []string{"ace"}, FirstAt: &sentAt}
step := nextRollout(state, []string{"ace", "g14"}, false, nil, sentAt.Add(31*time.Minute), 30*time.Minute)
if !strings.Contains(step.failed, "ace did not report it applied within 30m") ||
!reflect.DeepEqual(step.rest, []string{"g14"}) || len(step.send) != 0 {
t.Fatalf("a silent first machine: %+v", step)
}
}
// The first machine is the first by name among those heard from lately: a laptop that is away is
// not the one the rest wait on. When none has been heard from, the first by name.
func TestTheFirstMachineIsOneThatHasReportedLately(t *testing.T) {
now := time.Date(2026, 10, 5, 12, 0, 0, 0, time.UTC)
lately, long := now.Add(-time.Minute), now.Add(-3*time.Hour)
reports := []inventory.Reported{
{Node: "ace", At: &long}, {Node: "g14", At: &lately}, {Node: "novox", At: &lately},
}
step := nextRollout(inventory.PlanModule{}, []string{"novox", "ace", "g14"}, false, reports, now, 30*time.Minute)
if !step.first || !reflect.DeepEqual(step.send, []string{"g14"}) {
t.Fatalf("the first send was %+v, wanted g14, the first heard from lately", step)
}
}
// An announced move of a module an open plan is still rolling out is left to the plan: sending it
// here as well put the bundle on every machine at once (novox/hq issue 249).
func TestAnAnnouncedMoveIsLeftToThePlanRollingItOut(t *testing.T) {
sent := time.Now()
plans := []inventory.Plan{
{ID: "plan-done", State: inventory.PlanDone, Modules: map[string]*inventory.PlanModule{"agent": {}}},
{ID: "plan-1", State: inventory.PlanRolling, Modules: map[string]*inventory.PlanModule{
"agent": {State: "built", First: []string{"ace"}, FirstAt: &sent}, "gitea": {State: "built", SentAt: &sent}}},
}
if got := rolledOutByAPlan(plans, "agent"); got != "plan-1" {
t.Fatalf("a module the plan is rolling out was not left to it: %q", got)
}
for _, m := range []string{"gitea", "keycloak"} {
if got := rolledOutByAPlan(plans, m); got != "" {
t.Errorf("%s, which no plan will send, was left to %s", m, got)
}
}
}
// A plan that ends without sending what it built says so, with the remedy; a module whose rollout
// stopped at its first machine is not among them.
func TestAnEndedPlanSaysWhatItBuiltAndNeverSent(t *testing.T) {
at := time.Now()
p := inventory.Plan{State: inventory.PlanFailed, Note: "closed by hand at tier 1",
Modules: map[string]*inventory.PlanModule{
"agent": {State: "built"},
"stopped": {State: "built", First: []string{"ace"}, FirstAt: &at},
"sent": {State: "built", SentAt: &at},
"notes": {State: "built"},
"later": {},
}}
rollsOut := func(m string) bool { return m != "notes" }
sayUnsent(&p, rollsOut)
sayUnsent(&p, rollsOut)
if p.Note != "closed by hand at tier 1; built and never sent: agent — `push --behind` sends them" {
t.Fatalf("the note reads %q", p.Note)
}
}
@@ -0,0 +1,149 @@
package main
import (
"encoding/json"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// Defends novox/hq ADR 0207 at the controller's acts: `assign` refuses a module whose resources a
// seat nothing on the node holds applies, `unassign` refuses taking the last holder from under its
// dependents, and `status` reports what composition does not yet refuse.
func serviceManagerHolder() catalogue.Manifest {
return catalogue.Manifest{Module: "systemd", Version: "1",
Claims: []catalogue.Claim{{Name: catalogue.ServiceManagerSeat, Scope: catalogue.ScopeNode,
Serves: []string{"units", "status", "start", "stop", "restart", "enable", "disable", "journal"}}},
Resources: []map[string]any{{"id": "systemd", "type": "package", "package": "systemd"}}}
}
func packageManagerHolder() catalogue.Manifest {
return catalogue.Manifest{Module: "pacman", Version: "1",
Claims: []catalogue.Claim{{Name: catalogue.PackageManagerSeat, Scope: catalogue.ScopeNode}},
Resources: []map[string]any{{"id": "refresh", "type": "service", "unit": "pacman-refresh.timer"}}}
}
func aDaemon() catalogue.Manifest {
return catalogue.Manifest{Module: "sshd", Version: "1",
Resources: []map[string]any{{"id": "sshd", "type": "service", "unit": "sshd.service"}}}
}
func TestAnAssignmentWithoutItsHolderIsRefusedAndNotKept(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, serviceManagerHolder())
register(t, open, packageManagerHolder())
register(t, open, aDaemon())
_, err := assign(ctx, open, "laptop", "sshd")
if err == nil {
t.Fatal("sshd went onto a machine nothing holds the service manager of")
}
for _, want := range []string{catalogue.ServiceManagerSeat, "systemd", "ADR 0207"} {
if !strings.Contains(err.Error(), want) {
t.Errorf("the refusal does not say %q:\n%v", want, err)
}
}
assigned, err := open.inventory.Assigned(ctx, "laptop")
if err != nil {
t.Fatal(err)
}
if contains(assigned, "sshd") {
t.Fatalf("a refused assignment was kept: %v", assigned)
}
// The holders depend on each other, so neither goes on alone — and both go on in one act.
if _, err := assign(ctx, open, "laptop", "systemd"); err == nil {
t.Fatal("systemd went on alone though its package needs a package manager")
}
if said, err := assign(ctx, open, "laptop", "systemd", "pacman"); err != nil {
t.Fatalf("the two holders assigned together were refused: %v\n%s", err, said)
}
if said, err := assign(ctx, open, "laptop", "sshd"); err != nil {
t.Fatalf("sshd beside its holder was refused: %v\n%s", err, said)
}
}
func TestTheControllerSeatsAssignTakesSeveralModulesAsOneAct(t *testing.T) {
argv, err := argvFor("assign", map[string]any{"node": "laptop", "module": "systemd, pacman"})
if err != nil {
t.Fatal(err)
}
if strings.Join(argv, " ") != "assign laptop systemd pacman" {
t.Errorf("the seat's assign became %v", argv)
}
}
func TestUnassigningTheLastHolderUnderItsDependentsIsRefused(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, serviceManagerHolder())
register(t, open, packageManagerHolder())
register(t, open, aDaemon())
if _, err := assign(ctx, open, "laptop", "systemd", "pacman", "sshd"); err != nil {
t.Fatal(err)
}
_, err := unassign(ctx, open, "laptop", "systemd")
if err == nil {
t.Fatal("the service manager came off a machine still running services")
}
for _, want := range []string{catalogue.ServiceManagerSeat, "sshd", "pacman"} {
if !strings.Contains(err.Error(), want) {
t.Errorf("the refusal does not name %q:\n%v", want, err)
}
}
assigned, _ := open.inventory.Assigned(ctx, "laptop")
if !contains(assigned, "systemd") {
t.Fatalf("a refused unassignment took the module off anyway: %v", assigned)
}
if _, err := unassign(ctx, open, "laptop", "sshd"); err != nil {
t.Fatalf("a dependent could not come off: %v", err)
}
}
func TestStatusReportsAnUnheldDependencyWithoutRefusingTheMachine(t *testing.T) {
// The mesh as it ran before the switch (novox/hq ADR 0207 §4).
defer catalogue.EnforcingSeatDependencies(false)()
open := aMesh(t)
ctx := t.Context()
register(t, open, serviceManagerHolder())
register(t, open, aDaemon())
// Assigned straight into the store: a machine whose modules predate the rule, which is every
// machine on the day it ships.
if _, err := open.inventory.Assign(ctx, "laptop", "sshd"); err != nil {
t.Fatal(err)
}
asked, err := theThreeQuestions(ctx, open)
if err != nil {
t.Fatal(err)
}
if _, refused := asked.refused["laptop"]; refused {
t.Fatalf("an unmet dependency refused the machine before the switch: %s", asked.refused["laptop"])
}
if asked.well() {
t.Error("a mesh with an unheld dependency reads as all well")
}
got := printed(t, func() error { return printStatus(asked) })
for _, want := range []string{"unheld", "laptop", "sshd", catalogue.ServiceManagerSeat, "systemd"} {
if !strings.Contains(got, want) {
t.Errorf("status does not say %q:\n%s", want, got)
}
}
body, err := statusAsJSON(asked)
if err != nil {
t.Fatal(err)
}
var doc struct {
Unheld []catalogue.Unheld `json:"unheld"`
}
if err := json.Unmarshal(body, &doc); err != nil {
t.Fatal(err)
}
if len(doc.Unheld) != 1 || doc.Unheld[0].Module != "sshd" || doc.Unheld[0].Seat != catalogue.ServiceManagerSeat {
t.Errorf("the document's unheld is %+v", doc.Unheld)
}
}
+24 -3
View File
@@ -100,6 +100,9 @@ func argvFor(verb string, args map[string]any) ([]string, error) {
if id := str("stop"); id != "" {
return []string{"plans", "stop", id}, nil
}
if id := str("close"); id != "" {
return []string{"plans", "close", id}, nil
}
if id := str("id"); id != "" {
return []string{"plans", id}, nil
}
@@ -113,7 +116,9 @@ func argvFor(verb string, args map[string]any) ([]string, error) {
if err := need("node", "module"); err != nil {
return nil, err
}
return []string{verb, str("node"), str("module")}, nil
// Several modules comma-separated, judged as one act (novox/hq ADR 0207): the holders of
// the seats that apply resources depend on each other and go on together.
return append([]string{verb, str("node")}, splitModules(str("module"))...), nil
case "pin":
if err := need("node", "provision", "from", "module"); err != nil {
return nil, err
@@ -129,7 +134,12 @@ func argvFor(verb string, args map[string]any) ([]string, error) {
// what a person at a shell does too. A tool call that blocked for a push's whole apply would
// time out on every machine that takes a minute, and say nothing about the ones that did not.
if n := str("node"); n != "" {
return []string{"push", n, "--wait", "0"}, nil
argv := []string{"push", n, "--wait", "0"}
// A running module's data moving is sent only when said (novox/hq ADR 0217).
if m := str("move"); m != "" {
argv = append(argv, "--move", m)
}
return argv, nil
}
return []string{"push", "--behind", "--wait", "0"}, nil
case "rotate":
@@ -158,6 +168,17 @@ func argvFor(verb string, args map[string]any) ([]string, error) {
argv = []string{"settings", "clear", str("module")}
case str("values") != "":
argv = append(argv, str("values"))
// What a set removes is refused unless meant (novox/hq ADR 0217).
if str("replace") == "true" {
argv = append(argv, "--replace")
}
default:
// Neither values nor clear: the layer as it stands, which is what a caller reads before
// replacing it (novox/hq ADR 0217) — and with history, the layers it replaced.
argv = []string{"settings", "show", str("module")}
if str("history") == "true" {
argv = append(argv, "--history")
}
}
// Neither values nor clear: the command says its usage, which names both, and that is the
// answer the caller needs — the same as `rotate` given half of either shape.
@@ -196,7 +217,7 @@ func argvFor(verb string, args map[string]any) ([]string, error) {
}
// jsonVerbs are the verbs whose command speaks JSON, so the answer carries it as data as well.
var jsonVerbs = map[string]bool{"status": true, "seats": true, "plan": true}
var jsonVerbs = map[string]bool{"status": true, "seats": true, "plan": true, "collection": true}
// runVerb runs this binary with the given command line and gathers what it said.
func runVerb(ctx context.Context, argv []string) (verbAnswer, error) {
+4 -2
View File
@@ -86,9 +86,11 @@ func TestSettingsSetsOrClearsALayer(t *testing.T) {
if strings.Join(argv, " ") != "settings clear dnsmasq" {
t.Fatalf("clear for the mesh: %v", argv)
}
// Neither values nor clear reads the layer as it stands (novox/hq ADR 0217): what a caller reads
// before replacing it, where it used to fall to the command's usage.
argv, _ = argvFor("settings", map[string]any{"module": "dnsmasq"})
if strings.Join(argv, " ") != "settings set dnsmasq" {
t.Fatalf("a set with no values falls to the command's usage: %v", argv)
if strings.Join(argv, " ") != "settings show dnsmasq" {
t.Fatalf("a call with no values reads the layer: %v", argv)
}
}
+4
View File
@@ -31,6 +31,10 @@ type sendable struct {
// the same composition as its received files, and every machine's private-network address.
Received map[string]map[string][]catalogue.Contribution
Mesh []string
// BusUsers is the bus's user list this declaration carries, empty for every machine but the one
// holding the bus; not sent apart from the file it is in. Its digest is recorded once sent, so
// whether that machine must go first is read from the list alone (novox/hq issue 249).
BusUsers string
// LeftOut is every module of the machine's set left out of this declaration because a stored
// setting cannot compose with its definition (novox/hq ADR 0163, rule 6), sorted. The host
// keeps that module's held things and touches none of its containers; a machine is told
+31 -1
View File
@@ -282,6 +282,22 @@ func printStatus(asked answers) error {
fmt.Printf("\n `take <node> <module>` compares what runs against what it declares, and runs it\n\n")
}
if len(asked.unheld) > 0 {
// **Reported, and not refused yet** (novox/hq ADR 0207 §4). Each machine still resolves and
// is sent what it would be; this says which of its modules depend on a seat nothing there
// holds, until every machine has its holders and the switch makes it a refusal.
fmt.Printf("%d module dependenc(ies) on a seat nothing on the machine holds (unheld, ADR 0207):\n",
len(asked.unheld))
for _, u := range asked.unheld {
holders := "no module in the catalogue claims it yet"
if len(u.Holders) > 0 {
holders = "could be held by " + strings.Join(u.Holders, ", ")
}
fmt.Printf(" %-12s %-24s %-24s %s\n", u.Node, u.Module, u.Seat, holders)
}
fmt.Printf("\n `assign <node> <holder>` meets it; reported until every machine has its holders, then refused\n\n")
}
if adopted := adoptedNodes(nodes); len(adopted) > 0 {
// Said, because nothing forces the flip: a node left adopted is visible here rather than
// read as converged (novox/hq ADR 0100). Not a fault, so it does not break "all well".
@@ -386,6 +402,20 @@ func theThreeQuestions(ctx context.Context, open *stores) (answers, error) {
if err != nil {
return answers{}, err
}
// And which machines run a module whose resources a seat nothing there holds applies (novox/hq
// ADR 0207). Each machine resolved again rather than threaded through whoResolves, whose answer
// the private network is built from and should say nothing else; a machine that does not
// resolve is already in refused, and is passed over here.
for _, n := range out.nodes {
plan, _, err := planFor(ctx, open, n.Name)
if err != nil {
if unresolvable(err) {
continue
}
return answers{}, err
}
out.unheld = append(out.unheld, plan.Unheld...)
}
out.plans, err = inv.RecentPlans(ctx, 5)
if err != nil {
return answers{}, err
@@ -496,7 +526,7 @@ func untakenModules(ctx context.Context, inv *inventory.Inventory, nodes []inven
func (a answers) well() bool {
return len(a.wrong) == 0 && len(a.quiet) == 0 && len(a.behind) == 0 &&
len(a.waiting) == 0 && len(a.refused) == 0 && a.network == "" && len(a.untaken) == 0 &&
len(a.filtered) == 0
len(a.filtered) == 0 && len(a.unheld) == 0
}
// hostSplit is which machines report which host version, for every version more than one machine
+95
View File
@@ -0,0 +1,95 @@
package main
import (
"reflect"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/inventory"
)
// novox/hq issue 254, ADR 0218: a newer plan takes over what the older open plans of its repository
// and branch had not built, and closes them as superseded; another repository's plan, another
// branch's, and a plan made after it are left alone.
func TestANewerPlanSupersedesTheOlderOpenPlansOfItsRepository(t *testing.T) {
at := time.Date(2026, 10, 5, 12, 0, 0, 0, time.UTC)
sent := at.Add(time.Minute)
plan := func(id, repository, branch string, created time.Time, modules map[string]*inventory.PlanModule) inventory.Plan {
return inventory.Plan{ID: id, Repository: repository, Branch: branch, Commit: id + "-commit",
Created: created, State: inventory.PlanRolling, Modules: modules}
}
older := plan("plan-1", "novox/mesh-catalog", "main", at, map[string]*inventory.PlanModule{
"gitea": {State: "built", SentAt: &sent}, // done with: stays done
"keycloak": {State: "asked"}, // asked, not answered: folded
"plex": {}, // not yet asked: folded
"agent": {State: "built"}, // built, rolls out, not sent: folded
"notes": {State: "built"}, // built, records: nothing to send
})
stuck := plan("plan-0", "Novox/Mesh-Catalog", "", at.Add(-time.Hour), map[string]*inventory.PlanModule{
"runtime": {State: "asked"},
})
other := plan("plan-2", "novox/mesh-controller", "main", at, map[string]*inventory.PlanModule{"mesh-controller": {}})
release := plan("plan-3", "novox/mesh-catalog", "release", at, map[string]*inventory.PlanModule{"lemurs": {}})
later := plan("plan-5", "novox/mesh-catalog", "main", at.Add(2*time.Hour), map[string]*inventory.PlanModule{"later": {}})
done := plan("plan-6", "novox/mesh-catalog", "main", at, map[string]*inventory.PlanModule{"finished": {}})
done.State = inventory.PlanDone
newer := plan("plan-4", "novox/mesh-catalog", "main", at.Add(time.Hour), nil)
newer.Commit = "97b1b2b0c0ffee"
rollsOut := func(m string) bool { return m != "notes" }
folded, closed := supersededBy(newer, []inventory.Plan{stuck, older, other, release, later, done, newer}, rollsOut)
if want := []string{"agent", "keycloak", "plex", "runtime"}; !reflect.DeepEqual(folded, want) {
t.Fatalf("folded %v, wanted %v", folded, want)
}
var ids []string
for _, p := range closed {
ids = append(ids, p.ID)
if p.State != inventory.PlanSuperseded || p.Open() {
t.Errorf("%s was left %s", p.ID, p.State)
}
if !strings.Contains(p.Note, "plan-4") || !strings.Contains(p.Note, "97b1b2b0") {
t.Errorf("%s does not name the plan that superseded it: %q", p.ID, p.Note)
}
}
if want := []string{"plan-0", "plan-1"}; !reflect.DeepEqual(ids, want) {
t.Fatalf("superseded %v, wanted %v — another repository, another branch, a later plan and a "+
"finished one are left alone", ids, want)
}
if other.State != inventory.PlanRolling {
t.Fatal("the plan handed in was changed in place")
}
if line := planLine(closed[1], time.Now()); !strings.Contains(line, "superseded") {
t.Fatalf("a superseded plan reads %q", line)
}
}
// novox/hq issue 254: a person closes a plan that will not move again, by its id.
func TestAPersonClosesAStuckPlan(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
stuck := inventory.Plan{ID: "plan-97b1b2b", Repository: "novox/mesh-catalog", Commit: "97b1b2b",
Created: time.Now().UTC(), State: inventory.PlanRolling, Tier: 1, Tiers: [][]string{{"a"}, {"b"}},
Modules: map[string]*inventory.PlanModule{"a": {State: "built"}, "b": {}}}
if err := open.inventory.SavePlan(ctx, stuck); err != nil {
t.Fatal(err)
}
if err := plansCommand(ctx, []string{"close", stuck.ID}); err != nil {
t.Fatal(err)
}
closed, err := open.inventory.PlanByID(ctx, stuck.ID)
if err != nil {
t.Fatal(err)
}
if closed.State != inventory.PlanFailed || !strings.Contains(closed.Note, "closed by hand") {
t.Fatalf("the plan was left %s: %q", closed.State, closed.Note)
}
if err := plansCommand(ctx, []string{"close", stuck.ID}); err == nil {
t.Fatal("a plan already closed was closed again")
}
if argv, err := argvFor("plans", map[string]any{"close": stuck.ID}); err != nil ||
!reflect.DeepEqual(argv, []string{"plans", "close", stuck.ID}) {
t.Fatalf("the seat's verb does not close a plan: %v %v", argv, err)
}
}
+108
View File
@@ -0,0 +1,108 @@
package main
import (
"bytes"
"os"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// An act says what it changed about the node it acted on, and nothing about the rest of the mesh
// (novox/hq ADR 0207): after the seat dependencies shipped, every `push <node>` and `assign` printed
// every node's unmet dependencies, a hundred lines around the one about the module just assigned.
func aContainer(name string) catalogue.Manifest {
return catalogue.Manifest{Module: name, Version: "1",
Resources: []map[string]any{{"id": name, "type": "container", "image": name}}}
}
func TestAnAssignmentSaysOnlyWhatItChangedOnItsOwnNode(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, aContainer("web"))
register(t, open, aContainer("db"))
// anchor already lacks a runtime for db: true, and not this act's to say.
if _, err := open.inventory.Assign(ctx, "anchor", "db"); err != nil {
t.Fatal(err)
}
if _, err := open.inventory.Assign(ctx, "laptop", "db"); err != nil {
t.Fatal(err)
}
said, err := assign(ctx, open, "laptop", "web")
if err != nil {
t.Fatalf("%v\n%s", err, said)
}
if !strings.Contains(said, "web on laptop depends on "+catalogue.ContainerRuntimeSeat) {
t.Errorf("the assignment does not say what the module it assigned depends on:\n%s", said)
}
for _, not := range []string{"db on laptop", "db on anchor", "anchor:"} {
if strings.Contains(said, not) {
t.Errorf("the assignment says %q, which it did not change:\n%s", not, said)
}
}
}
func TestAnAssignmentThatMeetsADependencySaysSo(t *testing.T) {
shelf := map[string]catalogue.Manifest{
"web": aContainer("web"),
"docker": {Module: "docker", Claims: []catalogue.Claim{{Name: catalogue.ContainerRuntimeSeat}},
Resources: []map[string]any{{"id": "d", "type": "container", "image": "dind"}}},
}
lines := unheldChange(shelf, "laptop", []string{"web"}, []string{"web", "docker"})
if len(lines) != 1 || !strings.Contains(lines[0], "web on laptop now has "+catalogue.ContainerRuntimeSeat+" held") {
t.Errorf("meeting a dependency said %v", lines)
}
// Taking the dependent off says nothing: the dependency went with its module.
if lines := unheldChange(shelf, "laptop", []string{"web"}, nil); len(lines) != 0 {
t.Errorf("unassigning the dependent said %v", lines)
}
}
func TestAPushSaysANamedNodesDependenciesAndCountsTheRest(t *testing.T) {
unheld := map[string][]catalogue.Unheld{
"anchor": {{Node: "anchor", Module: "db", Seat: catalogue.ContainerRuntimeSeat}},
"laptop": {{Node: "laptop", Module: "web", Seat: catalogue.ContainerRuntimeSeat},
{Node: "laptop", Module: "sshd", Seat: catalogue.ServiceManagerSeat}},
}
var named bytes.Buffer
reportUnheldPushed(&named, true, []string{"laptop"}, unheld)
got := named.String()
if !strings.Contains(got, "laptop has 2 unmet") || !strings.Contains(got, "web on laptop") ||
!strings.Contains(got, "sshd on laptop") || strings.Contains(got, "anchor") {
t.Errorf("a named push said:\n%s", got)
}
var all bytes.Buffer
reportUnheldPushed(&all, false, []string{"anchor", "laptop", "quiet"}, unheld)
want := "anchor: 1 unmet seat dependenc(ies) — see `status`\nlaptop: 2 unmet seat dependenc(ies) — see `status`\n"
if all.String() != want {
t.Errorf("a push to every node said:\n%s\nwant\n%s", all.String(), want)
}
}
func TestOnlyTheServingControllerLogsEachChange(t *testing.T) {
read := func(f func()) string {
old := os.Stderr
r, w, _ := os.Pipe()
os.Stderr = w
f()
_ = w.Close()
os.Stderr = old
var b bytes.Buffer
_, _ = b.ReadFrom(r)
return b.String()
}
u := []catalogue.Unheld{{Node: "n1", Module: "web", Seat: catalogue.ContainerRuntimeSeat}}
if got := read(func() { logUnheld("n1", u) }); got != "" {
t.Errorf("a command logged:\n%s", got)
}
logUnheldChanges = true
defer func() { logUnheldChanges = false }()
if got := read(func() { logUnheld("n1", u) }); !strings.Contains(got, "web on n1") {
t.Errorf("the serving controller did not log a change:\n%s", got)
}
if got := read(func() { logUnheld("n1", u) }); got != "" {
t.Errorf("an unchanged report was logged again:\n%s", got)
}
}
+335
View File
@@ -0,0 +1,335 @@
package main
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"io"
"os"
"reflect"
"sort"
"strings"
"github.com/novox/mesh-controller/internal/inventory"
)
// No change to a machine takes effect unseen (novox/hq ADR 0217, to-be 44).
//
// Two incidents in two days had one shape: a change took effect that nobody saw before it did. A
// provisioner read an unreadable file as "nobody asks" and dropped seven databases (issue 241); one
// placement set for one module on one machine replaced its whole settings layer, and the plan then
// ran the module on an empty directory (issue 246). Here are the three guards: what a settings layer
// loses is said and refused unless meant; what a push will change can be read before it is sent; and
// a running container's data moving holds that machine's push until the operator says so. Each is
// silent when nothing is at stake, so an ordinary change goes exactly as before.
// ---- 1. what a settings layer loses ------------------------------------------------------------
// settingsChange is what replacing one layer with another adds, changes and removes, by dotted path
// to each leaf — `places.config`, not only `places` — because a layer that keeps a key and loses one
// of its entries has lost something just the same: on 2026-10-05 a node's `places` kept its name and
// lost three of its four directories.
func settingsChange(before, after map[string]any) (added, changed, removed []string) {
was, now := leaves(before, ""), leaves(after, "")
for path, v := range now {
old, had := was[path]
switch {
case !had:
added = append(added, path)
case !reflect.DeepEqual(old, v):
changed = append(changed, path)
}
}
for path := range was {
if _, kept := now[path]; !kept {
removed = append(removed, path)
}
}
sort.Strings(added)
sort.Strings(changed)
sort.Strings(removed)
return added, changed, removed
}
// leaves flattens a settings layer to its leaves by dotted path; a list is a leaf, compared whole.
func leaves(m map[string]any, prefix string) map[string]any {
out := map[string]any{}
for k, v := range m {
path := k
if prefix != "" {
path = prefix + "." + k
}
if inner, ok := v.(map[string]any); ok && len(inner) > 0 {
for p, leaf := range leaves(inner, path) {
out[p] = leaf
}
continue
}
out[path] = v
}
return out
}
// ---- 2. what a push will change -----------------------------------------------------------------
// sentResource is what is kept of one resource a machine was sent: enough to say what changed and
// whether a container's data moved, and nothing that could carry a secret — a file's content is
// kept as a digest, never as text (the record lands in the store's own backups).
type sentResource struct {
ID string `json:"id"`
Type string `json:"type"`
// Digest is of the whole resource as it was sent.
Digest string `json:"digest"`
// Fields is each field's digest, so a change can be named by field.
Fields map[string]string `json:"fields"`
// Volumes is a container's mounts as sent — paths, which are not secrets, and what a moved
// data directory is read from.
Volumes []string `json:"volumes,omitempty"`
}
// summarize is what is kept of a declaration body: its resources, in the order sent.
func summarize(body []byte) ([]sentResource, error) {
var envelope struct {
Resources []map[string]any `json:"resources"`
}
if err := json.Unmarshal(body, &envelope); err != nil {
return nil, fmt.Errorf("a declaration that is not one: %w", err)
}
out := make([]sentResource, 0, len(envelope.Resources))
for _, r := range envelope.Resources {
s := sentResource{ID: fmt.Sprint(r["id"]), Type: fmt.Sprint(r["type"]), Digest: digestValue(r),
Fields: map[string]string{}}
for k, v := range r {
s.Fields[k] = digestValue(v)
}
if s.Type == "container" {
if vols, ok := r["volumes"].([]any); ok {
for _, v := range vols {
s.Volumes = append(s.Volumes, fmt.Sprint(v))
}
}
}
out = append(out, s)
}
return out, nil
}
func digestValue(v any) string {
raw, _ := json.Marshal(v)
sum := sha256.Sum256(raw)
return hex.EncodeToString(sum[:8])
}
// changedResource is one resource sent differently, with the fields that differ.
type changedResource struct {
ID, Type string
Fields []string
}
// sentDiff is what would be sent now against what was sent last, by resource id.
type sentDiff struct {
Added, Removed []string
Changed []changedResource
}
func (d sentDiff) empty() bool {
return len(d.Added) == 0 && len(d.Removed) == 0 && len(d.Changed) == 0
}
func diffSent(before, after []sentResource) sentDiff {
was := map[string]sentResource{}
for _, r := range before {
was[r.ID] = r
}
var d sentDiff
seen := map[string]bool{}
for _, r := range after {
seen[r.ID] = true
old, had := was[r.ID]
if !had {
d.Added = append(d.Added, r.ID)
continue
}
if old.Digest == r.Digest {
continue
}
c := changedResource{ID: r.ID, Type: r.Type}
for k, v := range r.Fields {
if old.Fields[k] != v {
c.Fields = append(c.Fields, k)
}
}
for k := range old.Fields {
if _, kept := r.Fields[k]; !kept {
c.Fields = append(c.Fields, k)
}
}
sort.Strings(c.Fields)
d.Changed = append(d.Changed, c)
}
for _, r := range before {
if !seen[r.ID] {
d.Removed = append(d.Removed, r.ID)
}
}
sort.Strings(d.Added)
sort.Strings(d.Removed)
sort.Slice(d.Changed, func(a, b int) bool { return d.Changed[a].ID < d.Changed[b].ID })
return d
}
// writeDiff says a diff the way a person reads one: what is added, removed and changed, and a
// changed container's fields — a container sent differently is a container recreated.
func writeDiff(w io.Writer, node string, d sentDiff, moves []dataMove) {
if d.empty() {
fmt.Fprintf(w, "%s: nothing would change — it was last sent exactly this\n", node)
return
}
fmt.Fprintf(w, "%s would change:\n", node)
for _, id := range d.Added {
fmt.Fprintf(w, " + %s\n", id)
}
for _, id := range d.Removed {
fmt.Fprintf(w, " - %s\n", id)
}
for _, c := range d.Changed {
what := "changed"
if c.Type == "container" {
what = "recreated"
}
fmt.Fprintf(w, " ~ %s %s: %s\n", c.ID, what, strings.Join(c.Fields, ", "))
}
writeMoves(w, node, moves)
}
// ---- 3. a running module's data moving ------------------------------------------------------------
// dataMove is a container keeping a mount at the same place inside it with a different directory
// on the machine behind it: its data moving — or, as on 2026-10-05, a module about to start on an
// empty directory because the placement that pointed at its data was lost.
type dataMove struct {
Container, Inside, From, To string
}
// Module is the module the container belongs to: resource ids are `<module>.<id>`.
func (m dataMove) Module() string {
module, _, _ := strings.Cut(m.Container, ".")
return module
}
// movesIn is every data move between what a machine was sent and what it would be sent. A new
// container, a mount added or dropped and a changed image are not moves.
func movesIn(before, after []sentResource) []dataMove {
was := map[string]sentResource{}
for _, r := range before {
if r.Type == "container" {
was[r.ID] = r
}
}
var out []dataMove
for _, r := range after {
old, had := was[r.ID]
if r.Type != "container" || !had {
continue
}
from := mountSources(old.Volumes)
for inside, to := range mountSources(r.Volumes) {
if prev, mounted := from[inside]; mounted && prev != to {
out = append(out, dataMove{Container: r.ID, Inside: inside, From: prev, To: to})
}
}
}
sort.Slice(out, func(a, b int) bool {
if out[a].Container != out[b].Container {
return out[a].Container < out[b].Container
}
return out[a].Inside < out[b].Inside
})
return out
}
// mountSources is a container's mounts by the place inside it: `source:inside[:options]`.
func mountSources(volumes []string) map[string]string {
out := map[string]string{}
for _, v := range volumes {
parts := strings.SplitN(v, ":", 3)
if len(parts) < 2 {
continue
}
out[parts[1]] = parts[0]
}
return out
}
func writeMoves(w io.Writer, node string, moves []dataMove) {
for _, m := range moves {
fmt.Fprintf(w, " ! %s: %s moves from %s to %s\n", m.Container, m.Inside, m.From, m.To)
}
}
// heldMoves is the moves in a push to one machine that the operator has not said to send (`--move
// <module>`); empty when there is nothing to hold, including a machine never sent anything before.
func heldMoves(ctx context.Context, inv *inventory.Inventory, node string, body []byte, allowed map[string]bool) ([]dataMove, error) {
prev, err := inv.SentSummary(ctx, node)
if err != nil || len(prev) == 0 {
return nil, err
}
var before []sentResource
if err := json.Unmarshal(prev, &before); err != nil {
// A record this version cannot read compares with nothing: holding every push for it would
// stop the mesh on a format, which is not what is at stake.
return nil, nil
}
after, err := summarize(body)
if err != nil {
return nil, err
}
var held []dataMove
for _, m := range movesIn(before, after) {
if !allowed[m.Module()] {
held = append(held, m)
}
}
return held, nil
}
// sayHeld tells the operator why a machine was not sent, and how to send it.
func sayHeld(w io.Writer, node string, moves []dataMove) {
modules := map[string]bool{}
for _, m := range moves {
modules[m.Module()] = true
}
var names []string
for m := range modules {
names = append(names, m)
}
sort.Strings(names)
fmt.Fprintf(w, "held %s: a running module's data would move (novox/hq ADR 0217)\n", node)
writeMoves(w, node, moves)
fmt.Fprintf(w, " if that is meant: push %s --move %s\n", node, strings.Join(names, ","))
}
// writePlanDiff says what sending body to a machine would change against what it was last sent.
func writePlanDiff(ctx context.Context, inv *inventory.Inventory, node string, body []byte) error {
after, err := summarize(body)
if err != nil {
return err
}
prev, err := inv.SentSummary(ctx, node)
if err != nil {
return err
}
if len(prev) == 0 {
fmt.Printf("%s: what it was last sent is not kept yet — the first push from this version keeps "+
"it; %d resource(s) would be sent\n", node, len(after))
return nil
}
var before []sentResource
if err := json.Unmarshal(prev, &before); err != nil {
return fmt.Errorf("%s: what it was last sent is kept in a form this version does not read: %w", node, err)
}
writeDiff(os.Stdout, node, diffSent(before, after), movesIn(before, after))
return nil
}
+170
View File
@@ -0,0 +1,170 @@
package main
import (
"bytes"
"context"
"reflect"
"strings"
"testing"
)
// novox/hq ADR 0217: no change to a machine takes effect unseen.
// The layer of 2026-10-05: a media server's node layer, and the one that replaced it, which kept
// `places` and lost three of its four directories and every other key.
var before = map[string]any{
"puid": 1000.0, "pgid": 1000.0,
"expose": map[string]any{"32400": "anywhere"},
"places": map[string]any{
"config": map[string]any{"path": "/srv/media/config", "owner": "1000:1000"},
"data": map[string]any{"path": "/srv/media/data", "owner": "1000:1000"},
"transcode": map[string]any{"path": "/srv/media/temp", "owner": "1000:1000"},
},
}
func TestASettingThatKeepsAKeyAndLosesItsEntriesIsSaidToRemoveThem(t *testing.T) {
after := map[string]any{"places": map[string]any{
"previews": map[string]any{"path": "/srv/pool/previews", "owner": "1000:1000"},
}}
added, changed, removed := settingsChange(before, after)
if !reflect.DeepEqual(added, []string{"places.previews.owner", "places.previews.path"}) {
t.Errorf("added %v", added)
}
if len(changed) != 0 {
t.Errorf("changed %v", changed)
}
for _, lost := range []string{"expose.32400", "pgid", "places.config.path", "places.data.owner", "puid"} {
found := false
for _, r := range removed {
found = found || r == lost
}
if !found {
t.Errorf("removing %s was not said: %v", lost, removed)
}
}
}
func TestASettingThatOnlyAddsOrChangesRemovesNothing(t *testing.T) {
after := map[string]any{
"puid": 1001.0, "pgid": 1000.0,
"expose": map[string]any{"32400": "anywhere"},
"places": map[string]any{
"config": map[string]any{"path": "/srv/media/config", "owner": "1000:1000"},
"data": map[string]any{"path": "/srv/media/data", "owner": "1000:1000"},
"transcode": map[string]any{"path": "/srv/media/temp", "owner": "1000:1000"},
"previews": map[string]any{"path": "/srv/pool/previews", "owner": "1000:1000"},
},
}
_, changed, removed := settingsChange(before, after)
if len(removed) != 0 {
t.Errorf("an additive set was said to remove %v", removed)
}
if !reflect.DeepEqual(changed, []string{"puid"}) {
t.Errorf("changed %v", changed)
}
}
// What was sent, as a push would send it: a media server's container and a file with a secret.
func declaration(configFrom string, extra ...string) []byte {
vols := `"` + configFrom + `:/config", "/srv/media/data:/data"`
for _, e := range extra {
vols += `, "` + e + `"`
}
return []byte(`{"declaration":1,"sequence":7,"resources":[
{"id":"plex.server","type":"container","image":"plex@sha256:aa","volumes":[` + vols + `]},
{"id":"plex.env","type":"file","path":"/srv/media/env","content":"TOKEN=the-secret-itself"}]}`)
}
func summarized(t *testing.T, body []byte) []sentResource {
t.Helper()
s, err := summarize(body)
if err != nil {
t.Fatal(err)
}
return s
}
func TestWhatIsKeptOfASendHoldsNoFileContent(t *testing.T) {
s := summarized(t, declaration("/srv/media/config"))
var kept bytes.Buffer
for _, r := range s {
kept.WriteString(r.ID + r.Type + r.Digest + strings.Join(r.Volumes, ","))
for k, v := range r.Fields {
kept.WriteString(k + v)
}
}
if strings.Contains(kept.String(), "the-secret-itself") {
t.Fatal("a file's content was kept in the record of what was sent")
}
if len(s) != 2 || s[0].Volumes[0] != "/srv/media/config:/config" {
t.Fatalf("summary %+v", s)
}
}
func TestADiffOfAnUnchangedMachineIsEmptyAndAChangedContainerNamesItsField(t *testing.T) {
was := summarized(t, declaration("/srv/media/config"))
if d := diffSent(was, summarized(t, declaration("/srv/media/config"))); !d.empty() {
t.Fatalf("an unchanged machine differs: %+v", d)
}
d := diffSent(was, summarized(t, declaration("/var/lib/plex/config")))
if len(d.Changed) != 1 || d.Changed[0].ID != "plex.server" || !reflect.DeepEqual(d.Changed[0].Fields, []string{"volumes"}) {
t.Fatalf("diff %+v", d)
}
var out bytes.Buffer
writeDiff(&out, "home", d, movesIn(was, summarized(t, declaration("/var/lib/plex/config"))))
for _, want := range []string{"~ plex.server recreated: volumes", "! plex.server: /config moves from /srv/media/config to /var/lib/plex/config"} {
if !strings.Contains(out.String(), want) {
t.Errorf("diff does not say %q:\n%s", want, out.String())
}
}
}
// The incident: the configuration mount would move to an empty default directory.
func TestARunningContainersDataMovingIsAMoveAndAnAddedMountIsNot(t *testing.T) {
was := summarized(t, declaration("/srv/media/config"))
moves := movesIn(was, summarized(t, declaration("/var/lib/plex/config")))
want := []dataMove{{Container: "plex.server", Inside: "/config", From: "/srv/media/config", To: "/var/lib/plex/config"}}
if !reflect.DeepEqual(moves, want) {
t.Fatalf("moves %+v", moves)
}
if moves[0].Module() != "plex" {
t.Errorf("module %q", moves[0].Module())
}
// A mount added — the previews of 2026-10-05 — is not a move.
if m := movesIn(was, summarized(t, declaration("/srv/media/config", "/srv/pool/previews:/config/Media"))); len(m) != 0 {
t.Errorf("an added mount was held as a move: %+v", m)
}
// Nor is a container the machine never ran.
if m := movesIn(nil, was); len(m) != 0 {
t.Errorf("a first send was held as a move: %+v", m)
}
}
func TestAPushNamingNoMachineIsRefusedUnlessItSaysAll(t *testing.T) {
err := pushCommand(context.Background(), nil)
if err == nil || !strings.Contains(err.Error(), "--all") {
t.Fatalf("a bare push was not refused: %v", err)
}
if err := pushCommand(context.Background(), []string{"--all", "--behind"}); err == nil {
t.Fatal("--all with --behind was accepted")
}
}
func TestTheVerbsCarryReadReplaceAndMove(t *testing.T) {
for _, c := range []struct {
verb string
args map[string]any
want []string
}{
{"settings", map[string]any{"module": "plex", "node": "home"}, []string{"settings", "show", "plex", "--node", "home"}},
{"settings", map[string]any{"module": "plex", "history": "true"}, []string{"settings", "show", "plex", "--history"}},
{"settings", map[string]any{"module": "plex", "values": "{}", "replace": "true"}, []string{"settings", "set", "plex", "{}", "--replace"}},
{"push", map[string]any{"node": "home", "move": "plex"}, []string{"push", "home", "--wait", "0", "--move", "plex"}},
{"push", map[string]any{}, []string{"push", "--behind", "--wait", "0"}},
} {
got, err := argvFor(c.verb, c.args)
if err != nil || !reflect.DeepEqual(got, c.want) {
t.Errorf("%s %v: %v %v, want %v", c.verb, c.args, got, err, c.want)
}
}
}
+171 -11
View File
@@ -5,6 +5,8 @@ import (
"errors"
"flag"
"fmt"
"path"
"regexp"
"strings"
"time"
@@ -54,10 +56,26 @@ func (f following) Upgraded(ctx context.Context, u link.Upgraded) error {
return nil
}
// **A plan that holds the module rolls it out, and this does not** (novox/hq issue 249, ADR
// 0218). A merge's plan builds the module and sends it one machine first, the rest once that one
// has applied it; this announcement arrives as the build registers, and sending here too — one
// machine after another without waiting for any to apply — put the new bundle on every machine in
// the same minute, whatever the plan was waiting for. A move no plan answers (a build asked by
// hand) is still this handler's.
if plans, err := inv.OpenPlans(ctx); err != nil {
return notNow(err)
} else if id := rolledOutByAPlan(plans, u.Module); id != "" {
// Said with its remedy: a plan that ends without sending it — failed, or closed by hand — leaves
// these machines behind, which `status` lists and `push --behind` sends (novox/hq issue 249).
fmt.Printf("%s moved to %s; %s rolls it out to %s — if that plan ends without sending it, "+
"`status` lists them as behind and `push --behind` sends it\n",
u.Module, shortCommit(u.Commit), id, readableList(on))
return nil
}
if decision.Together {
fmt.Printf("%s moved to %s; sending %s together\n",
u.Module, shortCommit(u.Commit), readableList(on))
return sendTo(ctx, f.open, on)
return askAgainOnGrants(sendTo(ctx, f.open, on))
}
// One at a time, and stopping at the first that fails.
//
@@ -68,13 +86,34 @@ func (f following) Upgraded(ctx context.Context, u link.Upgraded) error {
u.Module, shortCommit(u.Commit), readableList(on))
for _, node := range on {
if err := sendTo(ctx, f.open, []string{node}); err != nil {
return fmt.Errorf("%s did not take %s, so the machines after it were left alone: %w",
node, u.Module, err)
return askAgainOnGrants(fmt.Errorf("%s did not take %s, so the machines after it were left alone: %w",
node, u.Module, err))
}
}
return nil
}
// askAgainOnGrants marks a send that stopped at its grants as one to ask again (novox/hq issue 249):
// an announcement handled by a send whose memberships could not be issued is held and redelivered,
// rather than taken as handled with the machines left on the old version.
func askAgainOnGrants(err error) error {
if err != nil && errors.Is(err, errGrants) && !errors.Is(err, link.ErrTryAgain) {
return fmt.Errorf("%w: %w", link.ErrTryAgain, err)
}
return err
}
// rolledOutByAPlan is the open plan that will send a module's machines its new build — one holding
// the module that has not finished sending it — or empty when none will (novox/hq issue 249).
func rolledOutByAPlan(plans []inventory.Plan, module string) string {
for _, p := range plans {
if s, holds := p.Modules[module]; p.Open() && holds && (s == nil || (s.SentAt == nil && s.State != "failed")) {
return p.ID
}
}
return ""
}
// readableList names machines the way a sentence does, because this is read by a person deciding
// whether an upgrade went where they expected.
func readableList(names []string) string {
@@ -283,6 +322,14 @@ func (f following) SourceMoved(ctx context.Context, m link.SourceMoved) error {
if isHistory(m.MergedAt, lastLookAt(entries, m)) {
packaging = nil
}
// Said, never silent (novox/hq 04-ISSUES/215): a module built from this repository that follows
// another branch is not part of this merge, and whoever is waiting for its change should read why.
for _, e := range entries {
if sameRepository(e.Source.Repository, m) && !sourceIs(e.Source, m) {
fmt.Printf(" %s is built from %s/%s and follows %s, not %s; this merge leaves it out\n",
e.Manifest.Module, m.Owner, m.Repo, e.Source.Ref, m.Base)
}
}
touched := whatTheMergeTouched(from, entries, m)
for _, e := range touched {
if err := inv.SourceMoved(ctx, e.Manifest.Module, m.Commit); err != nil {
@@ -306,7 +353,53 @@ func (f following) SourceMoved(ctx context.Context, m link.SourceMoved) error {
for _, e := range moved {
movedNames = append(movedNames, e.Manifest.Module)
}
// Written and its first tier asked as one act on the plans (novox/hq issue 213): a timer on
// another controller reading it between the two would ask the tier again.
release, err := inv.HoldPlans(ctx, true)
if err != nil {
return notNow(err)
}
defer release()
plan := planOfMerge(m, movedNames, edges)
// **A newer plan supersedes the older open plans of this repository and branch** (novox/hq issue
// 254, ADR 0218): what they had not built is planned here again, and they are closed, so one plan
// works a repository's modules at a time and a stuck one ends at the next merge.
working, err := inv.OpenPlans(ctx)
if err != nil {
return notNow(err)
}
rollsOut := func(module string) bool {
u, err := inv.UpgradeOf(ctx, module)
return err == nil && u.RollOut
}
folded, superseded := supersededBy(plan, working, rollsOut)
if len(folded) > 0 {
held := map[string]bool{}
for _, e := range entries {
held[e.Manifest.Module] = true
}
names := map[string]bool{}
for _, name := range movedNames {
names[name] = true
}
var also []string
for _, name := range folded {
// One the catalogue no longer holds would fail the newer plan's ask; it is not this
// merge's to build.
if held[name] && !names[name] {
names[name] = true
movedNames = append(movedNames, name)
also = append(also, name)
}
}
if len(also) > 0 {
again := planOfMerge(m, movedNames, edges)
again.ID, again.Created = plan.ID, plan.Created
plan = again
fmt.Printf(" %s, left unbuilt by an older plan of %s, are planned here again\n",
strings.Join(also, ", "), plan.Repository)
}
}
if hasCycle(plan.Tiers, edges) {
fmt.Printf(" the last tier depends on itself: %s — built together, in no order\n",
strings.Join(plan.Tiers[len(plan.Tiers)-1], ", "))
@@ -314,6 +407,14 @@ func (f following) SourceMoved(ctx context.Context, m link.SourceMoved) error {
if err := inv.SavePlan(ctx, plan); err != nil {
return notNow(err)
}
// Closed after the newer plan is kept, never before: a controller replaced between the two leaves
// both open, which the next merge settles, rather than neither.
for _, old := range superseded {
if err := inv.SavePlan(ctx, old); err != nil {
return notNow(err)
}
fmt.Printf(" %s (%s at %s) is %s\n", old.ID, old.Repository, short(old.Commit), old.Note)
}
var tiers []string
for i, t := range plan.Tiers {
tiers = append(tiers, fmt.Sprintf("%d: %s", i, strings.Join(t, ", ")))
@@ -345,7 +446,24 @@ func sourceIs(s inventory.Source, m link.SourceMoved) bool {
if !sameRepository(s.Repository, m) {
return false
}
return s.Ref == "" || s.Ref == m.Base
ref := followedBranch(s.Ref)
return ref == "" || ref == m.Base
}
// commitRef is a ref that names a commit rather than a branch: what `build --ref <commit>` asks for.
var commitRef = regexp.MustCompile(`^[0-9a-f]{7,40}$`)
// followedBranch is the branch a recorded ref means a module follows (novox/hq 04-ISSUES/215). **A
// commit is never a branch to follow.** A build asked at a commit — to try one, or to pin it during a
// fix — recorded that commit as the module's ref; every merge after it then failed to match the
// module, its plan left it out without saying so, and every plan that rebuilt it asked for that same
// old commit again. A commit recorded so is read as the repository's default branch, which is what
// the module followed before it; a branch is followed as named.
func followedBranch(ref string) string {
if commitRef.MatchString(strings.TrimSpace(ref)) {
return ""
}
return ref
}
// sameRepository is whether a recorded repository is the one a merge names, in either spelling it
@@ -390,25 +508,45 @@ func lastLookAt(entries []inventory.Entry, m link.SourceMoved) time.Time {
//
// A change inside *another* module's directory is that module's business and not this one's, even
// when the mesh does not hold that module: `known` is every module this repository is known to hold,
// whatever branch it was registered from. That is also the limit of this — a repository whose shared
// code sits inside a directory the mesh has never seen a module in reads as shared, and everything
// is rebuilt. Rebuilding too much is the safe direction: the fault this whole path exists for is a
// mesh that believes it is current and is not (novox/hq 04-ISSUES/131).
// whatever branch it was registered from.
//
// **And a module the mesh has never seen is still a module** (novox/hq issue 252). A merge adding a
// new module to the catalogue repository — `modules/newmod/module.json` and its files — read as a
// change to shared code, because `modules/newmod` was nobody's known directory, and every module
// built from the repository was rebuilt and rolled out for a module none of them is. So the
// directories that hold modules are known too: the parents of the known modules' directories
// (`modules`, never the root). A changed path `<parent>/<name>/…` belongs to the module at
// `<parent>/<name>` — held or not — and rebuilds nothing else, **provided it is shown to be a
// module**: its `module.json` is among the changed files (added, changed, or removed with it). A
// directory under the same parent whose manifest the merge did not touch may as well be a shared
// library (`modules/lib`), and that is still read as shared. Rebuilding too much remains the safe
// direction: the fault this whole path exists for is a mesh that believes it is current and is not
// (novox/hq 04-ISSUES/131). A file at the root, or directly in a parent, is shared as it always was.
func whatTheMergeTouched(candidates, known []inventory.Entry, m link.SourceMoved) []inventory.Entry {
// Nothing said about the files, or not all of them said: everything built from it is affected.
if len(m.Paths) == 0 || m.PathsTruncated {
return candidates
}
var dirs []string
parents := map[string]bool{}
for _, e := range known {
if e.Source.Path != "" && sameRepository(e.Source.Repository, m) {
dirs = append(dirs, e.Source.Path)
dir := strings.Trim(e.Source.Path, "/")
dirs = append(dirs, dir)
if parent := path.Dir(dir); parent != "." && parent != "/" {
parents[parent] = true
}
}
}
changed := map[string]bool{}
for _, p := range m.Paths {
if !insideAny(p, dirs) {
return candidates
changed[strings.TrimPrefix(p, "/")] = true
}
for _, p := range m.Paths {
if insideAny(p, dirs) || inAModuleOfItsOwn(p, parents, changed) {
continue
}
return candidates
}
var out []inventory.Entry
for _, e := range candidates {
@@ -419,6 +557,28 @@ func whatTheMergeTouched(candidates, known []inventory.Entry, m link.SourceMoved
return out
}
// inAModuleOfItsOwn is whether a changed file is inside a module directory the mesh does not know —
// `<parent>/<name>/…` under a directory known to hold modules, whose `module.json` the same merge
// changed (novox/hq issue 252). Such a file is that module's business and nobody else's.
func inAModuleOfItsOwn(p string, parents, changed map[string]bool) bool {
p = strings.TrimPrefix(p, "/")
for parent := range parents {
rest, under := strings.CutPrefix(p, parent+"/")
if !under {
continue
}
name, _, inADirectory := strings.Cut(rest, "/")
if !inADirectory || name == "" {
// A file directly in the parent — `modules/README.md` — is about all of them.
continue
}
if changed[parent+"/"+name+"/module.json"] {
return true
}
}
return false
}
// inside is whether a changed file is in a directory: that directory itself, or under it.
func inside(path, dir string) bool {
dir = strings.Trim(dir, "/")
+345
View File
@@ -0,0 +1,345 @@
package artifacts
import (
"bytes"
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"io"
"net/http"
"strconv"
"strings"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
)
// Every archive the store keeps is held by a manifest (novox/hq issue 253, ADR 0189).
//
// **The store's collector marks only from manifests.** The mesh's store is a stock registry, and
// its nightly `registry garbage-collect` walks every manifest in every repository, marks the blobs
// those manifests name, and deletes every blob it did not mark. An image is a manifest, so what
// the mesh keeps of an image survives. An archive was not: the builder put it in the store as a
// bare blob — upload, then `PUT ?digest=` — and nothing in the store names it. To the collector a
// bare blob is unreferenced, so the first real collection would have deleted every archive the
// mesh holds, kept or not, and every machine pinning a bundle would have found it gone. The
// collector runs `--dry-run` until this is true.
//
// **So each archive gets a holder**: the smallest OCI image manifest that names it — the empty
// config, one layer, nothing else — put in the archive's own repository, by digest, untagged. The
// collector marks it and so keeps the archive; the sweep lets go of an archive by deleting its
// holder first, which is what lets the bytes go at the next collection.
//
// **Nothing a machine reads changes.** The recorded reference stays
// `artifact-store://<module>/<artifact>/blobs/sha256:…`, and machines fetch the blob exactly as
// before. The holder is the store's bookkeeping, not a second way to reach anything.
//
// **Deterministic, so it never needs recording.** The holder is composed from the archive's digest
// and size alone, in a fixed field order with no timestamps or annotations, so the sweep can
// compute which manifest holds any archive from the reference it already has plus one HEAD for the
// size. No schema change, no second record that could disagree with the store.
const (
// mediaManifest is the type a holder is put and asked for as.
mediaManifest = "application/vnd.oci.image.manifest.v1+json"
// mediaEmpty is the OCI empty descriptor's type: a config that says nothing, for a manifest
// whose only purpose is to name its layer.
mediaEmpty = "application/vnd.oci.empty.v1+json"
// emptyDigest is the digest of `{}`, the empty config's content, fixed by the OCI spec.
emptyDigest = "sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a"
// mediaArchive is the layer type an archive is held as. Every archive the builder publishes is
// `pack`'s gzipped tar, so this is the true type and not a placeholder — and it is a constant,
// not read from anywhere, because the holder must be recomputable from the reference alone.
mediaArchive = "application/vnd.oci.image.layer.v1.tar+gzip"
)
// emptyConfig is the content emptyDigest names.
var emptyConfig = []byte("{}")
// manifestAccept is what a manifest is asked for as. A registry answers a manifest HEAD only in a
// type the caller named, and answers 404 to a bare one for a manifest it holds perfectly well
// (measured 2026-09-28; internal/builder/registry.go says how that was found).
var manifestAccept = []string{
mediaManifest,
"application/vnd.docker.distribution.manifest.v2+json",
}
type descriptor struct {
MediaType string `json:"mediaType"`
Digest string `json:"digest"`
Size int64 `json:"size"`
}
type holderManifest struct {
SchemaVersion int `json:"schemaVersion"`
MediaType string `json:"mediaType"`
Config descriptor `json:"config"`
Layers []descriptor `json:"layers"`
}
// Holder is the manifest that holds an archive in the store, and its digest.
//
// A pure function of the archive's digest and size: the same two in give the same bytes out,
// always, because `encoding/json` writes a struct's fields in their declared order and there is
// nothing here that varies by when or where it was composed.
func Holder(digest string, size int64) (body []byte, holder string) {
body, err := json.Marshal(holderManifest{
SchemaVersion: 2,
MediaType: mediaManifest,
Config: descriptor{MediaType: mediaEmpty, Digest: emptyDigest, Size: int64(len(emptyConfig))},
Layers: []descriptor{{MediaType: mediaArchive, Digest: digest, Size: size}},
})
if err != nil {
// Marshalling a struct of strings and integers cannot fail.
panic(err)
}
sum := sha256.Sum256(body)
return body, "sha256:" + hex.EncodeToString(sum[:])
}
// Hold makes sure the store holds this archive by a manifest, and says whether it had to write one.
//
// Takes a reference as the mesh records it. An image is its own manifest and needs no holder, so
// it answers false and nothing is asked. Idempotent: a holder already there is left alone, which
// is what lets the sweep run it over every kept archive on every build and so backfill the bare
// blobs published before holders existed (novox/hq issue 253).
//
// Gone when the store does not hold the archive at all: there is nothing to hold, and that is a
// fact the caller reports rather than one this invents a remedy for.
func (s Store) Hold(ctx context.Context, reference string) (bool, error) {
repository, digest, archive, err := s.archive(reference)
if err != nil || !archive {
return false, err
}
size, err := s.blobSize(ctx, repository, digest)
if err != nil {
return false, err
}
return s.HoldBlob(ctx, repository, digest, size)
}
// Held is whether the store holds this archive by its manifest. Asks and changes nothing — the
// question an operator needs answered with "none unheld" before the collector is let loose.
//
// An image answers true: it is its own manifest. An archive the store does not have answers Gone.
func (s Store) Held(ctx context.Context, reference string) (bool, error) {
repository, digest, archive, err := s.archive(reference)
if err != nil {
return false, err
}
if !archive {
return true, nil
}
size, err := s.blobSize(ctx, repository, digest)
if err != nil {
return false, err
}
_, holder := Holder(digest, size)
return s.has(ctx, s.url(repository, "manifests", holder), manifestAccept...)
}
// HoldBlob puts the holder for a blob of this digest and size into its repository, unless it is
// there already. Answers whether it wrote one.
//
// The builder calls this with the size it has just uploaded; the sweep, through Hold, with the size
// the store reports. Both arrive at the same holder, which is the point of composing it.
func (s Store) HoldBlob(ctx context.Context, repository, digest string, size int64) (bool, error) {
if s.Address == "" {
return false, fmt.Errorf("this mesh has no artifact store on its network to hold %s/%s in", repository, digest)
}
body, holder := Holder(digest, size)
there, err := s.has(ctx, s.url(repository, "manifests", holder), manifestAccept...)
if err != nil {
return false, err
}
if there {
return false, nil
}
// The config must be in the repository before a manifest naming it is accepted: a registry
// refuses a manifest whose blobs it cannot find there, which is the property that makes a
// holder mean something.
if err := s.putBlob(ctx, repository, emptyDigest, emptyConfig); err != nil {
return false, err
}
// **By digest, never by tag.** A tag would be one more name to move and one more thing the
// collector's `--delete-untagged` would read as meaningful; the mesh names nothing by tag that
// it pins by digest, and an untagged manifest is kept by plain collection.
request, err := http.NewRequestWithContext(ctx, http.MethodPut,
s.url(repository, "manifests", holder), bytes.NewReader(body))
if err != nil {
return false, err
}
request.Header.Set("Content-Type", mediaManifest)
response, err := s.client().Do(request)
if err != nil {
return false, err
}
defer response.Body.Close()
if response.StatusCode != http.StatusCreated {
said, _ := io.ReadAll(io.LimitReader(response.Body, 4096))
return false, fmt.Errorf("the artifact store refused to hold %s/%s: %s %s",
repository, digest, response.Status, strings.TrimSpace(string(said)))
}
return true, nil
}
// letGoOfHolder deletes the manifest holding an archive, before the archive's own link goes.
//
// **Holder first.** Deleting the blob link alone leaves a manifest still naming the blob, and the
// collector would keep its bytes for ever on the strength of it — the sweep would record the
// archive collected while the disk said otherwise. Deleting the holder first and failing before
// the link goes leaves an unheld archive that the next sweep still offers, which is safe.
//
// A store that no longer has the blob answers Gone: without its size the holder cannot be named,
// and without the blob there is nothing left for a holder to keep. A store that never had a holder
// for it — an archive published before holders, never backfilled — answers 404 to the delete, and
// that is the outcome wanted.
func (s Store) letGoOfHolder(ctx context.Context, repository, digest string) error {
size, err := s.blobSize(ctx, repository, digest)
if err != nil {
return err
}
_, holder := Holder(digest, size)
err = s.remove(ctx, s.url(repository, "manifests", holder), repository+"/manifests/"+holder)
if err == Gone {
return nil
}
return err
}
// archive reads a recorded reference into its repository and digest, and whether it is an archive
// at all. Refuses as ErrNotOurs anything the mesh did not put in its own store.
func (s Store) archive(reference string) (repository, digest string, archive bool, err error) {
path, kept := catalogue.InArtifactStore(reference)
if !kept {
return "", "", false, fmt.Errorf("%w: %s", ErrNotOurs, reference)
}
if s.Address == "" {
return "", "", false, fmt.Errorf("this mesh has no artifact store on its network to ask about %s", reference)
}
repository, kind, digest, err := split(path)
if err != nil {
return "", "", false, err
}
return repository, digest, kind == "blobs", nil
}
// blobSize is how large the store says a blob is; Gone when it does not have it.
func (s Store) blobSize(ctx context.Context, repository, digest string) (int64, error) {
request, err := http.NewRequestWithContext(ctx, http.MethodHead, s.url(repository, "blobs", digest), nil)
if err != nil {
return 0, err
}
response, err := s.client().Do(request)
if err != nil {
return 0, fmt.Errorf("cannot reach the artifact store at %s: %w", s.Address, err)
}
defer response.Body.Close()
switch response.StatusCode {
case http.StatusOK:
case http.StatusNotFound:
return 0, Gone
default:
return 0, fmt.Errorf("the artifact store answered %s for %s/blobs/%s", response.Status, repository, digest)
}
// Read from the header rather than ContentLength: a HEAD's ContentLength is what the response
// says it would have sent, which Go reports faithfully, but a proxy in between is free to drop
// it, and the header is what the registry itself wrote.
if length := response.Header.Get("Content-Length"); length != "" {
if n, err := strconv.ParseInt(length, 10, 64); err == nil && n >= 0 {
return n, nil
}
}
if response.ContentLength >= 0 {
return response.ContentLength, nil
}
return 0, fmt.Errorf("the artifact store holds %s/blobs/%s and will not say how large it is", repository, digest)
}
// putBlob uploads a small blob unless the repository already has it: ask where, then put it there
// naming the digest — the registry's own two steps, the same the builder takes for an archive.
func (s Store) putBlob(ctx context.Context, repository, digest string, body []byte) error {
if there, err := s.has(ctx, s.url(repository, "blobs", digest)); err != nil {
return err
} else if there {
return nil
}
start, err := http.NewRequestWithContext(ctx, http.MethodPost,
"http://"+s.Address+"/v2/"+repository+"/blobs/uploads/", nil)
if err != nil {
return err
}
begun, err := s.client().Do(start)
if err != nil {
return fmt.Errorf("cannot start an upload to %s: %w", repository, err)
}
begun.Body.Close()
if begun.StatusCode != http.StatusAccepted {
return fmt.Errorf("the artifact store answered %s when asked where to put a blob in %s", begun.Status, repository)
}
where := begun.Header.Get("Location")
if where == "" {
return fmt.Errorf("the artifact store accepted an upload to %s and said nowhere to put it", repository)
}
if strings.HasPrefix(where, "/") {
where = "http://" + s.Address + where
}
separator := "?"
if strings.Contains(where, "?") {
separator = "&"
}
put, err := http.NewRequestWithContext(ctx, http.MethodPut, where+separator+"digest="+digest, bytes.NewReader(body))
if err != nil {
return err
}
put.Header.Set("Content-Type", "application/octet-stream")
done, err := s.client().Do(put)
if err != nil {
return err
}
defer done.Body.Close()
if done.StatusCode != http.StatusCreated {
said, _ := io.ReadAll(io.LimitReader(done.Body, 4096))
return fmt.Errorf("the artifact store refused a blob in %s: %s %s", repository, done.Status, strings.TrimSpace(string(said)))
}
return nil
}
// has is whether the store answers 200 for a HEAD at that URL.
func (s Store) has(ctx context.Context, url string, accept ...string) (bool, error) {
request, err := http.NewRequestWithContext(ctx, http.MethodHead, url, nil)
if err != nil {
return false, err
}
for _, media := range accept {
request.Header.Add("Accept", media)
}
response, err := s.client().Do(request)
if err != nil {
return false, fmt.Errorf("cannot reach the artifact store at %s: %w", s.Address, err)
}
defer response.Body.Close()
switch response.StatusCode {
case http.StatusOK:
return true, nil
case http.StatusNotFound:
return false, nil
default:
return false, fmt.Errorf("the artifact store answered %s for %s", response.Status, url)
}
}
func (s Store) url(repository, kind, digest string) string {
return "http://" + s.Address + "/v2/" + repository + "/" + kind + "/" + digest
}
func (s Store) client() *http.Client {
if s.HTTP != nil {
return s.HTTP
}
return &http.Client{Timeout: 30 * time.Second}
}
+268
View File
@@ -0,0 +1,268 @@
package artifacts
import (
"bytes"
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/http/httptest"
"strconv"
"strings"
"sync"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// Every kept archive is held by a manifest (novox/hq issue 253, ADR 0189).
//
// Against an in-memory registry that keeps blobs and manifests per repository and refuses what a
// registry refuses — a blob whose digest does not match, a manifest whose digest does not match
// or whose blobs the repository does not have, a manifest asked for without an Accept naming its
// type. What is asserted is this side's decisions; the live test below asserts the registry's.
type memRegistry struct {
mu sync.Mutex
blobs map[string][]byte // repository + "@" + digest
manifests map[string][]byte // repository + "@" + digest
writes []string // every PUT and DELETE, as "METHOD path"
}
func digestOf(body []byte) string {
sum := sha256.Sum256(body)
return "sha256:" + hex.EncodeToString(sum[:])
}
func (m *memRegistry) serve(t *testing.T) Store {
t.Helper()
m.blobs = map[string][]byte{}
m.manifests = map[string][]byte{}
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
m.mu.Lock()
defer m.mu.Unlock()
path := strings.TrimPrefix(r.URL.Path, "/v2/")
if r.Method == http.MethodPut || r.Method == http.MethodDelete {
m.writes = append(m.writes, r.Method+" "+r.URL.Path)
}
switch {
case r.Method == http.MethodPost && strings.HasSuffix(path, "/blobs/uploads/"):
repository := strings.TrimSuffix(path, "/blobs/uploads/")
w.Header().Set("Location", "/upload/"+repository+"?state=x")
w.WriteHeader(http.StatusAccepted)
case r.Method == http.MethodPut && strings.HasPrefix(r.URL.Path, "/upload/"):
repository := strings.TrimPrefix(r.URL.Path, "/upload/")
body, _ := io.ReadAll(r.Body)
digest := r.URL.Query().Get("digest")
if digest != digestOf(body) {
w.WriteHeader(http.StatusBadRequest)
return
}
m.blobs[repository+"@"+digest] = body
w.WriteHeader(http.StatusCreated)
case strings.Contains(path, "/blobs/"):
repository, digest, _ := strings.Cut(path, "/blobs/")
key := repository + "@" + digest
body, ok := m.blobs[key]
if !ok {
w.WriteHeader(http.StatusNotFound)
return
}
switch r.Method {
case http.MethodHead:
w.Header().Set("Content-Length", strconv.Itoa(len(body)))
w.WriteHeader(http.StatusOK)
case http.MethodDelete:
delete(m.blobs, key)
w.WriteHeader(http.StatusAccepted)
default:
w.WriteHeader(http.StatusMethodNotAllowed)
}
case strings.Contains(path, "/manifests/"):
repository, digest, _ := strings.Cut(path, "/manifests/")
key := repository + "@" + digest
switch r.Method {
case http.MethodHead:
if _, ok := m.manifests[key]; !ok || !strings.Contains(r.Header.Get("Accept"), mediaManifest) {
w.WriteHeader(http.StatusNotFound)
return
}
w.WriteHeader(http.StatusOK)
case http.MethodPut:
body, _ := io.ReadAll(r.Body)
if digest != digestOf(body) {
w.WriteHeader(http.StatusBadRequest)
return
}
var named holderManifest
if err := json.Unmarshal(body, &named); err != nil {
w.WriteHeader(http.StatusBadRequest)
return
}
for _, d := range append([]descriptor{named.Config}, named.Layers...) {
if _, ok := m.blobs[repository+"@"+d.Digest]; !ok {
w.WriteHeader(http.StatusBadRequest)
fmt.Fprintf(w, "MANIFEST_BLOB_UNKNOWN %s", d.Digest)
return
}
}
m.manifests[key] = body
w.WriteHeader(http.StatusCreated)
case http.MethodDelete:
if _, ok := m.manifests[key]; !ok {
w.WriteHeader(http.StatusNotFound)
return
}
delete(m.manifests, key)
w.WriteHeader(http.StatusAccepted)
}
default:
w.WriteHeader(http.StatusNotFound)
}
}))
t.Cleanup(server.Close)
return Store{Address: strings.TrimPrefix(server.URL, "http://")}
}
// bare puts an archive in the store the way the builder did before holders: a blob, nothing more.
func (m *memRegistry) bare(repository string, body []byte) string {
m.mu.Lock()
defer m.mu.Unlock()
digest := digestOf(body)
m.blobs[repository+"@"+digest] = body
return catalogue.ArtifactStoreScheme + repository + "/blobs/" + digest
}
func TestTheHolderIsComposedFromTheDigestAndSizeAlone(t *testing.T) {
// The sweep must arrive at the very manifest the builder wrote, with nothing recorded between
// them. Same inputs, same bytes — and a different size is a different holder, so a holder can
// never be mistaken for one of a different blob.
digest := "sha256:" + strings.Repeat("a", 64)
one, first := Holder(digest, 42)
two, second := Holder(digest, 42)
if !bytes.Equal(one, two) || first != second {
t.Fatalf("the same archive composed two holders:\n%s\n%s", one, two)
}
if _, other := Holder(digest, 43); other == first {
t.Fatal("a different size composed the same holder")
}
want := `{"schemaVersion":2,"mediaType":"application/vnd.oci.image.manifest.v1+json",` +
`"config":{"mediaType":"application/vnd.oci.empty.v1+json",` +
`"digest":"sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a","size":2},` +
`"layers":[{"mediaType":"application/vnd.oci.image.layer.v1.tar+gzip","digest":"` + digest + `","size":42}]}`
if string(one) != want {
t.Fatalf("the holder is\n%s\nwant\n%s", one, want)
}
if digestOf(emptyConfig) != emptyDigest {
t.Fatalf("the empty config's digest is %s, not %s", digestOf(emptyConfig), emptyDigest)
}
}
func TestHoldBackfillsABareArchiveAndIsIdempotent(t *testing.T) {
// The archives published before this have no holder. Hold, run over every kept archive on
// every sweep, writes one the first time and nothing after.
m := &memRegistry{}
store := m.serve(t)
ctx := context.Background()
body := []byte("a theme")
reference := m.bare("shell/config", body)
if held, err := store.Held(ctx, reference); err != nil || held {
t.Fatalf("a bare blob reads as held=%v (%v)", held, err)
}
wrote, err := store.Hold(ctx, reference)
if err != nil {
t.Fatal(err)
}
if !wrote {
t.Fatal("holding a bare archive wrote nothing")
}
_, holder := Holder(digestOf(body), int64(len(body)))
if _, ok := m.manifests["shell/config@"+holder]; !ok {
t.Fatalf("the store holds manifests %v; want %s", m.manifests, holder)
}
if held, err := store.Held(ctx, reference); err != nil || !held {
t.Fatalf("after holding, held=%v (%v)", held, err)
}
writes := len(m.writes)
wrote, err = store.Hold(ctx, reference)
if err != nil {
t.Fatal(err)
}
if wrote || len(m.writes) != writes {
t.Fatalf("holding again wrote %v", m.writes[writes:])
}
}
func TestAnImageNeedsNoHolderAndAMissingArchiveIsGone(t *testing.T) {
m := &memRegistry{}
store := m.serve(t)
ctx := context.Background()
// An image is its own manifest: nothing is asked.
wrote, err := store.Hold(ctx, catalogue.ArtifactStoreScheme+"web/app@sha256:"+strings.Repeat("b", 64))
if err != nil || wrote || len(m.writes) != 0 {
t.Fatalf("holding an image wrote=%v err=%v writes=%v", wrote, err, m.writes)
}
// An archive the store does not have is a fact to report, not something to invent a holder for.
_, err = store.Hold(ctx, catalogue.ArtifactStoreScheme+"web/config/blobs/sha256:"+strings.Repeat("c", 64))
if !errors.Is(err, Gone) {
t.Fatalf("holding a missing archive answered %v, want Gone", err)
}
// And a reference that is not the mesh's is refused as such.
if _, err := store.Hold(ctx, "docker.io/library/registry@sha256:abc"); !errors.Is(err, ErrNotOurs) {
t.Fatalf("holding a vendor's image answered %v, want ErrNotOurs", err)
}
}
func TestLettingGoOfAnArchiveDeletesItsHolderFirst(t *testing.T) {
// A holder left behind would keep the bytes through every collection while the record said
// collected; the link deleted first and the holder failing after would be that exactly.
m := &memRegistry{}
store := m.serve(t)
ctx := context.Background()
body := []byte("an old theme")
reference := m.bare("shell/config", body)
if _, err := store.Hold(ctx, reference); err != nil {
t.Fatal(err)
}
m.writes = nil
if err := store.LetGo(ctx, reference); err != nil {
t.Fatal(err)
}
_, holder := Holder(digestOf(body), int64(len(body)))
want := []string{
"DELETE /v2/shell/config/manifests/" + holder,
"DELETE /v2/shell/config/blobs/" + digestOf(body),
}
if strings.Join(m.writes, "\n") != strings.Join(want, "\n") {
t.Fatalf("the store was asked\n%s\nwant\n%s", strings.Join(m.writes, "\n"), strings.Join(want, "\n"))
}
if len(m.manifests) != 0 {
t.Fatalf("a holder survived: %v", m.manifests)
}
// Asked again, the archive is already gone, which is the outcome wanted.
if err := store.LetGo(ctx, reference); !errors.Is(err, Gone) {
t.Fatalf("letting go twice answered %v, want Gone", err)
}
}
func TestLettingGoOfAnUnheldArchiveStillDeletesIt(t *testing.T) {
// An archive published before holders and let go of before any sweep held it: the holder's
// delete answers 404, which is the outcome wanted, and the blob still goes.
m := &memRegistry{}
store := m.serve(t)
reference := m.bare("shell/config", []byte("never held"))
if err := store.LetGo(context.Background(), reference); err != nil {
t.Fatal(err)
}
if len(m.blobs) != 0 {
t.Fatalf("the blob survived: %v", m.blobs)
}
}
+130
View File
@@ -0,0 +1,130 @@
package artifacts
import (
"bytes"
"context"
"fmt"
"io"
"net/http"
"os"
"os/exec"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
)
// The registry's own collector keeps a held archive and takes a bare one (novox/hq issue 253,
// ADR 0189).
//
// Everything else here is asserted against a fake, which can only say what this side asks. This is
// the one question a fake cannot answer — what `registry garbage-collect` actually does with what
// this side wrote — and it is the whole of whether the store's nightly step may stop being a dry
// run. Against the very image the mesh's store runs:
//
// docker run -d --rm --name mesh-controller-registry -p 15000:5000 \
// -e REGISTRY_STORAGE_DELETE_ENABLED=true registry:2.8.3
// MESH_TEST_REGISTRY=127.0.0.1:15000 MESH_TEST_REGISTRY_CONTAINER=mesh-controller-registry \
// go test -run Live ./internal/artifacts/
// docker stop mesh-controller-registry
//
// Skipped without both variables: it needs a registry it may write to and collect, and a container
// to run the collector in.
func TestLiveTheRegistrysCollectorKeepsWhatIsHeldAndTakesWhatIsNot(t *testing.T) {
address := os.Getenv("MESH_TEST_REGISTRY")
container := os.Getenv("MESH_TEST_REGISTRY_CONTAINER")
if address == "" || container == "" {
t.Skip("no MESH_TEST_REGISTRY / MESH_TEST_REGISTRY_CONTAINER; see this test's comment for the registry to raise")
}
ctx := context.Background()
store := Store{Address: address}
run := time.Now().UnixNano()
// Four archives in four repositories, each a different story. Distinct bytes per run, so a
// registry reused across runs cannot answer for an earlier one.
put := func(name string) (repository, digest string, body []byte) {
repository = fmt.Sprintf("live-%d/%s", run, name)
body = []byte(fmt.Sprintf("%s archive of run %d", name, run))
digest = digestOf(body)
if err := store.putBlob(ctx, repository, digest, body); err != nil {
t.Fatal(err)
}
return repository, digest, body
}
reference := func(repository, digest string) string {
return catalogue.ArtifactStoreScheme + repository + "/blobs/" + digest
}
// Published held — what PublishArchive now does.
heldRepo, heldDigest, heldBody := put("held")
if _, err := store.HoldBlob(ctx, heldRepo, heldDigest, int64(len(heldBody))); err != nil {
t.Fatalf("the registry refused a holder: %v", err)
}
// Published bare, as before, and never held: what the collector must take.
_, bareDigest, _ := put("bare")
// Published bare and then held by the sweep: the backfill.
backRepo, backDigest, backBody := put("backfilled")
if wrote, err := store.Hold(ctx, reference(backRepo, backDigest)); err != nil || !wrote {
t.Fatalf("backfilling wrote=%v: %v", wrote, err)
}
if held, err := store.Held(ctx, reference(backRepo, backDigest)); err != nil || !held {
t.Fatalf("after backfilling, held=%v: %v", held, err)
}
// Held, and then let go of by the sweep: holder first, then the link.
goneRepo, goneDigest, _ := put("let-go")
if _, err := store.Hold(ctx, reference(goneRepo, goneDigest)); err != nil {
t.Fatal(err)
}
if err := store.LetGo(ctx, reference(goneRepo, goneDigest)); err != nil {
t.Fatalf("letting go of a held archive: %v", err)
}
collected, err := exec.CommandContext(ctx, "docker", "exec", container,
"registry", "garbage-collect", "/etc/docker/registry/config.yml").CombinedOutput()
if err != nil {
t.Fatalf("the collector failed: %v\n%s", err, collected)
}
t.Logf("the collector said:\n%s", lastLines(string(collected), 12))
// What is asserted is the bytes on the store's disk, not what the running server answers: the
// server caches blob descriptors in memory and can answer for a blob the collector removed.
onDisk := func(digest string) bool {
hex := strings.TrimPrefix(digest, "sha256:")
path := "/var/lib/registry/docker/registry/v2/blobs/sha256/" + hex[:2] + "/" + hex + "/data"
return exec.CommandContext(ctx, "docker", "exec", container, "test", "-f", path).Run() == nil
}
if !onDisk(heldDigest) {
t.Error("the collector took an archive published held")
}
if !onDisk(backDigest) {
t.Error("the collector took an archive the sweep backfilled a holder for")
}
if onDisk(bareDigest) {
t.Error("the collector kept a bare archive — then the holders prove nothing, and this test is wrong")
}
if onDisk(goneDigest) {
t.Error("the collector kept an archive the sweep let go of: its holder outlived its link")
}
// And what survived is still fetched exactly as machines fetch it: the blob, by digest.
for repository, want := range map[string][]byte{heldRepo: heldBody, backRepo: backBody} {
digest := digestOf(want)
response, err := http.Get(catalogue.Routed(reference(repository, digest), address))
if err != nil {
t.Fatal(err)
}
got, _ := io.ReadAll(response.Body)
response.Body.Close()
if response.StatusCode != http.StatusOK || !bytes.Equal(got, want) {
t.Errorf("%s answered %s with %q after collection", repository, response.Status, got)
}
}
}
func lastLines(s string, n int) string {
lines := strings.Split(strings.TrimSpace(s), "\n")
if len(lines) > n {
lines = lines[len(lines)-n:]
}
return strings.Join(lines, "\n")
}
+123
View File
@@ -0,0 +1,123 @@
// Package artifacts speaks to the mesh's artifact store over its own door.
//
// Only what the mesh needs that nothing else does: letting go of something it put there
// (novox/hq ADR 0189, issue 108), and holding every archive it keeps by a manifest so the store's
// own collector does not take it (novox/hq issue 253). Pushing is the builder's, through the container runtime; reading
// is every machine's, through its runtime. This is the one operation that belongs to the thing
// holding the records, because it is the only one that is a decision rather than a transfer.
package artifacts
import (
"context"
"errors"
"fmt"
"net/http"
"strings"
"github.com/novox/mesh-controller/internal/catalogue"
)
// Store is the artifact store at an address, as this machine reaches it.
type Store struct {
// Address is `host:port` — the store as the caller reaches it now, composed and never
// recorded (novox/hq 04-ISSUES/102).
Address string
// HTTP is the client used; nil is a client with a modest timeout.
HTTP *http.Client
}
// Gone is the answer when the store does not hold it: the outcome wanted, already true.
var Gone = errors.New("the store does not hold it")
// ErrNotOurs is a reference this sweep will not address: not the mesh's own, or naming nothing
// the store holds by digest.
//
// **A fact about the record, not about the store** (novox/hq issue 226). The two deserve opposite
// responses — skip one and go on, abandon the sweep for the other — and collapsing them into "an
// error" is how a cautious loop became one that did nothing while reporting the right number.
var ErrNotOurs = errors.New("not a reference into the mesh's artifact store")
// LetGo asks the store to drop one artifact the mesh recorded making.
//
// Takes a reference as the mesh records it — `artifact-store://<module>/<artifact>@sha256:…` for
// an image, `…/blobs/sha256:…` for an archive — because that is the identity every record uses,
// and composes the address here at the moment of use.
//
// An archive is let go of in two deletes, its holder manifest and then the blob's link (novox/hq
// issue 253); an image in one.
//
// Returns Gone when the store answers that it does not have it. That is not a failure: the sweep
// wants the artifact absent, and it is. It is distinguished from success only so a caller can say
// which of the two happened.
func (s Store) LetGo(ctx context.Context, reference string) error {
// **Strict, and deliberately** (novox/hq issue 226). Only a reference the mesh keeps in its
// own vocabulary is addressed here. `Recorded` would read `docker.io/library/registry@sha256:…`
// as the mesh's too — it cannot tell one registry host from another — so normalising belongs
// where the provenance is known, which is the sweep reading its own build records, not here
// where the only job is to refuse anything that is not plainly ours.
path, kept := catalogue.InArtifactStore(reference)
if !kept {
// Nothing the mesh put in its own store. Refused rather than attempted: composing a
// delete for a reference of unknown shape is how a sweep reaches something that is not
// the mesh's. Distinguished from a store that refuses, so a sweep skips this and goes on.
return fmt.Errorf("%w: %s", ErrNotOurs, reference)
}
if s.Address == "" {
return fmt.Errorf("this mesh has no artifact store on its network to ask about %s", reference)
}
repository, kind, digest, err := split(path)
if err != nil {
return err
}
if kind == "blobs" {
// **An archive's holder goes before the archive** (novox/hq issue 253): a manifest left
// naming the blob would keep its bytes through every collection while the record said
// collected. Gone here means the store has no such blob, so there is nothing to let go.
if err := s.letGoOfHolder(ctx, repository, digest); err != nil {
return err
}
}
return s.remove(ctx, s.url(repository, kind, digest), reference)
}
// remove asks the store to delete what is at url. Gone when it has no such thing.
func (s Store) remove(ctx context.Context, url, what string) error {
request, err := http.NewRequestWithContext(ctx, http.MethodDelete, url, nil)
if err != nil {
return err
}
response, err := s.client().Do(request)
if err != nil {
return err
}
defer response.Body.Close()
switch response.StatusCode {
case http.StatusAccepted, http.StatusOK, http.StatusNoContent:
return nil
case http.StatusNotFound:
return Gone
case http.StatusMethodNotAllowed:
// The registry was started without deletion enabled. Said plainly, because the remedy is
// a setting on the store's module and not anything about this artifact.
return fmt.Errorf(
"the artifact store refuses deletion: its server was started without it enabled "+
"(REGISTRY_STORAGE_DELETE_ENABLED), so nothing can be collected until the store "+
"module is applied again (novox/hq ADR 0189). Asking about %s", what)
default:
return fmt.Errorf("the artifact store answered %s for %s", response.Status, what)
}
}
// split reads a recorded path into the repository, which endpoint names the thing, and the digest.
//
// Two shapes, which are the two the mesh records: `<repository>@sha256:<hex>` is a manifest, and
// `<repository>/blobs/sha256:<hex>` is a blob.
func split(path string) (repository, kind, digest string, err error) {
if before, after, ok := strings.Cut(path, "@sha256:"); ok {
return before, "manifests", "sha256:" + after, nil
}
if before, after, ok := strings.Cut(path, "/blobs/sha256:"); ok {
return before, "blobs", "sha256:" + after, nil
}
return "", "", "", fmt.Errorf("%w: %q names nothing the store holds by digest", ErrNotOurs, path)
}
+134
View File
@@ -0,0 +1,134 @@
package artifacts
import (
"context"
"errors"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// Asking the store to let go of what the mesh no longer keeps (novox/hq ADR 0189, issue 108).
//
// A fake store records what it was asked to delete, so what is asserted is the mesh's decision
// and the shape of the request — not the registry's behaviour, which is the registry's to test.
func fakeStore(t *testing.T, answer int) (Store, *[]string) {
t.Helper()
var asked []string
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodHead && strings.Contains(r.URL.Path, "/blobs/") {
// An archive's size, asked so its holder can be named (novox/hq issue 253). A store
// that does not have the thing does not have its blob either.
if answer == http.StatusNotFound {
w.WriteHeader(http.StatusNotFound)
return
}
w.Header().Set("Content-Length", "7")
w.WriteHeader(http.StatusOK)
return
}
if r.Method != http.MethodDelete {
t.Errorf("the store was asked %s %s; collecting is a delete", r.Method, r.URL.Path)
}
asked = append(asked, r.URL.Path)
w.WriteHeader(answer)
}))
t.Cleanup(server.Close)
return Store{Address: strings.TrimPrefix(server.URL, "http://")}, &asked
}
func TestAnImageAndAnArchiveAreAskedForAtTheirOwnEndpoints(t *testing.T) {
// The two shapes the mesh records: a manifest by digest, and a blob by digest. They are
// different endpoints, and asking at the wrong one answers 404 — which this would then
// record as collected, leaving the bytes on disk for ever while the record says otherwise.
store, asked := fakeStore(t, http.StatusAccepted)
ctx := context.Background()
image := catalogue.ArtifactStoreScheme + "web/app@sha256:abc123"
archive := catalogue.ArtifactStoreScheme + "web/config/blobs/sha256:def456"
if err := store.LetGo(ctx, image); err != nil {
t.Fatal(err)
}
if err := store.LetGo(ctx, archive); err != nil {
t.Fatal(err)
}
// The archive's holder goes first, then the archive (novox/hq issue 253).
_, holder := Holder("sha256:def456", 7)
want := []string{
"/v2/web/app/manifests/sha256:abc123",
"/v2/web/config/manifests/" + holder,
"/v2/web/config/blobs/sha256:def456",
}
if strings.Join(*asked, " ") != strings.Join(want, " ") {
t.Fatalf("the store was asked %v; want %v", *asked, want)
}
}
func TestAStoreThatDoesNotHaveItAnswersGone(t *testing.T) {
// The outcome wanted, already true. Told apart from success only so the sweep can say which
// happened; both are recorded, because retrying for ever is the thing to avoid.
store, _ := fakeStore(t, http.StatusNotFound)
err := store.LetGo(context.Background(), catalogue.ArtifactStoreScheme+"web/app@sha256:abc123")
if !errors.Is(err, Gone) {
t.Fatalf("a store that does not hold it answered %v, want Gone", err)
}
}
func TestAStoreWithDeletionOffSaysSoAndNamesTheRemedy(t *testing.T) {
// The registry answers 405 when it was started without deletion enabled. The remedy is a
// setting on the store's module, and saying "405" would send somebody to the wrong place.
store, _ := fakeStore(t, http.StatusMethodNotAllowed)
err := store.LetGo(context.Background(), catalogue.ArtifactStoreScheme+"web/app@sha256:abc123")
if err == nil {
t.Fatal("a store that refuses deletion was read as success")
}
if !strings.Contains(err.Error(), "REGISTRY_STORAGE_DELETE_ENABLED") {
t.Fatalf("the refusal does not name the remedy: %v", err)
}
}
func TestAReferenceThatIsNotTheMeshsOwnIsNeverAsked(t *testing.T) {
// The whole safety of the sweep is that it names only what the mesh recorded putting there.
// A reference of another shape — a vendor's image, a package version — is refused rather
// than composed into a delete somewhere that is not the mesh's store.
store, asked := fakeStore(t, http.StatusAccepted)
for _, reference := range []string{
"docker.io/library/registry@sha256:abc123",
"registry@sha256:abc123",
"1.4.2",
} {
if err := store.LetGo(context.Background(), reference); err == nil {
t.Errorf("%s was asked about; it is not a reference into the mesh's store", reference)
}
}
if len(*asked) != 0 {
t.Fatalf("the store was asked about %v", *asked)
}
}
// A reference this sweep will not address says so as ErrNotOurs, which is a fact about the
// record and not about the store (novox/hq issue 226).
//
// The sweep skips one and abandons itself for the other, so they cannot be the same error. The
// first live run met a reference recorded with the store's old address, read the refusal as "the
// store refuses everything", and collected none of the 1681 it had found.
func TestAReferenceThisSweepWillNotAddressIsToldApartFromAStoreRefusing(t *testing.T) {
store, asked := fakeStore(t, http.StatusAccepted)
for _, reference := range []string{
"docker.io/library/registry@sha256:abc123",
"127.0.0.1:5100/mesh-tools/build@sha256:abc123",
"1.4.2",
} {
err := store.LetGo(context.Background(), reference)
if !errors.Is(err, ErrNotOurs) {
t.Errorf("%s answered %v; a sweep must be able to skip it and go on", reference, err)
}
}
if len(*asked) != 0 {
t.Fatalf("the store was asked about %v", *asked)
}
}
+91
View File
@@ -0,0 +1,91 @@
package broker
import (
"fmt"
"os"
"testing"
"time"
"github.com/nats-io/nats.go"
)
// A consumer that reacts to announcements, made on a stream that keeps a week of them, starts from now:
// made from the start it replays every merge and build of that week (novox/hq issue 248). And a reset
// re-makes a stuck one from now, its configuration otherwise kept, and refuses a work queue.
//
// docker run -d --rm --name t -p 14231:4222 nats:2.10-alpine -js
// MESH_TEST_NATS=nats://127.0.0.1:14231 go test ./internal/broker/ -run TestAConsumerMadeFromNow
func TestAConsumerMadeFromNowNeverReplaysTheStreamsHistory(t *testing.T) {
url := os.Getenv("MESH_TEST_NATS")
if url == "" {
t.Skip("MESH_TEST_NATS unset")
}
js, err := Dial(url)
if err != nil {
t.Fatal(err)
}
defer js.Close()
const stream = "HISTORY_TEST"
_ = js.js.DeleteStream(stream)
if _, err := js.js.AddStream(&nats.StreamConfig{Name: stream, Subjects: []string{"history.>"}, Storage: nats.MemoryStorage}); err != nil {
t.Fatal(err)
}
defer func() { _ = js.js.DeleteStream(stream) }()
for i := 0; i < 5; i++ {
if _, err := js.js.Publish("history.merged", []byte(fmt.Sprint(i))); err != nil {
t.Fatal(err)
}
}
fresh := Consumer{Name: "fresh", Stream: stream, Filters: []string{"history.merged"}, Push: true,
AckWaitSeconds: 30, MaxDeliver: 5, MaxAckPending: 1, FromNow: true}
if err := js.EnsureConsumer(fresh); err != nil {
t.Fatal(err)
}
info, _ := js.js.ConsumerInfo(stream, "fresh")
if info.NumPending != 0 || info.Config.DeliverPolicy != nats.DeliverNewPolicy {
t.Fatalf("a consumer made from now holds %d of the stream's past (policy %v)", info.NumPending, info.Config.DeliverPolicy)
}
_, _ = js.js.Publish("history.merged", []byte("new"))
time.Sleep(100 * time.Millisecond)
if info, _ = js.js.ConsumerInfo(stream, "fresh"); info.NumPending != 1 {
t.Fatalf("a new announcement is not pending: %d", info.NumPending)
}
// Asserted again, it keeps where it is.
if err := js.EnsureConsumer(fresh); err != nil {
t.Fatal(err)
}
// One made from the start, as the server's default makes it, and stuck behind its history.
old := fresh
old.Name, old.FromNow = "old", false
if err := js.EnsureConsumer(old); err != nil {
t.Fatal(err)
}
if info, _ = js.js.ConsumerInfo(stream, "old"); info.NumPending != 6 {
t.Fatalf("the default should replay all six: %d", info.NumPending)
}
before, after, err := js.ResetConsumer(stream, "old")
if err != nil {
t.Fatal(err)
}
info, _ = js.js.ConsumerInfo(stream, "old")
if before.Pending != 6 || after.Pending != 0 || info.Config.MaxAckPending != 1 || info.Config.MaxDeliver != 5 ||
info.Config.AckWait != 30*time.Second || info.Config.DeliverSubject == "" {
t.Fatalf("before %+v after %+v config %+v", before, after, info.Config)
}
// Never a work queue: what is pending there is work.
const queue = "QUEUE_TEST"
_ = js.js.DeleteStream(queue)
if _, err := js.js.AddStream(&nats.StreamConfig{Name: queue, Subjects: []string{"queue.>"}, Retention: nats.WorkQueuePolicy, Storage: nats.MemoryStorage}); err != nil {
t.Fatal(err)
}
defer func() { _ = js.js.DeleteStream(queue) }()
if _, err := js.js.AddConsumer(queue, &nats.ConsumerConfig{Durable: "w", AckPolicy: nats.AckExplicitPolicy}); err != nil {
t.Fatal(err)
}
if _, _, err := js.ResetConsumer(queue, "w"); err == nil {
t.Fatal("a work queue's consumer was reset")
}
}
+45 -1
View File
@@ -47,7 +47,13 @@ type Consumer struct {
// and a merge that came back rebuilt what it had just built, five times over on 2026-09-30.
// With one outstanding, the server holds the rest, and the heartbeat is keeping the message.
MaxAckPending int
Why string
// FromNow makes a consumer that does not exist yet start at the stream's end rather than its
// beginning (novox/hq issue 248). For a consumer that reacts to announcements — a merge, a build's
// outcome — on a stream that keeps a week of them: the server's default, everything the stream
// holds, replays every merge and every build of that week as if it had just happened. A consumer
// that exists keeps where it is, whatever this says; only its making is decided here.
FromNow bool
Why string
}
// seatStreamName is the stream holding a seat's inbound work. Named after the seat rather than
@@ -144,6 +150,44 @@ func ConsumerFor(p Principal) (Consumer, bool) {
}, true
}
// ModuleConsumer is one module's durable consumer, with the module and node it is for.
type ModuleConsumer struct {
Node, Module string
Consumer Consumer
}
// ConsumersOf is every module's durable consumer the composed users imply: each module user's, and
// each module a runtime carries — a carried module with no account of its own is no user (novox/hq
// issue 195), and its consumer is still the controller's to make, since the runtime reads it on the
// module's behalf (ADR 0198). One per module and node, whichever of the two named it first.
func ConsumersOf(users []Principal) []ModuleConsumer {
var out []ModuleConsumer
seen := map[string]bool{}
add := func(p Principal) {
c, needed := ConsumerFor(p)
if !needed || seen[p.Node+"/"+p.Module] {
return
}
seen[p.Node+"/"+p.Module] = true
out = append(out, ModuleConsumer{Node: p.Node, Module: p.Module, Consumer: c})
}
for _, p := range users {
if p.Kind == KindModule {
add(p)
}
}
for _, p := range users {
if p.Kind != KindNodeTools {
continue
}
for _, d := range p.Carries {
add(Principal{Kind: KindModule, Node: p.Node, Module: d.Module, Emits: d.Emits,
Consumes: d.Consumes, Serves: d.Serves, Holds: d.Holds, Uses: d.Uses, Watches: d.Watches})
}
}
return out
}
// HolderConsumerFor is the worker a seat's holders share on that seat's work queue.
//
// **One worker for every holder, and each holder pulls one ask when it is idle** (novox/hq ADR
+104
View File
@@ -1,6 +1,7 @@
package broker
import (
"context"
"crypto/sha256"
"crypto/tls"
"crypto/x509"
@@ -12,6 +13,7 @@ import (
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
)
// The JetStream side of the controller: the one place the mesh's streams and consumers are
@@ -84,6 +86,13 @@ func pinnedTo(path string) (*tls.Config, error) {
return PinnedToFingerprint(want), nil
}
// OnConn is the JetStream handle over a connection the caller already holds — the control plane's
// link — for asserting what the bus holds without dialling a second time.
func OnConn(conn *nats.Conn) *JetStream {
js, _ := conn.JetStream()
return &JetStream{conn: conn, js: js}
}
// DialPinned is Dial with the server's certificate pinned by a fingerprint the caller already holds
// — a module or a build machine that was handed one beside its credential, and has no file.
func DialPinned(url, fingerprint string, opts ...nats.Option) (*JetStream, error) {
@@ -299,6 +308,9 @@ func (j *JetStream) EnsureConsumer(c Consumer) error {
}
return nil
case errors.Is(err, nats.ErrConsumerNotFound):
if c.FromNow {
want.DeliverPolicy = nats.DeliverNewPolicy
}
if _, err := j.js.AddConsumer(c.Stream, want); err != nil {
return fmt.Errorf("creating consumer %s on %s: %w", c.Name, c.Stream, err)
}
@@ -308,6 +320,51 @@ func (j *JetStream) EnsureConsumer(c Consumer) error {
}
}
// ConsumerState is what a reset says about a consumer, before and after.
type ConsumerState struct {
DeliverPolicy string
Delivered uint64
AckFloor uint64
Pending uint64
AckPending int
}
func stateOf(info *nats.ConsumerInfo) ConsumerState {
policy, _ := info.Config.DeliverPolicy.MarshalJSON()
return ConsumerState{DeliverPolicy: strings.Trim(string(policy), `"`), Delivered: info.Delivered.Stream,
AckFloor: info.AckFloor.Stream, Pending: info.NumPending, AckPending: info.NumAckPending}
}
// ResetConsumer re-makes a consumer to start from now, its configuration otherwise unchanged (novox/hq
// issue 248): what it had not yet delivered or acknowledged is dropped, which is the point — on a stream
// that keeps history, a consumer replaying a week of announcements does nothing anyone wants. Refused on
// a work queue, where what is pending is work nobody else will do.
func (j *JetStream) ResetConsumer(stream, name string) (before, after ConsumerState, err error) {
info, err := j.js.StreamInfo(stream)
if err != nil {
return before, after, fmt.Errorf("asking about stream %s: %w", stream, err)
}
if info.Config.Retention == nats.WorkQueuePolicy {
return before, after, fmt.Errorf("%s is a work queue: what its consumer has pending is work, and a reset would drop it", stream)
}
have, err := j.js.ConsumerInfo(stream, name)
if err != nil {
return before, after, fmt.Errorf("asking about consumer %s on %s: %w", name, stream, err)
}
before = stateOf(have)
want := have.Config
want.DeliverPolicy = nats.DeliverNewPolicy
want.OptStartSeq, want.OptStartTime = 0, nil
if err := j.js.DeleteConsumer(stream, name); err != nil {
return before, after, fmt.Errorf("removing consumer %s on %s: %w", name, stream, err)
}
made, err := j.js.AddConsumer(stream, &want)
if err != nil {
return before, after, fmt.Errorf("re-making consumer %s on %s — it is gone until the controller asserts it at its next start: %w", name, stream, err)
}
return before, stateOf(made), nil
}
func retentionOf(r Retention) nats.RetentionPolicy {
switch r {
case RetentionWorkQueue:
@@ -316,3 +373,50 @@ func retentionOf(r Retention) nats.RetentionPolicy {
return nats.LimitsPolicy
}
}
// EnsureBucket creates a module's bucket if it is absent and brings its options to match if it is
// present (novox/hq ADR 0201).
//
// **An update, never a delete and recreate**, for the reason a stream is updated: recreating
// discards what the bucket holds, and what a module's state holds is data. The mesh's caps are
// asserted with the owner's options, so a bucket made by hand converges to them.
func (j *JetStream) EnsureBucket(b Bucket) error {
history := b.History
if history == 0 {
history = 1
}
js, err := jetstream.New(j.conn)
if err != nil {
return err
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if _, err := js.CreateOrUpdateKeyValue(ctx, jetstream.KeyValueConfig{
Bucket: b.Bucket(),
Description: b.Why(),
History: uint8(history),
TTL: time.Duration(b.TTLSeconds) * time.Second,
MaxValueSize: StateMaxValueBytes,
MaxBytes: StateMaxBytes,
Storage: jetstream.FileStorage,
}); err != nil {
return fmt.Errorf("asserting bucket %s: %w", b.Bucket(), err)
}
return nil
}
// BucketNames is every key-value bucket on the server, the mesh's and anybody else's.
func (j *JetStream) BucketNames() ([]string, error) {
js, err := jetstream.New(j.conn)
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
lister := js.KeyValueStoreNames(ctx)
var out []string
for name := range lister.Name() {
out = append(out, name)
}
return out, lister.Error()
}
+6
View File
@@ -45,6 +45,11 @@ type Membership struct {
// module that must tell the mesh from the world, the route proxy serving an internal name, reads
// it here rather than keeping a definition of its own.
Mesh []string `json:"mesh,omitempty"`
// State is every bucket this module's code may reach, by the name it uses for each, and whether
// it may write it (novox/hq ADR 0201): the runtime answers a bundle's state verbs from this list
// and refuses, with the reason, what is not on it — the bus enforces only the union over every
// module on the machine.
State []StateIssued `json:"state,omitempty"`
}
// Served is one address a tool is answered on.
@@ -103,6 +108,7 @@ func MembershipFor(node string, d Declared, where Placements) Membership {
m.Seats = append(m.Seats, SeatServed{Seat: s.Name, Verb: verb, Subject: seatToolSubject(s, verb, node)})
}
}
m.State = stateIssuedFor(d)
if len(d.Invokes) > 0 {
m.Reaches = map[string][]string{}
for _, t := range d.Invokes {
+17
View File
@@ -103,6 +103,12 @@ type Principal struct {
// permission and nothing beside it.
Invokes []string
// State is the local names of the state this principal's module keeps, and Reads the state of
// others it reads as `<module>.<name>` (novox/hq ADR 0201): a bucket each, kept by the owner's
// instances and read by whoever declares it.
State []string
Reads []string
// PasswordHash is the bcrypt hash the mesh minted. The plaintext is sealed to the principal
// and never appears here: this file is written to a node's disk and read by a server, and a
// secret that can be read from a configuration file is a secret with a wider blast radius
@@ -421,6 +427,10 @@ func PermissionsFor(p Principal) (Permissions, error) {
}
}
// 5. Its state, and the state of others it reads (novox/hq ADR 0201): every one read and
// watched, its own written too.
pub = append(pub, stateGrants(p.Module, p.State, p.Reads)...)
case KindNodeTools:
// **One process serves what every module on the machine would have served for itself**
// (novox/hq ADR 0175). Each carried module's whole tool namespace — the same grant that
@@ -487,6 +497,13 @@ func PermissionsFor(p Principal) (Permissions, error) {
"$JS.API.CONSUMER.MSG.NEXT."+stream+"."+durable,
"$JS.ACK."+stream+"."+durable+".>")
}
// **And it keeps and reads state for the modules it carries** (novox/hq ADR 0201): the union
// of what each may do with a bucket — an owner's write, a reader's read. That one module's code
// does not write another's bucket through it is the runtime's to keep, from the membership
// each assignment is issued, as it keeps each module's events under that module's own name.
for _, d := range p.Carries {
pub = append(pub, stateGrants(d.Module, stateNames(d.State), d.Reads)...)
}
sub = unique(sub)
pub = unique(pub)
}
+169
View File
@@ -0,0 +1,169 @@
package broker
import (
"fmt"
"sort"
"strings"
)
// A module's state on the bus (novox/hq ADR 0201, design 32 §4, design 25 §3).
//
// A module names the state it keeps (`state`) and the state of others it reads (`reads`), and each
// is a key-value bucket: the server's own last-per-subject stream with direct reads, delete markers
// and watches, which is the state relationship the mesh already uses for declarations, opened to
// modules. The controller creates every bucket from the catalogue — from registration, like a
// seat's stream, so a reader may watch before the owner runs anywhere — and no module can.
//
// Pure, like everything else in this package that decides what the bus holds; jetstream.go is the
// part that asks a server.
// The mesh's caps on a bucket, the same for every module: a value is a piece of state, not a file,
// and a bucket that grew without bound would be one module filling the bus's disk for everyone.
const (
StateMaxValueBytes = 256 * 1024
StateMaxBytes = 64 * 1024 * 1024
)
// A Bucket is one module's declared state as the bus holds it.
type Bucket struct {
Module string
Name string
// History is how many values a key keeps; zero is one.
History int
// TTLSeconds is how long a value lives; zero is until replaced or deleted.
TTLSeconds int
}
// BucketName is the bucket a module's state lives in: the module and the local name joined by an
// underscore, which neither may contain, so two modules can never derive one bucket.
func BucketName(module, name string) string { return module + "_" + name }
// Bucket is this bucket's name on the bus.
func (b Bucket) Bucket() string { return BucketName(b.Module, b.Name) }
// Why is carried into the server's description of the bucket, so somebody reading the server's
// own state finds whose it is and why it is kept.
func (b Bucket) Why() string {
return fmt.Sprintf("%s's state %q (novox/hq ADR 0201): its current value per key, written by %s, "+
"read by whatever declares it reads it; kept when %s is unassigned, because it is data",
b.Module, b.Name, b.Module, b.Module)
}
// bucketOfRead is the bucket a read names, `<module>.<name>`, or false when it names none.
func bucketOfRead(read string) (string, bool) {
at := strings.LastIndex(read, ".")
if at <= 0 || at == len(read)-1 {
return "", false
}
module, name := read[:at], read[at+1:]
if !safeSubject.MatchString(module) || !safeSubject.MatchString(name) {
return "", false
}
return BucketName(module, name), true
}
// stateGrants is what a principal publishes to reach the state its modules keep and read: for every
// bucket, binding to it, reading a key directly, and an ordered consumer for listing and watching,
// created and deleted on the bucket's own stream, with its flow control answered; for a bucket an
// owner keeps, writing under the bucket's own subjects too.
//
// **Measured against a running server, 2026-10-04** (novox/hq research 024), and each one is there
// because leaving it out failed: without STREAM.INFO nothing binds; without DIRECT.GET nothing is
// read; without CONSUMER.CREATE no key is listed and nothing is watched; without CONSUMER.DELETE a
// watch cannot be stopped and lingers on the server. A write outside these is refused by the server
// — and reaches the writer as a timeout, not a refusal, which is why the runtime refuses first.
func stateGrants(module string, keeps []string, reads []string) []string {
var out []string
read := func(bucket string) {
stream := "KV_" + bucket
out = append(out,
"$JS.API.STREAM.INFO."+stream,
"$JS.API.DIRECT.GET."+stream+".>",
"$JS.API.CONSUMER.CREATE."+stream+".>",
"$JS.API.CONSUMER.DELETE."+stream+".>",
"$JS.FC."+stream+".>")
}
for _, name := range keeps {
if !safeSubject.MatchString(name) {
continue
}
bucket := BucketName(module, name)
read(bucket)
out = append(out, "$KV."+bucket+".>")
}
for _, r := range reads {
if bucket, ok := bucketOfRead(r); ok {
read(bucket)
}
}
return out
}
// StateIssued is one bucket an assignment may reach, by the name its module uses for it: its own
// state by the local name, another's as `<module>.<name>` (novox/hq ADR 0201).
type StateIssued struct {
Name string `json:"name"`
Bucket string `json:"bucket"`
Writes bool `json:"writes,omitempty"`
}
// stateIssuedFor is every bucket a module's code may reach, as its membership lists them.
func stateIssuedFor(d Declared) []StateIssued {
var out []StateIssued
for _, b := range d.State {
out = append(out, StateIssued{Name: b.Name, Bucket: BucketName(d.Module, b.Name), Writes: true})
}
for _, r := range d.Reads {
if bucket, ok := bucketOfRead(r); ok {
out = append(out, StateIssued{Name: r, Bucket: bucket})
}
}
return out
}
// stateNames is the local names of a module's own buckets.
func stateNames(buckets []Bucket) []string {
out := make([]string, 0, len(buckets))
for _, b := range buckets {
out = append(out, b.Name)
}
return out
}
// A BucketAsserter is the part of a JetStream connection bucket assertion needs.
type BucketAsserter interface {
// EnsureBucket creates the bucket if absent and brings its options to match if present, never
// discarding what it holds.
EnsureBucket(b Bucket) error
// BucketNames is every key-value bucket on the server.
BucketNames() ([]string, error)
}
// RaiseBuckets asserts every declared bucket and answers the buckets on the server that nothing
// declares any more.
//
// **Those are reported, never removed** (novox/hq ADR 0201, ADR 0030): what a module stored is
// data, and a manifest edited, a module renamed or a catalogue entry dropped is an ordinary day's
// work that must not take data with it. Removing one is a person's act.
func RaiseBuckets(a BucketAsserter, buckets []Bucket) (undeclared []string, err error) {
sorted := append([]Bucket(nil), buckets...)
sort.Slice(sorted, func(i, j int) bool { return sorted[i].Bucket() < sorted[j].Bucket() })
declared := map[string]bool{}
for _, b := range sorted {
if err := a.EnsureBucket(b); err != nil {
return nil, fmt.Errorf("asserting %s's state %q: %w", b.Module, b.Name, err)
}
declared[b.Bucket()] = true
}
names, err := a.BucketNames()
if err != nil {
return nil, fmt.Errorf("listing the bus's state: %w", err)
}
for _, n := range names {
if !declared[n] {
undeclared = append(undeclared, n)
}
}
sort.Strings(undeclared)
return undeclared, nil
}
+180
View File
@@ -0,0 +1,180 @@
package broker
import (
"slices"
"strings"
"testing"
"github.com/nats-io/nats.go"
)
// The grants measured against a running server (novox/hq research 024): an owner reads and writes
// its bucket, a reader only reads, and neither reaches any other bucket.
func TestAnOwnerWritesItsStateAndAReaderOnlyReads(t *testing.T) {
owner, err := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "claude-code",
State: []string{"servers"}, PasswordHash: "x"})
if err != nil {
t.Fatal(err)
}
for _, s := range []string{
"$KV.claude-code_servers.>",
"$JS.API.STREAM.INFO.KV_claude-code_servers",
"$JS.API.DIRECT.GET.KV_claude-code_servers.>",
"$JS.API.CONSUMER.CREATE.KV_claude-code_servers.>",
"$JS.API.CONSUMER.DELETE.KV_claude-code_servers.>",
"$JS.FC.KV_claude-code_servers.>",
} {
has(t, owner.Publish, s)
}
hasNot(t, owner.Publish, "$KV.>")
hasNot(t, owner.Publish, "$JS.API.>")
reader, err := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "console",
Reads: []string{"claude-code.servers"}, PasswordHash: "x"})
if err != nil {
t.Fatal(err)
}
has(t, reader.Publish, "$JS.API.DIRECT.GET.KV_claude-code_servers.>")
has(t, reader.Publish, "$JS.API.CONSUMER.CREATE.KV_claude-code_servers.>")
hasNot(t, reader.Publish, "$KV.claude-code_servers.>")
for _, s := range reader.Subscribe {
if s == "$KV.claude-code_servers.>" {
t.Fatalf("a reader subscribes the bucket's subjects directly: %v", reader.Subscribe)
}
}
}
// One runtime carries every module on its machine, so its grant is the union: the owner's write
// where an owner is carried, a read where only a reader is.
func TestTheRuntimeKeepsAndReadsStateForItsModules(t *testing.T) {
perms, err := PermissionsFor(Principal{Kind: KindNodeTools, Node: "one", Module: RuntimeModule,
Carries: []Declared{
{Module: "claude-code", State: []Bucket{{Module: "claude-code", Name: "servers"}},
Reads: []string{"licence-manager.bindings"}},
{Module: "audit"},
}, PasswordHash: "x"})
if err != nil {
t.Fatal(err)
}
has(t, perms.Publish, "$KV.claude-code_servers.>")
has(t, perms.Publish, "$JS.API.DIRECT.GET.KV_licence-manager_bindings.>")
hasNot(t, perms.Publish, "$KV.licence-manager_bindings.>")
}
// A module with no state is granted nothing of any bucket — the composition of every module that
// existed before this is unchanged.
func TestAModuleWithNoStateReachesNoBucket(t *testing.T) {
perms, err := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "billing",
Emits: []string{"order.placed"}, PasswordHash: "x"})
if err != nil {
t.Fatal(err)
}
for _, s := range perms.Publish {
if strings.HasPrefix(s, "$KV.") || strings.HasPrefix(s, "$JS.FC.") || strings.Contains(s, ".KV_") {
t.Fatalf("granted %q without declaring state", s)
}
}
}
// A read that names no bucket grants nothing rather than something that happens to parse.
func TestAReadThatNamesNoBucketGrantsNothing(t *testing.T) {
if got := stateGrants("a", nil, []string{"nodot", "x.", ".y", "a.b>"}); len(got) != 0 {
t.Fatalf("granted %v for reads that name no bucket", got)
}
}
// The membership lists every bucket the module's code may reach, by the name the module uses for
// it, and whether it may write it — the list the runtime refuses from.
func TestAMembershipListsTheStateItsModuleMayReach(t *testing.T) {
m := MembershipFor("one", Declared{Module: "claude-code",
State: []Bucket{{Module: "claude-code", Name: "servers"}},
Reads: []string{"licence-manager.bindings"}}, Placements{})
want := []StateIssued{
{Name: "servers", Bucket: "claude-code_servers", Writes: true},
{Name: "licence-manager.bindings", Bucket: "licence-manager_bindings"},
}
if !slices.Equal(m.State, want) {
t.Fatalf("issued %+v, want %+v", m.State, want)
}
if none := MembershipFor("one", Declared{Module: "audit"}, Placements{}); none.State != nil {
t.Fatalf("a module with no state was issued %+v", none.State)
}
}
type buckets struct {
ensured []string
on []string
}
func (b *buckets) EnsureBucket(x Bucket) error {
b.ensured = append(b.ensured, x.Bucket())
return nil
}
func (b *buckets) BucketNames() ([]string, error) { return b.on, nil }
// Every declared bucket is asserted; one on the server that nothing declares is said, not removed.
func TestRaisingStateReportsWhatNothingDeclares(t *testing.T) {
b := &buckets{on: []string{"claude-code_servers", "gone_old", "ours_by_hand"}}
undeclared, err := RaiseBuckets(b, []Bucket{{Module: "claude-code", Name: "servers"}, {Module: "a", Name: "b"}})
if err != nil {
t.Fatal(err)
}
if !slices.Equal(b.ensured, []string{"a_b", "claude-code_servers"}) {
t.Fatalf("asserted %v", b.ensured)
}
if !slices.Equal(undeclared, []string{"gone_old", "ours_by_hand"}) {
t.Fatalf("reported %v", undeclared)
}
}
// Against a real server: a bucket is created with the owner's options and the mesh's caps,
// asserting it again changes nothing and keeps what it holds, and a changed option is brought to
// match in place.
func TestABucketIsAssertedInPlace(t *testing.T) {
js := aLiveBus(t)
b := Bucket{Module: "statetest", Name: "servers"}
if _, err := RaiseBuckets(js, []Bucket{b}); err != nil {
t.Fatalf("a real server refused a module's bucket: %v", err)
}
kv, err := js.Context().KeyValue(b.Bucket())
if err != nil {
t.Fatal(err)
}
if _, err := kv.Put("all.one", []byte(`{"kept":true}`)); err != nil {
t.Fatal(err)
}
b.History = 3
if _, err := RaiseBuckets(js, []Bucket{b}); err != nil {
t.Fatalf("asserting the bucket again failed, so a restart would: %v", err)
}
got, err := kv.Get("all.one")
if err != nil || string(got.Value()) != `{"kept":true}` {
t.Fatalf("asserting again lost what the bucket held: %v %v", got, err)
}
status, err := kv.Status()
if err != nil {
t.Fatal(err)
}
if status.History() != 3 {
t.Fatalf("history is %d, the owner declared 3", status.History())
}
if s, ok := status.(*nats.KeyValueBucketStatus); ok {
if c := s.StreamInfo().Config; c.MaxMsgSize != StateMaxValueBytes || c.MaxBytes != StateMaxBytes {
t.Fatalf("the mesh's caps are not on the bucket: value %d, bucket %d", c.MaxMsgSize, c.MaxBytes)
}
}
}
// Against a real server: the handle over a connection the control plane already holds asserts a
// bucket as Dial's does — what a push uses, so a module registered since the last start has its
// bucket before its membership names it.
func TestABucketIsAssertedOverAHeldConnection(t *testing.T) {
js := aLiveBus(t)
held := OnConn(js.Conn())
if _, err := RaiseBuckets(held, []Bucket{{Module: "statetest", Name: "held"}}); err != nil {
t.Fatalf("asserting over a held connection failed: %v", err)
}
if _, err := js.Context().KeyValue("statetest_held"); err != nil {
t.Fatalf("the bucket is not there: %v", err)
}
}
+1
View File
@@ -270,6 +270,7 @@ func MeshConsumers() []Consumer {
// announcement handed over behind it must wait on the server, not time out on the
// client and come back to be acted on again.
MaxAckPending: 1,
FromNow: true,
Why: "the two events the mesh's own controller reacts to, one at a time; after " +
"max-deliver it dead-letters, because an announcement it cannot act on will not " +
"become actionable",
+18
View File
@@ -33,6 +33,14 @@ type Declared struct {
Watches []Seat
// Invokes are the tools it calls, `<module>.<tool>` or `*` (novox/hq ADR 0152).
Invokes []string
// State is the state it keeps, each a bucket its instances write (novox/hq ADR 0201).
State []Bucket
// Reads are other modules' state it reads, each `<module>.<name>` (novox/hq ADR 0201).
Reads []string
// NoAccount says the module declares no own secret named broker, so no account could ever be
// delivered to it and nothing can connect as it (novox/hq issue 195). Said in the negative so a
// record that does not say is composed as it always was.
NoAccount bool
}
// Records is what composing a user list needs to know about the mesh, and nothing more.
@@ -77,10 +85,20 @@ func Users(r Records) ([]Principal, error) {
if runtimeHere && d.Module == RuntimeModule {
continue
}
// **A module with nowhere to read an account is no user** (novox/hq issue 195). `module
// issue` refuses it one (issue 078: an account nothing reads is an orphan), so its user
// could only ever be left out of the file for want of a password — and every such module
// was named, on every status and plan, as a credential the mesh had not minted. Where the
// runtime is, it speaks for the module; where it is not, the module cannot speak at all,
// and a user would not change that.
if d.NoAccount {
continue
}
out = append(out, Principal{
Kind: KindModule, Node: node, Module: d.Module,
Emits: d.Emits, Consumes: d.Consumes, Serves: d.Serves,
Holds: d.Holds, Uses: d.Uses, Watches: d.Watches, Invokes: d.Invokes,
State: stateNames(d.State), Reads: d.Reads,
})
}
if runtimeHere {
+54
View File
@@ -296,3 +296,57 @@ func TestTheRuntimeModuleBecomesTheMachinesRuntimePrincipal(t *testing.T) {
t.Error("telegram lost its own principal when the runtime arrived on its node")
}
}
// A module that declares nowhere to read an account is no user: it could never be issued one, so it
// was only ever named as a credential the mesh had not minted (novox/hq issue 195). Where the runtime
// is, the runtime still carries it — its grants are the runtime's.
func TestAModuleThatCannotReadAnAccountIsNoUser(t *testing.T) {
r := someRecords()
r.Assigned["one"] = append(r.Assigned["one"],
Declared{Module: "packet-filter", NoAccount: true, Emits: []string{"rule.changed"}},
Declared{Module: RuntimeModule})
users, err := Users(r)
if err != nil {
t.Fatal(err)
}
for _, u := range users {
if u.Username() == "one.packet-filter" {
t.Fatalf("packet-filter declares no broker secret and was composed as a user: %v", namesOf(t, r))
}
if u.Kind == KindNodeTools {
carried := false
for _, d := range u.Carries {
carried = carried || d.Module == "packet-filter"
}
if !carried {
t.Error("the runtime stopped carrying a module that has no account of its own")
}
}
}
if names := strings.Join(namesOf(t, r), ","); !strings.Contains(names, "one.telegram") {
t.Errorf("a module that does read an account lost its user: %s", names)
}
}
// A carried module with no account still has its consumer made: the runtime reads it on the module's
// behalf (ADR 0198), and the consumer was derived from the module's own user until issue 195 took
// that user away.
func TestACarriedModuleWithNoAccountStillHasItsConsumer(t *testing.T) {
r := someRecords()
r.Assigned["one"] = append(r.Assigned["one"],
Declared{Module: "listener", NoAccount: true, Consumes: []string{"shop.order.placed"}},
Declared{Module: RuntimeModule})
r.Assigned["two"] = append(r.Assigned["two"],
Declared{Module: "auditor", Consumes: []string{"shop.order.placed"}})
users, err := Users(r)
if err != nil {
t.Fatal(err)
}
got := map[string]int{}
for _, c := range ConsumersOf(users) {
got[c.Node+"/"+c.Module]++
}
if got["one/listener"] != 1 || got["two/auditor"] != 1 || len(got) != 2 {
t.Fatalf("consumers: %v", got)
}
}
+106 -3
View File
@@ -215,7 +215,7 @@ func Build(ctx context.Context, run Runner, publish Publisher,
sort.Slice(artifacts, func(i, j int) bool { return artifacts[i].Name < artifacts[j].Name })
for _, a := range artifacts {
say("artifact", "%s (%s%s) — starting", a.Name, a.Kind, langSuffix(a))
made, err := one(ctx, run, publish, manifest.Module, within, workspace, commit, credentials, a, args, held, npmrcPath, seatBases, say)
made, err := one(ctx, run, publish, manifest.Module, within, workspace, commit, credentials, a, args, held, npmrcPath, npmrc, seatBases, say)
if err != nil {
say("artifact", "%s FAILED: %v", a.Name, err)
return Result{}, err
@@ -443,7 +443,7 @@ func wantsPackages(manifest catalogue.Manifest, within string) bool {
func one(ctx context.Context, run Runner, publish Publisher,
module, tree, workspace, commit, credentials string, a catalogue.Artifact, args []string,
held map[string]string, npmrc string, seats map[string]string,
held map[string]string, npmrc string, registry Npmrc, seats map[string]string,
say func(step, format string, args ...any)) (catalogue.Built, error) {
switch a.Kind {
@@ -582,6 +582,11 @@ func one(ctx context.Context, run Runner, publish Publisher,
"holds no copy of it. Build %s first",
module, a.Name, chain.Language, chain.Base, chain.Artifact, chain.Base)
}
// The module's own packages first, where the compiler and the bundler resolve them from
// (dependencies.go); nothing at all for a module whose package.json names only the SDK.
if err := installOwn(ctx, run, tree, chain, base, registry, say); err != nil {
return catalogue.Built{}, fmt.Errorf("%s: %s: %w", module, a.Name, err)
}
say("bundle", "compiling %s in %s's toolchain", a.Language, chain.Base)
compiled, err := compile(ctx, run, tree, chain, base, a)
if err != nil {
@@ -593,6 +598,12 @@ func one(ctx context.Context, run Runner, publish Publisher,
if err != nil {
return catalogue.Built{}, fmt.Errorf("%s: writing %s's launchers failed: %w", module, a.Name, err)
}
if chain.Bundler != "" {
say("bundle", "bundling each entrypoint into one file")
if compiled, err = bundled(ctx, run, tree, chain, base, a, launchers); err != nil {
return catalogue.Built{}, fmt.Errorf("%s: bundling %s failed: %w", module, a.Name, err)
}
}
say("bundle", "compiled, packing")
body, err := pack(compiled)
if err != nil {
@@ -974,7 +985,7 @@ func compile(ctx context.Context, run Runner, tree string, chain Toolchain,
if _, err := run(ctx, tree, "docker", invocation...); err != nil {
return "", err
}
if chain.Dependencies != "" {
if chain.Dependencies != "" && chain.Bundler == "" {
// **What the bundle runs with, from the image it was compiled in** (Toolchain.Dependencies).
// A second run in the same image rather than a shell wrapped around the compiler: the
// compile line stays a plain command a reader can run by hand, and the copy is one more
@@ -1224,3 +1235,95 @@ func writeLaunchers(root string, chain Toolchain, a catalogue.Artifact) (map[str
}
return out, nil
}
// bundledSuffix is where a bundle's one-file output is written, beside what the compiler wrote.
const bundledSuffix = ".bundled"
// bundled makes every entrypoint and every launcher of a compiled bundle ONE file, in the toolchain
// image's bundler, and answers the directory to pack (novox/hq ADR 0193).
//
// **What a launched bundle runs is what it imports, and nothing else.** Every served bundle is its
// own process, so it carries its own copy of the SDK and its own dependencies inlined — the
// toolchain's whole node_modules no longer travels in every bundle. An entrypoint a process runs by
// name (`node daemon/index.js`) is bundled in place under its own name; a launcher keeps its name
// and its first line, and stays executable. A package the bundler cannot inline is named by the
// artifact (`external`), kept as an import, and only then is the toolchain's runtime directory
// copied beside the files. CommonJS inlined into an ES module still finds `require`.
func bundled(ctx context.Context, run Runner, tree string, chain Toolchain, base string,
a catalogue.Artifact, launchers map[string]string) (string, error) {
const within = "/app/modules/module"
out, final := Out(a.Name), Out(a.Name)+bundledSuffix
if err := os.RemoveAll(filepath.Join(tree, final)); err != nil {
return "", err
}
if err := os.MkdirAll(filepath.Join(tree, final), 0o755); err != nil {
return "", err
}
common := []string{"--bundle", "--platform=node", "--format=esm", "--target=node22",
"--outbase=" + out, "--outdir=" + final, "--log-level=warning",
"--banner:js=import { createRequire as __meshRequire } from 'node:module'; const require = __meshRequire(import.meta.url);"}
for _, x := range a.External {
common = append(common, "--external:"+x)
}
var plain []string
for _, e := range a.Entrypoints {
if strings.HasSuffix(e, ".js") {
plain = append(plain, out+"/"+e)
}
}
var launch []string
for _, l := range sortedValues(launchers) {
launch = append(launch, out+"/"+l)
}
// Refused by name in an image that predates the bundler, as the dependencies copy is: a bundle
// packed without it would carry nothing it imports. Run as itself: npm installs esbuild's native
// binary in place of its script, which `node` cannot run.
guard := `test -x "$0" || { echo "the toolchain image carries no bundler at $0: it predates one-file bundles, rebuild mesh-tools first" >&2; exit 1; }; exec "$0" "$@"`
step := func(entries []string, extra ...string) error {
if len(entries) == 0 {
return nil
}
invocation := []string{"run", "--rm", "--volume", tree + ":" + within, "--workdir", within, base,
"sh", "-c", guard, chain.Bundler}
invocation = append(invocation, entries...)
invocation = append(invocation, common...)
invocation = append(invocation, extra...)
_, err := run(ctx, tree, "docker", invocation...)
return err
}
if err := step(plain); err != nil {
return "", err
}
if err := step(launch, "--out-extension:.js=.mjs"); err != nil {
return "", err
}
// Plain `.js` output is an ES module; said once, as the runtime directory used to say it.
if err := os.WriteFile(filepath.Join(tree, final, "package.json"), []byte(`{"type":"module","private":true}`+"\n"), 0o644); err != nil {
return "", err
}
for _, l := range launchers {
path := filepath.Join(tree, final, filepath.FromSlash(l))
if _, err := os.Stat(path); err == nil {
if err := os.Chmod(path, 0o755); err != nil {
return "", err
}
}
}
if len(a.External) > 0 && chain.Dependencies != "" {
copying := []string{"run", "--rm", "--volume", tree + ":" + within, "--workdir", within, base,
"sh", "-c", `cp -a "$0/node_modules" "$1/"`, chain.Dependencies, final}
if _, err := run(ctx, tree, "docker", copying...); err != nil {
return "", fmt.Errorf("copying the packages %s keeps external: %w", a.Name, err)
}
}
return filepath.Join(tree, final), nil
}
func sortedValues(m map[string]string) []string {
out := make([]string, 0, len(m))
for _, v := range m {
out = append(out, v)
}
sort.Strings(out)
return out
}
+38 -14
View File
@@ -83,25 +83,49 @@ func TestABundleIsCompiledAndPackedWithNoDockerfile(t *testing.T) {
t.Fatalf("the bundle was not pinned: %v", got.Manifest.Resources[0])
}
// **And what it runs with, from the image it was compiled in** (novox/hq to-be 38 WP3). A
// second run in the same toolchain image copies the toolchain's runtime directory — the
// `"type": "module"` package.json and the pruned node_modules — into the output's root, and
// refuses by name when the image carries none rather than packing a bundle that starts nowhere.
var copied string
// **One file per entrypoint and launcher, in the toolchain's bundler** (novox/hq ADR 0193). A
// second run in the same toolchain image bundles each into the artifact's bundled output, the SDK
// inlined, refusing by name in an image that predates the bundler; and the toolchain's
// node_modules is no longer copied into a bundle that keeps nothing external.
var bundling []string
for _, line := range r.ran {
if strings.HasPrefix(line, "docker run") && strings.Contains(line, "/app/runtime") {
copied = line
if strings.HasPrefix(line, "docker run") && strings.Contains(line, "esbuild") {
bundling = append(bundling, line)
}
}
if copied == "" {
t.Fatalf("the bundle's dependencies were not copied in after the compile:\n%s", strings.Join(r.ran, "\n"))
if len(bundling) != 2 {
t.Fatalf("want one bundling run for the entrypoints and one for the launchers:\n%s", strings.Join(r.ran, "\n"))
}
if !strings.Contains(copied, "mesh-tools/build@sha256:") || !strings.Contains(copied, "predates") ||
!strings.Contains(copied, Out("code")) {
t.Fatalf("the copy does not run in the same toolchain, refuse an older image by name, or land in the artifact's output: %s", copied)
for _, want := range []string{"mesh-tools/build@sha256:", "predates one-file bundles", "--bundle", "--format=esm",
"--platform=node", "--outdir=" + Out("code") + ".bundled", Out("code") + "/index.js"} {
if !strings.Contains(bundling[0], want) {
t.Errorf("the entrypoints' bundling lacks %q: %s", want, bundling[0])
}
}
if strings.Index(strings.Join(r.ran, "\n"), "--outDir") > strings.Index(strings.Join(r.ran, "\n"), "/app/runtime") {
t.Fatal("the dependencies were copied before the compile wrote its output")
if !strings.Contains(bundling[1], Out("code")+"/index.serve.mjs") || !strings.Contains(bundling[1], "--out-extension:.js=.mjs") {
t.Errorf("the launcher is not bundled under its own name: %s", bundling[1])
}
if strings.Contains(strings.Join(r.ran, "\n"), "/app/runtime") {
t.Errorf("the toolchain's node_modules was copied into a bundle that keeps nothing external:\n%s", strings.Join(r.ran, "\n"))
}
if strings.Index(strings.Join(r.ran, "\n"), "--outDir") > strings.Index(strings.Join(r.ran, "\n"), "esbuild") {
t.Fatal("the bundler ran before the compile wrote its output")
}
}
// A bundle naming packages it keeps external is bundled with them as imports, and carries the
// toolchain's node_modules for them — the one case it still does.
func TestABundleKeepingAPackageExternalCarriesTheToolchainsModules(t *testing.T) {
manifest := strings.Replace(aBundle, `"entrypoints":["index.js"]`, `"entrypoints":["index.js"],"external":["sharp"]`, 1)
r, workspace := aRepository(t, manifest, map[string]string{"index.ts": "console.log(1)"})
held := map[string]string{"mesh-tools/build": "registry.invalid/mesh-tools/build@sha256:" + strings.Repeat("b", 64)}
if _, err := Build(context.Background(), compiling{r}.run, r,
"https://forge.invalid/greeter.git", "", "", workspace, held, Npmrc{}, GitCredential{}, nil); err != nil {
t.Fatal(err)
}
all := strings.Join(r.ran, "\n")
if !strings.Contains(all, "--external:sharp") || !strings.Contains(all, "/app/runtime") {
t.Errorf("an external package was not kept as an import with the toolchain's modules beside it:\n%s", all)
}
}
+147
View File
@@ -0,0 +1,147 @@
package builder
import (
"context"
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"sort"
"strings"
)
// A module's own packages, installed before its bundle is compiled, so the bundler inlines them.
//
// **A bundle could only import what the toolchain happened to carry.** The compiler and the bundler
// resolve an import by walking up from the module's source: the module's own directory first, then
// the toolchain image's node_modules. Nothing ever put anything in the first, so a module needing a
// database driver (`pg`, `mongodb`, `mssql`) could not be a bundle at all, and kept a container whose
// recipe installed it by hand (novox/hq ADR 0198 §4: "the backend's own driver inside the bundle").
// Now the module's `package.json` says what it depends on, as any Node package does, and the build
// installs exactly that into the module's own directory before compiling.
//
// **The SDK the toolchain carries is the one a bundle is built with, whatever the module says**
// (novox/hq issue 212: the toolchain is rebuilt on every SDK release and every bundle after it). A
// module's `package.json` names `@novox/mesh-sdk` with a range — it has to, to type-check on a
// workstation — and installing that range would shadow the toolchain's copy for this module alone:
// one module compiled against an older SDK than its neighbours, chosen by a caret nobody re-reads.
// So the SDK is taken out of what is installed (and never fetched), and any copy something else
// pulls in is removed afterwards; every import of it resolves past the module's node_modules to the
// toolchain's. A module therefore cannot pin a different SDK, by design: the toolchain is the pin.
//
// **Correctness before speed.** Every build installs afresh into a fresh clone, from the lockfile
// when the module has one (`npm ci`, exact) and from its ranges otherwise; nothing installed is kept
// between builds. What is shared is npm's own download cache, a named volume, which is
// content-addressed and verified by integrity on every read — it saves the network, never the
// install. Install scripts do not run: the build node runs nobody's postinstall, and what a script
// would build natively could not be inlined into one file anyway.
// sdkPackage is the package a TypeScript bundle's launcher serves through, and the one package a
// module's own dependencies never supply (above).
const sdkPackage = "@novox/mesh-sdk"
// npmCache is the named volume npm's download cache lives in across builds on one build node.
const npmCache = "mesh-builder-npm-cache"
// ownDependencies is what a module's package.json depends on beyond the SDK, sorted; nothing when
// the module has no package.json or depends on nothing else — which builds exactly as before.
func ownDependencies(tree string) ([]string, error) {
raw, err := os.ReadFile(filepath.Join(tree, "package.json"))
if errors.Is(err, os.ErrNotExist) {
return nil, nil
}
if err != nil {
return nil, err
}
var p struct {
Dependencies map[string]string `json:"dependencies"`
}
if err := json.Unmarshal(raw, &p); err != nil {
return nil, fmt.Errorf("the module's package.json is not JSON: %w", err)
}
var names []string
for name := range p.Dependencies {
if name != sdkPackage {
names = append(names, name)
}
}
sort.Strings(names)
return names, nil
}
// installSteps is the script run inside the toolchain image, from the module's own directory ($0).
// It works in a scratch copy so the module's package.json and lockfile are never rewritten, takes
// the SDK out of what is installed, installs production dependencies only, removes any copy of the
// SDK something pulled in, and puts the result at the module's node_modules.
const installSteps = `set -e
work="$(mktemp -d)"
cp "$0/package.json" "$work/"
if [ -f "$0/package-lock.json" ]; then cp "$0/package-lock.json" "$work/"; fi
cd "$work"
node -e '
const fs = require("fs"), sdk = process.argv[1];
const p = JSON.parse(fs.readFileSync("package.json", "utf8"));
for (const k of ["dependencies", "peerDependencies", "optionalDependencies"]) if (p[k]) delete p[k][sdk];
delete p.devDependencies; delete p.scripts;
fs.writeFileSync("package.json", JSON.stringify(p));
' "$1"
shift
if [ -f package-lock.json ]; then
npm ci --omit=dev --omit=peer --ignore-scripts --no-audit --no-fund "$@"
else
npm install --omit=dev --omit=peer --ignore-scripts --no-audit --no-fund --no-package-lock "$@"
fi
find node_modules -depth -type d -path "*/node_modules/@novox/mesh-sdk" -exec rm -rf {} +
rm -rf "$0/node_modules"
cp -a node_modules "$0/node_modules"
`
// installOwn installs a TypeScript module's own production dependencies into its directory, in the
// toolchain image, before the compile — or does nothing at all for a module that has none.
func installOwn(ctx context.Context, run Runner, tree string, chain Toolchain, base string,
registry Npmrc, say func(step, format string, args ...any)) error {
if chain.Language != "typescript" {
return nil
}
deps, err := ownDependencies(tree)
if err != nil || len(deps) == 0 {
return err
}
scoped := strings.TrimSpace(registry.Scope)
if !registry.Enabled() {
// **No registry, no scoped package.** Without the mesh's registry a scoped name resolves on
// the public one, where anybody may have published it: a dependency that installs is not
// the dependency the module meant.
for _, d := range deps {
if strings.HasPrefix(d, "@novox/") {
return fmt.Errorf("the module depends on %s, and this build knows no package registry "+
"for its scope; it would resolve from the public registry, which is not where the "+
"mesh publishes it", d)
}
}
}
const within = "/app/modules/module"
invocation := []string{"run", "--rm",
"--volume", tree + ":" + within,
"--volume", npmCache + ":/root/.npm",
"--workdir", within}
var flags []string
if registry.Enabled() {
// The registry is reached where the binding says it is, which may be this machine's own
// loopback — the reason an image build that resolves packages runs on the host network too.
invocation = append(invocation, "--network", "host")
reg := strings.TrimSpace(registry.Registry)
if !strings.HasSuffix(reg, "/") {
reg += "/"
}
flags = append(flags, "--"+scoped+":registry="+reg)
}
invocation = append(invocation, base, "sh", "-c", installSteps, within, sdkPackage)
invocation = append(invocation, flags...)
say("bundle", "installing the module's own packages: %s", strings.Join(deps, ", "))
if _, err := run(ctx, tree, "docker", invocation...); err != nil {
return fmt.Errorf("installing the module's own packages (%s): %w", strings.Join(deps, ", "), err)
}
return nil
}
+131
View File
@@ -0,0 +1,131 @@
package builder
import (
"context"
"strings"
"testing"
)
// A module's own packages (dependencies.go): installed into its own directory, in the toolchain,
// before the compile, so the bundler inlines them — the SDK always the toolchain's.
func buildWithPackageJSON(t *testing.T, pkg string, extra map[string]string, registry Npmrc) (*recorded, error) {
t.Helper()
files := map[string]string{"index.ts": "console.log(1)"}
if pkg != "" {
files["package.json"] = pkg
}
for k, v := range extra {
files[k] = v
}
r, workspace := aRepository(t, aBundle, files)
held := map[string]string{"mesh-tools/build": "registry.invalid/mesh-tools/build@sha256:" + strings.Repeat("b", 64)}
_, err := Build(context.Background(), compiling{r}.run, r,
"https://forge.invalid/greeter.git", "", "", workspace, held, registry, GitCredential{}, nil)
return r, err
}
func installs(r *recorded) []string {
var out []string
for _, line := range r.ran {
if strings.HasPrefix(line, "docker run") && strings.Contains(line, "npm ci") {
out = append(out, line)
}
}
return out
}
func compileIndex(r *recorded) int {
for i, line := range r.ran {
if strings.Contains(line, "--outDir") {
return i
}
}
return -1
}
func TestAModulesOwnPackagesAreInstalledInTheToolchainBeforeTheCompile(t *testing.T) {
r, err := buildWithPackageJSON(t, `{"type":"module","dependencies":{"@novox/mesh-sdk":"^0.1.0","pg":"^8"},"devDependencies":{"typescript":"^5"}}`,
nil, Npmrc{Scope: "@novox", Registry: "https://forge.invalid/api/packages/novox/npm"})
if err != nil {
t.Fatal(err)
}
got := installs(r)
if len(got) != 1 {
t.Fatalf("want one install of the module's own packages:\n%s", strings.Join(r.ran, "\n"))
}
line := got[0]
for _, want := range []string{
"mesh-tools/build@sha256:", // in the toolchain image
":/app/modules/module", // into the module's own directory
"--workdir /app/modules/module", //
npmCache + ":/root/.npm", // npm's verified download cache, and only that
"--omit=dev", "--ignore-scripts", // production packages, no build-node scripts
"npm ci", "npm install", "--no-package-lock", // the lockfile when there is one, else the ranges
"--@novox:registry=https://forge.invalid/api/packages/novox/npm/", // the scope from the mesh's registry
"--network host",
"@novox/mesh-sdk", // named, to be taken out of what is installed
} {
if !strings.Contains(line, want) {
t.Errorf("the install lacks %q:\n%s", want, line)
}
}
// The SDK is the toolchain's: never installed from the module's range, and any copy removed.
if !strings.Contains(line, `delete p[k][sdk]`) || !strings.Contains(line, `-path "*/node_modules/@novox/mesh-sdk" -exec rm -rf`) {
t.Errorf("the module's own SDK range could shadow the toolchain's SDK:\n%s", line)
}
if i, c := strings.Index(strings.Join(r.ran, "\n"), "npm ci"), compileIndex(r); c < 0 ||
i > strings.Index(strings.Join(r.ran, "\n"), "--outDir") {
t.Fatalf("the install did not run before the compile:\n%s", strings.Join(r.ran, "\n"))
}
}
// **A module with nothing beyond the SDK builds exactly as before**: the same commands, no install.
func TestAModuleDependingOnlyOnTheSDKBuildsExactlyAsBefore(t *testing.T) {
without, err := buildWithPackageJSON(t, "", nil, Npmrc{})
if err != nil {
t.Fatal(err)
}
for _, pkg := range []string{
`{"type":"module","dependencies":{"@novox/mesh-sdk":"^0.1.0"},"devDependencies":{"typescript":"^5"}}`,
`{"type":"module"}`,
} {
with, err := buildWithPackageJSON(t, pkg, map[string]string{"package-lock.json": "{}"}, Npmrc{Scope: "@novox", Registry: "https://forge.invalid/npm/"})
if err != nil {
t.Fatal(err)
}
if strings.Contains(strings.Join(with.ran, "\n"), "npm ") {
t.Fatalf("a module depending on nothing but the SDK ran npm:\n%s", strings.Join(with.ran, "\n"))
}
if len(with.ran) != len(without.ran) {
t.Fatalf("a module depending only on the SDK built differently from one with no package.json:\n%s\n---\n%s",
strings.Join(with.ran, "\n"), strings.Join(without.ran, "\n"))
}
}
}
// Without the mesh's registry a scoped package would resolve on the public one: refused by name.
func TestAScopedPackageWithNoRegistryIsRefused(t *testing.T) {
r, err := buildWithPackageJSON(t, `{"dependencies":{"@novox/mesh-sdk":"^0.1.0","@novox/other":"^1"}}`, nil, Npmrc{})
if err == nil || !strings.Contains(err.Error(), "@novox/other") {
t.Fatalf("a scoped package was installed with no registry for its scope: %v", err)
}
if strings.Contains(strings.Join(r.ran, "\n"), "--outDir") {
t.Fatal("the compile ran after the refusal")
}
// A public package installs without one, from the public registry and nothing else.
r, err = buildWithPackageJSON(t, `{"dependencies":{"mssql":"^11"}}`, nil, Npmrc{})
if err != nil {
t.Fatal(err)
}
if got := installs(r); len(got) != 1 || strings.Contains(got[0], ":registry=") || strings.Contains(got[0], "--network host") {
t.Fatalf("a public package's install: %v", got)
}
}
func TestAnUnreadablePackageJSONIsRefusedByName(t *testing.T) {
_, err := buildWithPackageJSON(t, `{"dependencies":`, nil, Npmrc{})
if err == nil || !strings.Contains(err.Error(), "package.json") {
t.Fatalf("a broken package.json was not refused by name: %v", err)
}
}
+29 -7
View File
@@ -7,6 +7,8 @@ import (
"io"
"net/http"
"strings"
"github.com/novox/mesh-controller/internal/artifacts"
)
// Where built artifacts go.
@@ -66,22 +68,32 @@ func (r Registry) PublishImage(ctx context.Context, localTag, repository string)
return pinned, nil
}
// PublishArchive stores bytes as a blob and returns where to fetch them from.
// PublishArchive stores bytes as a blob, holds it by a manifest, and returns where to fetch them
// from.
//
// Two steps, which is the registry's own protocol: ask for somewhere to put it, then put it there
// naming the digest. The registry verifies the digest itself, so a blob that arrived corrupted is
// refused by the thing storing it rather than by the machine unpacking it a week later.
// Two steps for the blob, which is the registry's own protocol: ask for somewhere to put it, then
// put it there naming the digest. The registry verifies the digest itself, so a blob that arrived
// corrupted is refused by the thing storing it rather than by the machine unpacking it a week
// later.
//
// **Then a manifest that names it** (novox/hq issue 253, ADR 0189). The store's own collector
// marks only from manifests, and a blob no manifest names is collected however much the mesh
// means to keep it — so an archive published bare is an archive the first nightly collection
// deletes. The holder is composed from the digest and size alone (artifacts.Holder), which is
// what lets the sweep recompute it to backfill or let go without anything being recorded here.
// What a machine is told to fetch is the blob, exactly as before.
func (r Registry) PublishArchive(ctx context.Context, repository string, body []byte, digest string) (string, error) {
base := "http://" + r.Address + "/v2/" + repository
final := base + "/blobs/" + digest
// Already there. Blobs are immutable and named by their content, so this is not an
// optimisation — re-uploading would be asking the registry to store what it already has under
// the name it already has.
// the name it already has. It is still held: a blob published before holders existed is
// exactly the one a rebuild of the same source finds already there.
if there, err := r.has(ctx, final); err != nil {
return "", err
} else if there {
return final, nil
return final, r.hold(ctx, repository, digest, len(body))
}
start, err := http.NewRequestWithContext(ctx, http.MethodPost, base+"/blobs/uploads/", nil)
@@ -119,7 +131,17 @@ func (r Registry) PublishArchive(ctx context.Context, repository string, body []
said, _ := io.ReadAll(io.LimitReader(done.Body, 4096))
return "", fmt.Errorf("%s refused the blob: %s %s", base, done.Status, strings.TrimSpace(string(said)))
}
return final, nil
return final, r.hold(ctx, repository, digest, len(body))
}
// hold puts the manifest holding an archive beside it. A build whose archive could not be held is
// a failed build: recorded as published, it would be an archive the store's collector takes.
func (r Registry) hold(ctx context.Context, repository, digest string, size int) error {
store := artifacts.Store{Address: r.Address, HTTP: r.client()}
if _, err := store.HoldBlob(ctx, repository, digest, int64(size)); err != nil {
return fmt.Errorf("published %s/blobs/%s and could not hold it by a manifest: %w", repository, digest, err)
}
return nil
}
// has is whether this registry already holds what is at that URL.
+86 -6
View File
@@ -9,6 +9,8 @@ import (
"net/http/httptest"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/artifacts"
)
// An OCI registry as a content-addressed blob store, which is what it is.
@@ -18,9 +20,10 @@ import (
// there is not sent again, and that a tag is never accepted as a pin.
type fakeRegistry struct {
blobs map[string][]byte
uploads int
location string
blobs map[string][]byte
manifests map[string][]byte // "<repository>@<digest>"
puts map[string]int // blob uploads, by digest
location string
}
func (f *fakeRegistry) serve(t *testing.T) *httptest.Server {
@@ -28,6 +31,8 @@ func (f *fakeRegistry) serve(t *testing.T) *httptest.Server {
if f.blobs == nil {
f.blobs = map[string][]byte{}
}
f.manifests = map[string][]byte{}
f.puts = map[string]int{}
mux := http.NewServeMux()
server := httptest.NewServer(mux)
mux.HandleFunc("/v2/", func(w http.ResponseWriter, r *http.Request) {
@@ -39,8 +44,28 @@ func (f *fakeRegistry) serve(t *testing.T) *httptest.Server {
return
}
w.WriteHeader(http.StatusNotFound)
case strings.Contains(r.URL.Path, "/manifests/"):
// The archive's holder (novox/hq issue 253): asked for with an Accept, put by digest.
repository, digest, _ := strings.Cut(strings.TrimPrefix(r.URL.Path, "/v2/"), "/manifests/")
switch r.Method {
case http.MethodHead:
if _, ok := f.manifests[repository+"@"+digest]; ok &&
strings.Contains(r.Header.Get("Accept"), "application/vnd.oci.image.manifest.v1+json") {
w.WriteHeader(http.StatusOK)
return
}
w.WriteHeader(http.StatusNotFound)
case http.MethodPut:
body, _ := io.ReadAll(r.Body)
sum := sha256.Sum256(body)
if digest != "sha256:"+hex.EncodeToString(sum[:]) {
w.WriteHeader(http.StatusBadRequest)
return
}
f.manifests[repository+"@"+digest] = body
w.WriteHeader(http.StatusCreated)
}
case r.Method == http.MethodPost && strings.HasSuffix(r.URL.Path, "/blobs/uploads/"):
f.uploads++
where := f.location
if where == "" {
where = "/v2/upload/" + hex.EncodeToString([]byte("session"))
@@ -58,6 +83,7 @@ func (f *fakeRegistry) serve(t *testing.T) *httptest.Server {
return
}
f.blobs[digest] = body
f.puts[digest]++
w.WriteHeader(http.StatusCreated)
default:
w.WriteHeader(http.StatusNotFound)
@@ -92,6 +118,57 @@ func TestAnArchiveIsStoredAndFetchableByItsDigest(t *testing.T) {
}
}
func TestAnArchiveIsPublishedWithAManifestHoldingIt(t *testing.T) {
// The store's collector marks only from manifests, so a bare blob is one the first collection
// deletes, kept or not (novox/hq issue 253, ADR 0189). Every archive goes out held, by the
// holder the sweep can compute for itself from the digest and size.
f := &fakeRegistry{}
r := registryFor(t, f)
body := []byte("a theme")
sum := sha256.Sum256(body)
digest := "sha256:" + hex.EncodeToString(sum[:])
where, err := r.PublishArchive(context.Background(), "shell/config", body, digest)
if err != nil {
t.Fatal(err)
}
if !strings.HasSuffix(where, "/v2/shell/config/blobs/"+digest) {
t.Fatalf("a machine is told to fetch %q; the blob is still what is fetched", where)
}
manifest, holder := artifacts.Holder(digest, int64(len(body)))
if got := f.manifests["shell/config@"+holder]; string(got) != string(manifest) {
t.Fatalf("the store holds %v; want the holder %s in the archive's own repository", f.manifests, holder)
}
if !strings.Contains(string(manifest), `"digest":"`+digest+`"`) {
t.Fatalf("the holder does not name the archive: %s", manifest)
}
empty := sha256.Sum256([]byte("{}"))
if _, ok := f.blobs["sha256:"+hex.EncodeToString(empty[:])]; !ok {
t.Fatal("the holder's empty config was never put, and a registry refuses a manifest without it")
}
}
func TestAnArchiveAlreadyThereIsStillHeld(t *testing.T) {
// A rebuild of the same source finds its archive already there — often one published bare,
// before holders. It is not sent again, and it is held.
f := &fakeRegistry{}
r := registryFor(t, f)
body := []byte("published bare")
sum := sha256.Sum256(body)
digest := "sha256:" + hex.EncodeToString(sum[:])
f.blobs[digest] = body
if _, err := r.PublishArchive(context.Background(), "shell/config", body, digest); err != nil {
t.Fatal(err)
}
if f.puts[digest] != 0 {
t.Fatal("a blob already there was sent again")
}
if _, holder := artifacts.Holder(digest, int64(len(body))); f.manifests["shell/config@"+holder] == nil {
t.Fatal("a blob already there was left bare")
}
}
func TestABlobAlreadyThereIsNotSentAgain(t *testing.T) {
// Not an optimisation: blobs are named by their content, so re-uploading is asking the
// registry to store what it already has under the name it already has.
@@ -107,8 +184,11 @@ func TestABlobAlreadyThereIsNotSentAgain(t *testing.T) {
if _, err := r.PublishArchive(context.Background(), "shell/config", body, digest); err != nil {
t.Fatal(err)
}
if f.uploads != 1 {
t.Fatalf("the blob was uploaded %d times", f.uploads)
if f.puts[digest] != 1 {
t.Fatalf("the blob was uploaded %d times", f.puts[digest])
}
if len(f.manifests) != 1 {
t.Fatalf("publishing twice left %d holders; want the one", len(f.manifests))
}
}
+20
View File
@@ -200,3 +200,23 @@ func TestWhatABuildReadIsTheRepositoriesItsRecipesName(t *testing.T) {
t.Fatal("a module whose recipes name no other repository read one")
}
}
// novox/hq 04-ISSUES/212: a toolchain stands on the SDK's published package, and is built with the
// exact version the mesh published — an argument that changes when the SDK does, so a rebuild after
// a release never reuses an install of the version before it.
func TestAPackageTheMeshPublishedIsPassedByItsExactVersion(t *testing.T) {
manifest := catalogue.Manifest{
Module: "mesh-tools",
Build: &catalogue.Build{
On: []catalogue.BuildsOn{{Arg: "MESH_SDK", Module: "mesh-sdk", Artifact: "lib"}},
},
}
held := map[string]string{"mesh-sdk/lib": "@novox/mesh-sdk@0.1.6"}
args, resolved, err := standingOn(context.Background(), manifest, held, noMirror)
if err != nil {
t.Fatal(err)
}
if fmt.Sprint(args) != "[--build-arg MESH_SDK=@novox/mesh-sdk@0.1.6]" || fmt.Sprint(resolved) != "[@novox/mesh-sdk@0.1.6]" {
t.Errorf("the package was passed as %v, recorded as %v", args, resolved)
}
}
+13 -2
View File
@@ -67,13 +67,23 @@ type Toolchain struct {
// `package.json` saying `"type": "module"` — Node reads a bare `.js` as CommonJS otherwise, so a
// bundle with its dependencies and without that line still fails to start — and the pruned,
// production-only node_modules the runtime itself ships with: the SDK's and the runtime's
// dependencies, and nothing module-specific yet (novox/hq ADR 0188 §5: a skeleton; a module's
// own npm dependencies are a later step). Empty for a language whose bundle carries its own —
// dependencies, and nothing module-specific (a module's own npm dependencies are installed into
// its own directory before the compile and inlined by the bundler: dependencies.go). Empty for a
// language whose bundle carries its own —
// a Go binary is static, a Python bundle is installed with its dependencies.
//
// A toolchain image without the directory fails the build by name rather than packing a bundle
// that starts nowhere: the image predates this and must be rebuilt first.
//
// *Since the bundler (below):* copied only for a bundle that names packages it keeps external,
// which cannot be inlined; a bundle with none carries no node_modules at all.
Dependencies string
// Bundler is the bundler inside the toolchain image that makes each compiled entrypoint and each
// launcher ONE self-contained file (novox/hq ADR 0193): every served bundle is its own process
// now, so each carries its own copy of what it imports — the SDK included — and nothing else.
// A bundle shrinks from the toolchain's whole node_modules to the code it runs. Empty for a
// language whose build is already one file.
Bundler string
// SystemStamp is the variable this language's linker fills with the artifact's declared system,
// for a language whose binaries are pinned to one at link time (novox/hq ADR 0005).
//
@@ -139,6 +149,7 @@ var toolchains = []Toolchain{
Unit: UnitSources,
SourceExt: ".ts",
Dependencies: "/app/runtime",
Bundler: "/app/node_modules/esbuild/bin/esbuild",
},
{
Language: "go",
@@ -0,0 +1,166 @@
package catalogue
import (
"encoding/json"
"strings"
"testing"
)
// novox/hq issue 213 (ADR 0188 §1, §3): a module's own Go service is a bundle the host runs as a
// process, not an image. Each test holds one thing that had to change in the composer for the
// controller to be declared that way.
const aServiceDigest = "sha256:dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd"
// aServiceModule is the controller's shape in miniature: it answers tools of its own, its code is a
// Go bundle a process runs as an account it declares, its secrets belong to that account, it
// prepares its state, and its process replaces the container it used to run as.
func aServiceModule(t *testing.T) Manifest {
t.Helper()
raw := `{
"module": "svc", "version": "1", "tools": ["status"], "prepares": true,
"own-secrets": {"store": "${dir:state}/store"},
"secrets-owner": "svc",
"resources": [
{"id": "state", "type": "directory", "mode": "0700", "place": "mesh", "owner": "svc"},
{"id": "service", "type": "process", "name": "svc", "artifact": "code",
"run": ["./svc", "serve"], "user": "svc", "replaces": ["server"],
"env": {"SVC_STORE_FILE": "${dir:state}/store", "SVC_STORE_PORT": "${seat:mesh-store:5432}"}},
{"id": "account", "type": "user", "name": "svc", "shell": "/usr/bin/nologin", "home": "/var/lib/svc"}
],
"build": {"artifacts": [{"name": "code", "kind": "bundle", "language": "go", "system": "arch",
"from": "cmd/svc", "binary": "svc"}]}
}`
m, err := ParseManifest([]byte(raw))
if err != nil {
t.Fatalf("the service's manifest is refused: %v", err)
}
resolved, err := m.Resolve([]Built{{Name: "code", Kind: ArtifactBundle,
Reference: ArtifactStoreScheme + "svc/code@" + aServiceDigest, Digest: aServiceDigest}})
if err != nil {
t.Fatal(err)
}
return resolved
}
func composeTheService(t *testing.T, with Rendering) []map[string]any {
t.Helper()
with.Needed = map[string]map[string]string{"svc": {"store": "sealed-store"}}
with.ArtifactStore = "anchor.internal:5100"
out, err := Resolution{Node: "anchor", Modules: []Manifest{aServiceModule(t)}}.Declaration(with)
if err != nil {
t.Fatalf("the service does not compose: %v", err)
}
return out
}
func indexOf(out []map[string]any, id string) int {
for i, r := range out {
if r["id"] == id {
return i
}
}
return -1
}
// A module that declares tools has every bundle served by the node's runtime unless it says
// otherwise — and the controller declares the verbs it answers as tools. Its service bundle is run
// by its own process; launched a second time by the runtime it would be a second controller
// pretending to be an MCP server.
func TestABundleItsOwnProcessRunsIsNotServedByTheRuntime(t *testing.T) {
m := aServiceModule(t)
if len(m.Bundles) != 1 {
t.Fatalf("the service's bundle was not kept: %+v", m.Bundles)
}
if loads := m.Bundles[0].Loads; len(loads) != 0 {
t.Fatalf("the runtime would launch the service's own bundle as tools: %v", loads)
}
// And a bundle no resource runs still is served, as a module declaring tools always had it.
tools := Manifest{Module: "t", Version: "1", Tools: []string{"x"},
Build: &Build{Artifacts: []Artifact{{Name: "tools", Kind: ArtifactBundle, Language: "go",
System: "arch", From: "cmd/t"}}}}
resolved, err := tools.Resolve([]Built{{Name: "tools", Kind: ArtifactBundle,
Reference: ArtifactStoreScheme + "t/tools@" + aServiceDigest, Digest: aServiceDigest}})
if err != nil {
t.Fatal(err)
}
if loads := resolved.Bundles[0].Loads; len(loads) != 1 || loads[0] != "t" {
t.Fatalf("a tools bundle nothing runs is no longer served: %v", loads)
}
}
// The account is created before anything is given to it. Its secrets are mesh-computed and so
// placed before the module's own resources; given to a user the machine did not have yet, they were
// refused on the first apply and the process started without them.
func TestAModulesAccountComesBeforeWhatBelongsToIt(t *testing.T) {
out := composeTheService(t, Rendering{})
account, secret := indexOf(out, "svc.account"), indexOf(out, "svc."+NeedID("store"))
if account < 0 || secret < 0 {
t.Fatalf("the account or the secret is missing: %v", out)
}
if account > secret {
t.Fatalf("the secret owned by svc is written before svc exists: account at %d, secret at %d",
account, secret)
}
if owner := out[secret]["owner"]; owner != "svc" {
t.Errorf("the secret belongs to %v, not the account its process runs as", owner)
}
}
// The process is the module's program; its preparation is the same program asked to prepare, as a
// step before it — with the same account and environment, and handing nothing over.
func TestAProcessIsPreparedByItsOwnProgram(t *testing.T) {
out := composeTheService(t, Rendering{})
step, process := indexOf(out, "svc.service-prepare"), indexOf(out, "svc.service")
if step < 0 || process < 0 || step > process {
t.Fatalf("the preparation is not a step before the process (%d, %d): %v", step, process, out)
}
s := out[step]
if s["type"] != "process" || s["run-once"] != true || s["name"] != "svc-prepare" {
t.Errorf("the preparation is not a run-once process: %v", s)
}
if run, _ := json.Marshal(s["run"]); string(run) != `["./svc","prepare"]` {
t.Errorf("the preparation runs %s", run)
}
if s["user"] != "svc" || s["source"] != out[process]["source"] {
t.Errorf("the preparation does not run the same bundle as the same account: %v", s)
}
if env, _ := s["env"].(map[string]any); env["SVC_STORE_FILE"] == nil {
t.Errorf("the preparation is not given the process's environment: %v", s["env"])
}
if _, has := s["replaces"]; has {
t.Errorf("the preparation would hand over what the process replaces: %v", s)
}
if _, has := s["args"]; has {
t.Errorf("the preparation carries a container's args: %v", s)
}
}
// What the process replaces is named as the host recorded it, `<module>.<id>`; unprefixed, the host
// matches nothing and removes the container first, as before.
func TestWhatAProcessReplacesIsNamedAsTheHostRecordedIt(t *testing.T) {
out := composeTheService(t, Rendering{})
p := out[indexOf(out, "svc.service")]
if got, _ := json.Marshal(p["replaces"]); string(got) != `["svc.server"]` {
t.Fatalf("the process replaces %s", got)
}
}
func TestWhatReplacesMayNameIsRefusedNearItsAuthor(t *testing.T) {
for what, resource := range map[string]string{
"a container": `{"id":"c","type":"container","name":"c","image":"x@` + aServiceDigest + `","replaces":["old"]}`,
"a step": `{"id":"p","type":"process","name":"p","run":["./p"],"run-once":true,"replaces":["old"]}`,
"something declared": `{"id":"p","type":"process","name":"p","run":["./p"],"replaces":["p"]}`,
"another module's": `{"id":"p","type":"process","name":"p","run":["./p"],"replaces":["other.old"]}`,
"not a list": `{"id":"p","type":"process","name":"p","run":["./p"],"replaces":"old"}`,
} {
raw := `{"module":"m","version":"1","resources":[` + resource + `]}`
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), "replace") {
t.Errorf("replaces on %s was accepted: %v", what, err)
}
}
ok := `{"module":"m","version":"1","resources":[{"id":"p","type":"process","name":"p","run":["./p"],"replaces":["old"]}]}`
if _, err := ParseManifest([]byte(ok)); err != nil {
t.Errorf("a process replacing what its module no longer declares was refused: %v", err)
}
}
+72
View File
@@ -0,0 +1,72 @@
package catalogue
import (
"strings"
"testing"
)
// A store provider says how its data is backed up (novox/hq ADR 0214); a provider of something that
// holds nothing does not have to.
func TestAStoreProviderWithoutABackupIsRefusedByTheCheck(t *testing.T) {
bare, err := ParseManifest([]byte(`{"module":"pg","version":"1",
"provides":[{"name":"postgres-database","scope":"mesh"}]}`))
if err != nil {
t.Fatalf("parsing refused it, and the rule is the check's: %v", err)
}
if problems := CheckBackup(bare); len(problems) != 1 || !strings.Contains(problems[0], "node-backup") {
t.Fatalf("a store with no backup passed the check: %v", problems)
}
backed, err := ParseManifest([]byte(`{"module":"pg","version":"1",
"provides":[{"name":"postgres-database","scope":"mesh"}],
"resources":[{"id":"dumps","type":"directory","mode":"0700"}],
"contributions":[{"seat":"node-backup","kind":"backup","content":"path ${dir:dumps}"}]}`))
if err != nil {
t.Fatal(err)
}
if problems := CheckBackup(backed); len(problems) != 0 {
t.Fatalf("a store that contributes a backup was refused: %v", problems)
}
route, _ := ParseManifest([]byte(`{"module":"r","version":"1","provides":[{"name":"route","scope":"mesh"}]}`))
if problems := CheckBackup(route); len(problems) != 0 {
t.Fatalf("a route was asked for a backup: %v", problems)
}
}
// A backup contribution names its module's own directories; one it does not declare is refused
// where it is written rather than reaching the holder as the literal text.
func TestABackupNamingAnUndeclaredDirectoryIsRefused(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"pg","version":"1",
"contributions":[{"seat":"node-backup","kind":"backup","content":"path ${dir:nowhere}"}]}`))
if err == nil || !strings.Contains(err.Error(), "nowhere") {
t.Fatalf("a backup of an undeclared directory was accepted: %v", err)
}
}
// The holder receives every module's backup lines with each module's own directories filled, each
// module's under a comment naming it — and a kind written in a shell's grammar is left untouched.
func TestBackupContributionsArePlacedWithTheirModulesDirectories(t *testing.T) {
pg, err := ParseManifest([]byte(`{"module":"pg","version":"1",
"resources":[{"id":"dumps","type":"directory","path":"/srv/pg/dumps","mode":"0700"}],
"contributions":[{"seat":"node-backup","kind":"backup","content":"run pg-dump-all\npath ${dir:dumps}"}]}`))
if err != nil {
t.Fatal(err)
}
mail, err := ParseManifest([]byte(`{"module":"mail","version":"1",
"resources":[{"id":"spool","type":"directory","mode":"0700"}],
"contributions":[{"seat":"node-backup","kind":"backup","content":"path ${dir:spool}"}]}`))
if err != nil {
t.Fatal(err)
}
placed, err := seatContributions([]Manifest{pg, mail}, BackupSeat, "backup", Rendering{})
if err != nil {
t.Fatal(err)
}
for _, want := range []string{"# pg\nrun pg-dump-all\npath /srv/pg/dumps\n", "# mail\npath /var/lib/mail/spool\n"} {
if !strings.Contains(placed, want) {
t.Errorf("placed contributions lack %q:\n%s", want, placed)
}
}
if strings.Contains(placed, "${dir:") {
t.Errorf("a directory reached the holder unfilled:\n%s", placed)
}
}
+15 -6
View File
@@ -21,8 +21,9 @@ import (
// formats and gains no fields.
//
// **It stays name-agnostic** ([ADR 0027]). The mesh does not learn what a `postgres-database` is:
// `at`, `as` and `from` are facts about any provision at all, and everything else comes from what
// the provider said it serves — whose keys are agreed by the requirement's name, not by this file.
// `at`, `as`, `from` and `address` (the providing machine's private address, novox/hq ADR 0194) are
// facts about any provision at all, and everything else comes from what the provider said it
// serves — whose keys are agreed by the requirement's name, not by this file.
// bound is where a module says a value from one of its bindings belongs:
// ${bound:<provision>.<key>}.
@@ -46,7 +47,7 @@ func boundUsed(content string) [][2]string {
// Three facts the mesh states about any provision, plus whatever the provider said it serves. A
// module may not reach a binding it does not have — the same boundary as a secret, for the same
// reason.
func knownFor(m Manifest, needs []Needed, node string) map[string]map[string]string {
func knownFor(m Manifest, needs []Needed, node string) (map[string]map[string]string, error) {
out := map[string]map[string]string{}
for _, want := range m.Wants() {
for i := range needs {
@@ -54,12 +55,20 @@ func knownFor(m Manifest, needs []Needed, node string) map[string]map[string]str
if n.Name != want || n.For != m.Module {
continue
}
as := ConsumerIdentity(node, IdentitySource(m.Slug, m.Module))
values := map[string]string{
"at": n.At,
"from": n.From,
"as": ConsumerIdentity(node, IdentitySource(m.Slug, m.Module)),
"as": as,
}
for key, value := range n.Serves {
// What the provider derives for this consumer rather than for all of them
// (novox/hq ADR 0201). Filled here, the one place a provision and the module
// requiring it are both in hand.
served, err := ServedTo(n.Serves, as)
if err != nil {
return nil, fmt.Errorf("%s requires %s: %w", m.Module, want, err)
}
for key, value := range served {
// The provider's own vocabulary. Rendered plainly: a port is 5432, not 5432.000000,
// which is what a float would write and what a connection string would refuse.
values[key] = plainly(value)
@@ -67,7 +76,7 @@ func knownFor(m Manifest, needs []Needed, node string) map[string]map[string]str
out[want] = values
}
}
return out
return out, nil
}
// withOwnNames adds a module's own composed names to what it may name from one binding:
@@ -232,3 +232,34 @@ func TestTwoModulesOnOneNodeAreTwoIdentities(t *testing.T) {
t.Fatal("one module on two machines shares an identity")
}
}
// A machine's resolver configuration must name its resolver by address — it cannot resolve the name
// of the thing it resolves names with (novox/hq ADR 0194). So a binding offers the providing
// machine's private address beside its name, and only when the machine has one.
func TestABindingOffersTheProvidersAddress(t *testing.T) {
consumer := func() Resolution {
return Resolution{
Node: "workstation",
Modules: []Manifest{{
Module: "resolv-conf",
Requires: []string{"wildcard-resolution"},
Resources: []map[string]any{{
"id": "resolv", "type": "file", "path": "/etc/resolv.conf", "mode": "0644",
"content": "nameserver ${bound:wildcard-resolution:address}\n",
}},
}},
Needs: []Needed{{Name: "wildcard-resolution", From: "anchor", At: "anchor.internal", For: "resolv-conf"}},
}
}
out, err := consumer().Declaration(Rendering{Machines: map[string]string{"anchor.internal": "10.77.0.1"}})
if err != nil {
t.Fatal(err)
}
if got := fileNamed(out, "resolv-conf.resolv")["content"]; got != "nameserver 10.77.0.1\n" {
t.Fatalf("the resolver is not named by its address: %q", got)
}
// A machine with no address yet: refused, never written with a blank where the address belongs.
if _, err := consumer().Declaration(Rendering{}); err == nil {
t.Fatal("a file naming an address the mesh does not have was composed")
}
}
+60 -1
View File
@@ -77,6 +77,7 @@ func (m Manifest) Resolve(built []Built) (Manifest, error) {
// build compare equal.
out.Bundles = nil
if m.Build != nil {
run := runByAResource(m)
for _, a := range m.Build.Artifacts {
if a.Kind != ArtifactBundle {
continue
@@ -84,9 +85,20 @@ func (m Manifest) Resolve(built []Built) (Manifest, error) {
made := by[a.Name]
// What the runtime loads: what the artifact said, else every entrypoint of a module
// that declares tools, else nothing (the field's own rule; see Artifact.Loads).
//
// **Never, unasked, a bundle one of the module's own resources runs** (novox/hq issue 213).
// A process the host runs is the module's service, not its tools: the controller declares
// the verbs it answers as `tools` and serves them itself, and its bundle would otherwise
// have been launched a second time by the node's runtime, as an MCP child it is not.
loads := append([]string(nil), a.Loads...)
if a.Loads == nil && len(m.Tools) > 0 {
if a.Loads == nil && len(m.Tools) > 0 && !run[a.Name] {
loads = append([]string(nil), a.Entrypoints...)
// A bundle compiled to a binary has no entrypoints: the binary is what it is, and what
// the runtime starts to serve it (novox/hq ADR 0193). So a Go tools bundle is served
// as Go — the runtime execs it — exactly as a TypeScript one is through its launcher.
if bin := BinaryOf(a); bin != "" {
loads = []string{bin}
}
}
// **Kept, never routed** (ADR 0155): the builder publishes to the store at the address
// it reached it by, and a manifest carrying that address names an installation —
@@ -208,6 +220,11 @@ func (b *Build) problems(module string) []string {
// A bundle's source is the module's own directory by definition, and what it needs to say
// is which compiler — because the mesh chooses that, and cannot choose for a module that
// has not said.
if len(a.External) > 0 && (a.Kind != ArtifactBundle || a.Language != "typescript") {
problems = append(problems, fmt.Sprintf(
"%s: %q names packages it keeps external, and only a TypeScript bundle is bundled into "+
"one file with some kept out (novox/hq ADR 0193)", module, a.Name))
}
if len(a.Env) > 0 && a.Kind != ArtifactBundle {
problems = append(problems, fmt.Sprintf(
"%s: %q is a %q and says what it is given (env). Only a bundle the node's runtime "+
@@ -242,6 +259,11 @@ func (b *Build) problems(module string) []string {
for _, e := range a.Entrypoints {
found = found || e == load
}
// A bundle compiled to a binary is one executable: the runtime loads that or nothing
// (novox/hq ADR 0193).
if bin := BinaryOf(a); bin != "" {
found = load == bin
}
if !found {
problems = append(problems, fmt.Sprintf(
"%s: %q says the runtime loads %q, which is not among its entrypoints — "+
@@ -436,3 +458,40 @@ func BinaryOf(a Artifact) string {
}
return a.Name
}
// runByAResource is the artifacts one of a module's own resources names — a process that runs it,
// a step, an archive that unpacks it — by name.
func runByAResource(m Manifest) map[string]bool {
named := map[string]bool{}
for _, r := range m.Resources {
if a, ok := r["artifact"].(string); ok && a != "" {
named[a] = true
}
}
return named
}
// undeliveredBundles says which of a module's bundles nothing would ever put on a machine (novox/hq
// 04-ISSUES/216). A bundle reaches a machine three ways: the node's runtime serves it (it says
// `loads`, or its module declares `tools`), a resource names it (a process, a step, an archive), or
// it is the runtime itself. One reached by none of them was built, recorded and pushed as success,
// and was simply absent — seven modules' tools went missing that way on 2026-10-03. Refused here,
// naming the field that would deliver it.
func undeliveredBundles(m Manifest) []string {
if m.Build == nil || m.Module == RuntimeModule {
return nil
}
named := runByAResource(m)
var problems []string
for _, a := range m.Build.Artifacts {
if a.Kind != ArtifactBundle || named[a.Name] || len(a.Loads) > 0 || len(m.Tools) > 0 {
continue
}
problems = append(problems, fmt.Sprintf(
"%s: the bundle %q would be built and never reach a machine: nothing loads it, runs it or "+
"unpacks it. A tools bundle says `loads` (the entrypoints the node's runtime serves) or its "+
"module lists its `tools`; a daemon or a step is a resource naming it (novox/hq 04-ISSUES/216)",
m.Module, a.Name))
}
return problems
}
@@ -86,3 +86,36 @@ func TestNoCatalogueManifestNamesAnInstallation(t *testing.T) {
t.Fatalf("%d value(s) name an installation:\n %s", len(named), strings.Join(named, "\n "))
}
}
// TestEveryCatalogueStoreSaysHowItIsBackedUp is ADR 0214's check over the real catalogue: a module
// providing a store contributes a backup to node-backup, so a store added is a store backed up.
func TestEveryCatalogueStoreSaysHowItIsBackedUp(t *testing.T) {
root := catalogueRoot(t)
found, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
if err != nil || len(found) == 0 {
t.Fatalf("no manifests under %s: %v", root, err)
}
stores := 0
for _, p := range found {
raw, err := os.ReadFile(p)
if err != nil {
t.Fatalf("%s: %v", p, err)
}
m, err := ParseManifest(raw)
if err != nil {
continue // TestEveryCatalogueManifestParses says why
}
for _, o := range m.Provides {
if storeProvisions[o.Name] {
stores++
break
}
}
for _, problem := range CheckBackup(m) {
t.Error(problem)
}
}
if stores == 0 {
t.Fatal("no module in the catalogue provides a store, so this proved nothing")
}
}
+311
View File
@@ -0,0 +1,311 @@
package catalogue
import (
"fmt"
"regexp"
"sort"
"strings"
)
// What a provider derives for one consumer, said once in the provider's definition and delivered
// to both ends (novox/hq ADR 0201, issue 124).
//
// A `serves` block is otherwise literal: the same values for every consumer. Where the provider
// *names the resource* — a bucket, a database, a vhost — the name is derived from who is asking,
// and before this the mesh had no channel for it. The provider recomputed it in its own code and
// every consumer transcribed it into its own definition by hand, which is a copy of somebody
// else's rule kept in agreement by nobody. One of three transcriptions was wrong for months.
//
// **The mesh learns no protocol here; it spells its own name in an alphabet it already knows.**
// The only fact a served value may name is the identity the mesh itself minted for the consumer,
// in one of two alphabets: as it was minted, and as a DNS label. Everything a provider wants
// around it — a prefix, a suffix, a separator — it writes around the placeholder, because a
// served value is a string.
// consumerFact is `${consumer:<fact>}` or `${consumer:<fact>:<alphabet>}`.
var consumerFact = regexp.MustCompile(`\$\{consumer:([a-z][a-z0-9-]*)(?::([a-z][a-z0-9-]*))?\}`)
// consumerFacts are what a served value may name about the consumer it is being derived for.
// One entry, deliberately: the identity is the one thing about a consumer the mesh itself chose,
// so it is the one thing the mesh can hand to a provider without either end guessing.
var consumerFacts = []string{"as"}
// consumerAlphabets are the ways the mesh will write that identity. `dns` is the mesh's own
// identifier with its separator written `-` instead of `_` — the whole of the difference between
// the alphabet the mesh mints in and the one buckets, vhosts and hostnames accept.
var consumerAlphabets = []string{"dns"}
// ServedTo fills a provider's served values for one consumer.
//
// `as` is the identity the mesh minted for that consumer — the same string it is told to present
// as a login. Values with no placeholder are returned exactly as they were, and a block with no
// placeholder at all is returned unchanged, so this costs nothing for the providers that derive
// nothing.
//
// Only strings carry placeholders. A number, a boolean or a nested object is a value the provider
// stated outright, and is left alone.
func ServedTo(serves map[string]any, as string) (map[string]any, error) {
if len(serves) == 0 {
return serves, nil
}
var out map[string]any
for _, key := range sortedAnyKeys(serves) {
text, ok := serves[key].(string)
if !ok || !strings.Contains(text, "${consumer:") {
continue
}
filled, err := consumerInto(text, as)
if err != nil {
return nil, fmt.Errorf("the value served as %q: %w", key, err)
}
if out == nil {
// Copied only once something actually changes: the caller's map is the manifest's,
// and a provider that derives nothing must not have it rewritten underneath it.
out = make(map[string]any, len(serves))
for k, v := range serves {
out[k] = v
}
}
out[key] = filled
}
if out == nil {
return serves, nil
}
return out, nil
}
// consumerInto replaces every `${consumer:…}` in one value.
//
// **A fact or an alphabet the mesh does not have is refused, not left standing.** Written through,
// the literal `${consumer:as}` would reach a configuration file and be read as a bucket name,
// failing somewhere that names neither the module nor the mesh — the same reasoning `${bound:…}`
// is refused by (boundInto).
func consumerInto(value, as string) (string, error) {
var failed error
out := consumerFact.ReplaceAllStringFunc(value, func(match string) string {
parts := consumerFact.FindStringSubmatch(match)
fact, alphabet := parts[1], parts[2]
if fact != "as" {
if failed == nil {
failed = fmt.Errorf(
"says %s, and the mesh states %s about a consumer", match, orNothing(consumerFacts))
}
return match
}
switch alphabet {
case "":
return as
case "dns":
return asDNSLabel(as)
default:
if failed == nil {
failed = fmt.Errorf(
"says %s, and the mesh writes an identity as %s", match, orNothing(consumerAlphabets))
}
return match
}
})
if failed != nil {
return "", failed
}
return out, nil
}
// asDNSLabel writes a minted identity as a DNS label.
//
// The mesh's identities are already lower-case letters, digits and `_` (ConsumerIdentity), and
// already short enough for the tightest backend they reach (CheckIdentity, twenty characters). So
// this is the separator and nothing else — no lower-casing of what is already lower case, no
// truncation to a limit the identity is already inside, no padding of a name that is already long
// enough. Each of those would be the mesh guessing at a rule it has not been given.
func asDNSLabel(as string) string {
return strings.ReplaceAll(as, "_", "-")
}
// CheckServes refuses a `serves` block that names a consumer fact or an alphabet the mesh does not
// have, when the definition is parsed rather than when a consumer is resolved.
//
// A provision nobody consumes yet still has its rule read: a definition that would be refused the
// first time somebody required it is a definition that is wrong now.
func CheckServes(m Manifest) []string {
var problems []string
for _, provision := range sortedServes(m.Serves) {
for _, key := range sortedAnyKeys(m.Serves[provision]) {
text, ok := m.Serves[provision][key].(string)
if !ok {
continue
}
// A probe identity, because what is checked is the shape of the statement and not
// what any consumer is called.
if _, err := consumerInto(text, "mesh_node_module"); err != nil {
problems = append(problems, fmt.Sprintf(
"%s serves %s, and the value it serves as %q %s", m.Module, provision, key, err))
}
}
}
return problems
}
func sortedServes(serves map[string]map[string]any) []string {
out := make([]string, 0, len(serves))
for k := range serves {
out = append(out, k)
}
sort.Strings(out)
return out
}
func sortedAnyKeys(values map[string]any) []string {
out := make([]string, 0, len(values))
for k := range values {
out = append(out, k)
}
sort.Strings(out)
return out
}
// derivedFor is what the provider on this machine derives for one consumer of one provision
// (novox/hq ADR 0201).
//
// Settled first, then derived: an operator may set a prefix on what the provider serves and the
// mesh still fills the consumer's half of it ([ADR 0174]). Only the keys that actually name the
// consumer are returned — the rest of a `serves` block is the same for every consumer and is
// already in the provider's own definition, so repeating it here would be a second copy to go
// stale.
//
// The first module in the resolved order that says it serves the provision answers, which is the
// choice servedOnThisMachine makes for the consumer's half. Nothing serving it on this machine is
// not an error: a contribution can reach a machine whose provider is a record or an adapter, and
// then there is nothing derived to tell.
func (r Resolution) derivedFor(provision, as, consumer, local string, settings SettingsBy) (map[string]any, error) {
for _, m := range r.Modules {
serves, said := m.Serves[provision]
if !said {
continue
}
var names map[string]any
for key, value := range serves {
if text, ok := value.(string); ok && strings.Contains(text, "${consumer:") {
if names == nil {
names = map[string]any{}
}
names[key] = value
}
}
if names == nil {
return nil, nil
}
// **A consumer that keeps several holders of this provision is refused** — this is issue
// 124's own failure one case to the side, and it would be just as quiet.
//
// Each holder gets its own login, `…_<local>` (ADR 0094), and a provider derives from the
// login, so it would make one resource per holder. The consumer's side has no such
// dimension: one binding file per provision, one `${bound:<provision>:<key>}`, both
// derived from the un-suffixed identity. So the provider would create the holder's
// resource and the consumer would be configured against a name nothing made — it would
// authenticate successfully and be refused on every object, which reads like a credential
// fault and is not one.
//
// Lifting this means giving the consumer's side a local dimension. That is a decision,
// not an omission, and until it is taken the mesh says so rather than guessing.
if local != "" {
return nil, fmt.Errorf(
"%s keeps several holders of %s (this one is %q), and %s derives %s for each "+
"consumer from the login the mesh minted. Each holder has its own login, and a "+
"consumer is told one value per requirement — so the two ends would name "+
"different things and nothing would compare them (novox/hq ADR 0201)",
consumer, local, provision, m.Module, orNothing(sortedAnyKeys(names)))
}
settled, err := Settle(names, settings[m.Module])
if err != nil {
return nil, fmt.Errorf("%s serving %s: %w", m.Module, provision, err)
}
derived, err := ServedTo(settled, as)
if err != nil {
return nil, fmt.Errorf("%s serving %s to %s: %w", m.Module, provision, as, err)
}
return derived, nil
}
return nil, nil
}
// notTranscribed refuses a consumer's file that writes out the value its provider derives for it,
// instead of asking for it (novox/hq ADR 0201, issue 124).
//
// **What would have caught the one wrong instance.** The object store's three consumers each wrote
// their bucket into their own configuration by hand. One of them named a predecessor's bucket, and
// nothing compared it to what the provider would actually create: the module would have
// authenticated successfully and been refused on every object, which reads like a credential fault
// and is not one. It looked authoritative for months.
//
// The test is exact and costs one string search: a definition whose file already contains the
// value the mesh is about to derive for it has written down somebody else's rule. It cannot be a
// coincidence — a derived value carries the identity the mesh minted for this very consumer on
// this very machine, which nothing else would spell out — and it cannot be checked afterwards,
// because after substitution every consumer's file contains it legitimately.
//
// Only values that actually name the consumer are judged. A provider that serves a constant under
// the same key serves the same constant to everyone, and a consumer repeating it is redundant
// rather than wrong.
func notTranscribed(resource map[string]any, known map[string]map[string]string, module string) error {
if fmt.Sprint(resource["type"]) != "file" {
return nil
}
content, ok := resource["content"].(string)
if !ok || content == "" {
return nil
}
for _, provision := range sortedKnown(known) {
values := known[provision]
identity := values["as"]
if identity == "" {
continue
}
for _, key := range sortedStringKeys(values) {
if key == "as" {
// The login is not derived from itself, and a consumer that must present it in a
// connection string legitimately has it from `${bound:…}` — which is what it will
// be after substitution, so this would judge the substitution, not the module.
continue
}
value := values[key]
if value == "" || !namesTheConsumer(value, identity) {
continue
}
if !strings.Contains(content, value) {
continue
}
return fmt.Errorf(
"%s writes %q into %v, and that is exactly what %s derives for it — a definition "+
"keeping its own copy of somebody else's naming rule is one that can disagree "+
"with it, silently. Say ${bound:%s:%s} and be told",
module, value, resource["id"], provision, provision, key)
}
}
return nil
}
// namesTheConsumer is whether a derived value was built from this consumer's identity — in the
// alphabet it was minted in, or as a DNS label. A value that does not contain it was not derived
// from it, whatever else it may be.
func namesTheConsumer(value, identity string) bool {
return strings.Contains(value, identity) || strings.Contains(value, asDNSLabel(identity))
}
func sortedKnown(known map[string]map[string]string) []string {
out := make([]string, 0, len(known))
for k := range known {
out = append(out, k)
}
sort.Strings(out)
return out
}
func sortedStringKeys(values map[string]string) []string {
out := make([]string, 0, len(values))
for k := range values {
out = append(out, k)
}
sort.Strings(out)
return out
}
@@ -0,0 +1,121 @@
package catalogue
import (
"reflect"
"strings"
"testing"
)
// Defends novox/hq ADR 0210 §3: a contribution is a dependency on the seat that receives it.
func TestAContributionDependsOnTheSeatThatPlacesIt(t *testing.T) {
env := mod("theme", nil, nil, nil)
env.Environment = &Environment{Variables: map[string]string{"GTK_THEME": "Adwaita:dark"}}
path := mod("toolchain", nil, nil, nil)
path.Environment = &Environment{Path: []PathEntry{{Entry: "/opt/x/bin"}}}
shell := mod("prompt", nil, nil, nil)
shell.Shell = []ShellCode{{For: "zsh", Slot: "first", Code: "true"}}
session := mod("wallpaper", nil, nil, nil)
session.Shell = []ShellCode{{For: "xinitrc", Slot: "normal", Code: "true"}}
resources := mod("bar", nil, nil, nil)
resources.Shell = []ShellCode{{For: "xresources", Slot: "normal", Code: "x: y"}}
empty := mod("nothing", nil, nil, nil)
empty.Environment = &Environment{}
cases := map[string]struct {
m Manifest
want []string
}{
"a variable": {env, []string{EnvironmentSeat}},
"a path entry": {path, []string{EnvironmentSeat}},
"shell code": {shell, []string{LoginShellSeat}},
"the session's start": {session, []string{DisplayServerSeat}},
"the session's X resources": {resources, []string{DisplayServerSeat}},
"an empty environment": {empty, nil},
}
for name, c := range cases {
got := DependsOn(c.m)
if len(got) == 0 {
got = nil
}
if !reflect.DeepEqual(got, c.want) {
t.Errorf("%s depends on %v, want %v", name, got, c.want)
}
}
}
func TestAContributionIsMetByAHolderOnTheNodeAndRefusedWithout(t *testing.T) {
holder := mod("node-env", nil, nil, nil, Claim{Name: EnvironmentSeat})
contributor := mod("theme", nil, nil, nil)
contributor.Environment = &Environment{Variables: map[string]string{"GTK_THEME": "Adwaita:dark"}}
catalogue := map[string]Manifest{"node-env": holder, "theme": contributor}
if _, err := AssignRefusal(catalogue, "laptop", []string{"node-env"}, []string{"theme"}); err != nil {
t.Fatalf("a contributor beside the holder is refused: %v", err)
}
_, err := AssignRefusal(catalogue, "laptop", nil, []string{"theme"})
if err == nil {
t.Fatal("a contributor on a node without the holder was accepted, and its contribution would be written nowhere")
}
if !strings.Contains(err.Error(), EnvironmentSeat) || !strings.Contains(err.Error(), "node-env") {
t.Errorf("the refusal names neither the seat nor its holder: %v", err)
}
if _, err := AssignRefusal(catalogue, "laptop", nil, []string{"theme", "node-env"}); err != nil {
t.Errorf("the contributor and the holder assigned together are refused: %v", err)
}
}
func TestAHolderMeetsItsOwnContribution(t *testing.T) {
// The display server's module contributes nothing to its own seat today, but the zsh module's
// environment contributions do go to another seat: a claim meets only the seat it names.
zsh := mod("zsh", nil, nil, nil, Claim{Name: LoginShellSeat})
zsh.Shell = []ShellCode{{For: "zsh", Slot: "normal", Code: "true"}}
zsh.Environment = &Environment{Variables: map[string]string{"EDITOR": "vim"}}
catalogue := map[string]Manifest{"zsh": zsh,
"node-env": mod("node-env", nil, nil, nil, Claim{Name: EnvironmentSeat})}
unheld := UnheldDependencies(catalogue, "laptop", []Manifest{zsh}, nil)
if len(unheld) != 1 || unheld[0].Seat != EnvironmentSeat {
t.Errorf("zsh alone: unheld %v, want only %s (its shell code is its own seat's)", unheld, EnvironmentSeat)
}
}
func TestTwoModulesDeclaringOnePackageAreRefusedBeforeAnythingIsRecorded(t *testing.T) {
pacman := withResources(mod("pacman", nil, nil, nil, Claim{Name: PackageManagerSeat}),
res("package", "pacman-contrib"))
bar := withResources(mod("bar", nil, nil, nil), res("package", "bar"), res("package", "pacman-contrib"))
other := withResources(mod("other", nil, nil, nil), res("package", "other"))
catalogue := map[string]Manifest{"pacman": pacman, "bar": bar, "other": other}
err := CollisionRefusal(catalogue, "laptop", []string{"pacman"}, []string{"bar"})
if err == nil || !strings.Contains(err.Error(), "pacman-contrib") || !strings.Contains(err.Error(), "bar") {
t.Fatalf("the second owner of a package was not refused by name: %v", err)
}
if err := CollisionRefusal(catalogue, "laptop", []string{"pacman"}, []string{"other"}); err != nil {
t.Errorf("a module declaring nothing shared is refused: %v", err)
}
// A collision already on the node is status's, not a reason to refuse an unrelated assignment.
if err := CollisionRefusal(catalogue, "laptop", []string{"pacman", "bar"}, []string{"other"}); err != nil {
t.Errorf("an unrelated assignment is refused for a collision already there: %v", err)
}
}
func TestCodeForAPowerMomentDependsOnThePowerSeat(t *testing.T) {
for _, moment := range powerMoments {
m := mod("laptop", nil, nil, nil)
m.Shell = []ShellCode{{For: moment, Slot: "normal", Code: "true"}}
if got := DependsOn(m); !reflect.DeepEqual(got, []string{PowerSeat}) {
t.Errorf("code for %s depends on %v, want %s", moment, got, PowerSeat)
}
if p := m.shellProblems(); len(p) != 0 {
t.Errorf("code for %s is refused: %v", moment, p)
}
}
m := mod("laptop", nil, nil, nil)
m.Shell = []ShellCode{{For: "after-lunch", Slot: "normal", Code: "true"}}
if p := m.shellProblems(); len(p) == 0 {
t.Error("code for a moment that does not exist was accepted")
}
if s, ok := SeatNamed(PowerSeat); !ok || s.Scope != ScopeNode {
t.Errorf("%s is not a node seat of the mesh's own: %+v %v", PowerSeat, s, ok)
}
}
@@ -0,0 +1,131 @@
package catalogue
import (
"encoding/json"
"fmt"
"os"
"strings"
"testing"
)
// novox/hq issue 213: the controller is a Go program and was the one piece of the mesh's own Go
// code still shipped as an image (ADR 0188 §1). Its own manifest, composed for the machine that runs
// it, is a Go bundle run by the host as a process — and no container.
func TestTheControllerIsAProcessAndNoContainer(t *testing.T) {
raw, err := os.ReadFile("../../module.json")
if err != nil {
t.Fatal(err)
}
m, err := ParseManifest(raw)
if err != nil {
t.Fatalf("the controller's own manifest does not parse:\n%v", err)
}
if m.Build == nil || len(m.Build.Artifacts) != 1 {
t.Fatalf("the controller builds %+v; it is one bundle", m.Build)
}
a := m.Build.Artifacts[0]
if a.Kind != ArtifactBundle || a.Language != "go" || a.System == "" || BinaryOf(a) != "mesh-controller" {
t.Fatalf("the controller's artifact is %+v, not a Go bundle naming its system and binary", a)
}
for _, c := range m.Capabilities {
if c == "container-runtime" {
t.Error("the controller still requires a container runtime on its machine")
}
}
digest := "sha256:" + strings.Repeat("c", 64)
control, err := m.Resolve([]Built{{Name: a.Name, Kind: ArtifactBundle,
Reference: ArtifactStoreScheme + "mesh-controller/" + a.Name + "@" + digest, Digest: digest}})
if err != nil {
t.Fatal(err)
}
// The node's runtime does not launch it: it serves its seat's verbs itself.
if loads := control.Bundles[0].Loads; len(loads) != 0 {
t.Errorf("the node's runtime would launch the controller as a tools bundle: %v", loads)
}
needed := map[string]map[string]string{"mesh-controller": {}}
for name := range m.OwnSecrets {
needed["mesh-controller"][name] = "sealed-" + name
}
out, err := Resolution{Node: "anchor", Modules: []Manifest{control}}.Declaration(Rendering{
Needed: needed, ArtifactStore: "anchor.internal:5100",
Seats: map[string]map[int]int{"mesh-store": {5432: 6852}},
})
if err != nil {
t.Fatalf("the controller does not compose: %v", err)
}
var process, step map[string]any
account, firstSecret := -1, -1
for i, r := range out {
switch {
case r["type"] == "container":
t.Errorf("the controller's declaration still runs a container: %v", r)
case r["id"] == "mesh-controller.controller":
process = r
case r["id"] == "mesh-controller.controller-prepare":
step = r
if process != nil {
t.Error("the controller's preparation is placed after the process it prepares for")
}
case r["type"] == "user" && r["name"] == "mesh-controller":
account = i
case strings.HasPrefix(fmt.Sprint(r["id"]), "mesh-controller.needs-") && firstSecret < 0:
firstSecret = i
}
}
if process == nil {
t.Fatalf("the controller's process is not in its declaration: %v", out)
}
if run, _ := json.Marshal(process["run"]); string(run) != `["./mesh-controller","serve"]` {
t.Errorf("the controller is run as %s, not its own bundle's binary", run)
}
if process["source"] != "anchor.internal:5100/mesh-controller/"+a.Name+"@"+digest || process["digest"] != digest {
t.Errorf("the controller's bundle is fetched from %v (%v)", process["source"], process["digest"])
}
// The user: an account the host declares, which owns what the process reads.
if process["user"] != "mesh-controller" || account < 0 {
t.Errorf("the controller runs as %v, and the account declared is at %d", process["user"], account)
}
if firstSecret >= 0 && account > firstSecret {
t.Error("the controller's secrets are written before the account they belong to exists")
}
for _, r := range out {
if strings.HasPrefix(fmt.Sprint(r["id"]), "mesh-controller.needs-") && r["owner"] != "mesh-controller" {
t.Errorf("%v belongs to %v, which the controller's process cannot read", r["id"], r["owner"])
}
}
if dir := fileNamed(out, "mesh-controller.mesh-state"); dir == nil || dir["owner"] != "mesh-controller" {
t.Errorf("the controller's state directory is not its account's to enter: %v", dir)
}
// Each mount became a path the process reads: nothing it is told is a path inside a container.
state := fmt.Sprint(fileNamed(out, "mesh-controller.mesh-state")["path"])
env, _ := process["env"].(map[string]any)
for key, value := range env {
v := fmt.Sprint(value)
if strings.HasPrefix(v, "/run/secrets") || strings.HasPrefix(v, "/broker-tls") {
t.Errorf("%s=%s is a path inside the container the controller no longer runs in", key, v)
}
if strings.HasSuffix(key, "_FILE") && !strings.HasPrefix(v, state+"/") {
t.Errorf("%s=%s is not one of the files the mesh places for it", key, v)
}
}
if env["MESH_BROKER_CERTIFICATE"] != "/var/lib/mesh-broker-tls/tls.crt" {
t.Errorf("the controller reads the broker's certificate from %v", env["MESH_BROKER_CERTIFICATE"])
}
if env["MESH_STORE_INVENTORY_PORT"] != "6852" {
t.Errorf("the controller is told the store is on %v; the node put it on 6852", env["MESH_STORE_INVENTORY_PORT"])
}
// The handover: the container it ran as goes only once this is running.
if got, _ := json.Marshal(process["replaces"]); string(got) != `["mesh-controller.server"]` {
t.Errorf("the controller's process replaces %s, not the container it ran as", got)
}
// And its state is prepared first, by the same program as the same account.
if step == nil || step["run-once"] != true || step["user"] != "mesh-controller" {
t.Fatalf("the controller's preparation is %v", step)
}
if run, _ := json.Marshal(step["run"]); string(run) != `["./mesh-controller","prepare"]` {
t.Errorf("the controller's preparation runs %s", run)
}
}
+164 -12
View File
@@ -155,6 +155,10 @@ type Rendering struct {
// standing beside the machines and looking as real as they do.
Machines map[string]string
// Zones is every zone a module in the mesh answers itself, where it is answered (novox/hq ADR
// 0199): the mesh's resolver forwards each one there.
Zones []ZoneAt
Settings SettingsBy
Generators map[string]Generator
// Grants are the credentials this node must create, for the provisions it offers. Passed in
@@ -415,6 +419,13 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
filtering := AsNftables(rules, with.Mesh, r.PublicDomain != "", with.Foundation,
with.OutwardLinks, with.TunnelInterface)
// **A variable two modules set is refused whether or not anything places it** (novox/hq ADR
// 0203 §5): the account has one environment, and a machine whose holder arrives later should not
// be the moment two modules are found to disagree about it.
if err := variablesSetOnce(r.Modules); err != nil {
return nil, err
}
var out []map[string]any
for _, m := range r.Modules {
if with.Adopted && m.Filtering != nil {
@@ -603,14 +614,14 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
// and nothing would say so.
continue
}
first = append(first, map[string]any{
first = append(first, ownedBy(r.provisionsAs(m), map[string]any{
// One file per holder — the consumer's module with its local name after it
// where it keeps several (ADR 0094); the lab found two files with one id.
"id": GrantID(to, g.Consumer+"."+holderAs(g.From, g.Local)),
"type": "file",
"path": grantPath(m.Grants[to], g.Consumer, holderAs(g.From, g.Local)),
"sealed": g.Sealed,
})
}))
}
}
for _, to := range sortedKeys(m.Binds) {
@@ -645,7 +656,16 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
if err != nil {
return nil, err
}
file, err := boundFile(*found, m.Binds[to], ConsumerIdentity(r.Node, IdentitySource(m.Slug, m.Module)), own)
as := ConsumerIdentity(r.Node, IdentitySource(m.Slug, m.Module))
// What the provider derives for THIS consumer, filled here where the consumer is
// known (novox/hq ADR 0201). The same fill knownFor does below, so the binding file
// and the module's `${bound:…}` substitutions cannot say different things.
told := *found
told.Serves, err = ServedTo(told.Serves, as)
if err != nil {
return nil, fmt.Errorf("%s is told about %s: %w", m.Module, to, err)
}
file, err := boundFile(told, m.Binds[to], as, own)
if err != nil {
return nil, err
}
@@ -693,7 +713,15 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
// Now, and not before: a module whose resources are computed replaces them wholesale, and
// merging earlier would throw away the files it still needs.
resources = append(append([]map[string]any{}, first...), resources...)
//
// **Except the module's own accounts, which go before even those** (novox/hq issue 213). What
// the mesh computes may belong to one: a module whose code runs as an account it declares has
// its secrets written owned by that account, and a file given to a user the machine does not
// have yet fails — so on the first apply the secrets were refused, the process started without
// them, and the second apply healed it, which is the fault the paragraph above describes.
// An account depends on nothing the mesh computes.
accounts, rest := accountsFirst(resources)
resources = append(append(accounts, first...), rest...)
// No container is given the mesh's names (novox/hq ADR 0148). It used to be: every
// container got the whole roster as `--add-host` entries at creation, and a name that
@@ -711,7 +739,10 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
return nil, err
}
// And what its bindings say, for the half of a connection that is not secret.
known := knownFor(m, r.Needs, r.Node)
known, err := knownFor(m, r.Needs, r.Node)
if err != nil {
return nil, err
}
// A requirement answered on this same machine is not in r.Needs — its binding file is
// written from `here` (above) — and so `${bound:…}` could not name it, though the file
// beside it said the same facts. Filled from the same answer, so the two cannot disagree.
@@ -728,10 +759,24 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
}
local := *answered
local.For = m.Module
for provision, values := range knownFor(m, []Needed{local}, r.Node) {
here, err := knownFor(m, []Needed{local}, r.Node)
if err != nil {
return nil, err
}
for provision, values := range here {
known[provision] = values
}
}
// And the providing machine's private address, beside its name (novox/hq ADR 0194). A name is
// what nearly every consumer wants; the one that cannot use it is a machine's resolver
// configuration, which must reach the resolver before it can resolve anything — the resolver's
// own name included. Absent when the machine has no address yet, so a file naming it is refused
// rather than written with a blank where an address belongs.
for _, values := range known {
if address := with.Machines[values["at"]]; address != "" {
values["address"] = address
}
}
// And what the module is called through each requirement it contributes to (novox/hq
// 04-ISSUES/122) — the same composition its binding file carries.
for provision, values := range known {
@@ -753,6 +798,17 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
// And the machine underneath, which no binding of its own can tell it.
thisMachine := machineFacts(r, with.Names, with.MeshRange)
// **A definition that already holds the answer transcribed it** (novox/hq ADR 0201).
// Judged over what the module itself declares, and before anything is substituted: the
// mesh's own generated files — the binding, the contributions — legitimately carry the
// derived value, and after substitution so does every consumer's file, so this is the one
// moment the two can be told apart.
for _, own := range m.Resources {
if err := notTranscribed(own, known, m.Module); err != nil {
return nil, err
}
}
// Which of this module's files carry a secret, for the rule that a container may not read
// one of them as its environment without saying so (ADR 0086, issue 041).
secretFiles := secretFilesOf(resources)
@@ -854,6 +910,13 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
return nil, err
}
publishedOn(copied, m.Module, with)
// The account's environment and every module's shell code, where this module holds the
// seat that places them (novox/hq ADR 0203, ADR 0204). Gathered from every module on
// the node, as the jails are, and **last of every placeholder pass**: shell code is a
// shell's own syntax, full of `${…}` no pass above should ever be shown.
if err := contributionsInto(copied, m, r.Modules, thisMachine, with); err != nil {
return nil, err
}
copied["id"] = m.Module + "." + fmt.Sprint(resource["id"])
// A service saying what it reflects names resources within its own module, so those
// are prefixed too or they would point at nothing.
@@ -872,6 +935,21 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
if renamed := reflectsRenamed(m.Module, resource["reload-on"]); renamed != nil {
copied["reload-on"] = renamed
}
// And which of its module's containers a scheduled step holds still (novox/hq ADR 0189).
// **The loudest of the three when it is missed.** An unprefixed `restart-on` matches
// nothing and a service quietly never restarts; an unprefixed `while-stopped` names a
// container the declaration does not contain, and the host refuses the whole
// declaration — so the machine takes nothing at all, for every push, until this is
// right. That is what it did on the control node (2026-10-04).
if renamed := reflectsRenamed(m.Module, resource[WhileStopped]); renamed != nil {
copied[WhileStopped] = renamed
}
// And what a process replaces (novox/hq issue 213): a resource of this module's that it
// no longer declares, named as the host recorded it, or the host hands nothing over and
// removes it first.
if renamed := reflectsRenamed(m.Module, resource["replaces"]); renamed != nil {
copied["replaces"] = renamed
}
// **What reads one of this module's own secrets is restarted when it changes** (novox/hq
// issue 203, issue 206). A credential is re-issued by the mesh, and a container that
// mounted the old file keeps the old one open: the build machine ran for an hour on a
@@ -901,7 +979,7 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
// plane's; making a name resolve is the module's software. Emitted as ordinary files under
// this module's name, so they are applied, reported and removed exactly as anything else
// it declares.
given, err := FactsInto(m, r, with.Names, with.Machines, with.Accounts, with.Suffix)
given, err := FactsWithZonesInto(m, r, with.Names, with.Machines, with.Accounts, with.Suffix, with.Zones)
if err != nil {
return nil, err
}
@@ -1161,6 +1239,41 @@ type Contribution struct {
// requirement's name — everything providing `reverse-proxy` understands the same shape, which
// is what makes swapping one for another cost nothing.
Values map[string]any `json:"values"`
// Derived is what this provider's own definition said it derives for this consumer, already
// derived (novox/hq ADR 0201).
//
// **The provider is told, rather than recomputing it.** A served value may name the consumer's
// identity — a bucket named for who is asking, a database prefixed with it — and before this
// the rule lived twice: once in the provisioner's code, once transcribed into every consumer's
// definition. The mesh fills the provider's own statement here and delivers the same filled
// value to the consumer, so the two cannot disagree: there is no second computation to
// disagree with.
//
// Only the keys that are per-consumer. The rest of what the provider serves is the same for
// everyone and is in its own definition, where it already is.
Derived map[string]any `json:"derived,omitempty"`
}
// provisionsAs is the account that reads what the mesh writes for this provider: the one secret
// per consumer it must open to set that consumer's password (novox/hq issue 225).
//
// **A root-owned 0600 file is one that process cannot read**, which is the same sentence already
// written above for a module's own secrets — and the grant secret is the other kind of secret
// the mesh writes for a module, so it is the same rule.
//
// Which account depends on where the module's code runs. A module whose code is a bundle is run
// by the node's tool runtime, as the node's account ([ADR 0198](0198)); one still in a container
// is whatever it declares as its secrets owner. Nothing names these paths, so the rule that
// claims a bundle's other files by the words that name them (givenTo) cannot reach them: the
// harness composes a grant secret's path from the contributions file, not from a word.
//
// Empty is root, which is what it was and what a module with no bundle and no declared owner
// still wants.
func (r Resolution) provisionsAs(m Manifest) string {
if len(m.Bundles) > 0 && r.Account != "" {
return r.Account
}
return m.SecretsOwner
}
// grantPath is where one consumer's sealed credential lands on the providing machine.
@@ -1252,12 +1365,17 @@ func (r Resolution) contributions(settings SettingsBy, grants []Grant,
// told about it and withdraws the login on its next pass.
continue
}
as := holderAs(ConsumerIdentity(g.Consumer, IdentitySource(g.Slug, g.From)), g.Local)
derived, err := r.derivedFor(g.Provision, as, g.From, g.Local, settings)
if err != nil {
return nil, err
}
out[g.Provision] = append(out[g.Provision], Contribution{
From: g.From, Node: g.Consumer, At: g.At, Values: g.Values,
From: g.From, Node: g.Consumer, At: g.At, Values: g.Values, Derived: derived,
// One holder per local name: the identity the consumer is known by, and the local name
// after it where the module keeps several (ADR 0094). Not a login any backend checks —
// a secret is not a login — so the identity limit does not apply to the suffix.
As: holderAs(ConsumerIdentity(g.Consumer, IdentitySource(g.Slug, g.From)), g.Local),
As: as,
Secret: grantPath(directories[g.Provision], g.Consumer, holderAs(g.From, g.Local)),
})
if granted[g.Provision] == nil {
@@ -2010,12 +2128,18 @@ func preparationTarget(m Manifest) string {
return ""
}
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) != "container" || !ownArtifact(r, m.Module) {
// A container, or a process the host runs from a bundle the module built (novox/hq issue
// 213): the same program in the same context, hosted as a unit rather than a container.
kind := fmt.Sprint(r["type"])
if (kind != "container" && kind != "process") || !ownArtifact(r, m.Module) {
continue
}
if once, _ := r["run-once"].(bool); once {
continue
}
if r["schedule"] != nil {
continue
}
return fmt.Sprint(r["id"])
}
return ""
@@ -2029,7 +2153,10 @@ func ownArtifact(resource map[string]any, module string) bool {
return true
}
image, _ := resource["image"].(string)
return strings.HasPrefix(image, ArtifactStoreScheme+module+"/")
// A process or an archive carries what was built as its source (novox/hq issue 213).
source, _ := resource["source"].(string)
return strings.HasPrefix(image, ArtifactStoreScheme+module+"/") ||
strings.HasPrefix(source, ArtifactStoreScheme+module+"/")
}
// prepared is the module's own resource as the step that prepares its state: the same image, the same
@@ -2051,7 +2178,19 @@ func prepared(from map[string]any) map[string]any {
step["id"] = fmt.Sprint(from["id"]) + "-prepare"
step["name"] = fmt.Sprint(from["name"]) + "-prepare"
step["run-once"] = true
step["args"] = []any{PreparationArgument}
if fmt.Sprint(from["type"]) == "process" {
// A process says its whole command: the program, then its arguments. The step is the same
// program asked to prepare (novox/hq issue 213). It replaces nothing — what the process
// replaces is handed over to the process, never to the step that runs before it — and a
// step is not restarted, it runs again when what it reads changed, which `restart-on` says.
run := stringsIn(from["run"])
if len(run) > 0 {
step["run"] = []any{run[0], PreparationArgument}
}
delete(step, "replaces")
} else {
step["args"] = []any{PreparationArgument}
}
delete(step, "ports")
delete(step, "ip")
delete(step, "schedule")
@@ -2196,3 +2335,16 @@ func withRestartOn(have any, add []string) []any {
}
return out
}
// accountsFirst splits a module's resources into its accounts and everything else, each in the order
// written.
func accountsFirst(resources []map[string]any) (accounts, rest []map[string]any) {
for _, r := range resources {
if fmt.Sprint(r["type"]) == "user" {
accounts = append(accounts, r)
continue
}
rest = append(rest, r)
}
return accounts, rest
}
@@ -0,0 +1,343 @@
package catalogue
import (
"encoding/json"
"strings"
"testing"
)
// What a provider derives for each consumer, said once and delivered to both ends
// (novox/hq ADR 0201, issue 124).
//
// The failure these are written against: the object store's provisioner derived each consumer's
// bucket from the login the mesh minted, in its own code, and the mesh had no channel to tell the
// consumer which bucket that was — so all three consumers wrote the answer into their own
// definitions by hand. Two were right. One named a predecessor's bucket and would have
// authenticated successfully and been refused on every object. Each of them also named the
// machine the module happens to run on, which a definition may not do.
// store is an object store in the shape minio has: it serves a region and a port to everyone, and
// a bucket named for whoever is asking.
func store() Manifest {
return Manifest{
Module: "store", Version: "1",
Provides: FromAnywhere("s3-bucket"),
Listens: []Listening{{Port: 9000, Protocol: "tcp", From: FromMesh}},
Serves: map[string]map[string]any{"s3-bucket": {
"region": "eu-west",
"bucket": "${consumer:as:dns}",
}},
Receives: map[string]string{"s3-bucket": "/var/lib/store/grants/mesh.json"},
Grants: map[string]string{"s3-bucket": "/var/lib/store/grants"},
Resources: []map[string]any{{
"id": "server", "type": "container", "name": "store", "ports": []any{"9000"},
}},
}
}
// files is a consumer that writes the bucket into its own configuration — which is the thing it
// could not do before, and had to transcribe.
func files() Manifest {
return Manifest{
Module: "files", Version: "1", Slug: "files",
Requires: []string{"s3-bucket"},
Binds: map[string]string{"s3-bucket": "/var/lib/files/store.json"},
Secrets: map[string]string{"s3-bucket": "/var/lib/files/store.secret"},
Resources: []map[string]any{{
"id": "env", "type": "file", "path": "/var/lib/files/env", "mode": "0600",
"content": "BUCKET=${bound:s3-bucket:bucket}\nREGION=${bound:s3-bucket:region}\n",
}},
}
}
// pics is a second consumer of the same provider on the same machine: two derivations, neither
// the other's.
func pics() Manifest {
return Manifest{
Module: "pics", Version: "1", Slug: "pics",
Requires: []string{"s3-bucket"},
Binds: map[string]string{"s3-bucket": "/var/lib/pics/store.json"},
Secrets: map[string]string{"s3-bucket": "/var/lib/pics/store.secret"},
Resources: []map[string]any{{
"id": "env", "type": "file", "path": "/var/lib/pics/env", "mode": "0600",
"content": "BUCKET=${bound:s3-bucket:bucket}\n",
}},
}
}
// The three places the derived value lands must agree, because agreeing is the whole point: the
// consumer's own file, the binding it reads as JSON, and the provider's contributions entry.
func TestADerivedValueReachesBothEndsAndAgrees(t *testing.T) {
r, err := Resolve(shelf(store(), files()), []string{"store", "files"}, reachable(), World{})
if err != nil {
t.Fatal(err)
}
out, err := r.Declaration(Rendering{Grants: []Grant{{
Provision: "s3-bucket", Consumer: "workstation", From: "files", Slug: "files",
Values: map[string]any{}, Sealed: "c2VhbGVk",
}}})
if err != nil {
t.Fatal(err)
}
// The mesh minted this identity for the consumer; the bucket is that identity as a DNS label.
// Derived here with the mesh's own function, so the test cannot agree with a wrong rule.
as := ConsumerIdentity("workstation", IdentitySource("files", "files"))
want := strings.ReplaceAll(as, "_", "-")
if want == as || !strings.Contains(as, "_") {
t.Fatalf("the mesh's identity %q has no separator to rewrite; this test proves nothing", as)
}
env := fileNamed(out, "files.env")
if env == nil {
t.Fatalf("the consumer was given no file: %v", out)
}
if got := env["content"].(string); !strings.Contains(got, "BUCKET="+want+"\n") {
t.Errorf("the consumer's own file was not told the bucket:\n%s\nwant BUCKET=%s", got, want)
}
binding := fileNamed(out, "files.bound-s3-bucket")
if binding == nil {
t.Fatalf("the consumer was given no binding: %v", out)
}
var said struct {
Serves map[string]any `json:"serves"`
}
if err := json.Unmarshal([]byte(binding["content"].(string)), &said); err != nil {
t.Fatal(err)
}
if said.Serves["bucket"] != want {
t.Errorf("the binding says the bucket is %q, want %q", said.Serves["bucket"], want)
}
// And what is the same for everybody is still the same for everybody.
if said.Serves["region"] != "eu-west" {
t.Errorf("the binding lost what the provider serves to all: %v", said.Serves)
}
given := storeGrants(t, out)
if len(given) != 1 {
t.Fatalf("the provider was told about %d consumer(s): %v", len(given), given)
}
if given[0].Derived["bucket"] != want {
t.Errorf("the provider was told the bucket is %v, and the consumer was told %q — "+
"the two ends disagree, which is the whole failure", given[0].Derived["bucket"], want)
}
// Only the per-consumer half. The region is the same for everyone and is already in the
// provider's own definition; repeating it here would be a copy to go stale.
if _, carried := given[0].Derived["region"]; carried {
t.Errorf("the provider was handed back what it already says for everyone: %v", given[0].Derived)
}
}
// Two consumers of one provider on one machine get two buckets, and neither gets the other's.
func TestTwoConsumersOfOneProviderGetTheirOwnDerivation(t *testing.T) {
r, err := Resolve(shelf(store(), files(), pics()),
[]string{"store", "files", "pics"}, reachable(), World{})
if err != nil {
t.Fatal(err)
}
out, err := r.Declaration(Rendering{Grants: []Grant{
{Provision: "s3-bucket", Consumer: "workstation", From: "files", Slug: "files",
Values: map[string]any{}, Sealed: "c2VhbGVk"},
{Provision: "s3-bucket", Consumer: "workstation", From: "pics", Slug: "pics",
Values: map[string]any{}, Sealed: "c2VhbGVk"},
}})
if err != nil {
t.Fatal(err)
}
forFiles := strings.ReplaceAll(ConsumerIdentity("workstation", IdentitySource("files", "files")), "_", "-")
forPics := strings.ReplaceAll(ConsumerIdentity("workstation", IdentitySource("pics", "pics")), "_", "-")
if forFiles == forPics {
t.Fatal("the two consumers were given the same identity; this test proves nothing")
}
if got := fileNamed(out, "files.env")["content"].(string); !strings.Contains(got, "BUCKET="+forFiles+"\n") {
t.Errorf("files was not given its own bucket:\n%s", got)
}
if got := fileNamed(out, "pics.env")["content"].(string); !strings.Contains(got, "BUCKET="+forPics+"\n") {
t.Errorf("pics was not given its own bucket:\n%s", got)
}
var buckets []any
for _, g := range storeGrants(t, out) {
buckets = append(buckets, g.Derived["bucket"])
}
if len(buckets) != 2 || buckets[0] == buckets[1] {
t.Errorf("the provider was told %v; it must be told one bucket per consumer", buckets)
}
}
// An operator may still set what the provider serves, and the mesh still derives the rest: the
// setting is laid on first, then the consumer's half is filled.
func TestASettingComposesWithADerivedValue(t *testing.T) {
r, err := Resolve(shelf(store(), files()), []string{"store", "files"}, reachable(), World{})
if err != nil {
t.Fatal(err)
}
out, err := r.Declaration(Rendering{
Settings: SettingsBy{"store": {{From: "the operator",
Values: map[string]any{"bucket": "team-${consumer:as:dns}"}}}},
Grants: []Grant{{Provision: "s3-bucket", Consumer: "workstation", From: "files", Slug: "files",
Values: map[string]any{}, Sealed: "c2VhbGVk"}},
})
if err != nil {
t.Fatal(err)
}
want := "team-" + strings.ReplaceAll(ConsumerIdentity("workstation", IdentitySource("files", "files")), "_", "-")
if got := fileNamed(out, "files.env")["content"].(string); !strings.Contains(got, "BUCKET="+want+"\n") {
t.Errorf("the operator's prefix did not survive the derivation:\n%s\nwant BUCKET=%s", got, want)
}
if given := storeGrants(t, out); given[0].Derived["bucket"] != want {
t.Errorf("the provider was told %v, the consumer %q", given[0].Derived["bucket"], want)
}
}
// A fact or an alphabet the mesh does not have is refused where the definition is, not where a
// consumer happens to be resolved — and the refusal says what may be said instead.
func TestAServedValueNamingSomethingTheMeshDoesNotHaveIsRefused(t *testing.T) {
for _, c := range []struct{ value, says string }{
{"${consumer:node}", "as"},
{"${consumer:as:punycode}", "dns"},
} {
m := store()
m.Serves["s3-bucket"]["bucket"] = c.value
raw, err := json.Marshal(m)
if err != nil {
t.Fatal(err)
}
_, err = ParseManifest(raw)
if err == nil {
t.Fatalf("%s was accepted", c.value)
}
if !strings.Contains(err.Error(), c.value) {
t.Errorf("the refusal of %s does not quote it: %v", c.value, err)
}
if !strings.Contains(err.Error(), c.says) {
t.Errorf("the refusal of %s does not say what may be said (%q): %v", c.value, c.says, err)
}
}
}
// `dns` is checked against an identity the mesh actually mints, not an invented string.
func TestTheDNSAlphabetIsTheMintedIdentityWithItsSeparatorRewritten(t *testing.T) {
as := ConsumerIdentity("anchor", IdentitySource("ncloud", "nextcloud"))
if err := CheckIdentity("anchor", IdentitySource("ncloud", "nextcloud")); err != nil {
t.Fatalf("the mesh would not mint this identity at all: %v", err)
}
label := asDNSLabel(as)
if strings.Contains(label, "_") {
t.Errorf("%q is not a DNS label", label)
}
if strings.ReplaceAll(label, "-", "_") != as {
t.Errorf("%q is not %q with its separator rewritten", label, as)
}
}
// The check that would have caught the one wrong instance: a consumer that writes the derived
// value into its own definition instead of asking for it is refused, whether it transcribed the
// right answer or a predecessor's.
func TestAConsumerThatTranscribesWhatItsProviderDerivesIsRefused(t *testing.T) {
as := ConsumerIdentity("workstation", IdentitySource("files", "files"))
transcribed := strings.ReplaceAll(as, "_", "-")
m := files()
m.Resources = []map[string]any{{
"id": "env", "type": "file", "path": "/var/lib/files/env", "mode": "0600",
// Exactly what the provider will create — correct today, and a copy of a rule that is
// not this module's.
"content": "BUCKET=" + transcribed + "\n",
}}
r, err := Resolve(shelf(store(), m), []string{"store", "files"}, reachable(), World{})
if err != nil {
t.Fatal(err)
}
_, err = r.Declaration(Rendering{Grants: []Grant{{
Provision: "s3-bucket", Consumer: "workstation", From: "files", Slug: "files",
Values: map[string]any{}, Sealed: "c2VhbGVk",
}}})
if err == nil {
t.Fatal("a definition holding its own copy of the provider's naming rule was accepted")
}
if !strings.Contains(err.Error(), "${bound:s3-bucket:bucket}") {
t.Errorf("the refusal does not say what to write instead: %v", err)
}
// And a constant the provider serves to everyone is not a transcription: repeating it is
// redundant, not wrong, and refusing it would be the mesh policing style.
m.Resources = []map[string]any{{
"id": "env", "type": "file", "path": "/var/lib/files/env", "mode": "0600",
"content": "REGION=eu-west\n",
}}
r, err = Resolve(shelf(store(), m), []string{"store", "files"}, reachable(), World{})
if err != nil {
t.Fatal(err)
}
if _, err := r.Declaration(Rendering{Grants: []Grant{{
Provision: "s3-bucket", Consumer: "workstation", From: "files", Slug: "files",
Values: map[string]any{}, Sealed: "c2VhbGVk",
}}}); err != nil {
t.Errorf("a value the provider serves to everyone was judged a transcription: %v", err)
}
}
func storeGrants(t *testing.T, out []map[string]any) []Contribution {
t.Helper()
for _, r := range out {
if r["path"] != "/var/lib/store/grants/mesh.json" {
continue
}
var parsed struct {
Given []Contribution `json:"given"`
}
if err := json.Unmarshal([]byte(r["content"].(string)), &parsed); err != nil {
t.Fatal(err)
}
return parsed.Given
}
t.Fatalf("the provider was given no contributions file: %v", out)
return nil
}
// A consumer that keeps SEVERAL holders of one provision is refused, rather than told one thing
// while its provider is told another.
//
// **This is issue 124's own failure, one case to the side.** The mesh gives each holder its own
// login — `mesh_node_mod_<local>` (ADR 0094) — and the provider derives from the login, so it
// would make one resource per holder. The consumer's side has no such dimension: there is one
// binding file per provision and one `${bound:<provision>:<key>}`, both derived from the
// un-suffixed identity. So the provider would create `…-mod-cold` and the consumer would be
// configured against `…-mod`: it would authenticate successfully and be refused on every object,
// which is exactly the fault this whole record exists to end.
//
// Refused, loudly, at the one place that can see both halves. Lifting it means giving the
// consumer's side a local dimension, which is a decision and not an omission.
func TestAConsumerWithSeveralHoldersOfADerivingProviderIsRefused(t *testing.T) {
m := files()
// Two holders of the one provision, the shape ADR 0094 gives a module that keeps several.
m.Secrets = nil
m.SecretsMany = map[string]map[string]string{"s3-bucket": {
"hot": "/var/lib/files/hot.secret",
"cold": "/var/lib/files/cold.secret",
}}
m.Resources = []map[string]any{{
"id": "env", "type": "file", "path": "/var/lib/files/env", "mode": "0600",
"content": "BUCKET=${bound:s3-bucket:bucket}\n",
}}
r, err := Resolve(shelf(store(), m), []string{"store", "files"}, reachable(), World{})
if err != nil {
t.Fatal(err)
}
_, err = r.Declaration(Rendering{Grants: []Grant{
{Provision: "s3-bucket", Consumer: "workstation", From: "files", Slug: "files",
Local: "hot", Values: map[string]any{}, Sealed: "c2VhbGVk"},
{Provision: "s3-bucket", Consumer: "workstation", From: "files", Slug: "files",
Local: "cold", Values: map[string]any{}, Sealed: "c2VhbGVk"},
}})
if err == nil {
t.Fatal("a consumer with several holders of a deriving provider was accepted; " +
"its two ends would have disagreed in silence")
}
for _, want := range []string{"files", "s3-bucket", "bucket"} {
if !strings.Contains(err.Error(), want) {
t.Errorf("the refusal does not name %q: %v", want, err)
}
}
}
+593
View File
@@ -0,0 +1,593 @@
package catalogue
import (
"fmt"
"regexp"
"sort"
"strings"
)
// The account's environment and the login shell's code, composed from the modules a node runs
// (novox/hq ADR 0203, ADR 0204).
//
// **The same shape as the jails.** Every module may contribute — a toolchain its directory on PATH,
// a version manager a variable naming its home, a prompt the code that loads it — naming no node, no
// path and no file of the shell's (ADR 0112). The one module holding the matching seat places the
// result with a placeholder in its own file, and the controller fills it from every module on the
// node. A node not running a module has none of its contribution, and unassigning one takes its
// lines away at the next composition.
//
// **Two kinds of contribution, kept apart on purpose.** The environment is facts, which the
// controller writes in two standard formats — POSIX assignment and the service manager's
// environment.d — so a terminal, a script, the login shell's `execute` and a graphical session all
// read the same values (ADR 0203). Shell code is not a fact: it is text in one shell's syntax, which
// the controller sorts into a slot and pastes without reading, as it pastes a jail's stanza (ADR
// 0204).
// EnvironmentSeat and LoginShellSeat are the seats whose holders may place what the modules
// contributed: the account's environment, and the login shell's code.
const (
EnvironmentSeat = "node-environment"
LoginShellSeat = "node-login-shell"
// PowerSeat is the seat whose holder places code for the power moments (novox/hq ADR 0211).
PowerSeat = "node-power"
)
// Where an environment entry on PATH goes: before the account's existing PATH, or after it.
const (
PathAtStart = "start"
PathAtEnd = "end"
)
// Environment is what one module adds to the account's environment (novox/hq ADR 0203).
type Environment struct {
// Variables are names and literal values. A value may name the machine's own facts with
// ${machine:…}, resolved before anything is written, and nothing else that expands.
Variables map[string]string `json:"variables,omitempty"`
// Path is entries on the account's PATH, each at its start or its end, in the order declared.
Path []PathEntry `json:"path,omitempty"`
}
// PathEntry is one directory a module puts on the account's PATH.
type PathEntry struct {
Entry string `json:"entry"`
At string `json:"at"`
}
// ShellCode is one piece of code a module adds to a shell's startup (novox/hq ADR 0204).
type ShellCode struct {
// For is the shell the code is written in.
For string `json:"for"`
// Slot is where it runs among the other modules' code: first, normal or last. Named rather
// than numbered, because every contributor would guess a number and a collision says nothing.
Slot string `json:"slot"`
// Code is never interpreted — it is the shell's syntax, and only the shell reads it.
Code string `json:"code"`
}
// The shells and slots a contribution may name (novox/hq ADR 0204). Closed, so a typo is a refusal
// at the check rather than code that silently lands in no placeholder.
var (
knownShells = []string{"zsh", "bash", "fish"}
knownSlots = []string{"first", "normal", "last"}
// sessionFiles are the two files of the graphical session's start that read no directory, so a
// contribution to them is a slot rather than a drop-in (novox/hq ADR 0208 §4): `xinitrc` is POSIX
// code the session's start runs, `xresources` X resources merged at its start. Placed by the
// display server's holder, as a shell's slots are placed by the login shell's.
sessionFiles = []string{"xinitrc", "xresources"}
// powerMoments are the moments of a machine's power a module may run code at (novox/hq ADR
// 0211 §3): POSIX code run as root by node-power's holder, in module order, each piece bounded.
powerMoments = []string{"after-boot", "before-sleep", "after-wake", "before-shutdown", "on-mains", "on-battery"}
)
// contributionTargets is every name a contribution's `for` may take.
func contributionTargets() []string {
out := append(append([]string(nil), knownShells...), sessionFiles...)
return append(out, powerMoments...)
}
// placerOf is the seat whose holder places a contribution for this target (novox/hq ADR 0204,
// ADR 0208 §4).
func placerOf(target string) string {
if oneOf(sessionFiles, target) {
return DisplayServerSeat
}
if oneOf(powerMoments, target) {
return PowerSeat
}
return LoginShellSeat
}
// The two renderings of the environment a holder may place (novox/hq ADR 0203, decision 3).
const (
EnvironmentPOSIX = "posix"
EnvironmentSystemd = "systemd"
)
// ofEnvironment and ofShell are where a holder places what was contributed: ${environment:posix},
// ${environment:systemd} and ${shell:<shell>:<slot>}. Loose inside the braces on purpose, so a
// misspelt key is found and refused rather than left in a file as a literal nobody reads.
var (
ofEnvironment = regexp.MustCompile(`\$\{environment:([^}]*)\}`)
ofShell = regexp.MustCompile(`\$\{shell:([^}]*)\}`)
// ofContributed is either kind of contributed text, matched together so both fill in one pass.
ofContributed = regexp.MustCompile(`\$\{(shell|contribution):([^}]*)\}`)
)
// variableName is a POSIX shell variable name, which is also what environment.d accepts.
var variableName = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`)
// environmentProblems is what is wrong with this module's environment contribution, from the
// manifest alone.
func (m Manifest) environmentProblems() []string {
if m.Environment == nil {
return nil
}
var problems []string
for _, n := range sortedKeys(m.Environment.Variables) {
switch {
case !variableName.MatchString(n):
problems = append(problems, fmt.Sprintf(
"%s sets the variable %q, which is not a name a shell accepts: a letter or an "+
"underscore, then letters, digits and underscores", m.Module, n))
continue
case n == "PATH":
// PATH is the one variable every module shares, so no module may set it whole: a second
// setter would replace the first's entries, and the account's own PATH with them.
problems = append(problems, fmt.Sprintf(
"%s sets PATH as a variable; a module adds an entry under environment.path, at the "+
"start or the end, and PATH is composed from every module's (novox/hq ADR 0203)", m.Module))
continue
}
if why := literalProblem(m.Environment.Variables[n]); why != "" {
problems = append(problems, fmt.Sprintf(
"%s sets %s to %q, which %s — %s", m.Module, n, m.Environment.Variables[n], why, literalRule))
}
}
seen := map[string]bool{}
for i, p := range m.Environment.Path {
switch {
case p.Entry == "":
problems = append(problems, fmt.Sprintf("%s's PATH entry %d names no directory", m.Module, i+1))
case strings.Contains(ofMachine.ReplaceAllString(p.Entry, ""), ":"):
// A colon is PATH's own separator, so an entry holding one is two entries, and the
// check that it is already present would look for the wrong thing.
problems = append(problems, fmt.Sprintf(
"%s puts %q on PATH, which holds a colon, PATH's own separator", m.Module, p.Entry))
case seen[p.Entry]:
problems = append(problems, fmt.Sprintf("%s puts %q on PATH twice", m.Module, p.Entry))
default:
if why := literalProblem(p.Entry); why != "" {
problems = append(problems, fmt.Sprintf(
"%s puts %q on PATH, which %s — %s", m.Module, p.Entry, why, literalRule))
}
}
seen[p.Entry] = true
if p.At != PathAtStart && p.At != PathAtEnd {
problems = append(problems, fmt.Sprintf(
"%s puts %q on PATH at %q; an entry goes at %q or %q of the account's PATH",
m.Module, p.Entry, p.At, PathAtStart, PathAtEnd))
}
}
return problems
}
// literalRule is why a value must be literal, said with every refusal of one.
const literalRule = "a value is literal, so a POSIX shell and the service manager read it alike, and " +
"names the machine only through the mesh's own ${machine:…} facts (novox/hq ADR 0203)"
// literalProblem is why a value cannot be written, unquoted by either reader, as the same string in
// both formats — or nothing. A `$` would expand differently in each; a quote or a backslash is
// quoting in one and a character in the other; a line break ends the line in both.
func literalProblem(v string) string {
switch {
case strings.ContainsAny(v, `'"`):
return "holds a quote"
case strings.Contains(v, `\`):
return "holds a backslash"
case strings.ContainsAny(v, "\n\r"):
return "holds a line break"
case strings.ContainsRune(v, 0):
return "holds a NUL"
case strings.Contains(ofMachine.ReplaceAllString(v, ""), "$"):
return "holds a $ that is not one of the machine's ${machine:…} facts"
}
return ""
}
// shellProblems is what is wrong with this module's shell code, from the manifest alone. The code
// itself is not judged: it is the shell's syntax, which the controller does not read.
func (m Manifest) shellProblems() []string {
var problems []string
for i, c := range m.Shell {
if !oneOf(contributionTargets(), c.For) {
problems = append(problems, fmt.Sprintf(
"%s's shell code %d is for %q; the shells are %s, the session's files %s, and the power "+
"moments %s", m.Module, i+1, c.For, strings.Join(knownShells, ", "),
strings.Join(sessionFiles, ", "), strings.Join(powerMoments, ", ")))
}
if !oneOf(knownSlots, c.Slot) {
problems = append(problems, fmt.Sprintf(
"%s's shell code %d goes in the slot %q; the slots are %s", m.Module, i+1, c.Slot,
strings.Join(knownSlots, ", ")))
}
if strings.TrimSpace(c.Code) == "" {
problems = append(problems, fmt.Sprintf("%s's shell code %d has no code", m.Module, i+1))
}
}
return problems
}
// contributionPlaceholderProblems is every place this module's resources name the environment or
// the shell's code and may not — judged from the manifest, so the catalogue check refuses it before
// a mesh does, and again at composition in the same words.
func (m Manifest) contributionPlaceholderProblems() []string {
var problems []string
for _, r := range m.Resources {
problems = append(problems, placeholderProblems(m, r)...)
problems = append(problems, seatPlaceholderProblems(m, r)...)
}
return problems
}
// placeholderProblems is what is wrong with one resource's ${environment:…} and ${shell:…}.
//
// **The seat authorises it, not the placeholder** (novox/hq ADR 0203 §5, ADR 0204 §3), as the seat
// authorises the bus's user list: a module that does not hold the account's environment writing it
// would be a second writer of a file there is one of, and a module that does not hold the login
// shell writing every module's shell code would be a second shell.
func placeholderProblems(m Manifest, r map[string]any) []string {
var problems []string
for _, field := range sortedKeys(r) {
s, ok := r[field].(string)
if !ok {
continue
}
env := ofEnvironment.FindAllStringSubmatch(s, -1)
code := ofShell.FindAllStringSubmatch(s, -1)
if len(env)+len(code) == 0 {
continue
}
if field != "content" {
// Placed only where a file's bytes are, which is where every one of them is meant to go:
// a path or an owner holding several lines of shell is nothing the host could act on.
problems = append(problems, fmt.Sprintf(
"%s's resource %v names %s in its %s; the environment and the shell's code are placed "+
"only in a file's content", m.Module, r["id"], placeholderOf(env, code), field))
continue
}
for _, e := range env {
if e[1] != EnvironmentPOSIX && e[1] != EnvironmentSystemd {
problems = append(problems, fmt.Sprintf(
"%s's resource %v names %s; the environment is ${environment:%s} or ${environment:%s}",
m.Module, r["id"], e[0], EnvironmentPOSIX, EnvironmentSystemd))
}
}
if len(env) > 0 && !m.ClaimsSeat(EnvironmentSeat) {
problems = append(problems, fmt.Sprintf(
"%s's resource %v names %s and %s does not claim %s; the account's environment is "+
"written by that seat's holder alone (novox/hq ADR 0203)",
m.Module, r["id"], env[0][0], m.Module, EnvironmentSeat))
}
// Each placeholder judged by its own target: a shell's code is the login shell's holder's to
// place (ADR 0204), the session's files the display server's (ADR 0208 §4) — and a holder of
// one placing the other's would be a second writer of a file there is one of.
refusedFor := map[string]bool{}
for _, c := range code {
target, slot, two := strings.Cut(c[1], ":")
if !two || !oneOf(contributionTargets(), target) || !oneOf(knownSlots, slot) {
problems = append(problems, fmt.Sprintf(
"%s's resource %v names %s; shell code is ${shell:<shell>:<slot>}, the shell one of "+
"%s, the session's file one of %s or the power moment one of %s, and the slot one of %s",
m.Module, r["id"], c[0], strings.Join(knownShells, ", "), strings.Join(sessionFiles, ", "),
strings.Join(powerMoments, ", "), strings.Join(knownSlots, ", ")))
continue
}
seat := placerOf(target)
if m.ClaimsSeat(seat) || refusedFor[seat] {
continue
}
refusedFor[seat] = true
if seat == PowerSeat {
problems = append(problems, fmt.Sprintf(
"%s's resource %v names %s and %s does not claim %s; every module's code for a power "+
"moment is placed by the power seat's holder alone (novox/hq ADR 0211)",
m.Module, r["id"], c[0], m.Module, seat))
continue
}
if seat == DisplayServerSeat {
problems = append(problems, fmt.Sprintf(
"%s's resource %v names %s and %s does not claim %s; every module's %s is placed by "+
"the display server's holder alone (novox/hq ADR 0208)",
m.Module, r["id"], c[0], m.Module, seat, target))
continue
}
problems = append(problems, fmt.Sprintf(
"%s's resource %v names %s and %s does not claim %s; every module's shell code is "+
"placed by the login shell's holder alone (novox/hq ADR 0204)",
m.Module, r["id"], c[0], m.Module, seat))
}
}
return problems
}
func placeholderOf(env, code [][]string) string {
if len(env) > 0 {
return env[0][0]
}
return code[0][0]
}
// contributedEnvironment is one node's environment, gathered and in the order it is written.
type contributedEnvironment struct {
// variables is by module in name order, each module's sorted by name.
variables []setBy
// start and end are PATH's entries in their final order, each once.
start, end []placedOn
}
type setBy struct {
module string
names []string
values map[string]string
}
type placedOn struct {
module, entry string
}
// inModuleOrder is the modules sorted by name — the order contributions are written in (novox/hq
// ADR 0203, ADR 0204), so the same set composes byte for byte whatever order they were assigned in.
func inModuleOrder(modules []Manifest) []Manifest {
out := append([]Manifest(nil), modules...)
sort.SliceStable(out, func(a, b int) bool { return out[a].Module < out[b].Module })
return out
}
// variablesSetOnce refuses a variable two modules on one node both set (novox/hq ADR 0203 §5),
// naming both. Neither is chosen: whichever was written last would win in one reader and not
// necessarily in the other, and the module that lost would not be told.
func variablesSetOnce(modules []Manifest) error {
setter := map[string]string{}
for _, m := range inModuleOrder(modules) {
if m.Environment == nil {
continue
}
for _, n := range sortedKeys(m.Environment.Variables) {
if first, taken := setter[n]; taken {
return fmt.Errorf(
"%s and %s both set %s on this machine; the account has one environment, so one "+
"of them must stop setting it (novox/hq ADR 0203)", first, m.Module, n)
}
setter[n] = m.Module
}
}
return nil
}
// environmentOn gathers every module's environment on a node, with the machine's facts in place.
//
// A PATH entry two modules both add is written once, where the first puts it: two toolchains
// sharing ~/.local/bin is ordinary, and nothing about it is in conflict.
func environmentOn(modules []Manifest, facts map[string]string) (contributedEnvironment, error) {
var env contributedEnvironment
if err := variablesSetOnce(modules); err != nil {
return env, err
}
placed := map[string]bool{}
for _, m := range inModuleOrder(modules) {
if m.Environment == nil {
continue
}
if len(m.Environment.Variables) > 0 {
set := setBy{module: m.Module, values: map[string]string{}}
for _, n := range sortedKeys(m.Environment.Variables) {
v, err := factsIn(m.Environment.Variables[n], facts, m.Module, n)
if err != nil {
return env, err
}
set.names = append(set.names, n)
set.values[n] = v
}
env.variables = append(env.variables, set)
}
for _, p := range m.Environment.Path {
entry, err := factsIn(p.Entry, facts, m.Module, "a PATH entry")
if err != nil {
return env, err
}
if strings.Contains(entry, ":") {
return env, fmt.Errorf("%s puts %q on PATH on this machine, which holds a colon, PATH's own separator",
m.Module, entry)
}
if placed[entry] {
continue
}
placed[entry] = true
if p.At == PathAtEnd {
env.end = append(env.end, placedOn{m.Module, entry})
} else {
env.start = append(env.start, placedOn{m.Module, entry})
}
}
}
return env, nil
}
// factsIn resolves a contributed value's ${machine:…} facts with this machine's — first, before
// either format is written, so both say the same thing (novox/hq ADR 0203).
func factsIn(v string, facts map[string]string, module, what string) (string, error) {
for _, key := range machineUsed(v) {
value, has := facts[key]
if !has {
return "", fmt.Errorf("%s sets %s to a value that says ${machine:%s}, and this machine says %s",
module, what, key, orNothing(namesOfFacts(facts)))
}
v = strings.ReplaceAll(v, fmt.Sprintf("${machine:%s}", key), value)
}
// Judged again once filled: a fact is the mesh's, and still has to be a literal both readers
// take alike.
if why := literalProblem(v); why != "" {
return "", fmt.Errorf("%s sets %s to %q on this machine, which %s — %s", module, what, v, why, literalRule)
}
return v, nil
}
// posix is the environment as lines a POSIX shell sources (novox/hq ADR 0203 §3): every variable
// exported, every PATH entry added only when it is missing, so sourcing the file twice — a login
// shell that starts another — changes nothing. POSIX sh only, because sh, bash and zsh all read it.
//
// The start entries are written last-first: each is put in front of PATH, so the last written ends
// up first, and the result reads in module order, then the order each module declared.
func (e contributedEnvironment) posix() string {
var b strings.Builder
for _, set := range e.variables {
fmt.Fprintf(&b, "# %s\n", set.module)
for _, n := range set.names {
fmt.Fprintf(&b, "export %s='%s'\n", n, set.values[n])
}
}
named := ""
for i := len(e.start) - 1; i >= 0; i-- {
p := e.start[i]
if p.module != named {
fmt.Fprintf(&b, "# %s\n", p.module)
named = p.module
}
fmt.Fprintf(&b, "case \":${PATH}:\" in *':%s:'*) ;; *) PATH='%s'\"${PATH:+:${PATH}}\" ;; esac\n",
p.entry, p.entry)
}
named = ""
for _, p := range e.end {
if p.module != named {
fmt.Fprintf(&b, "# %s\n", p.module)
named = p.module
}
fmt.Fprintf(&b, "case \":${PATH}:\" in *':%s:'*) ;; *) PATH=\"${PATH:+${PATH}:}\"'%s' ;; esac\n",
p.entry, p.entry)
}
if len(e.start)+len(e.end) > 0 {
b.WriteString("export PATH\n")
}
return b.String()
}
// systemd is the same environment as the service manager's environment.d reads it (novox/hq ADR
// 0203 §3), for the account's user manager and so for everything a graphical session starts. Read
// once per manager start, so it needs no guard against running twice; the account's existing PATH
// sits between the start and the end entries.
func (e contributedEnvironment) systemd() string {
var b strings.Builder
for _, set := range e.variables {
fmt.Fprintf(&b, "# %s\n", set.module)
for _, n := range set.names {
fmt.Fprintf(&b, "%s=%s\n", n, set.values[n])
}
}
if len(e.start) > 0 {
fmt.Fprintf(&b, "# %s\nPATH=%s${PATH:+:$PATH}\n", modulesOf(e.start), entriesOf(e.start))
}
if len(e.end) > 0 {
fmt.Fprintf(&b, "# %s\nPATH=${PATH:+$PATH:}%s\n", modulesOf(e.end), entriesOf(e.end))
}
return b.String()
}
// modulesOf names who contributed a line holding several modules' entries, in the order they appear.
func modulesOf(entries []placedOn) string {
var names []string
seen := map[string]bool{}
for _, p := range entries {
if !seen[p.module] {
seen[p.module] = true
names = append(names, p.module)
}
}
return strings.Join(names, ", ")
}
func entriesOf(entries []placedOn) string {
out := make([]string, len(entries))
for i, p := range entries {
out[i] = p.entry
}
return strings.Join(out, ":")
}
// shellCode is every module's code for one shell and one slot (novox/hq ADR 0204 §3): in module
// order, each module's pieces in the order it declared them, each preceded by a line naming the
// module, and empty when nothing is contributed.
func shellCode(modules []Manifest, shell, slot string) string {
var b strings.Builder
for _, m := range inModuleOrder(modules) {
named := false
for _, c := range m.Shell {
if c.For != shell || c.Slot != slot {
continue
}
if !named {
fmt.Fprintf(&b, "# %s\n", m.Module)
named = true
}
b.WriteString(c.Code)
if !strings.HasSuffix(c.Code, "\n") {
b.WriteString("\n")
}
}
}
return b.String()
}
// contributionsInto fills a holder's file with the node's environment and its shell code.
//
// **Last, after every other placeholder pass, and in one pass each.** Shell code is contributed text
// in a shell's own syntax — `${XDG_CACHE_HOME:-$HOME/.cache}`, `${(%):-%n}` — and the rendered
// environment holds `${PATH:+…}`: a scanner for the mesh's own placeholders that ran after these
// were in place would read the shell's expansions as the mesh's and refuse them, or fill a
// `${machine:…}` some module wrote for its shell to see. So nothing runs after them, the environment
// is filled before the shell's code is, and each is replaced in a single pass over what the holder
// wrote, so a contributed piece is never scanned again.
func contributionsInto(resource map[string]any, m Manifest, modules []Manifest, facts map[string]string, with Rendering) error {
if problems := append(placeholderProblems(m, resource), seatPlaceholderProblems(m, resource)...); len(problems) > 0 {
return fmt.Errorf("%s", problems[0])
}
content, ok := resource["content"].(string)
if !ok {
return nil
}
if ofEnvironment.MatchString(content) {
env, err := environmentOn(modules, facts)
if err != nil {
return err
}
content = ofEnvironment.ReplaceAllStringFunc(content, func(placeholder string) string {
if ofEnvironment.FindStringSubmatch(placeholder)[1] == EnvironmentSystemd {
return env.systemd()
}
return env.posix()
})
}
// Shell code and seat contributions in one pass (novox/hq ADR 0212): both are contributed text
// the controller does not read, so neither may be scanned after the other is in place — a
// contributed line that happened to spell the other's placeholder would be filled.
if ofContributed.MatchString(content) {
var failed error
content = ofContributed.ReplaceAllStringFunc(content, func(placeholder string) string {
found := ofContributed.FindStringSubmatch(placeholder)
first, second, _ := strings.Cut(found[2], ":")
if found[1] == "contribution" {
placed, err := seatContributions(modules, first, second, with)
if err != nil && failed == nil {
failed = err
}
return placed
}
return shellCode(modules, first, second)
})
if failed != nil {
return failed
}
}
resource["content"] = content
return nil
}
+471
View File
@@ -0,0 +1,471 @@
package catalogue
import (
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
)
// Defends novox/hq ADR 0203 (the account's environment is one module's, and every module
// contributes to it) and ADR 0204 (shell code in named slots, placed by the login shell's holder).
// contributors is a fixed set of contributions, in no particular order: what the renderings are
// asserted against byte for byte. go-toolchain and zsh both put ~/.local/bin on PATH, which is the
// ordinary case of two modules sharing a directory, and is written once.
func contributors() []Manifest {
return []Manifest{
{Module: "zsh", Environment: &Environment{
Variables: map[string]string{"XDG_CONFIG_HOME": "${machine:account-home}/.config", "EDITOR": "vim"},
Path: []PathEntry{
{Entry: "${machine:account-home}/.local/bin", At: PathAtStart},
{Entry: "${machine:account-home}/bin", At: PathAtStart},
{Entry: "/opt/scripts", At: PathAtEnd},
},
}},
{Module: "go-toolchain", Environment: &Environment{
Variables: map[string]string{"GOPATH": "${machine:account-home}/go"},
Path: []PathEntry{
{Entry: "${machine:account-home}/go/bin", At: PathAtStart},
{Entry: "/usr/local/go/bin", At: PathAtStart},
{Entry: "${machine:account-home}/.local/bin", At: PathAtStart},
},
}},
{Module: "agent", Environment: &Environment{
Variables: map[string]string{"DISABLE_AUTOUPDATER": "1"},
Path: []PathEntry{{Entry: "/opt/agent/bin", At: PathAtEnd}},
}},
// A module contributing nothing is in the set and writes nothing.
{Module: "postgres"},
}
}
var operatorFacts = map[string]string{"name": "workstation", "account": "op", "account-home": "/home/op"}
// The final PATH this set composes, around whatever the account had: the start entries in module
// order and then declared order, the account's own, then the end entries.
const composedPOSIX = `# agent
export DISABLE_AUTOUPDATER='1'
# go-toolchain
export GOPATH='/home/op/go'
# zsh
export EDITOR='vim'
export XDG_CONFIG_HOME='/home/op/.config'
# zsh
case ":${PATH}:" in *':/home/op/bin:'*) ;; *) PATH='/home/op/bin'"${PATH:+:${PATH}}" ;; esac
# go-toolchain
case ":${PATH}:" in *':/home/op/.local/bin:'*) ;; *) PATH='/home/op/.local/bin'"${PATH:+:${PATH}}" ;; esac
case ":${PATH}:" in *':/usr/local/go/bin:'*) ;; *) PATH='/usr/local/go/bin'"${PATH:+:${PATH}}" ;; esac
case ":${PATH}:" in *':/home/op/go/bin:'*) ;; *) PATH='/home/op/go/bin'"${PATH:+:${PATH}}" ;; esac
# agent
case ":${PATH}:" in *':/opt/agent/bin:'*) ;; *) PATH="${PATH:+${PATH}:}"'/opt/agent/bin' ;; esac
# zsh
case ":${PATH}:" in *':/opt/scripts:'*) ;; *) PATH="${PATH:+${PATH}:}"'/opt/scripts' ;; esac
export PATH
`
const composedSystemd = `# agent
DISABLE_AUTOUPDATER=1
# go-toolchain
GOPATH=/home/op/go
# zsh
EDITOR=vim
XDG_CONFIG_HOME=/home/op/.config
# go-toolchain, zsh
PATH=/home/op/go/bin:/usr/local/go/bin:/home/op/.local/bin:/home/op/bin${PATH:+:$PATH}
# agent, zsh
PATH=${PATH:+$PATH:}/opt/agent/bin:/opt/scripts
`
func TestTheEnvironmentRendersForAPOSIXShellByteForByte(t *testing.T) {
env, err := environmentOn(contributors(), operatorFacts)
if err != nil {
t.Fatal(err)
}
if got := env.posix(); got != composedPOSIX {
t.Fatalf("the POSIX rendering is\n%s\nnot\n%s", got, composedPOSIX)
}
}
func TestTheEnvironmentRendersForTheServiceManagerByteForByte(t *testing.T) {
env, err := environmentOn(contributors(), operatorFacts)
if err != nil {
t.Fatal(err)
}
if got := env.systemd(); got != composedSystemd {
t.Fatalf("the environment.d rendering is\n%s\nnot\n%s", got, composedSystemd)
}
}
// Sourcing twice changes nothing (ADR 0203 §3): a login shell that starts another reads the file
// again, and a PATH that grew each time would be the symptom. Run by a real `sh`, because the claim
// is about what a shell does with the file, not about what the file looks like.
func TestThePOSIXEnvironmentSourcedTwiceLeavesPATHAsOnce(t *testing.T) {
sh, err := exec.LookPath("sh")
if err != nil {
t.Skip("no sh on this machine")
}
script := "PATH=/usr/bin:/bin\n" + composedPOSIX + "once=$PATH\n" + composedPOSIX +
`[ "$PATH" = "$once" ] || { echo "changed: $once -> $PATH"; exit 1; }` + "\n" +
`echo "$PATH"; echo "$GOPATH"`
out, err := exec.Command(sh, "-c", script).CombinedOutput()
if err != nil {
t.Fatalf("sourcing twice: %v\n%s", err, out)
}
lines := strings.Split(strings.TrimSpace(string(out)), "\n")
want := "/home/op/go/bin:/usr/local/go/bin:/home/op/.local/bin:/home/op/bin:/usr/bin:/bin:/opt/agent/bin:/opt/scripts"
if lines[0] != want {
t.Fatalf("PATH is %s, not %s", lines[0], want)
}
if lines[1] != "/home/op/go" {
t.Fatalf("GOPATH was not exported: %q", lines[1])
}
// And an entry the account already has stays where it is, and once.
out, err = exec.Command(sh, "-c", "PATH=/opt/scripts:/usr/bin\n"+composedPOSIX+`echo "$PATH"`).CombinedOutput()
if err != nil {
t.Fatalf("%v\n%s", err, out)
}
if got := strings.TrimSpace(string(out)); got !=
"/home/op/go/bin:/usr/local/go/bin:/home/op/.local/bin:/home/op/bin:/opt/scripts:/usr/bin:/opt/agent/bin" {
t.Fatalf("an entry already on PATH was added again or moved: %s", got)
}
}
// The environment.d rendering, read by the service manager's own generator where this machine has
// one — the same reader an account's user manager runs, so the PATH it composes is the one asserted.
func TestTheServiceManagerReadsTheSystemdRenderingAsMeant(t *testing.T) {
generator := "/usr/lib/systemd/user-environment-generators/30-systemd-environment-d-generator"
if _, err := os.Stat(generator); err != nil {
t.Skip("no environment.d generator on this machine")
}
config := t.TempDir()
if err := os.MkdirAll(filepath.Join(config, "environment.d"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(config, "environment.d", "50-mesh.conf"), []byte(composedSystemd), 0o644); err != nil {
t.Fatal(err)
}
cmd := exec.Command(generator)
cmd.Env = []string{"PATH=/usr/bin:/bin", "HOME=" + config, "XDG_CONFIG_HOME=" + config}
out, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("%v\n%s", err, out)
}
want := "PATH=/home/op/go/bin:/usr/local/go/bin:/home/op/.local/bin:/home/op/bin:/usr/bin:/bin:/opt/agent/bin:/opt/scripts"
if !strings.Contains(string(out), want+"\n") || !strings.Contains(string(out), "GOPATH=/home/op/go\n") {
t.Fatalf("the service manager read\n%s", out)
}
}
// Nothing contributed renders nothing, in both formats — not an empty `export PATH`.
func TestNoContributionsRenderNothing(t *testing.T) {
env, err := environmentOn([]Manifest{{Module: "postgres"}}, operatorFacts)
if err != nil {
t.Fatal(err)
}
if env.posix() != "" || env.systemd() != "" {
t.Fatalf("an empty environment rendered %q and %q", env.posix(), env.systemd())
}
}
// A ${machine:…} fact the machine does not have is refused naming the module, as a file's is.
func TestAContributedFactTheMachineLacksIsRefused(t *testing.T) {
_, err := environmentOn(contributors(), map[string]string{"name": "server"})
if err == nil || !strings.Contains(err.Error(), "go-toolchain sets GOPATH") ||
!strings.Contains(err.Error(), "${machine:account-home}") {
t.Fatalf("a missing account home was not refused by name: %v", err)
}
}
// ADR 0203 §5: two modules setting one variable are refused, both named — neither silently wins.
func TestAVariableTwoModulesSetIsRefusedNamingBoth(t *testing.T) {
modules := append(contributors(), Manifest{Module: "neovim", Environment: &Environment{
Variables: map[string]string{"EDITOR": "nvim"}}})
_, err := environmentOn(modules, operatorFacts)
if err == nil || err.Error() != "neovim and zsh both set EDITOR on this machine; the account has one "+
"environment, so one of them must stop setting it (novox/hq ADR 0203)" {
t.Fatalf("a variable set twice was not refused naming both: %v", err)
}
// And at composition, whether or not the node holds the environment.
r := Resolution{Node: "workstation", Account: "op", Modules: modules}
if _, err := r.Declaration(Rendering{}); err == nil || !strings.Contains(err.Error(), "neovim and zsh both set EDITOR") {
t.Fatalf("composition accepted a variable set twice: %v", err)
}
}
// shells contributes code for several shells and slots, in no order.
func shells() []Manifest {
return []Manifest{
{Module: "zsh-syntax-highlighting", Shell: []ShellCode{
{For: "zsh", Slot: "last", Code: "source /usr/share/zsh/plugins/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh"},
}},
{Module: "powerlevel10k", Shell: []ShellCode{
{For: "zsh", Slot: "first", Code: "if [[ -r \"${XDG_CACHE_HOME:-$HOME/.cache}/p10k-instant-prompt-${(%):-%n}.zsh\" ]]; then\n" +
" source \"${XDG_CACHE_HOME:-$HOME/.cache}/p10k-instant-prompt-${(%):-%n}.zsh\"\nfi\n"},
{For: "zsh", Slot: "normal", Code: "source ~/.local/share/powerlevel10k/powerlevel10k.zsh-theme"},
{For: "zsh", Slot: "normal", Code: "[[ -f ~/.local/share/powerlevel10k/p10k.zsh ]] && source ~/.local/share/powerlevel10k/p10k.zsh"},
}},
{Module: "zsh-autosuggestions", Shell: []ShellCode{
{For: "zsh", Slot: "normal", Code: "source /usr/share/zsh/plugins/zsh-autosuggestions/zsh-autosuggestions.zsh"},
{For: "bash", Slot: "normal", Code: "echo not for zsh"},
}},
{Module: "direnv", Shell: []ShellCode{
{For: "fish", Slot: "last", Code: "direnv hook fish | source"},
{For: "bash", Slot: "last", Code: "eval \"$(direnv hook bash)\""},
}},
}
}
// ADR 0204 §3: a slot holds that shell's code only, in module order, each module's pieces in the
// order it declared them under a line naming it; empty when nothing is contributed.
func TestShellCodeLandsInItsSlotInModuleOrderForItsShellOnly(t *testing.T) {
if got, want := shellCode(shells(), "zsh", "normal"), "# powerlevel10k\n"+
"source ~/.local/share/powerlevel10k/powerlevel10k.zsh-theme\n"+
"[[ -f ~/.local/share/powerlevel10k/p10k.zsh ]] && source ~/.local/share/powerlevel10k/p10k.zsh\n"+
"# zsh-autosuggestions\n"+
"source /usr/share/zsh/plugins/zsh-autosuggestions/zsh-autosuggestions.zsh\n"; got != want {
t.Fatalf("zsh's normal slot is\n%s\nnot\n%s", got, want)
}
if got, want := shellCode(shells(), "zsh", "last"), "# zsh-syntax-highlighting\n"+
"source /usr/share/zsh/plugins/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh\n"; got != want {
t.Fatalf("zsh's last slot is\n%s\nnot\n%s", got, want)
}
if got, want := shellCode(shells(), "bash", "last"), "# direnv\neval \"$(direnv hook bash)\"\n"; got != want {
t.Fatalf("bash's last slot is %q, not %q", got, want)
}
if got := shellCode(shells(), "fish", "first"); got != "" {
t.Fatalf("a slot nobody contributed to holds %q", got)
}
}
// The holder of node-login-shell, as WP3's zsh module writes its block, with its own zsh around the
// slots — which holds `${…}` of the shell's own that no mesh pass may touch either.
func zshHolder() Manifest {
return Manifest{Module: "zsh", Claims: []Claim{{Name: LoginShellSeat, Scope: ScopeNode}},
Resources: []map[string]any{
{"id": "zshrc", "type": "file", "path": "${machine:account-home}/.zshrc", "content": "" +
"${shell:zsh:first}" +
"PROMPT='%n@%m ${PWD/#$HOME/~} '\n" +
"${shell:zsh:normal}" +
"alias ll='ls -l'\n" +
"${shell:zsh:last}"},
}}
}
// The case the ordering exists for: contributed zsh code full of `${…}` reaches the file byte for
// byte, because the shell's code is placed after every other placeholder pass and in one pass — a
// scanner for the mesh's placeholders that ran after it would read `${XDG_CACHE_HOME:-…}` and
// `${(%):-%n}` as the mesh's, or fill a `${machine:…}` some module wrote for its shell to see.
func TestShellCodeReachesTheHoldersFileByteForByte(t *testing.T) {
modules := append(shells(), zshHolder(), Manifest{Module: "sly", Shell: []ShellCode{
{For: "zsh", Slot: "last", Code: "echo ${machine:account-home} ${secret:x} ${shell:zsh:first} ${environment:posix}"},
}})
r := Resolution{Node: "workstation", Account: "op", Modules: modules}
out, err := r.Declaration(Rendering{})
if err != nil {
t.Fatal(err)
}
var zshrc map[string]any
for _, res := range out {
if res["id"] == "zsh.zshrc" {
zshrc = res
}
}
if zshrc == nil {
t.Fatalf("the holder's file was not composed: %v", out)
}
if zshrc["path"] != "/home/op/.zshrc" {
t.Fatalf("the holder's own placeholders were not filled first: %v", zshrc["path"])
}
want := "# powerlevel10k\n" +
"if [[ -r \"${XDG_CACHE_HOME:-$HOME/.cache}/p10k-instant-prompt-${(%):-%n}.zsh\" ]]; then\n" +
" source \"${XDG_CACHE_HOME:-$HOME/.cache}/p10k-instant-prompt-${(%):-%n}.zsh\"\nfi\n" +
"PROMPT='%n@%m ${PWD/#$HOME/~} '\n" +
"# powerlevel10k\n" +
"source ~/.local/share/powerlevel10k/powerlevel10k.zsh-theme\n" +
"[[ -f ~/.local/share/powerlevel10k/p10k.zsh ]] && source ~/.local/share/powerlevel10k/p10k.zsh\n" +
"# zsh-autosuggestions\n" +
"source /usr/share/zsh/plugins/zsh-autosuggestions/zsh-autosuggestions.zsh\n" +
"alias ll='ls -l'\n" +
"# sly\n" +
"echo ${machine:account-home} ${secret:x} ${shell:zsh:first} ${environment:posix}\n" +
"# zsh-syntax-highlighting\n" +
"source /usr/share/zsh/plugins/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh\n"
if got := zshrc["content"]; got != want {
t.Fatalf("the holder's .zshrc is\n%s\nnot\n%s", got, want)
}
}
// The holder of node-environment places both renderings, and they are the same as rendered alone.
func TestTheEnvironmentHolderPlacesBothRenderings(t *testing.T) {
holder := Manifest{Module: "node-env", Claims: []Claim{{Name: EnvironmentSeat, Scope: ScopeNode}},
Resources: []map[string]any{
{"id": "posix", "type": "file", "path": "${machine:account-home}/.config/mesh/environment.sh",
"content": "# The mesh's environment.\n${environment:posix}"},
{"id": "systemd", "type": "file", "path": "${machine:account-home}/.config/environment.d/50-mesh.conf",
"content": "${environment:systemd}"},
}}
r := Resolution{Node: "workstation", Account: "op", Modules: append(contributors(), holder)}
out, err := r.Declaration(Rendering{})
if err != nil {
t.Fatal(err)
}
by := map[string]any{}
for _, res := range out {
by[res["id"].(string)] = res["content"]
}
if by["node-env.posix"] != "# The mesh's environment.\n"+composedPOSIX {
t.Fatalf("the POSIX file is\n%v", by["node-env.posix"])
}
if by["node-env.systemd"] != composedSystemd {
t.Fatalf("the environment.d file is\n%v", by["node-env.systemd"])
}
}
// ADR 0203 §5 and ADR 0204 §3: a placeholder outside the seat's holder is refused — by the parser,
// which is what the catalogue check and registration run, and again at composition, in the same words.
func TestAPlaceholderOutsideTheHolderIsRefused(t *testing.T) {
for _, c := range []struct{ content, want string }{
{"${environment:posix}", "toolchain's resource rc names ${environment:posix} and toolchain does not claim node-environment"},
{"${shell:zsh:normal}", "toolchain's resource rc names ${shell:zsh:normal} and toolchain does not claim node-login-shell"},
} {
raw := `{"module":"toolchain","resources":[{"id":"rc","type":"file","path":"/etc/rc","content":"` + c.content + `"}]}`
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("the catalogue check accepted %s outside its holder: %v", c.content, err)
}
m := Manifest{Module: "toolchain", Resources: []map[string]any{
{"id": "rc", "type": "file", "path": "/etc/rc", "content": c.content}}}
r := Resolution{Node: "workstation", Account: "op", Modules: []Manifest{m}}
if _, err := r.Declaration(Rendering{}); err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("composition accepted %s outside its holder: %v", c.content, err)
}
}
}
// A key nobody renders is refused, not left in the file as a literal.
func TestAnUnknownPlaceholderKeyIsRefused(t *testing.T) {
for _, c := range []struct{ content, want string }{
{"${environment:foo}", "names ${environment:foo}; the environment is ${environment:posix} or ${environment:systemd}"},
{"${shell:zsh:middle}", "names ${shell:zsh:middle}; shell code is ${shell:<shell>:<slot>}"},
{"${shell:tcsh:first}", "names ${shell:tcsh:first}; shell code is ${shell:<shell>:<slot>}"},
{"${shell:zsh}", "names ${shell:zsh}; shell code is ${shell:<shell>:<slot>}"},
} {
raw := `{"module":"holder","claims":[{"name":"node-environment","scope":"node"},{"name":"node-login-shell","scope":"node"}],` +
`"resources":[{"id":"rc","type":"file","path":"/etc/rc","content":"` + c.content + `"}]}`
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s was accepted: %v", c.content, err)
}
}
// And outside a file's content, where nothing could be placed.
raw := `{"module":"holder","claims":[{"name":"node-environment","scope":"node"}],` +
`"resources":[{"id":"rc","type":"file","path":"/etc/${environment:posix}","content":"x"}]}`
if _, err := ParseManifest([]byte(raw)); err == nil ||
!strings.Contains(err.Error(), "names ${environment:posix} in its path; the environment and the shell's code are placed only in a file's content") {
t.Errorf("a placeholder in a path was accepted: %v", err)
}
}
// What ADR 0203 §2 allows a contribution to say, refused at parse when it says anything else.
func TestAMalformedEnvironmentIsRefusedAtParse(t *testing.T) {
for _, c := range []struct{ environment, want string }{
{`{"variables":{"1X":"a"}}`, `tool sets the variable "1X", which is not a name a shell accepts`},
{`{"variables":{"MY-VAR":"a"}}`, `tool sets the variable "MY-VAR", which is not a name a shell accepts`},
{`{"variables":{"PATH":"/bin"}}`, `tool sets PATH as a variable; a module adds an entry under environment.path`},
{`{"variables":{"A":"$HOME/x"}}`, `tool sets A to "$HOME/x", which holds a $ that is not one of the machine's ${machine:…} facts`},
{`{"variables":{"A":"${HOME}/x"}}`, `tool sets A to "${HOME}/x", which holds a $`},
{`{"variables":{"A":"it's"}}`, `tool sets A to "it's", which holds a quote`},
{`{"variables":{"A":"say \"hi\""}}`, `which holds a quote`},
{`{"variables":{"A":"a\\b"}}`, `which holds a backslash`},
{`{"variables":{"A":"a\nb"}}`, `which holds a line break`},
{`{"variables":{"A":"a\u0000b"}}`, `which holds a NUL`},
{`{"path":[{"entry":"","at":"start"}]}`, `tool's PATH entry 1 names no directory`},
{`{"path":[{"entry":"/a:/b","at":"start"}]}`, `tool puts "/a:/b" on PATH, which holds a colon`},
{`{"path":[{"entry":"$HOME/bin","at":"start"}]}`, `tool puts "$HOME/bin" on PATH, which holds a $`},
{`{"path":[{"entry":"/a","at":"middle"}]}`, `tool puts "/a" on PATH at "middle"; an entry goes at "start" or "end"`},
{`{"path":[{"entry":"/a"}]}`, `tool puts "/a" on PATH at ""`},
{`{"path":[{"entry":"/a","at":"start"},{"entry":"/a","at":"end"}]}`, `tool puts "/a" on PATH twice`},
{`{"variables":{"A":"x"},"paths":[]}`, `unknown field "paths"`},
} {
_, err := ParseManifest([]byte(`{"module":"tool","environment":` + c.environment + `}`))
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s: want %q, got %v", c.environment, c.want, err)
}
}
// What is allowed: a literal, and the machine's own facts.
if _, err := ParseManifest([]byte(`{"module":"tool","environment":{` +
`"variables":{"GOPATH":"${machine:account-home}/go","DISABLE_X":"1","ANSWER":"a b+c=d"},` +
`"path":[{"entry":"${machine:account-home}/go/bin","at":"start"},{"entry":"/opt/x","at":"end"}]}}`)); err != nil {
t.Fatalf("a well-formed environment was refused: %v", err)
}
}
func TestMalformedShellCodeIsRefusedAtParse(t *testing.T) {
for _, c := range []struct{ shell, want string }{
{`[{"for":"tcsh","slot":"normal","code":"x"}]`, `tool's shell code 1 is for "tcsh"; the shells are zsh, bash, fish`},
{`[{"for":"zsh","slot":"middle","code":"x"}]`, `tool's shell code 1 goes in the slot "middle"; the slots are first, normal, last`},
{`[{"for":"zsh","slot":"last","code":"x"},{"for":"zsh","slot":"last","code":" \n"}]`, `tool's shell code 2 has no code`},
{`[{"for":"zsh","slot":"last","code":"x","order":1}]`, `unknown field "order"`},
} {
_, err := ParseManifest([]byte(`{"module":"tool","shell":` + c.shell + `}`))
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s: want %q, got %v", c.shell, c.want, err)
}
}
// The code itself is never judged: a shell's own `${…}` is not the mesh's.
if _, err := ParseManifest([]byte(`{"module":"tool","shell":[{"for":"zsh","slot":"first",` +
`"code":"source \"${XDG_CACHE_HOME:-$HOME/.cache}/p10k-instant-prompt-${(%):-%n}.zsh\""}]}`)); err != nil {
t.Fatalf("shell code was judged as if it were the mesh's: %v", err)
}
}
// ADR 0203 §1 and ADR 0204 §1: both seats are the mesh's own, held once per machine; the login
// shell's contract is `execute`, described and with a schema an agent can call.
func TestTheSeatTableCarriesTheEnvironmentAndTheLoginShell(t *testing.T) {
env, ok := SeatNamed("node-environment")
if !ok || env.Scope != ScopeNode || env.Decision != "novox/hq ADR 0203" ||
len(env.Serves)+len(env.Accepts)+len(env.Emits) != 0 || env.Delivers != "" {
t.Fatalf("node-environment is not a node seat with no protocol: %+v (defined %v)", env, ok)
}
shell, ok := SeatNamed("node-login-shell")
if !ok || shell.Scope != ScopeNode || shell.Decision != "novox/hq ADR 0204" {
t.Fatalf("node-login-shell is not a node seat: %+v (defined %v)", shell, ok)
}
if len(shell.Serves) != 1 || shell.Serves[0].Name != "execute" || shell.Serves[0].Description == "" {
t.Fatalf("the login shell serves %+v, not execute alone", shell.Serves)
}
props, _ := shell.Serves[0].Input["properties"].(map[string]any)
required, _ := shell.Serves[0].Input["required"].([]string)
if _, has := props["command"]; !has || len(required) != 1 || required[0] != "command" {
t.Fatalf("execute does not require a command: %v", shell.Serves[0].Input)
}
if _, has := props["timeout_seconds"]; !has {
t.Fatalf("execute takes no timeout: %v", props)
}
}
// ADR 0204 §1: the login shell is the mesh's, so no module declares it — neither under the mesh's
// name nor under the name a module gave it before.
func TestNoModuleMayDeclareTheLoginShell(t *testing.T) {
for _, n := range []string{"login-shell", "node-login-shell", "node-environment"} {
raw := `{"module":"zsh","seats":[{"name":"` + n + `","scope":"node","serves":["execute"]}],"tools":["execute"]}`
_, err := ParseManifest([]byte(raw))
if err == nil {
t.Errorf("a module declaring %q was accepted", n)
}
}
got := strings.Join(declaredSeatProblems(Manifest{Module: "zsh",
DefinesSeats: []SeatDeclaration{{Name: "login-shell", Scope: ScopeNode}}}), "; ")
if !strings.Contains(got, `zsh declares a seat named "login-shell"; the login shell is the mesh's own seat node-login-shell`) {
t.Fatalf("declaring login-shell was not refused by name: %q", got)
}
// And a shell module claiming the mesh's seat, serving execute, is what the seat is for.
m, err := ParseManifest([]byte(`{"module":"zsh","tools":["execute"],` +
`"claims":[{"name":"node-login-shell","scope":"node"}]}`))
if err != nil {
t.Fatal(err)
}
if err := CanHold(m, Seat{Name: LoginShellSeat, Scope: ScopeNode, Serves: loginShellVerbs()}); err != nil {
t.Fatalf("a shell module claiming the seat cannot hold it: %v", err)
}
}
+23 -17
View File
@@ -1,6 +1,7 @@
package catalogue
import (
"encoding/json"
"fmt"
"os"
"reflect"
@@ -170,7 +171,7 @@ func TestTheForgeHoldsTheNpmAndGitSeats(t *testing.T) {
// **And the forge's own address follows it**, composed from the manifest in the catalogue beside
// this checkout (novox/hq 04-ISSUES/088).
//
// The forge is reached a third way that neither test above covers: by its own sidecar, over the
// The forge is reached a third way that neither test above covers: by its own code, over the
// machine's loopback, told where to go in its environment. The `2999:3000` mapping that lets the
// forge go on binding 3000 does nothing for a caller dialling the machine — so a literal there is
// wrong on every node whose assignment differs, and wrong for a second reason on a node given the
@@ -178,13 +179,13 @@ func TestTheForgeHoldsTheNpmAndGitSeats(t *testing.T) {
// in an `env` at all is a declaration, not a manifest.
func TestTheForgesOwnAddressFollowsThePortTheNodeGaveIt(t *testing.T) {
forge, err := catalogueManifest(t, "gitea").Resolve([]Built{{
Name: "runtime", Kind: ArtifactImage,
Reference: "registry.example/gitea-runtime@sha256:" + strings.Repeat("a", 64),
Name: "code", Kind: ArtifactBundle,
Reference: ArtifactStoreScheme + "gitea/code/blobs/" + bundleDigest, Digest: bundleDigest,
}})
if err != nil {
t.Fatalf("the forge's manifest does not resolve against its own build: %v", err)
}
r := Resolution{Node: "anchor", Modules: []Manifest{forge}, Needs: []Needed{
r := Resolution{Node: "anchor", Modules: []Manifest{forge, theRuntime(t)}, Needs: []Needed{
{Name: "postgres-database", For: "gitea", From: "anchor", At: "127.0.0.1",
Serves: map[string]any{"port": float64(5432)}, Sealed: "sealed-db"},
{Name: "route", For: "gitea", From: "anchor"},
@@ -194,8 +195,8 @@ func TestTheForgesOwnAddressFollowsThePortTheNodeGaveIt(t *testing.T) {
// The number this node was given for the forge — the one the machine it is about to run on
// already publishes.
out, err := r.Declaration(Rendering{
Needed: map[string]map[string]string{"gitea": {"broker": "sealed-broker"}},
out, err := r.Declaration(Rendering{ArtifactStore: "anchor.internal:5101",
Needed: map[string]map[string]string{RuntimeModule: {"broker": "sealed-broker"}},
Given: map[string]map[int]int{"gitea": {3000: 2999}},
})
if err != nil {
@@ -210,14 +211,19 @@ func TestTheForgesOwnAddressFollowsThePortTheNodeGaveIt(t *testing.T) {
if published := fmt.Sprint(server["ports"]); !strings.Contains(published, "2999:3000") {
t.Fatalf("the forge is not published on the port this node gave it: %v", server["ports"])
}
runtime := fileNamed(out, "gitea.runtime")
// The forge's own code runs in the node's runtime (novox/hq ADR 0198), given its words there.
runtime := fileNamed(out, RuntimeModule+"."+RuntimeProcessID())
if runtime == nil {
t.Fatalf("the forge's sidecar is not in the declaration: %v", out)
t.Fatalf("the node's runtime is not in the declaration: %v", ids(out))
}
env, _ := runtime["env"].(map[string]any)
if env["MESH_GITEA_URL"] != "http://127.0.0.1:2999" {
t.Fatalf("the forge's sidecar dials %v while the machine publishes the forge on 2999 — "+
"whatever reads it dials a dead port", env["MESH_GITEA_URL"])
env, _ := runtime["env"].(map[string]string)
var given map[string]map[string]string
if err := json.Unmarshal([]byte(env[RuntimeToolEnv]), &given); err != nil {
t.Fatalf("the runtime's %s is not JSON: %q", RuntimeToolEnv, env[RuntimeToolEnv])
}
if given["gitea"]["MESH_GITEA_URL"] != "http://127.0.0.1:2999" {
t.Fatalf("the forge's code dials %v while the machine publishes the forge on 2999 — "+
"whatever reads it dials a dead port", given["gitea"]["MESH_GITEA_URL"])
}
}
@@ -229,13 +235,13 @@ func declaredGiteaSsh(t *testing.T, given map[int]int) map[string]any {
t.Helper()
forge := catalogueManifest(t, "gitea")
resolved, err := forge.Resolve([]Built{{
Name: "runtime", Kind: ArtifactImage,
Reference: "registry.example/gitea-runtime@sha256:" + strings.Repeat("a", 64),
Name: "code", Kind: ArtifactBundle,
Reference: ArtifactStoreScheme + "gitea/code/blobs/" + bundleDigest, Digest: bundleDigest,
}})
if err != nil {
t.Fatalf("the forge's manifest does not resolve against its own build: %v", err)
}
r := Resolution{Node: "anchor", Modules: []Manifest{resolved}, Needs: []Needed{
r := Resolution{Node: "anchor", Modules: []Manifest{resolved, theRuntime(t)}, Needs: []Needed{
{Name: "postgres-database", For: "gitea", From: "anchor", At: "127.0.0.1",
Serves: map[string]any{"port": float64(5432)}, Sealed: "sealed-db"},
{Name: "route", For: "gitea", From: "anchor"},
@@ -246,8 +252,8 @@ func declaredGiteaSsh(t *testing.T, given map[int]int) map[string]any {
for k, v := range given {
givenPorts[k] = v
}
out, err := r.Declaration(Rendering{
Needed: map[string]map[string]string{"gitea": {"broker": "sealed-broker"}},
out, err := r.Declaration(Rendering{ArtifactStore: "anchor.internal:5101",
Needed: map[string]map[string]string{RuntimeModule: {"broker": "sealed-broker"}},
Ports: map[string]map[int]int{"gitea": givenPorts},
Given: map[string]map[int]int{"gitea": given},
})
+29
View File
@@ -0,0 +1,29 @@
package catalogue
import (
"os"
"regexp"
"testing"
)
// novox/hq issue 223: genesis raises the controller as a container built from this repository's own
// Dockerfile, with no build arguments — the manifest no longer builds an image, so nothing passes a
// base in. The Dockerfile's own default must therefore be a Go that builds this module, pinned by
// digest, and the replacement the manifest's process names must be the container genesis raises.
func TestGenesisCanBuildTheControllersImageAsItStands(t *testing.T) {
raw, err := os.ReadFile("../../Dockerfile")
if err != nil {
t.Fatal(err)
}
if !regexp.MustCompile(`(?m)^ARG GO_BASE=golang@sha256:[0-9a-f]{64}$`).Match(raw) {
t.Fatal("the Dockerfile's default Go base is not pinned by digest; genesis builds it with no arguments")
}
makefile, err := os.ReadFile("../../Makefile")
if err != nil {
t.Fatal(err)
}
pin := regexp.MustCompile(`golang@sha256:[0-9a-f]{64}`)
if string(pin.Find(raw)) != string(pin.Find(makefile)) {
t.Errorf("the Dockerfile and the Makefile build on different Go: %s, %s", pin.Find(raw), pin.Find(makefile))
}
}
@@ -0,0 +1,103 @@
package catalogue
import (
"strings"
"testing"
)
// A grant secret is read by whatever provisions, and that stopped being root (novox/hq issue 225).
//
// The mesh seals one credential per consumer beside the provider's contributions file. The
// provider's harness reads both: the file to learn who asked, the secret to set their password.
// While a module's own code ran in a container as root, a root-owned 0600 file was readable by
// the thing that needed it. ADR 0198 moved that code under the node's runtime, which runs as the
// operator's account — and the secret stayed root's.
//
// **The cost was silence.** The harness says `secret not readable yet`, which is true and
// ordinary on the first pass, so four thousand refusals in three hours read as patience. No user
// was ever created, and two consumers crash-looped against a database that had never heard of
// them.
//
// The same reasoning is already written for a module's *own* secrets, three hundred lines above:
// "a root-owned 0600 file is one that process cannot read". This is that rule reaching the other
// kind of secret the mesh writes for a module.
// aProviderWithABundle is a provider whose code is a bundle the node's runtime runs — the shape
// every TypeScript provisioner has since ADR 0198.
func aProviderWithABundle() Manifest {
return Manifest{
Module: "mongodb", Version: "1",
Provides: FromAnywhere("mongodb-database"),
Receives: map[string]string{"mongodb-database": "/var/lib/mongodb/grants/mesh.json"},
Grants: map[string]string{"mongodb-database": "/var/lib/mongodb/grants"},
Bundles: []Bundle{{Name: "code", Language: "typescript"}},
Resources: []map[string]any{{
"id": "server", "type": "container", "name": "mongodb-server",
"image": "mongo@sha256:" + strings.Repeat("a", 64),
}},
}
}
func TestAGrantSecretIsOwnedByTheAccountThatProvisions(t *testing.T) {
r, err := Resolve(shelf(aProviderWithABundle()), []string{"mongodb"}, reachable(), World{})
if err != nil {
t.Fatal(err)
}
r.Account = "operator"
out, err := r.Declaration(Rendering{Grants: []Grant{{
Provision: "mongodb-database", Consumer: "workstation", From: "photos", Slug: "photos",
Values: map[string]any{}, Sealed: "c2VhbGVk",
}}})
if err != nil {
t.Fatal(err)
}
var secret map[string]any
for _, res := range out {
if res["type"] == "file" && strings.HasSuffix(fmtPath(res), ".secret") {
secret = res
}
}
if secret == nil {
t.Fatalf("no grant secret was composed at all: %v", out)
}
if got := secret["owner"]; got != "operator" {
t.Fatalf("the grant secret at %v belongs to %v; the provisioner runs as %q and a "+
"root-owned 0600 file is one it cannot read — which is silent, because the harness "+
"calls it \"not readable yet\"", fmtPath(secret), got, "operator")
}
}
// And a provider whose code still runs in a container keeps the owner it declares, so this
// changes nothing for the modules the runtime has not taken.
func TestAContainerProvidersGrantSecretKeepsItsDeclaredOwner(t *testing.T) {
m := aProviderWithABundle()
m.Bundles = nil
m.SecretsOwner = "65534:65534"
r, err := Resolve(shelf(m), []string{"mongodb"}, reachable(), World{})
if err != nil {
t.Fatal(err)
}
r.Account = "operator"
out, err := r.Declaration(Rendering{Grants: []Grant{{
Provision: "mongodb-database", Consumer: "workstation", From: "photos", Slug: "photos",
Values: map[string]any{}, Sealed: "c2VhbGVk",
}}})
if err != nil {
t.Fatal(err)
}
for _, res := range out {
if res["type"] == "file" && strings.HasSuffix(fmtPath(res), ".secret") {
if got := res["owner"]; got != "65534:65534" {
t.Fatalf("a container provider's grant secret belongs to %v, not what it declares", got)
}
return
}
}
t.Fatal("no grant secret was composed")
}
func fmtPath(r map[string]any) string {
p, _ := r["path"].(string)
return p
}
+109
View File
@@ -0,0 +1,109 @@
package catalogue
// The graphical session's seats (novox/hq ADR 0208): one module per piece of software, each piece's
// role a node seat in the mesh's own set, so i3 and sway, xterm and foot, rofi and dmenu compete for
// a role rather than each inventing one — and a machine running two of one role is refused at
// assignment instead of found by two bars on one screen.
const (
LoginManagerSeat = "node-login-manager"
DisplayServerSeat = "node-display-server"
DisplaySessionSeat = "node-display-session"
TerminalEmulatorSeat = "node-terminal-emulator"
LauncherSeat = "node-launcher"
NotifierSeat = "node-notifier"
LockScreenSeat = "node-lock-screen"
ClipboardSeat = "node-clipboard"
BarSeat = "node-bar"
CompositorSeat = "node-compositor"
SecretServiceSeat = "node-secret-service"
)
// graphicalSessionSeats are the eleven, in the order ADR 0208's table reads, each with the verbs
// research 026/05 starts it with. Three have none yet: the bar, the compositor and the secret
// service are roles a second holder competes for, and nothing has needed to ask them anything.
func graphicalSessionSeats() []Seat {
const decided = "novox/hq ADR 0208"
return []Seat{
{Name: LoginManagerSeat, Scope: ScopeNode, Decision: decided, Serves: []Verb{
{Name: "sessions", Description: "The sessions the login manager offers on this machine, and which " +
"one the operator account starts by default.",
Input: schema(map[string]string{}, nil)},
}},
{Name: DisplayServerSeat, Scope: ScopeNode, Decision: decided, Serves: []Verb{
{Name: "displays", Description: "The monitors connected now, each with its identity, its modes and " +
"where it is placed; and the layout profile in force, if one matches.",
Input: schema(map[string]string{}, nil)},
// Profiles are keyed by the monitors' identities and are the operator's data (ADR 0208 §6).
{Name: "layout", Description: "The monitor layout profiles, keyed by the connected monitors' " +
"identities: list them, save the current arrangement under a name, or apply one.",
Input: withEnum(schema(map[string]string{
"action": "list, save or apply",
"name": "the profile to save or apply (save and apply only)",
}, []string{"action"}), "action", "list", "save", "apply")},
}},
// It receives window-manager configuration lines from every other module (novox/hq ADR 0212):
// bindings, start-up commands, rules — placed by the session's holder, never written into
// its directory by the contributor.
{Name: DisplaySessionSeat, Scope: ScopeNode, Decision: decided,
Receives: []Receivable{{Kind: "config", Comment: "#"}}, Serves: []Verb{
{Name: "reload", Description: "Reload the session's configuration in place, keeping its windows.",
Input: schema(map[string]string{}, nil)},
{Name: "workspaces", Description: "The session's workspaces: each one's name, output, and whether " +
"it is visible or focused.",
Input: schema(map[string]string{}, nil)},
{Name: "windows", Description: "The session's windows: each one's title, class, workspace and " +
"whether it has focus; narrowed to one workspace when named.",
Input: schema(map[string]string{"workspace": "one workspace (optional)"}, nil)},
}},
{Name: TerminalEmulatorSeat, Scope: ScopeNode, Decision: decided, Serves: []Verb{
{Name: "open", Description: "Open a terminal window in the operator's session, running a command " +
"or the login shell, in a directory or the account's home.",
Input: schema(map[string]string{
"command": "what to run in it (optional; the login shell when absent)",
"directory": "where it starts (optional; the account's home when absent)",
}, nil)},
}},
{Name: LauncherSeat, Scope: ScopeNode, Decision: decided, Serves: []Verb{
{Name: "menu", Description: "Put a menu of choices in front of the operator and answer with the " +
"one chosen, or nothing when the menu was dismissed — the dmenu-compatible contract.",
Input: map[string]any{"type": "object", "required": []string{"choices"},
"properties": map[string]any{
"choices": map[string]any{"type": "array", "items": map[string]any{"type": "string"},
"description": "the lines to choose between, in order"},
"prompt": map[string]any{"type": "string", "description": "what the menu asks (optional)"},
}}},
}},
{Name: NotifierSeat, Scope: ScopeNode, Decision: decided, Serves: []Verb{
{Name: "send", Description: "Show the operator a notification.",
Input: withEnum(schema(map[string]string{
"title": "the notification's summary",
"body": "its text (optional)",
"urgency": "low, normal (the default) or critical",
}, []string{"title"}), "urgency", "low", "normal", "critical")},
{Name: "history", Description: "The notifications shown lately, newest first.",
Input: schema(map[string]string{"limit": "how many (optional, default 20)"}, nil)},
}},
{Name: LockScreenSeat, Scope: ScopeNode, Decision: decided, Serves: []Verb{
{Name: "lock", Description: "Lock the operator's session now.",
Input: schema(map[string]string{}, nil)},
}},
{Name: ClipboardSeat, Scope: ScopeNode, Decision: decided, Serves: []Verb{
{Name: "history", Description: "What the clipboard held lately, newest first.",
Input: schema(map[string]string{"limit": "how many (optional, default 20)"}, nil)},
{Name: "copy", Description: "Put text on the operator's clipboard.",
Input: schema(map[string]string{"text": "the text"}, []string{"text"})},
}},
{Name: BarSeat, Scope: ScopeNode, Decision: decided},
{Name: CompositorSeat, Scope: ScopeNode, Decision: decided},
{Name: SecretServiceSeat, Scope: ScopeNode, Decision: decided},
}
}
// withEnum narrows one string property of a schema to the values it may take, so a caller is told
// the choices by the schema rather than by a refusal.
func withEnum(s map[string]any, property string, values ...string) map[string]any {
props := s["properties"].(map[string]any)
p := props[property].(map[string]any)
p["enum"] = values
return s
}
@@ -0,0 +1,213 @@
package catalogue
import (
"encoding/json"
"reflect"
"strings"
"testing"
)
// Defends novox/hq ADR 0208: the graphical session is one module per piece, on the mesh's seats.
// §2: the eleven roles are the mesh's own node seats, each with the verbs it starts with.
func TestTheGraphicalSessionsSeatsAreTheMeshsOwnWithTheirVerbs(t *testing.T) {
want := map[string][]string{
LoginManagerSeat: {"sessions"},
DisplayServerSeat: {"displays", "layout"},
DisplaySessionSeat: {"reload", "workspaces", "windows"},
TerminalEmulatorSeat: {"open"},
LauncherSeat: {"menu"},
NotifierSeat: {"send", "history"},
LockScreenSeat: {"lock"},
ClipboardSeat: {"history", "copy"},
BarSeat: nil,
CompositorSeat: nil,
SecretServiceSeat: nil,
}
for name, verbs := range want {
s, ok := SeatNamed(name)
if !ok {
t.Errorf("%s is not in the mesh's set", name)
continue
}
if s.Scope != ScopeNode || s.Decision != "novox/hq ADR 0208" {
t.Errorf("%s is %s-scoped under %q", name, s.Scope, s.Decision)
}
var got []string
for _, v := range s.Serves {
got = append(got, v.Name)
if v.Description == "" || v.Input["type"] != "object" {
t.Errorf("%s.%s has no description or no object schema", name, v.Name)
}
}
if !reflect.DeepEqual(got, verbs) {
t.Errorf("%s serves %v, want %v", name, got, verbs)
}
}
// The launcher's menu takes a list, and the layout verb says its actions.
menu, _ := SeatNamed(LauncherSeat)
choices := menu.Serves[0].Input["properties"].(map[string]any)["choices"].(map[string]any)
if choices["type"] != "array" {
t.Errorf("menu's choices are %v, not a list", choices["type"])
}
display, _ := SeatNamed(DisplayServerSeat)
action := display.Serves[1].Input["properties"].(map[string]any)["action"].(map[string]any)
if !reflect.DeepEqual(action["enum"], []string{"list", "save", "apply"}) {
t.Errorf("layout's actions are %v", action["enum"])
}
// And they survive the store's JSON, which is where the live set comes from.
if _, err := json.Marshal(graphicalSessionSeats()); err != nil {
t.Fatal(err)
}
}
// §2: a module may claim one of them, and may not declare it as its own.
func TestNoModuleMayDeclareAGraphicalSessionSeat(t *testing.T) {
raw := `{"module":"xorg","seats":[{"name":"node-display-server","scope":"node"}]}`
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), "mesh's own namespace") {
t.Fatalf("a module declared node-display-server as its own: %v", err)
}
}
func displayServer(name, display string, claims ...string) Manifest {
m := Manifest{Module: name, Provides: []Offer{{Name: display, Reach: ReachMachine}}}
for _, c := range claims {
m.Claims = append(m.Claims, Claim{Name: c})
}
return m
}
func windowManager() Manifest {
return Manifest{Module: "i3", Requires: []string{"x11-display"}}
}
// §3: a display is resolved on the requiring module's own node.
func TestAMachineReachRequirementResolvesToTheProviderOnItsOwnNode(t *testing.T) {
cat := shelf(windowManager(), displayServer("xorg", "x11-display", DisplayServerSeat))
got, err := Resolve(cat, []string{"xorg", "i3"}, workstation(), World{})
if err != nil {
t.Fatalf("i3 beside xorg did not resolve: %v", err)
}
if !reflect.DeepEqual(names(got), []string{"xorg", "i3"}) && !reflect.DeepEqual(names(got), []string{"i3", "xorg"}) {
t.Errorf("resolved %v", names(got))
}
}
// §3: never answered by installing a provider, and never by another machine's.
func TestAMachineReachRequirementIsNotPulledInNorAnsweredFromAnotherNode(t *testing.T) {
cat := shelf(windowManager(),
displayServer("xorg", "x11-display", DisplayServerSeat),
displayServer("xwayland", "x11-display"))
// Another machine runs xorg and says so to the world; it does not count.
world := World{Offered: map[string][]Provider{
"x11-display": {{Node: "laptop", At: "laptop.mesh", Module: "xorg"}}}}
_, err := Resolve(cat, []string{"i3"}, workstation(), world)
if err == nil {
t.Fatal("i3 resolved on a machine with no display of its own")
}
for _, want := range []string{
`"x11-display" is wanted by i3`, "usable only on the machine that provides it",
"assign one to workstation", "xorg (holds node-display-server)", "xwayland",
} {
if !strings.Contains(err.Error(), want) {
t.Errorf("the refusal does not say %q:\n%v", want, err)
}
}
// With a single provider in the catalogue too: one candidate is still not a choice to make
// for somebody, unlike a node-scoped provision without the machine's reach.
_, err = Resolve(shelf(windowManager(), displayServer("xorg", "x11-display", DisplayServerSeat)),
[]string{"i3"}, workstation(), World{})
if err == nil {
t.Fatal("xorg was pulled in for i3")
}
// Not in the first pass, whose refusals take the machine off the network.
if _, err := Resolve(cat, []string{"i3"}, workstation(), World{Unchecked: true}); err != nil {
t.Errorf("the first pass refused: %v", err)
}
}
func TestTheMachinesReachIsAProvisionsOnlyReachAndIsNodeScoped(t *testing.T) {
for _, c := range []struct{ provides, want string }{
{`{"name":"x11-display","reach":"internal"}`, `with reach "internal"; a provision's reach is "machine" or nothing`},
{`{"name":"x11-display","scope":"mesh","reach":"machine"}`, `at scope "mesh" with the machine's reach`},
} {
_, err := ParseManifest([]byte(`{"module":"xorg","provides":[` + c.provides + `]}`))
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s: want %q, got %v", c.provides, c.want, err)
}
}
m, err := ParseManifest([]byte(`{"module":"xorg","provides":[{"name":"x11-display","reach":"machine"}]}`))
if err != nil {
t.Fatal(err)
}
if !m.Provides[0].MachineReach() {
t.Fatal("the reach was not read")
}
back, _ := json.Marshal(m.Provides[0])
if string(back) != `{"name":"x11-display","reach":"machine"}` {
t.Errorf("written back as %s", back)
}
}
func TestACatalogueDisagreeingAboutAProvisionsReachIsRefused(t *testing.T) {
cat := shelf(windowManager(), displayServer("xorg", "x11-display"),
Manifest{Module: "fake-x", Provides: Offers("x11-display")})
_, err := Resolve(cat, []string{"xorg", "i3"}, workstation(), World{})
if err == nil || !strings.Contains(err.Error(), `the catalogue disagrees about "x11-display"`) {
t.Fatalf("a provision with and without the machine's reach gave %v", err)
}
}
// §4: xinitrc and xresources slots, placed by the display server's holder alone.
func TestTheSessionsFilesArePlacedByTheDisplayServersHolderAlone(t *testing.T) {
xorg := Manifest{Module: "xorg", Claims: []Claim{{Name: DisplayServerSeat}},
Shell: []ShellCode{{For: "xinitrc", Slot: "first", Code: "xset s off"}},
Resources: []map[string]any{
{"id": "xinitrc", "type": "file", "path": "/home/op/.xinitrc",
"content": "${shell:xinitrc:first}${shell:xinitrc:normal}${shell:xinitrc:last}"},
{"id": "xresources", "type": "file", "path": "/home/op/.Xresources",
"content": "${shell:xresources:normal}"},
}}
i3 := Manifest{Module: "i3", Shell: []ShellCode{{For: "xinitrc", Slot: "last", Code: "exec i3"}}}
theme := Manifest{Module: "theme", Shell: []ShellCode{
{For: "xresources", Slot: "normal", Code: "Xft.dpi: 96"},
{For: "zsh", Slot: "normal", Code: "not for the session"},
}}
r := Resolution{Node: "workstation", Account: "op", Modules: []Manifest{xorg, i3, theme}}
out, err := r.Declaration(Rendering{})
if err != nil {
t.Fatal(err)
}
by := map[string]any{}
for _, res := range out {
by[res["id"].(string)] = res["content"]
}
if got := by["xorg.xinitrc"]; got != "# xorg\nxset s off\n# i3\nexec i3\n" {
t.Errorf("the .xinitrc is %q", got)
}
if got := by["xorg.xresources"]; got != "# theme\nXft.dpi: 96\n" {
t.Errorf("the .Xresources is %q", got)
}
// The contributions parse; the placeholders parse only in the holder.
if _, err := ParseManifest([]byte(`{"module":"i3","shell":[{"for":"xinitrc","slot":"last","code":"exec i3"},` +
`{"for":"xresources","slot":"normal","code":"i3.font: x"}]}`)); err != nil {
t.Fatalf("a session contribution was refused: %v", err)
}
for _, c := range []struct{ claims, content, want string }{
{``, "${shell:xinitrc:normal}", "does not claim node-display-server; every module's xinitrc is placed by the display server's holder alone"},
{`{"name":"node-login-shell"}`, "${shell:xresources:normal}", "does not claim node-display-server"},
{`{"name":"node-display-server"}`, "${shell:zsh:normal}", "does not claim node-login-shell"},
{`{"name":"node-display-server"}`, "${shell:xsession:normal}", "the session's file one of xinitrc, xresources"},
} {
raw := `{"module":"holder","claims":[` + c.claims + `],"resources":[{"id":"rc","type":"file","path":"/etc/rc","content":"` +
c.content + `"}]}`
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s with claims [%s]: want %q, got %v", c.content, c.claims, c.want, err)
}
}
if _, err := ParseManifest([]byte(`{"module":"xorg","claims":[{"name":"node-display-server"}],` +
`"resources":[{"id":"rc","type":"file","path":"/home/op/.xinitrc","content":"${shell:xinitrc:last}"}]}`)); err != nil {
t.Errorf("the display server's holder could not place the session's slots: %v", err)
}
}
+200 -5
View File
@@ -147,8 +147,17 @@ type Offer struct {
// shared by every consumer (novox/hq ADR 0158): software that holds one password or one key
// cannot give each consumer a login of its own. The named secret must say how it is taken.
Credential *OfferCredential `json:"credential,omitempty"`
// Reach is ReachMachine for a provision usable only on the provider's own machine — a display
// (novox/hq ADR 0208 §3). Node scope already keeps a provision off other machines; what this adds
// is that a requirement for it is never answered by installing a provider: the display server is
// a seat's holder gated by the machine's graphical session, and pulling one in for whatever asked
// is the misassignment research 026 found. Unmet, the requirement is refused naming who could.
Reach string `json:"reach,omitempty"`
}
// MachineReach is whether a provision is usable only on its provider's own machine.
func (o Offer) MachineReach() bool { return o.Reach == ReachMachine }
// OfferCredential names which of the provider's own secrets a provision's consumers receive.
type OfferCredential struct {
Own string `json:"own"`
@@ -196,27 +205,29 @@ func (o *Offer) UnmarshalJSON(raw []byte) error {
Name string `json:"name"`
Scope string `json:"scope,omitempty"`
Credential *OfferCredential `json:"credential,omitempty"`
Reach string `json:"reach,omitempty"`
}
dec := json.NewDecoder(bytes.NewReader(raw))
dec.DisallowUnknownFields()
if err := dec.Decode(&full); err != nil {
return fmt.Errorf("a provided name is either a string or {name, scope, credential}: %w", err)
return fmt.Errorf("a provided name is either a string or {name, scope, credential, reach}: %w", err)
}
o.Name, o.Scope, o.Credential = full.Name, full.Scope, full.Credential
o.Name, o.Scope, o.Credential, o.Reach = full.Name, full.Scope, full.Credential, full.Reach
return nil
}
// MarshalJSON writes back the short form when there is nothing else to say, so a manifest that
// went through the mesh comes out looking like the one that went in.
func (o Offer) MarshalJSON() ([]byte, error) {
if o.Scope == "" && o.Credential == nil {
if o.Scope == "" && o.Credential == nil && o.Reach == "" {
return json.Marshal(o.Name)
}
return json.Marshal(struct {
Name string `json:"name"`
Scope string `json:"scope,omitempty"`
Credential *OfferCredential `json:"credential,omitempty"`
}{o.Name, o.Scope, o.Credential})
Reach string `json:"reach,omitempty"`
}{o.Name, o.Scope, o.Credential, o.Reach})
}
// Manifest is everything a module says about itself.
@@ -325,6 +336,15 @@ type Manifest struct {
// person's account (design 25 §7) already had the same shape.
Invokes []string `json:"invokes,omitempty"`
// State is the current state this module keeps on the bus, by local name: each a key-value
// bucket the controller creates, which every instance of the module writes and reads
// (novox/hq ADR 0201). Not history — that is an event — and never a secret, sealed or not.
State []StateDeclaration `json:"state,omitempty"`
// Reads are other modules' state this module reads and watches, each `<module>.<name>`
// (novox/hq ADR 0201). Read-only: only the owner's instances write.
Reads []string `json:"reads,omitempty"`
// Capabilities the machine must have. A different field from Requires because the remedy
// differs: a missing module can be assigned, and a missing capability means the wrong
// machine.
@@ -508,6 +528,22 @@ type Manifest struct {
// holder. Like Filtering: one module per node gathers what every module declared and writes it.
Jailing *Jailing `json:"jailing,omitempty"`
// Environment is what this module adds to the operator account's environment: variables, and
// entries on PATH (novox/hq ADR 0203). Facts, not lines of one shell's syntax — the holder of
// node-environment places them, and the controller writes them in each reader's format. Like
// Jails: any module contributes, gathered from every module on the node, written by the holder.
Environment *Environment `json:"environment,omitempty"`
// Shell is code this module adds to the login shell's startup, for a named shell in a named
// slot (novox/hq ADR 0204). The controller never reads it: it is placed, in module order, where
// the holder of node-login-shell put the slot's placeholder.
Shell []ShellCode `json:"shell,omitempty"`
// Contributions are configuration this module gives the holder of a seat it does not hold, in
// that tool's own grammar (novox/hq ADR 0212): a seat, a kind the seat receives, and the text. The
// holder places them; the module depends on the seat (ADR 0210 §3).
Contributions []SeatContribution `json:"contributions,omitempty"`
// Guards are ports of this module's the mesh refuses on an adopted node except from the
// private network and from the machine itself (novox/hq ADR 0100) — the store's port and the
// broker's management port. The ports the software uses; the mesh guards where the machine
@@ -540,6 +576,10 @@ type Manifest struct {
// `restart-on` names to restart when the roster changes.
Facts map[string]RosterFile `json:"facts,omitempty"`
// Zone is the zone of names this module answers itself, and the listen that answers it (novox/hq
// ADR 0199). The mesh's resolver forwards the zone to it; nothing here names an address.
Zone *Zone `json:"zone,omitempty"`
// Certificate is where this module wants a certificate for its machine's name inside the
// mesh, and where the key that goes with it can be found.
//
@@ -757,6 +797,11 @@ type Artifact struct {
// list twice. A module declaring no tools has nothing the runtime loads, whatever it compiles.
Loads []string `json:"loads,omitempty"`
// External are packages a TypeScript bundle keeps as imports rather than inlining — a native
// addon, a package that reads its own files — and so carries the toolchain's node_modules for
// (novox/hq ADR 0193). Absent for nearly every bundle, which is then one file per entrypoint.
External []string `json:"external,omitempty"`
// Env is what a tools bundle is given on a machine (novox/hq ADR 0192): words and their values,
// paths and constants composed with ${dir:…} and ${port:…} exactly as a container's environment
// is, never a secret's content. The node's runtime hands it to this bundle and to no other.
@@ -1292,6 +1337,19 @@ func ParseManifest(raw []byte) (Manifest, error) {
"%s provides %q at scope %q; a provision is %q or %q",
m.Module, p, s, ScopeNode, ScopeMesh))
}
switch {
case offer.Reach == "":
case offer.Reach != ReachMachine:
// The one reach a provision has (novox/hq ADR 0208): a provision reached over the private
// network is mesh scope, and the world reaches nothing but a name.
problems = append(problems, fmt.Sprintf(
"%s provides %q with reach %q; a provision's reach is %q or nothing",
m.Module, p, offer.Reach, ReachMachine))
case offer.At() != ScopeNode:
problems = append(problems, fmt.Sprintf(
"%s provides %q at scope %q with the machine's reach; a provision usable only on its own "+
"machine is node-scoped (novox/hq ADR 0208)", m.Module, p, offer.At()))
}
if p == m.Module {
// Harmless and worth saying: a module always provides its own name, so writing it
// suggests the author expected it not to.
@@ -1311,6 +1369,8 @@ func ParseManifest(raw []byte) (Manifest, error) {
// module whose event names are wrong installs, starts, connects and reacts to nothing, with
// every log line saying it is fine (novox/hq 04-ISSUES/127).
problems = append(problems, EventProblems(m)...)
// And what it may call its state, and whose it may read (state.go, novox/hq ADR 0201).
problems = append(problems, StateProblems(m)...)
wellFormed := true
for _, c := range m.Claims {
if !name.MatchString(c.Name) {
@@ -1373,6 +1433,7 @@ func ParseManifest(raw []byte) (Manifest, error) {
}
}
problems = append(problems, m.Build.problems(m.Module)...)
problems = append(problems, undeliveredBundles(m)...)
// **What provides the artifact store cannot be delivered through it** (novox/hq 04-ISSUES/029).
//
// Building publishes to the store, and the builder will not start without one. So a module
@@ -1422,6 +1483,10 @@ func ParseManifest(raw []byte) (Manifest, error) {
"%s serves %q to whoever requires it, and does not provide it", m.Module, to))
}
}
// A served value may be derived for the consumer it is served to (novox/hq ADR 0201). Read
// here, where the definition is, rather than when somebody first requires it: a rule that
// would be refused at the first consumer is wrong from the moment it is written.
problems = append(problems, CheckServes(m)...)
for to, where := range m.Binds {
if !placedOrAbsolute(where) {
problems = append(problems, fmt.Sprintf(
@@ -1509,6 +1574,53 @@ func ParseManifest(raw []byte) (Manifest, error) {
"program that reads what the mesh delivered and reconciles",
m.Module, r["id"]))
}
// **What a process replaces is something the module no longer declares** (novox/hq issue 213).
// The host keeps it running until the process is, then removes it: so it is named by the id the
// module used to give it, it is never a resource the module still declares — that would be
// applied and removed by one declaration — and only a process that stays up has anything to
// hand over to. Said here, near the author, as the host would refuse it far away.
ids := map[string]bool{}
for _, r := range m.Resources {
ids[fmt.Sprint(r["id"])] = true
}
for _, r := range m.Resources {
raw, present := r["replaces"]
if !present {
continue
}
if fmt.Sprint(r["type"]) != "process" {
problems = append(problems, fmt.Sprintf(
"%s: %v says what it replaces, and only a process does", m.Module, r["id"]))
continue
}
if once, _ := r["run-once"].(bool); once || r["schedule"] != nil {
problems = append(problems, fmt.Sprintf(
"%s: %v replaces something and runs once or on a schedule — only a process that stays "+
"up is there a moment later to hand over to", m.Module, r["id"]))
}
list, ok := raw.([]any)
if !ok {
problems = append(problems, fmt.Sprintf(
"%s: %v replaces %v; replaces is a list of the ids this module no longer declares",
m.Module, r["id"], raw))
continue
}
for _, item := range list {
id, ok := item.(string)
switch {
case !ok || strings.TrimSpace(id) == "":
problems = append(problems, fmt.Sprintf(
"%s: %v replaces %v, which is not an id", m.Module, r["id"], item))
case strings.Contains(id, "."):
problems = append(problems, fmt.Sprintf(
"%s: %v replaces %q; a process replaces only a resource of its own module, named "+
"by its own id", m.Module, r["id"], id))
case ids[id]:
problems = append(problems, fmt.Sprintf(
"%s: %v replaces %q, which this module still declares", m.Module, r["id"], id))
}
}
}
// **A module that prepares its state must have code the mesh can run** (novox/hq ADR 0135). The
// preparation is the module's own program in its preparation mode, so it is derived from the
// resource that runs that program — and a module declaring none has asked for something the mesh
@@ -1516,7 +1628,7 @@ func ParseManifest(raw []byte) (Manifest, error) {
// quietly prepares nothing.
if m.Prepares && preparationTarget(m) == "" {
problems = append(problems, fmt.Sprintf(
"%s says it prepares its state, and declares no container running an artifact it built — "+
"%s says it prepares its state, and declares no container or process running an artifact it built — "+
"the preparation is this module's own program, so there has to be one for the mesh to "+
"run it in", m.Module))
}
@@ -1569,6 +1681,13 @@ func ParseManifest(raw []byte) (Manifest, error) {
}
}
}
// **A scheduled step may hold this module's own containers still while it runs**
// (novox/hq ADR 0189). What the host judges is the declaration it receives — whether each
// id is a container placed on that machine; what belongs here is what only the definition
// shows: that the ids are this module's, that they are containers, and that the step is
// scheduled. A module naming a neighbour's container would be a module that can stop the
// mesh, and the manifest is where that is visible.
problems = append(problems, whileStoppedProblems(m, r, hasSchedule(r))...)
}
for name, own := range m.OwnSecrets {
if !placedOrAbsolute(own.Path) {
@@ -1730,6 +1849,13 @@ func ParseManifest(raw []byte) (Manifest, error) {
problems = append(problems, m.unknownDirRefs()...)
problems = append(problems, m.unknownAccessRefs()...)
problems = append(problems, m.jailProblems()...)
// What a module adds to the account's environment and to the login shell, and the holder's
// placeholders for them (novox/hq ADR 0203, ADR 0204) — here, so the catalogue check refuses
// them in the words registration does.
problems = append(problems, m.environmentProblems()...)
problems = append(problems, m.shellProblems()...)
problems = append(problems, m.contributionPlaceholderProblems()...)
problems = append(problems, m.seatContributionProblems()...)
for i, r := range m.Resources {
id, _ := r["id"].(string)
@@ -1741,6 +1867,7 @@ func ParseManifest(raw []byte) (Manifest, error) {
}
}
problems = append(problems, zoneProblems(m)...)
if len(problems) > 0 {
sort.Strings(problems)
return Manifest{}, fmt.Errorf("this manifest cannot be used:\n - %s",
@@ -2100,3 +2227,71 @@ func (o OwnSecrets) Paths() map[string]string {
// InstancesInterchangeable is the one value of a definition's `instances`: the module is the same
// on every machine, so any instance may answer for the module.
const InstancesInterchangeable = "interchangeable"
// WhileStopped is the resource key naming the containers a scheduled step holds still while it
// runs (novox/hq ADR 0189). Carried to the host unchanged, like `schedule`.
const WhileStopped = "while-stopped"
// hasSchedule is whether a resource declares a cadence, as a string.
func hasSchedule(r map[string]any) bool {
s, _ := r["schedule"].(string)
return s != ""
}
// whileStoppedProblems judges one container's maintenance window against its own definition
// (novox/hq ADR 0189).
//
// Three things the manifest is the only place to see: that the step is scheduled (a one-time
// offline job says *before* rather than *instead of* — at apply the host already has a window,
// because the declaration is applied in order and a run-once step gates what follows); that every
// id it names is **this module's own** container; and that it does not name itself.
//
// The host checks the fourth — that the container is actually placed on that machine — because
// that is a fact about the declaration and not about the definition.
func whileStoppedProblems(m Manifest, r map[string]any, scheduled bool) []string {
raw, present := r[WhileStopped]
if !present {
return nil
}
ids, ok := raw.([]any)
if !ok {
return []string{fmt.Sprintf(
"%s declares %s on %v as a %T; it is a list of this module's container ids",
m.Module, WhileStopped, r["id"], raw)}
}
var problems []string
if len(ids) > 0 && !scheduled {
problems = append(problems, fmt.Sprintf(
"%s declares %s on %v, which has no schedule. A maintenance window is for a recurring "+
"step: at apply the mesh already has one, because a run-once step gates what is "+
"declared after it (novox/hq ADR 0189)", m.Module, WhileStopped, r["id"]))
}
containers := map[string]bool{}
for _, own := range m.Resources {
if fmt.Sprint(own["type"]) == "container" {
containers[fmt.Sprint(own["id"])] = true
}
}
for _, each := range ids {
id, ok := each.(string)
if !ok {
problems = append(problems, fmt.Sprintf(
"%s declares %s on %v naming a %T; each entry is a container's id",
m.Module, WhileStopped, r["id"], each))
continue
}
if id == fmt.Sprint(r["id"]) {
problems = append(problems, fmt.Sprintf(
"%s declares %s on %v naming itself", m.Module, WhileStopped, r["id"]))
continue
}
if !containers[id] {
problems = append(problems, fmt.Sprintf(
"%s declares %s on %v naming %q, which is not a container this module declares. "+
"A step may hold still its own module's containers and nobody else's — one "+
"that could quiesce a neighbour could stop the mesh",
m.Module, WhileStopped, r["id"], id))
}
}
return problems
}
+10 -5
View File
@@ -31,7 +31,7 @@ import (
//
// So a module asks. `${port:8080}` is "the machine-side port you gave me for the 8080 I said I
// listen on", and the module writes that where it would otherwise have written a literal — in a
// file's content, or in a value of a container's `env`.
// file's content, or in a value of a container's or a process's `env`.
//
// **The environment is filled by the control plane, exactly as a bound value is.** A port is not
// secret — the mesh holds it in the clear — so there is nothing for the host to be the only
@@ -64,7 +64,12 @@ func portsUsed(content string) []int {
}
// portInto replaces a resource's ${port:…} placeholders with what this machine assigned — in a
// file's content, and in a value of a container's environment.
// file's content, and in a value of a container's or a process's environment.
//
// **A process's environment is a container's** (novox/hq to-be 38 WP4c). A module's code moving out
// of its container becomes a process on the machine and still has to be told what the container
// was told; filled for one kind and not the other, the literal reached the process and was read as
// a port, and the modules that moved first wrote their run-once steps a 0600 env file instead.
//
// A port the module did not say it listens on is refused, for the same reason a binding's unknown
// key is: the module is asking about something it never declared, and the answer would be a guess.
@@ -84,7 +89,7 @@ func portInto(resource map[string]any, module string, listens []Listening, with
}
resource["content"] = filled
case "container":
case "container", "process":
env, ok := resource["env"].(map[string]any)
if !ok {
return nil
@@ -106,8 +111,8 @@ func portInto(resource map[string]any, module string, listens []Listening, with
continue
}
value, err := portsFilledInto(written,
fmt.Sprintf("%s's container %s sets %s to something that",
module, resource["name"], key), module, listens, with)
fmt.Sprintf("%s's %s %s sets %s to something that",
module, resource["type"], resource["name"], key), module, listens, with)
if err != nil {
return err
}
+80
View File
@@ -0,0 +1,80 @@
package catalogue
import (
"strings"
"testing"
)
// **A process's environment is composed as a container's is** (novox/hq to-be 38 WP4c).
//
// A module's code moving out of its container becomes a process on the machine, and what its
// container's environment asked for — the port this machine gave the module, the place it put the
// module's directory — it still has to be told. Filled for a container and not for a process, the
// literal `${port:8080}` reached the process as its environment and was read as a port; the modules
// that moved first wrote their run-once steps an env file instead.
func processModule(env map[string]any) Manifest {
return Manifest{
Module: "showcase",
Listens: []Listening{{Port: 8080, From: FromMesh}},
Resources: []map[string]any{
{"id": "data", "type": "directory", "mode": "0700"},
{"id": "setup", "type": "process", "name": "showcase-setup", "run-once": true,
"run": []any{"/usr/bin/showcase", "setup"}, "env": env},
},
}
}
func TestAProcessIsToldItsPortAndItsPlaceInItsEnvironment(t *testing.T) {
env := map[string]any{
"SHOWCASE_URL": "http://127.0.0.1:${port:8080}",
"SHOWCASE_DATA": "${dir:data}/objects",
"SHOWCASE_DB": "127.0.0.1:${seat:mesh-store:5432}",
"GREETING": "hello",
}
out, err := Resolution{Node: "anchor", Modules: []Manifest{processModule(env)}}.Declaration(Rendering{
Ports: map[string]map[int]int{"showcase": {8080: 21000}},
Seats: map[string]map[int]int{"mesh-store": {5432: 6852}},
})
if err != nil {
t.Fatalf("a process asking for its port and its place does not compose: %v", err)
}
setup := fileNamed(out, "showcase.setup")
if setup == nil {
t.Fatalf("the process is not in the declaration: %v", out)
}
got, _ := setup["env"].(map[string]any)
for key, want := range map[string]string{
"SHOWCASE_URL": "http://127.0.0.1:21000",
"SHOWCASE_DATA": "/var/lib/showcase/data/objects",
"SHOWCASE_DB": "127.0.0.1:6852",
"GREETING": "hello",
} {
if got[key] != want {
t.Errorf("the process is told %s=%v, want %q", key, got[key], want)
}
}
if env["SHOWCASE_URL"] != "http://127.0.0.1:${port:8080}" {
t.Fatalf("composing for one machine edited the module's own manifest: %v", env)
}
}
// An unknown reference in a process's environment is refused as a container's is, naming the
// process and the variable — left alone, it would reach the machine as a literal.
func TestAProcessAskingAboutAnUndeclaredPortIsRefused(t *testing.T) {
env := map[string]any{"SHOWCASE_URL": "http://127.0.0.1:${port:9999}"}
_, err := Resolution{Node: "anchor", Modules: []Manifest{processModule(env)}}.Declaration(Rendering{})
if err == nil {
t.Fatal("a process was told a port its module never said it listens on")
}
for _, said := range []string{"showcase-setup", "SHOWCASE_URL", "${port:9999}", "8080"} {
if !strings.Contains(err.Error(), said) {
t.Errorf("the refusal does not say %q: %v", said, err)
}
}
env = map[string]any{"SHOWCASE_DATA": "${dir:date}/objects"}
if _, err := (Resolution{Node: "anchor", Modules: []Manifest{processModule(env)}}).Declaration(Rendering{}); err == nil ||
!strings.Contains(err.Error(), "${dir:date}") {
t.Fatalf("a process naming no directory of its module was not refused: %v", err)
}
}
@@ -0,0 +1,75 @@
package catalogue
import (
"fmt"
"os"
"path/filepath"
"strconv"
"strings"
"testing"
)
// A container publishes only a port its module declares (novox/hq issue 227).
//
// **The short form is a question the mesh answers.** `"80"` means *publish what the software
// calls 80*, and the mesh fills in the machine's half from the port it assigned
// ([ADR 0038](0038)). It can only assign one for a port the module declared in `listens` — so a
// container publishing a number that appears nowhere in `listens` gets no assignment, and
// `publishedOn` falls back to the number as written. It escapes to the machine.
//
// That is how the photo module asked for port 80 on the control node, where the reverse proxy
// holds it: it declared its web endpoint at 4001, published a bare 80, and the container never
// started. Four other modules publish 80 quite safely — because they declare 80, so the mesh
// gives them a machine port for it. The difference is the declaration, not the number.
//
// A mapping written the long way is a module pinning both halves on purpose and is left alone.
func TestEveryPublishedPortIsOneItsModuleDeclares(t *testing.T) {
root := catalogueRoot(t)
entries, err := os.ReadDir(filepath.Join(root, "modules"))
if err != nil {
t.Fatal(err)
}
var escaped []string
for _, entry := range entries {
if !entry.IsDir() {
continue
}
raw, err := os.ReadFile(filepath.Join(root, "modules", entry.Name(), "module.json"))
if err != nil {
continue
}
m, err := ParseManifest(raw)
if err != nil {
// Whether every manifest parses is TestEveryCatalogueManifestParses's question.
continue
}
declared := map[int]bool{}
for _, l := range m.Listens {
declared[l.Port] = true
}
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) != "container" {
continue
}
listed, _ := r["ports"].([]any)
for _, p := range listed {
written := strings.Split(fmt.Sprint(p), "/")[0]
if strings.Contains(written, ":") {
continue // pinned by hand, both halves, on purpose
}
port, err := strconv.Atoi(strings.TrimSpace(written))
if err != nil || declared[port] {
continue
}
escaped = append(escaped, fmt.Sprintf(
"%s's %v publishes %d, and %s declares no such port — the mesh has nothing "+
"to assign, so %d reaches the machine as written",
m.Module, r["id"], port, m.Module, port))
}
}
}
if len(escaped) > 0 {
t.Fatalf("a container may publish only a port its module declares:\n - %s",
strings.Join(escaped, "\n - "))
}
}
+115
View File
@@ -147,6 +147,10 @@ type Resolution struct {
// dropped nor fatal to the rest. A module that is *required* by something running here is a
// different case — that set is incoherent and is refused (see checkCapabilities).
Unhostable []Unhostable
// Unheld is every dependency of this node's modules on a seat nothing here holds (novox/hq ADR
// 0207) — reported rather than refused while enforceSeatDependencies is off, so a node short of a
// holder still converges and `status` says what it is short of.
Unheld []Unheld
}
// Unhostable is one directly-assigned module the machine cannot run.
@@ -229,6 +233,27 @@ func Resolve(catalogue map[string]Manifest, assigned []string, node Node, world
local[o.Name] = true
}
}
// Which names are usable only on their provider's own machine (novox/hq ADR 0208 §3). Also a
// property of the name: a display one provider says is the machine's and another says is not
// would be pulled in for one consumer and refused for the next.
machineReach := map[string]bool{}
plainLocal := map[string]bool{}
for _, m := range catalogue {
for _, o := range m.Provides {
if o.MachineReach() {
machineReach[o.Name] = true
} else if o.At() == ScopeNode {
plainLocal[o.Name] = true
}
}
}
for want := range machineReach {
if plainLocal[want] {
problems = append(problems, fmt.Sprintf(
"the catalogue disagrees about %q: some modules provide it with the machine's reach and "+
"others without, so a requirement for it would be met differently by each", want))
}
}
for want := range brokered {
if local[want] {
problems = append(problems, fmt.Sprintf(
@@ -495,6 +520,23 @@ func Resolve(catalogue map[string]Manifest, assigned []string, node Node, world
continue
}
// Usable only on its provider's own machine, and not here: refused, never answered by
// installing a provider (novox/hq ADR 0208 §3). The display server is the machine's own role,
// gated by its graphical session; one pulled in for a window manager is the misassignment
// research 026 found. Another node's provider never counts — node scope is never brokered.
if machineReach[want] && !isModule(catalogue, want) {
reported[want] = true
if world.Unchecked {
// The first pass's refusals take a machine off the network; the second says it.
continue
}
problems = append(problems, fmt.Sprintf(
"%q is wanted by %s and is usable only on the machine that provides it, and nothing "+
"assigned to %s does — %s", want, because[want], node.Name,
machineReachRemedy(catalogue, want, node.Name)))
continue
}
candidates := offers[want]
switch len(candidates) {
case 0:
@@ -628,6 +670,22 @@ func Resolve(catalogue map[string]Manifest, assigned []string, node Node, world
problems = append(problems, checkResources(resolution.Modules)...)
resolution.Claims = claims
// Judged over the closure — what this node will actually run — so a holder pulled in by a
// requirement counts, and the holders' mutual dependence resolves (novox/hq ADR 0207 §3).
// Reported until the switch; refused after it, though never in the first pass, whose refusals
// make a machine vanish from the network rather than report anything.
resolution.Unheld = UnheldDependencies(catalogue, node.Name, resolution.Modules, nil)
// Only a dependency some module in the catalogue could meet is refused: one with no possible
// holder has no remedy to name, and stays a report in `status` (novox/hq ADR 0207 §4, read with
// the assign rule above it).
if enforceSeatDependencies && !world.Unchecked {
for _, u := range resolution.Unheld {
if len(u.Holders) > 0 {
problems = append(problems, u.String())
}
}
}
if len(problems) > 0 {
sort.Strings(problems)
return Resolution{}, &Refusal{Problems: problems}
@@ -804,6 +862,28 @@ func checkResources(modules []Manifest) []string {
// does not exist is the manifest's own problem, refused where it was made.
dirs := dirsFor(m, Rendering{})
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) == "user" {
// **An account is shared; what it is set to is not.** Several modules may need one
// login: the shell's module sets its shell, the container runtime's puts it in the
// `docker` group. The host only ever adds groups — it never takes the account out of
// one, not even when the resource that named it is undeclared — so groups from
// several modules cannot contradict each other and are not owned. A shell or a home
// is one value, and two modules setting it would each be undone by the other's
// apply: each stays one module's per node, and two are refused naming both.
name, _ := r["name"].(string)
for _, field := range []string{"shell", "home"} {
if v, ok := r[field].(string); !ok || v == "" || name == "" {
continue
}
key := "user " + field + " " + name
if other, taken := owner[key]; taken && other != m.Module {
problems = append(problems, fmt.Sprintf(
"%s and %s both set the %s of the user %q", other, m.Module, field, name))
}
owner[key] = m.Module
}
continue
}
for _, field := range []string{"path", "unit", "name", "package"} {
value, ok := r[field].(string)
if !ok || value == "" {
@@ -1019,3 +1099,38 @@ func eachLocal(needs []Needed, catalogue map[string]Manifest, n Needed) []Needed
}
return needs
}
// machineReachRemedy names what would meet a requirement with the machine's reach: every module in
// the catalogue that provides it, each with the node seats it holds — for a display, the holders of
// node-display-server (novox/hq ADR 0208 §3), named by the seat because that is the role being
// asked for, without this code knowing which seat any provision belongs to.
func machineReachRemedy(catalogue map[string]Manifest, want, node string) string {
var named []string
for _, name := range sortedKeys(catalogue) {
m := catalogue[name]
provides := false
for _, o := range m.Provides {
if o.Name == want {
provides = true
}
}
if !provides {
continue
}
var held []string
for _, c := range m.Claims {
if c.At() == ScopeNode {
held = append(held, c.Name)
}
}
if len(held) > 0 {
named = append(named, fmt.Sprintf("%s (holds %s)", name, strings.Join(held, ", ")))
} else {
named = append(named, name)
}
}
if len(named) == 0 {
return "and nothing in the catalogue provides it"
}
return fmt.Sprintf("assign one to %s: %s", node, strings.Join(named, "; "))
}
+44 -23
View File
@@ -48,9 +48,12 @@ func TestTheResolverForwardsToFixedUpstreamsAndNeverReadsResolvConf(t *testing.T
}
for _, want := range []string{
"\nno-resolv\n", "\nserver=1.1.1.1\n", "\nserver=8.8.8.8\n",
// Loopback is the mesh-wide setting's default; a machine answering its own LAN adds its
// address there (novox/hq issue 198).
"\nlisten-address=${setting:listen-addresses}\n", "\nlisten-address=${machine:address}\n", "\nbind-dynamic\n",
// The private address and loopback, never a LAN's (novox/hq ADR 0194): a device that is not a
// member cannot reach what the mesh's names point at.
"\nlisten-address=127.0.0.1\n", "\nlisten-address=${machine:address}\n", "\nbind-dynamic\n",
// No hosts file and no operator's files: the mesh's resolver answers every node (ADR 0199).
"\nno-hosts\n",
"\nconf-file=" + m.Facts["zones"].Path + "\n",
"\ndomain-needed\n", "\nbogus-priv\n",
"\nconf-file=" + m.Facts["node-zones"].Path + "\n",
} {
@@ -73,7 +76,15 @@ func TestTheResolverForwardsToFixedUpstreamsAndNeverReadsResolvConf(t *testing.T
t.Errorf("the resolver listens on %s", taken)
}
}
// And the file that decides what the machine asks names it there, alone.
// Never a directory or a file the operator keeps: a line written for one machine's programs would
// become an answer for every node (ADR 0199).
for _, never := range []string{"conf-dir=", "addn-hosts=", "listen-address=${setting:"} {
if strings.Contains(config, never) {
t.Errorf("the mesh's resolver still reads or listens on %q", never)
}
}
// And the file that decides what the machine asks names the mesh's resolver first, by address,
// and a public one second, asked only when the first is silent (ADR 0196).
var resolv string
for _, r := range catalogueManifest(t, "resolv-conf").Resources {
if r["path"] == "/etc/resolv.conf" {
@@ -86,13 +97,16 @@ func TestTheResolverForwardsToFixedUpstreamsAndNeverReadsResolvConf(t *testing.T
nameservers = append(nameservers, strings.TrimPrefix(line, "nameserver "))
}
}
if len(nameservers) != 1 || nameservers[0] != "127.0.0.1" {
t.Errorf("resolv.conf names %v; the predecessor's names the mesh's resolver alone at 127.0.0.1", nameservers)
if len(nameservers) != 2 || nameservers[0] != "${bound:wildcard-resolution:address}" || nameservers[1] != "1.1.1.1" {
t.Errorf("resolv.conf names %v; the mesh's resolver by address first, a public one second", nameservers)
}
// The split-DNS alternative points at the same address, or a machine that keeps
if !strings.Contains(resolv, "\noptions timeout:1 attempts:1") {
t.Errorf("the fallback is not reached after one short attempt:\n%s", resolv)
}
// The split-DNS alternative points at the same resolver, or a machine that keeps
// systemd-resolved in charge would route the mesh's suffix to nothing.
for _, r := range catalogueManifest(t, "resolved-split-dns").Resources {
if content, _ := r["content"].(string); content != "" && !strings.Contains(content, "DNS=127.0.0.1\n") {
if content, _ := r["content"].(string); content != "" && !strings.Contains(content, "DNS=${bound:wildcard-resolution:address}\n") {
t.Errorf("resolved-split-dns does not point at the resolver's address:\n%s", content)
}
}
@@ -100,7 +114,8 @@ func TestTheResolverForwardsToFixedUpstreamsAndNeverReadsResolvConf(t *testing.T
// The resolver and what points the machine at it compose on one machine, and what arrives is the
// mesh's account of every machine as a wildcard, the suffix kept local, the daemon restarting on
// that file, and the runtime pointed at this machine's own address.
// that file, the machine pointed at the resolver by address, and the runtime given no resolver of
// its own but kept running across a restart (ADR 0196).
func TestTheResolverAndWhatAsksItComposeOnOneMachine(t *testing.T) {
got, err := Resolve(resolverShelf(t), []string{"dnsmasq", "resolv-conf"},
Node{Name: "anchor", At: "anchor.internal"}, World{})
@@ -115,6 +130,7 @@ func TestTheResolverAndWhatAsksItComposeOnOneMachine(t *testing.T) {
// issue 111) — the resolver's zones read only the second, and in this scenario the two
// happen to be the same map, since nothing routed is part of it.
Names: twoMachines, Machines: twoMachines, Suffix: "internal",
Zones: []ZoneAt{{Zone: "incus", Address: "10.42.0.2", Port: 5353}},
Needed: map[string]map[string]string{"dnsmasq": {"broker": "sealed"}},
Settings: SettingsBy{"dnsmasq": {{From: "the mesh", Values: map[string]any{"listen-addresses": "127.0.0.1"}}}},
})
@@ -144,24 +160,29 @@ func TestTheResolverAndWhatAsksItComposeOnOneMachine(t *testing.T) {
for _, id := range service["restart-on"].([]any) {
reflects[id.(string)] = true
}
if !reflects["dnsmasq.config"] || !reflects["dnsmasq.fact-node-zones"] {
t.Errorf("the daemon does not restart on its configuration and the machines file both: %v", service["restart-on"])
if !reflects["dnsmasq.config"] || !reflects["dnsmasq.fact-node-zones"] || !reflects["dnsmasq.fact-zones"] {
t.Errorf("the daemon does not restart on its configuration, the machines and the zones: %v", service["restart-on"])
}
if z, _ := ids["dnsmasq.fact-zones"]["content"].(string); !strings.Contains(z, "server=/incus/10.42.0.2#5353\n") {
t.Errorf("the resolver was not told to forward the zone to its answerer:\n%s", z)
}
// The runtime's own file, written into (novox/hq ADR 0102) with the keys this module states:
// where containers resolve, and that a restart keeps them running — because the runtime reads
// `dns` only when it starts, and the one restart that needs is the operator's (issue 110).
runtime := ids["dnsmasq.runtime-dns"]
// The runtime's own file, written into (novox/hq ADR 0102) with one key, by what decides how the
// machine resolves: a restart keeps every container running. No `dns` — a container copies its
// machine's resolvers (ADR 0196), and the mesh's resolver is not written into the runtime twice.
if ids["dnsmasq.runtime-dns"] != nil {
t.Errorf("the resolver still writes the runtime's dns: %v", ids["dnsmasq.runtime-dns"])
}
runtime := ids["resolv-conf.runtime-config"]
if runtime == nil || runtime["path"] != "/etc/docker/daemon.json" || runtime["into"] != "json" {
t.Fatalf("the runtime's dns is not written into its file: %v", runtime)
t.Fatalf("live-restore is not written into the runtime's file: %v", runtime)
}
var keys map[string]any
if err := json.Unmarshal([]byte(runtime["content"].(string)), &keys); err != nil {
t.Fatalf("the runtime's keys are not JSON: %v", err)
}
dns, _ := keys["dns"].([]any)
if len(keys) != 2 || len(dns) != 1 || dns[0] != "10.42.0.1" || keys["live-restore"] != true {
t.Errorf("the runtime is given %v; containers resolve at this machine's own private-network address, a restart keeps them, and nothing else is written", keys)
if len(keys) != 1 || keys["live-restore"] != true {
t.Errorf("the runtime is given %v; live-restore and nothing else", keys)
}
// The runtime is reloaded when that file changes, and never restarted: a restart stops every
// container on the machine (ADR 0102), and a reload is what turns live-restore on.
@@ -171,10 +192,10 @@ func TestTheResolverAndWhatAsksItComposeOnOneMachine(t *testing.T) {
continue
}
if _, restarts := r["restart-on"]; restarts {
t.Errorf("the resolver orders the runtime restarted, which stops every container (ADR 0102): %v", r)
t.Errorf("the runtime is ordered restarted, which stops every container (ADR 0102): %v", r)
}
for _, on := range asStrings(r["reload-on"]) {
if on == "dnsmasq.runtime-dns" {
if on == "resolv-conf.runtime-config" {
reloaded = true
}
}
@@ -184,8 +205,8 @@ func TestTheResolverAndWhatAsksItComposeOnOneMachine(t *testing.T) {
}
resolv := ids["resolv-conf.resolv"]
if resolv == nil || !strings.Contains(resolv["content"].(string), "\nnameserver 127.0.0.1\n") {
t.Fatalf("the machine is not pointed at the resolver: %v", resolv)
if resolv == nil || !strings.Contains(resolv["content"].(string), "\nnameserver 10.42.0.1\nnameserver 1.1.1.1\n") {
t.Fatalf("the machine is not pointed at the resolver by address, with the public fallback: %v", resolv)
}
}
+27
View File
@@ -61,6 +61,16 @@ type rosterView struct {
Suffix string
Names []rosterEntry
Machines []rosterEntry
// Zones is every zone a module in the mesh answers itself, with where its answerer is (novox/hq
// ADR 0199) — what the mesh's resolver forwards. Ordered by zone.
Zones []rosterZone
}
// rosterZone is one zone as a template sees it: the zone, and the address and port answering it.
type rosterZone struct {
Zone string
Address string
Port int
}
// rosterEntry is one machine as a template sees it: its bare name, its full mesh name, its address,
@@ -80,6 +90,12 @@ type rosterEntry struct {
// `machines` is only the machines — the two must not be confused (novox/hq 04-ISSUES/111), so both
// are given and the template chooses.
func FactsInto(m Manifest, r Resolution, every, machines, accounts map[string]string, suffix string) ([]map[string]any, error) {
return FactsWithZonesInto(m, r, every, machines, accounts, suffix, nil)
}
// FactsWithZonesInto is FactsInto with the mesh's zones in the view, for a template that ranges them.
func FactsWithZonesInto(m Manifest, r Resolution, every, machines, accounts map[string]string, suffix string,
zones []ZoneAt) ([]map[string]any, error) {
if len(m.Facts) == 0 {
return nil, nil
}
@@ -94,6 +110,7 @@ func FactsInto(m Manifest, r Resolution, every, machines, accounts map[string]st
Suffix: strings.TrimPrefix(suffixOr(suffix), "."),
Names: entriesFrom(every, accounts, suffix),
Machines: entriesFrom(machines, accounts, suffix),
Zones: zonesFrom(zones),
}
out := make([]map[string]any, 0, len(names))
@@ -223,3 +240,13 @@ func sortedNames(addresses map[string]string) []string {
sort.Strings(out)
return out
}
// zonesFrom is the zones a template ranges, ordered by zone so two renderings of one mesh are one file.
func zonesFrom(zones []ZoneAt) []rosterZone {
out := make([]rosterZone, 0, len(zones))
for _, z := range zones {
out = append(out, rosterZone{Zone: z.Zone, Address: z.Address, Port: z.Port})
}
sort.Slice(out, func(i, j int) bool { return out[i].Zone < out[j].Zone })
return out
}
+68
View File
@@ -389,3 +389,71 @@ func TestARuntimeCompiledToABinaryRunsItself(t *testing.T) {
t.Errorf("the Go runtime is not told what to serve or whose it is: %v %v", env, process["user"])
}
}
// novox/hq 04-ISSUES/216: a bundle nothing loads, runs or unpacks is refused at registration; saying
// `loads`, listing `tools`, or a resource naming it admits it.
func TestABundleNothingDeliversIsRefused(t *testing.T) {
base := func() Manifest {
return Manifest{Module: "baserow", Version: "1", Build: &Build{Artifacts: []Artifact{
{Name: "tools", Kind: ArtifactBundle, Language: "typescript", Entrypoints: []string{"tools/index.js"}}}}}
}
if p := undeliveredBundles(base()); len(p) != 1 || !strings.Contains(p[0], "never reach a machine") {
t.Fatalf("a bundle nothing delivers was admitted: %v", p)
}
loads := base()
loads.Build.Artifacts[0].Loads = []string{"tools/index.js"}
tools := base()
tools.Tools = []string{"baserow_list_rows"}
run := base()
run.Resources = []map[string]any{{"id": "daemon", "type": "process", "artifact": "tools", "run": []any{"node", "tools/index.js"}}}
runtime := base()
runtime.Module = RuntimeModule
for name, m := range map[string]Manifest{"loads": loads, "tools": tools, "a process": run, "the runtime": runtime} {
if p := undeliveredBundles(m); len(p) != 0 {
t.Errorf("a bundle delivered by %s was refused: %v", name, p)
}
}
}
// novox/hq ADR 0193: a Go tools bundle is served — its binary is what the runtime starts, delivered
// like any tools bundle, named to the runtime where a TypeScript bundle names its launcher.
func TestAGoToolsBundleIsServedByItsBinary(t *testing.T) {
with := Rendering{ArtifactStore: "anchor.internal:5101",
Needed: map[string]map[string]string{RuntimeModule: {"broker": "sealed-credential"}}}
lamp := Manifest{Module: "lamp", Version: "1", Tools: []string{"on"},
Build: &Build{Artifacts: []Artifact{{Name: "tools", Kind: ArtifactBundle, Language: "go",
System: "arch", From: "cmd/lamp-tools"}}}}
if p := lamp.Build.problems("lamp"); len(p) != 0 {
t.Fatalf("a Go tools bundle was refused: %v", p)
}
lamp, err := lamp.Resolve([]Built{{Name: "tools", Kind: ArtifactBundle,
Reference: ArtifactStoreScheme + "lamp/tools/blobs/" + bundleDigest, Digest: bundleDigest}})
if err != nil {
t.Fatal(err)
}
if fmt.Sprint(lamp.Bundles[0].Loads) != "[lamp-tools]" {
t.Fatalf("the runtime loads %v from a Go bundle, want its binary", lamp.Bundles[0].Loads)
}
out, err := Resolution{Node: "anchor", Account: "ops", Modules: []Manifest{lamp, theRuntime(t)}}.Declaration(with)
if err != nil {
t.Fatal(err)
}
if fileNamed(out, "lamp."+BundleID("tools")) == nil {
t.Errorf("the Go bundle is not delivered: %v", ids(out))
}
env := fileNamed(out, RuntimeModule+"."+RuntimeProcessID())["env"].(map[string]string)
if env[RuntimeToolModules] != "lamp="+BundlePath("lamp", "tools")+"/lamp-tools" {
t.Errorf("the runtime is told %q, want the binary", env[RuntimeToolModules])
}
// An artifact may say it explicitly; naming anything but the binary is refused.
said := Manifest{Module: "lamp", Version: "1", Build: &Build{Artifacts: []Artifact{{Name: "tools",
Kind: ArtifactBundle, Language: "go", System: "arch", Binary: "lamp", Loads: []string{"lamp"}}}}}
if p := said.Build.problems("lamp"); len(p) != 0 {
t.Errorf("loads naming the binary was refused: %v", p)
}
said.Build.Artifacts[0].Loads = []string{"tools/index.js"}
if p := said.Build.problems("lamp"); len(p) == 0 {
t.Error("a Go bundle loading a file it does not contain was admitted")
}
}
+226
View File
@@ -0,0 +1,226 @@
package catalogue
import (
"fmt"
"regexp"
"strings"
)
// A module contributes to a seat it does not hold (novox/hq ADR 0212).
//
// ADR 0210 made a tool's configuration its seat holder's, and every other module's way in a
// contribution to the seat. The environment, the shell's slots and the power moments each became a
// field of their own; this is the general form, so a new seat that takes contributions is a row in
// the seat table rather than a change to the manifest: a contribution names a seat, a kind that
// seat receives, and text in the tool's own grammar, which the controller never reads.
// HotkeysSeat is the machine's hotkey daemon (novox/hq ADR 0212 §5).
const HotkeysSeat = "node-hotkeys"
// MessageBusSeat is the machine's D-Bus (novox/hq ADR 0215).
const MessageBusSeat = "node-message-bus"
// BackupSeat is the machine's backups (novox/hq ADR 0214, to-be 43).
const BackupSeat = "node-backup"
// storeProvisions are the provisions whose provider keeps its consumers' data (novox/hq ADR 0214):
// a module providing one must say how to back it up, or the data the mesh hands out is the data it
// cannot restore — issue 241's seven databases. A provision that holds nothing worth keeping (a
// route, a cache, a name) is not here; adding one is adding a store.
var storeProvisions = map[string]bool{
"postgres-database": true,
"mssql-database": true,
"mongodb-database": true,
"s3-bucket": true,
"influxdb-api": true,
"secret": true,
}
// CheckBackup is a store provider that contributes no backup (novox/hq ADR 0214, "How it is
// checked").
//
// **The catalogue check's, not parsing's.** A manifest already registered and running was written
// before the rule; refusing it on read would make the controller refuse the very providers whose
// data the rule protects. New definitions meet it in the catalogue check, where they are written.
func CheckBackup(m Manifest) []string {
backs := false
for _, c := range m.Contributions {
if s, known := SeatNamed(c.Seat); known && s.Name == BackupSeat && c.Kind == "backup" {
backs = true
}
}
if backs {
return nil
}
var problems []string
for _, o := range m.Provides {
if storeProvisions[o.Name] {
problems = append(problems, fmt.Sprintf(
"%s provides %s, which keeps its consumers' data, and contributes no backup to %s; a "+
"store says how its data is copied (novox/hq ADR 0214)", m.Module, o.Name, BackupSeat))
}
}
return problems
}
// SeatContribution is one piece of configuration a module gives a seat's holder to place.
type SeatContribution struct {
// Seat is the seat whose holder places it.
Seat string `json:"seat"`
// Kind is which of the seat's receivable kinds it is.
Kind string `json:"kind"`
// Content is the text, in the tool's own grammar. Never interpreted.
Content string `json:"content"`
}
// ofContribution is where a holder places a kind: ${contribution:<seat>:<kind>}. Loose inside the
// braces, so a misspelt seat or kind is found and refused rather than written out as text.
var ofContribution = regexp.MustCompile(`\$\{contribution:([^}]*)\}`)
// receivable is what a seat receives of a kind, with the seat's canonical name; false when the seat
// is unknown or does not receive it.
func receivable(seat, kind string) (Seat, Receivable, bool) {
s, known := SeatNamed(seat)
if !known {
return Seat{}, Receivable{}, false
}
for _, r := range s.Receives {
if r.Kind == kind {
return s, r, true
}
}
return s, Receivable{}, false
}
// kindsOf names a seat's receivable kinds for a refusal.
func kindsOf(s Seat) string {
if len(s.Receives) == 0 {
return "it receives no contributions"
}
var kinds []string
for _, r := range s.Receives {
kinds = append(kinds, r.Kind)
}
return "it receives " + strings.Join(kinds, ", ")
}
// seatContributionProblems is what is wrong with this module's contributions, from the manifest
// alone (novox/hq ADR 0212 §2).
func (m Manifest) seatContributionProblems() []string {
var problems []string
for i, c := range m.Contributions {
s, r, ok := receivable(c.Seat, c.Kind)
// A directory a contribution names must be one of this module's own, here rather than on the
// machine — where a `${dir:x}` nobody declared would reach the holder as the literal text.
if ok && r.Dirs {
declared := map[string]string{}
for _, res := range m.Resources {
if fmt.Sprint(res["type"]) == "directory" {
declared[fmt.Sprint(res["id"])] = ""
}
}
if _, err := dirFill(c.Content, declared, m.Module); err != nil {
problems = append(problems, fmt.Sprintf("%s's contribution %d: %v", m.Module, i+1, err))
}
}
switch {
case s.Name == "":
problems = append(problems, fmt.Sprintf(
"%s's contribution %d is to the seat %q, which the mesh does not define", m.Module, i+1, c.Seat))
case !ok:
problems = append(problems, fmt.Sprintf(
"%s's contribution %d to %s is of the kind %q; %s (novox/hq ADR 0212)",
m.Module, i+1, s.Name, c.Kind, kindsOf(s)))
}
if strings.TrimSpace(c.Content) == "" {
problems = append(problems, fmt.Sprintf("%s's contribution %d has no content", m.Module, i+1))
}
}
return problems
}
// seatPlaceholderProblems is what is wrong with one resource's ${contribution:…}: placed only in a
// file's content, naming a seat and a kind it receives, and only by a module that claims that seat
// — another would be a second writer of a file there is one of (novox/hq ADR 0212 §3).
func seatPlaceholderProblems(m Manifest, r map[string]any) []string {
var problems []string
for _, field := range sortedKeys(r) {
v, ok := r[field].(string)
if !ok {
continue
}
found := ofContribution.FindAllStringSubmatch(v, -1)
if len(found) == 0 {
continue
}
if field != "content" {
problems = append(problems, fmt.Sprintf(
"%s's resource %v names %s in its %s; contributions are placed only in a file's content",
m.Module, r["id"], found[0][0], field))
continue
}
for _, f := range found {
seat, kind, two := strings.Cut(f[1], ":")
s, _, ok := receivable(seat, kind)
if !two || !ok {
detail := "the mesh defines no seat " + fmt.Sprintf("%q", seat)
if s.Name != "" {
detail = s.Name + ": " + kindsOf(s)
}
problems = append(problems, fmt.Sprintf(
"%s's resource %v names %s; a contribution is ${contribution:<seat>:<kind>} (%s)",
m.Module, r["id"], f[0], detail))
continue
}
if !m.ClaimsSeat(s.Name) {
problems = append(problems, fmt.Sprintf(
"%s's resource %v names %s and %s does not claim %s; every module's contributions to a "+
"seat are placed by its holder alone (novox/hq ADR 0212)",
m.Module, r["id"], f[0], m.Module, s.Name))
}
}
}
return problems
}
// seatContributions is every module's contribution of one kind to one seat (novox/hq ADR 0212 §3):
// in module order, each module's pieces in the order it declared them, each module's preceded by a
// comment line naming it in the tool's grammar, and empty when nothing is contributed. A kind that
// takes directories has each contributor's `${dir:<id>}` filled with where that module's directory
// is on this machine (novox/hq to-be 43).
func seatContributions(modules []Manifest, seat, kind string, with Rendering) (string, error) {
s, r, ok := receivable(seat, kind)
if !ok {
return "", nil
}
var failed error
var b strings.Builder
for _, m := range inModuleOrder(modules) {
named := false
for _, c := range m.Contributions {
if c.Kind != kind {
continue
}
if cs, known := SeatNamed(c.Seat); !known || cs.Name != s.Name {
continue
}
if !named {
fmt.Fprintf(&b, "%s %s\n", r.Comment, m.Module)
named = true
}
content := c.Content
if r.Dirs {
filled, err := dirFill(content, dirsFor(m, with), m.Module)
if err != nil && failed == nil {
failed = err
}
content = filled
}
b.WriteString(content)
if !strings.HasSuffix(content, "\n") {
b.WriteString("\n")
}
}
}
return b.String(), failed
}
@@ -0,0 +1,117 @@
package catalogue
import (
"reflect"
"strings"
"testing"
)
// Defends novox/hq ADR 0212: a seat says what it receives, its holder places it, and a contribution
// depends on the seat.
func hotkeysHolder() Manifest {
return Manifest{Module: "triggerhappy", Claims: []Claim{{Name: HotkeysSeat}}, Resources: []map[string]any{
{"id": "triggers", "type": "file", "path": "/etc/triggerhappy/triggers.d/mesh.conf",
"content": "# the mesh's triggers\n${contribution:node-hotkeys:trigger}"},
}}
}
func TestAContributionReachesTheHoldersFileInModuleOrderNamedByModule(t *testing.T) {
laptop := Manifest{Module: "laptop", Contributions: []SeatContribution{
{Seat: HotkeysSeat, Kind: "trigger", Content: "KEY_PROG1 1 play ${machine:account-home}"},
{Seat: HotkeysSeat, Kind: "trigger", Content: "KEY_F21 1 touchpad\n"},
}}
another := Manifest{Module: "another", Contributions: []SeatContribution{
{Seat: HotkeysSeat, Kind: "trigger", Content: "KEY_MUTE 1 mute ${shell:zsh:first}"},
}}
unrelated := Manifest{Module: "bar", Contributions: []SeatContribution{
{Seat: DisplaySessionSeat, Kind: "config", Content: "bar { }"},
}}
r := Resolution{Node: "laptop", Account: "op", Modules: []Manifest{hotkeysHolder(), laptop, another, unrelated}}
out, err := r.Declaration(Rendering{})
if err != nil {
t.Fatal(err)
}
var file map[string]any
for _, res := range out {
if res["id"] == "triggerhappy.triggers" {
file = res
}
}
if file == nil {
t.Fatalf("the holder's file was not composed: %v", out)
}
// Module order; each module named once; text never read, so neither ${machine:…} nor ${shell:…}
// inside a contribution is filled.
want := "# the mesh's triggers\n" +
"# another\nKEY_MUTE 1 mute ${shell:zsh:first}\n" +
"# laptop\nKEY_PROG1 1 play ${machine:account-home}\nKEY_F21 1 touchpad\n"
if got := file["content"]; got != want {
t.Fatalf("the holder's file is\n%s\nnot\n%s", got, want)
}
}
func TestNoContributionPlacesNothing(t *testing.T) {
r := Resolution{Node: "laptop", Account: "op", Modules: []Manifest{hotkeysHolder()}}
out, err := r.Declaration(Rendering{})
if err != nil {
t.Fatal(err)
}
for _, res := range out {
if res["id"] == "triggerhappy.triggers" && res["content"] != "# the mesh's triggers\n" {
t.Fatalf("an empty kind left %q", res["content"])
}
}
}
func TestAContributionToASeatThatDoesNotReceiveItIsRefused(t *testing.T) {
cases := map[string]string{
`{"seat":"node-hotkeys","kind":"config","content":"x"}`: "it receives trigger",
`{"seat":"node-nothing","kind":"trigger","content":"x"}`: "the mesh does not define",
`{"seat":"node-hotkeys","kind":"trigger","content":" "}`: "has no content",
`{"seat":"node-display-session","kind":"trigger","content":"x"}`: "it receives config",
}
for c, want := range cases {
raw := `{"module":"laptop","contributions":[` + c + `]}`
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("%s: accepted, or refused without %q: %v", c, want, err)
}
}
}
func TestAContributionPlaceholderOutsideTheHolderIsRefused(t *testing.T) {
raw := `{"module":"laptop","resources":[{"id":"t","type":"file","path":"/etc/t","content":"${contribution:node-hotkeys:trigger}"}]}`
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), "laptop does not claim node-hotkeys") {
t.Errorf("a placeholder outside the holder was accepted: %v", err)
}
raw = `{"module":"triggerhappy","claims":[{"name":"node-hotkeys"}],"resources":[{"id":"t","type":"file","path":"/etc/t","content":"${contribution:node-hotkeys:keys}"}]}`
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), "it receives trigger") {
t.Errorf("a placeholder for a kind the seat does not receive was accepted: %v", err)
}
}
func TestAContributionDependsOnItsSeat(t *testing.T) {
m := Manifest{Module: "laptop", Contributions: []SeatContribution{
{Seat: HotkeysSeat, Kind: "trigger", Content: "x"},
{Seat: DisplaySessionSeat, Kind: "config", Content: "y"},
}}
if got, want := DependsOn(m), []string{DisplaySessionSeat, HotkeysSeat}; !reflect.DeepEqual(got, want) {
t.Errorf("depends on %v, want %v", got, want)
}
catalogue := map[string]Manifest{"laptop": m, "triggerhappy": hotkeysHolder()}
if _, err := AssignRefusal(catalogue, "laptop", nil, []string{"laptop"}); err == nil ||
!strings.Contains(err.Error(), HotkeysSeat) {
t.Errorf("a contributor without the holder was accepted: %v", err)
}
}
func TestTheHotkeysSeatIsTheMeshsAndReceivesTriggers(t *testing.T) {
s, ok := SeatNamed(HotkeysSeat)
if !ok || s.Scope != ScopeNode || len(s.Receives) != 1 || s.Receives[0].Kind != "trigger" {
t.Fatalf("%+v %v", s, ok)
}
d, _ := SeatNamed(DisplaySessionSeat)
if len(d.Receives) != 1 || d.Receives[0].Kind != "config" {
t.Fatalf("the display session receives %+v", d.Receives)
}
}
+349
View File
@@ -0,0 +1,349 @@
package catalogue
import (
"fmt"
"sort"
"strings"
)
// A module depends on the node seats that apply its resources (novox/hq ADR 0207), and on the
// seats it contributes to (novox/hq ADR 0210).
//
// Some of what a module declares is applied through software on the machine that is itself a
// module: a service through the service manager, a package through the package manager, a container
// through the container runtime. A *capability* only says that software is installed; it does not
// say that a module of the mesh holds the role and answers for it. So the dependency is derived from
// the resources — never stated in a manifest, because a module that adds a service and forgets a
// field would pass — and is met when some module assigned to the same node holds the seat.
// The three seats that apply resources (novox/hq ADR 0207 §1). Named once, because the derivation,
// the seed and the messages all turn on these strings.
const (
ServiceManagerSeat = "node-service-manager"
PackageManagerSeat = "node-package-manager"
ContainerRuntimeSeat = "node-container-runtime"
)
// appliedThrough is which seat applies a resource of each type. **Only these three**: ADR 0207
// names them and no more. A process is supervised by the host itself, a file, a directory, an
// archive, a user or an action is the host's own act, and a module's other kinds reach the machine
// without a role in between — adding one here is a decision, not a refinement.
var appliedThrough = map[string]string{
"service": ServiceManagerSeat,
"package": PackageManagerSeat,
"container": ContainerRuntimeSeat,
}
// enforceSeatDependencies is the one-line switch ADR 0207 §4 names. Off, an unmet dependency at
// composition is *reported* — in the resolution, in `status`, once in the log — and the node still
// resolves; on, it is refused like any unresolved requirement. Off until `status` reports none,
// which is when the three holders are assigned to every node: switching it before then would stop
// every machine lacking one from being sent anything at all.
//
// Switched on 2026-10-04, when `status` first reported no unmet dependency on any node: systemd,
// pacman and docker were assigned to all four machines that afternoon (novox/hq to-be 42).
//
// A variable rather than a constant only so a test can hold both behaviours; nothing else sets it.
var enforceSeatDependencies = true
// EnforcingSeatDependencies sets the switch and returns what puts it back. For tests in other
// packages that hold the behaviour from before the switch; nothing else calls it.
func EnforcingSeatDependencies(on bool) (restore func()) {
was := enforceSeatDependencies
enforceSeatDependencies = on
return func() { enforceSeatDependencies = was }
}
// foundationModules are the pieces genesis lays before any module exists (novox/hq ADR 0207 §5):
// the host and the private network. Registered as modules so they can be assigned, but what they
// declare is the installation's, not a module's, so it is never judged. The third piece, the
// bootstrap container runtime, is not a module at all: its package and service are in the genesis
// bundle the host applies itself, and never pass through a resolution here.
//
// The private network's module is overlay.Name, written out because the overlay package composes
// on top of this one; its resources are computed, which exempts it by the rule below as well.
var foundationModules = map[string]bool{
"mesh-host": true,
"mesh-wireguard": true,
}
// isFoundation is whether a module's declarations are the foundation's rather than its own. A
// module whose resources are computed is the mesh's by construction — the private network's peer
// list and its tools are the controller's, written per node — so it counts whatever its name.
func isFoundation(m Manifest) bool {
return foundationModules[m.Module] || m.Computed != ""
}
// DependsOn is every seat a module needs held on its node, derived from the resource types it
// declares itself (novox/hq ADR 0207 §2), sorted.
//
// **The module's own `resources` only.** What the controller composes around a module — its
// filter, jails, certificates, kept files, bundles, the guard — is the mesh's, put there because the
// module is assigned, and depending on it would make the module answer for the mesh's choices.
func DependsOn(m Manifest) []string {
if isFoundation(m) {
return nil
}
seen := map[string]bool{}
for _, r := range m.Resources {
if seat, applied := appliedThrough[fmt.Sprint(r["type"])]; applied {
seen[seat] = true
}
}
for _, seat := range contributedTo(m) {
seen[seat] = true
}
out := make([]string, 0, len(seen))
for s := range seen {
out = append(out, s)
}
sort.Strings(out)
return out
}
// contributedTo is every seat a module contributes to (novox/hq ADR 0210 §3): a contribution is
// configuration only the seat's holder applies, so it is a dependency on that seat exactly as a
// resource is on the seat that applies it. The environment goes to node-environment's holder
// (ADR 0203); shell code to the holder that places it for its target — the login shell's for a
// shell, the display server's for the session's start and resources (ADR 0204, ADR 0208 §4).
func contributedTo(m Manifest) []string {
var out []string
if e := m.Environment; e != nil && (len(e.Variables) > 0 || len(e.Path) > 0) {
out = append(out, EnvironmentSeat)
}
for _, c := range m.Shell {
out = append(out, placerOf(c.For))
}
// Any other seat's contribution (novox/hq ADR 0212 §4), by the seat's canonical name; one the
// mesh does not define is refused at registration and depends on nothing here.
for _, c := range m.Contributions {
if s, known := SeatNamed(c.Seat); known {
out = append(out, s.Name)
}
}
return out
}
// claimsSeat is whether a module claims a node seat, by its current name or one it used to have
// (ADR 0122), so a rename leaves the dependency met.
func claimsSeat(m Manifest, seat string) bool {
for _, c := range m.Claims {
if c.At() != ScopeNode {
continue
}
name := c.Name
if s, known := SeatNamed(name); known {
name = s.Name
}
if name == seat {
return true
}
}
return false
}
// PossibleHolders is every module in the catalogue that claims a seat at node scope — what a
// refusal names as the remedy.
func PossibleHolders(catalogue map[string]Manifest, seat string) []string {
var out []string
for name, m := range catalogue {
if claimsSeat(m, seat) {
out = append(out, name)
}
}
sort.Strings(out)
return out
}
// Unheld is one dependency of one module on a node that nothing on that node holds.
type Unheld struct {
Node string `json:"node"`
Module string `json:"module"`
Seat string `json:"seat"`
// Holders are the modules in the catalogue that could hold the seat: assigning one meets it.
Holders []string `json:"holders"`
}
// String is the line a refusal and a report both say, so the two never drift.
func (u Unheld) String() string {
remedy := "and no module in the catalogue claims it yet"
if len(u.Holders) > 0 {
remedy = "— assign one that holds it: " + strings.Join(u.Holders, ", ")
}
return fmt.Sprintf("%s on %s depends on %s, which nothing on %s holds (novox/hq ADR 0207) %s",
u.Module, u.Node, u.Seat, u.Node, remedy)
}
// UnheldDependencies is every dependency of the modules in `judged` that the node's whole set
// leaves unmet (novox/hq ADR 0207 §3).
//
// **Judged over the whole set, never one module at a time.** The holders depend on each other:
// the service manager's own package needs the package manager, and the package manager's timer
// needs the service manager. Asked one by one, neither could ever be first; asked of the set, the
// two assigned together meet each other. A module holding a seat it depends on meets its own
// dependency. `judged` nil judges every module of the set.
func UnheldDependencies(catalogue map[string]Manifest, node string, set []Manifest, judged map[string]bool) []Unheld {
held := map[string]bool{}
for _, m := range set {
for _, seat := range nodeSeatsClaimed(m) {
held[seat] = true
}
}
var out []Unheld
for _, m := range set {
if judged != nil && !judged[m.Module] {
continue
}
for _, seat := range DependsOn(m) {
if held[seat] {
continue
}
out = append(out, Unheld{Node: node, Module: m.Module, Seat: seat,
Holders: PossibleHolders(catalogue, seat)})
}
}
sort.Slice(out, func(i, j int) bool {
if out[i].Module != out[j].Module {
return out[i].Module < out[j].Module
}
return out[i].Seat < out[j].Seat
})
return out
}
// nodeSeatsClaimed is every node seat a module claims, each by its current name (ADR 0122), so
// a dependency on any of them — a resource's or a contribution's — is met by the claim.
func nodeSeatsClaimed(m Manifest) []string {
var out []string
for _, c := range m.Claims {
if c.At() != ScopeNode {
continue
}
name := c.Name
if s, known := SeatNamed(name); known {
name = s.Name
}
out = append(out, name)
}
return out
}
// manifestsOf is the catalogue's definitions of the named modules; a name the catalogue does not
// know contributes nothing, as it does to a resolution.
func manifestsOf(catalogue map[string]Manifest, names []string) []Manifest {
var out []Manifest
seen := map[string]bool{}
for _, n := range names {
if m, known := catalogue[n]; known && !seen[n] {
seen[n] = true
out = append(out, m)
}
}
return out
}
// AssignRefusal is why assigning `adding` to a node already running `assigned` is refused, or
// nothing (novox/hq ADR 0207 §4): each new module's dependency the node's assignments, the new ones
// included, leave unmet.
//
// **Only the new modules are judged.** A node already short of a holder is reported by `status`;
// refusing an unrelated assignment for it would make the remedy — assigning the holder — refused too.
//
// **A dependency nothing in the catalogue can meet is said, not refused.** A refusal names the
// module that would meet it; with none registered there is no remedy to name, and refusing would
// stop every assignment of that kind until a module that does not exist yet is written. The answer
// still says it, and `status` reports it — before the switch and after it alike: there is never a
// remedy to name for it. The first return is those lines.
func AssignRefusal(catalogue map[string]Manifest, node string, assigned, adding []string) ([]string, error) {
set := manifestsOf(catalogue, append(append([]string(nil), assigned...), adding...))
judged := map[string]bool{}
for _, a := range adding {
judged[a] = true
}
var refused, said []string
for _, u := range UnheldDependencies(catalogue, node, set, judged) {
if len(u.Holders) == 0 {
said = append(said, u.String())
continue
}
refused = append(refused, u.String())
}
if len(refused) > 0 {
return said, &Refusal{Problems: append(refused,
fmt.Sprintf("holders that depend on each other are assigned together: `assign %s <module> <module>…`", node))}
}
return said, nil
}
// UnassignRefusal is why taking `removing` off a node running `assigned` is refused, or nothing
// (novox/hq ADR 0207): a seat the removed modules hold that nothing left on the node holds, while
// a module left there depends on it. Names the dependents, because they are what must go first —
// or the holder's replacement come.
func UnassignRefusal(catalogue map[string]Manifest, node string, assigned, removing []string) error {
gone := map[string]bool{}
for _, r := range removing {
gone[r] = true
}
var left []string
for _, a := range assigned {
if !gone[a] {
left = append(left, a)
}
}
before := map[string]bool{}
for _, u := range UnheldDependencies(catalogue, node, manifestsOf(catalogue, assigned), nil) {
before[u.Module+"\x00"+u.Seat] = true
}
dependents := map[string][]string{}
for _, u := range UnheldDependencies(catalogue, node, manifestsOf(catalogue, left), nil) {
if before[u.Module+"\x00"+u.Seat] {
continue // unmet already; not this removal's doing
}
dependents[u.Seat] = append(dependents[u.Seat], u.Module)
}
if len(dependents) == 0 {
return nil
}
seats := make([]string, 0, len(dependents))
for s := range dependents {
seats = append(seats, s)
}
sort.Strings(seats)
var problems []string
for _, s := range seats {
problems = append(problems, fmt.Sprintf(
"%s holds %s on %s, and %s depend on it (novox/hq ADR 0207) — unassign them with it, "+
"or assign another holder first", strings.Join(removing, ", "), s, node,
strings.Join(dependents[s], ", ")))
}
return &Refusal{Problems: problems}
}
// CollisionRefusal is why assigning `adding` beside `assigned` is refused for what two modules
// would both declare, or nothing (novox/hq ADR 0210 §1, 04-ISSUES/235).
//
// **Refused, not kept like an unresolved provision.** An assignment is otherwise kept when the
// node does not resolve, because assignment is not an ordering: a consumer's provider can follow.
// A collision is not an order anything can complete — no further assignment makes two owners of one
// package one owner — and kept, it leaves the node unresolvable, so the next push of anything drops
// it from the mesh. Only collisions involving a module being added are refused; one already on the
// node is `status`'s, and refusing an unrelated assignment for it would block its own remedy.
func CollisionRefusal(catalogue map[string]Manifest, node string, assigned, adding []string) error {
before := map[string]bool{}
for _, p := range checkResources(manifestsOf(catalogue, assigned)) {
before[p] = true
}
var problems []string
for _, p := range checkResources(manifestsOf(catalogue, append(append([]string(nil), assigned...), adding...))) {
if before[p] {
continue // on the node already; not this assignment's doing
}
problems = append(problems, p+" (novox/hq ADR 0210: one owner per node; the other module "+
"depends on the owner's seat instead)")
}
if len(problems) == 0 {
return nil
}
sort.Strings(problems)
return &Refusal{Problems: append(problems, fmt.Sprintf("nothing was assigned to %s", node))}
}
@@ -0,0 +1,247 @@
package catalogue
import (
"errors"
"reflect"
"strings"
"testing"
)
// Defends novox/hq ADR 0207: a module depends on the node seats that apply its resources.
func res(kind, id string) map[string]any {
r := map[string]any{"id": id, "type": kind}
switch kind {
case "service":
r["unit"] = id + ".service"
case "package":
r["package"] = id
case "container":
r["image"] = id
case "file":
r["path"] = "/etc/" + id
}
return r
}
func withResources(m Manifest, rs ...map[string]any) Manifest {
m.Resources = rs
return m
}
// The three holders as to-be 42 names them, each declaring what it really does: systemd's own
// package needs the package manager, pacman's timer needs the service manager, docker's package and
// service need both.
func coreThree() []Manifest {
return []Manifest{
withResources(mod("systemd", nil, nil, nil, Claim{Name: ServiceManagerSeat}), res("package", "systemd")),
withResources(mod("pacman", nil, nil, nil, Claim{Name: PackageManagerSeat}), res("service", "pacman-refresh")),
withResources(mod("docker", nil, nil, nil, Claim{Name: ContainerRuntimeSeat}),
res("package", "docker"), res("service", "docker")),
}
}
func TestADependencyIsDerivedFromEachResourceTypeThatAppliesThroughASeat(t *testing.T) {
cases := map[string][]string{
"service": {ServiceManagerSeat},
"package": {PackageManagerSeat},
"container": {ContainerRuntimeSeat},
// The host's own acts, or the mesh's: nothing in between holds a role for them.
"file": nil, "directory": nil, "process": nil, "archive": nil, "user": nil,
"action": nil, "network": nil, "access": nil,
}
for kind, want := range cases {
got := DependsOn(withResources(mod("m", nil, nil, nil), res(kind, "x")))
if len(got) == 0 {
got = nil
}
if !reflect.DeepEqual(got, want) {
t.Errorf("a %s resource depends on %v, want %v", kind, got, want)
}
}
all := DependsOn(withResources(mod("m", nil, nil, nil),
res("container", "a"), res("service", "b"), res("package", "c"), res("package", "d")))
if want := []string{ContainerRuntimeSeat, PackageManagerSeat, ServiceManagerSeat}; !reflect.DeepEqual(all, want) {
t.Errorf("a module of every kind depends on %v, want each seat once: %v", all, want)
}
}
func TestTheSeatsThatApplyResourcesAreTheMeshsOwnAtNodeScope(t *testing.T) {
for _, name := range []string{ServiceManagerSeat, PackageManagerSeat, ContainerRuntimeSeat} {
s, ok := SeatNamed(name)
if !ok {
t.Fatalf("%s is not in the mesh's set", name)
}
if s.Scope != ScopeNode {
t.Errorf("%s is held per %s, want per node", name, s.Scope)
}
}
// No verbs yet for either new seat: ADR 0207 seeds the package manager without a protocol, and
// the runtime's verbs wait for ADR 0166's acceptance.
for _, name := range []string{PackageManagerSeat, ContainerRuntimeSeat} {
if s, _ := SeatNamed(name); len(s.Serves)+len(s.Accepts)+len(s.Emits) > 0 {
t.Errorf("%s carries a protocol; ADR 0207 seeds it with none", name)
}
}
}
func TestANodeWhoseAssignmentsHoldTheSeatsResolvesWithNothingUnheld(t *testing.T) {
web := withResources(mod("web", nil, nil, nil), res("container", "web"), res("service", "web-timer"))
cat := shelf(append(coreThree(), web)...)
got, err := Resolve(cat, []string{"systemd", "pacman", "docker", "web"}, workstation(), World{})
if err != nil {
t.Fatal(err)
}
if len(got.Unheld) != 0 {
t.Errorf("a node holding all three seats reports %v", got.Unheld)
}
if _, err := AssignRefusal(cat, "workstation", []string{"systemd", "pacman", "docker"}, []string{"web"}); err != nil {
t.Errorf("assigning beside the three holders was refused: %v", err)
}
}
func TestAnAssignmentMissingAHolderIsRefusedNamingTheSeatAndItsPossibleHolders(t *testing.T) {
web := withResources(mod("web", nil, nil, nil), res("container", "web"), res("service", "web-timer"))
cat := shelf(append(coreThree(), web)...)
_, err := AssignRefusal(cat, "workstation", []string{"systemd", "pacman"}, []string{"web"})
var refusal *Refusal
if !errors.As(err, &refusal) {
t.Fatalf("web assigned to a node without a container runtime was not refused: %v", err)
}
msg := err.Error()
for _, want := range []string{"web on workstation depends on " + ContainerRuntimeSeat, "docker", "ADR 0207"} {
if !strings.Contains(msg, want) {
t.Errorf("the refusal does not say %q:\n%s", want, msg)
}
}
// What the node does hold is not named as missing.
if strings.Contains(msg, "depends on "+ServiceManagerSeat) {
t.Errorf("the refusal names a seat systemd already holds:\n%s", msg)
}
}
func TestADependencyNoCatalogueModuleCanMeetIsSaidNeverRefused(t *testing.T) {
// No runtime module in the catalogue: refusing would stop every container's assignment until one
// is written, with no remedy to name — so it is said, with the switch on as with it off.
web := withResources(mod("web", nil, nil, nil), res("container", "web"))
cat := shelf(web)
for _, on := range []bool{false, true} {
enforceSeatDependencies = on
said, err := AssignRefusal(cat, "workstation", nil, []string{"web"})
if err != nil {
t.Fatalf("switch %v: a dependency nothing could meet was refused: %v", on, err)
}
if len(said) != 1 || !strings.Contains(said[0], "no module in the catalogue claims it yet") {
t.Errorf("switch %v: the assignment does not say what it depends on: %v", on, said)
}
if _, err := Resolve(cat, []string{"web"}, workstation(), World{}); err != nil {
t.Errorf("switch %v: a dependency nothing could meet refused the node: %v", on, err)
}
}
enforceSeatDependencies = true
}
func TestTheHoldersMutualDependenceResolvesWhenAssignedTogether(t *testing.T) {
cat := shelf(coreThree()...)
// Alone, each needs the other.
if _, err := AssignRefusal(cat, "workstation", nil, []string{"systemd"}); err == nil ||
!strings.Contains(err.Error(), "pacman") {
t.Errorf("systemd alone was not refused naming pacman: %v", err)
}
if _, err := AssignRefusal(cat, "workstation", nil, []string{"pacman"}); err == nil ||
!strings.Contains(err.Error(), "systemd") {
t.Errorf("pacman alone was not refused naming systemd: %v", err)
}
// Together, in one act, they meet each other — and docker meets its own seat.
if _, err := AssignRefusal(cat, "workstation", nil, []string{"systemd", "pacman", "docker"}); err != nil {
t.Errorf("the three holders assigned together were refused: %v", err)
}
got, err := Resolve(cat, []string{"systemd", "pacman"}, workstation(), World{})
if err != nil {
t.Fatal(err)
}
if len(got.Unheld) != 0 {
t.Errorf("systemd and pacman together report %v", got.Unheld)
}
}
func TestStatusIsToldOfAnUnmetDependencyAndTheNodeStillResolves(t *testing.T) {
// Before the switch (the state the mesh ran in until every node held the three seats).
enforceSeatDependencies = false
defer func() { enforceSeatDependencies = true }()
web := withResources(mod("web", nil, nil, nil), res("container", "web"))
cat := shelf(append(coreThree(), web)...)
got, err := Resolve(cat, []string{"web"}, workstation(), World{})
if err != nil {
t.Fatalf("an unmet dependency refused the node before the switch: %v", err)
}
want := []Unheld{{Node: "workstation", Module: "web", Seat: ContainerRuntimeSeat, Holders: []string{"docker"}}}
if !reflect.DeepEqual(got.Unheld, want) {
t.Errorf("reported %+v, want %+v", got.Unheld, want)
}
}
func TestWithTheSwitchFlippedAnUnmetDependencyRefusesTheNode(t *testing.T) {
enforceSeatDependencies = true
defer func() { enforceSeatDependencies = true }()
web := withResources(mod("web", nil, nil, nil), res("container", "web"))
cat := shelf(append(coreThree(), web)...)
_, err := Resolve(cat, []string{"web"}, workstation(), World{})
if err == nil || !strings.Contains(err.Error(), ContainerRuntimeSeat) || !strings.Contains(err.Error(), "docker") {
t.Fatalf("with the switch on, an unmet dependency gave %v", err)
}
// Never in the first pass, whose refusals take a machine off the network instead.
if _, err := Resolve(cat, []string{"web"}, workstation(), World{Unchecked: true}); err != nil {
t.Errorf("the first pass refused an unmet dependency: %v", err)
}
}
func TestTheFoundationsDeclarationsAreNotJudged(t *testing.T) {
// The private network, as the controller computes it: its tools' package and its service are
// the mesh's, written per node, and so are never a module's dependency.
network := withResources(mod("mesh-wireguard", []string{"private-network"}, nil, nil),
res("package", "wireguard-tools"), res("service", "overlay-up"))
network.Computed = "mesh-wireguard"
// The host, whatever it declares.
host := withResources(mod("mesh-host", nil, nil, nil), res("file", "launcher"), res("service", "nox-mesh-host"))
cat := shelf(append(coreThree(), network, host)...)
for _, m := range []Manifest{network, host} {
if d := DependsOn(m); len(d) != 0 {
t.Errorf("%s, the foundation's, depends on %v", m.Module, d)
}
}
got, err := Resolve(cat, []string{"mesh-wireguard", "mesh-host"}, workstation(), World{})
if err != nil {
t.Fatal(err)
}
if len(got.Unheld) != 0 {
t.Errorf("the foundation on a node with no holders reports %v", got.Unheld)
}
if _, err := AssignRefusal(cat, "workstation", nil, []string{"mesh-wireguard", "mesh-host"}); err != nil {
t.Errorf("assigning the foundation was refused: %v", err)
}
}
func TestUnassigningTheLastHolderWhileDependentsRemainIsRefused(t *testing.T) {
sshd := withResources(mod("sshd", nil, nil, nil), res("service", "sshd"))
cat := shelf(append(coreThree(), sshd)...)
on := []string{"systemd", "pacman", "docker", "sshd"}
err := UnassignRefusal(cat, "workstation", on, []string{"systemd"})
if err == nil {
t.Fatal("the last service manager came off a node still running services")
}
for _, want := range []string{ServiceManagerSeat, "sshd", "pacman", "docker"} {
if !strings.Contains(err.Error(), want) {
t.Errorf("the refusal does not name %q:\n%v", want, err)
}
}
// A dependent comes off freely, and the holders with everything depending on them in one act.
if err := UnassignRefusal(cat, "workstation", on, []string{"sshd"}); err != nil {
t.Errorf("a dependent's unassignment was refused: %v", err)
}
if err := UnassignRefusal(cat, "workstation", on, on); err != nil {
t.Errorf("unassigning everything together was refused: %v", err)
}
}
+5 -5
View File
@@ -43,8 +43,8 @@ import (
var ofSeat = regexp.MustCompile(`\$\{seat:([a-z0-9][a-z0-9-]*):([0-9]+)\}`)
// seatInto replaces a resource's ${seat:…} placeholders with where this machine put each seat's
// holder — in a file's content, and in a value of a container's environment. The same two places
// portInto fills, for the same reason: they are where a process reads a number from.
// holder — in a file's content, and in a value of a container's or a process's environment. The
// same places portInto fills, for the same reason: they are where a program reads a number from.
func seatInto(resource map[string]any, module string, with Rendering) error {
switch fmt.Sprint(resource["type"]) {
case "file":
@@ -58,7 +58,7 @@ func seatInto(resource map[string]any, module string, with Rendering) error {
}
resource["content"] = filled
case "container":
case "container", "process":
env, ok := resource["env"].(map[string]any)
if !ok {
return nil
@@ -78,8 +78,8 @@ func seatInto(resource map[string]any, module string, with Rendering) error {
continue
}
value, err := seatsFilledInto(written,
fmt.Sprintf("%s's container %s sets %s to something that",
module, resource["name"], key), with)
fmt.Sprintf("%s's %s %s sets %s to something that",
module, resource["type"], resource["name"], key), with)
if err != nil {
return err
}
+10 -9
View File
@@ -101,7 +101,7 @@ func TestTheControlPlanesOwnAddressesFollowTheNodesPorts(t *testing.T) {
}
// The manifest itself names them now; withSeatPorts is a no-op on it, and this holds it so.
for _, r := range m.Resources {
if r["type"] != "container" {
if r["type"] != "process" {
continue
}
env, _ := r["env"].(map[string]any)
@@ -113,8 +113,9 @@ func TestTheControlPlanesOwnAddressesFollowTheNodesPorts(t *testing.T) {
}
m = withSeatPorts(m)
control, err := m.Resolve([]Built{{
Name: "server", Kind: ArtifactImage,
Reference: ArtifactStoreScheme + "mesh-controller/server@sha256:" + strings.Repeat("c", 64),
Name: "controller", Kind: ArtifactBundle,
Reference: ArtifactStoreScheme + "mesh-controller/controller@sha256:" + strings.Repeat("c", 64),
Digest: "sha256:" + strings.Repeat("c", 64),
}})
if err != nil {
t.Fatal(err)
@@ -135,9 +136,9 @@ func TestTheControlPlanesOwnAddressesFollowTheNodesPorts(t *testing.T) {
if err != nil {
t.Fatalf("the control plane does not compose: %v", err)
}
server := fileNamed(out, "mesh-controller.server")
server := fileNamed(out, "mesh-controller.controller")
if server == nil {
t.Fatalf("the control plane's container is not in the declaration: %v", out)
t.Fatalf("the control plane's process is not in the declaration: %v", out)
}
env, _ := server["env"].(map[string]any)
for key, want := range map[string]string{
@@ -151,8 +152,8 @@ func TestTheControlPlanesOwnAddressesFollowTheNodesPorts(t *testing.T) {
t.Errorf("the control plane is told %s=%v; the node put it on %s", key, env[key], want)
}
}
if got := server["image"]; got != "anchor.internal:5100/mesh-controller/server@sha256:"+strings.Repeat("c", 64) {
t.Errorf("the control plane's own image is %v, not routed through the store", got)
if got := server["source"]; got != "anchor.internal:5100/mesh-controller/controller@sha256:"+strings.Repeat("c", 64) {
t.Errorf("the control plane's own bundle is fetched from %v, not routed through the store", got)
}
// And on a mesh where the foundation is where genesis raised it, nothing is added.
@@ -160,7 +161,7 @@ func TestTheControlPlanesOwnAddressesFollowTheNodesPorts(t *testing.T) {
if err != nil {
t.Fatal(err)
}
env, _ = fileNamed(out, "mesh-controller.server")["env"].(map[string]any)
env, _ = fileNamed(out, "mesh-controller.controller")["env"].(map[string]any)
if env["MESH_STORE_INVENTORY_PORT"] != "" {
t.Errorf("with no settings, the control plane is told %v", env)
}
@@ -186,7 +187,7 @@ func withSeatPorts(m Manifest) Manifest {
out := m
out.Resources = nil
for _, r := range m.Resources {
if r["type"] != "container" {
if r["type"] != "container" && r["type"] != "process" {
out.Resources = append(out.Resources, r)
continue
}
+128 -3
View File
@@ -40,16 +40,34 @@ type Seat struct {
// Serves carries each verb in full — name, description, schema — because a role's tools are the
// mesh's to define and an agent's to call (novox/hq ADR 0132, design 33 §2).
Serves []Verb
// Receives is what other modules may contribute to the seat's holder, by kind (novox/hq ADR
// 0212): each kind is text in the tool's own grammar, placed by the holder with
// ${contribution:<seat>:<kind>}. Compiled, never stored: like the protocol, it is the mesh's
// definition of the role, and the store's rows carry no column for it.
Receives []Receivable
// Decision is the record that made it a seat.
Decision string
}
// Receivable is one kind of contribution a seat receives (novox/hq ADR 0212 §2): its name, and the
// comment prefix of the tool's grammar, with which the controller names each contributing module.
type Receivable struct {
Kind string
Comment string
// Dirs says a contribution of this kind may name its contributor's own directories as
// `${dir:<id>}`, filled with where they are on the machine before the holder places it (novox/hq
// to-be 43): a module saying which of its data to back up names a directory the mesh placed, and
// only the mesh knows where. Off for every kind written in a tool's grammar that has its own
// `${…}` — a shell's — where the mesh filling one would change what the tool reads.
Dirs bool
}
// defaultSeats is the set the mesh ships with — the seed for the control plane's seat table and the
// fallback when it has none (novox/hq ADR 0122). It is the one place the closed set 0110 defines is
// written; the store's table is seeded from it and thereafter is the live, editable copy.
//
// In the order a person reads it: the mesh's own, then a node's.
var defaultSeats = []Seat{
var defaultSeats = append([]Seat{
// The control plane states what it did under the seat it holds (novox/hq ADR 0134): a role's
// events belong to the role, so they keep their address while the holder is replaced. No accepts,
// so no work queue is raised for it — only what its holder may say.
@@ -107,7 +125,31 @@ var defaultSeats = []Seat{
// that machine unresolvable in the meantime. Deleted once no registered manifest claims it.
{Name: "mesh-build-machine", Scope: ScopeMesh,
Accepts: []string{"build"}, Emits: []string{"started", "built", "log.*"}, Decision: "novox/hq ADR 0190"},
// **The mesh's one resolver** (novox/hq ADR 0194, 0196): every node's internal domain, held in one
// place, and every node and container asks it first. Delivers what a machine's resolver
// configuration requires, so that requirement resolves to the holder wherever it is placed.
{Name: "mesh-dns-resolver", Scope: ScopeMesh, Delivers: "wildcard-resolution", Decision: "novox/hq ADR 0194"},
// **Retired by ADR 0194, kept while a manifest still claims it** — the same reason as
// mesh-build-machine above: a machine still holds it until the mesh's resolver replaces it, and
// removing the row first would make that machine unresolvable. Deleted once nothing claims it.
{Name: "node-dns-resolver", Scope: ScopeNode, Decision: "novox/hq ADR 0121"},
// **A machine's /etc/hosts is one module's** (novox/hq ADR 0199): its holder writes the machine's
// own lines and keeps every other line as the operator's, changed through these three verbs on that
// machine alone. The controller holds none of it.
{Name: "node-hosts-file", Scope: ScopeNode, Decision: "novox/hq ADR 0199",
Serves: []Verb{
{Name: "entries", Description: "Every line of this machine's /etc/hosts, each marked whose it is: " +
"the operator's, or the block of the module or tool that writes it.",
Input: schema(map[string]string{}, nil)},
{Name: "add", Description: "Add one address and its names to the operator's lines of this machine's " +
"/etc/hosts — a name for this machine's own programs, not the mesh's.",
Input: schema(map[string]string{"address": "the IPv4 or IPv6 address",
"names": "the names for it, separated by spaces"}, []string{"address", "names"})},
{Name: "remove", Description: "Remove one name, or every line of one address, from the operator's " +
"lines of this machine's /etc/hosts. A line a module writes is refused, naming the module.",
Input: schema(map[string]string{"name": "a host name, or an address to remove every line of"},
[]string{"name"})},
}},
// The intrusion prevention's verbs (novox/hq ADR 0179): what a person asks a machine's ban list
// whatever keeps it — who is banned and why, ban one address, let one go. Every holder serves all
// four; the jails themselves are composed from the modules the machine runs (to-be 31).
@@ -148,8 +190,50 @@ var defaultSeats = []Seat{
// system or user scope; the holder answers questions and operator acts about them, each verb
// taking the unit and an optional scope. The holder runs nothing of its own: its verbs are
// served by the node tools runtime (ADR 0175).
{Name: "node-service-manager", Scope: ScopeNode, Decision: "novox/hq ADR 0177",
{Name: ServiceManagerSeat, Scope: ScopeNode, Decision: "novox/hq ADR 0177",
Serves: serviceManagerVerbs()},
// The machine's package manager (novox/hq ADR 0207). A module declaring a `package` depends on
// it being held on its node, as one declaring a `service` depends on node-service-manager: the
// mesh's word for "something on this machine answers for installing", where a capability only
// says the software is there. No verbs yet — the seat says who answers, and what may be asked
// of it is decided when someone needs to ask.
{Name: PackageManagerSeat, Scope: ScopeNode, Decision: "novox/hq ADR 0207"},
// The machine's container runtime (novox/hq ADR 0166, seeded now by ADR 0207): a module
// declaring a `container` depends on it being held on its node. Its verbs, and the host creating
// containers through its holder, wait for ADR 0166's acceptance — seeded without them so the
// dependency has a seat to name and the runtime's module has one to claim.
{Name: ContainerRuntimeSeat, Scope: ScopeNode, Decision: "novox/hq ADR 0166, ADR 0207"},
// The operator account's environment (novox/hq ADR 0203): one module per machine writes it, and
// every module contributes to it. No verbs — the seat says who places the environment's files,
// and their path is its protocol: a shell sources ~/.config/mesh/environment.sh without knowing
// which module wrote it.
{Name: EnvironmentSeat, Scope: ScopeNode, Decision: "novox/hq ADR 0203"},
// The login shell (novox/hq ADR 0204, replacing the module-declared `login-shell` of ADR 0176):
// the mesh's, so a second shell module claims the seat rather than declaring a second one, and
// the seat exists whether or not zsh's definition is registered. `execute` is the contract any
// node may call; the holder places every module's shell code in its slots.
{Name: LoginShellSeat, Scope: ScopeNode, Decision: "novox/hq ADR 0204",
Serves: loginShellVerbs()},
// A machine's power (novox/hq ADR 0211): its holder owns logind's power handling, places the
// code modules contribute for the power moments, and publishes the machine's power states as
// its events. Every machine has one — every machine boots and shuts down. No verbs yet.
{Name: PowerSeat, Scope: ScopeNode, Decision: "novox/hq ADR 0211"},
// The machine's hotkeys (novox/hq ADR 0212): the daemon that sees the keys the window manager
// does not — a laptop's vendor keys — run by one module per machine, which owns its
// configuration. Every other module with keys contributes trigger lines to it.
{Name: HotkeysSeat, Scope: ScopeNode, Decision: "novox/hq ADR 0212",
Receives: []Receivable{{Kind: "trigger", Comment: "#"}}},
// The machine's message bus (novox/hq ADR 0215): its holder owns the D-Bus implementation and its
// system service, never restarts it live, and publishes curated events about it, never its traffic.
// It receives nothing yet: packages ship their own policies.
{Name: MessageBusSeat, Scope: ScopeNode, Decision: "novox/hq ADR 0215"},
// The machine's backups (novox/hq ADR 0214, to-be 43): its holder keeps nightly restore points of
// the data every module on the machine declares, on the machine, against mistakes rather than
// disasters. A module contributes `backup` lines — what to run to take a consistent copy, and
// which of its directories to keep.
{Name: BackupSeat, Scope: ScopeNode, Decision: "novox/hq ADR 0214",
Serves: backupVerbs(),
Receives: []Receivable{{Kind: "backup", Comment: "#", Dirs: true}}},
// Deferred (novox/hq ADR 0121): renaming to mesh-private-network is a scope + server/client
// model change, not a rename, so it stays until that is built.
{Name: "the-private-network", Scope: ScopeNode, Decision: "novox/hq ADR 0110"},
@@ -161,7 +245,9 @@ var defaultSeats = []Seat{
// rather than a condition in the resolver's module, so a machine running two managers is
// refused at assignment instead of found by the resolver being rewritten (novox/hq ADR 0117).
{Name: "node-uplink", Scope: ScopeNode, Decision: "novox/hq ADR 0117"},
}
},
// The graphical session's roles (novox/hq ADR 0208), last because they are a workstation's.
graphicalSessionSeats()...)
// A system seat name is the control plane's namespace: `mesh-*` for a mesh-wide role, `node-*` for
// a per-node one (novox/hq ADR 0121). A claim to a system name the mesh does not define is refused;
@@ -210,6 +296,10 @@ func UseSeats(s []Seat) {
row.Accepts, row.Emits, row.Serves = d.Accepts, d.Emits, d.Serves
}
}
// What a seat receives is never stored (novox/hq ADR 0212), so it is always the compiled one.
if d, known := byName[row.Name]; known {
row.Receives = d.Receives
}
merged = append(merged, row)
}
seats = merged
@@ -456,3 +546,38 @@ func serviceManagerVerbs() []Verb {
Input: scoped(map[string]string{"unit": unit["unit"], "lines": "how many lines from the end (default 100)"}, []string{"unit"})},
}
}
// loginShellVerbs is the contract every holder of node-login-shell serves (novox/hq ADR 0176, ADR
// 0204): one command, run the way the operator's own terminal would run it, bounded below the
// runtime's thirty-second call limit so a hung command answers rather than times the caller out.
// backupVerbs is the node-backup seat's protocol (novox/hq to-be 43): what is kept, take one now,
// and restore beside the live data — never over it.
func backupVerbs() []Verb {
return []Verb{
{Name: "backed-up", Description: "What this machine backs up: each module, what it declared, " +
"its last good night, how many restore points are kept and the repository's size.",
Input: schema(map[string]string{"module": "one module (optional)"}, nil)},
{Name: "now", Description: "Take a backup now, of one module or of every module on this " +
"machine — before a migration, a retirement or anything else that could go wrong.",
Input: schema(map[string]string{"module": "one module (optional)"}, nil)},
{Name: "restore", Description: "Restore one module's data from a restore point BESIDE the live " +
"data, never over it: each directory as <path>.restored-<date>. Swapping it in is a " +
"person's act. Lists the restore points when none is named.",
Input: schema(map[string]string{
"module": "the module",
"snapshot": "the restore point (from `backed-up`; the newest when omitted)",
"path": "one of the module's directories (all of them when omitted)",
}, []string{"module"})},
}
}
func loginShellVerbs() []Verb {
return []Verb{
{Name: "execute", Description: "Run one command on this machine as the operator account, in a " +
"non-interactive login shell in its home; answers with what it printed and how it exited.",
Input: schema(map[string]string{
"command": "the command line, as you would type it",
"timeout_seconds": "give up after this long, at most 25 (default 20)",
}, []string{"command"})},
}
}
+20
View File
@@ -25,6 +25,10 @@ import (
// and nothing to keep in step when a mesh seat is added.
const meshSeatPrefix = "mesh-"
// retiredLoginShell is the one name outside the prefix a module may not declare: the login shell's,
// from when a module declared it (novox/hq ADR 0176), before it became the mesh's (ADR 0204).
const retiredLoginShell = "login-shell"
// A SeatDeclaration is a role a module offers on the bus: what may be sent to it, what it says,
// and what it answers. A caller declares that it uses the *seat*, never the module, so the
// implementation can be replaced under it.
@@ -88,6 +92,15 @@ func declaredSeatProblems(m Manifest) []string {
"seats (novox/hq ADR 0118)", m.Module, s.Name, meshSeatPrefix+"*"))
continue
}
if s.Name == retiredLoginShell {
// The name ADR 0176 gave the login shell when the zsh module declared it. The seat is
// the mesh's now, so a module declaring the old name would be a second login shell
// beside it, with a protocol of its own (novox/hq ADR 0204).
problems = append(problems, fmt.Sprintf(
"%s declares a seat named %q; the login shell is the mesh's own seat %s, which a shell "+
"module claims and none declares (novox/hq ADR 0204)", m.Module, s.Name, LoginShellSeat))
continue
}
if seen[s.Name] {
problems = append(problems, fmt.Sprintf(
"%s declares the seat %q twice", m.Module, s.Name))
@@ -215,6 +228,13 @@ func CatalogueProblems(shelf Shelf) []string {
}
}
}
// A read of a module's state that module does not keep (novox/hq ADR 0201) — said only where the
// owner is on the shelf, as a consumer may be installed before its emitter.
var manifests []Manifest
for _, module := range shelfOrder(shelf) {
manifests = append(manifests, shelf[module])
}
problems = append(problems, StateReadsNothingDeclares(manifests)...)
sort.Strings(problems)
return problems
}
+11 -5
View File
@@ -18,7 +18,9 @@ import (
// vocabulary test follows. If this fails because a seat was added, the fix is a record in novox/hq
// and a row in to-be 26, not a new number here.
func TestTheSeatsAreAClosedSetAndEachNamesItsDecision(t *testing.T) {
record := regexp.MustCompile(`^novox/hq ADR \d{4}$`)
// A seat a later record extends names both, "novox/hq ADR 0166, ADR 0207": the one that defined
// it and the one that seeded it.
record := regexp.MustCompile(`^novox/hq ADR \d{4}(, ADR \d{4})*$`)
seen := map[string]bool{}
delivered := map[string]string{}
for _, s := range Seats() {
@@ -44,10 +46,14 @@ func TestTheSeatsAreAClosedSetAndEachNamesItsDecision(t *testing.T) {
delivered[s.Delivers] = s.Name
}
}
// Seventeen since node-build-agent (novox/hq ADR 0190) — sixteen once the retired
// mesh-build-machine row goes, when no registered manifest claims it any more.
if len(Seats()) != 17 {
t.Errorf("the mesh defines %d seats rather than 17; the set is closed, so a change here is "+
// Thirty-eight with node-backup (novox/hq ADR 0214); thirty-seven with node-message-bus (novox/hq ADR 0215); thirty-six with mesh-dns-resolver (novox/hq ADR 0194) and node-hosts-file (ADR 0199); thirty-four
// with node-hotkeys (ADR 0212); thirty-three with node-power (ADR 0211); thirty-two since the
// graphical session's eleven (ADR 0208); twenty-one with node-package-manager and
// node-container-runtime (ADR 0207); nineteen with node-environment and node-login-shell (ADR 0203,
// ADR 0204); seventeen with node-build-agent (ADR 0190). Two fewer once the retired
// mesh-build-machine and node-dns-resolver rows go, when no registered manifest claims either.
if len(Seats()) != 38 {
t.Errorf("the mesh defines %d seats rather than 38; the set is closed, so a change here is "+
"a decision (novox/hq ADR 0110): %s", len(Seats()), seatNames())
}
}
+42
View File
@@ -0,0 +1,42 @@
package catalogue
import (
"strings"
"testing"
)
// One account, several modules: the shell's module sets its shell, the container runtime's adds it
// to a group. Groups are only ever added by the host, so they are contributed; a shell or a home is
// one value, owned by one module per node.
func userResource(fields map[string]any) map[string]any {
r := map[string]any{"id": "operator", "type": "user", "name": "op"}
for k, v := range fields {
r[k] = v
}
return r
}
func TestAShellAndAGroupOnOneAccountFromTwoModulesResolve(t *testing.T) {
zsh := Manifest{Module: "zsh", Resources: []map[string]any{userResource(map[string]any{"shell": "/usr/bin/zsh"})}}
docker := Manifest{Module: "docker", Resources: []map[string]any{userResource(map[string]any{"groups": []any{"docker"}})}}
other := Manifest{Module: "media", Resources: []map[string]any{userResource(map[string]any{"groups": []any{"video"}})}}
if problems := checkResources([]Manifest{zsh, docker, other}); len(problems) != 0 {
t.Fatalf("a shell and two modules' groups on one account were refused: %v", problems)
}
if _, err := Resolve(shelf(zsh, docker, other), []string{"zsh", "docker", "media"}, workstation(), World{}); err != nil {
t.Fatalf("the three did not resolve together: %v", err)
}
}
func TestTwoModulesSettingOneAccountsShellOrHomeAreRefused(t *testing.T) {
for _, field := range []string{"shell", "home"} {
a := Manifest{Module: "zsh", Resources: []map[string]any{userResource(map[string]any{field: "/one"})}}
b := Manifest{Module: "fish", Resources: []map[string]any{userResource(map[string]any{field: "/two", "groups": []any{"x"}})}}
problems := checkResources([]Manifest{a, b})
want := `zsh and fish both set the ` + field + ` of the user "op"`
if len(problems) != 1 || !strings.Contains(problems[0], want) {
t.Errorf("two modules setting %s gave %v, want %q", field, problems, want)
}
}
}
+41 -17
View File
@@ -5,20 +5,24 @@ import (
"testing"
)
// The ssh-client module, composed as a machine receives it (novox/hq to-be 29): every other node's
// Host block written into a marked region of the operator's ~/.ssh/config, owned by the account,
// with ~/.ssh created 0700 — the operator's own config kept.
// The ssh-client module, composed as a machine receives it (novox/hq to-be 29, research 027/03): a
// region at the START of the operator's ~/.ssh/config that includes ~/.ssh/config.d/* — first,
// because ssh takes the first value it finds for each option — and the mesh's Host blocks as the
// whole of ~/.ssh/config.d/00-mesh, every other node with its account. ~/.ssh and ~/.ssh/config.d
// are created 0700 and owned by the account; the operator's own config below the region is kept.
func TestSSHClientOwnsTheOperatorsSSHConfig(t *testing.T) {
shelf := shelf(catalogueManifest(t, "ssh-client"))
got, err := Resolve(shelf, []string{"ssh-client"},
Node{Name: "homer", At: "homer.internal", Account: "jo"}, World{})
Node{Name: "homer", At: "homer.internal", Account: "jo", AccountHome: "/home/jo"}, World{})
if err != nil {
t.Fatal(err)
}
names := map[string]string{"homer.internal": "10.10.0.1", "marge.internal": "10.10.0.2"}
names := map[string]string{"homer.internal": "10.10.0.1", "marge.internal": "10.10.0.2",
"bart.internal": "10.10.0.3"}
out, err := got.Declaration(Rendering{
Names: names, Machines: names, Accounts: map[string]string{"homer": "jo", "marge": "jo"},
Suffix: "internal",
Names: names, Machines: names,
Accounts: map[string]string{"homer": "jo", "marge": "jo", "bart": "op"},
Suffix: "internal",
})
if err != nil {
t.Fatal(err)
@@ -28,19 +32,39 @@ func TestSSHClientOwnsTheOperatorsSSHConfig(t *testing.T) {
by[r["id"].(string)] = r
}
dir := by["ssh-client.ssh-dir"]
if dir == nil || dir["path"] != "/home/jo/.ssh" || dir["owner"] != "jo" || dir["mode"] != "0700" {
t.Fatalf("~/.ssh is not created 0700 owned by the account: %v", dir)
for id, path := range map[string]string{
"ssh-client.ssh-dir": "/home/jo/.ssh", "ssh-client.config-d": "/home/jo/.ssh/config.d"} {
dir := by[id]
if dir == nil || dir["type"] != "directory" || dir["path"] != path || dir["owner"] != "jo" || dir["mode"] != "0700" {
t.Fatalf("%s is not created 0700 owned by the account: %v", path, dir)
}
}
cfg := by["ssh-client.fact-ssh-config"]
if cfg == nil || cfg["path"] != "/home/jo/.ssh/config" || cfg["owner"] != "jo" || cfg["into"] != "block" {
t.Fatalf("the ssh config is not written into the operator's ~/.ssh/config as a region: %v", cfg)
cfg := by["ssh-client.config"]
if cfg == nil || cfg["path"] != "/home/jo/.ssh/config" || cfg["owner"] != "jo" ||
cfg["into"] != "block" || cfg["at"] != "start" {
t.Fatalf("the mesh's region is not the first thing in the operator's ~/.ssh/config: %v", cfg)
}
body := cfg["content"].(string)
if !strings.Contains(body, "Host marge marge.internal") || !strings.Contains(body, "User jo") {
t.Fatalf("the config does not name the peer node and its account:\n%s", body)
if body := cfg["content"].(string); !strings.Contains(body, "\nInclude ~/.ssh/config.d/*\n") || strings.Contains(body, "\nHost ") {
t.Fatalf("the region does not just include config.d:\n%s", body)
}
var hosts map[string]any
for _, r := range out {
if r["path"] == "/home/jo/.ssh/config.d/00-mesh" {
hosts = r
}
}
if hosts == nil || hosts["type"] != "file" || hosts["into"] != nil {
t.Fatalf("the mesh's hosts are not the whole of ~/.ssh/config.d/00-mesh: %v", hosts)
}
body := hosts["content"].(string)
for _, want := range []string{"Host marge marge.internal", "Host bart bart.internal", "User jo", "User op"} {
if !strings.Contains(body, want) {
t.Fatalf("00-mesh does not say %q — every other node with its account:\n%s", want, body)
}
}
if strings.Contains(body, "Host homer ") {
t.Fatalf("the config names the machine itself, not only its peers:\n%s", body)
t.Fatalf("00-mesh names the machine itself, not only its peers:\n%s", body)
}
}
+150
View File
@@ -0,0 +1,150 @@
package catalogue
import (
"bytes"
"encoding/json"
"fmt"
"regexp"
"strings"
)
// What a module may call its state, and whose state it may ask to read (novox/hq ADR 0201).
//
// A module names its state **locally** — `servers`, never a bucket or a subject — and another
// module's as `<module>.<name>`, the way a consumed event names its emitter (design 32 §1). The
// mesh derives the bucket from the two names, so the module and the local name must each be one
// token: the bucket joins them with an underscore, which neither may contain, so two modules can
// never derive one bucket.
// stateName is one local name of a module's state: lower-case, no dot, no underscore.
var stateName = regexp.MustCompile(`^[a-z0-9][a-z0-9-]*$`)
// The mesh's caps on what a module may ask of a bucket's history.
const (
// StateMostHistory is the most past values a key may keep. The server's own limit.
StateMostHistory = 64
)
// StateDeclaration is one bucket a module owns: its local name, and the options that are the
// owner's to choose, as a seat chooses how long its backlog survives (design 32 §3).
type StateDeclaration struct {
Name string `json:"name"`
// History is how many values a key keeps, the current one included; zero is one.
History int `json:"history,omitempty"`
// TTLSeconds is how long a value lives once written; zero is until it is replaced or deleted.
TTLSeconds int `json:"ttl-seconds,omitempty"`
}
// UnmarshalJSON reads a bucket as its bare name, or as {name, history, ttl-seconds}.
func (s *StateDeclaration) UnmarshalJSON(raw []byte) error {
trimmed := bytes.TrimSpace(raw)
if len(trimmed) > 0 && trimmed[0] == '"' {
return json.Unmarshal(trimmed, &s.Name)
}
type plain StateDeclaration
var full plain
dec := json.NewDecoder(bytes.NewReader(trimmed))
dec.DisallowUnknownFields()
if err := dec.Decode(&full); err != nil {
return fmt.Errorf("a state is either a name or {name, history, ttl-seconds}: %w", err)
}
*s = StateDeclaration(full)
return nil
}
// MarshalJSON writes back the short form when there is nothing else to say.
func (s StateDeclaration) MarshalJSON() ([]byte, error) {
if s.History == 0 && s.TTLSeconds == 0 {
return json.Marshal(s.Name)
}
type plain StateDeclaration
return json.Marshal(plain(s))
}
// ReadState splits a read into the owning module and the local name, or says why it is not one.
func ReadState(read string) (module, local string, err error) {
at := strings.LastIndex(read, ".")
if at <= 0 || at == len(read)-1 {
return "", "", fmt.Errorf("%q does not name a module and its state: a read is <module>.<name>", read)
}
module, local = read[:at], read[at+1:]
if !stateName.MatchString(module) {
return "", "", fmt.Errorf("%q cannot own state: a module whose state is read is one plain name", module)
}
if !stateName.MatchString(local) {
return "", "", fmt.Errorf("%q is not a state name: lower-case letters, digits and hyphens", local)
}
return module, local, nil
}
// StateProblems is what is wrong with a manifest's state and reads.
//
// Refused at registration, because a bucket name the bus cannot hold is a module that installs,
// starts, and is refused on its first write with a reason about a bucket nobody named.
func StateProblems(m Manifest) []string {
var problems []string
if len(m.State) > 0 && !stateName.MatchString(m.Module) {
problems = append(problems, fmt.Sprintf(
"%s keeps state, and a module's name is part of its buckets' names, which take one plain "+
"name — no dot (novox/hq ADR 0201)", m.Module))
}
seen := map[string]bool{}
for _, s := range m.State {
switch {
case !stateName.MatchString(s.Name):
problems = append(problems, fmt.Sprintf(
"%s keeps state %q: a state is named locally — lower-case letters, digits and hyphens, "+
"no dot and no underscore; the mesh derives the bucket (novox/hq ADR 0201)", m.Module, s.Name))
case seen[s.Name]:
problems = append(problems, fmt.Sprintf("%s keeps state %q twice", m.Module, s.Name))
}
seen[s.Name] = true
if s.History < 0 || s.History > StateMostHistory {
problems = append(problems, fmt.Sprintf(
"%s keeps %d values of %q; a key keeps between 1 and %d", m.Module, s.History, s.Name, StateMostHistory))
}
if s.TTLSeconds < 0 {
problems = append(problems, fmt.Sprintf("%s gives %q a negative lifetime", m.Module, s.Name))
}
}
for _, r := range m.Reads {
module, _, err := ReadState(r)
if err != nil {
problems = append(problems, fmt.Sprintf("%s reads %v", m.Module, err))
continue
}
if module == m.Module {
problems = append(problems, fmt.Sprintf(
"%s reads %q, which is its own state: a module reads and writes what it keeps already", m.Module, r))
}
}
return problems
}
// StateReadsNothingDeclares is every read across a catalogue whose owner is present and declares no
// such state. An absent owner says nothing — a module may be installed long before the one whose
// state it reads, as a consumer may before its emitter (design 32 §1).
func StateReadsNothingDeclares(manifests []Manifest) []string {
declared := map[string]map[string]bool{}
for _, m := range manifests {
own := map[string]bool{}
for _, s := range m.State {
own[s.Name] = true
}
declared[m.Module] = own
}
var problems []string
for _, m := range manifests {
for _, r := range m.Reads {
module, local, err := ReadState(r)
if err != nil {
continue
}
if own, present := declared[module]; present && !own[local] {
problems = append(problems, fmt.Sprintf(
"%s reads %q, and %s keeps no state called %q", m.Module, r, module, local))
}
}
}
return problems
}
+94
View File
@@ -0,0 +1,94 @@
package catalogue
import (
"encoding/json"
"strings"
"testing"
)
// A module declares the state it keeps and the state it reads (novox/hq ADR 0201), a bucket by its
// bare name or with the owner's options.
func TestAManifestMaySayWhatStateItKeepsAndReads(t *testing.T) {
m, err := ParseManifest([]byte(`{"module":"claude-code","version":"1",` +
`"state":["servers",{"name":"seen","history":5,"ttl-seconds":3600}],` +
`"reads":["licence-manager.bindings"]}`))
if err != nil {
t.Fatal(err)
}
if len(m.State) != 2 || m.State[0].Name != "servers" || m.State[1].History != 5 || m.State[1].TTLSeconds != 3600 {
t.Fatalf("state not read: %+v", m.State)
}
if len(m.Reads) != 1 || m.Reads[0] != "licence-manager.bindings" {
t.Fatalf("reads not read: %v", m.Reads)
}
// Written back as it came in: the short form where nothing else is said.
out, _ := json.Marshal(m.State)
if string(out) != `["servers",{"name":"seen","history":5,"ttl-seconds":3600}]` {
t.Fatalf("written back as %s", out)
}
}
// A name the bus could not hold, or that would let two modules derive one bucket, is refused at
// registration in the manifest's words.
func TestAStateNameIsLocalAndOneToken(t *testing.T) {
for _, c := range []struct{ manifest, says string }{
{`{"module":"a","version":"1","state":["mesh.servers"]}`, `keeps state "mesh.servers": a state is named locally`},
{`{"module":"a","version":"1","state":["my_servers"]}`, `keeps state "my_servers"`},
{`{"module":"a","version":"1","state":["s","s"]}`, `keeps state "s" twice`},
{`{"module":"a","version":"1","state":[{"name":"s","history":65}]}`, `a key keeps between 1 and 64`},
{`{"module":"a.b","version":"1","state":["s"]}`, `no dot`},
{`{"module":"a","version":"1","reads":["bindings"]}`, `a read is <module>.<name>`},
{`{"module":"a","version":"1","reads":["a.s"]}`, `which is its own state`},
{`{"module":"a","version":"1","state":[{"name":"s","shared":true}]}`, `{name, history, ttl-seconds}`},
} {
_, err := ParseManifest([]byte(c.manifest))
if err == nil {
t.Errorf("%s was accepted", c.manifest)
continue
}
if !strings.Contains(err.Error(), c.says) {
t.Errorf("%s refused for the wrong reason: %v", c.manifest, err)
}
}
}
// A read whose owner is present must name a state that owner keeps; an absent owner says nothing,
// because a module may be installed before the one whose state it reads.
func TestAReadNamesStateItsOwnerKeeps(t *testing.T) {
owner := Manifest{Module: "licence-manager", State: []StateDeclaration{{Name: "bindings"}}}
good := Manifest{Module: "claude-code", Reads: []string{"licence-manager.bindings", "absent.anything"}}
bad := Manifest{Module: "other", Reads: []string{"licence-manager.tokens"}}
if p := StateReadsNothingDeclares([]Manifest{owner, good}); len(p) != 0 {
t.Fatalf("a read of declared state was refused: %v", p)
}
p := StateReadsNothingDeclares([]Manifest{owner, bad})
if len(p) != 1 || !strings.Contains(p[0], `licence-manager keeps no state called "tokens"`) {
t.Fatalf("a read of state nobody keeps was not named: %v", p)
}
}
// **Across the whole catalogue**: every state name is local, and every read whose owner is present
// names state that owner keeps.
func TestEveryManifestsStateIsLocalAndEveryReadIsKept(t *testing.T) {
manifests := theCatalogue(t)
var problems []string
for _, m := range manifests {
problems = append(problems, StateProblems(m)...)
}
problems = append(problems, StateReadsNothingDeclares(manifests)...)
if len(problems) > 0 {
t.Fatalf("the catalogue's state is not what ADR 0201 says:\n %s", strings.Join(problems, "\n "))
}
}
// `module check` says it too: the cross-catalogue pass names a read nothing on the shelf keeps.
func TestTheCataloguePassNamesAReadItsOwnerDoesNotKeep(t *testing.T) {
shelf := Shelf{
"licence-manager": {Module: "licence-manager", State: []StateDeclaration{{Name: "bindings"}}},
"claude-code": {Module: "claude-code", Reads: []string{"licence-manager.tokens"}},
}
problems := CatalogueProblems(shelf)
if len(problems) != 1 || !strings.Contains(problems[0], `keeps no state called "tokens"`) {
t.Fatalf("the catalogue pass said %v", problems)
}
}

Some files were not shown because too many files have changed in this diff Show More