Compare commits

...
Author SHA1 Message Date
jschoubben 344af664b2 WIP: recheck a take's filesystem, follow only the module's branch, count users in an access change, hold the identity until its data is taken (hq ADR 0325 review) 2026-10-12 21:49:30 +02:00
jschoubben e1c2b74442 Pin the node-engine at its ADR 0325 merge (#94), and capture it beside 2026-10-12 21:00:40 +02:00
jschoubben 6500139a2c Pin the node-engine at the ADR 0325 follow-up rebased onto #93, and capture it beside 2026-10-12 20:37:36 +02:00
jschoubben 1c117d1189 Read the engine's access wait by its reason and ACL, name the tool runner by name, and show a kept folder's entries (hq ADR 0325)
A keep wait on a filesystem that cannot carry ACLs is not excused, and nothing is asked; one whose
filesystem must first enable them says in the ask that the take sets it. The tool runner's account is sent
with no uid, for the engine to resolve by name.
2026-10-12 20:14:40 +02:00
jschoubben 09c6f92b4a Judge an access the engine cannot see, and entries left while a disk was away, by whether the move changes accesses (hq ADR 0325)
An unknown access carries no wait: of a module whose move leaves its accesses as they were it is not
counted; of one whose move changes them, or is not known, it is not yet healthy, held to the gate's bound
so the gate fails safe and never hangs. Entries left while a disk was away follow the same rule.
2026-10-12 19:40:47 +02:00
jschoubben 2b15cb57f0 Ask again for a folder that moved, and carry entries left while a disk was away (hq ADR 0325)
The engine names the filesystem a waiting folder is on; a warrant is recorded for that filesystem, and a
folder on another is asked about again and not taken there meanwhile. Entries the engine could not take
out while their disk was away, stated before a send, do not hold that delivery's gate. mesh-host pinned
to d2d9fbbd.
2026-10-12 19:24:34 +02:00
jschoubben 63602264ef Give every module an identity and ask once to keep a shared folder (hq ADR 0325, issue 575)
A module's containers that declare identity run as an identity the controller allocates once, mesh-wide,
from 50000-59999; each access names its users and whether its folder is taken; an access wait is excused
for a folder the mesh may not keep yet, and the operator is asked once per folder and machine, the folder
taken only on the warrant. All of it is sent only to an engine at contract 5. A placement for an access a
newer build brings is judged against that build (issue 575). mesh-host pinned to the ADR 0325 branch head.
2026-10-12 19:05:13 +02:00
mesh-admin f10aedac58 Merge pull request 'A merge check judges the head merged into its trunk, and names the trunk commit (hq ADR 0323, issue 341)' (#271) from feat/341-a-merge-check-judges-the-merge into main 2026-10-12 16:11:27 +00:00
jschoubben 875ebc093e Format PullUpdated, and keep the announced base head on a rechecked member
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group feat/341-a-merge-check-judges-the-merge delivered: every member is delivered
A head checked again through mesh-delivery reached the direct verdict with no
base head, so a head no build judges was answered with no trunk commit and the
merge tool would refuse it (novox/hq ADR 0323).
2026-10-12 17:52:12 +02:00
mesh-admin 1e28342ede Merge pull request 'Make no value for an own secret issued outside the mesh that nobody gave (issue 401)' (#262) from fix/401-an-outside-secret-not-given-is-not-made into main 2026-10-12 15:48:12 +00:00
jschoubben bc623bc1f9 Name the announced trunk commit on a verdict no build judges
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check fail: its merge-check.sh failed: not gofmt'd by the toolchain's gofmt: internal/link/events.go
mesh/delivery-group group feat/341-a-merge-check-judges-the-merge rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
A change touching no module, in a repository with no merge-check.sh, was
answered by the controller with no trunk commit, so the merge tool refused it,
asked again and refused it for ever (novox/mesh-tools-go; novox/hq ADR 0323).
2026-10-12 17:44:18 +02:00
jschoubben ce9dc8e157 Run no hook and no signature setting when a check makes its merge
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
A build machine's or a clone's git configuration must not run code or change
the merge a check judges (novox/hq ADR 0323).
2026-10-12 17:39:59 +02:00
mesh-admin c672b3df6d Merge pull request 'Stop composing ADR 0312's group grant, so deliveries to the media machine pass again (hq issue 567)' (#270) from fix/567-revert-the-group-grant into main 2026-10-12 15:36:12 +00:00
jschoubben 25deb499bf Say why a merge's changed files read as none on an error
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
2026-10-12 17:31:36 +02:00
jschoubben 83a9cf6e3a Judge a pull request's head merged into its trunk, and name the trunk commit
A check judged the head alone, so two pull requests that each passed could
break the trunk together (node-engine #66 on 2026-10-09), and a recheck after
the trunk moved judged the same head again. The build seat now merges the head
into the trunk's head in its own clone, never pushed, fails a head that does not
merge cleanly naming the files, and the verdict names the head, the trunk
commit and the commits read beside it, for the forge and mesh-delivery to weigh
against the trunk's next move (novox/hq ADR 0323, issue 341).
2026-10-12 17:27:01 +02:00
jschoubben 7b050abdcb Stop composing ADR 0312's group grant, so deliveries to the media machine pass again
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
Since #256 delivered, the node-engine judges every read-write access by the path's group alone and
states six media modules unhealthy on ace, though they write their folders as the owner; every
delivery there fails its gate (hq issue 567). ADR 0325 replaces ADR 0312; until it is built, the
controller sends no grant, as before #256.

This reverts commit 3cfc121670.
2026-10-12 17:25:46 +02:00
jochen c3b4f187e1 Name issue 401's two store tests, so a skip without the store fails the check (issue 459's rule)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery failed: its walk failed, carrying its own commit: a gate on a first machine (what it carried put back), a build, a machine
2026-10-12 17:15:18 +02:00
jochen edd33f21d2 Make no value for an own secret issued outside the mesh that nobody gave (issue 401)
The send minted a random value for a plain own secret marked issued-by outside and placed it, so a
module could not tell "not given" from "given and wrong" (ADR 0228). It now places only a value a
person gave: not given is no file, an empty value where a module's file names it, and nothing to
restart on, as ADR 0283 decided for a family's member. A value the mesh made for one earlier is not
read as given.
2026-10-12 17:15:12 +02:00
mesh-admin 3cfc121670 Merge pull request 'Compose each read-write access's grant for the node-engine (hq ADR 0312, issue 537)' (#256) from feat/537-a-read-write-access-grants-the-paths-group into main 2026-10-12 14:20:27 +00:00
mesh-admin db7e4869ae Merge pull request 'Raise a machine that refuses or only partly applies its declarations past one apply interval (issue 233)' (#263) from fix/233-a-failed-probe-fails-only-what-needs-the-tool-2 into main 2026-10-12 14:11:10 +00:00
mesh-admin 9eddf36503 Merge pull request 'A directory's owner may follow the account its module runs as (issue 279)' (#260) from fix/279-owner-follows-a-setting into main 2026-10-12 14:00:23 +00:00
jochen d4de4543cc Judge S21 by time since the first failing report while the machine is heard, not by how many reports said it (issue 233 review)
mesh/merge-gate pass: builds mesh-controller → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/233-a-failed-probe-fails-only-what-needs-the-tool-2 delivered: every member is delivered
2026-10-12 15:39:08 +02:00
jochen a371de37fe Keep a directory owner's settings at the controller's terminal (issue 279)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
Root chowns a directory to the number its owner's setting names, so a key
an owner asks for must not change through the settings verb: any caller
could have set puid to 0 or to its own uid. TerminalKeys now takes every
${setting:…} in a directory's owner, whatever else asks for it, and the
owner is written as the parsed number so "0001" never reaches the
node-engine as text (review 60).
2026-10-12 15:37:29 +02:00
jochen 77acb40092 Let a directory's owner follow the account its module runs as (issue 279)
A media manager runs as the puid/pgid its assignment sets per node, but a
declared directory's owner was literal, so a folder the app must write was
refused on a machine whose identity was not the image's default. A
directory's owner now takes ${setting:<key>} (user, or user:group), filled
at composition and refused unless the value is a whole number, and the
module check refuses any other shape or a key the module asks for nowhere
else.
2026-10-12 15:37:23 +02:00
mesh-admin 5facd8b08e Merge pull request 'An act's answer is said from one resolution after its write (hq issue 237)' (#253) from fix/237-one-resolution-for-assign into main 2026-10-12 13:31:03 +00:00
mesh-admin 72edb458cc Merge pull request 'The node-reads seat: its verbs, its holder's own account, and the node's secret places (hq ADR 0310, issue 418)' (#257) from fix/418-node-reads into main 2026-10-12 13:20:23 +00:00
mesh-admin 2843834813 Merge pull request 'The build queue answers as data too (issue 455)' (#259) from fix/455-the-queue-answers-as-data into main 2026-10-12 13:04:26 +00:00
jochen 3e10ad1535 Raise a machine that refuses or only partly applies its declarations past one apply interval, naming the package manager and its words, so the mesh says it rather than only status (issue 233)
mesh/delivery-group group fix/233-a-failed-probe-fails-only-what-needs-the-tool-2 ready: every member ready, and composed together they pass
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
2026-10-12 14:49:33 +02:00
mesh-admin 27b13e4586 Merge pull request 'Ask a node's engine which of its directories the walk would refuse, read-only (issue 358)' (#258) from fix/358-judge-directories-read-only into main 2026-10-12 12:48:56 +00:00
mesh-admin 142a5ad71d Merge pull request 'An archive may be an upstream release pinned by URL and sha256, mirrored by the build seat (hq issue 345, ADR 0316)' (#255) from fix/345-builder-archive-pins into main 2026-10-12 12:33:58 +00:00
jochen 63a99d4323 Name the vault's placed directories among a node's secret places, fail loudly on one that cannot be resolved, and keep the reads account out of every group (review 54, issue 418)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/418-node-reads delivered: every member is delivered
The real mesh-vault manifest declares its directories without a path, so the
list left its state and ledger out, and a place still holding a placeholder
vanished from it without a word. The provider's directories are now resolved
where the node places them, an unresolved place fails the composition naming
the module, and a node-reads holder whose account is put in a group is refused.
2026-10-12 14:30:00 +02:00
jschoubben 0cd8bc0bfb Answer the build queue as data too, so a check can say how far back it waits
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
The queue verb answered only a table, so nothing outside the controller
could say an ask's place in the queue. It now asks its command for JSON:
each ask's state, kind, repository, commit, machine, since and position
among the waiting, with the table carried beside it and answered as the
verb's output, both written from one read. A repository's password, if
one were ever written into it, is left out of what is shown.

novox/hq issue 455
2026-10-12 14:22:22 +02:00
jschoubben 7385d9c092 Ask a node's engine which of its directories the walk would refuse, read-only (issue 358)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/358-judge-directories-read-only delivered: every member is delivered
Before the node-engine gives a mesh directory its mode and owner only through
a walk that follows no link another account could make, the operator is to
see which directories that walk would refuse on every node, the control-node
included, where no tool reads a path. `node judge-directories <node>` signs
the ask as a hand-over is, under its own context, and prints the engine's
one line per directory; it changes nothing, so it is a seat verb an agent may
call. The node is granted the subject, the controller alone its publish.
2026-10-12 14:17:01 +02:00
mesh-admin db9e9c5b87 Merge pull request 'Advise a recheck for an unanswered check on a ready or rejected delivery (issue 451)' (#254) from fix/451-a-recheck-is-the-step into main 2026-10-12 12:14:43 +00:00
jochen 4ead5a3df7 Define the node-reads seat, refuse a holder without an account of its own, and tell it where its node keeps a secret (hq ADR 0310, issue 418)
mesh/delivery-group group fix/418-node-reads rejected: a member's own check failed
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
Reading a file, searching one, listing a directory, the owner and mode of each
component of a path and looking up an account had no mesh tool, so they went
through the login shell's execute as the operator's account. node-reads is a
node seat with six verbs, each saying the commands it replaces. Its holder must
run as an account of its own (ADR 0310 §3), and its own runtime is given
MESH_SECRET_PLACES: the directory of every module's own secret, every credential
file and grant directory, the vault's directories and the sealed-secret keep,
so it refuses them by name before any read (§5).
2026-10-12 13:57:24 +02:00
jschoubben a23b190747 Merge remote-tracking branch 'origin/main' into fix/451-a-recheck-is-the-step
mesh/merge-gate pass: builds mesh-controller, route-proxy → ace, novox; no bus step; every machine composes with the change as it did without (4 of 4 compo…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/451-a-recheck-is-the-step delivered: every member is delivered
2026-10-12 13:56:52 +02:00
jochen 3be6e86f77 Say an act's answer from one resolution after its write (issue 237)
mesh/merge-gate pass: builds mesh-controller, route-proxy → ace, novox; no bus step; every machine composes with the change as it did without (4 of 4 compo…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
assign read the whole mesh before its write, to record the seats held by
derivation, and that read printed on standard error that the node did not
resolve for lack of the seat the act was about to give it. The seat verb
answers with both streams, so the stale line came last and named the
assignment just made as its remedy.

The reads inside an act no longer print 'left out' lines; the node's
unmet seat dependencies and whether it resolves come from one planFor
taken after the write, and every other machine from blockedElsewhere.
unassign now says when its node does not resolve, from the same
resolution.
2026-10-12 13:53:19 +02:00
jochen b0c4b849a7 Judge an upstream pin's host like any name, and harden the upstream unpack (review 35, hq issue 345)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
The url was skipped by the installation check, so an installation's host or
an address passed; it is judged whole now, the vendor's host declared in the
artifact's names-on-purpose, and an address or a reserved name as the host
is refused. Only the version's dotted-quad reading is skipped. The unpack
caps its entries, drains xz before waiting for it, reads a pax global
header and a nested within's parents, and a redirect off https is not
followed.
2026-10-12 13:51:35 +02:00
jochen b849af53dd Let an archive be an upstream release pinned by URL and sha256, mirrored by the build seat (hq issue 345, ADR 0316)
A mail client's release is about 80 MB per version, too large to keep in a
repository at every upgrade, and an add-on whose licence forbids copying it
to third parties may not be in a repository at all. An archive artifact may
now name url, sha256 and version (and file or within) instead of from: the
build seat fetches it once, refuses it when its digest is not the pin's and
names both, keeps it in its workspace under the digest (a vendor that serves
only its newest version cannot break a later build of the same pin), and
packs it like any archive, so a node gets the mesh's own archive by digest
and never asks the vendor. The build seat's image carries xz, which Go's
standard library does not read. The installation check takes the vendor's
address and version of such a pin for the world's, as a registry is an
image's.
2026-10-12 13:51:35 +02:00
jschoubben 74bd90df7e Pin the node-engine at its merge of the access grant, and capture it beside (hq ADR 0312, issue 537)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery failed: its walk failed, carrying its own commit: a gate on a first machine (what it carried put back), a build, a machine
2026-10-12 13:47:28 +02:00
jschoubben 30599c1858 Pin the node-engine at its reviewed grant, and hold the mount match to the path itself (hq ADR 0312, issue 537 review)
A path beginning with the same letters is another path, and a read-only mount
of a read-write access is still granted: the grant follows what mounts it.
2026-10-12 13:47:17 +02:00
jschoubben 10d4712f82 Name who a read-write access grants its path's group to, so modules write shared data through the group the operator gave it (hq ADR 0312, issue 537)
Each read-write access is composed with a grant: the module's containers that
mount its path and the account its tools run as (the operator's, or the module's
own under ADR 0259 §8); that runtime restarts on the access, so it holds a group
the engine put its account in. A read access grants nothing. Sent only to an
engine whose statement says GrantContract (4), since an older one refuses the
field. Pins the node-engine at the branch that reads it.
2026-10-12 13:47:17 +02:00
jschoubben e737e13bbb Advise a recheck for an unanswered check on a ready or rejected delivery (issue 451)
mesh/merge-gate pass: builds mesh-controller → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check fail: its merge-check.sh failed: --- FAIL: TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose (0.66s)
mesh/delivery-group group fix/451-a-recheck-is-the-step rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
That check answered once, so the build queue is the wrong place to look.
mesh-delivery names the delivery's state as data (held); the controller
reads it and never the words. A line without it keeps the advice of 438.
2026-10-12 13:47:11 +02:00
mesh-admin 4b1dee6b79 Merge pull request 'A merge that shares nothing with the assembling batch, and whose group waits for no one, is cut at once (hq issue 525, ADR 0304 §1, ADR 0306 §2-3)' (#250) from feat/525-a-lone-merge-skips-the-window into main 2026-10-12 11:46:59 +00:00
mesh-admin 4ab441ce16 Merge pull request 'Ask only once the bus enforces the controller's own grant, judged for real and failing closed (review of hq issue 353)' (#188) from fix/353-the-asker-judges-the-controllers-own-grant into main 2026-10-12 11:34:14 +00:00
jochen f996cf7b6b Take an older bus module's allowed as held, and name only remedies that work (review 31 of PR 188, hq issue 353)
mesh/merge-gate pass: builds mesh-controller → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
After mesh-catalog #166 merges and before the bus upgrade, the bus's
machine cannot be pushed, so any change to the composed list made the
list check read behind for good: every ask was withheld and the condition
named a push that is refused. An older module's allowed now counts as
held, the unconfirmed reload said once; the worst case is one ask refused
in the seconds before a reload, which is asked again (issue 369).

The list check stays for a module that says not allowed or cannot answer,
and its remedy is the plan's own reason when the list cannot be composed,
the bus upgrade while sends there wait on it, and a push otherwise. An
in_force false is a wait only when the module gives both times; otherwise
what is not known is named.
2026-10-12 13:21:32 +02:00
jochen bcfcfaaaf3 Keep asking on an older bus module and name the real remedy for each grant not held (review of hq issue 353)
A bus module older than its in_force answer read as not held, so a merge
before mesh-catalog #166 is live would silence every ask until a person's
bus upgrade. Such an answer now falls back to the check main makes (is the
list sent to the bus's machine the one composed now), says the reload is
not confirmed, and each case names its own remedy: push <node> for a list
behind or not allowing it, a ~5 s wait for a reload, the bus upgrade when
an older module's list cannot be judged.
2026-10-12 13:21:32 +02:00
jschoubben a3181e356b Ask only once the bus enforces the controller's own grant, judged for real and failing closed (review of hq issue 353)
change held asking back, a list sent was taken for a list the bus enforces, a plan that could not be
worked out read as not behind, and no test held the real judgement. Now the asker asks only when the
composition grants the controller itself to publish its ask, and the bus's own module says that grant
is in force on the running server (nats_user_can's in_force, mesh-catalog); each fact not known is not
held, said with why. brokerBehind takes a list nobody could compose as behind.
2026-10-12 13:21:32 +02:00
mesh-admin dd2fd28c92 Merge pull request 'closedNames waits for the verb's own connection by name, not the first closed one (issue 543)' (#252) from fix/543-wait-for-the-verbs-own-connection-by-name into main 2026-10-12 11:21:18 +00:00
mesh-admin 9d165c7500 Merge pull request 'plan-moved says what a walk touches, its bound and its outcome when it starts and ends (ADR 0308, issue 532)' (#247) from adr-0308-walk-touches into main 2026-10-12 11:08:53 +00:00
jochen 7630072e63 Wait for the verb's own connection by name, not the first closed one (issue 543)
mesh/merge-gate pass: the change touches no module of the mesh's graph
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
closedNames returned as soon as the bus's closed-connections list held
anything, which could be the test's own setup connection closing before
the verb subprocess's connection did. That raced TestAVerbsOwnProcessNamesItsConnection
red about once in 80 runs. It now waits until the connection it is named
for appears, or the deadline passes.
2026-10-12 13:05:15 +02:00
jochen 1e9dcf0667 Say every walk a verb keeps, and clear a stop from a walk that walks again
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
cancel, clear and rebuild kept a walk's end or restart without saying it, as plans stop, retry and
release-backlog did before them. The controller's run now says every save a verb makes on the bus, so the
next verb cannot lose a mark the same way. A stop is cleared when a walk is resumed, so one that fails again
ends failed and the search for its merge runs (review 21, ADR 0308, issue 532).
2026-10-12 12:54:20 +02:00
jochen cd6610cdf7 Say a walk's start or end when a person stops, closes, retries or releases it, and a hand stop as stopped
plans stop|close, plans retry and upgrade release-backlog kept a walk's start or end mark without saying it,
so the router would hold what the walk touched until its bound; each now runs on the bus as plans go does.
A hand stop keeps who stopped it, so its outcome is stopped, not failed (review 15, ADR 0308, issue 532).
2026-10-12 12:54:20 +02:00
jochen facc551ff2 Say what a walk touches, its bound and its outcome when it starts and ends (ADR 0308, issue 532)
The router holds what an open walk touches and tells its result once; it can do so
only if plan-moved names the walk's start and end, the modules and machines it moves,
what a moved tool runner or node-engine carries there, and how long to hold.
2026-10-12 12:54:20 +02:00
jschoubben 71b3aeaf2c Keep a group member of the other kind's batch from being overtaken (hq issue 525)
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
A merge's group member counted as merged when its merge waited in any batch,
but kinds never share a batch: the later member got a batch of its own, cut
at once, and walked before the earlier member's batch was cut, breaking the
group's order (ADR 0276 §5). Only a member kept in a batch of the merge's own
kind counts as merged now; one kept in the other kind's keeps the merge
waiting, and its batch names that member, so the two are cut in the order
their windows close (review of #250).
2026-10-12 12:54:11 +02:00
mesh-admin fafa3fd9ef Merge pull request 'Require the service manager's reset-failed, wanted-by and unlink-dangling verbs (issue 332)' (#248) from fix/332-require-service-manager-verbs into main 2026-10-12 10:53:20 +00:00
jschoubben 6e5a1df575 Judge the installer's user list against mesh-host with the groups grant (hq issue 525)
mesh/repo-check fail: its merge-check.sh failed: --- FAIL: TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose (0.78s)
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate pass: builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step; every machine composes with the change as it…
The controller now asks mesh-delivery.groups, and mesh-host a76baa4b carries
that grant in the installer's first user list (mesh-host #85). The captured
copy is moved to that commit so the genesis test judges the list that ships.
2026-10-12 12:31:50 +02:00
jochen 61d00a234a Give the store tests' service manager holders the three verbs it now must serve (issue 332)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
They named only the eight verbs from before issue 332 and were refused once the seat required
reset-failed, wanted-by and unlink-dangling.
2026-10-12 12:25:05 +02:00
jschoubben 4d203b5a5d Ask mesh-delivery whether a merge's group still assembles (hq issue 525)
A merge from any head waited for the window because the controller could not
see a group's members, so the window skip changed nothing in production and
unrelated merges walked one after another. The controller now asks
mesh-delivery's groups, within 3 s and before the plans are held: a merge
waits only when its head's group has another member whose pull request is
open and whose merge it does not keep yet (ADR 0306 §2). No answer means it
waits, and its batch says why. A headed merge in no waiting group and related
to no batch is cut at once, as one from no head is (ADR 0304 §1, ADR 0306 §3).

The controller's grant names mesh-delivery.groups. The installer's first user
list in mesh-host must carry it too before the genesis test can pass.
2026-10-12 12:14:46 +02:00
jschoubben 6196a4815e Let a merge that shares nothing skip the merge window (hq issue 525)
A merge waited the whole window even when nothing assembling shared anything
with it. It now joins only a batch it is related to (ADR 0304 §1, read by ADR
0306 §3: repository, a module the cut would move, a graph edge, a group head),
folding the newer into the oldest when it relates to two. A merge related to
none starts a batch whose window is closed at once (PlanBatch.CutAtOnce),
unless its delivery group may still have a member to merge (ADR 0306 §2).

The controller cannot see a group's unmerged members (that is mesh-delivery's
record), so a merge from any head still waits, in a batch of its own: the safe
error is to wait. Walks stay one at a time; a batch cut at once queues behind
an open walk. A stop's merges walked again join a related batch, or their own.
2026-10-12 12:14:46 +02:00
jochen 0a0f5b4a5a Require the service manager's reset-failed, wanted-by and unlink-dangling verbs now every holder serves them (issue 332)
mesh/merge-gate pass: builds build-agent, mesh-controller → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without …
mesh/repo-check fail: its merge-check.sh failed: --- FAIL: TestAVerbsOwnProcessNamesItsConnection (4.62s); and TestASeatsVerbGainsTheArgumentsTheBinaryNames
mesh/delivery superseded: a newer head of the same pull request
ADR 0246 step 3: the systemd module, the seat's only holder, serves all three on every node,
so a holder that stops serving one is refused rather than leaving a node without them.
2026-10-12 12:03:37 +02:00
117 changed files with 10021 additions and 313 deletions
+2
View File
@@ -6,3 +6,5 @@
/mesh-controller
/postgres-provisioner
/redis-provisioner
# And where `go build` in cmd/mesh-controller leaves it (novox/hq issue 233).
/cmd/mesh-controller/mesh-controller
+3 -2
View File
@@ -16,7 +16,8 @@ RUN CGO_ENABLED=0 go build -trimpath -ldflags '-s -w' -o /mesh-builder ./cmd/mes
FROM ${ALPINE_BASE}
# git to clone what it is asked to build, and the docker client to build and push it. The daemon
# is the machine's, reached through its socket — a build machine shares the runtime it was given
# rather than running one inside itself.
RUN apk add --no-cache git docker-cli
# rather than running one inside itself. xz to read an upstream release an archive is pinned to
# (novox/hq ADR 0316): Go's standard library reads gzip and bzip2 and not xz.
RUN apk add --no-cache git docker-cli xz
COPY --from=build /mesh-builder /usr/local/bin/mesh-builder
ENTRYPOINT ["/usr/local/bin/mesh-builder"]
+8 -2
View File
@@ -256,7 +256,13 @@ func answer(ctx context.Context, publisher builder.Publisher, on, workspace stri
v, err = builder.Check(building, builder.Command, checkSpecOf(request), workspace, registry, forgeFrom(), say)
if err == nil {
result.Check = &link.CheckOutcome{Verdict: v.Verdict, Summary: v.Summary, Report: v.Report,
Took: v.Took.Round(time.Second).String(), Gate: layerOf(v.Gate), RepoCheck: layerOf(v.Repo)}
Took: v.Took.Round(time.Second).String(), Gate: layerOf(v.Gate), RepoCheck: layerOf(v.Repo),
Head: v.Head, Trunk: v.Trunk, Beside: v.Beside}
// What each of a group's heads was judged as (novox/hq ADR 0323 §6).
for _, m := range v.Members {
result.Check.Members = append(result.Check.Members, link.CheckedMember{Owner: m.Owner, Repo: m.Repo,
Commit: m.Head, Trunk: m.Trunk})
}
}
} else if err == nil {
// The package-registry credential is a build input, so it is resolved before the clone: a
@@ -354,7 +360,7 @@ func checkSpecOf(request link.BuildRequest) builder.CheckSpec {
}
for _, m := range c.Members {
spec.Group = append(spec.Group, builder.GroupHead{Owner: m.Owner, Repo: m.Repo, Repository: m.Repository,
Ref: m.Ref, Paths: m.Paths})
Ref: m.Ref, Paths: m.Paths, Base: m.Base})
}
return spec
}
+67 -15
View File
@@ -4,6 +4,7 @@ import (
"context"
"fmt"
"github.com/novox/mesh-controller/internal/broker"
"io"
"slices"
"sort"
"strings"
@@ -54,6 +55,9 @@ func assignWith(ctx context.Context, open *stores, node string, opts assignOptio
if err := refusedMovingTheController(modules); err != nil {
return "", err
}
// Every line of the answer comes from the one resolution after the write (novox/hq issue 237):
// what the reads before it would say about a machine left out is not the answer's.
ctx = leftOutSaidBy(ctx, io.Discard)
// Held while it is recorded, so it cannot land between a converge's preview and its flip and
// be taken without ever having been previewed (novox/hq ADR 0100).
ctx, release, err := holdNodes(ctx, open, []string{node})
@@ -125,22 +129,28 @@ func assignWith(ctx context.Context, open *stores, node string, opts assignOptio
for _, line := range settled {
answer += "\n " + line
}
// What this act changed about this node's unmet seat dependencies, and nothing else (novox/hq
// ADR 0207): a dependency of a module just assigned, or one this assignment met. The rest of the
// node's list, and every other node's, is `status`'s.
for _, line := range unheldChange(shelf, node, before, append(append([]string(nil), before...), added...)) {
answer += "\n " + line
}
// Its bus credential, in the same act (novox/hq issue 203): an assignment pushed before its
// credential exists delivers a process that cannot authenticate and crash-loops until somebody
// runs a second verb and a second push. Issued here when the module speaks on the bus and has
// no credential yet; kept when it has one, so re-assigning rotates nothing.
var issued []string
for _, module := range added {
if line := issueOnAssign(ctx, open, node, module); line != "" {
answer += "\n " + line
issued = append(issued, line)
}
}
plan, _, err := planFor(ctx, open, node)
// **The one resolution the rest of the answer is said from** (novox/hq issue 237), taken after
// the write: what this act changed about this node's unmet seat dependencies, and nothing else
// (novox/hq ADR 0207) — a dependency of a module just assigned, or one this assignment met; the
// rest of the node's list, and every other node's, is `status`'s — and whether the node resolves.
after := resolvedAfter(ctx, open, shelf, node)
for _, line := range unheldChangeTo(shelf, node, before, after.modules, after.unheld) {
answer += "\n " + line
}
for _, line := range issued {
answer += "\n " + line
}
plan, err := after.plan, after.err
if err != nil {
// Kept, and still refused. Both halves are the answer, and the rest of the mesh is still
// worth reporting: this machine's refusal is rarely the only consequence.
@@ -218,6 +228,8 @@ func unassign(ctx context.Context, open *stores, node string, modules ...string)
if len(modules) == 0 {
return "", fmt.Errorf("unassign %s names no module", node)
}
// One resolution after the write, as assign's answer (novox/hq issue 237).
ctx = leftOutSaidBy(ctx, io.Discard)
ctx, release, err := holdNodes(ctx, open, []string{node})
if err != nil {
return "", err
@@ -262,15 +274,22 @@ func unassign(ctx context.Context, open *stores, node string, modules ...string)
}
answer := fmt.Sprintf("%s no longer runs %s — run `push %s` to make it so",
node, strings.Join(modules, ", "), node)
var left []string
for _, a := range assigned {
if !slices.Contains(modules, a) {
left = append(left, a)
}
}
for _, line := range unheldChange(shelf, node, assigned, left) {
after := resolvedAfter(ctx, open, shelf, node)
for _, line := range unheldChangeTo(shelf, node, assigned, after.modules, after.unheld) {
answer += "\n " + line
}
// Whether the node itself still resolves, from the same resolution: before, only the rest of the
// mesh's reads said it, on standard error and once per other machine.
if after.err != nil {
if unresolvable(after.err) {
answer += fmt.Sprintf("\n but %s does not resolve now, so nothing will be sent to it:", node)
} else {
answer += fmt.Sprintf("\n but whether %s still resolves could not be read:", node)
}
for _, line := range strings.Split(strings.TrimRight(after.err.Error(), "\n"), "\n") {
answer += "\n " + strings.TrimSpace(line)
}
}
// What it leaves behind that is irreplaceable is kept and retired, never removed (novox/hq ADR 0233).
for _, line := range keptOnUnassign(ctx, open.inventory, node, modules) {
answer += "\n " + line
@@ -278,6 +297,39 @@ func unassign(ctx context.Context, open *stores, node string, modules ...string)
return answer + blockedElsewhere(ctx, open, node), nil
}
// afterAct is the one resolution of a node an act's answer is said from, taken after its write
// (novox/hq issue 237): the plan or why there is none, the modules it has on the node, and its unmet
// seat dependencies.
type afterAct struct {
plan catalogue.Resolution
err error
modules []string
unheld []catalogue.Unheld
}
// resolvedAfter resolves the node as `plan` does, once, after the act's write. When the node resolves,
// its modules and unmet seat dependencies are the resolution's — the closure, so a holder pulled in by
// a requirement counts as `plan` counts it. When it does not, they are judged over the assignments read
// after the write, which can only find more unmet than the closure would: a seat said held there is
// held in the closure too.
func resolvedAfter(ctx context.Context, open *stores, shelf map[string]catalogue.Manifest, node string) afterAct {
plan, _, err := planFor(ctx, open, node)
if err == nil {
out := afterAct{plan: plan, unheld: plan.Unheld}
for _, m := range plan.Modules {
out.modules = append(out.modules, m.Module)
}
return out
}
out := afterAct{err: err}
assigned, readErr := open.inventory.Assigned(ctx, node)
if readErr != nil {
return out
}
out.modules, out.unheld = assigned, unheldOver(shelf, node, assigned)
return out
}
// splitModules is a surface's one `module` field as the modules it names: several, comma-separated,
// are one act (novox/hq ADR 0207), so the holders that depend on each other go on together from the
// command API and the controller seat's verbs as they do from the command line.
+118
View File
@@ -0,0 +1,118 @@
package main
import (
"fmt"
"sort"
"strings"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
)
// kindApplyFailing is S21's kind: a machine whose declarations have been refused, or applied only in part,
// for longer than one apply interval (novox/hq issue 233).
const kindApplyFailing = "apply-failing"
// applyEvery is how often a node-engine applies what it was last told when nothing new arrives: its
// ReconcileEvery. A failure still there after one more of them is not one the next apply clears.
const applyEvery = 5 * time.Minute
// packageManagerKey is how a node-engine names a package manager that does not answer among what failed
// (mesh-host's probe.go): "the package manager (pacman)".
const packageManagerKey = "the package manager ("
// watchApplyFailing is S21: **a machine that does not apply what it was sent is said by the mesh, not only
// to someone who asks for `status`** (novox/hq issue 233, issue 187's family). A workstation refused every
// declaration for an hour and a half because its package manager did not answer, and the mesh's only word
// on it was a line in `status`. Raised once the machine's first report of this failure is older than one apply
// interval **while the machine is heard** — judged by time, not by how many reports said it: a node-engine
// says a re-apply only when how it went changed, so the same failure is one report however long it lasts.
// Names the refusal's or the failure's words; cleared by the clean apply the node-engine then says. Not while
// the machine is silent, asleep or not known: S1 says silent, and asleep is not lost.
func watchApplyFailing(f *signalFacts) []conditions.Observation {
heard := map[string]bool{}
for _, m := range f.machines {
if !m.asleep() && !m.lastHeard.IsZero() && f.now.Sub(m.lastHeard) <= heartbeatBound(m.every) {
heard[m.name] = true
}
}
var out []conditions.Observation
for _, d := range f.doing {
if !d.Wrong() || d.Since == nil || f.now.Sub(*d.Since) <= applyEvery || !heard[d.Node] {
continue
}
out = append(out, applyFailing(d, f.now))
}
return out
}
// applyFailing is one machine's S21 condition.
func applyFailing(d inventory.Doing, now time.Time) conditions.Observation {
m := d.Node
since := d.Since.UTC().Format("2006-01-02 15:04 MST")
o := conditions.Observation{Scope: conditions.ScopeMachine, ID: m, Kind: kindApplyFailing, Machine: m,
Severity: conditions.Warning, Said: doingWords(d),
Resolved: m + " applies what it is sent again"}
how := "refused its declaration whole"
if d.Outcome != inventory.OutcomeRefused {
how = "applied its declaration only in part"
}
o.Summary = fmt.Sprintf("%s has %s since %s (bound %s): %s", m, how, since, ago(applyEvery), failedNames(d))
o.Headline = m + " is not applying its update"
o.Explanation = fmt.Sprintf("%s has %s for %s, and trying again has not helped. The details say what fails.",
m, plainHow(d), humanDuration(now.Sub(*d.Since)))
if tool := unansweringTool(d); tool != "" {
o.Summary = fmt.Sprintf("%s: %s does not answer, so no package reaches it; everything else it was sent "+
"is applied, since %s (bound %s)", m, tool, since, ago(applyEvery))
o.Headline = fmt.Sprintf("%s cannot install packages", m)
o.Explanation = fmt.Sprintf("The package manager on %s (%s) has not answered for %s, so no package reaches "+
"it. Everything else is still applied, so an update that repairs it can land; if none does, it needs a "+
"look on the machine.", m, tool, humanDuration(now.Sub(*d.Since)))
}
return o
}
// plainHow is how a machine failed, as the operator reads it.
func plainHow(d inventory.Doing) string {
if d.Outcome == inventory.OutcomeRefused {
return "refused every update"
}
return "applied its updates only in part"
}
// unansweringTool is the package manager a machine said does not answer, or "".
func unansweringTool(d inventory.Doing) string {
for _, r := range d.Failed {
if rest, ok := strings.CutPrefix(r.ID, packageManagerKey); ok {
return strings.TrimSuffix(rest, ")")
}
}
return ""
}
// failedNames is what failed, by name: the refusal, or the failed resources' ids.
func failedNames(d inventory.Doing) string {
if d.Outcome == inventory.OutcomeRefused {
return "refused"
}
ids := make([]string, 0, len(d.Failed))
for _, r := range d.Failed {
ids = append(ids, r.ID)
}
sort.Strings(ids)
return "failed " + strings.Join(ids, ", ")
}
// doingWords is the machine's own words on what failed: the evidence, kept inside the mesh.
func doingWords(d inventory.Doing) string {
if d.Outcome == inventory.OutcomeRefused {
return "refused: " + oneLine(d.Refused)
}
var parts []string
for _, r := range d.Failed {
parts = append(parts, r.ID+": "+oneLine(r.Error))
}
sort.Strings(parts)
return strings.Join(parts, "; ")
}
+123
View File
@@ -0,0 +1,123 @@
package main
import (
"context"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
)
// unansweringPackageManager is anchor's last report as the workstation's read on 2026-10-04, once its
// node-engine applies the rest (novox/hq issue 233): the package failed and the package manager did not answer,
// the same failure on times reports since for before now.
func unansweringPackageManager(now time.Time, forHowLong time.Duration, times int) inventory.Doing {
since := now.Add(-forHowLong)
return inventory.Doing{Node: "anchor", Outcome: inventory.OutcomeFailed, At: now.Add(-time.Second),
Since: &since, Times: times, Applied: 41, Failed: []inventory.FailedResource{
{ID: "apply", Error: `applying "pacman.base": nothing can be said about "base": this is the arch host and ` +
"pacman does not answer here. Either this machine is not Arch, or its package database is broken: " +
"pacman exited 1: error: config file /etc/pacman.conf could not be read: No such file or directory"},
{ID: "the package manager (pacman)", Error: "pacman does not answer here, so every package in this " +
"declaration fails and the rest of it is applied (novox/hq issue 233): pacman exited 1: error: config " +
"file /etc/pacman.conf could not be read: No such file or directory"}}}
}
// **A machine whose package manager does not answer is raised by the mesh itself, naming the tool and the
// probe's words** (novox/hq issue 233): the words in the evidence, the path kept out of the summary.
func TestAPackageManagerThatDoesNotAnswerIsRaisedNamingItsWords(t *testing.T) {
now := time.Date(2026, 10, 4, 14, 0, 0, 0, time.UTC)
f := calm(now)
f.doing = []inventory.Doing{unansweringPackageManager(now, 90*time.Minute, 18)}
got := watchApplyFailing(f)
if len(got) != 1 {
t.Fatalf("raised %d conditions, want one: %+v", len(got), got)
}
o := got[0]
if o.Key() != "machine.anchor.apply-failing" || o.Machine != "anchor" {
t.Errorf("the condition is not the machine's: %s", o.Key())
}
if !strings.Contains(o.Summary, "pacman does not answer") || !strings.Contains(o.Summary, "since 2026-10-04 12:30") {
t.Errorf("the summary does not name the tool and since when: %s", o.Summary)
}
if strings.Contains(o.Summary, "/etc/") {
t.Errorf("the summary carries a path, which belongs in the evidence: %s", o.Summary)
}
if !strings.Contains(o.Said, "could not be read") {
t.Errorf("the evidence does not carry the probe's words: %s", o.Said)
}
if !strings.Contains(o.Headline, "cannot install packages") || o.Explanation == "" || o.Resolved == "" {
t.Errorf("the plain words do not say it: %+v", o)
}
}
// **A whole refusal is raised too**, with the node-engine's words: what the workstation's report said before the
// node-engine applied the rest.
func TestARefusalThatGoesOnIsRaised(t *testing.T) {
now := time.Date(2026, 10, 4, 14, 0, 0, 0, time.UTC)
since := now.Add(-time.Hour)
f := calm(now)
f.doing = []inventory.Doing{{Node: "anchor", Outcome: inventory.OutcomeRefused, Since: &since, Times: 12,
Refused: "this is the arch host and pacman does not answer here", At: now.Add(-time.Minute)}}
got := watchApplyFailing(f)
if len(got) != 1 || !strings.Contains(got[0].Summary, "refused its declaration whole") ||
!strings.Contains(got[0].Said, "pacman does not answer") {
t.Fatalf("a refusal that went on for an hour was not raised with its words: %+v", got)
}
}
// **Judged by time while the machine is heard, not by how many reports said it** (review of novox/hq issue 233):
// a node-engine says the same failure once, so one report an apply interval old is a fault that goes on; one
// younger is not yet; and a silent or sleeping machine is S1's.
func TestOneReportOlderThanAnApplyIsRaisedAndAQuietMachineIsNot(t *testing.T) {
now := time.Date(2026, 10, 4, 14, 0, 0, 0, time.UTC)
f := calm(now)
f.doing = []inventory.Doing{unansweringPackageManager(now, 6*time.Minute, 1)}
if got := watchApplyFailing(f); len(got) != 1 {
t.Errorf("one report of a failure six minutes old was not raised: %+v", got)
}
f.doing = []inventory.Doing{unansweringPackageManager(now, 4*time.Minute, 1)}
if got := watchApplyFailing(f); len(got) != 0 {
t.Errorf("a failure four minutes old was raised before one apply could clear it: %+v", got)
}
f.doing = []inventory.Doing{unansweringPackageManager(now, time.Hour, 1)}
f.machines[0].lastHeard = now.Add(-time.Hour)
if got := watchApplyFailing(f); len(got) != 0 {
t.Errorf("a silent machine was raised for its last report: %+v", got)
}
f = calm(now)
f.doing = []inventory.Doing{unansweringPackageManager(now, time.Hour, 1)}
f.doing[0].Node = "a-machine-the-mesh-does-not-know"
if got := watchApplyFailing(f); len(got) != 0 {
t.Errorf("a machine not heard of was raised: %+v", got)
}
}
// **A failed re-apply followed by a clean one clears S21** (review of novox/hq issue 233): the node-engine says
// the clean re-apply, the store keeps it as applied with no failure since, and the condition the failure raised
// is cleared through the keeper on the next tick.
func TestACleanReapplyAfterAFailedOneClearsTheCondition(t *testing.T) {
now := time.Date(2026, 10, 4, 14, 0, 0, 0, time.UTC)
store := conditions.NewInMemory()
k := conditions.NewKeeper(t.Context(), conditions.Options{Store: store, History: store,
Teller: &conditions.Told{}, Now: func() time.Time { return now }})
defer k.Close(context.Background())
w := &watchdogs{keeper: k, started: now.Add(-time.Hour)}
failing := calm(now)
failing.doing = []inventory.Doing{unansweringPackageManager(now, 20*time.Minute, 1)}
w.see(t.Context(), failing)
open, err := k.Open(t.Context())
if err != nil || len(open) != 1 || open[0].Kind != kindApplyFailing {
t.Fatalf("the failed re-apply raised %+v (%v), want apply-failing", open, err)
}
clean := calm(now.Add(5 * time.Minute))
clean.doing = nil // the clean re-apply recorded: NotDoingWhatTheyWereTold no longer lists the machine
w.see(t.Context(), clean)
if open, _ := k.Open(t.Context()); len(open) != 0 {
t.Fatalf("the condition stayed open after the clean re-apply: %+v", open)
}
}
+32 -2
View File
@@ -117,10 +117,13 @@ type asked struct {
// Proposal is a settings layer proposed through a verb (proposals.go, novox/hq ADR 0277): about no
// condition, set on Approve by the serving controller itself, and left alone by the reconciling of conditions.
Proposal *settingsProposal `json:"proposal,omitempty"`
// Take is a shared folder the operator is asked to let the mesh keep (kept_folders.go, novox/hq ADR 0325 §8):
// about no condition, taken on Approve by the serving controller itself, asked once per folder and machine.
Take *folderTake `json:"take,omitempty"`
}
// ofACondition says an ask is one of a condition's: not a rehearsal, not a proposal.
func (r asked) ofACondition() bool { return !r.Rehearsal && r.Proposal == nil }
func (r asked) ofACondition() bool { return !r.Rehearsal && r.Proposal == nil && r.Take == nil }
// partKey is an ask's place among what is asked: its condition and its part.
func partKey(condition, part string) string { return condition + "#" + part }
@@ -192,6 +195,11 @@ type asker struct {
call func(ctx context.Context, a conditions.Action, args map[string]string) error
// setLayer sets a proposed settings layer on the operator's warrant (novox/hq ADR 0277); nil cannot.
setLayer setOnWarrant
// folders is every shared folder waiting to be kept (novox/hq ADR 0325 §8), and take what takes one on the
// operator's warrant; nil asks about none. machines are the mesh's machine names, allowed in an ask's words.
folders func(ctx context.Context) ([]folderTake, error)
take takeOnWarrant
machines func(ctx context.Context) ([]string, error)
// record writes the hand-act log.
record func(ctx context.Context, act link.HandAct) error
// routerRecord reads the router's record of an ask for a warrant missed; nil reads nothing.
@@ -213,6 +221,8 @@ type asker struct {
saidNoRouter bool
saidNoGrant bool
// saidGrantCaveat is whether a grant held with a caveat was said, so it is said once.
saidGrantCaveat bool
mu sync.Mutex
nudged chan struct{}
@@ -317,6 +327,13 @@ func (a *asker) reconcile(ctx context.Context) error {
}
return a.sayUnasked(ctx, unasked, "the bus does not hold the controller's grant to ask yet: "+why)
}
if why != "" && !a.saidGrantCaveat {
// Held, with a caveat worth one line (an older bus module cannot confirm the reload).
a.logf("the controller asks: %s", why)
a.saidGrantCaveat = true
} else if why == "" {
a.saidGrantCaveat = false
}
a.saidNoGrant = false
}
channels := ""
@@ -545,6 +562,10 @@ func (a *asker) reconcile(ctx context.Context) error {
return err
}
}
// And every shared folder waiting to be kept, once per folder and machine (novox/hq ADR 0325 §8).
if err := a.askFolders(ctx); err != nil {
a.logf("the operator could not be asked about the folders waiting to be kept: %v", err)
}
why := "the router refused the ask"
if len(refusedWords) > 0 {
why += ": " + refusedWords[0]
@@ -809,7 +830,7 @@ func (a *asker) Decided(ctx context.Context, body []byte) error {
if err != nil {
return err
}
stillOpen := r.Rehearsal || r.Proposal != nil // a rehearsal and a proposal are about no condition
stillOpen := r.Rehearsal || r.Proposal != nil || r.Take != nil // a rehearsal, a proposal and a take are about no condition
for _, c := range open {
stillOpen = stillOpen || c.Key == r.Condition
}
@@ -867,6 +888,15 @@ func (a *asker) Decided(ctx context.Context, body []byte) error {
if acted == nil && outcome != "" && !strings.HasPrefix(outcome, "nothing") {
outcome = "done: " + outcome
}
case r.Take != nil && act.Verb == takeVerb:
// A shared folder taken on the operator's warrant (novox/hq ADR 0325 §8): the act is composed again from the
// record and held to what the option bound before the engine is asked.
outcome, acted = a.decideTake(ctx, *r, act, w)
if acted == nil && !strings.HasPrefix(outcome, "nothing") {
outcome = "done: " + outcome
}
case r.Take != nil:
acted = fmt.Errorf("the take %s offers no act %s", r.ID, act.Verb)
case act.Arguments["silence"] != "":
acted = a.silence(ctx, act.Arguments["silence"], conditions.MaxSilence, byWords(w), why)
default:
+205
View File
@@ -12,6 +12,7 @@ import (
"git.novox.be/novox/mesh-sdk/go/asks"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/link"
)
@@ -808,3 +809,207 @@ func TestARefusalIsAskedAgainAndTheUndeliveredConditionClearsOnceTaken(t *testin
t.Errorf("the explanation says its verdict twice: %q", e)
}
}
// hq issue 353, the review: the bus holds the controller's grant to ask only when both facts hold — the mesh
// composes the controller's own grant, and the bus's module says it is in force on the running server — and
// each fact not known is not held, said with why.
func TestTheGrantIsHeldOnlyWhenComposedAndInForceOnTheBus(t *testing.T) {
yes := func(context.Context) (bool, string, error) { return true, "", nil }
no := func(why string) grantJudge {
return func(context.Context) (bool, string, error) { return false, why, nil }
}
broken := func(context.Context) (bool, string, error) { return false, "", errors.New("the store is away") }
for _, c := range []struct {
name string
composed, inForce grantJudge
held bool
says string
}{
{"composed and in force", yes, yes, true, ""},
{"not composed", no("no router is assigned"), yes, false, "no router is assigned"},
{"composed, written, not reloaded", yes, no("the server has not reloaded"), false, "has not reloaded"},
{"the composition unknown", broken, yes, false, "could not be worked out: the store is away"},
{"the bus unknown", yes, broken, false, "could not be worked out: the store is away"},
{"nothing judges the bus", yes, nil, false, "not judged"},
} {
held, why, err := grantHeldBy(c.composed, c.inForce, time.Now)(context.Background())
if err != nil || held != c.held || !strings.Contains(why, c.says) {
t.Errorf("%s: held %v (%q, %v)", c.name, held, why, err)
}
}
// Remembered for grantLookEvery, then judged again.
now := time.Date(2026, 10, 9, 18, 0, 0, 0, time.UTC)
inForce := false
judge := grantHeldBy(yes, func(context.Context) (bool, string, error) { return inForce, "not yet", nil },
func() time.Time { return now })
if held, _, _ := judge(context.Background()); held {
t.Fatal("held before the bus enforces it")
}
inForce = true
if held, _, _ := judge(context.Background()); held {
t.Error("judged again inside grantLookEvery")
}
now = now.Add(grantLookEvery)
if held, _, _ := judge(context.Background()); !held {
t.Error("not judged again after grantLookEvery")
}
}
// The controller's own grant, as composed: present while a router that takes asks is assigned, absent
// otherwise — whatever else in the user list changes.
func TestTheControllersOwnGrantToAskIsReadFromTheComposition(t *testing.T) {
router := broker.Declared{Module: "messenger", Holds: []broker.Seat{{Name: broker.AsksSeat, Scope: "mesh",
Accepts: []string{"ask", "cancel"}, Emits: []string{"decided"}, ByCaller: []string{"ask", "cancel", "decided"}}}}
other := broker.Declared{Module: "dunst"}
with := broker.Records{Nodes: []string{"anchor"}, Assigned: map[string][]broker.Declared{"anchor": {router, other}}}
without := broker.Records{Nodes: []string{"anchor"}, Assigned: map[string][]broker.Declared{"anchor": {other}}}
if may, err := controllerMayAsk(with); err != nil || !may {
t.Errorf("with a router assigned: %v %v", may, err)
}
if may, err := controllerMayAsk(without); err != nil || may {
t.Errorf("with no router assigned: %v %v", may, err)
}
}
// The bus module's answer, with the record's word on the list sent as the check only where the answer does
// not settle it (reviews 22 and 31 of PR 188); each way it is not held names its own remedy.
func TestTheBusModulesAnswerNamesTheRemedyForEachCase(t *testing.T) {
const (
reloaded = `{"allowed":true,"in_force":true,"file_written":"2026-10-09T15:55:56Z","server_loaded":"2026-10-09T15:56:05Z"}`
notLoaded = `{"allowed":true,"in_force":false,"why":"the user list on the machine allows it, and the server has not reloaded since it was written","file_written":"2026-10-09T15:55:56Z","server_loaded":"2026-10-09T15:50:00Z"}`
noVarz = `{"allowed":true,"in_force":false,"why":"the server's last configuration load could not be read: refused"}`
noStat = `{"allowed":true,"in_force":false,"why":"when the bus's user list was written could not be read inside mesh-broker-nats: exit 1"}`
notAllowed = `{"allowed":false,"in_force":false,"why":"the user list on the machine does not allow it"}`
)
current := busListState{Machine: "anchor"}
behind := busListState{Machine: "anchor", Behind: true}
stepped := busListState{Machine: "anchor", Behind: true, BusStep: "sending anchor would replace the bus; `bus upgrade --why …`"}
uncomposed := busListState{Machine: "anchor", Behind: true, Uncomposed: errors.New("module foo requires a provision nobody provides")}
for _, c := range []struct {
name, answer string
askErr error
list busListState
listErr error
held bool
says, not string
}{
{"in force", reloaded, nil, behind, nil, true, "", "push"},
{"written, not reloaded", notLoaded, nil, current, nil, false, "within about 5 s", "push"},
{"in force not known: varz", noVarz, nil, current, nil, false, "is not known: the server's last configuration load", "wait"},
{"in force not known: stat", noStat, nil, current, nil, false, "is not known: when the bus's user list was written", "wait"},
{"not allowed, list behind", notAllowed, nil, behind, nil, false, "`push anchor` carries it", "5 s"},
{"not allowed, bus step pending", notAllowed, nil, stepped, nil, false, "`bus upgrade", "`push anchor`"},
{"not allowed, list uncomposed", notAllowed, nil, uncomposed, nil, false, "`plan anchor`", "`push anchor`"},
{"older module, allowed, list current", `{"allowed":true}`, nil, current, nil, true, "not confirmed", "push"},
{"older module, allowed, list behind", `{"allowed":true}`, nil, behind, nil, true, "not confirmed", "push"},
{"older module, allowed, list unknown", `{"allowed":true}`, nil, current, errors.New("the store is away"), true, "`bus upgrade`", "push"},
{"older module, not allowed, list behind", `{"allowed":false}`, nil, behind, nil, false, "`push anchor` carries it", "5 s"},
{"module cannot answer, list current", "", errors.New("no responders"), current, nil, true, "could not answer", "push"},
{"module cannot answer, list behind", "", errors.New("no responders"), behind, nil, false, "`push anchor` carries it", ""},
{"module cannot answer, list unknown", "", errors.New("no responders"), current, errors.New("the store is away"), false, "could not be worked out", ""},
{"answer unreadable, list behind", `not json`, nil, behind, nil, false, "could not be read", ""},
} {
judge := busGrantBy(
func(context.Context) (string, json.RawMessage, error) {
return "anchor", json.RawMessage(c.answer), c.askErr
},
func(context.Context) (busListState, error) { return c.list, c.listErr })
held, why, _ := judgeGrantOnce(judge)
if held != c.held || !strings.Contains(why, c.says) || (c.not != "" && strings.Contains(why, c.not)) {
t.Errorf("%s: held %v (%q)", c.name, held, why)
}
}
none := busGrantBy(func(context.Context) (string, json.RawMessage, error) { return "", nil, nil }, nil)
if held, why, _ := judgeGrantOnce(none); held || !strings.Contains(why, "no module holds the bus") {
t.Errorf("no bus: held %v (%q)", held, why)
}
}
// judgeGrantOnce is the whole judgement with the composed grant held, as the asker makes it.
func judgeGrantOnce(bus grantJudge) (bool, string, error) {
yes := func(context.Context) (bool, string, error) { return true, "", nil }
return grantHeldBy(yes, bus, time.Now)(context.Background())
}
// The orders reviews 22 and 31 of PR 188 found: this controller live before the bus module that answers
// in_force, with the bus's machine behind on its list — and, after mesh-catalog #166 merged, unpushable until a
// person's bus upgrade. An older module that allows the ask asks (once, with the caveat said); a new module that
// has not reloaded names the wait, not a push; one that cannot say whether it reloaded names what is not known.
func TestAnOlderBusModuleNeitherSilencesTheAsksNorNamesTheWrongRemedy(t *testing.T) {
for _, c := range []struct {
name, answer string
list busListState
asks bool
says, not string
}{
{"older module, list current", `{"allowed":true}`, busListState{Machine: "anchor"}, true, "not confirmed", ""},
{"older module, list behind, bus step pending", `{"allowed":true}`,
busListState{Machine: "anchor", Behind: true, BusStep: "a planned step"}, true, "not confirmed", ""},
{"new module, not reloaded",
`{"allowed":true,"in_force":false,"why":"not reloaded","file_written":"2026-10-09T15:55:56Z","server_loaded":"2026-10-09T15:50:00Z"}`,
busListState{Machine: "anchor"}, false, "within about 5 s", "push"},
{"new module, reload not known", `{"allowed":true,"in_force":false,"why":"the server's last configuration load could not be read: refused"}`,
busListState{Machine: "anchor"}, false, "is not known", "wait"},
} {
t.Run(c.name, func(t *testing.T) {
r := newAskerRig(t)
var said []string
r.a.logf = func(f string, a ...any) { said = append(said, fmt.Sprintf(f, a...)) }
var raised [][]conditions.Observation
r.a.raise = func(_ context.Context, obs []conditions.Observation) error {
raised = append(raised, obs)
return nil
}
yes := func(context.Context) (bool, string, error) { return true, "", nil }
r.a.grantHeld = grantHeldBy(yes, busGrantBy(
func(context.Context) (string, json.RawMessage, error) {
return "anchor", json.RawMessage(c.answer), nil
},
func(context.Context) (busListState, error) { return c.list, nil }),
func() time.Time { return r.now })
r.open = []conditions.Condition{heldCondition()}
_ = r.a.reconcile(context.Background())
_ = r.a.reconcile(context.Background())
sent := r.asksSent(t)
if c.asks {
if len(sent) != 1 || sent[0].About != heldCondition().Key {
t.Fatalf("not asked: %+v", sent)
}
n := 0
for _, s := range said {
if strings.Contains(s, c.says) {
n++
}
}
if n != 1 {
t.Errorf("the caveat said %d times: %q", n, said)
}
return
}
if len(sent) != 0 {
t.Fatalf("asked while the bus does not hold the grant: %+v", sent)
}
if len(raised) == 0 || len(raised[len(raised)-1]) != 1 ||
raised[len(raised)-1][0].Kind != "asks-undelivered" {
t.Fatalf("not said as undelivered: %+v", raised)
}
summary := raised[len(raised)-1][0].Summary
if !strings.Contains(summary, c.says) || (c.not != "" && strings.Contains(summary, c.not)) {
t.Errorf("the condition names the wrong remedy: %q", summary)
}
})
}
}
// The serving asker is wired to the real judgement: with records and a bus it cannot read, the grant is not
// held, and so nothing is asked (a stub saying yes, or no judgement at all, would ask).
func TestTheServingAskerJudgesTheGrantForReal(t *testing.T) {
a := servingAsker(&stores{}, nil, nil, nil)
if a.grantHeld == nil {
t.Fatal("the serving asker does not judge the bus's grant")
}
held, why, err := a.grantHeld(context.Background())
if err != nil || held || !strings.Contains(why, "could not be worked out") {
t.Errorf("held %v (%q, %v) with nothing to read", held, why, err)
}
}
+292 -21
View File
@@ -235,12 +235,27 @@ func routerHereIn(inv *inventory.Inventory) func(ctx context.Context) (bool, err
}
}
// grantHeldIn says whether the bus holds the controller's grant to ask (novox/hq issue 353): the user list the
// machine holding the bus was last sent is the one the mesh composes now (brokerBehind, the same judgement a
// push makes to send that machine first). While it is behind, the controller's ask is refused by the bus,
// whatever the record says of the router, so nothing is asked and the operator is told to push that machine.
// Judged at most every grantLookEvery: composing the list resolves the bus's machine whole.
func grantHeldIn(open *stores) func(ctx context.Context) (bool, string, error) {
// grantHeldIn says whether the bus holds the controller's grant to ask (novox/hq issue 353), judged on the two
// facts that make it so, and failing closed on each:
//
// 1. **composed**: the user list the mesh composes now grants the controller itself to publish its ask
// (controllerMayAsk) — the controller's own grant, not whether any user of the list changed;
// 2. **in force**: the bus's own module, on the machine holding the bus, says the user list there allows it
// and the server reloaded after that list was written (the nats module's `nats_user_can`, `in_force`) — a
// list sent is not a list the bus enforces until it reloads.
//
// Anything that cannot be worked out is not held, with why, so the conditions that need the operator are said
// as undelivered rather than asked into a refusal.
func grantHeldIn(open *stores, conn *nats.Conn) func(ctx context.Context) (bool, string, error) {
return grantHeldBy(composedGrantIn(open), busGrantIn(open, conn), time.Now)
}
// grantJudge is one of the two facts: whether it holds, and why not.
type grantJudge func(ctx context.Context) (bool, string, error)
// grantHeldBy is the judgement over the two facts, remembered for grantLookEvery: composing the list resolves
// the bus's machine whole, and asking the bus's module is a call.
func grantHeldBy(composed, inForce grantJudge, now func() time.Time) func(ctx context.Context) (bool, string, error) {
var mu sync.Mutex
var at time.Time
var held bool
@@ -248,22 +263,251 @@ func grantHeldIn(open *stores) func(ctx context.Context) (bool, string, error) {
return func(ctx context.Context) (bool, string, error) {
mu.Lock()
defer mu.Unlock()
if !at.IsZero() && time.Since(at) < grantLookEvery {
if !at.IsZero() && now().Sub(at) < grantLookEvery {
return held, why, nil
}
machine, behind, err := brokerBehind(ctx, open, nil)
held, why = judgeGrant(ctx, composed, inForce)
at = now()
return held, why, nil
}
}
// judgeGrant is held only when both facts hold; an error is a fact not known, never a yes. A fact that holds
// with a caveat (a reload not confirmed) gives that caveat as the why of a yes.
func judgeGrant(ctx context.Context, composed, inForce grantJudge) (bool, string) {
var caveat []string
for _, fact := range []struct {
judge grantJudge
what string
}{{composed, "whether the mesh composes the controller a grant to ask"}, {inForce, "whether the bus enforces it"}} {
if fact.judge == nil {
return false, fact.what + " is not judged here"
}
ok, why, err := fact.judge(ctx)
if err != nil {
return false, fact.what + " could not be worked out: " + err.Error()
}
if !ok {
return false, why
}
if why != "" {
caveat = append(caveat, why)
}
}
return true, strings.Join(caveat, "; ")
}
// controllerMayAsk says whether the controller's own grant, as composed from these records, lets it publish its
// ask (ADR 0259 §3: composed while a router that takes asks under its asker's name is assigned).
func controllerMayAsk(records broker.Records) (bool, error) {
users, err := broker.Users(records)
if err != nil {
return false, err
}
for _, u := range users {
if u.Kind != broker.KindController {
continue
}
perms, err := broker.PermissionsFor(u)
if err != nil {
return false, err
}
return broker.MayPublish(perms, asks.AskSubject(askerName)), nil
}
return false, errors.New("the composition holds no controller")
}
// composedGrantIn is the first fact, from the mesh's records.
func composedGrantIn(open *stores) grantJudge {
return func(ctx context.Context) (bool, string, error) {
if open == nil || open.inventory == nil {
return false, "", errors.New("the mesh's records are not open")
}
records, err := open.inventory.BusRecords(ctx)
if err != nil {
return false, "", err
}
at, held, why = time.Now(), !behind, ""
if behind {
why = fmt.Sprintf("the bus's user list on %s is behind what the mesh composes, so the bus has not been "+
"given the controller's grant to ask; `push %s` carries it", machine, machine)
may, err := controllerMayAsk(records)
if err != nil || may {
return may, "", err
}
return held, why, nil
return false, "the mesh composes the controller no grant to ask: no router that takes asks under its " +
"asker's name is assigned", nil
}
}
// busGrantIn is the second fact, asked of the bus's own module on the machine holding the bus, with the check
// main made before it (novox/hq issue 353, PR 186: is the user list sent to the bus's machine the one composed
// now) kept for the cases the module's word does not settle.
func busGrantIn(open *stores, conn *nats.Conn) grantJudge {
ask := func(ctx context.Context) (string, json.RawMessage, error) {
if open == nil || open.inventory == nil || conn == nil {
return "", nil, errors.New("the mesh's records or the bus are not open")
}
holders, err := seatHolders(ctx, open.inventory)
if err != nil {
return "", nil, err
}
h, held := holders[theBrokerSeat]
if !held || h.Node == "" || h.Module == "" {
return "", nil, nil
}
a, err := link.AskModuleToolOn(ctx, conn, h.Module, busUserCanTool, h.Node, map[string]any{
"user": broker.ControllerName, "action": "publish", "subject": asks.AskSubject(askerName)}, 15*time.Second)
if err != nil {
return h.Node, nil, err
}
if a.Error != "" {
return h.Node, nil, errors.New(a.Error)
}
return h.Node, a.Result, nil
}
list := func(ctx context.Context) (busListState, error) {
if open == nil || open.inventory == nil {
return busListState{}, errors.New("the mesh's records are not open")
}
s, err := brokerListState(ctx, open, nil)
if err != nil || s.Machine == "" || !s.Behind {
return s, err
}
step, err := busHeld(ctx, open.inventory, []string{s.Machine})
if err != nil {
return busListState{}, err
}
s.BusStep = step[s.Machine]
return s, nil
}
return busGrantBy(ask, list)
}
// busListState is the user list on the bus's machine as the controller's record knows it: whether the list
// sent there is behind the one composed now, why it could not be composed, and the bus step a send there waits
// on (that machine cannot be pushed until a person's `bus upgrade`).
type busListState struct {
Machine string
Behind bool
Uncomposed error
BusStep string
}
// remedy is the one thing that sends the bus's machine the list composed now: the plan's own reason when it
// cannot be composed, the bus upgrade while sends there wait on it, and otherwise a push of that machine.
func (s busListState) remedy(node string) string {
if s.Machine != "" {
node = s.Machine
}
switch {
case s.Uncomposed != nil:
return fmt.Sprintf("the user list for %s cannot be composed (%v); `plan %s` says what to fix first",
node, s.Uncomposed, node)
case s.BusStep != "":
return fmt.Sprintf("sends to %s wait on the bus's planned step, so a push will not carry it: %s", node, s.BusStep)
}
return fmt.Sprintf("`push %s` carries it", node)
}
// busGrantBy judges the bus's word, and names for each way it is not held the one thing that makes it held:
//
// - **in force**: held;
// - **allowed, from a module older than `in_force`**: held, with the caveat that the reload is not confirmed.
// The file allows the publish, and the bus reloads within about 5 s of a change to it, so the worst case is
// one ask refused in those seconds, which is asked again (issue 369). The list check is not consulted:
// while a bus step is pending the bus's machine cannot be pushed, so any change to the composed list would
// read as behind for good and silence every ask until a person's bus upgrade (the review of PR 188);
// - **allowed, written, not reloaded** (the module gives both times): wait about 5 s;
// - **allowed, in force not known** (the module could not read a time): what is not known, said;
// - **not allowed, or the module cannot answer**: the list check decides — the remedy that sends the list
// when it is behind (busListState.remedy), and, for a module that cannot answer, held while the list sent
// is current, as main judged it.
//
// ask answers the machine holding the bus and its module's answer ("" when nothing holds the bus); list is the
// record's word on the list sent to that machine.
func busGrantBy(ask func(ctx context.Context) (string, json.RawMessage, error),
list func(ctx context.Context) (busListState, error)) grantJudge {
return func(ctx context.Context) (bool, string, error) {
node, raw, askErr := ask(ctx)
if askErr == nil && node == "" {
return false, "no module holds the bus, so nothing says what it enforces", nil
}
var answer busAnswer
if askErr == nil {
if answer, askErr = readBusAnswer(raw); askErr != nil {
askErr = fmt.Errorf("its answer could not be read: %w", askErr)
}
}
if askErr != nil {
if node == "" {
node = "the bus's machine"
}
s, err := list(ctx)
if err != nil {
return false, "", fmt.Errorf("the bus module on %s could not answer (%v), and whether the list sent "+
"there is the one composed now could not be worked out: %w", node, askErr, err)
}
if s.Behind {
return false, fmt.Sprintf("the bus module on %s could not answer (%v), and the user list sent there is "+
"behind what the mesh composes; %s", node, askErr, s.remedy(node)), nil
}
return true, fmt.Sprintf("the bus module on %s could not answer (%v); judged by the user list sent there, "+
"which is the one composed now", node, askErr), nil
}
notAllowed := func(older string) (bool, string, error) {
s, err := list(ctx)
if err != nil {
return false, "", fmt.Errorf("the user list on %s does not allow the controller to ask%s, and whether "+
"the list sent there is the one composed now could not be worked out: %w", node, older, err)
}
if s.Behind {
return false, fmt.Sprintf("the user list on %s does not allow the controller to ask%s: it is behind "+
"what the mesh composes; %s", node, older, s.remedy(node)), nil
}
return false, fmt.Sprintf("the user list on %s does not allow the controller to ask%s, though the list "+
"sent there is the one composed now; %s again", node, older, s.remedy(node)), nil
}
switch {
case answer.InForce == nil && answer.Allowed:
return true, fmt.Sprintf("the user list on %s allows the controller to ask, but its reload is not "+
"confirmed: the bus module there is older than the answer whether a grant is in force, and the bus "+
"upgrade (`bus upgrade`) carries one that confirms it", node), nil
case answer.InForce == nil:
return notAllowed(" (the bus module there is older than the answer whether a grant is in force)")
case !answer.Allowed:
return notAllowed("")
case *answer.InForce:
return true, "", nil
case answer.FileWritten != "" && answer.ServerLoaded != "":
return false, fmt.Sprintf("the bus on %s has the controller's grant to ask written (%s) but has not "+
"reloaded since (last load %s); it reloads in place within about 5 s, so nothing is to be done but wait",
node, answer.FileWritten, answer.ServerLoaded), nil
}
why := answer.Why
if why == "" {
why = "it gives no reason"
}
return false, fmt.Sprintf("the user list on %s allows the controller to ask, but whether the bus enforces it "+
"is not known: %s", node, why), nil
}
}
// busUserCanTool is the bus module's answer to whether a user may do something, and whether it is in force.
const busUserCanTool = "nats_user_can"
// busAnswer is the bus module's answer to busUserCanTool; InForce is nil from a module older than that field,
// and FileWritten and ServerLoaded are given only when it could read both times (mesh-catalog's nats InForce).
type busAnswer struct {
Allowed bool `json:"allowed"`
InForce *bool `json:"in_force"`
Why string `json:"why"`
FileWritten string `json:"file_written"`
ServerLoaded string `json:"server_loaded"`
}
func readBusAnswer(raw json.RawMessage) (busAnswer, error) {
var answer busAnswer
err := json.Unmarshal(raw, &answer)
return answer, err
}
// grantLookEvery is how often the bus's user list is judged against the one its machine was last sent.
const grantLookEvery = 30 * time.Second
@@ -302,7 +546,19 @@ func startAsking(ctx context.Context, open *stores, server *link.Server, conn *n
fmt.Printf("the operator cannot be asked: %v\n", err)
return
}
a := &asker{
a := servingAsker(open, conn, js, keeper)
if err := server.Decides(a); err != nil {
fmt.Printf("the operator's answers cannot be heard, so nothing is asked: %v\n", err)
return
}
askerFrom = a
go a.keep(ctx)
}
// servingAsker is the serving controller's asker, wired to the mesh: what it reads, publishes, performs and
// records, and the judgements it asks by — the bus's grant to ask among them (novox/hq issue 353).
func servingAsker(open *stores, conn *nats.Conn, js jetstream.JetStream, keeper *conditions.Keeper) *asker {
return &asker{
open: keeper.Open,
silence: func(ctx context.Context, key string, d time.Duration, by, why string) error {
_, err := keeper.Silence(ctx, key, d, by, why)
@@ -316,13 +572,34 @@ func startAsking(ctx context.Context, open *stores, server *link.Server, conn *n
call: callAction(conn),
// A proposed settings layer is set by this controller itself on the warrant (novox/hq ADR 0277).
setLayer: setLayerIn(open),
// A shared folder waiting to be kept is asked about once, and taken on the warrant (novox/hq ADR 0325 §8).
folders: foldersIn(open),
take: takeFolderIn(open, func(ctx context.Context, act link.AccessActAsk) (link.HandOverAnswer, error) {
ident, err := open.Identity(ctx)
if err != nil {
return link.HandOverAnswer{}, fmt.Errorf("the mesh's signing key cannot be read, so nothing was asked of "+
"%s: %w", act.Node, err)
}
return link.AskTake(ctx, conn, ident, act, link.TakeWithin)
}),
machines: func(ctx context.Context) ([]string, error) {
nodes, err := open.inventory.Nodes(ctx)
if err != nil {
return nil, err
}
names := make([]string, 0, len(nodes))
for _, n := range nodes {
names = append(names, n.Name)
}
return names, nil
},
record: func(ctx context.Context, act link.HandAct) error {
_, err := link.RecordHandAct(ctx, conn, act)
return err
},
routerRecord: routerRecordOf(conn, open.inventory),
routerHere: routerHereIn(open.inventory),
grantHeld: grantHeldIn(open),
grantHeld: grantHeldIn(open, conn),
channels: channelsIn(open.inventory),
raise: func(ctx context.Context, obs []conditions.Observation) error {
return keeper.Reconcile(ctx, sourceAsker, obs)
@@ -330,10 +607,4 @@ func startAsking(ctx context.Context, open *stores, server *link.Server, conn *n
now: time.Now,
logf: func(format string, args ...any) { fmt.Printf(format+"\n", args...) },
}
if err := server.Decides(a); err != nil {
fmt.Printf("the operator's answers cannot be heard, so nothing is asked: %v\n", err)
return
}
askerFrom = a
go a.keep(ctx)
}
+232 -17
View File
@@ -37,6 +37,11 @@ import (
// own commit, the newest first, one walk each, until one is delivered; the older ones are then answered
// by that one.
//
// **A merge waits only for a batch it shares something with** (novox/hq ADR 0304 decision 1, ADR 0306 decisions
// 2 and 3, issue 525): it joins a batch it is related to, and one that is related to none and whose delivery
// group has no member still to merge is a batch whose window is closed at once (batchToJoin). Walks stay one at
// a time: such a batch is cut when no walk is open, and queued behind the one that is otherwise.
//
// A merge on the controller's own path (a module whose walk waits for nobody's word) never shares a batch with
// one that waits for mesh-delivery's: each kind has a batch of its own, an own-path batch is cut first and folds
// no waiting catalogue walk, and a catalogue walk let go while another walk runs starts once it ended — so no
@@ -173,6 +178,12 @@ func (f following) hearMerge(ctx context.Context, m link.SourceMoved, now time.T
fmt.Printf("%s merged into %s (%.8s); nothing the mesh holds reads it\n", repository, m.Base, m.Commit)
return nil
}
// **Its delivery group's word, before the plans are held** (ADR 0306 decision 2): a slow answer holds up no walk.
own := ownPath(touches)
waits, err := groupWait(ctx, inv, m, deliverySeatHeld(entries), own)
if err != nil {
return notNow(err)
}
release, err := inv.HoldPlans(ctx, true)
if err != nil {
return notNow(err)
@@ -198,7 +209,6 @@ func (f following) hearMerge(ctx context.Context, m link.SourceMoved, now time.T
// **A merge on the controller's own path never shares a batch with one that waits for the delivery's
// word** (decided during the build, 2026-10-10): batched together, the catalogue's deliveries would start
// with the controller's and skip their turn. Each kind has a batch of its own.
own := ownPath(touches)
if p, ok, err := answeredByALaterMerge(ctx, inv, heard, touches, own, deliverySeatHeld(entries)); err != nil {
return notNow(err)
} else if ok {
@@ -218,7 +228,7 @@ func (f following) hearMerge(ctx context.Context, m link.SourceMoved, now time.T
return cutBatchesHeld(ctx, f.open, now)
}
batch, err := theOpenBatch(ctx, inv, heard, now, own)
batch, why, err := batchToJoin(ctx, inv, heard, entries, read, now, own, waits)
if err != nil {
return notNow(err)
}
@@ -229,7 +239,7 @@ func (f following) hearMerge(ctx context.Context, m link.SourceMoved, now time.T
if err := keepBatch(ctx, inv, &batch, now, ""); err != nil {
return notNow(err)
}
fmt.Printf("%s merged into %s (%.8s); it joins %s — %s\n", repository, m.Base, m.Commit, batch.ID,
fmt.Printf("%s merged into %s (%.8s); %s, it joins %s — %s\n", repository, m.Base, m.Commit, why, batch.ID,
batchWords(batch, now))
return cutBatchesHeld(ctx, f.open, now)
}
@@ -320,22 +330,211 @@ func ownPath(touches []inventory.Entry) bool {
return false
}
// theOpenBatch is the batch a merge heard now joins: the one of its kind not yet cut, or a new one.
func theOpenBatch(ctx context.Context, inv *inventory.Inventory, first inventory.BatchedMerge, now time.Time, own bool) (inventory.Plan, error) {
// **A merge waits only for a batch it shares something with** (novox/hq ADR 0304 decision 1, as ADR 0306
// decisions 2 and 3 read it). batchToJoin is the batch a merge heard now joins, and why in words:
//
// - an assembling or queued batch of its kind it is related to (related.go): it joins it and waits for the
// window, which its coming restarts. Related to two or more, the newer are folded into the oldest, which
// answers all their merges: a merge never waits in two windows, and what it shares with each walks in one.
// - none, and its delivery group has a member still to merge, or mesh-delivery could not say (groupWait, waits
// names why): a new batch, which waits for the window, beside the ones it shares nothing with; its words say
// why.
// - none, and no member of its group is still to merge: a new batch whose window is closed at once. Cut at
// once means only that: walks stay one at a time (ADR 0276 decision 4), so with a walk open the batch is
// queued behind it as any closed batch is.
func batchToJoin(ctx context.Context, inv *inventory.Inventory, heard inventory.BatchedMerge, entries []inventory.Entry,
read map[string][]inventory.ReadRepository, now time.Time, own bool, waits string) (inventory.Plan, string, error) {
edges, err := inv.Dependencies(ctx)
if err != nil {
return inventory.Plan{}, "", err
}
joins, whys, _, err := relatedBatches(ctx, inv, []inventory.BatchedMerge{heard}, own, "", entries, read, edges)
if err != nil {
return inventory.Plan{}, "", err
}
if len(joins) > 0 {
into := joins[0]
why := "related to " + into.ID + " (" + whys[0] + ")"
if len(joins) > 1 {
plans, moves, err := foldInto(ctx, inv, &into, joins[1:])
if err != nil {
return inventory.Plan{}, "", err
}
if err := inv.SavePlansMoving(ctx, plans, moves); err != nil {
return inventory.Plan{}, "", err
}
var folded []string
for i, b := range joins[1:] {
folded = append(folded, b.ID+" ("+whys[i+1]+")")
}
why += ", and to " + strings.Join(folded, ", ") + ", folded into it"
}
into.Delivery.Batch.CutAtOnce = false
return into, why, nil
}
if waits != "" {
// Its own window, beside the batches it shares nothing with: waiting is for its group's members, which
// are related to it by their head and join it (ADR 0306 decision 2).
b, err := newBatch(ctx, inv, heard, now, own, false, waits)
return b, waits + ", it waits for the window in a batch of its own", err
}
b, err := newBatch(ctx, inv, heard, now, own, true, "")
return b, "it shares nothing with a batch assembling and no member of a group is still to merge: it is cut at once", err
}
// relatedBatches are the batches not yet cut of a kind, oldest first, parted into those related to merges (with
// the first reason of each) and the others. Each batch is judged by what its cut would move (ADR 0306 decision
// 3): its merges' moves as they were kept and as they read now — the catalogue may have moved since a merge was
// heard — widened to their dependents, and, for a catalogue batch, every walk waiting for its word, which the
// cut folds in (cutBatch's carry). A merge kept with no moves leaves its batch unknown, related to everything.
// except is a walk not to count as folded: the one being stopped.
func relatedBatches(ctx context.Context, inv *inventory.Inventory, merges []inventory.BatchedMerge, own bool,
except string, entries []inventory.Entry, read map[string][]inventory.ReadRepository,
edges []inventory.Edge) (joins []inventory.Plan, whys []string, others []inventory.Plan, err error) {
batches, err := inv.Batches(ctx)
if err != nil {
return inventory.Plan{}, err
return nil, nil, nil, err
}
var kind []inventory.Plan
for _, b := range batches {
if b.OwnPath() == own {
return b, nil
kind = append(kind, b)
}
}
if len(kind) == 0 {
return nil, nil, nil, nil
}
carry := newFootprint()
if !own {
// An own-path batch folds no waiting walk (cutBatchesHeld); a catalogue batch may fold every one.
plans, err := inv.OpenPlans(ctx)
if err != nil {
return nil, nil, nil, err
}
for _, p := range plans {
if p.Release != nil || !p.Waiting() || p.ID == except {
continue
}
kept, err := inv.MergesOf(ctx, p.ID)
if err != nil {
return nil, nil, nil, err
}
carry = unionFootprint(carry, footprintOfWalk(p, kept))
}
}
mine := footprintAsCut(merges, entries, read, edges)
for _, b := range kind {
held, err := inv.MergesOf(ctx, b.ID)
if err != nil {
return nil, nil, nil, err
}
if b.Delivery == nil {
b.Delivery = &inventory.PlanDelivery{}
}
if b.Delivery.Batch == nil {
b.Delivery.Batch = &inventory.PlanBatch{Own: own}
}
if ok, why := related(mine, unionFootprint(footprintAsCut(held, entries, read, edges), carry), edges); ok {
joins, whys = append(joins, b), append(whys, why)
} else {
others = append(others, b)
}
}
return joins, whys, others, nil
}
// footprintAsCut is a batch's footprint as a cut now would read it: footprintOfBatch, widened by the modules each
// merge moves as the catalogue reads now (planBatch judges each merge again at the cut, so what it moves may
// have changed since it was heard). Both are kept: when in doubt, related.
func footprintAsCut(merges []inventory.BatchedMerge, entries []inventory.Entry,
read map[string][]inventory.ReadRepository, edges []inventory.Edge) footprint {
f := footprintOfBatch(merges, edges)
var moved []string
for _, m := range merges {
s := announcedOf(m).SourceMoved
if s.Owner == "" {
continue
}
s.MergedAt = "" // news, whatever a cut marked seen since (as planBatch reads it)
for _, e := range wouldMove(s, entries, read) {
moved = append(moved, e.Manifest.Module)
}
}
for _, name := range reachableFrom(moved, edges) {
f.modules[name] = true
}
return f
}
// unionFootprint is what two changes reach together.
func unionFootprint(a, b footprint) footprint {
f := newFootprint()
for _, from := range []footprint{a, b} {
for k := range from.repos {
f.repos[k] = true
}
for k := range from.modules {
f.modules[k] = true
}
for k := range from.heads {
f.heads[k] = true
}
}
f.unknown = a.unknown
if f.unknown == "" {
f.unknown = b.unknown
}
return f
}
// foldInto folds batches into the oldest one related to a merge: the plans to write (into first, so no merge
// names a plan the store does not hold) and the merges to move, for one SavePlansMoving. The folded batches name
// the batch that took them over; a stop's closed window and its withheld modules (ADR 0299) go with their merges.
func foldInto(ctx context.Context, inv *inventory.Inventory, into *inventory.Plan, from []inventory.Plan) ([]*inventory.Plan,
[]inventory.MergeMove, error) {
if into.Delivery == nil {
into.Delivery = &inventory.PlanDelivery{}
}
if into.Delivery.Batch == nil {
into.Delivery.Batch = &inventory.PlanBatch{Own: into.OwnPath()}
}
plans := []*inventory.Plan{into}
var moves []inventory.MergeMove
for i := range from {
f := from[i]
merges, err := inv.MergesOf(ctx, f.ID)
if err != nil {
return nil, nil, err
}
for _, m := range merges {
moves = append(moves, inventory.MergeMove{Repository: m.Repository, Commit: m.Commit, Plan: into.ID, Alone: m.Alone})
}
if f.Delivery == nil {
f.Delivery = &inventory.PlanDelivery{}
}
if w := f.Delivery.Batch; w != nil {
into.Delivery.Batch.Closed = into.Delivery.Batch.Closed || w.Closed
into.Delivery.Batch.Withheld = union(into.Delivery.Batch.Withheld, w.Withheld)
sort.Strings(into.Delivery.Batch.Withheld)
}
f.State = inventory.PlanSuperseded
f.Delivery.Batch = nil
f.Delivery.TakenOverBy = into.ID
f.Note = fmt.Sprintf("folded into %s before it was cut: a merge related to both joined it, and %s answers "+
"its merges", into.ID, into.ID)
plans = append(plans, &f)
}
return plans, moves, nil
}
// newBatch is a new batch of a kind, kept before its first merge names it: a merge never names a plan the store
// does not hold. cutAtOnce closes its window at once; waits says why it waits for its group, when it does.
func newBatch(ctx context.Context, inv *inventory.Inventory, first inventory.BatchedMerge, now time.Time, own,
cutAtOnce bool, waits string) (inventory.Plan, error) {
b := inventory.Plan{ID: fmt.Sprintf("plan-%d", now.UnixNano()), Repository: first.Repository, Branch: first.Branch,
Commit: first.Commit, Merged: first.Merged, Created: now, State: inventory.PlanAssembling,
Tiers: [][]string{}, Modules: map[string]*inventory.PlanModule{},
Delivery: &inventory.PlanDelivery{Batch: &inventory.PlanBatch{Own: own}}}
// Kept before its first merge names it: a merge never names a plan the store does not hold.
Delivery: &inventory.PlanDelivery{Batch: &inventory.PlanBatch{Own: own, CutAtOnce: cutAtOnce, Waits: waits}}}
if err := inv.SavePlan(ctx, &b); err != nil {
return inventory.Plan{}, err
}
@@ -367,13 +566,16 @@ func keepBatch(ctx context.Context, inv *inventory.Inventory, b *inventory.Plan,
}
closes, latest := windowOf(merges, b.Created, window, atMost)
b.Delivery.Merges = named
var closed bool
var closed, cutAtOnce bool
var withheld []string
var waits string
if was := b.Delivery.Batch; was != nil {
closed, withheld = was.Closed, was.Withheld // a stop's, which no look takes back (novox/hq ADR 0299)
cutAtOnce = was.CutAtOnce // the merge's, until another joins it (ADR 0304 decision 1)
waits = was.Waits // its group's, as said when it was made (ADR 0306 decision 2)
}
b.Delivery.Batch = &inventory.PlanBatch{ClosesAt: closes, AtMost: latest, Behind: behind, Own: b.OwnPath(),
Closed: closed, Withheld: withheld}
Closed: closed, Withheld: withheld, CutAtOnce: cutAtOnce, Waits: waits}
b.State = inventory.PlanAssembling
if behind != "" && windowClosed(*b, now) {
b.State = inventory.PlanQueued
@@ -402,13 +604,14 @@ func windowOf(merges []inventory.BatchedMerge, created time.Time, window, atMost
return last.Add(window), created.Add(atMost)
}
// windowClosed says a batch's window is closed: no merge for the window's length, or its maximum reached.
// windowClosed says a batch's window is closed: closed by a stop, closed at once for a merge that shares
// nothing (ADR 0304 decision 1), no merge for the window's length, or its maximum reached.
func windowClosed(b inventory.Plan, now time.Time) bool {
if b.Delivery == nil || b.Delivery.Batch == nil {
return true
}
w := b.Delivery.Batch
return w.Closed || !now.Before(w.ClosesAt) || !now.Before(w.AtMost)
return w.Closed || w.CutAtOnce || !now.Before(w.ClosesAt) || !now.Before(w.AtMost)
}
// carriedOf is a set of merges as a walk carries them: the latest merge of each repository's branch, and
@@ -617,8 +820,9 @@ func cutBatchesHeld(ctx context.Context, open *stores, now time.Time) error {
return err
}
// The oldest batch whose window closed is cut, an own-path one before a catalogue one (it goes first, and
// the catalogue batch queues behind it rather than being cut and overtaken); one of each kind at most is
// open (theOpenBatch).
// the catalogue batch queues behind it rather than being cut and overtaken). Several of a kind may be open
// since a merge that shares nothing is cut at once (ADR 0304 decision 1): walks stay one at a time, so one
// is cut per look, and once its walk started the others are queued behind it.
sort.SliceStable(batches, func(i, j int) bool { return batches[i].OwnPath() && !batches[j].OwnPath() })
for i := range batches {
b := &batches[i]
@@ -886,6 +1090,9 @@ func walkTimesAtCut(batch, walk inventory.Plan, entries []inventory.Entry, now t
if !w.AtMost.IsZero() && w.AtMost.Before(closed) {
closed = w.AtMost
}
if w.CutAtOnce && !batch.Created.IsZero() {
closed = batch.Created // closed when it was made (ADR 0304 decision 1)
}
if !closed.IsZero() {
if closed.After(now) {
closed = now
@@ -1142,6 +1349,10 @@ func batchWords(b inventory.Plan, now time.Time) string {
if b.State == inventory.PlanQueued {
return fmt.Sprintf("queued behind %s; grouped: %s; plan not yet calculated", w.Behind, grouped)
}
if w.CutAtOnce {
return fmt.Sprintf("cut at once, sharing nothing with a batch assembling; grouped: %s; plan not yet calculated",
grouped)
}
left := w.ClosesAt.Sub(now)
if w.AtMost.Before(w.ClosesAt) {
left = w.AtMost.Sub(now)
@@ -1149,8 +1360,12 @@ func batchWords(b inventory.Plan, now time.Time) string {
if left < 0 {
left = 0
}
return fmt.Sprintf("assembling: %s left (at the latest %s); grouped: %s; plan not yet calculated",
secondsWords(left), w.AtMost.Local().Format("15:04:05"), grouped)
waits := ""
if w.Waits != "" {
waits = " — " + w.Waits
}
return fmt.Sprintf("assembling: %s left (at the latest %s)%s; grouped: %s; plan not yet calculated",
secondsWords(left), w.AtMost.Local().Format("15:04:05"), waits, grouped)
}
// groupedWords is a batch's commits, each with the earlier merges it answers.
+77 -13
View File
@@ -34,6 +34,7 @@ func windowed(t *testing.T) *stores {
t.Fatal(err)
}
t.Setenv(mergeWindowFileVar, settings)
noGroups(t)
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: "mesh-controller", Version: "1"},
inventory.Source{Repository: "novox/mesh-controller", Seat: "git", Ref: "main", BuiltFrom: "c0",
Head: "c0"}); err != nil {
@@ -58,19 +59,70 @@ func windowed(t *testing.T) *stores {
// t0 is the moment a test's clock starts: merges are dated by it, and heard after it.
var t0 = time.Now().UTC().Truncate(time.Second)
// catalogueMerge is a merge of the catalogue changing one module's directory, made at a moment.
// catalogueMerge is a merge of the catalogue changing one module's directory, made at a moment, from a pull
// request's head branch of its own, as the forge announces every merge. Whether its head names a delivery group
// with a member still to merge is mesh-delivery's word, which windowed stubs as no group at all (noGroups); a test
// whose merges wait for the window as a group's says so (groupsWaiting).
func catalogueMerge(commit, module string, made time.Time) link.SourceMoved {
return link.SourceMoved{Owner: "novox", Repo: "mesh-catalog", Base: "main", Commit: commit,
return link.SourceMoved{Owner: "novox", Repo: "mesh-catalog", Base: "main", Head: "pr/" + commit, Commit: commit,
MergedAt: made.Format(time.RFC3339Nano), Paths: []string{"modules/" + module + "/module.json"},
ModuleDirs: []string{"modules/" + module}, ModuleDirsSaid: true}
}
// repoMerge is a merge of a repository of one module's own.
// repoMerge is a merge of a repository of one module's own, from a head branch of its own (catalogueMerge).
func repoMerge(repo, commit string, made time.Time) link.SourceMoved {
return link.SourceMoved{Owner: "novox", Repo: repo, Base: "main", Commit: commit,
return link.SourceMoved{Owner: "novox", Repo: repo, Base: "main", Head: "pr/" + commit, Commit: commit,
MergedAt: made.Format(time.RFC3339Nano), Paths: []string{"main.go"}}
}
// noGroups stubs mesh-delivery's word on groups for a test: no head names a group.
func noGroups(t *testing.T) {
t.Helper()
was := deliveryGroupOf
deliveryGroupOf = func(context.Context, bool, string) ([]groupMember, bool, error) { return nil, false, nil }
t.Cleanup(func() { deliveryGroupOf = was })
}
// groupsWaiting stubs mesh-delivery's word on groups for a test: each head named is a group with a member of another
// repository whose pull request is open (novox/elsewhere, ready), so a merge from it waits for the window (ADR 0306
// decision 2); every other head names no group.
func groupsWaiting(t *testing.T, heads ...string) {
t.Helper()
was := deliveryGroupOf
deliveryGroupOf = func(_ context.Context, _ bool, head string) ([]groupMember, bool, error) {
if slices.Contains(heads, head) {
return []groupMember{{Repository: "novox/elsewhere", State: "ready"}}, true, nil
}
return nil, false, nil
}
t.Cleanup(func() { deliveryGroupOf = was })
}
// everyHeadWaits stubs mesh-delivery's word on groups for a test whose merges each wait for the window as they did
// before novox/hq ADR 0304: every head is a group with a member of another repository still to merge.
func everyHeadWaits(t *testing.T) {
t.Helper()
was := deliveryGroupOf
deliveryGroupOf = func(context.Context, bool, string) ([]groupMember, bool, error) {
return []groupMember{{Repository: "novox/elsewhere", State: "ready"}}, true, nil
}
t.Cleanup(func() { deliveryGroupOf = was })
}
// fromHead is a merge from a head branch a test names: merges from one head are members of one delivery group,
// related by it (novox/hq ADR 0304 decision 1), so they share a batch whatever else they share.
func fromHead(m link.SourceMoved, head string) link.SourceMoved {
m.Head = head
return m
}
// lone is a merge that names no head: known to be no delivery group's, it waits for no window unless it is
// related to a batch assembling (ADR 0304 decision 1).
func lone(m link.SourceMoved) link.SourceMoved {
m.Head = ""
return m
}
// hear is a merge heard at a moment.
func hear(t *testing.T, open *stores, m link.SourceMoved, now time.Time) {
t.Helper()
@@ -108,6 +160,7 @@ func walks(t *testing.T, open *stores) (walks, batches []inventory.Plan) {
// later commit, which names both merges — the earlier carried by the later.
func TestTwoMergesSecondsApartAreOneWalkAtTheLaterCommit(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asked := asksWithPaths(t)
claude := catalogueMerge("553b7191claude", "app", t0)
dunst := catalogueMerge("48bda475dunst", "notes", t0.Add(17*time.Second))
@@ -152,9 +205,10 @@ func TestTwoMergesSecondsApartAreOneWalkAtTheLaterCommit(t *testing.T) {
// Merges of three repositories in one window are one walk, one commit for each.
func TestThreeRepositoriesInOneWindowAreOneWalk(t *testing.T) {
open := windowed(t)
groupsWaiting(t, "feat/three")
asksRecorded(t)
for i, r := range []string{"one", "two", "three"} {
hear(t, open, repoMerge(r, "c-"+r, t0.Add(time.Duration(i)*20*time.Second)),
hear(t, open, fromHead(repoMerge(r, "c-"+r, t0.Add(time.Duration(i)*20*time.Second)), "feat/three"),
t0.Add(time.Duration(i)*20*time.Second+time.Second))
}
cutAt(t, open, t0.Add(41*time.Second+90*time.Second))
@@ -187,9 +241,10 @@ func TestThreeRepositoriesInOneWindowAreOneWalk(t *testing.T) {
// closes at most ten minutes after its first merge however busy.
func TestAMergeRestartsTheWindowAndTheMaximumClosesIt(t *testing.T) {
open := windowed(t)
groupsWaiting(t, "feat/both", "feat/busy")
asksWithPaths(t)
hear(t, open, repoMerge("one", "c1", t0), t0)
hear(t, open, repoMerge("two", "c2", t0.Add(80*time.Second)), t0.Add(80*time.Second))
hear(t, open, fromHead(repoMerge("one", "c1", t0), "feat/both"), t0)
hear(t, open, fromHead(repoMerge("two", "c2", t0.Add(80*time.Second)), "feat/both"), t0.Add(80*time.Second))
cutAt(t, open, t0.Add(100*time.Second))
if ws, _ := walks(t, open); len(ws) != 0 {
t.Fatal("the window closed 90 seconds after its first merge, not after its last")
@@ -200,12 +255,13 @@ func TestAMergeRestartsTheWindowAndTheMaximumClosesIt(t *testing.T) {
}
busy := windowed(t)
groupsWaiting(t, "feat/both", "feat/busy")
asksWithPaths(t)
var last time.Time
for i := 0; i < 12; i++ {
last = t0.Add(time.Duration(i) * time.Minute)
repo := []string{"one", "two", "three"}[i%3]
hear(t, busy, repoMerge(repo, "c"+string(rune('a'+i)), last), last)
hear(t, busy, fromHead(repoMerge(repo, "c"+string(rune('a'+i)), last), "feat/busy"), last)
if ws, _ := walks(t, busy); len(ws) > 0 {
if i != 10 {
t.Fatalf("a busy window closed at merge %d (minute %d), not at its maximum", i, i)
@@ -223,10 +279,11 @@ func TestAMergeRestartsTheWindowAndTheMaximumClosesIt(t *testing.T) {
// what is grouped, and that the plan is not yet calculated. It says so on the bus as plan-moved, too.
func TestTheAssemblingBatchIsShown(t *testing.T) {
open := windowed(t)
groupsWaiting(t, "pr/553b7191claude")
asksWithPaths(t)
var said []inventory.Plan
was := inventory.PlanSaved
inventory.PlanSaved = func(p inventory.Plan) { said = append(said, p) }
inventory.PlanSaved = func(_ *inventory.Inventory, p inventory.Plan) { said = append(said, p) }
t.Cleanup(func() { inventory.PlanSaved = was })
now := time.Now().UTC()
claude := catalogueMerge("553b7191claude", "app", now.Add(-20*time.Second))
@@ -235,7 +292,7 @@ func TestTheAssemblingBatchIsShown(t *testing.T) {
dunst.Number, dunst.Title = 175, "dunst: the font the operator chose"
hear(t, open, claude, now.Add(-19*time.Second))
hear(t, open, dunst, now.Add(-time.Second))
hear(t, open, repoMerge("one", "a6bc0931one", now), now)
hear(t, open, fromHead(repoMerge("one", "a6bc0931one", now), dunst.Head), now)
out := captured(t, func() error { return plansCommand(t.Context(), nil) })
lines := strings.Split(out, "\n")
first := lines[0]
@@ -360,6 +417,7 @@ func TestAWaitingWalkIsFoldedIntoTheNextBatch(t *testing.T) {
// moves — named at once on a walk done — and joins the next batch when it does not.
func TestALateMergeIsAnsweredByTheWalkOfTheLaterOne(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asksWithPaths(t)
ctx := t.Context()
later := catalogueMerge("48bda475dunst", "notes", t0.Add(17*time.Second))
@@ -391,6 +449,7 @@ func TestALateMergeIsAnsweredByTheWalkOfTheLaterOne(t *testing.T) {
// first; the first delivered answers the older ones, and the search stops.
func TestAFailedWalkWalksItsEarlierMergesAlone(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asked := asksWithPaths(t)
ctx := t.Context()
// Made before the cut marks their modules seen, as merges are: walked again, they are news all the same.
@@ -459,6 +518,7 @@ func TestAFailedWalkWalksItsEarlierMergesAlone(t *testing.T) {
func TestAGroupsOrderBecomesTiersInsideTheWalk(t *testing.T) {
for _, grouped := range []bool{true, false} {
open := windowed(t)
everyHeadWaits(t)
asksWithPaths(t)
ctx := t.Context()
for _, m := range []string{"mesh-host"} {
@@ -496,9 +556,10 @@ func TestAGroupsOrderBecomesTiersInsideTheWalk(t *testing.T) {
// store, a merge handed over again is not doubled, and the batch is cut when its window closes.
func TestABatchSurvivesARestart(t *testing.T) {
open := windowed(t)
groupsWaiting(t, "feat/both")
asksWithPaths(t)
hear(t, open, repoMerge("one", "c1", t0), t0)
hear(t, open, repoMerge("two", "c2", t0.Add(30*time.Second)), t0.Add(30*time.Second))
hear(t, open, fromHead(repoMerge("one", "c1", t0), "feat/both"), t0)
hear(t, open, fromHead(repoMerge("two", "c2", t0.Add(30*time.Second)), "feat/both"), t0.Add(30*time.Second))
// A new controller: nothing of the first one's but the store.
again, err := openStores(t.Context())
@@ -506,7 +567,7 @@ func TestABatchSurvivesARestart(t *testing.T) {
t.Fatal(err)
}
t.Cleanup(again.Close)
hear(t, again, repoMerge("one", "c1", t0), t0.Add(60*time.Second)) // the bus hands it over again
hear(t, again, fromHead(repoMerge("one", "c1", t0), "feat/both"), t0.Add(60*time.Second)) // the bus hands it over again
cutAt(t, again, t0.Add(119*time.Second))
if ws, bs := walks(t, again); len(ws) != 0 || len(bs) != 1 || len(bs[0].Delivery.Merges) != 2 {
t.Fatalf("the restarted controller's batch: %d walk(s), %+v", len(ws), bs)
@@ -563,6 +624,7 @@ func TestAWaitingWalkNamesTheMergesItAnswers(t *testing.T) {
// nothing, and the merge joins the next batch.
func TestALateMergeOfAOneModuleRepositoryIsNamed(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asksWithPaths(t)
ctx := t.Context()
c2 := repoMerge("one", "c2", t0.Add(-50*time.Second))
@@ -716,6 +778,7 @@ func TestTheCatchUpKeepsWhatACutMadeHistory(t *testing.T) {
// still waits for the word, so no catalogue delivery skips its turn.
func TestAMergeOnTheControllersPathNeverSharesABatch(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asked := asksWithPaths(t)
ctx := t.Context()
if err := open.inventory.RegisterModule(ctx, catalogue.Manifest{Module: "mesh-delivery", Version: "1",
@@ -860,6 +923,7 @@ func sameMerge(a, b inventory.PlanMerge) bool {
// its delivery would skip its turn. It joins the next catalogue batch.
func TestALateCatalogueMergeIsNotAnsweredByAnOwnPathWalk(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
asksWithPaths(t)
ctx := t.Context()
for _, m := range []struct {
+4
View File
@@ -48,6 +48,10 @@ func gatherBatches(ctx context.Context, inv *inventory.Inventory, now time.Time)
if f.atMost.Before(f.closed) {
f.closed = f.atMost
}
if w.Batch.CutAtOnce && !b.Created.IsZero() {
// Its window closed when it was made, which is its latest too (ADR 0304 decision 1).
f.atMost, f.closed = b.Created, b.Created
}
}
if f.behind != "" {
if walk, err := inv.PlanByID(ctx, f.behind); err == nil {
+10
View File
@@ -152,6 +152,16 @@ func moduleCheckFor(paths []string, longestMachine int, out io.Writer) error {
}
}
// **What ADR 0325 refuses once a module runs as its own identity is warned of until then** (novox/hq ADR 0325 §1–§3):
// a directory naming its owner, a container declaring no identity that mounts the module's own directory or a
// read-write access. Printed, never counted: every registered container module does one of them today, and the
// catalogues migrate against these lines. A module that declares identity is refused them instead (ParseManifest).
for _, name := range names {
for _, w := range catalogue.IdentityWarnings(shelf[name]) {
fmt.Fprintf(out, "warning: %s\n", w)
}
}
// **A file that asks for a setting and does not say whether it is trusted counts as trusted** (novox/hq issue
// 339): listed and counted, never refused, so an author can opt out a file nothing trusts.
unsaid := 0
+50 -7
View File
@@ -161,9 +161,7 @@ func (f following) PullUpdated(ctx context.Context, p link.PullUpdated) error {
return u, err == nil
})
fmt.Print(planText(plan))
direct := link.Checked{Owner: p.Owner, Repo: p.Repo, Number: p.Number, Commit: p.Commit,
ID: link.NewBuildID(time.Now()), Verdict: "pass", Summary: noModuleTouched,
Gate: &link.CheckLayer{Verdict: "pass", Summary: noModuleTouched}, Plan: &plan}
direct := directVerdict(p, plan)
switch {
case !scope.gated() && !scope.Mesh:
fmt.Printf("%s/%s#%d (%.8s): %s, and the repository is not the mesh's: said, nothing run\n",
@@ -201,6 +199,24 @@ func (f following) PullUpdated(ctx context.Context, p link.PullUpdated) error {
return nil
}
// directVerdict is the verdict of a change no build judges — it touches no module, and either the repository is not
// the mesh's or the head holds no merge-check.sh. **It names the trunk commit it holds for** (novox/hq ADR 0323 §3):
// the base branch's head the forge announced with the head, so the merge tool can weigh it against the trunk's head
// and does not refuse it for ever (a verdict naming no trunk is never the trunk's head's). Nothing was built to
// judge: the trunk commit is the one the forge said the change was announced against.
func directVerdict(p link.PullUpdated, plan link.ChangePlan) link.Checked {
trunk := p.BaseSHA
if len(trunk) > 12 {
trunk = trunk[:12]
}
if trunk != "" {
plan.Trunk = trunk
}
return link.Checked{Owner: p.Owner, Repo: p.Repo, Number: p.Number, Commit: p.Commit,
ID: link.NewBuildID(time.Now()), Verdict: "pass", Summary: noModuleTouched,
Gate: &link.CheckLayer{Verdict: "pass", Summary: noModuleTouched}, Plan: &plan, Trunk: trunk}
}
// checkRequestFor is the ask for one pull request's head: the repository as the mesh clones it, the head,
// and what is read beside it.
func checkRequestFor(ctx context.Context, open *stores, p link.PullUpdated, scope checkScope,
@@ -363,6 +379,9 @@ func checkedOf(result link.BuildResult) link.Checked {
case result.Check != nil:
c.Verdict, c.Summary, c.Report = result.Check.Verdict, result.Check.Summary, result.Check.Report
c.Gate, c.RepoCheck = result.Check.Gate, result.Check.RepoCheck
// What was judged (novox/hq ADR 0323 §3): the head merged into the trunk commit, and the commits of the
// repositories read beside it.
c.Trunk, c.Beside, c.Took = result.Check.Trunk, result.Check.Beside, result.Check.Took
if c.Gate == nil {
// A build seat from before the layers: its verdict is the gate's.
c.Gate = &link.CheckLayer{Verdict: c.Verdict, Summary: c.Summary}
@@ -388,14 +407,20 @@ func checkedOf(result link.BuildResult) link.Checked {
}
if result.Checked != nil {
c.Plan = result.Checked.Plan
if c.Plan != nil && c.Trunk != "" {
// The kept delivery plan records the trunk commit with its other inputs (ADR 0323 §3).
plan := *c.Plan
plan.Trunk = c.Trunk
c.Plan = &plan
}
// A delivery group's composed check (novox/hq ADR 0239): every head it judged, this one first.
if g := result.Checked; g.Group != "" {
c.Group = g.Group
c.Members = append(c.Members, link.CheckedMember{Owner: g.Owner, Repo: g.Repo, Number: g.Number,
Commit: result.Ref})
Commit: result.Ref, Trunk: c.Trunk})
for _, m := range g.Members {
c.Members = append(c.Members, link.CheckedMember{Owner: m.Owner, Repo: m.Repo, Number: m.Number,
Commit: m.Ref})
Commit: m.Ref, Trunk: memberTrunk(result.Check, m)})
}
}
}
@@ -410,6 +435,20 @@ func checkedOf(result link.BuildResult) link.Checked {
return c
}
// memberTrunk is the trunk commit the build seat merged one of a group's heads into (novox/hq ADR 0323 §6), empty
// when it said none.
func memberTrunk(outcome *link.CheckOutcome, m link.GroupMember) string {
if outcome == nil {
return ""
}
for _, j := range outcome.Members {
if strings.EqualFold(j.Owner, m.Owner) && strings.EqualFold(j.Repo, m.Repo) && strings.HasPrefix(m.Ref, j.Commit) {
return j.Trunk
}
}
return ""
}
func prefixedAll(prefix string, items []string) []string {
out := make([]string, 0, len(items))
for _, i := range items {
@@ -424,8 +463,12 @@ func sayChecked(ctx context.Context, c link.Checked) {
if c.RepoCheck != nil {
repo = strings.ToUpper(c.RepoCheck.Verdict) + " — " + c.RepoCheck.Summary
}
fmt.Printf("%s: %s/%s#%d at %.8s checked on %s: gate %s — %s; repository %s\n", c.ID, c.Owner, c.Repo, c.Number,
c.Commit, orSomewhere(c.On), strings.ToUpper(c.Verdict), c.Summary, repo)
judged := ""
if c.Trunk != "" {
judged = " merged into trunk " + c.Trunk
}
fmt.Printf("%s: %s/%s#%d at %.8s%s checked on %s: gate %s — %s; repository %s\n", c.ID, c.Owner, c.Repo, c.Number,
c.Commit, judged, orSomewhere(c.On), strings.ToUpper(c.Verdict), c.Summary, repo)
if checkEvents == nil {
return
}
+78
View File
@@ -210,3 +210,81 @@ func TestABuildOffTheTrunkIsNeverPublished(t *testing.T) {
t.Errorf("a build seat that said nothing was refused: %v", err)
}
}
// **A verdict names the trunk commit it judged** (novox/hq ADR 0323 §3, issue 341): the controller's `checked`
// carries the head and the trunk commit the build seat merged it into, the commits of the repositories read beside
// it, how long it took, and the kept delivery plan records the trunk commit with its other inputs. A group's
// composed check names, per member, the trunk commit that head was merged into (§6).
func TestAVerdictNamesTheTrunkCommitItJudged(t *testing.T) {
plan := &link.ChangePlan{Repository: "novox/mesh-host", Base: "main", Head: "dfb1415aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
Summary: "builds mesh-host"}
asked := &link.CheckRequest{Owner: "novox", Repo: "mesh-host", Number: 66, Base: "main", Plan: plan}
c := checkedOf(link.BuildResult{ID: "b", Ref: "dfb1415aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", Checked: asked,
Check: &link.CheckOutcome{Verdict: "pass", Took: "4m12s", Head: "dfb1415aaaaa", Trunk: "d28ca95bbbbb",
Gate: &link.CheckLayer{Verdict: "pass"}, RepoCheck: &link.CheckLayer{Verdict: "pass"},
Beside: map[string]string{"mesh-catalog": "0123456789ab", "mesh-lab": "ba9876543210"}}})
if c.Trunk != "d28ca95bbbbb" || c.Commit != "dfb1415aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" {
t.Fatalf("the verdict names head %s and trunk %q", c.Commit, c.Trunk)
}
if c.Beside["mesh-catalog"] != "0123456789ab" || c.Beside["mesh-lab"] != "ba9876543210" {
t.Fatalf("the verdict names the repositories read beside it as %v", c.Beside)
}
if c.Took != "4m12s" {
t.Fatalf("the verdict says it took %q", c.Took)
}
if c.Plan == nil || c.Plan.Trunk != "d28ca95bbbbb" || c.Plan.Head != plan.Head {
t.Fatalf("the kept plan records %+v", c.Plan)
}
if plan.Trunk != "" {
t.Fatal("the plan the controller asked with was changed: the verdict's is a copy")
}
group := &link.CheckRequest{Owner: "novox", Repo: "mesh-controller", Number: 7, Base: "main", Group: "feat/x",
Members: []link.GroupMember{{Owner: "novox", Repo: "mesh-host", Number: 8, Ref: "cccccccccccc0000", Base: "main"}}}
c = checkedOf(link.BuildResult{ID: "g", Ref: "aaaaaaaaaaaa0000", Checked: group, Check: &link.CheckOutcome{
Verdict: "pass", Head: "aaaaaaaaaaaa", Trunk: "111111111111", Gate: &link.CheckLayer{Verdict: "pass"},
Members: []link.CheckedMember{{Owner: "novox", Repo: "mesh-host", Commit: "cccccccccccc", Trunk: "222222222222"}}}})
if len(c.Members) != 2 || c.Members[0].Trunk != "111111111111" || c.Members[1].Trunk != "222222222222" ||
c.Members[1].Commit != "cccccccccccc0000" {
t.Fatalf("the composed check names its members %+v", c.Members)
}
}
// **The merge made for a check is never registered** (novox/hq ADR 0323 §2): a build of it — a commit on no branch
// of the forge — is refused, naming the trunk. The rule of ADR 0238 §1 stands unchanged; this holds it for the
// merge commit a check makes.
func TestABuildOfACheckMergeIsRefusedNamingTheTrunk(t *testing.T) {
merge := link.BuildResult{ID: "b", Commit: "4b5c6d7e8f90aabbccddeeff0011223344556677", Trunk: "main"}
err := publishable(merge, "")
if !errors.Is(err, errOffTheTrunk) || !strings.Contains(err.Error(), "not on main") {
t.Fatalf("a build of a check's merge was answered %v", err)
}
}
// **A verdict no build judges names its trunk commit too** (novox/hq ADR 0323 §3): a change touching no module of a
// repository with no merge-check.sh (novox/mesh-tools-go) is answered by the controller itself. Naming no trunk, the
// merge tool refused it, asked again, got the same verdict, and refused for ever.
func TestADirectVerdictNamesTheTrunkTheForgeAnnounced(t *testing.T) {
p := link.PullUpdated{Owner: "novox", Repo: "mesh-tools-go", Number: 3, Base: "main", Commit: "1a2b3c4d5e6f7a8b9c0d",
BaseSHA: "7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f5a6b"}
c := directVerdict(p, link.ChangePlan{Repository: "novox/mesh-tools-go", Base: "main", Head: p.Commit})
if c.Trunk != "7a8b9c0d1e2f" || c.Plan == nil || c.Plan.Trunk != "7a8b9c0d1e2f" {
t.Fatalf("the direct verdict names trunk %q, its plan %+v", c.Trunk, c.Plan)
}
if c.Verdict != "pass" || c.Commit != p.Commit {
t.Fatalf("the direct verdict is %+v", c)
}
if c := directVerdict(link.PullUpdated{Owner: "novox", Repo: "x", Commit: "abc"}, link.ChangePlan{}); c.Trunk != "" {
t.Fatalf("a forge that said no base head gave the trunk %q", c.Trunk)
}
}
// A head checked again through mesh-delivery (`delivery check --members`, a recheck) reaches the same direct verdict:
// it names the base head the forge last announced, kept on the member, or the merge tool would refuse it for ever.
func TestARecheckedHeadsDirectVerdictNamesItsTrunk(t *testing.T) {
m := orderMember{ID: "novox/mesh-tools-go@1a2b3c4d5e6f", Repository: "novox/mesh-tools-go", Base: "main",
Head: "1a2b3c4d5e6f7a8b9c0d", Number: 3, BaseSHA: "7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f5a6b"}
if c := directVerdict(m.pull(), link.ChangePlan{}); c.Trunk != "7a8b9c0d1e2f" {
t.Fatalf("a rechecked head's direct verdict names trunk %q", c.Trunk)
}
}
+110 -31
View File
@@ -10,6 +10,7 @@ import (
"slices"
"sort"
"strings"
"sync"
"time"
"github.com/nats-io/nats.go"
@@ -240,7 +241,7 @@ func stopWalkWithout(ctx context.Context, inv *inventory.Inventory, id, by, why
}
return p, said, inv.SavePlan(ctx, &p)
}
target, err := batchToWalkAgain(ctx, inv, again, now)
target, folded, foldMoves, err := batchToWalkAgain(ctx, inv, p.ID, again, now)
if err != nil {
return p, said, fmt.Errorf("%s is not stopped: %w", p.ID, err)
}
@@ -261,8 +262,10 @@ func stopWalkWithout(ctx context.Context, inv *inventory.Inventory, id, by, why
if len(said.Kept) > 0 {
p.Note += "; " + keptWords(said.Kept)
}
// The batch first, so no merge names a plan the store does not hold; all of it, or nothing (decision 7).
if err := inv.SavePlansMoving(ctx, []*inventory.Plan{&target, &p}, moves); err != nil {
// The batch first, so no merge names a plan the store does not hold; all of it, or nothing (decision 7) — the
// batches folded into it too.
if err := inv.SavePlansMoving(ctx, append([]*inventory.Plan{&target, &p}, folded...),
append(moves, foldMoves...)); err != nil {
return p, said, fmt.Errorf("%s is not stopped, and none of its merges moved: %w", p.ID, err)
}
// What the batch holds and names now: written again by the cutter's next look should this fail.
@@ -317,36 +320,42 @@ func stopInBatch(ctx context.Context, inv *inventory.Inventory, b *inventory.Pla
return nil
}
// batchToWalkAgain is the batch a stopped walk's merges join: the open batch of their kind, or a new one, not yet
// written. The kind is read from what the merges moved.
func batchToWalkAgain(ctx context.Context, inv *inventory.Inventory, again []inventory.BatchedMerge,
now time.Time) (inventory.Plan, error) {
// batchToWalkAgain is the batch a stopped walk's merges join (novox/hq ADR 0299): a batch of their kind not yet cut
// that they are related to (ADR 0304 decision 1, ADR 0306 decision 3), or a new one, not yet written. Related to
// two or more, the newer are folded into the oldest: the plans and merge moves that fold them, to be written in
// the stop's one act. The kind is read from what the merges moved; the stopped walk itself is no walk a cut folds.
func batchToWalkAgain(ctx context.Context, inv *inventory.Inventory, stopping string, again []inventory.BatchedMerge,
now time.Time) (inventory.Plan, []*inventory.Plan, []inventory.MergeMove, error) {
own := false
for _, n := range movesOf(again) {
if _, on := onTheControllersPath[n]; on {
own = true
}
}
batches, err := inv.Batches(ctx)
entries, read, edges, err := theGraph(ctx, inv)
if err != nil {
return inventory.Plan{}, err
return inventory.Plan{}, nil, nil, err
}
for _, b := range batches {
if b.OwnPath() == own {
if b.Delivery == nil {
b.Delivery = &inventory.PlanDelivery{}
}
if b.Delivery.Batch == nil {
b.Delivery.Batch = &inventory.PlanBatch{Own: own}
}
return b, nil
joins, _, _, err := relatedBatches(ctx, inv, again, own, stopping, entries, read, edges)
if err != nil {
return inventory.Plan{}, nil, nil, err
}
if len(joins) > 0 {
into := joins[0]
if len(joins) == 1 {
return into, nil, nil, nil
}
plans, moves, err := foldInto(ctx, inv, &into, joins[1:])
if err != nil {
return inventory.Plan{}, nil, nil, err
}
return into, plans[1:], moves, nil
}
first := again[0]
return inventory.Plan{ID: fmt.Sprintf("plan-%d", now.UnixNano()), Repository: first.Repository, Branch: first.Branch,
Commit: first.Commit, Merged: first.Merged, Created: now, State: inventory.PlanAssembling,
Tiers: [][]string{}, Modules: map[string]*inventory.PlanModule{},
Delivery: &inventory.PlanDelivery{Batch: &inventory.PlanBatch{Own: own}}}, nil
Delivery: &inventory.PlanDelivery{Batch: &inventory.PlanBatch{Own: own}}}, nil, nil, nil
}
// leaveOut splits a walk's merges by the merges named to leave out, each `<repository>@<commit>` (a commit's
@@ -571,6 +580,9 @@ type orderMember struct {
CloneURL string `json:"clone_url,omitempty"`
// After are the repositories its pull request says it goes after (`after: <repository>`).
After []string `json:"after,omitempty"`
// BaseSHA is the base branch's head the forge last announced the head with (novox/hq ADR 0323): the trunk commit
// a verdict no build judges names when the head is checked again through mesh-delivery.
BaseSHA string `json:"base_sha,omitempty"`
}
// pull is the member as the forge announced it.
@@ -580,7 +592,7 @@ func (m orderMember) pull() link.PullUpdated {
if base == "" {
base = "main"
}
return link.PullUpdated{Owner: owner, Repo: repo, Number: m.Number, Base: base, Commit: m.Head,
return link.PullUpdated{Owner: owner, Repo: repo, Number: m.Number, Base: base, Commit: m.Head, BaseSHA: m.BaseSHA,
CloneURL: m.CloneURL, Paths: m.Paths, PathsTruncated: m.PathsTruncated, Removed: m.Removed,
ModuleDirs: m.ModuleDirs, ModuleDirsSaid: m.ModuleDirsSaid}
}
@@ -940,7 +952,7 @@ func deliveryCommand(ctx context.Context, args []string) error {
if *group == "" && len(members) == 1 {
// One head, checked again as the forge's announcement would have it (a recheck): the controller's
// own path for a pull request, run because the delivery's owner asked.
if err := sayingOnTheBus(ctx, func() error { return (following{open}).PullUpdated(ctx, members[0].pull()) }); err != nil {
if err := sayingOnTheBus(ctx, inv, func() error { return (following{open}).PullUpdated(ctx, members[0].pull()) }); err != nil {
return err
}
return answer(map[string]any{"rechecked": members[0].ID})
@@ -958,7 +970,7 @@ func deliveryCommand(ctx context.Context, args []string) error {
return errors.New("delivery go <plan> [--by <who>] [--why <text>]")
}
var p inventory.Plan
if err := sayingOnTheBus(ctx, func() (err error) {
if err := sayingOnTheBus(ctx, inv, func() (err error) {
p, err = letGo(ctx, inv, positionals[0], *by, strings.TrimSpace(*why))
return err
}); err != nil {
@@ -973,7 +985,7 @@ func deliveryCommand(ctx context.Context, args []string) error {
}
var p inventory.Plan
var said stopAnswer
if err := sayingOnTheBus(ctx, func() (err error) {
if err := sayingOnTheBus(ctx, inv, func() (err error) {
p, said, err = stopWalkWithout(ctx, inv, positionals[0], *by, strings.TrimSpace(*why), splitList(*without),
time.Now().UTC())
return err
@@ -1176,7 +1188,7 @@ func askGroupCheck(ctx context.Context, open *stores, group string, members []or
return "", fmt.Errorf("%s cannot be cloned for the group's check: %w", m.ID, err)
}
request.Check.Members = append(request.Check.Members, link.GroupMember{Owner: mp.Owner, Repo: mp.Repo,
Number: mp.Number, Repository: url, Ref: mp.Commit, Paths: mp.Paths})
Number: mp.Number, Repository: url, Ref: mp.Commit, Paths: mp.Paths, Base: mp.Base})
}
seat := buildSeatHeld(ctx)
ask, err := askOverOn(seat)
@@ -1215,15 +1227,19 @@ func shelfOf(entries []inventory.Entry) map[string]catalogue.Manifest {
// delivery it walks reads its steps from. Never in the way of the walk: said beside it, and a save that could
// not be said is logged — the delivery's owner, which also asks for the walks it follows, finds it by
// comparison. Records arriving out of order are told apart by the plan's revision.
func sayPlanMoved(ctx context.Context, bus link.Bus, p inventory.Plan) {
go publishPlanMoved(ctx, bus, p)
//
// On the save that starts the walk and the one that ends it, the walk also says what it touches, its bound and,
// at the end, its outcome (novox/hq ADR 0308 decision 1, walk_said.go), read from facts; nil facts say only what
// the walk recorded.
func sayPlanMoved(ctx context.Context, bus link.Bus, facts walkFacts, p inventory.Plan) {
go publishPlanMoved(ctx, bus, facts, p)
}
func publishPlanMoved(ctx context.Context, bus link.Bus, p inventory.Plan) {
func publishPlanMoved(ctx context.Context, bus link.Bus, facts walkFacts, p inventory.Plan) {
// Its phases so far (novox/hq ADR 0282 decision 6): the rest's reports come after the walk ends, and are
// read by whoever asks for the walk (`delivery walks`).
p.Phases = p.WalkPhases(time.Now(), nil)
body, err := json.Marshal(p)
body, err := json.Marshal(planMovedBody{Plan: p, walkSaid: sayWalk(ctx, facts, p)})
if err != nil {
return
}
@@ -1238,7 +1254,7 @@ func publishPlanMoved(ctx context.Context, bus link.Bus, p inventory.Plan) {
// controller's, or a connection of the command's own — a verb runs as a command of its own, and what it kept
// would otherwise be said by nobody until the delivery's owner read the walks back. Without a bus the command
// still runs; what it did is found by comparison.
func sayingOnTheBus(ctx context.Context, f func() error) error {
func sayingOnTheBus(ctx context.Context, facts walkFacts, f func() error) error {
if checkEvents != nil {
return f()
}
@@ -1249,9 +1265,10 @@ func sayingOnTheBus(ctx context.Context, f func() error) error {
return err
}
bus := link.OverNATS{Conn: conn, JS: js}
was := inventory.PlanSaved
checkEvents = bus
inventory.PlanSaved = func(p inventory.Plan) { publishPlanMoved(ctx, bus, p) }
defer func() { checkEvents, inventory.PlanSaved = nil, nil }()
inventory.PlanSaved = func(_ *inventory.Inventory, p inventory.Plan) { publishPlanMoved(ctx, bus, facts, p) }
defer func() { checkEvents, inventory.PlanSaved = nil, was }()
ran = true
return f()
})
@@ -1261,3 +1278,65 @@ func sayingOnTheBus(ctx context.Context, f func() error) error {
}
return err
}
// sayingWhatAVerbKept is PlanSaved for a process that is not the serving controller: each walk the verb keeps is
// said as `plan-moved` before the save returns, with the facts of the inventory that kept it, on the bus the verb
// reaches. It reaches the bus on the first save and never for a verb that keeps no walk. When the bus cannot be
// reached, a save that started or ended a walk says so on the verb's error output: its mark is kept and no later
// save says it, so the router holds what the walk touched until its bound, or not at all for a start. done lets
// the bus go and takes the hook back.
func sayingWhatAVerbKept(ctx context.Context) (said func(*inventory.Inventory, inventory.Plan), done func()) {
var (
mu sync.Mutex
reached bool
bus link.Bus
closing func()
failed error
)
said = func(inv *inventory.Inventory, p inventory.Plan) {
mu.Lock()
defer mu.Unlock()
if !reached {
reached = true
if handActConn != nil {
bus, failed = overNATS(handActConn)
} else if js, err := aBus(); err != nil {
failed = err
} else if bus, failed = overNATS(js.Conn()); failed == nil {
closing = js.Close
} else {
js.Close()
}
}
if bus == nil {
switch p.WalkSaid {
case inventory.WalkStarted:
fmt.Fprintf(os.Stderr, "%s: its start is kept and could NOT be said as plan-moved (%v): the router "+
"does not hold what it touches\n", p.ID, failed)
case inventory.WalkEnded:
fmt.Fprintf(os.Stderr, "%s: its end is kept and could NOT be said as plan-moved (%v): the router holds "+
"what it touched until its bound\n", p.ID, failed)
}
return
}
publishPlanMoved(ctx, bus, inv, p)
}
done = func() {
mu.Lock()
defer mu.Unlock()
inventory.PlanSaved = nil
if closing != nil {
closing()
}
}
return said, done
}
// overNATS is the bus a connection carries, with its JetStream.
func overNATS(conn *nats.Conn) (link.Bus, error) {
js, err := conn.JetStream()
if err != nil {
return nil, err
}
return link.OverNATS{Conn: conn, JS: js}, nil
}
+8 -4
View File
@@ -47,10 +47,14 @@ type stalledLine struct {
// issue 443): a time inside a finished sentence cannot be said again in another zone. A mesh-delivery from before
// says none, and Waiting is said as it reads.
Checks []waitingCheck `json:"checks,omitempty"`
For string `json:"for"`
Bound string `json:"bound"`
H2 string `json:"h2"`
Says string `json:"says"`
// Held is the state of the delivery of a line of the state `unanswered`, ready or rejected, when one is held
// (novox/hq issues 441, 451): its check answered once, so a recheck is the step. Empty for a head with no
// delivery, and from a mesh-delivery that does not say it yet; the words are never read for it.
Held string `json:"held,omitempty"`
For string `json:"for"`
Bound string `json:"bound"`
H2 string `json:"h2"`
Says string `json:"says"`
}
// waitingCheck is one merge check a stalled line waits on, as mesh-delivery gives it: its context, its state —
@@ -119,6 +119,13 @@ func TestTheBusMachineIsBehindByItsUserListAlone(t *testing.T) {
if userListBehind("", "") {
t.Fatal("a machine sent no list reads as behind")
}
// A list that could not be composed is behind: not known is never "not behind" (the review of hq issue 353).
if !listBehind("", errors.New("the plan cannot be worked out"), digestOf([]byte(list))) {
t.Fatal("a list nobody could compose reads as current")
}
if listBehind(list, nil, digestOf([]byte(list))) || !listBehind(list, nil, "") {
t.Fatal("listBehind does not read the digest as userListBehind does")
}
}
// The machine holding the bus goes first: its declaration carries the user list the new grants are
+56
View File
@@ -154,6 +154,10 @@ type gateFacts struct {
// groupsAdded is, per module, whether the move judged puts an account in a group its previous build did
// not (issue 318 review): the only move whose wait for a new login is excused.
groupsAdded map[string]bool
// accessesChanged is, per module, whether the move judged changes its accesses (novox/hq ADR 0325): an access the
// engine cannot judge this look (unknown), or entries it left while a disk was away, of a module whose move leaves
// its accesses as they were is no fault of the move. A module absent here is taken to change them: judged as before.
accessesChanged map[string]bool
// waits is what the controller holds to check a module's wait for the operator (novox/hq ADR 0283): the
// manifest of each module's build judged, and the secrets given on each machine.
waits operatorWaitFacts
@@ -528,6 +532,7 @@ func judgeMoves(ctx context.Context, open *stores, g *inventory.PlanGate, pairs
return "", err
}
facts.groupsAdded = movesAddingGroups(ctx, open.inventory, g, pairs, shelf)
facts.accessesChanged = movesChangingAccesses(ctx, open.inventory, g, pairs, shelf)
facts.waits = gateWaitFacts(ctx, open.inventory, g, pairs, shelf, facts.health)
facts.sent = g.Sent
facts.commits, facts.sentBuilds = judgedCommits(g, pairs), map[string]map[string]string{}
@@ -783,6 +788,57 @@ func joinSaid(a, b string) string {
return a + "; " + b
}
// movesChangingAccesses is, per module a gate judges, whether its move changes its accesses — one added, dropped,
// or given another use — read from the builds' manifests (novox/hq ADR 0325). A move not known changes them: an
// access it cannot judge then holds the gate to its bound, and never passes unseen.
func movesChangingAccesses(ctx context.Context, inv *inventory.Inventory, g *inventory.PlanGate, pairs []judged,
shelf map[string]catalogue.Manifest) map[string]bool {
out := map[string]bool{}
for _, j := range pairs {
if _, done := out[j.module]; done {
continue
}
from, to := g.From, g.To
for _, c := range g.Carried {
if c.Module == j.module {
from, to = c.From, c.To
break
}
}
target, found, err := inv.ManifestAt(ctx, j.module, to)
if err != nil || !found {
target = shelf[j.module]
}
before, had, err := inv.ManifestAt(ctx, j.module, from)
if from == "" || err != nil || !had {
out[j.module] = true
continue
}
out[j.module] = !sameAccessUse(before, target)
}
return out
}
// sameAccessUse says a move leaves a module's accesses as they were, users included (the review of e1c2b744): the
// same accesses at the same use, and the same containers declaring identity and mounting each — a container that
// starts or stops running as the module's identity changes who uses the folder.
func sameAccessUse(a, b catalogue.Manifest) bool {
return sameAccesses(a.Accesses, b.Accesses) && catalogue.AccessUsersShape(a) == catalogue.AccessUsersShape(b)
}
// sameAccesses says two manifests declare the same accesses, at the same use, in any order.
func sameAccesses(a, b []catalogue.Access) bool {
key := func(as []catalogue.Access) string {
var out []string
for _, x := range as {
out = append(out, x.ID+"\x00"+x.Path+"\x00"+x.At())
}
sort.Strings(out)
return strings.Join(out, "\x01")
}
return key(a) == key(b)
}
// movesAddingGroups is, per module a gate judges, whether its move puts an account in a group the build it
// moved from did not: read from the builds' manifests. A module whose move is not known adds none.
func movesAddingGroups(ctx context.Context, inv *inventory.Inventory, g *inventory.PlanGate, pairs []judged,
+158
View File
@@ -0,0 +1,158 @@
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"strings"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// **A delivery group still assembles** (novox/hq ADR 0306 decision 2): a merge whose delivery group has a member
// not yet merged waits for the merge window, whether or not a batch is assembling, so the group's members walk in
// one batch, in their order (ADR 0276 decision 5). Which groups exist and where each member stands is
// mesh-delivery's record (ADR 0239): a group is the pull requests of two repositories or more sharing a head
// branch name, which is its id. The controller asks it, through its `groups` verb, when a merge from a head is
// heard; a member whose pull request is open (proposed, checked, ready or rejected) is still to merge. When
// mesh-delivery does not answer within groupAskWithin, or answers something unreadable, the merge waits: the safe
// error is to wait, and its batch says why.
// groupAskWithin is how long mesh-delivery is given to say a merge's group, while the merge is being heard.
var groupAskWithin = 3 * time.Second
// openPullStates are the states of a delivery whose pull request is open and not merged (mesh-delivery's table):
// a member in one of them is still to merge. Closed unmerged is stopped, final, and waits for nothing.
var openPullStates = map[string]bool{"proposed": true, "checked": true, "ready": true, "rejected": true}
// groupMember is one member of a delivery group as mesh-delivery's `groups` says it: `<owner/repo>@<commit> (state)`.
type groupMember struct {
Repository, State string
}
// deliveryGroupOf is what mesh-delivery says of the group a head names: its members, and whether the head names a
// group at all; an error when it cannot be asked or did not answer. A seam a test replaces.
var deliveryGroupOf = func(ctx context.Context, held bool, head string) ([]groupMember, bool, error) {
if !held {
return nil, false, fmt.Errorf("no holder of %s is on record to say it", catalogue.DeliverySeat)
}
serving := servingBus.Load()
if serving == nil {
return nil, false, errors.New("this controller is not on the bus")
}
asking, cancel := context.WithTimeout(ctx, groupAskWithin)
defer cancel()
raw, err := askDeliveryOwner(asking, serving.Conn(), "groups", map[string]any{})
if err != nil {
if errors.Is(err, link.ErrNothingServes) {
return nil, false, fmt.Errorf("nothing serves %s.groups", catalogue.DeliverySeat)
}
return nil, false, err
}
var lines []struct {
ID string `json:"id"`
Members []string `json:"members"`
}
if err := json.Unmarshal(raw, &lines); err != nil {
return nil, false, fmt.Errorf("%s.groups answered something unreadable: %w", catalogue.DeliverySeat, err)
}
for _, l := range lines {
if l.ID != head {
continue
}
var members []groupMember
for _, s := range l.Members {
members = append(members, memberOf(s))
}
return members, true, nil
}
return nil, false, nil
}
// memberOf reads one member as `groups` says it: `novox/mesh-sdk@d41d255f28ad (held)`.
func memberOf(s string) groupMember {
id, state := strings.TrimSpace(s), ""
if i := strings.LastIndex(id, " ("); i >= 0 && strings.HasSuffix(id, ")") {
id, state = id[:i], id[i+2:len(id)-1]
}
repository, _, _ := strings.Cut(id, "@")
return groupMember{Repository: repository, State: state}
}
// groupWait says whether a merge heard now waits for the window for its delivery group, and why in words; nothing
// when it names no head, its head names no group, or every other member is merged or closed. own is the merge's
// kind (ownPath).
//
// A member of another repository whose merge this controller already keeps in a batch not yet cut of the same
// kind is merged, whatever mesh-delivery says yet (it marks a member merged once its walk opened): the merge joins
// that batch by their head. One kept in a batch of the other kind keeps the merge waiting, whatever mesh-delivery
// says (review of #250): kinds never share a batch, so the merge's batch is a new one, and cut at once it would
// walk before that member's batch is cut and break the group's order (ADR 0276 decision 5). Waiting, the two
// batches are cut in the order their windows close, an own-path one first at one look, as cutBatchesHeld cuts
// every pair of kinds.
//
// Asked before the plans are held: a slow answer holds up no walk.
func groupWait(ctx context.Context, inv *inventory.Inventory, m link.SourceMoved, held, own bool) (string, error) {
if m.Head == "" {
return "", nil
}
same, other, err := keptFromHead(ctx, inv, m.Head, own)
if err != nil {
return "", err
}
if len(other) > 0 {
return "group " + m.Head + " has a member waiting in a batch of the other kind: " + strings.Join(other, ", "), nil
}
members, isGroup, err := deliveryGroupOf(ctx, held, m.Head)
if err != nil {
return fmt.Sprintf("%s did not say whether group %s has a member still to merge (%v): it waits for the "+
"window, the safe side", catalogue.DeliverySeat, m.Head, err), nil
}
if !isGroup {
return "", nil
}
var open []string
for _, g := range members {
if strings.EqualFold(g.Repository, m.Owner+"/"+m.Repo) || same[strings.ToLower(g.Repository)] ||
!openPullStates[g.State] {
continue
}
open = append(open, g.Repository+" ("+g.State+")")
}
if len(open) == 0 {
return "", nil
}
return "group " + m.Head + " has a member still to merge: " + strings.Join(open, ", "), nil
}
// keptFromHead are the merges from a head that batches not yet cut hold: the repositories (lower case) of those in
// a batch of the kind own says, and, in words, each of the other kind with its batch.
func keptFromHead(ctx context.Context, inv *inventory.Inventory, head string, own bool) (map[string]bool, []string, error) {
batches, err := inv.Batches(ctx)
if err != nil {
return nil, nil, err
}
same := map[string]bool{}
var other []string
for _, b := range batches {
merges, err := inv.MergesOf(ctx, b.ID)
if err != nil {
return nil, nil, err
}
for _, k := range merges {
if announcedOf(k).Head != head {
continue
}
if b.OwnPath() == own {
same[strings.ToLower(k.Repository)] = true
} else {
other = append(other, fmt.Sprintf("%s@%s in %s", k.Repository, short(k.Commit), b.ID))
}
}
}
return same, other, nil
}
+51
View File
@@ -167,3 +167,54 @@ func TestASetuidSearchAsksOnlyAKnownNodeAndFailsOnARefusal(t *testing.T) {
t.Fatalf("a start: %v, printed %q", err, out.String())
}
}
// The directory judgement's line asks only a known node, one name and nothing else, fails on a refusal, and says it
// changed nothing (novox/hq issue 358).
func TestADirectoryJudgementAsksOnlyAKnownNodeAndSaysItIsReadOnly(t *testing.T) {
t.Setenv(link.CallerVar, "jo through mesh-cli on anchor")
asked := 0
ask := func(answer link.HandOverAnswer) func(node, by string) (link.HandOverAnswer, error) {
return func(node, by string) (link.HandOverAnswer, error) {
asked++
if node != "novox" || by != "jo through mesh-cli on anchor" {
t.Fatalf("asked %q %q", node, by)
}
return answer, nil
}
}
known := func(string) error { return nil }
var out bytes.Buffer
for _, args := range [][]string{nil, {"novox", "extra"}, {"-x"}} {
if err := judgeDirectoriesAsked(args, known, ask(link.HandOverAnswer{Said: "x"}), &out); err == nil || asked != 0 {
t.Fatalf("%v was asked: %v", args, err)
}
}
if err := judgeDirectoriesAsked([]string{"novox"}, func(string) error { return errors.New("no node called novox") },
ask(link.HandOverAnswer{Said: "x"}), &out); err == nil || asked != 0 {
t.Fatalf("an unknown node: %v", err)
}
if err := judgeDirectoriesAsked([]string{"novox"}, known, ask(link.HandOverAnswer{Refused: "no; nothing was judged"}),
&out); err == nil || !strings.Contains(err.Error(), "novox refused") || out.Len() != 0 {
t.Fatalf("a refusal: %v, printed %q", err, out.String())
}
said := "2 directories declared: 1 placed, 1 would be refused\n/var/lib/a: placed\n/var/lib/b: would be refused: …"
if err := judgeDirectoriesAsked([]string{"novox"}, known, ask(link.HandOverAnswer{Said: said}), &out); err != nil ||
out.String() != "novox (read-only: nothing on novox was changed)\n"+said+"\n" {
t.Fatalf("a judgement: %v, printed %q", err, out.String())
}
}
// The judgement is a seat verb an agent may call — it reads, and changes nothing on the node — and runs exactly
// `node judge-directories <node>` (novox/hq issue 358).
func TestTheDirectoryJudgementIsAVerbThatOnlyReads(t *testing.T) {
argv, err := argvFor("judge-directories", map[string]any{"node": "novox"})
if err != nil || strings.Join(argv, " ") != "node judge-directories novox" {
t.Fatalf("the verb runs %v, %v", argv, err)
}
if _, err := argvFor("judge-directories", map[string]any{}); err == nil {
t.Fatal("the verb ran with no node")
}
if !readingVerbs["judge-directories"] {
t.Fatal("the judgement is not counted a reading verb")
}
}
+11 -5
View File
@@ -123,11 +123,12 @@ func TestDeliveryStopTakesTheMergesToLeaveOut(t *testing.T) {
// that builds the other repository's module alone.
func TestAStopWithoutAMergeWalksTheOthersAgain(t *testing.T) {
open := windowed(t)
groupsWaiting(t, "pr/cat1aaaaaaaa")
endsItsLine(t)
asksRecorded(t)
ctx := t.Context()
cat := catalogueMerge("cat1aaaaaaaa", "app", t0)
one := repoMerge("one", "one1bbbbbbbb", t0.Add(5*time.Second))
one := fromHead(repoMerge("one", "one1bbbbbbbb", t0.Add(5*time.Second)), cat.Head)
hear(t, open, cat, t0.Add(time.Second))
hear(t, open, one, t0.Add(6*time.Second))
cutAt(t, open, t0.Add(2*time.Minute))
@@ -181,12 +182,13 @@ func TestAStopWithoutAMergeWalksTheOthersAgain(t *testing.T) {
// ended. A merge the walk does not answer is refused, and nothing ends.
func TestAStopWithoutAMergeEndsTheMergesThatContainIt(t *testing.T) {
open := windowed(t)
groupsWaiting(t, "pr/cat1aaaaaaaa")
endsItsLine(t)
asksRecorded(t)
ctx := t.Context()
first := catalogueMerge("cat1aaaaaaaa", "app", t0)
later := catalogueMerge("cat2cccccccc", "notes", t0.Add(10*time.Second))
one := repoMerge("one", "one1bbbbbbbb", t0.Add(5*time.Second))
one := fromHead(repoMerge("one", "one1bbbbbbbb", t0.Add(5*time.Second)), first.Head)
hear(t, open, first, t0.Add(time.Second))
hear(t, open, one, t0.Add(6*time.Second))
hear(t, open, later, t0.Add(11*time.Second))
@@ -238,11 +240,12 @@ func TestAStopWithoutAMergeEndsTheMergesThatContainIt(t *testing.T) {
// (review of #232): its window closes, and its walk is at the newer commit, answering both.
func TestAStopWalksTheOthersAgainInTheOpenBatch(t *testing.T) {
open := windowed(t)
groupsWaiting(t, "pr/cat1aaaaaaaa")
endsItsLine(t)
asksRecorded(t)
ctx := t.Context()
cat := catalogueMerge("cat1aaaaaaaa", "app", t0)
one1 := repoMerge("one", "one1bbbbbbbb", t0.Add(5*time.Second))
one1 := fromHead(repoMerge("one", "one1bbbbbbbb", t0.Add(5*time.Second)), cat.Head)
hear(t, open, cat, t0.Add(time.Second))
hear(t, open, one1, t0.Add(6*time.Second))
cutAt(t, open, t0.Add(2*time.Minute))
@@ -290,11 +293,12 @@ func TestAStopWalksTheOthersAgainInTheOpenBatch(t *testing.T) {
// that walked its merges again is refused a retry, saying why.
func TestAStopNamesWhatStaysOnTheMachinesAndWhatStillCarriesIt(t *testing.T) {
open := windowed(t)
groupsWaiting(t, "pr/cat1aaaaaaaa")
endsItsLine(t)
asksRecorded(t)
ctx := t.Context()
cat := catalogueMerge("cat1aaaaaaaa", "app", t0)
one1 := repoMerge("one", "one1bbbbbbbb", t0.Add(5*time.Second))
one1 := fromHead(repoMerge("one", "one1bbbbbbbb", t0.Add(5*time.Second)), cat.Head)
hear(t, open, cat, t0.Add(time.Second))
hear(t, open, one1, t0.Add(6*time.Second))
cutAt(t, open, t0.Add(2*time.Minute))
@@ -335,11 +339,12 @@ func TestAStopNamesWhatStaysOnTheMachinesAndWhatStillCarriesIt(t *testing.T) {
// with the others and withholds the stopped merge's modules; with every merge left out, the batch is stopped.
func TestAStopWithoutAMergeLeavesItOutOfABatch(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
endsItsLine(t)
asksRecorded(t)
ctx := t.Context()
cat := catalogueMerge("cat1aaaaaaaa", "app", t0)
one := repoMerge("one", "one1bbbbbbbb", t0.Add(5*time.Second))
one := fromHead(repoMerge("one", "one1bbbbbbbb", t0.Add(5*time.Second)), cat.Head)
hear(t, open, cat, t0.Add(time.Second))
hear(t, open, one, t0.Add(6*time.Second))
_, bs := walks(t, open)
@@ -382,6 +387,7 @@ func TestAStopWithoutAMergeLeavesItOutOfABatch(t *testing.T) {
// else since it was read refuses all of it, and no merge moves, no plan is written.
func TestSavingPlansAndMovingMergesIsOneAct(t *testing.T) {
open := windowed(t)
everyHeadWaits(t)
endsItsLine(t)
ctx := t.Context()
hear(t, open, catalogueMerge("cat1aaaaaaaa", "app", t0), t0.Add(time.Second))
+394
View File
@@ -0,0 +1,394 @@
package main
// The mesh takes a shared folder that already holds data once, on the operator's word (novox/hq ADR 0325 §7, §8).
//
// **What waits.** A node-engine that keeps folders states an access waiting, naming it (inventory.Wait.Access), while
// its folder is there and has not been taken; the controller excuses that wait only for a declared, placed folder not
// yet taken (checkAccessWait), and the module is said as its needs-operator condition, in words with no path.
//
// **One ask per folder and machine.** The serving controller's asker asks the operator once for each such folder on
// each machine — however many modules wait on it — at the level approve (ADR 0259), in the clause's words: "Let the
// mesh keep <path> on <machine>, now for <modules>: …". It is kept in the controller's asks beside a proposal
// (asked.Take), never a condition's: the condition is the module's, and two modules waiting on one folder are one
// question. While it is open no second is asked; one that expired unanswered is asked again while the folder waits;
// one about a folder that no longer waits is taken back. None is asked for a folder already taken, so a module that
// joins a taken folder is asked nothing (its access wait is not excused, and its entries are added at apply).
//
// **On the warrant, and only on it**, the controller asks the machine's engine to take the folder (link.AskTake),
// naming the access by its resource id and the warrant; the engine adds the entries of the users it was declared for
// that folder. On its word that it started, the folder is recorded taken (kept_folder), and from the next send every
// access to it says `taken`. Where the engine's wait says the filesystem must first allow ACLs (acl
// "needs-enabling"), the ask says the take sets that; where it cannot carry them (acl "unsupported") the wait is not
// excused and nothing is asked.
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"path/filepath"
"sort"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/asks"
"github.com/novox/mesh-controller/internal/conditions"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/outward"
)
// folderTake is one folder the operator is asked to let the mesh keep, as the controller's ask keeps it
// (asked.Take). Every field the ask is composed from is here, so the act on the warrant is composed from the record.
type folderTake struct {
Node string `json:"node"`
Path string `json:"path"`
// Access is the resource id of one access to it in the machine's declaration: what the take act names.
Access string `json:"access"`
// Accesses are the access ids (each module's own) that place it, Modules the modules waiting on it, both sorted.
Accesses []string `json:"accesses"`
Modules []string `json:"modules"`
// EnablesACL says the folder's filesystem must first allow access entries (a ZFS dataset's acltype), which the
// take act sets: the ask says so.
EnablesACL bool `json:"enables-acl,omitempty"`
// Placed is the filesystem the folder is on, as the engine names it without a path: the warrant is for that one.
Placed string `json:"placed,omitempty"`
At time.Time `json:"at"`
}
// takeVerb is the verb a take's answer binds: the controller's own, performed by itself on the warrant.
const takeVerb = askerName + ".take"
// key is the folder on its machine: what one ask is kept for.
// The filesystem is part of it: a folder that moved to another disk is asked about again.
func (t folderTake) key() string { return t.Node + "\x00" + filepath.Clean(t.Path) + "\x00" + t.Placed }
// digest binds everything the take does and was said for: the machine, the folder, the access it names and the
// modules the operator was told.
func (t folderTake) digest() string {
raw, _ := json.Marshal(struct {
Node, Path, Access, Placed string
Modules []string
EnablesACL bool
}{t.Node, filepath.Clean(t.Path), t.Access, t.Placed, t.Modules, t.EnablesACL})
sum := sha256.Sum256(raw)
return "sha256:" + hex.EncodeToString(sum[:])
}
// actions is the take's one answer, at the level approve, binding the exact act through boundAct.
func (t folderTake) actions(id string) []conditions.Action {
return []conditions.Action{{Label: "Let the mesh keep it", Verb: takeVerb, Machine: t.Node,
Level: conditions.LevelApprove, Arguments: map[string]string{"take": id, "node": t.Node,
"path": filepath.Clean(t.Path), "access": t.Access, "digest": t.digest()}}}
}
// takeWords is the clause's ask (ADR 0325 §8), with the folder as shown: "Let the mesh keep <path> on <machine>, now
// for <modules>: …".
func takeWords(shown, machine string, modules []string, enablesACL bool) string {
words := fmt.Sprintf("Let the mesh keep %s on %s, now for %s: it adds their access entries, and those of any "+
"module given access later, to every folder and file there; the group bits `ls -l` shows become the "+
"entries' mask, and owners, groups and every other right stay as they are", shown, machine,
strings.Join(modules, ", "))
if enablesACL {
words += ". Its filesystem does not allow access entries yet (a ZFS dataset's acltype): approving sets that first"
}
return words
}
// ask is the take's ask, composed from it alone. The folder's path is said whole where the router's content rule
// lets it leave the mesh, and otherwise named by the shared folder it is ("the shared folder movies"), which is the
// access the module's wait names.
func (t folderTake) ask(id string, machines []string) (asks.Ask, map[string]int) {
shown := filepath.Clean(t.Path)
if _, ok := outward.Check(shown, machines...); !ok {
shown = "the shared folder " + strings.Join(t.Accesses, ", ")
}
headline := fmt.Sprintf("Let the mesh keep a shared folder on %s?", t.Node)
if len([]rune(headline)) > asks.HeadlineLength {
headline = "Let the mesh keep a shared folder?"
}
explanation := takeWords(shown, t.Node, t.Modules, t.EnablesACL) + "."
if _, ok := outward.Check(explanation, machines...); !ok {
explanation = sayable(explanation, machines)
}
q := asks.Ask{ID: id, Headline: headline, Explanation: explanation,
Who: asks.Operator, Expires: t.At.Add(askApproveFor),
OnExpiry: "nothing is changed, and you are asked again while the folder waits", About: "take." + t.Node}
options := map[string]int{}
for i, act := range t.actions(id) {
binds, _ := asks.ActDigest(boundAct(act))
oid := optionID(act.Label)
options[oid] = i
q.Options = append(q.Options, asks.Option{ID: oid, Label: act.Label, Level: asks.Level(act.Level), Binds: binds,
Does: "the mesh adds the access entries there, and keeps them from then on without asking"})
}
return q, options
}
// takeOnWarrant performs a take the operator approved: the engine is asked, and the folder recorded taken on its
// word. It answers what the engine said.
type takeOnWarrant func(ctx context.Context, t folderTake, askID, by string) (string, error)
// askFolders asks the operator about every folder waiting to be kept that is not asked about yet, and takes back an
// open ask about one that no longer waits. Called at the end of each reconcile, after the conditions' asks, within
// the asks the router holds open at once.
func (a *asker) askFolders(ctx context.Context) error {
if a.folders == nil {
return nil
}
wanted, err := a.folders(ctx)
if err != nil {
return err
}
now := a.now()
all, err := a.store.All(ctx)
if err != nil {
return err
}
open, openNow := map[string]asked{}, 0
for _, r := range all {
if r.State != askOpen || !now.Before(r.Ask.Expires) {
continue
}
openNow++
if r.Take != nil {
open[r.Take.key()] = r
}
}
want := map[string]bool{}
for _, t := range wanted {
want[t.key()] = true
}
for key, r := range open {
if !want[key] {
if err := a.cancel(ctx, r, "the folder no longer waits to be kept"); err != nil {
return err
}
openNow--
}
}
var machines []string
if a.machines != nil {
if machines, err = a.machines(ctx); err != nil {
return err
}
}
for _, t := range wanted {
if _, asking := open[t.key()]; asking {
continue // one ask per folder and machine (§8)
}
if openNow >= askMostOpen {
return nil // asked when one of the open ones ends
}
t.At = now
id := newAskID()
q, options := t.ask(id, machines)
if err := q.Check(now); err != nil {
a.logf("the operator could not be asked to let the mesh keep a folder on %s: %v", t.Node, err)
continue
}
body, err := json.Marshal(q)
if err != nil {
return err
}
take := t
if err := a.store.Create(ctx, asked{ID: id, Condition: q.About, Ask: q, Actions: t.actions(id),
Options: options, State: askOpen, Opened: now, Take: &take}); err != nil {
return fmt.Errorf("the take could not be kept in the controller's asks, so it was not asked: %w", err)
}
if err := a.publish(ctx, asks.AskSubject(askerName), body, "ask."+id); err != nil {
_, _ = a.store.Change(ctx, id, func(x *asked) bool {
if x.State != askOpen || x.Acted != "" {
return false
}
x.State, x.Ended, x.Acted = askUnsent, a.now(), "nothing: it could not be published: "+err.Error()
return true
})
a.logf("the operator could not be asked to let the mesh keep a folder on %s: %v", t.Node, err)
continue
}
openNow++
a.logf("asked the operator to let the mesh keep a folder on %s for %s (%s)", t.Node,
strings.Join(t.Modules, ", "), id)
}
return nil
}
// decideTake is what the asker does with a warrant for a take (asker.Decided): the act only when it is the one the
// option bound — composed again from the record, the users' digest among it — and then the engine is asked.
func (a *asker) decideTake(ctx context.Context, r asked, act conditions.Action, w asks.Warrant) (string, error) {
t := *r.Take
recomposed := t.actions(r.ID)
option, offered := r.Ask.Option(w.Option)
if !offered || len(recomposed) != 1 {
return "", fmt.Errorf("the take %s offers no answer %q: nothing is taken", r.ID, w.Option)
}
if err := option.Performs(boundAct(recomposed[0])); err != nil {
return "", fmt.Errorf("the take kept for %s is not the one the operator was shown: %v", r.ID, err)
}
if err := option.Performs(boundAct(act)); err != nil {
return "", fmt.Errorf("the act chosen for %s is not the one the operator was shown: %v", r.ID, err)
}
if a.take == nil {
return "", errors.New("this controller cannot take a folder on a warrant")
}
// **The folder must still wait on the filesystem the operator was asked about** (the review of e1c2b744): a take
// is about no condition, so nothing else rechecks it. A folder that moved to another filesystem, or no longer
// waits for the operator's word, is not taken: the ask is spent, and one for the filesystem it is on now is asked.
if a.folders != nil {
waiting, err := a.folders(ctx)
if err != nil {
return "", fmt.Errorf("whether the folder still waits on %s could not be read, so nothing is taken: %w",
t.Node, err)
}
still := false
for _, now := range waiting {
still = still || now.key() == t.key()
}
if !still {
return "nothing: the folder no longer waits to be kept on the filesystem the operator was asked about " +
"(it moved, or was taken or released meanwhile); asked again where it waits now", nil
}
}
by := fmt.Sprintf("approved by %s via %s at %s (ask %s)", byWords(w), viaWords(w),
w.At.Local().Format("2006-01-02 15:04"), r.ID)
return a.take(ctx, t, r.ID, by)
}
// takeFolderIn is takeOnWarrant on this controller's stores and bus: the engine is asked with the mesh's key, naming the
// ask the warrant answered; on its word that it started, the folder is recorded taken. A refusal records nothing.
func takeFolderIn(open *stores, ask func(ctx context.Context, act link.AccessActAsk) (link.HandOverAnswer, error)) takeOnWarrant {
return func(ctx context.Context, t folderTake, askID, by string) (string, error) {
// Who asked, as one short line the engine takes (at most link.MaxHandOverBy): the ask names the rest.
short := by
if len(short) > 180 {
short = "the operator, on the warrant for ask " + askID
}
answer, err := ask(ctx, link.AccessActAsk{Node: t.Node, Path: filepath.Clean(t.Path), By: short,
Access: t.Access, Warrant: askID})
if err != nil {
return "", err
}
if answer.Refused != "" {
return "", fmt.Errorf("%s refused to take the folder: %s", t.Node, answer.Refused)
}
if err := open.inventory.RecordTaken(ctx, t.Node, inventory.TakenFolder{Path: t.Path, By: by, Ask: askID,
Warrant: askID, Said: answer.Said, Placed: t.Placed}); err != nil {
return "", fmt.Errorf("%s started the take (%s), and it could NOT be recorded: %w", t.Node, answer.Said, err)
}
return fmt.Sprintf("%s takes the folder for %s: %s", t.Node, strings.Join(t.Modules, ", "), answer.Said), nil
}
}
// waitingFolder is one access wait excused for a folder not yet taken, as foldersWaiting gathers them.
type waitingFolder struct {
module, access string
// resource is the access's resource id on the machine, as the engine stated it; placed the filesystem its folder
// is on, without its path.
resource, placed string
enablesACL bool
}
// foldersWaiting is every folder waiting to be kept across the mesh (ADR 0325 §7, §8): from each machine's newest
// statement, each access wait of a module that is excused (checkWaiting) and waits for the operator's word rather
// than for a disk, grouped by machine and folder.
func foldersWaiting(ctx context.Context, inv *inventory.Inventory, healths map[string]inventory.NodeHealth) (
map[string]map[string][]waitingFolder, operatorWaitFacts) {
var f operatorWaitFacts
out := map[string]map[string][]waitingFolder{}
machines := make([]string, 0, len(healths))
for m := range healths {
machines = append(machines, m)
}
sort.Strings(machines)
for _, machine := range machines {
rs := healths[machine].Resources
var modules []string
for _, r := range rs {
for _, w := range r.Waits {
if r.State == link.StateWaiting && w.Access != "" && !w.Disk {
modules = append(modules, r.Module)
}
}
}
if len(modules) == 0 {
continue
}
readWaitFacts(ctx, inv, machine, modules, nil, &f)
for _, r := range checkWaiting(machine, rs, f) {
if r.State != link.StateWaiting {
continue
}
for _, w := range r.Waits {
if w.Access == "" || w.Disk {
continue
}
path, placed := f.placed[r.Module+"@"+machine][w.Access]
if !placed {
continue
}
if out[machine] == nil {
out[machine] = map[string][]waitingFolder{}
}
out[machine][path] = append(out[machine][path], waitingFolder{module: r.Module, access: w.Access,
resource: r.Resource, placed: w.Placed, enablesACL: w.ACL == link.ACLNeedsEnabling})
}
}
}
return out, f
}
// foldersIn is the asker's folders on this controller's stores: every folder waiting to be kept, each with the
// modules waiting on it and the access the take names (the first waiting module's, by its resource id). Who the
// take adds entries for is what the engine was declared for that folder: the ask names the modules.
func foldersIn(open *stores) func(ctx context.Context) ([]folderTake, error) {
return func(ctx context.Context) ([]folderTake, error) {
healths, err := open.inventory.Healths(ctx)
if err != nil {
return nil, err
}
waiting, _ := foldersWaiting(ctx, open.inventory, healths)
return takesOf(waiting), nil
}
}
// takesOf is one take per machine and folder from the waits gathered, sorted.
func takesOf(waiting map[string]map[string][]waitingFolder) []folderTake {
var out []folderTake
for machine, folders := range waiting {
for path, all := range folders {
// One take per filesystem the waits name: they agree but for a moment while a disk is swapped.
byPlaced := map[string][]waitingFolder{}
for _, w := range all {
byPlaced[w.placed] = append(byPlaced[w.placed], w)
}
for placed, ws := range byPlaced {
t := folderTake{Node: machine, Path: path, Placed: placed}
seenM, seenA := map[string]bool{}, map[string]bool{}
var resources []string
for _, w := range ws {
if !seenM[w.module] {
seenM[w.module] = true
t.Modules = append(t.Modules, w.module)
}
if !seenA[w.access] {
seenA[w.access] = true
t.Accesses = append(t.Accesses, w.access)
}
resources = append(resources, w.resource)
t.EnablesACL = t.EnablesACL || w.enablesACL
}
sort.Strings(t.Modules)
sort.Strings(t.Accesses)
sort.Strings(resources)
t.Access = resources[0]
out = append(out, t)
}
}
}
sort.Slice(out, func(i, j int) bool { return out[i].key() < out[j].key() })
return out
}
+563
View File
@@ -0,0 +1,563 @@
package main
import (
"context"
"crypto/ed25519"
"encoding/json"
"errors"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/asks"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
)
// novox/hq ADR 0325 §7 and §8 on the controller's side: an access wait is excused only for a declared, placed folder
// not yet taken, or one whose disk is not there; the gate passes it as a wait for a person and the operator reads it
// as needs-operator in words with no path; the operator is asked once per folder and machine, never about a folder
// taken; and the folder is taken only on the warrant.
var radarrManifest = catalogue.Manifest{Module: "radarr", Version: "1",
Accesses: []catalogue.Access{{ID: "movies", Path: "/services/media/movies", Mode: catalogue.AccessReadWrite}}}
var plexManifest = catalogue.Manifest{Module: "plex", Version: "1",
Accesses: []catalogue.Access{{ID: "films", Path: "/services/media/movies", Mode: catalogue.AccessRead}}}
const moviesAt = "/storage/media/movies"
func keepWait(access string) inventory.Wait {
return inventory.Wait{Part: "radarr", Access: access, Reason: link.WaitKeep,
What: "let the mesh keep the shared folder " + access}
}
func placedWait(access, placed string) inventory.Wait {
w := keepWait(access)
w.Placed = placed
return w
}
func aclWait(access, acl string) inventory.Wait {
w := keepWait(access)
w.ACL = acl
return w
}
func diskWait(access string) inventory.Wait {
return inventory.Wait{Part: "radarr", Access: access, Disk: true, Reason: link.WaitDisk,
What: "mount the disk that holds " + access}
}
// accessFacts is the facts with each folder given taken on the filesystem fs-000000000001.
func accessFacts(taken ...string) operatorWaitFacts {
kept := map[string]string{}
for _, p := range taken {
kept[p] = "fs-000000000001"
}
return operatorWaitFacts{
manifests: map[string]catalogue.Manifest{"radarr": radarrManifest, "plex": plexManifest},
given: map[string]map[string]time.Time{"radarr@ace": {}, "plex@ace": {}},
placed: map[string]map[string]string{"radarr@ace": {"movies": moviesAt}, "plex@ace": {"films": moviesAt},
"radarr@novox": {}},
taken: map[string]map[string]string{"ace": kept},
}
}
// Row 7: excused for a declared, placed folder not yet taken, and for an absent folder whose disk is not there (taken
// or not); judged unhealthy when undeclared, unplaced, already taken, or naming a secret beside it. That a folder is
// root's is the engine's to see: it states that unhealthy and never waiting (§12).
func TestAnAccessWaitIsExcusedOnlyForAFolderTheMeshMayNotKeepYet(t *testing.T) {
for _, c := range []struct {
name string
machine string
w inventory.Wait
f operatorWaitFacts
says string
}{
{"declared, placed, not taken", "ace", keepWait("movies"), accessFacts(), ""},
{"its disk not there", "ace", diskWait("movies"), accessFacts(), ""},
{"its disk not there, the folder taken before", "ace", diskWait("movies"), accessFacts(moviesAt), ""},
{"already taken", "ace", keepWait("movies"), accessFacts(moviesAt), "taken on ace already"},
{"already taken on the filesystem it is on", "ace", placedWait("movies", "fs-000000000001"),
accessFacts(moviesAt), "taken on ace already"},
{"taken on another filesystem: it moved, and is asked about again", "ace",
placedWait("movies", "fs-0000000000ff"), accessFacts(moviesAt), ""},
{"undeclared", "ace", keepWait("books"), accessFacts(), "does not declare"},
{"unplaced", "novox", keepWait("movies"), accessFacts(), "not placed on novox"},
{"taken folders unread", "ace", keepWait("movies"), operatorWaitFacts{manifests: accessFacts().manifests,
placed: accessFacts().placed}, "could not be read"},
{"its filesystem needs ACLs enabled: asked", "ace", aclWait("movies", link.ACLNeedsEnabling), accessFacts(), ""},
{"its filesystem cannot carry ACLs: not asked", "ace", aclWait("movies", link.ACLUnsupported), accessFacts(),
"cannot carry access entries"},
{"a reason the controller does not know", "ace", inventory.Wait{Part: "radarr", Access: "movies",
Reason: "later", What: "w"}, accessFacts(), "waits for the operator's word to keep its folder or for its disk"},
{"an access and a secret", "ace", inventory.Wait{Part: "p", Access: "movies", Secret: "x", What: "w"},
accessFacts(), "one secret or one setting, or one access"},
} {
err := checkWait("radarr", c.machine, c.w, c.f)
switch {
case c.says == "" && err != nil:
t.Errorf("%s: refused: %v", c.name, err)
case c.says != "" && (err == nil || !strings.Contains(err.Error(), c.says)):
t.Errorf("%s: %v; want a refusal saying %q", c.name, err, c.says)
}
}
}
func accessWaiting(module, access string, w inventory.Wait) inventory.ResourceHealth {
return inventory.ResourceHealth{Module: module, Resource: module + "." + catalogue.AccessID("/services/media/movies"),
Kind: "access", Target: moviesAt, State: link.StateWaiting, Waits: []inventory.Wait{w}}
}
// Row 7, the gate and the condition: both excused waits pass the first-node gate as a wait for a person, and the
// module is said as needs-operator — in words that carry no path.
func TestAnAccessWaitPassesTheGateAndAsksForTheOperatorWithoutAPath(t *testing.T) {
now := time.Now()
for _, w := range []inventory.Wait{keepWait("movies"), diskWait("movies")} {
w.What += " at " + moviesAt // an engine's words carrying a path never reach the operator's
f := gateFacts{now: now, waits: accessFacts(), groupsAdded: map[string]bool{"radarr": false},
health: map[string]inventory.NodeHealth{"ace": {Node: "ace", HeardAt: now,
Resources: []inventory.ResourceHealth{accessWaiting("radarr", "movies", w)}}}}
h, why := moduleHealthWord("radarr", "ace", now.Add(-time.Minute), f)
if h != healthPerson || !strings.Contains(why, "waits for the operator") {
t.Fatalf("an excused access wait reads %v %q; want a wait for a person", h, why)
}
if strings.Contains(why, "/") {
t.Errorf("the gate's verdict carries a path: %q", why)
}
o := needsOperatorObservation("radarr", "ace", []inventory.Wait{w}, nil)
if o.Kind != kindNeedsOperator {
t.Fatalf("raised as %s", o.Kind)
}
for _, words := range []string{o.Summary, o.Headline, o.Explanation, o.Needs} {
if strings.Contains(words, "/") {
t.Errorf("the operator reads a path: %q", words)
}
}
want := "let the mesh keep the shared folder movies on ace when you are asked"
if w.Disk {
want = "mount the disk that holds movies on ace"
}
if !strings.Contains(o.Needs, want) {
t.Errorf("needs-operator asks %q; want %q", o.Needs, want)
}
}
// Taken already: not excused, and the gate does not pass it.
f := gateFacts{now: now, waits: accessFacts(moviesAt), groupsAdded: map[string]bool{"radarr": false},
health: map[string]inventory.NodeHealth{"ace": {Node: "ace", HeardAt: now,
Resources: []inventory.ResourceHealth{accessWaiting("radarr", "movies", keepWait("movies"))}}}}
if h, why := moduleHealthWord("radarr", "ace", now.Add(-time.Minute), f); h != healthNotYet ||
!strings.Contains(why, "taken") {
t.Fatalf("a wait on a taken folder reads %v %q", h, why)
}
}
// waitingOn is foldersWaiting's gathering for one folder: the given modules waiting on it by their access.
func waitingOn(machine string, ws ...waitingFolder) map[string]map[string][]waitingFolder {
return map[string]map[string][]waitingFolder{machine: {moviesAt: ws}}
}
func takeRig(t *testing.T, waiting func() map[string]map[string][]waitingFolder) (*askerRig, *[]string) {
r := newAskerRig(t)
var taken []string
r.a.folders = func(context.Context) ([]folderTake, error) { return takesOf(waiting()), nil }
r.a.machines = func(context.Context) ([]string, error) { return []string{"ace", "novox"}, nil }
r.a.take = func(_ context.Context, ft folderTake, askID, by string) (string, error) {
taken = append(taken, ft.Node+" "+ft.Path+" "+ft.Access+" "+askID+" "+by)
return "taking", nil
}
return r, &taken
}
func takeAsks(r *askerRig) []asked {
var out []asked
for _, a := range r.store {
if a.Take != nil {
out = append(out, a)
}
}
return out
}
// Row 8, the controller's part: one ask per folder and machine however many modules wait on it, in the clause's
// words; none again while it is open; taken back when the folder no longer waits; none for a folder taken (its waits
// are not excused, so it is never gathered).
func TestTheOperatorIsAskedOncePerFolderAndMachine(t *testing.T) {
both := waitingOn("ace",
waitingFolder{module: "radarr", access: "movies", resource: "radarr.access-services/media/movies"},
waitingFolder{module: "plex", access: "films", resource: "plex.access-services/media/movies"})
current := both
r, _ := takeRig(t, func() map[string]map[string][]waitingFolder { return current })
ctx := context.Background()
for i := 0; i < 3; i++ {
if err := r.a.reconcile(ctx); err != nil {
t.Fatal(err)
}
}
kept := takeAsks(r)
if len(kept) != 1 {
t.Fatalf("%d asks for one folder on one machine", len(kept))
}
q := kept[0].Ask
if err := q.Check(r.now); err != nil {
t.Fatal(err)
}
if q.Highest() != asks.Approve || len(q.Options) != 1 {
t.Fatalf("the ask offers %+v; want one answer at approve", q.Options)
}
// The path may not leave the mesh (the router's content rule): the folder is named by the shared folder it is.
want := "Let the mesh keep the shared folder films, movies on ace, now for plex, radarr: it adds their access " +
"entries, and those of any module given access later, to every folder and file there; the group bits `ls -l` " +
"shows become the entries' mask, and owners, groups and every other right stay as they are."
if q.Explanation != want {
t.Errorf("the ask says\n %q\nwant\n %q", q.Explanation, want)
}
if kept[0].Take.Access != "plex.access-services/media/movies" {
t.Errorf("the take names %q", kept[0].Take.Access)
}
// The folder no longer waits (taken, or no module uses it): the open ask is taken back, and nothing is asked.
current = nil
if err := r.a.reconcile(ctx); err != nil {
t.Fatal(err)
}
if got := takeAsks(r); len(got) != 1 || got[0].State != askCancelled {
t.Fatalf("the ask about a folder that no longer waits: %+v", got)
}
if err := r.a.reconcile(ctx); err != nil {
t.Fatal(err)
}
if len(takeAsks(r)) != 1 {
t.Fatal("asked about a folder nothing waits on")
}
// A module joining a folder already taken is never gathered: its wait is not excused.
f := accessFacts(moviesAt)
if err := checkWait("radarr", "ace", keepWait("movies"), f); err == nil {
t.Fatal("a wait on a taken folder was excused, and would have been asked about")
}
}
// Row 8: nothing is taken without the operator's warrant — an ask that ends without a choice, a warrant for another
// ask, or an act other than the one the option bound takes nothing; the warrant takes it once.
func TestAFolderIsTakenOnlyOnTheWarrant(t *testing.T) {
r, taken := takeRig(t, func() map[string]map[string][]waitingFolder {
return waitingOn("ace", waitingFolder{module: "radarr", access: "movies", resource: "radarr.access-movies"})
})
ctx := context.Background()
if err := r.a.reconcile(ctx); err != nil {
t.Fatal(err)
}
a := takeAsks(r)[0]
if len(*taken) != 0 {
t.Fatal("taken before any warrant")
}
// A warrant for another asker's ask.
other := warrantOn(a, "let-the-mesh-keep-it", r.now)
other.Asker = "someone-else"
answerWith(t, r, other)
// The record changed after the ask: the act no longer the one bound.
changed := r.store[a.ID]
changed.Take = &folderTake{Node: "ace", Path: "/", Access: "radarr.access-movies", Modules: a.Take.Modules}
r.store[a.ID] = changed
answerWith(t, r, warrantOn(a, "let-the-mesh-keep-it", r.now))
if len(*taken) != 0 || !strings.HasPrefix(r.store[a.ID].Acted, "failed") {
t.Fatalf("a take whose record changed after the ask: %v, %q", *taken, r.store[a.ID].Acted)
}
// Asked afresh, and the ask expires unanswered: nothing.
r2, taken2 := takeRig(t, func() map[string]map[string][]waitingFolder {
return waitingOn("ace", waitingFolder{module: "radarr", access: "movies", resource: "radarr.access-movies"})
})
if err := r2.a.reconcile(ctx); err != nil {
t.Fatal(err)
}
b := takeAsks(r2)[0]
expired := asks.Warrant{Ask: b.ID, Asker: "mesh-controller", Outcome: asks.OutcomeExpired, At: r2.now}
answerWith(t, r2, expired)
if len(*taken2) != 0 {
t.Fatal("taken on an ask that expired")
}
// The warrant: taken once, naming the access, the ask as the warrant, and who approved.
r3, taken3 := takeRig(t, func() map[string]map[string][]waitingFolder {
return waitingOn("ace", waitingFolder{module: "radarr", access: "movies", resource: "radarr.access-movies"})
})
if err := r3.a.reconcile(ctx); err != nil {
t.Fatal(err)
}
c := takeAsks(r3)[0]
w := warrantOn(c, "let-the-mesh-keep-it", r3.now)
answerWith(t, r3, w)
answerWith(t, r3, w)
if len(*taken3) != 1 || !strings.Contains((*taken3)[0], "ace "+moviesAt+" radarr.access-movies "+c.ID) ||
!strings.Contains((*taken3)[0], "telegram") {
t.Fatalf("the warrant took %v", *taken3)
}
if !strings.HasPrefix(r3.store[c.ID].Acted, "done") {
t.Fatalf("the take was recorded %q", r3.store[c.ID].Acted)
}
}
// The engine is asked only on a warrant, and the ask carries it: one with no warrant is never signed.
func TestATakeAskCarriesItsWarrantAndIsSigned(t *testing.T) {
_, key, _ := ed25519.GenerateKey(nil)
signer := keySigner{key}
ask := link.AccessActAsk{Node: "ace", Path: moviesAt, By: "the operator", Access: "radarr.access-services/media/movies"}
if _, err := link.SignTake(context.Background(), signer, ask); err == nil {
t.Fatal("a take with no warrant was signed")
}
ask.Warrant = "c0123456789abcdef"
body, err := link.SignTake(context.Background(), signer, ask)
if err != nil {
t.Fatal(err)
}
var signed link.SignedHandOver
if err := json.Unmarshal(body, &signed); err != nil {
t.Fatal(err)
}
if !ed25519.Verify(key.Public().(ed25519.PublicKey), append([]byte(link.AccessActContext), signed.Ask...), signed.Signature) {
t.Fatal("the take is not signed over its own context")
}
var got map[string]any
if err := json.Unmarshal(signed.Ask, &got); err != nil {
t.Fatal(err)
}
if got["act"] != "take" || got["warrant"] != "c0123456789abcdef" || got["access"] != ask.Access ||
got["path"] != moviesAt || got["nonce"] == "" {
t.Fatalf("the ask carries %v", got)
}
// And the controller records it taken only on the engine's word.
refusing := takeFolderIn(nil, func(context.Context, link.AccessActAsk) (link.HandOverAnswer, error) {
return link.HandOverAnswer{Refused: "the disk that holds the folder is not there"}, nil
})
if _, err := refusing(context.Background(), folderTake{Node: "ace", Path: moviesAt}, "c01", "the operator"); err == nil {
t.Fatal("a refused take was answered as done")
}
unreachable := takeFolderIn(nil, func(context.Context, link.AccessActAsk) (link.HandOverAnswer, error) {
return link.HandOverAnswer{}, errors.New("nothing on ace answers")
})
if _, err := unreachable(context.Background(), folderTake{Node: "ace", Path: moviesAt}, "c01", "the operator"); err == nil {
t.Fatal("an unanswered take was answered as done")
}
}
type keySigner struct{ key ed25519.PrivateKey }
func (s keySigner) Sign(_ context.Context, b []byte) ([]byte, error) {
return ed25519.Sign(s.key, b), nil
}
// The envelope carries `identities` only when composed — for an engine at link.IdentityContract — and an older
// engine's body is what it was.
func TestTheDeclarationSaysItsIdentitiesOnlyWhenComposed(t *testing.T) {
older, err := sendable{Resources: []map[string]any{{"id": "a"}}}.Body()
if err != nil {
t.Fatal(err)
}
if strings.Contains(string(older), "identities") {
t.Fatalf("an older engine's body names identities: %s", older)
}
newer, err := sendable{Resources: []map[string]any{{"id": "a"}}, Identities: []map[string]any{
{"module": "radarr", "name": "mesh-radarr", "uid": 50000, "gid": 50000}}}.Body()
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(newer), `"identities":[{"gid":50000,"module":"radarr","name":"mesh-radarr","uid":50000}]`) {
t.Fatalf("the body: %s", newer)
}
if link.IdentityContract != 5 {
t.Fatalf("the contract the engine states for ADR 0325 is 5, and this controller reads %d", link.IdentityContract)
}
}
// A folder that moved to another filesystem is asked about again: one ask per folder, machine and filesystem, and the
// take records the filesystem it was for.
func TestAFolderThatMovedIsAskedAboutAgain(t *testing.T) {
placed := "fs-000000000001"
r, taken := takeRig(t, func() map[string]map[string][]waitingFolder {
return waitingOn("ace", waitingFolder{module: "radarr", access: "movies", resource: "radarr.access-movies",
placed: placed})
})
ctx := context.Background()
if err := r.a.reconcile(ctx); err != nil {
t.Fatal(err)
}
placed = "fs-0000000000ff"
if err := r.a.reconcile(ctx); err != nil {
t.Fatal(err)
}
var open []asked
for _, a := range takeAsks(r) {
if a.State == askOpen {
open = append(open, a)
}
}
if len(open) != 1 || open[0].Take.Placed != "fs-0000000000ff" || len(takeAsks(r)) != 2 {
t.Fatalf("after the folder moved: %d asks, open %+v", len(takeAsks(r)), open)
}
answerWith(t, r, warrantOn(open[0], "let-the-mesh-keep-it", r.now))
if len(*taken) != 1 {
t.Fatalf("taken %v", *taken)
}
}
// The mesh's entries left while a disk was away (kind kept-entries) do not hold a delivery whose move leaves the
// module's accesses as they were — they may name an access no longer declared; one that changes them is held.
func TestEntriesLeftWhileADiskWasAwayDoNotHoldADeliveryThatDidNotBringThem(t *testing.T) {
now := time.Now()
since := now.Add(-time.Minute)
left := inventory.ResourceHealth{Module: "radarr", Resource: "radarr.access-gone~entries", Kind: link.KindKeptEntries,
Target: moviesAt, State: link.StateUnhealthy, Reason: "the mesh's entries could not be taken out: its disk is away",
Since: since.Add(-time.Hour)}
f := gateFacts{now: now, waits: accessFacts(), groupsAdded: map[string]bool{"radarr": false},
accessesChanged: map[string]bool{"radarr": false}, health: map[string]inventory.NodeHealth{"ace": {Node: "ace",
HeardAt: now, Resources: []inventory.ResourceHealth{left}}}}
if h, why := moduleHealthWord("radarr", "ace", since, f); h != healthPerson || !strings.Contains(why, "access-gone~entries") {
t.Fatalf("entries left before the send read %v %q; want carried in a passing verdict", h, why)
}
left.Since = now
f.health["ace"] = inventory.NodeHealth{Node: "ace", HeardAt: now, Resources: []inventory.ResourceHealth{left}}
if h, _ := moduleHealthWord("radarr", "ace", since, f); h != healthPerson {
t.Fatalf("entries stated since the send by a move that leaves accesses as they were read %v", h)
}
f.accessesChanged = nil // a move not known changes them
if h, _ := moduleHealthWord("radarr", "ace", since, f); h != healthNotYet {
t.Fatalf("entries of a move not known read %v; want judged unhealthy", h)
}
}
// An access the engine reports unknown (its filesystem could not be named this look) carries no wait: of a module
// whose move leaves its accesses as they were it is pre-existing and not counted; of one whose move changes them, or
// whose move is not known, it is not yet healthy — held to the gate's bound, never waited on for ever. Entries left
// while a disk was away follow the same rule whenever they were stated.
func TestAnUnknownAccessAndLeftEntriesAreJudgedByWhetherTheMoveChangesAccesses(t *testing.T) {
now := time.Now()
since := now.Add(-time.Minute)
unknown := inventory.ResourceHealth{Module: "radarr", Resource: "radarr.access-movies", Kind: link.KindAccess,
Target: moviesAt, State: link.StateUnknown, Since: now}
left := inventory.ResourceHealth{Module: "radarr", Resource: "radarr.access-gone~entries", Kind: link.KindKeptEntries,
Target: moviesAt, State: link.StateUnhealthy, Since: now}
facts := func(changes map[string]bool, rs ...inventory.ResourceHealth) gateFacts {
return gateFacts{now: now, waits: accessFacts(), groupsAdded: map[string]bool{"radarr": false},
accessesChanged: changes, health: map[string]inventory.NodeHealth{"ace": {Node: "ace", HeardAt: now,
Resources: rs}}}
}
unchanged := map[string]bool{"radarr": false}
if h, why := moduleHealthWord("radarr", "ace", since, facts(unchanged, unknown)); h != healthGood {
t.Errorf("an unknown access of a move that leaves accesses as they were reads %v %q; want not counted", h, why)
}
for name, changes := range map[string]map[string]bool{"changes them": {"radarr": true}, "not known": nil} {
if h, _ := moduleHealthWord("radarr", "ace", since, facts(changes, unknown)); h != healthNotYet {
t.Errorf("an unknown access of a move that %s reads %v; want not yet, held to the bound", name, h)
}
}
if h, why := moduleHealthWord("radarr", "ace", since, facts(unchanged, left)); h != healthPerson ||
!strings.Contains(why, "access-gone~entries") {
t.Errorf("entries left, a move that leaves accesses as they were: %v %q; want carried", h, why)
}
if h, _ := moduleHealthWord("radarr", "ace", since, facts(map[string]bool{"radarr": true}, left)); h != healthNotYet {
t.Errorf("entries left since the send by a move that changes accesses read %v", h)
}
if !sameAccesses(radarrManifest.Accesses, []catalogue.Access{{ID: "movies", Path: "/services/media/movies",
Mode: catalogue.AccessReadWrite}}) || sameAccesses(radarrManifest.Accesses, plexManifest.Accesses) {
t.Error("sameAccesses does not tell a changed access from an unchanged one")
}
}
// A take whose folder's filesystem must first allow access entries says so in the ask (ADR 0325 §8).
func TestTheAskSaysWhenTheTakeEnablesACLs(t *testing.T) {
r, _ := takeRig(t, func() map[string]map[string][]waitingFolder {
return waitingOn("ace", waitingFolder{module: "radarr", access: "movies", resource: "radarr.access-movies",
enablesACL: true})
})
if err := r.a.reconcile(context.Background()); err != nil {
t.Fatal(err)
}
q := takeAsks(r)[0].Ask
if !strings.Contains(q.Explanation, "does not allow access entries yet (a ZFS dataset's acltype): approving sets that first") {
t.Fatalf("the ask says %q", q.Explanation)
}
if err := q.Check(r.now); err != nil {
t.Fatal(err)
}
}
// The engine's access waits are kept by their reason, never by their words, and a kept folder's entries are shown.
func TestTheEnginesAccessWaitsAndEntriesAreKept(t *testing.T) {
lines := healthLines(inventory.NodeHealth{Node: "ace", HeardAt: time.Now(), Resources: []inventory.ResourceHealth{{
Module: "radarr", Resource: "radarr.access-movies", Kind: link.KindAccess, Target: moviesAt,
State: link.StateHealthy, Entries: []inventory.KeptEntry{{Identity: "mesh-radarr", UID: 50000, Use: "read-write",
From: "container"}, {Identity: "jochen", UID: 1000, Use: "read-write", From: "tool-runner",
Note: "the tool runner's account stands for every module whose tools use this folder"}}}}}, true, time.Now())
all := strings.Join(lines, "\n")
if !strings.Contains(all, "entry mesh-radarr (uid 50000, read-write, from the container)") ||
!strings.Contains(all, "entry jochen (uid 1000, read-write, from the tool-runner) — the tool runner's account") {
t.Fatalf("the listing reads:\n%s", all)
}
}
// Review of e1c2b744: a warrant for a folder on one filesystem never takes it on another. When the warrant comes and
// the folder no longer waits on the filesystem the ask named, nothing is sent, the ask is spent, and the operator is
// asked again for the filesystem it is on now.
func TestAWarrantForOneFilesystemIsNotCarriedOutOnAnother(t *testing.T) {
placed := "fs-000000000001"
r, taken := takeRig(t, func() map[string]map[string][]waitingFolder {
return waitingOn("ace", waitingFolder{module: "radarr", access: "movies", resource: "radarr.access-movies",
placed: placed})
})
ctx := context.Background()
if err := r.a.reconcile(ctx); err != nil {
t.Fatal(err)
}
a := takeAsks(r)[0]
placed = "fs-0000000000ff" // the folder moved before the operator answered
answerWith(t, r, warrantOn(a, "let-the-mesh-keep-it", r.now))
if len(*taken) != 0 {
t.Fatalf("a warrant for %s was carried out on %s: %v", a.Take.Placed, placed, *taken)
}
if spent := r.store[a.ID]; spent.State == askOpen || !strings.HasPrefix(spent.Acted, "nothing") {
t.Fatalf("the ask after the folder moved: %s, %q", spent.State, spent.Acted)
}
if err := r.a.reconcile(ctx); err != nil {
t.Fatal(err)
}
var again []asked
for _, x := range takeAsks(r) {
if x.State == askOpen {
again = append(again, x)
}
}
if len(again) != 1 || again[0].Take.Placed != placed {
t.Fatalf("not asked again for the filesystem the folder is on now: %+v", again)
}
}
// Review of e1c2b744: a move that changes who uses an access — a container of the module that starts declaring
// identity — changes the access, so an access the engine cannot judge is not passed uncounted.
func TestAMoveThatChangesAnAccesssUsersChangesTheAccess(t *testing.T) {
before := catalogue.Manifest{Module: "radarr", Accesses: radarrManifest.Accesses, Resources: []map[string]any{
{"id": "app", "type": "container", "volumes": []any{"${access:movies}:/movies"}}}}
after := catalogue.Manifest{Module: "radarr", Accesses: radarrManifest.Accesses, Resources: []map[string]any{
{"id": "app", "type": "container", "identity": map[string]any{"user": "run"},
"volumes": []any{"${access:movies}:/movies"}}}}
if sameAccessUse(before, after) {
t.Fatal("a move that adds identity to the container mounting the access left it the same")
}
if !sameAccessUse(before, before) {
t.Fatal("an unchanged manifest reads changed")
}
// Entries left while a disk was away, stated before the send, of a move that changes the access: counted.
now := time.Now()
since := now.Add(-time.Minute)
left := inventory.ResourceHealth{Module: "radarr", Resource: "radarr.access-movies~entries", Kind: link.KindKeptEntries,
Target: moviesAt, State: link.StateUnhealthy, Since: since.Add(-time.Hour)}
f := gateFacts{now: now, waits: accessFacts(), groupsAdded: map[string]bool{"radarr": false},
accessesChanged: map[string]bool{"radarr": true}, health: map[string]inventory.NodeHealth{"ace": {Node: "ace",
HeardAt: now, Resources: []inventory.ResourceHealth{left}}}}
if h, _ := moduleHealthWord("radarr", "ace", since, f); h != healthNotYet {
t.Fatalf("entries left before the send by a move that changes the access read %v; want counted", h)
}
}
+398
View File
@@ -0,0 +1,398 @@
package main
import (
"context"
"errors"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
// The tests of novox/hq issue 525's window skip: a merge waits for the merge window only when a batch assembling
// holds a merge related to it, or its delivery group may have a member still to merge (ADR 0304 decision 1, ADR
// 0306 decisions 2 and 3). Walks stay one at a time.
// A merge from no head that shares nothing, with no walk open, is cut into its walk when it is heard: no window.
func TestALoneMergeSharingNothingIsCutAtOnce(t *testing.T) {
open := windowed(t)
asksWithPaths(t)
hear(t, open, lone(repoMerge("one", "c1", t0)), t0)
ws, bs := walks(t, open)
if len(ws) != 1 || len(bs) != 0 || !ws[0].Open() || ws[0].Commit != "c1" {
t.Fatalf("a lone merge sharing nothing: %d walk(s) %+v, %d batch(es)", len(ws), ws, len(bs))
}
if ws[0].Times == nil || ws[0].Times.WindowClosed == nil || !ws[0].Times.WindowClosed.Equal(t0) {
t.Fatalf("its window closed at %+v; want when it was heard, %s", ws[0].Times, t0)
}
}
// A merge related to the batch assembling joins it and waits for the window, which its coming restarts: by its
// repository, by a module in common (a dependent of what the batch moves), by an edge of the module graph between
// their modules (one that widens nothing), or by its delivery group's head.
func TestARelatedMergeJoinsTheBatchAndWaitsForTheWindow(t *testing.T) {
ctx := t.Context()
for _, c := range []string{"same repository", "a module in common", "an edge of the graph", "the same group head"} {
t.Run(c, func(t *testing.T) {
open := windowed(t)
groupsWaiting(t, "pr/c1")
asksWithPaths(t)
inv := open.inventory
first := repoMerge("one", "c1", t0)
second := lone(repoMerge("two", "c2", t0.Add(10*time.Second)))
why := map[string]string{"same repository": "both change novox/one", "a module in common": "both move two",
"an edge of the graph": "two depends on builder (built-by)",
"the same group head": "both are members of group " + first.Head}[c]
switch c {
case "same repository":
second = lone(repoMerge("one", "c2", t0.Add(10*time.Second)))
case "a module in common":
// two is built on one: a merge of one moves two too.
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: "two", Version: "1",
Build: &catalogue.Build{On: []catalogue.BuildsOn{{Arg: "BASE", Module: "one"}}}},
inventory.Source{Repository: "novox/two", Seat: "git", Ref: "main", BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
case "an edge of the graph":
// two is built by the build machine: an edge that orders and widens nothing.
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: "builder", Version: "1",
Claims: []catalogue.Claim{{Name: "mesh-build-machine", Scope: catalogue.ScopeMesh}}},
inventory.Source{Repository: "novox/builder", Seat: "git", Ref: "main", BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
first = repoMerge("builder", "c1", t0)
case "the same group head":
second = fromHead(repoMerge("two", "c2", t0.Add(10*time.Second)), first.Head)
}
hear(t, open, first, t0)
said := captured(t, func() error {
return (following{open: open}).hearMerge(ctx, second, t0.Add(10*time.Second))
})
if !strings.Contains(said, "("+why+")") {
t.Fatalf("the merge was not said related by %q: %s", why, said)
}
ws, bs := walks(t, open)
if len(ws) != 0 || len(bs) != 1 || len(bs[0].Delivery.Merges) != 2 {
t.Fatalf("the related merge: %d walk(s), batches %+v", len(ws), bs)
}
if bs[0].Delivery.Batch.CutAtOnce {
t.Fatal("a batch a related merge joined is cut at once")
}
cutAt(t, open, t0.Add(99*time.Second))
if ws, _ := walks(t, open); len(ws) != 0 {
t.Fatalf("cut before the window the related merge restarted closed: %+v", ws)
}
cutAt(t, open, t0.Add(100*time.Second))
ws, bs = walks(t, open)
if len(ws) != 1 || len(bs) != 0 || len(ws[0].Delivery.Merges) != 2 {
t.Fatalf("after the window: %d walk(s), %d batch(es)", len(ws), len(bs))
}
})
}
}
// A merge from a head whose delivery group has a member still to merge waits for the window, even with no batch
// assembling, and says why; its sibling joins the same batch by their head (ADR 0306 decision 2). A merge from a
// head that names no group shares nothing and is cut at once, as one from no head is.
func TestAMergeOfAGroupWaitsEvenAlone(t *testing.T) {
open := windowed(t)
asksWithPaths(t)
groupsWaiting(t, "feat/x")
hear(t, open, fromHead(repoMerge("one", "c1", t0), "feat/x"), t0)
ws, bs := walks(t, open)
if len(ws) != 0 || len(bs) != 1 || bs[0].Delivery.Batch.CutAtOnce || windowClosed(bs[0], t0) {
t.Fatalf("a group's merge alone: %d walk(s), batches %+v", len(ws), bs)
}
if words := batchWords(bs[0], t0); !strings.HasPrefix(words, "assembling: 90 s left") ||
!strings.Contains(words, "group feat/x has a member still to merge: novox/elsewhere (ready)") {
t.Fatalf("it reads %q", words)
}
hear(t, open, fromHead(repoMerge("two", "c2", t0.Add(5*time.Second)), "feat/x"), t0.Add(5*time.Second))
if _, bs := walks(t, open); len(bs) != 1 || len(bs[0].Delivery.Merges) != 2 {
t.Fatalf("the group's second member did not join the first: %+v", bs)
}
cutAt(t, open, t0.Add(94*time.Second))
if ws, _ := walks(t, open); len(ws) != 0 {
t.Fatalf("cut before its window closed: %+v", ws)
}
cutAt(t, open, t0.Add(95*time.Second))
ws, bs = walks(t, open)
if len(ws) != 1 || len(bs) != 0 || ws[0].CommitOf("novox/one") != "c1" || ws[0].CommitOf("novox/two") != "c2" {
t.Fatalf("at its window: %d walk(s) %+v, %d batch(es)", len(ws), ws, len(bs))
}
}
// A merge from a head that names no group, sharing nothing, with no walk open, is cut at once.
func TestAHeadedMergeInNoGroupIsCutAtOnce(t *testing.T) {
open := windowed(t)
asksWithPaths(t)
hear(t, open, fromHead(repoMerge("one", "c1", t0), "fix/alone"), t0)
if ws, bs := walks(t, open); len(ws) != 1 || len(bs) != 0 || !ws[0].Open() {
t.Fatalf("a headed merge in no group: %d walk(s), %d batch(es)", len(ws), len(bs))
}
}
// When mesh-delivery cannot say a head's group, the merge waits for the window, the safe side, and its batch says
// why.
func TestAGroupNobodyCanSayWaits(t *testing.T) {
open := windowed(t)
asksWithPaths(t)
was := deliveryGroupOf
deliveryGroupOf = func(context.Context, bool, string) ([]groupMember, bool, error) {
return nil, false, errors.New("nothing serves mesh-delivery.groups")
}
t.Cleanup(func() { deliveryGroupOf = was })
hear(t, open, fromHead(repoMerge("one", "c1", t0), "feat/x"), t0)
ws, bs := walks(t, open)
if len(ws) != 0 || len(bs) != 1 || bs[0].Delivery.Batch.CutAtOnce {
t.Fatalf("a merge whose group nobody could say: %d walk(s), batches %+v", len(ws), bs)
}
if words := batchWords(bs[0], t0); !strings.Contains(words, "mesh-delivery did not say whether group feat/x has a "+
"member still to merge (nothing serves mesh-delivery.groups): it waits for the window, the safe side") {
t.Fatalf("it reads %q", words)
}
}
// A member counts as still to merge only while its pull request is open, and not once this controller keeps its
// merge: a group whose other members are merged or closed makes nobody wait.
func TestAGroupsMergedOrClosedMembersMakeNobodyWait(t *testing.T) {
open := windowed(t)
asksWithPaths(t)
ctx := t.Context()
members := []groupMember{{Repository: "novox/one", State: "ready"}, {Repository: "novox/two", State: "published"},
{Repository: "novox/three", State: "stopped"}}
was := deliveryGroupOf
deliveryGroupOf = func(context.Context, bool, string) ([]groupMember, bool, error) { return members, true, nil }
t.Cleanup(func() { deliveryGroupOf = was })
m := fromHead(repoMerge("one", "c1", t0), "feat/x")
if why, err := groupWait(ctx, open.inventory, m, true, false); err != nil || why != "" {
t.Fatalf("the group's other members are merged or closed, yet: %q %v", why, err)
}
members = append(members, groupMember{Repository: "novox/four", State: "checked"})
if why, _ := groupWait(ctx, open.inventory, m, true, false); !strings.Contains(why, "novox/four (checked)") {
t.Fatalf("an open member: %q", why)
}
// novox/four's merge is kept in a batch already: merged, whatever mesh-delivery says yet.
hear(t, open, fromHead(repoMerge("two", "c4", t0), "feat/x"), t0)
members[3].Repository = "novox/two"
if why, _ := groupWait(ctx, open.inventory, m, true, false); why != "" {
t.Fatalf("a member this controller keeps the merge of still waits: %q", why)
}
if got := memberOf("novox/mesh-sdk@d41d255f28ad (held)"); got != (groupMember{Repository: "novox/mesh-sdk", State: "held"}) {
t.Fatalf("a member reads %+v", got)
}
}
// A merge related to two batches assembling folds the newer into the older, which answers every merge of both
// and names nothing twice; the folded one names the batch that took it over.
func TestAMergeRelatedToTwoBatchesFoldsThem(t *testing.T) {
open := windowed(t)
groupsWaiting(t, "feat/a", "feat/b")
asksWithPaths(t)
ctx := t.Context()
hear(t, open, fromHead(repoMerge("one", "c1", t0), "feat/a"), t0)
hear(t, open, fromHead(repoMerge("two", "c2", t0.Add(5*time.Second)), "feat/b"), t0.Add(5*time.Second))
_, bs := walks(t, open)
if len(bs) != 2 {
t.Fatalf("two merges sharing nothing: %d batch(es)", len(bs))
}
older, newer := bs[1], bs[0] // newest first
// Of one's repository and from two's head: related to both.
hear(t, open, fromHead(repoMerge("one", "c3", t0.Add(10*time.Second)), "feat/b"), t0.Add(10*time.Second))
_, bs = walks(t, open)
if len(bs) != 1 || bs[0].ID != older.ID || len(bs[0].Delivery.Merges) != 3 {
t.Fatalf("after a merge related to both: batches %+v; want %s answering three merges", bs, older.ID)
}
folded, err := open.inventory.PlanByID(ctx, newer.ID)
if err != nil || folded.State != inventory.PlanSuperseded || folded.Delivery == nil ||
folded.Delivery.TakenOverBy != older.ID {
t.Fatalf("the newer batch reads %s %+v (%v)", folded.State, folded.Delivery, err)
}
if left, _ := open.inventory.MergesOf(ctx, newer.ID); len(left) != 0 {
t.Fatalf("the folded batch still answers %+v", left)
}
cutAt(t, open, t0.Add(100*time.Second))
ws, bs := walks(t, open)
var cut []inventory.Plan
for _, w := range ws {
if w.State != inventory.PlanSuperseded {
cut = append(cut, w)
}
}
if len(cut) != 1 || len(bs) != 0 || cut[0].ID != older.ID || cut[0].CommitOf("novox/one") != "c3" ||
cut[0].CommitOf("novox/two") != "c2" {
t.Fatalf("the folded batch's walk: %+v, %d batch(es)", cut, len(bs))
}
}
// With a walk open, a batch cut at once still waits for it, said queued behind it (walks stay one at a time, ADR
// 0276 decision 4); it is cut once that walk ended, with no window to wait for.
func TestACutAtOnceBatchQueuesBehindAnOpenWalk(t *testing.T) {
open := windowed(t)
groupsWaiting(t, "pr/c1")
asksWithPaths(t)
hear(t, open, repoMerge("one", "c1", t0), t0)
cutAt(t, open, t0.Add(90*time.Second))
ws, _ := walks(t, open)
if len(ws) != 1 || !ws[0].Open() {
t.Fatalf("no walk open: %+v", ws)
}
first := ws[0]
heard := t0.Add(100 * time.Second)
hear(t, open, lone(repoMerge("two", "c2", heard)), heard)
ws, bs := walks(t, open)
if len(ws) != 1 || len(bs) != 1 || bs[0].State != inventory.PlanQueued || bs[0].Delivery.Batch.Behind != first.ID ||
!bs[0].Delivery.Batch.CutAtOnce {
t.Fatalf("a lone merge during a walk: %d walk(s), batches %+v", len(ws), bs)
}
if !strings.HasPrefix(batchWords(bs[0], heard), "queued behind "+first.ID) {
t.Fatalf("it reads %q", batchWords(bs[0], heard))
}
first.State = inventory.PlanDone
if err := open.inventory.SavePlan(t.Context(), &first); err != nil {
t.Fatal(err)
}
cutAt(t, open, heard.Add(time.Second))
ws, bs = walks(t, open)
if len(ws) != 2 || len(bs) != 0 || ws[0].Commit != "c2" {
t.Fatalf("the queued batch was not cut once the walk ended: %+v %+v", ws, bs)
}
if ws[0].Times == nil || ws[0].Times.WindowClosed == nil || !ws[0].Times.WindowClosed.Equal(heard) {
t.Fatalf("its window closed at %+v; want when its merge was heard, %s", ws[0].Times, heard)
}
}
// A stopped walk's other merges are walked again in a batch they are related to, and a batch of their own when
// they share nothing with the one assembling (ADR 0299, ADR 0304 decision 1): never in a batch of another
// change, whose window the stop would close.
func TestAStopWalksTheOthersAgainInABatchOfTheirOwnWhenTheyShareNothing(t *testing.T) {
open := windowed(t)
groupsWaiting(t, "pr/cat1aaaaaaaa")
endsItsLine(t)
asksRecorded(t)
ctx := t.Context()
cat := catalogueMerge("cat1aaaaaaaa", "app", t0)
one := fromHead(repoMerge("one", "one1bbbbbbbb", t0.Add(5*time.Second)), cat.Head)
hear(t, open, cat, t0.Add(time.Second))
hear(t, open, one, t0.Add(6*time.Second))
cutAt(t, open, t0.Add(2*time.Minute))
ws, _ := walks(t, open)
walk := ws[0]
hear(t, open, repoMerge("two", "two1cccccccc", t0.Add(3*time.Minute)), t0.Add(3*time.Minute))
_, bs := walks(t, open)
if len(bs) != 1 {
t.Fatalf("the merge heard while the walk ran is in %d batches", len(bs))
}
other := bs[0]
_, said, err := stopWalkWithout(ctx, open.inventory, walk.ID, "mesh-delivery for jochen", "its group stopped",
[]string{"novox/mesh-catalog@" + cat.Commit}, t0.Add(3*time.Minute+10*time.Second))
if err != nil {
t.Fatal(err)
}
if said.WalkedAgainBy == "" || said.WalkedAgainBy == other.ID {
t.Fatalf("walked again by %q; want a batch of its own, not %s", said.WalkedAgainBy, other.ID)
}
b, _ := open.inventory.PlanByID(ctx, other.ID)
if b.Delivery.Batch == nil || b.Delivery.Batch.Closed || len(b.Delivery.Merges) != 1 {
t.Fatalf("the other change's batch reads %+v; want it untouched by the stop", b.Delivery)
}
}
// A catalogue batch's cut folds in every walk waiting for its delivery's word (cutBatch's carry), so what that
// walk moves is the batch's too: a lone merge related to the waiting walk joins the batch assembling and waits for
// its window, rather than being cut at once as sharing nothing with what the batch's own merges move.
func TestALoneMergeRelatedToAWalkTheCutFoldsJoinsTheBatch(t *testing.T) {
open := windowed(t)
groupsWaiting(t, "pr/c3")
asksWithPaths(t)
ctx := t.Context()
if err := open.inventory.RegisterModule(ctx, catalogue.Manifest{Module: "mesh-delivery", Version: "1",
Claims: []catalogue.Claim{{Name: catalogue.DeliverySeat, Scope: catalogue.ScopeMesh}}},
inventory.Source{Repository: "novox/mesh-catalog", Seat: "git", Path: "modules/mesh-delivery", Ref: "main",
BuiltFrom: "c0", Head: "c0"}); err != nil {
t.Fatal(err)
}
if _, err := open.inventory.Assign(ctx, "anchor", "mesh-delivery"); err != nil {
t.Fatal(err)
}
hear(t, open, repoMerge("one", "c1", t0), t0)
cutAt(t, open, t0.Add(90*time.Second))
if ws, _ := walks(t, open); len(ws) != 1 || !ws[0].Waiting() {
t.Fatalf("the walk does not wait for its word: %+v", ws)
}
hear(t, open, repoMerge("three", "c3", t0.Add(2*time.Minute)), t0.Add(2*time.Minute))
hear(t, open, lone(repoMerge("one", "c2", t0.Add(2*time.Minute+5*time.Second))), t0.Add(2*time.Minute+5*time.Second))
_, bs := walks(t, open)
if len(bs) != 1 || len(bs[0].Delivery.Merges) != 2 || bs[0].Delivery.Batch.CutAtOnce {
t.Fatalf("a lone merge related to the waiting walk: batches %+v", bs)
}
}
// A delivery group with a member of each kind (review of #250): a merge on the controller's own path and a
// catalogue merge never share a batch, so the later member cannot join the earlier one's. Kept in a batch of the
// other kind, the earlier member keeps the later one waiting, whatever mesh-delivery says yet — never cut at once
// to walk before it. The two batches are then cut in the order their windows close (cutBatchesHeld: the oldest
// closed batch first, an own-path one first at one look, and the other queued behind a walk that started): the
// earlier member's window closes first, so it walks first, in both orders.
func TestAGroupAcrossBothKindsKeepsItsOrder(t *testing.T) {
for _, ownFirst := range []bool{true, false} {
name := "catalogue member first"
if ownFirst {
name = "controller member first"
}
t.Run(name, func(t *testing.T) {
open := windowed(t)
asksWithPaths(t)
ctx := t.Context()
was := deliveryGroupOf
// mesh-delivery still says both members ready: it marks one merged only once its walk opened.
deliveryGroupOf = func(context.Context, bool, string) ([]groupMember, bool, error) {
return []groupMember{{Repository: "novox/mesh-controller", State: "ready"},
{Repository: "novox/mesh-catalog", State: "ready"}}, true, nil
}
t.Cleanup(func() { deliveryGroupOf = was })
own := fromHead(repoMerge("mesh-controller", "k1", t0), "feat/both-kinds")
cat := fromHead(catalogueMerge("cat1aaaaaaaa", "app", t0), "feat/both-kinds")
first, second := cat, own
if ownFirst {
first, second = own, cat
}
second.MergedAt = t0.Add(10 * time.Second).Format(time.RFC3339Nano)
hear(t, open, first, t0)
hear(t, open, second, t0.Add(10*time.Second))
ws, bs := walks(t, open)
if len(ws) != 0 || len(bs) != 2 {
t.Fatalf("both members heard: %d walk(s), %d batch(es); want two batches, nothing cut", len(ws), len(bs))
}
later := bs[0] // newest first
if later.Delivery.Batch.CutAtOnce || !strings.Contains(later.Delivery.Batch.Waits,
"has a member waiting in a batch of the other kind: novox/"+first.Repo+"@") {
t.Fatalf("the later member's batch reads %+v; want it waiting for the earlier member", later.Delivery.Batch)
}
// The earlier member's window closes first: its batch alone is cut.
cutAt(t, open, t0.Add(90*time.Second))
ws, bs = walks(t, open)
if len(ws) != 1 || !ws[0].Open() || ws[0].CommitOf("novox/"+first.Repo) != first.Commit || len(bs) != 1 ||
bs[0].ID != later.ID {
t.Fatalf("at the earlier window: walks %+v, batches %+v; want the earlier member alone cut", ws, bs)
}
earlier := ws[0]
// The later member's window closes while the earlier walk runs: it queues behind it.
cutAt(t, open, t0.Add(100*time.Second))
if _, bs := walks(t, open); len(bs) != 1 || bs[0].State != inventory.PlanQueued || bs[0].Delivery.Batch.Behind != earlier.ID {
t.Fatalf("the later member's batch does not queue behind the earlier walk: %+v", bs)
}
earlier.State = inventory.PlanDone
if err := open.inventory.SavePlan(ctx, &earlier); err != nil {
t.Fatal(err)
}
cutAt(t, open, t0.Add(101*time.Second))
got, err := open.inventory.PlanByID(ctx, later.ID)
if err != nil || got.Batch() || got.CommitOf("novox/"+second.Repo) != second.Commit {
t.Fatalf("the later member was not walked after the earlier: %s %v", got.State, err)
}
})
}
}
+9
View File
@@ -63,6 +63,15 @@ func run() error {
// 45 §6), so the next controller takes it at once rather than after its age.
defer theLease.release()
// Every walk a verb keeps is said on the bus, whichever verb kept it (novox/hq ADR 0308 decision 1): the save
// that starts or ends a walk keeps its mark, and no later save says it again. The serving controller says its
// own saves.
if args[0] != "serve" {
said, done := sayingWhatAVerbKept(ctx)
inventory.PlanSaved = said
defer done()
}
switch args[0] {
case "build":
return buildCommand(ctx, args[1:])
+42 -2
View File
@@ -77,6 +77,16 @@ func stateHealth(ctx context.Context, inv *inventory.Inventory, k *conditions.Ke
for _, w := range r.Waits {
kept.Waits = append(kept.Waits, inventory.Wait{Part: w.Part, Secret: w.Secret, Setting: w.Setting, What: w.What})
}
// An access's waits (novox/hq ADR 0325 §7), kept beside the others as one more thing the operator gives: the
// access by its module's id, and whether it waits for a disk rather than for the operator's word.
for _, w := range r.AccessWaits {
kept.Waits = append(kept.Waits, inventory.Wait{Part: w.Part, Access: w.Access, Disk: w.Reason == link.WaitDisk,
Placed: w.Placed, Reason: w.Reason, ACL: w.ACL, What: w.What})
}
for _, e := range r.Entries {
kept.Entries = append(kept.Entries, inventory.KeptEntry{Identity: e.Identity, UID: e.UID, Use: e.Use, From: e.From,
Note: e.Note})
}
resources = append(resources, kept)
}
// **A wait for the operator is checked before it is excused** (novox/hq ADR 0283): a waiting resource whose
@@ -644,7 +654,7 @@ func moduleHealthWord(module, machine string, since time.Time, f gateFacts) (hea
if waits && !f.groupsAdded[module] {
waits = false
}
var found, onWaiting []string
var found, onWaiting, leftover []string
var forOperator []inventory.Wait
for _, r := range resources {
if r.Module != module {
@@ -667,6 +677,24 @@ func moduleHealthWord(module, machine string, since time.Time, f gateFacts) (hea
found = append(found, r.Resource)
continue
}
// **The mesh's entries left on a folder whose disk was away are a condition this build found** when its move
// leaves the module's accesses and their users as they were (novox/hq ADR 0325 §8): they may name an access no
// longer declared, they wait for the disk, and a delivery that did not bring them is not held for them — said in
// the verdict, as a directory used as found is. A move that changes the accesses is held to them, whenever they
// were stated (the review of e1c2b744).
changed, known := f.accessesChanged[module]
changes := !known || changed
if r.Kind == link.KindKeptEntries && r.State == link.StateUnhealthy && !changes {
leftover = append(leftover, r.Resource)
continue
}
// **An access the engine could not judge this look** (its filesystem could not be named — a hung mount) carries
// no wait. Of a module whose move leaves its accesses as they were it is a condition the move found, not
// counted; of one whose move changes them it is not yet healthy, held to the gate's bound like anything still
// unsettled, so the gate fails safe at the bound and never waits for ever.
if r.Kind == link.KindAccess && r.State == link.StateUnknown && !changes {
continue
}
switch r.State {
case link.StateHealthy:
case link.StateStarting:
@@ -690,7 +718,7 @@ func moduleHealthWord(module, machine string, since time.Time, f gateFacts) (hea
reasonAfter(r.Reason))
}
}
if waits || len(found) > 0 || len(forOperator) > 0 || len(onWaiting) > 0 {
if waits || len(found) > 0 || len(forOperator) > 0 || len(onWaiting) > 0 || len(leftover) > 0 {
said := onWaiting
if waits {
said = append(said, wait)
@@ -702,6 +730,10 @@ func moduleHealthWord(module, machine string, since time.Time, f gateFacts) (hea
said = append(said, fmt.Sprintf("on %s, %s uses %s as found and waits for the operator to hand it over "+
"(`nox node hand-over %s <directory>` on the control-node)", machine, module, strings.Join(found, ", "), machine))
}
if len(leftover) > 0 {
said = append(said, fmt.Sprintf("on %s, the mesh's entries %s could not be taken out while their disk was "+
"away, stated before this send: they are taken out when it is back", machine, strings.Join(leftover, ", ")))
}
return healthPerson, strings.Join(said, "; ")
}
return healthGood, ""
@@ -742,6 +774,14 @@ func healthLines(h inventory.NodeHealth, had bool, now time.Time) []string {
line += " (unconfirmed: said once)"
}
out = append(out, line)
// A kept folder's users, as the engine resolved them (novox/hq ADR 0325 §4), the runner's note beside it.
for _, e := range r.Entries {
entry := fmt.Sprintf(" entry %s (uid %d, %s, from the %s)", e.Identity, e.UID, e.Use, e.From)
if e.Note != "" {
entry += " — " + e.Note
}
out = append(out, entry)
}
}
return out
}
+9
View File
@@ -487,6 +487,15 @@ func settingsCommand(ctx context.Context, args []string) error {
if err := inv.SetSettingsBy(ctx, *node, positionals[0], values, setByWords()); err != nil {
return err
}
// An owner a placement names is warned of (novox/hq ADR 0325 §1): accepted while registered modules still
// name one, refused once they have moved.
if shelf, err := inv.Catalogue(ctx); err == nil {
if m, known := shelf[positionals[0]]; known {
for _, w := range catalogue.PlacesOwnerWarnings(m, []catalogue.Layer{{From: where, Values: values}}) {
fmt.Println(w)
}
}
}
fmt.Printf("%s on %s:\n", positionals[0], where)
if len(added)+len(changed)+len(removed) == 0 {
fmt.Println(" nothing changed")
+61 -2
View File
@@ -127,9 +127,14 @@ func nodeCommand(ctx context.Context, args []string) error {
// and whoever may call a verb includes agents.
return nodeSetuidSearch(ctx, open, args[1:])
case "judge-directories":
// Which of a node's declared directories the walk that follows no link another account could make would
// refuse (novox/hq issue 358). Read-only on the node and here, so a verb may run it too.
return nodeJudgeDirectories(ctx, open, args[1:])
default:
return fmt.Errorf("node has no %q; it has add, list, show, public-domain, account, agent-account, hand-over "+
"and setuid-search", args[0])
return fmt.Errorf("node has no %q; it has add, list, show, public-domain, account, agent-account, hand-over, "+
"setuid-search and judge-directories", args[0])
}
}
@@ -937,3 +942,57 @@ func printJSONTo(w io.Writer, v any) error {
fmt.Fprintln(w, string(body))
return nil
}
const judgeDirectoriesUsage = "node judge-directories <node> — read-only: which of the directories <node> is " +
"declared the node-engine's walk that follows no link another account could make would refuse, one line each " +
"(novox/hq issue 358). Nothing is made, chmodded or chowned"
// nodeJudgeDirectories asks the node's engine to judge its declared directories, signed with the mesh's key as a
// hand-over is. Read-only: the engine changes nothing and keeps nothing.
func nodeJudgeDirectories(ctx context.Context, open *stores, args []string) error {
known := func(node string) error {
_, err := open.inventory.NodeByName(ctx, node)
return err
}
ask := func(node, by string) (link.HandOverAnswer, error) {
ident, err := open.Identity(ctx)
if err != nil {
return link.HandOverAnswer{}, fmt.Errorf("the mesh's signing key cannot be read, so nothing was asked of %s: %w",
node, err)
}
address, err := broker.BusAddress()
if err != nil {
return link.HandOverAnswer{}, err
}
js, err := broker.Dial(address)
if err != nil {
return link.HandOverAnswer{}, fmt.Errorf("cannot reach the bus, so nothing was asked of %s: %w", node, err)
}
defer js.Close()
return link.AskJudgeDirectories(ctx, js.Conn(), ident, node, by, link.HandOverWithin)
}
return judgeDirectoriesAsked(args, known, ask, os.Stdout)
}
// judgeDirectoriesAsked is the line with its two acts given: whether the mesh knows the node, and the ask. The
// engine's refusal is this command's failure.
func judgeDirectoriesAsked(args []string, known func(node string) error,
ask func(node, by string) (link.HandOverAnswer, error), out io.Writer) error {
if len(args) != 1 || args[0] == "" || strings.HasPrefix(args[0], "-") {
return errors.New(judgeDirectoriesUsage)
}
node := args[0]
if err := known(node); err != nil {
return fmt.Errorf("nothing was asked: %w", err)
}
answer, err := ask(node, handOverBy())
if err != nil {
return err
}
if answer.Refused != "" {
return fmt.Errorf("%s refused: %s", node, answer.Refused)
}
fmt.Fprintf(out, "%s (read-only: nothing on %s was changed)\n", node, node)
fmt.Fprintln(out, answer.Said)
return nil
}
+112
View File
@@ -0,0 +1,112 @@
package main
import (
"bytes"
"context"
"errors"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// novox/hq issue 237: every line of an act's answer comes from one resolution of the node taken after
// its write, so the answer agrees with `plan` for the node right after.
func aProviderAndItsConsumer(t *testing.T, open *stores) {
t.Helper()
register(t, open, catalogue.Manifest{Module: "db", Version: "1",
Provides: []catalogue.Offer{{Name: "rows", Scope: catalogue.ScopeNode}}})
register(t, open, catalogue.Manifest{Module: "app", Version: "1", Requires: []string{"rows"}})
}
// twoRivalsAndANote is a node that does not resolve — two modules claim one node seat there — and a
// module beside them that changes nothing about it.
func twoRivalsAndANote(t *testing.T, open *stores) {
t.Helper()
one, two := rivals()
register(t, open, one)
register(t, open, two)
register(t, open, catalogue.Manifest{Module: "notes", Version: "1"})
}
func TestAnUnassignThatLeavesItsNodeUnresolvedSaysSoOnceInItsAnswer(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
twoRivalsAndANote(t, open)
// Straight into the store, as a node that came to this before the act.
for _, m := range []string{"rival-one", "rival-two", "notes"} {
if _, err := open.inventory.Assign(ctx, "laptop", m); err != nil {
t.Fatal(err)
}
}
said := answered(t, func() error { return assignCommand(ctx, "unassign", []string{"laptop", "notes"}) })
if _, _, err := planFor(ctx, open, "laptop"); err == nil || !unresolvable(err) {
t.Fatalf("laptop resolves with two claimants of one seat: %v", err)
}
if n := strings.Count(said, "laptop does not resolve now"); n != 1 {
t.Errorf("the answer says laptop does not resolve %d times, want once:\n%s", n, said)
}
if strings.Contains(said, "is left out of the rest of the mesh") {
t.Errorf("the answer carries a line from a read that is not its resolution:\n%s", said)
}
}
func TestAnUnassignThatLeavesItsNodeResolvedSaysNothingOfIt(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
aProviderAndItsConsumer(t, open)
if _, err := assign(ctx, open, "laptop", "db", "app"); err != nil {
t.Fatal(err)
}
said, err := unassign(ctx, open, "laptop", "app")
if err != nil {
t.Fatal(err)
}
if _, _, err := planFor(ctx, open, "laptop"); err != nil {
t.Fatal(err)
}
if strings.Contains(said, "does not resolve") || strings.Contains(said, "could not be read") {
t.Errorf("a node that resolves is said not to:\n%s", said)
}
}
func TestAnAssignThatLeavesItsNodeUnresolvedSaysOnlyThePlansRefusal(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
twoRivalsAndANote(t, open)
if _, err := assign(ctx, open, "laptop", "rival-one"); err != nil {
t.Fatal(err)
}
// Kept and not resolvable: a second claimant of the seat, which assignment keeps.
said, err := assign(ctx, open, "laptop", "rival-two")
if err == nil {
t.Fatalf("two claimants of one node seat resolved:\n%s", said)
}
_, _, planErr := planFor(ctx, open, "laptop")
if planErr == nil || planErr.Error() != err.Error() {
t.Errorf("the act's refusal is not the plan's right after:\nact: %v\nplan: %v", err, planErr)
}
if strings.Contains(said, "now has") {
t.Errorf("the answer says a seat is now held on a node that does not resolve:\n%s", said)
}
}
// The other readers of the whole mesh still say a machine they leave out, on standard error (novox/hq
// issue 188); an act sends it elsewhere.
func TestAMachineLeftOutIsSaidWhereTheContextSays(t *testing.T) {
var buf bytes.Buffer
sayLeftOut(leftOutSaidBy(context.Background(), &buf), "laptop", errors.New("why"))
if buf.String() != "laptop is left out of the rest of the mesh: it does not resolve: why\n" {
t.Errorf("said %q", buf.String())
}
got := answered(t, func() error {
sayLeftOut(context.Background(), "laptop", errors.New("why"))
return nil
})
if !strings.Contains(got, "laptop is left out") {
t.Errorf("without an act's writer the line did not reach standard error: %q", got)
}
}
+105 -2
View File
@@ -44,6 +44,12 @@ func needsOperatorKey(module, node string) string {
type operatorWaitFacts struct {
manifests map[string]catalogue.Manifest
given map[string]map[string]time.Time
// placed is, per "<module>@<machine>", where each access of the module is on that machine, by id (novox/hq ADR
// 0325 §7); taken is, per machine, every folder the operator let the mesh keep there, with the filesystem it was
// taken on (empty when the engine named none). A machine absent from taken could not be read, and no access wait
// of it is excused.
placed map[string]map[string]string
taken map[string]map[string]string
}
// checkWait is nil when a wait is excused, and otherwise why not, in words. Pure.
@@ -52,9 +58,17 @@ func checkWait(module, machine string, w inventory.Wait, f operatorWaitFacts) er
if !known {
return fmt.Errorf("says it waits for %s, and the mesh holds no manifest of %s to check it against", waitNames(w), module)
}
named := 0
for _, n := range []string{w.Secret, w.Setting, w.Access} {
if n != "" {
named++
}
}
switch {
case w.Secret != "" && w.Setting != "", w.Secret == "" && w.Setting == "":
return fmt.Errorf("says it waits, naming %s, where a wait names one secret or one setting", waitNames(w))
case named != 1:
return fmt.Errorf("says it waits, naming %s, where a wait names one secret or one setting, or one access", waitNames(w))
case w.Access != "":
return checkAccessWait(module, machine, m, w, f)
case w.Setting != "":
if _, declared := m.Settings[w.Setting]; !declared {
return fmt.Errorf("says it waits for the setting %s, which %s does not declare", w.Setting, module)
@@ -80,6 +94,49 @@ func checkWait(module, machine string, w inventory.Wait, f operatorWaitFacts) er
return nil
}
// checkAccessWait is nil when an access wait is excused (novox/hq ADR 0325 §7): the module's manifest declares the
// access, its folder is placed on that machine, and either the folder is absent and its disk is not there (§9), or
// it has not yet been taken under the operator's warrant (§8). That a folder that is there is not owned by root or a
// root group is the node-engine's to see: it states such an access unhealthy and never waiting (§12). Any other
// access wait is judged unhealthy, saying why: a module that gains an access to a folder already taken is given its
// entries without a wait.
func checkAccessWait(module, machine string, m catalogue.Manifest, w inventory.Wait, f operatorWaitFacts) error {
declared := false
for _, a := range m.Accesses {
declared = declared || a.ID == w.Access
}
if !declared {
return fmt.Errorf("says it waits for the access %s, which %s does not declare", w.Access, module)
}
path, placed := f.placed[module+"@"+machine][w.Access]
if !placed {
return fmt.Errorf("says it waits for the access %s, which is not placed on %s", w.Access, machine)
}
switch {
case w.Reason == link.WaitDisk || (w.Reason == "" && w.Disk):
return nil
case w.Reason != link.WaitKeep && w.Reason != "":
return fmt.Errorf("says it waits for the access %s for %q, and an access waits for the operator's word to "+
"keep its folder or for its disk", w.Access, w.Reason)
case w.ACL == link.ACLUnsupported:
// No act can make this filesystem carry access entries: nothing to ask; the operator places the data elsewhere.
return fmt.Errorf("says it waits for the access %s, whose folder's filesystem cannot carry access entries: "+
"the operator places the data on one that can, or mounts it with ACLs", w.Access)
}
taken, readable := f.taken[machine]
if !readable {
return fmt.Errorf("says it waits for the access %s, and which folders are taken on %s could not be read",
w.Access, machine)
}
// A folder taken on another filesystem than the one it is on now (it moved to another disk) is not taken by that
// warrant: the wait is excused, and the operator is asked again (ADR 0325 §8).
if on, was := taken[path]; was && (w.Placed == "" || on == w.Placed) {
return fmt.Errorf("says it waits for the access %s, whose folder is taken on %s already: a module that gains "+
"an access to a taken folder is given its entries without a wait", w.Access, machine)
}
return nil
}
// waitRefusedPrefix opens every reason checkWait gives, so the words of a refused wait are told from a check's own.
const waitRefusedPrefix = "says it waits"
@@ -88,6 +145,10 @@ func waitNames(w inventory.Wait) string {
switch {
case w.Secret != "" && w.Setting != "":
return "the secret " + w.Secret + " and the setting " + w.Setting
case w.Access != "" && (w.Secret != "" || w.Setting != ""):
return "the access " + w.Access + " and a secret or a setting"
case w.Access != "":
return "the access " + w.Access
case w.Secret != "":
return "the secret " + w.Secret
case w.Setting != "":
@@ -147,6 +208,14 @@ func operatorWaitSaid(module, machine string, waits []inventory.Wait) string {
var parts []string
for _, w := range waits {
part := fmt.Sprintf("%s (%s", w.What, waitNames(w))
// An access in the controller's own words, never the engine's: words with no path (novox/hq ADR 0325 §7).
switch {
case w.Access != "" && w.Disk:
part = fmt.Sprintf("the disk that holds %s mounted (%s", w.Access, waitNames(w))
case w.Access != "":
part = fmt.Sprintf("leave to keep the shared folder %s (%s, asked of you once for the folder", w.Access,
waitNames(w))
}
if w.Secret != "" {
part += fmt.Sprintf(", given with `nox secret ask %s %s %s`", machine, module, w.Secret)
}
@@ -180,6 +249,11 @@ func needsOperatorWords(module, node string, waits []inventory.Wait) words {
act, secret = fmt.Sprintf("type %s at %s's desk prompt", w.What, node), true
case w.Setting != "":
act = fmt.Sprintf("approve %s of %s on %s when it is proposed to you", w.Setting, module, node)
// An access, in words with no path (novox/hq ADR 0325 §7): the folder by the access's name.
case w.Access != "" && w.Disk:
act = fmt.Sprintf("mount the disk that holds %s on %s", w.Access, node)
case w.Access != "":
act = fmt.Sprintf("let the mesh keep the shared folder %s on %s when you are asked", w.Access, node)
}
if act != "" && !seen[act] {
seen[act] = true
@@ -217,6 +291,12 @@ func readWaitFacts(ctx context.Context, inv *inventory.Inventory, machine string
if f.given == nil {
f.given = map[string]map[string]time.Time{}
}
if f.placed == nil {
f.placed = map[string]map[string]string{}
}
if f.taken == nil {
f.taken = map[string]map[string]string{}
}
if inv == nil {
return
}
@@ -238,6 +318,20 @@ func readWaitFacts(ctx context.Context, inv *inventory.Inventory, machine string
}
}
}
// Where each access of it is placed there (ADR 0325 §7): read once per module and machine.
if m, known := f.manifests[module]; known && len(m.Accesses) > 0 {
if _, read := f.placed[module+"@"+machine]; !read {
if layers, err := inv.SettingsFor(ctx, machine, module); err == nil {
placed := map[string]string{}
for _, a := range m.Accesses {
if path, has := catalogue.AccessPathOn(m, layers, a.ID); has && a.ID != "" {
placed[a.ID] = path
}
}
f.placed[module+"@"+machine] = placed
}
}
}
if _, read := f.given[module+"@"+machine]; read {
continue
}
@@ -245,6 +339,15 @@ func readWaitFacts(ctx context.Context, inv *inventory.Inventory, machine string
f.given[module+"@"+machine] = given
}
}
if _, read := f.taken[machine]; !read {
if taken, err := inv.KeptFolders(ctx, machine); err == nil {
kept := map[string]string{}
for path, t := range taken {
kept[path] = t.Placed
}
f.taken[machine] = kept
}
}
}
// waitingModules is every module with a waiting resource in a statement.
@@ -0,0 +1,120 @@
package main
import (
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// An own secret issued outside the mesh that nobody gave is never made (novox/hq issue 401, ADR 0228). Seen on
// 2026-10-10: the send made a random value for the mounts module's smb-credentials and placed it, and the module read
// it as a credential given and wrong. The send now makes none, places none, and places a value only once a person
// gave one — and a value the mesh made for it before is not placed either.
// anAssistant is a module with a secret the mesh makes and one only an outside party issues.
func anAssistant() catalogue.Manifest {
return catalogue.Manifest{Module: "assistant", Version: "1",
OwnSecrets: catalogue.OwnSecrets{
"password": {Path: "/var/lib/mesh/assistant/password", Taken: catalogue.TakenAtStart},
"vendor-key": {Path: "/var/lib/mesh/assistant/vendor-key", Taken: catalogue.TakenAtStart, IssuedBy: catalogue.IssuedOutside},
},
Resources: []map[string]any{
{"id": "state", "type": "directory", "path": "/var/lib/mesh/assistant", "mode": "0700"},
}}
}
// sentTo composes what a push sends to a machine, on the send path.
func sentTo(t *testing.T, open *stores, node string) []map[string]any {
t.Helper()
plan, settings, err := planFor(t.Context(), open, node)
if err != nil {
t.Fatal(err)
}
declared, err := declarationFor(t.Context(), open, node, plan, settings)
if err != nil {
t.Fatalf("the send to %s did not compose: %v", node, err)
}
return declared.Resources
}
func placedAt(resources []map[string]any, path string) map[string]any {
for _, r := range resources {
if r["path"] == path && r["sealed"] != nil {
return r
}
}
return nil
}
func TestTheSendMakesNoValueForAnOutsideSecretNobodyGave(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, anAssistant())
if _, err := assign(ctx, open, "laptop", "assistant"); err != nil {
t.Fatal(err)
}
sent := sentTo(t, open, "laptop")
if placedAt(sent, "/var/lib/mesh/assistant/password") == nil {
t.Fatal("the secret the mesh makes was not placed")
}
if got := placedAt(sent, "/var/lib/mesh/assistant/vendor-key"); got != nil {
t.Fatalf("a secret nobody gave was placed: %v", got)
}
if _, held, err := open.inventory.ModuleSecretIfIssued(ctx, "laptop", "assistant", "vendor-key"); err != nil || held {
t.Fatalf("the send made a value for a secret issued outside the mesh (held %v, %v)", held, err)
}
// A value the mesh made before this rule is not one a person gave: it is not placed.
if _, err := open.inventory.SecretForModule(ctx, "laptop", "assistant", "vendor-key"); err != nil {
t.Fatal(err)
}
if got := placedAt(sentTo(t, open, "laptop"), "/var/lib/mesh/assistant/vendor-key"); got != nil {
t.Fatalf("a value the mesh made for a secret issued outside was placed: %v", got)
}
// Given, it is placed.
if err := open.inventory.AcceptSecretForModule(ctx, "laptop", "assistant", "vendor-key", "the vendor's key"); err != nil {
t.Fatal(err)
}
given, ok, err := open.inventory.GivenOwnSecret(ctx, "laptop", "assistant", "vendor-key")
if err != nil || !ok || !given.Current {
t.Fatalf("the value given is not read as given: %+v %v %v", given, ok, err)
}
placed := placedAt(sentTo(t, open, "laptop"), "/var/lib/mesh/assistant/vendor-key")
if placed == nil || placed["sealed"] != given.Sealed {
t.Fatalf("the value given is placed as %v", placed)
}
}
// The read paths answer as the send does: a question about the machine composes without the secret nobody gave, and
// asking writes nothing.
func TestAQuestionComposesWithoutAnOutsideSecretNobodyGave(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, anAssistant())
if _, err := assign(ctx, open, "laptop", "assistant"); err != nil {
t.Fatal(err)
}
pushedBy(t, open, "laptop") // the mesh's own secret is made by the send
plan, settings, err := planFor(ctx, open, "laptop")
if err != nil {
t.Fatal(err)
}
gens, err := generators(ctx, open)
if err != nil {
t.Fatal(err)
}
for _, choosing := range []Choosing{Reading, Foreseeing} {
got, err := declarationWith(ctx, open, "laptop", plan, settings, gens, choosing)
if err != nil {
t.Fatalf("%v: %v", choosing, err)
}
if placedAt(got.Resources, "/var/lib/mesh/assistant/vendor-key") != nil || len(got.foreseen) != 0 {
t.Fatalf("%v: a secret nobody gave is composed (foreseen %v)", choosing, got.foreseen)
}
}
if _, held, err := open.inventory.ModuleSecretIfIssued(ctx, "laptop", "assistant", "vendor-key"); err != nil || held {
t.Fatalf("a question made a value (held %v, %v)", held, err)
}
}
+27 -5
View File
@@ -81,6 +81,13 @@ var plainWordings = map[string]func(conditions.Observation) words{
"It usually catches up by itself.", m, m),
Resolved: m + " applied the change"}
}),
kindApplyFailing: worded(func(o conditions.Observation) words {
m := machineOr(o, "a machine")
return words{Headline: m + " is not applying its update",
Explanation: fmt.Sprintf("%s has refused or only partly applied what the mesh sent it, again and again. "+
"The details say what fails.", conditions.Capital(m)),
Resolved: m + " applies what it is sent again"}
}),
"tools-silent": worded(func(o conditions.Observation) words {
m := machineOr(o, "a machine")
return words{Headline: "Tools on " + m + " are not answering",
@@ -885,6 +892,11 @@ func stalledWords(l stalledLine, o conditions.Observation) (headline, explanatio
// within its bound (novox/hq issue 438): mesh-delivery holds no delivery of it, so there is nothing to stop, release
// or close, and no action is offered. The operator's acts are a new commit, which asks the check again, or a look
// at the build queue, where a check that never ran may still wait.
//
// A head whose delivery is held in ready or rejected is said the same way (novox/hq issue 441), and mesh-delivery
// names that state as data (Held). Its check answered once, so the build queue is the wrong place to look: the step
// is a recheck of the delivery, named by its id, which asks the check again (novox/hq issue 451). It is advised, not
// offered as an action, since the controller is not granted the recheck.
func unansweredWords(l stalledLine, o conditions.Observation) (headline, explanation, resolved, needs string,
actions []conditions.Action) {
repository, _, _ := strings.Cut(l.ID, "@")
@@ -899,13 +911,23 @@ func unansweredWords(l stalledLine, o conditions.Observation) (headline, explana
if d, err := time.ParseDuration(l.Bound); err == nil {
limit = ", past its limit of " + humanDuration(d)
}
which := "Its merge check"
check := "merge check"
if said := uncheckedWaitWords(l); len(said) > 0 {
which = "Its merge check (" + strings.Join(said, ", ") + ")"
check = "merge check (" + strings.Join(said, ", ") + ")"
}
if o.Resolver == conditions.ResolverOperator {
needs = "push a new commit to its branch, which asks the check again, or see whether its check still waits " +
"in the build queue: mesh-controller.queue."
which := "Its " + check
switch l.Held {
case "ready", "rejected":
which = "Its delivery is " + l.Held + ", but on the forge its " + check
if o.Resolver == conditions.ResolverOperator {
needs = "recheck its delivery, which asks its check again — mesh-delivery.recheck with id " + l.ID +
" and why — or push a new commit to its branch, " + FromMeshMCPServer
}
default:
if o.Resolver == conditions.ResolverOperator {
needs = "push a new commit to its branch, which asks the check again, or see whether its check still " +
"waits in the build queue: mesh-controller.queue."
}
}
return fmt.Sprintf("%s's merge check has not answered %s", pull, long),
fmt.Sprintf("%s is open on a branch that requires the merge check. %s was asked and has not answered %s%s. "+
+46
View File
@@ -452,3 +452,49 @@ func TestAnUnansweredMergeChecksTimeIsSaidInLocalTime(t *testing.T) {
t.Fatalf("a pending check without its time reads %q", o.Explanation)
}
}
// **An unanswered merge check on a ready or rejected delivery is told to recheck it** (novox/hq issue 451):
// mesh-delivery says such a line too (issue 441) and names the delivery's state as data, `held`. That check already
// answered once, so the build queue is the wrong place to look; a recheck of the delivery, named by its id, asks the
// check again. A line with no delivery held keeps the advice of issue 438, and so does a line from a mesh-delivery
// that does not say `held` yet: the controller never guesses the state from the words.
func TestAnUnansweredMergeCheckOnAReadyOrRejectedDeliveryOffersARecheck(t *testing.T) {
line := func(held string) stalledLine {
var l stalledLine
raw := `{"id":"novox/mesh-controller@c11222026a3b","number":212,"state":"unanswered",` +
`"waiting":["mesh/merge-gate pending since 2026-10-11T00:11:39Z"],"for":"1h1m0s","bound":"1h0m0s",` +
`"h2":"none: its delivery is ` + held + `, a state the operator moves — the operator's",` +
`"says":"novox/mesh-controller#212 is open on main, which requires the merge check, and its delivery is ` +
held + ` here, but on the forge its check has not answered: mesh/merge-gate pending since ` +
`2026-10-11T00:11:39Z. The pull request waits on that status and nothing asks it again. A recheck of the ` +
`delivery asks its check again, as does a new commit on its branch"`
if held != "" {
raw += `,"held":"` + held + `"`
}
if err := json.Unmarshal([]byte(raw+`}`), &l); err != nil {
t.Fatal(err)
}
return l
}
for _, held := range []string{"ready", "rejected"} {
o := stalledObservations([]stalledLine{line(held)})[0]
t.Logf("%s: %s\nneeds: %s", held, o.Explanation, o.Needs)
if !strings.Contains(o.Needs, "mesh-delivery.recheck") || !strings.Contains(o.Needs, "novox/mesh-controller@c11222026a3b") {
t.Fatalf("a %s delivery's unanswered check is not offered its recheck: %q", held, o.Needs)
}
if strings.Contains(o.Needs, "build queue") || strings.Contains(o.Needs, "mesh-controller.queue") {
t.Fatalf("a %s delivery's unanswered check, which answered once, is sent to the build queue: %q", held, o.Needs)
}
if !strings.Contains(o.Explanation, "Its delivery is "+held) {
t.Fatalf("a %s delivery's explanation does not say its state: %q", held, o.Explanation)
}
if len(o.Actions) != 0 {
t.Fatalf("it offers an action the controller is not granted: %+v", o.Actions)
}
}
// No delivery held, or a mesh-delivery that does not say: the advice of issue 438, and no recheck.
o := stalledObservations([]stalledLine{line("")})[0]
if !strings.Contains(o.Needs, "in the build queue: mesh-controller.queue") || strings.Contains(o.Needs, "recheck") {
t.Fatalf("a line with no delivery held says to %q", o.Needs)
}
}
+178 -23
View File
@@ -9,6 +9,7 @@ import (
"fmt"
"io"
"os"
"path/filepath"
"slices"
"sort"
"strings"
@@ -292,7 +293,7 @@ func theRestOfTheMesh(ctx context.Context, inv *inventory.Inventory,
if err != nil {
// Said, not skipped: a machine dropped here offers nothing and holds nothing as far
// as every other machine's plan can tell (novox/hq issue 188).
fmt.Fprintf(os.Stderr, "%s is left out of the rest of the mesh: it does not resolve: %v\n", o.node.Name, err)
sayLeftOut(ctx, o.node.Name, err)
continue
}
firstHeld = append(firstHeld, got.Claims...)
@@ -337,7 +338,7 @@ func theRestOfTheMesh(ctx context.Context, inv *inventory.Inventory,
// resolved without its offers, and one that consumes them cannot resolve here by
// design — that is not the machine being dropped, it is the view being partial.
if exclude == "" {
fmt.Fprintf(os.Stderr, "%s is left out of the rest of the mesh: it does not resolve: %v\n", o.node.Name, err)
sayLeftOut(ctx, o.node.Name, err)
}
continue
}
@@ -347,6 +348,35 @@ func theRestOfTheMesh(ctx context.Context, inv *inventory.Inventory,
return world, nil
}
// leftOutKey carries where theRestOfTheMesh says a machine it leaves out, for an act that answers with
// its own resolution instead (novox/hq issue 237).
type leftOutKey struct{}
// leftOutSaidBy is ctx with theRestOfTheMesh's "left out" lines written to w rather than to standard
// error.
//
// **An act's answer is one resolution taken after its write** (novox/hq issue 237, decided 2026-10-11).
// `assign` read the whole mesh once before its write — to record the seats held by derivation — and that
// read said, on standard error, that the node did not resolve for lack of the seat the act was about to
// give it. The controller seat's verb answers with both streams, so the stale line came last, after the
// act's own "now has it held", and named as its remedy the assignment just made. An act gives io.Discard
// here: what the node resolves to after the write is said by its own planFor, and every other machine
// that does not resolve by blockedElsewhere, both read after the write.
func leftOutSaidBy(ctx context.Context, w io.Writer) context.Context {
return context.WithValue(ctx, leftOutKey{}, w)
}
// sayLeftOut says that a machine is left out of the rest of the mesh, where ctx says to: standard
// error unless an act said otherwise. Said, not skipped: a machine dropped there offers nothing and
// holds nothing as far as every other machine's plan can tell (novox/hq issue 188).
func sayLeftOut(ctx context.Context, node string, err error) {
w, ok := ctx.Value(leftOutKey{}).(io.Writer)
if !ok {
w = os.Stderr
}
fmt.Fprintf(w, "%s is left out of the rest of the mesh: it does not resolve: %v\n", node, err)
}
// declarationFor is everything a node would be sent.
//
// One place, because there were three and one of them was written before credentials existed and
@@ -418,7 +448,7 @@ func declarationWith(ctx context.Context, open *stores, node string,
if err != nil {
return sendable{}, err
}
out := sendable{Resources: composed.Resources, Adoption: adoption,
out := sendable{Resources: composed.Resources, Adoption: adoption, Identities: composed.Identities,
Received: composed.Received, Mesh: with.Mesh, BusUsers: with.BusUsers,
LeftOut: sortedKeysOf(composed.LeftOut), leftOutWhy: composed.LeftOut, withheld: with.Withheld,
unbound: with.Unbound, foreseen: composed.Foreseen, unplaced: composed.Unplaced,
@@ -561,6 +591,13 @@ func busCredentialIssued(ctx context.Context, inv *inventory.Inventory, node, mo
return nil
}
// givenUnderAnOldKey is the refusal for a value a person gave — a family's member, or an own secret issued outside
// the mesh — sealed to a key the machine no longer holds: the mesh cannot make another, so it is given again.
func givenUnderAnOldKey(module, node, name string) error {
return fmt.Errorf("%s on %s holds %q, which was given to the mesh rather than made by it, and %s has "+
"since generated a new sealing key. The mesh cannot make another; give it again", module, node, name, node)
}
// renderingFor is everything a node's declaration is composed with, and the node's record.
func renderingFor(ctx context.Context, open *stores, node string,
plan catalogue.Resolution, settings catalogue.SettingsBy,
@@ -679,10 +716,7 @@ func renderingFor(ctx context.Context, open *stores, node string,
}
if !g.Current {
if choosing == Allocating {
return catalogue.Rendering{}, inventory.Node{}, fmt.Errorf(
"%s on %s holds %q, which was given to the mesh rather than made by it, and %s has "+
"since generated a new sealing key. The mesh cannot make another; give it again",
m.Module, node, g.Name, node)
return catalogue.Rendering{}, inventory.Node{}, givenUnderAnOldKey(m.Module, node, g.Name)
}
continue
}
@@ -692,7 +726,31 @@ func renderingFor(ctx context.Context, open *stores, node string,
needed[m.Module][g.Name] = g.Sealed
}
}
for name := range m.OwnSecrets.Plain() {
for name, own := range m.OwnSecrets.Plain() {
// **An own secret issued outside the mesh is never made** (novox/hq issue 401, ADR 0228; ADR 0283 for
// a family): only a value a person gave is placed. One not given is nothing — no file, and an empty
// value where a file of the module names it — so the module can tell it was not given, where a value
// the mesh made looked given and wrong. A value the mesh made for it before is not one either.
if own.IssuedBy == catalogue.IssuedOutside {
g, given, err := inv.GivenOwnSecret(ctx, node, m.Module, name)
if err != nil {
return catalogue.Rendering{}, inventory.Node{}, err
}
if !given {
continue
}
if !g.Current {
if choosing == Allocating {
return catalogue.Rendering{}, inventory.Node{}, givenUnderAnOldKey(m.Module, node, name)
}
continue
}
if needed[m.Module] == nil {
needed[m.Module] = map[string]string{}
}
needed[m.Module][name] = g.Sealed
continue
}
// Minted on the send path and only read on every other. Making one is an insert, and
// a question that writes is a question that can block against the machine it is about.
var sealed string
@@ -918,9 +976,18 @@ func renderingFor(ctx context.Context, open *stores, node string,
if err != nil {
return catalogue.Rendering{}, inventory.Node{}, err
}
// Each module's identity and the folders taken here (novox/hq ADR 0325 §3, §8), to an engine that keeps folders.
keepsFolders, identities, keptFolders, readyFolders, err := folderFacts(ctx, inv, node, plan, choosing)
if err != nil {
return catalogue.Rendering{}, inventory.Node{}, err
}
return catalogue.Rendering{
ReadsHealth: readsHealth,
JudgesRoot: judgesRoot,
KeepsFolders: keepsFolders,
Identities: identities,
KeptFolders: keptFolders,
ReadyFolders: readyFolders,
BusMembership: memberships[node],
Settings: settings, Generators: gens, Grants: grants, Needed: needed, Foreseen: foreseen, Ports: ports,
Certificate: certificate, Authority: authority, Mesh: private, Names: names,
@@ -953,6 +1020,79 @@ func engineJudgesRoot(ctx context.Context, inv *inventory.Inventory, node string
return had && stated.Contract >= link.RootContract, nil
}
// engineKeepsFolders says whether a machine's node-engine gives modules their identities and keeps shared folders
// (novox/hq ADR 0325), by its own newest statement, for the same reason as engineReadsHealth: an older engine parses
// strictly, and is sent none of it.
func engineKeepsFolders(ctx context.Context, inv *inventory.Inventory, node string) (bool, error) {
stated, had, err := inv.HealthOf(ctx, node)
if err != nil {
return false, err
}
return had && stated.Contract >= link.IdentityContract, nil
}
// folderFacts is what a machine's composition needs of ADR 0325: whether its engine keeps folders, each identity of
// a module of it that declares one, and the folders taken there. **An identity is allocated only on the send path**
// (Allocating), as a port is: a question about what would be sent reads what was allocated, and a module that has
// none yet is composed without one. Nothing is read for an engine that does not keep folders.
func folderFacts(ctx context.Context, inv *inventory.Inventory, node string, plan catalogue.Resolution,
choosing Choosing) (bool, map[string]catalogue.ModuleIdentity, map[string]bool, map[string]bool, error) {
keeps, err := engineKeepsFolders(ctx, inv, node)
if err != nil || !keeps {
return false, nil, nil, nil, err
}
identities := map[string]catalogue.ModuleIdentity{}
for _, m := range plan.Modules {
if !catalogue.DeclaresIdentity(m) {
continue
}
var id catalogue.ModuleIdentity
had := true
if choosing == Allocating {
id, err = inv.AllocateIdentity(ctx, m.Module)
} else {
id, had, err = inv.IdentityOf(ctx, m.Module)
}
if err != nil {
return false, nil, nil, nil, err
}
if had {
identities[m.Module] = id
}
}
taken, err := inv.KeptFolders(ctx, node)
if err != nil {
return false, nil, nil, nil, err
}
kept := map[string]bool{}
for path := range taken {
kept[path] = true
}
// **A folder that moved to another filesystem is not taken there until its new warrant** (ADR 0325 §8): the
// machine's newest statement names, without a path, the filesystem each folder waiting for the operator is on now.
// And every folder the engine says it keeps (its access healthy): one it found empty, or took itself — what, with
// a taken one, lets a container switch to its identity (identityHeld).
ready := map[string]bool{}
if stated, had, err := inv.HealthOf(ctx, node); err != nil {
return false, nil, nil, nil, err
} else if had {
for _, r := range stated.Resources {
if r.Kind == link.KindAccess && r.State == link.StateHealthy && r.Target != "" {
ready[filepath.Clean(r.Target)] = true
}
}
for _, r := range stated.Resources {
for _, w := range r.Waits {
path := filepath.Clean(r.Target)
if t, was := taken[path]; was && w.Access != "" && !w.Disk && w.Placed != "" && w.Placed != t.Placed {
kept[path] = false
}
}
}
}
return true, identities, kept, ready, nil
}
// zonesInTheMesh is every zone a module in the mesh declares, where the mesh placed it (novox/hq ADR
// 0199): the zone settled from that node's settings, the node's private address, the port the
// answering listen is published on there.
@@ -1751,28 +1891,43 @@ func (l *unheldLogger) log(node string, unheld []catalogue.Unheld) {
// which includes every one of a module just assigned — and each now met that was not. Nothing about
// any other node, and nothing that was already true before the act.
func unheldChange(shelf map[string]catalogue.Manifest, node string, before, after []string) []string {
judge := func(names []string) map[string]catalogue.Unheld {
var set []catalogue.Manifest
for _, n := range names {
if m, known := shelf[n]; known {
set = append(set, m)
}
return unheldChangeTo(shelf, node, before, after, unheldOver(shelf, node, after))
}
// unheldOver is the node's unmet seat dependencies judged over a set of module names alone.
func unheldOver(shelf map[string]catalogue.Manifest, node string, names []string) []catalogue.Unheld {
var set []catalogue.Manifest
for _, n := range names {
if m, known := shelf[n]; known {
set = append(set, m)
}
out := map[string]catalogue.Unheld{}
for _, u := range catalogue.UnheldDependencies(shelf, node, set, nil) {
out[u.Module+" "+u.Seat] = u
}
return out
}
was, now := judge(before), judge(after)
return catalogue.UnheldDependencies(shelf, node, set, nil)
}
// unheldChangeTo is unheldChange with the node's dependencies as they now stand given: `now` is what
// the act's one resolution after its write says is unmet, and `after` the modules that resolution has
// on the node (novox/hq issue 237). A line about how things stand after the act is never judged from
// anything else, so it cannot disagree with `plan` for the node.
func unheldChangeTo(shelf map[string]catalogue.Manifest, node string, before, after []string,
now []catalogue.Unheld) []string {
key := func(u catalogue.Unheld) string { return u.Module + " " + u.Seat }
was := map[string]catalogue.Unheld{}
for _, u := range unheldOver(shelf, node, before) {
was[key(u)] = u
}
is := map[string]catalogue.Unheld{}
for _, u := range now {
is[key(u)] = u
}
var lines []string
for _, k := range sortedNames(now) {
for _, k := range sortedNames(is) {
if _, already := was[k]; !already {
lines = append(lines, "but "+now[k].String())
lines = append(lines, "but "+is[k].String())
}
}
for _, k := range sortedNames(was) {
if _, still := now[k]; still {
if _, still := is[k]; still {
continue
}
u := was[k]
+12
View File
@@ -251,6 +251,16 @@ func resumed(p *inventory.Plan, why string) {
if p.State == inventory.PlanFailed && len(failedIn(*p)) == 0 {
p.State = inventory.PlanBuilding
p.Note = why
goesOnAgain(p)
}
}
// goesOnAgain clears a stop from a walk that walks again: a stop said why the walk ended, and a walk resumed has
// not ended. Kept, it would end a retried walk that fails again as stopped, not failed (novox/hq ADR 0308
// decision 1), and keep the search for the merge that brought the failure from running (searchAfterFailures).
func goesOnAgain(p *inventory.Plan) {
if p.Delivery != nil {
p.Delivery.Stopped, p.Delivery.StoppedWhy = "", ""
}
}
@@ -460,6 +470,7 @@ func retryRollouts(ctx context.Context, open *stores, p *inventory.Plan) (string
said = append(said, m+" to "+strings.Join(again, ", "))
}
p.State = inventory.PlanRolling
goesOnAgain(p)
p.Note = fmt.Sprintf("tier %d retried by hand; sent %s first again", p.Tier, strings.Join(said, "; "))
if err := open.inventory.SavePlan(ctx, p); err != nil {
return "", err
@@ -525,6 +536,7 @@ func retryTierWhole(ctx context.Context, open *stores, p *inventory.Plan, again
}
if p.State == inventory.PlanFailed && len(failedIn(*p)) == 0 {
p.State = inventory.PlanBuilding
goesOnAgain(p)
p.Note = fmt.Sprintf("tier %d retried by hand: %s asked again; %s sent again with the tier", p.Tier,
strings.Join(again, ", "), orNone(strings.Join(resent, ", ")))
}
+1 -1
View File
@@ -591,7 +591,7 @@ func proposeCommand(ctx context.Context, module, node, valuesArg string, clear,
return err
},
routerHere: routerHereIn(open.inventory),
grantHeld: grantHeldIn(open),
grantHeld: grantHeldIn(open, conn),
routerRecord: func(ctx context.Context, id string) (string, error) {
bucket, err := asksRecords(ctx, open.inventory)
if err != nil || bucket == "" {
+30 -14
View File
@@ -249,7 +249,7 @@ func serve(ctx context.Context) (err error) {
// (novox/hq to-be 45 §9).
checkEvents = bus
// And every walk this controller keeps in a new state, for the delivery it walks (novox/hq ADR 0239).
inventory.PlanSaved = func(p inventory.Plan) { sayPlanMoved(ctx, bus, p) }
inventory.PlanSaved = func(_ *inventory.Inventory, p inventory.Plan) { sayPlanMoved(ctx, bus, open.inventory, p) }
if err := server.Checks(following{open}); err != nil {
return err
}
@@ -1084,42 +1084,58 @@ func brokerFirst(names []string, holder string, behind bool) []string {
// send anywhere, and a module running there always put it in its first wave. A digest of the list
// last sent is kept for this (ADR 0043: the list is composed on each push, never kept itself).
func brokerBehind(ctx context.Context, open *stores, names []string) (string, bool, error) {
s, err := brokerListState(ctx, open, names)
return s.Machine, s.Behind, err
}
// brokerListState is brokerBehind with why the list could not be composed, when it could not: the plan's own
// reason, which a remedy names rather than a push that would fail on it (the review of hq issue 353).
func brokerListState(ctx context.Context, open *stores, names []string) (busListState, error) {
inv := open.inventory
holders, err := seatHolders(ctx, inv)
if err != nil {
return "", false, err
return busListState{}, err
}
h, held := holders[theBrokerSeat]
if !held || h.Node == "" {
return "", false, nil
return busListState{}, nil
}
shelf, err := inv.Catalogue(ctx)
if err != nil {
return "", false, err
return busListState{}, err
}
if m, known := shelf[h.Module]; !known || m.BusUsers == "" {
// A holder that is sent no user list carries no grant: nothing to send first.
return "", false, nil
return busListState{}, nil
}
for _, n := range names {
if n == h.Node {
return h.Node, false, nil
return busListState{Machine: h.Node}, nil
}
}
// **What cannot be worked out is behind** (the review of hq issue 353). The list it would be sent cannot be
// composed, so whether it carries the grants a send relies on is not known; it is taken as behind, so a push
// sends that machine first and says why it cannot (`plan` says it too), and a rollout that needs the list
// refuses rather than sending code whose grants the bus may not hold. It said "not behind" until then: a
// send went ahead as if the bus held a list nobody could compose.
list, composeErr := "", error(nil)
plan, _, err := planFor(ctx, open, h.Node)
if err != nil {
// It cannot be worked out: sending it would refuse the whole send, and `plan` says why.
return h.Node, false, nil
}
list, _, err := busUserList(ctx, inv, plan.Modules)
if err != nil {
return h.Node, false, nil
composeErr = err
} else if list, _, err = busUserList(ctx, inv, plan.Modules); err != nil {
composeErr = err
}
sent, err := inv.SentBusUsers(ctx, h.Node)
if err != nil {
return "", false, err
return busListState{}, err
}
return h.Node, userListBehind(list, sent), nil
return busListState{Machine: h.Node, Behind: listBehind(list, composeErr, sent), Uncomposed: composeErr}, nil
}
// listBehind is whether the bus's machine is behind: the list composed now is not the one last sent, or it
// could not be composed at all — not known is behind, never "not behind".
func listBehind(list string, composeErr error, sentDigest string) bool {
return composeErr != nil || userListBehind(list, sentDigest)
}
// userListBehind is whether the user list composed now is not the one last sent, by its digest. An
+81 -3
View File
@@ -8,6 +8,7 @@ import (
"fmt"
"io"
"os"
"regexp"
"sort"
"strings"
"time"
@@ -67,18 +68,95 @@ func listQueue(ctx context.Context, args []string, w io.Writer) error {
if err != nil {
return err
}
now := time.Now()
if *asJSON {
body, err := json.MarshalIndent(q, "", " ")
body, err := json.MarshalIndent(queueAnswerOf(q, now), "", " ")
if err != nil {
return err
}
fmt.Fprintln(w, string(body))
return nil
}
fmt.Fprint(w, queueText(q, time.Now()))
fmt.Fprint(w, queueText(q, now))
return nil
}
// queueAnswer is the queue as data (novox/hq issue 455): what a caller reads to say how far back an ask
// waits — "waiting in the build queue behind 2" — which the table alone could not tell it. Text is the
// table, written from the same read: the verb answers it as its output and the rest as its answer, so the
// two never disagree.
type queueAnswer struct {
Seat string `json:"seat"`
Waiting int `json:"waiting"`
InFlight int `json:"in-flight"`
Dead int `json:"dead"`
MaxDeliver int `json:"max-deliver"`
Asks []queuedAnswer `json:"asks"`
Text string `json:"text"`
}
// queuedAnswer is one ask as data. Never what the request carries beyond its source — not `held`, every
// artifact the mesh has built, nor a check's checkouts beside it.
type queuedAnswer struct {
ID string `json:"id"`
State string `json:"state"`
// Kind is "check" for a pull request's merge check, "build" for anything else.
Kind string `json:"kind"`
Repository string `json:"repository"`
Path string `json:"path,omitempty"`
Ref string `json:"ref,omitempty"`
// Commit is the ref when it names a commit; empty when the ask follows a branch or the default.
Commit string `json:"commit,omitempty"`
// Machine is the one building an ask in flight; WasOn the one that started it and handed it back.
Machine string `json:"machine,omitempty"`
WasOn string `json:"was-on,omitempty"`
AskedAt time.Time `json:"asked-at,omitzero"`
// Since is when the ask came to its state: its start on the machine for one in flight that a
// machine said it started, otherwise when it was asked.
Since time.Time `json:"since,omitzero"`
// Position is its place among the waiting, 1 being handed out next; 0 for an ask not waiting.
Position int `json:"position,omitempty"`
Seq uint64 `json:"seq"`
}
// aCommitRef is a ref that names a commit by its full hash (SHA-1 or SHA-256).
var aCommitRef = regexp.MustCompile(`^(?:[0-9a-f]{40}|[0-9a-f]{64})$`)
// queueAnswerOf is the queue as data, in the table's order: in flight, the waiting from the next one to
// be handed out, then the dead. A pure function of one read (novox/hq issue 455).
func queueAnswerOf(q link.Queue, now time.Time) queueAnswer {
waiting, running, dead := q.Of(link.AskWaiting), q.Of(link.AskInFlight), q.Of(link.AskDead)
out := queueAnswer{Seat: q.Seat, Waiting: len(waiting), InFlight: len(running), Dead: len(dead),
MaxDeliver: q.MaxDeliver, Asks: []queuedAnswer{}, Text: queueText(q, now)}
one := func(a link.QueuedAsk, position int) queuedAnswer {
kind := "build"
if a.Request.Check != nil {
kind = "check"
}
since := a.AskedAt
if a.State == link.AskInFlight && a.On != "" && !a.Started.IsZero() {
since = a.Started
}
commit := ""
if aCommitRef.MatchString(a.Ref) {
commit = a.Ref
}
return queuedAnswer{ID: a.ID, State: a.State, Kind: kind, Repository: link.WithoutPassword(a.Repository), Path: a.Path,
Ref: a.Ref, Commit: commit, Machine: a.On, WasOn: a.Was, AskedAt: a.AskedAt, Since: since,
Position: position, Seq: a.Seq}
}
for _, a := range running {
out.Asks = append(out.Asks, one(a, 0))
}
for i, a := range waiting {
out.Asks = append(out.Asks, one(a, i+1))
}
for _, a := range dead {
out.Asks = append(out.Asks, one(a, 0))
}
return out
}
// queueText is the queue as a person reads it: a summary line, then each kind in queue order.
// Never what an ask carries as `held` — that is every artifact the mesh has built.
func queueText(q link.Queue, now time.Time) string {
@@ -92,7 +170,7 @@ func queueText(q link.Queue, now time.Time) string {
return "asked " + now.Sub(t).Round(time.Second).String() + " ago"
}
what := func(a link.QueuedAsk) string {
s := a.Repository
s := link.WithoutPassword(a.Repository)
if a.Path != "" {
s += " at " + a.Path
}
+137
View File
@@ -0,0 +1,137 @@
package main
import (
"encoding/json"
"reflect"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/link"
)
// The queue answers as data as well as text (novox/hq issue 455): each ask's repository, commit, kind,
// machine, since and position among the waiting, read from the same queue the table is written from.
const aCommit = "0123456789abcdef0123456789abcdef01234567"
func aQueueOfEachKind(now time.Time) link.Queue {
asked := now.Add(-5 * time.Minute)
started := now.Add(-2 * time.Minute)
return link.Queue{Seat: link.TheBuildMachine, Delivered: 2, AckPending: 1, MaxDeliver: 5, Asks: []link.QueuedAsk{
{Seq: 1, ID: "build-dead", Repository: "ssh://forge/novox/x", AskedAt: asked, State: link.AskDead},
{Seq: 2, ID: "build-running", Repository: "ssh://forge/novox/mesh-host", Ref: aCommit, AskedAt: asked,
State: link.AskInFlight, On: "ace", Started: started,
Request: link.BuildRequest{ID: "build-running", Check: &link.CheckRequest{Owner: "novox", Repo: "mesh-host"}}},
{Seq: 3, ID: "build-first", Repository: "ssh://forge/novox/mesh-catalog", Ref: "main", AskedAt: asked,
State: link.AskWaiting},
{Seq: 4, ID: "build-second", Repository: "ssh://forge/novox/mesh-catalog", Path: "modules/gitea", Ref: aCommit,
AskedAt: now.Add(-time.Minute), State: link.AskWaiting,
Request: link.BuildRequest{ID: "build-second", Check: &link.CheckRequest{Owner: "novox", Repo: "mesh-catalog"}}},
}}
}
func TestTheQueueVerbAnswersEachAskAsData(t *testing.T) {
now := time.Date(2026, 10, 12, 12, 0, 0, 0, time.UTC)
q := aQueueOfEachKind(now)
got := queueAnswerOf(q, now)
if got.Waiting != 2 || got.InFlight != 1 || got.Dead != 1 || got.Seat != link.TheBuildMachine {
t.Fatalf("counts: %+v", got)
}
if got.Text != queueText(q, now) {
t.Fatalf("the data's text is not the table read from the same queue:\n%s", got.Text)
}
byID := map[string]queuedAnswer{}
for _, a := range got.Asks {
byID[a.ID] = a
}
running := byID["build-running"]
if running.Kind != "check" || running.Machine != "ace" || running.Commit != aCommit || running.Position != 0 ||
!running.Since.Equal(now.Add(-2*time.Minute)) || running.State != link.AskInFlight {
t.Errorf("in flight: %+v", running)
}
first, second := byID["build-first"], byID["build-second"]
if first.Position != 1 || first.Kind != "build" || first.Commit != "" || first.Ref != "main" || first.Machine != "" ||
!first.Since.Equal(now.Add(-5*time.Minute)) {
t.Errorf("first waiting: %+v", first)
}
if second.Position != 2 || second.Kind != "check" || second.Commit != aCommit || second.Path != "modules/gitea" ||
second.Repository != "ssh://forge/novox/mesh-catalog" {
t.Errorf("second waiting: %+v", second)
}
if dead := byID["build-dead"]; dead.Position != 0 || dead.State != link.AskDead {
t.Errorf("dead: %+v", dead)
}
// In the table's order: in flight, then the waiting by position, then the dead.
var order []string
for _, a := range got.Asks {
order = append(order, a.ID)
}
if want := []string{"build-running", "build-first", "build-second", "build-dead"}; !reflect.DeepEqual(order, want) {
t.Errorf("order %v, want %v", order, want)
}
}
// The verb asks its command for JSON, and its answer carries the data while its output stays the table a
// person reads, so a text caller reads what it always did.
func TestTheQueueVerbKeepsTheTableAndCarriesTheData(t *testing.T) {
argv, err := argvFor("queue", nil)
if err != nil || !reflect.DeepEqual(argv, []string{"queue", "--json"}) {
t.Fatalf("queue's command line: %v %v", argv, err)
}
now := time.Now()
q := aQueueOfEachKind(now)
body, err := json.Marshal(queueAnswerOf(q, now))
if err != nil {
t.Fatal(err)
}
answer := answerOf(argv, body, "", true)
if answer.Output != queueText(q, now) {
t.Fatalf("the output is not the table:\n%s", answer.Output)
}
data, ok := answer.Answer.(map[string]any)
if !ok {
t.Fatalf("no data: %#v", answer.Answer)
}
if _, doubled := data["text"]; doubled {
t.Error("the table is said twice: once as the output and again inside the data")
}
asks, _ := data["asks"].([]any)
if len(asks) != 4 || data["waiting"] != float64(2) {
t.Fatalf("data: %v", data)
}
second := asks[2].(map[string]any)
if second["position"] != float64(2) || second["kind"] != "check" || second["commit"] != aCommit {
t.Errorf("second waiting: %v", second)
}
for _, a := range asks {
if _, held := a.(map[string]any)["held"]; held {
t.Error("an ask carries what the mesh holds")
}
}
// A failure is said as text, as before.
failed := answerOf(argv, nil, "mesh-controller: the bus cannot be reached\n", false)
if failed.Answer != nil || !strings.Contains(failed.Output, "cannot be reached") {
t.Errorf("a failure: %+v", failed)
}
}
// A repository named with a password never shows it, in the table or the data.
func TestTheQueueNeverShowsARepositorysPassword(t *testing.T) {
if got := link.WithoutPassword("https://agent:s3cret@forge.example/novox/x.git"); strings.Contains(got, "s3cret") ||
!strings.Contains(got, "agent@forge.example/novox/x.git") {
t.Errorf("redacted: %s", got)
}
now := time.Now()
q := link.Queue{Seat: link.TheBuildMachine, Asks: []link.QueuedAsk{{Seq: 1, ID: "build-1",
Repository: "https://agent:s3cret@forge.example/novox/x.git", State: link.AskWaiting}}}
body, _ := json.Marshal(queueAnswerOf(q, now))
if strings.Contains(string(body), "s3cret") {
t.Errorf("the queue said a password: %s", body)
}
for _, kept := range []string{"ssh://git@forge:222/novox/x.git", "novox/x", "git@forge:novox/x.git", ""} {
if got := link.WithoutPassword(kept); got != kept {
t.Errorf("%q became %q", kept, got)
}
}
}
+1 -1
View File
@@ -326,7 +326,7 @@ func TestTheQueueVerbsRunTheirCommands(t *testing.T) {
args map[string]any
want []string
}{
{"queue", nil, []string{"queue"}},
{"queue", nil, []string{"queue", "--json"}},
{"cancel", map[string]any{"id": "build-1"}, []string{"cancel", "build-1"}},
{"clear", nil, []string{"clear"}},
{"clear", map[string]any{"dead": "true"}, []string{"clear", "--dead"}},
+1 -1
View File
@@ -80,7 +80,7 @@ func TestAVerbsOwnProcessNamesItsConnection(t *testing.T) {
// A silence acts, so it is still its own process; it dials, and finds no such condition.
_, _ = handlers["conditions"](context.Background(),
json.RawMessage(`{"silence":"bus.nothing.here","for":"1h","why":"a test"}`))
names := closedNames(t, bus)
names := closedNames(t, bus, "mesh-controller verb conditions")
found := false
for _, n := range names {
found = found || n == "mesh-controller verb conditions"
+12 -7
View File
@@ -812,14 +812,19 @@ func backlogCommand(ctx context.Context, sub string, args []string) error {
}
defer open.Close()
if sub == "release-backlog" {
release, err := open.inventory.HoldPlans(ctx, true)
if err != nil {
// Said on the bus (novox/hq ADR 0308 decision 1): the release plan starts walking here, and its start is
// said by the verb that opens it.
var p *inventory.Plan
if err := sayingOnTheBus(ctx, open.inventory, func() error {
release, err := open.inventory.HoldPlans(ctx, true)
if err != nil {
return err
}
defer release()
why.record(ctx, "upgrade release-backlog", nil)
p, err = releaseBacklog(ctx, open, link.Caller())
return err
}
defer release()
why.record(ctx, "upgrade release-backlog", nil)
p, err := releaseBacklog(ctx, open, link.Caller())
if err != nil {
}); err != nil {
return err
}
if p == nil {
+43 -23
View File
@@ -1559,7 +1559,7 @@ func plansCommand(ctx context.Context, args []string) error {
if len(positionals) == 2 && positionals[0] == "go" {
why.record(ctx, "plans go", positionals[1:])
var p inventory.Plan
if err := sayingOnTheBus(ctx, func() (err error) {
if err := sayingOnTheBus(ctx, inv, func() (err error) {
p, err = letGo(ctx, inv, positionals[1], "a person ("+whoAsked()+")", strings.TrimSpace(*why.why))
return err
}); err != nil {
@@ -1571,8 +1571,13 @@ func plansCommand(ctx context.Context, args []string) error {
}
// `retry` (novox/hq ADR 0219): a failed plan's failed builds asked again, and the plan goes on.
if len(positionals) == 2 && positionals[0] == "retry" {
said, err := retryPlan(ctx, open, positionals[1])
if err != nil {
// Said on the bus (novox/hq ADR 0308 decision 1): a retried walk starts again, and its start is said
// by the verb that starts it.
var said string
if err := sayingOnTheBus(ctx, inv, func() (err error) {
said, err = retryPlan(ctx, open, positionals[1])
return err
}); err != nil {
return err
}
fmt.Println(said)
@@ -1586,26 +1591,29 @@ func plansCommand(ctx context.Context, args []string) error {
if positionals[0] == "close" {
how = "closed"
}
release, err := inv.HoldPlans(ctx, true)
if err != nil {
return err
}
defer release()
p, err := inv.PlanByID(ctx, positionals[1])
if err != nil {
return err
}
if !p.Open() {
return fmt.Errorf("%s is already %s", p.ID, p.State)
}
why.record(ctx, "plans "+positionals[0], positionals[1:])
p.State = inventory.PlanFailed
p.Note = how + " by hand at tier " + fmt.Sprint(p.Tier) + ": " + strings.TrimSpace(*why.why)
sayUnsent(&p, func(m string) bool {
u, err := inv.UpgradeOf(ctx, m)
return err == nil && u.RollOut
})
if err := inv.SavePlan(ctx, &p); err != nil {
// Said on the bus (novox/hq ADR 0308 decision 1): the walk's end, outcome stopped, is said by the verb
// that ends it, so what it touched is not held until its bound.
var p inventory.Plan
if err := sayingOnTheBus(ctx, inv, func() error {
release, err := inv.HoldPlans(ctx, true)
if err != nil {
return err
}
defer release()
if p, err = inv.PlanByID(ctx, positionals[1]); err != nil {
return err
}
if !p.Open() {
return fmt.Errorf("%s is already %s", p.ID, p.State)
}
why.record(ctx, "plans "+positionals[0], positionals[1:])
stopByHand(&p, how, "a person ("+whoAsked()+")", strings.TrimSpace(*why.why))
sayUnsent(&p, func(m string) bool {
u, err := inv.UpgradeOf(ctx, m)
return err == nil && u.RollOut
})
return inv.SavePlan(ctx, &p)
}); err != nil {
return err
}
fmt.Printf("%s %s at tier %d of %d; what was asked still builds and registers, nothing further is asked\n",
@@ -1924,3 +1932,15 @@ func forgetDeleted(ctx context.Context, inv *inventory.Inventory, module string,
"`module forget %s`: %v\n", p.ID, module, module, firstLine(err.Error()))
}
}
// stopByHand ends a walk a person stopped or closed (`plans stop|close`): failed, with who stopped it and why
// kept as the delivery's owner keeps its own stop, so the walk ends with outcome stopped, not failed (novox/hq
// ADR 0308 decision 1).
func stopByHand(p *inventory.Plan, how, by, why string) {
p.State = inventory.PlanFailed
p.Note = how + " by hand at tier " + fmt.Sprint(p.Tier) + ": " + why
if p.Delivery == nil {
p.Delivery = &inventory.PlanDelivery{}
}
p.Delivery.Stopped, p.Delivery.StoppedWhy = by, why
}
+85
View File
@@ -0,0 +1,85 @@
package main
import (
"bytes"
"io"
"os"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// TestReplay237 replays novox/hq issue 237 (2026-10-04): a workstation already ran a module that
// depends on a node seat nothing there held, and the module holding that seat was assigned to it. The
// answer said the seat was now held, and then that the workstation was left out of the rest of the mesh
// for lack of it, naming as the remedy the assignment just made. `plan` for the node, right after,
// resolved. The outcome asserted: every line of the answer, as the controller seat's verb gives it
// (standard output, then standard error), agrees with `plan` for the node right after the act. Written
// with only what the controller had before its fix — register, assign's command, planFor — so it is laid
// over the commit before.
func TestReplay237(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, aContainer("web"))
register(t, open, catalogue.Manifest{Module: "docker", Version: "1",
Claims: []catalogue.Claim{{Name: catalogue.ContainerRuntimeSeat, Scope: catalogue.ScopeNode}}})
// Already on the node, from before the rule refused it: the node does not resolve until the
// holder comes.
if _, err := open.inventory.Assign(ctx, "laptop", "web"); err != nil {
t.Fatal(err)
}
said := answered(t, func() error { return assignCommand(ctx, "assign", []string{"laptop", "docker"}) })
if _, _, err := planFor(ctx, open, "laptop"); err != nil {
t.Fatalf("laptop does not resolve right after the act: %v\nthe answer was:\n%s", err, said)
}
if !strings.Contains(said, "web on laptop now has "+catalogue.ContainerRuntimeSeat+" held") {
t.Errorf("the answer does not say the seat is now held:\n%s", said)
}
for _, not := range []string{"laptop is left out", "does not resolve", "which nothing on laptop holds"} {
if strings.Contains(said, not) {
t.Errorf("the answer says %q, which `plan` right after contradicts:\n%s", not, said)
}
}
}
// answered is what a command says as the controller seat's verb answers it: standard output, then
// standard error (answerOf).
func answered(t *testing.T, f func() error) string {
t.Helper()
read := func(w **os.File) (func() string, error) {
old := *w
r, pw, err := os.Pipe()
if err != nil {
return nil, err
}
*w = pw
got := make(chan string)
go func() {
var buf bytes.Buffer
_, _ = io.Copy(&buf, r)
got <- buf.String()
}()
return func() string {
_ = pw.Close()
*w = old
return <-got
}, nil
}
stdout, err := read(&os.Stdout)
if err != nil {
t.Fatal(err)
}
stderr, err := read(&os.Stderr)
if err != nil {
t.Fatal(err)
}
runErr := f()
said := stdout() + stderr()
if runErr != nil {
t.Fatalf("%v\n%s", runErr, said)
}
return said
}
+11 -3
View File
@@ -84,8 +84,12 @@ func asAProcess(t *testing.T, url string) {
t.Setenv(broker.CertificateVar, "")
}
// closedNames are the names of the connections the bus saw closed.
func closedNames(t *testing.T, bus *server.Server) []string {
// closedNames are the names of the connections the bus saw closed, waited for by want's own name: the
// bus can register another connection's close first (the test's own setup connection, say), so returning
// on the first non-empty list names whichever closed first, not necessarily the one being waited for
// (novox/hq issue 543). closedNames instead waits until want itself appears among the closed connections,
// or the deadline passes, and returns whatever names it last saw either way.
func closedNames(t *testing.T, bus *server.Server, want string) []string {
t.Helper()
deadline := time.Now().Add(5 * time.Second)
var names []string
@@ -95,10 +99,14 @@ func closedNames(t *testing.T, bus *server.Server) []string {
t.Fatal(err)
}
names = names[:0]
found := false
for _, c := range connz.Conns {
names = append(names, c.Name)
if c.Name == want {
found = true
}
}
if len(names) > 0 {
if found {
return names
}
time.Sleep(50 * time.Millisecond)
@@ -15,7 +15,8 @@ import (
func serviceManagerHolder() catalogue.Manifest {
return catalogue.Manifest{Module: "systemd", Version: "1",
Claims: []catalogue.Claim{{Name: catalogue.ServiceManagerSeat, Scope: catalogue.ScopeNode,
Serves: []string{"units", "status", "start", "stop", "restart", "enable", "disable", "journal"}}},
Serves: []string{"units", "status", "start", "stop", "restart", "enable", "disable", "journal",
"reset-failed", "wanted-by", "unlink-dangling"}}},
Resources: []map[string]any{{"id": "systemd", "type": "package", "package": "systemd"}}}
}
+24 -3
View File
@@ -314,6 +314,11 @@ func (a *verbArguments) commandLine() ([]string, error) {
return nil, err
}
return []string{"node", "show", str("node")}, nil
case "judge-directories":
if err := need("node"); err != nil {
return nil, err
}
return []string{"node", "judge-directories", str("node")}, nil
case "modules":
return []string{"module", "list", "--json"}, nil
case "seats":
@@ -426,7 +431,8 @@ func (a *verbArguments) commandLine() ([]string, error) {
return argv, nil
// The build queue (novox/hq ADR 0219).
case "queue":
return []string{"queue"}, nil
// As data, the table its output (novox/hq issue 455).
return []string{"queue", "--json"}, nil
case "cancel", "kill":
if err := need("id"); err != nil {
return nil, err
@@ -977,7 +983,14 @@ var jsonVerbs = map[string]bool{"status": true, "seats": true, "plan": true, "co
// What the records say the registries may keep (novox/hq ADR 0251).
"artifacts": true, "collect": true, "images": true, "mirrors": true,
// The delivery's owner's verbs answer JSON where they read (plan, order, check, walks) — novox/hq ADR 0239.
"delivery": true}
"delivery": true,
// The build queue, each ask with its position among the waiting (novox/hq issue 455).
"queue": true}
// tableBesideVerbs are the JSON verbs whose document carries the table a person reads as its `text`: the
// verb answers that table as its output, as it did before it answered data, and the rest as `answer` —
// both from the one read, so they never disagree (novox/hq issue 455).
var tableBesideVerbs = map[string]bool{"queue": true}
// overviewVerbs are the JSON verbs whose answer is said once, as data, and not again as the text the
// command printed: the overviews, which grow with the mesh (novox/hq issue 314). Every reader of theirs
@@ -1097,6 +1110,12 @@ func answerOf(argv []string, stdout []byte, stderr string, ok bool) verbAnswer {
var parsed any
if json.Unmarshal(bytes.TrimSpace(stdout), &parsed) == nil {
answer.Answer = parsed
if doc, isDoc := parsed.(map[string]any); isDoc && tableBesideVerbs[argv[0]] {
if table, said := doc["text"].(string); said {
answer.Output = table + stderr
delete(doc, "text")
}
}
if overviewVerbs[argv[0]] {
// Once, as data: the same document again as text doubled an answer that already
// outgrew one message of the bus (novox/hq issue 314).
@@ -1479,7 +1498,9 @@ func seatAnnouncement(handlers map[string]link.ToolHandler) micro.Info {
}
// nodeReads are the `node` subcommands a verb may run: the ones that only read.
var nodeReads = map[string]bool{"list": true, "show": true}
var nodeReads = map[string]bool{"list": true, "show": true,
// Read-only on the node too: the engine judges its directories and changes nothing (novox/hq issue 358).
"judge-directories": true}
// flagsOnly says a command line's rest names no subcommand: empty, or beginning with a flag. For a command
// with no subcommands every word is a flag, its value or a name it reads.
+1 -1
View File
@@ -256,7 +256,7 @@ func stringLit(e ast.Expr) (string, bool) {
var accountedFlags = map[string]map[string]string{
"status": {"json": "set by the verb: the answer is data"},
"seats": {"json": "set by the verb: the answer is data"},
"queue": {"json": "not set: the verb answers the table a person reads"},
"queue": {"json": "set by the verb: the answer is data, its output the table (novox/hq issue 455)"},
"plan": {"json": "set by the verb unless files is asked"},
"push": {
"wait": "set by the verb to 0: a tool call cannot hold a connection for a whole apply",
+6
View File
@@ -18,6 +18,9 @@ import (
// make a machine look out of date for ever, or send something `plan` never showed.
type sendable struct {
Resources []map[string]any
// Identities is the declaration's `identities` (novox/hq ADR 0325 §3): composed only for an engine that keeps
// folders, and not sent at all when empty — an older engine refuses a key it does not know, whole.
Identities []map[string]any
// Sequence orders this send against every other to the same node: one higher each time, taken
// under the node's hold just before the body is made (novox/hq 04-ISSUES/107). Zero is not sent
// at all, which a host reads as "no order claimed" — the shape of every declaration before this.
@@ -110,6 +113,9 @@ func (s sendable) Body() ([]byte, error) {
if s.Generation > 0 {
envelope["generation"] = s.Generation
}
if len(s.Identities) > 0 {
envelope["identities"] = s.Identities
}
// An empty declaration is deliberate here — the node owns nothing the mesh put there
// (novox/hq issue 127) — and the host refuses an empty body unless it is told the emptiness
// is meant, so a truncated or mis-composed body is never mistaken for "own nothing".
+10
View File
@@ -254,6 +254,16 @@ var signalsTable = []signalRow{
newest: func(f *signalFacts) time.Time {
return newestOf(f.bus.waits, func(w busWaitFacts) time.Time { return w.since })
}},
{Row: "S21", Signal: "a clean apply", Emitter: "node-engine", Trigger: "each report of a declaration refused or applied only in part (novox/hq issue 233)",
Bound: "one apply interval (5 min) past the first report of the same failure, judged by time while the machine " +
"is heard — the node-engine says a re-apply only when how it went changed; not while it is silent or " +
"asleep; naming the refusal, or the package manager that does not answer with its words; cleared by the " +
"clean apply it then says",
Kind: kindApplyFailing, Severity: conditions.Warning, Phase: 2,
needs: func(f *signalFacts) error { return f.doingErr }, watch: watchApplyFailing,
newest: func(f *signalFacts) time.Time {
return newestOf(f.doing, func(d inventory.Doing) time.Time { return d.At })
}},
}
// watchFacts is S14: the snapshot a merge check is fed is older than its bound, or none was kept since
+8
View File
@@ -186,6 +186,14 @@ var suppressions = map[string]suppression{
},
past: func(f *signalFacts) { f.refusedSends = []inventory.Send{refusedSend(f.now.Add(-time.Second))} },
},
// A machine whose package manager does not answer (novox/hq issue 233): said once its first report of the
// failure is older than one apply interval while it is heard. Inside: exactly five minutes old.
"S21": {
inside: func(f *signalFacts) { f.doing = []inventory.Doing{unansweringPackageManager(f.now, 5*time.Minute, 1)} },
past: func(f *signalFacts) {
f.doing = []inventory.Doing{unansweringPackageManager(f.now, 5*time.Minute+time.Second, 1)}
},
},
// Twice by hand within a fortnight is a healer wanted; once, or the first of two a day too old, is not.
"S15": {
inside: func(f *signalFacts) {
+1 -1
View File
@@ -172,7 +172,7 @@ func composeStatus(open *stores) func(context.Context) ([]byte, error) {
var readingVerbs = map[string]bool{
"tools": true, "calls": true, "status": true, "nodes": true, "node": true, "modules": true,
"seats": true, "builds": true, "plan": true, "queue": true, "durations": true, "hand-acts": true,
"doctor": true, "conditions": true, "healers": true,
"doctor": true, "conditions": true, "healers": true, "judge-directories": true,
}
// nudgingListener is the enrolment, nudging the summary when a machine said something new.
@@ -344,3 +344,58 @@ func TestAnyKeyOfAModuleWithATrustedMergeableFileIsRefusedThroughAVerb(t *testin
t.Fatalf("a file marked untrusted through the verb: %v", err)
}
}
// Who owns a module's directory is said at the terminal alone (novox/hq issue 279): a directory's owner may name
// the settings the module runs as, root chowns the folder to them at the next push, and through the settings verb
// any caller could have set puid to 0 or to its own uid. The key is refused through every verb route even where a
// file nothing trusts also asks for it, and taken at the terminal; the layer a refused call meets is unchanged.
func TestADirectoryOwnersKeyIsRefusedThroughAVerb(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
register(t, open, catalogue.Manifest{Module: "sonarr", Version: "1",
Resources: []map[string]any{
{"id": "recycle-bin", "type": "directory", "mode": "0755", "owner": "${setting:puid}:${setting:pgid}"},
{"id": "identity", "type": "file", "path": "/etc/sonarr.env", "mode": "0644", "trusted": false,
"content": "PUID=${setting:puid}\nPGID=${setting:pgid}\nX=${setting:x}\n"}}})
if err := atTheTerminal(t, "settings", "set", "sonarr", `{"puid":1001,"pgid":2000,"x":0}`,
"--node", "anchor"); err != nil {
t.Fatalf("the owner at the terminal: %v", err)
}
if _, err := assign(ctx, open, "anchor", "sonarr"); err != nil {
t.Fatal(err)
}
layer := func() string {
t.Helper()
values, _, err := open.inventory.Layer(ctx, "anchor", "sonarr")
if err != nil {
t.Fatal(err)
}
raw, _ := json.Marshal(values)
return string(raw)
}
kept := layer()
refused := func(what string, err error) {
t.Helper()
if err == nil || !strings.Contains(err.Error(), "controller's terminal") {
t.Fatalf("%s: %v", what, err)
}
}
refused("puid 0 through the verb", throughVerb(t, "settings", map[string]any{"module": "sonarr",
"node": "anchor", "values": `{"puid":0,"pgid":2000,"x":0}`}))
refused("pgid through the verb, mesh-wide", throughVerb(t, "settings", map[string]any{"module": "sonarr",
"values": `{"pgid":0}`}))
refused("cleared through the verb", throughVerb(t, "settings", map[string]any{"module": "sonarr",
"node": "anchor", "clear": "true"}))
if err := throughVerb(t, "command", map[string]any{
"command": `settings set sonarr '{"puid":0,"pgid":2000,"x":0}' --node anchor`}); err == nil {
t.Fatal("the command verb set the owner")
}
if got := layer(); got != kept {
t.Fatalf("a refused call changed the layer: %s, was %s", got, kept)
}
// Another key, the owner kept, is still the verb's.
if err := throughVerb(t, "settings", map[string]any{"module": "sonarr", "node": "anchor",
"values": `{"puid":1001,"pgid":2000,"x":1}`}); err != nil {
t.Fatalf("another key through the verb, the owner kept: %v", err)
}
}
+334
View File
@@ -0,0 +1,334 @@
package main
import (
"context"
"fmt"
"os"
"slices"
"sort"
"time"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
)
// What a walk touches, its bound and its outcome (novox/hq ADR 0308 decision 1, decision 5): said in
// `plan-moved` on the save that starts the walk and the save that ends it, beside the walk whole, for the holder
// of the operator's channel to hold what the walk touches while it runs and tell its result once.
//
// **What a walk touches** is every module it moves on every machine it sends it to, and every module a moved
// tool runner (node-tools) or node-engine (mesh-host) carries on that machine: what restarts there. At the start
// that is what the walk will send — each module of its tiers whose policy rolls it out, on every machine running
// it; at the end it is what it reached — the machines each module was sent to, first and rest, and what a gated
// send carried with it.
//
// **Its bound** is twice the largest delivery budget among the walk's deliveries (deliveryBudgets: a core
// module's ten minutes, a leaf module's budget in force), and fifteen minutes, not doubled, for a walk none of
// whose deliveries has a budget: a release plan (builds waiting for a person's gate), a walk whose every module's
// policy records rather than rolls out (held for a person, ADR 0282 decision 1), or a walk kept without a class.
// walkNoBudgetBound is the bound of a walk with no budget (ADR 0308 decision 5): not doubled.
const walkNoBudgetBound = 15 * time.Minute
// budgetNone is the budget class of a walk none of whose deliveries has a budget.
const budgetNone = "none"
// walkSaid is what `plan-moved` adds on a walk's start and end, beside the walk whole.
type walkSaid struct {
Walk string `json:"walk"`
Phase string `json:"phase"`
// Touches is, per machine, the modules the walk moves or reaches there and what a moved tool runner or
// node-engine carries there; never null.
Touches []walkTouch `json:"touches"`
// Outcome is on the end only: delivered, failed or stopped.
Outcome string `json:"outcome,omitempty"`
// BoundSeconds and BudgetClass are said on both, read from the walk's class at the start.
BoundSeconds int `json:"bound_seconds"`
BudgetClass string `json:"budget_class"`
// StartedAt is when the walk was said started, on both; EndedAt when it was said ended, on the end: the
// bound runs from StartedAt, whenever the event is heard.
StartedAt *time.Time `json:"started_at,omitempty"`
EndedAt *time.Time `json:"ended_at,omitempty"`
}
// walkTouch is what a walk touches on one machine.
type walkTouch struct {
Machine string `json:"machine"`
Modules []string `json:"modules"`
// Carried names those of Modules that are there only because a moved tool runner or node-engine carries
// them: what restarts with it, not what the walk builds.
Carried []string `json:"carried,omitempty"`
}
// planMovedBody is `plan-moved`'s body: the walk whole, as before, and on its start and end what it touches.
type planMovedBody struct {
inventory.Plan
*walkSaid
}
// walkFacts is what the touches are read from: the controller's inventory.
type walkFacts interface {
Running(ctx context.Context, module string) ([]string, error)
UpgradeOf(ctx context.Context, module string) (inventory.Upgrade, error)
Assigned(ctx context.Context, node string) ([]string, error)
SentBuilds(ctx context.Context, node string) (map[string]string, bool, error)
Catalogue(ctx context.Context) (map[string]catalogue.Manifest, error)
}
// sayWalk is what `plan-moved` adds to the save that started or ended the walk; nil on every other save. A fact
// that cannot be read is said on the controller's log and left out, never in the way of the event: a walk said
// with fewer touches is held less, never longer.
func sayWalk(ctx context.Context, facts walkFacts, p inventory.Plan) *walkSaid {
if p.WalkSaid == "" {
return nil
}
w := &walkSaid{Walk: p.ID, Phase: p.WalkSaid}
if p.Times != nil {
w.StartedAt = p.Times.Started
if p.WalkSaid == inventory.WalkEnded {
w.EndedAt, w.Outcome = p.Times.Ended, p.Outcome()
}
}
read := readWalkFacts(ctx, facts, p)
w.BudgetClass = walkBudgetClass(p, read.rollsOut)
w.BoundSeconds = int(walkBound(w.BudgetClass) / time.Second)
var moved map[string][]string
if p.WalkSaid == inventory.WalkStarted {
moved = walkSends(p, read.running, read.rollsOut)
} else {
moved = walkReached(p, read.running)
}
var shelf map[string]catalogue.Manifest
if len(moved[broker.RuntimeModule]) > 0 {
// Which modules the tool runner serves is the catalogue's: read only when the walk moves it.
shelf = read.shelf()
}
w.Touches = touchesOf(moved, read.onMachine(moved), shelf)
return w
}
// walkBudgetClass is the largest budget class among the walk's deliveries: its class read at the cut when one of
// its modules rolls out, and none when it has no budget.
func walkBudgetClass(p inventory.Plan, rollsOut func(string) bool) string {
if p.Release != nil || p.Times == nil {
return budgetNone
}
if _, budgeted := deliveryBudgets[p.Times.Class]; !budgeted {
return budgetNone
}
for _, tier := range p.Tiers {
for _, m := range tier {
if rollsOut(m) {
return p.Times.Class
}
}
}
return budgetNone
}
// walkBound is a walk's bound by its budget class: twice its budget, or fifteen minutes with none.
func walkBound(class string) time.Duration {
if b, ok := deliveryBudgets[class]; ok {
return 2 * b
}
return walkNoBudgetBound
}
// walkSends is what a walk will send at its start, per module the machines it goes to: each module of its tiers
// whose policy rolls it out, on every machine running it (for a release plan, those of its order).
func walkSends(p inventory.Plan, running func(string) []string, rollsOut func(string) bool) map[string][]string {
out := map[string][]string{}
for _, tier := range p.Tiers {
for _, m := range tier {
if s := p.Modules[m]; s != nil && s.State == planDeleted {
continue
}
if p.Release == nil && !rollsOut(m) {
continue
}
for _, n := range running(m) {
if p.Release != nil && !slices.Contains(p.Release.Order, n) {
continue
}
out[m] = appendNew(out[m], n)
}
}
}
return out
}
// walkReached is what a walk sent by its end, per module the machines it reached: its first machines, its rest,
// and every module a gated send carried with it. A release plan reached the machines it walked: those done and
// the one being judged.
func walkReached(p inventory.Plan, running func(string) []string) map[string][]string {
out := map[string][]string{}
carried := func(g *inventory.PlanGate) {
if g == nil {
return
}
for _, c := range g.Carried {
out[c.Module] = appendNew(out[c.Module], c.Node)
}
}
if r := p.Release; r != nil {
reached := append([]string{}, r.Done...)
if r.Gate != nil {
reached = appendNew(reached, r.Gate.Machines...)
carried(r.Gate)
}
for _, tier := range p.Tiers {
for _, m := range tier {
for _, n := range running(m) {
if slices.Contains(reached, n) {
out[m] = appendNew(out[m], n)
}
}
}
}
return out
}
for m, s := range p.Modules {
if s == nil {
continue
}
out[m] = appendNew(out[m], s.First...)
for n := range s.Rest {
out[m] = appendNew(out[m], n)
}
carried(s.Gate)
if len(out[m]) == 0 {
delete(out, m)
}
}
return out
}
// touchesOf turns what a walk moves, per module, into what it touches, per machine: the modules it moves there,
// and, where it moves the tool runner, every module there whose tools it serves, and where it moves the
// node-engine, every module there. Sorted by machine and module.
func touchesOf(moved map[string][]string, on map[string][]string, shelf map[string]catalogue.Manifest) []walkTouch {
byMachine := map[string][]string{}
for m, machines := range moved {
for _, n := range machines {
byMachine[n] = appendNew(byMachine[n], m)
}
}
out := make([]walkTouch, 0, len(byMachine))
for n, modules := range byMachine {
t := walkTouch{Machine: n, Modules: modules}
engine, runner := slices.Contains(modules, hostModule), slices.Contains(modules, broker.RuntimeModule)
if engine || runner {
for _, m := range on[n] {
if slices.Contains(t.Modules, m) || (!engine && !servesTools(shelf[m])) {
continue
}
t.Modules = append(t.Modules, m)
t.Carried = append(t.Carried, m)
}
}
sort.Strings(t.Modules)
sort.Strings(t.Carried)
out = append(out, t)
}
sort.Slice(out, func(i, j int) bool { return out[i].Machine < out[j].Machine })
return out
}
// servesTools says the tool runner serves something of this module: a bundle it loads entrypoints from, or the
// module's own tools.
func servesTools(m catalogue.Manifest) bool {
if len(m.Tools) > 0 {
return true
}
for _, b := range m.Bundles {
if len(b.Loads) > 0 {
return true
}
}
return false
}
// walkRead is the facts a walk's touches are worked out from, read once per event.
type walkRead struct {
running func(string) []string
rollsOut func(string) bool
onMachine func(moved map[string][]string) map[string][]string
shelf func() map[string]catalogue.Manifest
}
// readWalkFacts reads what the walk's touches need. Without facts (a test, a command with no store), nothing runs
// anywhere and every module rolls out: the touches are then what the walk itself recorded.
func readWalkFacts(ctx context.Context, facts walkFacts, p inventory.Plan) walkRead {
cantRead := func(what string, err error) {
fmt.Fprintf(os.Stderr, "%s: %s cannot be read for what the walk touches: %v\n", p.ID, what, err)
}
r := walkRead{
running: func(string) []string { return nil },
rollsOut: func(string) bool { return true },
onMachine: func(map[string][]string) map[string][]string { return nil },
shelf: func() map[string]catalogue.Manifest { return nil },
}
if facts == nil {
return r
}
running := map[string][]string{}
r.running = func(m string) []string {
if on, read := running[m]; read {
return on
}
on, err := facts.Running(ctx, m)
if err != nil {
cantRead("where "+m+" runs", err)
}
running[m] = on
return on
}
rolls := map[string]bool{}
r.rollsOut = func(m string) bool {
if yes, read := rolls[m]; read {
return yes
}
u, err := facts.UpgradeOf(ctx, m)
if err != nil {
cantRead(m+"'s policy", err)
}
// A policy that cannot be read is taken to roll out: the walk is then held more, never less.
rolls[m] = err != nil || u.RollOut
return rolls[m]
}
r.onMachine = func(moved map[string][]string) map[string][]string {
out := map[string][]string{}
for _, core := range []string{hostModule, broker.RuntimeModule} {
for _, n := range moved[core] {
if _, read := out[n]; read {
continue
}
var there []string
if sent, known, err := facts.SentBuilds(ctx, n); err != nil {
cantRead("what "+n+" was sent", err)
} else if known {
for m := range sent {
there = append(there, m)
}
}
if assigned, err := facts.Assigned(ctx, n); err != nil {
cantRead("what "+n+" is assigned", err)
} else {
there = appendNew(there, assigned...)
}
sort.Strings(there)
out[n] = there
}
}
return out
}
r.shelf = func() map[string]catalogue.Manifest {
shelf, err := facts.Catalogue(ctx)
if err != nil {
cantRead("the catalogue", err)
}
return shelf
}
return r
}
+514
View File
@@ -0,0 +1,514 @@
package main
import (
"context"
"encoding/json"
"os"
"reflect"
"testing"
"time"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
"github.com/novox/mesh-controller/internal/link"
"github.com/novox/mesh-controller/internal/testbus"
)
// fakeWalkFacts is the inventory as far as a walk's touches read it.
type fakeWalkFacts struct {
running map[string][]string
records map[string]bool // modules whose policy records rather than rolls out
sent map[string]map[string]string
assigned map[string][]string
shelf map[string]catalogue.Manifest
}
func (f fakeWalkFacts) Running(_ context.Context, m string) ([]string, error) {
return f.running[m], nil
}
func (f fakeWalkFacts) UpgradeOf(_ context.Context, m string) (inventory.Upgrade, error) {
return inventory.Upgrade{RollOut: !f.records[m]}, nil
}
func (f fakeWalkFacts) Assigned(_ context.Context, n string) ([]string, error) {
return f.assigned[n], nil
}
func (f fakeWalkFacts) SentBuilds(_ context.Context, n string) (map[string]string, bool, error) {
s, ok := f.sent[n]
return s, ok, nil
}
func (f fakeWalkFacts) Catalogue(context.Context) (map[string]catalogue.Manifest, error) {
return f.shelf, nil
}
// theMesh is four machines: the tool runner on each, messenger and mesh-issues on novox, gitea on ace.
func theMesh() fakeWalkFacts {
return fakeWalkFacts{
running: map[string][]string{
broker.RuntimeModule: {"ace", "g14", "novox", "shanks"},
"messenger": {"novox"},
"gitea": {"ace"},
"plex": {"ace"},
},
sent: map[string]map[string]string{
"novox": {broker.RuntimeModule: "c0", "messenger": "c1", "mesh-issues": "c2", "postgres": "c3"},
"ace": {broker.RuntimeModule: "c0", "gitea": "c4"},
},
assigned: map[string][]string{"g14": {broker.RuntimeModule, "desktop"}, "shanks": {broker.RuntimeModule}},
shelf: map[string]catalogue.Manifest{
"messenger": {Module: "messenger", Tools: []string{"notify"}},
"mesh-issues": {Module: "mesh-issues", Bundles: []catalogue.Bundle{{Name: "tools", Loads: []string{"main"}}}},
"postgres": {Module: "postgres"},
"gitea": {Module: "gitea", Tools: []string{"gitea_api"}},
"desktop": {Module: "desktop", Tools: []string{"notify_send"}},
},
}
}
func walkOfTiers(class string, tiers ...[]string) inventory.Plan {
cut := time.Date(2026, 10, 12, 11, 0, 0, 0, time.UTC)
started := cut.Add(time.Second)
p := inventory.Plan{ID: "plan-308", State: inventory.PlanBuilding, Tiers: tiers,
Modules: map[string]*inventory.PlanModule{}, Times: &inventory.PlanTimes{Cut: &cut, Class: class, Started: &started}}
for _, tier := range tiers {
for _, m := range tier {
p.Modules[m] = &inventory.PlanModule{}
}
}
p.WalkSaid = inventory.WalkStarted
return p
}
// novox/hq ADR 0308 How it is checked §1: the start names the modules the walk moves and the machines it sends
// them to, and every module a moved tool runner carries there.
func TestAWalksStartNamesWhatItTouchesWithWhatTheToolRunnerCarries(t *testing.T) {
p := walkOfTiers(inventory.ClassCore, []string{broker.RuntimeModule}, []string{"messenger"})
w := sayWalk(t.Context(), theMesh(), p)
if w == nil || w.Walk != "plan-308" || w.Phase != inventory.WalkStarted || w.Outcome != "" {
t.Fatalf("the start is said: %+v", w)
}
want := []walkTouch{
{Machine: "ace", Modules: []string{"gitea", broker.RuntimeModule}, Carried: []string{"gitea"}},
{Machine: "g14", Modules: []string{"desktop", broker.RuntimeModule}, Carried: []string{"desktop"}},
// postgres serves no tools: the tool runner does not carry it.
{Machine: "novox", Modules: []string{"mesh-issues", "messenger", broker.RuntimeModule}, Carried: []string{"mesh-issues"}},
{Machine: "shanks", Modules: []string{broker.RuntimeModule}},
}
if !reflect.DeepEqual(w.Touches, want) {
t.Fatalf("touches:\n got %+v\nwant %+v", w.Touches, want)
}
if w.BudgetClass != inventory.ClassCore || w.BoundSeconds != 1200 || w.StartedAt == nil {
t.Fatalf("a core walk is bound at 20 minutes: %+v", w)
}
}
// A moved node-engine carries every module on its machine; a module whose policy records is not sent, and not
// touched.
func TestAMovedNodeEngineCarriesEverythingOnItsMachine(t *testing.T) {
f := theMesh()
f.running[hostModule] = []string{"novox"}
f.records = map[string]bool{"plex": true}
p := walkOfTiers(inventory.ClassCore, []string{hostModule, "plex"})
w := sayWalk(t.Context(), f, p)
want := []walkTouch{{Machine: "novox",
Modules: []string{"mesh-host", "mesh-issues", "messenger", "node-tools", "postgres"},
Carried: []string{"mesh-issues", "messenger", "node-tools", "postgres"}}}
if !reflect.DeepEqual(w.Touches, want) {
t.Fatalf("touches:\n got %+v\nwant %+v", w.Touches, want)
}
}
// The end names what the walk reached — its first machines, its rest, what a gated send carried — and its
// outcome; the modules a reached tool runner carries are touched there too.
func TestAWalksEndNamesWhatItReachedAndItsOutcome(t *testing.T) {
p := walkOfTiers(inventory.ClassCore, []string{broker.RuntimeModule}, []string{"messenger"})
ended := p.Times.Started.Add(4 * time.Minute)
p.Times.Ended = &ended
p.State, p.WalkSaid = inventory.PlanDone, inventory.WalkEnded
p.Modules[broker.RuntimeModule] = &inventory.PlanModule{First: []string{"shanks"},
Rest: map[string]inventory.SentDeclaration{"novox": {}},
Gate: &inventory.PlanGate{Carried: []inventory.CarriedMove{{Module: "gitea", Node: "shanks"}}}}
// messenger was built and found unchanged: no move, not reached.
p.Modules["messenger"] = &inventory.PlanModule{Why: "no move"}
w := sayWalk(t.Context(), theMesh(), p)
if w.Phase != inventory.WalkEnded || w.Outcome != inventory.WalkDelivered || w.EndedAt == nil {
t.Fatalf("the end is said with its outcome: %+v", w)
}
want := []walkTouch{
{Machine: "novox", Modules: []string{"mesh-issues", "messenger", broker.RuntimeModule},
Carried: []string{"mesh-issues", "messenger"}},
{Machine: "shanks", Modules: []string{"gitea", broker.RuntimeModule}},
}
if !reflect.DeepEqual(w.Touches, want) {
t.Fatalf("touches:\n got %+v\nwant %+v", w.Touches, want)
}
for outcome, end := range map[string]func(*inventory.Plan){
inventory.WalkFailed: func(p *inventory.Plan) { p.State = inventory.PlanFailed },
inventory.WalkStopped: func(p *inventory.Plan) {
p.State, p.Delivery = inventory.PlanFailed, &inventory.PlanDelivery{Stopped: "a person"}
},
} {
q := p
end(&q)
if got := sayWalk(t.Context(), theMesh(), q).Outcome; got != outcome {
t.Errorf("outcome %q, want %q", got, outcome)
}
}
}
// novox/hq ADR 0308 How it is checked §1 and decision 5: the bound is twice the largest budget in force among
// the walk's deliveries — 20 minutes for a core walk, twice the leaf budget for a leaf walk — and 15 minutes, not
// doubled, for a walk none of whose deliveries has a budget.
func TestAWalksBoundIsTwiceItsBudgetOrFifteenMinutesWithNone(t *testing.T) {
f := theMesh()
f.records = map[string]bool{"plex": true}
release := walkOfTiers("", []string{"gitea"})
release.Times, release.Release = &inventory.PlanTimes{}, &inventory.PlanRelease{Order: []string{"ace"}}
for name, c := range map[string]struct {
p inventory.Plan
class string
bound int
}{
"core": {walkOfTiers(inventory.ClassCore, []string{"messenger"}), inventory.ClassCore, 1200},
"leaf": {walkOfTiers(inventory.ClassLeaf, []string{"messenger"}), inventory.ClassLeaf, int(2 * deliveryBudgets[inventory.ClassLeaf] / time.Second)},
"held for a person": {walkOfTiers(inventory.ClassLeaf, []string{"plex"}), budgetNone, 900},
"a release": {release, budgetNone, 900},
"kept without a class": {walkOfTiers("", []string{"messenger"}), budgetNone, 900},
} {
w := sayWalk(t.Context(), f, c.p)
if w.BudgetClass != c.class || w.BoundSeconds != c.bound {
t.Errorf("%s: %s bound %ds, want %s %ds", name, w.BudgetClass, w.BoundSeconds, c.class, c.bound)
}
}
// The leaf budget in force is three minutes (ADR 0304 decision 4): a leaf walk is bound at six.
if deliveryBudgets[inventory.ClassLeaf] == 3*time.Minute {
if w := sayWalk(t.Context(), f, walkOfTiers(inventory.ClassLeaf, []string{"messenger"})); w.BoundSeconds != 360 {
t.Errorf("a leaf walk at a three-minute budget is bound at 360 s: %d", w.BoundSeconds)
}
}
}
// plan-moved's body: the walk whole as before, and on its start the walk's fields beside it; on any other save
// none of them.
func TestPlanMovedCarriesTheWalkWholeAndOnItsStartWhatItTouches(t *testing.T) {
p := walkOfTiers(inventory.ClassLeaf, []string{"messenger"})
body, err := json.Marshal(planMovedBody{Plan: p, walkSaid: sayWalk(t.Context(), theMesh(), p)})
if err != nil {
t.Fatal(err)
}
var said map[string]any
if err := json.Unmarshal(body, &said); err != nil {
t.Fatal(err)
}
for _, k := range []string{"id", "state", "tiers", "modules", "times", "walk", "phase", "touches", "bound_seconds", "budget_class", "started_at"} {
if _, ok := said[k]; !ok {
t.Errorf("plan-moved lacks %q: %s", k, body)
}
}
if _, ok := said["outcome"]; ok {
t.Errorf("a start says no outcome: %s", body)
}
if said["walk"] != "plan-308" || said["phase"] != "started" || said["budget_class"] != "leaf" {
t.Errorf("the walk's fields: %s", body)
}
p.WalkSaid = ""
body, _ = json.Marshal(planMovedBody{Plan: p, walkSaid: sayWalk(t.Context(), theMesh(), p)})
said = map[string]any{}
_ = json.Unmarshal(body, &said)
for _, k := range []string{"walk", "phase", "touches", "bound_seconds"} {
if _, ok := said[k]; ok {
t.Errorf("a save neither starting nor ending the walk says %q: %s", k, body)
}
}
// A walk that touches nothing says an empty list, never null.
q := walkOfTiers(inventory.ClassLeaf, []string{"nobody-runs-this"})
body, _ = json.Marshal(planMovedBody{Plan: q, walkSaid: sayWalk(t.Context(), theMesh(), q)})
said = map[string]any{}
_ = json.Unmarshal(body, &said)
if touches, ok := said["touches"].([]any); !ok || len(touches) != 0 {
t.Errorf("touches nothing: %s", body)
}
}
// Live, with the store and a bus: a walk let go by the delivery's verb is said started with what it touches and
// its bound, and stopped by the verb is said ended, stopped (novox/hq ADR 0308 How it is checked §1).
func TestAWalkLetGoAndStoppedByVerbsIsSaidStartedAndEnded(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
next := planMovedHeard(t)
register(t, open, catalogue.Manifest{Module: "app", Version: "1"})
if _, err := assign(ctx, open, "anchor", "app"); err != nil {
t.Fatal(err)
}
cut := time.Now().UTC()
waiting := inventory.Plan{ID: "plan-308-live", Repository: "novox/mesh-catalog", Branch: "main", Commit: "c8c8c8c8",
Created: cut, State: inventory.PlanBuilding, Tiers: [][]string{{"app"}},
Modules: map[string]*inventory.PlanModule{"app": {}},
Delivery: &inventory.PlanDelivery{Awaits: catalogue.DeliverySeat},
Times: &inventory.PlanTimes{Cut: &cut, Class: inventory.ClassLeaf}}
if err := open.inventory.SavePlan(ctx, &waiting); err != nil {
t.Fatal(err)
}
if err := deliveryCommand(ctx, []string{"go", waiting.ID, "--why", "its turn"}); err != nil {
t.Fatal(err)
}
started := next(inventory.WalkStarted)
touches, _ := json.Marshal(started["touches"])
if started["walk"] != waiting.ID || string(touches) != `[{"machine":"anchor","modules":["app"]}]` ||
started["budget_class"] != inventory.ClassLeaf ||
started["bound_seconds"] != float64(2*deliveryBudgets[inventory.ClassLeaf]/time.Second) {
t.Fatalf("the start: %v", started)
}
if err := deliveryCommand(ctx, []string{"stop", waiting.ID, "--why", "a test", "--by", "a person"}); err != nil {
t.Fatal(err)
}
ended := next(inventory.WalkEnded)
touches, _ = json.Marshal(ended["touches"])
if ended["walk"] != waiting.ID || ended["outcome"] != inventory.WalkStopped || string(touches) != `[]` {
t.Fatalf("the end, which sent nothing: %v", ended)
}
}
// planMovedHeard raises a bus of the test's own, points the verbs' hand-act connection at it, and returns what
// waits for the next plan-moved said with the given phase, failing the test when none comes within 5 s.
func planMovedHeard(t *testing.T) func(phase string) map[string]any {
t.Helper()
js, err := broker.Dial(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
t.Cleanup(js.Close)
if err := broker.AssertMeshStreams(js); err != nil {
t.Fatal(err)
}
before := handActConn
handActConn = js.Conn()
t.Cleanup(func() { handActConn = before })
heard := make(chan *nats.Msg, 64)
sub, err := js.Conn().ChanSubscribe(link.SeatEventSubject(link.MeshControllerSeat, link.KeyPlanMoved), heard)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = sub.Unsubscribe() })
return func(phase string) map[string]any {
t.Helper()
deadline := time.After(5 * time.Second)
for {
select {
case m := <-heard:
var said map[string]any
if err := json.Unmarshal(m.Data, &said); err != nil {
t.Fatal(err)
}
if said["phase"] == phase {
return said
}
case <-deadline:
t.Fatalf("no walk was said %s", phase)
}
}
}
}
// Live, with the store and a bus: a walk a person stops with `plans stop` is said ended, outcome stopped, by the
// verb itself, and is kept stopped by that person; `plans close` ends one the same way (novox/hq ADR 0308
// decision 1, review 15 on mesh-controller#247).
func TestAWalkStoppedOrClosedByAPersonIsSaidEndedStopped(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
next := planMovedHeard(t)
register(t, open, catalogue.Manifest{Module: "app", Version: "1"})
if _, err := assign(ctx, open, "anchor", "app"); err != nil {
t.Fatal(err)
}
for _, verb := range []string{"stop", "close"} {
cut := time.Now().UTC()
walk := inventory.Plan{ID: "plan-308-" + verb, Repository: "novox/mesh-catalog", Branch: "main",
Commit: "c8c8c8c8", Created: cut, State: inventory.PlanBuilding, Tiers: [][]string{{"app"}},
Modules: map[string]*inventory.PlanModule{"app": {}},
Times: &inventory.PlanTimes{Cut: &cut, Started: &cut, Class: inventory.ClassLeaf}}
if err := open.inventory.SavePlan(ctx, &walk); err != nil {
t.Fatal(err)
}
if err := plansCommand(ctx, []string{verb, walk.ID, "--why", "a test"}); err != nil {
t.Fatal(err)
}
ended := next(inventory.WalkEnded)
if ended["walk"] != walk.ID || ended["outcome"] != inventory.WalkStopped {
t.Fatalf("plans %s: the end is %v", verb, ended)
}
p, err := open.inventory.PlanByID(ctx, walk.ID)
if err != nil {
t.Fatal(err)
}
if p.Outcome() != inventory.WalkStopped || p.Delivery == nil || p.Delivery.StoppedWhy != "a test" {
t.Fatalf("plans %s kept the walk %s, %+v", verb, p.Outcome(), p.Delivery)
}
}
}
// Live, with the store and a bus: a failed walk retried with `plans retry` starts again, and the verb says its
// start (novox/hq ADR 0308 decision 1, review 15 on mesh-controller#247).
func TestAWalkRetriedByAPersonIsSaidStartedAgain(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
asksRecorded(t)
next := planMovedHeard(t)
twoTiers(t, open)
before := time.Now().UTC().Add(-time.Hour)
ended := before.Add(time.Minute)
failed := inventory.Plan{ID: "plan-308-retry", Repository: "novox/a", Branch: "main", Commit: "c0ffee",
Created: before, State: inventory.PlanFailed, Tier: 0, Tiers: [][]string{{"a"}, {"b"}},
Note: "a failed to build in tier 0",
Modules: map[string]*inventory.PlanModule{"a": {State: "failed", AskedAt: &before, Build: "build-1",
Why: link.KilledByHand}},
Times: &inventory.PlanTimes{Cut: &before, Started: &before, Ended: &ended, Class: inventory.ClassLeaf}}
if err := open.inventory.SavePlan(ctx, &failed); err != nil {
t.Fatal(err)
}
if err := plansCommand(ctx, []string{"retry", failed.ID}); err != nil {
t.Fatal(err)
}
started := next(inventory.WalkStarted)
if started["walk"] != failed.ID || started["budget_class"] != inventory.ClassLeaf {
t.Fatalf("the retried start: %v", started)
}
}
// Live, with the store and a bus: the release plan `upgrade release-backlog` opens is said started by the verb
// (novox/hq ADR 0308 decision 1, review 15 on mesh-controller#247).
func TestABacklogReleasedByAPersonIsSaidStarted(t *testing.T) {
b := aBacklog(t)
next := planMovedHeard(t)
if err := backlogCommand(t.Context(), "release-backlog", []string{"--why", "a test"}); err != nil {
t.Fatal(err)
}
started := next(inventory.WalkStarted)
if p := b.release(t); started["walk"] != p.ID {
t.Fatalf("the release's start: %v", started)
}
}
// asTyped runs a verb as the controller's command line runs it, through run, as a person's terminal would.
func asTyped(t *testing.T, args ...string) error {
t.Helper()
was := os.Args
os.Args = append([]string{"mesh-controller"}, args...)
t.Cleanup(func() { os.Args = was })
err := run()
// run gives back the lease as the process would end; the test's process goes on, and takes it again.
theLease.mu.Lock()
theLease.held, theLease.kv, theLease.noBus = nil, nil, false
theLease.mu.Unlock()
return err
}
// aStartedWalkAsking keeps a live walk, said started, whose first tier asked a as build.
func aStartedWalkAsking(t *testing.T, open *stores, id, build string, asked time.Time) inventory.Plan {
t.Helper()
walk := inventory.Plan{ID: id, Repository: "novox/a", Branch: "main", Commit: "c0ffee", Created: asked,
State: inventory.PlanBuilding, Tiers: [][]string{{"a"}, {"b"}},
Modules: map[string]*inventory.PlanModule{"a": {State: "asked", AskedAt: &asked, Build: build}},
Times: &inventory.PlanTimes{Cut: &asked, Started: &asked, Class: inventory.ClassLeaf}}
if err := open.inventory.SavePlan(t.Context(), &walk); err != nil {
t.Fatal(err)
}
return walk
}
// Live, with the store and a bus: `cancel <build>` and `clear` fail the walk that asked the build, and the verb
// says its end, failed (novox/hq ADR 0308 decision 1, review 21 on mesh-controller#247).
func TestAWalkEndedByCancelOrClearIsSaidEnded(t *testing.T) {
_, ask := aBuildQueue(t)
open := aMesh(t)
next := planMovedHeard(t)
twoTiers(t, open)
for _, verb := range [][]string{{"cancel"}, {"clear"}} {
id := link.NewBuildID(time.Now())
ask(id, "https://forge.example/novox/a.git")
asked, _ := link.BuildAskedAt(id)
walk := aStartedWalkAsking(t, open, "plan-308-"+verb[0], id, asked)
args := verb
if verb[0] == "cancel" {
args = append(args, id)
}
if err := asTyped(t, args...); err != nil {
t.Fatal(err)
}
ended := next(inventory.WalkEnded)
if ended["walk"] != walk.ID || ended["outcome"] != inventory.WalkFailed {
t.Fatalf("%s: the end is %v", verb[0], ended)
}
// Ended, and so the next walk may be cut on the same repository.
p, _ := open.inventory.PlanByID(t.Context(), walk.ID)
if p.State != inventory.PlanFailed {
t.Fatalf("%s left the walk %s", verb[0], p.State)
}
}
}
// Live, with the store and a bus: `rebuild <module>` joining a failed walk sets it building again, and the verb
// says its start (novox/hq ADR 0308 decision 1, review 21 on mesh-controller#247).
func TestAWalkRestartedByARebuildIsSaidStarted(t *testing.T) {
open := aMesh(t)
asksRecorded(t)
next := planMovedHeard(t)
twoTiers(t, open)
before := time.Now().UTC().Add(-time.Hour)
ended := before.Add(time.Minute)
failed := inventory.Plan{ID: "plan-308-rebuild", Repository: "novox/a", Branch: "main", Commit: "c0ffee",
Created: before, State: inventory.PlanFailed, Tiers: [][]string{{"a"}, {"b"}},
Modules: map[string]*inventory.PlanModule{"a": {State: "failed", AskedAt: &before, Build: "build-1",
Why: link.CancelledByHand}},
Times: &inventory.PlanTimes{Cut: &before, Started: &before, Ended: &ended, Class: inventory.ClassLeaf}}
if err := open.inventory.SavePlan(t.Context(), &failed); err != nil {
t.Fatal(err)
}
if err := asTyped(t, "rebuild", "a"); err != nil {
t.Fatal(err)
}
if started := next(inventory.WalkStarted); started["walk"] != failed.ID {
t.Fatalf("the rebuilt walk's start: %v", started)
}
}
// Live, with the store: a walk a person stopped and then retried, by `plans retry` or by `rebuild`, walks again
// without the stop, so when it fails again it ends failed, not stopped, and the search for the merge that
// brought the failure is not kept from it (novox/hq ADR 0308 decision 1, review 21 on mesh-controller#247).
func TestAWalkStoppedByHandAndRetriedEndsFailedWhenItFailsAgain(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
asksRecorded(t)
twoTiers(t, open)
for _, again := range [][]string{{"plans", "retry"}, {"rebuild", "a"}} {
before := time.Now().UTC().Add(-time.Hour)
walk := aStartedWalkAsking(t, open, "plan-308-again-"+again[0], "build-1", before)
if err := asTyped(t, "plans", "stop", walk.ID, "--why", "a test"); err != nil {
t.Fatal(err)
}
// The build asked before the stop fails after it.
p, _ := open.inventory.PlanByID(ctx, walk.ID)
p.Modules["a"].State, p.Modules["a"].Why = "failed", "it broke"
if err := open.inventory.SavePlan(ctx, &p); err != nil {
t.Fatal(err)
}
args := again
if again[0] == "plans" {
args = append(append([]string(nil), again...), walk.ID)
}
if err := asTyped(t, args...); err != nil {
t.Fatal(err)
}
p, _ = open.inventory.PlanByID(ctx, walk.ID)
if p.State != inventory.PlanBuilding || (p.Delivery != nil && p.Delivery.Stopped != "") {
t.Fatalf("%s: the walk is %s and keeps its stop: %+v", again[0], p.State, p.Delivery)
}
asking, _ := link.BuildAskedAt(p.Modules["a"].Build)
planBuilt(ctx, open, "a", "c0ffee", "it broke again", asking, p.Modules["a"].Build)
if p, _ = open.inventory.PlanByID(ctx, walk.ID); p.State != inventory.PlanFailed || p.Outcome() != inventory.WalkFailed {
t.Fatalf("%s: failing again, the walk is %s and ends %s", again[0], p.State, p.Outcome())
}
}
}
+5
View File
@@ -59,6 +59,10 @@ type signalFacts struct {
// from, each naming its sender (S20, novox/hq issue 234).
refusedSends []inventory.Send
refusedSendsErr error
// doing is every machine whose last report was not a clean apply, with since when it has said the same
// failure (S21, novox/hq issue 233).
doing []inventory.Doing
doingErr error
loop loopFacts
loopErr error
@@ -369,6 +373,7 @@ func (w *watchdogs) gather(ctx context.Context) *signalFacts {
}
f.handActs, f.handActsErr = w.gatherHandActs(ctx, now)
f.refusedSends, f.refusedSendsErr = inv.RefusedSendsSince(ctx, now.Add(-generationRefusalsSaid))
f.doing, f.doingErr = inv.NotDoingWhatTheyWereTold(ctx)
f.facts.taken, _, f.facts.began, f.facts.err = exportedFacts.last()
return f
}
+1 -1
View File
@@ -35,4 +35,4 @@ require (
// committed. Every build (the build agent's `go build`, the Dockerfile) compiles from vendor/ and
// fetches nothing; go refuses to build when vendor/ and this file disagree, so a pin moved without
// `go mod vendor` fails loudly, at once, everywhere.
replace github.com/novox/mesh-host => git.novox.be/novox/mesh-host v0.0.0-20261011092230-20d5af9b2d5f
replace github.com/novox/mesh-host => git.novox.be/novox/mesh-host v0.0.0-20261012175741-303d4037e84e
+2 -2
View File
@@ -1,5 +1,5 @@
git.novox.be/novox/mesh-host v0.0.0-20261011092230-20d5af9b2d5f h1:8N5OW2mdTNIck2pe4EciTYX5NsrPsHrTLENGNIWYNTU=
git.novox.be/novox/mesh-host v0.0.0-20261011092230-20d5af9b2d5f/go.mod h1:tcTK4LMs1d6JpZUwy3hSHMFG/MIuDr/XFs7/nbeaW/c=
git.novox.be/novox/mesh-host v0.0.0-20261012175741-303d4037e84e h1:RxvY7T12EizGs2e9W1dVZFEnQFJBPNKRY+oUPgovcBc=
git.novox.be/novox/mesh-host v0.0.0-20261012175741-303d4037e84e/go.mod h1:tcTK4LMs1d6JpZUwy3hSHMFG/MIuDr/XFs7/nbeaW/c=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010191001-33917f91ac3c h1:l5onJwoIeH8yE/PjVlEeoPyXBsHp2NQiWLllz2fwX7s=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261010191001-33917f91ac3c/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
+21 -2
View File
@@ -262,9 +262,12 @@ func MaySubscribe(perms Permissions, subject string) bool {
// delivery held past its bound, and calls them on the operator's warrant, with its why.
//
// And, since novox/hq ADR 0282, `times`: the self-check reads the delivery times to say a delivery over its budget.
//
// And, since novox/hq ADR 0306, `groups`: a merge heard from a head waits for the window while its delivery group
// has a member still to merge.
var VerbsTheControllerAsksTheDeliveryOwner = []SeatVerb{{Seat: "mesh-delivery", Verb: "stalled"},
{Seat: "mesh-delivery", Verb: "close"}, {Seat: "mesh-delivery", Verb: "release"}, {Seat: "mesh-delivery", Verb: "stop"},
{Seat: "mesh-delivery", Verb: "times"}}
{Seat: "mesh-delivery", Verb: "times"}, {Seat: "mesh-delivery", Verb: "groups"}}
// VerbsTheControllerActsOnAWarrant are the other seat verbs the controller calls when the operator's warrant
// chooses them (novox/hq ADR 0259): a machine's service restarted, and a walk started or stopped through the
@@ -333,6 +336,18 @@ func AskHandOverSubject(node string) string { return "mesh.node." + node + ".ask
// a hand-over is (link.SetuidSearchContext), for the same reason.
func AskSetuidSearchSubject(node string) string { return "mesh.node." + node + ".ask.setuid-search" }
// AskJudgeDirectoriesSubject is where the controller asks one machine's node-engine to judge, read-only, which of
// its declared directories the walk that follows no link another account could make would refuse (novox/hq issue
// 358): a request answered once, signed as a hand-over is (link.JudgeDirectoriesContext), for the same reason.
func AskJudgeDirectoriesSubject(node string) string {
return "mesh.node." + node + ".ask.judge-directories"
}
// AskAccessActSubject is where the controller asks one machine's node-engine to take a shared folder on the
// operator's warrant (novox/hq ADR 0325 §8; mesh-host internal/link AskAccessActSubject): a request answered once,
// signed as a hand-over is (link.AccessActContext), for the same reason.
func AskAccessActSubject(node string) string { return "mesh.node." + node + ".ask.access-act" }
// inbox is a principal's own reply space. No user is ever granted a bare `_INBOX.>` (design 25
// §4): with one account, inbox privacy is the permission list or it is nothing, so each user's
// inbox is derived from its own identity and its permissions name that prefix and no other.
@@ -611,7 +626,11 @@ func PermissionsFor(p Principal) (Permissions, error) {
// issue 356), which it answers on the request's reply: the one request a node is asked.
AskHandOverSubject(p.Node),
// And asking it for a fresh search for setuid programs (novox/hq issue 361), answered the same way.
AskSetuidSearchSubject(p.Node)}
AskSetuidSearchSubject(p.Node),
// And asking it to judge its declared directories, read-only (novox/hq issue 358), answered the same way.
AskJudgeDirectoriesSubject(p.Node),
// And asking it to take a shared folder on the operator's warrant (novox/hq ADR 0325 §8), answered the same way.
AskAccessActSubject(p.Node)}
// The node-engine witnesses the core builds it places (novox/hq to-be 45 §8, ADR 0236; the
// contract is lease/witness.go): it asks its own machine's node tools PING, and, where the
// machine runs the controller, reads the lease's one key — read, never written.
+3 -1
View File
@@ -114,7 +114,9 @@ func TestOnlyWhatCanBeAskedMayAnswer(t *testing.T) {
node, _ := PermissionsFor(Principal{Kind: KindNode, Node: "one", PasswordHash: "x"})
if !node.AllowResponses || !slices.Contains(node.Subscribe, AskHandOverSubject("one")) ||
!slices.Contains(node.Subscribe, AskSetuidSearchSubject("one")) ||
slices.Contains(node.Subscribe, AskSetuidSearchSubject("two")) {
slices.Contains(node.Subscribe, AskSetuidSearchSubject("two")) ||
!slices.Contains(node.Subscribe, AskJudgeDirectoriesSubject("one")) ||
slices.Contains(node.Subscribe, AskJudgeDirectoriesSubject("two")) {
t.Fatalf("a node cannot answer the hand-over it is asked: %v %v", node.AllowResponses, node.Subscribe)
}
person, _ := PermissionsFor(Principal{Kind: KindPerson, Node: "one", Module: "jo", PasswordHash: "x"})
+2 -2
View File
@@ -24,7 +24,7 @@ accounts {
jetstream: enabled
users = [
{ user: "controller", password: "$2a$11$cccccccccccccccccccccc", permissions: {
publish: { allow: ["$JS.ACK.CONTROL.controller.>", "$JS.ACK.DEAD_LETTER_NOTICES.controller.>", "$JS.ACK.EVENTS.controller.>", "$JS.API.>", "$KV.SEAT_MESH_BUILD_MACHINE_cancelled.>", "$KV.SEAT_NODE_BUILD_AGENT_cancelled.>", "$KV.mesh-controller_asked.>", "$KV.mesh-controller_calls.>", "$KV.mesh-controller_condition-history.>", "$KV.mesh-controller_conditions.>", "$KV.mesh-controller_hand-acts.>", "$KV.mesh-controller_lease.>", "$SRV.INFO", "_INBOX.enrol.>", "mesh.again.>", "mesh.assignment.>", "mesh.events.dead.>", "mesh.mod.*.tool.>", "mesh.node.>", "mesh.seat.mesh-build-machine.accept.>", "mesh.seat.mesh-build-machine.tool.>", "mesh.seat.mesh-controller.event.applied", "mesh.seat.mesh-controller.event.built-before", "mesh.seat.mesh-controller.event.checked", "mesh.seat.mesh-controller.event.condition-changed", "mesh.seat.mesh-controller.event.condition-cleared", "mesh.seat.mesh-controller.event.condition-raised", "mesh.seat.mesh-controller.event.doctor-heartbeat", "mesh.seat.mesh-controller.event.healer-acted", "mesh.seat.mesh-controller.event.plan-moved", "mesh.seat.mesh-controller.event.refused", "mesh.seat.mesh-controller.event.rolled-back", "mesh.seat.mesh-controller.event.secret-replaced", "mesh.seat.mesh-controller.tool.plans", "mesh.seat.mesh-delivery.tool.close", "mesh.seat.mesh-delivery.tool.release", "mesh.seat.mesh-delivery.tool.stalled", "mesh.seat.mesh-delivery.tool.stop", "mesh.seat.mesh-delivery.tool.times", "mesh.seat.node-backup.tool.backed-up.*", "mesh.seat.node-backup.tool.now.*", "mesh.seat.node-build-agent.accept.>", "mesh.seat.node-build-agent.tool.>", "mesh.seat.node-intrusion-prevention.tool.banned.*", "mesh.seat.node-launcher.tool.secret.*", "mesh.seat.node-service-manager.tool.restart.*"] }
publish: { allow: ["$JS.ACK.CONTROL.controller.>", "$JS.ACK.DEAD_LETTER_NOTICES.controller.>", "$JS.ACK.EVENTS.controller.>", "$JS.API.>", "$KV.SEAT_MESH_BUILD_MACHINE_cancelled.>", "$KV.SEAT_NODE_BUILD_AGENT_cancelled.>", "$KV.mesh-controller_asked.>", "$KV.mesh-controller_calls.>", "$KV.mesh-controller_condition-history.>", "$KV.mesh-controller_conditions.>", "$KV.mesh-controller_hand-acts.>", "$KV.mesh-controller_lease.>", "$SRV.INFO", "_INBOX.enrol.>", "mesh.again.>", "mesh.assignment.>", "mesh.events.dead.>", "mesh.mod.*.tool.>", "mesh.node.>", "mesh.seat.mesh-build-machine.accept.>", "mesh.seat.mesh-build-machine.tool.>", "mesh.seat.mesh-controller.event.applied", "mesh.seat.mesh-controller.event.built-before", "mesh.seat.mesh-controller.event.checked", "mesh.seat.mesh-controller.event.condition-changed", "mesh.seat.mesh-controller.event.condition-cleared", "mesh.seat.mesh-controller.event.condition-raised", "mesh.seat.mesh-controller.event.doctor-heartbeat", "mesh.seat.mesh-controller.event.healer-acted", "mesh.seat.mesh-controller.event.plan-moved", "mesh.seat.mesh-controller.event.refused", "mesh.seat.mesh-controller.event.rolled-back", "mesh.seat.mesh-controller.event.secret-replaced", "mesh.seat.mesh-controller.tool.plans", "mesh.seat.mesh-delivery.tool.close", "mesh.seat.mesh-delivery.tool.groups", "mesh.seat.mesh-delivery.tool.release", "mesh.seat.mesh-delivery.tool.stalled", "mesh.seat.mesh-delivery.tool.stop", "mesh.seat.mesh-delivery.tool.times", "mesh.seat.node-backup.tool.backed-up.*", "mesh.seat.node-backup.tool.now.*", "mesh.seat.node-build-agent.accept.>", "mesh.seat.node-build-agent.tool.>", "mesh.seat.node-intrusion-prevention.tool.banned.*", "mesh.seat.node-launcher.tool.secret.*", "mesh.seat.node-service-manager.tool.restart.*"] }
subscribe: { allow: ["$JS.API.>", "$JS.EVENT.ADVISORY.CONSUMER.DELETED.>", "$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES.>", "$SRV.INFO", "$SRV.INFO.mesh-controller", "$SRV.INFO.mesh-controller.>", "$SRV.PING", "$SRV.PING.mesh-controller", "$SRV.PING.mesh-controller.>", "$SRV.STATS", "$SRV.STATS.mesh-controller", "$SRV.STATS.mesh-controller.>", "_DELIVER.controller", "_DELIVER.controller.>", "_INBOX.controller.>", "mesh.control.>", "mesh.mod.*.event.provisioner.failing", "mesh.mod.*.event.provisioner.recovered", "mesh.mod.*.event.provisioner.retirement", "mesh.mod.gitea.event.pull.merged", "mesh.mod.gitea.event.pull.updated", "mesh.mod.mesh-catalog.event.catching-up", "mesh.mod.mesh-catalog.event.upgraded", "mesh.seat.mesh-build-machine.event.built", "mesh.seat.mesh-controller.tool.>", "mesh.seat.node-build-agent.event.built", "mesh.seat.operator-channel.event.decided.mesh-controller"] }
allow_responses: { max: 1, ttl: "1m" }
} }
@@ -34,7 +34,7 @@ accounts {
} }
{ user: "node.one", password: "$2a$11$nnnnnnnnnnnnnnnnnnnnnn", permissions: {
publish: { allow: ["$JS.ACK.NODES.one.>", "$JS.API.CONSUMER.INFO.NODES.one", "$SRV.PING.node-tools.one", "mesh.control.one.>"] }
subscribe: { allow: ["_DELIVER.one", "_DELIVER.one.>", "_INBOX.node.one.>", "mesh.node.one.ask.hand-over", "mesh.node.one.ask.report", "mesh.node.one.ask.setuid-search", "mesh.node.one.declare"] }
subscribe: { allow: ["_DELIVER.one", "_DELIVER.one.>", "_INBOX.node.one.>", "mesh.node.one.ask.access-act", "mesh.node.one.ask.hand-over", "mesh.node.one.ask.judge-directories", "mesh.node.one.ask.report", "mesh.node.one.ask.setuid-search", "mesh.node.one.declare"] }
allow_responses: { max: 1, ttl: "1m" }
} }
{ user: "one.nats", password: "$2a$11$bbbbbbbbbbbbbbbbbbbbbb", permissions: {
+10
View File
@@ -96,6 +96,16 @@ var WritersTable = []WriterRow{
{State: "a fresh setuid search asked of a machine", Writer: "controller, at its terminal",
KeptIn: "the machine, which throws its last search away", Others: "the engine verifies the mesh's signature and starts it, or refuses",
Subjects: []string{"mesh.node.*.ask.setuid-search"}, Writes: isController},
// A read-only judgement of a machine's declared directories asked of its engine (novox/hq issue 358): signed as a
// hand-over is, under a signing context of its own. It changes nothing on the machine.
{State: "a judgement of a machine's directories asked of it", Writer: "controller",
KeptIn: "nowhere: the engine answers it and keeps nothing", Others: "the engine verifies the mesh's signature and answers, or refuses",
Subjects: []string{"mesh.node.*.ask.judge-directories"}, Writes: isController},
// The take of a shared folder asked of a machine's engine on the operator's warrant (novox/hq ADR 0325 §8):
// signed as a hand-over is, under a signing context of its own.
{State: "the take of a shared folder asked of a machine", Writer: "controller, on the operator's warrant",
KeptIn: "the machine, which records the act", Others: "the engine verifies the mesh's signature and takes it, or refuses",
Subjects: []string{"mesh.node.*.ask.access-act"}, Writes: isController},
{State: "a machine's applied state and its report", Writer: "the node-engine's apply queue",
KeptIn: "the machine; the report on the bus", Others: "the reconcile and a delivery enqueue, never apply",
// And its health statement between reports (novox/hq ADR 0240): the same writer stating the same
+20
View File
@@ -17,6 +17,8 @@ var designRows = []string{
"a machine's declaration",
"a hand-over asked of a machine",
"a fresh setuid search asked of a machine",
"a judgement of a machine's directories asked of it",
"the take of a shared folder asked of a machine",
"a machine's applied state and its report",
"the controller lease",
"plans and their tiers",
@@ -186,3 +188,21 @@ func TestASetuidSearchAskHasOnePublisher(t *testing.T) {
}
}
}
// **A judgement of a machine's directories asked of it has one publisher, the controller** (novox/hq issue 358), as a
// hand-over has.
func TestADirectoryJudgementAskHasOnePublisher(t *testing.T) {
for _, p := range []Principal{
{Kind: KindNode, Node: "laptop"},
{Kind: KindNodeTools, Node: "laptop", Module: RuntimeModule},
{Kind: KindModule, Node: "laptop", Module: "notes"},
} {
err := CheckWriters(p, []string{AskJudgeDirectoriesSubject("laptop")})
if err == nil || !strings.Contains(err.Error(), "a judgement of a machine's directories asked of it") {
t.Errorf("%s may ask a judgement of directories: %v", p.Username(), err)
}
}
if err := CheckWriters(Principal{Kind: KindController}, []string{AskJudgeDirectoriesSubject("laptop")}); err != nil {
t.Fatalf("the controller may not ask a judgement: %v", err)
}
}
+13 -2
View File
@@ -971,8 +971,19 @@ func one(ctx context.Context, run Runner, publish Publisher,
return catalogue.Built{Name: a.Name, Kind: a.Kind, Reference: reference}, nil
case catalogue.ArtifactArchive:
src.ownHas(cleanEntry(a.From) + "/**")
body, err := pack(filepath.Join(tree, a.From))
var body []byte
var err error
if a.URL != "" {
// An upstream file pinned by URL and sha256 (novox/hq ADR 0316): its build source is the pin, which
// is in module.json, always part of it.
body, err = upstreamArchive(ctx, workspace, a, say)
if err != nil {
return catalogue.Built{}, fmt.Errorf("%s: %s: %w", module, a.Name, err)
}
} else {
src.ownHas(cleanEntry(a.From) + "/**")
body, err = pack(filepath.Join(tree, a.From))
}
if err != nil {
return catalogue.Built{}, fmt.Errorf("%s: packing %s failed: %w", module, a.Name, err)
}
+81 -8
View File
@@ -49,8 +49,10 @@ import (
//
// One check:
//
// 1. clones the repository at the pull request's head, and beside it the repositories its check
// reads — the controller the mesh runs, the catalogue, the host, the lab — each at the ref asked;
// 1. clones the repository at the pull request's head and **makes the merge of that head into the trunk's
// head**, the check's trunk commit, in its own clone (novox/hq ADR 0323, issue 341; check_merge.go): what
// is judged is what merging makes, never pushed. Beside it the repositories its check reads — the
// controller the mesh runs, the catalogue, the host, the lab — each at the ref asked;
// 2. reads the facts snapshot the controller keeps, and with it **the versions the mesh runs**: the
// store a check's tests stand on is the store's own release, and the bus the bus's;
// 3. raises a throwaway PostgreSQL and a throwaway bus of those versions, labelled with the ask so a
@@ -79,7 +81,8 @@ type CheckSpec struct {
Modules []string
New []string
Manifests []string
// Base is the branch the pull request merges into: what the gate compares the change against.
// Base is the branch the pull request merges into: what the gate compares the change against, and the trunk
// whose head the change is merged into before it is judged (novox/hq ADR 0323). Empty judges the head alone.
Base string
// Judge is who judges the gate: "" the controller the mesh runs, "self" the change's own, "validator"
// the running one with the change's validator.
@@ -104,6 +107,9 @@ type GroupHead struct {
Repository string
Ref string
Paths []string
// Base is the branch it merges into: the head is merged into that trunk's head before it is composed
// (novox/hq ADR 0323 §6). Empty composes the head alone, as a controller from before it asks.
Base string
}
// Gated is whether the change touches the mesh's graph, and so whether the gate runs.
@@ -139,6 +145,16 @@ type CheckVerdict struct {
Took time.Duration
Gate *Layer
Repo *Layer
// Head and Trunk are what was judged (novox/hq ADR 0323 §3): the pull request's head merged into the trunk
// commit, each as twelve characters. Trunk is empty for a check asked with no trunk, which judged the head
// alone. Merge is the merge commit made for the check, whole: only checked, never pushed or registered.
Head, Trunk, Merge string
// Beside are the commits of the repositories the check read beside its own, by the directory each was
// cloned under, as twelve characters: named, not judged against (ADR 0323 §3, issue 568).
Beside map[string]string
// Members are a delivery group's other heads as the composed check judged them, each with the trunk
// commit it was merged into (ADR 0323 §6).
Members []JudgedHead
}
// Layer is one layer of a check, judged.
@@ -281,6 +297,34 @@ func Check(ctx context.Context, run Runner, spec CheckSpec, workspace, registry
return CheckVerdict{}, err
}
tree := filepath.Join(root, name)
// **What merging makes is judged, not the head** (novox/hq ADR 0323, issue 341): the head merged into the
// trunk's head as this clone found it, in this clone alone. A head that does not merge cleanly fails here.
judged := CheckVerdict{Head: twelve(spec.Ref)}
paths := spec.Paths
if spec.Base != "" {
m, err := mergeIntoTrunk(ctx, run, tree, spec.Base, spec.Ref)
if err != nil {
return CheckVerdict{}, err
}
judged.Head, judged.Trunk = twelve(m.Head), twelve(m.Trunk)
if len(m.Conflicts) > 0 {
said := conflictWords(m)
say("check", "%s", said)
return CheckVerdict{Verdict: "fail", Summary: said, Report: said, Took: time.Since(began),
Gate: &Layer{Verdict: "fail", Summary: said}, Repo: &Layer{Verdict: "fail", Summary: said},
Head: judged.Head, Trunk: judged.Trunk}, nil
}
judged.Merge = m.Merge
if changed := changedByMerge(ctx, run, tree, m); len(changed) > 0 {
paths = changed
}
say("check", "head %s merged into trunk %s (%s): the merge is judged, and never pushed", judged.Head,
judged.Trunk, spec.Base)
}
named := func(v CheckVerdict) CheckVerdict {
v.Head, v.Trunk, v.Merge, v.Beside, v.Members = judged.Head, judged.Trunk, judged.Merge, judged.Beside, judged.Members
return v
}
// **The base branch's check judges the change** (novox/hq issue 310), never the one the change brings.
judging, err := TheCheckThatJudges(ctx, tree, spec.Base)
if err != nil {
@@ -297,7 +341,7 @@ func Check(ctx context.Context, run Runner, spec CheckSpec, workspace, registry
v := CheckVerdict{Verdict: "pass", Summary: noModule, Took: time.Since(began),
Gate: &Layer{Verdict: "pass", Summary: noModule}, Repo: &Layer{Verdict: "warning", Summary: noScript}}
say("check", "%s; %s", noModule, noScript)
return v, nil
return named(v), nil
}
for dir, b := range spec.Beside {
if dir == name || !safeName.MatchString(dir) {
@@ -306,6 +350,13 @@ func Check(ctx context.Context, run Runner, spec CheckSpec, workspace, registry
if err := clone(b.Repository, b.Ref, dir); err != nil {
return CheckVerdict{}, fmt.Errorf("beside it, %w", err)
}
// The commit each one read is named in the verdict (ADR 0323 §3): its move is issue 568's question.
if at := headAt(ctx, run, filepath.Join(root, dir)); at != "" {
if judged.Beside == nil {
judged.Beside = map[string]string{}
}
judged.Beside[dir] = at
}
say("check", "beside it %s at %s", dir, short(b.Ref))
}
@@ -322,6 +373,26 @@ func Check(ctx context.Context, run Runner, spec CheckSpec, workspace, registry
return CheckVerdict{}, fmt.Errorf("a head of the group, %w", err)
}
say("check", "with it, of its group, %s/%s at %s", g.Owner, g.Repo, short(g.Ref))
// Each member is composed as its merge into its own trunk's head (ADR 0323 §6).
member := JudgedHead{Owner: g.Owner, Repo: g.Repo, Head: twelve(g.Ref)}
if g.Base != "" {
m, err := mergeIntoTrunk(ctx, run, filepath.Join(root, dir), g.Base, g.Ref)
if err != nil {
return CheckVerdict{}, fmt.Errorf("a head of the group, %s/%s: %w", g.Owner, g.Repo, err)
}
member.Head, member.Trunk = twelve(m.Head), twelve(m.Trunk)
if len(m.Conflicts) > 0 {
said := fmt.Sprintf("%s/%s: %s", g.Owner, g.Repo, conflictWords(m))
say("check", "%s", said)
judged.Members = append(judged.Members, member)
return named(CheckVerdict{Verdict: "fail", Summary: said, Report: said, Took: time.Since(began),
Gate: &Layer{Verdict: "fail", Summary: said}, Repo: &Layer{Verdict: "fail", Summary: said}}), nil
}
if changed := changedByMerge(ctx, run, filepath.Join(root, dir), m); len(changed) > 0 {
g.Paths = changed
}
}
judged.Members = append(judged.Members, member)
heads = append(heads, map[string]any{"repository": g.Owner + "/" + g.Repo,
"tree": filepath.Join(root, dir), "changed": g.Paths})
}
@@ -433,7 +504,7 @@ func Check(ctx context.Context, run Runner, spec CheckSpec, workspace, registry
verdictFile := filepath.Join(root, "verdict.json")
env := []string{EnvFacts + "=" + factsFile, EnvGate + "=" + gate, EnvGateStore + "=" + storeURL,
EnvTestStore + "=" + storeURL, EnvTestBus + "=nats://" + bus, EnvRepository + "=" + spec.Owner + "/" + spec.Repo,
EnvChanged + "=" + strings.Join(spec.Paths, ","), EnvBeside + "=" + root,
EnvChanged + "=" + strings.Join(paths, ","), EnvBeside + "=" + root,
EnvModules + "=" + strings.Join(append(append([]string{}, spec.Modules...), spec.New...), ","),
EnvGroup + "=" + groupFile,
"GOCACHE=" + filepath.Join(workspace, "go-cache"), "GOMODCACHE=" + filepath.Join(workspace, "go-modules")}
@@ -501,7 +572,7 @@ func Check(ctx context.Context, run Runner, spec CheckSpec, workspace, registry
v.Report, v.Took = redact.redact(withWhatFailed(out.String(), v.Repo)), time.Since(began)
say("check", "gate %s — %s; repository %s — %s (%s)", strings.ToUpper(v.Gate.Verdict), v.Gate.Summary,
strings.ToUpper(v.Repo.Verdict), v.Repo.Summary, v.Took.Round(time.Second))
return v, nil
return named(v), nil
}
// ownCheck runs the repository's own check, part by part, each in its toolchain's own container: the
@@ -868,14 +939,16 @@ func replaceGoFiles(from, into string) error {
}
// problemLines are the lines of a module check's output that name a problem: not a manifest found ok, not
// the count, not the closing note.
// the count, not the closing note, not a warning.
func problemLines(s string) []string {
var out []string
for _, line := range strings.Split(s, "\n") {
line = strings.TrimSpace(line)
switch {
case line == "", strings.Contains(line, ": ok"), strings.Contains(line, "problem(s) in"),
strings.Contains(line, "manifest(s) checked"):
strings.Contains(line, "manifest(s) checked"),
// A warning is said and fails nothing (novox/hq ADR 0325: what is refused once the catalogues moved).
strings.HasPrefix(line, "warning: "):
continue
}
out = append(out, line)
+115
View File
@@ -0,0 +1,115 @@
package builder
import (
"context"
"fmt"
"strings"
)
// **A merge check judges the merge, not the head** (novox/hq ADR 0323, issue 341).
//
// On 2026-10-09 a node-engine pull request was checked green at its head and broke the trunk when it merged:
// a merge an hour earlier had added a field under the same JSON key on other lines, git merged the two edits
// without a conflict, and the result did not compile. The check had judged the head alone. So a check now
// takes the trunk the pull request merges into, its head at the moment the check clones it as the check's
// **trunk commit**, and makes in its own clone the merge of the head into that commit, as the forge's
// default merge would. The gate and the repository's own check judge that merge.
//
// **The merge is only checked** (ADR 0323 §2, ADR 0238 §1): it is a commit off the trunk, made in the
// check's own clone, never pushed to the forge and never registered; the clone is removed when the check
// ends. Nothing here writes to a remote.
//
// The forge keeps no merge reference for a pull request, only its head, so the builder makes it itself.
// TrunkMerge is the merge a check made: the head and the trunk commit it was merged into, whole, and the merge
// commit; or the files in conflict when the head does not merge cleanly.
type TrunkMerge struct {
Head, Trunk, Merge string
Conflicts []string
}
// JudgedHead is one head a check judged, with the trunk commit it merged that head into, each as twelve
// characters (ADR 0323 §3, §6).
type JudgedHead struct {
Owner, Repo string
Head, Trunk string
}
// mergeIdentity is who a check's merge is made as: it is never pushed, so it names nobody's account. No hook of the
// clone's or the build machine's git configuration runs, and no setting there changes the merge.
var mergeIdentity = []string{"-c", "user.name=mesh merge check", "-c", "user.email=merge-check@mesh.invalid",
"-c", "commit.gpgsign=false", "-c", "core.hooksPath=/dev/null", "-c", "merge.verifySignatures=false"}
// mergeIntoTrunk makes, in the clone at tree, the merge of head into the head of origin/<base> as it stands in
// that fresh clone: the trunk commit. A head that does not merge cleanly answers the files in conflict and leaves
// the tree at the trunk commit; an error is that the merge could not be tried at all.
func mergeIntoTrunk(ctx context.Context, run Runner, tree, base, head string) (TrunkMerge, error) {
var m TrunkMerge
revParse := func(ref string) (string, error) {
out, err := run(ctx, tree, "git", "rev-parse", "--verify", "--quiet", ref+"^{commit}")
return strings.TrimSpace(out), err
}
var err error
if m.Head, err = revParse(head); err != nil || m.Head == "" {
return m, fmt.Errorf("the head %s cannot be read in the check's clone: %v", head, err)
}
if m.Trunk, err = revParse("refs/remotes/origin/" + base); err != nil || m.Trunk == "" {
return m, fmt.Errorf("the trunk %s cannot be read, and a check judges the head merged into it "+
"(novox/hq ADR 0323): %v", base, err)
}
if _, err := run(ctx, tree, "git", "checkout", "--quiet", "--detach", m.Trunk); err != nil {
return m, fmt.Errorf("the trunk commit %s cannot be checked out: %w", twelve(m.Trunk), err)
}
message := fmt.Sprintf("Check the merge of %s into %s at %s (made for a merge check, never pushed)",
twelve(m.Head), base, twelve(m.Trunk))
args := append(append([]string{}, mergeIdentity...), "merge", "--no-ff", "--no-edit", "--no-verify", "--quiet", "-m", message, m.Head)
if _, err := run(ctx, tree, "git", args...); err != nil {
out, _ := run(ctx, tree, "git", "diff", "--name-only", "--diff-filter=U")
m.Conflicts = strings.Fields(out)
_, _ = run(ctx, tree, "git", "merge", "--abort")
if len(m.Conflicts) == 0 {
return m, fmt.Errorf("the head %s could not be merged into trunk %s: %w", twelve(m.Head), twelve(m.Trunk), err)
}
return m, nil
}
if m.Merge, err = revParse("HEAD"); err != nil {
return m, fmt.Errorf("the merge made for the check cannot be read: %w", err)
}
return m, nil
}
// changedByMerge is every path the merge changes against the trunk commit: what the gate's delivery plan is
// computed from (ADR 0323 §1). Nil when it cannot be read; the caller keeps the pull request's own list.
func changedByMerge(ctx context.Context, run Runner, tree string, m TrunkMerge) []string {
if m.Merge == "" || m.Trunk == "" {
return nil
}
out, err := run(ctx, tree, "git", "diff", "--name-only", "--no-renames", m.Trunk, m.Merge)
if err != nil {
return nil // empty-on-error: the caller keeps the pull request's own list of changed files, the safe one
}
return strings.Fields(out)
}
// conflictWords is a head that does not merge cleanly, said as its check's failure.
func conflictWords(m TrunkMerge) string {
return fmt.Sprintf("head %s does not merge cleanly into trunk %s: in conflict %s", twelve(m.Head), twelve(m.Trunk),
strings.Join(m.Conflicts, ", "))
}
// headAt is the commit a clone is checked out at, as twelve characters; empty when it cannot be read.
func headAt(ctx context.Context, run Runner, tree string) string {
out, err := run(ctx, tree, "git", "rev-parse", "--verify", "--quiet", "HEAD^{commit}")
if err != nil {
return ""
}
return twelve(strings.TrimSpace(out))
}
// twelve is a commit as a verdict names it: its first twelve characters.
func twelve(commit string) string {
if len(commit) > 12 {
return commit[:12]
}
return commit
}
+212
View File
@@ -0,0 +1,212 @@
package builder
import (
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
)
// **A merge check judges the merge, not the head** (novox/hq ADR 0323, issue 341): the head merged into the trunk
// commit in the check's own clone, never pushed; a head that does not merge cleanly fails, naming the files; the
// verdict names the head, the trunk commit and the commits of the repositories read beside it.
// aTrunkAndABranch is a repository whose main moved after the branch was cut: main holds trunkFiles over the
// common files, the branch headFiles. It answers the repository, the branch's head and main's head.
func aTrunkAndABranch(t *testing.T, common, trunkFiles, headFiles map[string]string) (string, string, string) {
t.Helper()
dir := t.TempDir()
git := func(args ...string) string {
t.Helper()
cmd := exec.Command("git", args...)
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GIT_AUTHOR_NAME=t", "GIT_AUTHOR_EMAIL=t@example.org",
"GIT_COMMITTER_NAME=t", "GIT_COMMITTER_EMAIL=t@example.org")
out, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("git %v: %v\n%s", args, err, out)
}
return strings.TrimSpace(string(out))
}
write := func(files map[string]string) {
for name, body := range files {
if err := os.MkdirAll(filepath.Dir(filepath.Join(dir, name)), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, name), []byte(body), 0o755); err != nil {
t.Fatal(err)
}
}
git("add", "-A")
}
git("init", "--quiet", "-b", "main")
write(common)
git("commit", "--quiet", "-m", "common")
git("checkout", "--quiet", "-b", "change")
write(headFiles)
git("commit", "--quiet", "-m", "the change")
head := git("rev-parse", "HEAD")
git("checkout", "--quiet", "main")
write(trunkFiles)
git("commit", "--quiet", "-m", "the trunk moved")
return dir, head, git("rev-parse", "HEAD")
}
func TestACheckJudgesTheHeadMergedIntoTheTrunk(t *testing.T) {
// main's script passes only in a tree holding both the trunk's file and the head's: the merge.
repo, head, trunk := aTrunkAndABranch(t,
map[string]string{CheckScript: "test -f trunk-only && test -f head-only || { echo 'FAIL: not the merge'; exit 1; }\n"},
map[string]string{"trunk-only": "t"}, map[string]string{"head-only": "h"})
beside, besideHead := aCheckedRepository(t, map[string]string{"README": "the catalogue"})
registry := aFactsRegistry(t)
aRuntimeOnThisMachine(t)
v, err := Check(t.Context(), Command, CheckSpec{ID: "check-341", Repository: repo, Ref: head, Owner: "novox",
Repo: "mesh-lab", Number: 1, Base: "main", Toolchain: "golang",
Beside: map[string]Beside{"mesh-catalog": {Repository: beside, Ref: besideHead}}}, t.TempDir(), registry,
GitCredential{}, nil)
if err != nil {
t.Fatal(err)
}
if v.Repo == nil || v.Repo.Verdict != "pass" {
t.Fatalf("the check did not judge the head merged into the trunk: %+v\n%s", v.Repo, v.Report)
}
if v.Head != head[:12] || v.Trunk != trunk[:12] {
t.Fatalf("the verdict names head %q and trunk %q, not %s and %s", v.Head, v.Trunk, head[:12], trunk[:12])
}
if v.Beside["mesh-catalog"] != besideHead[:12] {
t.Fatalf("the verdict names the catalogue read beside it as %q, not %s: %v", v.Beside["mesh-catalog"],
besideHead[:12], v.Beside)
}
if v.Merge == "" || v.Merge == head || v.Merge == trunk {
t.Fatalf("no merge of the head into the trunk was made for the check: %q", v.Merge)
}
}
func TestAHeadThatDoesNotMergeCleanlyFailsNamingTheFiles(t *testing.T) {
repo, head, trunk := aTrunkAndABranch(t,
map[string]string{"store.go": "package store\n\nconst Declared = 1\n", "other.go": "package store\n"},
map[string]string{"store.go": "package store\n\nconst Declared = 2\n"},
map[string]string{"store.go": "package store\n\nconst Declared = 3\n"})
// No artifact store and no container runtime: a head that does not merge is judged before either is needed.
v, err := Check(t.Context(), Command, CheckSpec{ID: "check-341-conflict", Repository: repo, Ref: head,
Owner: "novox", Repo: "mesh-lab", Number: 2, Base: "main", Toolchain: "golang"}, t.TempDir(), "",
GitCredential{}, nil)
if err != nil {
t.Fatalf("a head that does not merge cleanly was not judged: %v", err)
}
want := "does not merge cleanly into trunk " + trunk[:12]
for name, l := range map[string]*Layer{"gate": v.Gate, "repository": v.Repo} {
if l == nil || l.Verdict != "fail" || !strings.Contains(l.Summary, want) || !strings.Contains(l.Summary, "store.go") {
t.Fatalf("its %s layer said %+v, not a failure saying %q and naming store.go", name, l, want)
}
}
if strings.Contains(v.Summary, "other.go") {
t.Fatalf("a file not in conflict is named: %s", v.Summary)
}
if v.Head != head[:12] || v.Trunk != trunk[:12] {
t.Fatalf("the failure names head %q and trunk %q", v.Head, v.Trunk)
}
}
// **The merge made for a check is only checked** (ADR 0323 §2): the forge never receives it.
func TestTheMergeMadeForACheckIsNeverPushed(t *testing.T) {
repo, head, trunk := aTrunkAndABranch(t, map[string]string{CheckScript: "true\n"},
map[string]string{"trunk-only": "t"}, map[string]string{"head-only": "h"})
registry := aFactsRegistry(t)
aRuntimeOnThisMachine(t)
refsBefore := gitOut(t, repo, "for-each-ref")
v, err := Check(t.Context(), Command, CheckSpec{ID: "check-341-push", Repository: repo, Ref: head, Owner: "novox",
Repo: "mesh-lab", Number: 3, Base: "main", Toolchain: "golang"}, t.TempDir(), registry, GitCredential{}, nil)
if err != nil {
t.Fatal(err)
}
if v.Merge == "" {
t.Fatal("the check made no merge of the head into the trunk")
}
if err := exec.Command("git", "-C", repo, "cat-file", "-e", v.Merge+"^{commit}").Run(); err == nil {
t.Fatalf("the merge %s made for the check reached the forge's repository", v.Merge)
}
if after := gitOut(t, repo, "for-each-ref"); after != refsBefore {
t.Fatalf("the check moved a ref of the forge's repository:\nbefore %s\nafter %s", refsBefore, after)
}
if got := gitOut(t, repo, "rev-parse", "main"); got != trunk {
t.Fatalf("the trunk moved to %s", got)
}
}
// A delivery group's composed check merges each member's head into its own trunk's head (ADR 0323 §6), and names
// both for each; a member that does not merge cleanly fails the composed check, naming it.
func TestAGroupsComposedCheckMergesEachMemberIntoItsTrunk(t *testing.T) {
repo, head, trunk := aTrunkAndABranch(t, map[string]string{CheckScript: "true\n"},
map[string]string{"trunk-only": "t"}, map[string]string{"head-only": "h"})
member, memberHead, memberTrunk := aTrunkAndABranch(t, map[string]string{"a": "1"},
map[string]string{"b": "trunk"}, map[string]string{"c": "head"})
registry := aFactsRegistry(t)
aRuntimeOnThisMachine(t)
v, err := Check(t.Context(), Command, CheckSpec{ID: "check-341-group", Repository: repo, Ref: head, Owner: "novox",
Repo: "mesh-controller", Number: 4, Base: "main", Toolchain: "golang", Group: []GroupHead{{Owner: "novox",
Repo: "mesh-host", Repository: member, Ref: memberHead, Base: "main"}}}, t.TempDir(), registry,
GitCredential{}, nil)
if err != nil {
t.Fatal(err)
}
if v.Head != head[:12] || v.Trunk != trunk[:12] {
t.Fatalf("the composed check names its first head %q into %q", v.Head, v.Trunk)
}
if len(v.Members) != 1 || v.Members[0].Head != memberHead[:12] || v.Members[0].Trunk != memberTrunk[:12] {
t.Fatalf("the composed check names its members %+v, not mesh-host %s into %s", v.Members, memberHead[:12],
memberTrunk[:12])
}
conflicting, conflictHead, _ := aTrunkAndABranch(t, map[string]string{"a": "1"}, map[string]string{"a": "2"},
map[string]string{"a": "3"})
v, err = Check(t.Context(), Command, CheckSpec{ID: "check-341-group-conflict", Repository: repo, Ref: head,
Owner: "novox", Repo: "mesh-controller", Number: 4, Base: "main", Toolchain: "golang", Group: []GroupHead{{
Owner: "novox", Repo: "mesh-host", Repository: conflicting, Ref: conflictHead, Base: "main"}}}, t.TempDir(),
registry, GitCredential{}, nil)
if err != nil {
t.Fatal(err)
}
if v.Verdict != "fail" || !strings.Contains(v.Summary, "novox/mesh-host") ||
!strings.Contains(v.Summary, "does not merge cleanly") {
t.Fatalf("a member that does not merge cleanly was answered %s: %s", v.Verdict, v.Summary)
}
}
func gitOut(t *testing.T, dir string, args ...string) string {
t.Helper()
out, err := exec.Command("git", append([]string{"-C", dir}, args...)...).CombinedOutput()
if err != nil {
t.Fatalf("git %v: %v\n%s", args, err, out)
}
return strings.TrimSpace(string(out))
}
// No hook runs when a check makes its merge: not one in the clone, whatever the build machine's git configuration says.
func TestACheckMergeRunsNoHook(t *testing.T) {
repo, head, _ := aTrunkAndABranch(t, map[string]string{"a": "1"}, map[string]string{"b": "trunk"},
map[string]string{"c": "head"})
clone := filepath.Join(t.TempDir(), "clone")
if out, err := exec.Command("git", "clone", "--quiet", repo, clone).CombinedOutput(); err != nil {
t.Fatalf("%v\n%s", err, out)
}
ran := filepath.Join(t.TempDir(), "ran")
for _, hook := range []string{"pre-merge-commit", "commit-msg", "prepare-commit-msg", "post-merge"} {
if err := os.WriteFile(filepath.Join(clone, ".git", "hooks", hook),
[]byte("#!/bin/sh\necho "+hook+" >> "+ran+"\n"), 0o755); err != nil {
t.Fatal(err)
}
}
m, err := mergeIntoTrunk(t.Context(), Command, clone, "main", head)
if err != nil || m.Merge == "" {
t.Fatalf("the merge was not made: %+v, %v", m, err)
}
if said, err := os.ReadFile(ran); err == nil {
t.Fatalf("a hook ran when the check made its merge: %s", said)
}
}
+139
View File
@@ -0,0 +1,139 @@
package builder
import (
"net"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
)
// TestReplay341 is the replay of novox/hq issue 341, written with only what the build agent had before its fix,
// so mesh-lab's prover can lay it over the commit before. On 2026-10-09 node-engine #64/#65 merged at 00:15 and
// added Applied.Declared, under the JSON key `declared`, to the store record. #66, checked at 00:49 at its head,
// had added its own Applied.Declared under the same key on other lines. Its check passed; git merged the two
// edits without a conflict, and the trunk no longer compiled: "Declared redeclared". Before the fix the check
// judged the head alone and passed. After it (novox/hq ADR 0323), the check judges the head merged into the
// trunk commit that held the other field, and fails saying the trunk does not compile.
//
// It runs the check whole, with a container runtime that runs each container's command on this machine
// (aRuntimeOnThisMachine): the repository's own check is `go build ./...`.
func TestReplay341(t *testing.T) {
origin, head := theNinthOfOctober(t)
registry := aFactsRegistry(t)
aRuntimeOnThisMachine(t)
v, err := Check(t.Context(), Command, CheckSpec{ID: "replay-341", Repository: origin, Ref: head, Owner: "novox",
Repo: "mesh-host", Number: 66, Base: "main", Toolchain: "golang"}, t.TempDir(), registry, GitCredential{}, nil)
if err != nil {
t.Fatalf("the check could not run: %v", err)
}
if v.Repo == nil || v.Repo.Verdict != "fail" {
t.Fatalf("node-engine #66 at its head was answered %+v: the merge it makes into the trunk that holds #65's "+
"field does not compile, and the check let it merge", v.Repo)
}
if said := v.Repo.Summary + "\n" + v.Repo.Failed + "\n" + v.Report; !strings.Contains(said, "Declared redeclared") {
t.Fatalf("the check failed without saying the trunk does not compile:\n%s", said)
}
}
// theNinthOfOctober is the node-engine's repository as it stood at 00:49 on 2026-10-09, in miniature: a trunk
// holding #65's Applied.Declared (a string, key `declared`), and #66's head, branched before it, holding its own
// Applied.Declared (a *FoundUnit, the same key) on other lines. Each compiles alone; their merge does not.
func theNinthOfOctober(t *testing.T) (string, string) {
t.Helper()
origin := t.TempDir()
git := func(args ...string) string {
t.Helper()
cmd := exec.Command("git", args...)
cmd.Dir = origin
cmd.Env = append(os.Environ(), "GIT_AUTHOR_NAME=t", "GIT_AUTHOR_EMAIL=t@example.org",
"GIT_COMMITTER_NAME=t", "GIT_COMMITTER_EMAIL=t@example.org")
out, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("git %v: %v\n%s", args, err, out)
}
return strings.TrimSpace(string(out))
}
write := func(name, body string) {
t.Helper()
if err := os.MkdirAll(filepath.Dir(filepath.Join(origin, name)), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(origin, name), []byte(body), 0o755); err != nil {
t.Fatal(err)
}
}
store := func(declared65, declared66 string) string {
return "package store\n\n// FoundUnit is a unit found on the machine.\ntype FoundUnit struct{ Name string }\n\n" +
"// Applied is what the node-engine applied.\ntype Applied struct {\n\tGeneration int `json:\"generation\"`\n" +
declared66 + "\tModules []string `json:\"modules\"`\n\tFiles []string `json:\"files\"`\n" +
"\tUnits []string `json:\"units\"`\n\tAt string `json:\"at\"`\n" + declared65 + "}\n"
}
git("init", "--quiet", "-b", "main")
write("go.mod", "module example.org/mesh-host\n\ngo 1.21\n")
write("internal/store/store.go", store("", ""))
write(CheckScript, "# mesh-check-toolchain: go\nset -e\nGOFLAGS= GOWORK=off GOTOOLCHAIN=local GOPROXY=off go build ./...\n")
git("add", "-A")
git("commit", "--quiet", "-m", "#63: the store record")
git("checkout", "--quiet", "-b", "adr-0269")
write("internal/store/store.go", store("", "\tDeclared *FoundUnit `json:\"declared\"`\n"))
git("commit", "--quiet", "-am", "#66: the unit a declaration names")
head := git("rev-parse", "HEAD")
git("checkout", "--quiet", "main")
write("internal/store/store.go", store("\tDeclared string `json:\"declared\"`\n", ""))
git("commit", "--quiet", "-am", "#65: the declaration applied")
return origin, head
}
// aRuntimeOnThisMachine puts a container runtime on PATH that runs a container's command on this machine, in the
// directory and with the environment the check gives it: a store and a bus that answer, a judge that is never
// built, and the repository's own check run as written. Only for a check whose code is the test's own.
func aRuntimeOnThisMachine(t *testing.T) {
t.Helper()
bus, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { bus.Close() })
go func() {
for {
c, err := bus.Accept()
if err != nil {
return
}
c.Close()
}
}()
dir := t.TempDir()
runtime := `#!/bin/sh
cmd="$1"; shift
case "$cmd" in
ps|rm|exec) exit 0 ;;
port) echo "` + bus.Addr().String() + `"; exit 0 ;;
run)
workdir=""; detached=""
while [ $# -gt 0 ]; do
case "$1" in
-d) detached=1; shift ;;
--rm) shift ;;
--workdir) workdir="$2"; shift 2 ;;
--env) export "$2"; shift 2 ;;
--network|--volume|--user|--label|--name|-p|-e) shift 2 ;;
*) break ;;
esac
done
if [ -n "$detached" ]; then echo replay-container; exit 0; fi
shift
if [ "$1" = "sh" ] && [ -n "$workdir" ]; then cd "$workdir" && exec "$@"; fi
echo "no such image on this machine" >&2
exit 1 ;;
esac
exit 1
`
if err := os.WriteFile(filepath.Join(dir, "docker"), []byte(runtime), 0o755); err != nil {
t.Fatal(err)
}
t.Setenv("PATH", dir+string(os.PathListSeparator)+os.Getenv("PATH"))
}
+326
View File
@@ -0,0 +1,326 @@
package builder
import (
"archive/tar"
"bufio"
"bytes"
"compress/bzip2"
"compress/gzip"
"context"
"crypto/sha256"
"encoding/hex"
"errors"
"fmt"
"io"
"net/http"
"os"
"os/exec"
"path"
"path/filepath"
"strings"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
)
// An archive whose source is somebody else's published file, pinned by URL and sha256 (novox/hq ADR 0316
// §2, §6, to-be 56 §7): a mail client's release, an add-on's xpi.
//
// **Fetched by the build seat, checked against the pin before anything is read from it, and packed as any
// archive is packed.** What a machine receives is the mesh's own archive from the artifact store, by digest,
// exactly as for a directory of the repository; no machine ever asks the vendor. A file whose bytes are not
// the pin's fails the build and names both digests, because the address serving something else is the one
// thing the pin exists to catch.
//
// **Kept in the build seat's workspace under its sha256**, so it is fetched once: a vendor that serves only
// its newest version at one address (Owl) would otherwise break every later build of the same pin.
// maxUpstream is the largest upstream file fetched, and the largest a release may unpack to: the node-engine
// unpacks no archive larger than this either (mesh-host internal/apply/archive.go, maxArchive).
const maxUpstream = 512 << 20
// maxEntries is the most entries an upstream archive may hold: millions of empty ones would fill the build
// seat's inodes and the node's. Betterbird's release holds 64.
const maxEntries = 100_000
// noDowngrade refuses a redirect off https: the digest still guards the bytes, and a fetch in the clear says
// to whoever is on the path what the build seat fetches.
func noDowngrade(req *http.Request, via []*http.Request) error {
if req.URL.Scheme != "https" {
return fmt.Errorf("%s redirected to %s, which is not https", via[0].URL, req.URL)
}
if len(via) >= 10 {
return errors.New("more than 10 redirects")
}
return nil
}
// upstreamClient fetches upstream files: a variable so a test serves its own.
var upstreamClient = &http.Client{Timeout: 15 * time.Minute}
// upstreamDir is where the build seat keeps the upstream files it fetched, by sha256.
func upstreamDir(workspace string) string { return filepath.Join(workspace, "upstream") }
// upstreamArchive is the packed archive of an artifact pinned by URL: the file fetched (or kept from an
// earlier build) and checked, then laid out as the artifact says and packed.
func upstreamArchive(ctx context.Context, workspace string, a catalogue.Artifact,
say func(step, format string, args ...any)) ([]byte, error) {
file, err := upstreamFile(ctx, workspace, a, say)
if err != nil {
return nil, err
}
laid := filepath.Join(workspace, "upstream-"+a.Name)
if err := os.RemoveAll(laid); err != nil {
return nil, err
}
defer os.RemoveAll(laid)
if err := os.MkdirAll(laid, 0o755); err != nil {
return nil, err
}
if a.File != "" {
body, err := os.ReadFile(file)
if err != nil {
return nil, err
}
if err := os.WriteFile(filepath.Join(laid, a.File), body, 0o644); err != nil {
return nil, err
}
} else if err := unpackUpstream(ctx, file, laid, a.Within); err != nil {
return nil, fmt.Errorf("%s (%s): %w", a.URL, a.Version, err)
}
return pack(laid)
}
// upstreamFile is the path of the pinned file in the workspace: kept from an earlier build when its bytes
// are still the pin's, else fetched now and kept only when they are.
func upstreamFile(ctx context.Context, workspace string, a catalogue.Artifact,
say func(step, format string, args ...any)) (string, error) {
want := strings.ToLower(a.SHA256)
dir := upstreamDir(workspace)
kept := filepath.Join(dir, want)
if got, err := sumOf(kept); err == nil {
if got == want {
say("upstream", "%s %s: kept from an earlier build (sha256:%s)", a.Name, a.Version, short(want))
return kept, nil
}
// Changed on disk since it was kept: fetched again rather than trusted.
os.Remove(kept)
}
if err := os.MkdirAll(dir, 0o755); err != nil {
return "", err
}
say("upstream", "fetching %s (%s)", a.URL, a.Version)
request, err := http.NewRequestWithContext(ctx, http.MethodGet, a.URL, nil)
if err != nil {
return "", err
}
client := *upstreamClient
client.CheckRedirect = noDowngrade
response, err := client.Do(request)
if err != nil {
return "", fmt.Errorf("cannot fetch %s: %w", a.URL, err)
}
defer response.Body.Close()
if response.StatusCode != http.StatusOK {
return "", fmt.Errorf("%s answered %s", a.URL, response.Status)
}
part, err := os.CreateTemp(dir, want+".*.part")
if err != nil {
return "", err
}
defer os.Remove(part.Name())
hash := sha256.New()
n, err := io.Copy(io.MultiWriter(part, hash), io.LimitReader(response.Body, maxUpstream+1))
if cerr := part.Close(); err == nil {
err = cerr
}
if err != nil {
return "", fmt.Errorf("fetching %s: %w", a.URL, err)
}
if n > maxUpstream {
return "", fmt.Errorf("%s is larger than %d bytes, more than a machine unpacks; nothing was kept", a.URL, maxUpstream)
}
got := hex.EncodeToString(hash.Sum(nil))
if got != want {
return "", fmt.Errorf("%s was pinned as sha256:%s and what it served is sha256:%s (%d bytes); nothing "+
"was kept. The address serves another file than the one reviewed: compare it, then move the pin "+
"in a reviewed change (novox/hq ADR 0316)", a.URL, want, got, n)
}
if err := os.Rename(part.Name(), kept); err != nil {
return "", err
}
say("upstream", "%s %s: %d bytes, sha256:%s as pinned", a.Name, a.Version, n, short(want))
return kept, nil
}
// sumOf is the sha256 of a file, in hexadecimal.
func sumOf(file string) (string, error) {
f, err := os.Open(file)
if err != nil {
return "", err
}
defer f.Close()
hash := sha256.New()
if _, err := io.Copy(hash, f); err != nil {
return "", err
}
return hex.EncodeToString(hash.Sum(nil)), nil
}
// unpackUpstream unpacks a tar archive, compressed with gzip, bzip2 or xz (told by its first bytes), into a
// directory: of its entries only those within `within` (all, when empty), with that prefix taken off.
//
// **Only files and directories, and nothing outside.** A link, a device or an entry naming a path outside
// the archive, or outside `within`, refuses the build by name: the node-engine unpacks only files and
// directories, and an archive quietly unpacked without what it held would be a release that is not the one
// pinned.
func unpackUpstream(ctx context.Context, file, into, within string) error {
f, err := os.Open(file)
if err != nil {
return err
}
defer f.Close()
buffered := bufio.NewReader(f)
magic, _ := buffered.Peek(6)
var stream io.Reader
var wait func() error
switch {
case bytes.HasPrefix(magic, []byte{0x1f, 0x8b}):
zipped, err := gzip.NewReader(buffered)
if err != nil {
return err
}
defer zipped.Close()
stream = zipped
case bytes.HasPrefix(magic, []byte("BZh")):
stream = bzip2.NewReader(buffered)
case bytes.HasPrefix(magic, []byte{0xfd, '7', 'z', 'X', 'Z', 0x00}):
// Go's standard library reads no xz; the build seat's image carries xz (cmd/mesh-builder/Dockerfile).
if _, err := exec.LookPath("xz"); err != nil {
return errors.New("it is compressed with xz, and this build seat has no xz to read it")
}
cmd := exec.CommandContext(ctx, "xz", "-dc")
cmd.Stdin = buffered
var stderr bytes.Buffer
cmd.Stderr = &stderr
out, err := cmd.StdoutPipe()
if err != nil {
return err
}
if err := cmd.Start(); err != nil {
return err
}
stream = out
wait = func() error {
if err := cmd.Wait(); err != nil {
return fmt.Errorf("xz could not read it: %v %s", err, strings.TrimSpace(stderr.String()))
}
return nil
}
defer func() {
if wait != nil {
_ = cmd.Process.Kill()
_ = cmd.Wait()
}
}()
default:
return errors.New("it is not a tar archive compressed with gzip, bzip2 or xz; a single file is kept " +
"whole by naming it in `file`")
}
if err := untar(stream, into, within); err != nil {
return err
}
if wait != nil {
// Whatever follows the tar's end is read before xz is waited for: xz blocks on a full pipe otherwise.
if _, err := io.Copy(io.Discard, stream); err != nil {
return err
}
w := wait
wait = nil
return w()
}
return nil
}
// untar writes a tar stream's files and directories within `within` into a directory.
func untar(stream io.Reader, into, within string) error {
reader := tar.NewReader(stream)
prefix := ""
if within != "" {
prefix = path.Clean(within) + "/"
}
var total int64
files, entries := 0, 0
for {
header, err := reader.Next()
if err == io.EOF {
break
}
if err != nil {
return err
}
if entries++; entries > maxEntries {
return fmt.Errorf("it holds more than %d entries", maxEntries)
}
if header.Typeflag == tar.TypeXGlobalHeader {
// A pax global header (git archive writes one) names no file.
continue
}
name := path.Clean(strings.TrimPrefix(header.Name, "./"))
if path.IsAbs(name) || name == ".." || strings.HasPrefix(name, "../") {
return fmt.Errorf("%s names a path outside the archive", header.Name)
}
if name == "." || name+"/" == prefix ||
(header.Typeflag == tar.TypeDir && strings.HasPrefix(prefix, name+"/")) {
// The archive's root, the directory taken, or one of its parents.
continue
}
if prefix != "" {
if !strings.HasPrefix(name, prefix) {
return fmt.Errorf("%s is outside %s, the directory the artifact is taken from", header.Name, within)
}
name = strings.TrimPrefix(name, prefix)
}
target := filepath.Join(into, filepath.FromSlash(name))
switch header.Typeflag {
case tar.TypeDir:
if err := os.MkdirAll(target, 0o755); err != nil {
return err
}
case tar.TypeReg:
total += header.Size
if total > maxUpstream {
return fmt.Errorf("it unpacks to more than %d bytes, more than a machine unpacks", maxUpstream)
}
if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil {
return err
}
mode := os.FileMode(0o644)
if header.Mode&0o111 != 0 {
mode = 0o755
}
out, err := os.OpenFile(target, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, mode)
if err != nil {
return err
}
if _, err := io.Copy(out, io.LimitReader(reader, header.Size)); err != nil {
out.Close()
return err
}
if err := out.Close(); err != nil {
return err
}
files++
default:
return fmt.Errorf("%s is a %c, and an archive carries only files and directories", header.Name,
header.Typeflag)
}
}
if files == 0 {
if within != "" {
return fmt.Errorf("it holds no file within %s", within)
}
return errors.New("it holds no file")
}
return nil
}
+385
View File
@@ -0,0 +1,385 @@
package builder
import (
"archive/tar"
"bytes"
"compress/gzip"
"context"
"crypto/sha256"
"encoding/hex"
"fmt"
"io"
"net"
"net/http"
"net/http/httptest"
"os"
"os/exec"
"path/filepath"
"sort"
"strings"
"testing"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
)
// An archive pinned by URL and sha256 (novox/hq ADR 0316 §2, §6; How it is checked: "an upstream artifact
// whose download does not match its sha256 fails the build and names both digests").
type tarEntry struct {
name, body string
mode int64
kind byte
}
func tarOf(t *testing.T, entries []tarEntry) []byte {
t.Helper()
var raw bytes.Buffer
w := tar.NewWriter(&raw)
for _, e := range entries {
kind := e.kind
if kind == 0 {
kind = tar.TypeReg
}
h := &tar.Header{Name: e.name, Mode: e.mode, Typeflag: kind, Size: int64(len(e.body))}
if kind == tar.TypeSymlink {
h.Linkname, h.Size = "/usr/share/hunspell", 0
}
if kind == tar.TypeDir {
h.Size = 0
}
if err := w.WriteHeader(h); err != nil {
t.Fatal(err)
}
if kind == tar.TypeReg {
if _, err := w.Write([]byte(e.body)); err != nil {
t.Fatal(err)
}
}
}
if err := w.Close(); err != nil {
t.Fatal(err)
}
return raw.Bytes()
}
func gzipped(t *testing.T, body []byte) []byte {
t.Helper()
var out bytes.Buffer
z := gzip.NewWriter(&out)
if _, err := z.Write(body); err != nil {
t.Fatal(err)
}
if err := z.Close(); err != nil {
t.Fatal(err)
}
return out.Bytes()
}
func sumHex(b []byte) string {
s := sha256.Sum256(b)
return hex.EncodeToString(s[:])
}
// vendor serves files by path over https and counts what it was asked.
type vendor struct {
files map[string][]byte
asked int
}
func serveVendor(t *testing.T, v *vendor) string {
t.Helper()
server := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
v.asked++
body, ok := v.files[r.URL.Path]
if !ok {
http.NotFound(w, r)
return
}
w.Write(body)
}))
t.Cleanup(server.Close)
before := upstreamClient
// Under a public name, as a manifest must say it: example.com, which the test server's certificate
// names, dialled at the server.
transport := server.Client().Transport.(*http.Transport).Clone()
addr := server.Listener.Addr().String()
transport.DialContext = func(ctx context.Context, network, _ string) (net.Conn, error) {
return (&net.Dialer{}).DialContext(ctx, network, addr)
}
upstreamClient = &http.Client{Transport: transport}
t.Cleanup(func() { upstreamClient = before })
return "https://example.com"
}
// entriesOf lists a packed archive: each file with its mode and content.
func entriesOf(t *testing.T, packed []byte) map[string]string {
t.Helper()
z, err := gzip.NewReader(bytes.NewReader(packed))
if err != nil {
t.Fatal(err)
}
r := tar.NewReader(z)
out := map[string]string{}
for {
h, err := r.Next()
if err == io.EOF {
return out
}
if err != nil {
t.Fatal(err)
}
body, _ := io.ReadAll(r)
out[h.Name] = string(body) + " " + os.FileMode(h.Mode).String()
}
}
func release(t *testing.T) []byte {
return gzipped(t, tarOf(t, []tarEntry{
{name: "betterbird/", kind: tar.TypeDir, mode: 0o755},
{name: "betterbird/application.ini", body: "[App]\nVersion=153.4.0\n", mode: 0o644},
{name: "betterbird/betterbird", body: "#!/bin/sh\n", mode: 0o755},
{name: "betterbird/defaults/pref/channel-prefs.js", body: "pref(\"app.update.channel\", \"default\");\n", mode: 0o644},
}))
}
func pinnedManifest(base, sum string) string {
return `{"module":"betterbird","version":"1",
"build":{"artifacts":[
{"name":"archive","kind":"archive","url":"` + base + `/release.tar.gz","sha256":"` + sum + `","version":"153.4.0esr-bb10","within":"betterbird"}]},
"resources":[{"id":"installation","type":"archive","path":"/opt/betterbird","artifact":"archive"}]}`
}
func TestAnArchivePinnedByURLIsFetchedCheckedAndPackedWithoutItsOwnDirectory(t *testing.T) {
body := release(t)
v := &vendor{files: map[string][]byte{"/release.tar.gz": body}}
base := serveVendor(t, v)
r, workspace := aRepository(t, pinnedManifest(base, sumHex(body)), nil)
got, err := Build(context.Background(), r.run, r, "https://forge.invalid/catalogue.git", "", "", workspace, nil,
Npmrc{}, GitCredential{}, nil)
if err != nil {
t.Fatal(err)
}
digest, _ := got.Manifest.Resources[0]["digest"].(string)
if !strings.HasPrefix(digest, "sha256:") || r.archives["betterbird/archive"] != digest {
t.Fatalf("the release was not published and pinned as the module's archive: %v, published %v",
got.Manifest.Resources[0], r.archives)
}
packed, err := upstreamArchive(context.Background(), workspace, catalogue.Artifact{Name: "archive", Kind: "archive",
URL: base + "/release.tar.gz", SHA256: sumHex(body), Version: "153.4.0esr-bb10", Within: "betterbird"},
func(string, string, ...any) {})
if err != nil {
t.Fatal(err)
}
if "sha256:"+sumHex(packed) != digest {
t.Fatalf("a second packing of one pin gave another digest: %s and %s", digest, sumHex(packed))
}
entries := entriesOf(t, packed)
want := map[string]string{
"application.ini": "[App]\nVersion=153.4.0\n -rw-r--r--",
"betterbird": "#!/bin/sh\n -rwxr-xr-x",
"defaults/pref/channel-prefs.js": "pref(\"app.update.channel\", \"default\");\n -rw-r--r--",
}
if len(entries) != len(want) {
t.Fatalf("the archive holds %v, want %v", entries, want)
}
for name, w := range want {
if entries[name] != w {
t.Errorf("%s is %q in the archive, want %q", name, entries[name], w)
}
}
}
func TestAnUpstreamFileIsFetchedOnceAndKeptByItsDigest(t *testing.T) {
// The vendor of an add-on serves only its newest version at one address: a pin that was fetched once
// must still build after the address moved on.
body := release(t)
v := &vendor{files: map[string][]byte{"/release.tar.gz": body}}
base := serveVendor(t, v)
r, workspace := aRepository(t, pinnedManifest(base, sumHex(body)), nil)
first, err := Build(context.Background(), r.run, r, "https://forge.invalid/catalogue.git", "", "", workspace, nil,
Npmrc{}, GitCredential{}, nil)
if err != nil {
t.Fatal(err)
}
v.files["/release.tar.gz"] = []byte("the vendor's next release")
second, err := Build(context.Background(), r.run, r, "https://forge.invalid/catalogue.git", "", "", workspace, nil,
Npmrc{}, GitCredential{}, nil)
if err != nil {
t.Fatalf("a pin fetched before failed to build once the address served something else: %v", err)
}
if v.asked != 1 {
t.Fatalf("the vendor was asked %d times for one pin, want once", v.asked)
}
if first.Manifest.Resources[0]["digest"] != second.Manifest.Resources[0]["digest"] {
t.Fatal("two builds of one pin gave two digests")
}
}
func TestAnUpstreamFileWhoseDownloadDoesNotMatchItsPinFailsAndNamesBothDigests(t *testing.T) {
pinned := release(t)
served := gzipped(t, tarOf(t, []tarEntry{{name: "betterbird/betterbird", body: "not what was reviewed", mode: 0o755}}))
v := &vendor{files: map[string][]byte{"/release.tar.gz": served}}
base := serveVendor(t, v)
r, workspace := aRepository(t, pinnedManifest(base, sumHex(pinned)), nil)
_, err := Build(context.Background(), r.run, r, "https://forge.invalid/catalogue.git", "", "", workspace, nil,
Npmrc{}, GitCredential{}, nil)
if err == nil {
t.Fatal("a download that is not the pinned file was built")
}
for _, digest := range []string{sumHex(pinned), sumHex(served)} {
if !strings.Contains(err.Error(), "sha256:"+digest) {
t.Errorf("the refusal does not name sha256:%s: %v", digest, err)
}
}
if len(r.archives) != 0 {
t.Fatalf("something was published: %v", r.archives)
}
kept, _ := filepath.Glob(filepath.Join(upstreamDir(workspace), "*"))
if len(kept) != 0 {
t.Fatalf("the refused download was kept: %v", kept)
}
}
func TestAnUpstreamFileThatIsNotAnArchiveIsKeptWholeUnderItsName(t *testing.T) {
xpi := []byte("PK\x03\x04 an add-on")
v := &vendor{files: map[string][]byte{"/owl.xpi": xpi}}
base := serveVendor(t, v)
packed, err := upstreamArchive(context.Background(), t.TempDir(), catalogue.Artifact{Name: "owl", Kind: "archive",
URL: base + "/owl.xpi", SHA256: sumHex(xpi), Version: "1.5.6.1", File: "owl.xpi"}, func(string, string, ...any) {})
if err != nil {
t.Fatal(err)
}
entries := entriesOf(t, packed)
if len(entries) != 1 || entries["owl.xpi"] != string(xpi)+" -rw-r--r--" {
t.Fatalf("the add-on was not kept whole as owl.xpi: %v", entries)
}
}
func TestAnUpstreamArchiveHoldingALinkOrAnEntryOutsideItsDirectoryIsRefused(t *testing.T) {
cases := map[string][]tarEntry{
// The community package links the dictionaries; a release carrying a link is not one a machine unpacks.
"is a 2": {{name: "betterbird/betterbird", body: "x", mode: 0o755},
{name: "betterbird/dictionaries", kind: tar.TypeSymlink}},
"outside betterbird": {{name: "betterbird/betterbird", body: "x", mode: 0o755},
{name: "README", body: "x", mode: 0o644}},
"outside the archive": {{name: "../etc/passwd", body: "x", mode: 0o644}},
"no file within": {{name: "betterbird/", kind: tar.TypeDir, mode: 0o755}},
}
for want, entries := range cases {
body := gzipped(t, tarOf(t, entries))
v := &vendor{files: map[string][]byte{"/r.tar.gz": body}}
base := serveVendor(t, v)
_, err := upstreamArchive(context.Background(), t.TempDir(), catalogue.Artifact{Name: "archive", Kind: "archive",
URL: base + "/r.tar.gz", SHA256: sumHex(body), Version: "1", Within: "betterbird"}, func(string, string, ...any) {})
if err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("an archive whose entry is %q was not refused for it: %v", want, err)
}
}
}
func TestAnUpstreamArchiveCompressedWithXzIsRead(t *testing.T) {
// Betterbird publishes its Linux release as .tar.xz; the build seat's image carries xz for it.
if _, err := exec.LookPath("xz"); err != nil {
t.Skip("NOT TESTED here: no xz on this machine (the build seat's image carries it)")
}
plain := tarOf(t, []tarEntry{{name: "betterbird/betterbird", body: "#!/bin/sh\n", mode: 0o755}})
cmd := exec.Command("xz", "-zc")
cmd.Stdin = bytes.NewReader(plain)
compressed, err := cmd.Output()
if err != nil {
t.Fatal(err)
}
v := &vendor{files: map[string][]byte{"/r.tar.xz": compressed}}
base := serveVendor(t, v)
packed, err := upstreamArchive(context.Background(), t.TempDir(), catalogue.Artifact{Name: "archive", Kind: "archive",
URL: base + "/r.tar.xz", SHA256: sumHex(compressed), Version: "1", Within: "betterbird"}, func(string, string, ...any) {})
if err != nil {
t.Fatal(err)
}
var names []string
for n := range entriesOf(t, packed) {
names = append(names, n)
}
sort.Strings(names)
if strings.Join(names, ",") != "betterbird" {
t.Fatalf("the xz release unpacked to %v", names)
}
}
func TestAnUpstreamTarWithAGlobalHeaderNestedWithinAndTrailingDataIsRead(t *testing.T) {
// git archive writes a pax global header; a release may unpack under a/b with a/ listed first; xz may be
// followed by bytes after the tar's end, which must be read before xz is waited for.
var raw bytes.Buffer
w := tar.NewWriter(&raw)
w.WriteHeader(&tar.Header{Typeflag: tar.TypeXGlobalHeader, Name: "pax_global_header", PAXRecords: map[string]string{"comment": "abc"}})
w.WriteHeader(&tar.Header{Name: "a/", Typeflag: tar.TypeDir, Mode: 0o755})
w.WriteHeader(&tar.Header{Name: "a/b/", Typeflag: tar.TypeDir, Mode: 0o755})
w.WriteHeader(&tar.Header{Name: "a/b/f", Typeflag: tar.TypeReg, Mode: 0o644, Size: 1})
w.Write([]byte("x"))
w.Close()
body := gzipped(t, raw.Bytes())
v := &vendor{files: map[string][]byte{"/r.tar.gz": body}}
base := serveVendor(t, v)
packed, err := upstreamArchive(context.Background(), t.TempDir(), catalogue.Artifact{Name: "a", Kind: "archive",
URL: base + "/r.tar.gz", SHA256: sumHex(body), Version: "1", Within: "a/b"}, func(string, string, ...any) {})
if err != nil {
t.Fatal(err)
}
if e := entriesOf(t, packed); len(e) != 1 || e["f"] == "" {
t.Fatalf("%v", e)
}
if _, err := exec.LookPath("xz"); err != nil {
t.Skip("NOT TESTED here: no xz")
}
// A megabyte after the tar's end: more than a pipe holds.
trailing := append(append([]byte{}, raw.Bytes()...), bytes.Repeat([]byte{0}, 1<<20)...)
cmd := exec.Command("xz", "-zc")
cmd.Stdin = bytes.NewReader(trailing)
compressed, err := cmd.Output()
if err != nil {
t.Fatal(err)
}
v.files["/r.tar.xz"] = compressed
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if _, err := upstreamArchive(ctx, t.TempDir(), catalogue.Artifact{Name: "a", Kind: "archive", URL: base + "/r.tar.xz",
SHA256: sumHex(compressed), Version: "1", Within: "a/b"}, func(string, string, ...any) {}); err != nil {
t.Fatal(err)
}
}
func TestAnUpstreamArchiveWithTooManyEntriesIsRefused(t *testing.T) {
var raw bytes.Buffer
w := tar.NewWriter(&raw)
for i := 0; i <= maxEntries; i++ {
w.WriteHeader(&tar.Header{Name: fmt.Sprintf("d%d/", i), Typeflag: tar.TypeDir, Mode: 0o755})
}
w.Close()
body := gzipped(t, raw.Bytes())
v := &vendor{files: map[string][]byte{"/r.tar.gz": body}}
base := serveVendor(t, v)
_, err := upstreamArchive(context.Background(), t.TempDir(), catalogue.Artifact{Name: "a", Kind: "archive",
URL: base + "/r.tar.gz", SHA256: sumHex(body), Version: "1"}, func(string, string, ...any) {})
if err == nil || !strings.Contains(err.Error(), "entries") {
t.Fatalf("%v", err)
}
}
func TestARedirectOffHttpsIsNotFollowed(t *testing.T) {
plain := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { w.Write([]byte("x")) }))
t.Cleanup(plain.Close)
secure := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, plain.URL+"/r", http.StatusFound)
}))
t.Cleanup(secure.Close)
before := upstreamClient
upstreamClient = secure.Client()
t.Cleanup(func() { upstreamClient = before })
_, err := upstreamFile(context.Background(), t.TempDir(), catalogue.Artifact{Name: "a", URL: secure.URL + "/r",
SHA256: sumHex([]byte("x")), Version: "1", File: "a"}, func(string, string, ...any) {})
if err == nil || !strings.Contains(err.Error(), "not https") {
t.Fatalf("%v", err)
}
}
+83 -1
View File
@@ -2,7 +2,10 @@ package catalogue
import (
"fmt"
"net"
"net/url"
"path"
"regexp"
"sort"
"strings"
)
@@ -312,7 +315,7 @@ func (b *Build) problems(module string) []string {
module, a.Name, a.System, spokenSystems()))
}
} else {
if a.From == "" {
if a.From == "" && a.URL == "" {
problems = append(problems, fmt.Sprintf(
"%s: %q says nothing about what it is built from", module, a.Name))
}
@@ -350,6 +353,85 @@ func (b *Build) problems(module string) []string {
problems = append(problems, fmt.Sprintf(
"%s: %q mirrors %q, which names no tag or digest", module, a.Name, a.From))
}
problems = append(problems, upstreamFileProblems(module, a)...)
}
return problems
}
// reservedSuffixes are names no public vendor publishes under: an installation's own and documentation's.
var reservedSuffixes = []string{".internal", ".local", ".lan", ".home", ".corp", ".localhost", ".home.arpa",
".test", ".invalid", ".example", ".intranet", ".private"}
// notAPublicHost says why a host is not a public vendor's name, or "".
func notAPublicHost(host string) string {
h := strings.ToLower(strings.TrimSuffix(host, "."))
if net.ParseIP(h) != nil {
return "an address rather than a name"
}
if !strings.Contains(h, ".") {
return "a name with no domain"
}
for _, s := range reservedSuffixes {
if strings.HasSuffix(h, s) {
return "a name under " + s + ", which is nobody's public domain"
}
}
return ""
}
// sha256Hex is a pinned sha256 as a manifest writes it: 64 lower-case hexadecimal characters.
var sha256Hex = regexp.MustCompile(`^[0-9a-f]{64}$`)
// upstreamFileProblems is what is wrong with an archive pinned by URL (novox/hq ADR 0316 §2, §6): it is an
// archive, fetched over https, pinned by a sha256 and a version, and it names no directory of the repository
// besides; a file name or a directory within is a plain path that stays where it is put.
func upstreamFileProblems(module string, a Artifact) []string {
pinned := a.URL != "" || a.SHA256 != "" || a.Version != "" || a.File != "" || a.Within != ""
if !pinned {
return nil
}
var problems []string
say := func(format string, args ...any) {
problems = append(problems, fmt.Sprintf("%s: %q ", module, a.Name)+fmt.Sprintf(format, args...))
}
if a.Kind != ArtifactArchive {
say("is a %q and names an upstream file (url, sha256, version, file, within): only an archive is "+
"mirrored from one (novox/hq ADR 0316)", a.Kind)
return problems
}
if a.URL == "" {
say("names a sha256, a version, a file or a directory within, and no url to fetch it from")
return problems
}
if a.From != "" {
say("is built from %q and fetched from %s: an archive is one or the other", a.From, a.URL)
}
if u, err := url.Parse(a.URL); err != nil || u.Scheme != "https" || u.Host == "" {
say("is fetched from %q, and an upstream file is fetched over https from a named host", a.URL)
} else if why := notAPublicHost(u.Hostname()); why != "" {
say("is fetched from %s, %s: an upstream file is somebody else's release, published under a public name", a.URL, why)
}
if !sha256Hex.MatchString(a.SHA256) {
say("is fetched from %s without a pinned sha256 (64 lower-case hexadecimal characters): what that "+
"address serves changes under the mesh, and only the digest says it is what was reviewed", a.URL)
}
if strings.TrimSpace(a.Version) == "" {
say("pins %s and says no version, so a person reading the pin cannot tell which release it is", a.URL)
}
if a.File != "" && a.Within != "" {
say("names a file (%q) and a directory within (%q): a file is kept whole, a directory is taken out "+
"of an archive", a.File, a.Within)
}
for field, p := range map[string]string{"file": a.File, "within": a.Within} {
if p == "" {
continue
}
clean := path.Clean(p)
if clean != p || strings.HasPrefix(p, "/") || clean == "." || clean == ".." || strings.HasPrefix(clean, "../") ||
(field == "file" && strings.Contains(p, "/")) {
say("names %s %q, which is not a plain relative %s", field, p, map[string]string{
"file": "file name", "within": "directory"}[field])
}
}
return problems
}
+62 -6
View File
@@ -246,6 +246,20 @@ type Rendering struct {
// statement's contract is link.RootContract or later). To an older, strict engine the field is not
// sent, and the account it names is not judged — which the self-check says, as not judged.
JudgesRoot bool
// KeepsFolders says this machine's node-engine gives modules their identities and keeps shared folders
// (novox/hq ADR 0325: its statement's contract is link.IdentityContract or later). To an older, strict engine none of
// it is sent — no `identities`, no container `identity`, no access `users` or `taken`, no owner derived from an
// identity — and what it runs is composed as before.
KeepsFolders bool
// Identities is each module's allocated identity, by module (ADR 0325 §3): the same on every machine. A module
// that declares `identity` and has none here is composed without one, as to an older engine.
Identities map[string]ModuleIdentity
// KeptFolders is every folder the operator let the mesh keep on this machine (ADR 0325 §8), by its clean path.
KeptFolders map[string]bool
// ReadyFolders is every folder the engine states it keeps on this machine (its access healthy), by clean path: a
// folder it found empty and kept, or took itself. With KeptFolders, what lets a container switch to its identity.
ReadyFolders map[string]bool
}
// RootField is a user resource's field saying the account must never become root without a person
@@ -320,6 +334,9 @@ type Composed struct {
// naming its module and why (novox/hq ADR 0255): that piece is left out and the machine is told
// everything else. Push and plan say it; the merge gate refuses a change that adds one.
Unplaced []string
// Identities is the declaration's `identities` (novox/hq ADR 0325 §3): [{module, name, uid, gid}] for each
// module composed with an identity. Empty to an engine that does not keep folders.
Identities []map[string]any
}
// ForeseenSealed is what stands for an own secret a send will make, in a composition asked ahead of
@@ -358,7 +375,9 @@ func (r Resolution) LeftOutBecause(settings SettingsBy, adopted bool) map[string
out[m.Module] = &UnreadManifestError{Module: m.Module, said: why}
continue
}
if err := JudgeSettings(m, settings[m.Module], adopted); err != nil {
// A placement stored ahead of the build that declares its access is passed by, not a reason to leave the
// module out (novox/hq issue 575): it was judged against that build where it was stored.
if err := ComposesWith(m, settings[m.Module], adopted); err != nil {
out[m.Module] = err
}
}
@@ -380,7 +399,8 @@ func (r Resolution) Compose(with Rendering) (Composed, error) {
received := map[string]map[string][]Contribution{}
leftOut := map[string]string{}
var foreseen, unplaced []string
resources, err := r.compose(with, owner, received, leftOut, &foreseen, &unplaced)
var identities []ModuleIdentity
resources, err := r.compose(with, owner, received, leftOut, &foreseen, &unplaced, &identities)
if err != nil {
return Composed{}, err
}
@@ -394,8 +414,12 @@ func (r Resolution) Compose(with Rendering) (Composed, error) {
})
}
sort.Strings(unplaced)
return Composed{Resources: resources, Owner: owner, Received: received, LeftOut: leftOut,
Foreseen: foreseen, Unplaced: unplaced}, nil
composed := Composed{Resources: resources, Owner: owner, Received: received, LeftOut: leftOut,
Foreseen: foreseen, Unplaced: unplaced}
if len(identities) > 0 {
composed.Identities = identitiesOf(identities)
}
return composed, nil
}
// BusMembershipID names the resource carrying a machine's membership for the new bus, and
@@ -406,7 +430,7 @@ const BusMembershipPath = "/var/lib/mesh/membership-next.json"
func (r Resolution) compose(with Rendering, owner map[string]string,
received map[string]map[string][]Contribution, leftOut map[string]string,
foreseen, unplaced *[]string) ([]map[string]any, error) {
foreseen, unplaced *[]string, identities *[]ModuleIdentity) ([]map[string]any, error) {
// **A setting is judged where it is stored, and an impossible one costs a module, not a
// machine** (novox/hq ADR 0163, rule 6). A definition that moved under a stored setting makes
// this module uncomposable; it is left out of the declaration — its held things kept, its
@@ -553,6 +577,19 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
}
resources := m.Resources
// The identity it runs as here (novox/hq ADR 0325 §3, §11): none to an older engine, nor without an allocation.
ident, hasIdent, err := identityFor(m, with)
if err != nil {
return nil, err
}
if hasIdent && identities != nil {
*identities = append(*identities, ident)
}
mountedByIdentity := dirsMountedByIdentity(m)
// Its containers run as it only once every read-write folder they mount is taken (the operator's question on
// ADR 0325): until then the identity is allocated, sent and named in the take's users, and not switched to.
runsAsIdent := hasIdent && !identityHeld(m, with)
// What the mesh computes for this module goes FIRST, before the module's own resources.
//
// **Order is stated, not derived — the host does not sort** (novox/hq ADR 0005), so
@@ -645,6 +682,11 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
sealed = ForeseenSealed
*foreseen = append(*foreseen, m.Module+"/"+name)
}
if sealed == "" && m.OwnSecrets[name].IssuedBy == IssuedOutside {
// **Not given, and the mesh never makes it** (novox/hq issue 401, ADR 0228): no file, as for a
// family's member (ADR 0283), so the module reads its absence as not given.
continue
}
if sealed == "" {
// Declared and not made. Refused rather than skipped: a module whose own
// credential is silently absent starts, fails to authenticate, and the reason is
@@ -998,11 +1040,16 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
return nil, err
}
ownerInto(copied, placed)
// And its identity (novox/hq ADR 0325 §2, §3): a container's resolved, or taken out where none is composed;
// the owner of a directory its identity's containers mount. After the placement's owner: the identity wins.
if err := identityInto(copied, m, ident, runsAsIdent, mountedByIdentity); err != nil {
return nil, err
}
// **After settings, and that is the whole reason it is here.** A module's file
// content is where a setting lands, so a placeholder may only exist once the setting
// has been put in — filling secrets first would look at content that is not yet what
// the machine receives.
if err := intoFile(copied, sealed, m.Module); err != nil {
if err := intoFile(copied, sealed, outsideNotGiven(m, with.Needed[m.Module]), m.Module); err != nil {
return nil, err
}
// **Substituted here, not on the machine.** A bound value is not secret — the mesh
@@ -1115,6 +1162,12 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
out = append(out, copied)
}
// **Who uses each of its shared folders** (novox/hq ADR 0325 §4, §5, §8): the access's `users` and `taken`, to
// an engine that keeps folders.
if with.KeepsFolders {
usersInto(out, owner, m, ident, hasIdent, r.toolsAccount(m, with), with)
}
// **What only the mesh knows, where this module asked for it.** The graph is the control
// plane's; making a name resolve is the module's software. Emitted as ordinary files under
// this module's name, so they are applied, reported and removed exactly as anything else
@@ -2431,6 +2484,9 @@ func secretsReadBy(resource map[string]any, m Manifest, given map[string]string)
}
paths := map[string]string{}
for name, own := range m.OwnSecrets.Plain() {
if own.IssuedBy == IssuedOutside && given[name] == "" {
continue // not given, so not placed: nothing to restart on (novox/hq issue 401)
}
paths[name] = own.Path
}
for name, sealed := range given {
+7 -1
View File
@@ -71,6 +71,9 @@ var hostname = regexp.MustCompile(
`(?:be|nl|de|fr|uk|eu|com|net|org|io|dev|app|cloud|site|online|me|co|ch|at|lu|` +
`internal|example|tld|test|invalid)`)
// upstreamVersion is where an artifact pinned by URL says which release it is (Artifact.Version).
var upstreamVersion = regexp.MustCompile(`^build\.artifacts\[[0-9]+\]\.version$`)
// address is a dotted quad.
var address = regexp.MustCompile(`(?:[0-9]{1,3}\.){3}[0-9]{1,3}`)
@@ -120,7 +123,10 @@ func InstallationProblems(m Manifest) []string {
problems = append(problems, fmt.Sprintf("%s names %s at %s", m.Module, name, at))
}
for _, ip := range addressesIn(value) {
if meant[ip] {
// An upstream pin's version may have four dotted numbers (Owl 1.5.6.1), which is no address; a name
// in it is still judged above. Its URL is judged whole: the vendor's host goes through the artifact's
// names-on-purpose like any name a definition means (novox/hq ADR 0316).
if meant[ip] || upstreamVersion.MatchString(at) {
continue
}
problems = append(problems, fmt.Sprintf("%s names the public address %s at %s", m.Module, ip, at))
+33
View File
@@ -976,6 +976,35 @@ type Artifact struct {
// success, and was invisible to the launcher (novox/hq 04-ISSUES/142).
Binary string `json:"binary,omitempty"`
// URL is where an archive's upstream file is fetched from, for an archive that is a release
// somebody else published rather than a directory of this repository (novox/hq ADR 0316 §2, §6).
// With it come SHA256 and Version, and From stays empty.
//
// **Pinned by the digest of what the vendor published, mirrored by the build seat.** A release of
// about 80 MB copied into a repository is kept for ever in its history at every upgrade, and an
// add-on whose licence forbids copying it to third parties may not be in a repository at all. So the
// manifest names the file, the build seat fetches it, refuses it when its sha256 is not the pin's and
// names both digests, and packs it as any archive is packed: what reaches a machine is the mesh's own
// archive in the artifact store, fetched by digest like every other, and a machine never asks the
// vendor. Upgrading is changing the pin, reviewed like any other change.
URL string `json:"url,omitempty"`
// SHA256 is the sha256 of the file at URL, as 64 lower-case hexadecimal characters.
SHA256 string `json:"sha256,omitempty"`
// Version is what the vendor calls the release pinned, said where a person reads the pin; the build
// does not read it.
Version string `json:"version,omitempty"`
// File is the name an upstream file that is not an archive is kept under: the archive built from it
// holds that one file (an add-on's xpi). Empty means the file is a tar archive (gzip, bzip2 or xz)
// whose files are the artifact.
File string `json:"file,omitempty"`
// Within is the directory inside an upstream tar archive whose content is the artifact, for a
// release that unpacks into a directory of its own name (`betterbird/`). An entry outside it refuses
// the build. Empty means the archive's root.
Within string `json:"within,omitempty"`
// NamesOnPurpose are the names this artifact means to name, each with why: the vendor's host an upstream
// file is fetched from, which the installation check judges like any name (InstallationProblems).
NamesOnPurpose map[string]string `json:"names-on-purpose,omitempty"`
// Entrypoints are the compiled files a tool host should load from this module, relative to the
// bundle's root.
//
@@ -1689,6 +1718,7 @@ func ParseManifest(raw []byte) (Manifest, error) {
// facts about the catalogue and are checked at registration (CatalogueProblems).
problems = append(problems, declaredSeatProblems(m)...)
problems = append(problems, RunsAsProblems(m)...)
problems = append(problems, ReadsHolderProblems(m)...)
if m.Computed != "" && len(m.Resources) > 0 {
// One or the other. A module that both ships files and has them computed would leave
// nobody able to say where a given file came from.
@@ -1829,6 +1859,9 @@ func ParseManifest(raw []byte) (Manifest, error) {
problems = append(problems, invokeProblems(m)...)
problems = append(problems, replacesProblems(m)...)
problems = append(problems, UnitSettingProblems(m)...)
problems = append(problems, OwnerSettingProblems(m)...)
// How each container's image is told who to run as, and what a module that says so is held to (novox/hq ADR 0325).
problems = append(problems, identityProblems(m)...)
problems = append(problems, endpointNameProblems(m)...)
problems = append(problems, RouteProblems(m)...)
for _, port := range m.Guards {
+575
View File
@@ -0,0 +1,575 @@
package catalogue
import (
"fmt"
"path/filepath"
"regexp"
"sort"
"strings"
)
// Every module is given an identity of its own, and a container that declares `identity` runs as it (novox/hq ADR
// 0325 §2, §3, §11).
//
// **The manifest says how its image is told who to run as, and nothing else.** A container resource may declare
// `identity`: `{"user": "env:PUID", "group": "env:PGID"}` for an image that reads its user and group from its
// environment, or `{"user": "run"}` for the container runtime's own user option. It is the image's interface, as its
// ports are. The mesh fills it in: the controller allocates the module one user and group number, mesh-wide, once,
// from a range of its own (IdentityLow–IdentityHigh), and the node-engine makes the account `mesh-<module>` on each
// machine the module is assigned to and tells the image those numbers.
//
// **The range is 50000–59999, and a module's user and group numbers are one number.** Chosen above the numbers
// distributions give people (1000 up) and below the 60000s Debian keeps for its own daemons, and far from root's 0
// and every root group a distribution ships (0, adm's 4, wheel's 10): no number in it is 0 or a root group's, so §11
// holds by the range alone, and the store refuses a row outside it. Both numbers come from one sequence, so a
// module's uid and gid are equal; a number once given is never given again, even after the module is forgotten
// (the store keeps the row, inventory.AllocateIdentity).
//
// **What the controller sends, and only to an engine that reads it** (Rendering.KeepsFolders: its statement's
// contract is link.IdentityContract or later):
//
// - the declaration's `identities`: [{module, name, uid, gid}] for each module of the machine that declares
// `identity` on a container;
// - a container's `identity` resolved: {"mode": "env", "user_env", "group_env", "uid", "gid"} or {"mode": "run",
// "uid", "gid"};
// - the owner of each of the module's directories a container declaring `identity` mounts: the identity's
// uid:gid (§3). A directory only a process or the tools write keeps the owner it had: what writes it is not the
// identity, and owning it by the identity would lock its writer out;
// - an access's `users`: the module's identity where a container of it that declares `identity` mounts the
// access, and the tool runner's account where the module's tools run there as named, its number the engine reads by name,
// each at the access's use — the node-engine keeps one entry per user on a folder at the widest use any access
// to it gives (§4, §5), so the runner's entry is the widest of the modules whose tools use it; and `taken`,
// whether the operator let the mesh keep that folder on that machine (§8).
//
// To an older engine none of it is sent, and a container's `identity` is taken out: it runs as its image says, as
// before. A module that declares no `identity` is composed exactly as before, on any engine.
// The range module identities are allocated from (novox/hq ADR 0325 §3, §11). The store holds the same bounds.
const (
IdentityLow = 50000
IdentityHigh = 59999
)
// AllocatableID says whether n is a number the mesh may give a module's identity: inside the range, so never 0
// nor a root group's.
func AllocatableID(n int) bool { return n >= IdentityLow && n <= IdentityHigh }
// IdentityField is a container resource's field saying how its image is told who to run as.
const IdentityField = "identity"
// The two ways an image is told who to run as.
const (
IdentityEnv = "env"
IdentityRun = "run"
)
// IdentitySpec is a container's `identity` as its manifest declares it.
type IdentitySpec struct {
// Mode is IdentityEnv or IdentityRun.
Mode string
// UserEnv and GroupEnv are the environment variables an env image reads, such as PUID and PGID.
UserEnv, GroupEnv string
}
// ModuleIdentity is one module's allocated identity: its numbers, the same on every machine.
type ModuleIdentity struct {
Module string
UID int
GID int
}
// IdentityName is the account a module's identity is on every machine it is assigned to.
func IdentityName(module string) string { return "mesh-" + module }
// Where an access's user comes from (novox/hq ADR 0325 §4, §5).
const (
FromContainer = "container"
FromToolRunner = "tool-runner"
)
// AccessUser is one identity that uses a shared folder, as an access's `users` names it (mesh-host internal/declaration
// AccessUser): the account, its user number, the use, and where it runs. The tool runner's account is named with no
// number: it is the machine's, and the node-engine resolves it by name, refusing one missing or root's.
type AccessUser struct {
Identity string `json:"identity"`
UID int `json:"uid,omitempty"`
Use string `json:"use"`
From string `json:"from"`
}
var envName = regexp.MustCompile(`^[A-Z_][A-Z0-9_]*$`)
// ParseIdentity reads a container's `identity`, or refuses it in words.
func ParseIdentity(v any) (IdentitySpec, error) {
fields, ok := v.(map[string]any)
if !ok {
return IdentitySpec{}, fmt.Errorf("identity is {\"user\": \"env:<NAME>\", \"group\": \"env:<NAME>\"} or " +
"{\"user\": \"run\"}")
}
for k := range fields {
if k != "user" && k != "group" {
return IdentitySpec{}, fmt.Errorf("identity says %q, and it says only how the image is told its user "+
"and group: \"user\" and \"group\", never an owner or a number (%s §1, §2)", k, ADR0325)
}
}
user, _ := fields["user"].(string)
group, hasGroup := fields["group"]
switch {
case user == IdentityRun:
if hasGroup {
return IdentitySpec{}, fmt.Errorf("identity {\"user\": \"run\"} takes no group: the runtime's user " +
"option carries both numbers")
}
return IdentitySpec{Mode: IdentityRun}, nil
case strings.HasPrefix(user, "env:"):
g, _ := group.(string)
if !strings.HasPrefix(g, "env:") {
return IdentitySpec{}, fmt.Errorf("identity names its user as %q and its group as %v; an image told "+
"its user by its environment is told its group the same way, \"env:<NAME>\"", user, group)
}
u, gr := strings.TrimPrefix(user, "env:"), strings.TrimPrefix(g, "env:")
if !envName.MatchString(u) || !envName.MatchString(gr) || u == gr {
return IdentitySpec{}, fmt.Errorf("identity names the variables %q and %q; each is an environment "+
"variable's name, upper-case letters, digits and underscores, and the two differ", u, gr)
}
return IdentitySpec{Mode: IdentityEnv, UserEnv: u, GroupEnv: gr}, nil
}
return IdentitySpec{}, fmt.Errorf("identity says its user is %q; it is \"env:<NAME>\" or \"run\"", user)
}
// DeclaresIdentity says a module has a container that declares `identity`: it is given one (§3).
func DeclaresIdentity(m Manifest) bool {
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) == "container" {
if _, has := r[IdentityField]; has {
return true
}
}
}
return false
}
// containerMounts is what a container of the manifest mounts, read from its volumes as written: the module's own
// directories by id, and its accesses by id — or, for a literal source, as "path:<path>".
func containerMounts(r map[string]any) (dirs, accesses []string) {
volumes, _ := r["volumes"].([]any)
for _, v := range volumes {
s, _ := v.(string)
source := volumeSource(s)
if strings.HasPrefix(source, "/") {
accesses = append(accesses, "path:"+filepath.Clean(source))
continue
}
for _, m := range dirRef.FindAllStringSubmatch(source, -1) {
dirs = append(dirs, m[1])
}
for _, m := range accessRef.FindAllStringSubmatch(source, -1) {
accesses = append(accesses, m[1])
}
}
return dirs, accesses
}
// volumeSource is a volume's source as written: up to its first colon outside a ${…} placeholder, so
// "${dir:config}:/config" is "${dir:config}".
func volumeSource(v string) string {
depth := 0
for i, r := range v {
switch {
case r == '{' && i > 0 && v[i-1] == '$':
depth++
case r == '}' && depth > 0:
depth--
case r == ':' && depth == 0:
return v[:i]
}
}
return v
}
// AccessUsersShape is who among a module's containers uses which access as the module's identity, in a form that
// compares: each container declaring identity with how it is told, and the accesses it mounts, sorted. Two manifests
// with the same shape give their accesses the same container users.
func AccessUsersShape(m Manifest) string {
var out []string
for _, r := range m.Resources {
v, declares := r[IdentityField]
if fmt.Sprint(r["type"]) != "container" || !declares {
continue
}
_, accesses := containerMounts(r)
sort.Strings(accesses)
out = append(out, fmt.Sprint(r["id"], v, accesses))
}
sort.Strings(out)
return strings.Join(out, "\x01")
}
// accessMode is the mode an access a container mounts was declared at, by id or by "path:<path>"; "" when the
// manifest declares no such access.
func accessMode(m Manifest, ref string) string {
for _, a := range m.Accesses {
if (a.ID != "" && a.ID == ref) || (a.Path != "" && ref == "path:"+filepath.Clean(a.Path)) {
return a.At()
}
}
return ""
}
// ADR0325 is how the module check names the decision its identity rules come from.
const ADR0325 = "novox/hq ADR 0325"
// identityProblems is what the module check refuses of a manifest's identities (novox/hq ADR 0325 §1, §2, §3):
//
// - an `identity` on anything but a container, or one that is neither env:<NAME> for both nor run;
// - **in a module that declares `identity` on a container** — it says it runs as its own identity, so it is held
// to the decision whole: a directory of it naming an owner (§1, §3), and a container of it that declares none
// and mounts one of the module's directories or a read-write access (§2).
//
// A module that declares no `identity` is warned of the last two (IdentityWarnings), never refused for them yet:
// every registered container module mounts its own directories without one, and refusing them would stop the
// catalogues composing before they can migrate. **This is the narrowest rule that breaks nothing registered**: no
// registered manifest declares `identity`, so no registered module is refused anything new.
func identityProblems(m Manifest) []string {
var problems []string
for _, r := range m.Resources {
v, has := r[IdentityField]
if !has {
continue
}
if fmt.Sprint(r["type"]) != "container" {
problems = append(problems, fmt.Sprintf("%s: %v is a %v and says identity; only a container says how its "+
"image is told who to run as (%s §2)", m.Module, r["id"], r["type"], ADR0325))
continue
}
if _, err := ParseIdentity(v); err != nil {
problems = append(problems, fmt.Sprintf("%s: the container %v: %v (%s §2)", m.Module, r["id"], err, ADR0325))
}
}
if !DeclaresIdentity(m) {
return problems
}
return append(problems, identityRules(m)...)
}
// IdentityWarnings is what the module check warns of a module that declares no `identity` (novox/hq ADR 0325): each
// thing the decision refuses once the module migrates, said now so the catalogues can. Refusing comes when no
// registered module still does any of it. Empty for a module that declares `identity`: it is refused these instead.
func IdentityWarnings(m Manifest) []string {
if DeclaresIdentity(m) {
return nil
}
var out []string
for _, p := range identityRules(m) {
out = append(out, p+"; accepted while registered modules still do it, refused once they have moved")
}
return out
}
// identityRules is every place a manifest breaks what ADR 0325 §1–§3 refuse: a directory naming an owner, and a
// container declaring no identity that mounts the module's own directory or a read-write access.
func identityRules(m Manifest) []string {
var out []string
for _, r := range m.Resources {
switch fmt.Sprint(r["type"]) {
case "directory":
if owner, said := r["owner"]; said {
out = append(out, fmt.Sprintf("%s: the directory %v names its owner %v; a directory a module owns is "+
"owned by the module's identity, and no manifest names an owner (%s §1, §3)", m.Module, r["id"],
owner, ADR0325))
}
case "container":
if _, has := r[IdentityField]; has {
continue
}
dirs, accesses := containerMounts(r)
seen := map[string]bool{}
for _, d := range dirs {
if !seen[d] {
seen[d] = true
out = append(out, fmt.Sprintf("%s: the container %v declares no identity and mounts the module's "+
"own directory %s; only a container that declares identity mounts its module's directories, "+
"so what writes them is the module's identity (%s §2)", m.Module, r["id"], d, ADR0325))
}
}
for _, a := range accesses {
if accessMode(m, a) == AccessReadWrite && !seen["access:"+a] {
seen["access:"+a] = true
out = append(out, fmt.Sprintf("%s: the container %v declares no identity and mounts the read-write "+
"access %s; a container that runs as its image says is given no entry on a shared folder, so "+
"it is refused write (%s §2)", m.Module, r["id"], strings.TrimPrefix(a, "path:"), ADR0325))
}
}
}
}
return out
}
// PlacesOwnerProblem refuses an owner a `places` setting names for a module that declares `identity` (novox/hq ADR
// 0325 §1, §3): its directories are owned by its identity. Nil for a module that declares none, whose placed owner
// is still applied, and warned of by PlacesOwnerWarnings.
func PlacesOwnerProblem(m Manifest, layers []Layer) error {
if !DeclaresIdentity(m) {
return nil
}
if owned := placedOwners(m, layers); len(owned) > 0 {
return fmt.Errorf("%s places %s with an owner, and declares identity: its directories are owned by its "+
"identity, and no setting names an owner (%s §1, §3)", m.Module, strings.Join(owned, ", "), ADR0325)
}
return nil
}
// PlacesOwnerWarnings is a warning for each directory a `places` setting gives an owner, for a module that declares
// no `identity` (novox/hq ADR 0325 §1): accepted while registered modules still do it, refused once they have moved.
func PlacesOwnerWarnings(m Manifest, layers []Layer) []string {
if DeclaresIdentity(m) {
return nil
}
var out []string
for _, id := range placedOwners(m, layers) {
out = append(out, fmt.Sprintf("warning: %s's setting places.%s.owner names an owner; a directory a module owns "+
"is owned by the module's identity, and no setting names an owner (%s §1, §3). Accepted while "+
"registered modules still do it, refused once they have moved", m.Module, id, ADR0325))
}
return out
}
func placedOwners(m Manifest, layers []Layer) []string {
placed, err := Places(m, layers)
if err != nil {
// empty-on-error: a placement Places refuses is refused by name where it is judged and composed; it names no owner here
return nil
}
var out []string
for id, p := range placed {
if p.Owner != "" {
out = append(out, id)
}
}
sort.Strings(out)
return out
}
// resolvedIdentity is a container's `identity` as the node-engine is told it: the mode, the variables an env image
// reads, and the module's numbers.
func resolvedIdentity(spec IdentitySpec, id ModuleIdentity) map[string]any {
out := map[string]any{"mode": spec.Mode, "uid": id.UID, "gid": id.GID}
if spec.Mode == IdentityEnv {
out["user_env"], out["group_env"] = spec.UserEnv, spec.GroupEnv
}
return out
}
// identityFor is the identity module m runs as here, and whether one is composed: a module that declares `identity`,
// to an engine that keeps folders, with an allocation the composition was given. A number outside the range is
// refused (§11), never sent.
func identityFor(m Manifest, with Rendering) (ModuleIdentity, bool, error) {
if !with.KeepsFolders || !DeclaresIdentity(m) {
return ModuleIdentity{}, false, nil
}
id, allocated := with.Identities[m.Module]
if !allocated {
return ModuleIdentity{}, false, nil
}
if !AllocatableID(id.UID) || !AllocatableID(id.GID) {
return ModuleIdentity{}, false, fmt.Errorf("%s's identity is %d:%d, outside %d–%d: no module identity is root "+
"or a root group's, and none is sent (%s §11)", m.Module, id.UID, id.GID, IdentityLow, IdentityHigh, ADR0325)
}
id.Module = m.Module
return id, true, nil
}
// identityInto composes one resource of module m for its identity: a container's `identity` resolved, or taken out
// where none is composed; and the owner of a directory one of its identity's containers mounts.
func identityInto(resource map[string]any, m Manifest, id ModuleIdentity, has bool, mountedByIdentity map[string]bool) error {
switch fmt.Sprint(resource["type"]) {
case "container":
v, declared := resource[IdentityField]
if !declared {
return nil
}
if !has {
delete(resource, IdentityField)
return nil
}
spec, err := ParseIdentity(v)
if err != nil {
return fmt.Errorf("%s: the container %v: %w", m.Module, resource["id"], err)
}
resource[IdentityField] = resolvedIdentity(spec, id)
case "directory":
if has && mountedByIdentity[fmt.Sprint(resource["id"])] {
resource["owner"] = fmt.Sprintf("%d:%d", id.UID, id.GID)
}
}
return nil
}
// identityHeld says module m's containers are not switched to its identity yet (the operator's question on ADR 0325):
// a container declaring `identity` mounts a read-write access whose folder is neither taken (KeptFolders) nor kept by
// the engine (ReadyFolders). Switched before the take adds the identity's entry, the app would run as the new uid and
// lose write on the operator's existing files; held, it runs as it did (its environment's PUID and PGID), the take's
// users already name the identity, and the switch comes in the apply after every such folder is taken.
func identityHeld(m Manifest, with Rendering) bool {
_, byID, err := accessesFor(m, with.Settings[m.Module])
if err != nil {
return true // refused where the module itself is composed; never switched on a guess
}
for _, r := range m.Resources {
if _, declares := r[IdentityField]; !declares || fmt.Sprint(r["type"]) != "container" {
continue
}
_, refs := containerMounts(r)
for _, ref := range refs {
if accessMode(m, ref) != AccessReadWrite {
continue
}
path := byID[ref]
if strings.HasPrefix(ref, "path:") {
path = strings.TrimPrefix(ref, "path:")
}
path = filepath.Clean(path)
if path != "." && !with.KeptFolders[path] && !with.ReadyFolders[path] {
return true
}
}
}
return false
}
// identityContainers is the composed ids of module m's containers that declare `identity`, whether or not it is sent.
func identityContainers(m Manifest) map[string]bool {
out := map[string]bool{}
for _, r := range m.Resources {
if _, declares := r[IdentityField]; declares && fmt.Sprint(r["type"]) == "container" {
out[m.Module+"."+fmt.Sprint(r["id"])] = true
}
}
return out
}
// dirsMountedByIdentity is the ids of module m's directories one of its containers declaring `identity` mounts.
func dirsMountedByIdentity(m Manifest) map[string]bool {
out := map[string]bool{}
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) != "container" {
continue
}
if _, has := r[IdentityField]; !has {
continue
}
dirs, _ := containerMounts(r)
for _, d := range dirs {
out[d] = true
}
}
return out
}
// identitiesOf is the declaration's `identities`, by module.
func identitiesOf(ids []ModuleIdentity) []map[string]any {
sort.Slice(ids, func(i, j int) bool { return ids[i].Module < ids[j].Module })
out := make([]map[string]any, 0, len(ids))
for _, id := range ids {
out = append(out, map[string]any{"module": id.Module, "name": IdentityName(id.Module), "uid": id.UID,
"gid": id.GID})
}
return out
}
// toolsAccount is the account module m's tools run as on this machine, or empty where they run nowhere as one: no
// runtime here, no bundle it loads, or a module left out of an adopted machine's declaration.
func (r Resolution) toolsAccount(m Manifest, with Rendering) string {
if !r.runtimeHere() || (with.Adopted && m.Filtering != nil) {
return ""
}
loads := false
for _, b := range m.Bundles {
if len(b.Loads) > 0 {
loads = true
}
}
if !loads {
return ""
}
if m.RunsAs != "" {
return m.RunsAs
}
return r.Account
}
// usersInto writes `users` and `taken` on every access of module m among the resources composed so far: its identity
// where a container of it that declares `identity` mounts the access's path, and the tool runner's account where the
// module's tools run as it, each at the access's use (§4: the engine keeps the widest).
func usersInto(out []map[string]any, owner map[string]string, m Manifest, id ModuleIdentity, has bool, account string,
with Rendering) {
// The containers that declare identity, whether or not it is sent yet: the take adds the entries of the identity
// to come, so they exist before the switch.
declaring := identityContainers(m)
var containers []map[string]any
for _, res := range out {
if res["type"] == "container" && owner[fmt.Sprint(res["id"])] == m.Module && declaring[fmt.Sprint(res["id"])] && has {
containers = append(containers, res)
}
}
for _, res := range out {
if res["type"] != "access" || owner[fmt.Sprint(res["id"])] != m.Module {
continue
}
path, _ := res["path"].(string)
mode, _ := res["mode"].(string)
if mode == "" {
mode = AccessRead
}
users := []AccessUser{}
for _, c := range containers {
if mounts(c, path) {
users = append(users, AccessUser{Identity: IdentityName(m.Module), UID: id.UID, Use: mode, From: FromContainer})
break // one identity per module, whichever of its containers mounts it
}
}
if account != "" {
users = append(users, AccessUser{Identity: account, Use: mode, From: FromToolRunner})
}
if len(users) > 0 {
res["users"] = users
}
res["taken"] = with.KeptFolders[filepath.Clean(path)]
}
}
// mounts says whether a composed container mounts path, or something below it, from the machine. A path that merely
// begins with the same letters — /storage/media2 beside /storage/media — is another path.
func mounts(container map[string]any, path string) bool {
volumes, _ := container["volumes"].([]any)
path = filepath.Clean(path)
for _, v := range volumes {
s, ok := v.(string)
if !ok {
continue
}
source, _, _ := strings.Cut(s, ":")
if !strings.HasPrefix(source, "/") {
continue // a named volume: the runtime's, not the operator's path
}
source = filepath.Clean(source)
if source == path || strings.HasPrefix(source, path+"/") {
return true
}
}
return false
}
// AccessPathOn is where access id of module m is on a machine with these settings: its placement, else the
// definition's default; false where neither says, or the module declares no such access.
func AccessPathOn(m Manifest, layers []Layer, id string) (string, bool) {
_, byID, err := accessesFor(m, layers)
if err != nil {
return "", false
}
path, has := byID[id]
if !has || path == "" {
return "", false
}
return filepath.Clean(path), true
}
+370
View File
@@ -0,0 +1,370 @@
package catalogue
import (
"encoding/json"
"fmt"
"sort"
"strings"
"testing"
"github.com/novox/mesh-host/validate"
)
// Defends novox/hq ADR 0325 on the controller's side, row by row of its "How it is checked" table: the module check
// (rows 1, 2), the identity composed (row 3), the tool runner's entry (row 4), no identity root (row 11) — and none
// of it sent to a node-engine older than the contract, which would refuse the whole declaration.
const identityImage = "x@sha256:1111111111111111111111111111111111111111111111111111111111111111"
// aLibrarian files films: a container told its identity by PUID and PGID that mounts its config and the movies it
// writes; a sidecar that declares none and reads the music; a directory only its tools write; and tools served by the
// machine's runtime.
func aLibrarian(t *testing.T) Manifest {
t.Helper()
m := aToolsModule(t, "librarian", "tools/index.js")
m.Accesses = []Access{
{ID: "movies", Path: "/storage/media/movies", Mode: AccessReadWrite},
{ID: "music", Path: "/storage/media/music", Mode: AccessRead},
}
m.Resources = []map[string]any{
{"id": "config", "type": "directory", "mode": "0750"},
{"id": "notes", "type": "directory", "mode": "0700"},
{"id": "app", "type": "container", "name": "librarian", "image": identityImage,
"identity": map[string]any{"user": "env:PUID", "group": "env:PGID"},
"volumes": []any{"${dir:config}:/config", "${access:movies}:/movies"}},
{"id": "sidecar", "type": "container", "name": "librarian-sidecar", "image": identityImage,
"volumes": []any{"${access:music}:/music:ro"}},
}
return m
}
// aPlayer reads the movies the librarian writes, as its own identity through the runtime's user option, and has
// tools of its own that read them too.
func aPlayer(t *testing.T) Manifest {
t.Helper()
m := aToolsModule(t, "player", "tools/index.js")
m.Accesses = []Access{{ID: "movies", Path: "/storage/media/movies", Mode: AccessRead}}
m.Resources = []map[string]any{
{"id": "app", "type": "container", "name": "player", "image": identityImage,
"identity": map[string]any{"user": "run"}, "volumes": []any{"${access:movies}:/movies:ro"}},
}
return m
}
func keepingFolders(ids map[string]ModuleIdentity, kept ...string) Rendering {
with := Rendering{KeepsFolders: true, Identities: ids, KeptFolders: map[string]bool{}}
for _, k := range kept {
with.KeptFolders[k] = true
}
return with
}
func composeFolders(t *testing.T, r Resolution, with Rendering) Composed {
t.Helper()
with.Needed = map[string]map[string]string{RuntimeModule: {"broker": "sealed-credential"}}
with.ArtifactStore = "anchor.internal:5101"
composed, err := r.Compose(with)
if err != nil {
t.Fatal(err)
}
return composed
}
var identities = map[string]ModuleIdentity{"librarian": {UID: 50001, GID: 50001}, "player": {UID: 50002, GID: 50002}}
// Row 3: the identity composed — the same on every machine, the container told it, the module's directories its
// containers mount owned by it, and the declaration's identities naming the account.
func TestAModuleRunsAsItsIdentityTheSameOnEveryMachine(t *testing.T) {
var seen []string
for _, node := range []string{"ace", "novox"} {
r := Resolution{Node: node, Account: "jochen", Modules: []Manifest{aLibrarian(t), aPlayer(t), theRuntime(t)}}
composed := composeFolders(t, r, keepingFolders(identities, "/storage/media/movies")) // its read-write folder taken: switched
got := byID(composed.Resources)
app := got["librarian.app"]
ident, _ := app[IdentityField].(map[string]any)
want := map[string]any{"mode": "env", "user_env": "PUID", "group_env": "PGID", "uid": 50001, "gid": 50001}
if fmt.Sprint(ident) != fmt.Sprint(want) {
t.Errorf("%s: the container is told %v; want %v", node, ident, want)
}
player, _ := got["player.app"][IdentityField].(map[string]any)
if fmt.Sprint(player) != fmt.Sprint(map[string]any{"mode": "run", "uid": 50002, "gid": 50002}) {
t.Errorf("%s: the player is told %v", node, player)
}
if _, has := got["librarian.sidecar"][IdentityField]; has {
t.Errorf("%s: a container declaring no identity was given one", node)
}
if owner := got["librarian.config"]["owner"]; owner != "50001:50001" {
t.Errorf("%s: the directory its identity's container mounts is owned by %v", node, owner)
}
if owner, has := got["librarian.notes"]["owner"]; has {
t.Errorf("%s: a directory no identity container mounts was given the owner %v: what writes it is not "+
"the identity", node, owner)
}
raw, _ := json.Marshal(composed.Identities)
seen = append(seen, string(raw))
if !strings.Contains(string(raw), `"name":"mesh-librarian"`) || !strings.Contains(string(raw), `"uid":50001`) {
t.Errorf("%s: the declaration's identities are %s", node, raw)
}
}
// What the node-engine this controller pins (the ADR 0325 half) makes of it: nothing refused.
r := Resolution{Node: "ace", Account: "jochen", Modules: []Manifest{aLibrarian(t), aPlayer(t), theRuntime(t)}}
with := keepingFolders(identities, "/storage/media/movies")
composed := composeFolders(t, r, with)
body, err := json.Marshal(map[string]any{"declaration": validate.Version, "resources": composed.Resources,
"identities": composed.Identities})
if err != nil {
t.Fatal(err)
}
if problems := validate.Declaration(body); len(problems) > 0 {
t.Fatalf("the pinned node-engine refuses it: %v", problems)
}
if seen[0] != seen[1] {
t.Fatalf("the identities differ between machines: %s and %s", seen[0], seen[1])
}
}
// Rows 2, 4 and 8's composed part: each access names its users — the module's identity where a container declaring
// identity mounts it, never a container declaring none — and the tool runner's account by name, with no number,
// each at the access's own use (the node-engine keeps one entry per user at the widest use any access gives it, so
// the runner's is the widest among the modules whose tools use the folder); and whether the folder is taken.
func TestAnAccessNamesItsUsers(t *testing.T) {
r := Resolution{Node: "ace", Account: "jochen", Modules: []Manifest{aLibrarian(t), aPlayer(t), theRuntime(t)}}
with := keepingFolders(identities, "/storage/media/movies")
got := byID(composeFolders(t, r, with).Resources)
users := func(id string) string {
raw, _ := json.Marshal(got[id]["users"])
return string(raw)
}
movies := "librarian." + AccessID("/storage/media/movies")
if want := `[{"identity":"mesh-librarian","uid":50001,"use":"read-write","from":"container"},` +
`{"identity":"jochen","use":"read-write","from":"tool-runner"}]`; users(movies) != want {
t.Errorf("the librarian's movies name %s; want %s", users(movies), want)
}
played := "player." + AccessID("/storage/media/movies")
if want := `[{"identity":"mesh-player","uid":50002,"use":"read","from":"container"},` +
`{"identity":"jochen","use":"read","from":"tool-runner"}]`; users(played) != want {
t.Errorf("the player's movies name %s; want %s", users(played), want)
}
// The sidecar declares no identity: no entry for it (§2); the runner reads the music for the librarian's tools.
music := "librarian." + AccessID("/storage/media/music")
if want := `[{"identity":"jochen","use":"read","from":"tool-runner"}]`; users(music) != want {
t.Errorf("the music names %s; want only the tool runner's read", users(music))
}
if got[movies]["taken"] != true || got[music]["taken"] != false {
t.Errorf("taken is %v for the movies (taken) and %v for the music (not)", got[movies]["taken"], got[music]["taken"])
}
// The runner is named with no number: the engine resolves it by name, never as uid 0 sent from here.
if strings.Contains(users(music), `"uid"`) {
t.Errorf("the tool runner's entry carries a number: %s", users(music))
}
}
// The contract: to an engine that does not state link.IdentityContract nothing of ADR 0325 is sent — no identities, no
// container identity, no users, no taken, no owner from an identity — and what is sent is what the pinned node-engine
// accepts, as before.
func TestNothingOfTheIdentityIsSentToAnOlderEngine(t *testing.T) {
r := Resolution{Node: "ace", Account: "jochen", Modules: []Manifest{aLibrarian(t), aPlayer(t), theRuntime(t)}}
composed := composeFolders(t, r, Rendering{Identities: identities})
if len(composed.Identities) > 0 {
t.Fatalf("identities were composed for an older engine: %v", composed.Identities)
}
for _, res := range composed.Resources {
for _, field := range []string{IdentityField, "users", "taken"} {
if _, has := res[field]; has {
t.Errorf("%v carries %s for an older engine", res["id"], field)
}
}
if res["id"] == "librarian.config" && res["owner"] != nil {
t.Errorf("the directory was given the owner %v for an older engine", res["owner"])
}
}
body, err := json.Marshal(map[string]any{"declaration": validate.Version, "resources": composed.Resources})
if err != nil {
t.Fatal(err)
}
if problems := validate.Declaration(body); len(problems) > 0 {
t.Fatalf("the pinned node-engine refuses it: %v", problems)
}
// An engine that keeps folders, with no allocation yet (a question asked before the first send): composed
// without an identity, never with a made-up one.
none := composeFolders(t, r, keepingFolders(nil))
if len(none.Identities) > 0 || byID(none.Resources)["librarian.app"][IdentityField] != nil {
t.Fatal("an identity was composed with no allocation")
}
}
// Row 11: no identity outside the range is ever sent — 0 above all — and the composition says why.
func TestNoModuleIdentityIsRoot(t *testing.T) {
for _, n := range []int{0, 4, 10, 1000, IdentityLow - 1, IdentityHigh + 1} {
if AllocatableID(n) {
t.Errorf("%d is allocatable", n)
}
r := Resolution{Node: "ace", Account: "jochen", Modules: []Manifest{aLibrarian(t), theRuntime(t)}}
with := keepingFolders(map[string]ModuleIdentity{"librarian": {UID: n, GID: n}})
with.Needed = map[string]map[string]string{RuntimeModule: {"broker": "sealed-credential"}}
with.ArtifactStore = "anchor.internal:5101"
if _, err := r.Compose(with); err == nil || !strings.Contains(err.Error(), "§11") {
t.Errorf("an identity of %d composed: %v", n, err)
}
}
for _, n := range []int{IdentityLow, IdentityHigh} {
if !AllocatableID(n) {
t.Errorf("%d is not allocatable", n)
}
}
}
func manifestJSON(t *testing.T, m map[string]any) []byte {
t.Helper()
raw, err := json.Marshal(m)
if err != nil {
t.Fatal(err)
}
return raw
}
func aContainerModule(name string, resources ...map[string]any) map[string]any {
return map[string]any{"module": name, "version": "1", "capabilities": []any{"container-runtime"},
"accesses": []any{map[string]any{"id": "movies", "path": "/storage/media/movies", "mode": "read-write"}},
"resources": resources}
}
// Rows 1 and 2: the module check accepts identity as env:<NAME> or run and refuses every other shape and an owner in
// it; a module that declares identity is refused a directory naming its owner and a container declaring none that
// mounts its own directory or a read-write access; a directory's mode is accepted.
func TestTheModuleCheckHoldsAnIdentityModuleToTheDecision(t *testing.T) {
ok := func(identity any) map[string]any {
return map[string]any{"id": "app", "type": "container", "name": "app", "image": identityImage,
"identity": identity, "volumes": []any{"${dir:config}:/config", "${access:movies}:/movies"}}
}
config := map[string]any{"id": "config", "type": "directory", "mode": "0750"}
for _, identity := range []any{
map[string]any{"user": "env:PUID", "group": "env:PGID"},
map[string]any{"user": "run"},
} {
if _, err := ParseManifest(manifestJSON(t, aContainerModule("app", config, ok(identity)))); err != nil {
t.Errorf("identity %v refused: %v", identity, err)
}
}
for _, c := range []struct {
identity any
want string
}{
{map[string]any{"user": "1000", "group": "1000"}, `"env:<NAME>" or "run"`},
{map[string]any{"user": "env:PUID"}, "told its group the same way"},
{map[string]any{"user": "run", "group": "env:PGID"}, "takes no group"},
{map[string]any{"user": "env:puid", "group": "env:PGID"}, "environment variable's name"},
{map[string]any{"user": "env:PUID", "group": "env:PGID", "owner": "1000:1000"}, "never an owner or a number"},
{"run", "identity is"},
} {
_, err := ParseManifest(manifestJSON(t, aContainerModule("app", config, ok(c.identity))))
if err == nil || !strings.Contains(err.Error(), c.want) || !strings.Contains(err.Error(), "ADR 0325") {
t.Errorf("identity %v: %v, want it refused saying %q", c.identity, err, c.want)
}
}
// An identity on anything but a container.
onAFile := map[string]any{"id": "f", "type": "directory", "mode": "0750", "identity": map[string]any{"user": "run"}}
if _, err := ParseManifest(manifestJSON(t, aContainerModule("app", onAFile))); err == nil {
t.Error("an identity on a directory was accepted")
}
// Held to the decision whole once it declares identity (§1, §2, §3).
owned := map[string]any{"id": "config", "type": "directory", "mode": "0750", "owner": "1000:1000"}
bare := map[string]any{"id": "helper", "type": "container", "name": "helper", "image": identityImage,
"volumes": []any{"${dir:config}:/config", "${access:movies}:/movies"}}
_, err := ParseManifest(manifestJSON(t, aContainerModule("app", owned, ok(map[string]any{"user": "run"}), bare)))
for _, want := range []string{"names its owner 1000:1000", "declares no identity and mounts the module's own directory config",
"declares no identity and mounts the read-write access movies"} {
if err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("an identity module: %v, want it refused saying %q", err, want)
}
}
// A setting's owner too.
settingOwned := map[string]any{"id": "config", "type": "directory", "mode": "0750", "owner": "${setting:puid}:${setting:pgid}"}
identityModule := aContainerModule("app", settingOwned, ok(map[string]any{"user": "run"}))
identityModule["settings"] = map[string]any{"puid": map[string]any{"default": "1000"}, "pgid": map[string]any{"default": "1000"}}
if _, err := ParseManifest(manifestJSON(t, identityModule)); err == nil || !strings.Contains(err.Error(), "names its owner") {
t.Errorf("a setting's owner in an identity module: %v", err)
}
}
// Rows 1 and 2 for what is registered today: a module that declares no identity is warned, never refused — a
// directory naming its owner, a container mounting its own directory or a read-write access — and each warning
// names ADR 0325. A placement's owner is refused for an identity module and warned of for any other.
func TestAModuleWithoutIdentityIsWarnedNotRefused(t *testing.T) {
owned := map[string]any{"id": "config", "type": "directory", "mode": "0750", "owner": "1000:1000"}
bare := map[string]any{"id": "app", "type": "container", "name": "app", "image": identityImage,
"volumes": []any{"${dir:config}:/config", "${access:movies}:/movies"}}
m, err := ParseManifest(manifestJSON(t, aContainerModule("legacy", owned, bare)))
if err != nil {
t.Fatalf("a registered shape was refused: %v", err)
}
warnings := IdentityWarnings(m)
sort.Strings(warnings)
if len(warnings) != 3 {
t.Fatalf("warned %d times: %q", len(warnings), warnings)
}
for _, w := range warnings {
if !strings.Contains(w, "ADR 0325") || !strings.Contains(w, "refused once they have moved") {
t.Errorf("a warning that does not name ADR 0325 or say refusal comes later: %q", w)
}
}
placed := []Layer{{From: "ace", Values: map[string]any{PlacesSetting: map[string]any{
"config": map[string]any{"path": "/srv/legacy", "owner": "1000:1000"}}}}}
if err := PlacesOwnerProblem(m, placed); err != nil {
t.Errorf("a placement's owner refused for a module without identity: %v", err)
}
if w := PlacesOwnerWarnings(m, placed); len(w) != 1 || !strings.Contains(w[0], "places.config.owner") {
t.Errorf("a placement's owner warned %q", w)
}
withIdentity := aLibrarian(t)
placedLibrarian := []Layer{{From: "ace", Values: map[string]any{PlacesSetting: map[string]any{
"config": map[string]any{"path": "/srv/librarian", "owner": "1000:1000"}}}}}
if err := PlacesOwnerProblem(withIdentity, placedLibrarian); err == nil || !strings.Contains(err.Error(), "ADR 0325") {
t.Errorf("a placement's owner for an identity module: %v", err)
}
if len(IdentityWarnings(withIdentity)) != 0 {
t.Error("an identity module was warned rather than refused")
}
}
// A container is not switched to its identity while a read-write folder it mounts holds data the mesh has not taken
// (the operator's question on ADR 0325): the app would run as the new uid before the take adds its entry, and lose
// write on the existing files. Its identity is held — no `identity` sent, its directories keep their owner — while
// the take's users already name the identity, so the entries exist before the switch; once every such folder is
// taken (or kept by the engine), the identity is sent in one apply.
func TestAContainerKeepsItsOldIdentityUntilItsDataIsTaken(t *testing.T) {
r := Resolution{Node: "ace", Account: "jochen", Modules: []Manifest{aLibrarian(t), theRuntime(t)}}
held := byID(composeFolders(t, r, keepingFolders(identities)).Resources)
if _, has := held["librarian.app"][IdentityField]; has {
t.Fatalf("the container was told its identity while its movies are not taken: %v", held["librarian.app"][IdentityField])
}
if owner, has := held["librarian.config"]["owner"]; has {
t.Fatalf("its directory was given the owner %v while its identity is held", owner)
}
users, _ := json.Marshal(held["librarian."+AccessID("/storage/media/movies")]["users"])
if !strings.Contains(string(users), `"identity":"mesh-librarian","uid":50001`) {
t.Fatalf("the access does not name the identity to come, so the take would not add its entry: %s", users)
}
for name, with := range map[string]Rendering{
"taken": keepingFolders(identities, "/storage/media/movies"),
"kept by the engine": func() Rendering {
w := keepingFolders(identities)
w.ReadyFolders = map[string]bool{"/storage/media/movies": true}
return w
}(),
} {
got := byID(composeFolders(t, r, with).Resources)
if _, has := got["librarian.app"][IdentityField]; !has {
t.Errorf("%s: the container is not told its identity", name)
}
if got["librarian.config"]["owner"] != "50001:50001" {
t.Errorf("%s: its directory is owned by %v", name, got["librarian.config"]["owner"])
}
}
}
+34 -19
View File
@@ -68,9 +68,10 @@ func TestFailedIsAnOptionalVerbOfTheServiceManager(t *testing.T) {
holder := func(serves ...string) Manifest {
return Manifest{Module: "systemd", Version: "1", Claims: []Claim{{Name: ServiceManagerSeat, Scope: ScopeNode, Serves: serves}}}
}
eight := []string{"units", "status", "start", "stop", "restart", "enable", "disable", "journal"}
eight := []string{"units", "status", "start", "stop", "restart", "enable", "disable", "journal",
"reset-failed", "wanted-by", "unlink-dangling"}
if err := CanHold(holder(eight...), seat); err != nil {
t.Fatalf("today's holder, without failed, is refused: %v", err)
t.Fatalf("a holder without failed is refused: %v", err)
}
if err := CanHold(holder(append(eight, "failed")...), seat); err != nil {
t.Fatalf("a holder serving failed is refused: %v", err)
@@ -81,7 +82,7 @@ func TestFailedIsAnOptionalVerbOfTheServiceManager(t *testing.T) {
if err := CanHold(holder(eight[1:]...), seat); err == nil || !strings.Contains(err.Error(), "does not serve units") {
t.Fatalf("a holder missing a required verb was accepted: %v", err)
}
optional := map[string]bool{"failed": true, "reset-failed": true, "wanted-by": true, "unlink-dangling": true}
optional := map[string]bool{"failed": true}
for _, v := range seat.Serves {
if v.Optional != optional[v.Name] {
t.Errorf("%s optional: %v", v.Name, v.Optional)
@@ -89,26 +90,35 @@ func TestFailedIsAnOptionalVerbOfTheServiceManager(t *testing.T) {
}
}
// **`reset-failed` and `wanted-by` join the seat optional** (novox/hq issue 332, ADR 0246 step 1): the
// holder running today, serving the nine verbs, still holds the seat; a holder serving all eleven is not
// refused; each replaces the shell command it names; and read back from the store's row they stay optional.
func TestResetFailedAndWantedByAreOptionalVerbsOfTheServiceManager(t *testing.T) {
// **`reset-failed`, `wanted-by` and `unlink-dangling` are required** (novox/hq issue 332, ADR 0246 step 3):
// every holder serves them, so a holder serving only the nine verbs before them is refused by the verb it
// lacks; a holder serving all twelve holds the seat; each replaces the shell command it names; and read
// back from the store's row they stay required.
func TestResetFailedWantedByAndUnlinkDanglingAreRequiredVerbsOfTheServiceManager(t *testing.T) {
defer UseSeats(DefaultSeats())
nine := []string{"units", "status", "start", "stop", "restart", "enable", "disable", "journal", "failed"}
all := append(slices.Clone(nine), "reset-failed", "wanted-by", "unlink-dangling")
holder := func(serves ...string) Manifest {
return Manifest{Module: "systemd", Version: "1", Claims: []Claim{{Name: ServiceManagerSeat, Scope: ScopeNode, Serves: serves}}}
}
refusedWithout := func(seat Seat) {
t.Helper()
for _, verb := range []string{"reset-failed", "wanted-by", "unlink-dangling"} {
serves := slices.DeleteFunc(slices.Clone(all), func(v string) bool { return v == verb })
if err := CanHold(holder(serves...), seat); err == nil || !strings.Contains(err.Error(), "does not serve "+verb) {
t.Errorf("a holder not serving %s holds the seat: %v", verb, err)
}
}
}
seat, _ := SeatNamed(ServiceManagerSeat)
if err := CanHold(holder(nine...), seat); err != nil {
t.Fatalf("today's holder, without the two verbs, is refused: %v", err)
if err := CanHold(holder(nine...), seat); err == nil {
t.Fatal("a holder serving only the nine verbs before issue 332 holds the seat")
}
if err := CanHold(holder(append(nine, "reset-failed", "wanted-by")...), seat); err != nil {
t.Fatalf("a holder serving both is refused: %v", err)
refusedWithout(seat)
if err := CanHold(holder(all...), seat); err != nil {
t.Fatalf("a holder serving all twelve is refused: %v", err)
}
if err := CanHold(holder(append(nine, "reset-failed", "wanted-by", "unlink-dangling")...), seat); err != nil {
t.Fatalf("a holder serving unlink-dangling too is refused: %v", err)
}
if err := CanHold(holder(append(nine, "why-started")...), seat); err == nil || !strings.Contains(err.Error(), "does not promise") {
if err := CanHold(holder(append(slices.Clone(all), "why-started")...), seat); err == nil || !strings.Contains(err.Error(), "does not promise") {
t.Fatalf("a verb the seat does not promise was accepted: %v", err)
}
replaces := map[string]string{"reset-failed": "systemctl reset-failed", "wanted-by": "systemctl list-dependencies --reverse",
@@ -118,6 +128,9 @@ func TestResetFailedAndWantedByAreOptionalVerbsOfTheServiceManager(t *testing.T)
if !ok {
continue
}
if v.Optional {
t.Errorf("%s is still optional", v.Name)
}
if !slices.Contains(v.Replaces, want) {
t.Errorf("%s does not say it replaces %q: %v", v.Name, want, v.Replaces)
}
@@ -148,8 +161,9 @@ func TestResetFailedAndWantedByAreOptionalVerbsOfTheServiceManager(t *testing.T)
}
UseSeats(rows)
seat, _ = SeatNamed(ServiceManagerSeat)
if err := CanHold(holder(nine...), seat); err != nil {
t.Fatalf("read back from the row, the two verbs are required: %v", err)
refusedWithout(seat)
if err := CanHold(holder(all...), seat); err != nil {
t.Fatalf("read back from the row, a holder serving all twelve is refused: %v", err)
}
}
@@ -169,8 +183,9 @@ func TestAnOptionalVerbStaysOptionalInASetReadFromTheStore(t *testing.T) {
UseSeats(rows)
seat, _ := SeatNamed(ServiceManagerSeat)
holder := Manifest{Module: "systemd", Version: "1", Claims: []Claim{{Name: ServiceManagerSeat, Scope: ScopeNode,
Serves: []string{"units", "status", "start", "stop", "restart", "enable", "disable", "journal"}}}}
Serves: []string{"units", "status", "start", "stop", "restart", "enable", "disable", "journal",
"reset-failed", "wanted-by", "unlink-dangling"}}}}
if err := CanHold(holder, seat); err != nil {
t.Fatalf("a set read from rows refuses today's holder: %v", err)
t.Fatalf("a set read from rows refuses a holder without the optional failed: %v", err)
}
}
@@ -0,0 +1,106 @@
package catalogue
import (
"strings"
"testing"
)
// An own secret issued outside the mesh that nobody gave (novox/hq issue 401, ADR 0228): the mesh makes no value for it,
// so the composition places no file, a file of the module that names it says empty there, and nothing restarts on it.
// Given, it is placed and filled like any own secret.
func outsideManifest(t *testing.T) Manifest {
t.Helper()
m, err := ParseManifest([]byte(`{"module":"assistant","version":"1",
"own-secrets":{
"password":{"path":"/var/lib/assistant/password.secret","taken":"at-start"},
"vendor-key":{"path":"/var/lib/assistant/vendor-key.secret","taken":"at-start","issued-by":"outside"}},
"resources":[
{"id":"env","type":"file","path":"/var/lib/assistant/env","mode":"0600",
"content":"PASSWORD=${secret:password}\nVENDOR_KEY=${secret:vendor-key}\n"},
{"id":"server","type":"container","name":"assistant","image":"assistant:1",
"env-file":["/var/lib/assistant/env"],
"volumes":["/var/lib/assistant/vendor-key.secret:/run/vendor-key:ro"],
"secrets-in-environment":"reads its settings from the environment only"}]}`))
if err != nil {
t.Fatal(err)
}
return m
}
func resourceAt(out []map[string]any, path string) map[string]any {
for _, r := range out {
if r["path"] == path {
return r
}
}
return nil
}
func resourceNamed(out []map[string]any, name string) map[string]any {
for _, r := range out {
if r["name"] == name {
return r
}
}
return nil
}
func TestAnOutsideSecretNotGivenIsNoFileAndAFileNamingItSaysEmpty(t *testing.T) {
r := Resolution{Node: "server", Modules: []Manifest{outsideManifest(t)}}
out, err := r.Declaration(Rendering{Needed: map[string]map[string]string{"assistant": {"password": "sealed-password"}}})
if err != nil {
t.Fatalf("a module whose outside secret nobody gave did not compose: %v", err)
}
if got := resourceAt(out, "/var/lib/assistant/vendor-key.secret"); got != nil {
t.Fatalf("a secret nobody gave was placed: %v", got)
}
env := resourceAt(out, "/var/lib/assistant/env")
if env == nil {
t.Fatal("the module's own file is not composed")
}
if content := env["content"].(string); content != "PASSWORD=${secret:password}\nVENDOR_KEY=\n" {
t.Fatalf("the file names the secret nobody gave as %q", content)
}
secrets, _ := env["secrets"].(map[string]any)
if len(secrets) != 1 || secrets["password"] != "sealed-password" {
t.Fatalf("the file is filled from %v", secrets)
}
server := resourceNamed(out, "assistant")
if on := strings.Join(stringsIn(server["restart-on"]), " "); strings.Contains(on, "vendor-key") {
t.Fatalf("the container restarts on a secret nobody gave: %q", on)
}
}
func TestAnOutsideSecretGivenIsPlacedAndFilled(t *testing.T) {
r := Resolution{Node: "server", Modules: []Manifest{outsideManifest(t)}}
out, err := r.Declaration(Rendering{Needed: map[string]map[string]string{"assistant": {
"password": "sealed-password", "vendor-key": "sealed-vendor-key"}}})
if err != nil {
t.Fatal(err)
}
placed := resourceAt(out, "/var/lib/assistant/vendor-key.secret")
if placed == nil || placed["sealed"] != "sealed-vendor-key" {
t.Fatalf("the secret given is placed as %v", placed)
}
env := resourceAt(out, "/var/lib/assistant/env")
if content := env["content"].(string); content != "PASSWORD=${secret:password}\nVENDOR_KEY=${secret:vendor-key}\n" {
t.Fatalf("the file was changed: %q", content)
}
if secrets, _ := env["secrets"].(map[string]any); secrets["vendor-key"] != "sealed-vendor-key" {
t.Fatalf("the file is filled from %v", secrets)
}
server := resourceNamed(out, "assistant")
if on := strings.Join(stringsIn(server["restart-on"]), " "); !strings.Contains(on, "assistant."+NeedID("vendor-key")) {
t.Fatalf("the container does not restart on the secret given: %q", on)
}
}
// A secret the mesh makes is still refused when it was not made: only an outside one is left out.
func TestAnOwnSecretTheMeshMakesIsStillRefusedWhenNotMade(t *testing.T) {
r := Resolution{Node: "server", Modules: []Manifest{outsideManifest(t)}}
_, err := r.Declaration(Rendering{})
if nm, ok := err.(*NotMadeError); !ok || nm.Name != "password" {
t.Fatalf("a secret the mesh makes, not made: %v", err)
}
}
+32 -3
View File
@@ -1,6 +1,7 @@
package catalogue
import (
"errors"
"fmt"
"path/filepath"
"regexp"
@@ -270,6 +271,31 @@ func Places(m Manifest, layers []Layer) (map[string]Placement, error) {
// It refuses an id the module declares no access under, and a path that is not absolute. An
// access declared by path alone cannot be placed — it has no name to place it by.
func AccessPlaces(m Manifest, layers []Layer) (map[string]string, error) {
return accessPlaces(m, layers, true)
}
// UndeclaredAccessError is a layer placing an access the definition does not declare (novox/hq issue 575): refused
// where a setting is stored, unless a newer build of the module declares it; passed by where a machine is composed.
type UndeclaredAccessError struct {
Module, ID string
said string
}
func (e *UndeclaredAccessError) Error() string { return e.said }
// UndeclaredAccessPlaces refuses a layer that places an access the definition does not declare.
func UndeclaredAccessPlaces(m Manifest, layers []Layer) error {
_, err := accessPlaces(m, layers, true)
var undeclared *UndeclaredAccessError
if errors.As(err, &undeclared) {
return err
}
return nil
}
// accessPlaces is AccessPlaces; where strict is false an access the definition does not declare is passed by
// (novox/hq issue 575), as a placement stored ahead of the build that declares it.
func accessPlaces(m Manifest, layers []Layer, strict bool) (map[string]string, error) {
declared := map[string]bool{}
for _, a := range m.Accesses {
if a.ID != "" {
@@ -289,9 +315,12 @@ func AccessPlaces(m Manifest, layers []Layer) (map[string]string, error) {
}
for id, body := range blocks {
if !declared[id] {
return nil, fmt.Errorf(
if !strict {
continue
}
return nil, &UndeclaredAccessError{Module: m.Module, ID: id, said: fmt.Sprintf(
"%s places the access %q, which it does not declare — the setting reaches "+
"nothing. It declares %s", m.Module, id, orNothing(namesOfAccesses(m)))
"nothing. It declares %s", m.Module, id, orNothing(namesOfAccesses(m)))}
}
path, _ := body.(string)
path = strings.TrimSpace(path)
@@ -324,7 +353,7 @@ type placedAccess struct {
// placed one, the definition's where it carries a default, and refused where neither says — an
// access that resolves to nowhere would reach the machine as a mount of nothing.
func accessesFor(m Manifest, layers []Layer) ([]placedAccess, map[string]string, error) {
placed, err := AccessPlaces(m, layers)
placed, err := accessPlaces(m, layers, false)
if err != nil {
return nil, nil, err
}
+220
View File
@@ -0,0 +1,220 @@
package catalogue
import (
"encoding/json"
"fmt"
"path/filepath"
"sort"
"strings"
)
// A node's files and accounts, read (novox/hq ADR 0310, extending ADR 0268 §5; hq to-be 41 WP8).
//
// **A seat of its own on every node.** Reading a file, searching one, listing a directory, the owner and mode
// of each component of a path, and looking up an account or a group are the reads ADR 0268 §5 has narrow tools
// take over from the login shell's `execute`, so an approval is asked for an act and never for a read. Every
// node needs them, the control-node included, and another module could offer them instead of the tool
// runner's (ADR 0126 §9): a seat, so the mesh finds them on every node by one name.
//
// **Read as an account that holds no secret.** The holder runs as an account of its own (runs-as, ADR 0259
// §8), so the node's file modes bound what it sees — never the operator's account, which sees the
// operator's keys and the agent's credentials. On top, it refuses by name the places a secret is kept,
// which the controller names for it (secretPlaces), and every answer passes the content rule's secret class
// (mesh-sdk's outward, ADR 0310 §4).
// ReadsSeat is the role that reads a node's files and accounts (novox/hq ADR 0310).
const ReadsSeat = "node-reads"
// ReadsSecretPlaces is the word the holder of node-reads is given: the places on its node where a secret is
// kept, as a JSON list of absolute paths, each refused by name before any read (novox/hq ADR 0310 §5).
const ReadsSecretPlaces = "MESH_SECRET_PLACES"
// readsVerbs is the contract every holder of node-reads serves (novox/hq ADR 0310 §2). The bounds are the
// build's to set and each verb's description to state (ADR 0310, Consequences); the holder states them in
// every answer it cuts.
func readsVerbs() []Verb {
return []Verb{
{Name: "stat", Description: "Each component of one path or several, from / down: its type, owner and group " +
"(name and number), mode, whether it is a link and to what, and whether it is a mount point and of which " +
"file system. A link is not followed unless follow is \"true\". Read as the holder's own account, which " +
"holds no secret: a component it may not reach says so. At most 200 paths in one call.",
Input: schema(map[string]string{
"paths": "one absolute path, or several separated by line breaks",
"follow": "\"true\": also say where the path leads once every link is followed",
}, []string{"paths"}, "follow"),
Replaces: []string{"stat", "ls -ld", "namei -l", "findmnt -T"}},
{Name: "readlink", Description: "What one link points to, as written, and where the path leads once every " +
"link is followed, and whether that exists.",
Input: schema(map[string]string{"path": "an absolute path"}, []string{"path"}),
Replaces: []string{"readlink", "readlink -f", "realpath"}},
{Name: "list", Description: "Every entry of one directory with its type, owner, group, mode, size and, for a " +
"link, its target. At most 2000 entries; a longer directory is cut and says so. A place a secret is kept " +
"is refused by name.",
Input: schema(map[string]string{"path": "an absolute path to a directory"}, []string{"path"}),
Replaces: []string{"ls", "ls -la"}},
{Name: "read", Description: "One text file, or a part of it by line: at most 256 KiB in one answer " +
"(64 KiB unless more is asked), cut at a line and said so. A binary file is refused; a place a secret is " +
"kept, or a known credential file, is refused by name; a line shaped like a secret is withheld and its " +
"number said. /proc and /sys are read as far as the holder's account may.",
Input: schema(map[string]string{
"path": "an absolute path to a file",
"from": "the first line to answer, counted from 1 (optional)",
"lines": "how many lines to answer (optional)",
"bytes": "the most bytes to answer, up to 262144 (optional, 65536 unless given)",
}, []string{"path"}),
Replaces: []string{"cat", "head", "tail", "less"}},
{Name: "search", Description: "Find a fixed string (never a pattern) in one file or under one directory: " +
"every matching line with its file and number. Bounded: at most 500 matches (100 unless more is asked), " +
"8 directories deep (4 unless more is asked), 5000 files and 4 MiB per file; links are not followed and " +
"binary files are skipped. A place a secret is kept is skipped and named; a matching line shaped like a " +
"secret is withheld. The answer says which bound it met.",
Input: schema(map[string]string{
"text": "the fixed string to find",
"path": "an absolute path to a file or a directory",
"depth": "how many directories deep, up to 8 (optional)",
"matches": "the most matches to answer, up to 500 (optional)",
"case": "\"false\": match whatever the case (optional)",
}, []string{"text", "path"}, "case"),
Replaces: []string{"grep", "grep -r", "grep -F"}},
{Name: "account", Description: "One account or group by name or number, as this node's account files say: " +
"an account's number, group, home, shell and every group it is in; a group's number and members. With " +
"neither, every account and group. Never a password field.",
Input: schema(map[string]string{
"name": "an account's name or number (optional)",
"group": "a group's name or number (optional)",
}, nil),
Replaces: []string{"getent passwd", "getent group", "id", "groups"}},
}
}
// ReadsHolderProblems is what a holder of node-reads is held to (novox/hq ADR 0310 §3): it reads as an
// account of its own, so the node's file modes bound what it sees. Carried by the machine's runtime it would
// read as the operator's account, which sees the operator's keys and the agent's credentials.
func ReadsHolderProblems(m Manifest) []string {
if !m.ClaimsSeat(ReadsSeat) {
return nil
}
if m.RunsAs == "" {
return []string{fmt.Sprintf("%s claims %s and must run as an account of its own (runs-as): its reads are "+
"bounded by that account's file modes, and the machine's runtime runs as the operator's account "+
"(novox/hq ADR 0310 §3)", m.Module, ReadsSeat)}
}
// In no group beside its own: a group is a way to a secret (docker's is root's), and the account's file modes
// are what bound its reads.
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) != "user" || fmt.Sprint(r["name"]) != m.RunsAs {
continue
}
if groups, said := r["groups"]; said {
if list, ok := groups.([]any); !ok || len(list) > 0 {
return []string{fmt.Sprintf("%s claims %s, and its account %s is put in groups %v: the account that "+
"reads is in no group beside its own (novox/hq ADR 0310 §3)", m.Module, ReadsSeat, m.RunsAs, groups)}
}
}
}
return nil
}
// secretPlaces is every place on this node a secret is kept, as the placed manifests say (novox/hq ADR 0310
// §5): the directory of every module's own secret, every credential file a module is given and every grant
// directory a provider is handed its consumers' credentials in, the place a module keeps the operator-sealed
// secrets, and every directory of the module that provides `secret` — the vault — wherever the node places it,
// a directory declared without a path included. Sorted, each once, and a place under another dropped, so the
// list says each refusal once.
//
// **Failure is loud.** A place still holding a placeholder, or not absolute, fails the composition with the
// module named: a list that silently shrank would be a refusal the decision promises and nothing does.
func (r Resolution) secretPlaces(with Rendering) ([]string, error) {
seen := map[string]bool{}
var problems []string
add := func(module, what, path string) {
path = strings.TrimSpace(path)
if path == "" {
return
}
if !strings.HasPrefix(path, "/") || strings.Contains(path, "${") {
problems = append(problems, fmt.Sprintf("%s's %s is %q, which is not a place on %s", module, what, path, r.Node))
return
}
path = filepath.Clean(path)
if path == "/" {
problems = append(problems, fmt.Sprintf("%s's %s is the root directory", module, what))
return
}
seen[path] = true
}
for _, m := range r.Modules {
for name, own := range m.OwnSecrets {
add(m.Module, "own secret "+name, own.Path)
if strings.HasPrefix(own.Path, "/") {
add(m.Module, "own secret "+name+"'s directory", filepath.Dir(own.Path))
}
}
for _, to := range m.SecretRequirements() {
for _, f := range m.SecretFiles(to) {
add(m.Module, "credential file for "+to, f.Path)
}
}
for to, where := range m.Grants {
add(m.Module, "grant directory for "+to, where)
}
add(m.Module, "place for the operator-sealed secrets (keeps)", m.Keeps)
if providesSecret(m) {
dirs := dirsFor(m, with)
for _, id := range sortedKeys(dirs) {
add(m.Module, "directory "+id, dirs[id])
}
}
}
if len(problems) > 0 {
sort.Strings(problems)
return nil, fmt.Errorf("the holder of %s on %s cannot be told where a secret is kept, so it would read "+
"one (novox/hq ADR 0310 §5): %s", ReadsSeat, r.Node, strings.Join(problems, "; "))
}
all := make([]string, 0, len(seen))
for p := range seen {
all = append(all, p)
}
sort.Strings(all)
var out []string
for _, p := range all {
if n := len(out); n > 0 && strings.HasPrefix(p, out[n-1]+"/") {
continue
}
out = append(out, p)
}
return out, nil
}
func providesSecret(m Manifest) bool {
for _, p := range m.Provides {
if p.Name == "secret" {
return true
}
}
return false
}
// readsWords is what the holder of node-reads is given beside its own words: the node's secret places.
func (r Resolution) readsWords(m Manifest, words map[string]string, with Rendering) (map[string]string, error) {
if !m.ClaimsSeat(ReadsSeat) {
return words, nil
}
places, err := r.secretPlaces(with)
if err != nil {
return nil, err
}
body, err := json.Marshal(places)
if err != nil {
return nil, err
}
if words == nil {
words = map[string]string{}
}
if _, has := words[ReadsSecretPlaces]; has {
return nil, fmt.Errorf("%s gives its own %s; the node's secret places are the mesh's to say (novox/hq ADR 0310 §5)",
m.Module, ReadsSecretPlaces)
}
words[ReadsSecretPlaces] = string(body)
return words, nil
}
+136
View File
@@ -0,0 +1,136 @@
package catalogue
import (
"encoding/json"
"strings"
"testing"
)
// novox/hq ADR 0310 §1, §2: node-reads is a node seat, and its verbs say the commands they replace, so
// mesh_search finds them by `cat`, `grep` and `getent passwd` (ADR 0245 §3).
func TestNodeReadsIsANodeSeatWhoseVerbsSayWhatTheyReplace(t *testing.T) {
s, ok := SeatNamed(ReadsSeat)
if !ok {
t.Fatalf("the mesh defines no seat %s", ReadsSeat)
}
if s.Scope != ScopeNode || s.Delivers != "" || !strings.Contains(s.Decision, "0310") {
t.Errorf("%s: scope %q, delivers %q, decision %q", ReadsSeat, s.Scope, s.Delivers, s.Decision)
}
replaced := map[string]string{}
for _, v := range s.Serves {
if v.Optional {
t.Errorf("%s.%s is optional: a new seat's holder serves every verb", ReadsSeat, v.Name)
}
if len(v.Replaces) == 0 {
t.Errorf("%s.%s says nothing it replaces", ReadsSeat, v.Name)
}
for _, c := range v.Replaces {
replaced[c] = v.Name
}
}
for command, verb := range map[string]string{"cat": "read", "grep": "search", "getent passwd": "account",
"getent group": "account", "id": "account", "stat": "stat", "readlink": "readlink", "ls": "list"} {
if replaced[command] != verb {
t.Errorf("%q is replaced by %q, want %s.%s", command, replaced[command], ReadsSeat, verb)
}
}
}
// novox/hq ADR 0310 §3: a holder of node-reads reads as an account of its own; one the machine's runtime would
// carry — as the operator's account — is refused at parse.
func TestAHolderOfNodeReadsWithoutAnAccountOfItsOwnIsRefused(t *testing.T) {
_, err := ParseManifest([]byte(`{"module": "reads", "version": "1",
"claims": [{"name": "node-reads", "scope": "node"}]}`))
if err == nil || !strings.Contains(err.Error(), "runs-as") || !strings.Contains(err.Error(), "0310") {
t.Errorf("a holder of node-reads without runs-as was not refused for it: %v", err)
}
if p := ReadsHolderProblems(Manifest{Module: "reads", RunsAs: "mesh-reads",
Claims: []Claim{{Name: ReadsSeat}}}); len(p) != 0 {
t.Errorf("a holder running as its own account was refused: %v", p)
}
}
// novox/hq ADR 0310 §5: the holder is told, by the composition, where its node keeps a secret — the directory
// of every module's own secret, every credential file and grant directory, the vault's directories and the
// place the operator-sealed secrets are kept — and no other module is told.
func TestTheHolderOfNodeReadsIsToldWhereItsNodeKeepsASecret(t *testing.T) {
with := Rendering{ArtifactStore: "anchor.internal:5101",
Needed: map[string]map[string]string{RuntimeModule: {"broker": "sealed-credential"},
"reads": {"broker": "own"}, "desk": {"broker": "own"}}}
goRuntime := Manifest{Module: RuntimeModule, Version: "1",
OwnSecrets: OwnSecrets{"broker": {Path: "/var/lib/mesh/" + RuntimeModule + "/broker"}},
Build: &Build{Artifacts: []Artifact{{Name: "runtime", Kind: ArtifactBundle, Language: "go",
System: "arch", From: "cmd/node-tools"}}}}
goRuntime, err := goRuntime.Resolve([]Built{{Name: "runtime", Kind: ArtifactBundle,
Reference: ArtifactStoreScheme + RuntimeModule + "/runtime/blobs/" + bundleDigest, Digest: bundleDigest}})
if err != nil {
t.Fatal(err)
}
reads := aToolsModule(t, "reads", "tools/index.js")
reads.RunsAs, reads.SecretsOwner = "mesh-reads", "mesh-reads"
reads.Claims = []Claim{{Name: ReadsSeat}}
reads.OwnSecrets = OwnSecrets{"broker": {Path: "/var/lib/mesh-reads/broker"}}
desk := aToolsModule(t, "desk", "tools/index.js")
desk.RunsAs, desk.SecretsOwner = "desk", "desk"
desk.OwnSecrets = OwnSecrets{"broker": {Path: "/var/lib/desk/broker"}}
// The vault as the catalogue declares it: its directories have no path of their own, and the node places them.
vault, err := placedManifest(catalogueManifest(t, "mesh-vault"), with)
if err != nil {
t.Fatal(err)
}
app := Manifest{Module: "app", Version: "1", Secrets: map[string]string{"postgres-database": "/etc/app/db.secret"},
OwnSecrets: OwnSecrets{"token": {Path: "/home/ops/.config/app/token"}}}
r := Resolution{Node: "anchor", Account: "ops", Modules: []Manifest{reads, desk, vault, app, goRuntime}}
owns, err := r.ownRuntimes(with)
if err != nil {
t.Fatal(err)
}
var given map[string]map[string]string
for _, p := range owns {
if p["id"] == "reads."+OwnRuntimeID() {
if err := json.Unmarshal([]byte(p["env"].(map[string]string)[RuntimeToolEnv]), &given); err != nil {
t.Fatal(err)
}
}
if p["id"] == "desk."+OwnRuntimeID() && strings.Contains(p["env"].(map[string]string)[RuntimeToolEnv], ReadsSecretPlaces) {
t.Error("a module not holding node-reads is told the node's secret places")
}
}
var places []string
if err := json.Unmarshal([]byte(given["reads"][ReadsSecretPlaces]), &places); err != nil {
t.Fatalf("the holder is not told its node's secret places: %v (%v)", err, given)
}
want := []string{"/etc/app/db.secret", "/home/ops/.config/app", "/var/lib/desk", "/var/lib/mesh-reads",
"/var/lib/mesh-vault", "/var/lib/mesh/" + RuntimeModule}
if strings.Join(places, " ") != strings.Join(want, " ") {
t.Errorf("secret places:\n got %v\nwant %v", places, want)
}
}
// The review of #257: a secret place the controller cannot resolve fails the composition, naming the module,
// rather than leaving the list one place shorter.
func TestASecretPlaceThatCannotBeResolvedFailsTheComposition(t *testing.T) {
r := Resolution{Node: "anchor", Modules: []Manifest{
{Module: "app", Version: "1", Keeps: "${dir:keep}/sealed"},
}}
if _, err := r.secretPlaces(Rendering{}); err == nil || !strings.Contains(err.Error(), "app") {
t.Errorf("an unresolved place was dropped: %v", err)
}
r.Modules[0].Keeps = "relative/place"
if _, err := r.secretPlaces(Rendering{}); err == nil {
t.Error("a relative place was dropped")
}
}
// ADR 0310 §3: the account that reads is in no group beside its own; one put in a group is refused.
func TestTheReadsAccountIsInNoGroup(t *testing.T) {
m := Manifest{Module: "reads", RunsAs: "mesh-reads", Claims: []Claim{{Name: ReadsSeat}},
Resources: []map[string]any{{"id": "account", "type": "user", "name": "mesh-reads", "groups": []any{"docker"}}}}
if p := ReadsHolderProblems(m); len(p) != 1 || !strings.Contains(p[0], "docker") {
t.Errorf("an account in docker was not refused: %v", p)
}
m.Resources[0]["groups"] = []any{}
if p := ReadsHolderProblems(m); len(p) != 0 {
t.Errorf("an account in no group was refused: %v", p)
}
}
+4
View File
@@ -303,6 +303,10 @@ func (r Resolution) ownRuntimes(with Rendering) ([]map[string]any, error) {
if err != nil {
return nil, err
}
// The holder of node-reads is told where this node keeps its secrets (novox/hq ADR 0310 §5).
if words, err = r.readsWords(m, words, with); err != nil {
return nil, err
}
if len(words) > 0 {
body, err := json.Marshal(map[string]map[string]string{m.Module: words})
if err != nil {
+10 -7
View File
@@ -310,6 +310,9 @@ var defaultSeats = append([]Seat{
// node-mounts writes a mount and an automount unit per share on a machine that asks (shares.go).
{Name: NFSServerSeat, Scope: ScopeNode, Decision: "novox/hq ADR 0263", Serves: nfsServerVerbs()},
{Name: MountsSeat, Scope: ScopeNode, Decision: "novox/hq ADR 0263", Serves: mountsVerbs()},
// A node's files and accounts, read (novox/hq ADR 0310, reads.go): by an account of the holder's own that holds
// no secret, on every node, so the reads ADR 0268 §5 takes over from the login shell's execute have a tool.
{Name: ReadsSeat, Scope: ScopeNode, Decision: "novox/hq ADR 0310", Serves: readsVerbs()},
},
// The graphical session's roles (novox/hq ADR 0208), last because they are a workstation's.
graphicalSessionSeats()...)
@@ -678,13 +681,13 @@ func serviceManagerVerbs() []Verb {
Replaces: []string{"systemctl --failed"}},
// **Clearing a failed record, and finding what starts a unit** (novox/hq issue 332): a unit whose
// file is gone stays failed in its manager until the record is reset, and what keeps asking for it
// is a dependency or an enable link no verb could show. Both optional while their holders catch up
// (ADR 0246 step 1); a later change requires them.
{Name: "reset-failed", Optional: true, Description: "Clear one unit's failed record in its manager — " +
// is a dependency or an enable link no verb could show. Required since every holder serves them
// (ADR 0246 step 3).
{Name: "reset-failed", Description: "Clear one unit's failed record in its manager — " +
"its failed state and its start-limit count — and answer its state after. It removes no file and " +
"starts nothing.",
Input: scoped(unit, []string{"unit"}), Replaces: []string{"systemctl reset-failed"}},
{Name: "wanted-by", Optional: true, Description: "What wants, requires or triggers one unit, read-only: the " +
{Name: "wanted-by", Description: "What wants, requires or triggers one unit, read-only: the " +
"manager's reverse dependencies (WantedBy, RequiredBy, UpheldBy, BoundBy, TriggeredBy and the rest) and " +
"every enable link naming it in the scope's configuration directories (*.wants, *.requires, *.upholds), " +
"each with where it points and whether it dangles — the account's own directory included in user scope.",
@@ -692,9 +695,9 @@ func serviceManagerVerbs() []Verb {
Replaces: []string{"systemctl list-dependencies --reverse", "systemctl show -p WantedBy",
"ls ~/.config/systemd/user/*.wants"}},
// **A dangling enable link removed on the person's word** (novox/hq issue 332): `systemctl disable`
// leaves a link whose unit file is gone, so the manager keeps asking for the unit. Optional until its
// holders serve it (ADR 0246 step 1).
{Name: "unlink-dangling", Optional: true, Description: "Remove the dangling enable links named for one " +
// leaves a link whose unit file is gone, so the manager keeps asking for the unit. Required since
// every holder serves it (ADR 0246 step 3).
{Name: "unlink-dangling", Description: "Remove the dangling enable links named for one " +
"unit — links in a *.wants, *.requires or *.upholds directory of the account's or the machine's own " +
"configuration whose target does not exist, which `disable` leaves once the unit's file is gone — on " +
"the person's word: `why` is required. Each is recorded (its path, its target, the reason) before it is " +
+3 -3
View File
@@ -48,7 +48,7 @@ func TestTheSeatsAreAClosedSetAndEachNamesItsDecision(t *testing.T) {
delivered[s.Delivers] = s.Name
}
}
// Forty with node-nfs-server and node-mounts (novox/hq ADR 0263); thirty-eight with node-resolver (novox/hq ADR 0247); thirty-seven with mesh-delivery (novox/hq ADR 0239); thirty-six since node-resolver-config retired
// Forty-one with node-reads (novox/hq ADR 0310); forty with node-nfs-server and node-mounts (novox/hq ADR 0263); thirty-eight with node-resolver (novox/hq ADR 0247); thirty-seven with mesh-delivery (novox/hq ADR 0239); thirty-six since node-resolver-config retired
// into node-uplink (novox/hq ADR 0223); thirty-seven
// since the retired node-dns-resolver went (novox/hq ADR 0220); thirty-eight with
// node-backup (novox/hq ADR 0214); thirty-seven with node-message-bus (novox/hq ADR 0215);
@@ -59,8 +59,8 @@ func TestTheSeatsAreAClosedSetAndEachNamesItsDecision(t *testing.T) {
// node-container-runtime (ADR 0207); nineteen with node-environment and node-login-shell (ADR 0203,
// ADR 0204); seventeen with node-build-agent (ADR 0190). One fewer once the retired
// mesh-build-machine row goes, when no registered manifest claims it.
if len(Seats()) != 40 {
t.Errorf("the mesh defines %d seats rather than 40; the set is closed, so a change here is "+
if len(Seats()) != 41 {
t.Errorf("the mesh defines %d seats rather than 41; the set is closed, so a change here is "+
"a decision (novox/hq ADR 0110): %s", len(Seats()), seatNames())
}
}
+25 -2
View File
@@ -156,7 +156,11 @@ func secretFilesOf(resources []map[string]any) map[string]bool {
// A name the module never declared is refused here rather than on the machine. The host would
// refuse it too — but it would do so having already been handed a declaration, which reads as the
// mesh sending something broken, and the name it could not find is a manifest's typo.
func intoFile(resource map[string]any, sealed map[string]string, module string) error {
//
// **An own secret issued outside the mesh that nobody gave is filled with nothing** (novox/hq issue 401): the mesh
// never makes one (ADR 0228), and a file is where the module reads it, so the place says empty — the module sees it
// was not given, where a value the mesh made looked given and wrong. notGiven names those secrets.
func intoFile(resource map[string]any, sealed map[string]string, notGiven map[string]bool, module string) error {
if fmt.Sprint(resource["type"]) != "file" {
return nil
}
@@ -171,6 +175,10 @@ func intoFile(resource map[string]any, sealed map[string]string, module string)
into := map[string]any{}
for _, name := range used {
value := sealed[name]
if value == "" && notGiven[name] {
content = strings.ReplaceAll(content, "${secret:"+name+"}", "")
continue
}
if value == "" {
return fmt.Errorf(
"%s has a file that says ${secret:%s}, and %s has no secret of its own by that "+
@@ -179,10 +187,25 @@ func intoFile(resource map[string]any, sealed map[string]string, module string)
}
into[name] = value
}
resource["secrets"] = into
resource["content"] = content
if len(into) > 0 {
resource["secrets"] = into
}
return nil
}
// outsideNotGiven is each own secret of a module issued outside the mesh that this composition holds no value for
// (novox/hq issue 401): what intoFile fills with nothing, and what nothing restarts on.
func outsideNotGiven(m Manifest, given map[string]string) map[string]bool {
out := map[string]bool{}
for name, own := range m.OwnSecrets.Plain() {
if own.IssuedBy == IssuedOutside && given[name] == "" {
out[name] = true
}
}
return out
}
// namesOr says what a module could have written, because the answer to "that name is wrong" is
// almost always one of two or three right ones.
func namesOr(sealed map[string]string) string {
+181
View File
@@ -0,0 +1,181 @@
package catalogue
import (
"strings"
"testing"
)
// A declared directory may be owned by the account its module runs as (novox/hq issue 279): a media
// manager runs as the puid and pgid its assignment sets per node, and a folder it must write was owned
// by the manifest's literal, so the app refused it on a machine whose identity was not the image's.
func recycler() Manifest {
return Manifest{Module: "sonarr",
Resources: []map[string]any{
{"id": "recycle-bin", "type": "directory", "path": "/srv/sonarr/recycle", "mode": "0755",
"owner": "${setting:puid}:${setting:pgid}"},
{"id": "identity", "type": "file", "path": "/srv/sonarr/identity.env", "mode": "0644",
"content": "PUID=${setting:puid}\nPGID=${setting:pgid}\n"},
},
}
}
func copied(r map[string]any) map[string]any {
out := map[string]any{}
for k, v := range r {
out[k] = v
}
return out
}
// The node's identity reaches the directory's owner, the node's layer over the mesh's, a number given
// as a string or as a JSON number alike.
func TestASettingOwnsADirectory(t *testing.T) {
m := recycler()
layers := []Layer{
{From: "the mesh", Values: map[string]any{"puid": "1000", "pgid": "1000"}},
{From: "this node", Values: map[string]any{"puid": float64(1001), "pgid": "2000"}},
}
r := copied(m.Resources[0])
if err := settingInto(r, layers, m.Module); err != nil {
t.Fatal(err)
}
if r["owner"] != "1001:2000" {
t.Fatalf("owned by %q, not 1001:2000", r["owner"])
}
user := copied(m.Resources[0])
user["owner"] = "${setting:puid}"
if err := settingInto(user, layers, m.Module); err != nil || user["owner"] != "1001" {
t.Fatalf("user alone: %v, %v", user["owner"], err)
}
}
// Composed for a machine, the way a push writes it: the directory reaches the node-engine owned by the
// number the node's settings name, never by the placeholder.
func TestAComposedMachineGetsTheOwnerTheSettingNames(t *testing.T) {
r := anAdoptedAnchor()
r.Modules = append(r.Modules, recycler())
with := anchorRendering(false)
with.Settings["sonarr"] = []Layer{{From: "anchor", Values: map[string]any{"puid": "1001", "pgid": "2000"}}}
composed, err := r.Compose(with)
if err != nil {
t.Fatal(err)
}
if why, left := composed.LeftOut["sonarr"]; left {
t.Fatalf("left out: %s", why)
}
got := byID(composed.Resources)["sonarr.recycle-bin"]
if got["owner"] != "1001:2000" {
t.Fatalf("composed owned by %v, not 1001:2000", got["owner"])
}
}
// Only a whole number owns a directory: an account's name, text, a sign, a fraction or a value past the
// kernel's ids is refused by name, and the directory is left as it was. So is a key nothing sets.
func TestAnOwnerSettingThatIsNoNumberIsRefused(t *testing.T) {
m := recycler()
for _, bad := range []any{"abc", "root", "", "-1", "+5", "1000:1000", "10 00", "1000\nx", float64(1.5),
float64(-3), "4294967295", "99999999999", true} {
r := copied(m.Resources[0])
err := settingInto(r, []Layer{{From: "ace", Values: map[string]any{"puid": bad, "pgid": "1000"}}}, m.Module)
if err == nil || !strings.Contains(err.Error(), `"puid"`) || !strings.Contains(err.Error(), "whole number") {
t.Errorf("%v: %v", bad, err)
}
if r["owner"] != m.Resources[0]["owner"] {
t.Errorf("%v: the owner was changed on refusal: %v", bad, r["owner"])
}
}
r := copied(m.Resources[0])
if err := settingInto(r, nil, m.Module); err == nil || !strings.Contains(err.Error(), "${setting:puid}") {
t.Errorf("nothing set: %v", err)
}
if err := JudgeSettings(m, []Layer{{From: "ace", Values: map[string]any{"puid": "media", "pgid": "1000"}}}, false); err == nil ||
!strings.Contains(err.Error(), "whole number") {
t.Errorf("a stored name is not refused when judged: %v", err)
}
}
// Where the manifest is read: an owner holding a setting is a whole placeholder or a whole number on
// each side of the colon, and every key is one the module asks for elsewhere or declares a default for.
func TestAnOwnerSettingIsJudgedWhereTheManifestIsRead(t *testing.T) {
ok := []string{"${setting:puid}:${setting:pgid}", "${setting:puid}", "${setting:puid}:1000", "1000:${setting:pgid}",
"1000:1000", "media:media"}
bad := []string{
"${setting:puid}x",
"x${setting:puid}",
"${setting:puid}:${setting:pgid}:1",
"${setting:puid}${setting:pgid}",
"media:${setting:pgid}",
"${setting:puid}:",
"${setting:other}:${setting:pgid}",
}
for _, owner := range append(ok, bad...) {
m := recycler()
m.Resources[0] = copied(m.Resources[0])
m.Resources[0]["owner"] = owner
err := parsed(t, m)
refused := err != nil && strings.Contains(err.Error(), "owner")
want := false
for _, b := range bad {
want = want || b == owner
}
if refused != want {
t.Errorf("%s: refused %v, want %v (%v)", owner, refused, want, err)
}
}
// A key with a default is declared, though nothing else asks for it.
m := recycler()
m.Resources = m.Resources[:1]
m.Resources[0] = copied(m.Resources[0])
m.Resources[0]["owner"] = "${setting:workers}"
m.Settings = map[string]SettingDeclaration{"workers": {Kind: KindPreference, Default: float64(4), Why: "a test"}}
if err := parsed(t, m); err != nil && strings.Contains(err.Error(), "owner") {
t.Errorf("a declared key was refused: %v", err)
}
// A file's owner takes no setting: the sweep says so.
f := recycler()
f.Resources[1] = copied(f.Resources[1])
f.Resources[1]["owner"] = "${setting:puid}"
if err := parsed(t, f); err == nil || !strings.Contains(err.Error(), "not filled in this field") {
t.Errorf("a file's owner setting passed: %v", err)
}
}
// A key a directory's owner asks for is a destination, so setting it is not called stray.
func TestASettingAnOwnerAsksForIsNotStray(t *testing.T) {
m := recycler()
m.Resources = m.Resources[:1]
stray := strings.Join(UnusedSettings(m, []Layer{{From: "ace", Values: map[string]any{"puid": "1001", "pgid": "2000"}}}), "; ")
if strings.Contains(stray, "puid") || strings.Contains(stray, "pgid") {
t.Errorf("called stray: %s", stray)
}
}
// An owner is written as the number parsed: "0001" and "00" reach the node-engine as 1 and 0, never as text.
// Uid 0 is allowed, because a key an owner asks for is the terminal's alone.
func TestAnOwnerSettingIsWrittenAsTheNumber(t *testing.T) {
m := recycler()
r := copied(m.Resources[0])
if err := settingInto(r, []Layer{{From: "ace", Values: map[string]any{"puid": "0001", "pgid": "00"}}}, m.Module); err != nil {
t.Fatal(err)
}
if r["owner"] != "1:0" {
t.Fatalf("owned by %q, not 1:0", r["owner"])
}
}
// Every key a directory's owner asks for is the controller's terminal's alone (novox/hq issue 279, issue 339):
// root chowns the folder to it, so it is terminal-only whatever else asks for it — a file nothing trusts, a
// default, or nothing at all.
func TestAnOwnersKeyIsTheTerminals(t *testing.T) {
m := recycler()
m.Resources[1] = copied(m.Resources[1])
m.Resources[1]["trusted"] = false
m.Settings = map[string]SettingDeclaration{"pgid": {Kind: KindPreference, Default: "1000", Why: "a test"}}
keys := strings.Join(TerminalKeys(m), ",")
for _, k := range []string{"puid", "pgid"} {
if !strings.Contains(","+keys+",", ","+k+",") {
t.Errorf("%s is not the terminal's: %s", k, keys)
}
}
}
+125 -3
View File
@@ -4,6 +4,7 @@ import (
"fmt"
"regexp"
"sort"
"strconv"
"strings"
)
@@ -15,7 +16,8 @@ import (
// (novox/hq issues 122, 134). ADR 0112 names the operator as one of the four providers; this is the
// operator answering.
//
// `${setting:<key>}` in a file's content, and in a service's unit name, is filled from the module's settings layers — the mesh's,
// `${setting:<key>}` in a file's content, in a service's unit name and in a directory's owner (a uid or
// gid only, issue 279) is filled from the module's settings layers — the mesh's,
// then this node's — the same layers a mergeable JSON file and a contribution already take, so
// `settings set <module>` is the one place a person's values go. **Refused when no layer sets it**,
// naming the key and the remedy: a definition that carried a default for a mail domain would be
@@ -59,6 +61,9 @@ func settingInto(resource map[string]any, layers []Layer, module string) error {
if fmt.Sprint(resource["type"]) == "service" {
return settingIntoUnit(resource, layers, module)
}
if fmt.Sprint(resource["type"]) == "directory" {
return settingIntoOwner(resource, layers, module)
}
if fmt.Sprint(resource["type"]) != "file" {
return nil
}
@@ -143,6 +148,107 @@ func settingIntoUnit(resource map[string]any, layers []Layer, module string) err
return nil
}
// A declared directory may be owned by the account its module runs as (novox/hq issue 279).
//
// A containerised module runs as the identity its assignment sets per node — its puid and pgid
// settings, written into the environment file its image reads (ADR 0112, issue 153) — and a directory
// its app must write, beyond the one its image re-owns at start, was owned by whatever the manifest
// spelt: right on a machine whose identity matched the image's default, refused by the app on one whose
// did not. So a directory's `owner` may name the settings that identity is: `${setting:<key>}` for the
// user alone, `${setting:<user>}:${setting:<group>}` for both, filled from the same layers at
// composition.
//
// **Whole numbers only.** An owner a setting fills is a uid or a gid — never an account's name, never
// text — so no value can put anything into the ownership the node-engine applies but a number; one that
// is not is refused at composition, naming the key, and the directory is left as it was. The shape is
// judged where the manifest is read (OwnerSettingProblems): each side of the colon a whole placeholder
// or a whole number, and every key one the module itself asks for elsewhere — in a file, a unit, a
// contribution or a served fact — or declares a default for, so the owner follows the identity its
// process is told it runs as, never a key that reaches nothing else.
// ownerSide is one side of an owner a setting may fill: a whole placeholder or a whole number.
var ownerSide = `(?:\$\{setting:[a-z0-9][a-z0-9_.-]*\}|[0-9]+)`
// ownerWithSetting is the only shape an owner holding a setting may have: user, or user:group.
var ownerWithSetting = regexp.MustCompile(`^` + ownerSide + `(?::` + ownerSide + `)?$`)
// wholeID is a value as a uid or a gid, and whether it is one: a whole number below the kernel's "no id"
// (2^32-1). Uid 0 is one: a key an owner asks for is the controller's terminal's alone (TerminalKeys).
func wholeID(v string) (uint64, bool) {
if v == "" || strings.TrimLeft(v, "0123456789") != "" {
return 0, false
}
n, err := strconv.ParseUint(v, 10, 32)
return n, err == nil && n < 1<<32-1
}
// OwnerSettingProblems refuses, where a manifest is read, a directory whose owner carries a setting
// in any shape but a whole placeholder or number on each side of the colon, or names a key the module
// asks for nowhere else and declares no default for.
func OwnerSettingProblems(m Manifest) []string {
var problems []string
asked := settingKeysAskedOutsideOwners(m)
for k := range m.Settings {
asked[k] = true
}
for _, r := range m.Resources {
owner, _ := r["owner"].(string)
if !strings.Contains(owner, "${setting:") {
continue
}
if fmt.Sprint(r["type"]) != "directory" {
// Said by the placeholder sweep: only a directory's owner takes a setting.
continue
}
if !ownerWithSetting.MatchString(owner) {
problems = append(problems, fmt.Sprintf("%s: the directory %v's owner %q carries a setting in a shape "+
"the mesh does not fill; an owner names the account by number only, each side of the colon a whole "+
"${setting:key} or a whole number (${setting:puid}:${setting:pgid}) (novox/hq issue 279)",
m.Module, r["id"], owner))
continue
}
for _, key := range settingsUsed(owner) {
if !asked[key] {
problems = append(problems, fmt.Sprintf("%s: the directory %v's owner says ${setting:%s}, and the "+
"module asks for %q nowhere else and declares no default for it: an owner follows the identity "+
"the module's process is told it runs as (its identity file's ${setting:%s}), never a key of its "+
"own (novox/hq issue 279)", m.Module, r["id"], key, key, key))
}
}
}
return problems
}
// settingIntoOwner fills ${setting:…} in a directory's owner, refusing — with the key and why, the
// resource left as it was — a key nothing sets and a value that is not a whole number.
func settingIntoOwner(resource map[string]any, layers []Layer, module string) error {
owner, ok := resource["owner"].(string)
if !ok {
return nil
}
for _, key := range settingsUsed(owner) {
value, set := settingValue(layers, key)
if !set {
return &UnsetSettingError{Module: module, Setting: key, said: fmt.Sprintf(
"%s has a directory %v whose owner says ${setting:%s}, and nothing sets %q for it — an operator's "+
"value is the assignment's, never the definition's (novox/hq ADR 0112): "+
"`settings set %s <file>` with {%q: …}%s",
module, resource["id"], key, key, module, key, orNoSettings(layers))}
}
v := plainly(value)
id, whole := wholeID(v)
if !whole {
return fmt.Errorf("%s: the setting %q is %q, which cannot own the directory %v: an owner a setting "+
"names is a uid or a gid, a whole number, never an account's name or other text (novox/hq issue 279)",
module, key, v, resource["id"])
}
// Written as the number parsed, so "0001" reaches the node-engine as 1, never as text.
owner = strings.ReplaceAll(owner, "${setting:"+key+"}", strconv.FormatUint(id, 10))
}
resource["owner"] = owner
return nil
}
func settingValue(layers []Layer, key string) (any, bool) {
var value any
set := false
@@ -171,9 +277,25 @@ func orNoSettings(layers []Layer) string {
return "; set today: " + strings.Join(keys, ", ")
}
// settingKeysUsedBy is every key a module's files, contributions and served facts ask for, so a
// setting that lands in one is not called stray.
// settingKeysUsedBy is every key a module's files, units, directory owners, contributions and served
// facts ask for, so a setting that lands in one is not called stray.
func settingKeysUsedBy(m Manifest) map[string]bool {
used := settingKeysAskedOutsideOwners(m)
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) == "directory" {
if owner, ok := r["owner"].(string); ok {
for _, k := range settingsUsed(owner) {
used[k] = true
}
}
}
}
return used
}
// settingKeysAskedOutsideOwners is every key a module's files, units, contributions and served facts
// ask for: what a directory's owner may follow (OwnerSettingProblems).
func settingKeysAskedOutsideOwners(m Manifest) map[string]bool {
used := map[string]bool{}
note := func(s string) {
for _, k := range settingsUsed(s) {
+12
View File
@@ -509,6 +509,18 @@ var networkName = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9_.-]*$`)
// refused where it is stored (SetSettings, with UnusedSettings) and said where a plan is read,
// and never costs a module its place.
func JudgeSettings(m Manifest, layers []Layer, adopted bool) error {
if err := UndeclaredAccessPlaces(m, layers); err != nil {
return err
}
return ComposesWith(m, layers, adopted)
}
// ComposesWith is JudgeSettings for a layer stored ahead of the definition that reads it (novox/hq issue 575): an
// access placed that this definition does not declare yet is not refused, because composition passes it by — it
// reaches nothing until the build that declares it registers, and then it places that access. Where a machine is
// declared (LeftOutBecause), and where a setting judged against a newer build of the module is held to still compose
// with the one registered.
func ComposesWith(m Manifest, layers []Layer, adopted bool) error {
if err := settingsHoldOneLine(m.Module, layers); err != nil {
return err
}
+12 -2
View File
@@ -33,6 +33,9 @@ import (
// 4. **A module's whole layer, when it has a mergeable file not marked `"trusted": false`** (novox/hq issue 340,
// TrustedMergeable). A mergeable file takes any key a layer sets, not only those its content names, so no list
// of keys covers it: an empty runtime configuration a provider reads would take a `url` of the caller's.
// 5. **Every setting a directory's owner asks for** (novox/hq issue 279): root chowns the directory to the number
// at the next push, so through a verb any caller could hand itself — or uid 0 — a module's folder. Whatever
// else asks for the key, a file marked `"trusted": false` included, the owner makes it the terminal's.
//
// Derived from the manifest, never listed by hand, so a provider or a trusted file added tomorrow is covered.
@@ -42,8 +45,8 @@ import (
const TrustedField = "trusted"
// TerminalKeys are the settings keys of a module that are set at the controller's terminal alone: places and
// accesses, every key its provisions serve and every setting a served value asks for, and every setting a file
// asks for unless it says `"trusted": false`. Places and accesses first, then the rest sorted.
// accesses, every key its provisions serve and every setting a served value asks for, every setting a file
// asks for unless it says `"trusted": false`, and every setting a directory's owner asks for. Places and accesses first, then the rest sorted.
func TerminalKeys(m Manifest) []string {
keys := map[string]bool{}
for _, served := range m.Serves {
@@ -57,6 +60,13 @@ func TerminalKeys(m Manifest) []string {
}
}
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) == "directory" {
owner, _ := r["owner"].(string)
for _, asked := range settingsUsed(owner) {
keys[asked] = true
}
continue
}
if fmt.Sprint(r["type"]) != "file" {
continue
}
+1 -1
View File
@@ -56,7 +56,7 @@ type filler struct {
// seatInto, machineInto and contributionsInto. A pass that comes to read another field adds it here,
// or the sweep refuses the placeholder it would have filled.
var fillers = []filler{
{namespace: "setting", patterns: []*regexp.Regexp{settingRef}, fields: map[string][]string{"file": {"content"}, "service": {"unit"}}},
{namespace: "setting", patterns: []*regexp.Regexp{settingRef}, fields: map[string][]string{"file": {"content"}, "service": {"unit"}, "directory": {"owner"}}},
{namespace: "dir", patterns: []*regexp.Regexp{dirRef}, fields: map[string][]string{"*": {"path", "content", "volumes", "env", "env-file"}}},
{namespace: "access", patterns: []*regexp.Regexp{accessRef}, fields: map[string][]string{"*": {"path", "content", "volumes", "env", "env-file"}}},
{namespace: "secret", patterns: []*regexp.Regexp{placeholder}, fields: map[string][]string{"file": {"content"}},
+101
View File
@@ -0,0 +1,101 @@
package catalogue
import (
"strings"
"testing"
)
// An archive pinned by URL and sha256 (novox/hq ADR 0316 §2, §6): the vendor's file, mirrored by the build
// seat, never a copy in the repository.
const pinnedSum = "38dd24890a66b4cf9a45bc24e15192e00c26b929e1bb8df7974c16a4539129e2"
func upstreamManifest(artifact string) string {
return `{"module":"a","version":"1","build":{"artifacts":[` + artifact + `]}}`
}
func TestAnArchivePinnedByURLAndSha256IsAccepted(t *testing.T) {
for _, a := range []string{
`{"name":"archive","kind":"archive","url":"https://vendor.example.org/release-1.tar.xz","sha256":"` + pinnedSum +
`","version":"1.0","within":"release"}`,
`{"name":"addon","kind":"archive","url":"https://vendor.example.org/addon.xpi","sha256":"` + pinnedSum +
`","version":"1.5.6.1","file":"addon.xpi"}`,
} {
if _, err := ParseManifest([]byte(upstreamManifest(a))); err != nil {
t.Errorf("%s was refused: %v", a, err)
}
}
}
func TestAnUpstreamFileThatIsNotPinnedIsRefused(t *testing.T) {
cases := map[string]string{
// No digest: what the address serves changes under the mesh.
`{"name":"x","kind":"archive","url":"https://vendor.invalid/r.tar.xz","version":"1"}`: "without a pinned sha256",
`{"name":"x","kind":"archive","url":"https://vendor.invalid/r.tar.xz","sha256":"ABC","version":"1"}`: "without a pinned sha256",
// No version: a person reading the pin cannot tell which release.
`{"name":"x","kind":"archive","url":"https://vendor.invalid/r.tar.xz","sha256":"` + pinnedSum + `"}`: "says no version",
// Not https.
`{"name":"x","kind":"archive","url":"http://vendor.invalid/r.tar.xz","sha256":"` + pinnedSum + `","version":"1"}`: "over https",
// Both a directory of the repository and a URL.
`{"name":"x","kind":"archive","from":"files","url":"https://vendor.invalid/r.tar.xz","sha256":"` + pinnedSum +
`","version":"1"}`: "one or the other",
// A pin without a URL.
`{"name":"x","kind":"archive","from":"files","sha256":"` + pinnedSum + `"}`: "no url",
// Only an archive is mirrored from a file.
`{"name":"x","kind":"image","from":"Dockerfile","url":"https://vendor.invalid/r.tar.xz","sha256":"` + pinnedSum +
`","version":"1"}`: "only an archive",
// A file name that is a path, a directory within that leaves the archive, and both at once.
`{"name":"x","kind":"archive","url":"https://vendor.invalid/a.xpi","sha256":"` + pinnedSum +
`","version":"1","file":"../a.xpi"}`: "not a plain relative file name",
`{"name":"x","kind":"archive","url":"https://vendor.invalid/r.tar.xz","sha256":"` + pinnedSum +
`","version":"1","within":"../up"}`: "not a plain relative directory",
`{"name":"x","kind":"archive","url":"https://vendor.invalid/r.tar.xz","sha256":"` + pinnedSum +
`","version":"1","within":"r","file":"r.tar.xz"}`: "a file is kept whole",
}
for artifact, want := range cases {
_, err := ParseManifest([]byte(upstreamManifest(artifact)))
if err == nil {
t.Errorf("%s was accepted", artifact)
continue
}
if !strings.Contains(err.Error(), want) {
t.Errorf("%s was refused for another reason than %q: %v", artifact, want, err)
}
}
}
func TestAnUpstreamFilesVendorIsNamedOnPurposeAndAnInstallationsIsRefused(t *testing.T) {
// The vendor's host is judged like any name, and passes through names-on-purpose; a version of four dotted
// numbers is not an address, and a name in it is still judged.
m, err := ParseManifest([]byte(upstreamManifest(`{"name":"owl","kind":"archive","url":"https://vendor.be/owl/owl.xpi","sha256":"` +
pinnedSum + `","version":"1.5.6.1","file":"owl.xpi","names-on-purpose":{"vendor.be":"Owl's vendor publishes it there"}}`)))
if err != nil {
t.Fatal(err)
}
if problems := InstallationProblems(m); len(problems) != 0 {
t.Fatalf("the declared vendor and the version were taken for an installation: %v", problems)
}
m.Build.Artifacts[0].NamesOnPurpose = nil
if problems := InstallationProblems(m); len(problems) != 1 || !strings.Contains(problems[0], "vendor.be") {
t.Fatalf("an undeclared vendor host is judged: %v", problems)
}
m.Build.Artifacts[0].Version = "git.novox.be 10.0.0.1"
if problems := InstallationProblems(m); len(problems) != 2 {
t.Fatalf("a name in the version is judged: %v", problems)
}
m.Build.Artifacts[0].URL, m.Build.Artifacts[0].Version = "https://mail.novox.be/owl.xpi", "1"
if problems := InstallationProblems(m); len(problems) != 1 || !strings.Contains(problems[0], "mail.novox.be") {
t.Fatalf("an installation's host in the url is judged: %v", problems)
}
}
func TestAnUpstreamFileFromAnAddressOrAReservedNameIsRefused(t *testing.T) {
for _, u := range []string{"https://git.novox.internal/r.tar.xz", "https://10.1.2.3/r.tar.xz",
"https://203.0.113.9/r.tar.xz", "https://[2001:db8::1]/r.tar.xz", "https://nas.lan/r.tar.xz", "https://vendor/r.tar.xz"} {
_, err := ParseManifest([]byte(upstreamManifest(`{"name":"x","kind":"archive","url":"` + u + `","sha256":"` +
pinnedSum + `","version":"1"}`)))
if err == nil || !strings.Contains(err.Error(), "published under a public name") {
t.Errorf("%s: %v", u, err)
}
}
}
+10 -1
View File
@@ -333,7 +333,9 @@ var ControllerVerbs = []Verb{
// The build queue, controlled by hand (novox/hq ADR 0219). Every verb that drops an ask leaves a
// failed outcome for it, so a plan waiting on it fails visibly instead of hanging.
{Name: "queue", Description: "Every ask in the build queue: waiting, in flight (on which machine, for how long), " +
"and dead (handed out as often as allowed and never settled) — each with its id, repository, path, ref and when it was asked.",
"and dead (handed out as often as allowed and never settled) — each with its id, repository, path, ref and when it was asked. " +
"Answers the table as text and, as data, each ask's state, kind (check or build), repository, commit, machine, since " +
"and position among the waiting (novox/hq issue 455).",
Input: schema(nil, nil)},
{Name: "cancel", Description: "Drop one waiting or dead ask from the build queue; its outcome is recorded failed, " +
"cancelled by hand, and a plan that asked for it fails. One in flight is refused: `kill` ends it where it runs.",
@@ -490,6 +492,13 @@ var ControllerVerbs = []Verb{
"why": "with consumer or older-than: why — required, and recorded in the hand-act log",
"cause": "with consumer or older-than: the cause in a word (cleanup-waiting when absent)",
}, nil, "confirm")},
{Name: "judge-directories", Description: "Read-only: which of the directories one machine is declared its " +
"node-engine's walk that follows no link another account could make would refuse (novox/hq issue 358), one line " +
"each: placed, would be refused and why (the link and who could replace it), refused already by the placement " +
"guard, or below a home. The engine judges the declaration it keeps and makes, chmods and chowns nothing.",
Input: schema(map[string]string{
"node": "the machine's name",
}, []string{"node"})},
// What a consumer gave up on (novox/hq issue 330): kept in DEAD_LETTERS until a person acts on it.
{Name: "root-free", Description: "Whether each machine named is root-free now (novox/hq ADR 0259 §8): no agent " +
"there can become root without a person. Judged when asked, never from a condition: free only when the machine " +
+67 -1
View File
@@ -958,7 +958,39 @@ func (i *Inventory) judgeSettings(ctx context.Context, nodeName, module string,
}
layers = append(layers, catalogue.Layer{From: from, Values: values})
if err := catalogue.JudgeSettings(m, layers, adopted); err != nil {
return fmt.Errorf("refused: %s on %s cannot compose with the layer %q — %w", module, where, from, err)
// **A placement for an access the module's newer build brings** (novox/hq issue 575): judged against the
// manifest of the module's newest build on the branch it follows, where the registered one does not declare
// the access yet — so an access can be placed before or with the delivery that adds it. It must still compose
// with the registered manifest, which passes the placement by until the build registers.
var undeclared *catalogue.UndeclaredAccessError
if !errors.As(err, &undeclared) {
return fmt.Errorf("refused: %s on %s cannot compose with the layer %q — %w", module, where, from, err)
}
ahead, has, aerr := i.NewestManifestOnBranch(ctx, module)
if aerr != nil {
return aerr
}
if !has {
follows, ferr := i.followsABranch(ctx, module)
if ferr != nil {
return ferr
}
if !follows {
return fmt.Errorf("refused: %s on %s cannot compose with the layer %q — %w; and %s follows no branch "+
"(it was registered by hand), so no build of it can be judged ahead of what is registered", module,
where, from, err, module)
}
}
if !has || catalogue.JudgeSettings(ahead, layers, adopted) != nil || catalogue.ComposesWith(m, layers, adopted) != nil {
return fmt.Errorf("refused: %s on %s cannot compose with the layer %q — %w; nor does the newest build of "+
"%s on the branch it follows declare it", module, where, from, err, module)
}
m = ahead
}
// A module that runs as its own identity owns its directories by it: no setting names their owner (novox/hq ADR
// 0325 §1, §3). A module that does not yet keeps its placed owner, warned of where it is set.
if err := catalogue.PlacesOwnerProblem(m, layers); err != nil {
return fmt.Errorf("refused: %w", err)
}
// And a key that reaches nothing, refused here where somebody can still fix the spelling:
// stored, it would be a setting somebody believes they made.
@@ -969,6 +1001,40 @@ func (i *Inventory) judgeSettings(ctx context.Context, nodeName, module string,
return nil
}
// followsABranch says a module's record names the branch it follows.
func (i *Inventory) followsABranch(ctx context.Context, module string) (bool, error) {
var ref string
err := i.store.Pool().QueryRow(ctx, `select coalesce(ref, '') from module where name = $1`, module).Scan(&ref)
if errors.Is(err, pgx.ErrNoRows) {
return false, nil
}
return ref != "", err
}
// NewestManifestOnBranch is the manifest of a module's newest successful build on the branch its record follows
// (novox/hq issue 575): what the delivery under way brings, before it registers — or the registered one's own, when
// nothing newer was built. Only a build of that very branch: never one of another branch or of none. False when no
// such build is kept, and for a module whose record follows no branch (registered by hand).
func (i *Inventory) NewestManifestOnBranch(ctx context.Context, module string) (catalogue.Manifest, bool, error) {
var raw []byte
err := i.store.Pool().QueryRow(ctx,
`select b.manifest from build b join module m on m.name = b.module
where b.module = $1 and b.failed = '' and b.manifest is not null and b.manifest::text <> 'null'
and coalesce(m.ref, '') <> '' and b.ref = m.ref
order by b.at desc limit 1`, module).Scan(&raw)
if errors.Is(err, pgx.ErrNoRows) {
return catalogue.Manifest{}, false, nil
}
if err != nil {
return catalogue.Manifest{}, false, err
}
m, err := catalogue.ParseManifest(raw)
if err != nil {
return catalogue.Manifest{}, false, nil // a build the mesh cannot read judges nothing
}
return m, true, nil
}
// givenIn is the machine ports a node-level settings layer gives a module, software port →
// machine port (novox/hq ADR 0100). Nothing when the layer gives none; what is not a port is left
// for composition to refuse in its own words.
+25 -3
View File
@@ -36,15 +36,37 @@ type ResourceHealth struct {
Root string `json:"root,omitempty"`
// Waits is what a waiting resource waits for the operator to give (novox/hq ADR 0283).
Waits []Wait `json:"waits,omitempty"`
// Entries is a kept folder's listing as the engine resolved it (novox/hq ADR 0325 §4): each user with its uid.
Entries []KeptEntry `json:"entries,omitempty"`
}
// Wait is one thing a waiting resource waits for the operator to give: exactly one of Secret and Setting
// (novox/hq ADR 0283), as link.Wait carries it.
// KeptEntry is one user of a kept folder as the engine lists it, with a note beside the tool runner's.
type KeptEntry struct {
Identity string `json:"identity"`
UID int `json:"uid"`
Use string `json:"use"`
From string `json:"from"`
Note string `json:"note,omitempty"`
}
// Wait is one thing a waiting resource waits for the operator to give: exactly one of Secret, Setting and Access
// (novox/hq ADR 0283, ADR 0325 §7), as link.Wait carries it.
type Wait struct {
Part string `json:"part"`
Secret string `json:"secret,omitempty"`
Setting string `json:"setting,omitempty"`
What string `json:"what"`
// Access is an access of the module by its own id for it, from the engine's access-waits, and Disk that it
// waits for the disk that holds its folder rather than for the operator's word (novox/hq ADR 0325 §7, §9).
Access string `json:"access,omitempty"`
Disk bool `json:"disk,omitempty"`
// Placed is the filesystem the folder is on now, without its path (ADR 0325 §8): a warrant for another does not
// answer the wait.
Placed string `json:"placed,omitempty"`
// Reason is the engine's reason for an access wait ("keep" or "disk"), ACL what its folder's filesystem says of
// access entries ("needs-enabling", "unsupported", or empty when it allows them).
Reason string `json:"reason,omitempty"`
ACL string `json:"acl,omitempty"`
What string `json:"what"`
}
// NodeHealth is a machine's newest statement, as kept.
+127
View File
@@ -0,0 +1,127 @@
package inventory
import (
"context"
"errors"
"fmt"
"path/filepath"
"strings"
"github.com/jackc/pgx/v5"
"github.com/novox/mesh-controller/internal/catalogue"
)
// Module identities and taken folders (novox/hq ADR 0325 §3, §8, §11; migration 0096).
// AllocateIdentity is module's identity, allocated now when it has none: one number from the range of its own,
// mesh-wide, kept for ever. Two controllers allocating at once agree: the second's insert finds the first's row.
func (i *Inventory) AllocateIdentity(ctx context.Context, module string) (catalogue.ModuleIdentity, error) {
if strings.TrimSpace(module) == "" {
return catalogue.ModuleIdentity{}, errors.New("an identity is allocated for a module, and none was named")
}
if id, had, err := i.IdentityOf(ctx, module); err != nil || had {
return id, err
}
// The number is drawn only when the row is new; a draw lost to a concurrent insert is a gap in the range, never
// a number given twice.
_, err := i.store.Pool().Exec(ctx,
`insert into module_identity (module, uid, gid)
select $1, n, n from (select nextval('module_identity_number')::integer as n) drawn
on conflict (module) do nothing`, module)
if err != nil {
return catalogue.ModuleIdentity{}, fmt.Errorf("%s could not be given an identity: %w", module, err)
}
id, had, err := i.IdentityOf(ctx, module)
if err == nil && !had {
err = fmt.Errorf("%s was given an identity and it cannot be read back", module)
}
return id, err
}
// IdentityOf is module's identity, and false when it was never allocated one.
func (i *Inventory) IdentityOf(ctx context.Context, module string) (catalogue.ModuleIdentity, bool, error) {
id := catalogue.ModuleIdentity{Module: module}
err := i.store.Pool().QueryRow(ctx, `select uid, gid from module_identity where module = $1`, module).
Scan(&id.UID, &id.GID)
if errors.Is(err, pgx.ErrNoRows) {
return catalogue.ModuleIdentity{}, false, nil
}
if err != nil {
return catalogue.ModuleIdentity{}, false, err
}
return id, true, nil
}
// Identities is every module's identity, by module.
func (i *Inventory) Identities(ctx context.Context) (map[string]catalogue.ModuleIdentity, error) {
rows, err := i.store.Pool().Query(ctx, `select module, uid, gid from module_identity`)
if err != nil {
return nil, err
}
defer rows.Close()
out := map[string]catalogue.ModuleIdentity{}
for rows.Next() {
var id catalogue.ModuleIdentity
if err := rows.Scan(&id.Module, &id.UID, &id.GID); err != nil {
return nil, err
}
out[id.Module] = id
}
return out, rows.Err()
}
// TakenFolder is one folder the operator let the mesh keep on a machine (ADR 0325 §8).
type TakenFolder struct {
Path string
By string
Ask string
Warrant string
Said string
// Placed is the filesystem the folder was on when taken, without its path (ADR 0325 §8).
Placed string
}
// KeptFolders is every folder taken on a machine, by its clean path.
func (i *Inventory) KeptFolders(ctx context.Context, nodeName string) (map[string]TakenFolder, error) {
rows, err := i.store.Pool().Query(ctx,
`select k.path, k.taken_by, k.ask, k.warrant, k.said, k.placed from kept_folder k join node n on n.id = k.node
where n.name = $1`, nodeName)
if err != nil {
return nil, err
}
defer rows.Close()
out := map[string]TakenFolder{}
for rows.Next() {
var t TakenFolder
if err := rows.Scan(&t.Path, &t.By, &t.Ask, &t.Warrant, &t.Said, &t.Placed); err != nil {
return nil, err
}
out[t.Path] = t
}
return out, rows.Err()
}
// RecordTaken keeps that a folder was taken on a machine, under the operator's warrant. A take is recorded once per
// filesystem: a second for the same folder on the same filesystem keeps the first, and one for the folder on another
// filesystem (it moved, and the operator was asked again) replaces it.
func (i *Inventory) RecordTaken(ctx context.Context, nodeName string, t TakenFolder) error {
switch {
case !strings.HasPrefix(t.Path, "/"):
return fmt.Errorf("a taken folder is an absolute path, not %q", t.Path)
case strings.TrimSpace(t.Warrant) == "" || strings.TrimSpace(t.Ask) == "":
return errors.New("a folder is taken only on the operator's warrant, and this names none")
}
node, err := i.NodeByName(ctx, nodeName)
if err != nil {
return err
}
_, err = i.store.Pool().Exec(ctx,
`insert into kept_folder (node, path, taken_by, ask, warrant, said, placed) values ($1, $2, $3, $4, $5, $6, $7)
on conflict (node, path) do update
set taken_at = now(), taken_by = excluded.taken_by, ask = excluded.ask, warrant = excluded.warrant,
said = excluded.said, placed = excluded.placed
where kept_folder.placed is distinct from excluded.placed`, node.ID, filepath.Clean(t.Path), t.By, t.Ask,
t.Warrant, t.Said, t.Placed)
return err
}
+224
View File
@@ -0,0 +1,224 @@
package inventory
import (
"encoding/json"
"fmt"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// novox/hq ADR 0325 §3, §11: a module gets one allocation, the same whoever asks and however often, from the range
// of its own; no allocation is 0 or a root group's; a number once given is never given again.
func TestAModuleIsGivenOneIdentityOnceAndNeverRoot(t *testing.T) {
inv := fresh(t)
ctx := t.Context()
radarr, err := inv.AllocateIdentity(ctx, "radarr")
if err != nil {
t.Fatal(err)
}
again, err := inv.AllocateIdentity(ctx, "radarr")
if err != nil {
t.Fatal(err)
}
if radarr != again {
t.Fatalf("radarr was given %+v, then %+v: one allocation per module", radarr, again)
}
plex, err := inv.AllocateIdentity(ctx, "plex")
if err != nil {
t.Fatal(err)
}
for _, id := range []catalogue.ModuleIdentity{radarr, plex} {
if id.UID == 0 || id.GID == 0 || !catalogue.AllocatableID(id.UID) || !catalogue.AllocatableID(id.GID) ||
id.UID != id.GID {
t.Errorf("%s was given %d:%d: within %d–%d, never 0, uid and gid one number", id.Module, id.UID, id.GID,
catalogue.IdentityLow, catalogue.IdentityHigh)
}
}
if radarr.UID == plex.UID {
t.Fatalf("radarr and plex share %d", plex.UID)
}
// Read back as composition reads it, from another connection's view: the same numbers (stable across restarts
// and machines, since every machine's composition reads this one row).
all, err := inv.Identities(ctx)
if err != nil {
t.Fatal(err)
}
if all["radarr"].UID != radarr.UID || all["plex"].UID != plex.UID {
t.Fatalf("the identities read back are %+v", all)
}
// A module forgotten keeps its row, and the next module is given a new number: never one given before.
if _, err := inv.store.Pool().Exec(ctx, `delete from module_identity where module = 'plex'`); err != nil {
t.Fatal(err)
}
sonarr, err := inv.AllocateIdentity(ctx, "sonarr")
if err != nil {
t.Fatal(err)
}
if sonarr.UID == plex.UID || sonarr.UID == radarr.UID {
t.Fatalf("sonarr was given %d, a number given before", sonarr.UID)
}
// The store itself refuses a number outside the range, root's above all: no write by hand makes one root.
for _, n := range []int{0, 10, 1000, 49999, 60000} {
if _, err := inv.store.Pool().Exec(ctx,
`insert into module_identity (module, uid, gid) values ($1, $2, $2)`, "byhand", n); err == nil {
t.Errorf("the store took %d as an identity", n)
}
}
}
// novox/hq ADR 0325 §8: a folder taken is recorded once per machine, under a warrant, and read back by its path.
func TestAFolderIsTakenOnceUnderAWarrant(t *testing.T) {
inv := fresh(t)
ctx := t.Context()
if _, err := inv.AddNode(ctx, "ace"); err != nil {
t.Fatal(err)
}
if err := inv.RecordTaken(ctx, "ace", TakenFolder{Path: "/storage/media/movies/", By: "the operator"}); err == nil {
t.Fatal("a folder was recorded taken with no warrant")
}
first := TakenFolder{Path: "/storage/media/movies/", By: "the operator", Ask: "c01", Warrant: "c01", Said: "started"}
if err := inv.RecordTaken(ctx, "ace", first); err != nil {
t.Fatal(err)
}
if err := inv.RecordTaken(ctx, "ace", TakenFolder{Path: "/storage/media/movies", By: "someone else", Ask: "c02",
Warrant: "c02"}); err != nil {
t.Fatal(err)
}
kept, err := inv.KeptFolders(ctx, "ace")
if err != nil {
t.Fatal(err)
}
got, has := kept["/storage/media/movies"]
if !has || got.Ask != "c01" || got.By != "the operator" {
t.Fatalf("the taken folder reads %+v (%v): the first take, by its clean path", got, has)
}
// The folder moved to another filesystem and the operator was asked again: the new take replaces the old.
if err := inv.RecordTaken(ctx, "ace", TakenFolder{Path: "/storage/media/movies", By: "the operator", Ask: "c03",
Warrant: "c03", Placed: "fs-0000000000ff"}); err != nil {
t.Fatal(err)
}
kept, err = inv.KeptFolders(ctx, "ace")
if err != nil {
t.Fatal(err)
}
if got := kept["/storage/media/movies"]; got.Ask != "c03" || got.Placed != "fs-0000000000ff" {
t.Fatalf("after a take on another filesystem the folder reads %+v", got)
}
}
// novox/hq issue 575: a module registered without the access movies, whose newest build on its branch declares it —
// the build the delivery under way brings — takes a proposal placing movies: judged against that build, not refused.
// Without such a build it is refused, as before.
func TestAPlacementIsJudgedAgainstTheBuildThatBringsTheAccess(t *testing.T) {
inv := fresh(t)
ctx := t.Context()
if _, err := inv.AddNode(ctx, "ace"); err != nil {
t.Fatal(err)
}
registered := catalogue.Manifest{Module: "media-importer", Version: "1",
Accesses: []catalogue.Access{{ID: "downloads", Path: "/services/media/downloads", Mode: catalogue.AccessReadWrite}},
Resources: []map[string]any{{"id": "server", "type": "container", "name": "importer", "image": "x",
"volumes": []any{"${access:downloads}:/downloads"}}}}
if err := inv.RegisterModule(ctx, registered, Source{Repository: "https://forge.invalid/media.git",
Ref: "main", BuiltFrom: "c0ffee33"}); err != nil {
t.Fatal(err)
}
placing := map[string]any{catalogue.AccessesSetting: map[string]any{"movies": "/storage/media/movies"}}
err := inv.JudgeSettings(ctx, "ace", "media-importer", placing)
if err == nil || !strings.Contains(err.Error(), `places the access "movies", which it does not declare`) {
t.Fatalf("a placement no build declares: %v", err)
}
// main's manifest — the build of #34 — declares movies.
ahead := registered
ahead.Accesses = append([]catalogue.Access{}, registered.Accesses...)
ahead.Accesses = append(ahead.Accesses, catalogue.Access{ID: "movies", Path: "/services/media/movies",
Mode: catalogue.AccessReadWrite})
raw, err := json.Marshal(ahead)
if err != nil {
t.Fatal(err)
}
b := aBuild("b34", "media-importer", "")
b.Ref, b.Manifest = "main", raw
if err := inv.RecordBuild(ctx, b); err != nil {
t.Fatal(err)
}
if err := inv.JudgeSettings(ctx, "ace", "media-importer", placing); err != nil {
t.Fatalf("the proposal placing movies, which main's build declares, was refused: %v", err)
}
if err := inv.SetSettings(ctx, "ace", "media-importer", placing); err != nil {
t.Fatalf("the layer placing movies was not stored: %v", err)
}
// Stored ahead, it does not cost the registered module its place: composition passes it by.
layers, err := inv.SettingsFor(ctx, "ace", "media-importer")
if err != nil {
t.Fatal(err)
}
if err := catalogue.ComposesWith(registered, layers, false); err != nil {
t.Fatalf("the registered module no longer composes with the layer stored ahead: %v", err)
}
// A typo is still a typo: an access neither build declares is refused.
typo := map[string]any{catalogue.AccessesSetting: map[string]any{"moveis": "/storage/media/movies"}}
if err := inv.JudgeSettings(ctx, "ace", "media-importer", typo); err == nil {
t.Fatal("a placement no build declares was taken")
}
}
// Review of e1c2b744: only a build of the branch the module's record follows judges a placement ahead of it — never one
// from another branch, nor one with no branch, and a module registered with no branch is refused as before, saying so.
func TestOnlyABuildOfTheFollowedBranchJudgesAPlacementAhead(t *testing.T) {
inv := fresh(t)
ctx := t.Context()
if _, err := inv.AddNode(ctx, "ace"); err != nil {
t.Fatal(err)
}
registered := catalogue.Manifest{Module: "importer", Version: "1",
Resources: []map[string]any{{"id": "server", "type": "container", "name": "importer", "image": "x"}}}
ahead := registered
ahead.Accesses = []catalogue.Access{{ID: "movies", Path: "/services/media/movies", Mode: catalogue.AccessReadWrite}}
raw, err := json.Marshal(ahead)
if err != nil {
t.Fatal(err)
}
placing := map[string]any{catalogue.AccessesSetting: map[string]any{"movies": "/storage/media/movies"}}
if err := inv.RegisterModule(ctx, registered, Source{Repository: "https://forge.invalid/media.git", Ref: "main",
BuiltFrom: "c0ffee"}); err != nil {
t.Fatal(err)
}
for i, ref := range []string{"feature", ""} {
b := aBuild(fmt.Sprintf("elsewhere%d", i), "importer", "")
b.Ref, b.Manifest = ref, raw
if err := inv.RecordBuild(ctx, b); err != nil {
t.Fatal(err)
}
}
if err := inv.JudgeSettings(ctx, "ace", "importer", placing); err == nil {
t.Fatal("a build of another branch, or of none, judged the placement")
}
// Registered by hand, with no branch: refused, saying why.
byHand := registered
byHand.Module = "byhand"
if err := inv.RegisterModule(ctx, byHand, Source{}); err != nil {
t.Fatal(err)
}
aheadByHand := ahead
aheadByHand.Module = "byhand"
if raw, err = json.Marshal(aheadByHand); err != nil {
t.Fatal(err)
}
b := aBuild("onmain", "byhand", "")
b.Ref, b.Manifest = "main", raw
if err := inv.RecordBuild(ctx, b); err != nil {
t.Fatal(err)
}
if err := inv.JudgeSettings(ctx, "ace", "byhand", placing); err == nil ||
!strings.Contains(err.Error(), "follows no branch") {
t.Fatalf("a module with no branch recorded: %v", err)
}
}
@@ -0,0 +1,44 @@
-- Every module is given an identity of its own, and the mesh takes a shared folder once, on the operator's word
-- (novox/hq ADR 0325 §3, §8, §11).
--
-- **One number per module, mesh-wide, allocated once and never again.** The node-engine makes the account
-- mesh-<module> on each machine the module is assigned to, with these numbers, so a container that declares
-- `identity` runs as the same uid and gid everywhere and a folder's entries mean the same on every machine. The
-- numbers come from a sequence of their own, 50000–59999 (catalogue.IdentityLow and IdentityHigh): above the
-- numbers distributions give people, below the 60000s Debian keeps for its daemons, and nowhere near root's 0 or a
-- root group's (0, 4, 10). The checks refuse a row outside the range, so no write, by hand or by a later bug, can make
-- an identity root (§11). uid and gid are one number. A sequence never hands a number back, and a row is kept after
-- its module is forgotten, so a number once given is never given to another module: a file left owned by a removed
-- module's identity is never inherited by a new one.
create sequence module_identity_number as integer minvalue 50000 maxvalue 59999 start 50000 no cycle;
create table module_identity (
module text primary key,
uid integer not null unique check (uid between 50000 and 59999),
gid integer not null unique check (gid between 50000 and 59999),
allocated timestamptz not null default now(),
check (uid = gid)
);
-- **A folder taken, per machine** (§8). The first time a placed folder that already holds data is to be kept, the
-- operator is asked once for that folder on that machine; on the warrant the node-engine adds the entries to
-- everything there, and the folder is taken: kept from then on without asking, a module that gains the access
-- added with no wait. The row is the record of that act — who approved it, under which warrant — and what tells a
-- later module's access that it joins a taken folder (`taken` on the access). The path is the folder as placed,
-- clean and absolute. A new warrant for the same folder on another filesystem replaces the row.
create table kept_folder (
node uuid not null references node (id) on delete cascade,
path text not null check (path like '/%'),
taken_at timestamptz not null default now(),
-- Who approved it, as the hand-act log says it, and the ask and warrant it was taken under.
taken_by text not null,
ask text not null,
warrant text not null,
-- What the node-engine answered when it started the take.
said text not null default '',
-- The filesystem the folder was on when it was taken, as the engine names it without a path ("fs-<12 hex>"):
-- a wait naming another (the folder moved to another disk) is not answered by this take, and the operator is
-- asked again; until the new warrant the folder is not taken there. Empty from an engine that names none.
placed text not null default '',
primary key (node, path)
);
+40 -8
View File
@@ -61,6 +61,10 @@ type Plan struct {
// Phases is the walk's time from its merge, phase by phase (ADR 0282 decision 6): never kept, worked out
// from the walk's own moments by whoever says the walk (`delivery walks`, `plan-moved`).
Phases *WalkPhases `json:"phases,omitempty"`
// WalkSaid is, on the save that started or ended the walk, which of the two it was (novox/hq ADR 0308
// decision 1): set by the save, never kept, read by whoever says the save (`plan-moved`). Empty on every
// other save.
WalkSaid string `json:"-"`
}
// PlanTimes are a walk's own moments beside its modules' (novox/hq ADR 0282 decision 6).
@@ -73,6 +77,10 @@ type PlanTimes struct {
// Class is the walk's module class, read at its cut: core when it moves a module on the controller's own
// path or the mesh's resolver, leaf otherwise (ADR 0282 decision 1).
Class string `json:"class,omitempty"`
// Started is when the walk was said started, and Ended when it was said ended (novox/hq ADR 0308 decision
// 1): each said once, on the save that marked it.
Started *time.Time `json:"started,omitempty"`
Ended *time.Time `json:"ended,omitempty"`
}
// PlanCommit is one repository's commit a walk carries: the latest merge of its branch in the batch.
@@ -117,6 +125,14 @@ type PlanBatch struct {
// the batch's walk does not build them.
Closed bool `json:"closed,omitempty"`
Withheld []string `json:"withheld,omitempty"`
// CutAtOnce says the batch's window closed when it was made: its merge shares nothing with a batch
// assembling and has no delivery group member still to merge, so it waits for no window (novox/hq ADR 0304
// decision 1, ADR 0306 decisions 2 and 3). A merge that joins it later opens its window again. It is not
// Closed, which is a stop's.
CutAtOnce bool `json:"cut_at_once,omitempty"`
// Waits says why a batch made for one merge waits for the window: its delivery group has a member still to
// merge, or mesh-delivery could not say whether it has (novox/hq ADR 0306 decision 2).
Waits string `json:"waits,omitempty"`
}
// OwnPath says the record is a batch of merges on the controller's own path.
@@ -183,8 +199,8 @@ type PlanDelivery struct {
Go *time.Time `json:"go,omitempty"`
By string `json:"by,omitempty"`
Why string `json:"why,omitempty"`
// Stopped is who ended the walk through the delivery's owner, and StoppedWhy why: the walk is failed,
// and the delivery reads it as stopped, not as a build that failed.
// Stopped is who ended the walk, through the delivery's owner or by hand (`plans stop|close`), and StoppedWhy
// why: the walk is failed, and the delivery reads it as stopped, not as a build that failed.
Stopped string `json:"stopped,omitempty"`
StoppedWhy string `json:"stopped_why,omitempty"`
// Merges is every merge the walk or batch answers (novox/hq ADR 0276): its own commits, and the earlier
@@ -236,10 +252,12 @@ func (p Plan) Waiting() bool {
return p.Delivery != nil && p.Delivery.Awaits != "" && p.Delivery.Go == nil
}
// PlanSaved is told every plan this process kept, after it is kept (novox/hq ADR 0239): the serving
// controller says it on the bus as `plan-moved`. Nil in a command, which says nothing; whoever follows a
// walk also asks for it, so a save made by a command is found by comparison.
var PlanSaved func(Plan)
// PlanSaved is told every plan this process kept, after it is kept, with the inventory that kept it (novox/hq
// ADR 0239): the serving controller says it on the bus as `plan-moved`, and so does a command (the controller's
// run installs it for every verb, novox/hq ADR 0308 decision 1), so the save that starts or ends a walk is said
// whichever process made it. Nil says nothing; whoever follows a walk also asks for it, and finds an unsaid save
// by comparison.
var PlanSaved func(*Inventory, Plan)
// ErrPlanMoved is a save against a plan written by somebody else since it was read.
var ErrPlanMoved = errors.New("the plan was written by somebody else since it was read")
@@ -451,7 +469,7 @@ type MergeMove struct {
// leaves a merge answered by two plans, nor a walk stopped whose merges were not moved). A plan written by
// somebody else since it was read refuses the whole act with ErrPlanMoved. Each plan is said (PlanSaved) once
// the transaction is committed, in order.
func (i *Inventory) SavePlansMoving(ctx context.Context, plans []*Plan, moves []MergeMove) error {
func (i *Inventory) SavePlansMoving(ctx context.Context, plans []*Plan, moves []MergeMove) (err error) {
epoch, err := i.actingEpoch(ctx)
if err != nil {
if len(plans) > 0 {
@@ -472,6 +490,20 @@ func (i *Inventory) SavePlansMoving(ctx context.Context, plans []*Plan, moves []
entered time.Time
}
done := make([]written, len(plans))
// **The save that starts or ends a walk is marked** (novox/hq ADR 0308 decision 1), in the same write, and
// unmarked should the write not be made, so the next save marks it again.
marked := time.Now()
undo := make([]func(), 0, len(plans))
defer func() {
if err != nil {
for _, u := range undo {
u()
}
}
}()
for _, p := range plans {
undo = append(undo, p.markWalk(marked))
}
for n, p := range plans {
revision, entered, err := writePlan(ctx, tx, p, epoch)
if err != nil {
@@ -497,7 +529,7 @@ func (i *Inventory) SavePlansMoving(ctx context.Context, plans []*Plan, moves []
p.Epoch = 0
}
if PlanSaved != nil {
PlanSaved(*p)
PlanSaved(i, *p)
}
}
return nil
+4 -3
View File
@@ -202,14 +202,15 @@ func TestASeatsVerbGainsTheArgumentsTheBinaryNames(t *testing.T) {
t.Fatal("failed was not added to the row")
}
// The optional mark is not stored; the working set read from the rows takes it from the compiled seat,
// so today's holder, which does not serve failed, still holds the seat.
// so a holder that does not serve the optional failed still holds the seat.
catalogue.UseSeats(seats)
defer catalogue.UseSeats(catalogue.DefaultSeats())
live, _ := catalogue.SeatNamed(catalogue.ServiceManagerSeat)
holder := catalogue.Manifest{Module: "systemd", Version: "1", Claims: []catalogue.Claim{{Name: catalogue.ServiceManagerSeat,
Scope: catalogue.ScopeNode, Serves: []string{"units", "status", "start", "stop", "restart", "enable", "disable", "journal"}}}}
Scope: catalogue.ScopeNode, Serves: []string{"units", "status", "start", "stop", "restart", "enable", "disable", "journal",
"reset-failed", "wanted-by", "unlink-dangling"}}}}
if err := catalogue.CanHold(holder, live); err != nil {
t.Fatalf("the seat read from the store refuses today's holder: %v", err)
t.Fatalf("the seat read from the store refuses a holder without the optional failed: %v", err)
}
// Seeding again changes nothing more.
before := verbs["journal"]

Some files were not shown because too many files have changed in this diff Show More