Compare commits

..
Author SHA1 Message Date
jschoubben 1747899419 delivery: the bus node hands the loaded list's hashes to the vault, sealed, in the bus module's have (ADR 0547 clause 9, issue 1945)
The vault holds no value for the controller's own account, the node accounts and the enrolled devices, so it cannot
hash them; the one part that has their hashes is the node-engine of the bus node, in the list the bus loaded. The have
of the bus module now carries that list's hashes sealed to the vault's key (UserListWritten.Handed), which every reader
of the have bucket sees as a count, a list digest and a key fingerprint, and only the vault opens. HandedAccountsOf
reads them from a composed list, HandAccounts seals, OpenHanded opens and refuses a box sealed to another key.
2026-10-26 11:28:38 +01:00
mesh-tools 85fa8511c4 Merge pull request 'Build steps: run every test step of a level and name every failure in the verdict (issue 1921)' (#87) from fix/1921-every-failure-r into main 2026-10-26 10:18:05 +00:00
jschoubben 79649c3fa9 Run every test step of a check's level after a failure, and let a verdict name every failed test (issue 1921)
mesh/delivery merged: source.main moved to novox/mesh-sdk@85fa8511c4b3
mesh/delivery-group group fix/1921-every-failure-r merged: every member is merged
mesh/check checking 79649c3fa9f4 on c3544b6a17a2 | novox/mesh-sdk#87 in group fix/1921-every-failure-r
A check ask stopped its level at the first test step that did not pass,
so the packages after the changed ones never ran and a check found one
fault per round. Every step now runs; the level still fails, and no
later level runs (ADR 0484 clause 13). The verdict gains Failures, every
failed test with its first line, at most MaxFailures and a count.
2026-10-26 10:57:02 +01:00
mesh-tools c3544b6a17 Merge pull request 'buscompose: the grants part may create a placed module's declared buckets, logs and streams, creation only (issue 1790, ADR 0547 clause 1 exception)' (#86) from fix/1790-grants-create-buckets into main 2026-10-26 08:17:16 +00:00
jschoubben 136fa8b2a8 Say how a declared bucket and module stream are created, so the grants part makes the same stream the controller does
mesh/delivery merged: source.main moved to novox/mesh-sdk@c3544b6a17a2
mesh/delivery-group group fix/1790-grants-create-buckets merged: every member is merged
mesh/check checking 136fa8b2a896 on 095d41fa3e81 | novox/mesh-sdk#86 in group fix/1790-grants-create-buckets
BucketStreamConfig and ModuleStreamConfig are the stream configurations the controller's
CreateOrUpdateKeyValue (BucketConfig) and StreamConfigFor assert at its start, pure, for the
grants part's create of a bucket or stream a manifest declares and the bus lacks. A log already
has Log.Config. Never used for an update.

Issue: 1790
2026-10-26 09:04:43 +01:00
jschoubben 8f6ee2fd18 Let the grants part create the streams manifests declare, so a new module state no longer waits for a controller start
The copied bus composer grants mesh-grants' principal $JS.API.STREAM.CREATE.<stream> and
$JS.API.STREAM.INFO.<stream> for each bucket (KV_<module>_<state>), log (LOG_<module>_<log>) and
module stream a placed module's manifest declares, each by its own name: NATS wildcards are whole
tokens, so nothing narrower than an exact name exists and nothing wider is granted. The writers
table lets the grants part create, never update or delete. The operator allowed this one grant
on 2026-10-26 08:54, beside the memberships grant of 22:31 (ADR 0547 clause 1 and 5).

The expected lists were written again by the controller's own internal/broker with the same
change (mesh-control fix/1790-grants-create-buckets); a fixture now places mesh-grants beside the
builder's state, log and stream.

Issue: 1790
2026-10-26 09:01:32 +01:00
jschoubben 095d41fa3e Merge pull request 'buscompose: the controller's bus composer in the SDK, byte-equal to the controller's, with the mesh-grants grant (ADR 0547)' (#85) from fix/1945-memberships into main
Reviewed-on: #85
2026-10-25 22:43:57 +00:00
jschoubben b7b5c6ef27 The copied composer grants the grants part reading every membership, and its memberships are held to the controller's too (ADR 0547 §5, review of mesh-grants)
mesh/delivery merged: source.main moved to novox/mesh-sdk@095d41fa3e81
mesh/delivery-group group fix/1945-memberships merged: every member is merged
mesh/check failed: integration: github.com/novox/mesh-controller/cmd/mesh-controller.TestTheDeclarationsStreamGoesOnceEveryNodeRunsTheNewEngine: with …
The grants part reads each module's last membership before it publishes, so it needs the direct get on every
membership subject, as the controller's composer now grants (fix/1945-mesh-grants-publishes-memberships). The equality
test compares every membership the fixtures issue with the controller's MembershipFor, written by its own code.
2026-10-25 22:50:21 +01:00
mesh-tools 7ac4493ce9 Merge pull request 'process: open a part's buckets as essential (stop, naming the grant) or optional (log what is not heard, keep running) (issue 1945)' (#84) from fix/1945-ask-again-and-why into main 2026-10-25 21:43:08 +00:00
jschoubben 881da7f599 The copied bus composer grants the grants part publishing memberships, as the controller's does (ADR 0547 §5)
The same line as mesh-controller f67d1553; the fixtures name a mesh-grants module and the lists were written again by
the controller's own composer on that branch, so the copy is held byte-equal to it.
2026-10-25 22:33:20 +01:00
mesh-tools 3db051be57 Merge pull request 'A node's have says which bus user list it wrote: delivery.Have's user-list field (ADR 0547 bridge, issue 1945)' (#83) from fix/1945-user-list-follows into main 2026-10-25 21:16:03 +00:00
jschoubben e0d1de0475 process: a part opens what it cannot work without as essential, the rest as optional (issue 1945)
mesh/delivery merged: source.main moved to novox/mesh-sdk@7ac4493ce9ab
mesh/delivery-group group fix/1945-ask-again-and-why merged: every member is merged
mesh/check pass e0d1de047585 on 3db051be57ca | unit 1642 run, 752 kept, 2 skipped | integration 1710 run, 803 kept, 2 skipped | over bound: unit, inte…
The delivery builder exited twice on 2026-10-25 for a bucket the bus
refused at start, though it could build without it: a grant reaches the
bus with a delivery, so a refusal at start is often only a grant not
there yet. EssentialBucket stops a start naming the bucket and its grant;
OptionalBucket and NotWatched say what is not heard and go on. Here so
the seven delivery parts take one copy, not seven.
2026-10-25 22:12:20 +01:00
jschoubben b5f8604d03 Copy the controller's bus composer into mesh-sdk as buscompose, held byte for byte to the controller's lists (issue 1377, ADR 0547 §5)
The grants part composes the bus's user list with the composer the controller uses (ADR 0547 §5), copied rather than
moved: the controller is not changed and keeps its own until it loses the lease. Left out: the bus client, the broker's
address from the environment and the client's two methods; the lease's two subjects are copied in.

One table test holds the copy to four meshes' lists and the server golden, written by the controller's own
internal/broker from the same fixtures (testdata/controller-user-lists.json).
2026-10-25 21:54:58 +01:00
jschoubben 0be0ffd615 Compare the have's user list by what it says, so a have is not written again for the same list (ADR 0547 §8)
mesh/delivery merged: source.main moved to novox/mesh-sdk@3db051be57ca
mesh/delivery-group group fix/1945-user-list-follows merged: every member is merged
mesh/check pass 0be0ffd6159e on db26f7e8b6f5 | unit 1467 run, 1696 kept, 7 skipped | integration 1471 run, 1740 kept, 6 skipped | over bound: unit, in…
2026-10-25 21:47:30 +01:00
jschoubben f63fa86c74 Say in the bus module's have the user list its node wrote alone, so the bridge of ADR 0547 §8 reports what the bus loads (issue 1945) 2026-10-25 21:42:13 +01:00
mesh-tools db26f7e8b6 Merge pull request 'The put's words: delivery.Put, a want's put and hold marks, a grant that says which node's accounts it names (ADR 0546)' (#82) from feat/put-version into main 2026-10-25 20:35:28 +00:00
jschoubben d4ae4b6790 Say the put's tools, not a seat's verbs, in the put's words (ADR 0546)
mesh/delivery merged: source.main moved to novox/mesh-sdk@db26f7e8b6f5
mesh/check pass d4ae4b6790b8 on db26f7e8b6f5 | unit 1102 run, 680 kept, 2 skipped | integration 1103 run, 722 kept, 2 skipped | over bound: unit (slow…
2026-10-25 21:08:10 +01:00
jschoubben 2f6741b7f8 No seat and no compose verb for the put: the controller takes no change, so the assigner serves the put as its own tools from compositions the controller already writes (ADR 0546) 2026-10-25 21:07:58 +01:00
jschoubben 20a6dc4920 A grant says whether it names a module's accounts on a node, for the assigner and the compose verb (ADR 0546) 2026-10-25 20:50:51 +01:00
jschoubben c4f37c279a The assigner's seat with put, release and puts, the controller's compose verb, a put's key and value, and the view answers no option that authorises (ADR 0546) 2026-10-25 20:49:29 +01:00
jschoubben 7f89cb51de Merge pull request 'Keep a build node's pause in a state of its own, and pull as not paused when the pause cannot be read (steps bucket full, 2026-10-25)' (#81) from fix/steps-bucket-ages into main
Reviewed-on: #81
2026-10-25 18:06:32 +00:00
jschoubben 518dae5ec9 Pull as not paused when the pause cannot be read, and say so once, instead of ending the puller silently
mesh/delivery held for a person: merged without a passing check: only a person decides what follows
mesh/delivery-group group fix/steps-bucket-ages held: a member is held: novox/mesh-sdk@518dae5ec9c8 (merged without a passing check: only a person decides what…
mesh/check checking 518dae5ec9c8 on 7f89cb51de15 | novox/mesh-sdk#81 in group fix/steps-bucket-ages
A Load error ended Pull, and the modules run it in a goroutine whose error nobody reads: every
build node would stop pulling when the new pauses state could not be read (or when Pull reached
restore before Serve). A node kept from building is held by capacity 0, not by a pause.
2026-10-25 18:56:17 +01:00
jschoubben e8f8ef63fa Keep a build node's pause in a pauses state of its own, so the steps state can drop its old keys without ending a pause
The steps state of the build modules filled to the bus's 64 MiB cap (27,370 keys of mesh-build-go,
none ever removed) and refused a passed check's end. Its keys get an age in the catalogue; the
pause key lived among them and would then have ended by itself. Puller.PausePut writes it to
delivery.PausesState instead; Put stays the fallback for a module not yet moved.
2026-10-25 18:51:04 +01:00
jschoubben 4f4d5ca9c9 Merge pull request 'Wake a parked build puller when its capacity changes, so a node raised from capacity 0 takes asks (issue 1887)' (#80) from fix/puller-wakes-on-capacity into main
Reviewed-on: #80
2026-10-25 17:10:33 +00:00
jschoubben a8fbabbfc3 Fold the puller-wake and Served tests into existing tests to stay within the test budget (issue 1887)
mesh/delivery merged: source.main moved to novox/mesh-sdk@4f4d5ca9c956
mesh/delivery-group group fix/puller-wakes-on-capacity merged: every member is merged
mesh/check pass a8fbabbfc373 on 4f4d5ca9c956 | unit 1102 run, 679 kept, 2 skipped | integration 1103 run, 721 kept, 2 skipped | over bound: unit, inte…
2026-10-25 17:44:03 +01:00
jschoubben b9911dc1c1 Start the puller's capacity watch once Serve has the runtime's channel, since a build module pulls before it serves (issue 1887)
A watch asked before Serve is refused, which would have left every build
module on its settings file again. stdio.Served closes when Serve has the
channel; WatchConfigServed waits on it, and Pull starts the watch beside its
loop.
2026-10-25 17:26:10 +01:00
jschoubben e6b2df6506 Wake a parked build puller on its capacity's own change, so a node started at capacity 0 takes asks once raised (issue 1887)
g14's Go builder sat idle at capacity 2 while delivery builds waited: a puller
with no room waited on its wake alone, which only a pause, a resume or a step's
end gives, and it read its capacity from the settings file only when woken.
The puller now follows the module's config value capacity through the
runtime's watch of what mesh-config resolved (stdio.WatchConfig), uses the
value it hands, and wakes on each change: no timer, no file watch, and no wait
for the node-engine to fill the file again. It says when it parks and when it
wakes, with its capacity.
2026-10-25 17:23:28 +01:00
jschoubben f9d11acf23 Merge pull request 'A node's own composition may carry filled paths; a composition that would leave a provisioner file empty is refused (issue 1944)' (#79) from fix/r6-node-composition into main
Reviewed-on: #79
2026-10-25 00:17:36 +00:00
jschoubben 10b6e20232 Let the node's own composition carry its files' filled paths, and refuse a composition that would have a provisioner's file written empty (issue 1945)
mesh/delivery held for a person: merged without a passing check: only a person decides what follows
mesh/delivery-group group fix/r6-node-composition held: a member is held: novox/mesh-sdk@10b6e202320f (merged without a passing check: only a person decides wh…
mesh/check checking 10b6e202320f on da7f0455759a
2026-10-25 01:46:36 +02:00
jschoubben da7f045575 Merge pull request 'R6: the composition's layout, its signature and digest, the want's digest, the judgement values; the hand tool is mesh-hand-compose (issue 1943)' (#78) from feat/r6 into main
Reviewed-on: #78
2026-10-24 23:17:32 +00:00
jschoubben 748c564cdf Merge main (the R4g finish: the composer command) into feat/r6 (issue 1943)
mesh/delivery delivering: 0 machine step(s) passed
mesh/delivery-group group feat/r6 merged: every member is merged
mesh/check failed: the composition: the composer of the mesh-sdk mesh-controller at bdb8193456e17c9df59215c7f3e69a83a2e5a262 pins did not run: failed …
main's mesh-compose is the composer command (ADR 0482 clause 4) and keeps its name, which composer.Package and the
build steps name; feat/r6's hand path tool, which composes one module for one node and signs it with the operator's key
(ADR 0539 clause 10), moves to cmd/mesh-hand-compose.
2026-10-24 23:01:18 +02:00
jschoubben 312878befb Merge pull request 'R4g finish: the composer's step contract and the reads-outside marker (issues 1621-1623, 1935)' (#77) from feat/r4g-finish into main
Reviewed-on: #77
2026-10-24 20:56:35 +00:00
jschoubben d7a9b9688b Let the judgement beside a composition say unchanged: equal to the declaration the node last applied (issue 1943; ADR 0540 clause 9)
The first want at the switch skips the record hold only where the composition changes nothing on the node.
2026-10-24 21:17:14 +02:00
jschoubben 7f9cc0e55c Keep plan-moved and sent among the controller seat's events until the catalogue's main consumes them no more (issue 1943)
The controller's broker tests read the catalogue's main, whose gate and messenger still consume plan-moved; nothing
emits them (ADR 0539).
2026-10-24 20:38:24 +02:00
jschoubben e45bdc0891 Drop plan-moved and sent from the controller seat's events (issue 1943; the review's notes on 1945)
No plan is kept or sent in release 6 (ADR 0539), and the messenger now hears the delivery view's events instead.
2026-10-24 20:20:48 +02:00
jschoubben a523a8cbb0 Say beside a composition that it changes only the bus's user list (issue 1943; the review's A2, for issue 1906)
A revision that changes only the user list is written into the bus node's want at once, outside the record hold
(ADR 0539 clause 7), so the assigner must tell it apart from a move that recreates or moves data.
2026-10-24 18:15:32 +02:00
jschoubben 5da87a0dda Let the controller say a bucket it deleted at its first start (issue 1943; for issue 1945, B13)
ADR 0539 clause 13 has the controller delete the plan-state buckets once and say each as its event.
2026-10-24 17:03:54 +02:00
jschoubben e64486b0e6 Sign a composition through a signer that keeps its private half (issue 1943, S5)
The controller's identity store signs a message and never hands its key out, as it signed declarations.
2026-10-24 16:42:41 +02:00
jschoubben 64a502d16a Carry the module's upgrade policy and its source beside a composition's judgement (issue 1943; for issue 1906)
The assigner holds a move of a module of policy record, and only the controller knows a person's own choice of
policy (ADR 0539 clause 14).
2026-10-24 16:40:45 +02:00
jschoubben de5b33843f Run the captured declarations' test per node in parallel, inside the unit level's bound (issue 1943) 2026-10-24 16:36:42 +02:00
jschoubben d8da8f1d36 Sign a composition with a trusted key and compose one by hand with mesh-compose (issue 1943, S5)
The node-engine applies only a composition signed by the controller or the operator, and when the controller cannot
compose its own fix the operator composes, signs and writes one at the terminal (ADR 0539 clauses 3 and 10).
2026-10-24 16:36:14 +02:00
jschoubben d506f2196a Refuse the plan verbs by name on the controller and delivery seats (issue 1943, S4)
With no delivery plan nothing starts, stops, retries or reads one; each removed verb stays a row with no input so a
caller is told its replacement, optional on the delivery seat so the gate of before and of release 6 both hold it
(ADR 0539 clauses 15, 17 and 18).
2026-10-24 16:32:11 +02:00
jschoubben d6cef5829b Have a want name its composition and a backup answer; give the node its own want, the seventh rule, the held phase and the backup and record asks (issue 1943, S3)
The node agent applies only the composition its want names, and the assigner writes a want only once that
composition exists (ADR 0539 clauses 1, 6, 14 and 16).
2026-10-24 16:29:24 +02:00
jschoubben 826fd73cf2 Give a composition its value, its canonical digest and its files by digest; key a dotted module escaped (issue 1943, S2; issue 1150)
A node applies one module's composition at a time, so each carries only its own module's resources, checked
resource by resource as the node-engine reads them, and its digest follows its content alone (ADR 0539 clauses 1
and 4). The fixtures are the controller's plan answers for ace and g14, their sealed resources taken out.
2026-10-24 16:26:32 +02:00
jschoubben aee971c198 Name a node's composition bucket, its keys, and the digest with the move's judgement beside it (issue 1943, S1)
The controller writes one module's composition per node and fingerprint, and the assigner reads its digest and
judgement before it writes a want (ADR 0539 clauses 1, 2 and 14).
2026-10-24 16:21:02 +02:00
jschoubben bfc3883af9 Read a manifest's fallback to a predefined key in the composer as the indexer reads it, or against the list a request or -catalogue gives: a run never holds the catalogue's predefined.json, and the composition refused pacman and xterm for it (ADR 0348 §7)
mesh/delivery delivered
mesh/delivery-group group feat/r4g-finish merged: every member is merged
mesh/check checking bfc3883af980 on 8c6e18e82286 | novox/mesh-sdk#77 in group feat/r4g-finish
2026-10-24 14:57:10 +02:00
jschoubben 19c1d35939 Name the pinned mesh-sdk version when an SDK command is built from a reader's tree, not a bare (devel): the compose job builds mesh-compose there and its verdict must name the reader's SDK (ADR 0482 clause 5)
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: the composition: the manifest of pacman cannot be read: this manifest cannot be used: | 19c1d35939bc on 8c6e18e82286 | unit 4799 ru…
2026-10-24 14:11:20 +02:00
jschoubben bc0c25e510 Merge main into feat/r4g-finish: the controller seat's verb held and event artifacts-gone (#76)
mesh/delivery-group group feat/r4g-finish rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: integration: meshbuildgo/cmd/mesh-build-go.TestAComposeAskIsBuiltOnceAndAnsweredOnTheBus: ask 1: answer built true key composer-066…
2026-10-24 13:25:44 +02:00
jschoubben 8c6e18e822 Merge pull request 'Give the controller seat the verb held and the event artifacts-gone (issue 1937)' (#76) from fix/1937-done-key-blobs into main
Reviewed-on: #76
2026-10-24 10:35:07 +00:00
jschoubben d9fa5536a8 Mark the packages whose tests read files outside their directory, found by the review (issue 1935) 2026-10-24 04:38:58 +02:00
jschoubben dd0d156b9e Wait for the third run to start, not only for its take, so the capacity test does not race under a loaded suite 2026-10-24 04:25:12 +02:00
jschoubben 51bc6a65b6 Merge remote-tracking branch 'origin/feat/r4g-finish-1935' into join/r4g-finish-sdk 2026-10-24 04:23:53 +02:00
jschoubben 66bf26c0d7 Let the composer read its request from a file, since a run step hands it no standard input (issue 1622) 2026-10-24 04:10:31 +02:00
jschoubben 34f210e5eb Give the controller seat the verb held and the event artifacts-gone, so the builder rebuilds a done key whose blob the artifact store no longer holds (issue 1937)
mesh/delivery covered after its stop: a later walk that contains it was delivered: plan-1792875397682979170 (novox/mesh-sdk at 312878be into main, done, …
mesh/delivery-group group fix/1937-done-key-blobs closed: a member was closed before the group merged
mesh/check pass 34f210e5eba1 on e1f393bbb12e | unit 202 run, 3523 kept | integration 2223 run, 2682 kept, 7 skipped | over bound: unit, integration (s…
The store's nightly collector deletes every bare blob no manifest names, and a done key the controller never took in holds
nothing: node-tools, mesh-host and the forge bridge kept done keys naming blobs that answer 404. held lets the builder ask
before it reuses a key; artifacts-gone is said when a take-in finds a gone blob. delivery/held.go is the shared contract.
2026-10-24 04:07:22 +02:00
jschoubben 4e0036cc95 Mark the packages whose tests read files outside their directory, so mesh-build-go never keeps their verdict (issue 1935) 2026-10-24 04:00:48 +02:00
jschoubben 4784762a70 Name every repository's siblings in beside's comment, mesh-controller beside mesh-host's tests, as the checker places them (issue 1295) 2026-10-24 03:51:35 +02:00
jschoubben 668fc2a722 Let the Go build seat carry a compose ask and its answer, so the assigner composes a group with its mesh-sdk head (issue 1622) 2026-10-24 03:51:35 +02:00
jschoubben 20d18eb4f5 Give mesh-sdk a composer command, so a program linking one mesh-sdk can have another commit's composition judged (issue 1621) 2026-10-24 03:51:35 +02:00
jschoubben e1f393bbb1 Merge pull request 'Raise @novox/mesh-sdk to 0.1.17, and a unit test refuses a packed change that keeps its version (issue 1899, met twice)' (#75) from fix/1899-version-guard into main
Reviewed-on: #75
2026-10-23 22:35:42 +00:00
jschoubben 35e86b00d3 Raise @novox/mesh-sdk to 0.1.17 and guard that a changed package carries a new version (issue 1899)
mesh/delivery delivered
mesh/check checking 35e86b00d3f0 on e1f393bbb12e
2026-10-24 00:15:27 +02:00
jschoubben 53a1de2aaa Merge pull request 'A level's digest is stored as a list, and an end the state refuses is said, so a check of a repository with a package named token ends (issue 1930, with mesh-catalog and mesh-delivery)' (#74) from fix/1930-digest-shape into main
Reviewed-on: #74
2026-10-23 22:08:06 +00:00
jschoubben 403a5d72ab Keep a level's digest as a list and say an end the state refuses, so a check of a repository with a package named token ends (issue 1930)
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1792794944815912745 (novox/mesh-sdk at e1f393bb into main, don…
mesh/delivery-group group fix/1930-digest-shape merged: every member is merged
mesh/check pass 403a5d72ab93 on 1ff6d339780c | unit 4545 run, 2097 kept, 13 skipped | integration 4595 run, 2188 kept, 21 skipped | over bound: unit, …
A check ask's answer kept the digest's packages as a map keyed by Go import path;
the node runtime refuses a state value with a field ending in a credential word,
so the end of any check of mesh-controller (internal/token) was never written and
the ask was acked anyway. The level's digest is now delivery.LevelDigest, packages
a sorted list of {package, digest}, reading the old map form too. finish logs an
end refused every time, writes a small failed end naming the write's error, and
the puller gives the ask back instead of acking it when no end could be written.
2026-10-23 23:32:00 +02:00
jschoubben 1ff6d33978 Merge pull request 'R4g+R5 (mesh-sdk): asks on the build seats' streams, the assigner's layouts, mesh-pack (design 47, ADR 0524-0530, 0537)' (#73) from feat/r4g into main
Reviewed-on: #73
2026-10-23 17:48:34 +00:00
jschoubben a2c813a6b4 Stand in node-tools beside the toolchain modules in the composition test, as the catalogue's build modules require it since issue 1911, so the test judges the toolchain alone (issue 1887, group check of #73)
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1792794944815912745 (novox/mesh-sdk at e1f393bb into main, don…
mesh/delivery-group group feat/r4g merged: every member is merged
mesh/check checking a2c813a6b468 on 07462c4caa12 | novox/mesh-sdk#73 in group feat/r4g
2026-10-23 17:03:21 +02:00
jschoubben a9560890ee Name the unit level's lint time in a verdict's times, so the check's line can say what gofmt and go vet cost (issue 1887 D7-go, review round 2 item 5)
mesh/delivery-group group feat/r4g rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: g14: fetch of novox/mesh-catalog#464: build_go_fetch on g14: the step g14.mvl3o921790ccf failed: go mod verify in checks/config: ch…
2026-10-23 16:50:22 +02:00
jschoubben bec9585d50 Run an ask no more once its stream delivered it past AskRuns, failing its step with why, so a run that kills its node is not offered to each build node in turn (issue 1907) 2026-10-23 16:41:12 +02:00
jschoubben fc4f57c4b3 Restart an ask's bound at each run's start, also on the same node, so a step redelivered after its build module was replaced is not cut by the killed run's time (issue 1887, review: issue 734 path; ADR 0525 §6) 2026-10-23 16:36:46 +02:00
jschoubben 88fa74ce5e Bound an ask's step from its start on the node that took it, not from its put, so an ask waiting in its stream during a large rebuild is never failed and asked twice (issue 1887, review round 2 item 1; ADR 0525 §3, §6) 2026-10-23 16:32:21 +02:00
jschoubben 9205222a98 Merge branch 'feat/r4g-fallback' into feat/r4g 2026-10-23 16:19:25 +02:00
jschoubben 285566eed1 Merge branch 'feat/r4g-scripts' into feat/r4g 2026-10-23 16:19:25 +02:00
jschoubben 51756ee9a4 Merge branch 'feat/r4g-asks' into feat/r4g 2026-10-23 16:19:24 +02:00
jschoubben d7375c9811 Add mesh-pack: a hand build with the build module's recipe and mesh-sdk's own pack (F1) 2026-10-23 16:12:23 +02:00
jschoubben 232ded26ea Delete both merge-check scripts; the type check runs before npm test (issue 1295, S8; ADR 0529)
gofmt and vet are the Go build module's, the unit bound the checker's.
2026-10-23 16:10:46 +02:00
jschoubben 691c97ec31 Ask a check's digest as the candidate under the runner and plan as any other when the installed module ignores it, as ADR 0535 §2 and §4 judge (issue 1887 A5a) 2026-10-23 16:04:01 +02:00
jschoubben ba092d543f Run the levels of one check in one ask after one fetch, keeping nothing under the candidate's runner or a digest without the siblings, as ADR 0525 §3 says a check ask is (issue 1887 A5a) 2026-10-23 15:59:15 +02:00
jschoubben c3915ffb11 Merge branch 'feat/r4g-assigner' into feat/r4g 2026-10-23 15:58:11 +02:00
jschoubben 34338df950 Key a building key's asks by build seat language, so a builder taking it over knows whose steps to watch (issue 1887 A4b) 2026-10-23 15:44:57 +02:00
jschoubben 07462c4caa Merge pull request 'Checks under 5 minutes (mesh-sdk): a kept mirror per repository for build steps' fetches, and @novox/mesh-sdk 0.1.16 (issues 890, 1899; ADR 0536)' (#72) from feat/fastcheck into main
Reviewed-on: #72
2026-10-23 13:44:13 +00:00
jschoubben 6003da0a58 Put an ask on its build seat's stream and read its step, and name a building key's asks, so the builder and the checker ask no node and a restarted builder never asks twice (issue 1887 A4a, A4b) 2026-10-23 15:43:59 +02:00
jschoubben 3e5d4c30ff Let a composition not run say why, the assigner not running among it, and name the indexer's words for a removed module, so a check never waits for a composition nobody makes (issue 1297, B3d) 2026-10-23 15:41:55 +02:00
jschoubben 1672f40233 Serve a build seat's current, pause and resume from the SDK, naming the asks running, so the three build modules hold their seats one way (issue 1887 A3node, A9a, ADR 0525 §10) 2026-10-23 15:38:03 +02:00
jschoubben e1dca0a39f Release an ask's workspace when its job ends, so a pulled build or check leaves nothing for the sweep (issue 1887 A3go, ADR 0509) 2026-10-23 15:30:32 +02:00
jschoubben 4b503f8d0d Pull a build seat's ask through the bundle's runtime one at a time and settle it by id, so a tools bundle pulls by its capacity and order (issue 1887 A11) 2026-10-23 15:29:05 +02:00
jschoubben 8cd22892a3 Read a node's build capacity from the build module's own setting at every pull, defaulting to the builder's values, so the control-node pulls nothing (issue 1887 A7, ADR 0525 §5) 2026-10-23 15:23:03 +02:00
jschoubben 1f6fc027c0 Run a pulled ask's whole job with a build module's own step tools, kept verdicts taken after its digest, so both build modules run asks one way and their tools stay the hand path (issue 1887 A3, ADR 0525 §3, §8) 2026-10-23 15:22:25 +02:00
jschoubben f625d98713 Lay out the assigner's ask value and name the asking a view answer is for, so a stale answer never fills a later asking of the same key (issue 1297 subtask B1c, for the view lane's B9 and B10) 2026-10-23 15:20:58 +02:00
jschoubben c2967432e6 Refuse the build queue's verbs by name and drop the rows R4e removed, so a caller is told the backlog is the delivery view's and a pause the build seat's (issue 1887 A9b, ADR 0525 §9) 2026-10-23 15:20:27 +02:00
jschoubben 69041c5302 Retire the seat node-build-agent for one build seat per language, so a node's builds are pulled and paused at its build modules and a claim on the old seat names its replacement (issue 1887 A9a, ADR 0525 §2, §10) 2026-10-23 15:19:41 +02:00
jschoubben 792fb2ccfd Keep a long ask alive with in-progress signals and leave a stopped holder's ask to the stream, so a node that dies hands its ask on (issue 1887 A2c, ADR 0525 §6) 2026-10-23 15:17:18 +02:00
jschoubben 89aa6fc17c Pull nothing at capacity 0, take a new capacity at the next pull, and keep a node's pause on its pause key, so a paused node stays paused across a restart (issue 1887 A2b) 2026-10-23 15:16:46 +02:00
jschoubben a1d73bdcf0 Pull a build seat's asks delivery first, at most the node's capacity, each ended on its own step key, so no part chooses a node or holds a check (issue 1887 A2a, ADR 0525 §4-§6) 2026-10-23 15:15:57 +02:00
jschoubben d9384bcb23 Lay out the asks the view may answer and the view's answer key, refusing the merge's ask, so the view and the teller answer only what needs no proof (issue 1297 subtask B1c; ADR 0528 clause 6, ADR 0437 clause 10) 2026-10-23 15:12:40 +02:00
jschoubben b9b9c5120c Name an ask's step key <node>.<ask id> and its watch, so the asker learns which node took it and refuses other keys (issue 1887 A1b, ADR 0525 §6) 2026-10-23 15:11:44 +02:00
jschoubben 37eff6fe5c Lay out the ask a build node pulls: its subjects, stream, workers and job, so builder, checker and build modules share one form (issue 1887 A1a, ADR 0525, ADR 0536 §4) 2026-10-23 15:11:39 +02:00
jschoubben 687d715e99 Lay out the controller's grants and the assigner's composition verdict under the checker's key, so the assigner's rule 6 and the checker's composition read one layout (issue 1297 subtask B1b, issue 1394; ADR 0528 clauses 2 and 5) 2026-10-23 15:11:34 +02:00
jschoubben ec2bd3b990 Lay out the assigner's wants, holds and differences in mesh-sdk, so the assigner, the watchman and the view read one layout (issue 1297, subtask B1a; ADR 0528 clauses 1 and 4) 2026-10-23 15:10:09 +02:00
jschoubben 2248b985bf Raise @novox/mesh-sdk to 0.1.16, so its package build is not refused as a second, different 0.1.15 (issue 1899)
mesh/delivery delivered
mesh/delivery-group group feat/fastcheck merged: every member is merged
mesh/check pass 2248b985bfac on 7aacbc162014 | unit 6632 run, 0 kept, 16 skipped | integration 6757 run, 0 kept, 24 skipped | over bound: unit, integr…
2026-10-23 14:15:26 +02:00
jschoubben 780be2fae5 Fetch through a bare mirror with git alternates, so a check transfers only new objects (issue 890, ADR 0536 §2) 2026-10-23 14:03:06 +02:00
jschoubben 5ab7ccf47e Merge pull request 'A build step is refused under a 20 GiB free-space floor, and a workspace is held, released and swept after 2 h (issue 1839, ADR 0532)' (#71) from fix/1839-build-floor into main
Reviewed-on: #71
2026-10-23 11:28:31 +00:00
jschoubben 6147a20454 Refuse a build step under a free-space floor and release workspaces, so a build node's disk is never filled (issue 1839)
mesh/merge-gate head 6147a2045467 merged into trunk 7aacbc162014 — pass: builds mesh-sdk (+41 dependent(s)) → ace, g14, novox, shanks; no bus step; 9 wait(…
mesh/repo-check head 6147a2045467 merged into trunk 7aacbc162014 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (typescript)…
mesh/delivery delivered
mesh/delivery-group group fix/1839-build-floor merged: every member is merged
mesh/check failed: integration: meshbuildnode/cmd/mesh-build-node.TestAFixtureBundleBuiltTwiceHasOneDigest: build_integration_test.go:71: no toolchain…
buildstep.Floor refuses a step, naming the node, what is free and the floor
(20 GiB, build-agent's), while the work or cache filesystem is under it.
buildstep.Workspaces holds a workspace while a step runs on it, removes it
when its caller releases it, and sweeps one left unheld past ADR 0509's
two-hour bound.
2026-10-23 08:28:28 +02:00
mesh-tools 7aacbc1620 Merge pull request 'R4e (mesh-sdk): the narrowed tree, the build modules' shared steps and layouts, ParseManifestForInputs, and the vendor directory's leaving' (#70) from feat/r4e into main 2026-10-22 22:13:10 +00:00
jschoubben 22b70c82b0 Stand in the artifact store in the toolchain test's shelf: R4e's build modules require it (ADR 0513), and the group's composed check judges them at the catalogue's group head
mesh/repo-check head 22b70c82b0ab merged into trunk cdfc63cf102b beside mesh-catalog#457@896bd883 +1 more — pass: THE CHANGE ALTERS ITS OWN CHECK (go/merge…
mesh/merge-gate head 22b70c82b0ab merged into trunk cdfc63cf102b — pass: group feat/r4e composed: every machine composes with the change as it did without …
mesh/delivery covered after its stop: a later walk that contains it was delivered: plan-1792707174627079192 (novox/mesh-sdk at 7aacbc16 into main, done, …
mesh/delivery-group group feat/r4e closed: a member was closed before the group merged
mesh/check failed: unit: ..test: node:internal/modules/esm/resolve:989 | 22b70c82b0ab on 7aacbc162014 | unit 1675 run, 0 kept, 3 skipped | integration…
2026-10-23 00:03:06 +02:00
jschoubben 6aa8e869a3 Name no paths in mesh-sdk's own manifest until R4e is delivered: today's controller judges R4e's group and refuses the word (issue 1758, ADR 0499 §7, ADR 0503 §4)
mesh/merge-gate waits for its group's composed check
mesh/repo-check head 6aa8e869a3b6 merged into trunk cdfc63cf102b beside mesh-catalog#457@896bd883 +1 more — fail: THE CHANGE ALTERS ITS OWN CHECK (go/merge…
mesh/delivery-group group feat/r4e rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: ace: novox/mesh-catalog#457: modules/gitea/go.mod names mesh-sdk 104533e8c0c9, which is neither on mesh-sdk's trunk nor the group's…
2026-10-22 23:57:26 +02:00
131 changed files with 17485 additions and 580 deletions
+400
View File
@@ -0,0 +1,400 @@
package buildstep
import (
"context"
"encoding/json"
"fmt"
"path"
"slices"
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// **One ask, one whole job** (novox/hq ADR 0525 §3). A build module runs a pulled ask with its own step tools' bodies,
// the same ones its request and reply tools run (ADR 0525 §8), in the order the job names: a build ask fetches, then
// builds or copies each artifact; a check ask fetches, digests the level, takes the kept verdicts and tests the rest,
// the packages whose tests the change touches first. The answers are the tools' own, whole, so the asker judges them as
// it judged the tools' answers.
// StepFunc is one step tool's body: its arguments in, its answer out.
type StepFunc func(ctx context.Context, args map[string]any) (any, error)
// AskSteps are the step tools a build module runs an ask with. Copy is the image module's alone; Digest the Go module's.
type AskSteps struct {
Fetch, Build, Copy, Digest, Test StepFunc
// Toolchain is the toolchain the module builds with now; a build ask naming another is refused. Nil checks nothing
// (an image or an archive names none).
Toolchain func(ctx context.Context) (delivery.ToolchainUsed, error)
// Kept looks up a kept verdict by level and digest (the checker's kept bucket, delivery.KeptKey): whether one is
// kept and how many tests it counted. Nil takes none.
Kept func(ctx context.Context, level, digest string) (tests int, found bool, err error)
// Release removes a workspace once the ask's steps on it ended (the module's release tool's body); nil leaves it to
// the module's sweep.
Release func(workspace string) error
Logf func(format string, args ...any)
}
// RunAsk runs ask's job with steps and answers a delivery.BuildAnswer or a delivery.CheckAnswer.
func RunAsk(ctx context.Context, steps AskSteps, ask delivery.StepAsk) (any, error) {
if ask.Build != nil {
return steps.build(ctx, *ask.Build)
}
if ask.Check != nil {
return steps.check(ctx, *ask.Check)
}
return nil, Refuse("the ask %s holds no job", ask.ID)
}
func raw(v any) (json.RawMessage, error) {
if r, ok := v.(json.RawMessage); ok {
return r, nil
}
return json.Marshal(v)
}
// fetched is what an ask reads of a fetch's answer.
type fetched struct {
Workspace string `json:"workspace"`
Merge string `json:"merge"`
Conflicts []string `json:"conflicts"`
Changed []string `json:"changed"`
Modules []string `json:"modules"`
}
func (s AskSteps) fetch(ctx context.Context, args map[string]any) (json.RawMessage, fetched, error) {
if s.Fetch == nil {
return nil, fetched{}, Refuse("this build module fetches nothing")
}
out, err := s.Fetch(ctx, args)
if err != nil {
return nil, fetched{}, err
}
r, err := raw(out)
if err != nil {
return nil, fetched{}, err
}
var f fetched
if err := json.Unmarshal(r, &f); err != nil || f.Workspace == "" {
return r, f, fmt.Errorf("the fetch answered no workspace: %s", r)
}
return r, f, nil
}
func (s AskSteps) build(ctx context.Context, job delivery.BuildJob) (any, error) {
answer := delivery.BuildAnswer{}
if job.Toolchain != nil && s.Toolchain != nil {
tc, err := s.Toolchain(ctx)
if err != nil {
return nil, err
}
if tc.Version != job.Toolchain.Version || tc.Recipe != job.Toolchain.Recipe {
return nil, Refuse("the ask is for %s (recipe %s) and this node builds with %s (recipe %s)",
job.Toolchain.Version, job.Toolchain.Recipe, tc.Version, tc.Recipe)
}
answer.Toolchain = &tc
}
var f fetched
defer func() { s.release(f.Workspace) }()
for _, a := range job.Artifacts {
if a.From == "" && f.Workspace == "" {
r, got, err := s.fetch(ctx, job.Fetch)
if err != nil {
return nil, err
}
if _, candidate := job.Fetch["base"]; candidate {
// A candidate is built from its head merged onto the trunk; a head that does not merge leaves the tree at the
// trunk, and building that would store the trunk's build under the candidate's fingerprint.
if len(got.Conflicts) > 0 {
return nil, fmt.Errorf("the head does not merge into the trunk: files in conflict: %s", strings.Join(got.Conflicts, ", "))
}
if got.Merge == "" {
return nil, fmt.Errorf("the fetch with a base answered no merge: %s", r)
}
}
f = got
}
var (
out any
err error
)
if a.From != "" {
if s.Copy == nil {
return nil, Refuse("this build module copies no artifact (%s)", a.Artifact)
}
out, err = s.Copy(ctx, map[string]any{"module": job.Module, "artifact": a.Artifact, "from": a.From})
} else {
if s.Build == nil {
return nil, Refuse("this build module builds no artifact (%s)", a.Artifact)
}
args := map[string]any{"workspace": f.Workspace, "module": job.Path, "artifact": a.Artifact}
if job.Publish {
args["publish"] = true
}
out, err = s.Build(ctx, args)
}
if err != nil {
return nil, fmt.Errorf("%s: %w", a.Artifact, err)
}
r, err := raw(out)
if err != nil {
return nil, err
}
var built struct {
Digest string `json:"digest"`
Reference string `json:"reference"`
}
if err := json.Unmarshal(r, &built); err != nil || built.Digest == "" {
return nil, fmt.Errorf("the build of %s answered no digest: %s", a.Artifact, r)
}
answer.Artifacts = append(answer.Artifacts, delivery.AskBuilt{Artifact: a.Artifact, Digest: built.Digest, Reference: built.Reference})
}
return answer, nil
}
func (s AskSteps) check(ctx context.Context, job delivery.CheckJob) (any, error) {
r, f, err := s.fetch(ctx, job.Fetch)
if err != nil {
return nil, err
}
defer s.release(f.Workspace)
answer := delivery.CheckAnswer{Fetch: r}
if len(f.Conflicts) > 0 {
return answer, nil // the asker reads the conflict from the fetch's answer: a verdict, not an error
}
if s.Test == nil {
return nil, Refuse("this build module tests nothing")
}
runner := ""
if job.Runner != "" && (job.RunnerWhen == "" || changesUnder(f.Changed, job.RunnerWhen)) {
runner = job.Runner
}
for _, level := range job.Levels {
la := delivery.LevelAnswer{Level: level}
passed, err := s.level(ctx, job, level, f, runner, &la)
answer.Levels = append(answer.Levels, la)
if err != nil {
return nil, err
}
if !passed {
return answer, nil // the first level that did not pass ends the check (ADR 0484 clause 13)
}
}
return answer, nil
}
// level runs one level of a check ask and says whether it passed.
func (s AskSteps) level(ctx context.Context, job delivery.CheckJob, level string, f fetched, runner string, la *delivery.LevelAnswer) (bool, error) {
withRunner := func(args map[string]any) map[string]any {
if runner != "" {
args["runner"] = runner
}
return args
}
test := func(args map[string]any) (bool, error) {
out, err := s.Test(ctx, args)
if err != nil {
return false, err
}
r, err := raw(out)
if err != nil {
return false, err
}
la.Tests = append(la.Tests, r)
var t struct {
Passed bool `json:"passed"`
}
_ = json.Unmarshal(r, &t)
return t.Passed, nil
}
// Every test step of the level runs, also after one that did not pass, so one check names every failing test of the
// level and not only the first (issue 1921); the level as a whole passes only when each of its steps did.
all := true
if s.Digest == nil {
// A module with no digest (Node) tests each module directory of the tree, those whose tests changed first.
for _, priority := range []bool{true, false} {
for _, dir := range f.Modules {
if changesTestIn(f.Changed, dir) != priority {
continue
}
passed, err := test(withRunner(map[string]any{"workspace": f.Workspace, "module": dir, "level": level}))
if err != nil {
return false, fmt.Errorf("the %s level of %s: %w", level, dir, err)
}
all = all && passed
}
}
return all, nil
}
todo, whole, keepRunner := s.plan(ctx, job, level, f, runner, withRunner, la)
if !keepRunner {
runner = "" // an installed module that ignores the runner judges the level as it judges any other (ADR 0535 §2)
}
for _, priority := range []bool{true, false} {
args := withRunner(map[string]any{"workspace": f.Workspace, "level": level})
if job.Run != "" {
args["run"] = job.Run
}
if whole {
if priority {
continue // nothing to put first: the level runs whole after
}
} else {
var run []string
for _, p := range todo {
if changesTestOf(f.Changed, p) == priority {
run = append(run, p)
}
}
if len(run) == 0 {
continue
}
args["packages"] = run
}
passed, err := test(args)
if err != nil {
return false, fmt.Errorf("the %s level: %w", level, err)
}
all = all && passed // the packages left still run (issue 1921)
}
return all, nil
}
// release removes an ask's workspace, saying a failure: the module's sweep removes it later.
func (s AskSteps) release(workspace string) {
if workspace == "" || s.Release == nil {
return
}
if err := s.Release(workspace); err != nil && s.Logf != nil {
s.Logf("the workspace %s is left to the sweep: %v", workspace, err)
}
}
// plan digests the level and takes the kept verdicts: the packages left to test, or all when the level runs whole, and
// whether the runner stays. A try of some tests runs whole, digesting nothing (a partial run proves no package). Under the
// candidate's runner the digest is asked as the candidate: when the candidate answers it, every package runs and nothing
// is digested for keeping (ADR 0535 §4); when the installed module answers it (it does not take the runner), the level is
// planned as any other and the runner is dropped. The level runs whole and takes nothing when the digest names no package
// or does not cover the siblings the fetch placed (ADR 0444, L5).
func (s AskSteps) plan(ctx context.Context, job delivery.CheckJob, level string, f fetched, runner string, withRunner func(map[string]any) map[string]any, la *delivery.LevelAnswer) ([]string, bool, bool) {
if job.Run != "" {
return nil, true, true
}
out, err := s.Digest(ctx, withRunner(map[string]any{"workspace": f.Workspace, "level": level}))
if err != nil {
if s.Logf != nil {
s.Logf("no digest of the %s level, so it runs whole: %v", level, err)
}
return nil, true, true
}
r, err := raw(out)
if err != nil {
return nil, true, true
}
var d delivery.LevelDigest // packages kept as a sorted list, never keyed by a package path (issue 1930)
if err := json.Unmarshal(r, &d); err != nil {
return nil, true, true
}
if runner != "" {
if strings.HasPrefix(d.JudgedBy, "candidate ") {
la.Judged = d.JudgedBy
return nil, true, true // the candidate decides which verdicts would be taken: all run, nothing kept
}
runner = ""
}
if len(d.Packages) == 0 || !slices.Equal(d.Beside, besideOf(job.Fetch)) {
return nil, true, runner != "" // a digest that names nothing or not the siblings keys no verdict: the level runs whole
}
la.Digest = &d
var todo []string
for _, pd := range d.Packages {
p := pd.Package
if job.Kept && s.Kept != nil {
tests, found, err := s.Kept(ctx, level, pd.Digest)
if err != nil && s.Logf != nil {
s.Logf("the kept verdict of %s at %s cannot be read, so it runs: %v", p, level, err)
}
if err == nil && found {
la.Kept = append(la.Kept, delivery.KeptTaken{Package: p, Digest: pd.Digest, Tests: tests})
continue
}
}
todo = append(todo, p)
}
return todo, false, false
}
// besideOf are the siblings a fetch's arguments place, as a digest echoes them: `<owner/repo> <commit>`, sorted.
func besideOf(fetch map[string]any) []string {
raw, err := json.Marshal(fetch["beside"])
if err != nil {
return nil
}
var refs []struct {
Repository string `json:"repository"`
Commit string `json:"commit"`
}
_ = json.Unmarshal(raw, &refs)
var out []string
for _, r := range refs {
out = append(out, r.Repository+" "+r.Commit)
}
sort.Strings(out)
return out
}
// changesUnder says whether a path the merge changes is under prefix.
func changesUnder(changed []string, prefix string) bool {
for _, p := range changed {
if strings.HasPrefix(p, prefix) {
return true
}
}
return false
}
// isTestFile says whether a repository-relative path is a test file: Go's _test.go, and a TypeScript module's .test.
// and .spec. files and test directories.
func isTestFile(p string) bool {
switch {
case strings.HasSuffix(p, "_test.go"):
return true
case strings.Contains(p, ".test.") || strings.Contains(p, ".spec."):
return true
}
for _, seg := range strings.Split(path.Dir(p), "/") {
if seg == "__tests__" || seg == "test" || seg == "tests" {
return true
}
}
return false
}
// changesTestOf says whether the merge changes a test file of the Go package with the import path pkg: the file's
// directory and the import path agree on their last segments.
func changesTestOf(changed []string, pkg string) bool {
for _, f := range changed {
if !strings.HasSuffix(f, "_test.go") {
continue
}
dir := path.Dir(f)
if dir == pkg || strings.HasSuffix(pkg, "/"+dir) || strings.HasSuffix(dir, "/"+pkg) {
return true
}
}
return false
}
// changesTestIn says whether the merge changes a test file inside the directory of a Node module.
func changesTestIn(changed []string, dir string) bool {
for _, f := range changed {
if !isTestFile(f) || strings.HasSuffix(f, "_test.go") {
continue
}
if dir == "." || dir == "" || strings.HasPrefix(f, strings.TrimSuffix(dir, "/")+"/") {
return true
}
}
return false
}
+245
View File
@@ -0,0 +1,245 @@
package buildstep
import (
"context"
"encoding/json"
"errors"
"fmt"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
const runCommit = "0123456789abcdef0123456789abcdef01234567"
// recorder is a build module's step tools that record each call.
type recorder struct {
calls []string
fetch map[string]any
tests map[string]bool // package or module directory -> passed
digests map[string]string
}
func (r *recorder) steps() AskSteps {
call := func(name string, answer func(args map[string]any) any) StepFunc {
return func(ctx context.Context, args map[string]any) (any, error) {
b, _ := json.Marshal(args)
r.calls = append(r.calls, name+" "+string(b))
return answer(args), nil
}
}
return AskSteps{
Fetch: call("fetch", func(map[string]any) any { return r.fetch }),
Build: call("build", func(args map[string]any) any {
return map[string]any{"digest": "sha256:" + args["artifact"].(string), "reference": "artifact-store://" + args["artifact"].(string)}
}),
Copy: call("copy", func(args map[string]any) any { return map[string]any{"digest": "sha256:copied"} }),
Digest: call("digest", func(map[string]any) any { return map[string]any{"level": "unit", "packages": r.digests} }),
Test: call("test", func(args map[string]any) any {
passed := true
if pkgs, ok := args["packages"].([]string); ok {
for _, p := range pkgs {
passed = passed && r.tests[p]
}
}
if dir, ok := args["module"].(string); ok {
passed = r.tests[dir]
}
return map[string]any{"passed": passed, "tests": 1}
}),
Kept: func(ctx context.Context, level, digest string) (int, bool, error) { return 7, digest == "d-kept", nil },
}
}
func ask(job any) delivery.StepAsk {
a := delivery.StepAsk{ID: NewID(), Kind: delivery.AskCheck, Language: delivery.AskGo, Asker: "t", At: time.Now()}
switch j := job.(type) {
case delivery.BuildJob:
a.Build = &j
case delivery.CheckJob:
a.Check = &j
}
return a
}
// 1887/A3: a build ask fetches once, builds each artifact from the workspace with publish, copies a pinned one without a
// fetch, and answers each artifact's digest and reference.
func TestRunAskBuilds(t *testing.T) {
r := &recorder{fetch: map[string]any{"workspace": "/w/1"}}
steps := r.steps()
var released []string
steps.Release = func(ws string) error { released = append(released, ws); return nil }
out, err := RunAsk(context.Background(), steps, ask(delivery.BuildJob{Module: "m", Path: "modules/m", Publish: true,
Fetch: map[string]any{"repository": "novox/x", "commit": runCommit},
Artifacts: []delivery.AskArtifact{{Artifact: "base", From: "registry/b@sha256:00"}, {Artifact: "bundle"}, {Artifact: "lib"}}}))
if err != nil {
t.Fatal(err)
}
got := strings.Join(r.calls, "\n")
want := `copy {"artifact":"base","from":"registry/b@sha256:00","module":"m"}
fetch {"commit":"` + runCommit + `","repository":"novox/x"}
build {"artifact":"bundle","module":"modules/m","publish":true,"workspace":"/w/1"}
build {"artifact":"lib","module":"modules/m","publish":true,"workspace":"/w/1"}`
if got != want {
t.Fatalf("calls:\n%s\nwant:\n%s", got, want)
}
if len(released) != 1 || released[0] != "/w/1" {
t.Errorf("released %v, want the one workspace", released)
}
a := out.(delivery.BuildAnswer)
if len(a.Artifacts) != 3 || a.Artifacts[1].Reference != "artifact-store://bundle" {
t.Fatalf("answer %+v", a)
}
}
// A build ask for another toolchain is refused, so its step ends refused and the asker reads why.
func TestRunAskRefusesAnotherToolchain(t *testing.T) {
r := &recorder{fetch: map[string]any{"workspace": "/w"}}
s := r.steps()
s.Toolchain = func(context.Context) (delivery.ToolchainUsed, error) {
return delivery.ToolchainUsed{Version: "go1.27", Recipe: "3"}, nil
}
_, err := RunAsk(context.Background(), s, ask(delivery.BuildJob{Module: "m", Path: ".", Fetch: map[string]any{"repository": "novox/x", "commit": runCommit},
Artifacts: []delivery.AskArtifact{{Artifact: "bundle"}}, Toolchain: &delivery.ToolchainUsed{Version: "go1.26", Recipe: "3"}}))
var ref *Refusal
if !errors.As(err, &ref) || len(r.calls) != 0 {
t.Fatalf("err %v, calls %v", err, r.calls)
}
}
// 1887/A3, A5a: a check ask runs the levels of one check after one fetch: it digests the unit level, takes the kept
// verdicts, tests the packages whose tests changed first, then the rest also after a failure (issue 1921), and runs no
// level after one that did not pass (ADR 0484 clause 13).
func TestRunAskChecksTheLevelsOfOneCheck(t *testing.T) {
r := &recorder{
fetch: map[string]any{"workspace": "/w", "changed": []string{"pkg/b/b_test.go"}},
digests: map[string]string{"x/pkg/a": "d-a", "x/pkg/b": "d-b", "x/pkg/k": "d-kept"},
tests: map[string]bool{"x/pkg/a": true, "x/pkg/b": false},
}
out, err := RunAsk(context.Background(), r.steps(), ask(delivery.CheckJob{Levels: []string{delivery.LevelUnit, delivery.LevelIntegration}, Kept: true,
Runner: "workspace", RunnerWhen: "modules/mesh-build-go/", Fetch: map[string]any{"repository": "novox/x", "commit": runCommit, "base": runCommit}}))
if err != nil {
t.Fatal(err)
}
got := strings.Join(r.calls, "\n")
want := `fetch {"base":"` + runCommit + `","commit":"` + runCommit + `","repository":"novox/x"}
digest {"level":"unit","workspace":"/w"}
test {"level":"unit","packages":["x/pkg/b"],"workspace":"/w"}
test {"level":"unit","packages":["x/pkg/a"],"workspace":"/w"}`
if got != want {
t.Fatalf("calls:\n%s\nwant:\n%s", got, want)
}
a := out.(delivery.CheckAnswer)
if len(a.Levels) != 1 {
t.Fatalf("levels %+v, want the unit level alone", a.Levels)
}
u := a.Levels[0]
if u.Level != delivery.LevelUnit || len(u.Kept) != 1 || u.Kept[0].Package != "x/pkg/k" || u.Kept[0].Tests != 7 || len(u.Tests) != 2 || u.Digest == nil {
t.Fatalf("the unit level %+v", u)
}
}
// ADR 0535 §2, §4: a merge that changes the build module's own code is judged by the candidate's runner: its digest says
// the candidate ran, and every package runs with nothing kept; an installed module that ignores the runner is planned as
// any other, the runner dropped. A digest that does not cover the fetch's siblings runs the level whole too.
func TestRunAskRunsWholeUnderTheRunnerOrWithoutTheSiblings(t *testing.T) {
for _, tc := range []struct{ judged, want string }{
{"candidate 999", `digest {"level":"unit","runner":"workspace","workspace":"/w"}
test {"level":"unit","runner":"workspace","workspace":"/w"}`},
{"installed", `digest {"level":"unit","runner":"workspace","workspace":"/w"}`},
} {
r := &recorder{fetch: map[string]any{"workspace": "/w", "changed": []string{"modules/mesh-build-go/x.go"}},
digests: map[string]string{"x/pkg/a": "d-kept"}, tests: map[string]bool{}}
steps := r.steps()
digest := steps.Digest
steps.Digest = func(ctx context.Context, args map[string]any) (any, error) {
_, _ = digest(ctx, args)
return map[string]any{"level": "unit", "packages": r.digests, "judged_by": tc.judged}, nil
}
out, err := RunAsk(context.Background(), steps, ask(delivery.CheckJob{Levels: []string{delivery.LevelUnit}, Kept: true,
Runner: "workspace", RunnerWhen: "modules/mesh-build-go/", Fetch: map[string]any{"repository": "novox/x", "commit": runCommit}}))
if err != nil {
t.Fatal(err)
}
if got := strings.Join(r.calls[1:], "\n"); got != tc.want {
t.Errorf("judged by %s: calls\n%s\nwant\n%s", tc.judged, got, tc.want)
}
l := out.(delivery.CheckAnswer).Levels[0]
if tc.judged == "installed" && (len(l.Kept) != 1 || l.Judged != "") {
t.Errorf("an installed module's level %+v, want the kept verdict taken", l)
}
if tc.judged != "installed" && (len(l.Kept) != 0 || l.Digest != nil || l.Judged != tc.judged) {
t.Errorf("the candidate's level %+v, want nothing kept and the program named", l)
}
}
r := &recorder{fetch: map[string]any{"workspace": "/w"}, digests: map[string]string{"x/pkg/a": "d-kept"}, tests: map[string]bool{}}
out, err := RunAsk(context.Background(), r.steps(), ask(delivery.CheckJob{Levels: []string{delivery.LevelUnit}, Kept: true,
Fetch: map[string]any{"repository": "novox/x", "commit": runCommit, "beside": []any{map[string]any{"repository": "novox/mesh-sdk", "commit": runCommit}}}}))
if err != nil {
t.Fatal(err)
}
if l := out.(delivery.CheckAnswer).Levels[0]; len(l.Kept) != 0 || l.Digest != nil {
t.Fatalf("a digest without the siblings kept %+v", l)
}
}
// A Node check tests each module directory, the one whose tests changed first; a conflict ends the ask with the fetch
// alone, a verdict for the asker.
func TestRunAskChecksNodeModulesAndConflicts(t *testing.T) {
r := &recorder{fetch: map[string]any{"workspace": "/w", "modules": []string{"a", "b"}, "changed": []string{"b/test/x.test.ts"}},
tests: map[string]bool{"a": true, "b": true}}
s := r.steps()
s.Digest = nil
if _, err := RunAsk(context.Background(), s, ask(delivery.CheckJob{Levels: []string{delivery.LevelUnit}, Fetch: map[string]any{"repository": "novox/x", "commit": runCommit}})); err != nil {
t.Fatal(err)
}
if fmt.Sprint(r.calls[1:]) != `[test {"level":"unit","module":"b","workspace":"/w"} test {"level":"unit","module":"a","workspace":"/w"}]` {
t.Fatalf("calls %v", r.calls)
}
r = &recorder{fetch: map[string]any{"workspace": "/w", "conflicts": []string{"go.mod"}}}
out, err := RunAsk(context.Background(), r.steps(), ask(delivery.CheckJob{Levels: []string{delivery.LevelUnit}, Fetch: map[string]any{"repository": "novox/x", "commit": runCommit}}))
if err != nil || len(r.calls) != 1 || len(out.(delivery.CheckAnswer).Levels) != 0 {
t.Fatalf("a conflict: %v %v", err, r.calls)
}
}
// Issue 1921: a failure in the packages whose tests changed does not stop the level: the other packages still run, so two
// failing packages, one changed and one not, are two test results of the one level, and the level does not pass. A Node
// check runs every module directory after one fails too.
func TestRunAskRunsEveryPackageAfterAFailure(t *testing.T) {
r := &recorder{
fetch: map[string]any{"workspace": "/w", "changed": []string{"pkg/b/b_test.go"}},
digests: map[string]string{"x/pkg/a": "d-a", "x/pkg/b": "d-b"},
tests: map[string]bool{"x/pkg/a": false, "x/pkg/b": false},
}
out, err := RunAsk(context.Background(), r.steps(), ask(delivery.CheckJob{Levels: []string{delivery.LevelIntegration, delivery.LevelUnit},
Fetch: map[string]any{"repository": "novox/x", "commit": runCommit}}))
if err != nil {
t.Fatal(err)
}
a := out.(delivery.CheckAnswer)
if len(a.Levels) != 1 || len(a.Levels[0].Tests) != 2 {
t.Fatalf("levels %+v, want the one level with both packages' results", a.Levels)
}
for i, raw := range a.Levels[0].Tests {
var res struct{ Passed bool }
if err := json.Unmarshal(raw, &res); err != nil || res.Passed {
t.Errorf("result %d %s, want a failure", i, raw)
}
}
r = &recorder{fetch: map[string]any{"workspace": "/w", "modules": []string{"a", "b"}, "changed": []string{"b/test/x.test.ts"}},
tests: map[string]bool{"a": true, "b": false}}
s := r.steps()
s.Digest = nil
out, err = RunAsk(context.Background(), s, ask(delivery.CheckJob{Levels: []string{delivery.LevelUnit, delivery.LevelIntegration},
Fetch: map[string]any{"repository": "novox/x", "commit": runCommit}}))
if err != nil {
t.Fatal(err)
}
if a := out.(delivery.CheckAnswer); len(r.calls) != 3 || len(a.Levels) != 1 || len(a.Levels[0].Tests) != 2 {
t.Fatalf("calls %v, levels %+v: want both directories of the unit level and no integration level", r.calls, a.Levels)
}
}
+61
View File
@@ -0,0 +1,61 @@
package buildstep
import (
"encoding/json"
"fmt"
"os"
"git.novox.be/novox/mesh-sdk/go/delivery/buildrun"
)
// CapacitySetting is the build modules' setting of each node's build capacity (novox/hq ADR 0525 §5): `<node>=<asks at
// once>` pairs separated by commas, as the builder's was (ADR 0485 §1). Each build module declares it with
// DefaultCapacities as its default, and the controller lays it into the module's settings file.
const CapacitySetting = "capacity"
// DefaultCapacities is the setting's default, the builder's before R4g: the control-node (novox) and the laptop (g14)
// build nothing.
const DefaultCapacities = "shanks=3,ace=2,g14=0,novox=0"
// NodeCapacity is node's build capacity, read from the module's settings file at path at every call, so a change takes
// effect at the next pull; with no path it reads fallback. A node the setting does not name runs one ask at a time (as
// the builder's did). A file or a setting that cannot be read gives 0 and says why: pulling on a setting nobody can read
// could build on a node that must not.
func NodeCapacity(path, fallback, node string, logf func(string, ...any)) func() int {
say := func(format string, args ...any) {
if logf != nil {
logf(format, args...)
}
}
return func() int {
setting := fallback
if path != "" {
raw, err := os.ReadFile(path)
if err != nil {
say("the build capacity cannot be read, so %s pulls nothing: %v", node, err)
return 0
}
var s map[string]any
if err := json.Unmarshal(raw, &s); err != nil {
say("the settings file %s is not JSON, so %s pulls nothing: %v", path, node, err)
return 0
}
setting, _ = s[CapacitySetting].(string)
}
n, err := capacityOf(setting, node)
if err != nil {
say("%v, so %s pulls nothing", err, node)
return 0
}
return n
}
}
// capacityOf is node's capacity in a setting's `<node>=<asks>` pairs: DefaultCapacity for a node it does not name.
func capacityOf(setting, node string) (int, error) {
caps, err := buildrun.ParseCapacity(setting)
if err != nil {
return 0, fmt.Errorf("the build capacity: %w", err)
}
return (&buildrun.Places{Capacity: caps}).CapacityOf(node), nil
}
+38
View File
@@ -0,0 +1,38 @@
package buildstep
import (
"os"
"path/filepath"
"testing"
)
// 1887/A7: the node's build capacity is the build module's setting, read at every pull: the control-node at 0 pulls
// nothing, a change takes effect at the next read, and a setting that cannot be read pulls nothing.
func TestNodeCapacity(t *testing.T) {
if got := NodeCapacity("", DefaultCapacities, "novox", nil)(); got != 0 {
t.Errorf("novox = %d by default, want 0", got)
}
if got := NodeCapacity("", DefaultCapacities, "shanks", nil)(); got != 3 {
t.Errorf("shanks = %d by default, want 3", got)
}
file := filepath.Join(t.TempDir(), "settings.json")
os.WriteFile(file, []byte(`{"capacity":"shanks=1,ace=2"}`), 0o600)
capacity := NodeCapacity(file, DefaultCapacities, "shanks", nil)
if got := capacity(); got != 1 {
t.Errorf("shanks = %d from the file, want 1", got)
}
os.WriteFile(file, []byte(`{"capacity":"shanks=4"}`), 0o600)
if got := capacity(); got != 4 {
t.Errorf("shanks = %d after the change, want 4", got)
}
if got := NodeCapacity(file, "", "pi", nil)(); got != 1 {
t.Errorf("a node not named = %d, want 1", got)
}
os.WriteFile(file, []byte(`{"capacity":"shanks=lots"}`), 0o600)
if got := capacity(); got != 0 {
t.Errorf("an unreadable setting = %d, want 0", got)
}
if got := NodeCapacity(filepath.Join(t.TempDir(), "none"), DefaultCapacities, "shanks", nil)(); got != 0 {
t.Errorf("a missing file = %d, want 0", got)
}
}
+234
View File
@@ -0,0 +1,234 @@
package buildstep
import (
"context"
"encoding/json"
"errors"
"fmt"
"sort"
"strings"
"sync"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// credentialField is the node runtime's rule for a state value (mesh-tools node-tools/internal/bus/state.go, novox/hq
// ADR 0201), copied whole: the first field anywhere in the value whose name, lower-cased and without - _ and spaces, ends
// in a credential word, or "". A step's end is refused when it answers a field.
func credentialField(value json.RawMessage) string {
endings := []string{"password", "passwd", "secret", "token", "credential", "credentials",
"authorization", "apikey", "privatekey", "accesskey", "cookie"}
var v any
if json.Unmarshal(value, &v) != nil {
return ""
}
var walk func(any) string
walk = func(v any) string {
switch t := v.(type) {
case map[string]any:
keys := make([]string, 0, len(t))
for k := range t {
keys = append(keys, k)
}
sort.Strings(keys)
for _, k := range keys {
norm := strings.NewReplacer("-", "", "_", "", " ", "").Replace(strings.ToLower(k))
for _, end := range endings {
if strings.HasSuffix(norm, end) {
return k
}
}
if found := walk(t[k]); found != "" {
return found
}
}
case []any:
for _, x := range t {
if found := walk(x); found != "" {
return found
}
}
}
return ""
}
return walk(v)
}
// runtimeState is a steps state that refuses a value as the node runtime does, and keeps every value it took.
type runtimeState struct {
mu sync.Mutex
took map[string][]delivery.Step
refused int
}
func newRuntimeState() *runtimeState { return &runtimeState{took: map[string][]delivery.Step{}} }
func (s *runtimeState) put(key string, value any) (uint64, error) {
raw, _ := value.(json.RawMessage)
if f := credentialField(raw); f != "" {
s.mu.Lock()
s.refused++
s.mu.Unlock()
return 0, fmt.Errorf("steps.%s carries a field %q, which names a credential", key, f)
}
st, err := delivery.DecodeStep(raw)
if err != nil {
return 0, err
}
s.mu.Lock()
defer s.mu.Unlock()
s.took[key] = append(s.took[key], st)
return uint64(len(s.took[key])), nil
}
func (s *runtimeState) last(key string) (delivery.Step, bool) {
s.mu.Lock()
defer s.mu.Unlock()
t := s.took[key]
if len(t) == 0 {
return delivery.Step{}, false
}
return t[len(t)-1], true
}
func fastEnds(t *testing.T) {
attempts, retry := EndAttempts, EndRetry
EndAttempts, EndRetry = 2, time.Millisecond
t.Cleanup(func() { EndAttempts, EndRetry = attempts, retry })
}
// 1930/AC1: given a check of a repository with the package .../internal/token, when its ask's answer is stored as the
// step's end, then no field is keyed by a package path: the runtime's credential rule takes the value, and the package
// and its digest read back from the stored end.
func TestADigestOfAPackageNamedTokenIsStoredAsAList(t *testing.T) {
fastEnds(t)
r := &recorder{
fetch: map[string]any{"workspace": "/w"},
digests: map[string]string{"x/internal/token": "d-token", "x/pkg/a": "d-a"},
tests: map[string]bool{"x/internal/token": true, "x/pkg/a": true},
}
out, err := RunAsk(context.Background(), r.steps(), ask(delivery.CheckJob{Levels: []string{delivery.LevelUnit},
Fetch: map[string]any{"repository": "novox/x", "commit": runCommit}}))
if err != nil {
t.Fatal(err)
}
old, _ := json.Marshal(map[string]any{"levels": []any{map[string]any{"level": "unit", "digest": map[string]any{"packages": r.digests}}}})
if credentialField(old) != "x/internal/token" {
t.Fatalf("the rule does not refuse the form before issue 1930 (%s): the test proves nothing", old)
}
st := newRuntimeState()
if err := finish(st.put, "shanks.a1", delivery.Step{ID: "a1", Tool: "check ask", Node: "shanks", State: delivery.StepRunning, Started: time.Now()}, out, nil, t.Logf); err != nil {
t.Fatal(err)
}
s, ok := st.last("shanks.a1")
if !ok || s.State != delivery.StepPassed || st.refused != 0 {
t.Fatalf("the end %+v (written %v, refused %d), want it passed and taken at once", s, ok, st.refused)
}
var a delivery.CheckAnswer
if err := json.Unmarshal(s.Result, &a); err != nil {
t.Fatal(err)
}
if len(a.Levels) != 1 || a.Levels[0].Digest == nil {
t.Fatalf("levels %+v, want the unit level with its digest", a.Levels)
}
want := []delivery.PackageDigest{{Package: "x/internal/token", Digest: "d-token"}, {Package: "x/pkg/a", Digest: "d-a"}}
if got := a.Levels[0].Digest.Packages; fmt.Sprint(got) != fmt.Sprint(want) {
t.Errorf("packages %v, want %v", got, want)
}
if !strings.Contains(string(s.Result), `"package":"x/internal/token"`) {
t.Errorf("the stored end does not name the package as a value: %s", s.Result)
}
}
// 1930/AC2: given an end the state refuses every time, when finish writes it, then it logs the refusal and writes a small
// failed end without the result that names the write's error; and when no end can be written at all it answers an error.
func TestARefusedEndIsSaidAndEndsTheStepFailed(t *testing.T) {
fastEnds(t)
big := errors.New("the value carries a field \"x/internal/token\", which names a credential")
var refused int
var stored []delivery.Step
put := func(key string, value any) (uint64, error) {
st, _ := delivery.DecodeStep(value.(json.RawMessage))
if len(st.Result) > 0 {
refused++
return 0, big
}
stored = append(stored, st)
return 1, nil
}
var logged []string
logf := func(format string, args ...any) { logged = append(logged, fmt.Sprintf(format, args...)) }
err := finish(put, "shanks.a2", delivery.Step{ID: "a2", Tool: "check ask", Node: "shanks", State: delivery.StepRunning, Started: time.Now()}, map[string]any{"levels": 1}, nil, logf)
if err != nil {
t.Fatalf("finish answered %v, want the small end written", err)
}
if refused != EndAttempts || len(stored) != 1 {
t.Fatalf("refused %d, stored %d, want %d refusals and one small end", refused, len(stored), EndAttempts)
}
s := stored[0]
if s.State != delivery.StepFailed || len(s.Result) != 0 || !strings.Contains(s.Reason, "could not be written") ||
!strings.Contains(s.Reason, big.Error()) || !strings.Contains(s.Reason, "passed") {
t.Errorf("the small end %+v, want failed, no result, naming the write's error and that it passed", s)
}
if len(logged) == 0 || !strings.Contains(logged[0], big.Error()) {
t.Errorf("logged %q, want the refusal said", logged)
}
none := func(string, any) (uint64, error) { return 0, errors.New("the bus is gone") }
if err := finish(none, "shanks.a3", delivery.Step{ID: "a3", Tool: "check ask", Node: "shanks", Started: time.Now()}, nil, nil, logf); err == nil || !strings.Contains(err.Error(), "the bus is gone") {
t.Errorf("finish with no write possible answered %v, want the error", err)
}
}
// 1930/AC2: given an ask whose end the state refuses, when it runs, then the ask is acknowledged only once its small
// failed end is written; when no end can be written at all, it is given back, never acknowledged as done.
func TestAnAskWhoseEndCannotBeWrittenIsNotAcked(t *testing.T) {
fastEnds(t)
for _, tc := range []struct {
name string
smallOK bool
wantAcked bool
}{{"the small end written", true, true}, {"no end written", false, false}} {
t.Run(tc.name, func(t *testing.T) {
src := NewFakeAsks()
a := testAsk(delivery.AskCheck)
_ = src.PublishAsk(a)
var mu sync.Mutex
var ends []delivery.Step
put := func(key string, value any) (uint64, error) {
st, err := delivery.DecodeStep(value.(json.RawMessage))
if err != nil {
return 0, err
}
if st.State != delivery.StepRunning && (len(st.Result) > 0 || !tc.smallOK) {
return 0, errors.New("refused")
}
mu.Lock()
ends = append(ends, st)
mu.Unlock()
return 1, nil
}
p := &Puller{Source: src, Language: delivery.AskGo, Node: "shanks", Put: put, Capacity: func() int { return 1 },
PullWait: 10 * time.Millisecond, Logf: t.Logf,
Run: func(ctx context.Context, a delivery.StepAsk) (any, error) { return map[string]any{"ran": true}, nil }}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go p.Pull(ctx)
waitFor(t, "the ask's fate", func() bool { ac, tm, nk := src.Fates(); return len(ac)+len(tm)+len(nk) > 0 })
cancel()
acked, termed, naked := src.Fates()
if tc.wantAcked && (len(acked) != 1 || len(naked) != 0) || !tc.wantAcked && (len(acked) != 0 || len(naked) == 0) || len(termed) != 0 {
t.Fatalf("acked %v naked %v termed %v", acked, naked, termed)
}
mu.Lock()
defer mu.Unlock()
if tc.smallOK {
last := ends[len(ends)-1]
if last.State != delivery.StepFailed || !strings.Contains(last.Reason, "refused") {
t.Errorf("the end %+v, want the small failed end", last)
}
}
})
}
}
+88 -1
View File
@@ -4,6 +4,8 @@ import (
"context"
"fmt"
"net/url"
"os"
"path/filepath"
"regexp"
"strings"
@@ -13,12 +15,37 @@ import (
// Option changes FetchCommit.
type Option func(*fetchOptions)
type fetchOptions struct{ allowFile bool }
type fetchOptions struct {
allowFile bool
mirror string
}
// AllowFileRemote lets FetchCommit take a file:// remote. It is for a test's local repository; production
// code does not pass it, and FetchCommit then refuses anything but ssh://.
func AllowFileRemote() Option { return func(o *fetchOptions) { o.allowFile = true } }
// WithMirror makes FetchCommit (and MergeInto) borrow objects from mirror, a bare repository that UpdateMirror keeps
// in the build module's own cache (novox/hq ADR 0536 §2; issue 890): the workspace's .git/objects/info/alternates
// names the mirror's objects directory, so git transfers only the objects the mirror lacks. It is a plain file, never
// a symlink. mirror must be an absolute path of an existing directory; the workspace never writes to it.
func WithMirror(mirror string) Option { return func(o *fetchOptions) { o.mirror = mirror } }
// useMirror writes the alternates file of the repository in dir, when a mirror is set.
func useMirror(o fetchOptions, dir string) error {
if o.mirror == "" {
return nil
}
info := filepath.Join(dir, ".git", "objects", "info")
if err := os.MkdirAll(info, 0o755); err != nil {
return fmt.Errorf("buildstep: the mirror cannot be used: %w", err)
}
line := filepath.Join(o.mirror, "objects") + "\n"
if err := os.WriteFile(filepath.Join(info, "alternates"), []byte(line), 0o644); err != nil {
return fmt.Errorf("buildstep: the mirror cannot be used: %w", err)
}
return nil
}
var (
commitOK = regexp.MustCompile(`^[0-9a-f]{40}$`)
refOK = regexp.MustCompile(`^refs/[A-Za-z0-9._/-]+$`)
@@ -51,6 +78,9 @@ func FetchCommit(ctx context.Context, run forgekey.Runner, dir, remote, commit,
if _, err := git("init", "-q"); err != nil {
return err
}
if err := useMirror(o, dir); err != nil {
return err
}
if _, err := git("remote", "add", "origin", remote); err != nil {
return err
}
@@ -78,10 +108,67 @@ func FetchCommit(ctx context.Context, run forgekey.Runner, dir, remote, commit,
return nil
}
// UpdateMirror brings the bare repository mirror to commit: it makes the mirror when it is absent (git init --bare,
// origin set to remote), then fetches the commit, or the ref when only the ref holds it, with the same rules as
// FetchCommit (no prompt, no credential helper, ssh:// or an allowed file:// remote, a 40-hex commit). The mirror
// keeps every commit it was brought to, so later fetches through WithMirror transfer only what is new. Callers
// serialise updates of one mirror.
func UpdateMirror(ctx context.Context, run forgekey.Runner, mirror, remote, commit, ref string, opts ...Option) error {
var o fetchOptions
for _, opt := range opts {
opt(&o)
}
if !filepath.IsAbs(mirror) {
return fmt.Errorf("buildstep: the mirror %q is not an absolute path", mirror)
}
if err := checkFetch(o, mirror, remote, commit, ref); err != nil {
return err
}
env := []string{"GIT_TERMINAL_PROMPT=0"}
git := func(args ...string) error {
r := run(ctx, env, "git", append([]string{"-c", "credential.helper="}, args...)...)
if r.Status != 0 || r.Err != "" {
return fmt.Errorf("git %s: %s", args[0], firstLine(r.Err+" "+r.Stderr))
}
return nil
}
in := func(args ...string) error { return git(append([]string{"-C", mirror}, args...)...) }
if _, err := os.Stat(filepath.Join(mirror, "HEAD")); err != nil {
if err := git("init", "-q", "--bare", mirror); err != nil {
return err
}
if err := in("remote", "add", "origin", remote); err != nil {
return err
}
} else if err := in("remote", "set-url", "origin", remote); err != nil {
return err
}
if err := in("fetch", "--no-tags", "origin", commit); err != nil {
if ref == "" {
return err
}
if rerr := in("fetch", "--no-tags", "origin", ref); rerr != nil {
return fmt.Errorf("%v; then the ref %s: %w", err, ref, rerr)
}
}
if err := in("cat-file", "-e", commit+"^{commit}"); err != nil {
return fmt.Errorf("the commit %s is not in the mirror: %w", commit, err)
}
return nil
}
func checkFetch(o fetchOptions, dir, remote, commit, ref string) error {
if dir == "" {
return fmt.Errorf("buildstep: no workspace directory")
}
if o.mirror != "" {
if !filepath.IsAbs(o.mirror) {
return fmt.Errorf("buildstep: the mirror %q is not an absolute path", o.mirror)
}
if fi, err := os.Stat(o.mirror); err != nil || !fi.IsDir() {
return fmt.Errorf("buildstep: the mirror %q is not an existing directory", o.mirror)
}
}
if !commitOK.MatchString(commit) {
return fmt.Errorf("buildstep: %q is not a full commit (40 hex digits)", commit)
}
+3
View File
@@ -44,6 +44,9 @@ func MergeInto(ctx context.Context, run forgekey.Runner, dir, remote, base, comm
if err := checkFetch(o, dir, remote, commit, ""); err != nil {
return res, err
}
if err := useMirror(o, dir); err != nil {
return res, err
}
env := []string{"GIT_TERMINAL_PROMPT=0"}
git := func(args ...string) (string, error) {
full := append([]string{"-c", "credential.helper=", "-C", dir}, args...)
+65
View File
@@ -0,0 +1,65 @@
//go:build integration
package buildstep
import (
"context"
"os"
"path/filepath"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/forgekey"
)
// Issue 890 C1 (AC3): a mirror brought to two commits in turn; a fetch through it checks out each commit, the
// alternates name the mirror, and no symlink is in the workspace.
func TestMirrorServesTwoCommitsWithoutSymlinks(t *testing.T) {
root := t.TempDir()
src := filepath.Join(root, "src.git")
work := filepath.Join(root, "work")
for _, d := range []string{src, work} {
if err := os.MkdirAll(d, 0o755); err != nil {
t.Fatal(err)
}
}
gitIn(t, src, "init", "-q", "--bare")
gitIn(t, work, "init", "-q")
var commits []string
for _, c := range []string{"one", "two"} {
if err := os.WriteFile(filepath.Join(work, "f"), []byte(c), 0o644); err != nil {
t.Fatal(err)
}
gitIn(t, work, "add", "f")
gitIn(t, work, "commit", "-q", "-m", c)
commits = append(commits, gitIn(t, work, "rev-parse", "HEAD"))
gitIn(t, work, "push", "-q", src, "HEAD:refs/heads/main")
}
remote := "file://" + src
mirror := filepath.Join(root, "mirror.git")
for i, commit := range commits {
if err := UpdateMirror(context.Background(), forgekey.ExecRunner, mirror, remote, commit, "", AllowFileRemote()); err != nil {
t.Fatal(err)
}
ws := filepath.Join(root, "ws"+string(rune('0'+i)))
if err := os.MkdirAll(ws, 0o755); err != nil {
t.Fatal(err)
}
if err := FetchCommit(context.Background(), forgekey.ExecRunner, ws, remote, commit, "", AllowFileRemote(), WithMirror(mirror)); err != nil {
t.Fatal(err)
}
if head := gitIn(t, ws, "rev-parse", "HEAD"); head != commit {
t.Errorf("HEAD = %s, want %s", head, commit)
}
b, _ := os.ReadFile(filepath.Join(ws, ".git", "objects", "info", "alternates"))
if !strings.Contains(string(b), filepath.Join(mirror, "objects")) {
t.Errorf("alternates %q", b)
}
_ = filepath.Walk(ws, func(p string, fi os.FileInfo, err error) error {
if err == nil && fi.Mode()&os.ModeSymlink != 0 {
t.Errorf("symlink %s", p)
}
return nil
})
}
}
+106
View File
@@ -0,0 +1,106 @@
package buildstep
import (
"context"
"os"
"path/filepath"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/forgekey"
)
// Issue 890 C1 (ADR 0536 §2): a fetch through a bare mirror names the mirror in .git/objects/info/alternates, a
// plain file, and never links it.
func TestFetchCommitWithMirrorWritesAlternates(t *testing.T) {
mirror := t.TempDir()
ws := t.TempDir()
f := &fakeGit{}
// the fake's init does not create .git; make the objects directory as git init would.
if err := os.MkdirAll(filepath.Join(ws, ".git", "objects", "info"), 0o755); err != nil {
t.Fatal(err)
}
err := FetchCommit(context.Background(), f.run, ws, "ssh://mesh-forge/novox/mesh-sdk.git", sha, "", WithMirror(mirror))
if err != nil {
t.Fatal(err)
}
p := filepath.Join(ws, ".git", "objects", "info", "alternates")
b, err := os.ReadFile(p)
if err != nil {
t.Fatal(err)
}
if got, want := strings.TrimSpace(string(b)), filepath.Join(mirror, "objects"); got != want {
t.Errorf("alternates %q, want %q", got, want)
}
if fi, err := os.Lstat(p); err != nil || !fi.Mode().IsRegular() {
t.Errorf("alternates is not a plain file: %v %v", fi, err)
}
// the alternates exist before the fetch runs
if len(f.calls) < 3 || !strings.Contains(f.calls[2], "fetch") {
t.Errorf("calls %v", f.calls)
}
}
func TestWithMirrorRefusesARelativeOrMissingMirror(t *testing.T) {
for _, m := range []string{"rel/mirror", filepath.Join(t.TempDir(), "absent")} {
f := &fakeGit{}
err := FetchCommit(context.Background(), f.run, t.TempDir(), "ssh://mesh-forge/novox/x.git", sha, "", WithMirror(m))
if err == nil || len(f.calls) != 0 {
t.Errorf("mirror %q: err %v, calls %v", m, err, f.calls)
}
}
}
func TestUpdateMirrorInitsBareThenFetches(t *testing.T) {
mirror := filepath.Join(t.TempDir(), "mirror.git")
f := &fakeGit{}
if err := UpdateMirror(context.Background(), f.run, mirror, "ssh://mesh-forge/novox/x.git", sha, ""); err != nil {
t.Fatal(err)
}
pm := "git -c credential.helper= -C " + mirror + " "
want := []string{
"git -c credential.helper= init -q --bare " + mirror,
pm + "remote add origin ssh://mesh-forge/novox/x.git",
pm + "fetch --no-tags origin " + sha,
pm + "cat-file -e " + sha + "^{commit}",
}
if strings.Join(f.calls, "\n") != strings.Join(want, "\n") {
t.Errorf("commands:\n%s\nwant:\n%s", strings.Join(f.calls, "\n"), strings.Join(want, "\n"))
}
}
func TestUpdateMirrorKeepsAnExistingMirrorAndFallsBackToTheRef(t *testing.T) {
mirror := t.TempDir() // exists: no init
if err := os.WriteFile(filepath.Join(mirror, "HEAD"), []byte("ref: refs/heads/main\n"), 0o644); err != nil {
t.Fatal(err)
}
f := &fakeGit{}
f.rule = func(args string) (r forgekey.Ran) {
if strings.HasSuffix(args, "origin "+sha) {
r.Status = 1
r.Stderr = "not our ref"
}
return
}
if err := UpdateMirror(context.Background(), f.run, mirror, "ssh://mesh-forge/novox/x.git", sha, "refs/pull/3/head"); err != nil {
t.Fatal(err)
}
all := strings.Join(f.calls, "\n")
if strings.Contains(all, "init") || strings.Contains(all, "remote add") || !strings.Contains(all, "fetch --no-tags origin refs/pull/3/head") {
t.Errorf("commands:\n%s", all)
}
}
func TestUpdateMirrorRefusesBeforeRunning(t *testing.T) {
f := &fakeGit{}
for _, c := range []struct{ remote, commit string }{
{"https://x/y.git", sha}, {"ssh://u@h/o/r.git", sha}, {"ssh://h/o/r.git", "abc"},
} {
if err := UpdateMirror(context.Background(), f.run, t.TempDir(), c.remote, c.commit, ""); err == nil {
t.Errorf("%v accepted", c)
}
}
if len(f.calls) != 0 {
t.Errorf("ran %v", f.calls)
}
}
+445
View File
@@ -0,0 +1,445 @@
package buildstep
import (
"context"
"encoding/json"
"errors"
"fmt"
"sort"
"sync"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// **The pull loop the build modules share** (novox/hq ADR 0525 §4-§6, ADR 0536 §4; issue 1887). A build module on a node
// whose build capacity is above zero pulls the asks of its language's seat: from the delivery worker first, then check,
// then recheck, holding at most its capacity at once. Each ask runs as a step under the ask's own id, so its end is the
// key <node>.<ask id> in the module's steps state, which the asker watches; the ask is acknowledged when that end is
// written, kept alive while it runs by an in-progress signal, and redelivered by the stream to another node if this one
// stops. No part keeps a timer for an ask: the pull's own wait and the stream's acknowledgement wait are the bus's.
// Pulled is one ask a source handed over, with what decides its fate on the bus.
type Pulled struct {
// Kind is the worker it came from, filled by the puller.
Kind string
Data []byte
// Delivered is how many times the bus has handed it over, this time included.
Delivered uint64
// Ack ends it: it is never delivered again. InProgress tells the bus it is still being worked, restarting its
// acknowledgement wait. Nak hands it back to be delivered again at once (to this node or another). Term drops it.
Ack, InProgress, Nak, Term func() error
}
// AskSource is where a puller takes asks from: the seat's workers on the bus (JetStreamAsks) or a test double
// (FakeAsks).
type AskSource interface {
// Next takes one ask of kind: with wait 0 only one already waiting, else waiting for one at most wait. It answers
// nil and no error when none came.
Next(ctx context.Context, kind string, wait time.Duration) (*Pulled, error)
}
// Defaults of a Puller.
var (
// DefaultPullWait is how long one pull of the delivery worker waits for an ask while every worker is empty: the
// pull's own wait (as process.Consume's), after which the check and recheck workers are looked at again.
DefaultPullWait = 5 * time.Second
// DefaultHeartbeat is how often a running ask tells the bus it is still being worked; under the seat worker's
// acknowledgement wait of 60 s (mesh-controller internal/broker HolderConsumerFor).
DefaultHeartbeat = 20 * time.Second
)
// Puller pulls a build seat's asks and runs them.
type Puller struct {
Source AskSource
Language string // the seat's language: delivery.AskGo, AskNode or AskImage
Node string // the node this holder runs on, the first part of its step keys
Put Put // writes the module's steps state
// Capacity is how many asks the node runs at once; read before every pull, so a change takes effect at the next
// one. 0 pulls nothing.
Capacity func() int
// Run runs one ask's job; a *Refusal ends its step refused. Its result is the step's result.
Run func(ctx context.Context, ask delivery.StepAsk) (any, error)
// PullWait and Heartbeat default to DefaultPullWait and DefaultHeartbeat.
PullWait time.Duration
Heartbeat time.Duration
Logf func(format string, args ...any)
// Load reads one key of the state the node's pause is kept in (nil and no error when it is absent). When set, Pull
// reads the node's pause key (delivery.PauseKey) before it pulls, and Pause and Resume write it with PausePut, so a
// pause outlives a restart.
Load func(key string) ([]byte, error)
// PausePut writes the node's pause key: the module's pauses state (delivery.PausesState), which has no age, so a
// pause never ends by itself while the steps state drops its old keys. Put when nil.
PausePut Put
// WatchConfig, when set, follows the module's config value `capacity` (CapacitySetting) as mesh-config resolves it
// for this node, and wakes the puller on each change (watchCapacity): WatchConfigServed in a module, a test's own
// double. Until it hands a value the puller judges by Capacity.
WatchConfig func(key string, handle func(stdio.ProvisionChange) error) error
mu sync.Mutex
inHand int
running map[string]RunningAsk
paused bool
wake chan struct{}
once sync.Once
// watched is the capacity the config watch handed last, used in place of Capacity once hasWatched.
watched int
hasWatched bool
}
func (p *Puller) init() {
p.once.Do(func() { p.wake = make(chan struct{}, 1) })
}
func (p *Puller) logf(format string, args ...any) {
if p.Logf != nil {
p.Logf(format, args...)
}
}
// Wake makes a waiting puller look again: after a capacity change, a resume, or a step's end.
func (p *Puller) Wake() {
p.init()
select {
case p.wake <- struct{}{}:
default:
}
}
// Pause stops the puller taking asks; the asks it holds run to their end. by names who paused it. With Load set the
// pause is written on the node's pause key first, and an error there leaves the puller as it was.
func (p *Puller) Pause(by string) error { return p.setPaused(true, by) }
// Resume lets the puller take asks again, written as Pause's is.
func (p *Puller) Resume(by string) error { return p.setPaused(false, by) }
func (p *Puller) setPaused(paused bool, by string) error {
if p.Load != nil {
key, err := delivery.PauseKey(p.Node)
if err != nil {
return err
}
raw, err := delivery.EncodePause(delivery.Pause{Paused: paused, By: by, At: now()})
if err != nil {
return err
}
put := p.PausePut
if put == nil {
put = p.Put
}
if _, err := put(key, json.RawMessage(raw)); err != nil {
return fmt.Errorf("buildstep: writing the pause of %s: %w", p.Node, err)
}
}
p.mu.Lock()
p.paused = paused
p.mu.Unlock()
p.Wake()
return nil
}
// restore reads the node's pause key, when Load is set.
func (p *Puller) restore() error {
if p.Load == nil {
return nil
}
key, err := delivery.PauseKey(p.Node)
if err != nil {
return err
}
raw, err := p.Load(key)
if err != nil {
return fmt.Errorf("buildstep: reading the pause of %s: %w", p.Node, err)
}
if raw == nil {
return nil
}
v, err := delivery.DecodePause(raw)
if err != nil {
return fmt.Errorf("buildstep: the pause of %s: %w", p.Node, err)
}
p.mu.Lock()
p.paused = v.Paused
p.mu.Unlock()
return nil
}
// Paused says whether the puller takes no asks.
func (p *Puller) Paused() bool {
p.mu.Lock()
defer p.mu.Unlock()
return p.paused
}
// InHand is how many asks run now.
func (p *Puller) InHand() int {
p.mu.Lock()
defer p.mu.Unlock()
return p.inHand
}
// RunningAsk is an ask a puller runs now, as its seat's current verb says it.
type RunningAsk struct {
ID string `json:"id"`
Kind string `json:"kind"`
Module string `json:"module,omitempty"`
Started time.Time `json:"started"`
}
// Running are the asks the puller runs now, oldest first.
func (p *Puller) Running() []RunningAsk {
p.mu.Lock()
defer p.mu.Unlock()
out := make([]RunningAsk, 0, len(p.running))
for _, r := range p.running {
out = append(out, r)
}
sort.Slice(out, func(i, j int) bool { return out[i].Started.Before(out[j].Started) })
return out
}
// room says whether the puller may take one more ask now.
func (p *Puller) room() bool {
ok, _, _ := p.roomNow()
return ok
}
// roomNow is room, with the capacity and the asks in hand it was judged on.
func (p *Puller) roomNow() (ok bool, capacity, inHand int) {
p.mu.Lock()
watched, has := p.watched, p.hasWatched
p.mu.Unlock()
if has {
capacity = watched
} else if p.Capacity != nil {
capacity = p.Capacity()
}
p.mu.Lock()
defer p.mu.Unlock()
return !p.paused && p.inHand < capacity, capacity, p.inHand
}
// CapacityNow is the capacity the puller judges by: the config watch's last value, else Capacity.
func (p *Puller) CapacityNow() int {
_, capacity, _ := p.roomNow()
return capacity
}
// Pull takes and runs asks until ctx ends. An ask still running then is neither ended nor acknowledged: the stream
// delivers it again, to another node or to this one restarted.
func (p *Puller) Pull(ctx context.Context) error {
p.init()
if p.Source == nil || p.Put == nil || p.Run == nil {
return errors.New("buildstep: a puller needs its source, its steps' put and its run")
}
if err := p.restore(); err != nil {
// An unreadable pause degrades to not paused, said once: a puller that stops silently is the worse failure, and a
// node kept from building is held by its capacity (0), not by a pause.
p.logf("the pause of %s cannot be read: %v; pulling as not paused", p.Node, err)
}
go p.watchCapacity() // WatchConfigServed waits for Serve, which a module starts after its pull
wait := p.PullWait
if wait <= 0 {
wait = DefaultPullWait
}
for {
if err := ctx.Err(); err != nil {
return err
}
if ok, capacity, inHand := p.roomNow(); !ok {
// Parked until something wakes it: a capacity change (the config watch), a resume, or a step's end. Every
// park logs once, so a puller that sits idle says why (issue 1887).
p.logf("the %s puller on %s parks: %d of capacity %d in hand, paused %v", p.Language, p.Node, inHand, capacity, p.Paused())
select {
case <-ctx.Done():
return ctx.Err()
case <-p.wake:
}
_, capacity, inHand = p.roomNow()
p.logf("the %s puller on %s wakes: %d of capacity %d in hand, paused %v", p.Language, p.Node, inHand, capacity, p.Paused())
continue
}
msg, err := p.next(ctx, wait)
if err != nil {
if ctx.Err() != nil {
return ctx.Err()
}
p.logf("the %s asks cannot be pulled: %v", p.Language, err)
select { // the pull's own wait before the next try, as an empty pull's
case <-ctx.Done():
return ctx.Err()
case <-p.wake:
case <-time.After(wait):
}
continue
}
if msg == nil {
continue
}
p.take(ctx, msg)
}
}
// next is one ask from the workers in their order: one already waiting on delivery, check or recheck, and when none
// waits, a pull of the delivery worker that waits for one.
func (p *Puller) next(ctx context.Context, wait time.Duration) (*Pulled, error) {
for _, kind := range delivery.AskKinds {
msg, err := p.Source.Next(ctx, kind, 0)
if err != nil || msg != nil {
return withKind(msg, kind), err
}
}
msg, err := p.Source.Next(ctx, delivery.AskDelivery, wait)
return withKind(msg, delivery.AskDelivery), err
}
func withKind(msg *Pulled, kind string) *Pulled {
if msg != nil {
msg.Kind = kind
}
return msg
}
// take starts one ask: it decodes it, refuses one that is not of this seat, and runs it as a step under its own id.
func (p *Puller) take(ctx context.Context, msg *Pulled) {
kind := msg.Kind
ask, err := delivery.DecodeStepAsk(msg.Data)
if err == nil && (ask.Language != p.Language || (kind != "" && ask.Kind != kind)) {
err = fmt.Errorf("the ask %s is a %s ask of %s, pulled from the %s %s worker", ask.ID, ask.Kind, ask.Language, p.Language, kind)
}
if err != nil {
// An ask that is not one runs nowhere: delivering it again would only fail again.
p.logf("an ask dropped: %v", err)
_ = msg.Term()
return
}
step, key, err := begin(p.Put, p.Node, ask.Kind+" ask", ask.ID)
if err != nil {
p.logf("the ask %s cannot start on %s: %v", ask.ID, p.Node, err)
_ = msg.Nak()
return
}
if msg.Delivered > delivery.AskRuns {
// Run AskRuns times and never ended: each run took its node down or died with it. It is run no more, and said
// (issue 1907): the asker's key fails with why.
why := fmt.Errorf("the ask was taken %d times and no run of it ended: each took its build node down or died with it, so it is run no more (poison)", msg.Delivered-1)
p.logf("the ask %s is ended on %s: %v", ask.ID, p.Node, why)
_ = finish(p.Put, key, step, nil, why, p.Logf) // run no more either way: the asker's bound says it then
_ = msg.Term()
return
}
module := ""
if ask.Build != nil {
module = ask.Build.Module
}
p.mu.Lock()
p.inHand++
if p.running == nil {
p.running = map[string]RunningAsk{}
}
p.running[ask.ID] = RunningAsk{ID: ask.ID, Kind: ask.Kind, Module: module, Started: step.Started}
p.mu.Unlock()
go p.run(ctx, msg, ask, key, step)
}
func (p *Puller) run(ctx context.Context, msg *Pulled, ask delivery.StepAsk, key string, step delivery.Step) {
defer func() {
p.mu.Lock()
p.inHand--
delete(p.running, ask.ID)
p.mu.Unlock()
p.Wake()
}()
beat := p.Heartbeat
if beat <= 0 {
beat = DefaultHeartbeat
}
done := make(chan struct{})
go func() {
t := time.NewTicker(beat)
defer t.Stop()
for {
select {
case <-done:
return
case <-t.C:
if msg.InProgress != nil {
_ = msg.InProgress()
}
}
}
}()
result, err := protect(ctx, func(ctx context.Context) (any, error) { return p.Run(ctx, ask) })
close(done)
if ctx.Err() != nil {
// The process stops: the ask is neither ended nor acknowledged here, so the stream gives it to a node that runs.
return
}
if ferr := finish(p.Put, key, step, result, err, p.Logf); ferr != nil {
// No end was written: the ask is not acknowledged as done, so the stream gives it again and a run that can
// write its end (or the poison rule, issue 1907) ends it where its asker sees it (issue 1930).
p.logf("the ask %s ran on %s and no end of it could be written, so it is given back: %v", ask.ID, p.Node, ferr)
_ = msg.Nak()
return
}
if aerr := msg.Ack(); aerr != nil {
p.logf("the ask %s ended on %s and its acknowledgement failed: %v", ask.ID, p.Node, aerr)
}
}
// watchCapacity follows the module's config value `capacity` when WatchConfig is set (issue 1887, 2026-10-25: g14's
// puller started at capacity 0 sat parked with capacity 2 set, because nothing woke it to read its settings file
// again). The value comes as mesh-config resolved it for this node, the event of the change itself: the runtime hands
// the value now and then each change its machine's provisioner client says moved, so nothing polls, and the puller does
// not wait for the node-engine to fill its settings file again. Each value handed is kept, used in place of Capacity,
// and wakes the puller. A watch that cannot start leaves the puller on Capacity, said once.
func (p *Puller) watchCapacity() {
if p.WatchConfig == nil {
return
}
err := p.WatchConfig(CapacitySetting, func(c stdio.ProvisionChange) error {
n, why := p.capacityFrom(c)
p.mu.Lock()
before, had := p.watched, p.hasWatched
p.watched, p.hasWatched = n, true
p.mu.Unlock()
switch {
case why != "":
p.logf("the %s puller on %s pulls nothing: %s", p.Language, p.Node, why)
case !had || before != n:
p.logf("the %s build capacity of %s is %d, as mesh-config resolved it", p.Language, p.Node, n)
}
p.Wake()
return nil
})
if err != nil {
p.logf("the %s build capacity of %s cannot be followed on the bus, so it is read from the settings file when the "+
"puller wakes: %v", p.Language, p.Node, err)
}
}
// WatchConfigServed is stdio.WatchConfig once Serve has the runtime's channel (stdio.Served): a build module starts its
// pull before it serves, and a watch asked before that would be refused.
func WatchConfigServed(key string, handle func(stdio.ProvisionChange) error) error {
<-stdio.Served()
return stdio.WatchConfig(key, handle)
}
// capacityFrom is the node's capacity in one value the watch handed, and why it is 0 when it cannot be read.
func (p *Puller) capacityFrom(c stdio.ProvisionChange) (int, string) {
if c.Refused != "" {
return 0, "its capacity is no longer answered: " + c.Refused
}
setting := ""
if len(c.Value) > 0 && string(c.Value) != "null" {
if err := json.Unmarshal(c.Value, &setting); err != nil {
return 0, fmt.Sprintf("its capacity %s is not a string", c.Value)
}
}
n, err := capacityOf(setting, p.Node)
if err != nil {
return 0, err.Error()
}
return n, ""
}
+102
View File
@@ -0,0 +1,102 @@
package buildstep
import (
"context"
"fmt"
"sync"
"time"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// JetStreamAsks is a build seat's workers on the bus: the shared pull consumer of each kind on the seat's stream
// (delivery.AskStream, AskWorker), which the broker makes when the seat is declared; a holder only binds them.
type JetStreamAsks struct {
JS jetstream.JetStream
Language string
mu sync.Mutex
cons map[string]jetstream.Consumer
}
func (s *JetStreamAsks) consumer(ctx context.Context, kind string) (jetstream.Consumer, error) {
s.mu.Lock()
defer s.mu.Unlock()
if c, ok := s.cons[kind]; ok {
return c, nil
}
stream, err := delivery.AskStream(s.Language)
if err != nil {
return nil, err
}
worker, err := delivery.AskWorker(s.Language, kind)
if err != nil {
return nil, err
}
c, err := s.JS.Consumer(ctx, stream, worker)
if err != nil {
return nil, fmt.Errorf("binding the %s worker %s on %s: %w", kind, worker, stream, err)
}
if s.cons == nil {
s.cons = map[string]jetstream.Consumer{}
}
s.cons[kind] = c
return c, nil
}
// Next is AskSource's: a fetch of one with no wait when wait is 0, else one that waits at most wait.
func (s *JetStreamAsks) Next(ctx context.Context, kind string, wait time.Duration) (*Pulled, error) {
c, err := s.consumer(ctx, kind)
if err != nil {
return nil, err
}
var batch jetstream.MessageBatch
if wait <= 0 {
batch, err = c.FetchNoWait(1)
} else {
batch, err = c.Fetch(1, jetstream.FetchMaxWait(wait))
}
if err != nil {
return nil, err
}
select {
case msg, ok := <-batch.Messages():
if !ok {
return nil, batch.Error()
}
var delivered uint64 = 1
if meta, err := msg.Metadata(); err == nil && meta != nil {
delivered = meta.NumDelivered
}
return &Pulled{Data: append([]byte(nil), msg.Data()...), Delivered: delivered,
Ack: msg.Ack, InProgress: msg.InProgress, Nak: msg.Nak, Term: msg.Term}, nil
case <-ctx.Done():
return nil, ctx.Err()
}
}
// DeclareAskWorkers makes language's seat stream and its three workers on a bus with no broker: a test's (the build
// modules' integration tests start one). On the mesh the broker declares them from the seat, as it declares every
// seat's stream (mesh-controller internal/broker SeatStreams); a holder never does.
func DeclareAskWorkers(ctx context.Context, js jetstream.JetStream, language string, ackWait time.Duration) error {
stream, err := delivery.AskStream(language)
if err != nil {
return err
}
seat, _ := delivery.BuildSeat(language)
if _, err := js.CreateOrUpdateStream(ctx, jetstream.StreamConfig{Name: stream, Subjects: []string{"mesh.seat." + seat + ".accept.>"},
Retention: jetstream.WorkQueuePolicy}); err != nil {
return fmt.Errorf("the stream %s: %w", stream, err)
}
for _, kind := range delivery.AskKinds {
worker, _ := delivery.AskWorker(language, kind)
subject, _ := delivery.AskSubject(language, kind)
if _, err := js.CreateOrUpdateConsumer(ctx, stream, jetstream.ConsumerConfig{Durable: worker, FilterSubject: subject,
AckPolicy: jetstream.AckExplicitPolicy, AckWait: ackWait, MaxDeliver: 5}); err != nil {
return fmt.Errorf("the worker %s: %w", worker, err)
}
}
return nil
}
+39
View File
@@ -0,0 +1,39 @@
package buildstep
import (
"encoding/json"
"sync"
"testing"
stdio "git.novox.be/novox/mesh-sdk/go"
)
// configWatch is a double of stdio.WatchConfig: it hands the value now at the start, as the runtime does, and a test
// hands each change after with set.
type configWatch struct {
mu sync.Mutex
value string
handle func(stdio.ProvisionChange) error
asked string
}
func (w *configWatch) watch(key string, handle func(stdio.ProvisionChange) error) error {
w.mu.Lock()
w.asked, w.handle = key, handle
value := w.value
w.mu.Unlock()
return handle(stdio.ProvisionChange{Kind: "config", Key: key, Value: json.RawMessage(`"` + value + `"`)})
}
func (w *configWatch) set(t *testing.T, value string) {
t.Helper()
w.mu.Lock()
handle, key := w.handle, w.asked
w.mu.Unlock()
if handle == nil {
t.Fatal("the puller watches no config value")
}
if err := handle(stdio.ProvisionChange{Kind: "config", Key: key, Value: json.RawMessage(`"` + value + `"`)}); err != nil {
t.Fatal(err)
}
}
+113
View File
@@ -0,0 +1,113 @@
package buildstep
import (
"context"
"sync"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// FakeAsks is a test double of a seat's workers for the build modules' unit tests: Publish puts an ask on a kind's
// worker, Next hands them out in their order, and the fates the puller gives them are counted. Safe for concurrent use.
type FakeAsks struct {
mu sync.Mutex
waiting map[string][][]byte
added chan struct{}
// Acked, Termed and Naked are the asks' ids or data by fate, InProgress how many heartbeats were given.
Acked, Termed, Naked []string
InProgress int
// Delivered, when set, is the delivery count every ask handed over says (1 when 0): a test of a poisoned ask sets it.
Delivered uint64
}
// NewFakeAsks is an empty double.
func NewFakeAsks() *FakeAsks {
return &FakeAsks{waiting: map[string][][]byte{}, added: make(chan struct{}, 1)}
}
// Publish puts an encoded ask on kind's worker.
func (f *FakeAsks) Publish(kind string, data []byte) {
f.mu.Lock()
f.waiting[kind] = append(f.waiting[kind], data)
f.mu.Unlock()
select {
case f.added <- struct{}{}:
default:
}
}
// PublishAsk encodes a and puts it on its kind's worker.
func (f *FakeAsks) PublishAsk(a delivery.StepAsk) error {
raw, err := delivery.EncodeStepAsk(a)
if err != nil {
return err
}
f.Publish(a.Kind, raw)
return nil
}
// Waiting is how many asks wait on kind's worker.
func (f *FakeAsks) Waiting(kind string) int {
f.mu.Lock()
defer f.mu.Unlock()
return len(f.waiting[kind])
}
// Fates is a copy of the acked, termed and naked lists.
func (f *FakeAsks) Fates() (acked, termed, naked []string) {
f.mu.Lock()
defer f.mu.Unlock()
return append([]string(nil), f.Acked...), append([]string(nil), f.Termed...), append([]string(nil), f.Naked...)
}
// Next is AskSource's.
func (f *FakeAsks) Next(ctx context.Context, kind string, wait time.Duration) (*Pulled, error) {
var deadline <-chan time.Time
if wait > 0 {
t := time.NewTimer(wait)
defer t.Stop()
deadline = t.C
}
for {
f.mu.Lock()
if q := f.waiting[kind]; len(q) > 0 {
data := q[0]
f.waiting[kind] = q[1:]
f.mu.Unlock()
return f.pulled(data), nil
}
f.mu.Unlock()
if wait <= 0 {
return nil, nil
}
select {
case <-ctx.Done():
return nil, ctx.Err()
case <-deadline:
return nil, nil
case <-f.added:
}
}
}
func (f *FakeAsks) pulled(data []byte) *Pulled {
id := string(data)
if a, err := delivery.DecodeStepAsk(data); err == nil {
id = a.ID
}
fate := func(list *[]string) func() error {
return func() error {
f.mu.Lock()
*list = append(*list, id)
f.mu.Unlock()
return nil
}
}
delivered := f.Delivered
if delivered == 0 {
delivered = 1
}
return &Pulled{Data: data, Delivered: delivered, Ack: fate(&f.Acked), Nak: fate(&f.Naked), Term: fate(&f.Termed),
InProgress: func() error { f.mu.Lock(); f.InProgress++; f.mu.Unlock(); return nil }}
}
+144
View File
@@ -0,0 +1,144 @@
//go:build integration
package buildstep
import (
"context"
"strings"
"sync"
"testing"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/testbus"
)
func askBus(t *testing.T, ackWait time.Duration) jetstream.JetStream {
t.Helper()
nc, err := nats.Connect(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
t.Cleanup(nc.Close)
js, err := jetstream.New(nc)
if err != nil {
t.Fatal(err)
}
if err := DeclareAskWorkers(context.Background(), js, delivery.AskGo, ackWait); err != nil {
t.Fatal(err)
}
return js
}
func publishAsk(t *testing.T, js jetstream.JetStream, a delivery.StepAsk) {
t.Helper()
raw, err := delivery.EncodeStepAsk(a)
if err != nil {
t.Fatal(err)
}
subject, _ := delivery.AskSubject(a.Language, a.Kind)
if _, err := js.Publish(context.Background(), subject, raw); err != nil {
t.Fatal(err)
}
}
// 1887/A2a, 1887/AC-order, 1887/AC-pull on the real server: a check ask published before a delivery ask runs after it;
// with capacity 2 and three asks two run, the third waits in the stream and starts when one ends.
func TestPullOrderAndCapacityOnTheBus(t *testing.T) {
js := askBus(t, 30*time.Second)
check, first, second := testAsk(delivery.AskCheck), testAsk(delivery.AskDelivery), testAsk(delivery.AskDelivery)
publishAsk(t, js, check)
publishAsk(t, js, first)
publishAsk(t, js, second)
release := make(chan struct{})
var mu sync.Mutex
var order []string
st := newSteps()
p := &Puller{Source: &JetStreamAsks{JS: js, Language: delivery.AskGo}, Language: delivery.AskGo, Node: "shanks", Put: st.put,
Capacity: func() int { return 2 }, PullWait: 200 * time.Millisecond,
Run: func(ctx context.Context, a delivery.StepAsk) (any, error) {
mu.Lock()
order = append(order, a.ID)
mu.Unlock()
<-release
return nil, nil
}}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go p.Pull(ctx)
waitFor(t, "two in hand", func() bool { return p.InHand() == 2 })
time.Sleep(500 * time.Millisecond)
mu.Lock()
if len(order) != 2 || order[0] != first.ID || order[1] != second.ID {
t.Fatalf("ran %v, want the two delivery asks %s %s first", order, first.ID, second.ID)
}
mu.Unlock()
release <- struct{}{}
waitFor(t, "the check taken", func() bool { mu.Lock(); defer mu.Unlock(); return len(order) == 3 })
close(release)
waitFor(t, "every step ended", func() bool {
for _, a := range []delivery.StepAsk{check, first, second} {
key, _ := delivery.AskStepKey("shanks", a.ID)
if s, ok := st.get(key); !ok || s.State != delivery.StepPassed {
return false
}
}
return true
})
info, err := js.Stream(ctx, "SEAT_NODE_BUILD_GO")
if err != nil {
t.Fatal(err)
}
waitFor(t, "the stream empty (every ask acknowledged)", func() bool {
si, err := info.Info(ctx)
return err == nil && si.State.Msgs == 0
})
}
// 1887/A2c, ADR 0525 §6: a killed puller's ask is redelivered by the stream's acknowledgement wait to the other node,
// whose step key names the reference.
func TestPullRedeliversAKilledPullersAsk(t *testing.T) {
js := askBus(t, 2*time.Second)
a := testAsk(delivery.AskDelivery)
publishAsk(t, js, a)
st := newSteps()
taken := make(chan struct{})
dying := &Puller{Source: &JetStreamAsks{JS: js, Language: delivery.AskGo}, Language: delivery.AskGo, Node: "ace", Put: st.put,
Capacity: func() int { return 1 }, PullWait: 200 * time.Millisecond, Heartbeat: time.Hour,
Run: func(ctx context.Context, a delivery.StepAsk) (any, error) {
close(taken)
<-ctx.Done()
return nil, ctx.Err()
}}
kill, killed := context.WithCancel(context.Background())
go dying.Pull(kill)
<-taken
killed()
other := &Puller{Source: &JetStreamAsks{JS: js, Language: delivery.AskGo}, Language: delivery.AskGo, Node: "shanks", Put: st.put,
Capacity: func() int { return 1 }, PullWait: 200 * time.Millisecond,
Run: func(ctx context.Context, a delivery.StepAsk) (any, error) {
return delivery.BuildAnswer{Artifacts: []delivery.AskBuilt{{Artifact: "bundle", Digest: "sha256:00", Reference: "artifact-store://x"}}}, nil
}}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go other.Pull(ctx)
key, _ := delivery.AskStepKey("shanks", a.ID)
deadline := time.Now().Add(10 * time.Second)
for {
if s, ok := st.get(key); ok && s.State == delivery.StepPassed {
if want := `artifact-store://x`; !contains(string(s.Result), want) {
t.Fatalf("the step's result %s names no %s", s.Result, want)
}
return
}
if time.Now().After(deadline) {
t.Fatalf("the killed puller's ask was not taken by shanks within 10 s")
}
time.Sleep(20 * time.Millisecond)
}
}
func contains(s, sub string) bool { return strings.Contains(s, sub) }
+44
View File
@@ -0,0 +1,44 @@
package buildstep
import (
"context"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// SeatAsks is a build seat's workers as a tools bundle reaches them: through its runtime's seat pull and settle
// (stdio.SeatPull, SeatSettle; issue 1887 A11), one message per pull, only from the workers the module's membership
// lists.
type SeatAsks struct {
Language string
// Pull and Settle are stdio.SeatPull and stdio.SeatSettle when nil; a test gives its own.
Pull func(worker string, wait time.Duration) (*stdio.Pulled, error)
Settle func(id, fate string) error
}
// Next is AskSource's.
func (s SeatAsks) Next(ctx context.Context, kind string, wait time.Duration) (*Pulled, error) {
pull, settle := s.Pull, s.Settle
if pull == nil {
pull = stdio.SeatPull
}
if settle == nil {
settle = stdio.SeatSettle
}
worker, err := delivery.AskWorker(s.Language, kind)
if err != nil {
return nil, err
}
if err := ctx.Err(); err != nil {
return nil, err
}
p, err := pull(worker, wait)
if err != nil || p == nil {
return nil, err
}
fate := func(f string) func() error { return func() error { return settle(p.ID, f) } }
return &Pulled{Data: p.Body, Delivered: p.Delivered, Ack: fate(stdio.SettleAck), InProgress: fate(stdio.SettleProgress),
Nak: fate(stdio.SettleNak), Term: fate(stdio.SettleTerm)}, nil
}
+40
View File
@@ -0,0 +1,40 @@
package buildstep
import (
"context"
"testing"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// 1887/A11: a bundle's source pulls the kind's worker by its seat's name and settles each ask by the runtime's id.
func TestSeatAsksPullsTheKindsWorkerAndSettlesByID(t *testing.T) {
var pulled []string
var settled []string
src := SeatAsks{Language: delivery.AskGo,
Pull: func(worker string, wait time.Duration) (*stdio.Pulled, error) {
pulled = append(pulled, worker)
if worker != "SEAT_NODE_BUILD_GO_CHECK_worker" {
return nil, nil
}
return &stdio.Pulled{ID: "p9", Worker: worker, Body: []byte(`{"v":1}`), Delivered: 2}, nil
},
Settle: func(id, fate string) error { settled = append(settled, id+" "+fate); return nil }}
if p, err := src.Next(context.Background(), delivery.AskDelivery, 0); err != nil || p != nil {
t.Fatalf("delivery = %+v, %v", p, err)
}
p, err := src.Next(context.Background(), delivery.AskCheck, time.Second)
if err != nil || p == nil || string(p.Data) != `{"v":1}` || p.Delivered != 2 {
t.Fatalf("check = %+v, %v", p, err)
}
_ = p.InProgress()
_ = p.Ack()
if len(pulled) != 2 || pulled[0] != "SEAT_NODE_BUILD_GO_DELIVERY_worker" {
t.Errorf("pulled %v", pulled)
}
if len(settled) != 2 || settled[0] != "p9 progress" || settled[1] != "p9 ack" {
t.Errorf("settled %v", settled)
}
}
+364
View File
@@ -0,0 +1,364 @@
package buildstep
import (
"context"
"encoding/json"
"errors"
"fmt"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// steps records a steps state's puts: the last value of each key.
type steps struct {
mu sync.Mutex
keys map[string]delivery.Step
}
func newSteps() *steps { return &steps{keys: map[string]delivery.Step{}} }
func (s *steps) put(key string, value any) (uint64, error) {
raw, _ := value.(json.RawMessage)
st, err := delivery.DecodeStep(raw)
if err != nil {
return 0, err
}
s.mu.Lock()
s.keys[key] = st
s.mu.Unlock()
return 1, nil
}
func (s *steps) get(key string) (delivery.Step, bool) {
s.mu.Lock()
defer s.mu.Unlock()
st, ok := s.keys[key]
return st, ok
}
func testAsk(kind string) delivery.StepAsk {
id := NewID()
a := delivery.StepAsk{ID: id, Kind: kind, Language: delivery.AskGo, Asker: "mesh-delivery-builder", At: time.Now()}
fetch := map[string]any{"repository": "novox/x", "commit": "0123456789abcdef0123456789abcdef01234567"}
if kind == delivery.AskDelivery {
b := buildJobFixture
b.Fetch = fetch
a.Build = &b
} else {
a.Check = &delivery.CheckJob{Fetch: fetch, Levels: []string{delivery.LevelUnit}}
}
return a
}
var buildJobFixture = delivery.BuildJob{Module: "m", Path: ".", Artifacts: []delivery.AskArtifact{{Artifact: "bundle"}}}
func waitFor(t *testing.T, what string, cond func() bool) {
t.Helper()
deadline := time.Now().Add(2 * time.Second)
for !cond() {
if time.Now().After(deadline) {
t.Fatalf("waited 2 s for %s", what)
}
time.Sleep(time.Millisecond)
}
}
// 1887/A2a: given a check ask published before a delivery ask and one slot, the delivery ask runs first.
func TestPullTakesDeliveryBeforeCheck(t *testing.T) {
src, st := NewFakeAsks(), newSteps()
check, recheck, deliver := testAsk(delivery.AskCheck), testAsk(delivery.AskRecheck), testAsk(delivery.AskDelivery)
for _, a := range []delivery.StepAsk{recheck, check, deliver} {
if err := src.PublishAsk(a); err != nil {
t.Fatal(err)
}
}
var mu sync.Mutex
var order []string
p := &Puller{Source: src, Language: delivery.AskGo, Node: "shanks", Put: st.put, Capacity: func() int { return 1 },
PullWait: 10 * time.Millisecond, Run: func(ctx context.Context, a delivery.StepAsk) (any, error) {
mu.Lock()
order = append(order, a.Kind)
mu.Unlock()
return map[string]string{"ran": a.ID}, nil
}}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go p.Pull(ctx)
waitFor(t, "three asks acked", func() bool { a, _, _ := src.Fates(); return len(a) == 3 })
mu.Lock()
defer mu.Unlock()
if want := []string{delivery.AskDelivery, delivery.AskCheck, delivery.AskRecheck}; len(order) != 3 || order[0] != want[0] || order[1] != want[1] || order[2] != want[2] {
t.Fatalf("order = %v, want %v", order, want)
}
key, _ := delivery.AskStepKey("shanks", deliver.ID)
s, ok := st.get(key)
if !ok || s.State != delivery.StepPassed || string(s.Result) != `{"ran":"`+deliver.ID+`"}` {
t.Fatalf("the step %s = %+v (%v)", key, s, ok)
}
}
// 1887/A2a, 1887/AC-pull: given capacity 2 and three asks waiting, two run and the third waits, and it starts when
// one ends, with no timer.
func TestPullHoldsAtMostItsCapacity(t *testing.T) {
src, st := NewFakeAsks(), newSteps()
for i := 0; i < 3; i++ {
_ = src.PublishAsk(testAsk(delivery.AskDelivery))
}
release := make(chan struct{})
var mu sync.Mutex
started := 0
p := &Puller{Source: src, Language: delivery.AskGo, Node: "shanks", Put: st.put, Capacity: func() int { return 2 },
PullWait: 10 * time.Millisecond, Run: func(ctx context.Context, a delivery.StepAsk) (any, error) {
mu.Lock()
started++
mu.Unlock()
<-release
return nil, nil
}}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go p.Pull(ctx)
waitFor(t, "two in hand", func() bool { return p.InHand() == 2 })
time.Sleep(30 * time.Millisecond) // three pull waits: the third must still wait
if p.InHand() != 2 || src.Waiting(delivery.AskDelivery) != 1 {
t.Fatalf("in hand %d, waiting %d; want 2 and 1", p.InHand(), src.Waiting(delivery.AskDelivery))
}
release <- struct{}{}
waitFor(t, "the third taken", func() bool { return src.Waiting(delivery.AskDelivery) == 0 })
// Taken from the source comes a moment before its run starts: wait for the start, not only the take.
waitFor(t, "the third started", func() bool { mu.Lock(); defer mu.Unlock(); return started == 3 })
close(release)
waitFor(t, "all acked", func() bool { a, _, _ := src.Fates(); return len(a) == 3 })
}
// An ask that is not one, or not this seat's, is dropped and runs nowhere.
func TestPullDropsWhatIsNoAsk(t *testing.T) {
src, st := NewFakeAsks(), newSteps()
src.Publish(delivery.AskDelivery, []byte(`{"v":1}`))
other := testAsk(delivery.AskDelivery)
other.Language = delivery.AskNode
_ = src.PublishAsk(other)
ran := false
p := &Puller{Source: src, Language: delivery.AskGo, Node: "shanks", Put: st.put, Capacity: func() int { return 1 },
PullWait: 10 * time.Millisecond, Run: func(ctx context.Context, a delivery.StepAsk) (any, error) { ran = true; return nil, nil }}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go p.Pull(ctx)
waitFor(t, "two termed", func() bool { _, tm, _ := src.Fates(); return len(tm) == 2 })
if ran {
t.Fatal("a wrong ask ran")
}
}
// load reads the recorded steps state's raw values: the pause key is not a step, so it is kept apart.
type kv struct {
mu sync.Mutex
vals map[string][]byte
}
func (k *kv) put(key string, value any) (uint64, error) {
raw, _ := value.(json.RawMessage)
k.mu.Lock()
defer k.mu.Unlock()
if k.vals == nil {
k.vals = map[string][]byte{}
}
k.vals[key] = append([]byte(nil), raw...)
return 1, nil
}
func (k *kv) load(key string) ([]byte, error) {
k.mu.Lock()
defer k.mu.Unlock()
return k.vals[key], nil
}
// 1887/A2b, 1887/AC-zero, 1887/AC-wake: at capacity 0 the puller takes nothing; a capacity raised takes effect at the
// next pull, or, with the config watch (g14 idle at capacity 2, 2026-10-25), at once when the value moves, with no
// pause, resume or step end to wake it; a refused value pulls nothing, and a watch that cannot start leaves the
// settings file's capacity.
func TestPullAtCapacityZeroTakesNothing(t *testing.T) {
t.Run("settings file", func(t *testing.T) {
src, st := NewFakeAsks(), &kv{}
_ = src.PublishAsk(testAsk(delivery.AskDelivery))
var mu sync.Mutex
capacity := 0
p := &Puller{Source: src, Language: delivery.AskGo, Node: "novox", Put: st.put, PullWait: 10 * time.Millisecond,
Capacity: func() int { mu.Lock(); defer mu.Unlock(); return capacity },
Run: func(ctx context.Context, a delivery.StepAsk) (any, error) { return nil, nil }}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go p.Pull(ctx)
time.Sleep(50 * time.Millisecond)
if src.Waiting(delivery.AskDelivery) != 1 {
t.Fatal("a node at capacity 0 took an ask")
}
mu.Lock()
capacity = 1
mu.Unlock()
p.Wake()
waitFor(t, "the ask taken at capacity 1", func() bool { a, _, _ := src.Fates(); return len(a) == 1 })
})
t.Run("config watch wakes", func(t *testing.T) {
src, st := NewFakeAsks(), newSteps()
_ = src.PublishAsk(testAsk(delivery.AskDelivery))
w := &configWatch{value: "shanks=3,g14=0"}
p := &Puller{Source: src, Language: delivery.AskGo, Node: "g14", Put: st.put, PullWait: 10 * time.Millisecond,
Capacity: func() int { return 0 }, // the settings file, not refilled yet
WatchConfig: w.watch,
Run: func(ctx context.Context, a delivery.StepAsk) (any, error) { return nil, nil }}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go p.Pull(ctx)
waitFor(t, "the watch of the capacity", func() bool { w.mu.Lock(); defer w.mu.Unlock(); return w.handle != nil })
time.Sleep(30 * time.Millisecond)
if src.Waiting(delivery.AskDelivery) != 1 {
t.Fatal("a node at capacity 0 took an ask")
}
if w.asked != CapacitySetting {
t.Fatalf("the puller watches %q, want %q", w.asked, CapacitySetting)
}
w.set(t, "shanks=3,g14=2")
waitFor(t, "the ask taken once the capacity moved to 2", func() bool { a, _, _ := src.Fates(); return len(a) == 1 })
})
for _, c := range []struct {
name string
watch func(string, func(stdio.ProvisionChange) error) error
room bool
}{
{"refused value leaves no room", func(key string, handle func(stdio.ProvisionChange) error) error {
return handle(stdio.ProvisionChange{Kind: "config", Key: key, Refused: "the module is no longer on g14"})
}, false},
{"watch that cannot start keeps the settings file", func(string, func(stdio.ProvisionChange) error) error {
return errors.New("no runtime")
}, true},
} {
t.Run(c.name, func(t *testing.T) {
p := &Puller{Source: NewFakeAsks(), Language: delivery.AskGo, Node: "g14", Put: newSteps().put,
Capacity: func() int { return 1 }, WatchConfig: c.watch,
Run: func(ctx context.Context, a delivery.StepAsk) (any, error) { return nil, nil }}
p.watchCapacity()
if p.room() != c.room {
t.Fatalf("room = %v, want %v", p.room(), c.room)
}
})
}
}
// 1887/A2b: a paused puller takes nothing, its pause is kept on the pause key, and a puller started again on that state
// stays paused until resumed.
func TestPullPauseOutlivesARestart(t *testing.T) {
src, st, steps := NewFakeAsks(), &kv{}, &kv{}
run := func(ctx context.Context, a delivery.StepAsk) (any, error) { return nil, nil }
p := &Puller{Source: src, Language: delivery.AskGo, Node: "shanks", Put: steps.put, PausePut: st.put, Load: st.load,
PullWait: 10 * time.Millisecond, Capacity: func() int { return 2 }, Run: run}
if err := p.Pause("the operator"); err != nil {
t.Fatal(err)
}
key, _ := delivery.PauseKey("shanks")
raw, _ := st.load(key)
if v, err := delivery.DecodePause(raw); err != nil || !v.Paused || v.By != "the operator" {
t.Fatalf("the pause key holds %s (%v)", raw, err)
}
if raw, _ := steps.load(key); raw != nil {
t.Fatalf("the pause was written in the steps state, whose keys age out: %s", raw)
}
// The restart: a new puller on the same state.
again := &Puller{Source: src, Language: delivery.AskGo, Node: "shanks", Put: steps.put, PausePut: st.put, Load: st.load,
PullWait: 10 * time.Millisecond, Capacity: func() int { return 2 }, Run: run}
_ = src.PublishAsk(testAsk(delivery.AskDelivery))
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go again.Pull(ctx)
time.Sleep(50 * time.Millisecond)
if !again.Paused() || src.Waiting(delivery.AskDelivery) != 1 {
t.Fatalf("restarted: paused %v, waiting %d; want paused and 1", again.Paused(), src.Waiting(delivery.AskDelivery))
}
if err := again.Resume("the operator"); err != nil {
t.Fatal(err)
}
waitFor(t, "the ask taken after the resume", func() bool { a, _, _ := src.Fates(); return len(a) == 1 })
t.Run("a pause that cannot be read is said and pulled as not paused", func(t *testing.T) {
src := NewFakeAsks()
var said atomic.Value
p := &Puller{Source: src, Language: delivery.AskGo, Node: "shanks", Put: steps.put, PausePut: st.put,
Load: func(string) ([]byte, error) { return nil, errors.New("no runtime yet") },
PullWait: 10 * time.Millisecond, Capacity: func() int { return 2 }, Run: run,
Logf: func(f string, a ...any) {
if m := fmt.Sprintf(f, a...); strings.Contains(m, "cannot be read") {
said.Store(m)
}
}}
_ = src.PublishAsk(testAsk(delivery.AskDelivery))
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go p.Pull(ctx)
waitFor(t, "the ask taken", func() bool { a, _, _ := src.Fates(); return len(a) == 1 })
if m, _ := said.Load().(string); !strings.Contains(m, "pulling as not paused") {
t.Fatalf("the unreadable pause was not said: %q", m)
}
})
}
// 1887/A2c: a long step tells the bus it is still being worked; a holder that stops mid-step neither ends nor
// acknowledges its ask, so the stream gives it to another node.
func TestPullHeartbeatAndStop(t *testing.T) {
src, st := NewFakeAsks(), newSteps()
a := testAsk(delivery.AskDelivery)
_ = src.PublishAsk(a)
running := make(chan struct{})
p := &Puller{Source: src, Language: delivery.AskGo, Node: "shanks", Put: st.put, Capacity: func() int { return 1 },
PullWait: 10 * time.Millisecond, Heartbeat: 5 * time.Millisecond,
Run: func(ctx context.Context, a delivery.StepAsk) (any, error) {
close(running)
<-ctx.Done()
return nil, ctx.Err()
}}
ctx, cancel := context.WithCancel(context.Background())
stopped := make(chan struct{})
go func() { p.Pull(ctx); close(stopped) }()
<-running
waitFor(t, "two heartbeats", func() bool { src.mu.Lock(); defer src.mu.Unlock(); return src.InProgress >= 2 })
cancel()
<-stopped
waitFor(t, "the step's goroutine gone", func() bool { return p.InHand() == 0 })
acked, termed, naked := src.Fates()
if len(acked)+len(termed)+len(naked) != 0 {
t.Fatalf("a stopped holder gave its ask a fate: acked %v termed %v naked %v", acked, termed, naked)
}
key, _ := delivery.AskStepKey("shanks", a.ID)
if s, ok := st.get(key); !ok || s.State != delivery.StepRunning {
t.Fatalf("the step %s = %+v, want left running", key, s)
}
}
// 1907: an ask taken past AskRuns (each run took its node down) is run no more: its step ends failed saying why, so the
// asker's key fails, and it is ended on the stream, not offered to another node.
func TestAPoisonedAskIsSaidAndRunNoMore(t *testing.T) {
src, st := NewFakeAsks(), newSteps()
src.Delivered = delivery.AskRuns + 1
a := testAsk(delivery.AskDelivery)
_ = src.PublishAsk(a)
ran := false
p := &Puller{Source: src, Language: delivery.AskGo, Node: "shanks", Put: st.put, Capacity: func() int { return 1 },
PullWait: 10 * time.Millisecond, Run: func(ctx context.Context, a delivery.StepAsk) (any, error) { ran = true; return nil, nil }}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go p.Pull(ctx)
waitFor(t, "the ask termed", func() bool { _, tm, _ := src.Fates(); return len(tm) == 1 })
key, _ := delivery.AskStepKey("shanks", a.ID)
s, ok := st.get(key)
if ran || !ok || s.State != delivery.StepFailed || !strings.Contains(s.Reason, "taken 3 times") || !strings.Contains(s.Reason, "poison") {
t.Fatalf("ran %v, step %+v", ran, s)
}
if acked, _, naked := src.Fates(); len(acked)+len(naked) != 0 {
t.Errorf("acked %v naked %v, want it termed alone", acked, naked)
}
}
+5
View File
@@ -1,5 +1,7 @@
package buildstep
import "fmt"
// Refusal is a step that is not run on purpose, as against one that failed: the toolchain installed is not the
// recipe's, the integration level's postgres major is not the one it is built for (novox/hq ADR 0469 §1, ADR 0470
// §4; issue 1473). A step's run returns one and the runner ends the step refused with its text.
@@ -11,3 +13,6 @@ type Refusal struct {
}
func (r *Refusal) Error() string { return r.Text }
// Refuse is a Refusal with its text formatted.
func Refuse(format string, args ...any) *Refusal { return &Refusal{Text: fmt.Sprintf(format, args...)} }
+36
View File
@@ -0,0 +1,36 @@
package buildstep
import (
stdio "git.novox.be/novox/mesh-sdk/go"
)
// SeatTools are a build seat's verbs on one node, as its holder serves them (novox/hq ADR 0525 §10; mesh-sdk catalogue
// buildSeatVerbs): `<seat>.current` (what the node runs, its capacity, whether it is paused), `<seat>.pause` and
// `<seat>.resume`, the pause kept on the node's pause key by the puller. what names the asks in the descriptions ("Go").
func SeatTools(seat, what string, p *Puller) []stdio.Tool {
verb := func(name, description string, run func() (any, error)) stdio.Tool {
return stdio.Tool{Name: seat + "." + name, Description: description,
Input: map[string]any{"type": "object", "properties": map[string]any{}},
Run: func(map[string]any) (any, error) { return run() }}
}
current := func() (any, error) {
return map[string]any{"node": p.Node, "capacity": p.CapacityNow(), "running": p.Running(), "paused": p.Paused()}, nil
}
return []stdio.Tool{
verb("current", "The "+what+" asks this node runs now (each one's id, kind and when it started), its build capacity, "+
"and whether it is paused.", current),
verb("pause", "Take no new "+what+" ask on this node until resumed; an ask running here ends in its place. Kept "+
"across a restart (the node's pause key).", func() (any, error) {
if err := p.Pause("the seat's pause verb"); err != nil {
return nil, err
}
return current()
}),
verb("resume", "Take "+what+" asks again on this node.", func() (any, error) {
if err := p.Resume("the seat's resume verb"); err != nil {
return nil, err
}
return current()
}),
}
}
+52
View File
@@ -0,0 +1,52 @@
package buildstep
import (
"context"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// 1887/A9a, ADR 0525 §10: a build seat's verbs are <seat>.current, .pause and .resume; current names the ask running,
// pause stops the puller taking asks, resume lets it take them again.
func TestSeatToolsCurrentPauseResume(t *testing.T) {
src, st := NewFakeAsks(), newSteps()
release := make(chan struct{})
p := &Puller{Source: src, Language: delivery.AskGo, Node: "shanks", Put: st.put, Capacity: func() int { return 1 },
PullWait: 10 * time.Millisecond, Run: func(ctx context.Context, a delivery.StepAsk) (any, error) { <-release; return nil, nil }}
tools := map[string]func(map[string]any) (any, error){}
for _, tool := range SeatTools("node-build-go", "Go", p) {
tools[tool.Name] = tool.Run
}
for _, name := range []string{"node-build-go.current", "node-build-go.pause", "node-build-go.resume"} {
if tools[name] == nil {
t.Fatalf("no %s", name)
}
}
a := testAsk(delivery.AskDelivery)
_ = src.PublishAsk(a)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go p.Pull(ctx)
waitFor(t, "the ask running", func() bool { return p.InHand() == 1 })
out, _ := tools["node-build-go.current"](nil)
cur := out.(map[string]any)
running := cur["running"].([]RunningAsk)
if len(running) != 1 || running[0].ID != a.ID || running[0].Module != "m" || cur["capacity"] != 1 || cur["paused"] != false {
t.Fatalf("current = %+v", cur)
}
if out, err := tools["node-build-go.pause"](nil); err != nil || out.(map[string]any)["paused"] != true {
t.Fatalf("pause = %v, %v", out, err)
}
close(release)
_ = src.PublishAsk(testAsk(delivery.AskDelivery))
time.Sleep(50 * time.Millisecond)
if src.Waiting(delivery.AskDelivery) != 1 {
t.Fatal("a paused node took an ask")
}
if _, err := tools["node-build-go.resume"](nil); err != nil {
t.Fatal(err)
}
waitFor(t, "the ask taken after the resume", func() bool { return src.Waiting(delivery.AskDelivery) == 0 })
}
+78 -31
View File
@@ -10,6 +10,7 @@ import (
"encoding/json"
"errors"
"fmt"
"log"
"strconv"
"time"
@@ -42,43 +43,89 @@ func NewID() string {
// the error's text, or refused when the error is a *Refusal; a panic in run ends the step failed, naming it, and
// the process goes on. If the running write fails, nothing was started and Start answers the error.
func Start(ctx context.Context, put Put, node, tool string, run func(ctx context.Context) (result any, err error)) (string, error) {
step := delivery.Step{ID: NewID(), Tool: tool, Node: node, State: delivery.StepRunning, Started: now().UTC()}
key, err := delivery.StepKey(node, step.ID)
step, key, err := begin(put, node, tool, NewID())
if err != nil {
return "", err
}
if err := write(put, key, step); err != nil {
return "", fmt.Errorf("buildstep: writing the running step %s: %w", key, err)
}
go func() {
result, err := protect(ctx, run)
step.Ended = now().UTC()
switch ref := (*Refusal)(nil); {
case err == nil:
step.State = delivery.StepPassed
if result != nil {
raw, merr := json.Marshal(result)
if merr != nil {
step.State, step.Reason = delivery.StepFailed, "the result is not JSON: "+merr.Error()
} else {
step.Result = raw
}
}
case errors.As(err, &ref):
step.State, step.Reason = delivery.StepRefused, err.Error()
default:
step.State, step.Reason = delivery.StepFailed, err.Error()
}
for attempt := 1; ; attempt++ {
if werr := write(put, key, step); werr == nil || attempt >= EndAttempts {
return
}
time.Sleep(EndRetry)
}
}()
go end(ctx, put, key, step, run)
return step.ID, nil
}
// begin writes the step running under id.
func begin(put Put, node, tool, id string) (delivery.Step, string, error) {
step := delivery.Step{ID: id, Tool: tool, Node: node, State: delivery.StepRunning, Started: now().UTC()}
key, err := delivery.StepKey(node, step.ID)
if err != nil {
return delivery.Step{}, "", err
}
if err := write(put, key, step); err != nil {
return delivery.Step{}, "", fmt.Errorf("buildstep: writing the running step %s: %w", key, err)
}
return step, key, nil
}
// end runs run and writes the step's end.
func end(ctx context.Context, put Put, key string, step delivery.Step, run func(ctx context.Context) (any, error)) {
result, err := protect(ctx, run)
_ = finish(put, key, step, result, err, log.Printf) // finish says a refused end itself
}
// finish writes the step's end from run's result or error: passed, failed or refused (a *Refusal). A write that fails is
// tried EndAttempts times. When the whole end is refused every time (a result the state will not take: issue 1930), the
// refusal is logged and a small failed end without the result is written instead, naming the write's error, so the
// step's reader learns it is over and why. finish answers an error only when no end could be written at all, and then
// the step's ask is not acknowledged as done.
func finish(put Put, key string, step delivery.Step, result any, err error, logf func(format string, args ...any)) error {
step.Ended = now().UTC()
switch ref := (*Refusal)(nil); {
case err == nil:
step.State = delivery.StepPassed
if result != nil {
raw, merr := json.Marshal(result)
if merr != nil {
step.State, step.Reason = delivery.StepFailed, "the result is not JSON: "+merr.Error()
} else {
step.Result = raw
}
}
case errors.As(err, &ref):
step.State, step.Reason = delivery.StepRefused, err.Error()
default:
step.State, step.Reason = delivery.StepFailed, err.Error()
}
werr := writeEnd(put, key, step)
if werr == nil {
return nil
}
if logf != nil {
logf("the end of the step %s (%s) could not be written in %d tries: %v", key, step.State, EndAttempts, werr)
}
ended := step.State
why := fmt.Sprintf("the step ended %s, and its end could not be written: %v", ended, werr)
if step.Reason != "" {
why = fmt.Sprintf("the step ended %s (%s), and its end could not be written: %v", ended, step.Reason, werr)
}
step.State, step.Reason, step.Result = delivery.StepFailed, why, nil
if ferr := writeEnd(put, key, step); ferr != nil {
if logf != nil {
logf("the failed end of the step %s could not be written either: %v", key, ferr)
}
return fmt.Errorf("buildstep: no end of the step %s could be written: %w", key, ferr)
}
return nil
}
// writeEnd writes the step's end, trying EndAttempts times.
func writeEnd(put Put, key string, step delivery.Step) error {
for attempt := 1; ; attempt++ {
werr := write(put, key, step)
if werr == nil || attempt >= EndAttempts {
return werr
}
time.Sleep(EndRetry)
}
}
// protect is run with a panic turned into the error that ends the step.
func protect(ctx context.Context, run func(ctx context.Context) (any, error)) (result any, err error) {
defer func() {
+160
View File
@@ -0,0 +1,160 @@
package buildstep
// A build module's room to work and its workspaces (novox/hq issue 1839, ADR 0509): a step is refused while the
// filesystem it writes to has less free than the floor, and a workspace is removed when its caller releases it, or,
// unheld, once its last use is past a bound (a lost caller's), never while a running step holds it.
import (
"fmt"
"os"
"path/filepath"
"regexp"
"sort"
"sync"
"syscall"
"time"
)
// FreeSpaceFloor is the free space a build module needs on its work and cache filesystems to take a step: the floor
// build-agent keeps (mesh-controller internal/builder/floor.go, issue 1834).
const FreeSpaceFloor uint64 = 20 << 30
// UnheldBound is how long a workspace no step holds is kept after its last use, when its caller never released it
// (ADR 0509: a lost caller's workspace goes after two hours).
const UnheldBound = 2 * time.Hour
// FreeBytes reads the bytes an unprivileged writer can still use on the filesystem holding path.
func FreeBytes(path string) (uint64, error) {
var s syscall.Statfs_t
if err := syscall.Statfs(path, &s); err != nil {
return 0, err
}
return s.Bavail * uint64(s.Bsize), nil
}
// Floor refuses a step while a filesystem it writes to has less free than Bytes. A zero Bytes checks nothing.
type Floor struct {
// Node is named in the refusal.
Node string
Bytes uint64
// Free reads a path's free bytes; FreeBytes when nil. A test answers for a disk it does not have.
Free func(path string) (uint64, error)
}
// Check answers a *Refusal, "refused: <node> has N GiB free under the floor of F GiB (on <path>)", when any path's
// filesystem has less free than the floor, or its free space cannot be read; nil otherwise. A path not made yet is
// read at its nearest folder that is; an empty path is skipped.
func (f Floor) Check(paths ...string) error {
if f.Bytes == 0 {
return nil
}
free := f.Free
if free == nil {
free = FreeBytes
}
for _, p := range paths {
if p == "" {
continue
}
at := p
for {
if _, err := os.Stat(at); err == nil || filepath.Dir(at) == at {
break
}
at = filepath.Dir(at)
}
n, err := free(at)
if err != nil {
return &Refusal{Text: fmt.Sprintf("refused: the free space of %s on %s cannot be read: %v", p, f.Node, err)}
}
if n < f.Bytes {
return &Refusal{Text: fmt.Sprintf("refused: %s has %d GiB free under the floor of %d GiB (on %s)", f.Node, n>>30, f.Bytes>>30, p),
Want: fmt.Sprintf("%d GiB", f.Bytes>>30), Have: fmt.Sprintf("%d GiB", n>>30)}
}
}
return nil
}
var workspaceID = regexp.MustCompile(`^[a-z0-9]+$`)
// Workspaces are a build module's workspaces, each <Dir>/<id>, and which of them running steps hold.
type Workspaces struct {
Dir string
mu sync.Mutex
held map[string]int
}
// Hold marks the workspace held by a running step until the answered func is called; both mark it used now, so the
// sweep counts its bound from the step's end. An id that is not a workspace's holds nothing.
func (w *Workspaces) Hold(id string) func() {
if !workspaceID.MatchString(id) {
return func() {}
}
w.mu.Lock()
if w.held == nil {
w.held = map[string]int{}
}
w.held[id]++
w.mu.Unlock()
w.touch(id)
var once sync.Once
return func() {
once.Do(func() {
w.touch(id)
w.mu.Lock()
if w.held[id]--; w.held[id] <= 0 {
delete(w.held, id)
}
w.mu.Unlock()
})
}
}
func (w *Workspaces) touch(id string) {
now := time.Now()
_ = os.Chtimes(filepath.Join(w.Dir, id), now, now) // a workspace not made yet has nothing to mark
}
// Release removes the workspace at once: its caller's chain of steps has ended. One already gone is no error; one a
// running step holds is refused and kept; an id that is not a workspace's is refused.
func (w *Workspaces) Release(id string) error {
if !workspaceID.MatchString(id) {
return fmt.Errorf("%q is not a workspace id", id)
}
w.mu.Lock()
defer w.mu.Unlock()
if w.held[id] > 0 {
return fmt.Errorf("the workspace %s is held by a running step", id)
}
if err := os.RemoveAll(filepath.Join(w.Dir, id)); err != nil {
return fmt.Errorf("the workspace %s could not be removed: %w", id, err)
}
return nil
}
// Sweep removes every workspace no step holds whose last use is older than bound, and answers the ids it removed,
// sorted. It is the safety net for a caller that never released its workspace.
func (w *Workspaces) Sweep(now time.Time, bound time.Duration) []string {
entries, err := os.ReadDir(w.Dir)
if err != nil {
return nil
}
removed := []string{}
w.mu.Lock()
defer w.mu.Unlock()
for _, e := range entries {
if !e.IsDir() || !workspaceID.MatchString(e.Name()) || w.held[e.Name()] > 0 {
continue
}
info, err := e.Info()
if err != nil || now.Sub(info.ModTime()) < bound {
continue
}
if os.RemoveAll(filepath.Join(w.Dir, e.Name())) == nil {
removed = append(removed, e.Name())
}
}
sort.Strings(removed)
return removed
}
+113
View File
@@ -0,0 +1,113 @@
package buildstep
import (
"errors"
"os"
"path/filepath"
"testing"
"time"
)
// 1839/AC1: a filesystem under the floor refuses the step, naming the node, what is free and the floor.
func TestAFilesystemUnderTheFloorRefusesNamingNodeFreeAndFloor(t *testing.T) {
dir := t.TempDir()
f := Floor{Node: "ace", Bytes: 20 << 30, Free: func(string) (uint64, error) { return 4<<30 + 5, nil }}
err := f.Check(dir)
var ref *Refusal
if !errors.As(err, &ref) {
t.Fatalf("Check = %v, want a *Refusal", err)
}
if want := "refused: ace has 4 GiB free under the floor of 20 GiB (on " + dir + ")"; err.Error() != want {
t.Errorf("Check = %q, want %q", err, want)
}
}
// 1839/AC1: at or above the floor, and with no floor set, nothing is refused.
func TestAtTheFloorOrWithNoFloorNothingIsRefused(t *testing.T) {
dir := t.TempDir()
if err := (Floor{Node: "ace", Bytes: 20 << 30, Free: func(string) (uint64, error) { return 20 << 30, nil }}).Check(dir); err != nil {
t.Errorf("at the floor: %v", err)
}
if err := (Floor{Node: "ace", Free: func(string) (uint64, error) { return 0, nil }}).Check(dir); err != nil {
t.Errorf("no floor: %v", err)
}
}
// 1839/AC1: a path not made yet is read at its nearest folder that is; a free space that cannot be read is a refusal.
func TestAPathNotMadeIsReadAtItsParentAndAnUnreadableOneRefuses(t *testing.T) {
dir := t.TempDir()
var read string
f := Floor{Node: "ace", Bytes: 1, Free: func(p string) (uint64, error) { read = p; return 2, nil }}
if err := f.Check(filepath.Join(dir, "not", "made")); err != nil || read != dir {
t.Errorf("Check = %v, read %q, want nil and %q", err, read, dir)
}
f.Free = func(string) (uint64, error) { return 0, errors.New("statfs: boom") }
var ref *Refusal
if err := f.Check(dir); !errors.As(err, &ref) {
t.Errorf("Check = %v, want a refusal", err)
}
}
func makeWorkspace(t *testing.T, w *Workspaces, id string, age time.Duration) string {
t.Helper()
p := filepath.Join(w.Dir, id)
if err := os.MkdirAll(filepath.Join(p, "tree"), 0o700); err != nil {
t.Fatal(err)
}
at := time.Now().Add(-age)
if err := os.Chtimes(p, at, at); err != nil {
t.Fatal(err)
}
return p
}
func gone(p string) bool { _, err := os.Stat(p); return os.IsNotExist(err) }
// 1839/AC2: a released workspace is gone at once; one already gone is no error; one a running step holds is refused
// and kept; an id that is not a workspace's is refused.
func TestReleaseRemovesAWorkspaceAtOnce(t *testing.T) {
w := &Workspaces{Dir: t.TempDir()}
p := makeWorkspace(t, w, "abc1", 0)
if err := w.Release("abc1"); err != nil || !gone(p) {
t.Fatalf("Release = %v, gone %v", err, gone(p))
}
if err := w.Release("abc1"); err != nil {
t.Errorf("a second release = %v, want nil", err)
}
held := makeWorkspace(t, w, "abc2", 0)
done := w.Hold("abc2")
if err := w.Release("abc2"); err == nil || gone(held) {
t.Errorf("Release of a held workspace = %v, gone %v; want refused and kept", err, gone(held))
}
done()
if err := w.Release("abc2"); err != nil || !gone(held) {
t.Errorf("Release after the hold ended = %v, gone %v", err, gone(held))
}
if err := w.Release("../etc"); err == nil {
t.Errorf("Release of ../etc answered nil")
}
}
// 1839/AC3: the sweep removes a workspace no step holds whose last use is past the bound, and keeps a held one and a
// recent one; a hold ending counts as a use.
func TestSweepRemovesUnheldWorkspacesPastTheBound(t *testing.T) {
w := &Workspaces{Dir: t.TempDir()}
old := makeWorkspace(t, w, "old1", 3*time.Hour)
heldOld := makeWorkspace(t, w, "held1", 3*time.Hour)
recent := makeWorkspace(t, w, "new1", time.Minute)
if err := os.WriteFile(filepath.Join(w.Dir, "not-a-workspace.txt"), nil, 0o600); err != nil {
t.Fatal(err)
}
done := w.Hold("held1")
removed := w.Sweep(time.Now(), 2*time.Hour)
if !gone(old) || gone(heldOld) || gone(recent) {
t.Errorf("after the sweep: old gone %v, held kept %v, recent kept %v", gone(old), !gone(heldOld), !gone(recent))
}
if len(removed) != 1 || removed[0] != "old1" {
t.Errorf("Sweep answered %v, want [old1]", removed)
}
done()
if w.Sweep(time.Now(), 2*time.Hour); gone(heldOld) {
t.Errorf("a workspace whose hold just ended was swept: its end is a use")
}
}
+125
View File
@@ -0,0 +1,125 @@
package buscompose
import (
"fmt"
"sort"
"strings"
)
// Do the emitters and the consumers of a catalogue agree?
//
// **The check that was missing** (novox/hq 04-ISSUES/127). Every manifest was individually
// well-formed and every derivation individually correct, and no cross-module subscription in the
// mesh matched anything: a consumer's declaration derived into a namespace nobody publishes to.
// Nothing failed, because a subscription that matches nothing is not an error — it is silence.
//
// The comparison has to be over the whole catalogue, because the two halves live in different
// manifests, and it cannot simply demand that every consumed event have a live emitter: a module
// may be installed long before the one whose events it wants. So the rule is narrower and still
// catches this: **where the emitter is present, it must emit what the consumer asked for.**
// AConsumer is one module's interest in another's events, as this check needs it.
type AConsumer struct {
Module string
Consumes []string
}
// AnEmitter is one module's events.
type AnEmitter struct {
Module string
Emits []string
}
// Disagreements are the consumed events whose emitter is in the catalogue and does not emit them.
//
// Returned as sentences rather than as structs: every one of them is read by a person deciding
// whether a manifest or a catalogue is wrong, and a pair of names without the reason is a puzzle.
func Disagreements(emitters []AnEmitter, consumers []AConsumer, seats []DeclaredSeat) []string {
emits := map[string]map[string]bool{}
for _, e := range emitters {
if emits[e.Module] == nil {
emits[e.Module] = map[string]bool{}
}
for _, name := range e.Emits {
emits[e.Module][name] = true
}
}
// A seat's events are published by its holder under the seat's name, so a consumer naming the
// seat is naming something real even though no module declares it as its own.
for _, s := range seats {
if len(s.Emits) == 0 {
continue
}
if emits[s.Name] == nil {
emits[s.Name] = map[string]bool{}
}
for _, name := range s.Emits {
emits[s.Name][name] = true
}
}
var out []string
for _, c := range consumers {
for _, pattern := range c.Consumes {
emitter, event, named := strings.Cut(pattern, ".")
// Every event from everyone, or every event from one module: both are deliberate and
// neither names a particular event to check.
if !named || emitter == "*" || emitter == catalogueTheRest || event == catalogueTheRest {
continue
}
known, present := emits[emitter]
if !present {
// Not installed here, which is ordinary: a module lives in its own repository and
// may be registered later. Nothing to compare, so nothing to say.
continue
}
if matchesAny(event, known) {
continue
}
out = append(out, fmt.Sprintf(
"%s consumes %q and %s emits %s — so that subscription would match nothing, and "+
"nothing would report it",
c.Module, pattern, emitter, listOf(known)))
}
}
sort.Strings(out)
return out
}
// matchesAny says whether one of an emitter's event names satisfies a consumer's pattern.
func matchesAny(pattern string, emitted map[string]bool) bool {
want := strings.Split(pattern, ".")
for name := range emitted {
if matches(want, strings.Split(name, ".")) {
return true
}
}
return false
}
func matches(pattern, name []string) bool {
for i, part := range pattern {
if part == catalogueTheRest {
return i < len(name)
}
if i >= len(name) {
return false
}
if part != "*" && part != name[i] {
return false
}
}
return len(pattern) == len(name)
}
func listOf(names map[string]bool) string {
if len(names) == 0 {
return "nothing"
}
out := make([]string, 0, len(names))
for n := range names {
out = append(out, n)
}
sort.Strings(out)
return strings.Join(out, ", ")
}
+82
View File
@@ -0,0 +1,82 @@
package buscompose
import (
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/announce"
)
// Who may read a module's state is what the module announced (novox/hq ADR 0495 clauses 2 and 3, ADR 0513 §3; issue
// 1790). A module says on the bus which states it keeps and which others may read (announce.Subject); the controller
// keeps the last word of each (announce.KeptBucket) and grants a reader's `reads: <owner>.<name>` only when the owner
// announced that state readable. An owner that has announced nothing yet is read as before, from the reader's `reads`
// (the transition, until every module announces), and is named as not yet announced.
// StatesAnnouncedBucket is the controller's bucket of the last announcement of each module (announce.KeptBucket).
const StatesAnnouncedBucket = announce.KeptBucket
// controllerReadable are the controller's states others may read: the record mirrors (the packages each node declares
// among them, which the pacman module reads, ADR 0507 §5), the chain's buckets, and the grants issued and the
// compositions' digests, which the assigner reads (ADR 0528 clause 2, ADR 0539 clause 6).
var controllerReadable = []string{MirrorAssignments, MirrorDeclaredConfig, MirrorPredefined, ChainBuilds, ChainRollouts,
MirrorDeclaredPackages, ControllerGrants, ControllerComposed}
// ControllerAnnouncement is what the controller announces it keeps (ADR 0495 clause 2): every bucket of its own by its local
// name, the ones others may read marked readable. It announces itself, as any module does from its code.
func ControllerAnnouncement(node string) announce.States {
s := announce.States{Module: ControllerSeat, Node: node}
readable := map[string]bool{}
for _, n := range controllerReadable {
readable[n] = true
}
for _, bucket := range ControllerBuckets() {
local, mine := strings.CutPrefix(bucket, ControllerSeat+"_")
if !mine {
continue
}
s.States = append(s.States, announce.State{Name: local, Readable: readable[local]})
}
sort.Slice(s.States, func(i, j int) bool { return s.States[i].Name < s.States[j].Name })
return s
}
// ReadsAnnounced splits a reader's `reads` (`<owner>.<name>`) by what each owner announced: granted when the owner
// announced the state readable, or announced nothing yet (the transition); refused when the owner announced and did
// not announce that state readable.
func ReadsAnnounced(reads []string, announced map[string]announce.States) (granted, refused []string) {
for _, r := range reads {
owner, name, ok := strings.Cut(r, ".")
s, said := announced[owner]
switch {
case !ok || !said || s.Readable(name):
granted = append(granted, r)
default:
refused = append(refused, r)
}
}
return granted, refused
}
// NotAnnounced are the owners some module reads the state of and that have announced nothing yet, sorted: read from the
// readers' `reads` as before, until they announce.
func NotAnnounced(r Records) []string {
seen := map[string]bool{}
var out []string
for _, ds := range r.Assigned {
for _, d := range ds {
for _, read := range append(append([]string(nil), d.Reads...), d.ReadsRefused...) {
owner, _, ok := strings.Cut(read, ".")
if !ok || seen[owner] {
continue
}
if _, said := r.Announced[owner]; !said {
seen[owner] = true
out = append(out, owner)
}
}
}
}
sort.Strings(out)
return out
}
+47
View File
@@ -0,0 +1,47 @@
package buscompose
import "git.novox.be/novox/mesh-sdk/go/delivery"
// The view alone reads the build seats' queues (novox/hq ADR 0525 §2, issue 1887 A8a): the shared workers' counts and the
// oldest waiting ask, to show the backlog. It may not publish an ask: only the builder and the checker ask, by their seat use.
// The watchman is granted nothing here: a waiting ask has no pulse, a taken one is read from the build module's steps (issue
// 1908). Nothing is a wildcard over a stream or a worker.
const (
// ViewModule is the module of the delivery view (mesh-delivery-view).
ViewModule = "mesh-delivery-view"
// BuilderModule is the module of the builder (mesh-delivery-builder).
BuilderModule = "mesh-delivery-builder"
)
// BuildAskReads are the API subjects the builder requests when it takes over, at its start, a key naming an ask (issue
// 1910): each build seat's stream's information and its messages by sequence, to tell whether the ask is on its stream.
// Reads only: no consumer, no pull, no acknowledgement; the builder puts its asks by its seat use, as before.
func BuildAskReads() []string {
var out []string
for _, lang := range delivery.AskLanguages {
if stream, err := delivery.AskStream(lang); err == nil {
out = append(out, "$JS.API.STREAM.INFO."+stream, "$JS.API.STREAM.MSG.GET."+stream)
}
}
return out
}
// BuildBacklogReads are the API subjects the view requests: the information of each build seat's stream and of each of its
// shared workers (delivery.AskStream, delivery.AskWorker), and the message of the stream by sequence. Reads only: no
// message is pulled, none acknowledged, none published.
func BuildBacklogReads() []string {
var out []string
for _, lang := range delivery.AskLanguages {
stream, err := delivery.AskStream(lang)
if err != nil {
continue
}
out = append(out, "$JS.API.STREAM.INFO."+stream, "$JS.API.STREAM.MSG.GET."+stream)
for _, kind := range delivery.AskKinds {
if w, err := delivery.AskWorker(lang, kind); err == nil {
out = append(out, "$JS.API.CONSUMER.INFO."+stream+"."+w)
}
}
}
return out
}
+102
View File
@@ -0,0 +1,102 @@
package buscompose
import (
"encoding/json"
"os"
"path/filepath"
"reflect"
"strings"
"testing"
)
// Defends issue 1377 and novox/hq ADR 0547 §5: this is the controller's bus composer copied, and it composes byte for
// byte what the controller's composes. testdata/controller-user-lists.json was written by the controller's own
// internal/broker (mesh-controller branch fix/1945-mesh-grants-publishes-memberships, and fix/1790-grants-create-buckets for the grants part creating declared streams), lists and memberships, from fixtures_test.go compiled there unchanged; composed.conf is the
// controller's golden of the whole server configuration. A row that differs is the copy drifting from the controller,
// which is a fault while both compose (until the controller loses its lease, ADR 0547 §13).
func TestTheCopiedComposerComposesAsTheControllers(t *testing.T) {
raw, err := os.ReadFile(filepath.Join("testdata", "controller-user-lists.json"))
if err != nil {
t.Fatal(err)
}
var want []struct {
Name, List string
Missing []string
Err string
Memberships json.RawMessage
}
if err := json.Unmarshal(raw, &want); err != nil {
t.Fatal(err)
}
fixtures := UserListFixtures()
if len(want) != len(fixtures) {
t.Fatalf("%d fixtures, %d lists the controller composed", len(fixtures), len(want))
}
for i, f := range fixtures {
t.Run(f.Name, func(t *testing.T) {
if want[i].Name != f.Name {
t.Fatalf("fixture %q against the controller's %q", f.Name, want[i].Name)
}
list, missing, err := ComposedFor(f)
got := ""
if err != nil {
got = err.Error()
}
if got != want[i].Err {
t.Fatalf("error %q, the controller's %q", got, want[i].Err)
}
if strings.Join(missing, ",") != strings.Join(want[i].Missing, ",") {
t.Errorf("left out %v, the controller %v", missing, want[i].Missing)
}
if list != want[i].List {
t.Errorf("the list differs from the controller's:\n%s", list)
}
// And every membership the records issue, as the controller's MembershipFor makes it (ADR 0547 §5): the
// grants part publishes them.
raw, err := json.Marshal(MembershipsFor(f))
if err != nil {
t.Fatal(err)
}
var gotM, wasM any
if err := json.Unmarshal(raw, &gotM); err != nil {
t.Fatal(err)
}
if err := json.Unmarshal(want[i].Memberships, &wasM); err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(gotM, wasM) {
t.Errorf("the memberships differ from the controller's:\n%s", raw)
}
})
}
t.Run("the controller's golden server configuration", func(t *testing.T) {
seat := Seat{Name: "telegram-sender", Accepts: []string{"send"}, Emits: []string{"delivered", "failed"}}
got, err := Compose(
Server{ClientPort: 4222, MonitoringPort: 8222, StoreDir: "/data",
TLSCert: "/tls/tls.crt", TLSKey: "/tls/tls.key", TLSCA: "/tls/ca.crt"},
[]Principal{
{Kind: KindController, PasswordHash: "$2a$11$cccccccccccccccccccccc"},
{Kind: KindEnrolment, Node: "one", PasswordHash: "$2a$11$eeeeeeeeeeeeeeeeeeeeee"},
{Kind: KindNode, Node: "one", PasswordHash: "$2a$11$nnnnnnnnnnnnnnnnnnnnnn"},
{Kind: KindModule, Node: "one", Module: "telegram", Holds: []Seat{seat},
Serves: []string{"status"}, PasswordHash: "$2a$11$tttttttttttttttttttttt"},
{Kind: KindModule, Node: "two", Module: "shop", Uses: []Seat{seat},
Emits: []string{"order.placed"}, PasswordHash: "$2a$11$ssssssssssssssssssssss"},
{Kind: KindModule, Node: "two", Module: "audit",
Consumes: []string{"shop.order.placed"}, PasswordHash: "$2a$11$aaaaaaaaaaaaaaaaaaaaaa"},
{Kind: KindModule, Node: "one", Module: WatchmanModule, PasswordHash: "$2a$11$wwwwwwwwwwwwwwwwwwwwww"},
{Kind: KindModule, Node: "one", Module: "nats", SnapshotsTheBus: true,
Serves: []string{"nats_streams"}, PasswordHash: "$2a$11$bbbbbbbbbbbbbbbbbbbbbb"},
})
if err != nil {
t.Fatal(err)
}
want, err := os.ReadFile(filepath.Join("testdata", "composed.conf"))
if err != nil {
t.Fatal(err)
}
if got != string(want) {
t.Errorf("the configuration differs from the controller's golden:\n%s", got)
}
})
}
+117
View File
@@ -0,0 +1,117 @@
package buscompose
import (
"regexp"
"strings"
)
// A tool call names its caller (novox/hq issue 365, by ADR 0259 §3's precedent for asks).
//
// mesh.mod.<module>.call.<tool>[.<node>].<caller>
// mesh.seat.<seat>.call.<verb>[.<node>].<caller>
//
// The last token is the bus user that published the call, and each user is granted these subjects with its own
// name there and no other — the caller is a fact the server enforces, and the tool runtime hands it to the
// module from the subject a call arrived on (mesh-tools node-tools internal/bus caller.go holds the same shape).
//
// **A kind of its own, `call`, not the `tool` subject with a token appended.** Every grant to call a tool is a
// wildcard over the `tool` kind — `.tool.<t>.*` for the instance on any machine, `mesh.mod.*.tool.>` for every
// tool — and either would match a `tool` subject with any caller appended, so a caller could name another.
// Under `call` nothing is granted but the caller-named subjects. No stream's filter covers it: a tool call is
// never persisted (design 25 §3).
//
// **Derived from the `tool` grants, in one place** (callerNamed): wherever a principal may call or answer a
// tool, it may call it in its own name, or answer it naming any caller — so no kind of principal is left
// unable to call in its own name, and a new grant to call is one in both shapes by construction.
//
// The `tool` grants stay beside these for one release, so every caller and every runtime moves without a gap:
// their retirement is hq issue 464.
const CallKind = "call"
var userName = regexp.MustCompile(`^[A-Za-z0-9_-]+(\.[A-Za-z0-9_-]+)*$`)
// CallerToken is a bus user's name as the last token of a call it publishes: each dot written `~`, which no part
// of a user's name may hold (safeSubject), so the token names that user and no other. "" for a name that is not
// a user's.
func CallerToken(user string) string {
if !userName.MatchString(user) {
return ""
}
return strings.ReplaceAll(user, ".", "~")
}
// toolGrant splits a grant on the `tool` kind — `mesh.mod.<m>.tool.<rest>` or `mesh.seat.<s>.tool.<rest>`, any
// part possibly a wildcard — at the kind; ok is false for any other subject.
func toolGrant(subject string) (head, rest string, ok bool) {
parts := strings.SplitN(subject, ".", 5)
if len(parts) != 5 || parts[0] != "mesh" || (parts[1] != "mod" && parts[1] != "seat") || parts[3] != "tool" ||
parts[2] == "" || parts[4] == "" {
return "", "", false
}
return parts[0] + "." + parts[1] + "." + parts[2], parts[4], true
}
// CalledSubject is where a call to a tool subject goes naming its caller; "" when the subject is not a tool's
// or the user is not a bus user.
func CalledSubject(subject, user string) string {
head, rest, ok := toolGrant(subject)
token := CallerToken(user)
if !ok || token == "" || strings.ContainsAny(rest, "*>") {
return ""
}
return head + "." + CallKind + "." + rest + "." + token
}
// CalledPattern is what a holder answering a tool subject also subscribes, to hear the calls that name their
// caller: the same address under the `call` kind, any caller last. "" for a subject that is not a tool's.
func CalledPattern(subject string) string {
head, rest, ok := toolGrant(subject)
if !ok || strings.ContainsAny(rest, "*>") {
return ""
}
return head + "." + CallKind + "." + rest + ".*"
}
// calledPublish is a grant to call on the `tool` kind, as the same grant in the caller's own name: its last
// token the caller's, and nothing that reaches past it. A `>` is every tail a call carries — the tool alone or
// the tool and the machine — so it becomes both, each ending in the caller.
func calledPublish(grant, token string) []string {
head, rest, ok := toolGrant(grant)
if !ok || token == "" {
return nil
}
base := head + "." + CallKind + "."
switch {
case rest == ">":
return []string{base + "*." + token, base + "*.*." + token}
case strings.HasSuffix(rest, ".>"):
return []string{base + strings.TrimSuffix(rest, ">") + "*." + token}
}
return []string{base + rest + "." + token}
}
// calledSubscribe is a grant to answer on the `tool` kind, as the same grant for the calls naming any caller.
func calledSubscribe(grant string) []string {
head, rest, ok := toolGrant(grant)
if !ok {
return nil
}
base := head + "." + CallKind + "."
if strings.HasSuffix(rest, ">") {
return []string{base + rest}
}
return []string{base + rest + ".*"}
}
// callerNamed adds, beside a principal's grants on the `tool` kind, the same grants under `call`: to publish in
// its own name, to subscribe naming anybody.
func callerNamed(user string, pub, sub []string) ([]string, []string) {
token := CallerToken(user)
for _, g := range pub {
pub = append(pub, calledPublish(g, token)...)
}
for _, g := range sub {
sub = append(sub, calledSubscribe(g)...)
}
return unique(pub), unique(sub)
}
+11
View File
@@ -0,0 +1,11 @@
package buscompose
import "strings"
// A seat's cancelled set (novox/hq ADR 0219) is a bucket the mesh no longer raises or grants (R4g, ADR 0525 clause 6): the
// controller asks no work queue, so nothing is cancelled. A bus that still holds one is not reported as state nothing declares.
// IsCancelledSet says a bucket is a seat's cancelled set rather than a module's state.
func IsCancelledSet(bucket string) bool {
return strings.HasPrefix(bucket, "SEAT_") && strings.HasSuffix(bucket, "_cancelled")
}
+196
View File
@@ -0,0 +1,196 @@
package buscompose
import (
"context"
"fmt"
"git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/nats-io/nats.go/jetstream"
)
// The compositions' buckets (novox/hq ADR 0539 clauses 2 and 4): one per node, `composed-<node>`, holding that node's
// compositions, which only that node's node-engine reads, since a composition can carry a sealed value (ADR 0488 §2's
// reason); and `mesh-controller_composed`, holding each composition's digest and the judgement of its move, which the
// assigner reads. The controller writes both, by compare-and-set while it holds the lease; each keeps five revisions
// of a key. The names and layouts are mesh-sdk's (delivery.ComposedBucketFor, ControllerComposedBucket, ComposedKey).
// ControllerComposed is the local name of the compositions' digests, as a reader names it in `reads` after
// `mesh-controller.`: delivery.ControllerComposedBucket.
const ControllerComposed = "composed"
// ComposedValueBytes is the most a composition may hold: a file over 64 KiB travels by digest, so a module's
// composition is far smaller; just under the server's largest message, which also carries the subject and headers.
const ComposedValueBytes = 1<<20 - 8<<10
// ComposedBucketBytes is a node's composition bucket's size: five revisions of every module's compositions of the node.
const ComposedBucketBytes = 256 << 20
// ComposedDigestsBytes is the size of the bucket of digests: a value of a few hundred bytes per node, module and
// fingerprint kept.
const ComposedDigestsBytes = 64 << 20
// ComposedConfig is a node's composition bucket as the controller asserts it.
func ComposedConfig(node string) (jetstream.KeyValueConfig, error) {
bucket, err := delivery.ComposedBucketFor(node)
if err != nil {
return jetstream.KeyValueConfig{}, err
}
return jetstream.KeyValueConfig{
Bucket: bucket,
Description: "the compositions of " + node + " (novox/hq ADR 0539): each module's resources for this node at one " +
"fingerprint, and the node's own, signed; written by the controller alone, read by this node's node-engine " +
"alone; deleted when outside the module's five newest fingerprints and named by no want",
History: delivery.ComposedHistory,
MaxValueSize: ComposedValueBytes,
MaxBytes: ComposedBucketBytes,
Storage: jetstream.FileStorage,
}, nil
}
// ComposedDigestsConfig is the bucket of the compositions' digests as the controller asserts it.
func ComposedDigestsConfig() jetstream.KeyValueConfig {
return jetstream.KeyValueConfig{
Bucket: delivery.ControllerComposedBucket,
Description: "each composition's digest and the judgement of its move from what the node runs (novox/hq ADR 0539 " +
"clauses 2 and 14): written by the controller alone, read by the assigner",
History: delivery.ControllerComposedHistory,
MaxValueSize: 4 << 10,
MaxBytes: ComposedDigestsBytes,
Storage: jetstream.FileStorage,
}
}
// EnsureComposed creates the bucket of digests and each named node's composition bucket if absent and brings their
// options to match. An update, never a delete.
func EnsureComposed(ctx context.Context, js jetstream.JetStream, nodes ...string) error {
if _, err := js.CreateOrUpdateKeyValue(ctx, ComposedDigestsConfig()); err != nil {
return fmt.Errorf("asserting bucket %s: %w", delivery.ControllerComposedBucket, err)
}
for _, node := range nodes {
cfg, err := ComposedConfig(node)
if err != nil {
return err
}
if _, err := js.CreateOrUpdateKeyValue(ctx, cfg); err != nil {
return fmt.Errorf("asserting bucket %s: %w", cfg.Bucket, err)
}
}
return nil
}
// ComposedWrites is what the controller publishes to write a node's compositions: the bucket's own subjects.
func ComposedWrites(node string) (string, error) {
bucket, err := delivery.ComposedBucketFor(node)
if err != nil {
return "", err
}
return "$KV." + bucket + ".>", nil
}
// ComposedReads is what a node's node-engine publishes to follow its wants and compositions (ADR 0539 clauses 1, 2 and
// 9): every key of its own composition bucket, and its own wants (`want.<node>.<module>` and `want.<node>.node`) in the
// assigner's bucket; nothing of another node's, nor the bucket of digests.
func ComposedReads(node string) ([]string, error) {
bucket, err := delivery.ComposedBucketFor(node)
if err != nil {
return nil, err
}
if !safeSubject.MatchString(node) {
return nil, fmt.Errorf("%q cannot be part of a subject: a permission is a subject pattern, and this would widen it", node)
}
return append(readGrants(bucket, ""), readGrants(delivery.AssignerWantsBucket, "want."+node+".>")...), nil
}
// VerbsTheNodeAgentAsks are the seat verbs a node's node-engine calls of its own machine while it applies a want (ADR
// 0539 clause 16, ADR 0429 §14): its backup holder backs a data module up before it moves, and says when it was. Asked
// of its own machine only, `mesh.seat.<seat>.tool.<verb>.<node>`.
var VerbsTheNodeAgentAsks = []SeatVerb{{Seat: "node-backup", Verb: "now"}, {Seat: "node-backup", Verb: "backed-up"}}
// NodeAgentAsks are the subjects of VerbsTheNodeAgentAsks for one node.
func NodeAgentAsks(node string) ([]string, error) {
if !safeSubject.MatchString(node) {
return nil, fmt.Errorf("%q cannot be part of a subject: a permission is a subject pattern, and this would widen it", node)
}
var out []string
for _, v := range VerbsTheNodeAgentAsks {
out = append(out, "mesh.seat."+v.Seat+".tool."+v.Verb+"."+node)
}
return out, nil
}
// NodeAgentHears are the events a node's node-engine subscribes while it applies a want: the backup holder's word that a
// backup is taken (`backed-up`), from whichever module holds node-backup on its machine (restic today).
var NodeAgentHears = []string{"mesh.mod." + BackupHolderModule + ".event.backed-up"}
// BackupHolderModule is the module that holds node-backup on every node, restic: the one whose word a node-engine hears.
// A second holder of the seat is a change of this grant.
const BackupHolderModule = "restic"
// administers says a person may invoke every tool (`*`): the operator.
func administers(invokes []string) bool {
for _, i := range invokes {
if i == "*" {
return true
}
}
return false
}
// HandPathWrites is what the operator's account publishes for release 6's hand path (ADR 0539 clauses 2 and 10): a want
// of any node, written and read back, in the assigner's bucket; each named node's compositions; the digests; and each
// node's object bucket for a file over 64 KiB, whose put reads an object's metadata and purges a replaced one's chunks.
func HandPathWrites(nodes []string) ([]string, error) {
out := append([]string{"$KV." + delivery.AssignerWantsBucket + ".want.>"}, readGrants(delivery.AssignerWantsBucket, "want.>")...)
out = append(out, "$KV."+delivery.ControllerComposedBucket+".>")
out = append(out, readGrants(delivery.ControllerComposedBucket, "")...)
for _, node := range nodes {
w, err := ComposedWrites(node)
if err != nil {
return nil, err
}
bucket, _ := delivery.ComposedBucketFor(node)
out = append(out, w)
out = append(out, readGrants(bucket, "")...)
files, err := DeclarationFilesWrites(node)
if err != nil {
return nil, err
}
stream := "OBJ_" + DeclarationFilesBucket + node
out = append(out, files, "$JS.API.STREAM.INFO."+stream, "$JS.API.DIRECT.GET."+stream+".>",
"$JS.API.STREAM.MSG.GET."+stream, "$JS.API.STREAM.PURGE."+stream)
}
return out, nil
}
// DeclarationsStream is the stream of each node's last declaration, `mesh.node.<node>.declare` (issue 107). Release 6
// sends none (ADR 0539 clauses 5 and 11): the stream is kept, with each node's consumer, only so a node away during the
// switch finds its last declaration and with it the new node-engine; the controller deletes it once every node's have
// names the new node-engine, and from then on neither asserts it nor any node's consumer on it; nor makes it again once
// it is gone from the bus for any reason, since a stream made anew would hold no node's last declaration (issue 1945).
const DeclarationsStream = "NODES"
// RaiseAfterDeclarations is Raise once the declarations' stream is retired: the mesh's streams and consumers but the
// declarations' stream, and no node's consumer of a declaration.
func RaiseAfterDeclarations(r Raiser) error {
for _, s := range MeshStreams() {
if s.Name == DeclarationsStream {
continue
}
if err := r.EnsureStream(s); err != nil {
return fmt.Errorf("asserting stream %s: %w", s.Name, err)
}
}
return AssertMeshConsumers(r)
}
// TakeInConsumer is the durable consumer the controller takes done keys in from, on the stream of each of TakeInBuckets
// (novox/hq ADR 0544 §1).
const TakeInConsumer = "mesh-controller-take-in"
// TakeInBuckets are the buckets the take-in consumes: the indexer's modules (each module's trunk value) and the builder's
// builds (each build's key).
var TakeInBuckets = []string{delivery.ModulesBucket, delivery.BuilderBuildsBucket}
// ConfigFollowConsumer is the durable consumer the controller follows mesh-config's resolved values with, on that
// bucket's stream (novox/hq ADR 0545).
const ConfigFollowConsumer = "mesh-controller-config-follow"
+247
View File
@@ -0,0 +1,247 @@
package buscompose
import (
"context"
"fmt"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
"github.com/nats-io/nats.go/jetstream"
)
// The controller's own key-value buckets (novox/hq to-be 45 §1, §6, §7).
//
// **What the controller must remember across its own restart, it keeps on the bus.** A call's
// outcome lived in the memory of the process that served it (novox/hq issue 265), so a controller
// replaced while a push ran answered "no such call" for the one thing its caller had been told to
// ask about. The bus already outlives the controller and is the shape ADR 0201 gives a module's
// current state: one value per key, written by one owner, read by anybody granted it. These are the
// controller's, written by it alone — the writers table of to-be 45 §1 — and asserted on every start
// like the streams, so a bus raised from nothing has them before the first call is served.
// CallsBucket keeps every call of the mesh's own verbs and what came of it; HandActsBucket every act
// a person did by hand, with why; ConditionsBucket every condition open now (to-be 45 §2), one key
// each, and ConditionHistoryBucket every transition of one — raised, changed, silenced, cleared —
// for ninety days.
//
// **The history is a bucket of its own** because its keys expire and an open condition's must not:
// a bucket has one age for every key, and a condition open longer than the history is kept would
// otherwise vanish from the store while still true.
var (
CallsBucket = BucketName(ControllerSeat, "calls")
HandActsBucket = BucketName(ControllerSeat, "hand-acts")
ConditionsBucket = BucketName(ControllerSeat, "conditions")
ConditionHistoryBucket = BucketName(ControllerSeat, "condition-history")
// LeaseBucket holds the controller's lease (to-be 45 §6): one key, `holder`, which the instance
// allowed to act writes by compare-and-set and renews; its revision when taken is the epoch.
LeaseBucket = BucketName(ControllerSeat, "lease")
// AskedBucket keeps what the controller asked the operator about its conditions (novox/hq ADR 0259):
// each ask by its id, its options and the actions they stand for, how it ended and whether the
// controller acted on its warrant — so a restart neither asks twice nor acts twice.
AskedBucket = BucketName(ControllerSeat, "asked")
// DeliveryMergesBucket holds the merges mesh-delivery announced it made, each with the ask whose Approve it merged
// on (novox/hq ADR 0420 §7, issue 1025): per merge a key `<owner>/<repo>/<number>/<head sha>` holding the
// announcement, kept DeliveryMergesKeptFor. What lets the build of a moved module register it from its new
// repository: the controller reads the ask's warrant from the router and accepts only the operator's Approve of
// those heads. Written by the controller alone, from the announcement, when it holds the lease.
DeliveryMergesBucket = BucketName(ControllerSeat, "delivery-merges")
)
// DeliveryMergesKeptFor is how long a merge mesh-delivery announced is kept: a week, as the events are.
const DeliveryMergesKeptFor = 7 * 24 * time.Hour
// The controller's record mirrors (novox/hq ADR 0289, design 54, design 57 package F3): a bucket per kind of
// record, one key per record, written from the store by the lease holder only when the record changed, with
// one event on the controller's seat per write. Read by whatever names them in `reads` as
// `mesh-controller.<mirror>` (ADR 0201 §1), written by nobody else. **No secret in either** (ADR 0201): an
// assignment's value names its module and node; a module's declared config is what its manifest, which is
// public, declares.
//
// AssignmentsBucket holds which module is assigned to which node, keyed `<node>.<module>`; and
// DeclaredConfigBucket each registered module's config declarations as its manifest holds them, keyed by
// the module: what mesh-config resolves a config value from (ADR 0348 clauses 2 and 4); and PredefinedBucket the
// list of predefined keys the mesh published (the newest trunk build's of the catalogue, ADR 0348 §7), one key
// `list`: what mesh-config publishes as mesh-config_predefined (design 57 K3, issue 662).
var (
AssignmentsBucket = BucketName(ControllerSeat, MirrorAssignments)
DeclaredConfigBucket = BucketName(ControllerSeat, MirrorDeclaredConfig)
PredefinedBucket = BucketName(ControllerSeat, MirrorPredefined)
// DeclaredPackagesBucket holds, keyed by the node, each package the modules assigned to it declare, present or absent,
// with the declaring modules: what the pacman module's verbs read to say a package is declared (ADR 0507 §5).
DeclaredPackagesBucket = BucketName(ControllerSeat, MirrorDeclaredPackages)
)
// The mirrors' local names, as a reader names them in `reads` after `mesh-controller.`.
const (
MirrorAssignments = "assignments"
MirrorDeclaredConfig = "declared-config"
MirrorPredefined = "predefined"
// MirrorDeclaredPackages is the local name of the packages each node's modules declare (ADR 0507 §5, issue 1840/S3).
MirrorDeclaredPackages = "declared-packages"
)
// The chain's buckets (novox/hq ADR 0408, issue 1089): builds, rollouts and assignments are one chain kept on the bus
// alone, one bucket each, one key per record, written by the lease holder by compare-and-set. The assignments bucket is
// AssignmentsBucket above, which keeps its name and its readers (ADR 0408 §3). Nothing is removed from them (§4), they keep
// ChainHistory earlier revisions of every key (§5), and a value is never cut (§6).
var (
BuildsBucket = BucketName(ControllerSeat, ChainBuilds)
RolloutsBucket = BucketName(ControllerSeat, ChainRollouts)
)
// **The assignments bucket keeps the marker age (MirrorMarkerTTL) it was made with as a mirror.** A stream that allows
// messages an age cannot be made to disallow them (the server refuses the update: "message TTL status can not be
// disabled", nats-server stream.go), so asserting the bucket without the age would fail the controller's start on a
// mesh that has the bucket. The chain never purges or removes a key (ADR 0408 §4), so no marker is ever written and the
// age ends nothing; it is kept only so the stream's configuration stays one the server accepts.
//
// The chain buckets' local names, as a reader names them in `reads` after `mesh-controller.`.
const (
ChainBuilds = "builds"
ChainRollouts = "rollouts"
)
// The chain buckets' bounds. ChainHistory is the bus's largest history (ADR 0408 §5). A value is at most ChainValueBytes,
// just under the server's largest message of a mebibyte, which also carries the subject and the headers (ADR 0408 §6):
// the controller writes none larger and the move refuses to. The buckets' size is the arithmetic of the survey of
// issue 981: 8585 builds of 1 to 40 KB, 1049 rollouts of about 2 KB a revision and 64 revisions, so 512 MiB for the two
// that grow, and the assignments, 285 of under a kilobyte, 64 MiB.
const (
ChainHistory = 64
ChainValueBytes = 1<<20 - 8<<10
ChainBuildsBytes = 512 << 20
ChainRolloutsBytes = 512 << 20
ChainAssignmentBytes = 64 << 20
)
// GrantsBucket holds, keyed `<module>.<fingerprint hex>`, each fingerprint of a module whose bus credential the controller
// has issued (novox/hq ADR 0528 clause 2, ADR 0429 §6 rule 6, issue 1394): written by the controller alone once the bus's
// user list composed with the module's manifest at that fingerprint was sent to the bus's machine; read by the assigner,
// whose sixth rule moves a node's want to a fingerprint only once its grant exists. The name and layout are mesh-sdk's
// (delivery.ControllerGrantsBucket, ControllerGrantKey).
var GrantsBucket = BucketName(ControllerSeat, ControllerGrants)
// ControllerGrants is the grants bucket's local name, as a reader names it in `reads` after `mesh-controller.`.
const ControllerGrants = "grants"
// ChainBuckets are the three buckets of the chain: builds, rollouts and assignments.
func ChainBuckets() []string { return []string{BuildsBucket, RolloutsBucket, AssignmentsBucket} }
// MirrorsWrittenByAnother are the controller's mirrors whose writer is not the controller, each with its reason: the rule
// that a mirror is written by the controller alone (TestTheMirrorsListAndTheWritersTableAgree) has these exceptions and no
// other.
var MirrorsWrittenByAnother = map[string]string{}
// The mirrors' caps (design 54 §5): one value per key, no age, 64 KiB a value, 64 MiB a bucket.
const (
MirrorValueBytes = 64 << 10
MirrorBytes = 64 << 20
// MirrorMarkerTTL is the longest a purge's marker is kept (design 54 §2): the bucket allows a
// per-message age for it, and the writer gives each marker a day.
MirrorMarkerTTL = 24 * time.Hour
)
// PlanStateBuckets are the buckets of the plans' state that release 6 deletes (novox/hq ADR 0539 clause 13): the merge
// window's clock, the silent sends' waits and the building plans' hold. The controller deletes each at its first start
// and says it once (composed_start.go); none is asserted, granted or written any more.
var PlanStateBuckets = []string{BucketName(ControllerSeat, "merge-window"), BucketName(ControllerSeat, "rollout-silent"),
BucketName(ControllerSeat, "plans-building")}
// AskedKeptFor is how long an ask is kept after it was made: a month, as the router keeps its own.
const AskedKeptFor = 30 * 24 * time.Hour
// LeaseTTL is how long the lease's key lives unrenewed (to-be 45 §6): fifteen seconds, renewed
// every five. The bucket's age, so the bus forgets a holder that stopped renewing.
const LeaseTTL = 15 * time.Second
// The bounds to-be 45 §6 sets for calls: the last thousand, or fourteen days, whichever is fewer.
// A call is two keys — its record, and its answer apart so a listing does not read every answer —
// so the stream holds twice as many messages as it keeps calls.
const (
KeptCallsDurably = 1000
CallsKeptFor = 14 * 24 * time.Hour
// CallAnswerBytes is the most of one answer kept: a whole declaration is far smaller, and an
// answer larger is cut and says so.
CallAnswerBytes = 64 << 10
// HandActsKeptFor is as long as a condition's history (to-be 45 §2): an act by hand is read
// back beside what it addressed.
HandActsKeptFor = 90 * 24 * time.Hour
// ConditionHistoryKeptFor is how long a condition's transitions are kept (to-be 45 §2).
ConditionHistoryKeptFor = 90 * 24 * time.Hour
)
// IsControllerBucket says a bucket is the controller's own, not a module's state nothing declares.
func IsControllerBucket(bucket string) bool {
return bucket == CallsBucket || bucket == HandActsBucket || bucket == ConditionsBucket ||
bucket == ConditionHistoryBucket || bucket == LeaseBucket || bucket == AskedBucket ||
bucket == AssignmentsBucket || bucket == DeclaredConfigBucket || bucket == PredefinedBucket ||
bucket == DeclaredPackagesBucket ||
bucket == DeliveryMergesBucket || bucket == BuildsBucket || bucket == RolloutsBucket ||
bucket == delivery.HaveBucket || bucket == StatesAnnouncedBucket ||
bucket == GrantsBucket || bucket == delivery.ControllerComposedBucket
}
// ControllerBuckets are the controller's own buckets, in the order they are asserted.
func ControllerBuckets() []string {
return []string{LeaseBucket, CallsBucket, HandActsBucket, ConditionsBucket, ConditionHistoryBucket, AskedBucket,
AssignmentsBucket, DeclaredConfigBucket, PredefinedBucket, DeclaredPackagesBucket,
DeliveryMergesBucket, BuildsBucket, RolloutsBucket, delivery.HaveBucket, StatesAnnouncedBucket, GrantsBucket,
delivery.ControllerComposedBucket}
}
// ControllerBucketsAsserter is what raising the controller's buckets needs of a connection.
type ControllerBucketsAsserter interface {
EnsureControllerBuckets() error
}
// chainBucket is the options of a bucket of the chain: sixty-four revisions of each key, no age, a value up to
// ChainValueBytes, on disk.
func chainBucket(bucket, why string, bytes int64, markers time.Duration) jetstream.KeyValueConfig {
return jetstream.KeyValueConfig{
Bucket: bucket,
Description: why,
History: ChainHistory,
MaxValueSize: ChainValueBytes,
MaxBytes: bytes,
Storage: jetstream.FileStorage,
LimitMarkerTTL: markers,
}
}
// EnsureLeaseBucket creates the lease's bucket if absent and brings its options to match (to-be 45
// §6). **Before the lease is taken, by any candidate**: it is the lease's own precondition, and asserting
// a bucket that exists changes nothing. File storage, so its revisions — the epochs — outlive a restart of
// the bus; one raised again from nothing is moved past the highest epoch issued when the lease is taken.
func EnsureLeaseBucket(ctx context.Context, js jetstream.JetStream) error {
if _, err := js.CreateOrUpdateKeyValue(ctx, jetstream.KeyValueConfig{
Bucket: LeaseBucket,
Description: "the controller's lease (novox/hq to-be 45 §6): the key `holder`, written by compare-and-set " +
"by the one controller instance that may act and renewed every five seconds; its revision when taken " +
"is the epoch every declaration and plan write carries",
History: 1,
TTL: LeaseTTL,
MaxValueSize: 4 << 10,
MaxBytes: 1 << 20,
Storage: jetstream.FileStorage,
}); err != nil {
return fmt.Errorf("asserting bucket %s: %w", LeaseBucket, err)
}
return nil
}
// HaveBucketConfig is the options of the have bucket (novox/hq ADR 0429 §1): delivery.HaveHistory revisions of each
// key, no age, 64 MiB on disk like the assignments. A module's have is JSON of about 500 bytes, but a node's own key
// carries its whole facts (its profile among them), several kilobytes on a desktop: a value is as large as the chain's
// buckets take, since a value is never cut (novox/hq ADR 0408 §6, issue 1162).
func HaveBucketConfig() jetstream.KeyValueConfig {
return jetstream.KeyValueConfig{
Bucket: delivery.HaveBucket,
Description: "what each machine has of each module, one key `<node>.<module>` (and `<node>.node` for the machine's " +
"own facts) (novox/hq ADR 0429, design 47): written by each machine's node-engine under its own node's keys " +
"alone, never by the controller; read by the delivery view and the watchman",
History: delivery.HaveHistory,
MaxValueSize: ChainValueBytes,
MaxBytes: 64 << 20,
Storage: jetstream.FileStorage,
}
}
+88
View File
@@ -0,0 +1,88 @@
package buscompose
import (
"time"
"github.com/nats-io/nats.go/jetstream"
)
// The streams a manifest declares, as the server is asked to create them (novox/hq ADR 0547 §5, issue 1790): the
// grants part creates each one the bus does not hold yet, with what the controller asserts at its start, so the two
// make the same stream. Pure. Only a create is ever asked with these: a stream the bus holds is never updated by the
// grants part (CreateStreamGrants grants no update).
// BucketStreamConfig is a module's bucket as the stream the controller's CreateOrUpdateKeyValue makes of its options
// (mesh-controller internal/broker BucketConfig, with no marker age held): the client library's key-value shape
// (rollup, no delete, direct reads, discard new, a duplicate window of two minutes or the age when shorter) with the
// mesh's caps, file storage and one replica.
func BucketStreamConfig(b Bucket) jetstream.StreamConfig {
history := b.History
if history < 1 {
history = 1
}
age := time.Duration(b.TTLSeconds) * time.Second
dupes := 2 * time.Minute
if age > 0 && age < dupes {
dupes = age
}
cfg := jetstream.StreamConfig{
Name: "KV_" + b.Bucket(),
Description: b.Why(),
Subjects: []string{"$KV." + b.Bucket() + ".>"},
MaxMsgsPerSubject: int64(history),
MaxBytes: StateMaxBytes,
MaxAge: age,
MaxMsgSize: StateMaxValueBytes,
Storage: jetstream.FileStorage,
Replicas: 1,
AllowRollup: true,
DenyDelete: true,
Duplicates: dupes,
MaxMsgs: -1,
MaxConsumers: -1,
AllowDirect: true,
Discard: jetstream.DiscardNew,
}
if b.KeyTTL {
cfg.AllowMsgTTL, cfg.SubjectDeleteMarkerTTL = true, KeyTTLMarkerTTL
}
return cfg
}
// ModuleStreamConfig is a module's own stream as the controller asserts it (mesh-controller internal/broker
// StreamConfigFor of ModuleStream.Stream).
func ModuleStreamConfig(m ModuleStream) jetstream.StreamConfig {
s := m.Stream()
cfg := jetstream.StreamConfig{
Name: s.Name,
Subjects: s.Subjects,
Retention: jetstream.LimitsPolicy,
MaxAge: time.Duration(s.MaxAge) * time.Second,
MaxMsgsPerSubject: int64(s.MaxMsgsPerSubject),
Description: s.Why,
AllowDirect: s.Direct,
AllowMsgTTL: s.AllowMsgTTL,
}
if s.Retention == RetentionWorkQueue {
cfg.Retention = jetstream.WorkQueuePolicy
}
if s.Restricted || s.WorkQueue {
cfg.Metadata = map[string]string{}
if s.Restricted {
cfg.Metadata[RestrictedMetadata] = "true"
}
if s.WorkQueue {
cfg.Metadata[WorkQueueMetadata] = "true"
}
}
if s.MaxBytes > 0 {
cfg.MaxBytes = s.MaxBytes
}
if s.DiscardNew {
cfg.Discard = jetstream.DiscardNew
}
if s.DuplicatesSeconds > 0 {
cfg.Duplicates = time.Duration(s.DuplicatesSeconds) * time.Second
}
return cfg
}
+46
View File
@@ -0,0 +1,46 @@
package buscompose
import (
"testing"
"time"
"github.com/nats-io/nats.go/jetstream"
)
// Defends issue 1790 (ADR 0547 §5): the grants part creates a declared bucket and stream as the controller asserts them.
func TestDeclaredStreamsAreCreatedAsTheControllerAssertsThem(t *testing.T) {
plain := BucketStreamConfig(Bucket{Module: "mesh-vault", Name: "bus-accounts"})
keyed := BucketStreamConfig(Bucket{Module: "m", Name: "presence", History: 5, TTLSeconds: 60, KeyTTL: true})
queue := ModuleStreamConfig(ModuleStream{Module: "b", Name: "ASKS", Events: []string{"ask"}, MaxAgeDays: 7, MaxMiB: 32,
WorkQueue: true, Duplicates: 1})
for _, row := range []struct {
what string
got, want any
}{
{"bucket name", plain.Name, "KV_mesh-vault_bus-accounts"},
{"bucket subjects", plain.Subjects[0], "$KV.mesh-vault_bus-accounts.>"},
{"bucket history defaults to one", plain.MaxMsgsPerSubject, int64(1)},
{"bucket cap", plain.MaxBytes, int64(StateMaxBytes)},
{"bucket value cap", plain.MaxMsgSize, int32(StateMaxValueBytes)},
{"bucket duplicate window", plain.Duplicates, 2 * time.Minute},
{"bucket discards new", plain.Discard, jetstream.DiscardNew},
{"bucket direct and rollup, no delete", plain.AllowDirect && plain.AllowRollup && plain.DenyDelete, true},
{"bucket without key ttl", plain.AllowMsgTTL, false},
{"history", keyed.MaxMsgsPerSubject, int64(5)},
{"age", keyed.MaxAge, time.Minute},
{"duplicate window under the age", keyed.Duplicates, time.Minute},
{"key ttl markers", keyed.SubjectDeleteMarkerTTL, KeyTTLMarkerTTL},
{"key ttl", keyed.AllowMsgTTL, true},
{"queue retention", queue.Retention, jetstream.WorkQueuePolicy},
{"queue subjects", queue.Subjects[0], "mesh.stream.b.event.ask"},
{"queue age", queue.MaxAge, 7 * 24 * time.Hour},
{"queue cap", queue.MaxBytes, int64(32 << 20)},
{"queue discards new", queue.Discard, jetstream.DiscardNew},
{"queue duplicates", queue.Duplicates, time.Hour},
{"queue metadata", queue.Metadata[WorkQueueMetadata] + queue.Metadata[RestrictedMetadata], "truetrue"},
} {
if row.got != row.want {
t.Errorf("%s: %v, want %v", row.what, row.got, row.want)
}
}
}
+77
View File
@@ -0,0 +1,77 @@
package buscompose
import "testing"
// Defends issue 1790 and novox/hq ADR 0547 §5 (the operator, 2026-10-26 08:54): the grants part may create the stream
// of each bucket, log and stream a placed module's manifest declares, by its own name, and read it back; nothing
// wider, nothing updated or deleted, and no other principal so. Held byte-equal in the controller's copy.
func TestTheGrantsPartCreatesOnlyDeclaredStreams(t *testing.T) {
records := Records{
Nodes: []string{"novox"},
Assigned: map[string][]Declared{"novox": {
{Module: "mesh-delivery-builder",
State: []Bucket{{Module: "mesh-delivery-builder", Name: "steps", History: 5}},
Logs: []Log{{Module: "mesh-delivery-builder", Name: "builds"}},
Streams: []ModuleStream{{Module: "mesh-delivery-builder", Name: "BUILDASKS", Events: []string{"ask"}}}},
{Module: GrantsModule, State: []Bucket{{Module: GrantsModule, Name: "assertions"}}},
}},
}
users, err := Users(records)
if err != nil {
t.Fatal(err)
}
perms := map[string]Permissions{}
for _, u := range users {
if u.Kind != KindModule {
continue
}
p, err := PermissionsFor(u)
if err != nil {
t.Fatalf("%s: %v", u.Username(), err)
}
perms[u.Module] = p
}
for _, row := range []struct {
who, subject string
may bool
}{
{GrantsModule, "$JS.API.STREAM.CREATE.KV_mesh-delivery-builder_steps", true},
{GrantsModule, "$JS.API.STREAM.INFO.KV_mesh-delivery-builder_steps", true},
{GrantsModule, "$JS.API.STREAM.CREATE.LOG_mesh-delivery-builder_builds", true},
{GrantsModule, "$JS.API.STREAM.CREATE.BUILDASKS", true},
{GrantsModule, "$JS.API.STREAM.CREATE.KV_mesh-grants_assertions", true},
{GrantsModule, "$JS.API.STREAM.UPDATE.KV_mesh-delivery-builder_steps", false},
{GrantsModule, "$JS.API.STREAM.DELETE.KV_mesh-delivery-builder_steps", false},
{GrantsModule, "$JS.API.STREAM.PURGE.KV_mesh-delivery-builder_steps", false},
{GrantsModule, "$JS.API.STREAM.CREATE.KV_mesh-delivery-builder_undeclared", false},
{GrantsModule, "$JS.API.STREAM.CREATE.EVENTS", false},
{GrantsModule, "$JS.API.STREAM.CREATE.ASSIGNMENTS", false},
{GrantsModule, "$JS.API.STREAM.INFO.EVENTS", false},
{"mesh-delivery-builder", "$JS.API.STREAM.CREATE.KV_mesh-delivery-builder_steps", false},
} {
if got := MayPublish(perms[row.who], row.subject); got != row.may {
t.Errorf("%s may publish %s: %v, want %v", row.who, row.subject, got, row.may)
}
}
// A name that is no plain token is never granted, and a module other than the grants part is never granted one.
if got := CreateStreamGrants([]string{"*", ">", "KV_a.b", "KV_a_b"}); len(got) != 2 {
t.Errorf("grants for wildcard names: %v", got)
}
other, err := PermissionsFor(Principal{Kind: KindModule, Node: "novox", Module: "shop", CreatesStreams: []string{"KV_shop_x"}})
if err != nil {
t.Fatal(err)
}
if MayPublish(other, "$JS.API.STREAM.CREATE.KV_shop_x") {
t.Error("a module other than the grants part was granted a create")
}
// The writers table refuses the grants part an update, and a create of a stream it was not given.
g := Principal{Kind: KindModule, Node: "novox", Module: GrantsModule, CreatesStreams: []string{"KV_a_b"}}
if err := CheckWriters(g, []string{"$JS.API.STREAM.CREATE.KV_a_b"}); err != nil {
t.Errorf("the declared create refused: %v", err)
}
for _, refused := range []string{"$JS.API.STREAM.UPDATE.KV_a_b", "$JS.API.STREAM.CREATE.KV_a_c", "$JS.API.STREAM.CREATE.*"} {
if CheckWriters(g, []string{refused}) == nil {
t.Errorf("%s was not refused", refused)
}
}
}
+111
View File
@@ -0,0 +1,111 @@
package buscompose
import (
"context"
"fmt"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/nats-io/nats.go/jetstream"
)
// The object buckets a declaration's large files travel in (novox/hq ADR 0486, issues 1696 and 1697): **one bucket for each
// node**, `declaration-files-<node>`.
//
// **The controller alone writes them, and a node-engine reads an object by its name from its own bucket** (ADR 0486 §2): its
// account may neither list the bucket nor write it, no account may touch another node's, and no other account has any access.
// Why a bucket each: a subject pattern cannot tell a name from a wildcard. A grant of `...M.*` matches a publish to the literal
// subject `...M.*` too (probed on nats-server 2.11.17 by the review of this issue: a node created a consumer filtered on
// `$O.<bucket>.C.*` and read every object of the bucket), so in one shared bucket a node could read every other node's files.
// With a bucket each, the same trick reads only the node's own files, which are the files of its own declaration.
//
// A bucket of the bus's object store is a stream, OBJ_<bucket>, whose messages are the objects' metadata
// (`$O.<bucket>.M.<base64 of the name>`) and chunks (`$O.<bucket>.C.<nuid>`).
// DeclarationFilesBucket is the prefix of the buckets' names, which mesh-sdk fixes for the node-engine and the controller alike.
const DeclarationFilesBucket = declaration.FileBucketPrefix
// DeclarationFilesBucketOf is the bucket a node's files are in: mesh-sdk's FileBucketFor, the one place the name is composed. A
// node name it refuses (one that could carry a subject wildcard or separator into a grant) is an error, never a bucket.
func DeclarationFilesBucketOf(node string) (string, error) { return declaration.FileBucketFor(node) }
// DeclarationFilesStreamOf is the stream under a node's bucket: mesh-sdk's FileStreamFor.
func DeclarationFilesStreamOf(node string) (string, error) { return declaration.FileStreamFor(node) }
// DeclarationFilesMaxBytes is a bucket's size: the objects are immutable and regenerable, and one node's hold well under a hundred
// megabytes.
const DeclarationFilesMaxBytes = 1 << 30
// DeclarationFilesKeptFor is how long an object stays after no node's latest declaration names it (ADR 0486 §3).
const DeclarationFilesKeptFor = 24 * time.Hour
// DeclarationFilesWrites is the one pattern the controller may publish to write a node's bucket: every object's metadata and
// chunks. The server's API for the stream is the controller's `$JS.API.>`.
func DeclarationFilesWrites(node string) (string, error) {
bucket, err := DeclarationFilesBucketOf(node)
if err != nil {
return "", err
}
return "$O." + bucket + ".>", nil
}
// DeclarationFilesReads is what a node-engine's account may publish to read an object of its own bucket by its name, and nothing
// more. The set is the one node-engine's own call (mesh-host internal/link FileObject) makes, for B the node's bucket and S its
// stream (issue 1697, notes 4 and 5):
//
// - the object's metadata, by a direct get of the last message of the one subject that names it. **Only the suffixed
// form**: the bare `$JS.API.DIRECT.GET.S` takes a sequence number and would read any message of the stream.
// - the ordered consumer on that object's chunks: `$JS.API.CONSUMER.CREATE.S.<name>.$O.B.C.<nuid>`. **Not `S.>`**:
// a consumer filtered on `$O.B.M.>` would read every object's name.
// - the flow-control replies of that consumer, without which a large object stalls at the flow-control window.
// - the consumer's deletion, best effort: the consumer's inactivity threshold removes it if this is refused.
//
// **Never granted:** STREAM.INFO (a `subjects_filter` of `>` lists every name), STREAM.MSG.GET (any message by its sequence), a
// publish to `$O.B.>` (a write), and any subject of another node's stream. A literal `*` in a subject matches the grant's `*`: it
// reaches the objects of this node's own bucket and no other (above).
func DeclarationFilesReads(node string) ([]string, error) {
bucket, err := DeclarationFilesBucketOf(node)
if err != nil {
return nil, err
}
stream, err := DeclarationFilesStreamOf(node)
if err != nil {
return nil, err
}
return []string{
"$JS.API.DIRECT.GET." + stream + ".$O." + bucket + ".M.*",
"$JS.API.CONSUMER.CREATE." + stream + ".*.$O." + bucket + ".C.*",
"$JS.API.CONSUMER.DELETE." + stream + ".*",
"$JS.FC." + stream + ".>",
}, nil
}
// EnsureDeclarationFiles is the same on a JetStream the caller holds.
func EnsureDeclarationFiles(ctx context.Context, js jetstream.JetStream, nodes ...string) error {
for _, node := range nodes {
cfg, err := DeclarationFilesConfig(node)
if err != nil {
return err
}
if _, err := js.CreateOrUpdateObjectStore(ctx, cfg); err != nil {
return fmt.Errorf("asserting the object bucket %s: %w", cfg.Bucket, err)
}
}
return nil
}
// DeclarationFilesConfig is a node's bucket as the controller asserts it.
func DeclarationFilesConfig(node string) (jetstream.ObjectStoreConfig, error) {
bucket, err := DeclarationFilesBucketOf(node)
if err != nil {
return jetstream.ObjectStoreConfig{}, err
}
return jetstream.ObjectStoreConfig{
Bucket: bucket,
Description: "the contents of the files over 64 KiB of the declaration of " + node + ", each under the hex of its sha256 " +
"digest (novox/hq ADR 0486): written by the controller alone, read by the node-engine of that node by name, removed " +
"a day after the node's latest declaration no longer names it",
MaxBytes: DeclarationFilesMaxBytes,
Storage: jetstream.FileStorage,
}, nil
}
+357
View File
@@ -0,0 +1,357 @@
package buscompose
import (
"fmt"
"sort"
"strings"
)
// Streams and consumers derived from what modules declare.
//
// The mesh's own four exist before any module does (streams.go). Everything here is the other
// half: a seat's stream comes into being when the module declaring it is **registered**, and a
// consumer when a module is **assigned** — which is why ADR 0116's task 1.4 had to be narrowed to
// the foundation set. Neither has happened at genesis.
//
// All of it is a pure function of declarations. The controller is still the only writer; this is
// only what it writes.
// A Consumer is a durable subscription the controller creates on a module's behalf. A module
// declares what it reacts to, never how delivery works, so it does not name these and cannot
// misconfigure them.
type Consumer struct {
Name string
Stream string
// Filters are the subjects this consumer receives. One consumer per module with several
// filters, rather than one per consumed event: its ack subject is derived from its name, and
// a module with five consumers would need five ack permissions to ack its own deliveries.
Filters []string
// Queue is the queue group, set for a seat's worker so that "exactly one holder" survives a
// seat later being relaxed to several. Authority and delivery are kept separate on purpose.
Queue string
// Push asks the server to deliver to a subject rather than wait to be pulled.
//
// For the mesh's own consumer, where the controller wants every message to arrive in the one
// loop it already runs: pulling would mean a second goroutine fetching batches and handing
// them over, and a loop that acts on one message at a time is the property the store window
// depends on. A queue group implies this, because a group has nothing to pull from.
Push bool
// TakesOverFrom names a consumer of the same stream whose filter this one takes over (issue 973): made while
// that consumer still filters this one's subject, it starts after what that consumer has acknowledged, so a
// message the old one held unacknowledged is not lost to the change. Otherwise it starts as FromNow says.
TakesOverFrom string
// DeliverSubject, when set, is where a push consumer delivers instead of DeliverSubjectFor's.
DeliverSubject string
// AckWaitSeconds before an unacknowledged delivery is redelivered.
AckWaitSeconds int
// MaxDeliver is how often a message is handed over before the consumer gives it up; zero for no
// bound. What a consumer gives up is kept in DEAD_LETTERS by the controller (novox/hq issue 330).
MaxDeliver int
// MaxAckPending is how many deliveries the server lets stand unacknowledged at once; zero for
// the server's default, which is many. **One, for a consumer handled one at a time**
// (novox/hq issue 175): a handler that builds for minutes keeps its own message alive with a
// heartbeat, but everything handed over behind it times out unacknowledged and comes back —
// and a merge that came back rebuilt what it had just built, five times over on 2026-09-30.
// With one outstanding, the server holds the rest, and the heartbeat is keeping the message.
MaxAckPending int
// FromNow makes a consumer that does not exist yet start at the stream's end rather than its
// beginning (novox/hq issue 248). For a consumer that reacts to announcements — a merge, a build's
// outcome — on a stream that keeps a week of them: the server's default, everything the stream
// holds, replays every merge and every build of that week as if it had just happened. A consumer
// that exists keeps where it is, whatever this says; only its making is decided here.
FromNow bool
Why string
// Resettable says why this consumer may be re-made to deliver from now by the mesh itself, with
// nobody asked (novox/hq to-be 45 §7, healer H4): what a reset drops, something else catches up.
// Empty for every consumer where nothing would — a module's, whose events would be lost to it.
// Not a property of the consumer on the bus: nothing here is sent to the server.
Resettable string
}
// seatStreamName is the stream holding a seat's inbound work. Named after the seat rather than
// the module holding it, because the holder can change and the queued work must not care — which
// is the whole reason a caller addresses a seat instead of a module.
func seatStreamName(seat string) string { return "SEAT_" + upperSnake(seat) }
// SeatStreams is one work queue per declared seat, created when the declaring module is
// registered rather than when it is assigned.
//
// **The stream exists before anyone holds the seat, and that is the point.** Work queues until a
// holder appears, so installing the telegram module a week after something started sending to it
// flushes the backlog instead of having lost it. A stream created at assignment would make "the
// holder is not here yet" mean "your messages are gone".
func SeatStreams(seats []DeclaredSeat) []Stream {
sorted := append([]DeclaredSeat(nil), seats...)
sort.Slice(sorted, func(i, j int) bool { return sorted[i].Name < sorted[j].Name })
var out []Stream
for _, s := range sorted {
if len(s.Accepts) == 0 {
// A seat that only emits and serves needs no stream: its events ride EVENTS and its
// tools are core request/reply, which is never persisted.
continue
}
retain := s.RetainSeconds
if retain == 0 {
retain = 7 * 24 * 60 * 60
}
out = append(out, Stream{
Name: seatStreamName(s.Name),
Subjects: []string{"mesh.seat." + s.Name + ".accept.>"},
Retention: RetentionWorkQueue,
MaxAge: retain,
Why: fmt.Sprintf("work submitted to the %s seat; one holder consumes it, and it "+
"queues while nobody does", s.Name),
})
}
return out
}
// A DeclaredSeat is a seat as the catalogue knows it. Mirrored here rather than imported so this
// package stays free of the catalogue's own types — the same reason the host mirrors the
// contracts instead of importing the sdk.
type DeclaredSeat struct {
Name string
Accepts []string
// Emits are the verbs the seat's holder publishes under the seat's own name. An event about a
// role belongs here rather than in the holder's namespace, because the name then outlives
// whoever fills it (novox/hq ADR 0121, 04-ISSUES/127).
Emits []string
// Serves are the verbs the holder answers, request and reply.
Serves []string
RetainSeconds int
// PulledByKind is Seat's: a worker per accepted kind, derived from the holders' seat traffic, not here.
PulledByKind bool
}
// ConsumerFor is the durable consumer a module's declarations imply, or false when it subscribes
// to nothing and needs none.
//
// One per module, with every consumed subject as a filter, because its ack permission is derived
// from its name: a module with a consumer per event would need an ack permission per consumer,
// and the permission list would stop being derivable from the declaration.
func ConsumerFor(p Principal) (Consumer, bool) {
// A module that reacts to anything — a module's events or a role's (novox/hq ADR 0121). Watching
// a role was missing here, so the one module that does it got no consumer at all: it started,
// connected, and its graph stayed empty with nothing anywhere reporting why.
// And one that hears its own answers on a seat it uses (novox/hq ADR 0259 §3): an asker's warrants.
hearsItsOwn := len(SeatTrafficOf(p.Module, nil, p.Uses, nil).Subscribe) > 0
hearsAStream := false
for _, h := range p.Hears {
if len(h.Filters) > 0 {
hearsAStream = true
}
}
if p.Kind != KindModule || (len(p.Consumes) == 0 && len(p.Watches) == 0 && !hearsItsOwn && !hearsAStream) {
return Consumer{}, false
}
perms, err := PermissionsFor(p)
if err != nil {
return Consumer{}, false
}
// Events, wherever they live: a module's own namespace, and the namespace of any role it watches
// (novox/hq ADR 0121). Tool subjects and inboxes are subscribed directly and are not a consumer's
// business, which is why this is a filter and not the whole list.
var filters []string
for _, s := range perms.Subscribe {
// Not a module stream's, which its own consumer on that stream hears (novox/hq ADR 0363).
if strings.Contains(s, ".event.") && !strings.HasPrefix(s, moduleStreamPrefix) {
filters = append(filters, s)
}
}
// **A module that hears only a module's stream still has one here** (novox/hq ADR 0363 §7): what it
// gives up on there is delivered again under `mesh.again.<consumer>.` on EVENTS, and only this filters it.
if len(filters) == 0 && !hearsAStream {
return Consumer{}, false
}
sort.Strings(filters)
// And the events given up on and delivered again to this consumer alone (novox/hq issue 330).
filters = append(filters, AgainFilter(consumerDurable(p)))
return Consumer{
Name: consumerDurable(p),
Stream: consumerStream(p),
Filters: filters,
AckWaitSeconds: 30,
MaxDeliver: 5,
// From now, as the controller's own consumers have been since issue 248: a module made on a stream that
// keeps a week replays none of it. What a part needs of the present it reads from its owner
// (novox/hq ADR 0443 §1, §3). A consumer that exists keeps its place whatever this says.
FromNow: true,
Why: "what " + p.Module + " declared it consumes; after max-deliver it gives an event up, and the " +
"controller keeps it in DEAD_LETTERS until a person delivers it again or drops it",
}, true
}
// ModuleConsumer is one module's durable consumer, with the module and node it is for.
type ModuleConsumer struct {
Node, Module string
Consumer Consumer
}
// ConsumersOf is every module's durable consumer the composed users imply: each module user's, and
// each module a runtime carries — a carried module with no account of its own is no user (novox/hq
// issue 195), and its consumer is still the controller's to make, since the runtime reads it on the
// module's behalf (ADR 0198). One per module and node, whichever of the two named it first.
func ConsumersOf(users []Principal) []ModuleConsumer {
var out []ModuleConsumer
seen := map[string]bool{}
add := func(p Principal) {
key := p.Node + "/" + p.Module
if seen[key] {
return
}
c, needed := ConsumerFor(p)
// **A module with a work-queue consumer of its own needs no other reason to be visited here**
// (novox/hq ADR 0436 §1, §2, §6): mesh-mail's sender consumes and watches nothing of its own, so
// ConsumerFor alone would have found nothing needed and skipped it before its one queue consumer
// was ever composed.
if !needed && len(p.Queues) == 0 {
return
}
seen[key] = true
if needed {
out = append(out, ModuleConsumer{Node: p.Node, Module: p.Module, Consumer: c})
// And one on each module stream it hears (novox/hq ADR 0363 §5).
for _, h := range p.Hears {
if len(h.Filters) == 0 {
continue
}
out = append(out, ModuleConsumer{Node: p.Node, Module: p.Module, Consumer: heardConsumer(p, h)})
}
}
// And the one durable consumer of each of its own work-queue streams, for the one process its
// bus.queues names (novox/hq ADR 0436 §1, §2, §6).
for _, q := range p.Queues {
out = append(out, ModuleConsumer{Node: p.Node, Module: p.Module, Consumer: QueueConsumer(p, q)})
}
}
for _, p := range users {
if p.Kind == KindModule {
add(p)
}
}
for _, p := range users {
if p.Kind != KindNodeTools {
continue
}
for _, d := range p.Carries {
// **A carried module's work-queue consumer is still the controller's to make** (novox/hq ADR
// 0436 §1, §2, §6), the same reason its ordinary consumer is (ADR 0198): mesh-mail has no
// account of its own (NoAccount) and is carried here, but its process does, and the consumer
// it reads through must exist and survive PruneHeard whether or not the module itself speaks.
add(Principal{Kind: KindModule, Node: p.Node, Module: d.Module, Emits: d.Emits,
Consumes: d.Consumes, Serves: d.Serves, Holds: d.Holds, Uses: d.Uses, Watches: d.Watches,
Streams: d.Streams, Hears: ComposedHears(d), Queues: d.Queues})
}
}
return out
}
// HolderConsumerFor is the worker a seat's holders share on that seat's work queue.
//
// **One worker for every holder, and each holder pulls one ask when it is idle** (novox/hq ADR
// 0190). The seat is *authority* — who may be the telegram sender — and the worker is *delivery*,
// kept separate so that relaxing one changes nothing about the other: a node-scoped seat has a
// holder per machine, and all of them take from this one consumer, so the work is shared without
// any holder knowing about the others. Pulled rather than pushed because a push consumer hands the
// next ask to whichever subscriber the server picks, busy or not, and a pulled one is asked for by
// a holder that has just become free. Which is also what ends the race issue 186 describes — asks
// delivered behind the one being worked, expiring unacknowledged and dropped after the fifth
// redelivery: nothing is delivered that nobody asked for. A long build keeps its own ask alive
// (stillWorking); the ack wait is for a holder that died.
func HolderConsumerFor(node, module string, seat DeclaredSeat) (Consumer, bool) {
if len(seat.Accepts) == 0 || seat.PulledByKind {
// A seat pulled by kind has a worker per kind, which its holders' seat traffic derives (SeatTrafficObjects): one
// worker filtered on every accept would overlap them on the work queue.
return Consumer{}, false
}
return Consumer{
Name: "SEAT_" + upperSnake(seat.Name) + "_worker",
Stream: seatStreamName(seat.Name),
Filters: []string{"mesh.seat." + seat.Name + ".accept.>"},
AckWaitSeconds: 60,
MaxDeliver: 5,
// As many in flight as there are holders working, which pulling bounds by itself: a holder
// fetches one and fetches again only after it acknowledged. The server's default stands.
Why: fmt.Sprintf("%s on %s holds %s; every holder pulls one ask at a time from this worker "+
"and acknowledges after the work is done, so a crash mid-work redelivers rather than "+
"loses and an idle holder is the one that takes the next ask", module, node, seat.Name),
}, true
}
// NodeConsumer is the durable consumer a node reads its own declaration through.
//
// **Derived from a node existing, and created by the controller, because a host cannot create it.**
// A host's account may subscribe its own declaration subject and publish its own ack subject, and
// reaches no part of the JetStream API — which is correct (the controller is the only writer of
// consumer definitions, design 25 §3) and means the consumer must be waiting before the host binds
// to it. Named after the node, because the node's ack grant is `$JS.ACK.NODES.<node>.>` and a
// consumer named anything else is one the host cannot acknowledge a delivery from.
//
// **No max-deliver, and a long ack wait.** A declaration is settled only after the node has applied
// it and reported, which is minutes on a machine pulling images; and a declaration the mesh cannot
// get a node to accept is not one to give up on, because the stream keeps only the newest per node
// anyway — so there is exactly one message per node to redeliver, for as long as that node is away.
func NodeConsumer(node string) Consumer {
return Consumer{
Name: node,
Stream: "NODES",
Filters: []string{"mesh.node." + node + ".declare"},
Push: true,
AckWaitSeconds: 300,
Why: "how " + node + " hears what it should be; last-per-subject, so a node that was away " +
"gets exactly the current declaration and nothing older",
}
}
// AssertNodeConsumers brings every known node's declaration consumer into being.
//
// Asserted on start as well as created at enrolment, for the reason the streams are: a mesh raised
// from a restored backup, or one whose bus was recreated, has node records and no consumers, and a
// node whose consumer is missing hears nothing while everything else about it looks correct.
func AssertNodeConsumers(e Ensurer, nodes []string) error {
for _, n := range nodes {
if err := e.EnsureConsumer(NodeConsumer(n)); err != nil {
return fmt.Errorf("asserting how %s hears its declaration: %w", n, err)
}
}
return nil
}
// AllOverlaps reports subject filters claimed by more than one stream, across the mesh's own and
// every derived one.
//
// NATS refuses an overlapping stream rather than merging it (verified against nats-server 2.10:
// "subjects overlap with an existing stream"), so this is not a subtle divergence — it is a
// registration that fails. Catching it here names both streams, before a half-applied mesh does.
func AllOverlaps(seats []DeclaredSeat) []string {
all := append(MeshStreams(), SeatStreams(seats)...)
seen := map[string]string{}
var clashes []string
for _, s := range all {
for _, subject := range s.Subjects {
if first, ok := seen[subject]; ok {
clashes = append(clashes, fmt.Sprintf("%s and %s both claim %s", first, s.Name, subject))
continue
}
seen[subject] = s.Name
}
}
sort.Strings(clashes)
return clashes
}
// upperSnake makes a stream name from a seat name. NATS stream names may not contain a dot,
// a space or a wildcard, and a hyphen is legal but reads badly beside the mesh's own.
func upperSnake(s string) string {
out := []rune(s)
for i, r := range out {
switch {
case r >= 'a' && r <= 'z':
out[i] = r - 32
case r == '-' || r == '.':
out[i] = '_'
}
}
return string(out)
}
+12
View File
@@ -0,0 +1,12 @@
// Package buscompose is the bus's composer: who may say what on the bus, as the user list and the server configuration
// the bus loads, derived from the mesh's records (Users, WithPasswords, ComposeAccounts, Compose), with the grants,
// streams, state and logs it composes them from.
//
// **A copy of the controller's internal/broker, composer half** (issue 1377, novox/hq ADR 0547 §5), taken at
// mesh-controller 7cd4d18b with the grants part's membership grant of f67d1553 (ADR 0547 §5) and its grant to create the streams manifests declare (issue 1790): the bus client (jetstream.go), the broker's address from the environment (broker.go, onnats.go
// but its one constant) and the two methods on that client are left out; the lease's two subjects are copied in
// (nats.go). The controller keeps its own copy, unchanged, until it loses its lease at ADR 0547's third milestone, so
// for that time two composers state who may say what and must say the same: TestTheCopiedComposerComposesAsTheControllers
// holds this copy to lists the controller's own code wrote from the same fixtures. A change to what is granted is made
// here and, while the controller composes, in the controller's copy too, never in one alone.
package buscompose
+119
View File
@@ -0,0 +1,119 @@
package buscompose
// The fixtures the copied bus composer is held to (issue 1377, ADR 0547 §5): the same source is compiled into the
// controller's internal/broker to write the expected lists, and into mesh-sdk's buscompose to compare against them.
// UserListFixture is one mesh's records, and the hashes its accounts carry.
type UserListFixture struct {
Name string
Records Records
// Hashless are users given no hash, so they are left out and named.
Hashless []string
}
func fixtureTelegramSeat() Seat {
return Seat{Name: "telegram-sender", Accepts: []string{"send"}, Emits: []string{"delivered", "failed"}}
}
// UserListFixtures are the meshes, from the controller's own tests' records outward.
func UserListFixtures() []UserListFixture {
return []UserListFixture{
{Name: "empty"},
{Name: "the controller's someRecords", Records: Records{
Nodes: []string{"two", "one"},
Assigned: map[string][]Declared{
"one": {{Module: "telegram", Serves: []string{"status"}}},
"two": {{Module: "shop", Emits: []string{"order.placed"}}},
},
Enrolling: []string{"three"},
People: map[string][]string{"ada": {"mesh-catalog.catalog_tools"}},
}},
{Name: "seats, consumers, people and a hashless user", Hashless: []string{"two.audit"}, Records: Records{
Nodes: []string{"one", "two"},
Assigned: map[string][]Declared{
"one": {
{Module: "telegram", Holds: []Seat{fixtureTelegramSeat()}, Serves: []string{"status"}},
{Module: "nats", SnapshotsTheBus: true, Serves: []string{"nats_streams"}},
{Module: "mesh-watchman"},
{Module: "mesh-grants", Emits: []string{"membership-changed"}},
},
"two": {
{Module: "shop", Uses: []Seat{fixtureTelegramSeat()}, Emits: []string{"order.placed"}},
{Module: "audit", Consumes: []string{"shop.order.placed"}},
},
},
People: map[string][]string{"ada": {"*"}, "bob": {"shop.status"}},
Interchangeable: map[string]bool{"shop": true},
}},
{Name: "state, logs, streams, queues and processes", Records: Records{
Nodes: []string{"ace", "novox"},
Assigned: map[string][]Declared{
"novox": {
{Module: "mesh-delivery-builder", Emits: []string{"built"},
State: []Bucket{{Module: "mesh-delivery-builder", Name: "steps", History: 5},
{Module: "mesh-delivery-builder", Name: "presence", PerMachine: true, KeyTTL: true, TTLSeconds: 60}},
Logs: []Log{{Module: "mesh-delivery-builder", Name: "builds", MaxMiB: 64}},
Streams: []ModuleStream{{Module: "mesh-delivery-builder", Name: "asks", Events: []string{"ask"}, MaxAgeDays: 7, MaxMiB: 32, WorkQueue: true}},
Queues: []QueueOwner{{Process: "worker", Stream: "asks", MaxDeliver: 5}},
Processes: []ProcessPrincipal{{Process: "worker", Emits: []string{"built"},
Queues: []QueueOwner{{Process: "worker", Stream: "asks", MaxDeliver: 5}},
State: []string{"steps"}, Invokes: []string{"gitea.gitea_commit"}}},
Invokes: []string{"gitea.gitea_commit"},
},
{Module: "mesh-delivery-view", Reads: []string{"mesh-delivery-builder.steps"},
KeyedReads: []KeyedRead{{Read: "mesh-delivery-builder.presence", Key: "novox"}}},
// The grants part, which may create every stream declared here (ADR 0547 §5, issue 1790).
{Module: "mesh-grants", Emits: []string{"membership-changed"},
State: []Bucket{{Module: "mesh-grants", Name: "assertions", History: 5}}},
},
"ace": {
{Module: "plex", Serves: []string{"plex_status"}, Checks: []string{"plex.plex_status"}},
{Module: "silent", NoAccount: true},
},
},
RootFree: map[string]bool{"ace": true},
}},
}
}
// HashOf is the stand-in hash a fixture's user carries.
func HashOf(user string) string {
h := "$2a$11$"
for len(h) < 29 {
h += user
}
return h[:29] + "fixturefixturefixturefixture00"
}
// ComposedFor is a fixture's user list as the composer writes it, and the users left out.
func ComposedFor(f UserListFixture) (string, []string, error) {
users, err := Users(f.Records)
if err != nil {
return "", nil, err
}
hashes := map[string]string{}
skip := map[string]bool{}
for _, u := range f.Hashless {
skip[u] = true
}
for _, u := range users {
if !skip[u.Username()] {
hashes[u.Username()] = HashOf(u.Username())
}
}
filled, missing := WithPasswords(users, hashes)
list, err := ComposeAccounts(filled)
return list, missing, err
}
// MembershipsFor is every membership a fixture's records issue, by `<node>.<module>`, as the composer makes them.
func MembershipsFor(f UserListFixture) map[string]Membership {
where := PlacementsOf(f.Records, f.Records.Interchangeable)
out := map[string]Membership{}
for node, ds := range f.Records.Assigned {
for _, d := range ds {
out[node+"."+d.Module] = MembershipFor(node, d, where)
}
}
return out
}
+184
View File
@@ -0,0 +1,184 @@
package buscompose
import (
"fmt"
"sort"
"strings"
"github.com/nats-io/nats.go/jetstream"
)
// A module's log on the bus (novox/hq ADR 0297): its record of operations, each entry appended under
// a key and kept as long as the log is.
//
// A log follows a bucket in every respect (ADR 0201): the module names it locally, the mesh derives
// its stream and subjects, the controller creates it from the catalogue on every raise and from
// registration, never a module, and **no path of the mesh removes one** — a log whose declaration
// is gone is reported, as a bucket is. Unlike a bucket, a log is its owner's alone: no other module
// reads it, so there is no read of a log to grant or issue.
//
// Pure, but for the stream's configuration, which is the server's own type so that what is tested
// is exactly what is sent; jetstream.go is the part that asks a server.
// The mesh's caps on a log: what an entry's value may weigh, what a message on the log's stream may
// weigh — the value and its headers, which the server counts in a message's size, so a value of the
// full size still fits with the expected-last-sequence header an append carries — and what a log holds
// when its module says nothing and at most.
const (
LogMaxEntryBytes = 256 * 1024
LogMaxMessageBytes = LogMaxEntryBytes + 4*1024
LogDefaultMiB = 1024
LogMostMiB = 8192
)
// A Log is one module's declared log as the bus holds it.
type Log struct {
Module string
Name string
// MaxMiB is its cap in MiB; zero is LogDefaultMiB.
MaxMiB int
}
// LogStreamPrefix starts every log's stream name, which no other stream of the mesh's starts with.
const LogStreamPrefix = "LOG_"
// LogStreamName is the stream a module's log lives in: `LOG_<module>_<name>`. The module and the
// local name are each one token with no underscore, so two modules can never derive one stream.
func LogStreamName(module, name string) string { return LogStreamPrefix + module + "_" + name }
// LogSubject is the subject a log's entries are published under, without the key: an entry for key K
// is on `<LogSubject>.<K>`.
func LogSubject(module, name string) string { return "mesh.log." + module + "." + name }
// Stream is this log's stream name.
func (l Log) Stream() string { return LogStreamName(l.Module, l.Name) }
// Subject is this log's subject, without the key.
func (l Log) Subject() string { return LogSubject(l.Module, l.Name) }
// MaxBytes is this log's cap in bytes.
func (l Log) MaxBytes() int64 {
mib := l.MaxMiB
if mib <= 0 {
mib = LogDefaultMiB
}
return int64(mib) * 1024 * 1024
}
// Why is carried into the server's description of the stream, so somebody reading the server's own
// state finds whose it is and why it is kept.
func (l Log) Why() string {
return fmt.Sprintf("%s's log %q (novox/hq ADR 0297): its record of operations, one entry per operation "+
"under its key, appended by %s alone and kept as long as the log; never removed by the mesh, because "+
"it is data", l.Module, l.Name, l.Module)
}
// Config is the stream a log is, field by field as novox/hq ADR 0297 fixes it: a file stream kept by
// limits, with no maximum age and no cap per key, whose message is an entry's value and 4 KiB of
// headers at most, that refuses a new entry when full rather than
// drop an old one, and that refuses deleting an entry or purging it. Direct gets are allowed, which
// is how the runtime reads it.
func (l Log) Config() jetstream.StreamConfig {
return jetstream.StreamConfig{
Name: l.Stream(),
Description: l.Why(),
Subjects: []string{l.Subject() + ".>"},
Storage: jetstream.FileStorage,
Retention: jetstream.LimitsPolicy,
MaxAge: 0,
MaxMsgs: -1,
MaxMsgsPerSubject: -1,
MaxBytes: l.MaxBytes(),
MaxMsgSize: LogMaxMessageBytes,
Discard: jetstream.DiscardNew,
AllowDirect: true,
DenyDelete: true,
DenyPurge: true,
Replicas: 1,
}
}
// logsIssuedFor is every log a module's code may reach, as its membership lists them: its own.
func logsIssuedFor(d Declared) []LogIssued {
var out []LogIssued
for _, l := range d.Logs {
if !safeSubject.MatchString(d.Module) || !safeSubject.MatchString(l.Name) {
continue
}
out = append(out, LogIssued{Name: l.Name, Stream: LogStreamName(d.Module, l.Name),
Subject: LogSubject(d.Module, l.Name), Writes: true})
}
return out
}
// logGrants is what a principal publishes to reach the logs its module keeps: for each, appending
// under the log's subjects, binding to its stream, and reading it directly. Nothing more — the
// runtime reads a log by direct gets alone and makes no consumer on it — and nothing of any other
// module's log, because a log is its owner's alone. Replies come on the principal's inbox, as for a
// bucket.
func logGrants(module string, names []string) []string {
if !safeSubject.MatchString(module) {
return nil
}
var out []string
for _, name := range names {
if !safeSubject.MatchString(name) {
continue
}
stream := LogStreamName(module, name)
out = append(out,
LogSubject(module, name)+".>",
"$JS.API.STREAM.INFO."+stream,
"$JS.API.DIRECT.GET."+stream,
"$JS.API.DIRECT.GET."+stream+".>")
}
return out
}
// logNames is the local names of a module's logs.
func logNames(logs []Log) []string {
out := make([]string, 0, len(logs))
for _, l := range logs {
out = append(out, l.Name)
}
return out
}
// A LogAsserter is the part of a JetStream connection log assertion needs. It has no way to remove a
// log, by design: nothing the mesh runs asks for one.
type LogAsserter interface {
// EnsureLog creates the log's stream if absent and brings its configuration to match if present,
// never deleting or recreating it.
EnsureLog(l Log) error
// LogStreams is every log stream on the server: every stream named with LogStreamPrefix.
LogStreams() ([]string, error)
}
// RaiseLogs asserts every declared log and answers the log streams on the server that nothing
// declares any more.
//
// **Those are reported, never removed** (novox/hq ADR 0297 §2, ADR 0201 §15–16): a log is a module's
// record, and a manifest edited, a module renamed or unassigned is an ordinary day's work that must
// not take a record with it. Removing one is a person's act, outside the mesh.
func RaiseLogs(a LogAsserter, logs []Log) (undeclared []string, err error) {
sorted := append([]Log(nil), logs...)
sort.Slice(sorted, func(i, j int) bool { return sorted[i].Stream() < sorted[j].Stream() })
declared := map[string]bool{}
for _, l := range sorted {
if err := a.EnsureLog(l); err != nil {
return nil, fmt.Errorf("asserting %s's log %q: %w", l.Module, l.Name, err)
}
declared[l.Stream()] = true
}
names, err := a.LogStreams()
if err != nil {
return nil, fmt.Errorf("listing the bus's logs: %w", err)
}
for _, n := range names {
if strings.HasPrefix(n, LogStreamPrefix) && !declared[n] {
undeclared = append(undeclared, n)
}
}
sort.Strings(undeclared)
return undeclared, nil
}
+178
View File
@@ -0,0 +1,178 @@
package buscompose
import (
"slices"
"sort"
"strings"
)
// What the mesh issues an assignment to serve and to reach (novox/hq ADR 0160).
//
// A module's code names its tools and its events; **where they land is the mesh's to decide**, and
// it decided it twice — once in the runtime, once here, by one rule compiled into both. Now the
// controller composes a membership for every module on every machine and publishes it to a subject
// only that assignment reads; the runtime serves exactly what the membership says, and the account's
// grant is the same composition read the other way. The shape issued today is the shape the mesh
// already had, so nothing moves when a membership first arrives; only who decides it moves.
// Placements is where every module runs, for deciding which instance answers for the module.
type Placements struct {
// Nodes is each module's machines.
Nodes map[string][]string
// Interchangeable is each module whose definition says its instances are the same anywhere,
// so the module's plain subject is issued to all of them in one queue.
Interchangeable map[string]bool
// Kinds is every kind held of a kinded bench, by whom and with what capabilities (ADR 0259 §5).
Kinds []KindHeld
}
// AnswersForTheModule says whether an instance of a module on one machine is issued the module's
// plain subject: when it is the only instance, or when the definition says instances are
// interchangeable. A stateful module on two machines gets only its machines' subjects, so a call
// that names none reaches nothing rather than the wrong store.
func (p Placements) AnswersForTheModule(module string) bool {
return len(p.Nodes[module]) <= 1 || p.Interchangeable[module]
}
// MembershipFor composes one assignment's membership from what it declared and where everything
// runs. The subjects are the ones PermissionsFor grants, derived here once more only until the
// grant itself is read from the membership — which is the next step, not this one.
func MembershipFor(node string, d Declared, where Placements) Membership {
own := "mesh.mod." + d.Module
m := Membership{
Node: node, Module: d.Module,
Emits: own + ".event.{event}",
Tools: own + ".tool.tools",
}
// This machine's address always; the module's when this instance answers for the module.
m.Serves = append(m.Serves, Served{Subject: own + ".tool.{tool}." + node})
if where.AnswersForTheModule(d.Module) {
m.Serves = append(m.Serves, Served{Subject: own + ".tool.{tool}", Queue: "serve." + d.Module})
}
for _, s := range d.Holds {
if servedOnlyByTheController(s) {
continue // answered by the serving controller alone, never through a membership
}
for _, verb := range s.Serves {
m.Seats = append(m.Seats, SeatServed{Seat: s.Name, Verb: verb, Subject: seatToolSubject(s, verb, node)})
}
}
m.State = stateIssuedFor(d, node)
m.Logs = logsIssuedFor(d)
m.Streams = streamsIssuedFor(d)
m.Hears = HeardIssuedFor(node, d)
m.Waits = waitsIssuedFor(d)
t := SeatTrafficOf(d.Module, d.Holds, d.Uses, d.Watches)
for _, s := range append(append([]Seat{}, d.Uses...), d.Watches...) {
if !s.Kinded {
continue
}
for _, k := range where.Kinds {
if k.Seat == s.Name && !kindListed(t.Kinds, k) {
t.Kinds = append(t.Kinds, k)
}
}
}
if len(t.Publish)+len(t.Subscribe)+len(t.Answers)+len(t.Workers)+len(t.Records)+len(t.Kinds) > 0 {
m.SeatTraffic = &t
}
if len(d.Invokes) > 0 {
m.Reaches = map[string][]string{}
for _, t := range d.Invokes {
if t == "*" || strings.HasPrefix(t, "seat:") {
continue // every tool, or a role's: addressed by name, not resolved per instance
}
module, tool, ok := strings.Cut(t, ".")
if !ok {
continue
}
var reach []string
if where.AnswersForTheModule(module) {
reach = append(reach, "mesh.mod."+module+".tool."+tool)
}
nodes := append([]string{}, where.Nodes[module]...)
sort.Strings(nodes)
for _, n := range nodes {
reach = append(reach, "mesh.mod."+module+".tool."+tool+"."+n)
}
m.Reaches[t] = reach
}
}
return m
}
// PlacementsOf reads where everything runs from the records the bus's accounts are composed from.
func PlacementsOf(r Records, interchangeable map[string]bool) Placements {
p := Placements{Nodes: map[string][]string{}, Interchangeable: interchangeable}
for node, declared := range r.Assigned {
for _, d := range declared {
p.Nodes[d.Module] = append(p.Nodes[d.Module], node)
}
}
for _, nodes := range p.Nodes {
sort.Strings(nodes)
}
// Where the router runs: a verified sender is believed only while its machine is root-free too. A router
// placed nowhere, or on more than one machine, frees nothing.
var routerNodes []string
for node, declared := range r.Assigned {
for _, d := range declared {
for _, s := range d.Holds {
if s.Name == routerSeat && !slices.Contains(routerNodes, node) {
routerNodes = append(routerNodes, node)
}
}
}
}
routerFree := len(routerNodes) == 1 && r.RootFree[routerNodes[0]]
for node, declared := range r.Assigned {
for _, d := range declared {
for _, s := range d.Holds {
if s.Kinded && s.Kind != "" {
p.Kinds = append(p.Kinds, KindHeld{Seat: s.Name, Kind: s.Kind, Module: d.Module, Node: node,
Capabilities: placedCapabilities(s.Capabilities, d.RunsAs, routerFree && r.RootFree[node])})
}
}
}
}
sort.Slice(p.Kinds, func(i, j int) bool {
a, b := p.Kinds[i], p.Kinds[j]
if a.Seat != b.Seat {
return a.Seat < b.Seat
}
if a.Kind != b.Kind {
return a.Kind < b.Kind
}
return a.Node < b.Node
})
return p
}
// routerSeat is the seat the router of asks holds (novox/hq ADR 0259 §3).
const routerSeat = "operator-channel"
// placedCapabilities is what a kind's claim promises, as far as its placement lets the router believe it
// (novox/hq ADR 0259 §8): `verified-sender` only from a holder that runs as an account of its own, on a bus
// account of its own — never one the machine's runtime carries as the operator's account — and only while
// its machine and the router's were root-free when composed (rootFree; the review of 2026-10-09, H3). The
// membership carrying it is composed at a push, so it can outlive a pass that later fails: the router asks
// the controller's root-free verb again before it honours an approval, and that is the check that holds.
func placedCapabilities(declared []string, runsAs string, rootFree bool) []string {
var out []string
for _, c := range declared {
if c == "verified-sender" && (runsAs == "" || !rootFree) {
continue
}
out = append(out, c)
}
return out
}
func kindListed(list []KindHeld, k KindHeld) bool {
for _, x := range list {
if x.Seat == k.Seat && x.Kind == k.Kind && x.Module == k.Module && x.Node == k.Node {
return true
}
}
return false
}
+343
View File
@@ -0,0 +1,343 @@
package buscompose
import (
"fmt"
"sort"
"strings"
)
// A module's own stream on the bus (novox/hq ADR 0363).
//
// A module declares a stream as it declares a log: named locally, created by the controller from the
// catalogue on every raise and every send, never removed by the mesh. Its events are published on
// `mesh.stream.<module>.event.<event>`, a subject space the shared EVENTS stream (`mesh.mod.*.event.>`)
// does not take — the bus refuses two streams claiming one subject, so a module's stream cannot share
// EVENTS' — and a consumer hears it through a durable consumer of its own on that stream, named as on
// EVENTS. Pure; the controller's raise asserts what is composed here.
// moduleStreamPrefix starts every subject a module's own stream takes.
const moduleStreamPrefix = "mesh.stream."
// ModuleStreamSubject is where a module's event on its own stream lands: `mesh.stream.<module>.event.<event>`.
// The token before `.event.` is the emitter, as on EVENTS, so a runtime reads the same key from it.
func ModuleStreamSubject(module, event string) string {
return moduleStreamPrefix + module + ".event." + event
}
// OnAModuleStream says a subject is one a module's own stream takes.
func OnAModuleStream(subject string) bool { return strings.HasPrefix(subject, moduleStreamPrefix) }
// WirePattern is a local event pattern as the bus spells it: `**` is `>`.
func WirePattern(local string) string {
parts := strings.Split(local, ".")
for i, p := range parts {
if p == "**" {
parts[i] = ">"
}
}
return strings.Join(parts, ".")
}
// A ModuleStream is one module's declared stream as the bus holds it.
type ModuleStream struct {
Module string
// Name is its name on the bus, `MAIL`.
Name string
// Events are the events it carries, as the bus spells them (WirePattern).
Events []string
MaxAgeDays int
MaxMiB int
// Restricted says a trusted config value of the module grants who hears it (ADR 0363 §4).
Restricted bool
// WorkQueue says this stream holds ADR 0436 §6's work-queue retention: a message is removed once a
// consumer acknowledges it, and the bus refuses a new one when full rather than dropping its oldest.
WorkQueue bool
// Duplicates is the work-queue's duplicate window in hours (novox/hq ADR 0436 §6); meaningless unless
// WorkQueue.
Duplicates int
}
// Subjects are the subjects the stream takes.
func (s ModuleStream) Subjects() []string {
var out []string
for _, e := range s.Events {
out = append(out, ModuleStreamSubject(s.Module, e))
}
sort.Strings(out)
return out
}
// Stream is the stream the controller asserts: kept by limits, its age and size the module's, dropping
// its oldest when full — a consumer that was away catches up within the age, and the module's own record
// answers for anything older.
func (s ModuleStream) Stream() Stream {
why := fmt.Sprintf("%s's own stream (novox/hq ADR 0363): its events, kept %d days and at most %d MiB so a "+
"consumer that was away catches up", s.Module, s.MaxAgeDays, s.MaxMiB)
if s.Restricted {
why += "; heard only by the consumers its trusted config value grants"
}
if s.WorkQueue {
why += "; a work-queue (novox/hq ADR 0436 §6): a message is removed once its one consumer " +
"acknowledges it, and a full stream refuses a new one rather than dropping what it holds"
}
out := Stream{
Name: s.Name,
Subjects: s.Subjects(),
Retention: RetentionLimits,
MaxAge: s.MaxAgeDays * 24 * 60 * 60,
MaxBytes: int64(s.MaxMiB) * 1024 * 1024,
Why: why,
// Said on the bus, so a dead letter kept from it ages out with it (novox/hq ADR 0376 §3).
Restricted: s.Restricted,
}
if s.WorkQueue {
out.Retention = RetentionWorkQueue
out.DiscardNew = true
out.DuplicatesSeconds = s.Duplicates * 60 * 60
// Said on the bus too, reusing the one flag a restricted stream's dead letters already age and hide
// by (novox/hq ADR 0436 §6, ADR 0376 §3-4): a work-queue stream's grant is always empty (ADR 0436
// §6 refuses one that carries a grant), so this never collides with a trusted-config restriction.
out.Restricted = true
// And its own metadata, apart from the bus's work-queue retention SEAT_* and other mesh-internal
// streams share without being one of this clause's: only this flag says a dead letter kept from the
// stream is drop-only (novox/hq ADR 0436 §6), never the stream's Retention alone.
out.WorkQueue = true
}
return out
}
// QueueOwner is the one process of a module that is the durable consumer of one of its own work-queue
// streams (novox/hq ADR 0436 §1, §2, §6): named in that process's bus.queues, and checked at registration
// to be the only one (catalogue.WorkQueueStreamProblems).
type QueueOwner struct {
// Process is the process resource's id — its bus account's name, with the node and module (novox/hq ADR
// 0436 §2).
Process string
// Stream is the work-queue stream's name on the bus.
Stream string
// MaxDeliver bounds how many times a message is handed over before it is given up as a dead letter
// (novox/hq ADR 0436 §6).
MaxDeliver int
}
// QueueConsumer is the durable consumer the one process a module's bus.queues names is granted on its own
// work-queue stream (novox/hq ADR 0436 §1, §2, §6): no filter, since it is the stream's only consumer —
// unlike heardConsumer, which is one of possibly several hearing a module's ordinary stream.
func QueueConsumer(p Principal, q QueueOwner) Consumer {
return Consumer{
Name: p.Node + "_" + p.Module + "_" + q.Process,
Stream: q.Stream,
AckWaitSeconds: 30,
MaxDeliver: q.MaxDeliver,
Why: fmt.Sprintf("%s's process %s is the one consumer %s's bus.queues names for its work-queue "+
"stream %s (novox/hq ADR 0436 §6); after max-deliver %d it gives a message up, and the "+
"controller keeps it in %s, offered only drop", p.Module, q.Process, p.Module, q.Stream,
q.MaxDeliver, DeadLettersStream),
}
}
// ModuleStreams is every declared module stream as the controller asserts it, in name order.
func ModuleStreams(declared []ModuleStream) []Stream {
sorted := append([]ModuleStream(nil), declared...)
sort.Slice(sorted, func(i, j int) bool { return sorted[i].Name < sorted[j].Name })
out := make([]Stream, 0, len(sorted))
for _, s := range sorted {
out = append(out, s.Stream())
}
return out
}
// Heard is a module's stream another module hears, and the subjects it hears of it: its `consumes`
// within the stream and, for a restricted stream, within what the provider's trusted config value grants it
// (ADR 0363 §5). Composed only with at least one subject.
type Heard struct {
Stream string
Emitter string
Filters []string
// Restricted says the stream is restricted by its provider's trusted config value (ADR 0363 §4): then it
// is heard only by a module on an account of its own (ADR 0376 §1, ComposedHears).
Restricted bool
}
// ComposedHears is what of a module's hearing the controller composes (novox/hq ADR 0376 §1): every stream it
// hears, but a restricted one only for a module that runs in a runtime of its own on a bus account of its own
// (`runs-as`, ADR 0259). **The tool runner never hears a restricted stream**: a module it carries reaches the bus
// through the tool runner's account, which every agent session on that machine reaches the bus as too, so a
// consumer, its grants or its bind word composed there would hand the stream to any session. A module that
// declares no `runs-as` and keeps its own container credential gets none either: only `runs-as` is an account
// the mesh keeps from agents.
func ComposedHears(d Declared) []Heard {
if d.RunsAs != "" {
return d.Hears
}
var out []Heard
for _, h := range d.Hears {
if !h.Restricted {
out = append(out, h)
}
}
return out
}
// Withheld is a restricted stream a grant gives a module that the controller composes nothing for, because the
// module has no account of its own (novox/hq ADR 0376 §2): said in `doctor` and in the controller's log, never
// silent.
type Withheld struct {
Node, Module, Stream, Emitter string
}
// Said is a withheld hearing in the operator's words: the module, the stream and the remedy.
func (w Withheld) Said() string {
return fmt.Sprintf("%s on %s is granted part of %s's restricted stream %s, and hears none of it: it is carried "+
"by the machine's tool runner, whose bus account every agent session there shares, so nothing is composed "+
"for it. Declare runs-as in %s's manifest, so it runs as an account of its own (novox/hq ADR 0376)",
w.Module, w.Node, w.Emitter, w.Stream, w.Module)
}
// WithheldOf is every granted hearing of a restricted stream composed for nothing, by machine and module.
func WithheldOf(r Records) []Withheld {
var out []Withheld
for _, node := range sortedCopy(r.Nodes) {
for _, d := range r.Assigned[node] {
if d.RunsAs != "" {
continue
}
for _, h := range d.Hears {
if h.Restricted && len(h.Filters) > 0 {
out = append(out, Withheld{Node: node, Module: d.Module, Stream: h.Stream, Emitter: h.Emitter})
}
}
}
}
return out
}
// heardConsumer is the durable consumer a module hears a module's stream through: named as its consumer
// on EVENTS, on that stream, with the subjects it was granted as its filters.
func heardConsumer(p Principal, h Heard) Consumer {
filters := append([]string(nil), h.Filters...)
sort.Strings(filters)
return Consumer{
Name: consumerDurable(p),
Stream: h.Stream,
Filters: filters,
AckWaitSeconds: 30,
MaxDeliver: 5,
// From now, like a module's consumer on EVENTS (novox/hq ADR 0443 §1).
FromNow: true,
Why: "what " + p.Module + " consumes of " + h.Emitter + "'s stream " + h.Stream + " and is granted " +
"(novox/hq ADR 0363); after max-deliver it gives an event up, and the controller keeps it in " +
"DEAD_LETTERS until a person delivers it again or drops it",
}
}
// heardGrants are what a principal publishes to read and acknowledge the consumers it hears module streams
// through: asking about each, pulling from it, acknowledging it — its own name on each stream, no other's.
func heardGrants(p Principal, hears []Heard) []string {
var out []string
durable := consumerDurable(p)
for _, h := range hears {
if !safeSubject.MatchString(h.Stream) || len(h.Filters) == 0 {
continue
}
out = append(out,
"$JS.API.CONSUMER.INFO."+h.Stream+"."+durable,
"$JS.API.CONSUMER.MSG.NEXT."+h.Stream+"."+durable,
"$JS.ACK."+h.Stream+"."+durable+".>")
}
return out
}
// ownStreamGrants are the subjects a module publishes its own streams' events on, and no other's.
func ownStreamGrants(module string, streams []ModuleStream) []string {
var out []string
for _, s := range streams {
if s.Module != module {
continue
}
out = append(out, s.Subjects()...)
}
return out
}
// streamsIssuedFor is every stream of its own a module's membership names.
func streamsIssuedFor(d Declared) []StreamIssued {
var out []StreamIssued
for _, s := range d.Streams {
if s.Module != d.Module || !safeSubject.MatchString(d.Module) {
continue
}
out = append(out, StreamIssued{Stream: s.Name, Events: append([]string(nil), s.Events...),
Subject: ModuleStreamSubject(d.Module, "{event}")})
}
return out
}
// HeardIssuedFor is every module stream an assignment hears, as its membership names it.
func HeardIssuedFor(node string, d Declared) []HeardIssued {
var out []HeardIssued
// A restricted stream's bind word only for a module of its own account (novox/hq ADR 0376 §1).
for _, h := range ComposedHears(d) {
if len(h.Filters) == 0 {
continue
}
out = append(out, HeardIssued{Stream: h.Stream, Consumer: node + "_" + d.Module})
}
sort.Slice(out, func(i, j int) bool { return out[i].Stream < out[j].Stream })
return out
}
// Pruner is the part of a JetStream connection that removes a module's consumer on a module's stream it
// may no longer hear (ADR 0363 §6). Optional: what only records what it is asked to assert has none.
type Pruner interface {
ConsumerNames(stream string) ([]string, error)
RemoveConsumer(stream, name string) error
}
// PruneHeard removes, from each module stream, every durable consumer the composition no longer asks for,
// and answers what it removed as `<stream>/<consumer>`. A consumer that loses its grant loses what it reads
// through, whatever its runtime still holds; a stream that does not exist yet has nothing to remove.
func PruneHeard(p Pruner, streams []ModuleStream, wanted []Consumer) ([]string, error) {
keep := map[string]bool{}
for _, c := range wanted {
keep[c.Stream+"/"+c.Name] = true
}
var removed []string
for _, s := range streams {
names, err := p.ConsumerNames(s.Name)
if err != nil {
return removed, fmt.Errorf("listing the consumers on %s: %w", s.Name, err)
}
for _, n := range names {
if keep[s.Name+"/"+n] {
continue
}
if err := p.RemoveConsumer(s.Name, n); err != nil {
return removed, fmt.Errorf("removing %s's consumer on %s, which is no longer granted: %w", n, s.Name, err)
}
removed = append(removed, s.Name+"/"+n)
}
}
return removed, nil
}
// EnsureHeard asserts a consumer on a module's stream (ADR 0363 §5, §6). **On a restricted stream, a consumer
// the bus will not bring to match is removed**: a pull consumer delivers to whoever pulls it, so one left with
// its wider filters after a narrowing was refused would keep handing over what the grant no longer names,
// whatever the user list now says. The next assertion makes it again with the granted filters. The refusal is
// returned either way, saying what was done, so the send says it and raises it.
func EnsureHeard(r Raiser, c Consumer, restricted bool) error {
err := r.EnsureConsumer(c)
if err == nil || !restricted {
return err
}
p, prunes := r.(Pruner)
if !prunes {
return fmt.Errorf("%w; it could not be removed here, and may still deliver more than it is granted", err)
}
if rerr := p.RemoveConsumer(c.Stream, c.Name); rerr != nil {
return fmt.Errorf("%w; REMOVING IT FAILED TOO, so it may still deliver more than it is granted: %v", err, rerr)
}
return fmt.Errorf("%w; REMOVED, so it delivers nothing it is no longer granted: the next assertion makes it "+
"again with its granted filters (novox/hq ADR 0363 §6)", err)
}
File diff suppressed because it is too large Load Diff
+5
View File
@@ -0,0 +1,5 @@
package buscompose
// NATSVar names the environment variable that says the controller is on the bus (mesh-controller internal/broker
// onnats.go); kept with the composer for the readiness it composes.
const NATSVar = "MESH_BUS_NATS"
+52
View File
@@ -0,0 +1,52 @@
package buscompose
// The bus account of one process of a module that declares bus (novox/hq ADR 0436 §1, §2, §3, §5).
//
// **Never the module's own account's grants, never the tool runner's.** A process's authority is read
// straight from its bus declaration — its emits, its queue's consumer, its state, whether it reads the
// records of its module's own asks, and the tools and seat verbs it calls — and nothing else: it holds no
// seat, serves no tool, hears no module stream, and answers nothing a caller asked (§3). Modelled on
// provisionerGrants, the other kind of process account the mesh mints (ADR 0349 clauses 6 and 11).
// processGrants is one process's whole authority: to publish, and to subscribe.
func processGrants(p Principal) (pub, sub []string, err error) {
// 1. Its own events, under its module's own namespace — the same subjects the module's own principal
// would publish them on, had it declared them there instead (novox/hq ADR 0436 §1).
own := "mesh.mod." + p.Module
for _, e := range p.Emits {
pub = append(pub, own+".event."+e)
}
// 2. The one durable consumer of each work-queue stream its bus names (novox/hq ADR 0436 §1, §2, §6):
// asked about, pulled from and acknowledged, by the same name ConsumersOf composes the consumer under
// (QueueConsumer) — and no other stream's.
for _, q := range p.Queues {
consumer := p.Node + "_" + p.Module + "_" + q.Process
pub = append(pub,
"$JS.API.CONSUMER.INFO."+q.Stream+"."+consumer,
"$JS.API.CONSUMER.MSG.NEXT."+q.Stream+"."+consumer,
"$JS.ACK."+q.Stream+"."+consumer+".>")
}
// 3. Its module's own state, by the local names its bus names alone — never the whole of what the
// module keeps (novox/hq ADR 0436 §1), read and written as stateGrants gives any module's own bucket.
pub = append(pub, stateGrants(stateAccess{Module: p.Module, Node: p.Node, Keeps: p.State})...)
// 4. The router's records of its module's own asks (novox/hq ADR 0436 §1, ADR 0259 §3): the same
// direct-get subjects a module's own principal reads for the seats its Uses names that keep them —
// read here from the one list, so the two cannot disagree.
if p.Asks {
pub = append(pub, SeatTrafficOf(p.Module, nil, p.Uses, nil).Records...)
}
// 5. The tools and seat verbs it calls (novox/hq ADR 0152, ADR 0436 §1): the same derivation a
// module's own principal's invokes takes, so `callerNamed` (applied to every non-controller principal)
// adds the caller-named `call` variants for it exactly as it would for a module.
invoked, err := invokedSubjects(p.Invokes)
if err != nil {
return nil, nil, err
}
pub = append(pub, invoked...)
return pub, sub, nil
}
+217
View File
@@ -0,0 +1,217 @@
package buscompose
import (
"errors"
"fmt"
"git.novox.be/novox/mesh-sdk/go/delivery"
"strings"
"github.com/nats-io/nats.go"
)
// The provisioner client's own bus account (novox/hq ADR 0349 clauses 6 and 11, design 57 package P3).
//
// **Not the tool runner's account.** The client fills a machine's files and asks the vault for the secrets that
// machine's declaration names; the vault hands a secret only to an account of its own, because a call under the
// tool runner's account cannot be told from an agent's on that machine (ADR 0305 §3, §12). So each machine that
// runs the client has a principal of its own, KindProvisioner, named for the machine, and granted what the client
// does and nothing else: read its own machine's keys in each domain's bucket, write its own machine's keys of the
// provisioner server's needs, applied and collections (design 57 P4), read the server's registry, say what it
// applied on its own machine, and ask the vault in its own name.
// NodeProvisionerSeat is the seat the provisioner client holds on every machine (ADR 0349 clause 3).
const NodeProvisionerSeat = "node-provisioner"
// ProvisionerSecret is the own secret of the module running the client that its bus account is delivered in, as
// `broker` is a module's own (novox/hq issue 195): a module that claims NodeProvisionerSeat and declares no own
// secret of this name has nowhere to read an account, and is given none.
const ProvisionerSecret = "provisioner-broker"
// ConfigResolvedBucket is the config domain's resolved values, keyed `<node>.<module>.<key>` (ADR 0348, research
// 051): mesh-config's state `resolved`, written by mesh-config alone.
var ConfigResolvedBucket = BucketName("mesh-config", "resolved")
// ProvisionerReads are the domains' buckets a provisioner client reads its own machine's keys of, each keyed
// `<node>.…` (research 051): placement, the controller's assignments mirror (`<node>.<module>`), whose value carries
// the provisioner's plan of a module handed to it (inventory.Provisioning, design 57 P5); and config,
// mesh-config's resolved values. **Named one by one**, as the controller's asks are: a domain that moves to the
// provisioner adds its bucket here, and a bucket not named is one the client cannot read.
var ProvisionerReads = []string{ConfigResolvedBucket}
// ProvisionerReadsWhole are the buckets a provisioner client reads every key of: the controller's assignments mirror,
// where a provider's consumers are bound to it in their own machines' plans, which the provider machine's client
// reads to fill ${consumers:<provision>} with each one's credential (novox/hq ADR 0373, issue 679). It holds placement
// and plans, never a secret's value; the client writes none of it.
var ProvisionerReadsWhole = []string{AssignmentsBucket}
// BoundState is a providing module's state of entries, one per consumer (mesh-sdk go/provider, issue 678): a
// consumer's machine's client reads its own keys of it, `<node>.<module>.<provision>[.<local>]`, to fill
// ${bound:<provision>:<key>} (issue 679).
const BoundState = "bound"
// ProvisionerServerModule is the module holding the seat mesh-provisioner once for the mesh (ADR 0349 clause 3,
// design 57 P4), whose buckets the clients write their own machine's keys of.
const ProvisionerServerModule = "mesh-provisioner"
// The provisioner server's buckets (ADR 0349 clause 3): its state, created by the controller from its manifest
// like any module's, and written by its own account whole.
var (
// ProvisionerRegistryBucket is which domain answers which kind of placeholder, keyed by the kind; the
// server's alone, read whole by every client.
ProvisionerRegistryBucket = BucketName(ProvisionerServerModule, "registry")
// ProvisionerNeedsBucket is what each module on each machine needs, `<node>.<module>.<kind>.<name>`.
ProvisionerNeedsBucket = BucketName(ProvisionerServerModule, "needs")
// ProvisionerAppliedBucket is what is applied where, `<node>.<module>`.
ProvisionerAppliedBucket = BucketName(ProvisionerServerModule, "applied")
// ProvisionerCollectBucket is the items of every collection, `<node>.<collection>.<contributor>`.
ProvisionerCollectBucket = BucketName(ProvisionerServerModule, "collect")
)
// ProvisionerWrites are the server's buckets a client writes, and reads, its own machine's keys of (`<node>.>`)
// and no other's: the needs it records, what it applied, and the items of the collections on its machine. A
// client cannot write another machine's needs, or say another machine applied a file.
var ProvisionerWrites = []string{ProvisionerNeedsBucket, ProvisionerAppliedBucket, ProvisionerCollectBucket}
// VerbsTheProvisionerAsksTheVault are the vault's verbs a provisioner client asks in its own name (ADR 0349
// clause 11): `get`, a secret its machine's declaration names, answered sealed to the asker's key. On the subject
// that names its caller, so the vault knows which machine's client asks from the bus's word.
var VerbsTheProvisionerAsksTheVault = []SeatVerb{{Seat: "mesh-vault", Verb: "get"}}
// VaultSecretChangedSubject is where the vault says a secret it holds changed (`secret-changed`: which, by whom,
// when, its generation; never a value, ADR 0349 clause 11), as link.EventSubject addresses mesh-vault's event. A
// provisioner client listens to it so a rotated secret reaches its file without a new declaration (design 57 V4):
// it asks the vault again, in its own name, for a secret its machine's declaration names.
const VaultSecretChangedSubject = "mesh.mod.mesh-vault.event.secret-changed"
// ProvisionerAppliedSubject is where a machine's client says what it applied for one module, applied or failed
// with the reason (ADR 0349 clauses 4 and 15): `prov.applied.<node>.<module>`.
func ProvisionerAppliedSubject(node, module string) string {
return "prov.applied." + node + "." + module
}
// ProvisionerAskSubject is where a machine's tool runner asks that machine's provisioner client for a config value
// or a non-secret bound value of a module it carries, naming the module from the bundle that asks (novox/hq ADR
// 0370 §1, §3, §4; issue 685): `prov.ask.<node>`, which the client subscribes and the tool runner's account of the
// same machine alone may publish. The bus proves only that the tool runner asks; the module is the runtime's word,
// so the client answers no secret here (ADR 0370 §2).
func ProvisionerAskSubject(node string) string { return "prov.ask." + node }
// ProvisionerChangedSubject is where a machine's client says that a value of one module it answers moved, so the
// tool runner asks again for the module's code's watches (ADR 0349 §14, ADR 0370 §1): `prov.changed.<node>.<module>`,
// carrying which value and never the value itself, heard by that machine's tool runner alone.
func ProvisionerChangedSubject(node, module string) string {
return "prov.changed." + node + "." + module
}
// provisionerGrants is a provisioner client's whole authority: to publish, and to subscribe.
func provisionerGrants(p Principal) (pub, sub []string, err error) {
if p.Node == "" {
// Refused rather than composed into `$KV.<bucket>..>`, a pattern no machine's keys are under, and a
// name every nameless client would share.
return nil, nil, errors.New("a provisioner client names no machine, so it would have no keys of its own " +
"to read and an account it shared with every other")
}
for _, bucket := range ProvisionerReads {
pub = append(pub, machineKeysRead(bucket, p.Node)...)
}
for _, bucket := range ProvisionerReadsWhole {
pub = append(pub, wholeBucketRead(bucket)...)
}
for _, bucket := range p.BoundBuckets {
module, name, ok := strings.Cut(bucket, "_")
if !ok || name != BoundState || !safeSubject.MatchString(module) || strings.Contains(module, "_") {
return nil, nil, fmt.Errorf("%q is not a provider's bucket of entries, <module>_%s", bucket, BoundState)
}
pub = append(pub, machineKeysRead(bucket, p.Node)...)
}
// The server's buckets (design 57 P4): its own machine's keys of the needs, the applied and the
// collections, read and written, `$KV.<bucket>.<node>.>` and nothing wider; and the registry, read whole,
// which says which domain answers which kind and is the same for every machine.
for _, bucket := range ProvisionerWrites {
pub = append(pub, machineKeysRead(bucket, p.Node)...)
pub = append(pub, "$KV."+bucket+"."+p.Node+".>")
}
pub = append(pub, wholeBucketRead(ProvisionerRegistryBucket)...)
// Its own node's own have, `<node>.node`, read and never written, and no other key of the bucket: it fills every
// module again when the node-engine moves to a new composition of the node's own (issue 1945), since an apply can
// leave a filled file as the composition had it.
pub = append(pub, readGrants(delivery.HaveBucket, p.Node+".node")...)
pub = append(pub, "prov.applied."+p.Node+".>")
token := CallerToken(p.Username())
if token == "" {
return nil, nil, fmt.Errorf("%q cannot name a caller", p.Username())
}
for _, v := range VerbsTheProvisionerAsksTheVault {
pub = append(pub, "mesh.seat."+v.Seat+"."+CallKind+"."+v.Verb+"."+token)
}
// What moved of a value it answers its own machine's tool runner (ADR 0370 §1), under its own machine's name.
pub = append(pub, "prov.changed."+p.Node+".>")
// Its own reply space, for the vault's answers and its reads' and watches' deliveries; what the vault says
// changed; and its own machine's tool runner asking it for a module's config and non-secret bound values (ADR
// 0370 §3), which it answers on the ask's reply and nowhere else.
return pub, []string{p.inbox(), VaultSecretChangedSubject, ProvisionerAskSubject(p.Node)}, nil
}
// machineKeysRead is reading every key of a bucket under one machine's name, `<node>.>`, and no other key: a
// direct get of a key's own subject, and a consumer whose filter is under it — the server's create-with-filter
// form, which it holds to the filter the subject names (as stateGrants does for one key, novox/hq ADR 0260).
// Binding, deleting a consumer and flow control cannot be narrowed by key and read no value.
func machineKeysRead(bucket, node string) []string {
stream := "KV_" + bucket
keys := "$KV." + bucket + "." + node + ".>"
return []string{
"$JS.API.STREAM.INFO." + stream,
"$JS.API.DIRECT.GET." + stream + "." + keys,
"$JS.API.CONSUMER.CREATE." + stream + ".*." + keys,
"$JS.API.CONSUMER.DELETE." + stream + ".>",
"$JS.FC." + stream + ".>",
}
}
// wholeBucketRead is reading every key of a bucket and writing none: what stateGrants grants a module for a
// bucket it `reads`.
func wholeBucketRead(bucket string) []string {
stream := "KV_" + bucket
return []string{
"$JS.API.STREAM.INFO." + stream,
"$JS.API.DIRECT.GET." + stream + ".>",
"$JS.API.CONSUMER.CREATE." + stream + ".>",
"$JS.API.CONSUMER.DELETE." + stream + ".>",
"$JS.FC." + stream + ".>",
}
}
// LastProvApplied is what a machine's provisioner client last said of one module's files, as it published it on
// ProvisionerAppliedSubject and ProvAppliedStream keeps it (design 57 P6): false when it has said nothing of it in
// the stream's week.
func LastProvApplied(js nats.JetStreamContext, node, module string) ([]byte, bool, error) {
msg, err := js.GetLastMsg(ProvAppliedStream, ProvisionerAppliedSubject(node, module))
if errors.Is(err, nats.ErrMsgNotFound) {
return nil, false, nil
}
if err != nil {
return nil, false, fmt.Errorf("what %s's provisioner client said of %s cannot be read: %w", node, module, err)
}
return msg.Data, true, nil
}
// ProvAppliedReports is the newest word of every machine's provisioner client on every module it said one of, kept by
// ProvAppliedStream, by subject (`prov.applied.<node>.<module>`).
func ProvAppliedReports(js nats.JetStreamContext) (map[string][]byte, error) {
info, err := js.StreamInfo(ProvAppliedStream, &nats.StreamInfoRequest{SubjectsFilter: ProvAppliedSubjects})
if err != nil {
return nil, fmt.Errorf("the provisioner clients' reports cannot be listed: %w", err)
}
out := map[string][]byte{}
for subject := range info.State.Subjects {
msg, err := js.GetLastMsg(ProvAppliedStream, subject)
if errors.Is(err, nats.ErrMsgNotFound) {
continue // aged out between the two reads
}
if err != nil {
return nil, fmt.Errorf("the provisioner client's report on %s cannot be read: %w", subject, err)
}
out[subject] = msg.Data
}
return out, nil
}
+73
View File
@@ -0,0 +1,73 @@
package buscompose
import "fmt"
// Bringing the bus's own objects into being, in the one order that works.
//
// **Asserted on every start rather than created once at genesis.** A stream somebody deleted, a mesh
// raised from a restored backup, or a bus whose data directory was replaced all have records and no
// objects — and a node whose consumer is missing hears nothing while everything else about it looks
// correct. Idempotence is the whole requirement, and the parts are already idempotent; this is the
// order they have to be asked in.
// Raiser is everything asserting the bus's objects needs of a connection to it.
type Raiser interface {
Asserter
Ensurer
}
// Raise asserts the mesh's streams, the controller's own consumers, and one consumer per node.
//
// **The order is not a preference.** A consumer on a stream that does not exist is refused, and the
// refusal names the stream rather than the order — so somebody reading it goes looking for a deleted
// stream instead of a reversed pair of lines. Nodes last, because the one a node reads lives on a
// stream the mesh's own set defines.
func Raise(r Raiser, nodes []string) error {
if err := AssertMeshStreams(r); err != nil {
return err
}
if err := AssertMeshConsumers(r); err != nil {
return err
}
if err := AssertNodeConsumers(r, nodes); err != nil {
return err
}
return nil
}
// RaiseSeats asserts one work queue per declared seat, and the worker of whoever holds it.
//
// Separate from Raise because it is answered by a different question: the mesh's own objects exist
// because the mesh does, and a seat's exist because a module declaring one was registered. Kept
// beside it so the order is visible — a holder's worker needs the seat's stream, and a seat's stream
// needs nothing.
func RaiseSeats(r Raiser, seats []DeclaredSeat, holders map[string]Holder) error {
for _, s := range SeatStreams(seats) {
if err := r.EnsureStream(s); err != nil {
return fmt.Errorf("asserting the work queue for %s: %w", s.Name, err)
}
}
for _, s := range seats {
h, held := holders[s.Name]
if !held {
// **The stream exists and the consumer does not, on purpose.** Work queues until a
// holder appears, so installing the module a week after something started sending to it
// flushes the backlog instead of having lost it.
continue
}
c, needed := HolderConsumerFor(h.Node, h.Module, s)
if !needed {
continue
}
if err := r.EnsureConsumer(c); err != nil {
return fmt.Errorf("asserting how %s on %s works %s: %w", h.Module, h.Node, s.Name, err)
}
}
return nil
}
// Holder is which module on which machine holds a seat.
type Holder struct {
Node string
Module string
}
+139
View File
@@ -0,0 +1,139 @@
package buscompose
import (
"fmt"
"sort"
"strings"
)
// Whether a mesh could move its bus, and what is missing if not.
//
// **Asked before anything moves, and answerable from records alone.** The rollout moves every node at
// once (novox/hq ADR 0116 step 5), so there is no partial state to inspect afterwards and no half to
// roll back: either the mesh was ready or it was not. That makes a readiness question the most
// valuable thing here — it costs nothing, it can be asked of a running mesh any number of times, and
// every answer is a thing somebody can go and fix.
//
// Deliberately pure. It is handed what the mesh knows and returns sentences; nothing here connects to
// anything, so it can be asked on a workstation about a mesh it has never reached.
// Readiness is what the mesh knows about its own ability to move.
type Readiness struct {
// TheBus is the address the mesh's own traffic would move to, empty when nothing names one.
TheBus string
// ServerStanding is whether a bus is reachable at that address, as somebody checked.
ServerStanding bool
// Holder is the node running the module that holds the bus seat, empty when nothing does.
Holder string
// AccountsComposed is whether that node has been sent the composed user list.
AccountsComposed bool
// Nodes is every machine the mesh knows.
Nodes []string
// Credentialled is which of them has a credential for the new bus.
Credentialled map[string]bool
// Modules is every assigned module, as `<node>/<module>`.
Modules []string
// ModuleCredentialled is which of those has one.
ModuleCredentialled map[string]bool
}
// NotReady is every reason this mesh cannot move its bus yet, in the order somebody would fix them.
//
// Empty means ready. **Each entry names one thing and what to do about it**, because a readiness check
// that says "not ready" is a check nobody can act on — and this is read at the point where the next
// step is irreversible.
func NotReady(r Readiness) []string {
var why []string
if strings.TrimSpace(r.TheBus) == "" {
why = append(why, "nothing names the bus to move to: set "+NATSVar+" on the control node "+
"to the address the new server answers on")
}
if !r.ServerStanding {
why = append(why, "no bus is answering at that address. Step 2 of the change raises it beside "+
"the one the mesh is on, carrying nothing — assign the module that holds "+
"mesh-broker and push the machine that runs it")
}
if r.Holder == "" {
why = append(why, "no machine holds mesh-broker, so nothing would compose the bus's user "+
"list. Assign the module that claims it")
} else if !r.AccountsComposed {
why = append(why, fmt.Sprintf(
"%s holds mesh-broker and has not been sent the composed user list, so the bus would "+
"refuse every connection. `push %s`", r.Holder, r.Holder))
}
// A node with no credential cannot come back after the move, and a node that cannot come back is
// a machine the mesh has lost until somebody goes to it.
var missing []string
for _, n := range r.Nodes {
if !r.Credentialled[n] {
missing = append(missing, n)
}
}
sort.Strings(missing)
if len(missing) > 0 {
why = append(why, fmt.Sprintf(
"%d machine(s) have no credential for the new bus and would not come back: %s. Each needs "+
"one minted before the move, not after — after, there is no bus to ask over",
len(missing), strings.Join(missing, ", ")))
}
// A module without one keeps running and stops being reachable, which is a smaller fault and still
// one somebody should choose rather than discover.
var quiet []string
for _, m := range r.Modules {
if !r.ModuleCredentialled[m] {
quiet = append(quiet, m)
}
}
sort.Strings(quiet)
if len(quiet) > 0 {
why = append(why, fmt.Sprintf(
"%d module(s) have no credential for the new bus: %s. Each keeps serving and stops "+
"answering tools and hearing events until it is issued one",
len(quiet), strings.Join(quiet, ", ")))
}
return why
}
// WhatMoves is what the rollout would do, in order, for somebody reading before they commit.
//
// **Written out rather than summarised.** This is the one step with nothing to inspect afterwards, so
// the last useful moment to disagree with it is while reading this.
func WhatMoves(r Readiness) []string {
out := []string{
fmt.Sprintf("compose the bus's user list and send it to %s", holderOr(r.Holder)),
fmt.Sprintf("move this control plane to %s, and confirm it is heard", busOr(r.TheBus)),
}
nodes := append([]string(nil), r.Nodes...)
sort.Strings(nodes)
for _, n := range nodes {
out = append(out, fmt.Sprintf("move %s, and confirm it reports", n))
}
if len(r.Modules) > 0 {
out = append(out, fmt.Sprintf("move %d module runtime(s), and confirm each answers",
len(r.Modules)))
}
// **The old broker goes, and it goes last** (novox/hq ADR 0131). AMQP is not a provision, so once
// every machine reports on the new bus nothing of the mesh is left speaking to it, and its module
// is unassigned. Said as a step so nobody reads the move as leaving a second bus behind.
out = append(out, "then unassign the old broker's module: AMQP is not a provision (ADR 0131), and "+
"once every machine reports on the new bus nothing of the mesh speaks to it")
return out
}
func holderOr(node string) string {
if node == "" {
return "whichever machine holds mesh-broker"
}
return node
}
func busOr(address string) string {
if address == "" {
return "the new bus"
}
return address
}
+38
View File
@@ -0,0 +1,38 @@
package buscompose
import (
"git.novox.be/novox/mesh-sdk/go/membership"
"git.novox.be/novox/mesh-sdk/go/process"
)
// The membership the mesh issues an assignment (novox/hq ADR 0160), its subject, its stream, the bare address and the
// pin are mesh-sdk's, public so that a module's program outside this Go module, the route proxy, reads them too
// (novox/hq ADR 0480 clause 4, issue 1633). This package keeps no second copy: what follows only names them where the
// controller already did.
type (
Membership = membership.Membership
Served = membership.Served
SeatServed = membership.SeatServed
StateIssued = membership.StateIssued
LogIssued = membership.LogIssued
Worker = membership.Worker
SeatTraffic = membership.SeatTraffic
KindHeld = membership.KindHeld
StreamIssued = membership.StreamIssued
HeardIssued = membership.HeardIssued
WaitIssued = membership.WaitIssued
)
// AssignmentsStream holds every assignment's membership, the newest per subject.
const AssignmentsStream = membership.AssignmentsStream
var (
// MembershipSubject is where an assignment's own membership is published.
MembershipSubject = membership.Subject
// CredentialIn reads the user and password out of a bus address, and the address without them.
CredentialIn = membership.CredentialIn
// BareAddress is a bus address with any credential stripped.
BareAddress = membership.BareAddress
// PinnedToFingerprint accepts exactly the certificate with this SHA-256 and no other.
PinnedToFingerprint = process.PinnedToFingerprint
)
+290
View File
@@ -0,0 +1,290 @@
package buscompose
import (
"fmt"
"sort"
)
// What a module may say and take on the seats it holds, uses and watches, beyond tools (novox/hq ADR
// 0259 §3, to-be 46 §10).
//
// Three rules are added to the ones a seat always had, each a subject whose last token names who may
// publish it, granted to that publisher alone — the way a node seat's event about a machine carries the
// machine (ADR 0219):
//
// - **A verb named by its caller** (`by-caller`). A user of the seat submits that accept, and hears that
// event, under its own module's name and no other: `accept.ask.<module>`, `event.decided.<module>`. The
// holder takes every caller's accept and says the event to any caller. So an ask's asker is a fact the
// server enforces, and a warrant reaches only the asker it is for.
// - **A kinded bench** (ADR 0234 §2). A holder claims one kind and takes its own kind's accepts, says its
// own kind's events and asks its own kind's proofs, and nothing of another kind; a user submits to any
// kind. Each kind has its own worker on the seat's queue, so a holder that is away keeps its work and
// holds up no other kind.
// - **A proof** (`proofs`): core request and reply on `mesh.seat.<seat>.proof.<verb>.<kind>`. No stream's
// subjects cover it, so what travels there — a code typed by the operator — is never persisted. A kinded
// holder asks with its own kind; the modules that watch the seat answer.
//
// And one read: **a holder's records**, a bucket the seat names, read by each user under its own name only
// (`$KV.<bucket>.<module>.>`), so an asker reads the state of its own asks and no other asker's.
// BuildAskMaxDeliver bounds the deliveries of a build seat's ask (issue 1907): mesh-sdk delivery.AskMaxDeliver, written
// here as the broker mirrors the catalogue rather than importing it.
const BuildAskMaxDeliver = 5
// KindedBenches are the seats that may be kinded (ADR 0234 §2): making another is a decision, recorded.
var KindedBenches = map[string]bool{"channel": true, "intake": true}
// WorkerName is the worker a seat's holders pull from: one for the seat, or one per kind on a kinded bench.
func WorkerName(seat, kind string) string {
if kind == "" {
return "SEAT_" + upperSnake(seat) + "_worker"
}
return "SEAT_" + upperSnake(seat) + "_" + upperSnake(kind) + "_worker"
}
func namesVerb(list []string, s string) bool {
for _, x := range list {
if x == s {
return true
}
}
return false
}
// SeatTrafficOf derives one module's seat traffic from the seats it holds, uses and watches. Only seats
// carrying one of the rules above are read: every other seat is composed as it always was.
func SeatTrafficOf(module string, holds, uses, watches []Seat) SeatTraffic {
var t SeatTraffic
for _, s := range holds {
if !s.isNewTraffic() {
continue
}
kind := ""
if s.Kinded {
kind = s.Kind
if kind == "" || !safeSubject.MatchString(kind) {
// A kinded claim without a usable kind is refused at registration; here it is granted
// nothing, which is the same answer at the last place it could be asked.
continue
}
}
if len(s.Accepts) > 0 && s.PulledByKind {
// One worker per accepted kind, which every holder pulls in the order of Accepts (ADR 0525 §4, ADR 0536 §4):
// `SEAT_<SEAT>_<KIND>_worker`, filtered to `accept.<kind>` (mesh-sdk delivery.AskWorker).
for _, a := range s.Accepts {
t.Workers = append(t.Workers, Worker{Stream: seatStreamName(s.Name), Consumer: WorkerName(s.Name, a),
Filter: "mesh.seat." + s.Name + ".accept." + a})
}
} else if len(s.Accepts) > 0 {
stream := seatStreamName(s.Name)
filter := "mesh.seat." + s.Name + ".accept.>"
if kind != "" {
filter = "mesh.seat." + s.Name + ".accept.*." + kind
}
t.Workers = append(t.Workers, Worker{Stream: stream, Consumer: WorkerName(s.Name, kind), Filter: filter})
}
for _, a := range s.Accepts {
switch {
case kind != "":
t.Subscribe = append(t.Subscribe, seatSubject(s, "accept", a+"."+kind))
case namesVerb(s.ByCaller, a):
t.Subscribe = append(t.Subscribe, seatSubject(s, "accept", a+".*"))
}
}
for _, e := range s.Emits {
switch {
case kind != "":
t.Publish = append(t.Publish, seatSubject(s, "event", e+"."+kind))
case namesVerb(s.ByCaller, e):
t.Publish = append(t.Publish, seatSubject(s, "event", e+".*"))
}
}
if kind != "" {
for _, v := range s.Proofs {
t.Publish = append(t.Publish, seatSubject(s, "proof", v+"."+kind))
}
}
}
for _, s := range uses {
if !s.isNewTraffic() {
continue
}
for _, a := range s.Accepts {
switch {
case namesVerb(s.ByCaller, a):
t.Publish = append(t.Publish, seatSubject(s, "accept", a+"."+module))
case s.Kinded && module == s.DeclaredBy:
// Work for a kind is put on its queue by the bench's own router, and by no other user.
t.Publish = append(t.Publish, seatSubject(s, "accept", a+".*"))
}
}
for _, e := range s.Emits {
if namesVerb(s.ByCaller, e) {
t.Subscribe = append(t.Subscribe, seatSubject(s, "event", e+"."+module))
}
}
for _, b := range s.Records {
if !safeSubject.MatchString(b) {
continue
}
t.Records = append(t.Records, "$JS.API.DIRECT.GET.KV_"+b+".$KV."+b+"."+module+".>")
}
}
for _, w := range watches {
if w.Kinded {
for _, e := range w.Emits {
t.Subscribe = append(t.Subscribe, seatSubject(w, "event", e+".*"))
}
if module == w.DeclaredBy {
// A code is answered by the bench's own router, and by no other watcher.
for _, v := range w.Proofs {
t.Answers = append(t.Answers, seatSubject(w, "proof", v+".*"))
}
}
}
}
t.Publish = unique(t.Publish)
t.Subscribe = unique(t.Subscribe)
t.Answers = unique(t.Answers)
t.Records = unique(t.Records)
sort.Slice(t.Workers, func(i, j int) bool { return t.Workers[i].Consumer < t.Workers[j].Consumer })
return t
}
// seatTrafficGrants is the bus permissions seat traffic needs: the subjects themselves, and the JetStream API a
// worker is pulled and acknowledged through and a record is read through.
func seatTrafficGrants(t SeatTraffic) (pub, sub []string) {
pub = append(pub, t.Publish...)
sub = append(sub, t.Subscribe...)
sub = append(sub, t.Answers...)
for _, w := range t.Workers {
pub = append(pub,
"$JS.API.CONSUMER.INFO."+w.Stream+"."+w.Consumer,
"$JS.API.CONSUMER.MSG.NEXT."+w.Stream+"."+w.Consumer,
"$JS.ACK."+w.Stream+"."+w.Consumer+".>")
}
pub = append(pub, t.Records...)
return pub, sub
}
// isNewTraffic says whether a seat carries any of the rules above, so a seat that carries none is
// composed exactly as before them.
func (s Seat) isNewTraffic() bool {
return s.Kinded || len(s.ByCaller) > 0 || len(s.Proofs) > 0 || len(s.Records) > 0 || s.PulledByKind
}
// plainVerbs is a seat's accepts or emits with those the rules above compose taken out: a verb named by
// its caller and every verb of a kinded bench are composed by SeatTrafficOf and nowhere else.
func plainVerbs(s Seat, verbs []string) []string {
if s.Kinded {
return nil
}
var out []string
for _, v := range verbs {
if !namesVerb(s.ByCaller, v) {
out = append(out, v)
}
}
return out
}
// SeatTrafficObjects is the work queues and workers the seat traffic of every composed user implies
// (novox/hq ADR 0259 §3): a queue for each seat a holder takes work from, and each holder's worker on it —
// one per kind on a kinded bench, filtered to that kind, so the kinds never take each other's work. Only
// seats carrying the rules above; the mesh's own seats' queues are RaiseSeats'.
func SeatTrafficObjects(users []Principal) ([]Stream, []Consumer) {
streams := map[string]Stream{}
consumers := map[string]Consumer{}
add := func(module string, holds []Seat) {
for _, w := range SeatTrafficOf(module, holds, nil, nil).Workers {
seat, byKind := "", false
for _, s := range holds {
if seatStreamName(s.Name) == w.Stream {
seat, byKind = s.Name, s.PulledByKind
}
}
streams[w.Stream] = Stream{
Name: w.Stream,
Subjects: []string{"mesh.seat." + seat + ".accept.>"},
Retention: RetentionWorkQueue,
MaxAge: 7 * 24 * 60 * 60,
Why: "work submitted to the " + seat + " seat; its holders take it, each kind its own, and it queues while nobody does",
}
c := Consumer{
Name: w.Consumer,
Stream: w.Stream,
Filters: []string{w.Filter},
AckWaitSeconds: 60,
// No bound on redelivery: a channel away for a day keeps its work, offered again later and
// later by its holder's runtime (novox/hq ADR 0259; the correctness review of 2026-10-08).
MaxDeliver: 0,
Why: module + " holds " + seat + "; it pulls one ask at a time and acknowledges once it has " +
"recorded it, so a crash redelivers rather than loses",
}
if byKind {
// A build seat's ask is offered a bounded number of times (issue 1907): the holder that takes it past
// mesh-sdk delivery.AskRuns ends it failed saying it is poison, and the bound stops a run that kills
// every node it reaches even when that holder dies too.
c.MaxDeliver = BuildAskMaxDeliver
c.Why = module + " holds " + seat + "; each holder pulls an ask when it has a free slot, and an ask that " +
"no run ends is offered at most " + fmt.Sprint(BuildAskMaxDeliver) + " times, its step failed as poison"
}
consumers[w.Consumer] = c
}
}
for _, p := range users {
switch p.Kind {
case KindModule:
add(p.Module, p.Holds)
case KindNodeTools:
for _, d := range p.Carries {
add(d.Module, d.Holds)
}
}
}
var ss []Stream
for _, s := range streams {
ss = append(ss, s)
}
sort.Slice(ss, func(i, j int) bool { return ss[i].Name < ss[j].Name })
var cs []Consumer
for _, c := range consumers {
cs = append(cs, c)
}
sort.Slice(cs, func(i, j int) bool { return cs[i].Name < cs[j].Name })
return ss, cs
}
// trustedTraffic is why a module's seat traffic is the trusted holder's (novox/hq ADR 0259 §8), or "": it
// says a seat's event to one caller each (a warrant), or holds a kind of a kinded bench that proves its sender.
func trustedTraffic(d Declared) string {
for _, s := range d.Holds {
for _, e := range s.Emits {
if namesVerb(s.ByCaller, e) {
return "it says " + s.Name + "'s " + e + " to one caller each"
}
}
if s.Kinded && namesVerb(s.Capabilities, "verified-sender") {
return "it holds " + s.Name + " of kind " + s.Kind + ", which proves its sender"
}
}
return ""
}
// TrafficQueues is the work queue of every seat naming its caller or its kind that accepts work, held or not
// (novox/hq ADR 0259 §3): what is submitted before a holder is assigned waits for it.
func TrafficQueues(seats []Seat) []Stream {
var out []Stream
seen := map[string]bool{}
for _, s := range seats {
if len(s.Accepts) == 0 || !s.isNewTraffic() || seen[s.Name] {
continue
}
seen[s.Name] = true
out = append(out, Stream{Name: seatStreamName(s.Name), Subjects: []string{"mesh.seat." + s.Name + ".accept.>"},
Retention: RetentionWorkQueue, MaxAge: 7 * 24 * 60 * 60,
Why: "work submitted to the " + s.Name + " seat; its holders take it, each kind its own, and it queues while nobody does"})
}
sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name })
return out
}
+265
View File
@@ -0,0 +1,265 @@
package buscompose
import (
"fmt"
"git.novox.be/novox/mesh-sdk/go/announce"
"sort"
"strings"
"time"
)
// A module's state on the bus (novox/hq ADR 0201, design 32 §4, design 25 §3).
//
// A module names the state it keeps (`state`) and the state of others it reads (`reads`), and each
// is a key-value bucket: the server's own last-per-subject stream with direct reads, delete markers
// and watches, which is the state relationship the mesh already uses for declarations, opened to
// modules. The controller creates every bucket from the catalogue — from registration, like a
// seat's stream, so a reader may watch before the owner runs anywhere — and no module can.
//
// Pure, like everything else in this package that decides what the bus holds; jetstream.go is the
// part that asks a server.
// The mesh's caps on a bucket, the same for every module: a value is a piece of state, not a file,
// and a bucket that grew without bound would be one module filling the bus's disk for everyone.
const (
StateMaxValueBytes = 256 * 1024
StateMaxBytes = 64 * 1024 * 1024
)
// A Bucket is one module's declared state as the bus holds it.
type Bucket struct {
Module string
Name string
// History is how many values a key keeps; zero is one.
History int
// TTLSeconds is how long a value lives; zero is until replaced or deleted.
TTLSeconds int
// PerMachine says each machine's instance writes only the key named for its machine (novox/hq ADR
// 0260), and is granted that key and no other.
PerMachine bool
// KeyTTL says the bucket allows a time to live per key (novox/hq ADR 0433 §1): its owner writes each key with its
// own, and the server writes a delete marker, kept KeyTTLMarkerTTL, when it ends.
KeyTTL bool
}
// KeyTTLMarkerTTL is how long the marker an expired key leaves is kept: long enough for a watcher to hear it.
const KeyTTLMarkerTTL = time.Minute
// BucketName is the bucket a module's state lives in: the module and the local name joined by an
// underscore, which neither may contain, so two modules can never derive one bucket.
func BucketName(module, name string) string { return module + "_" + name }
// Bucket is this bucket's name on the bus.
func (b Bucket) Bucket() string { return BucketName(b.Module, b.Name) }
// Why is carried into the server's description of the bucket, so somebody reading the server's
// own state finds whose it is and why it is kept.
func (b Bucket) Why() string {
return fmt.Sprintf("%s's state %q (novox/hq ADR 0201): its current value per key, written by %s, "+
"read by whatever declares it reads it; kept when %s is unassigned, because it is data",
b.Module, b.Name, b.Module, b.Module)
}
// bucketOfRead is the bucket a read names, `<module>.<name>`, or false when it names none.
func bucketOfRead(read string) (string, bool) {
at := strings.LastIndex(read, ".")
if at <= 0 || at == len(read)-1 {
return "", false
}
module, name := read[:at], read[at+1:]
if !safeSubject.MatchString(module) || !safeSubject.MatchString(name) {
return "", false
}
return BucketName(module, name), true
}
// readGrants is what reaching a bucket's keys needs: binding to it, a direct get, an ordered consumer for listing and
// watching and the deletion of that consumer and its flow control. key is one key's subject tail, a pattern such as
// `one.>` for a prefix of keys, or empty for every key. The bucket is made with AllowDirect, so a get is the direct get
// and never the stream's message get.
func readGrants(bucket, key string) []string {
stream := "KV_" + bucket
out := []string{"$JS.API.STREAM.INFO." + stream}
if key == "" {
out = append(out,
"$JS.API.DIRECT.GET."+stream+".>",
"$JS.API.CONSUMER.CREATE."+stream+".>")
} else {
out = append(out,
"$JS.API.DIRECT.GET."+stream+".$KV."+bucket+"."+key,
"$JS.API.CONSUMER.CREATE."+stream+".*.$KV."+bucket+"."+key)
}
return append(out,
"$JS.API.CONSUMER.DELETE."+stream+".>",
"$JS.FC."+stream+".>")
}
// NodeKeysGrants is what a node's node-engine is granted of a bucket it writes by node, `<node>.>` and no other key
// (novox/hq ADR 0429 §1: each key one writer, enforced by credentials): the writes of those keys and what a get,
// create, update and watch of them need. The node name is checked as every subject token is.
func NodeKeysGrants(bucket, node string) ([]string, error) {
if !safeSubject.MatchString(node) {
return nil, fmt.Errorf("%q cannot be part of a subject: a permission is a subject pattern, and this would widen it", node)
}
return append([]string{"$KV." + bucket + "." + node + ".>"}, readGrants(bucket, node+".>")...), nil
}
// stateGrants is what a principal publishes to reach the state its modules keep and read: for every
// bucket, binding to it, reading a key directly, and an ordered consumer for listing and watching,
// created and deleted on the bucket's own stream, with its flow control answered; for a bucket an
// owner keeps, writing under the bucket's own subjects too.
//
// **Measured against a running server, 2026-10-04** (novox/hq research 024), and each one is there
// because leaving it out failed: without STREAM.INFO nothing binds; without DIRECT.GET nothing is
// read; without CONSUMER.CREATE no key is listed and nothing is watched; without CONSUMER.DELETE a
// watch cannot be stopped and lingers on the server. A write outside these is refused by the server
// — and reaches the writer as a timeout, not a refusal, which is why the runtime refuses first.
//
// **One key, where the state is per machine** (novox/hq ADR 0260). The owner's instance on a machine
// writes and reads the key named for that machine, and a holder granted a read for its machine reads
// that key: a direct get of the key's own subject, and a consumer whose filter is that subject — the
// server's create-with-filter form, which a client uses for a watch of one key and which the server
// holds to the filter it names. What cannot be narrowed by key: binding (STREAM.INFO), deleting a
// consumer (named at random by the client) and flow control; none of them reads or writes a value.
func stateGrants(a stateAccess) []string {
var out []string
read := func(bucket, key string) { out = append(out, readGrants(bucket, key)...) }
machineKey := ""
if safeSubject.MatchString(a.Node) {
machineKey = a.Node
}
perMachine := map[string]bool{}
for _, n := range a.PerMachine {
perMachine[n] = true
}
for _, name := range a.Keeps {
if !safeSubject.MatchString(name) {
continue
}
bucket := BucketName(a.Module, name)
if perMachine[name] {
if machineKey == "" {
continue // a machine with no usable name reaches none of it
}
read(bucket, machineKey)
out = append(out, "$KV."+bucket+"."+machineKey)
continue
}
read(bucket, "")
out = append(out, "$KV."+bucket+".>")
}
// **And it says what it keeps** (novox/hq ADR 0495 clause 2, issue 1790): a module keeping state announces it on its
// own event subject, from its code; refused, its readers are read as before.
if len(a.Keeps) > 0 && safeSubject.MatchString(a.Module) {
out = append(out, announce.Subject(a.Module))
}
for _, r := range a.Reads {
if bucket, ok := bucketOfRead(r); ok {
read(bucket, "")
}
}
for _, k := range a.KeyedReads {
if bucket, ok := bucketOfRead(k.Read); ok && safeSubject.MatchString(k.Key) {
read(bucket, k.Key)
}
}
return out
}
// KeyedRead is one key of another module's state a principal reads, and no other key of it.
type KeyedRead struct {
Read string `json:"read"`
Key string `json:"key"`
}
// stateAccess is what one principal may do with state: the module's own buckets (some per machine),
// what it reads whole, and what it reads for its machine's key alone.
type stateAccess struct {
Module, Node string
Keeps, PerMachine []string
Reads []string
KeyedReads []KeyedRead
}
// perMachineNames is the local names of a module's buckets keyed by machine.
func perMachineNames(buckets []Bucket) []string {
var out []string
for _, b := range buckets {
if b.PerMachine {
out = append(out, b.Name)
}
}
return out
}
// stateIssuedFor is every bucket a module's code may reach on a machine, as its membership lists them.
func stateIssuedFor(d Declared, node string) []StateIssued {
var out []StateIssued
for _, b := range d.State {
issued := StateIssued{Name: b.Name, Bucket: BucketName(d.Module, b.Name), Writes: true}
if b.PerMachine {
issued.Key = node
}
out = append(out, issued)
}
for _, r := range d.Reads {
if bucket, ok := bucketOfRead(r); ok {
out = append(out, StateIssued{Name: r, Bucket: bucket})
}
}
for _, k := range d.KeyedReads {
if bucket, ok := bucketOfRead(k.Read); ok {
out = append(out, StateIssued{Name: k.Read, Bucket: bucket, Key: k.Key})
}
}
return out
}
// stateNames is the local names of a module's own buckets.
func stateNames(buckets []Bucket) []string {
out := make([]string, 0, len(buckets))
for _, b := range buckets {
out = append(out, b.Name)
}
return out
}
// A BucketAsserter is the part of a JetStream connection bucket assertion needs.
type BucketAsserter interface {
// EnsureBucket creates the bucket if absent and brings its options to match if present, never
// discarding what it holds.
EnsureBucket(b Bucket) error
// BucketNames is every key-value bucket on the server.
BucketNames() ([]string, error)
}
// RaiseBuckets asserts every declared bucket and answers the buckets on the server that nothing
// declares any more.
//
// **Those are reported, never removed** (novox/hq ADR 0201, ADR 0030): what a module stored is
// data, and a manifest edited, a module renamed or a catalogue entry dropped is an ordinary day's
// work that must not take data with it. Removing one is a person's act.
func RaiseBuckets(a BucketAsserter, buckets []Bucket) (undeclared []string, err error) {
sorted := append([]Bucket(nil), buckets...)
sort.Slice(sorted, func(i, j int) bool { return sorted[i].Bucket() < sorted[j].Bucket() })
declared := map[string]bool{}
for _, b := range sorted {
if err := a.EnsureBucket(b); err != nil {
return nil, fmt.Errorf("asserting %s's state %q: %w", b.Module, b.Name, err)
}
declared[b.Bucket()] = true
}
names, err := a.BucketNames()
if err != nil {
return nil, fmt.Errorf("listing the bus's state: %w", err)
}
for _, n := range names {
// A seat's cancelled set is the mesh's own (novox/hq ADR 0219), not a module's state; so are
// the controller's own buckets (novox/hq to-be 45 §1).
if !declared[n] && !IsCancelledSet(n) && !IsControllerBucket(n) {
undeclared = append(undeclared, n)
}
}
sort.Strings(undeclared)
return undeclared, nil
}
+612
View File
@@ -0,0 +1,612 @@
package buscompose
import (
"fmt"
"sort"
"strings"
)
// The mesh's own streams.
//
// **These and no more** (novox/hq ADR 0116 task 1.4, as revised by ADR 0118; the two that keep what a
// consumer gave up on added for issue 330). An earlier
// reading had the controller create *every* stream at genesis, from a fixed set. That is only the
// mesh's own half: a seat's streams are created when the module declaring it is registered, and a
// module's durable consumers when it is assigned — neither of which has happened at genesis. What
// is here is the foundation, which exists before any module does.
//
// The controller is the only writer of stream definitions (design 25 §3). A module declares
// nothing about them and cannot reach the JetStream API to make one.
// Retention is how a stream decides what to keep, which is the whole of what distinguishes the
// mesh's four relationships on the wire (design 29 §4).
type Retention string
const (
// RetentionWorkQueue: a message is removed once a consumer acknowledges it. Exactly one
// worker does the work, and a worker that dies has its message redelivered.
RetentionWorkQueue Retention = "workqueue"
// RetentionLastPerSubject: only the newest message on each subject survives. This is the
// state shape — a node that was away gets exactly the current declaration and nothing older.
RetentionLastPerSubject Retention = "last_per_subject"
// RetentionLimits: kept until it ages or the stream fills. Events, where a subscriber that
// was down catches up and nobody is obliged to act.
RetentionLimits Retention = "limits"
)
// A Stream is one of the mesh's own, as the controller asserts it.
type Stream struct {
Name string
Subjects []string
Retention Retention
// MaxAge in seconds, zero for unbounded. Per stream — JetStream has no per-subject age,
// which is why differing retention between modules would mean a stream each.
MaxAge int
// MaxMsgsPerSubject caps each subject independently, so one noisy emitter cannot push
// another's events out of a shared stream. Verified: with a cap of 3, ten messages on one
// subject and one on another leave four in the stream, not three.
MaxMsgsPerSubject int
// Why is carried into the assertion so an operator reading the server's own state finds the
// reason there, rather than only in a repository they may not have.
Why string
// Direct lets a client read a subject's last message without a consumer, which is how a
// runtime reads its own membership with no JetStream API beyond one request (ADR 0160).
Direct bool
// MaxBytes bounds the stream's size, zero for unbounded. With DiscardNew a full stream refuses
// what comes next rather than dropping what it holds: the publisher is told, and says so.
MaxBytes int64
DiscardNew bool
// DuplicatesSeconds is the window in which a message id published twice is kept once; zero for the
// server's default (two minutes).
DuplicatesSeconds int
// AllowMsgTTL lets a message carry a time to live of its own (`Nats-TTL`), for what must not outlive the
// stream it came from (novox/hq ADR 0376 §3, §5).
AllowMsgTTL bool
// Restricted marks a module's stream restricted by a grant (novox/hq ADR 0363 §4), in the stream's own
// metadata, so whoever keeps one of its messages reads it from the bus (ADR 0376 §3).
Restricted bool
// WorkQueue marks a module stream declared retention: work-queue (novox/hq ADR 0436 §6), in the
// stream's own metadata, so a dead letter kept from it is read as drop-only from the bus, by the
// declaration and not by the bus's work-queue retention alone — which SEAT_* and other mesh-internal
// streams share without being one of ADR 0436's, and whose dead letters are still delivered again
// (ADR 0264 §4).
WorkQueue bool
}
// RestrictedMetadata is the key a restricted stream's metadata says it under.
const RestrictedMetadata = "mesh-restricted"
// WorkQueueMetadata is the key a module stream declared retention: work-queue (novox/hq ADR 0436 §6) says
// it under: never set by a mesh-internal work-queue stream (a seat's, a check's), only by Stream.WorkQueue.
const WorkQueueMetadata = "mesh-work-queue"
// What a durable consumer gave up on is kept (novox/hq issue 330, design 25 §3).
//
// **The server says it and keeps it; the controller copies it.** A consumer that handed a message over
// as often as it may stops offering it and publishes `$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES` with
// the stream and the message's sequence. DeadLetterNoticesStream captures those advisories as the
// server publishes them, so one said while no controller listens is still there when one starts. The
// controller's consumer on it fetches the given-up message by its sequence, while the source stream
// still holds it, and keeps a copy in DeadLettersStream under DeadLetterSubject, with the consumer,
// the subject, how often it was handed over and when it was given up. It stays there until a person
// delivers it again or drops it, with why; a condition is open for as long as it does.
const (
DeadLetterNoticesStream = "DEAD_LETTER_NOTICES"
DeadLettersStream = "DEAD_LETTERS"
// MaxDeliveriesAdvisories is the subject the server says a given-up message on.
MaxDeliveriesAdvisories = "$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES.>"
deadLetterPrefix = "mesh.events.dead."
againPrefix = "mesh.again."
// DeadLettersBytes bounds the kept copies. Full, the stream refuses the next copy, which is said
// as the consumer's condition; it never drops one it holds.
DeadLettersBytes = 256 << 20
)
// DeadLetterSubject is where a message one consumer gave up on is kept: `mesh.events.dead.<stream>.<consumer>`.
func DeadLetterSubject(stream, consumer string) string {
return deadLetterPrefix + stream + "." + consumer
}
// DeadLetterOf is the stream and consumer a kept message's subject names; false for any other subject.
func DeadLetterOf(subject string) (stream, consumer string, ok bool) {
rest, found := strings.CutPrefix(subject, deadLetterPrefix)
if !found {
return "", "", false
}
stream, consumer, ok = strings.Cut(rest, ".")
return stream, consumer, ok && stream != "" && consumer != "" && !strings.Contains(consumer, ".")
}
// AgainSubject is where an event given up on is delivered again to the one consumer that gave it up,
// and to nobody else: `mesh.again.<consumer>.` and the original subject without its `mesh.`. Every
// consumer on EVENTS filters its own (AgainFilter), and a module's runtime reads the event's key from
// the tokens around `.event.`, so the handler sees the same key it saw the first time.
func AgainSubject(consumer, original string) string {
return againPrefix + consumer + "." + strings.TrimPrefix(original, "mesh.")
}
// AgainFilter is the one consumer's share of the subjects events are delivered again on.
func AgainFilter(consumer string) string { return againPrefix + consumer + ".>" }
// OriginalOfAgain is the subject an event delivered again was first published on; false for a subject
// that is not one delivered again.
func OriginalOfAgain(subject string) (string, bool) {
rest, found := strings.CutPrefix(subject, againPrefix)
if !found {
return "", false
}
_, original, ok := strings.Cut(rest, ".")
if !ok || original == "" {
return "", false
}
return "mesh." + original, true
}
// MeshStreams is the foundation set, in the order a person reads it.
//
// **CONTROL names its subjects rather than taking `mesh.control.>`**, because heartbeats live
// under that prefix and must not be persisted: a lost heartbeat is the next heartbeat, and a
// stream of them is a stream of the least valuable messages the mesh sends, competing for the
// same retention as the ones that matter.
//
// **EVENTS filters on the `event` token**, which is the reason that token exists. A module's
// namespace carries both its events and its tool calls; a filter of `mesh.mod.*.>` would persist
// every tool invocation in the mesh, and a tool call must never be persisted (design 25 §3 keeps
// tools on core NATS, where a lost call is a timeout the caller already handles).
func MeshStreams() []Stream {
return []Stream{
{
Name: "CONTROL",
// A build's outcome is no longer here: it is the build-machine seat's own event, so one
// publish reaches whoever asked, the controller and the catalogue (novox/hq ADR 0121).
Subjects: []string{"mesh.control.*.report", "mesh.control.enrol"},
Retention: RetentionWorkQueue,
Why: "the store-window guarantee (ADR 0083): the controller naks with a delay while its " +
"store is away and the message is redelivered; nothing is dropped",
},
{
Name: "NODES",
Subjects: []string{"mesh.node.*.declare"},
Retention: RetentionLastPerSubject,
Why: "one declaration per node, always the newest; a node that sees sequence n refuses " +
"n-1 by construction (issue 107)",
},
{
Name: AssignmentsStream,
Subjects: []string{"mesh.assignment.*.*"},
Retention: RetentionLastPerSubject,
Direct: true,
Why: "one membership per assignment, always the newest: what the mesh issued this module " +
"on this machine to serve and to reach (ADR 0160); read directly by the runtime it is for",
},
{
Name: EventsStream,
// A seat's own events ride here too: they are 1:many like any event, and the
// `event` token keeps them clear of both the seat's work queue (`accept`) and its
// tools (`tool`), which must not be persisted.
// And an event given up on, delivered again to the one consumer that gave it up
// (novox/hq issue 330): under `mesh.again.<consumer>.`, which only that consumer filters.
Subjects: []string{"mesh.mod.*.event.>", "mesh.seat.*.event.>", againPrefix + ">"},
Retention: RetentionLimits,
MaxAge: 7 * 24 * 60 * 60,
MaxMsgsPerSubject: 10000,
// A restricted dead letter delivered again keeps what is left of its stream's age (ADR 0376 §5).
AllowMsgTTL: true,
Why: "a subscriber that was down catches up; tool traffic under the same prefix is " +
"excluded by the event token; per-subject caps keep a noisy emitter from " +
"evicting a quiet one without splitting the stream",
},
{
Name: DeadLetterNoticesStream,
Subjects: []string{MaxDeliveriesAdvisories},
Retention: RetentionWorkQueue,
MaxAge: 7 * 24 * 60 * 60,
Why: "the server's word that a consumer gave up on a message, kept until the controller has " +
"copied the message into DEAD_LETTERS (novox/hq issue 330); a week, the longest the source " +
"streams keep what they are about",
},
{
Name: DeadLettersStream,
Subjects: []string{deadLetterPrefix + ">"},
Retention: RetentionLimits,
MaxBytes: DeadLettersBytes,
DiscardNew: true,
DuplicatesSeconds: 24 * 60 * 60,
AllowMsgTTL: true,
Why: "every message a consumer gave up on, with its consumer, subject, deliveries and when, kept " +
"until a person delivers it again or drops it with why (novox/hq issue 330); no age but a restricted " +
"stream's message, which ages out with its stream (ADR 0376), and full it refuses the next copy " +
"rather than drop one it holds",
},
{
Name: ProvAppliedStream,
Subjects: []string{ProvAppliedSubjects},
Retention: RetentionLastPerSubject,
Direct: true,
MaxAge: 7 * 24 * 60 * 60,
Why: "each machine's provisioner client's newest word on each module's files, applied, waiting or " +
"failed with the reason (novox/hq ADR 0349 §4): what a walk's end and a gate wait for (ADR 0362, " +
"ADR 0349 §15), read by subject; a week, as the events are",
},
}
}
// ProvAppliedStream keeps what each machine's provisioner client last said of each module's files, newest per
// subject (novox/hq ADR 0349 §4, ADR 0362, design 57 P6): `prov.applied.<node>.<module>` lies outside `mesh.*`, so
// no other stream captures it, and a report said while no controller listened is still there to be read.
const ProvAppliedStream = "PROV_APPLIED"
// ProvAppliedSubjects is every machine's client's reports: `prov.applied.<node>.<module>`, a module's name dots and
// all (ProvisionerAppliedSubject).
const ProvAppliedSubjects = "prov.applied.>"
// eventsTheControllerFollows are the events the controller's events consumer takes: all it follows but the forge's
// merges, which the merges consumer takes (MergesConsumerName).
func eventsTheControllerFollows() []string {
var out []string
for _, subject := range ControllerFollows {
if subject != ControllerFollows[3] {
out = append(out, subject)
}
}
return append(out, ControllerHears...)
}
// DeliverSubjectFor is where a push consumer's messages land.
//
// **Per consumer, which means per stream as well as per name** (novox/hq 04-ISSUES/146). A push
// consumer delivers onto an ordinary subject, and everything subscribed to that subject gets a
// copy. The controller holds a consumer called `controller` on CONTROL and another called
// `controller` on EVENTS; while both were given `_DELIVER.controller`, the one process holding
// both subscriptions acted on every message twice — a joining machine was enrolled twice from one
// request, and the second enrolment minted a credential that replaced the one the machine had just
// been handed. Every report and every followed event doubled the same way, silently: nothing is
// redelivered, no count is wrong, the work simply happens twice.
//
// The stream belongs in it because the pair is what identifies a consumer — the server scopes a
// durable's name to its stream, and this subject was the one place that scoping was dropped. It
// stays inside what a controller may already subscribe (`_DELIVER.controller.>`).
func DeliverSubjectFor(c Consumer) string {
if c.DeliverSubject != "" {
return c.DeliverSubject
}
return "_DELIVER." + c.Name + "." + c.Stream
}
// An Asserter is the part of a JetStream connection stream assertion needs. Narrow on purpose: it
// keeps this testable without a server, and keeps the client library out of everything that only
// wants to know what the streams are.
type Asserter interface {
// EnsureStream creates the stream if absent and updates it to match if present. It must be
// idempotent: the controller asserts on every start, not only at genesis.
EnsureStream(s Stream) error
}
// AssertMeshStreams brings the foundation set into being, in order, and says which one failed
// rather than that something did.
//
// Asserted on every start rather than created once at genesis, because a stream that was deleted,
// or a mesh raised from a restored backup, must converge rather than run without the guarantee
// its messages assume. Idempotence is the whole requirement.
func AssertMeshStreams(a Asserter) error {
for _, s := range MeshStreams() {
if err := a.EnsureStream(s); err != nil {
return fmt.Errorf("asserting stream %s: %w", s.Name, err)
}
}
return nil
}
// Overlaps reports subject filters claimed by more than one stream.
//
// **Corrected against the server**: an earlier version of this comment said NATS accepts
// overlapping streams and stores the message twice. It does not — it refuses the second stream
// with "subjects overlap with an existing stream" (verified against nats-server 2.10). The check
// still earns its place, for a different reason: the server's refusal arrives when the controller
// is applying, naming one stream, at a moment when the mesh is half-configured. This one arrives
// where the set is written, names both, and cannot reach a running mesh.
//
// It also decides a design question. Because overlap is refused rather than merged, a shared
// EVENTS stream and a per-module stream cannot coexist — the module's would be refused — so
// "one stream for most, its own for a module that wants different retention" is not an option
// the server allows. It is all of one or all of the other.
func Overlaps() []string {
seen := map[string]string{}
var clashes []string
for _, s := range MeshStreams() {
for _, subject := range s.Subjects {
if first, ok := seen[subject]; ok {
clashes = append(clashes, fmt.Sprintf("%s and %s both claim %s", first, s.Name, subject))
continue
}
seen[subject] = s.Name
}
}
sort.Strings(clashes)
return clashes
}
// The mesh's own consumers.
//
// A seat's streams and a module's consumers are derived from declarations (derived.go). These two
// are not: **the controller is not a module and files no manifest**, so its authority and its
// subscriptions cannot come from a declaration that does not exist. They are named here, where the
// mesh's own streams are named, and narrowly — a controller subscribing `mesh.mod.*.event.>` would
// hear every event in the mesh, which it has no business doing and which would make its permission
// list stop explaining anything.
// ControllerName is the controller's durable consumer on each stream it reads, and the name its
// ack subject is derived from (nats.go: `$JS.ACK.<stream>.controller.>`).
const ControllerName = "controller"
// MergesConsumerName is the controller's durable consumer of the forge's merged event on EVENTS, apart from the
// events consumer (novox/hq ADR 0407 decision 3, issue 973). A merge is never dropped, so this consumer has no
// max-deliver, which would send a merge held five times to DEAD_LETTERS to wait for a person, and no healer
// resets it, which would drop the merges it holds. A consumer of its own because both of those are properties of
// a consumer, not of a subject.
const MergesConsumerName = "controller-merges"
// ControllerSeat is the role the control plane holds, and ControllerStates are the facts it states
// under it (novox/hq ADR 0134).
//
// **Written here as well as in `link`, and a test keeps them agreeing.** `link` imports `broker`, so
// `broker` cannot import `link`; a grant naming a subject the controller never publishes is authority
// nobody uses, and a controller publishing one the grant omits is refused at the moment it has
// something to say.
const ControllerSeat = "mesh-controller"
var ControllerStates = []string{"applied", "refused", "built-before",
// What is wrong, said as it changes (novox/hq to-be 45 §2): a condition raised, changed in
// severity, resolver or silence, and cleared. The operator-channel's holder and any other surface
// consume them; the controller tells nobody itself.
"condition-raised", "condition-changed", "condition-cleared",
// And the self-check's heartbeat, at the end of every run (to-be 45 §4, S10): watched from a
// machine that is not the control node, so the controller going quiet is itself said.
"doctor-heartbeat",
// And a value given by hand, replaced after its module's first good start (novox/hq ADR 0228).
"secret-replaced",
// And every act a healer takes on a condition (novox/hq to-be 45 §7, Phase 3): a repair the mesh
// made by itself is said like one a person made, never quietly.
"healer-acted",
// And a build put back after its gate failed on its first machine (novox/hq ADR 0236, to-be 45 §8).
"rolled-back",
// And a pull request's merge check, judged (novox/hq to-be 45 §9): what the forge's holder sets as
// the pull request's status, and anybody else may read.
"checked",
// (A walk kept in a new state and a tier sent to a machine are gone with the plans, novox/hq ADR 0539.)
// And a key of a record mirror written or removed (novox/hq ADR 0289, design 54 §3): the assignments and
// each registered module's declared config (design 57 F3).
"assignment-changed", "declared-config-changed", "predefined-changed",
// And a builder's key not taken in because the artifact store does not hold what it names (novox/hq issue 1937).
"artifacts-gone",
// And a bucket of the plans' state deleted at the controller's first start, and the declarations' stream once no node
// reads one (novox/hq ADR 0539 clauses 11 and 13).
"bucket-deleted"}
// BusAdvisories are what the bus server says about the mesh's own account that the controller
// reads (novox/hq to-be 45 §3, S9): a durable consumer that handed a message over as often as it
// may and gave up on it, and one that was deleted. Read-only: an advisory is the server's to
// publish, and the controller's subscription changes nothing on the bus.
var BusAdvisories = []string{
"$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES.>",
"$JS.EVENT.ADVISORY.CONSUMER.DELETED.>",
}
// ControllerFollows are the events the controller reacts to: the catalogue saying a module's
// current version moved, and a catalogue that has just started saying it may have missed builds.
//
// **Derived the same way a module's subscription is**, from the emitter and the bare local event
// name, rather than written out. They were written out while the catalogue still spelled its events
// as the old bus's routing keys, and the moment those were converted (novox/hq 04-ISSUES/127) a
// hard-coded pair became a controller listening to a subject nothing publishes — the same fault, from
// the other side. Deriving them means the conversion could not leave these behind.
var ControllerFollows = []string{
moduleEventSubject("mesh-catalog", "upgraded"),
moduleEventSubject("mesh-catalog", "catching-up"),
// A build's outcome, which is the build-machine role's own event now (ADR 0121) rather than a
// message on the control branch. Same three audiences, one publish: whoever asked, this, and the
// catalogue.
seatEventSubject("node-build-agent", "built"),
// The forge's merges: what moved a source, so the mesh builds what that source produces
// without anybody telling it (novox/hq 04-ISSUES/131). Appended, because the index is a name.
moduleEventSubject("gitea", "pull.merged"),
// The retired build role's outcome too, while the handover runs (novox/hq ADR 0190): the one
// build machine keeps answering on its seat until build-agent replaces it, and the outcome that
// registers build-agent itself comes from there. Appended, for the same reason as above; goes
// with the retired seat row.
seatEventSubject("mesh-build-machine", "built"),
// **Every provider's standing** (novox/hq ADR 0224): a consumer it has failed for minutes, and
// that consumer recovered. The one pattern on this list, and a narrow one — two named events,
// from whichever module provides — because the rule is about every provider, and a list of
// providers here would be a list somebody forgets to extend. On 2026-10-05 the identity provider
// failed every consumer for a day and only its journal said so (issue 179). Appended, because
// the index is a name.
moduleEventSubject("*", ProvisionerFailing),
moduleEventSubject("*", ProvisionerRecovered),
// **What becomes of a consumer the mesh stopped asking for** (novox/hq ADR 0230): retired, waiting
// for a person, re-enabled, deleted — a provider's third word, from whichever module provides.
// Appended, because the index is a name.
moduleEventSubject("*", ProvisionerRetirement),
// **A pull request's head, announced** (novox/hq to-be 45 §9): what the controller asks the build
// seat to check before it merges — every machine of the facts snapshot composed with the change.
// Appended, because the index is a name.
moduleEventSubject("gitea", "pull.updated"),
// **The operator's answers to what the controller asked** (novox/hq ADR 0259): the router's warrant, or
// the end of an ask without one, said to the controller alone under its own name. On the stream, so a
// controller that was away hears what was decided meanwhile. Appended, because the index is a name.
DecidedSubject,
}
// ControllerHears are events the controller's events consumer takes beside the ones it follows, and that need no
// subscribe grant of their own. **A push consumer delivers onto its deliver subject** (`_DELIVER.controller.>`, already
// granted), so the subject a consumer filters on is never one the controller subscribes: listing this one in
// ControllerFollows would add a subscribe grant the installer's carried user list, which lives in mesh-host, would
// have to carry too (TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose), for nothing the consumer uses.
//
// A merge mesh-delivery made, announced (novox/hq ADR 0420 §7, issue 1025): with the ask whose Approve it merged on and
// the heads that Approve covered, kept a week to read the warrant a moved module's registration needs.
var ControllerHears = []string{DeliveryMergedSubject}
// DeliveryMergedEvent is the event a module says when it merged a pull request.
const DeliveryMergedEvent = "merged"
// DeliveryMergedSubject is where the delivery's owner says it merged a pull request: one wildcard subject, the
// module that claims the delivery seat being whichever the catalogue says (novox/hq ADR 0437 §3). Who said it is
// read from the subject, and whether it claims the seat is the controller's to check.
var DeliveryMergedSubject = moduleEventSubject("*", DeliveryMergedEvent)
// DeliveryMergedSubjectOf is where one module says it merged a pull request.
func DeliveryMergedSubjectOf(module string) string {
return moduleEventSubject(module, DeliveryMergedEvent)
}
// AsksSeat is the seat an ask is made on and its warrant heard from (novox/hq ADR 0259): the router's.
const AsksSeat = "operator-channel"
// DecidedSubject is where the router says the controller's warrants: the seat's event named by the
// controller as its caller.
var DecidedSubject = seatEventSubject(AsksSeat, "decided."+ControllerSeat)
// The provider standing events, by their local names. Written here as well as in the catalogue
// (catalogue.ProvisionerEvents), which this package cannot import; a test keeps them agreeing.
const (
ProvisionerFailing = "provisioner.failing"
ProvisionerRecovered = "provisioner.recovered"
ProvisionerRetirement = "provisioner.retirement"
)
// moduleEventSubject is where one module's event lands. The same derivation PermissionsFor uses, so
// what the controller subscribes and what the emitter is permitted to publish cannot drift apart.
func moduleEventSubject(module, event string) string {
return "mesh.mod." + module + ".event." + event
}
// seatEventSubject is where a role's own event lands, derived the same way a holder's permission is.
func seatEventSubject(seat, verb string) string {
return "mesh.seat." + seat + ".event." + verb
}
// EventsStream holds every module's and every role's events, a build's log among them.
const EventsStream = "EVENTS"
// MeshConsumers is what the controller consumes, in the order a person reads it.
//
// **Unlimited redelivery on CONTROL, deliberately.** The store window's bound is the controller's,
// not the server's (window.go): a message is held with a nak-and-delay until the controller either
// takes it or gives up and says so. A max-deliver here would give up on a push that was being
// held through a store restart — the exact message the stream exists to protect — some minutes
// before the controller had finished deciding about it.
func MeshConsumers() []Consumer {
return ControllerConsumers()
}
// ControllerConsumers are the consumers the controller's own event loop takes from: MeshConsumers without the build
// seat's check workers, which the builders take from and the loop never does (issue 1063). A consumer added to the
// controller is counted here without a change elsewhere.
func ControllerConsumers() []Consumer {
return []Consumer{
{
Name: ControllerName,
Stream: "CONTROL",
Push: true,
AckWaitSeconds: 30,
Why: "the controller is the single consumer of what nodes say; explicit ack and no " +
"max-deliver, because the store window's bound is the controller's own",
},
// The forge's merges (novox/hq ADR 0407 decision 3, issue 973): never dropped, so no max-deliver and no
// reset; a merge the controller cannot record yet is held with a growing delay until it can. **Asserted before
// the events consumer**, which stops filtering merges: the order of this list is the order of assertion.
{
Name: MergesConsumerName,
Stream: "EVENTS",
// Within what the controller may subscribe already (`_DELIVER.controller.>`).
DeliverSubject: "_DELIVER." + ControllerName + ".merges",
Filters: []string{ControllerFollows[3]},
Push: true,
AckWaitSeconds: 30,
MaxAckPending: 1,
FromNow: true,
TakesOverFrom: ControllerName,
Why: "the forge's merges, one at a time; no max-deliver and never reset, because a merge is never " +
"dropped and no person releases one",
},
{
Name: ControllerName,
Stream: "EVENTS",
Filters: append(eventsTheControllerFollows(), AgainFilter(ControllerName)),
Push: true,
AckWaitSeconds: 30,
MaxDeliver: 5,
// One at a time (novox/hq issue 175): acting on a merge builds for minutes, and an
// announcement handed over behind it must wait on the server, not time out on the
// client and come back to be acted on again.
MaxAckPending: 1,
FromNow: true,
// **The one consumer the mesh resets by itself** (healer H4, issue 248): it fell a week
// behind once and held every merge after it. What a reset drops is caught up elsewhere — a
// build's outcome from the build records a plan settles from (issue 214), a provider's failing
// word by the provider saying it again every quarter of an hour (ADR 0224). **Not the merges**:
// they are the merges consumer's below, which a reset leaves alone (ADR 0407 decision 3).
Resettable: "what it drops is caught up: build outcomes from the build records (issue 214), a " +
"provider's failing word said again (ADR 0224); the forge's merges are not its, they have a " +
"consumer of their own that no reset touches",
Why: "the events the mesh's own controller reacts to, one at a time; after " +
"max-deliver it gives the event up, and the controller keeps it in DEAD_LETTERS until " +
"a person delivers it again or drops it",
},
// What the server said a consumer gave up on (novox/hq issue 330): copied into DEAD_LETTERS and
// acknowledged. No max-deliver: a notice the controller could not copy is offered again, and said.
{
Name: ControllerName,
Stream: DeadLetterNoticesStream,
Push: true,
AckWaitSeconds: 30,
Why: "the controller copies each message a consumer gave up on into DEAD_LETTERS; no max-deliver, " +
"because a notice it gave up on would lose the message it is about",
},
}
}
// NoticesConsumer is the controller's consumer on DEAD_LETTER_NOTICES (novox/hq issue 330).
func NoticesConsumer() Consumer {
for _, c := range MeshConsumers() {
if c.Stream == DeadLetterNoticesStream {
return c
}
}
panic("the mesh's consumers carry none on " + DeadLetterNoticesStream)
}
// AssertServingConsumers are the controller's own consumers its serving cannot go without: all but the
// one on DEAD_LETTER_NOTICES, which the keeper of dead letters asserts and retries by itself, so a fault
// there never stops the controller serving (novox/hq issue 330).
func AssertServingConsumers(e Ensurer) error {
for _, c := range MeshConsumers() {
if c.Stream == DeadLetterNoticesStream {
continue
}
if err := e.EnsureConsumer(c); err != nil {
return fmt.Errorf("asserting consumer %s on %s: %w", c.Name, c.Stream, err)
}
}
return nil
}
// Ensurer is the part of a JetStream connection consumer assertion needs, narrow for the reason
// Asserter is.
type Ensurer interface {
EnsureConsumer(c Consumer) error
}
// AssertMeshConsumers brings the controller's own consumers into being, and says which one failed.
//
// After the streams, necessarily: a consumer on a stream that does not exist is refused, and the
// refusal names the stream rather than the order.
func AssertMeshConsumers(e Ensurer) error {
for _, c := range MeshConsumers() {
if err := e.EnsureConsumer(c); err != nil {
return fmt.Errorf("asserting consumer %s on %s: %w", c.Name, c.Stream, err)
}
}
return nil
}
+78
View File
@@ -0,0 +1,78 @@
# Composed by the mesh controller. Do not edit: the next composition overwrites it.
# Accounts and permissions are derived from what each module declares and nothing
# else (novox/hq ADR 0043, design 29 §2).
port: 4222
http: 127.0.0.1:8222
tls {
cert_file: "/tls/tls.crt"
key_file: "/tls/tls.key"
ca_file: "/tls/ca.crt"
}
jetstream {
store_dir: "/data"
}
# The mesh's users, composed by the controller. Do not edit: the next
# composition overwrites it. Permissions are derived from what each module
# declares and nothing else (novox/hq ADR 0043, design 29 §2).
accounts {
MESH {
jetstream: enabled
imports = [
{ stream: { account: "$SYS", subject: "$SYS.ACCOUNT.MESH.CONNECT" }, to: "mesh.bus.connect" }
{ stream: { account: "$SYS", subject: "$SYS.ACCOUNT.MESH.DISCONNECT" }, to: "mesh.bus.disconnect" }
{ service: { account: "$SYS", subject: "$SYS.REQ.ACCOUNT.MESH.CONNZ" }, to: "mesh.bus.connz" }
]
users = [
{ user: "controller", password: "$2a$11$cccccccccccccccccccccc", permissions: {
publish: { allow: ["$JS.ACK.CONTROL.controller.>", "$JS.ACK.DEAD_LETTER_NOTICES.controller.>", "$JS.ACK.EVENTS.controller-merges.>", "$JS.ACK.EVENTS.controller.>", "$JS.ACK.KV_mesh-delivery-builder_builds.mesh-controller-take-in.>", "$JS.ACK.KV_mesh-delivery-indexer_modules.mesh-controller-take-in.>", "$JS.API.>", "$KV.mesh-controller_asked.>", "$KV.mesh-controller_assignments.>", "$KV.mesh-controller_builds.>", "$KV.mesh-controller_calls.>", "$KV.mesh-controller_composed.>", "$KV.mesh-controller_condition-history.>", "$KV.mesh-controller_conditions.>", "$KV.mesh-controller_declared-config.>", "$KV.mesh-controller_declared-packages.>", "$KV.mesh-controller_delivery-merges.>", "$KV.mesh-controller_grants.>", "$KV.mesh-controller_hand-acts.>", "$KV.mesh-controller_lease.>", "$KV.mesh-controller_predefined.>", "$KV.mesh-controller_rollouts.>", "$KV.mesh-controller_states-announced.>", "$SRV.INFO", "_INBOX.enrol.>", "mesh.again.>", "mesh.assignment.>", "mesh.events.dead.>", "mesh.mod.*.tool.>", "mesh.mod.mesh-controller.event.states", "mesh.node.>", "mesh.seat.mesh-controller.event.applied", "mesh.seat.mesh-controller.event.artifacts-gone", "mesh.seat.mesh-controller.event.assignment-changed", "mesh.seat.mesh-controller.event.bucket-deleted", "mesh.seat.mesh-controller.event.built-before", "mesh.seat.mesh-controller.event.checked", "mesh.seat.mesh-controller.event.condition-changed", "mesh.seat.mesh-controller.event.condition-cleared", "mesh.seat.mesh-controller.event.condition-raised", "mesh.seat.mesh-controller.event.declared-config-changed", "mesh.seat.mesh-controller.event.doctor-heartbeat", "mesh.seat.mesh-controller.event.healer-acted", "mesh.seat.mesh-controller.event.predefined-changed", "mesh.seat.mesh-controller.event.refused", "mesh.seat.mesh-controller.event.rolled-back", "mesh.seat.mesh-controller.event.secret-replaced", "mesh.seat.mesh-controller.tool.plans", "mesh.seat.mesh-controller.tool.push", "mesh.seat.mesh-controller.tool.upgrade", "mesh.seat.mesh-delivery.tool.close", "mesh.seat.mesh-delivery.tool.groups", "mesh.seat.mesh-delivery.tool.release", "mesh.seat.mesh-delivery.tool.stalled", "mesh.seat.mesh-delivery.tool.stop", "mesh.seat.mesh-delivery.tool.times", "mesh.seat.node-backup.tool.backed-up.*", "mesh.seat.node-backup.tool.now.*", "mesh.seat.node-intrusion-prevention.tool.banned.*", "mesh.seat.node-launcher.tool.secret.*", "mesh.seat.node-service-manager.tool.restart.*", "mesh.seat.operator-channel.tool.tell"] }
subscribe: { allow: ["$JS.API.>", "$JS.EVENT.ADVISORY.CONSUMER.DELETED.>", "$JS.EVENT.ADVISORY.CONSUMER.MAX_DELIVERIES.>", "$SRV.INFO", "$SRV.INFO.mesh-controller", "$SRV.INFO.mesh-controller.>", "$SRV.PING", "$SRV.PING.mesh-controller", "$SRV.PING.mesh-controller.>", "$SRV.STATS", "$SRV.STATS.mesh-controller", "$SRV.STATS.mesh-controller.>", "_DELIVER.controller", "_DELIVER.controller.>", "_INBOX.controller.>", "mesh.control.>", "mesh.mod.*.event.provisioner.failing", "mesh.mod.*.event.provisioner.recovered", "mesh.mod.*.event.provisioner.retirement", "mesh.mod.gitea.event.pull.merged", "mesh.mod.gitea.event.pull.updated", "mesh.mod.mesh-catalog.event.catching-up", "mesh.mod.mesh-catalog.event.upgraded", "mesh.seat.mesh-build-machine.event.built", "mesh.seat.mesh-controller.tool.>", "mesh.seat.node-build-agent.event.built", "mesh.seat.operator-channel.event.decided.mesh-controller"] }
allow_responses: { max: 1, ttl: "1m" }
} }
{ user: "enrol.one", password: "$2a$11$eeeeeeeeeeeeeeeeeeeeee", permissions: {
publish: { allow: ["mesh.control.enrol"] }
subscribe: { allow: ["_INBOX.enrol.one.>"] }
} }
{ user: "node.one", password: "$2a$11$nnnnnnnnnnnnnnnnnnnnnn", permissions: {
publish: { allow: ["$JS.ACK.NODES.one.>", "$JS.API.CONSUMER.CREATE.KV_composed-one.>", "$JS.API.CONSUMER.CREATE.KV_mesh-delivery-assigner_wants.*.$KV.mesh-delivery-assigner_wants.want.one.>", "$JS.API.CONSUMER.CREATE.KV_node-engine_have.*.$KV.node-engine_have.one.>", "$JS.API.CONSUMER.CREATE.OBJ_declaration-files-one.*.$O.declaration-files-one.C.*", "$JS.API.CONSUMER.DELETE.KV_composed-one.>", "$JS.API.CONSUMER.DELETE.KV_mesh-delivery-assigner_wants.>", "$JS.API.CONSUMER.DELETE.KV_node-engine_have.>", "$JS.API.CONSUMER.DELETE.OBJ_declaration-files-one.*", "$JS.API.CONSUMER.INFO.NODES.one", "$JS.API.DIRECT.GET.KV_composed-one.>", "$JS.API.DIRECT.GET.KV_mesh-delivery-assigner_wants.$KV.mesh-delivery-assigner_wants.want.one.>", "$JS.API.DIRECT.GET.KV_node-engine_have.$KV.node-engine_have.one.>", "$JS.API.DIRECT.GET.OBJ_declaration-files-one.$O.declaration-files-one.M.*", "$JS.API.STREAM.INFO.KV_composed-one", "$JS.API.STREAM.INFO.KV_mesh-delivery-assigner_wants", "$JS.API.STREAM.INFO.KV_node-engine_have", "$JS.FC.KV_composed-one.>", "$JS.FC.KV_mesh-delivery-assigner_wants.>", "$JS.FC.KV_node-engine_have.>", "$JS.FC.OBJ_declaration-files-one.>", "$KV.node-engine_have.one.>", "$SRV.PING.node-tools.one", "mesh.control.one.>", "mesh.seat.node-backup.call.backed-up.one.node~one", "mesh.seat.node-backup.call.now.one.node~one", "mesh.seat.node-backup.tool.backed-up.one", "mesh.seat.node-backup.tool.now.one"] }
subscribe: { allow: ["_DELIVER.one", "_DELIVER.one.>", "_INBOX.node.one.>", "mesh.mod.restic.event.backed-up", "mesh.node.one.ask.access-act", "mesh.node.one.ask.hand-over", "mesh.node.one.ask.judge-directories", "mesh.node.one.ask.move-secret", "mesh.node.one.ask.report", "mesh.node.one.ask.setuid-search", "mesh.node.one.declare"] }
allow_responses: { max: 1, ttl: "1m" }
} }
{ user: "one.mesh-delivery-watchman", password: "$2a$11$wwwwwwwwwwwwwwwwwwwwww", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.one_mesh-delivery-watchman.>", "$JS.API.CONSUMER.INFO.EVENTS.one_mesh-delivery-watchman", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.one_mesh-delivery-watchman", "$JS.API.DIRECT.GET.ASSIGNMENTS.mesh.assignment.one.mesh-delivery-watchman", "mesh.bus.connz"] }
subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.mesh-delivery-watchman", "$SRV.INFO.mesh-delivery-watchman.>", "$SRV.PING", "$SRV.PING.mesh-delivery-watchman", "$SRV.PING.mesh-delivery-watchman.>", "$SRV.STATS", "$SRV.STATS.mesh-delivery-watchman", "$SRV.STATS.mesh-delivery-watchman.>", "_INBOX.one.mesh-delivery-watchman.>", "mesh.assignment.one.mesh-delivery-watchman", "mesh.bus.connect", "mesh.bus.disconnect", "mesh.mod.mesh-delivery-watchman.call.>", "mesh.mod.mesh-delivery-watchman.tool.>"] }
allow_responses: { max: 1, ttl: "1m" }
} }
{ user: "one.nats", password: "$2a$11$bbbbbbbbbbbbbbbbbbbbbb", permissions: {
publish: { allow: ["$JS.API.STREAM.INFO.*", "$JS.API.STREAM.NAMES", "$JS.API.STREAM.SNAPSHOT.*", "$JS.SNAPSHOT.ACK.>"] }
subscribe: { allow: ["_INBOX.one.nats.>"] }
} }
{ user: "one.telegram", password: "$2a$11$tttttttttttttttttttttt", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.one_telegram.>", "$JS.ACK.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker.>", "$JS.API.CONSUMER.INFO.EVENTS.one_telegram", "$JS.API.CONSUMER.INFO.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.one_telegram", "$JS.API.CONSUMER.MSG.NEXT.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker", "$JS.API.DIRECT.GET.ASSIGNMENTS.mesh.assignment.one.telegram", "mesh.seat.telegram-sender.event.delivered", "mesh.seat.telegram-sender.event.failed"] }
subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.telegram", "$SRV.INFO.telegram.>", "$SRV.PING", "$SRV.PING.telegram", "$SRV.PING.telegram.>", "$SRV.STATS", "$SRV.STATS.telegram", "$SRV.STATS.telegram.>", "_INBOX.one.telegram.>", "mesh.assignment.one.telegram", "mesh.mod.telegram.call.>", "mesh.mod.telegram.tool.>", "mesh.seat.telegram-sender.accept.send"] }
allow_responses: { max: 1, ttl: "1m" }
} }
{ user: "two.audit", password: "$2a$11$aaaaaaaaaaaaaaaaaaaaaa", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.two_audit.>", "$JS.API.CONSUMER.INFO.EVENTS.two_audit", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_audit", "$JS.API.DIRECT.GET.ASSIGNMENTS.mesh.assignment.two.audit"] }
subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.audit", "$SRV.INFO.audit.>", "$SRV.PING", "$SRV.PING.audit", "$SRV.PING.audit.>", "$SRV.STATS", "$SRV.STATS.audit", "$SRV.STATS.audit.>", "_INBOX.two.audit.>", "mesh.assignment.two.audit", "mesh.mod.audit.call.>", "mesh.mod.audit.tool.>", "mesh.mod.shop.event.order.placed"] }
allow_responses: { max: 1, ttl: "1m" }
} }
{ user: "two.shop", password: "$2a$11$ssssssssssssssssssssss", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.two_shop.>", "$JS.API.CONSUMER.INFO.EVENTS.two_shop", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_shop", "$JS.API.DIRECT.GET.ASSIGNMENTS.mesh.assignment.two.shop", "mesh.mod.shop.event.order.placed", "mesh.seat.telegram-sender.accept.send"] }
subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.shop", "$SRV.INFO.shop.>", "$SRV.PING", "$SRV.PING.shop", "$SRV.PING.shop.>", "$SRV.STATS", "$SRV.STATS.shop", "$SRV.STATS.shop.>", "_INBOX.two.shop.>", "mesh.assignment.two.shop", "mesh.mod.shop.call.>", "mesh.mod.shop.tool.>"] }
allow_responses: { max: 1, ttl: "1m" }
} }
]
}
"$SYS" {
exports = [
{ stream: "$SYS.ACCOUNT.MESH.CONNECT", accounts: ["MESH"] }
{ stream: "$SYS.ACCOUNT.MESH.DISCONNECT", accounts: ["MESH"] }
{ service: "$SYS.REQ.ACCOUNT.MESH.CONNZ", accounts: ["MESH"] }
]
}
}
File diff suppressed because one or more lines are too long
+451
View File
@@ -0,0 +1,451 @@
package buscompose
import (
"fmt"
"git.novox.be/novox/mesh-sdk/go/announce"
"slices"
"sort"
)
// Every user the composed file should contain, derived from what the mesh knows.
//
// **The list is derived, never kept.** A stored user list would be a second account of who may
// reach the bus, able to disagree with the records it came from — and the disagreement would be
// invisible, because both would look internally consistent. So this is a pure function of the
// mesh's records, run again every time the file is written.
//
// Records are mirrored into this package's own types rather than imported from the catalogue, for
// the reason DeclaredSeat is: composing authority is a different job from parsing a manifest, and
// this package stays free of the other's types so a change to a manifest field cannot quietly widen
// a permission.
// Declared is one module on one node, as composing its authority needs it.
type Declared struct {
Module string
Emits []string
Consumes []string
Serves []string
// Holds are the seats this module claims, with the protocol each seat declares. A seat the
// mesh defines for itself declares no protocol, so holding one grants nothing on the bus —
// which is right: those seats are about who does a job, not about who may say what.
Holds []Seat
// Uses are the seats this module sends to.
Uses []Seat
// Issued are the tools that answer a credential its software issued, one per offer that says so (novox/hq
// ADR 0305): the controller is granted to ask each in its own name, and no other tool so.
Issued []string
// Watches are the seats whose events it consumes.
Watches []Seat
// Invokes are the tools it calls, `<module>.<tool>` or `*` (novox/hq ADR 0152).
Invokes []string
// Waits are the event families a session may wait on through the tool runner, the tool runner's module's
// alone (novox/hq ADR 0367 §2): its account subscribes them for a call, through no consumer.
Waits []Wait
// State is the state it keeps, each a bucket its instances write (novox/hq ADR 0201).
State []Bucket
// Logs are the logs it keeps, each a stream its instances append to (novox/hq ADR 0297).
Logs []Log
// Streams are the streams it owns, each its events' one place (novox/hq ADR 0363).
Streams []ModuleStream
// Queues are its own work-queue streams and the one process each is the consumer of (novox/hq ADR 0436
// §1, §2, §6): one durable consumer per entry, named `<node>_<module>_<process id>`.
Queues []QueueOwner
// Hears are the module streams it hears on this machine, each with the subjects it was granted
// (novox/hq ADR 0363 §5); HearsAStream says its `consumes` names a module that owns a stream, granted
// or not, so its membership is issued again when a grant changes.
Hears []Heard
HearsAStream bool
// Reads are other modules' state it reads, each `<module>.<name>` (novox/hq ADR 0201).
Reads []string
// ReadsRefused are the reads its manifest names that the owner's announcement does not make readable (novox/hq ADR
// 0495 clause 3, issue 1790): granted nothing, kept to be said.
ReadsRefused []string
// KeyedReads are keys of other modules' state it reads, each one key alone: what a seat's holder is
// granted for the pieces the modules beside it offer (novox/hq ADR 0260).
KeyedReads []KeyedRead
// NoAccount says the module declares no own secret named broker, so no account could ever be
// delivered to it and nothing can connect as it (novox/hq issue 195). Said in the negative so a
// record that does not say is composed as it always was.
NoAccount bool
// SnapshotsTheBus says the module holds mesh-broker — it is the bus — and so is the one module
// granted the snapshot API, to copy the bus's streams for the night's backup (novox/hq ADR 0235).
SnapshotsTheBus bool
// Checks are the module's own tools its health asks, each `<module>.<tool>` (novox/hq ADR 0240, to-be
// 48 §3): the machine's node-engine asks them of its own node tools, and is granted that and no more.
Checks []string
// Loads says the module has a compiled bundle whose entrypoints the machine's tool runtime loads and runs
// (novox/hq ADR 0175; issue 1329): a helper that reaches the bus on the runner's account, so the runner
// must carry the module although it declares no tools.
Loads bool
// RunsAs is the account the module runs as in a runtime of its own (novox/hq ADR 0259 §8): it is never
// carried by the machine's runtime, and reaches the bus on its own account.
RunsAs string
// ProvisionsNode says the module runs the machine's provisioner client: it claims NodeProvisionerSeat and
// declares the own secret ProvisionerSecret its account is delivered in (novox/hq ADR 0349 clause 11, design 57
// P3). The client is given a bus account of its own, never the module's or the tool runner's.
ProvisionsNode bool
// Processes are this module's own processes that declare bus (novox/hq ADR 0436 §1), each getting a
// principal of its own: granted exactly what its bus says, and nothing of this module's other grants —
// composed whether or not the module itself has an account (NoAccount), because a process account is
// delivered in its own secret, broker-<process id>, never the module's `broker`.
Processes []ProcessPrincipal
}
// ProcessPrincipal is one process of a module that declares bus (novox/hq ADR 0436 §1, §2), as the broker
// composes its own account from: its own emits, its own work-queue consumers, its own state, whether it
// reads the router's records of its module's own asks, and the tools and seat verbs it calls.
type ProcessPrincipal struct {
// Process is the process resource's id, naming the account <node>.<module>.<process id>.
Process string
// Emits are the events this process may publish, named as the module's own Emits are.
Emits []string
// Queues are the module's own work-queue streams this process is the one consumer of.
Queues []QueueOwner
// State is the module's own state, by local name, this process reads and writes.
State []string
// Asks says whether this process reads the records of its module's own asks (novox/hq ADR 0259 §3), on
// whichever seats the module uses that keep them (the router's, operator-channel, among others).
Asks bool
// Invokes are the tools and seat verbs this process calls, each <module>.<tool> or a role's
// seat:<seat>.<verb>, or * for every tool on the mesh.
Invokes []string
}
// Records is what composing a user list needs to know about the mesh, and nothing more.
type Records struct {
// Nodes is every machine the mesh knows. Each gets a host user.
Nodes []string
// Assigned is the modules on each node, as they declare themselves.
Assigned map[string][]Declared
// Enrolling is every node with a live token — one enrolment user each, because the inbox an
// answer goes to is scoped to the token and a shared one is one machine reading another's
// sealed credentials (design 25 §6).
Enrolling []string
// People is each person's name against the tools they may invoke, `*` for an administrator.
People map[string][]string
// Interchangeable is each module whose definition says its instances are the same anywhere
// (ADR 0160), which decides whether the module's plain subject is issued to every instance.
Interchangeable map[string]bool
// RootFree is each machine judged root-free when this was composed (novox/hq ADR 0259 §8): it names an
// account agents run as, judged unable to become root by its node-engine, and serves no login shell
// execute. A machine absent is not free: no record of a pass is no pass.
RootFree map[string]bool
// Announced is each module's last announcement of the states it keeps (novox/hq ADR 0495, issue 1790), as the
// controller keeps it; a module absent has announced nothing yet. Nil where none was read.
Announced map[string]announce.States
}
// Users is every user the composed file should contain, in the order it will be written.
//
// The controller is always first and always present: a mesh whose own controller is not in the file
// is a mesh that cannot be told anything, and there is no state of the records in which that is
// correct.
func Users(r Records) ([]Principal, error) {
out := []Principal{{Kind: KindController, Uses: asksSeatOf(r), AsksIssued: issuedOf(r), AsksTheVault: vaultVerbsOf(r),
AsksTheConfig: servedOf(r, VerbsTheControllerAsksTheConfig), FilesFor: sortedCopy(r.Nodes)}}
for _, node := range sortedCopy(r.Nodes) {
witness := false
var checks []string
for _, d := range r.Assigned[node] {
if d.Module == controllerModule {
witness = true
}
checks = append(checks, d.Checks...)
}
out = append(out, Principal{Kind: KindNode, Node: node, WitnessesController: witness, Checks: checks,
MovesIntoTheVault: servesMoveIn(r), StatesItsKeyToTheVault: servesVerb(r, VaultNodeKey)})
// **And the machine's provisioner client, on an account of its own** (novox/hq ADR 0349 clause 11): one
// per machine that runs it, delivered to the module running it.
for _, d := range r.Assigned[node] {
if d.ProvisionsNode {
out = append(out, Principal{Kind: KindProvisioner, Node: node, Module: d.Module,
BoundBuckets: boundBucketsOf(r)})
break
}
}
// **Where the runtime is assigned, the machine gets one runtime principal in place of the
// runtime module's own** (novox/hq ADR 0175, to-be 38). It carries every module on the
// node: its serving grants are the union of theirs. Every other module keeps its own
// principal — a module still serving tools from its own container holds its own
// credential until it moves, and the two serve side by side in the meantime.
runtimeHere := false
for _, d := range r.Assigned[node] {
if d.Module == RuntimeModule {
runtimeHere = true
}
}
for _, d := range r.Assigned[node] {
// **A process account is its own, delivered in its own secret** (broker-<process id>, novox/hq
// ADR 0436 §1, §2), never the module's `broker` — so it is composed whether or not the module
// itself has an account, and is never carried by the machine's runtime, which runs as the
// operator's account (ADR 0436 §5): added here, before either of those is judged.
for _, pr := range d.Processes {
out = append(out, Principal{
Kind: KindProcess, Node: node, Module: d.Module, Process: pr.Process,
Emits: pr.Emits, Queues: pr.Queues, State: pr.State, Uses: d.Uses, Asks: pr.Asks,
Invokes: pr.Invokes,
})
}
if runtimeHere && d.Module == RuntimeModule {
continue
}
// **A module with nowhere to read an account is no user** (novox/hq issue 195). `module
// issue` refuses it one (issue 078: an account nothing reads is an orphan), so its user
// could only ever be left out of the file for want of a password — and every such module
// was named, on every status and plan, as a credential the mesh had not minted. Where the
// runtime is, it speaks for the module; where it is not, the module cannot speak at all,
// and a user would not change that.
if d.NoAccount {
continue
}
out = append(out, Principal{
Kind: KindModule, Node: node, Module: d.Module,
Emits: d.Emits, Consumes: d.Consumes, Serves: d.Serves,
Holds: d.Holds, Uses: d.Uses, Watches: d.Watches, Invokes: d.Invokes,
State: stateNames(d.State), Reads: d.Reads, SnapshotsTheBus: d.SnapshotsTheBus,
PerMachine: perMachineNames(d.State), KeyedReads: d.KeyedReads,
Logs: logNames(d.Logs), Streams: d.Streams, Hears: ComposedHears(d), Queues: d.Queues,
})
// **The grants part creates what a manifest declares** (novox/hq ADR 0547 §5; the operator, 2026-10-26
// 08:54): every placed module's buckets, logs and streams, each by its stream's name.
if d.Module == GrantsModule {
out[len(out)-1].CreatesStreams = declaredStreamsOf(r)
}
// **The vault's holder asks the desks and the issuing providers itself** (issue 723), under the
// account its own runtime runs on (ADR 0259 §8's runs-as): the one module granted it.
if holdsSeat(d, VaultSeat) {
out[len(out)-1].AsksTheDesk = true
out[len(out)-1].AsksIssued = issuedOf(r)
}
}
if runtimeHere {
var carried []Declared
for _, d := range r.Assigned[node] {
if d.RunsAs == "" && runnerServes(d) {
// Never a restricted stream for what the tool runner carries (novox/hq ADR 0376 §1).
d.Hears = ComposedHears(d)
carried = append(carried, d)
}
}
out = append(out, Principal{Kind: KindNodeTools, Node: node, Module: RuntimeModule, Carries: carried})
}
}
for _, node := range sortedCopy(r.Enrolling) {
out = append(out, Principal{Kind: KindEnrolment, Node: node})
}
for _, person := range sortedNames(r.People) {
p := Principal{Kind: KindPerson, Module: person, Invokes: r.People[person]}
if administers(p.Invokes) {
// The operator's hand path writes each node's compositions (ADR 0539 clause 10).
p.FilesFor = sortedCopy(r.Nodes)
}
out = append(out, p)
}
// Refused here rather than discovered by the server. Two users with one name is a file the
// server reads as one of them, and which one depends on the order — so a module assigned to a
// node twice, or a person named after nothing, is a composition that must not be written.
seen := map[string]string{}
for _, p := range out {
name := p.Username()
if name == "" || name == "." {
return nil, fmt.Errorf("a %s user has no name, so nothing could authenticate as it", p.Kind)
}
if first, already := seen[name]; already {
return nil, fmt.Errorf(
"two users would be called %q (a %s and a %s): the server would read the file as "+
"one of them, and which one depends on the order", name, first, p.Kind)
}
seen[name] = string(p.Kind)
}
return out, nil
}
// WithPasswords fills each user's hash from what the mesh minted, and says which users have none.
//
// **Separated from Users because they fail differently.** A user missing from the records is a bug
// in deriving them; a user with no password is a step that has not happened yet — a module assigned
// but never given a credential, a node enrolled before this existed. The second is ordinary and its
// remedy is to mint one, so it is named rather than returned as an error, and the caller decides
// whether a partial composition is worth writing.
func WithPasswords(principals []Principal, hashes map[string]string) (filled []Principal, missing []string) {
for _, p := range principals {
hash, ok := hashes[p.Username()]
if !ok || hash == "" {
missing = append(missing, p.Username())
continue
}
p.PasswordHash = hash
filled = append(filled, p)
}
return filled, missing
}
func sortedCopy(in []string) []string {
out := append([]string(nil), in...)
sort.Strings(out)
return out
}
func sortedNames(in map[string][]string) []string {
out := make([]string, 0, len(in))
for k := range in {
out = append(out, k)
}
sort.Strings(out)
return out
}
// controllerModule is the controller's module: the machine assigned it witnesses its upgrades.
const controllerModule = "mesh-controller"
// asksSeatOf is the seat an ask is made on, as its holder declares it (novox/hq ADR 0259): the controller
// asks the operator through it like any other user, and is granted what its declaration names for a caller.
// None while nothing holds it.
// issuedOf is every tool of a module assigned anywhere that answers a credential its software issued, as
// `<module>.<tool>` (novox/hq ADR 0305): what the controller asks in its own name, and nothing beside.
func issuedOf(r Records) []string {
seen := map[string]bool{}
var out []string
for _, node := range sortedCopy(r.Nodes) {
for _, d := range r.Assigned[node] {
for _, t := range d.Issued {
if key := d.Module + "." + t; !seen[key] {
seen[key] = true
out = append(out, key)
}
}
}
}
sort.Strings(out)
return out
}
func asksSeatOf(r Records) []Seat {
for _, node := range sortedCopy(r.Nodes) {
for _, d := range r.Assigned[node] {
for _, s := range d.Holds {
if s.Name == AsksSeat && namesVerb(s.ByCaller, "ask") {
seat := s
seat.Kind, seat.Capabilities = "", nil
return []Seat{seat}
}
}
}
}
return nil
}
// VaultSeat is the vault's seat: its holder alone asks the desks and the issuing providers in its own name.
const VaultSeat = "mesh-vault"
func holdsSeat(d Declared, seat string) bool {
for _, s := range d.Holds {
if s.Name == seat {
return true
}
}
return false
}
// vaultVerbsOf are the vault's intake verbs its holder serves, as the records say (issue 675): what the controller
// is granted to ask it in its own name. None while no holder serves them.
func vaultVerbsOf(r Records) []string { return servedOf(r, VerbsTheControllerAsksTheVault) }
// servedOf are the verbs of asked that a seat's holder serves, as the records say, by name.
func servedOf(r Records, asked []SeatVerb) []string {
seen := map[string]bool{}
var out []string
for _, node := range sortedCopy(r.Nodes) {
for _, d := range r.Assigned[node] {
for _, s := range d.Holds {
for _, v := range asked {
if s.Name == v.Seat && slices.Contains(s.Serves, v.Verb) && !seen[v.Verb] {
seen[v.Verb] = true
out = append(out, v.Verb)
}
}
}
}
}
sort.Strings(out)
return out
}
// servesMoveIn says the vault's holder serves move-in, as the records say (issue 676): what lets each machine's
// node-engine ask it in its own name.
func servesMoveIn(r Records) bool { return servesVerb(r, VaultMoveIn) }
// servesVerb says a holder of verb's seat serves it, as the records say.
func servesVerb(r Records, verb SeatVerb) bool {
for _, node := range r.Nodes {
for _, d := range r.Assigned[node] {
for _, s := range d.Holds {
if s.Name == verb.Seat && slices.Contains(s.Serves, verb.Verb) {
return true
}
}
}
}
return false
}
// boundBucketsOf is every provider's bucket of entries, `<module>_bound` (mesh-sdk go/provider, issue 678): the state
// named bound of every module placed anywhere, once each, in name order. A consumer on any machine may be bound to
// it, so each machine's provisioner client reads its own keys of each (issue 679).
func boundBucketsOf(r Records) []string {
seen := map[string]bool{}
for _, mods := range r.Assigned {
for _, d := range mods {
for _, b := range d.State {
if b.Name == BoundState {
seen[b.Bucket()] = true
}
}
}
}
out := make([]string, 0, len(seen))
for b := range seen {
out = append(out, b)
}
sort.Strings(out)
return out
}
// runnerServes says the machine's tool runner has something of this module to serve (novox/hq ADR 0436 §1,
// ADR 0437 §4, issue 1329): tools or loads it answers, a seat it holds, event families a session waits on,
// a bundle the runtime loads (Loads), or a check its health asks. A module with none of these is a process of its own on its own credential, and
// carrying it would hand every session on the machine the runner's account with that module's state, seat
// traffic and streams. A module with no account of its own (NoAccount) is spoken for by the runner, so it
// stays carried, and so does the runtime module itself. Whether the module has its own broker secret is not
// the test: a module with tools and its own credential is still carried (ADR 0175).
func runnerServes(d Declared) bool {
return d.Module == RuntimeModule || d.NoAccount || d.Loads || len(d.Serves) > 0 || len(d.Holds) > 0 || len(d.Waits) > 0 || len(d.Checks) > 0
}
// declaredStreamsOf is the stream of every bucket, log and stream a module placed anywhere declares, once each, in name
// order (novox/hq ADR 0547 §5, issue 1790): `KV_<module>_<state>`, `LOG_<module>_<log>` and each module stream's own
// name. What the grants part may create, and nothing beside.
func declaredStreamsOf(r Records) []string {
seen := map[string]bool{}
for _, mods := range r.Assigned {
for _, d := range mods {
for _, b := range d.State {
seen["KV_"+b.Bucket()] = true
}
for _, l := range d.Logs {
seen[l.Stream()] = true
}
for _, s := range d.Streams {
seen[s.Name] = true
}
}
}
out := make([]string, 0, len(seen))
for s := range seen {
out = append(out, s)
}
sort.Strings(out)
return out
}
+94
View File
@@ -0,0 +1,94 @@
package buscompose
import (
"fmt"
"strings"
)
// What a session may wait on through the mesh MCP server (novox/hq ADR 0367).
//
// **A grant apart from `consumes`, and the tool runner's alone.** The tool runner's module names in its
// manifest the event families a session may wait on, one by one; the runtime's account is granted a plain
// subscription on those subjects and nothing wider — no durable consumer, no queue of its own — and its
// membership names the same list, from which `mesh_wait` refuses any other. Both are read from this one
// composition, so the grant and what the tool runner offers cannot disagree, and nothing but the manifest
// adds a family (§6).
// Wait is one event family the tool runner's module declares: its name as the manifest gives it, and the
// emitter — a seat or a module — and the event, as the inventory resolved them.
type Wait struct {
// Name is the family as the manifest names it, `<emitter>.<event>`: what a session asks for.
Name string
Emitter string
// Seat says the emitter is a seat, whose events live under the seat's namespace (novox/hq ADR 0121).
Seat bool
// Event is the event, with any further token the subject carries: `decided.claude-code`, the asker of a
// seat that names it last (ADR 0259 §3).
Event string
// Last names what the seat's event subject ends in — "kind" for a kinded bench, "asker" for a seat naming
// its caller (ADR 0259 §3) — and is empty for an event that ends at its name. A family needing it is waited
// on with it given, and refused without.
Last string
}
// WaitSubject is where a family's events land. Each token is checked before it becomes a subject: a
// wildcard, a dot inside a name or an empty token would widen the grant.
func WaitSubject(w Wait) (string, error) {
if !safeSubject.MatchString(w.Emitter) {
return "", fmt.Errorf("%q cannot name an emitter of a waited event: it would widen the subject", w.Emitter)
}
for _, t := range strings.Split(w.Event, ".") {
if !safeSubject.MatchString(t) {
return "", fmt.Errorf("%q cannot name a waited event: each of its names is plain, never a wildcard "+
"(novox/hq ADR 0367 §2)", w.Event)
}
}
if w.Last != "" && !strings.Contains(w.Event, ".") {
return "", fmt.Errorf("%s names no %s: %s's events are said for each %s, so a family is %s.%s.<%s>; "+
"without it no event would ever be heard (novox/hq ADR 0367 §5)",
w.Name, w.Last, w.Emitter, w.Last, w.Emitter, w.Event, w.Last)
}
if w.Seat {
return "mesh.seat." + w.Emitter + ".event." + w.Event, nil
}
return "mesh.mod." + w.Emitter + ".event." + w.Event, nil
}
// waitGrants is the subscription the tool runner's account is granted for its module's families, and an
// error for a carried module other than the tool runner's that declares any (ADR 0367 §2).
func waitGrants(carries []Declared) ([]string, error) {
var sub []string
for _, d := range carries {
if len(d.Waits) == 0 {
continue
}
if d.Module != RuntimeModule {
return nil, fmt.Errorf("%s declares what a session waits on; only %s, the tool runner's module, "+
"does (novox/hq ADR 0367 §2)", d.Module, RuntimeModule)
}
for _, w := range d.Waits {
s, err := WaitSubject(w)
if err != nil {
return nil, err
}
sub = append(sub, s)
}
}
return sub, nil
}
// waitsIssuedFor is the families a membership names: the tool runner's module's, and nobody else's.
func waitsIssuedFor(d Declared) []WaitIssued {
if d.Module != RuntimeModule {
return nil
}
var out []WaitIssued
for _, w := range d.Waits {
s, err := WaitSubject(w)
if err != nil {
continue // refused where the account is composed, which names it
}
out = append(out, WaitIssued{Event: w.Name, Subject: s})
}
return out
}
+320
View File
@@ -0,0 +1,320 @@
package buscompose
import (
"fmt"
"slices"
"strings"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// The writers table (novox/hq to-be 45 §1, ADR 0227 rule 1), compiled in.
//
// **Every kind of state the core keeps has one writer; anyone else asks it.** The table is the design's,
// row for row, with what each row writes on the bus where it writes there. Two things read it: the
// composition of every principal's grants (PermissionsFor), which **refuses a grant that lets a
// principal publish on a subject another writes** — so a second writer cannot be granted by accident,
// and the composition says which state and whose — and the test beside it, which walks the rows
// against the design's list and the composition of a whole mesh. Changing a writer is a change to
// this table, through a decision.
// WriterRow is one row of the writers table.
type WriterRow struct {
State string
Writer string
KeptIn string
Others string
// Subjects are the bus subjects a write of this state is a publish to; none for state kept off the
// bus (the controller's store, a module's own) or not built yet.
Subjects []string
// Writes says a principal granted a publish pattern overlapping Subjects is this state's writer —
// given the pattern, because a machine writes its own report and no other's.
Writes func(p Principal, pattern string) bool
// Shared says why more than one principal may publish here; empty for one writer.
Shared string
}
// isController, and the other writers' tests, are who a row's writer is as a principal.
func isController(p Principal, _ string) bool { return p.Kind == KindController }
// createsDeclared is the controller, or the grants part creating one stream a manifest declares, by its own name: the
// create alone, never an update or a delete (novox/hq ADR 0547 §5; the operator, 2026-10-26 08:54).
func createsDeclared(p Principal, pattern string) bool {
if isController(p, pattern) {
return true
}
name, ok := strings.CutPrefix(pattern, "$JS.API.STREAM.CREATE.")
return ok && p.Kind == KindModule && p.Module == GrantsModule && slices.Contains(p.CreatesStreams, name) &&
safeSubject.MatchString(name)
}
// ownMachine is a node writing a subject whose third token is its own name, and no wildcard there.
func ownMachine(p Principal, pattern string) bool {
tokens := strings.Split(pattern, ".")
return p.Kind == KindNode && len(tokens) > 2 && tokens[2] == p.Node
}
// holdsSeatOf is a principal holding the seat a `mesh.seat.<seat>.…` pattern names: its module's own
// principal, or the node tools carrying a module that holds it (ADR 0175, the runtime is its modules).
func holdsSeatOf(p Principal, pattern string) bool {
tokens := strings.Split(pattern, ".")
if len(tokens) < 3 {
return false
}
seat := tokens[2]
holds := func(seats []Seat) bool {
return slices.ContainsFunc(seats, func(s Seat) bool { return s.Name == seat })
}
switch p.Kind {
case KindModule:
return holds(p.Holds)
case KindNodeTools:
return slices.ContainsFunc(p.Carries, func(d Declared) bool { return holds(d.Holds) })
}
return false
}
// ownModule is a module publishing under its own name, or the node tools carrying it.
func ownModule(p Principal, pattern string) bool {
tokens := strings.Split(pattern, ".")
if len(tokens) < 3 {
return false
}
module := tokens[2]
switch p.Kind {
case KindModule:
return p.Module == module
case KindNodeTools:
return slices.ContainsFunc(p.Carries, func(d Declared) bool { return d.Module == module })
}
return false
}
// ownAnnouncer is a module, a process of it or the node tools carrying it saying what that module keeps, or the
// controller saying what it keeps itself (novox/hq ADR 0495 clause 2): each under its own name alone.
func ownAnnouncer(p Principal, pattern string) bool {
tokens := strings.Split(pattern, ".")
if len(tokens) < 3 {
return false
}
switch p.Kind {
case KindController:
return tokens[2] == ControllerSeat
case KindProcess:
return p.Module == tokens[2]
}
return ownModule(p, pattern)
}
// ownProvisioner is a machine's provisioner client saying a report about its own machine: the third token of
// `prov.applied.<node>.…` is its machine (novox/hq ADR 0349 §4, design 57 P3).
func ownProvisioner(p Principal, pattern string) bool {
tokens := strings.Split(pattern, ".")
return p.Kind == KindProvisioner && len(tokens) > 3 && tokens[2] == p.Node
}
// ownNodeKeys is a node writing the have keys of its own node: `$KV.<bucket>.<node>.…`, the node in the third token
// and no wildcard there (novox/hq ADR 0429 §1).
func ownNodeKeys(p Principal, pattern string) bool {
tokens := strings.Split(pattern, ".")
return p.Kind == KindNode && len(tokens) > 2 && tokens[2] == p.Node
}
// kvOf is a bucket's write subjects.
func kvOf(bucket string) []string { return []string{"$KV." + bucket + ".>"} }
// WritersTable is to-be 45 §1, in its order.
var WritersTable = []WriterRow{
{State: "a machine's declaration", Writer: "controller (lease holder)", KeptIn: "the bus, last per subject",
Others: "read", Subjects: []string{"mesh.node.*.declare"}, Writes: isController},
// The operator's hand-over of a directory used as found, asked of the machine's engine at the controller's
// terminal (novox/hq issue 356). One publisher; and because a responder can still reach the subject through a
// reply, the ask is signed with the mesh's key and the engine verifies it — the row bounds who is granted the
// publish, the signature who is believed.
{State: "a hand-over asked of a machine", Writer: "controller, at its terminal", KeptIn: "the machine, beside its state",
Others: "the engine verifies the mesh's signature and records it, or refuses",
Subjects: []string{"mesh.node.*.ask.hand-over"}, Writes: isController},
// The operator asking a machine's engine for a fresh search for setuid programs, at the controller's
// terminal (novox/hq issue 361): signed as a hand-over is, under a signing context of its own.
{State: "a fresh setuid search asked of a machine", Writer: "controller, at its terminal",
KeptIn: "the machine, which throws its last search away", Others: "the engine verifies the mesh's signature and starts it, or refuses",
Subjects: []string{"mesh.node.*.ask.setuid-search"}, Writes: isController},
// A read-only judgement of a machine's declared directories asked of its engine (novox/hq issue 358): signed as a
// hand-over is, under a signing context of its own. It changes nothing on the machine.
{State: "a judgement of a machine's directories asked of it", Writer: "controller",
KeptIn: "nowhere: the engine answers it and keeps nothing", Others: "the engine verifies the mesh's signature and answers, or refuses",
Subjects: []string{"mesh.node.*.ask.judge-directories"}, Writes: isController},
{State: "a machine's applied state and its report", Writer: "the node-engine's apply queue",
KeptIn: "the machine; the report on the bus", Others: "the reconcile and a delivery enqueue, never apply",
// And its health statement between reports (novox/hq ADR 0240): the same writer stating the same
// machine, inside the grant it already had (`mesh.control.<its own>.>`). And a line mesh-cli was given
// on the machine (novox/hq ADR 0272): the node is what the controller judges the terminal by, so that
// subject is this machine's engine's alone.
Subjects: []string{"mesh.control.*.report", "mesh.control.*.health", "mesh.control.*.cli"}, Writes: ownMachine},
{State: "the controller lease", Writer: "the controller instance holding it", KeptIn: "key-value " + LeaseBucket,
Others: "a candidate waits", Subjects: kvOf(LeaseBucket), Writes: isController},
// The merges mesh-delivery announced, for a moved module's registration (ADR 0420 §7), written by the lease holder.
{State: "the merges mesh-delivery announced", Writer: "controller (lease holder)",
KeptIn: "key-value " + DeliveryMergesBucket, Others: "read at a moved module's registration",
Subjects: kvOf(DeliveryMergesBucket), Writes: isController},
// And the compositions (novox/hq ADR 0539 clauses 2 and 4): each node's in its own bucket, which only that node's
// node-engine reads, and each digest with the judgement of its move, which the assigner reads.
{State: "the compositions and their digests", Writer: "controller (lease holder), compare-and-set",
KeptIn: "key-value composed-<node> for each node, and " + delivery.ControllerComposedBucket,
Others: "a node-engine reads its own node's; the assigner reads the digests",
Subjects: kvOf(delivery.ControllerComposedBucket), Writes: isControllerOrTheHandPath},
{State: "conditions", Writer: "controller", KeptIn: "key-value " + ConditionsBucket + " (and its history, " +
ConditionHistoryBucket + "); how often each was observed, when last and its newest evidence in the " +
"controller's store, so a look that changes nothing writes no bucket", Others: "raise or clear only through observations the controller reads",
Subjects: append(kvOf(ConditionsBucket), kvOf(ConditionHistoryBucket)...), Writes: isController},
{State: "calls and their outcomes", Writer: "controller", KeptIn: "key-value " + CallsBucket,
Others: "read by id", Subjects: kvOf(CallsBucket), Writes: isController},
{State: "the hand-act log", Writer: "controller, through the verbs that act", KeptIn: "key-value " + HandActsBucket,
Others: "—", Subjects: kvOf(HandActsBucket), Writes: isController},
// What the healers did (novox/hq to-be 45 §7, Phase 3): the controller's alone, in its store — the
// budgets and the mesh-wide brake are counted from it, so a controller restarting cannot reset them.
{State: "the healers' acts and their brake", Writer: "controller (lease holder), each act begun before it is made",
KeptIn: "the controller's store", Others: "read through healers; each act said as healer-acted and in its condition's tried"},
{State: "stream definitions and bus permissions", Writer: "controller", KeptIn: "the bus", Others: "—",
// A stream's definition, and a durable consumer's by the API that names it so. Not every
// consumer create: a module watching its own bucket makes and deletes an ordered consumer on the
// bucket's stream (ADR 0201), which defines nothing the mesh keeps.
Subjects: []string{"$JS.API.STREAM.CREATE.>", "$JS.API.STREAM.UPDATE.>", "$JS.API.STREAM.DELETE.>",
"$JS.API.CONSUMER.DURABLE.CREATE.>"},
Writes: createsDeclared,
Shared: "the grants part creates a stream a manifest declares, by its name, and never updates or deletes one (novox/hq ADR 0547 §5; the operator, 2026-10-26 08:54)"},
{State: "builds and their outcomes", Writer: "the build seat's holder", KeptIn: "its own state",
Others: "the controller asks",
Subjects: []string{"mesh.seat.node-build-agent.event.built", "mesh.seat.mesh-build-machine.event.built"},
Writes: holdsSeatOf,
Shared: "every machine holding the build seat answers the asks it took; each outcome names its ask"},
// What a module keeps and who may read it (novox/hq ADR 0495 clauses 2 and 3, issue 1790): said by the module, kept by
// the controller.
{State: "a module's states announced", Writer: "the module itself, from its code, under its own name",
KeptIn: "the bus (EVENTS)", Others: "the controller keeps the last of each and grants readers from it",
Subjects: []string{"mesh.mod.*.event.states"}, Writes: ownAnnouncer},
{State: "the states announced", Writer: "controller, from each module's announcement",
KeptIn: "key-value " + StatesAnnouncedBucket, Others: "the composition of the bus's users reads it",
Subjects: kvOf(StatesAnnouncedBucket), Writes: isController},
{State: "a merge announced", Writer: "one announcer per forge (the hook, or the poll when the hook is absent — never both)",
KeptIn: "the bus", Others: "—", Subjects: []string{"mesh.mod.*.event.pull.merged"}, Writes: ownModule},
// A pull request's head, before it merges (novox/hq to-be 45 §9): the same one announcer.
{State: "a pull request's head announced", Writer: "the forge's announcer, the one that announces its merges",
KeptIn: "the bus", Others: "the controller asks the build seat to check it", Subjects: []string{"mesh.mod.*.event.pull.updated"},
Writes: ownModule},
{State: "a pull request's merge check", Writer: "the build seat's holder that ran it, said as the controller's `checked`",
KeptIn: "the bus", Others: "the forge's holder sets it as the pull request's status"},
{State: "a provider's standing", Writer: "the provider", KeptIn: "the provider's events",
Others: "the controller keeps the newest word as a condition",
Subjects: []string{"mesh.mod.*.event.provisioner.failing", "mesh.mod.*.event.provisioner.recovered",
"mesh.mod.*.event.provisioner.retirement"},
Writes: ownModule},
{State: "the operator-channel's open messages", Writer: "the seat's holder", KeptIn: "its own key-value state",
Others: "—"},
{State: "the facts snapshot", Writer: "controller", KeptIn: "the artifact store, facts/latest",
Others: "the build seat reads"},
// What a consumer gave up on (novox/hq issue 330): copied by the controller from the server's notice,
// and delivered again or dropped only through its verb, with why.
{State: "a message a consumer gave up on", Writer: "controller", KeptIn: "the bus, the stream " + DeadLettersStream,
Others: "read, delivered again or dropped through the controller's dead-letters verb",
Subjects: []string{deadLetterPrefix + ">", againPrefix + ">"}, Writes: isController},
// The record mirrors (novox/hq ADR 0289 §9, design 54 §2): rows of this table, so a grant that lets
// anything else write them is refused at composition. Built so far: the assignments and each registered
// module's declared config (design 57 F3), and the list of predefined keys kept (design 57 K3).
{State: "the assignments mirror", Writer: "controller (lease holder): the verbs that assign, and the sends that record what runs; " +
"a mirror of its store only until the assignments are moved to the bus (novox/hq ADR 0408)",
KeptIn: "key-value " + AssignmentsBucket + ", the record once moved", Others: "read through `reads` as mesh-controller." + MirrorAssignments,
Subjects: kvOf(AssignmentsBucket), Writes: isController},
// The chain (novox/hq ADR 0408): builds and rollouts beside the assignments above, which keep their bucket. Not
// mirrors of the store but the record, written by the lease holder by compare-and-set and removed by nobody.
{State: "the builds of the chain", Writer: "controller (lease holder), compare-and-set on the key's revision",
KeptIn: "key-value " + BuildsBucket + ", sixty-four revisions of each key, none removed",
Others: "read through `reads` as mesh-controller." + ChainBuilds, Subjects: kvOf(BuildsBucket), Writes: isController},
{State: "the rollouts of the chain", Writer: "controller (lease holder), compare-and-set on the key's revision",
KeptIn: "key-value " + RolloutsBucket + ", sixty-four revisions of each key, none removed",
Others: "read through `reads` as mesh-controller." + ChainRollouts, Subjects: kvOf(RolloutsBucket), Writes: isController},
// The grants issued (novox/hq ADR 0528 clause 2, issue 1394): the controller's, read by the assigner for its sixth rule.
{State: "the grants issued", Writer: "controller (lease holder), once the bus's user list carrying a module's manifest at a fingerprint was sent",
KeptIn: "key-value " + GrantsBucket + ", one key a module's fingerprint, none removed",
Others: "read through `reads` as mesh-controller." + ControllerGrants, Subjects: kvOf(GrantsBucket), Writes: isController},
// What each machine has of each module (novox/hq ADR 0429 §1, design 47): written by that machine's node-engine under
// its own node's keys, never by the controller; the view and the watchman read it.
{State: "a machine's have", Writer: "the node-engine of the machine, under its own keys `<node>.>`",
KeptIn: "key-value " + delivery.HaveBucket + ", twenty revisions of each key",
Others: "read through `reads` as node-engine.have", Subjects: kvOf(delivery.HaveBucket), Writes: ownNodeKeys},
// The contents of a declaration's files over 64 KiB (novox/hq ADR 0486 §2): the controller alone writes the bucket, each
// object under the digest of its content; a node-engine reads one by its name, with no list and no write.
{State: "a declaration's large files", Writer: "controller, each object under the sha256 digest of its content; the " +
"operator's account on release 6's hand path alone (ADR 0539 clause 10)",
KeptIn: "the object bucket " + DeclarationFilesBucket + "-<node> of each node, an object removed a day after the " +
"node's latest declaration no longer names it",
Others: "a node-engine reads an object of its own bucket by its name and may neither list the bucket nor write it",
Subjects: []string{"$O.*.>"}, Writes: isControllerOrTheHandPath},
{State: "the declared-config mirror", Writer: "controller (lease holder), from its store, only on a change",
KeptIn: "key-value " + DeclaredConfigBucket, Others: "read through `reads` as mesh-controller." + MirrorDeclaredConfig,
Subjects: kvOf(DeclaredConfigBucket), Writes: isController},
{State: "the predefined-keys mirror", Writer: "controller (lease holder), from its store, only on a change",
KeptIn: "key-value " + PredefinedBucket, Others: "read through `reads` as mesh-controller." + MirrorPredefined,
Subjects: kvOf(PredefinedBucket), Writes: isController},
{State: "the declared-packages mirror", Writer: "controller (lease holder), from the composition of each node, only on a change",
KeptIn: "key-value " + DeclaredPackagesBucket, Others: "read through `reads` as mesh-controller." + MirrorDeclaredPackages,
Subjects: kvOf(DeclaredPackagesBucket), Writes: isController},
// What a machine's provisioner client placed of a module's files (novox/hq ADR 0349 §4, design 57 P6): its own
// machine's word alone, which a walk's end and a gate wait for (ADR 0362, ADR 0349 §15). Kept newest per module
// in the stream ProvAppliedStream, and by the client in the provisioner server's `applied` bucket.
{State: "a module's files placed on a machine", Writer: "that machine's provisioner client",
KeptIn: "the bus, the stream " + ProvAppliedStream + ", newest per module; and key-value " +
ProvisionerAppliedBucket,
Others: "the controller reads it at a walk's end and a gate; a failure is that machine's failed apply",
Subjects: []string{"prov.applied.*.>"}, Writes: ownProvisioner},
}
// CheckWriters refuses a grant that lets a principal publish on a subject the writers table gives
// another writer (to-be 45 §1): which state, whose, and the pattern that would make a second writer.
func CheckWriters(p Principal, publish []string) error {
var problems []string
for _, pattern := range publish {
for _, row := range WritersTable {
if row.Writes == nil {
continue
}
for _, subject := range row.Subjects {
if !SubjectsOverlap(pattern, subject) || row.Writes(p, pattern) {
continue
}
problems = append(problems, fmt.Sprintf("%s may publish %s, which writes %s (%s), whose writer is %s",
p.Username(), pattern, row.State, subject, row.Writer))
}
}
}
if len(problems) == 0 {
return nil
}
return fmt.Errorf("a second writer would be granted (novox/hq to-be 45 §1, one writer per piece of state):\n %s",
strings.Join(problems, "\n "))
}
// SubjectsOverlap says some subject matches both patterns: `*` is one token, `>` one or more to the end.
func SubjectsOverlap(a, b string) bool {
x, y := strings.Split(a, "."), strings.Split(b, ".")
for i := 0; ; i++ {
switch {
case i == len(x) && i == len(y):
return true
case i == len(x) || i == len(y):
return false
case x[i] == ">" || y[i] == ">":
return true
case x[i] == "*" || y[i] == "*" || x[i] == y[i]:
continue
default:
return false
}
}
}
// isControllerOrTheHandPath is the controller, or the operator's account (a person who may invoke every tool) writing
// release 6's hand path: the one second writer ADR 0539 clauses 2 and 10 name, for a composition made by hand.
func isControllerOrTheHandPath(p Principal, pattern string) bool {
return isController(p, pattern) || (p.Kind == KindPerson && administers(p.Invokes))
}
+25 -46
View File
@@ -9,7 +9,26 @@ package catalogue
// DeliverySeat is the seat mesh-delivery holds.
const DeliverySeat = "mesh-delivery"
// deliveryVerbs are the delivery seat's tools: six that read, and the acts of a person, a reviewer and healer H2.
// RemovedDeliveryVerbs are the delivery seat's verbs release 6 removed (novox/hq ADR 0539 clause 17): with no plan the
// gate keeps the merge's states and derives nothing after the merge. Each stays in deliveryVerbs as an optional row
// with no input, so the holder of before and the holder of release 6 both hold the seat and a caller is answered by
// name with its replacement, given here.
var RemovedDeliveryVerbs = map[string]string{
"times": "the delivery view (nox-delivery): each machine's want and have, and since when",
"what-if": "the merge check's composition, which composes every machine a change touches",
"release": "the assigner's asks (hold, restore, backup, record), answered on the phone or in the delivery view",
"stop": "a bad fingerprint is held and its canary set back by the assigner (ADR 0429 §6, ADR 0539 clause 18)",
"retire-history": "nothing to retire: the gate keeps no delivery after its merge",
}
// deliveryRemovedInR6 is the row of a verb release 6 removed from the delivery seat.
func deliveryRemovedInR6(name string) Verb {
return Verb{Name: name, Description: "Removed in R6 (ADR 0539): refused by name, naming its replacement: " +
RemovedDeliveryVerbs[name], Input: schema(nil, nil), Optional: true}
}
// deliveryVerbs are the delivery seat's tools: what reads the merges and checks, and the acts of a person and healer H2
// on a merge (ADR 0539 clause 17).
func deliveryVerbs() []Verb {
return []Verb{
{Name: "deliveries", Description: "Every delivery not final, and those that ended in the last day, one line " +
@@ -18,15 +37,7 @@ func deliveryVerbs() []Verb {
Input: schema(map[string]string{"state": "one state, e.g. delivering or held",
"repository": "owner/repository", "group": "a group's id (its branch name)",
"all": "\"true\": the final ones of the last thirty days too"}, nil, "all")},
// Delivery time against the delivery budgets (novox/hq ADR 0282): what probe D16 reads, optional until its
// holder serves it.
{Name: "times", Description: "Delivery time: from a merge to every machine of its walk running the build. " +
"Each class's delivery budget (a leaf module three minutes, a core module ten), the last days' median and " +
"worst, how many within and over, the newest delivery of each class, every delivery over its budget with its " +
"longest phase, and the delivered ones whose time is not known. Given a delivery's id, its time phase by phase.",
Input: schema(map[string]string{"days": "how many days back (default 7)", "id": "one delivery's id: its phases"}, nil),
// Optional until mesh-delivery serves it: its holder lives in the catalogue (design 33 §7).
Optional: true},
deliveryRemovedInR6("times"),
{Name: "show", Description: "One delivery or group whole: its delivery plan (what it builds, what each " +
"machine receives, what is not an ordinary send), every transition with when and why, the machine " +
"steps of its walk, its group and its order.",
@@ -35,12 +46,7 @@ func deliveryVerbs() []Verb {
"(declared, built by, version skew, engine before controller, by name), its composed check and its " +
"state, derived from its members.",
Input: schema(map[string]string{"all": "\"true\": the groups that ended too"}, nil, "all")},
{Name: "what-if", Description: "The delivery plan a change would have, asked of the controller's planner " +
"and kept nowhere: the modules it moves and their dependents in tiers, what each machine would receive, " +
"the steps that are not an ordinary send.",
Input: schema(map[string]string{"repository": "owner/repository",
"paths": "the files it changes, comma-separated, from the repository's root",
"base": "the branch it merges into (default main)"}, []string{"repository", "paths"})},
deliveryRemovedInR6("what-if"),
{Name: "checks", Description: "What the mesh's checks said of a pull request's head or of one commit: each " +
"of the commit's mesh statuses (mesh/check, the one a main requires from R4e, hq's mesh/repo-check, " +
"mesh/delivery, …) with its state, " +
@@ -62,40 +68,13 @@ func deliveryVerbs() []Verb {
"controller is asked for its check. With why.",
Input: schema(map[string]string{"id": "the delivery's id", "why": "why, kept with the transition"},
[]string{"id", "why"})},
{Name: "release", Description: "A person's word that a held delivery goes on: it starts delivering. With why.",
Input: schema(map[string]string{"id": "the delivery's id", "why": "why, kept with the transition"},
[]string{"id", "why"})},
{Name: "stop", Description: "Stop a delivery that is not final, with why; its walk is ended through the " +
"controller, and in a group every member after it is stopped too, naming it.",
Input: schema(map[string]string{"id": "the delivery's id", "why": "why, kept with the transition"},
[]string{"id", "why"})},
deliveryRemovedInR6("release"),
deliveryRemovedInR6("stop"),
{Name: "close", Description: "Healer H2's verb: take the one transition the table names for a delivery held " +
"past its bound. Refused for any other.",
Input: schema(map[string]string{"id": "the delivery's id", "why": "the condition it answers"},
[]string{"id", "why"})},
{Name: "retire-history", Description: "Retire held deliveries that are history, on a person's word, with " +
"why: each made from the forge's word for a merge on the trunk heard long after it happened — replayed " +
"when the holder first held the seat — with no walk of it open. Never released, delivered or put back: " +
"retired, final. One by id, refused when the rule does not take it; or every held one, of one " +
"repository when named, answering how many it retired and naming those it left held. dry \"true\" " +
"answers what it would do and changes nothing.",
Input: schema(map[string]string{"why": "why, kept with every transition",
"id": "one delivery's id; every held one when empty",
"repository": "owner/repository: only its held deliveries",
"dry": "\"true\": what it would retire, changing nothing"}, []string{"why"}, "dry"),
// Added after the seat's holder shipped (novox/hq issue 313): optional until mesh-delivery serves it.
Optional: true},
{Name: "mark-ready", Description: "A reviewer's word that a delivery's head is approved: the holder puts the " +
"label mesh/ready on its pull request and then writes \"ready at <head>\", the order the forge counts, " +
"as the mesh's own forge account, naming who reviewed it and why. Refused when the delivery's commit is " +
"no longer its pull request's head. mesh-delivery then merges it once its checks pass (novox/hq ADR 0364).",
Input: schema(map[string]string{"id": "the delivery's id (owner/repository@commit): the head reviewed",
"reviewed-by": "who reviewed it, as who@node", "why": "the review's verdict in a line"},
[]string{"id", "reviewed-by", "why"}),
// Kept one more release, optional (novox/hq issue 1022): mesh-delivery no longer serves it, but the manifest
// it still has on the mesh does until its catalogue member merges, and a seat refuses a holder that serves
// a verb it does not promise. The next release removes this entry.
Optional: true},
deliveryRemovedInR6("retire-history"),
}
}
-104
View File
@@ -58,107 +58,3 @@ func TestTheDeliverySeatPromisesChecks(t *testing.T) {
t.Fatalf("a holder without %s: %v", before[0], err)
}
}
// The seat retires history on a person's word (novox/hq issue 313): `retire-history`, with why, one delivery by
// id or every held one, dry to see first. Added after the holder shipped, it is optional, so the holder that
// does not serve it yet still holds the seat, and one that does is not refused for a verb the seat lacks.
func TestTheDeliverySeatPromisesRetireHistoryOptionally(t *testing.T) {
seat, _ := SeatNamed(DeliverySeat)
var retire *Verb
for i := range seat.Serves {
if seat.Serves[i].Name == "retire-history" {
retire = &seat.Serves[i]
}
}
if retire == nil || !retire.Optional {
t.Fatalf("the delivery seat promises %v, retire-history optionally", VerbNames(seat.Serves))
}
props, _ := retire.Input["properties"].(map[string]any)
for _, arg := range []string{"why", "id", "repository", "dry"} {
if _, has := props[arg]; !has {
t.Errorf("retire-history takes no %q", arg)
}
}
if req, _ := retire.Input["required"].([]string); !reflect.DeepEqual(req, []string{"why"}) {
t.Errorf("retire-history requires %v, wanted only why", req)
}
var without []string
for _, v := range VerbNames(seat.Serves) {
if v != "retire-history" {
without = append(without, v)
}
}
m := Manifest{Module: "mesh-delivery", Claims: []Claim{{Name: DeliverySeat, Scope: ScopeMesh, Serves: without}}}
if err := CanHold(m, seat); err != nil {
t.Fatalf("a holder without retire-history yet: %v", err)
}
m.Claims[0].Serves = VerbNames(seat.Serves)
if err := CanHold(m, seat); err != nil {
t.Fatalf("a holder serving retire-history: %v", err)
}
}
// The seat says delivery times (novox/hq ADR 0282, issue 382): `times`, over some days or for one delivery. Added
// after the holder shipped, it is optional, so the holder that does not serve it yet still holds the seat, and one
// that does is not refused for a verb the seat lacks.
func TestTheDeliverySeatPromisesTimesOptionally(t *testing.T) {
seat, _ := SeatNamed(DeliverySeat)
var times *Verb
for i := range seat.Serves {
if seat.Serves[i].Name == "times" {
times = &seat.Serves[i]
}
}
if times == nil || !times.Optional {
t.Fatalf("the delivery seat promises %v, times optionally", VerbNames(seat.Serves))
}
props, _ := times.Input["properties"].(map[string]any)
for _, arg := range []string{"days", "id"} {
if _, has := props[arg]; !has {
t.Errorf("times takes no %q", arg)
}
}
var without []string
for _, v := range VerbNames(seat.Serves) {
if v != "times" {
without = append(without, v)
}
}
m := Manifest{Module: "mesh-delivery", Claims: []Claim{{Name: DeliverySeat, Scope: ScopeMesh, Serves: without}}}
if err := CanHold(m, seat); err != nil {
t.Fatalf("a holder without times yet: %v", err)
}
m.Claims[0].Serves = VerbNames(seat.Serves)
if err := CanHold(m, seat); err != nil {
t.Fatalf("a holder serving times: %v", err)
}
}
// The seat still promises `mark-ready`, optionally, for one more release (issue 1022); it was added by novox/hq issue 913: `mark-ready`, with the delivery's id, who reviewed it
// and why, so a reviewer marks a pull request through one audited verb, label and comment in the order the forge
// counts, instead of two raw forge calls. Added after the holder shipped, it is optional.
func TestTheDeliverySeatPromisesMarkReadyOptionally(t *testing.T) {
seat, _ := SeatNamed(DeliverySeat)
var mark *Verb
for i := range seat.Serves {
if seat.Serves[i].Name == "mark-ready" {
mark = &seat.Serves[i]
}
}
if mark == nil || !mark.Optional {
t.Fatalf("the delivery seat promises %v, mark-ready optionally", VerbNames(seat.Serves))
}
if req, _ := mark.Input["required"].([]string); !reflect.DeepEqual(req, []string{"id", "reviewed-by", "why"}) {
t.Errorf("mark-ready requires %v, wanted id, reviewed-by and why", req)
}
var without []string
for _, v := range VerbNames(seat.Serves) {
if v != "mark-ready" {
without = append(without, v)
}
}
m := Manifest{Module: "mesh-delivery", Claims: []Claim{{Name: DeliverySeat, Scope: ScopeMesh, Serves: without}}}
if err := CanHold(m, seat); err != nil {
t.Fatalf("a holder without mark-ready yet: %v", err)
}
}
+4 -4
View File
@@ -170,10 +170,10 @@ func TestAMembershipForTheNewBusIsComposedAsASealedFile(t *testing.T) {
func TestAStoreRowWithoutAProtocolKeepsTheCompiledOne(t *testing.T) {
was := Seats()
t.Cleanup(func() { UseSeats(was) })
UseSeats([]Seat{{Name: "mesh-build-machine", Scope: ScopeMesh, Decision: "row"}})
got, ok := SeatNamed("mesh-build-machine")
if !ok || len(got.Accepts) == 0 {
t.Fatalf("the build machine's seat lost what it accepts when loaded from the store: %+v", got)
UseSeats([]Seat{{Name: "node-build-go", Scope: ScopeNode, Decision: "row"}})
got, ok := SeatNamed("node-build-go")
if !ok || len(got.Accepts) == 0 || !got.PulledByKind {
t.Fatalf("the Go build seat lost what it accepts when loaded from the store: %+v", got)
}
if got.Decision != "row" {
t.Fatalf("the store's own columns were not kept: %+v", got)
+29
View File
@@ -0,0 +1,29 @@
package catalogue
import (
"os"
"testing"
)
// mesh-sdk's own manifest is judged by the controller that runs when it is checked, and that controller reads it
// strictly: a manifest word ships one release ahead of its first user. R4e's group is judged by today's controller
// (novox/hq ADR 0503 §4), which does not know `paths` (ADR 0499), so mesh-sdk's lib names no paths until a release
// after R4e's delivery adds them back (ADR 0499 §6, issue 1758).
func TestOwnManifestUsesNoWordTheRunningControllerRefuses(t *testing.T) {
raw, err := os.ReadFile("../../module.json")
if err != nil {
t.Fatal(err)
}
m, err := ParseManifest(raw)
if err != nil {
t.Fatalf("mesh-sdk's module.json does not parse: %v", err)
}
if m.Build == nil || len(m.Build.Artifacts) == 0 {
t.Fatal("mesh-sdk's module.json declares no artifact")
}
for _, a := range m.Build.Artifacts {
if a.Paths != nil {
t.Errorf("artifact %q names paths %v: the running controller refuses the word until R4e is delivered", a.Name, a.Paths)
}
}
}
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the repository's root files or other directories): mesh-build-go never keeps this package's verdict (issue 1935).
+50 -41
View File
@@ -45,6 +45,10 @@ type Seat struct {
// today — which the bus refuses, because a namespace belongs to who it is named for.
Accepts []string
Emits []string
// PulledByKind says each accepted kind is a worker of its own on the seat's stream, which its holders pull in the
// order Accepts lists them (novox/hq ADR 0525 §2, ADR 0536 §4): the build seats'. Compiled, never stored, like
// Receives.
PulledByKind bool
// Serves carries each verb in full — name, description, schema — because a role's tools are the
// mesh's to define and an agent's to call (novox/hq ADR 0132, design 33 §2).
Serves []Verb
@@ -106,12 +110,18 @@ var defaultSeats = append([]Seat{
"rolled-back",
// A pull request's merge check, judged (novox/hq to-be 45 §9).
"checked",
// A walk kept in a new state, for the delivery it walks (novox/hq ADR 0239).
// A walk kept in a new state, and a tier sent to a machine: kept until the catalogue's main consumes them no
// more; nothing emits them (ADR 0539).
"plan-moved",
// A tier sent to a machine, once the send is kept (novox/hq ADR 0406 §5, issue 974).
"sent",
// A key of a record mirror written or removed (novox/hq ADR 0289, design 54 §3; design 57 F3).
"assignment-changed", "declared-config-changed", "predefined-changed"},
"assignment-changed", "declared-config-changed", "predefined-changed",
// A builder's key not taken in because the artifact store does not hold an artifact it names (novox/hq
// issue 1937): the builder builds it again.
"artifacts-gone",
// A bucket of a state this release deleted, deleted at the controller's first start (novox/hq ADR 0539
// clause 13): the merge window's, the silent sends' and the building plans' hold, each said once.
"bucket-deleted"},
Serves: ControllerVerbs},
// The store's first verbs (novox/hq ADR 0159): the smallest set that makes the store askable,
// served by whichever module holds the seat with tools of these names.
@@ -238,31 +248,19 @@ var defaultSeats = append([]Seat{
// They keep their names until that migration is done deliberately, apart from the node-* pass.
{Name: "npm-package-registry", Scope: ScopeMesh, Delivers: "npm-package-registry", Decision: "novox/hq ADR 0109"},
{Name: "git", Scope: ScopeMesh, Delivers: "git", Decision: "novox/hq ADR 0111"},
// A build is work submitted to this role and its outcome is the role's own event (ADR 0129).
// One publish reaches whoever asked, the controller that records it, and the catalogue that
// places it in the graph — what the old bus's shared exchange did for free.
// A build says what it does as it does it (novox/hq ADR 0157): `started` when work is taken,
// `log.<build id>` for every line, `built` for the outcome. The log's tail token is the build's
// id, so a reader follows one build by subject alone.
// **Node-scoped, and every holder takes from one queue** (novox/hq ADR 0190): a build is asked of
// the role, and whichever machine holding the seat is idle pulls it. One holder per machine is
// what the scope says; sharing the work is what a seat's queue has always done.
//
// **And its holder answers for the build it is running** (novox/hq ADR 0219): what it is
// building, kill it, take nothing new, take again. The queue as a whole is the controller's to
// show and change (`queue`, `cancel`, `clear`); what one machine does with an ask it already
// took only that machine can do. `paused.<node>` is each holder saying whether it takes work, so
// the controller can tell a plan waiting on a paused seat from one that is late.
{Name: "node-build-agent", Scope: ScopeNode,
Accepts: []string{"build"}, Emits: []string{"started", "built", "log.*", "paused.*"},
Serves: buildAgentVerbs(),
Decision: "novox/hq ADR 0190, ADR 0219"},
// **Retired by ADR 0190, kept while a manifest still claims it.** The one build machine's seat.
// A claim to a seat the mesh no longer defines is refused, and the module holding this one is
// assigned on a live machine until build-agent replaces it — removing the row first would make
// that machine unresolvable in the meantime. Deleted once no registered manifest claims it.
{Name: "mesh-build-machine", Scope: ScopeMesh,
Accepts: []string{"build"}, Emits: []string{"started", "built", "log.*"}, Decision: "novox/hq ADR 0190"},
// **A build node's work, one seat per build language** (novox/hq ADR 0525 §2, §10; ADR 0536 §4; issue 1887). The
// build module of each language claims its seat on every node it runs on; the builder and the checker put their
// asks on the seat's accepted kinds, which the broker derives as one work queue with a shared worker per kind, and
// each holder pulls delivery first, then check, then recheck, at most its node's build capacity at once. Its verbs
// say what the node runs and pause or resume it there: every holder pulls from the same worker, so a node's pause is
// its holder taking nothing, kept on the module's own pause key (issue 1887's design limit 2). They replace
// node-build-agent, retired with build-agent (ADR 0524 §5).
{Name: "node-build-go", Scope: ScopeNode, Accepts: buildAskKinds(), PulledByKind: true, Serves: buildSeatVerbs("Go"),
Decision: "novox/hq ADR 0525, ADR 0536"},
{Name: "node-build-node", Scope: ScopeNode, Accepts: buildAskKinds(), PulledByKind: true, Serves: buildSeatVerbs("Node"),
Decision: "novox/hq ADR 0525, ADR 0536"},
{Name: "node-build-image", Scope: ScopeNode, Accepts: buildAskKinds(), PulledByKind: true, Serves: buildSeatVerbs("image"),
Decision: "novox/hq ADR 0525, ADR 0536"},
// **The mesh's resolvers** (novox/hq ADR 0194, 0196, 0223): every node's internal domain, and
// every node and container asks them and nothing else. Replicated since ADR 0223: held on more
// than one machine, each answering the same names from the same roster, and every machine's
@@ -498,6 +496,7 @@ func UseSeats(s []Seat) {
row.Receives = d.Receives
row.Paths = d.Paths
row.Replicated = d.Replicated
row.PulledByKind = d.PulledByKind
row.Serves = optionalAsCompiled(row.Serves, d.Serves)
}
merged = append(merged, row)
@@ -623,6 +622,10 @@ func claimProblems(m Manifest) []string {
// after UseSeats has replaced the set with the store's.
continue
}
if replaced, retired := RetiredSeats[c.Name]; retired {
problems = append(problems, fmt.Sprintf("%s claims the seat %q, which is retired: its work is %s", m.Module, c.Name, replaced))
continue
}
if isSystemSeatName(c.Name) {
problems = append(problems, fmt.Sprintf(
"%s claims %q, which is a seat in the mesh's own namespace (mesh-*/node-*) that it "+
@@ -903,22 +906,28 @@ func loginShellVerbs() []Verb {
}
}
// buildAgentVerbs are what a holder of node-build-agent answers on its own machine (novox/hq ADR
// 0219). One build at a time per holder (ADR 0190), so "the build running here" is one or none.
func buildAgentVerbs() []Verb {
// buildAskKinds are a build seat's accepted kinds, in the order a holder pulls them (ADR 0536 §4; mesh-sdk
// delivery.AskKinds).
func buildAskKinds() []string { return []string{"delivery", "check", "recheck"} }
// RetiredSeats are seats the mesh no longer defines, each with what replaced it: a claim or a use of one is refused
// naming its replacement, not as a seat nobody declares.
var RetiredSeats = map[string]string{
"node-build-agent": "node-build-go, node-build-node and node-build-image, held by the build modules (novox/hq ADR 0525 §10)",
"mesh-build-machine": "node-build-go, node-build-node and node-build-image, held by the build modules (novox/hq ADR 0190, ADR 0525 §10)",
}
// buildSeatVerbs are what a build module answers on its own node for its seat (ADR 0525 §10).
func buildSeatVerbs(language string) []Verb {
return []Verb{
{Name: "current", Description: "The build this machine is running — its id, repository, path, " +
"the step it is at, when it started and for how long — or none; and whether this machine is " +
"paused, taking no new build.",
{Name: "current", Description: "The " + language + " asks this node runs now (each ask's id, kind, module and when it " +
"started), its build capacity, and whether it is paused, taking no new ask.",
Input: schema(map[string]string{}, nil)},
{Name: "kill", Description: "End the build with this id, running here: its commands and the " +
"containers it started are stopped, and its outcome is announced as failed, killed by hand — " +
"settled, so it is not handed to another machine.",
Input: schema(map[string]string{"id": "the build's id, as `queue` or `current` says it"}, []string{"id"})},
{Name: "pause", Description: "Take no new build on this machine until resumed; a build running " +
"here finishes. Kept across a restart of the holder.",
{Name: "pause", Description: "Take no new " + language + " ask on this node until resumed; an ask running here " +
"ends in its place. Kept across a restart of the build module (its pause key). A capacity of 0 is the " +
"lasting way to keep a node from building.",
Input: schema(map[string]string{}, nil)},
{Name: "resume", Description: "Take builds again on this machine.",
{Name: "resume", Description: "Take " + language + " asks again on this node.",
Input: schema(map[string]string{}, nil)},
}
}
+8
View File
@@ -364,6 +364,10 @@ func CatalogueProblems(shelf Shelf) []string {
// A `uses` naming nothing is where ADR 0110's guarantee lands under a derived set: the
// same refusal, at the same moment, from a set nobody maintains by hand.
for _, u := range m.Uses {
if replaced, retired := RetiredSeats[u]; retired && !exists(u) {
problems = append(problems, fmt.Sprintf("%s uses the seat %q, which is retired: its work is %s", module, u, replaced))
continue
}
if !exists(u) {
problems = append(problems, fmt.Sprintf(
"%s uses the seat %q, "+UndeclaredSeat,
@@ -372,6 +376,10 @@ func CatalogueProblems(shelf Shelf) []string {
}
for _, c := range m.Claims {
if replaced, retired := RetiredSeats[c.Name]; retired && !exists(c.Name) {
problems = append(problems, fmt.Sprintf("%s claims the seat %q, which is retired: its work is %s", module, c.Name, replaced))
continue
}
if !exists(c.Name) {
problems = append(problems, fmt.Sprintf(
"%s claims the seat %q, "+UndeclaredSeat,
+36 -3
View File
@@ -43,7 +43,8 @@ func TestTheSeatsAreAClosedSetAndEachNamesItsDecision(t *testing.T) {
delivered[s.Delivers] = s.Name
}
}
// Forty-five with node-files (novox/hq ADR 0394); forty-four again once node-reads goes.
// Forty-six with the three build seats replacing node-build-agent and the retired mesh-build-machine (novox/hq
// ADR 0525 §2, §10); forty-five with node-files (novox/hq ADR 0394); forty-four again once node-reads goes.
// Forty-four with node-provisioner (novox/hq ADR 0349 clause 3); forty-three with mesh-provisioner (novox/hq ADR 0349 clause 3); forty-two with mesh-config (novox/hq ADR 0348 §9); forty-one with node-reads (novox/hq ADR 0310); forty with node-nfs-server and node-mounts (novox/hq ADR 0263); thirty-eight with node-resolver (novox/hq ADR 0247); thirty-seven with mesh-delivery (novox/hq ADR 0239); thirty-six since node-resolver-config retired
// into node-uplink (novox/hq ADR 0223); thirty-seven
// since the retired node-dns-resolver went (novox/hq ADR 0220); thirty-eight with
@@ -55,8 +56,8 @@ func TestTheSeatsAreAClosedSetAndEachNamesItsDecision(t *testing.T) {
// node-container-runtime (ADR 0207); nineteen with node-environment and node-login-shell (ADR 0203,
// ADR 0204); seventeen with node-build-agent (ADR 0190). One fewer once the retired
// mesh-build-machine row goes, when no registered manifest claims it.
if len(Seats()) != 45 {
t.Errorf("the mesh defines %d seats rather than 45; the set is closed, so a change here is "+
if len(Seats()) != 46 {
t.Errorf("the mesh defines %d seats rather than 46; the set is closed, so a change here is "+
"a decision (novox/hq ADR 0110): %s", len(Seats()), seatNames())
}
}
@@ -300,3 +301,35 @@ func TestAFormerNameResolvesAfterARename(t *testing.T) {
t.Fatalf("the holder recorded under the former name was not matched: %+v found=%v", holder, found)
}
}
// 1887/A9a: the catalogue defines no seat node-build-agent, and a manifest claiming it is refused naming its
// replacement; the build modules' seats accept the three kinds in their pull order and serve current, pause, resume.
func TestNodeBuildAgentIsRetiredForTheBuildSeats(t *testing.T) {
for _, gone := range []string{"node-build-agent", "mesh-build-machine"} {
if _, ok := SeatNamed(gone); ok {
t.Errorf("the mesh still defines %s", gone)
}
}
_, err := ParseManifest([]byte(`{"module":"build-agent","version":"1","claims":[{"name":"node-build-agent","scope":"node"}]}`))
if err == nil || !strings.Contains(err.Error(), `"node-build-agent", which is retired`) || !strings.Contains(err.Error(), "node-build-go") {
t.Fatalf("the claim is not refused naming its replacement: %v", err)
}
// And over a catalogue whose store still holds no row for it, a use of it is refused the same way.
user := Manifest{Module: "watcher", Uses: []string{"node-build-agent"}}
if joined := strings.Join(CatalogueProblems(Shelf{user.Module: user}), "; "); !strings.Contains(joined, "which is retired: its work is node-build-go") {
t.Fatalf("a use is not refused naming the replacement: %s", joined)
}
for _, name := range []string{"node-build-go", "node-build-node", "node-build-image"} {
s, ok := SeatNamed(name)
if !ok || s.Scope != ScopeNode || !s.PulledByKind || strings.Join(s.Accepts, ",") != "delivery,check,recheck" {
t.Fatalf("%s = %+v", name, s)
}
var verbs []string
for _, v := range s.Serves {
verbs = append(verbs, v.Name)
}
if strings.Join(verbs, ",") != "current,pause,resume" {
t.Errorf("%s serves %v", name, verbs)
}
}
}
+4 -2
View File
@@ -10,7 +10,9 @@ import (
// the provision, and does not find out at its first build. The manifests are the catalogue's, as they are.
// builderNode is a machine that can run a build module: the package manager and user namespaces it asks of
// the machine, and the forge account (mesh-scope, provided by the forge) stood in for.
// the machine, and the forge account (mesh-scope, provided by the forge) and the artifact store the build
// modules publish to (ADR 0513, provided by distribution) stood in for, and node-tools, the tool runner they pull their
// asks through (mesh-catalog issue 1911; mesh-tools' module, not in the catalogue).
func builderNode() Node {
return Node{Name: "builder", Site: "house",
Capabilities: map[string]bool{"package-manager": true, "user-namespaces": true}}
@@ -19,7 +21,7 @@ func builderNode() Node {
func toolchainShelf(t *testing.T) map[string]Manifest {
t.Helper()
m := map[string]Manifest{
"forge": {Module: "forge", Version: "1", Provides: []Offer{{Name: "forge-account"}, {Name: "npm-package-registry"}}},
"forge": {Module: "forge", Version: "1", Provides: []Offer{{Name: "forge-account"}, {Name: "npm-package-registry"}, {Name: "artifact-store"}, {Name: "node-tools"}}},
}
for _, name := range []string{"go", "nodejs", "mesh-build-go", "mesh-build-node", "dev-tools"} {
m[name] = catalogueManifest(t, name)
+67 -139
View File
@@ -74,6 +74,29 @@ func VerbNames(verbs []Verb) []string {
return out
}
// RemovedControllerVerbs are the controller seat's verbs release 6 removed (novox/hq ADR 0539, ADR 0429 §8): no
// delivery plan, batch, tier or push exists, so nothing starts, stops, retries, follows or reads one. Each stays in
// ControllerVerbs as a row with no input, so a caller is answered by name with its replacement, given here.
var RemovedControllerVerbs = map[string]string{
"plans": "the delivery view (nox-delivery): each machine's want and have, read from the assigner's wants and the " +
"node-engine's haves",
"plan": "the node's compositions: mesh-controller_composed holds each module's digest and judgement per node, and " +
"mesh-hand-compose prints one module's composition for one node",
"push": "a want: the assigner writes want.<node>.<module> by its rules, and by hand the operator's credential writes " +
"it, naming a kept composition (ADR 0539 clause 10)",
"deliver": "nothing to start: a merged change reaches each machine by its want",
"delivery-stop": "a bad fingerprint is held and its canary set back by the assigner (ADR 0429 §6, ADR 0539 clause 18)",
"delivery-backups-again": "the assigner's backup ask, answered on the phone or in the delivery view (ADR 0539 clause 16)",
"delivery-walks": "the delivery view (nox-delivery) and the wants, haves and compositions on the bus",
"delivery-order": "no order to keep: each module reaches its machines by its own want, its providers first by rule 5",
}
// removedInR6 is the row of a verb release 6 removed from the controller's seat: no input, refused by name.
func removedInR6(name string) Verb {
return Verb{Name: name, Description: "Removed in R6 (ADR 0539): refused by name, naming its replacement: " +
RemovedControllerVerbs[name], Input: schema(nil, nil)}
}
// ControllerSeatName is the seat the control plane holds, whose tools are the mesh's own verbs.
const ControllerSeatName = "mesh-controller"
@@ -115,87 +138,23 @@ var ControllerVerbs = []Verb{
"log": "a build's id (as `builds` lists it): print what the build machine said, line by line",
"limit": "how many builds to list (default 20); not with log",
}, nil)},
{Name: "plans", Description: "What the last merges produced and where each stands (novox/hq ADR 0162): " +
"the tiers, the tier a plan is at, what it waits for and since when; one plan whole, given its id; " +
"or, given wait, one plan followed until it moves.",
Input: schema(map[string]string{
"id": "a plan's id (as `plans` lists them): that plan, tier by tier",
"wait": "a plan's id: wait until it moves, then answer it as data — the plan, and what ended the wait " +
"(moved, tier, done, or bound); ask again to follow it further (novox/hq issue 229)",
"until": "with wait: moved (any change, the default), tier (it reaches another tier) or done (it ends)",
"bound": "with wait: how long to wait at most, as 15s or seconds — 20s, the default, is the longest, " +
"inside what a call is answered within",
"stop": "a plan's id: stop it — what was asked still builds, nothing further is asked",
"close": "a plan's id: close a plan that will not move again, as failed by hand (novox/hq issue 254)",
"retry": "a failed plan's id: ask its failed builds again under new ids, and carry the plan on from that tier (novox/hq ADR 0219)",
"go": "a walk's id that waits for its delivery's word: start it by hand, in place of mesh-delivery — with why, " +
"recorded in the hand-act log (novox/hq ADR 0239)",
"repository": "owner/repository: the plan a merge there would produce, saving nothing (what-if); with paths or modules",
"paths": "with repository: the files the merge would change, comma-separated, from the repository's root",
"modules": "with repository: or the modules it would change, comma-separated",
"module-dirs": "with repository and paths: the directories holding a module.json at the commit, comma-separated, " +
"as the forge's announcer says them (novox/hq issue 278); without it, a directory is a module only when the paths hold its manifest",
"data": "with repository and paths: true answers the change plan as data, the document delivery-plan answers " +
"(moved, dependents, new, tiers, machines with what each receives and waits for, steps, summary, class, head, base); " +
"without it the answer is text (novox/hq ADR 0450 §2)",
"head": "with data: the commit at hand",
"base": "with data: the branch it merges into (default main)",
"removed": "with data: the files among paths the change deletes, comma-separated",
"limit": "how many plans to list (default 10); only when listing",
"why": "with stop, close or go: why it is done by hand — required, and recorded in the hand-act log (novox/hq to-be 45 §7)",
"cause": "with stop, close or go: the cause in a word, or a condition's kind (optional)",
}, nil, "data")},
removedInR6("plans"),
// The verbs the delivery's owner asks with (novox/hq ADR 0239): the planner's answer, a group's order and
// composed check, and the start and end of a walk. Callable by a person too; mesh-delivery is who needs them.
{Name: "delivery-plan", Description: "Removed in R4e (ADR 0501 §5 / ADR 0503 §3): refused by name, naming its replacement",
Input: schema(nil, nil)},
{Name: "delivery-order", Description: "A delivery group's order (novox/hq ADR 0419): its members in the order " +
"they are delivered, every pair and why — built by, verb before caller, version skew, engine before " +
"controller — read from the module graph and the manifests' change alone; no `after:` line is read. The " +
"cycle, with every module and edge of it, is the one refusal.",
Input: schema(map[string]string{"members": "the members, as JSON: [{id, repository, base, head, number, paths, " +
"module_dirs, module_dirs_said, removed}]"}, []string{"members"})},
removedInR6("delivery-order"),
{Name: "delivery-check", Description: "Answers that the checker (mesh-delivery-checker) judges a group (novox/hq " +
"ADR 0503 §3); asks nothing. Its inputs stay, since hq's repository check still names it (ADR 0513 §4).",
Input: schema(map[string]string{"group": "the group's id (none for one head checked again)",
"members": "the members, as JSON, as delivery-order takes them",
"ready": "\"true\" when the pull request is marked ready: its check is queued before every other check (novox/hq ADR 0406 §3)"},
[]string{"members"})},
{Name: "deliver", Description: "The delivery's word that a walk waiting for it may start (novox/hq ADR 0239): its " +
"first tier is asked at the controller's next pass. Refused for a walk that waits for nobody.",
Input: schema(map[string]string{"plan": "the walk's id", "why": "what lets it go: its turn, a person's release"},
[]string{"plan"})},
{Name: "delivery-stop", Description: "End a walk on its delivery's word (novox/hq ADR 0239): failed, said as stopped " +
"by whom and why; what it asked still builds and registers, nothing further is asked or sent. Given merges to " +
"leave out, every other merge the walk answered is walked again — in the open batch of its kind, or a new one — " +
"without them, without each later merge of their repository, which contains one, and without each merge moving " +
"a module one moves, each left out too and named (novox/hq ADR 0299); a batch not yet cut drops the merges. The " +
"answer names the batch, the merges walked again, the merges left out, the modules the stopped walk sent that " +
"stay on their machines, and the merges elsewhere that will still deliver a stopped change.",
Input: schema(map[string]string{"plan": "the walk's id", "why": "why", "by": "who stopped the delivery",
"without": "merges to leave out, <repository>@<commit>, comma-separated"},
[]string{"plan", "why"})},
{Name: "delivery-backups-again", Description: "Ask again the backups a walk was held for (novox/hq ADR 0404 " +
"decision 4): a walk whose backup before its send failed or did not answer is held, nothing asked or sent; this " +
"clears those failed backups and the next look at the plans asks them again, and only them. Refused for a walk " +
"with no failed backup. The delivery's release calls it. The answer names the walk and the backups asked again.",
Input: schema(map[string]string{"plan": "the held walk's id", "why": "a person's reason, as release passes it",
"by": "who released the delivery"}, []string{"plan", "why"})},
{Name: "delivery-walks", Description: "The walks the controller keeps (novox/hq ADR 0239): every open one and the " +
"last ended ones, each whole — its tiers, each module's state, first machines and first-node gate with its readings, and its " +
"phases from its merge to every machine running it (novox/hq ADR 0282) — and whether the delivery seat has a " +
"holder on record. Given a plan, that one.",
Input: schema(map[string]string{"plan": "one walk's id", "limit": "how many ended walks beside the open ones (default 50)"},
nil)},
{Name: "plan", Description: "What one machine would run, and why: the declaration the mesh would send it — " +
"or, with files, the files it would be given; or, with diff, what a push would change there against what " +
"it was last sent: resources added, removed and changed, a container's changed fields (it is recreated), " +
"and any mount whose directory on the machine would move (novox/hq ADR 0217).",
Input: schema(map[string]string{
"node": "the machine's name",
"files": "\"true\": the files this machine would be given, instead of the declaration as JSON",
"diff": "\"true\": what a push would change on this machine, against what it was last sent; not with files",
}, []string{"node"}, "files", "diff")},
removedInR6("deliver"),
removedInR6("delivery-stop"),
removedInR6("delivery-backups-again"),
removedInR6("delivery-walks"),
removedInR6("plan"),
{Name: "assign", Description: "Put a module on a machine. Refused with the mesh's own words when it cannot resolve there, " +
"or when a seat its resources are applied through is held by nothing on the machine (novox/hq ADR 0207). " +
"A module not registered yet is looked for (novox/hq issue 325, ADR 0261): while its build runs — a merge " +
@@ -221,22 +180,7 @@ var ControllerVerbs = []Verb{
}, []string{"node", "provision", "from", "module"})},
{Name: "unpin", Description: "Take that choice back, putting the question to the mesh again.",
Input: schema(map[string]string{"node": "the machine's name", "provision": "the provision"}, []string{"node", "provision"})},
{Name: "push", Description: "Send one machine everything it should be. With no machine named it is a push of the " +
"WHOLE mesh — every machine that is behind — and the answer says so first; behind says that outright. " +
"Answers at once that it is running, with a call id: `calls` with that id says what it sent and, for a " +
"machine named, how that machine applied it — applied, refused with its reason, or each failed resource — " +
"once it reports within wait (a push can reload the bus, which then refuses any answer still to come). A push by hand is a repair, " +
"and says why: recorded in the hand-act log (novox/hq to-be 45 §7). A machine whose push would give a " +
"running module's data another directory is held and named, the others sent (novox/hq ADR 0217): read " +
"`plan` with diff, and name the machine with move to send it.",
Input: schema(map[string]string{
"node": "the machine's name; without it, every machine that is behind",
"move": "with node: the modules whose data may move with this push, comma-separated (optional)",
"wait": "with node: how long the call waits for the machine to report how it applied what it was sent, as \"90s\" or \"2m\"; \"0\" waits for nothing; without it, the controller's setting push-wait (optional)",
"behind": "\"true\": every machine that is behind, the whole mesh — the same as naming none, said outright; not with node",
"why": "why this is pushed by hand: recorded in the hand-act log",
"cause": "the cause in a word, or a condition's kind — the word a second push for the same reason uses (optional)",
}, []string{"why"}, "behind")},
removedInR6("push"),
{Name: "rotate", Description: "Replace a credential. A pair credential, by provision (and a consuming machine and module, " +
"else every holder): both ends are re-sent together. Or a module's own secret, by machine, module and " +
"name: made anew and the machine sent, so the module starts again on it — for a secret its definition " +
@@ -350,36 +294,24 @@ var ControllerVerbs = []Verb{
"its shell — `node show ace`, `module list`, `plans`, `conditions show <key>` — and answer what it " +
"printed. The generic verb beside the named ones (novox/hq ADR 0154), and since ADR 0266 it only reads: " +
"status, version, help, seats, healers, hand-acts, durations, collection, images, artifacts, data, builds, " +
"queue, plan, plans (not stop/close/go), doctor (not run), conditions list/show/history, node list/show, " +
"plan, plans (not stop/close/go), doctor (not run), conditions list/show/history, node list/show, " +
"module list, settings show/preferences, retire list, cleanup list, delivery plan/walks, bus, mirrors (not " +
"--record/--confirm). What writes has its named verb; what sets a key, issues a credential or a token, or " +
"accepts, recovers or exports a secret is the controller's terminal's alone.",
Input: schema(map[string]string{
"command": "the command line, as the controller's binary takes it; quotes group a word with spaces",
}, []string{"command"})},
// The build queue, controlled by hand (novox/hq ADR 0219). Every verb that drops an ask leaves a
// failed outcome for it, so a plan waiting on it fails visibly instead of hanging.
{Name: "queue", Description: "Every ask in the build queue: waiting, in flight (on which machine, for how long), " +
"and dead (handed out as often as allowed and never settled) — each with its id, repository, path, ref and when it was asked. " +
"Answers the table as text and, as data, each ask's state, kind (check or build), repository, commit, machine, since " +
"and position among the waiting (novox/hq issue 455).",
// The build queue went with build-agent (novox/hq ADR 0525 §9): asks are the build seats' streams, their backlog the
// streams' own state, shown in the delivery view. The verbs are refused by name for one release, naming that.
{Name: "queue", Description: "Removed in R4g (ADR 0525 §9): refused by name, naming its replacement: the build seats' backlog in the delivery view",
Input: schema(nil, nil)},
{Name: "cancel", Description: "Drop one waiting or dead ask from the build queue; its outcome is recorded failed, " +
"cancelled by hand, and a plan that asked for it fails. One in flight is refused: `kill` ends it where it runs.",
Input: schema(map[string]string{"id": "the ask's build id, as `queue` lists it"}, []string{"id"})},
{Name: "clear", Description: "Cancel every waiting ask in the build queue — and with dead, every dead one too — each " +
"recorded failed, cancelled by hand. Never touches one in flight.",
Input: schema(map[string]string{"dead": "\"true\" to cancel the dead asks as well"}, nil, "dead")},
{Name: "rebuild", Description: "Removed in R4e (ADR 0501 §5 / ADR 0503 §3): refused by name, naming its replacement",
Input: schema(nil, nil)},
{Name: "replay", Description: "Ask a recorded build's repository and path again at the commit it built, under a new id: " +
"a dry run only, nothing recorded or registered (novox/hq ADR 0429 §18: a fingerprint's key is built again by the " +
"builder, never registered by hand).",
Input: schema(map[string]string{"id": "the build's id"}, []string{"id"})},
{Name: "cancel", Description: "Removed in R4g (ADR 0525 §9): refused by name, naming its replacement", Input: schema(nil, nil)},
{Name: "clear", Description: "Removed in R4g (ADR 0525 §9): refused by name, naming its replacement", Input: schema(nil, nil)},
{Name: "replay", Description: "Removed in R4g (ADR 0525 §9): refused by name, naming its replacement", Input: schema(nil, nil)},
// novox/hq issue 686: a builder ran a branch's store tests for the first time in its pull request's merge check.
{Name: "try", Description: "Try a pushed branch or commit before its pull request (novox/hq issue 686): the " +
"checker (mesh-delivery-checker) merges the head onto the repository's trunk and runs its levels on the merge, " +
"as it judges a pull request (ADR 0503 §3); no merge-check.sh is run and build-agent is not asked. Answers at " +
"as it judges a pull request (ADR 0503 §3), its levels asks on the build seats (ADR 0525 §1); no merge-check.sh is run. Answers at " +
"once with an id; asked again with that id it waits up to 20 s and answers pass, fail or error, how long it " +
"took and the lines naming what failed — or that it is still waiting or running, to ask again. Sets no " +
"status on the forge, registers and builds nothing for delivery. One try of a commit at a time — a second " +
@@ -388,23 +320,14 @@ var ControllerVerbs = []Verb{
"repository": "the repository on the forge, owner/name; with ref",
"ref": "the branch or commit to try, as pushed to the forge",
"base": "the branch it is merged into (the repository's default branch when none is named)",
"tests": "only the tests this `go test -run` pattern names, where the repository's merge-check.sh reads " +
"MESH_CHECK_TESTS; a script that does not runs whole",
"tests": "only the tests this `go test -run` pattern names",
"store-only": "\"true\": only the integration level",
"id": "instead of a repository: the id a try answered, to read what came of it",
}, nil, "store-only"),
Replaces: []string{"sh merge-check.sh", "docker run postgres"}},
// novox/hq ADR 0350 §5: asked by the delivery's owner when the forge says a pull request closed unmerged.
{Name: "forget-checks", Description: "Removed in R4e (ADR 0501 §5 / ADR 0503 §3): refused by name, naming its replacement",
Input: schema(nil, nil)},
{Name: "kill", Description: "End a build where it runs: the machine that took it stops its commands and containers and " +
"announces it failed, killed by hand — settled, never handed to another machine.",
Input: schema(map[string]string{"id": "the build's id"}, []string{"id"})},
{Name: "pause", Description: "The build seat's holder on one machine — or every holder — takes no new build until resumed; " +
"a build running finishes. Kept across a restart of the holder. A plan waiting on a paused seat says so and is not late.",
Input: schema(map[string]string{"node": "one machine; every holder when absent"}, nil)},
{Name: "resume", Description: "The build seat's holder on one machine — or every holder — takes builds again.",
Input: schema(map[string]string{"node": "one machine; every holder when absent"}, nil)},
{Name: "kill", Description: "Removed in R4g (ADR 0525 §9): refused by name, naming its replacement: a build seat's ask ends where it runs", Input: schema(nil, nil)},
{Name: "pause", Description: "Removed in R4g (ADR 0525 §9): refused by name, naming its replacement: <node>/node-build-<language>.pause", Input: schema(nil, nil)},
{Name: "resume", Description: "Removed in R4g (ADR 0525 §9): refused by name, naming its replacement: <node>/node-build-<language>.resume", Input: schema(nil, nil)},
// Acts done by hand, and what the bounds are set from (novox/hq to-be 45 §7, Phase 0).
{Name: "hand-act", Description: "Record an act done by hand outside the mesh — a container restarted, a file " +
"edited, a service started on a machine — with why and its cause, in the hand-act log beside the pushes and " +
@@ -478,26 +401,22 @@ var ControllerVerbs = []Verb{
"probe": "one probe's id (D14): its verdict alone, with every finding — the verdict shows at most twenty a probe",
}, nil, "run", "probes", "signals")},
// How a module's new builds reach its machines, and the bus's planned step (novox/hq ADR 0236).
{Name: "upgrade", Description: "How each module's new builds reach its machines (novox/hq ADR 0236): rolled " +
"out one machine first and judged there at the gate, then the rest — or recorded, waiting for a person's " +
"push — with where that comes from (a person, the module, the bus, its irreplaceable data, the default) and " +
"why. With module, that one; with policy, a person's choice for it — roll-out, record (with why) or default " +
"to take the choice back. The bus is never rolled out.",
{Name: "upgrade", Description: "Whether each module's new builds wait for a person (novox/hq ADR 0236, ADR 0539 clause 14): " +
"for a module of policy record, the assigner writes no want naming a composition judged to recreate a container or move " +
"data until the operator answers its ask; with where the policy comes from (a person, the module, the bus, its " +
"irreplaceable data, the default) and why. With module, that one; with policy, a person's choice for it — roll-out, " +
"record (with why) or default to take the choice back. The bus is never moved by the assigner (ADR 0539 clause 15).",
Input: schema(map[string]string{
"module": "one module",
"policy": "with module: roll-out, record or default",
"together": "\"true\": with roll-out, every machine at once instead of one machine first",
"why": "with policy or release-backlog: why — required for record and for a release, kept",
"backlog": "\"true\": every build waiting for a gate, per machine — what a release plan would send",
"release-backlog": "\"true\": release what waits now, one machine at a time, each judged — after a release " +
"plan failed, the mesh waits for this; with why",
"cause": "with release-backlog: the cause in a word (optional)",
}, nil, "together", "backlog", "release-backlog")},
{Name: "bus", Description: "The bus as a planned step (novox/hq to-be 45 §8, ADR 0236): what a bus upgrade " +
"would do — the bus's build on each machine against the one the mesh holds — and how the last step went. " +
"With upgrade, start one: a person's act with why, after the streams are snapshotted (snapshot-taken says " +
"where, while the mesh takes none itself), saying first whether it can be reverted; bus-maintenance is open " +
"while it runs and every stream, consumer and a round trip are checked after.",
"module": "one module",
"policy": "with module: roll-out, record or default",
"why": "with policy: why — required for record, kept",
}, nil)},
{Name: "bus", Description: "The bus as a person's step (novox/hq to-be 45 §8, ADR 0236, ADR 0539 clause 15): what a bus " +
"upgrade would do — the bus's build on each machine against the one the mesh holds — and how the last step went. " +
"With upgrade, check one: a person's act with why, after the streams are snapshotted (snapshot-taken says where, " +
"while the mesh takes none itself), saying first whether it can be reverted; then it answers the want the operator " +
"writes with the operator's credential, want.<node>.<bus module>, recorded — nothing is sent, and the assigner never " +
"moves the bus.",
Input: schema(map[string]string{
"upgrade": "\"true\": start the bus's upgrade",
"why": "with upgrade: why — required, recorded in the hand-act log",
@@ -587,6 +506,15 @@ var ControllerVerbs = []Verb{
"repository": "only this repository's references, as <module>/<artifact> (the counts stay whole)",
"collected": "\"true\": list the collected references too (they only grow; counted always)",
}, nil, "collected")},
// Whether the artifact store holds what a builder's done key names, before the builder reuses it (novox/hq issue
// 1937): the store's collector deletes a blob nothing holds, and a key the controller never took in holds nothing.
{Name: "held", Description: "Whether the artifact store holds each reference given, asked with one HEAD each: " +
"held, gone (the store answered 404) or unknown with why. The builder asks it before it reuses a done key, and " +
"builds a key whose artifact is gone again. Reads only: nothing is held or written (novox/hq issue 1937).",
Input: listed(schema(map[string]string{
"references": "the artifacts' references, artifact-store://<module>/<artifact>/blobs/sha256:<hex>: a list, " +
"or one text separated by commas",
}, []string{"references"}), "references")},
{Name: "collect", Description: "The artifact store's sweep, now: what the mesh made and keeps for no reason " +
"is let go of. Without confirm a dry run — every kept archive asked about with HEADs, what would be let go " +
"listed, nothing held or deleted. With confirm and why: every kept archive held first, then each eligible " +
+31
View File
@@ -0,0 +1,31 @@
package catalogue
import (
"strings"
"testing"
)
// 1887/A9b: the build queue's verbs are refused by name, naming what replaced them, and the verbs R4e removed are gone.
func TestTheBuildQueueVerbsAreRefusedByName(t *testing.T) {
rows := map[string]Verb{}
for _, v := range ControllerVerbs {
rows[v.Name] = v
}
for _, gone := range []string{"rebuild", "forget-checks"} {
if _, ok := rows[gone]; ok {
t.Errorf("the controller still lists %s", gone)
}
}
for _, refused := range []string{"queue", "cancel", "clear", "kill", "pause", "resume", "replay"} {
v, ok := rows[refused]
if !ok || !strings.HasPrefix(v.Description, "Removed in R4g (ADR 0525 §9): refused by name") {
t.Errorf("%s = %q, want refused by name", refused, v.Description)
}
}
if !strings.Contains(rows["queue"].Description, "delivery view") || !strings.Contains(rows["pause"].Description, "node-build-") {
t.Errorf("queue and pause do not name their replacements: %q, %q", rows["queue"].Description, rows["pause"].Description)
}
if strings.Contains(rows["try"].Description, "build-agent") {
t.Errorf("try still names build-agent: %q", rows["try"].Description)
}
}
+115
View File
@@ -0,0 +1,115 @@
package catalogue
import (
"strings"
"testing"
)
// 1943/AC4: the controller's seat no longer offers a verb that starts, stops, retries, follows or reads a delivery
// plan, nor a push: each is a row with no input whose description refuses it by name and names its replacement, and
// each is in the removed list the holder answers from. upgrade keeps its policy and drops the plan's backlog.
func TestTheControllerSeatHasNoPlanVerbs(t *testing.T) {
rows := map[string]Verb{}
for _, v := range ControllerVerbs {
rows[v.Name] = v
}
gone := []string{"plans", "plan", "push", "deliver", "delivery-stop", "delivery-backups-again", "delivery-walks", "delivery-order"}
for _, name := range gone {
v, ok := rows[name]
if !ok {
t.Errorf("%s is not in the table, so a caller is not answered by name", name)
continue
}
replacement, removed := RemovedControllerVerbs[name]
if !removed || replacement == "" {
t.Errorf("%s is not in the removed list with its replacement", name)
}
if !strings.HasPrefix(v.Description, "Removed in R6 (ADR 0539): refused by name") || !strings.Contains(v.Description, replacement) {
t.Errorf("%s = %q, want refused by name, naming %q", name, v.Description, replacement)
}
if props, _ := v.Input["properties"].(map[string]any); len(props) > 0 {
t.Errorf("%s still takes %v", name, props)
}
}
props, _ := rows["upgrade"].Input["properties"].(map[string]any)
for _, arg := range []string{"together", "backlog", "release-backlog", "cause"} {
if _, has := props[arg]; has {
t.Errorf("upgrade still takes %s, a plan's", arg)
}
}
for _, arg := range []string{"module", "policy", "why"} {
if _, has := props[arg]; !has {
t.Errorf("upgrade no longer takes %s", arg)
}
}
for _, word := range []string{"rolled", "gate", "backlog", "push"} {
if strings.Contains(rows["upgrade"].Description, word) {
t.Errorf("upgrade's description still speaks of %q: %q", word, rows["upgrade"].Description)
}
}
if !strings.Contains(rows["bus"].Description, "want") {
t.Errorf("bus does not say the operator writes the bus's want: %q", rows["bus"].Description)
}
for name := range RemovedControllerVerbs {
if v := rows[name]; !strings.HasPrefix(v.Description, "Removed in R6") {
t.Errorf("%s is in the removed list and its row says %q", name, v.Description)
}
}
}
// 1943/AC4: the delivery seat keeps the merge's verbs and no longer offers times, what-if, release, stop or
// retire-history: each is a row with no input, optional so the holder of before and the holder of release 6 both hold
// the seat, refusing it by name; mark-ready, served by nobody since issue 1022, is gone.
func TestTheDeliverySeatHasNoPlanVerbs(t *testing.T) {
seat, ok := SeatNamed(DeliverySeat)
if !ok {
t.Fatal("the delivery seat is not in the set")
}
rows := map[string]Verb{}
for _, v := range seat.Serves {
rows[v.Name] = v
}
for _, name := range []string{"times", "what-if", "release", "stop", "retire-history"} {
v, ok := rows[name]
replacement, removed := RemovedDeliveryVerbs[name]
if !ok || !removed || replacement == "" {
t.Errorf("%s: in the table %v, in the removed list %v", name, ok, removed)
continue
}
if !strings.HasPrefix(v.Description, "Removed in R6 (ADR 0539): refused by name") || !strings.Contains(v.Description, replacement) || !v.Optional {
t.Errorf("%s = %q (optional %v), want an optional row refusing it by name", name, v.Description, v.Optional)
}
if props, _ := v.Input["properties"].(map[string]any); len(props) > 0 {
t.Errorf("%s still takes %v", name, props)
}
}
if _, ok := rows["mark-ready"]; ok {
t.Errorf("mark-ready is still promised")
}
for _, kept := range []string{"deliveries", "show", "groups", "checks", "table", "recheck", "close", "stalled"} {
if v, ok := rows[kept]; !ok || strings.HasPrefix(v.Description, "Removed") {
t.Errorf("%s is not kept (ADR 0539 clause 17)", kept)
}
}
gate := Manifest{Module: "mesh-delivery", Claims: []Claim{{Name: DeliverySeat, Scope: ScopeMesh,
Serves: []string{"deliveries", "show", "groups", "checks", "table", "stalled", "recheck", "close"}}}}
if err := CanHold(gate, seat); err != nil {
t.Errorf("the gate of release 6 cannot hold the seat: %v", err)
}
gate.Claims[0].Serves = append(gate.Claims[0].Serves, "times", "what-if", "release", "stop", "retire-history")
if err := CanHold(gate, seat); err != nil {
t.Errorf("the gate of release 5 cannot hold the seat: %v", err)
}
}
// 1943 (for 1945/AC14): the controller's seat says a bucket it deleted at its first start as its event, bucket-deleted
// (ADR 0539 clause 13).
func TestTheControllerSaysABucketItDeleted(t *testing.T) {
seat, ok := SeatNamed(ControllerSeatName)
if !ok {
t.Fatal("no controller seat")
}
if !SeatSays(seat.Emits, "bucket-deleted") {
t.Errorf("the controller's seat does not emit bucket-deleted: %v", seat.Emits)
}
}
+163
View File
@@ -0,0 +1,163 @@
// mesh-compose is mesh-sdk's composer command (novox/hq ADR 0482 clause 4, issue 1621): it reads a composer.Request on
// standard input, validates every manifest with this mesh-sdk's reader, composes the machines named in each case and
// writes one composer.Verdict on standard output. Built at any commit of mesh-sdk (or at the mesh-sdk a module pins),
// it answers what that commit's composition answers, so a program that links one mesh-sdk can have another's judged.
//
// mesh-compose < request.json > verdict.json
// mesh-compose -request request.json > verdict.json
// mesh-compose -catalogue <dir> [-node <name>]... [-depth resolve|compose] [-declarations] [< request.json]
//
// -catalogue reads <dir>/modules/*/module.json and a module.json at <dir>'s root as the first case's manifests (a
// checkout or `git archive` of the catalogue repository at a ref: the command fetches nothing). -node adds a machine to
// compose; -declarations answers each machine's declared paths with their owners and modes.
//
// It opens no network connection, bus, store or file beyond its standard input, the -request file and the -catalogue
// directory, and reads
// no environment. Exit codes: 0 the verdict is written (a machine that does not compose is a verdict, not an error);
// 1 the request cannot be read or run, said on standard error; 2 the flags are wrong.
package main
import (
"encoding/json"
"errors"
"flag"
"fmt"
"io"
"os"
"path/filepath"
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/composer"
)
func main() {
os.Exit(run(os.Args[1:], os.Stdin, os.Stdout, os.Stderr))
}
type nodes []string
func (n *nodes) String() string { return strings.Join(*n, ",") }
func (n *nodes) Set(v string) error { *n = append(*n, v); return nil }
// run is the command: its arguments, standard input, output and error; it answers the exit code.
func run(args []string, stdin io.Reader, stdout, stderr io.Writer) int {
fs := flag.NewFlagSet("mesh-compose", flag.ContinueOnError)
fs.SetOutput(stderr)
catalogueDir := fs.String("catalogue", "", "a catalogue checkout: <dir>/modules/*/module.json are the first case's manifests")
var machines nodes
fs.Var(&machines, "node", "a machine to compose (repeatable)")
depth := fs.String("depth", "", "resolve or compose (default: the request's, else compose)")
declarations := fs.Bool("declarations", false, "answer each composed machine's declared paths, owners and modes")
requestFile := fs.String("request", "", "read the request from this file instead of standard input (a run step that hands no standard input)")
if err := fs.Parse(args); err != nil {
return 2
}
if fs.NArg() > 0 {
fmt.Fprintf(stderr, "mesh-compose: unexpected arguments %q\n", fs.Args())
return 2
}
var raw []byte
var err error
if *requestFile != "" {
raw, err = os.ReadFile(*requestFile)
} else {
raw, err = io.ReadAll(stdin)
}
if err != nil {
fmt.Fprintln(stderr, "mesh-compose: the request cannot be read:", err)
return 1
}
var req composer.Request
if strings.TrimSpace(string(raw)) == "" {
if *catalogueDir == "" {
fmt.Fprintln(stderr, "mesh-compose: no request on standard input and no -catalogue")
return 1
}
req = composer.Request{V: composer.Version}
} else if req, err = composer.Decode(raw); err != nil {
fmt.Fprintln(stderr, "mesh-compose:", err)
return 1
}
if *catalogueDir != "" {
read, err := readCatalogue(*catalogueDir)
if err != nil {
fmt.Fprintln(stderr, "mesh-compose:", err)
return 1
}
if len(req.Cases) == 0 {
req.Cases = []composer.Case{{Name: "catalogue", Manifests: map[string]composer.Manifest{}}}
}
if req.Cases[0].Manifests == nil {
req.Cases[0].Manifests = map[string]composer.Manifest{}
}
for name, m := range read {
if _, given := req.Cases[0].Manifests[name]; !given {
req.Cases[0].Manifests[name] = m
}
}
// The catalogue's list of predefined keys is the one its manifests' fallbacks are judged against (ADR 0348 §7).
if req.Predefined == nil {
list, err := catalogue.ReadPredefined(*catalogueDir)
if err != nil {
fmt.Fprintln(stderr, "mesh-compose:", err)
return 1
}
req.Predefined = list
}
}
req.Machines = append(req.Machines, machines...)
if *depth != "" {
req.Depth = *depth
}
if *declarations {
req.Declarations = true
}
v, err := composer.Run(req)
if err != nil {
fmt.Fprintln(stderr, "mesh-compose:", err)
return 1
}
enc := json.NewEncoder(stdout)
if err := enc.Encode(v); err != nil {
fmt.Fprintln(stderr, "mesh-compose: the verdict cannot be written:", err)
return 1
}
return 0
}
// readCatalogue reads every module.json of a catalogue checkout by its module's name: the root's and each of
// modules/*/. A manifest whose name cannot be read is kept under its directory's name, so the reader's refusal is said.
func readCatalogue(dir string) (map[string]composer.Manifest, error) {
paths, err := filepath.Glob(filepath.Join(dir, "modules", "*", "module.json"))
if err != nil {
return nil, err
}
if _, err := os.Stat(filepath.Join(dir, "module.json")); err == nil {
paths = append(paths, filepath.Join(dir, "module.json"))
}
if len(paths) == 0 {
return nil, fmt.Errorf("the catalogue %s holds no module.json", dir)
}
sort.Strings(paths)
out := map[string]composer.Manifest{}
for _, p := range paths {
raw, err := os.ReadFile(p)
if err != nil {
return nil, err
}
var probe struct {
Module string `json:"module"`
}
name := filepath.Base(filepath.Dir(p))
if json.Unmarshal(raw, &probe) == nil && probe.Module != "" {
name = probe.Module
}
if _, twice := out[name]; twice {
return nil, errors.New("the catalogue " + dir + " defines " + name + " twice")
}
out[name] = composer.Manifest{Raw: raw}
}
return out, nil
}
+347
View File
@@ -0,0 +1,347 @@
package main
import (
"bytes"
"encoding/json"
"go/build"
"os"
"os/exec"
"path/filepath"
"reflect"
"runtime"
"sort"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
"git.novox.be/novox/mesh-sdk/go/composer"
"git.novox.be/novox/mesh-sdk/go/internal/beside"
)
// captured is every manifest of the catalogue captured in testdata/beside (internal/beside), raw, by module name.
func captured(t *testing.T) map[string]composer.Manifest {
t.Helper()
read, err := readCatalogue(filepath.Dir(beside.Catalogue(t)))
if err != nil {
t.Fatal(err)
}
if len(read) < 100 {
t.Fatalf("the captured catalogue holds %d manifests, fewer than the catalogue's", len(read))
}
return read
}
// meshOf is a mesh of three machines over the captured catalogue, each assigned real modules, the input the assigner
// composes with (nodes and their assignments, holdings from the claims): anchor runs the bus and the store, laptop a
// desktop's modules, ace a module whose requirement is provided elsewhere.
func meshOf() compose.Input {
return compose.Input{Nodes: []compose.Node{
{Name: "anchor", Assigned: []string{"nats", "postgres"}},
{Name: "laptop", Assigned: []string{"sudo", "zsh"}},
{Name: "ace", Assigned: []string{"mesh-reads", "nats"}},
}}
}
func request(t *testing.T, depth string) composer.Request {
t.Helper()
return composer.Request{V: composer.Version, Input: meshOf(), Depth: depth, HoldingsFromClaims: true,
Machines: []string{"anchor", "laptop", "ace"},
Cases: []composer.Case{{Name: "trunk", Manifests: captured(t)}}}
}
func runJSON(t *testing.T, args []string, in []byte) (int, composer.Verdict, string) {
t.Helper()
var out, errOut bytes.Buffer
code := run(args, bytes.NewReader(in), &out, &errOut)
var v composer.Verdict
if code == 0 {
if err := json.Unmarshal(out.Bytes(), &v); err != nil {
t.Fatalf("the verdict is not JSON: %v\n%s", err, out.String())
}
}
return code, v, errOut.String()
}
func encode(t *testing.T, r composer.Request) []byte {
t.Helper()
raw, err := json.Marshal(r)
if err != nil {
t.Fatal(err)
}
return raw
}
// 1621/AC1: the command reads the plain input and the manifests on standard input, validates every manifest, composes
// the machines named and writes one verdict: per machine composes or the reason, per manifest accepted or the field
// refused.
func TestTheCommandWritesOneVerdictPerManifestAndMachine(t *testing.T) {
r := request(t, composer.DepthResolve)
r.Cases[0].Manifests["broken"] = composer.Manifest{Raw: json.RawMessage(`{"module":"broken","version":"1","shares":["x"]}`)}
code, v, stderr := runJSON(t, nil, encode(t, r))
if code != 0 {
t.Fatalf("exit %d: %s", code, stderr)
}
if len(v.Cases) != 1 || v.Cases[0].Name != "trunk" || v.SDK == "" {
t.Fatalf("verdict = %+v", v)
}
c := v.Cases[0]
if got := c.Manifests["broken"]; got.Accepted || !strings.Contains(got.Refused, `"shares"`) {
t.Errorf("the manifest with an unknown field reads %+v, want refused naming the field", got)
}
if got := c.Manifests["nats"]; !got.Accepted {
t.Errorf("nats reads %+v, want accepted", got)
}
if len(c.Machines) != 3 {
t.Fatalf("machines = %v, want the three named", c.Machines)
}
for name, m := range c.Machines {
if !m.Composes && m.Reason == "" {
t.Errorf("%s does not compose and says no reason", name)
}
}
}
// 1621/AC3: over the captured catalogue and a mesh of three machines, the command's verdict equals, machine for
// machine and manifest for manifest, what the linked reader, resolution and composition answer, at both depths.
func TestTheCommandAnswersWhatTheLinkedCompositionAnswers(t *testing.T) {
for _, depth := range []string{composer.DepthResolve, composer.DepthCompose} {
r := request(t, depth)
code, v, stderr := runJSON(t, nil, encode(t, r))
if code != 0 {
t.Fatalf("%s: exit %d: %s", depth, code, stderr)
}
in := meshOf()
in.Catalogue = map[string]catalogue.Manifest{}
for name, m := range r.Cases[0].Manifests {
// A request without a list of predefined keys reads a fallback as the indexer does (Request.Predefined).
parsed, err := catalogue.ParseManifestForInputs(m.Raw)
got := v.Cases[0].Manifests[name]
if (err == nil) != got.Accepted || (err != nil && err.Error() != got.Refused) {
t.Errorf("%s: the manifest of %s: linked %v, command %+v", depth, name, err, got)
}
if err == nil {
in.Catalogue[name] = parsed
}
}
in.Holdings = composer.HoldingsOf(in.Catalogue, in.Nodes)
composed := 0
for _, node := range r.Machines {
got := v.Cases[0].Machines[node]
var want composer.Machine
if depth == composer.DepthResolve {
if _, err := compose.Resolve(in, node); err != nil {
want = composer.Machine{Reason: err.Error()}
} else {
want = composer.Machine{Composes: true}
}
} else {
d, problems, err := compose.Compose(in, node)
switch {
case err != nil:
want = composer.Machine{Reason: err.Error()}
default:
want = composer.Machine{Composes: len(problems) == 0, Problems: problems, Refusals: d.Refusals()}
if len(problems) > 0 {
want.Reason = "the node-engine's validator refuses the declaration"
}
}
}
if !reflect.DeepEqual(got, want) {
t.Errorf("%s: %s: command %+v, linked %+v", depth, node, got, want)
}
if got.Composes {
composed++
} else {
t.Logf("%s: %s does not compose: %s %v", depth, node, got.Reason, got.Problems)
}
}
t.Logf("%s: %d of %d machines compose", depth, composed, len(r.Machines))
}
}
// 1621 note 6: a person composes a catalogue checkout (repository@ref checked out or archived: the command fetches
// nothing) for a node and reads its declaration: each path with its kind, owner and mode.
func TestACatalogueCheckoutIsComposedForANode(t *testing.T) {
dir := t.TempDir()
if err := os.MkdirAll(filepath.Join(dir, "modules", "demo"), 0o755); err != nil {
t.Fatal(err)
}
manifest := `{"module": "demo", "version": "1", "resources": [{"id": "state", "type": "directory", ` +
`"path": "/var/lib/demo", "mode": "0750", "owner": "root"}]}`
if err := os.WriteFile(filepath.Join(dir, "modules", "demo", "module.json"), []byte(manifest), 0o644); err != nil {
t.Fatal(err)
}
in := encode(t, composer.Request{V: composer.Version, Input: compose.Input{Nodes: []compose.Node{{Name: "laptop", Assigned: []string{"demo"}}}}})
code, v, stderr := runJSON(t, []string{"-catalogue", dir, "-node", "laptop", "-declarations"}, in)
if code != 0 {
t.Fatalf("exit %d: %s", code, stderr)
}
m := v.Cases[0].Machines["laptop"]
want := []composer.Path{{Path: "/var/lib/demo", Kind: "directory", Owner: "root", Mode: "0750", Module: ""}}
if !m.Composes || len(m.Declared) != 1 || m.Declared[0].Path != want[0].Path || m.Declared[0].Mode != "0750" ||
m.Declared[0].Owner != "root" || m.Declared[0].Kind != "directory" {
t.Fatalf("laptop = %+v, want it composed declaring %+v", m, want)
}
}
// A run step hands the command no standard input: the request is read from the file -request names, and a file that is
// not there is a request it cannot read.
func TestTheRequestIsReadFromAFile(t *testing.T) {
file := filepath.Join(t.TempDir(), "request.json")
r := composer.Request{V: composer.Version, Cases: []composer.Case{{Name: "only", Manifests: map[string]composer.Manifest{
"demo": {Raw: json.RawMessage(`{"module":"demo","version":"1"}`)}}}}}
if err := os.WriteFile(file, encode(t, r), 0o600); err != nil {
t.Fatal(err)
}
code, v, stderr := runJSON(t, []string{"-request", file}, nil)
if code != 0 || len(v.Cases) != 1 || !v.Cases[0].Manifests["demo"].Accepted {
t.Fatalf("exit %d (%s), verdict %+v", code, stderr, v)
}
if code, _, stderr := runJSON(t, []string{"-request", file + ".gone"}, nil); code != 1 || !strings.Contains(stderr, "cannot be read") {
t.Errorf("a request file that is not there: exit %d, %q", code, stderr)
}
}
// The exit codes: 1 for a request that cannot be read or run, 2 for wrong flags, each said on standard error.
func TestTheExitCodesSayWhatFailed(t *testing.T) {
unfilled := encode(t, composer.Request{V: composer.Version, Cases: []composer.Case{{Name: "c", Manifests: map[string]composer.Manifest{"x": {Digest: "abc"}}}}})
for _, c := range []struct {
name string
args []string
in string
code int
says string
}{
{"not JSON", nil, "{", 1, "not JSON"},
{"another version", nil, `{"v":2}`, 1, "version 2"},
{"a digest left unfilled", nil, string(unfilled), 1, "digest"},
{"an unknown depth", nil, `{"v":1,"depth":"deep"}`, 1, "depth"},
{"nothing to read", nil, "", 1, "no request"},
{"a wrong flag", []string{"-nope"}, "", 2, "nope"},
{"an argument", []string{"extra"}, "", 2, "unexpected"},
} {
var out, errOut bytes.Buffer
if got := run(c.args, strings.NewReader(c.in), &out, &errOut); got != c.code || !strings.Contains(errOut.String(), c.says) {
t.Errorf("%s: exit %d saying %q, want %d saying %q", c.name, got, errOut.String(), c.code, c.says)
}
}
}
// sdkPackages is every package of this module the command imports, directly or not, with the packages each imports.
func sdkPackages(t *testing.T) map[string][]string {
t.Helper()
_, file, _, _ := runtime.Caller(0)
root := filepath.Join(filepath.Dir(file), "..", "..")
const module = "git.novox.be/novox/mesh-sdk/go"
out := map[string][]string{}
var walk func(path string)
walk = func(path string) {
if _, done := out[path]; done {
return
}
rel := strings.TrimPrefix(strings.TrimPrefix(path, module), "/")
pkg, err := build.Default.ImportDir(filepath.Join(root, rel), 0)
if err != nil {
t.Fatalf("%s: %v", path, err)
}
out[path] = pkg.Imports
for _, imp := range pkg.Imports {
if strings.HasPrefix(imp, module+"/") {
walk(imp)
}
}
}
walk(composer.Package)
return out
}
// 1621/AC2 and AC4: the command imports, through every package of mesh-sdk it reaches, only the standard library and
// mesh-sdk's own packages (so `go build` from mesh-sdk's module needs nothing beyond its lock), and none that reaches
// a bus, a store, a network service or another process.
func TestTheCommandImportsNoBusStoreNetworkOrProcess(t *testing.T) {
forbidden := []string{"net/http", "net/rpc", "net/smtp", "os/exec", "database/sql", "plugin", "syscall"}
for path, imports := range sdkPackages(t) {
for _, imp := range imports {
std := !strings.Contains(strings.SplitN(imp, "/", 2)[0], ".")
if !std && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-sdk/go/") {
t.Errorf("%s imports %s, which is neither the standard library nor mesh-sdk", path, imp)
}
for _, f := range forbidden {
if imp == f {
t.Errorf("%s imports %s", path, imp)
}
}
}
}
}
// 1621/AC2: run with an empty network namespace, the command still writes its verdict: it needs no network. The test
// runs its own binary again under `unshare --net` as an unprivileged user's namespace; where the kernel or the sandbox
// allows no new namespace (a build module's run step is already one with no network), it says so and skips.
func TestTheCommandRunsWithNoNetwork(t *testing.T) {
if os.Getenv("MESH_COMPOSE_HELPER") == "1" {
os.Exit(run(nil, os.Stdin, os.Stdout, os.Stderr))
}
unshare, err := exec.LookPath("unshare")
if err != nil {
t.Skip("no unshare on this machine, so no empty network namespace can be made here")
}
if out, err := exec.Command(unshare, "--net", "--map-root-user", "true").CombinedOutput(); err != nil {
t.Skipf("this machine makes no unprivileged network namespace (%v: %s); a run step's own has no network", err, out)
}
r := request(t, composer.DepthResolve)
cmd := exec.Command(unshare, "--net", "--map-root-user", os.Args[0], "-test.run=^TestTheCommandRunsWithNoNetwork$")
cmd.Env = []string{"MESH_COMPOSE_HELPER=1"}
cmd.Stdin = bytes.NewReader(encode(t, r))
var out, errOut bytes.Buffer
cmd.Stdout, cmd.Stderr = &out, &errOut
if err := cmd.Run(); err != nil {
t.Fatalf("the command in an empty network namespace: %v: %s", err, errOut.String())
}
var v composer.Verdict
if err := json.Unmarshal(out.Bytes(), &v); err != nil || len(v.Cases) != 1 || len(v.Cases[0].Machines) != 3 {
t.Fatalf("no verdict from the command with no network: %v\n%s", err, out.String())
}
names := make([]string, 0, len(v.Cases[0].Machines))
for n := range v.Cases[0].Machines {
names = append(names, n)
}
sort.Strings(names)
t.Logf("composed with no network: %v", names)
}
// -catalogue judges a fallback to a predefined key against the predefined.json at the checkout's root (ADR 0348 §7):
// xterm (mesh-catalog at 835bf22b) falls back to "font-size", accepted where the root lists it and refused where the
// root's list does not.
func TestACatalogueCheckoutJudgesFallbacksAgainstItsPredefinedKeys(t *testing.T) {
raw, err := os.ReadFile("../../catalogue/testdata/catalogue-835bf22b-xterm.module.json")
if err != nil {
t.Fatal(err)
}
for _, c := range []struct {
list string
want bool
}{
{`{"font-size": {"type": "number", "description": "the desktop's font size"}}`, true},
{`{"font-family": {"type": "string", "description": "the desktop's font"}}`, false},
} {
dir := t.TempDir()
if err := os.MkdirAll(filepath.Join(dir, "modules", "xterm"), 0o700); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "modules", "xterm", "module.json"), raw, 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, catalogue.PredefinedFile), []byte(c.list), 0o600); err != nil {
t.Fatal(err)
}
code, v, stderr := runJSON(t, []string{"-catalogue", dir}, nil)
if code != 0 {
t.Fatalf("exit %d: %s", code, stderr)
}
if got := v.Cases[0].Manifests["xterm"]; got.Accepted != c.want {
t.Errorf("predefined.json %s: xterm %+v, want accepted %v", c.list, got, c.want)
}
}
}
+1
View File
@@ -0,0 +1 @@
These tests read files outside this directory (the whole catalogue, the repository, or its root files): mesh-build-go never keeps this package's verdict (issue 1935).
+209
View File
@@ -0,0 +1,209 @@
// mesh-hand-compose composes one module for one node by hand, from the inputs the controller keeps, and signs it with the
// operator's key, for the hand path of novox/hq ADR 0539 clause 10: a fingerprint the controller never composed (a
// hand build), while the controller cannot compose its own fix. It composes with mesh-sdk's composition, the one the
// controller writes with (compose.Compositions), and prints the composition, its key and its digest; only with -write
// does it write them, with the operator's credential: the composition into the node's bucket composed-<node>, each
// file over 64 KiB into the node's object bucket, and the digest into mesh-controller_composed. The want is the
// operator's next act, naming that digest; each act is recorded in the hand-act log.
//
// mesh-hand-compose -input <inputs.json> -node <node> -module <module> -fp <src2:...> -key <operator key>
// [-write -nats <url> -creds <operator credential>]
//
// The inputs are a compose.Input as JSON. The key is the operator's ed25519 private key, PKCS#8 in PEM; the
// node-engine trusts it beside the controller's (ADR 0539 clause 3).
package main
import (
"context"
"crypto/ed25519"
"crypto/x509"
"encoding/json"
"encoding/pem"
"errors"
"flag"
"fmt"
"io"
"os"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/compose"
"git.novox.be/novox/mesh-sdk/go/declaration"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
func main() {
if err := run(os.Args[1:], os.Stdout); err != nil {
fmt.Fprintln(os.Stderr, "mesh-hand-compose:", err)
os.Exit(1)
}
}
// printed is what mesh-hand-compose prints: where the composition goes, its digest, the composition, and whether it was
// written.
type printed struct {
Bucket string `json:"bucket"`
Key string `json:"key"`
Digest string `json:"digest"`
Composition json.RawMessage `json:"composition"`
Objects []string `json:"objects,omitempty"`
Written bool `json:"written"`
Next string `json:"next"`
}
func run(args []string, stdout io.Writer) error {
fs := flag.NewFlagSet("mesh-hand-compose", flag.ContinueOnError)
input := fs.String("input", "", "the controller's kept inputs, a compose.Input as JSON")
node := fs.String("node", "", "the node to compose for")
module := fs.String("module", "", "the module to compose")
fp := fs.String("fp", "", "the module's fingerprint, src2: and 64 hex digits")
keyFile := fs.String("key", "", "the operator's ed25519 private key, PKCS#8 in PEM")
write := fs.Bool("write", false, "write the composition, its files and its digest, with the operator's credential")
url := fs.String("nats", nats.DefaultURL, "with -write: the bus")
creds := fs.String("creds", "", "with -write: the operator's credential file")
if err := fs.Parse(args); err != nil {
return err
}
switch {
case *input == "" || *node == "" || *module == "" || *fp == "":
return errors.New("-input, -node, -module and -fp are each required")
case *keyFile == "":
return errors.New("-key is required: a composition made by hand is signed by the operator's key, and the node-engine applies no other")
case *write && *creds == "":
return errors.New("-write needs -creds: the operator's credential is the only hand writer of a composition")
}
key, err := readKey(*keyFile)
if err != nil {
return err
}
key2, err := delivery.ComposedKey(*node, *module, *fp)
if err != nil {
return err
}
bucket, err := delivery.ComposedBucketFor(*node)
if err != nil {
return err
}
raw, err := os.ReadFile(*input)
if err != nil {
return err
}
var in compose.Input
if err := json.Unmarshal(raw, &in); err != nil {
return fmt.Errorf("the inputs in %s are not a compose.Input: %w", *input, err)
}
all, _, err := compose.Compositions(in, *node, map[string]string{*module: *fp}, compose.Reading)
if err != nil {
return err
}
c, ok := all[key2]
if !ok {
return fmt.Errorf("%s does not run %s: its composition holds none of it", *node, *module)
}
c, objects, err := c.ByDigest()
if err != nil {
return err
}
if c, err = delivery.SignComposed(c, key); err != nil {
return err
}
encoded, err := delivery.EncodeComposed(c)
if err != nil {
return err
}
digest, err := delivery.ComposedDigestOf(c)
if err != nil {
return err
}
out := printed{Bucket: bucket, Key: key2, Digest: digest, Composition: encoded,
Next: fmt.Sprintf("write want.%s.%s naming fp %s and composed %s with the operator's credential, and record each act in the hand-act log", *node, *module, *fp, digest)}
for d := range objects {
out.Objects = append(out.Objects, d)
}
if *write {
if err := writeIt(*url, *creds, *node, bucket, key2, encoded, digest, objects); err != nil {
return err
}
out.Written = true
}
enc := json.NewEncoder(stdout)
enc.SetIndent("", " ")
return enc.Encode(out)
}
func readKey(path string) (ed25519.PrivateKey, error) {
raw, err := os.ReadFile(path)
if err != nil {
return nil, err
}
block, _ := pem.Decode(raw)
if block == nil {
return nil, fmt.Errorf("%s holds no PEM block", path)
}
parsed, err := x509.ParsePKCS8PrivateKey(block.Bytes)
if err != nil {
return nil, fmt.Errorf("%s: %w", path, err)
}
key, ok := parsed.(ed25519.PrivateKey)
if !ok {
return nil, fmt.Errorf("%s is not an ed25519 key", path)
}
return key, nil
}
// writeIt writes the files by digest, then the composition, then its digest: a reader that finds the digest finds the
// composition and its files already there.
func writeIt(url, creds, node, bucket, key string, encoded []byte, digest string, objects map[string][]byte) error {
ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
defer cancel()
nc, err := nats.Connect(url, nats.UserCredentials(creds), nats.Name("mesh-hand-compose"))
if err != nil {
return err
}
defer nc.Close()
js, err := jetstream.New(nc)
if err != nil {
return err
}
if len(objects) > 0 {
filesBucket, err := declaration.FileBucketFor(node)
if err != nil {
return err
}
store, err := js.ObjectStore(ctx, filesBucket)
if err != nil {
return fmt.Errorf("the node's object bucket %s: %w", filesBucket, err)
}
for d, body := range objects {
name, err := declaration.ObjectName(d)
if err != nil {
return err
}
if _, err := store.PutBytes(ctx, name, body); err != nil {
return fmt.Errorf("the file %s: %w", d, err)
}
}
}
composed, err := js.KeyValue(ctx, bucket)
if err != nil {
return fmt.Errorf("the bucket %s: %w", bucket, err)
}
if _, err := composed.Put(ctx, key, encoded); err != nil {
return fmt.Errorf("%s in %s: %w", key, bucket, err)
}
value, err := delivery.EncodeComposedDigest(delivery.ComposedDigest{Digest: digest, Judgement: delivery.JudgedCannot,
Words: "composed by hand with mesh-hand-compose (ADR 0539 clause 10): the operator writes the want"})
if err != nil {
return err
}
digests, err := js.KeyValue(ctx, delivery.ControllerComposedBucket)
if err != nil {
return fmt.Errorf("the bucket %s: %w", delivery.ControllerComposedBucket, err)
}
if _, err := digests.Put(ctx, key, value); err != nil {
return fmt.Errorf("%s in %s: %w", key, delivery.ControllerComposedBucket, err)
}
return nil
}
+145
View File
@@ -0,0 +1,145 @@
package main
import (
"bytes"
"crypto/ed25519"
"crypto/rand"
"crypto/x509"
"encoding/base64"
"encoding/json"
"encoding/pem"
"os"
"path/filepath"
"sort"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/overlay"
)
// aMesh is the Input of mesh-controller's aMesh, as mesh-sdk's compose tests make it (compose/fixture_test.go): two
// machines on the private network, the module the mesh provides for itself assigned to both. The manifest is the one
// the controller provides (overlay.Manifest), read back as the catalogue reads it.
func aMesh(t *testing.T) compose.Input {
t.Helper()
var m catalogue.Manifest
b, _ := json.Marshal(overlay.Manifest())
if err := json.Unmarshal(b, &m); err != nil {
t.Fatal(err)
}
in := compose.Input{Catalogue: map[string]catalogue.Manifest{overlay.Name: m}}
for i, name := range []string{"anchor", "laptop"} {
pub, _, _ := ed25519.GenerateKey(rand.Reader)
in.Nodes = append(in.Nodes, compose.Node{Name: name,
Place: compose.Place{Key: base64.StdEncoding.EncodeToString(pub), Endpoint: name + ".example:51820", Site: "here", Hub: name == "anchor", Address: []string{"10.77.0.1", "10.77.0.2"}[i]},
Assigned: []string{overlay.Name}, Capabilities: map[string]bool{"container-runtime": true, "wireguard": true, "systemd": true}})
}
return in
}
func writeKey(t *testing.T, dir string) ed25519.PublicKey {
t.Helper()
pub, priv, _ := ed25519.GenerateKey(rand.Reader)
der, err := x509.MarshalPKCS8PrivateKey(priv)
if err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "operator.key"), pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: der}), 0o600); err != nil {
t.Fatal(err)
}
return pub
}
// 1943/AC5: mesh-hand-compose, given the kept inputs, a node, a module and its fingerprint, prints that module's
// composition for that node signed by the operator's key and its digest: its resources are the controller's for that
// module on that node (compose.Compose), and it verifies against the operator's key. Without --write it writes nothing.
func TestMeshComposeGivesTheControllersResourcesSignedByTheOperator(t *testing.T) {
dir := t.TempDir()
in := aMesh(t)
raw, err := json.Marshal(in)
if err != nil {
t.Fatal(err)
}
input := filepath.Join(dir, "input.json")
if err := os.WriteFile(input, raw, 0o600); err != nil {
t.Fatal(err)
}
operator := writeKey(t, dir)
fp := delivery.SourcePrefix2 + strings.Repeat("a", 64)
var out bytes.Buffer
if err := run([]string{"-input", input, "-node", "laptop", "-module", overlay.Name, "-fp", fp,
"-key", filepath.Join(dir, "operator.key")}, &out); err != nil {
t.Fatal(err)
}
var printed struct {
Key string `json:"key"`
Bucket string `json:"bucket"`
Digest string `json:"digest"`
Composition json.RawMessage `json:"composition"`
Written bool `json:"written"`
}
if err := json.Unmarshal(out.Bytes(), &printed); err != nil {
t.Fatalf("%s: %v", out.String(), err)
}
c, err := delivery.DecodeComposed(printed.Composition)
if err != nil {
t.Fatal(err)
}
if digest, err := delivery.VerifyComposed(c, []ed25519.PublicKey{operator}); err != nil || digest != printed.Digest {
t.Errorf("the composition verifies %q, %v against the operator's key; printed %s", digest, err, printed.Digest)
}
wantKey, _ := delivery.ComposedKey("laptop", overlay.Name, fp)
if printed.Key != wantKey || printed.Bucket != "composed-laptop" || printed.Written {
t.Errorf("printed key %s in %s, written %v", printed.Key, printed.Bucket, printed.Written)
}
d, _, err := compose.Compose(in, "laptop")
if err != nil {
t.Fatal(err)
}
var want, got []string
for _, r := range d.Resources {
want = append(want, r["id"].(string))
}
for _, r := range c.Resources {
var head struct{ ID string }
_ = json.Unmarshal(r, &head)
got = append(got, head.ID)
}
sort.Strings(want)
sort.Strings(got)
if len(got) == 0 || strings.Join(got, " ") != strings.Join(want, " ") {
t.Errorf("mesh-hand-compose gives %v, the controller's composition %v", got, want)
}
}
// 1943/AC5: mesh-hand-compose refuses what it cannot compose, naming why: no key, a fingerprint not of the new form, a
// module the node does not run, a node the mesh does not have.
func TestMeshComposeRefuses(t *testing.T) {
dir := t.TempDir()
raw, _ := json.Marshal(aMesh(t))
input := filepath.Join(dir, "input.json")
_ = os.WriteFile(input, raw, 0o600)
writeKey(t, dir)
key := filepath.Join(dir, "operator.key")
fp := delivery.SourcePrefix2 + strings.Repeat("a", 64)
for _, c := range []struct {
args []string
why string
}{
{[]string{"-input", input, "-node", "laptop", "-module", overlay.Name, "-fp", fp}, "-key"},
{[]string{"-input", input, "-node", "laptop", "-module", overlay.Name, "-fp", "src2:1", "-key", key}, "fingerprint"},
{[]string{"-input", input, "-node", "laptop", "-module", "mesh-mail", "-fp", fp, "-key", key}, "does not run mesh-mail"},
{[]string{"-input", input, "-node", "nowhere", "-module", overlay.Name, "-fp", fp, "-key", key}, "nowhere"},
{[]string{"-input", input, "-node", "laptop", "-module", overlay.Name, "-fp", fp, "-key", key, "-write"}, "-creds"},
} {
var out bytes.Buffer
if err := run(c.args, &out); err == nil || !strings.Contains(err.Error(), c.why) {
t.Errorf("mesh-hand-compose %v = %v, want a refusal saying %q", c.args, err, c.why)
}
}
}
+111
View File
@@ -0,0 +1,111 @@
// mesh-pack builds a Go bundle by hand with the recipe of the mesh-build-go module and packs it with mesh-sdk's own
// buildstep.Pack, printing the digest, so a hand build has the form of a built artifact and, with the same Go version,
// its digest (novox/hq ADR 0526; fallback record F1; issue 1499 for the Go version).
//
// mesh-pack -dir <module dir> -from <package> -binary <name> [-system arch] [-out <archive file>]
// mesh-pack -pack <directory> [-out <archive file>] (a `files` archive: pack a tree as it is)
//
// The recipe is mesh-catalog modules/mesh-build-go cmd/mesh-build-go/build.go (recipeFlags, recipeLinkerFlags,
// recipeSystemStamp, buildEnv): CGO_ENABLED=0, GOFLAGS "-trimpath -buildvcs=false -mod=mod" (-mod=vendor when the
// directory vendors), GOTOOLCHAIN=local, GOPROXY=off, and -ldflags "-s -w -X main.builtFor=<system>".
package main
import (
"flag"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"git.novox.be/novox/mesh-sdk/go/buildstep"
)
const (
recipeFlags = "-trimpath -buildvcs=false"
recipeLinkerFlags = "-s -w"
recipeSystemStamp = "main.builtFor"
)
// Env is the build module's environment for the compile of dir, added to the caller's own.
func Env(dir string) []string {
mod := "-mod=mod"
if _, err := os.Stat(filepath.Join(dir, "vendor", "modules.txt")); err == nil {
mod = "-mod=vendor"
}
return append(os.Environ(),
"CGO_ENABLED=0",
"GOFLAGS="+recipeFlags+" "+mod,
"GOTOOLCHAIN=local",
"GOPROXY=off",
)
}
// Args is the arguments of the go command of the recipe.
func Args(dir, from, out, binary, system string) []string {
linker := recipeLinkerFlags
if s := strings.TrimSpace(system); s != "" {
linker += " -X " + recipeSystemStamp + "=" + s
}
return []string{"-C", dir, "build", "-ldflags", linker, "-o", filepath.Join(out, binary), "./" + strings.Trim(from, "./")}
}
// Build compiles the package into a fresh directory and answers it.
func Build(dir, from, binary, system string) (string, error) {
out, err := os.MkdirTemp("", "mesh-pack-")
if err != nil {
return "", err
}
cmd := exec.Command("go", Args(dir, from, out, binary, system)...)
cmd.Env = Env(dir)
cmd.Stdout, cmd.Stderr = os.Stderr, os.Stderr
if err := cmd.Run(); err != nil {
os.RemoveAll(out)
return "", fmt.Errorf("go build of %s: %w", from, err)
}
return out, nil
}
func run(args []string) (string, error) {
fs := flag.NewFlagSet("mesh-pack", flag.ContinueOnError)
dir := fs.String("dir", ".", "the Go module's directory")
from := fs.String("from", "", "the package to build, relative to -dir (the artifact's from)")
binary := fs.String("binary", "", "the binary's name in the archive")
system := fs.String("system", "", "the system stamped into main.builtFor (an artifact's system, e.g. arch)")
pack := fs.String("pack", "", "pack this directory as it is (a files archive) instead of building")
outFile := fs.String("out", "", "write the archive here")
if err := fs.Parse(args); err != nil {
return "", err
}
root := *pack
if root == "" {
if *from == "" || *binary == "" {
return "", fmt.Errorf("-from and -binary are needed (or -pack <directory>)")
}
out, err := Build(*dir, *from, *binary, *system)
if err != nil {
return "", err
}
defer os.RemoveAll(out)
root = out
}
archive, digest, err := buildstep.Pack(root)
if err != nil {
return "", err
}
if *outFile != "" {
if err := os.WriteFile(*outFile, archive, 0o644); err != nil {
return "", err
}
}
return digest, nil
}
func main() {
digest, err := run(os.Args[1:])
if err != nil {
fmt.Fprintln(os.Stderr, "mesh-pack:", err)
os.Exit(1)
}
fmt.Println(digest)
}
+52
View File
@@ -0,0 +1,52 @@
package main
import (
"os"
"path/filepath"
"testing"
"git.novox.be/novox/mesh-sdk/go/buildstep"
)
func fixture(t *testing.T) string {
t.Helper()
d := t.TempDir()
os.MkdirAll(filepath.Join(d, "sub"), 0o755)
os.WriteFile(filepath.Join(d, "a.txt"), []byte("alpha"), 0o644)
os.WriteFile(filepath.Join(d, "sub", "run"), []byte("#!/bin/sh\n"), 0o755)
return d
}
func TestPackFixtureTwiceGivesBuildstepsDigest(t *testing.T) {
d := fixture(t)
_, want, err := buildstep.Pack(d)
if err != nil {
t.Fatal(err)
}
for i := 0; i < 2; i++ {
got, err := run([]string{"-pack", d, "-out", filepath.Join(t.TempDir(), "x.tgz")})
if err != nil || got != want {
t.Fatalf("run %d: %q, %v; want %q", i, got, err, want)
}
}
}
func TestBuildTwiceGivesOneDigestAndRecipeArgs(t *testing.T) {
m := t.TempDir()
os.WriteFile(filepath.Join(m, "go.mod"), []byte("module x\n\ngo 1.21\n"), 0o644)
os.MkdirAll(filepath.Join(m, "cmd", "p"), 0o755)
os.WriteFile(filepath.Join(m, "cmd", "p", "main.go"), []byte("package main\n\nvar builtFor string\n\nfunc main() { println(builtFor) }\n"), 0o644)
args := []string{"-dir", m, "-from", "cmd/p", "-binary", "p", "-system", "arch"}
a, err := run(args)
if err != nil {
t.Skipf("no go toolchain that builds offline: %v", err)
}
b, err := run(args)
if err != nil || a != b {
t.Fatalf("two builds: %q, %q, %v", a, b, err)
}
got := Args("/d", "./cmd/p/", "/o", "p", "arch")
if got[len(got)-1] != "./cmd/p" || got[4] != "-s -w -X main.builtFor=arch" {
t.Fatalf("args %v", got)
}
}
+94
View File
@@ -0,0 +1,94 @@
package compose
import (
"encoding/json"
"fmt"
"sort"
"git.novox.be/novox/mesh-sdk/go/declaration"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// Compositions is one node's compositions (novox/hq ADR 0539 clauses 1 and 2): its declaration composed over in, as
// the controller composes it, split into each module's composition at the fingerprint fps gives it and the node's own
// (`<node>.node`), by key (delivery.ComposedKey). Each resource says its module, whatever the node reported it reads.
// A module fps gives no fingerprint for is not composed and is named in skipped, so a caller composing one module
// (mesh-hand-compose) gets that one. Each module's Data says whether its manifest declares data. The compositions are not
// signed and not yet by digest: the writer does both (delivery.SignComposed, Composed.ByDigest).
func Compositions(in Input, node string, fps map[string]string, mode Mode) (out map[string]delivery.Composed, skipped []string, err error) {
reading := in
reading.Nodes = append([]Node(nil), in.Nodes...)
found := false
for i := range reading.Nodes {
if reading.Nodes[i].Name == node {
reading.Nodes[i].ReadsModules = true
found = true
}
}
if !found {
return nil, nil, fmt.Errorf("compose: the mesh has no node %q", node)
}
d, problems, err := ComposeIn(reading, node, mode)
if err != nil {
return nil, nil, err
}
if len(problems) > 0 {
return nil, nil, fmt.Errorf("compose: %s's declaration is refused by the node-engine's validator: %v", node, problems)
}
body, err := d.Body()
if err != nil {
return nil, nil, err
}
var envelope struct {
Resources []json.RawMessage `json:"resources"`
Adoption *declaration.Adoption `json:"adoption"`
Filled []string `json:"filled_files"`
Provisioner string `json:"provisioner_account"`
}
if err := json.Unmarshal(body, &envelope); err != nil {
return nil, nil, err
}
var kept []json.RawMessage
keptPaths := map[string]bool{}
left := map[string]bool{}
for _, raw := range envelope.Resources {
var head struct {
Module string `json:"module"`
Type string `json:"type"`
Path string `json:"path"`
}
if err := json.Unmarshal(raw, &head); err != nil {
return nil, nil, err
}
if _, ok := fps[head.Module]; head.Module != "" && !ok {
left[head.Module] = true
continue
}
if head.Type == string(declaration.TypeFile) {
keptPaths[head.Path] = true
}
kept = append(kept, raw)
}
var filled []string
for _, p := range envelope.Filled {
if keptPaths[p] {
filled = append(filled, p)
}
}
out, err = delivery.SplitComposed(node, kept, fps, filled,
delivery.NodeOwn{Adoption: envelope.Adoption, Provisioner: envelope.Provisioner})
if err != nil {
return nil, nil, err
}
for key, c := range out {
if m, ok := in.Catalogue[c.Module]; ok && c.Module != "" {
c.Data = len(m.DataItems()) > 0
out[key] = c
}
}
for m := range left {
skipped = append(skipped, m)
}
sort.Strings(skipped)
return out, skipped, nil
}
+88
View File
@@ -0,0 +1,88 @@
package compose
import (
"encoding/json"
"sort"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
func fpFor(c byte) string { return delivery.SourcePrefix2 + strings.Repeat(string(c), 64) }
// 1943/AC5: a node's compositions are its declaration as Compose gives it, split by the module each resource belongs
// to: every resource once, each in its module's composition at the fingerprint given, the module's ownership said on
// it; a module given no fingerprint is named as skipped and composed nowhere.
func TestCompositionsAreTheDeclarationSplitByModule(t *testing.T) {
in := aSmallMesh(t)
d, _, err := Compose(in, "laptop")
if err != nil {
t.Fatal(err)
}
var want []string
for _, r := range d.Resources {
want = append(want, r["type"].(string)+":"+r["id"].(string))
}
sort.Strings(want)
fps := map[string]string{"mesh-wireguard": fpFor('a'), "files": fpFor('b')}
out, skipped, err := Compositions(in, "laptop", fps, Foreseeing)
if err != nil || len(skipped) != 0 {
t.Fatalf("Compositions = %v, skipped %v", err, skipped)
}
var got []string
for key, c := range out {
node, module, fp, isNode, err := delivery.ParseComposedKey(key)
if err != nil || node != "laptop" || module != c.Module || fp != c.FP || (isNode != (module == "")) {
t.Errorf("key %s holds %s at %s (%v)", key, c.Module, c.FP, err)
}
if !isNode && fp != fps[module] {
t.Errorf("%s is composed at %s, given %s", module, fp, fps[module])
}
if _, err := delivery.EncodeComposed(c); err != nil {
t.Errorf("%s: %v", key, err)
}
for _, raw := range c.Resources {
var r struct{ ID, Type, Module string }
_ = json.Unmarshal(raw, &r)
if r.Module != module {
t.Errorf("%s holds %s of %q", key, r.ID, r.Module)
}
got = append(got, r.Type+":"+r.ID)
}
}
sort.Strings(got)
if strings.Join(got, " ") != strings.Join(want, " ") {
t.Errorf("the compositions hold %v, the declaration %v", got, want)
}
one, skipped, err := Compositions(in, "laptop", map[string]string{"files": fpFor('b')}, Foreseeing)
if err != nil || strings.Join(skipped, ",") != "mesh-wireguard" {
t.Fatalf("one module: %v, skipped %v", err, skipped)
}
for _, c := range one {
if c.Module == "mesh-wireguard" {
t.Errorf("a module given no fingerprint was composed")
}
}
if _, _, err := Compositions(in, "nowhere", fps, Foreseeing); err == nil {
t.Errorf("a node the mesh does not have was composed")
}
}
// 1943/AC1: the upgrade policies and sources a composition's value may name are the catalogue's own words.
func TestTheComposedPoliciesAreTheCatalogues(t *testing.T) {
for _, p := range []string{catalogue.PolicyRoll, catalogue.PolicyTogether, catalogue.PolicyRecord} {
for _, from := range []string{catalogue.FromPerson, catalogue.FromModule, catalogue.FromBus, catalogue.FromData, catalogue.FromDefault} {
v := delivery.ComposedDigest{Digest: "sha256:" + strings.Repeat("a", 64), Judgement: delivery.JudgedNeither, Policy: p, PolicyFrom: from}
if _, err := delivery.EncodeComposedDigest(v); err != nil {
t.Errorf("%s from %s: %v", p, from, err)
}
}
}
if delivery.PolicyRecord != catalogue.PolicyRecord {
t.Errorf("delivery.PolicyRecord = %q, the catalogue's %q", delivery.PolicyRecord, catalogue.PolicyRecord)
}
}
+347
View File
@@ -0,0 +1,347 @@
// Package composer is mesh-sdk's composition and validator as one function over a plain request (novox/hq ADR 0482
// clause 4, issue 1621): what the command mesh-compose runs, so a program that links any one mesh-sdk can have the
// composition of another commit judged by building that commit's command and reading its verdict.
//
// **It reads its request and nothing else.** No bus, store, network, file or environment: the manifests come inside the
// request (raw, as their repository holds them), the rest of the mesh is a compose.Input, and the answer is a value.
// A manifest named only by its digest in the delivery indexer's bucket is filled before the run by Fill, by whoever
// holds the bus (the Go build module's compose job); Run refuses a digest left unfilled.
package composer
import (
"encoding/json"
"errors"
"fmt"
"slices"
"sort"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
"git.novox.be/novox/mesh-sdk/go/sdkversion"
)
// Version is the version of a Request and of a Verdict; a reader refuses another.
const Version = 1
// Package is the import path of the command, which a compose job builds at a commit (in mesh-sdk's own module, or in
// a module that requires mesh-sdk, at the version it pins).
const Package = "git.novox.be/novox/mesh-sdk/go/cmd/mesh-compose"
// The depths of a composition.
const (
// DepthResolve resolves each machine's set over the rest of the mesh (compose.Resolve): what the assigner judges
// while the facts snapshot is not on the bus (issue 1461).
DepthResolve = "resolve"
// DepthCompose composes each machine's whole declaration and runs the node-engine's validator over it
// (compose.Compose).
DepthCompose = "compose"
)
// Request is everything one run composes: the rest of the mesh, one or more cases of manifests composed over it, and
// the machines composed in each case.
type Request struct {
V int `json:"v"`
// Input is the composition's input without its catalogue (and without each node's kept manifests): the cases give
// those. A Catalogue or Kept it holds is ignored.
Input compose.Input `json:"input"`
// Cases are composed one after the other over Input, each with its own manifests; their names are unique.
Cases []Case `json:"cases"`
// Machines are the nodes composed in every case; none means the manifests are only read.
Machines []string `json:"machines,omitempty"`
// Depth is DepthResolve or DepthCompose; empty is DepthCompose.
Depth string `json:"depth,omitempty"`
// HoldingsFromClaims adds to Input's holdings every seat an assigned module claims, held on its node (the
// assigner's input, which has no holdings of record).
HoldingsFromClaims bool `json:"holdings_from_claims,omitempty"`
// Declarations asks each composed machine's declared paths (DepthCompose only).
Declarations bool `json:"declarations,omitempty"`
// Predefined is the list of predefined keys at the catalogue's root (predefined.json, novox/hq ADR 0348 §7) that
// every config value falling back to a key is judged against, when the asker has it (mesh-compose -catalogue
// reads it). Without it a fallback is read as catalogue.ParseManifestForInputs reads it: against the list the mesh
// published when this process holds one, else each key as its value declares it, for the merge gate judges the
// fallback in the catalogue and a manifest that only carries one is not refused here.
Predefined catalogue.PredefinedList `json:"predefined,omitempty"`
}
// Case is one catalogue composed: every module's manifest by name, and per node the manifests kept at the build it runs.
type Case struct {
Name string `json:"name"`
Manifests map[string]Manifest `json:"manifests"`
Kept map[string]map[string]Manifest `json:"kept,omitempty"`
}
// Manifest is a module's manifest as its repository holds it (Raw, the bytes), or the digest the delivery indexer keeps
// it under (Digest), which Fill replaces by the bytes. On the wire it is the manifest itself, or {"digest": "..."}.
type Manifest struct {
Raw json.RawMessage
Digest string
}
// MarshalJSON writes the raw manifest, or {"digest": ...} when only the digest is known.
func (m Manifest) MarshalJSON() ([]byte, error) {
if len(m.Raw) > 0 {
return m.Raw, nil
}
return json.Marshal(map[string]string{"digest": m.Digest})
}
// UnmarshalJSON reads {"digest": "..."} (an object with that one key) as a digest, and any other value as the raw
// manifest.
func (m *Manifest) UnmarshalJSON(raw []byte) error {
var probe map[string]json.RawMessage
if json.Unmarshal(raw, &probe) == nil && len(probe) == 1 {
if d, ok := probe["digest"]; ok {
var digest string
if err := json.Unmarshal(d, &digest); err != nil {
return fmt.Errorf("a manifest's digest is not a string: %w", err)
}
*m = Manifest{Digest: digest}
return nil
}
}
*m = Manifest{Raw: slices.Clone(raw)}
return nil
}
// Verdict is what one run answers: the mesh-sdk the command was built with, and each case's verdict in the order asked.
type Verdict struct {
V int `json:"v"`
SDK string `json:"sdk"`
Cases []CaseVerdict `json:"cases"`
}
// CaseVerdict is one case's: every manifest read or refused, every machine composed or not.
type CaseVerdict struct {
Name string `json:"name"`
Manifests map[string]ManifestRead `json:"manifests"`
Machines map[string]Machine `json:"machines,omitempty"`
}
// ManifestRead is one manifest as this mesh-sdk reads it: accepted, or refused with the reader's words.
type ManifestRead struct {
Accepted bool `json:"accepted"`
Refused string `json:"refused,omitempty"`
}
// Machine is one machine composed: whether it composes (resolves, at DepthResolve), why not, the node-engine
// validator's problems and the composition's refusals (DepthCompose), and its declared paths when asked.
type Machine struct {
Composes bool `json:"composes"`
Reason string `json:"reason,omitempty"`
Problems []string `json:"problems,omitempty"`
Refusals []string `json:"refusals,omitempty"`
Declared []Path `json:"declared,omitempty"`
}
// Path is one path a declaration puts on the machine, with its kind (the resource's type), owner, group and mode as
// declared, and the module it is of.
type Path struct {
Path string `json:"path"`
Kind string `json:"kind"`
Owner string `json:"owner,omitempty"`
Group string `json:"group,omitempty"`
Mode string `json:"mode,omitempty"`
Module string `json:"module,omitempty"`
}
// ErrUnfilled is a manifest named by a digest that nobody replaced by its bytes.
var ErrUnfilled = errors.New("composer: a manifest is named by its digest and was not filled")
// Decode reads a request; it refuses one that is not JSON or of another version.
func Decode(data []byte) (Request, error) {
var r Request
if err := json.Unmarshal(data, &r); err != nil {
return Request{}, fmt.Errorf("composer: the request is not JSON of its shape: %w", err)
}
if r.V != Version {
return Request{}, fmt.Errorf("composer: a request of version %d, this composer reads %d", r.V, Version)
}
return r, nil
}
// Fill replaces every manifest named by its digest by the bytes get answers for that digest; a digest get cannot answer
// stops it, named.
func Fill(r *Request, get func(digest string) ([]byte, error)) error {
fill := func(m map[string]Manifest) error {
for name, man := range m {
if len(man.Raw) > 0 {
continue
}
raw, err := get(man.Digest)
if err != nil {
return fmt.Errorf("composer: the manifest of %s (digest %s): %w", name, man.Digest, err)
}
m[name] = Manifest{Raw: raw}
}
return nil
}
for _, c := range r.Cases {
if err := fill(c.Manifests); err != nil {
return err
}
for _, kept := range c.Kept {
if err := fill(kept); err != nil {
return err
}
}
}
return nil
}
// Run composes every case of r and answers the verdict, its SDK this binary's mesh-sdk. An error is a request it
// cannot run (an unknown depth, a case named twice, a manifest left as a digest); a machine that does not compose is a
// verdict.
func Run(r Request) (Verdict, error) {
depth := r.Depth
if depth == "" {
depth = DepthCompose
}
if depth != DepthResolve && depth != DepthCompose {
return Verdict{}, fmt.Errorf("composer: the depth %q is not resolve or compose", r.Depth)
}
v := Verdict{V: Version, SDK: sdkversion.Version()}
seen := map[string]bool{}
for _, c := range r.Cases {
if seen[c.Name] {
return Verdict{}, fmt.Errorf("composer: the case %q is named twice", c.Name)
}
seen[c.Name] = true
cv, err := runCase(r, c, depth)
if err != nil {
return Verdict{}, err
}
v.Cases = append(v.Cases, cv)
}
return v, nil
}
// readManifest reads one manifest as this run reads every one: against the request's list of predefined keys when it
// has one, else as the indexer and the builder read it (Request.Predefined).
func readManifest(raw []byte, predefined catalogue.PredefinedList) (catalogue.Manifest, error) {
if predefined != nil {
return catalogue.ParseManifestWith(raw, predefined)
}
return catalogue.ParseManifestForInputs(raw)
}
func parse(m map[string]Manifest, reads map[string]ManifestRead, predefined catalogue.PredefinedList) (map[string]catalogue.Manifest, error) {
out := map[string]catalogue.Manifest{}
for name, man := range m {
if len(man.Raw) == 0 {
return nil, fmt.Errorf("%w: %s (digest %s)", ErrUnfilled, name, man.Digest)
}
parsed, err := readManifest(man.Raw, predefined)
if reads != nil {
if err != nil {
reads[name] = ManifestRead{Refused: err.Error()}
} else {
reads[name] = ManifestRead{Accepted: true}
}
}
if err == nil {
out[name] = parsed
}
}
return out, nil
}
func runCase(r Request, c Case, depth string) (CaseVerdict, error) {
cv := CaseVerdict{Name: c.Name, Manifests: map[string]ManifestRead{}}
shelf, err := parse(c.Manifests, cv.Manifests, r.Predefined)
if err != nil {
return CaseVerdict{}, err
}
in := r.Input
in.Catalogue = shelf
in.Nodes = slices.Clone(in.Nodes)
in.Holdings = slices.Clone(in.Holdings)
for i := range in.Nodes {
in.Nodes[i].Kept = nil
if kept := c.Kept[in.Nodes[i].Name]; len(kept) > 0 {
k, err := parse(kept, nil, r.Predefined)
if err != nil {
return CaseVerdict{}, err
}
in.Nodes[i].Kept = k
}
}
if r.HoldingsFromClaims {
in.Holdings = append(in.Holdings, HoldingsOf(shelf, in.Nodes)...)
}
if len(r.Machines) == 0 {
return cv, nil
}
cv.Machines = map[string]Machine{}
for _, node := range r.Machines {
cv.Machines[node] = machine(in, node, depth, r.Declarations)
}
return cv, nil
}
// HoldingsOf is every seat an assigned module claims, held on the node it is assigned to, in node order: the holdings
// of an input that has none of record (the assigner's, mesh-delivery-assigner internal/assign input).
func HoldingsOf(shelf map[string]catalogue.Manifest, nodes []compose.Node) []catalogue.Held {
var out []catalogue.Held
sorted := slices.Clone(nodes)
sort.Slice(sorted, func(i, j int) bool { return sorted[i].Name < sorted[j].Name })
for _, n := range sorted {
for _, mod := range n.Assigned {
for _, cl := range shelf[mod].Claims {
scope := cl.Scope
if seat, ok := catalogue.SeatNamed(cl.Name); ok && scope == "" {
scope = seat.Scope
}
out = append(out, catalogue.Held{Claim: cl.Name, Scope: scope, Node: n.Name, Module: mod})
}
}
}
return out
}
func machine(in compose.Input, node, depth string, declarations bool) Machine {
if depth == DepthResolve {
if _, err := compose.Resolve(in, node); err != nil {
return Machine{Reason: err.Error()}
}
return Machine{Composes: true}
}
d, problems, err := compose.Compose(in, node)
if err != nil {
return Machine{Reason: err.Error()}
}
m := Machine{Composes: len(problems) == 0, Problems: problems, Refusals: d.Refusals()}
if len(problems) > 0 {
m.Reason = "the node-engine's validator refuses the declaration"
}
if declarations {
m.Declared = pathsOf(d)
}
return m
}
// pathsOf is every resource of a declaration that names a path, with its owner, group and mode as declared, sorted by
// path.
func pathsOf(d compose.Declared) []Path {
var out []Path
str := func(r map[string]any, key string) string {
if v, ok := r[key]; ok && v != nil {
return fmt.Sprint(v)
}
return ""
}
for _, r := range d.Resources {
p := str(r, "path")
if p == "" {
continue
}
out = append(out, Path{Path: p, Kind: str(r, "type"), Owner: str(r, "owner"), Group: str(r, "group"),
Mode: str(r, "mode"), Module: d.ModuleOf[str(r, "id")]})
}
sort.Slice(out, func(i, j int) bool {
if out[i].Path != out[j].Path {
return out[i].Path < out[j].Path
}
return out[i].Kind < out[j].Kind
})
return out
}
+137
View File
@@ -0,0 +1,137 @@
package composer
import (
"encoding/json"
"errors"
"os"
"reflect"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/compose"
)
const demo = `{"module": "demo", "version": "1", "claims": [{"name": "node-files"}]}`
// A manifest named by its digest goes on the wire as {"digest": ...}, and Fill puts the bytes the indexer keeps in its
// place; a raw manifest goes as itself.
func TestAManifestIsItsBytesOrItsDigest(t *testing.T) {
r := Request{V: Version, Cases: []Case{{Name: "change", Manifests: map[string]Manifest{
"demo": {Digest: "d1"}, "raw": {Raw: json.RawMessage(`{"module":"raw","version":"1"}`)}},
Kept: map[string]map[string]Manifest{"laptop": {"demo": {Digest: "d1"}}}}}}
wire, err := json.Marshal(r)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(wire), `"demo":{"digest":"d1"}`) || !strings.Contains(string(wire), `"raw":{"module":"raw","version":"1"}`) {
t.Fatalf("on the wire: %s", wire)
}
back, err := Decode(wire)
if err != nil {
t.Fatal(err)
}
if _, err := Run(back); !errors.Is(err, ErrUnfilled) {
t.Fatalf("a digest left unfilled runs: %v", err)
}
asked := 0
if err := Fill(&back, func(d string) ([]byte, error) {
asked++
if d != "d1" {
return nil, errors.New("no such digest")
}
return []byte(demo), nil
}); err != nil {
t.Fatal(err)
}
if asked != 2 || string(back.Cases[0].Manifests["demo"].Raw) != demo || string(back.Cases[0].Kept["laptop"]["demo"].Raw) != demo {
t.Fatalf("filled %d times: %+v", asked, back.Cases[0])
}
bad := Request{V: Version, Cases: []Case{{Name: "c", Manifests: map[string]Manifest{"x": {Digest: "nope"}}}}}
if err := Fill(&bad, func(string) ([]byte, error) { return nil, errors.New("not there") }); err == nil || !strings.Contains(err.Error(), "nope") {
t.Fatalf("a digest the indexer does not hold: %v", err)
}
}
// The input travels as JSON and comes back equal, so the command composes what its caller meant.
func TestTheInputTravelsAsJSON(t *testing.T) {
in := compose.Input{
Nodes: []compose.Node{{Name: "anchor", Assigned: []string{"nats"}, Capabilities: map[string]bool{"systemd": true},
Place: compose.Place{Endpoint: "anchor.example:51820", Site: "here", Hub: true, Address: "10.77.0.1"},
Pins: map[string]catalogue.Chosen{"bus": {Node: "anchor", Module: "nats"}},
Settings: map[string][]catalogue.Layer{"nats": {{Values: map[string]any{"port": float64(4222)}}}}}},
Holdings: []catalogue.Held{{Claim: "mesh-broker", Scope: "mesh", Node: "anchor", Module: "nats"}},
Issued: []compose.Issued{{Provision: "bus", Consumer: "laptop", ConsumerModule: "x", Provider: "anchor"}},
}
raw, err := json.Marshal(Request{V: Version, Input: in})
if err != nil {
t.Fatal(err)
}
back, err := Decode(raw)
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(back.Input, in) {
t.Fatalf("the input came back as\n%+v\nwant\n%+v", back.Input, in)
}
}
// The seats a node's assigned modules claim are held there when the request says the holdings come from the claims.
func TestHoldingsComeFromTheClaims(t *testing.T) {
m, err := catalogue.ParseManifest([]byte(demo))
if err != nil {
t.Fatal(err)
}
got := HoldingsOf(map[string]catalogue.Manifest{"demo": m}, []compose.Node{{Name: "b", Assigned: []string{"demo"}}, {Name: "a", Assigned: []string{"demo"}}})
if len(got) != 2 || got[0].Node != "a" || got[0].Claim != "node-files" || got[0].Scope != "node" {
t.Fatalf("holdings = %+v", got)
}
}
// A request the command cannot run is refused, never answered as a verdict: another version, a case named twice, an
// unknown depth.
func TestARequestItCannotRunIsRefused(t *testing.T) {
if _, err := Decode([]byte(`{"v": 2}`)); err == nil {
t.Error("a request of version 2 is read")
}
if _, err := Run(Request{V: Version, Cases: []Case{{Name: "a"}, {Name: "a"}}}); err == nil {
t.Error("a case named twice runs")
}
if _, err := Run(Request{V: Version, Depth: "deep"}); err == nil {
t.Error("an unknown depth runs")
}
v, err := Run(Request{V: Version, Cases: []Case{{Name: "only-read", Manifests: map[string]Manifest{"demo": {Raw: json.RawMessage(demo)}}}}})
if err != nil || len(v.Cases) != 1 || !v.Cases[0].Manifests["demo"].Accepted || v.Cases[0].Machines != nil || v.SDK == "" {
t.Fatalf("a request with no machines: %+v, %v", v, err)
}
}
// A manifest whose config value falls back to a predefined key (novox/hq ADR 0348 §7) is read, not refused for the
// list of predefined keys at the catalogue's root that a run is never handed with its manifests: the merge gate judges
// the fallback in the catalogue. xterm's module.json from mesh-catalog at 835bf22b falls back to "font-size"; a group
// touching xterm or pacman failed its composition on this (group check 18df81380db3). A list in the request is the one
// judged against, so a fallback it does not name is still refused.
func TestAManifestFallingBackToAPredefinedKeyIsRead(t *testing.T) {
raw, err := os.ReadFile("../catalogue/testdata/catalogue-835bf22b-xterm.module.json")
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(raw), `"predefined"`) {
t.Fatal("the fixture names no predefined key")
}
r := Request{V: Version, Cases: []Case{{Name: "change", Manifests: map[string]Manifest{"xterm": {Raw: raw}}}}}
v, err := Run(r)
if err != nil {
t.Fatal(err)
}
if got := v.Cases[0].Manifests["xterm"]; !got.Accepted {
t.Fatalf("xterm: %+v, want accepted", got)
}
r.Predefined = catalogue.PredefinedList{"something-else": {Type: "string", Description: "x"}}
if v, err = Run(r); err != nil {
t.Fatal(err)
}
if got := v.Cases[0].Manifests["xterm"]; got.Accepted || !strings.Contains(got.Refused, "not on the list of predefined keys") {
t.Fatalf("xterm judged against a list without its key: %+v, want refused naming the list", got)
}
}
+47
View File
@@ -0,0 +1,47 @@
package declaration
import (
"encoding/json"
"fmt"
)
// ParseResource reads one resource as Parse reads each of a declaration's resources: by its type, strictly, refusing
// a field its kind does not use and anything its kind refuses, and never an action (the link may not carry one, ADR
// 0005). It judges the resource alone: what Parse judges across resources (an opening on a node not adopted, the
// files the provisioner fills, what replaces what) is for the whole a node applies (novox/hq ADR 0539 clause 1: a
// composition is one module's resources, and a node applies its compositions together with its own key).
func ParseResource(raw json.RawMessage) (Resource, error) {
var head struct {
ID string `json:"id"`
Type Type `json:"type"`
}
if err := json.Unmarshal(raw, &head); err != nil {
return nil, &RefusalError{Problems: []string{"not a resource: " + err.Error()}}
}
where := "resource"
if head.ID == "" {
return nil, &RefusalError{Problems: []string{where + ": no id. Identity is what lets the host know this is " +
"the same resource it applied last time"}}
}
where = fmt.Sprintf("resource %q", head.ID)
resource := newOf(head.Type)
if resource == nil {
return nil, &RefusalError{Problems: []string{fmt.Sprintf("%s: unknown type %q. This host understands %s",
where, head.Type, vocabulary())}}
}
if unknown := unknownFields(raw, resource); len(unknown) > 0 {
var problems []string
for _, field := range unknown {
problems = append(problems, fmt.Sprintf("%s: a %s does not use %q, and it is set. Refused rather "+
"than ignored", where, head.Type, field))
}
return nil, &RefusalError{Problems: problems}
}
if err := json.Unmarshal(raw, resource); err != nil {
return nil, &RefusalError{Problems: []string{fmt.Sprintf("%s: %s", where, err)}}
}
if problems := resource.validate(where, false); len(problems) > 0 {
return nil, &RefusalError{Problems: problems}
}
return resource, nil
}
+510
View File
@@ -0,0 +1,510 @@
package delivery
import (
"bytes"
"encoding/json"
"fmt"
"reflect"
"strings"
)
// The assigner's keys (novox/hq ADR 0528 clauses 1 and 4, ADR 0429 §6, issue 1297). The assigner writes, by the six
// rules of ADR 0429 §6, what each node should run: `want.<node>.<module>` in its bucket wants, and beside it
// `held.<module>.<fingerprint>` for a fingerprint it holds after a failed health check. A module that leaves a node's
// modules gets its want written removed, never deleted (ADR 0408 §4). For each node and module it compares the want with
// what the controller sends (the build the controller's assignment names) and writes every difference under
// `<node>.<module>` in its bucket differences, with the rule that explains it or unexplained. In release 5 wants are
// only compared; the node agent applying them is release 6 (ADR 0429 §7). Only the assigner's credential writes these
// buckets; the watchman, the view and the controller read them.
//
// **Release 6** (ADR 0539 clauses 1, 6 and 16): a want also names the digest of the one composition it allows, written
// only once that composition exists (the seventh rule, RuleComposed), and carries the answer to a failed backup; the
// node's own want `want.<node>.node` names the digest of the node's own composition (NodeWant).
//
// **The bucket is the first word, the key the rest** (as the builder's `builds.<module>.<fp>` is the key
// `<module>.<fp>` in its builds bucket): the differences bucket's keys are `<node>.<module>`. The wants bucket holds
// two kinds of key, so each keeps its word: `want.<node>.<module>` and `held.<module>.<fingerprint hex>`.
const (
// AssignerModule is the module of the assigner part.
AssignerModule = "mesh-delivery-assigner"
// AssignerWantsBucket is the assigner's bucket of wants and holds, `<owner module>_<name>`.
AssignerWantsBucket = AssignerModule + "_wants"
// AssignerWantsHistory is the revisions the wants bucket keeps per key, so a want's history shows each move and
// the operator's manual write (ADR 0429 §6, the one escape).
AssignerWantsHistory = 20
// AssignerDifferencesBucket is the assigner's bucket of differences between a want and what the controller sends.
AssignerDifferencesBucket = AssignerModule + "_differences"
// AssignerDifferencesHistory is the revisions the differences bucket keeps per key: acceptance test 17 of design 47
// reads a day of them.
AssignerDifferencesHistory = 64
)
// The words that open the wants bucket's two kinds of key.
const (
wantWord = "want"
heldWord = "held"
)
// WantKey is the key, in AssignerWantsBucket, of a node's want for a module: `want.<node>.<module>`. A module named
// `node` is refused, since `want.<node>.node` is the node's own key (ADR 0429 §4), as HaveKey refuses it.
func WantKey(node, module string) (string, error) {
have, err := HaveKey(node, module)
if err != nil {
return "", err
}
return wantWord + "." + have, nil
}
// WantNodeKey is the key, in AssignerWantsBucket, of a node's own want: `want.<node>.node`, naming the digest of the
// node's own composition (ADR 0539 clause 6; ADR 0429 §4's node key). The assigner writes it once the node's own
// composition exists.
func WantNodeKey(node string) (string, error) {
own, err := HaveNodeKey(node)
if err != nil {
return "", err
}
return wantWord + "." + own, nil
}
// ParseWantNodeKey is the inverse of WantNodeKey; it refuses a module's want key and any key of another bucket,
// naming it.
func ParseWantNodeKey(key string) (node string, err error) {
rest, ok := strings.CutPrefix(key, wantWord+".")
if !ok {
return "", fmt.Errorf("delivery: %q is not a node's own want key, want.<node>.node", key)
}
node, _, isNode, err := ParseHaveKey(rest)
if err != nil || !isNode {
return "", fmt.Errorf("delivery: %q is not a node's own want key, want.<node>.node", key)
}
return node, nil
}
// ParseWantKey is the inverse of WantKey; it refuses a held key, a have key, the node's own want key and any key of
// another bucket, naming it.
func ParseWantKey(key string) (node, module string, err error) {
rest, ok := strings.CutPrefix(key, wantWord+".")
if !ok {
return "", "", fmt.Errorf("delivery: %q is not a want key, want.<node>.<module>", key)
}
node, module, isNode, err := ParseHaveKey(rest)
if err != nil || isNode {
return "", "", fmt.Errorf("delivery: %q is not a want key, want.<node>.<module>", key)
}
return node, module, nil
}
// HeldKey is the key, in AssignerWantsBucket, of a module's fingerprint the assigner holds: `held.<module>.<hex>`,
// the hex being the fingerprint's key part (FingerprintKeyPart).
func HeldKey(module, fingerprint string) (string, error) {
key, err := BuilderBuildKey(module, fingerprint)
if err != nil {
return "", err
}
return heldWord + "." + key, nil
}
// ParseHeldKey is the inverse of HeldKey: the module and the fingerprint (`src2:` and the hex); it refuses a want key
// and any key of another bucket, naming it.
func ParseHeldKey(key string) (module, fingerprint string, err error) {
rest, ok := strings.CutPrefix(key, heldWord+".")
if !ok {
return "", "", fmt.Errorf("delivery: %q is not a held key, held.<module>.<fingerprint hex>", key)
}
if module, fingerprint, err = ParseBuilderBuildKey(rest); err != nil {
return "", "", fmt.Errorf("delivery: %q is not a held key: %w", key, err)
}
return module, fingerprint, nil
}
// DifferenceKey is the key, in AssignerDifferencesBucket, of the difference for a module on a node: `<node>.<module>`.
func DifferenceKey(node, module string) (string, error) {
return HaveKey(node, module)
}
// ParseDifferenceKey is the inverse of DifferenceKey; it refuses a want or held key and a node's own key, naming it.
func ParseDifferenceKey(key string) (node, module string, err error) {
node, module, isNode, err := ParseHaveKey(key)
if err != nil || isNode {
return "", "", fmt.Errorf("delivery: %q is not a difference key, <node>.<module>", key)
}
return node, module, nil
}
// The rules of ADR 0429 §6 and ADR 0539 clause 6, by number, as a want or a difference names the one that holds a node
// back.
const (
RuleBuildDone = 1 // the build of F is done
RuleNotHeld = 2 // F is not held
RuleCanary = 3 // the node is the module's canary, or the canary runs F healthy
RuleStep = 4 // at most step other nodes are ahead (step 0: no limit, ADR 0528 §3)
RuleProviders = 5 // every provider the module requires runs healthy on that node, or at its canary
RuleGrant = 6 // the module's credential for F exists (grants.<module>.<fp>)
RuleComposed = 7 // the module's composition for that node at F exists (mesh-controller_composed, ADR 0539 §6)
)
func checkRule(rule int, of string) error {
if rule < RuleBuildDone || rule > RuleComposed {
return fmt.Errorf("delivery: %s names the rule %d, which is not one of the seven (1 to 7)", of, rule)
}
return nil
}
// The answers a want carries to a failed backup before an apply (ADR 0539 clause 16): the node agent reads them from
// its own want and from nothing else.
const (
// BackupAgain is the answer to back the module up again before the apply.
BackupAgain = "again"
// BackupSkip is the answer to go on without a backup.
BackupSkip = "skip"
)
func checkFingerprint(fp, of string) error {
if _, err := FingerprintKeyPart(fp); err != nil {
return fmt.Errorf("delivery: %s: %w", of, err)
}
return nil
}
// Want is a node's want for a module: the fingerprint FP the node should run and the digest Composed of the one
// composition of it the node may apply (ADR 0539 clause 1), or Removed for a module that left the node's modules (then
// both are empty). When the module's newest fingerprint is not yet allowed on this node, Newest names it and Rule the
// rule that holds the node back; both are empty when FP is the newest. Backup is the answer to a failed backup before
// the apply, BackupAgain or BackupSkip (clause 16), empty when none was asked. Words say why in a sentence. The
// entry's own time says since when.
type Want struct {
V int `json:"v"`
FP string `json:"fp,omitempty"`
Composed string `json:"composed,omitempty"`
Removed bool `json:"removed,omitempty"`
Newest string `json:"newest,omitempty"`
Rule int `json:"rule,omitempty"`
Backup string `json:"backup,omitempty"`
Words string `json:"words,omitempty"`
// Put says the want was written by a put (ADR 0546), and Hold that the put holds it: the rules pass over it until it
// is released. A put's want names no newest fingerprint or rule.
Put bool `json:"put,omitempty"`
Hold bool `json:"hold,omitempty"`
}
func (w Want) check() error {
if w.Removed {
if w.FP != "" || w.Composed != "" || w.Newest != "" || w.Rule != 0 || w.Backup != "" || w.Put || w.Hold {
return fmt.Errorf("delivery: a removed want names no fingerprint, composition, rule, backup answer or put")
}
return nil
}
if w.Hold && !w.Put {
return fmt.Errorf("delivery: only a put holds a want")
}
if w.Put && (w.Newest != "" || w.Rule != 0) {
return fmt.Errorf("delivery: a put's want is held back by no rule")
}
if err := checkFingerprint(w.FP, "a want's fingerprint"); err != nil {
return err
}
if w.Composed == "" {
return fmt.Errorf("delivery: the want of %s names no composition: a want names the digest of the one composition it allows (ADR 0539 clause 1)", w.FP)
}
if err := checkDigest(w.Composed, "a want's composition digest"); err != nil {
return err
}
switch w.Backup {
case "", BackupAgain, BackupSkip:
default:
return fmt.Errorf("delivery: the backup answer %q is not %s or %s", w.Backup, BackupAgain, BackupSkip)
}
if w.Newest == "" && w.Rule == 0 {
return nil
}
if err := checkFingerprint(w.Newest, "a want's newest fingerprint"); err != nil {
return err
}
if w.Newest == w.FP {
return fmt.Errorf("delivery: a want at its newest fingerprint names no rule holding it back")
}
return checkRule(w.Rule, "a want held back")
}
// Held is a fingerprint the assigner holds: the node whose health check failed on it, and the check's words.
type Held struct {
V int `json:"v"`
Node string `json:"node"`
Words string `json:"words,omitempty"`
}
func (h Held) check() error {
return checkName("node", h.Node)
}
// Difference is where a want and what the controller sends differ for a module on a node (ADR 0528 clause 4): the
// want's fingerprint (Want, empty when the node has no want for the module, WantRemoved when the want is removed), the
// fingerprint of the build the controller's assignment names (Sent, empty when it sends none or the build names no
// fingerprint; SentBuild names that build), and the rule that explains the difference, or Unexplained.
type Difference struct {
V int `json:"v"`
Want string `json:"want,omitempty"`
WantRemoved bool `json:"want-removed,omitempty"`
Sent string `json:"sent,omitempty"`
SentBuild string `json:"sent-build,omitempty"`
Rule int `json:"rule,omitempty"`
Unexplained bool `json:"unexplained,omitempty"`
Words string `json:"words,omitempty"`
}
func (d Difference) check() error {
if d.WantRemoved && d.Want != "" {
return fmt.Errorf("delivery: a difference with a removed want names no wanted fingerprint")
}
if d.Want != "" {
if err := checkFingerprint(d.Want, "a difference's want"); err != nil {
return err
}
}
if d.Sent != "" {
if err := checkFingerprint(d.Sent, "a difference's sent fingerprint"); err != nil {
return err
}
}
if !d.WantRemoved && d.Want == d.Sent && (d.Want != "" || d.SentBuild == "") {
return fmt.Errorf("delivery: a difference whose want and send are both %q is no difference", d.Want)
}
if d.Unexplained {
if d.Rule != 0 {
return fmt.Errorf("delivery: an unexplained difference names no rule")
}
return nil
}
return checkRule(d.Rule, "a difference")
}
// EncodeWant is w as JSON at Version; it refuses a fingerprint not of the new form, a removed want naming one, and a
// want held back without its rule.
func EncodeWant(w Want) ([]byte, error) {
if err := w.check(); err != nil {
return nil, err
}
w.V = Version
return json.Marshal(w)
}
// DecodeWant reads a want; it refuses an unknown version (ErrUnknownVersion) and what EncodeWant refuses.
func DecodeWant(data []byte) (Want, error) {
var w Want
if err := decode(data, &w); err != nil {
return Want{}, err
}
if err := w.check(); err != nil {
return Want{}, err
}
return w, nil
}
// SameWant says whether two wants are the same value, so the assigner writes only when they differ.
func SameWant(a, b Want) bool { return a == b }
// NodeWant is a node's own want (WantNodeKey): the digest Composed of the node's own composition it may apply. Words
// say why in a sentence.
type NodeWant struct {
V int `json:"v"`
Composed string `json:"composed"`
Words string `json:"words,omitempty"`
}
// EncodeNodeWant is w as JSON at Version; it refuses a digest not of the sha256 form.
func EncodeNodeWant(w NodeWant) ([]byte, error) {
if err := checkDigest(w.Composed, "the node's own want's composition digest"); err != nil {
return nil, err
}
w.V = Version
return json.Marshal(w)
}
// DecodeNodeWant reads a node's own want; it refuses an unknown version (ErrUnknownVersion), a field a node's own want
// does not have (a fingerprint among them), and what EncodeNodeWant refuses.
func DecodeNodeWant(data []byte) (NodeWant, error) {
if err := checkVersion(data); err != nil {
return NodeWant{}, err
}
var w NodeWant
d := json.NewDecoder(bytes.NewReader(data))
d.DisallowUnknownFields()
if err := d.Decode(&w); err != nil {
return NodeWant{}, fmt.Errorf("delivery: a node's own want does not fit its shape: %w", err)
}
if err := checkDigest(w.Composed, "the node's own want's composition digest"); err != nil {
return NodeWant{}, err
}
return w, nil
}
// EncodeHeld is h as JSON at Version; it refuses a node name a key refuses.
func EncodeHeld(h Held) ([]byte, error) {
if err := h.check(); err != nil {
return nil, err
}
h.V = Version
return json.Marshal(h)
}
// DecodeHeld reads a hold; it refuses an unknown version (ErrUnknownVersion) and what EncodeHeld refuses.
func DecodeHeld(data []byte) (Held, error) {
var h Held
if err := decode(data, &h); err != nil {
return Held{}, err
}
if err := h.check(); err != nil {
return Held{}, err
}
return h, nil
}
// SameHeld says whether two holds are the same value.
func SameHeld(a, b Held) bool { return a == b }
// EncodeDifference is d as JSON at Version; it refuses a difference with neither a rule nor unexplained, one with both,
// a rule outside the six, and a want equal to the send.
func EncodeDifference(d Difference) ([]byte, error) {
if err := d.check(); err != nil {
return nil, err
}
d.V = Version
return json.Marshal(d)
}
// DecodeDifference reads a difference; it refuses an unknown version (ErrUnknownVersion) and what EncodeDifference
// refuses.
func DecodeDifference(data []byte) (Difference, error) {
var d Difference
if err := decode(data, &d); err != nil {
return Difference{}, err
}
if err := d.check(); err != nil {
return Difference{}, err
}
return d, nil
}
// SameDifference says whether two differences are the same value.
func SameDifference(a, b Difference) bool { return reflect.DeepEqual(a, b) }
// The assigner's composition check (novox/hq ADR 0446 clauses 1 to 5, ADR 0528 clause 5, ADR 0482 clauses 3 and 9): for
// each group the checker judges, the assigner composes the machines the change touches and writes its verdict under
// the checker's key of that check, `<escaped group>.<heads hash>` (CheckerVerdictKey's form), in its bucket
// compositions. The checker watches that key and reads it into its verdict's composition (Composition); a key not yet
// written is a composition still running.
const (
// AssignerCompositionsBucket is the assigner's bucket of composition verdicts.
AssignerCompositionsBucket = AssignerModule + "_compositions"
// AssignerCompositionsHistory is the revisions the bucket keeps per key.
AssignerCompositionsHistory = 5
// CompositionWaits is the state of a composition that waits for what it composes from (ADR 0446 clause 3: the
// indexer holds no candidates for the group yet), its Words saying what; it is no verdict.
CompositionWaits = "waits"
// CompositionRebuildWarning is the most modules a merge may rebuild before the composition warns (ADR 0446 clause 1).
CompositionRebuildWarning = 12
)
// CompositionKey is the key, in AssignerCompositionsBucket, of the composition of a group's check at its heads: the
// checker's verdict key of the same check (CheckerVerdictKey).
func CompositionKey(group, headsHash string) (string, error) {
return CheckerVerdictKey(group, headsHash)
}
// ParseCompositionKey is the inverse of CompositionKey; it refuses a want, held or difference key, naming it.
func ParseCompositionKey(key string) (group, headsHash string, err error) {
if group, headsHash, err = ParseCheckerVerdictKey(key); err != nil {
return "", "", fmt.Errorf("delivery: %q is not a composition key, <group>.<heads hash>: %w", key, err)
}
return group, headsHash, nil
}
// CompositionVerdict is the value of a composition key: the group and its heads hash, the State (pass, fail or waits),
// the Words of a refusal or of what it waits for, the mesh-sdk commit it composed with (the assigner's own, or the
// group's mesh-sdk member's head, ADR 0482 clause 3), the machines composed, the first machine that failed, the
// warnings that fail nothing (more than twelve modules rebuilt; what was already broken before the change, ADR 0446
// clause 1), and the time it took in integer milliseconds.
type CompositionVerdict struct {
V int `json:"v"`
Group string `json:"group"`
Heads string `json:"heads"`
State string `json:"state"`
Words string `json:"words,omitempty"`
SDK string `json:"sdk,omitempty"`
Machines []string `json:"machines,omitempty"`
Machine string `json:"machine,omitempty"`
Warnings []string `json:"warnings,omitempty"`
Took int64 `json:"took,omitempty"`
}
func (c CompositionVerdict) check() error {
if c.Group == "" {
return fmt.Errorf("delivery: a composition names no group")
}
if !isHex12(c.Heads) {
return fmt.Errorf("delivery: the composition's heads hash %q is not 12 lowercase hex digits", c.Heads)
}
if c.Took < 0 {
return fmt.Errorf("delivery: the composition's time is negative")
}
switch c.State {
case CompositionPass:
if c.SDK == "" {
return fmt.Errorf("delivery: a composition that passed names no mesh-sdk commit")
}
if c.Machine != "" || c.Words != "" {
return fmt.Errorf("delivery: a composition that passed names no failure")
}
case CompositionFail:
if c.Words == "" {
return fmt.Errorf("delivery: a composition that failed gives no words")
}
if c.SDK == "" {
return fmt.Errorf("delivery: a composition that failed names no mesh-sdk commit")
}
case CompositionWaits:
if c.Words == "" {
return fmt.Errorf("delivery: a composition that waits does not say for what")
}
if c.Machine != "" || len(c.Machines) > 0 {
return fmt.Errorf("delivery: a composition that waits composed no machine")
}
default:
return fmt.Errorf("delivery: the composition state %q is not %s, %s or %s", c.State, CompositionPass, CompositionFail, CompositionWaits)
}
return nil
}
// Composition is what the checker writes into its verdict from c: nil while c waits (the check is still running).
func (c CompositionVerdict) Composition() *Composition {
if c.State == CompositionWaits {
return nil
}
return &Composition{State: c.State, Words: c.Words, SDK: c.SDK, Machines: c.Machines}
}
// EncodeCompositionVerdict is c as JSON at Version; it refuses an unknown state, a pass or fail naming no mesh-sdk
// commit, a fail without words, a pass naming a failure, and a wait that composed a machine.
func EncodeCompositionVerdict(c CompositionVerdict) ([]byte, error) {
if err := c.check(); err != nil {
return nil, err
}
c.V = Version
return json.Marshal(c)
}
// DecodeCompositionVerdict reads a composition; it refuses an unknown version (ErrUnknownVersion) and what
// EncodeCompositionVerdict refuses.
func DecodeCompositionVerdict(data []byte) (CompositionVerdict, error) {
var c CompositionVerdict
if err := decode(data, &c); err != nil {
return CompositionVerdict{}, err
}
if err := c.check(); err != nil {
return CompositionVerdict{}, err
}
return c, nil
}
// SameCompositionVerdict says whether two compositions are the same value.
func SameCompositionVerdict(a, b CompositionVerdict) bool { return reflect.DeepEqual(a, b) }
+317
View File
@@ -0,0 +1,317 @@
package delivery
import (
"errors"
"strings"
"testing"
"time"
)
// 1297/B1a: a want key round-trips through its parser; a held key, a have key and a node's own key are refused,
// naming the key.
func TestWantKey(t *testing.T) {
key, err := WantKey("ace", "mesh-mail")
if err != nil || key != "want.ace.mesh-mail" {
t.Fatalf("WantKey = %q, %v", key, err)
}
node, module, err := ParseWantKey(key)
if err != nil || node != "ace" || module != "mesh-mail" {
t.Errorf("ParseWantKey = %q, %q, %v", node, module, err)
}
for _, n := range [][2]string{{"ace", "node"}, {"a.b", "m"}, {"ace", ""}, {"", "m"}, {"ace", "m*"}} {
if _, err := WantKey(n[0], n[1]); err == nil {
t.Errorf("WantKey(%q, %q) accepted", n[0], n[1])
}
}
held, _ := HeldKey("mesh-mail", fp2of('a'))
for _, bad := range []string{"ace.mesh-mail", "want.ace.node", "want.ace", "want.ace.m.x", held, "wants.ace.m", ""} {
if _, _, err := ParseWantKey(bad); err == nil || !strings.Contains(err.Error(), "\""+bad+"\"") {
t.Errorf("ParseWantKey(%q) = %v, want a refusal naming it", bad, err)
}
}
}
// 1297/B1a: a held key round-trips; a want key, a builder build key and a fingerprint not of the new form are refused.
func TestHeldKey(t *testing.T) {
fp := fp2of('b')
key, err := HeldKey("mesh-mail", fp)
if err != nil || key != "held.mesh-mail."+strings.Repeat("b", 64) {
t.Fatalf("HeldKey = %q, %v", key, err)
}
module, back, err := ParseHeldKey(key)
if err != nil || module != "mesh-mail" || back != fp {
t.Errorf("ParseHeldKey = %q, %q, %v", module, back, err)
}
src1, _ := SourceFingerprint(SourceInputs{Module: "m", Tree: "t"})
if _, err := HeldKey("mesh-mail", src1); err == nil {
t.Errorf("a src1 fingerprint was held")
}
build, _ := BuilderBuildKey("mesh-mail", fp)
for _, bad := range []string{build, "want.ace.mesh-mail", "held.mesh-mail.abc", "held." + strings.Repeat("b", 64), ""} {
if _, _, err := ParseHeldKey(bad); err == nil || !strings.Contains(err.Error(), "\""+bad+"\"") {
t.Errorf("ParseHeldKey(%q) = %v, want a refusal naming it", bad, err)
}
}
}
// 1297/B1a: a difference key round-trips; a want or held key and a node's own key are refused.
func TestDifferenceKey(t *testing.T) {
key, err := DifferenceKey("g14", "mesh-mail")
if err != nil || key != "g14.mesh-mail" {
t.Fatalf("DifferenceKey = %q, %v", key, err)
}
node, module, err := ParseDifferenceKey(key)
if err != nil || node != "g14" || module != "mesh-mail" {
t.Errorf("ParseDifferenceKey = %q, %q, %v", node, module, err)
}
held, _ := HeldKey("mesh-mail", fp2of('a'))
for _, bad := range []string{"want.g14.mesh-mail", held, "g14.node", "g14", ""} {
if _, _, err := ParseDifferenceKey(bad); err == nil || !strings.Contains(err.Error(), "\""+bad+"\"") {
t.Errorf("ParseDifferenceKey(%q) = %v, want a refusal naming it", bad, err)
}
}
}
// 1297/B1a: wants, holds and differences round-trip through their encoders unchanged, at Version.
func TestAssignerValuesRoundTrip(t *testing.T) {
wants := []Want{
{FP: fp2of('a'), Composed: hashOf('1')},
{FP: fp2of('a'), Composed: hashOf('1'), Newest: fp2of('b'), Rule: RuleCanary, Words: "the canary ace does not run it healthy yet"},
{Removed: true, Words: "the module left g14"},
{FP: fp2of('a'), Composed: hashOf('1'), Put: true, Hold: true, Words: "put by the operator: the bus's user list"},
}
for _, w := range wants {
data, err := EncodeWant(w)
if err != nil {
t.Fatalf("EncodeWant(%+v): %v", w, err)
}
got, err := DecodeWant(data)
w.V = Version
if err != nil || !SameWant(got, w) {
t.Errorf("want %+v came back %+v, %v", w, got, err)
}
}
at := time.Date(2026, 10, 25, 21, 0, 0, 0, time.UTC)
p := Put{FP: fp2of('a'), Hold: true, Why: "the bus's user list", By: "shanks.node-tools", At: at}.Said(at, PutAsked, "put asked")
p = p.Said(at, PutAsked, "put asked").Said(at.Add(time.Second), PutWanted, "want written")
p.Composed = hashOf('1')
if len(p.Steps) != 2 || p.Step != PutWanted || PutEnded(p.Step) || !PutEnded(PutReleased) {
t.Errorf("a put's steps are %+v at %s", p.Steps, p.Step)
}
pkey, _ := PutKey("novox", "nats")
if n, m, err := ParsePutKey(pkey); err != nil || n != "novox" || m != "nats" {
t.Errorf("the put key %s reads %s %s, %v", pkey, n, m, err)
}
pdata, err := EncodePut(p)
if err != nil {
t.Fatal(err)
}
if gotP, err := DecodePut(pdata); err != nil || gotP.FP != p.FP || gotP.Step != PutWanted || len(gotP.Steps) != 2 || !gotP.Hold {
t.Errorf("the put came back %+v, %v", gotP, err)
}
h := Held{Node: "ace", Words: "its health check failed: connection refused"}
data, err := EncodeHeld(h)
if err != nil {
t.Fatal(err)
}
gotH, err := DecodeHeld(data)
h.V = Version
if err != nil || !SameHeld(gotH, h) {
t.Errorf("held came back %+v, %v", gotH, err)
}
diffs := []Difference{
{Want: fp2of('a'), Sent: fp2of('b'), Rule: RuleCanary},
{Sent: fp2of('b'), Rule: RuleBuildDone},
{WantRemoved: true, Sent: fp2of('b'), Unexplained: true},
{SentBuild: "build-17", Rule: RuleGrant},
}
for _, d := range diffs {
data, err := EncodeDifference(d)
if err != nil {
t.Fatalf("EncodeDifference(%+v): %v", d, err)
}
got, err := DecodeDifference(data)
d.V = Version
if err != nil || !SameDifference(got, d) {
t.Errorf("difference %+v came back %+v, %v", d, got, err)
}
}
}
// 1297/B1a: what a want, a hold and a difference may not say is refused on encode and on decode.
func TestAssignerValuesRefused(t *testing.T) {
src1, _ := SourceFingerprint(SourceInputs{Module: "m", Tree: "t"})
for _, w := range []Want{
{},
{FP: src1},
{Removed: true, FP: fp2of('a')},
{FP: fp2of('a'), Newest: fp2of('b')},
{FP: fp2of('a'), Newest: fp2of('a'), Rule: 3},
{FP: fp2of('a'), Composed: hashOf('1'), Newest: fp2of('b'), Rule: 8},
{FP: fp2of('a'), Composed: hashOf('1'), Hold: true},
{FP: fp2of('a'), Composed: hashOf('1'), Put: true, Newest: fp2of('b'), Rule: 3},
{Removed: true, Put: true},
} {
if _, err := EncodeWant(w); err == nil {
t.Errorf("EncodeWant(%+v) accepted", w)
}
}
if _, err := EncodeHeld(Held{Node: "a.b"}); err == nil {
t.Errorf("a held naming a dotted node was accepted")
}
for _, d := range []Difference{
{Want: fp2of('a'), Sent: fp2of('b')},
{Want: fp2of('a'), Sent: fp2of('b'), Rule: 3, Unexplained: true},
{Want: fp2of('a'), Sent: fp2of('a'), Rule: 3},
{},
{Want: fp2of('a'), Sent: fp2of('b'), Rule: 9},
{WantRemoved: true, Want: fp2of('a'), Rule: 1},
} {
if _, err := EncodeDifference(d); err == nil {
t.Errorf("EncodeDifference(%+v) accepted", d)
}
}
for _, p := range []Put{
{FP: fp2of('a'), Why: "w", Step: "walking"},
{FP: fp2of('a'), Step: PutAsked},
{FP: "src2:abc", Why: "w", Step: PutAsked},
{FP: fp2of('a'), Why: "w", Step: PutWanted, Composed: "sha256:x"},
} {
if _, err := EncodePut(p); err == nil {
t.Errorf("EncodePut(%+v) accepted", p)
}
}
if _, _, err := ParsePutKey("ace.node"); err == nil {
t.Errorf("a node's own key was read as a put")
}
if _, err := DecodeWant([]byte(`{"v":2,"fp":"` + fp2of('a') + `"}`)); !errors.Is(err, ErrUnknownVersion) {
t.Errorf("a want of version 2 was read: %v", err)
}
if _, err := DecodeDifference([]byte(`{"v":1,"want":"` + fp2of('a') + `","sent":"` + fp2of('b') + `"}`)); err == nil {
t.Errorf("a difference naming no rule was read")
}
}
// 1297/B1b: a composition key is the checker's verdict key of the same check, and round-trips; a want, held or
// difference key is refused, naming it.
func TestCompositionKey(t *testing.T) {
key, err := CompositionKey("feat/r4g", "0123456789ab")
verdict, _ := CheckerVerdictKey("feat/r4g", "0123456789ab")
if err != nil || key != verdict {
t.Fatalf("CompositionKey = %q, %v; the checker's key is %q", key, err, verdict)
}
group, heads, err := ParseCompositionKey(key)
if err != nil || group != "feat/r4g" || heads != "0123456789ab" {
t.Errorf("ParseCompositionKey = %q, %q, %v", group, heads, err)
}
held, _ := HeldKey("mesh-mail", fp2of('a'))
for _, bad := range []string{"want.ace.mesh-mail", held, "ace.mesh-mail", "g.0123456789AB", ""} {
if _, _, err := ParseCompositionKey(bad); err == nil || !strings.Contains(err.Error(), "\""+bad+"\"") {
t.Errorf("ParseCompositionKey(%q) = %v, want a refusal naming it", bad, err)
}
}
}
// 1297/B1b: a composition in each state round-trips; the checker's composition of a pass or fail carries its state,
// words, mesh-sdk commit and machines, and a wait gives none.
func TestCompositionVerdictRoundTrip(t *testing.T) {
for _, c := range []CompositionVerdict{
{Group: "g", Heads: "0123456789ab", State: CompositionPass, SDK: "7aacbc1", Machines: []string{"ace"}, Took: 120},
{Group: "g", Heads: "0123456789ab", State: CompositionFail, SDK: "7aacbc1", Machines: []string{"ace", "g14"}, Machine: "g14",
Words: "g14 runs mesh-mail, which the change removes from its source", Warnings: []string{"the merge rebuilds 13 modules"}},
{Group: "g", Heads: "0123456789ab", State: CompositionWaits, Words: "the indexer holds no candidates for the group yet"},
} {
data, err := EncodeCompositionVerdict(c)
if err != nil {
t.Fatalf("EncodeCompositionVerdict(%+v): %v", c, err)
}
got, err := DecodeCompositionVerdict(data)
c.V = Version
if err != nil || !SameCompositionVerdict(got, c) {
t.Errorf("composition %+v came back %+v, %v", c, got, err)
}
comp := got.Composition()
if c.State == CompositionWaits {
if comp != nil {
t.Errorf("a wait gave the checker a composition %+v", comp)
}
continue
}
if comp == nil || comp.State != c.State || comp.SDK != c.SDK || comp.Words != c.Words || len(comp.Machines) != len(c.Machines) {
t.Errorf("the checker's composition of %+v is %+v", c, comp)
}
if err := comp.check(); err != nil {
t.Errorf("the checker's composition of %+v is refused by the verdict: %v", c, err)
}
}
}
// 1297/B1b: what a composition may not say is refused.
func TestCompositionVerdictRefused(t *testing.T) {
for _, c := range []CompositionVerdict{
{Heads: "0123456789ab", State: CompositionPass, SDK: "x"},
{Group: "g", Heads: "0123", State: CompositionPass, SDK: "x"},
{Group: "g", Heads: "0123456789ab", State: CompositionPass},
{Group: "g", Heads: "0123456789ab", State: CompositionPass, SDK: "x", Machine: "ace"},
{Group: "g", Heads: "0123456789ab", State: CompositionFail, SDK: "x"},
{Group: "g", Heads: "0123456789ab", State: CompositionFail, Words: "w"},
{Group: "g", Heads: "0123456789ab", State: CompositionWaits},
{Group: "g", Heads: "0123456789ab", State: CompositionWaits, Words: "w", Machines: []string{"ace"}},
{Group: "g", Heads: "0123456789ab", State: CompositionNotRun},
{Group: "g", Heads: "0123456789ab", State: CompositionPass, SDK: "x", Took: -1},
} {
if _, err := EncodeCompositionVerdict(c); err == nil {
t.Errorf("EncodeCompositionVerdict(%+v) accepted", c)
}
}
}
// 1394/B1b: a grant key round-trips; a want or held key is refused, naming it; a grant round-trips and one naming an
// empty account is refused.
func TestControllerGrant(t *testing.T) {
fp := fp2of('c')
key, err := ControllerGrantKey("mesh-delivery-assigner", fp)
if err != nil || key != "mesh-delivery-assigner."+strings.Repeat("c", 64) {
t.Fatalf("ControllerGrantKey = %q, %v", key, err)
}
module, back, err := ParseControllerGrantKey(key)
if err != nil || module != "mesh-delivery-assigner" || back != fp {
t.Errorf("ParseControllerGrantKey = %q, %q, %v", module, back, err)
}
held, _ := HeldKey("mesh-mail", fp)
for _, bad := range []string{"want.ace.mesh-mail", held, "ace.mesh-mail", ""} {
if _, _, err := ParseControllerGrantKey(bad); err == nil || !strings.Contains(err.Error(), "\""+bad+"\"") {
t.Errorf("ParseControllerGrantKey(%q) = %v, want a refusal naming it", bad, err)
}
}
for _, g := range []ControllerGrant{{}, {Accounts: []string{"UABC", "UDEF"}}} {
data, err := EncodeControllerGrant(g)
if err != nil {
t.Fatal(err)
}
got, err := DecodeControllerGrant(data)
g.V = Version
if err != nil || !SameControllerGrant(got, g) {
t.Errorf("grant %+v came back %+v, %v", g, got, err)
}
}
if _, err := EncodeControllerGrant(ControllerGrant{Accounts: []string{""}}); err == nil {
t.Errorf("a grant naming an empty account was accepted")
}
g := ControllerGrant{Accounts: []string{"ace.mesh-build-go", "shanks.mesh-mail.reader"}}
for _, c := range []struct {
g ControllerGrant
node, module string
names bool
}{
{g, "ace", "mesh-build-go", true},
{g, "shanks", "mesh-mail", true},
{g, "g14", "mesh-build-go", false},
{g, "ace", "mesh-build", false},
{ControllerGrant{}, "novox", "nats", true},
} {
if got := c.g.NamesNode(c.node, c.module); got != c.names {
t.Errorf("%v names %s.%s: %v, want %v", c.g.Accounts, c.node, c.module, got, c.names)
}
}
}
+15
View File
@@ -184,6 +184,10 @@ type BuilderBuild struct {
// does not yet hold: the key is not claimed until that package's trunk key is done and published (novox/hq ADR 0500
// clause 6). A reader that does not know the field reads the key as queued.
Waits string `json:"waits,omitempty"`
// Asks are, on a building key, the asks the builder put for it by build seat language (AskGo, AskNode, AskImage), each
// its id (novox/hq ADR 0525 §1; issue 1887 A4b): a builder that takes the key over at its start waits for those asks'
// steps and never puts them again. A reader that does not know the field reads the key as before.
Asks map[string]string `json:"asks,omitempty"`
}
func (b BuilderBuild) check() error {
@@ -216,6 +220,17 @@ func (b BuilderBuild) check() error {
return fmt.Errorf("delivery: the reference %q of the %s %q %s", a.Reference, a.Kind, a.Name, p)
}
}
for language, id := range b.Asks {
if b.State != BuildBuilding {
return fmt.Errorf("delivery: a %s build names the ask %s; only a building key does", b.State, id)
}
if err := checkAskLanguage(language); err != nil {
return err
}
if err := checkAskID(id); err != nil {
return err
}
}
if b.Waits != "" {
if b.State != BuildQueued {
return fmt.Errorf("delivery: a %s build waits for %q; only a queued key waits for a package", b.State, b.Waits)
+265
View File
@@ -0,0 +1,265 @@
package buildrun
import (
"context"
"errors"
"fmt"
"sync"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
// **An asker puts an ask on its build seat's stream and waits for its step** (novox/hq ADR 0525 §1, §6; issue 1887 A4,
// A5). It chooses no node: whichever build node pulls the ask writes `<node>.<ask id>` in its build module's steps state,
// and the asker reads the first such key that is not running. The builder and the checker share it.
// AskPublisher puts an encoded ask on subject, de-duplicated by its id.
type AskPublisher interface {
PublishAsk(ctx context.Context, subject, id string, data []byte) error
}
// AskWaiter watches an ask's steps: every value a key `*.<ask id>` of the build module's steps state takes, running or
// ended, on the channel it answers, until ctx ends. A watch, not a poll.
type AskWaiter interface {
WatchAsk(ctx context.Context, module, askID string) (<-chan delivery.Step, error)
}
// Asker puts asks and waits for their steps.
type Asker struct {
Publisher AskPublisher
Waiter AskWaiter
// Bound is how long a step may run once a build node took the ask (its key written running) before the asker stops
// waiting; zero waits as long as ctx. An ask waiting in its stream is not bounded: the stream's backlog is no fault
// (ADR 0525 §6, §9), and the bound starts again when another node takes the ask.
Bound time.Duration
}
// Put puts a on its stream: the seat's accept subject of its language and kind.
func (k Asker) Put(ctx context.Context, a delivery.StepAsk) error {
raw, err := delivery.EncodeStepAsk(a)
if err != nil {
return err
}
subject, err := delivery.AskSubject(a.Language, a.Kind)
if err != nil {
return err
}
if err := k.Publisher.PublishAsk(ctx, subject, a.ID, raw); err != nil {
return fmt.Errorf("putting the %s ask %s: %w", a.Kind, a.ID, err)
}
return nil
}
// Wait waits for the step of an ask already put (a builder that started again takes over its own asks with it, never
// putting them twice) and answers it when it ended, whatever its state. The bound runs from the step's start on the node
// that took the ask, read from its running key, so an ask that waits in the stream for a free slot waits as long as ctx.
func (k Asker) Wait(ctx context.Context, a delivery.StepAsk) (delivery.Step, error) {
module, err := delivery.BuildModuleOf(a.Language)
if err != nil {
return delivery.Step{}, err
}
watch, cancel := context.WithCancel(ctx)
defer cancel()
steps, err := k.Waiter.WatchAsk(watch, module, a.ID)
if err != nil {
return delivery.Step{}, err
}
var bound <-chan time.Time
var timer *time.Timer
defer func() {
if timer != nil {
timer.Stop()
}
}()
running, since := "", time.Time{}
for {
select {
case <-ctx.Done():
return delivery.Step{}, ctx.Err()
case <-bound:
return delivery.Step{}, fmt.Errorf("the ask %s's step on %s did not end within %s of its start", a.ID, running, k.Bound)
case step, ok := <-steps:
if !ok {
return delivery.Step{}, errors.New("the watch of the ask's step ended")
}
if step.State != delivery.StepRunning {
return step, nil
}
if k.Bound <= 0 || (step.Node == running && step.Started.Equal(since)) {
continue
}
// A node took the ask, or took it again after its run stopped (another node, or the same node's build module
// replaced mid-run: a new start): its step is bounded from its own start.
running, since = step.Node, step.Started
left := k.Bound
if !step.Started.IsZero() {
left = time.Until(step.Started.Add(k.Bound))
}
if timer != nil {
timer.Stop()
}
timer = time.NewTimer(max(left, 0))
bound = timer.C
}
}
}
// Ask puts a and waits for its step; a step that did not pass is an error naming its node, its state and its reason.
func (k Asker) Ask(ctx context.Context, a delivery.StepAsk) (delivery.Step, error) {
if err := k.Put(ctx, a); err != nil {
return delivery.Step{}, err
}
return k.Passed(k.Wait(ctx, a))
}
// Passed is step when it passed, else an error naming where it ended and why.
func (k Asker) Passed(step delivery.Step, err error) (delivery.Step, error) {
if err != nil {
return step, err
}
if step.State != delivery.StepPassed {
return step, fmt.Errorf("the ask %s on %s %s: %s", step.ID, step.Node, step.State, step.Reason)
}
return step, nil
}
// JetStreamAsks is the bus side of an Asker: a JetStream publish de-duplicated by the ask's id (Nats-Msg-Id), and a watch
// of `*.<ask id>` in the build module's steps bucket.
type JetStreamAsks struct {
JS jetstream.JetStream
}
// PublishAsk is AskPublisher's.
func (s JetStreamAsks) PublishAsk(ctx context.Context, subject, id string, data []byte) error {
msg := &nats.Msg{Subject: subject, Data: data, Header: nats.Header{}}
msg.Header.Set(jetstream.MsgIDHeader, id)
_, err := s.JS.PublishMsg(ctx, msg)
return err
}
// WatchAsk is AskWaiter's: the watch hands over what the keys hold now, so a step that ended before it began is read at
// once, and every change after.
func (s JetStreamAsks) WatchAsk(ctx context.Context, module, askID string) (<-chan delivery.Step, error) {
pattern, err := delivery.AskStepWatch(askID)
if err != nil {
return nil, err
}
kv, err := s.JS.KeyValue(ctx, delivery.StepsBucket(module))
if err != nil {
return nil, fmt.Errorf("the bucket %s: %w", delivery.StepsBucket(module), err)
}
w, err := kv.Watch(ctx, pattern)
if err != nil {
return nil, err
}
out := make(chan delivery.Step)
go func() {
defer close(out)
defer w.Stop()
for {
select {
case <-ctx.Done():
return
case e, ok := <-w.Updates():
if !ok {
return
}
if e == nil || e.Operation() != jetstream.KeyValuePut {
continue
}
step, err := delivery.DecodeStep(e.Value())
if err != nil {
continue // not a step (a node's pause key never matches an ask id; anything else is not this ask's)
}
select {
case out <- step:
case <-ctx.Done():
return
}
}
}
}()
return out, nil
}
// FakeAsks is a test double of the bus for an asker's tests: each ask put is kept and, when Answer is set, its step is
// written at once with what Answer says a build node wrote (running or ended). Write writes a step later. Safe for
// concurrent use.
type FakeAsks struct {
Answer func(a delivery.StepAsk) delivery.Step
mu sync.Mutex
asks []delivery.StepAsk
steps map[string][]delivery.Step
watchers map[string][]chan delivery.Step
}
// PublishAsk is AskPublisher's.
func (f *FakeAsks) PublishAsk(_ context.Context, _, id string, data []byte) error {
a, err := delivery.DecodeStepAsk(data)
if err != nil {
return err
}
f.mu.Lock()
f.asks = append(f.asks, a)
f.mu.Unlock()
if f.Answer != nil {
f.Write(f.Answer(a))
}
return nil
}
// Write writes an ask's step, as a build node does.
func (f *FakeAsks) Write(step delivery.Step) {
f.mu.Lock()
if f.steps == nil {
f.steps, f.watchers = map[string][]delivery.Step{}, map[string][]chan delivery.Step{}
}
f.steps[step.ID] = append(f.steps[step.ID], step)
ws := append([]chan delivery.Step(nil), f.watchers[step.ID]...)
f.mu.Unlock()
for _, w := range ws {
w <- step
}
}
// End is Write: kept for the tests that end a step.
func (f *FakeAsks) End(step delivery.Step) { f.Write(step) }
// WatchAsk is AskWaiter's: what was written so far, then every write.
func (f *FakeAsks) WatchAsk(ctx context.Context, _ string, askID string) (<-chan delivery.Step, error) {
out := make(chan delivery.Step, 64)
f.mu.Lock()
if f.steps == nil {
f.steps, f.watchers = map[string][]delivery.Step{}, map[string][]chan delivery.Step{}
}
for _, s := range f.steps[askID] {
out <- s
}
f.watchers[askID] = append(f.watchers[askID], out)
f.mu.Unlock()
go func() {
<-ctx.Done()
f.mu.Lock()
ws := f.watchers[askID]
for i, w := range ws {
if w == out {
f.watchers[askID] = append(ws[:i], ws[i+1:]...)
break
}
}
f.mu.Unlock()
}()
return out, nil
}
// Asks are the asks put so far.
func (f *FakeAsks) Asks() []delivery.StepAsk {
f.mu.Lock()
defer f.mu.Unlock()
return append([]delivery.StepAsk(nil), f.asks...)
}
@@ -0,0 +1,61 @@
//go:build integration
package buildrun
import (
"context"
"encoding/json"
"testing"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/testbus"
)
// 1887/A4a, 1887/AC3 on the real bus: an ask put twice under one id is one message, and the asker reads the step a node
// wrote under `<node>.<ask id>`.
func TestAnAskIsPutOnceAndItsStepIsRead(t *testing.T) {
nc, err := nats.Connect(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
defer nc.Close()
js, _ := jetstream.New(nc)
ctx := context.Background()
if _, err := js.CreateStream(ctx, jetstream.StreamConfig{Name: "SEAT_NODE_BUILD_GO", Subjects: []string{"mesh.seat.node-build-go.accept.>"},
Retention: jetstream.WorkQueuePolicy}); err != nil {
t.Fatal(err)
}
kv, err := js.CreateKeyValue(ctx, jetstream.KeyValueConfig{Bucket: delivery.StepsBucket("mesh-build-go")})
if err != nil {
t.Fatal(err)
}
bus := JetStreamAsks{JS: js}
k := Asker{Publisher: bus, Waiter: bus, Bound: 5 * time.Second}
a := goAsk("mvl0frv18d1960")
for i := 0; i < 2; i++ {
if err := k.Put(ctx, a); err != nil {
t.Fatal(err)
}
}
s, _ := js.Stream(ctx, "SEAT_NODE_BUILD_GO")
if info, _ := s.Info(ctx); info.State.Msgs != 1 {
t.Fatalf("%d messages for one ask put twice", info.State.Msgs)
}
go func() {
time.Sleep(100 * time.Millisecond)
key, _ := delivery.AskStepKey("ace", a.ID)
running, _ := delivery.EncodeStep(delivery.Step{ID: a.ID, Tool: "delivery ask", Node: "ace", State: delivery.StepRunning, Started: time.Now()})
_, _ = kv.Put(ctx, key, running)
passed, _ := delivery.EncodeStep(delivery.Step{ID: a.ID, Tool: "delivery ask", Node: "ace", State: delivery.StepPassed, Started: time.Now(),
Result: json.RawMessage(`{"artifacts":[]}`)})
_, _ = kv.Put(ctx, key, passed)
}()
step, err := k.Passed(k.Wait(ctx, a))
if err != nil || step.Node != "ace" {
t.Fatalf("step %+v, %v", step, err)
}
}
+71
View File
@@ -0,0 +1,71 @@
package buildrun
import (
"context"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
func goAsk(id string) delivery.StepAsk {
return delivery.StepAsk{ID: id, Kind: delivery.AskDelivery, Language: delivery.AskGo, Asker: "mesh-delivery-builder", At: time.Now(),
Build: &delivery.BuildJob{Module: "m", Path: ".", Fetch: map[string]any{"repository": "novox/x", "commit": strings.Repeat("a", 40)},
Artifacts: []delivery.AskArtifact{{Artifact: "program"}}}}
}
// 1887/A4a: an asker puts the ask once and answers the step a build node wrote for it, naming the node; one that did not
// pass is an error naming the node and why; a wait of an ask already put puts nothing.
func TestAnAskerPutsOnceAndReadsTheStep(t *testing.T) {
f := &FakeAsks{Answer: func(a delivery.StepAsk) delivery.Step {
return delivery.Step{ID: a.ID, Node: "ace", State: delivery.StepPassed, Result: []byte(`{"artifacts":[]}`)}
}}
k := Asker{Publisher: f, Waiter: f}
step, err := k.Ask(context.Background(), goAsk("mvl0frv18d1957"))
if err != nil || step.Node != "ace" || len(f.Asks()) != 1 {
t.Fatalf("step %+v, %v, asks %d", step, err, len(f.Asks()))
}
f.Answer = func(a delivery.StepAsk) delivery.Step {
return delivery.Step{ID: a.ID, Node: "shanks", State: delivery.StepRefused, Reason: "go 1.27 installed"}
}
if _, err := k.Ask(context.Background(), goAsk("mvl0frv18d1958")); err == nil || !strings.Contains(err.Error(), "shanks refused: go 1.27 installed") {
t.Fatalf("a refused step: %v", err)
}
// review round 2 item 1: an ask that waits in its stream for a free slot is not bounded; once a node writes its step
// running, the step is bounded from its start, and the error names the node.
g := &FakeAsks{}
k = Asker{Publisher: g, Waiter: g, Bound: 20 * time.Millisecond}
waiting, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer cancel()
if _, err := k.Wait(waiting, goAsk("mvl0frv18d1959")); err != context.DeadlineExceeded {
t.Fatalf("an ask no node took: %v, want it still waiting when ctx ended", err)
}
g.Write(delivery.Step{ID: "mvl0frv18d1961", Node: "ace", State: delivery.StepRunning, Started: time.Now()})
if _, err := k.Wait(context.Background(), goAsk("mvl0frv18d1961")); err == nil || !strings.Contains(err.Error(), "on ace did not end within") {
t.Fatalf("a step that never ends: %v", err)
}
if len(g.Asks()) != 0 {
t.Fatal("a wait put an ask")
}
}
// Review (issue 734 path): an ask taken again by the same node (its build module replaced mid-run) is bounded from the
// new run's start, so its pass is answered after the first run's bound would have ended the wait.
func TestARedeliveryToTheSameNodeRestartsTheBound(t *testing.T) {
f := &FakeAsks{}
k := Asker{Publisher: f, Waiter: f, Bound: 60 * time.Millisecond}
const id = "mvl0frv18d1962"
first := time.Now()
f.Write(delivery.Step{ID: id, Node: "ace", State: delivery.StepRunning, Started: first})
go func() {
time.Sleep(40 * time.Millisecond)
f.Write(delivery.Step{ID: id, Node: "ace", State: delivery.StepRunning, Started: time.Now()})
time.Sleep(50 * time.Millisecond) // past the first run's bound, inside the second's
f.Write(delivery.Step{ID: id, Node: "ace", State: delivery.StepPassed, Started: time.Now()})
}()
step, err := k.Wait(context.Background(), goAsk(id))
if err != nil || step.State != delivery.StepPassed {
t.Fatalf("step %+v, %v, want the redelivered run's pass", step, err)
}
}
+35 -11
View File
@@ -96,10 +96,16 @@ type LevelResult struct {
// The states of a Composition (ADR 0484 clause 6): the composition check is the assigner's, so until it delivers a
// verdict says CompositionNotRun, and a line never says the composition passed on that.
//
// A composition not run may say why in its Words: AssignerNotRunning when no instance of the assigner is online, so a
// check never waits for a composition nobody is making (issue 1297).
const (
CompositionNotRun = "not-run"
CompositionPass = "pass"
CompositionFail = "fail"
// AssignerNotRunning is the Words of a composition not run because no instance of the assigner is online.
AssignerNotRunning = "the assigner is not running"
)
// Composition is what the check of the machines' composition made of a verdict: its State (not-run, pass or fail), the
@@ -114,9 +120,12 @@ type Composition struct {
// Took is the time of each part of a check in integer milliseconds (ADR 0447 clause 3): waiting in the queue, cloning,
// the unit level, the integration level's setup and its running, and the composition.
type Took struct {
Queue int64 `json:"queue"`
Clone int64 `json:"clone"`
Unit int64 `json:"unit"`
Queue int64 `json:"queue"`
Clone int64 `json:"clone"`
Unit int64 `json:"unit"`
// Lint is the unit level's gofmt and go vet before its tests (ADR 0529 §1; issue 1887 D7-go); absent from a verdict
// of a build module that runs none.
Lint int64 `json:"lint,omitempty"`
IntegrationSetup int64 `json:"integration-setup"`
IntegrationRun int64 `json:"integration-run"`
Composition int64 `json:"composition"`
@@ -130,16 +139,25 @@ type FailedFirst struct {
Words string `json:"words,omitempty"`
}
// MaxFailures is the most entries a verdict's Failures holds (issue 1921): fifty tests named one by one, and an entry
// after them saying how many more failed.
const MaxFailures = 50
// Verdict is the value of a verdict key: the group and its heads hash, the trunk head of each repository it was judged
// against (repository to full sha), the State (running, pass or fail), what failed first, the tests run and kept per
// level, each part's time, the machines composed and the mesh-sdk version the composition ran with (ADR 0446 clause 5).
type Verdict struct {
V int `json:"v"`
Group string `json:"group"`
Heads string `json:"heads"`
Trunk map[string]string `json:"trunk,omitempty"`
State string `json:"state"`
Failed FailedFirst `json:"failed,omitzero"`
V int `json:"v"`
Group string `json:"group"`
Heads string `json:"heads"`
Trunk map[string]string `json:"trunk,omitempty"`
State string `json:"state"`
Failed FailedFirst `json:"failed,omitzero"`
// Failures names every test that failed, in the order judged, each with the first line of its words (issue 1921):
// Failed keeps the first one's words whole, and a check that ran every test says all it found here. At most
// MaxFailures entries, the last of a longer list saying how many more failed. Absent in a value written before it,
// and from a verdict whose failure is no test.
Failures []FailedFirst `json:"failures,omitempty"`
Levels map[string]LevelResult `json:"levels,omitempty"`
Took Took `json:"took,omitzero"`
Machines []string `json:"machines,omitempty"`
@@ -156,8 +174,8 @@ func (c *Composition) check() error {
}
switch c.State {
case CompositionNotRun:
if c.Words != "" || c.SDK != "" || len(c.Machines) > 0 {
return fmt.Errorf("delivery: the verdict's composition is not run and says nothing of words, the mesh-sdk commit or machines")
if c.SDK != "" || len(c.Machines) > 0 {
return fmt.Errorf("delivery: the verdict's composition is not run and names no mesh-sdk commit or machines")
}
case CompositionPass:
if c.SDK == "" {
@@ -213,6 +231,12 @@ func (v Verdict) check() error {
}
}
named := v.Failed.Test != "" || v.Failed.Machine != ""
if len(v.Failures) > 0 && v.State != VerdictFail {
return fmt.Errorf("delivery: a %s verdict names no failures", v.State)
}
if len(v.Failures) > MaxFailures+1 {
return fmt.Errorf("delivery: the verdict names %d failures, more than %d and the count of the rest", len(v.Failures), MaxFailures)
}
switch v.State {
case VerdictRunning:
if v.Failed != (FailedFirst{}) {
+46
View File
@@ -3,6 +3,7 @@ package delivery
import (
"encoding/json"
"errors"
"strconv"
"strings"
"testing"
"time"
@@ -262,6 +263,16 @@ func TestVerdictCarriesSkippedCompositionAndOverBound(t *testing.T) {
if _, err := EncodeVerdict(notRun); err != nil {
t.Errorf("a composition not yet run refused: %v", err)
}
// 1297: a composition not run says why when the assigner is not running, and round-trips with its words.
notRunning := passVerdict()
notRunning.Composition = &Composition{State: CompositionNotRun, Words: AssignerNotRunning}
data, err = EncodeVerdict(notRunning)
if err != nil {
t.Fatalf("a composition not run because the assigner is not running refused: %v", err)
}
if back, err := DecodeVerdict(data); err != nil || back.Composition.Words != AssignerNotRunning {
t.Errorf("its words came back as %+v, %v", back.Composition, err)
}
failed := passVerdict()
failed.State, failed.Failed = VerdictFail, FailedFirst{Machine: "ace", Words: "x"}
failed.Composition = &Composition{State: CompositionFail, Words: "path /etc/x is owned by two modules", SDK: failed.SDK, Machines: []string{"ace"}}
@@ -590,3 +601,38 @@ func TestPreviewOfAHeadInConflict(t *testing.T) {
t.Errorf("the old reader: %+v %v", old, err)
}
}
// Issue 1921: a failed verdict names every failed test in Failures and reads back the same; an older reader's value
// without it reads too; Failures on a verdict that is no fail, and more than MaxFailures and the count of the rest, are
// refused.
func TestVerdictNamesEveryFailure(t *testing.T) {
v := passVerdict()
v.State = VerdictFail
v.Failed = FailedFirst{Test: "integration: x/a.TestA", Words: "a_test.go:9: want 1, got 2\nmore"}
v.Failures = []FailedFirst{{Test: "integration: x/a.TestA", Words: "a_test.go:9: want 1, got 2"}, {Test: "integration: x/b.TestB", Words: "b_test.go:3: boom"}}
data, err := EncodeVerdict(v)
if err != nil {
t.Fatal(err)
}
got, err := DecodeVerdict(data)
v.V = Version
if err != nil || !SameVerdict(got, v) || !strings.Contains(string(data), `"failures":[{"test":"integration: x/a.TestA"`) {
t.Fatalf("%s: %+v %v", data, got, err)
}
old := strings.Replace(string(data), string(data[strings.Index(string(data), `,"failures":`):strings.Index(string(data), `,"levels":`)]), "", 1)
if got, err := DecodeVerdict([]byte(old)); err != nil || len(got.Failures) != 0 || got.Failed != v.Failed {
t.Fatalf("a value without failures: %+v %v (%s)", got, err, old)
}
p := passVerdict()
p.Failures = v.Failures
if _, err := EncodeVerdict(p); err == nil {
t.Error("a pass with failures was written")
}
v.Failures = make([]FailedFirst, MaxFailures+2)
for i := range v.Failures {
v.Failures[i] = FailedFirst{Test: "unit: x.T" + strconv.Itoa(i)}
}
if _, err := EncodeVerdict(v); err == nil {
t.Error("a verdict naming more than MaxFailures and the rest was written")
}
}
+241
View File
@@ -0,0 +1,241 @@
package delivery
import (
"encoding/json"
"fmt"
"regexp"
"strings"
)
// The controller's compositions (novox/hq ADR 0539 clauses 1 to 4). A **composition** is one module's resources
// composed for one node at one fingerprint, written by the controller, which keeps the inputs. It lives in a bucket of
// each node, `composed-<node>`, which only that node's node-engine reads (a composition can carry a sealed value), under
// `<node>.<module>.<fingerprint hex>`, and the node's own resources (adoption, firewall openings, accounts, the
// resolver, identities and groups) under `<node>.node`. Beside it, under the same key in `mesh-controller_composed`,
// which the assigner reads, the controller writes the composition's digest and its judgement of the move from the
// composition the node runs now (ComposedDigest). The controller writes both by compare-and-set while it holds the
// lease; the operator's credential is a second writer on the hand path alone (clause 10, mesh-hand-compose).
//
// A composition is derived state (clause 4): its key and its digest are deleted by the controller when its fingerprint
// is no longer among the module's five newest and no want names it, and each bucket keeps five revisions of a key.
const (
// ComposedBucketPrefix starts the name of a node's bucket of compositions, ComposedBucketFor(node).
ComposedBucketPrefix = "composed-"
// ComposedHistory is the revisions a node's composition bucket keeps per key (ADR 0539 clause 4).
ComposedHistory = 5
// ControllerModule is the module of the controller, the writer of the compositions.
ControllerModule = "mesh-controller"
// ControllerComposedBucket is the controller's bucket of each composition's digest and judgement, `<owner
// module>_<name>`, read by the assigner (ADR 0539 clauses 2 and 6).
ControllerComposedBucket = ControllerModule + "_composed"
// ControllerComposedHistory is the revisions that bucket keeps per key.
ControllerComposedHistory = 5
// ComposedKeep is how many of a module's newest fingerprints keep their compositions though no want names them
// (ADR 0539 clause 4).
ComposedKeep = 5
)
// nodeWord is the last part of a node's own key, in the have bucket and in a node's composition bucket alike.
const nodeWord = "node"
var bucketNode = regexp.MustCompile(`^[A-Za-z0-9_-]+$`)
// ComposedBucketFor is the name of a node's bucket of compositions, "composed-<node>", as declaration.FileBucketFor
// names a node's object bucket (ADR 0488 §4). The node name must be one a bucket and a subject token accept: letters,
// digits, '_' and '-'.
func ComposedBucketFor(node string) (string, error) {
if node == "" {
return "", fmt.Errorf("delivery: the node name is empty; a node's composition bucket is named for its node")
}
if !bucketNode.MatchString(node) {
return "", fmt.Errorf("delivery: the node name %q is not one a bucket accepts: letters, digits, '_' and '-' only", node)
}
return ComposedBucketPrefix + node, nil
}
// ComposedKey is the key, in ComposedBucketFor(node) and in ControllerComposedBucket, of a module's composition for a
// node at a fingerprint: `<node>.<module>.<hex>`, the hex being the fingerprint's key part (FingerprintKeyPart). A
// module named `node` is refused, since `<node>.node` is the node's own key.
func ComposedKey(node, module, fingerprint string) (string, error) {
have, err := HaveKey(node, module)
if err != nil {
return "", err
}
part, err := FingerprintKeyPart(fingerprint)
if err != nil {
return "", err
}
return have + "." + part, nil
}
// ComposedNodeKey is the key of a node's own composition: `<node>.node`.
func ComposedNodeKey(node string) (string, error) {
return HaveNodeKey(node)
}
// ParseComposedKey is the inverse of ComposedKey and ComposedNodeKey: isNode is true for `<node>.node`, with module
// and fingerprint empty. It refuses any other shape, naming the key.
func ParseComposedKey(key string) (node, module, fingerprint string, isNode bool, err error) {
refused := func(why error) (string, string, string, bool, error) {
return "", "", "", false, fmt.Errorf("delivery: %q is not a composition key, <node>.<module>.<fingerprint hex> or <node>.node: %w", key, why)
}
parts := strings.Split(key, ".")
switch len(parts) {
case 2:
if parts[1] != nodeWord {
return refused(fmt.Errorf("a module's composition names its fingerprint"))
}
if err := checkName("node", parts[0]); err != nil {
return refused(err)
}
return parts[0], "", "", true, nil
case 3:
node, module, isNode, err := ParseHaveKey(parts[0] + "." + parts[1])
if err != nil {
return refused(err)
}
if isNode {
return refused(fmt.Errorf("the node's own key names no fingerprint"))
}
fp, err := ParseFingerprintKeyPart(parts[2])
if err != nil {
return refused(err)
}
return node, module, fp, false, nil
}
return refused(fmt.Errorf("it has %d parts", len(parts)))
}
// Judgement is the controller's judgement of a move from the composition a node runs, as its have names it, to a new
// one (ADR 0539 clauses 2 and 14, by ADR 0377 §3's tests).
type Judgement string
// The judgements of a move.
const (
// JudgedRecreates is a move that recreates a container.
JudgedRecreates Judgement = "recreates"
// JudgedMovesData is a move that gives the module's data another place.
JudgedMovesData Judgement = "moves-data"
// JudgedNeither is a move that does neither, and the judgement of a composition the node never ran.
JudgedNeither Judgement = "neither"
// JudgedCannot is a move that cannot be judged, because the composition the node runs is gone or cannot be read.
JudgedCannot Judgement = "cannot-judge"
// JudgedUnchanged is a composition equal to the module's slice of the declaration the node last applied, for a node
// whose have names no composition yet (the first want at the switch, novox/hq ADR 0540 clause 9): applying it
// changes nothing on the node, so it needs no person's yes.
JudgedUnchanged Judgement = "unchanged"
)
var digestForm = regexp.MustCompile(`^sha256:[0-9a-f]{64}$`)
func checkDigest(digest, of string) error {
if !digestForm.MatchString(digest) {
return fmt.Errorf("delivery: %s %q is not sha256:<64 lowercase hex digits>", of, digest)
}
return nil
}
// ComposedDigest is the value, in ControllerComposedBucket, beside a composition: its Digest (ComposedDigestOf), and
// the Judgement of the move to it from the composition the node runs now, whose digest is Against (empty when the node
// runs no composition of the module). Words say why, and are required for a move that cannot be judged. Rewritten when
// the node's have changes.
//
// Policy is the module's upgrade policy as the controller holds it (ADR 0236, ADR 0539 clause 14), with PolicyFrom
// where it comes from and PolicyWhy why: the assigner holds a move of a module of policy record from it, the operator's
// own choice (`upgrade` with policy) included. Rewritten when the policy changes. Empty on a composition made by hand
// (mesh-hand-compose), whose want the operator writes.
type ComposedDigest struct {
V int `json:"v"`
Digest string `json:"digest"`
Judgement Judgement `json:"judgement"`
Against string `json:"against,omitempty"`
Words string `json:"words,omitempty"`
// UserListOnly says the composition differs from the one the node's have names only in the bus's user list (the
// bus module's file `<module>.bus-users`), whatever the judgement says of its mount: the assigner writes such a
// revision into the bus node's want at once, outside the record hold, at the fingerprint the node runs (ADR 0539
// clause 7).
UserListOnly bool `json:"user-list-only,omitempty"`
Policy string `json:"policy,omitempty"`
PolicyFrom string `json:"policy-from,omitempty"`
PolicyWhy string `json:"policy-why,omitempty"`
}
// The upgrade policies and their sources a composition's value names: catalogue.PolicyRoll, PolicyTogether and
// PolicyRecord, and catalogue.FromPerson, FromModule, FromBus, FromData and FromDefault, the same words (the compose
// package's test holds them equal; this package does not import the catalogue).
var (
composedPolicies = []string{"roll", "together", "record"}
composedPolicyFroms = []string{"person", "module", "the bus", "irreplaceable data", "default"}
)
func oneOf(v string, of []string) bool {
for _, o := range of {
if v == o {
return true
}
}
return false
}
func (c ComposedDigest) check() error {
if err := checkDigest(c.Digest, "the composition's digest"); err != nil {
return err
}
if c.Against != "" {
if err := checkDigest(c.Against, "the digest a move is judged against"); err != nil {
return err
}
}
switch c.Judgement {
case JudgedRecreates, JudgedMovesData:
if c.Against == "" {
return fmt.Errorf("delivery: a move judged %s names the composition it is judged against", c.Judgement)
}
case JudgedNeither, JudgedUnchanged:
case JudgedCannot:
if c.Words == "" {
return fmt.Errorf("delivery: a move that cannot be judged says why")
}
default:
return fmt.Errorf("delivery: the judgement %q is not %s, %s, %s, %s or %s", c.Judgement, JudgedRecreates,
JudgedMovesData, JudgedNeither, JudgedCannot, JudgedUnchanged)
}
if c.Policy != "" || c.PolicyFrom != "" {
if !oneOf(c.Policy, composedPolicies) {
return fmt.Errorf("delivery: the upgrade policy %q is not one of %v", c.Policy, composedPolicies)
}
if !oneOf(c.PolicyFrom, composedPolicyFroms) {
return fmt.Errorf("delivery: the upgrade policy's source %q is not one of %v", c.PolicyFrom, composedPolicyFroms)
}
}
return nil
}
// PolicyRecord is the policy whose moves the assigner holds for a person's yes (ADR 0539 clause 14).
const PolicyRecord = "record"
// EncodeComposedDigest is c as JSON at Version; it refuses an unknown judgement, a digest not of the sha256 form, a
// move judged to recreate or move data against nothing, and a move that cannot be judged without its reason.
func EncodeComposedDigest(c ComposedDigest) ([]byte, error) {
if err := c.check(); err != nil {
return nil, err
}
c.V = Version
return json.Marshal(c)
}
// DecodeComposedDigest reads the value beside a composition; it refuses an unknown version (ErrUnknownVersion) and
// what EncodeComposedDigest refuses.
func DecodeComposedDigest(data []byte) (ComposedDigest, error) {
var c ComposedDigest
if err := decode(data, &c); err != nil {
return ComposedDigest{}, err
}
if err := c.check(); err != nil {
return ComposedDigest{}, err
}
return c, nil
}
// SameComposedDigest says whether two values are the same, so the controller writes only when they differ.
func SameComposedDigest(a, b ComposedDigest) bool { return a == b }
+125
View File
@@ -0,0 +1,125 @@
package delivery
import (
"errors"
"strings"
"testing"
)
// 1943/AC1: a node's composition bucket is named for the node, and a node with no name, or one a bucket refuses,
// is refused, naming why.
func TestComposedBucketFor(t *testing.T) {
b, err := ComposedBucketFor("ace")
if err != nil || b != "composed-ace" {
t.Fatalf("ComposedBucketFor(ace) = %q, %v", b, err)
}
for _, bad := range []string{"", "a.b", "a*", "a b", "a>"} {
if _, err := ComposedBucketFor(bad); err == nil || !strings.Contains(err.Error(), "node") {
t.Errorf("ComposedBucketFor(%q) = %v, want a refusal naming the node", bad, err)
}
}
}
// 1943/AC1: a module's composition key and the node's own key round-trip through one parser; an empty node, a
// module named node, a bad fingerprint and a key of another shape are refused, naming why.
func TestComposedKeys(t *testing.T) {
fp := fp2of('c')
key, err := ComposedKey("ace", "mesh-mail", fp)
if err != nil || key != "ace.mesh-mail."+strings.Repeat("c", 64) {
t.Fatalf("ComposedKey = %q, %v", key, err)
}
node, module, back, isNode, err := ParseComposedKey(key)
if err != nil || node != "ace" || module != "mesh-mail" || back != fp || isNode {
t.Errorf("ParseComposedKey(%q) = %q, %q, %q, %v, %v", key, node, module, back, isNode, err)
}
own, err := ComposedNodeKey("ace")
if err != nil || own != "ace.node" {
t.Fatalf("ComposedNodeKey = %q, %v", own, err)
}
node, module, back, isNode, err = ParseComposedKey(own)
if err != nil || node != "ace" || module != "" || back != "" || !isNode {
t.Errorf("ParseComposedKey(%q) = %q, %q, %q, %v, %v", own, node, module, back, isNode, err)
}
refusals := []struct {
node, module, fp, why string
}{
{"", "mesh-mail", fp, "node name is empty"},
{"ace", "node", fp, `"node" is refused`},
{"ace", "", fp, "module name is empty"},
{"ace", "mesh-mail", "src2:abc", "not a fingerprint of the new form"},
{"ace", "mesh-mail", "src1:" + strings.Repeat("c", 64), "not a fingerprint of the new form"},
{"a.b", "mesh-mail", fp, "node name"},
}
for _, r := range refusals {
if _, err := ComposedKey(r.node, r.module, r.fp); err == nil || !strings.Contains(err.Error(), r.why) {
t.Errorf("ComposedKey(%q, %q, %q) = %v, want a refusal saying %q", r.node, r.module, r.fp, err, r.why)
}
}
if _, err := ComposedNodeKey(""); err == nil {
t.Errorf("ComposedNodeKey(\"\") accepted")
}
for _, bad := range []string{"", "ace", "ace.mesh-mail", "ace.node.x", "ace.mesh-mail.abc", ".mesh-mail." + strings.Repeat("c", 64),
"ace.node." + strings.Repeat("c", 64), "ace.mesh-mail." + strings.Repeat("c", 64) + ".x", "want.ace.mesh-mail"} {
if _, _, _, _, err := ParseComposedKey(bad); err == nil || !strings.Contains(err.Error(), "\""+bad+"\"") {
t.Errorf("ParseComposedKey(%q) = %v, want a refusal naming it", bad, err)
}
}
}
// 1943/AC1: the controller's value beside a composition, its digest and the judgement of the move, round-trips; an
// unknown judgement, a digest not of the sha256 form, a move judged against nothing, a cannot-judge without its
// reason, and an upgrade policy or source outside the catalogue's words are refused, naming why; an unknown version is
// refused as one. The module's upgrade policy and its source ride beside the judgement, for the assigner's hold.
func TestComposedDigestValue(t *testing.T) {
digest := "sha256:" + strings.Repeat("d", 64)
against := "sha256:" + strings.Repeat("e", 64)
for _, v := range []ComposedDigest{
{Digest: digest, Judgement: JudgedNeither},
{Digest: digest, Judgement: JudgedNeither, Against: against},
{Digest: digest, Judgement: JudgedRecreates, Against: against, Words: "the container's image changes"},
{Digest: digest, Judgement: JudgedMovesData, Against: against},
{Digest: digest, Judgement: JudgedCannot, Words: "the composition the node runs is gone"},
{Digest: digest, Judgement: JudgedUnchanged, Words: "equal to the declaration the node last applied"},
{Digest: digest, Judgement: JudgedNeither, Policy: "record", PolicyFrom: "person", PolicyWhy: "the operator: its data"},
{Digest: digest, Judgement: JudgedNeither, Policy: "roll", PolicyFrom: "default"},
{Digest: digest, Judgement: JudgedNeither, Against: against, UserListOnly: true, Words: "only the bus's user list changes"},
} {
raw, err := EncodeComposedDigest(v)
if err != nil {
t.Fatalf("EncodeComposedDigest(%+v): %v", v, err)
}
back, err := DecodeComposedDigest(raw)
v.V = Version
if err != nil || !SameComposedDigest(back, v) {
t.Errorf("round trip of %+v = %+v, %v", v, back, err)
}
}
refusals := []struct {
v ComposedDigest
why string
}{
{ComposedDigest{Digest: digest, Judgement: "maybe"}, `judgement "maybe"`},
{ComposedDigest{Digest: digest}, `judgement ""`},
{ComposedDigest{Digest: "sha256:abc", Judgement: JudgedNeither}, "not sha256"},
{ComposedDigest{Judgement: JudgedNeither}, "not sha256"},
{ComposedDigest{Digest: digest, Judgement: JudgedRecreates}, "against"},
{ComposedDigest{Digest: digest, Judgement: JudgedMovesData}, "against"},
{ComposedDigest{Digest: digest, Judgement: JudgedCannot}, "why"},
{ComposedDigest{Digest: digest, Judgement: JudgedNeither, Against: "sha256:zz"}, "not sha256"},
{ComposedDigest{Digest: digest, Judgement: JudgedNeither, Policy: "sometimes", PolicyFrom: "person"}, `policy "sometimes"`},
{ComposedDigest{Digest: digest, Judgement: JudgedNeither, Policy: "record"}, `source ""`},
{ComposedDigest{Digest: digest, Judgement: JudgedNeither, PolicyFrom: "person"}, `policy ""`},
}
for _, r := range refusals {
if _, err := EncodeComposedDigest(r.v); err == nil || !strings.Contains(err.Error(), r.why) {
t.Errorf("EncodeComposedDigest(%+v) = %v, want a refusal saying %q", r.v, err, r.why)
}
}
if _, err := DecodeComposedDigest([]byte(`{"v":2,"digest":"` + digest + `","judgement":"neither"}`)); !errors.Is(err, ErrUnknownVersion) {
t.Errorf("version 2 = %v, want ErrUnknownVersion", err)
}
if _, err := DecodeComposedDigest([]byte(`{"v":1,"digest":"` + digest + `","judgement":"perhaps"}`)); err == nil {
t.Errorf("an unknown judgement was read")
}
}
+100
View File
@@ -0,0 +1,100 @@
package delivery
import (
"crypto/ed25519"
"encoding/base64"
"fmt"
)
// A composition's signature (novox/hq ADR 0539 clause 3): the controller signs each composition it writes with its
// key, and the operator signs one made by hand with mesh-hand-compose with theirs. The node-engine trusts exactly these two
// signers and applies nothing of a composition whose signature fails or whose signer is neither. The signature is an
// ed25519 signature, as a declaration's is, of the composition's digest (ComposedDigestOf) after a context line, so a
// signature made for anything else never verifies here.
const composedSigningContext = "novox composition v1\n"
// Signature is who signed a composition and the signature: Signer the signer's ed25519 public key and Sig the
// signature, both in standard base64.
type Signature struct {
Signer string `json:"signer"`
Sig string `json:"sig"`
}
func (s Signature) check() error {
if key, err := base64.StdEncoding.DecodeString(s.Signer); err != nil || len(key) != ed25519.PublicKeySize {
return fmt.Errorf("delivery: a composition's signer %q is not an ed25519 public key in base64", s.Signer)
}
if sig, err := base64.StdEncoding.DecodeString(s.Sig); err != nil || len(sig) != ed25519.SignatureSize {
return fmt.Errorf("delivery: a composition's signature is not an ed25519 signature in base64")
}
return nil
}
// SignComposed is c signed with key: its digest signed, and the key's public half named as the signer. A signature c
// already carries is replaced.
func SignComposed(c Composed, key ed25519.PrivateKey) (Composed, error) {
if len(key) != ed25519.PrivateKeySize {
return Composed{}, fmt.Errorf("delivery: the key to sign a composition with is not an ed25519 private key")
}
return SignComposedBy(c, key.Public().(ed25519.PublicKey), func(message []byte) ([]byte, error) {
return ed25519.Sign(key, message), nil
})
}
// SignComposedBy is SignComposed for a signer that keeps its private half to itself (the controller's identity store
// signs a message and hands back the signature): sign signs the message it is given with the key whose public half is
// signer. A signature that does not verify against signer is refused, so a composition is never written signed by a
// key other than the one it names.
func SignComposedBy(c Composed, signer ed25519.PublicKey, sign func(message []byte) ([]byte, error)) (Composed, error) {
if len(signer) != ed25519.PublicKeySize {
return Composed{}, fmt.Errorf("delivery: the signer of a composition is not an ed25519 public key")
}
c.Signature = nil
digest, err := ComposedDigestOf(c)
if err != nil {
return Composed{}, err
}
message := []byte(composedSigningContext + digest)
sig, err := sign(message)
if err != nil {
return Composed{}, err
}
if !ed25519.Verify(signer, message, sig) {
return Composed{}, fmt.Errorf("delivery: the signature made for the composition of %q on %s does not verify against its signer", c.Module, c.Node)
}
c.Signature = &Signature{Signer: base64.StdEncoding.EncodeToString(signer), Sig: base64.StdEncoding.EncodeToString(sig)}
return c, nil
}
// VerifyComposed is c's digest when c is signed by one of the trusted keys and its signature verifies against its
// digest; otherwise it refuses, saying why, and nothing of c is to be applied: an unsigned composition, one whose
// signer is not a trusted key, and one changed after it was signed.
func VerifyComposed(c Composed, trusted []ed25519.PublicKey) (string, error) {
if c.Signature == nil {
return "", fmt.Errorf("delivery: the composition of %q on %s is unsigned, and only a signed one is applied", c.Module, c.Node)
}
if err := c.Signature.check(); err != nil {
return "", err
}
signer, _ := base64.StdEncoding.DecodeString(c.Signature.Signer)
var key ed25519.PublicKey
for _, t := range trusted {
if t.Equal(ed25519.PublicKey(signer)) {
key = t
}
}
if key == nil {
return "", fmt.Errorf("delivery: the composition of %q on %s is signed by %s, which is not a trusted signer (the controller's key or the operator's)",
c.Module, c.Node, c.Signature.Signer)
}
digest, err := ComposedDigestOf(c)
if err != nil {
return "", err
}
sig, _ := base64.StdEncoding.DecodeString(c.Signature.Sig)
if !ed25519.Verify(key, []byte(composedSigningContext+digest), sig) {
return "", fmt.Errorf("delivery: the composition of %q on %s does not match its signature: it was changed after it was signed, or signed for another",
c.Module, c.Node)
}
return digest, nil
}
+103
View File
@@ -0,0 +1,103 @@
package delivery
import (
"crypto/ed25519"
"crypto/rand"
"encoding/json"
"strings"
"testing"
)
func aSigningKey(t *testing.T) (ed25519.PublicKey, ed25519.PrivateKey) {
t.Helper()
pub, priv, err := ed25519.GenerateKey(rand.Reader)
if err != nil {
t.Fatal(err)
}
return pub, priv
}
func aComposition() Composed {
return Composed{Node: "ace", Module: "m", FP: fp2of('a'), Resources: []json.RawMessage{
json.RawMessage(`{"id":"m.f","type":"file","module":"m","path":"/etc/m","content":"x"}`)}}
}
// 1943/AC5: a composition signed by the controller's key or by the operator's verifies against the two trusted keys,
// keeps its digest, and survives its encoding; one signed by another key, unsigned, or changed by one byte after it was
// signed does not verify, and nothing of it is to be applied.
func TestACompositionIsSignedByATrustedKey(t *testing.T) {
controllerPub, controller := aSigningKey(t)
operatorPub, operator := aSigningKey(t)
_, stranger := aSigningKey(t)
trusted := []ed25519.PublicKey{controllerPub, operatorPub}
c := aComposition()
want, _ := ComposedDigestOf(c)
for name, key := range map[string]ed25519.PrivateKey{"controller": controller, "operator": operator} {
signed, err := SignComposed(c, key)
if err != nil {
t.Fatalf("%s: %v", name, err)
}
raw, err := EncodeComposed(signed)
if err != nil {
t.Fatal(err)
}
back, err := DecodeComposed(raw)
if err != nil {
t.Fatal(err)
}
digest, err := VerifyComposed(back, trusted)
if err != nil || digest != want {
t.Errorf("%s's signature: %q, %v; want the digest %s", name, digest, err, want)
}
if d, _ := ComposedDigestOf(back); d != want {
t.Errorf("%s's signature changed the digest: %s", name, d)
}
changed := strings.Replace(string(raw), `"content":"x"`, `"content":"y"`, 1)
if changed == string(raw) {
t.Fatal("the test changed nothing")
}
if tampered, err := DecodeComposed([]byte(changed)); err != nil {
t.Fatal(err)
} else if _, err := VerifyComposed(tampered, trusted); err == nil || !strings.Contains(err.Error(), "signature") {
t.Errorf("%s: a composition changed by one byte verified: %v", name, err)
}
}
byStranger, _ := SignComposed(c, stranger)
if _, err := VerifyComposed(byStranger, trusted); err == nil || !strings.Contains(err.Error(), "not a trusted") {
t.Errorf("a stranger's signature = %v, want refused as no trusted signer", err)
}
if _, err := VerifyComposed(c, trusted); err == nil || !strings.Contains(err.Error(), "unsigned") {
t.Errorf("an unsigned composition = %v, want refused", err)
}
signed, _ := SignComposed(c, controller)
forged := signed
forged.Signature = &Signature{Signer: signed.Signature.Signer, Sig: byStranger.Signature.Sig}
if _, err := VerifyComposed(forged, trusted); err == nil {
t.Errorf("a stranger's signature under the controller's name verified")
}
if _, err := EncodeComposed(Composed{Node: "ace", Module: "m", FP: fp2of('a'), Resources: c.Resources,
Signature: &Signature{Signer: "not base64!", Sig: "x"}}); err == nil {
t.Errorf("a signature naming no key was encoded")
}
}
// 1943/AC5: a signer that keeps its private half signs through a function, and the composition verifies as one signed
// with the key; a function that signs with another key than the one named is refused.
func TestACompositionIsSignedByAKeptKey(t *testing.T) {
pub, priv := aSigningKey(t)
_, other := aSigningKey(t)
c := aComposition()
signed, err := SignComposedBy(c, pub, func(m []byte) ([]byte, error) { return ed25519.Sign(priv, m), nil })
if err != nil {
t.Fatal(err)
}
if _, err := VerifyComposed(signed, []ed25519.PublicKey{pub}); err != nil {
t.Errorf("a composition signed through the function does not verify: %v", err)
}
if _, err := SignComposedBy(c, pub, func(m []byte) ([]byte, error) { return ed25519.Sign(other, m), nil }); err == nil {
t.Errorf("a signature by another key than the one named was accepted")
}
}
+330
View File
@@ -0,0 +1,330 @@
package delivery
import (
"bytes"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"io"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
// NodeOwn is what a node's own composition (`<node>.node`) carries beside its resources: what the declaration says of
// the node as a whole (novox/hq ADR 0539 clause 2: adoption, accounts, identities and groups). A module's composition
// carries none of it.
type NodeOwn struct {
// Adoption says the node is adopted and which of its modules are taken (ADR 0100, ADR 0103); nil for a converged node.
Adoption *declaration.Adoption `json:"adoption,omitempty"`
// Identities is the identity of every module assigned to the node (ADR 0328 §4), Groups and Libraries every library
// it is reached by (ADR 0328 §3).
Identities []declaration.Identity `json:"identities,omitempty"`
Groups []declaration.Group `json:"groups,omitempty"`
Libraries []declaration.Library `json:"libraries,omitempty"`
// Provisioner is the account the provisioner's client runs as on the node (ADR 0349 §4).
Provisioner string `json:"provisioner_account,omitempty"`
}
func (o NodeOwn) empty() bool {
return o.Adoption == nil && len(o.Identities) == 0 && len(o.Groups) == 0 && len(o.Libraries) == 0 && o.Provisioner == ""
}
// Composed is a composition (ADR 0539 clause 1): one module's resources composed for one node at one fingerprint, or,
// with Module and FP empty, the node's own resources and parts (Own). Each resource is a declaration resource as the
// node-engine reads it, saying its module; it is kept as JSON and written canonically (its keys sorted, its spacing
// dropped), so its digest (ComposedDigestOf) is a function of its content alone. Data says the module declares data,
// so the node agent backs it up before it moves (ADR 0429 §14). Filled names the files of the module the provisioner
// fills (ADR 0349 §5). A file over 64 KiB travels by digest and size (ByDigest), its content in the node's object
// bucket (ADR 0539 clause 4).
type Composed struct {
V int
Node string
Module string
FP string
Data bool
Resources []json.RawMessage
Filled []string
Own NodeOwn
// Signature is the controller's or, made by hand, the operator's (ADR 0539 clause 3, SignComposed); it signs the
// digest and is never part of it.
Signature *Signature
}
// composedWire is a composition as it is written.
type composedWire struct {
V int `json:"v"`
Node string `json:"node"`
Module string `json:"module,omitempty"`
FP string `json:"fp,omitempty"`
Data bool `json:"data,omitempty"`
Resources []json.RawMessage `json:"resources"`
Filled []string `json:"filled_files,omitempty"`
Own *NodeOwn `json:"own,omitempty"`
Signature *Signature `json:"signature,omitempty"`
}
// canonicalJSON is raw with its object keys sorted and its spacing dropped, its numbers kept as written.
func canonicalJSON(raw json.RawMessage) (json.RawMessage, error) {
var v any
d := json.NewDecoder(bytes.NewReader(raw))
d.UseNumber()
if err := d.Decode(&v); err != nil {
return nil, err
}
if _, err := d.Token(); err != io.EOF {
return nil, fmt.Errorf("more after the value")
}
return json.Marshal(v)
}
func (c Composed) check() error {
if _, err := HaveNodeKey(c.Node); err != nil {
return err
}
if c.Module == "" {
if c.FP != "" {
return fmt.Errorf("delivery: the node's own composition names no fingerprint, and this one names %s", c.FP)
}
// It carries the filled paths of its own files, those of a module with no fingerprint of the new form moved
// into it (issue 1945), and never a module's data.
if c.Data {
return fmt.Errorf("delivery: the node's own composition holds no module's data")
}
} else {
if _, err := ComposedKey(c.Node, c.Module, c.FP); err != nil {
return err
}
if !c.Own.empty() {
return fmt.Errorf("delivery: the composition of %s carries the node's own parts, which only <node>.node carries", c.Module)
}
}
seen := map[string]bool{}
files := map[string][]*declaration.File{}
for _, raw := range c.Resources {
r, err := declaration.ParseResource(raw)
if err != nil {
return fmt.Errorf("delivery: the composition of %q on %s: %w", c.Module, c.Node, err)
}
if seen[r.Identity()] {
return fmt.Errorf("delivery: the composition of %q on %s holds the resource %q twice", c.Module, c.Node, r.Identity())
}
seen[r.Identity()] = true
if r.DeclaredModule() != c.Module {
return fmt.Errorf("delivery: the resource %q is of %q, and this is the composition of %q on %s",
r.Identity(), r.DeclaredModule(), c.Module, c.Node)
}
if f, ok := r.(*declaration.File); ok {
if f.TravelsByDigest() {
return fmt.Errorf("delivery: the file %q is over %d bytes and travels by digest (ByDigest), not whole",
f.ID, declaration.FileReferenceThreshold)
}
files[f.Path] = append(files[f.Path], f)
}
}
// A file the provisioner fills says no content, so a composition that does not say it is filled would have the
// node-engine write it empty (issue 1945, live on 2026-10-25): refused, naming it.
filledHere := make(map[string]bool, len(c.Filled))
for _, path := range c.Filled {
filledHere[path] = true
}
for path, fs := range files {
for _, f := range fs {
if f.OwnerAtRuntime == declaration.RuntimeOwned && !filledHere[path] && !saysContent(f) {
return fmt.Errorf("delivery: the file %q says no content and the composition of %q on %s does not say "+
"the provisioner fills %s: the node-engine would write it empty", f.ID, c.Module, c.Node, path)
}
}
}
for _, path := range c.Filled {
if len(files[path]) == 0 {
return fmt.Errorf("delivery: the composition of %q fills %q, and declares no file there", c.Module, path)
}
for _, f := range files[path] {
if f.OwnerAtRuntime == "" {
return fmt.Errorf("delivery: the file %q is filled by the provisioner and says no owner-at-runtime", f.ID)
}
}
}
if c.Signature != nil {
if err := c.Signature.check(); err != nil {
return err
}
}
return nil
}
// saysContent says a file names what to write: content, bytes, a digest or a sealed value. A file the provisioner fills
// names none of them.
func saysContent(f *declaration.File) bool {
return f.Content != "" || f.Bytes != "" || f.Digest != "" || f.Sealed != "" || f.VaultSealed != nil
}
// wire is c checked and made canonical, ready to marshal.
func (c Composed) wire() (composedWire, error) {
if err := c.check(); err != nil {
return composedWire{}, err
}
w := composedWire{V: Version, Node: c.Node, Module: c.Module, FP: c.FP, Data: c.Data, Filled: c.Filled,
Signature: c.Signature, Resources: make([]json.RawMessage, len(c.Resources))}
for i, r := range c.Resources {
canon, err := canonicalJSON(r)
if err != nil {
return composedWire{}, fmt.Errorf("delivery: a resource of %q is not JSON: %w", c.Module, err)
}
w.Resources[i] = canon
}
if !c.Own.empty() {
own := c.Own
w.Own = &own
}
return w, nil
}
// EncodeComposed is c as JSON at Version, each resource canonical. It refuses a composition whose node or fingerprint
// a key refuses, a resource its kind refuses (an action among them), a resource of another module, a module's
// composition carrying the node's own parts, a file over 64 KiB carried whole, and a filled path no file declares.
func EncodeComposed(c Composed) ([]byte, error) {
w, err := c.wire()
if err != nil {
return nil, err
}
return json.Marshal(w)
}
// DecodeComposed reads a composition; it refuses an unknown version (ErrUnknownVersion), a field it does not know, and
// what EncodeComposed refuses.
func DecodeComposed(data []byte) (Composed, error) {
if err := checkVersion(data); err != nil {
return Composed{}, err
}
var w composedWire
d := json.NewDecoder(bytes.NewReader(data))
d.DisallowUnknownFields()
if err := d.Decode(&w); err != nil {
return Composed{}, fmt.Errorf("delivery: a composition does not fit its shape: %w", err)
}
c := Composed{V: w.V, Node: w.Node, Module: w.Module, FP: w.FP, Data: w.Data, Resources: w.Resources,
Filled: w.Filled, Signature: w.Signature}
if w.Own != nil {
c.Own = *w.Own
}
if err := c.check(); err != nil {
return Composed{}, err
}
return c, nil
}
// ComposedDigestOf is a composition's digest, "sha256:<hex>", of its encoding without its signature. A want names it
// (ADR 0539 clause 1), the controller writes it beside the composition, and the signature signs it.
func ComposedDigestOf(c Composed) (string, error) {
c.Signature = nil
raw, err := EncodeComposed(c)
if err != nil {
return "", err
}
sum := sha256.Sum256(raw)
return "sha256:" + hex.EncodeToString(sum[:]), nil
}
// SameComposed says whether two compositions are the same value, their signatures included, so the controller writes
// only when they differ.
func SameComposed(a, b Composed) bool {
ra, errA := EncodeComposed(a)
rb, errB := EncodeComposed(b)
return errA == nil && errB == nil && bytes.Equal(ra, rb)
}
// ByDigest is c with every file over 64 KiB given by its digest and size, and the content of each by digest, for the
// writer to put in the node's object bucket (declaration.FileBucketFor) under declaration.ObjectName before it writes
// the composition (ADR 0539 clause 4, ADR 0488 §3). A file that is sealed, filled with secrets or written into another
// stays as it is (declaration.File.TravelsByDigest).
func (c Composed) ByDigest() (Composed, map[string][]byte, error) {
out := c
out.Resources = make([]json.RawMessage, len(c.Resources))
objects := map[string][]byte{}
for i, raw := range c.Resources {
out.Resources[i] = raw
r, err := declaration.ParseResource(raw)
if err != nil {
return Composed{}, nil, err
}
f, ok := r.(*declaration.File)
if !ok || !f.TravelsByDigest() {
continue
}
ref, body, err := f.AsReference()
if err != nil {
return Composed{}, nil, err
}
var fields map[string]json.RawMessage
if err := json.Unmarshal(raw, &fields); err != nil {
return Composed{}, nil, err
}
delete(fields, "content")
delete(fields, "bytes")
fields["digest"], _ = json.Marshal(ref.Digest)
fields["size"], _ = json.Marshal(ref.Size)
rewritten, err := json.Marshal(fields)
if err != nil {
return Composed{}, nil, err
}
out.Resources[i] = rewritten
objects[ref.Digest] = body
}
return out, objects, nil
}
// SplitComposed splits a node's composed resources into its compositions, by key (ComposedKey, ComposedNodeKey): each
// resource into the composition of the module it says, under that module's fingerprint in fps, and a resource that
// says no module into the node's own, with own. Each filled path goes with the module whose file declares it. The order
// of the resources is kept within each composition. A resource of a module fps names no fingerprint for is refused,
// naming it. Data is the caller's to set, from the module's manifest.
func SplitComposed(node string, resources []json.RawMessage, fps map[string]string, filled []string, own NodeOwn) (map[string]Composed, error) {
nodeKey, err := ComposedNodeKey(node)
if err != nil {
return nil, err
}
out := map[string]Composed{nodeKey: {Node: node, Own: own, Resources: []json.RawMessage{}}}
pathOf := map[string]string{}
for _, raw := range resources {
var head struct {
ID string `json:"id"`
Type string `json:"type"`
Module string `json:"module"`
Path string `json:"path"`
}
if err := json.Unmarshal(raw, &head); err != nil {
return nil, fmt.Errorf("delivery: a resource of %s is not JSON: %w", node, err)
}
key := nodeKey
if head.Module != "" {
fp, ok := fps[head.Module]
if !ok {
return nil, fmt.Errorf("delivery: the resource %q is of %q, whose fingerprint on %s is not given", head.ID, head.Module, node)
}
if key, err = ComposedKey(node, head.Module, fp); err != nil {
return nil, err
}
if _, ok := out[key]; !ok {
out[key] = Composed{Node: node, Module: head.Module, FP: fp}
}
}
if head.Type == string(declaration.TypeFile) && head.Module != "" {
pathOf[head.Path] = key
}
c := out[key]
c.Resources = append(c.Resources, raw)
out[key] = c
}
for _, path := range filled {
key, ok := pathOf[path]
if !ok {
return nil, fmt.Errorf("delivery: the provisioner fills %q on %s, and no module's file declares it", path, node)
}
c := out[key]
c.Filled = append(c.Filled, path)
out[key] = c
}
return out, nil
}
+364
View File
@@ -0,0 +1,364 @@
package delivery
import (
"bytes"
"compress/gzip"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"sort"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
// capturedDeclaration is a node's declaration as the controller's `plan` verb answered it on 2026-10-24 at 16:19
// (testdata/declaration-<node>.json.gz; the resources carrying a sealed value or a secret taken out, and the filled
// files that named them), never typed by hand.
type capturedDeclaration struct {
Builds map[string]string `json:"builds"`
Resources []json.RawMessage `json:"resources"`
Filled []string `json:"filled_files"`
Provisioner string `json:"provisioner_account"`
}
func readCaptured(t *testing.T, node string) capturedDeclaration {
t.Helper()
f, err := os.Open("testdata/declaration-" + node + ".json.gz")
if err != nil {
t.Fatal(err)
}
defer f.Close()
z, err := gzip.NewReader(f)
if err != nil {
t.Fatal(err)
}
raw, err := io.ReadAll(z)
if err != nil {
t.Fatal(err)
}
var c capturedDeclaration
if err := json.Unmarshal(raw, &c); err != nil {
t.Fatal(err)
}
return c
}
// fpOfBuild stands for each module's fingerprint in a captured declaration, which names builds, not fingerprints.
func fpOfBuild(build string) string {
var b strings.Builder
for b.Len() < 64 {
b.WriteString(fmt.Sprintf("%x", []byte(build)))
}
return SourcePrefix2 + b.String()[:64]
}
func splitCaptured(t *testing.T, node string, c capturedDeclaration) map[string]Composed {
t.Helper()
fps := map[string]string{}
for m, b := range c.Builds {
fps[m] = fpOfBuild(b)
}
// mesh-wireguard's resources are composed with no build of it named; its fingerprint is the indexer's, as every
// module's is, and stood in for here by its name.
for _, r := range c.Resources {
var head struct {
Module string `json:"module"`
}
_ = json.Unmarshal(r, &head)
if _, ok := fps[head.Module]; head.Module != "" && !ok {
fps[head.Module] = fpOfBuild(head.Module)
}
}
split, err := SplitComposed(node, c.Resources, fps, c.Filled, NodeOwn{Provisioner: c.Provisioner})
if err != nil {
t.Fatalf("SplitComposed(%s): %v", node, err)
}
return split
}
func canonical(t *testing.T, raw json.RawMessage) string {
t.Helper()
var v any
d := json.NewDecoder(bytes.NewReader(raw))
d.UseNumber()
if err := d.Decode(&v); err != nil {
t.Fatal(err)
}
out, _ := json.Marshal(v)
return string(out)
}
// 1943/AC2: each module's slice of a captured declaration of each node, and the node's own resources, round-trip
// through the composition's encoding with the same digest; together they are the declaration's resources, each once,
// each in the composition of the module it says.
func TestCompositionsOfACapturedDeclaration(t *testing.T) {
t.Parallel()
for _, node := range []string{"ace", "g14"} {
t.Run(node, func(t *testing.T) {
t.Parallel()
oneCapturedDeclaration(t, node)
})
}
}
func oneCapturedDeclaration(t *testing.T, node string) {
{
c := readCaptured(t, node)
split := splitCaptured(t, node, c)
modules := map[string]bool{}
for m := range c.Builds {
modules[m] = true
}
var union []string
for key, comp := range split {
n, module, fp, isNode, err := ParseComposedKey(key)
if err != nil || n != node || comp.Node != node || comp.Module != module || comp.FP != fp {
t.Fatalf("%s: key %q holds %s/%s/%s (%v)", node, key, comp.Node, comp.Module, comp.FP, err)
}
if !isNode && !modules[module] && module != "mesh-wireguard" {
t.Errorf("%s: a composition of %s, which the declaration names no build of", node, module)
}
raw, err := EncodeComposed(comp)
if err != nil {
t.Fatalf("%s: EncodeComposed(%s): %v", node, key, err)
}
back, err := DecodeComposed(raw)
if err != nil {
t.Fatalf("%s: DecodeComposed(%s): %v", node, key, err)
}
if !SameComposed(back, comp) {
t.Errorf("%s: %s did not round-trip", node, key)
}
// SameComposed encodes both, so the digest of what was read back, a hash of its encoding, is the one written.
if d, err := ComposedDigestOf(back); err != nil || !digestForm.MatchString(d) {
t.Errorf("%s: %s digests %q (%v)", node, key, d, err)
}
for _, r := range back.Resources {
var head struct {
Module string `json:"module"`
}
_ = json.Unmarshal(r, &head)
if head.Module != module {
t.Errorf("%s: %s holds a resource of %q", node, key, head.Module)
}
union = append(union, canonical(t, r))
}
}
var want []string
for _, r := range c.Resources {
want = append(want, canonical(t, r))
}
sort.Strings(union)
sort.Strings(want)
if strings.Join(union, "\n") != strings.Join(want, "\n") {
t.Errorf("%s: the compositions hold %d resources, the declaration %d, or they differ", node, len(union), len(want))
}
}
}
// 1943/AC2: one changed byte of a resource is another digest; the same composition written with its keys in another
// order is the same digest.
func TestComposedDigestFollowsTheContent(t *testing.T) {
c := readCaptured(t, "g14")
split := splitCaptured(t, "g14", c)
var comp Composed
for _, v := range split {
if v.Module != "" && len(v.Resources) > 1 {
comp = v
break
}
}
d1, _ := ComposedDigestOf(comp)
var m map[string]json.RawMessage
_ = json.Unmarshal(comp.Resources[0], &m)
var keys []string
for k := range m {
keys = append(keys, k)
}
sort.Sort(sort.Reverse(sort.StringSlice(keys)))
var parts []string
for _, k := range keys {
parts = append(parts, fmt.Sprintf("%q : %s", k, m[k]))
}
reordered := json.RawMessage("{ " + strings.Join(parts, ",\n ") + " }")
shuffled := comp
shuffled.Resources = append([]json.RawMessage{reordered}, comp.Resources[1:]...)
if d2, _ := ComposedDigestOf(shuffled); d2 != d1 {
t.Errorf("the same resources in another order and spacing digest %s and %s", d1, d2)
}
changed := comp
changed.Resources = append([]json.RawMessage{json.RawMessage(strings.Replace(canonical(t, comp.Resources[0]), `"id":"`, `"id":"x`, 1))}, comp.Resources[1:]...)
if d3, _ := ComposedDigestOf(changed); d3 == d1 {
t.Errorf("a changed resource kept its digest")
}
revised := comp
revised.Data = !comp.Data
if d4, _ := ComposedDigestOf(revised); d4 == d1 {
t.Errorf("a changed data flag kept its digest")
}
}
// 1943/AC2: a file over 64 KiB is carried by digest and size, its content handed back to be put in the node's object
// bucket; a composition that carries such a file whole is refused.
func TestAComposedFileOver64KiBIsCarriedByDigest(t *testing.T) {
c := readCaptured(t, "g14")
split := splitCaptured(t, "g14", c)
var comp Composed
var at int
for _, v := range split {
for i, r := range v.Resources {
var f map[string]any
_ = json.Unmarshal(r, &f)
if f["type"] == "file" && v.Module != "" && f["bytes"] == nil && f["into"] == nil {
comp, at = v, i
break
}
}
if comp.Module != "" {
break
}
}
var f map[string]any
_ = json.Unmarshal(comp.Resources[at], &f)
// Grown past the threshold from a captured file: no captured file of this node is over 64 KiB.
body := strings.Repeat("x", declaration.FileReferenceThreshold+1)
f["content"] = body
big, _ := json.Marshal(f)
comp.Resources = append([]json.RawMessage{}, comp.Resources...)
comp.Resources[at] = big
if _, err := EncodeComposed(comp); err == nil || !strings.Contains(err.Error(), "by digest") {
t.Fatalf("a file over 64 KiB carried whole = %v, want refused", err)
}
byDigest, objects, err := comp.ByDigest()
if err != nil {
t.Fatal(err)
}
var ref map[string]any
_ = json.Unmarshal(byDigest.Resources[at], &ref)
digest := declaration.DigestOf([]byte(body))
if ref["digest"] != digest || ref["size"] != float64(len(body)) || ref["content"] != nil {
t.Errorf("the file became %v, want digest %s and size %d", ref, digest, len(body))
}
if string(objects[digest]) != body || len(objects) != 1 {
t.Errorf("the objects handed back are %d, want the file's content under its digest", len(objects))
}
if _, err := EncodeComposed(byDigest); err != nil {
t.Errorf("the composition by digest is refused: %v", err)
}
}
// 1943/AC2: an unknown version is refused as one; a module's composition carrying the node's own parts, a resource of
// another module, a resource its kind refuses, or no fingerprint, is refused, naming why.
func TestComposedRefusals(t *testing.T) {
fp := fp2of('a')
file := json.RawMessage(`{"id":"m.f","type":"file","module":"m","path":"/etc/m","content":"x"}`)
ok := Composed{Node: "ace", Module: "m", FP: fp, Resources: []json.RawMessage{file}}
raw, err := EncodeComposed(ok)
if err != nil {
t.Fatal(err)
}
if _, err := DecodeComposed(bytes.Replace(raw, []byte(`"v":1`), []byte(`"v":2`), 1)); !errors.Is(err, ErrUnknownVersion) {
t.Errorf("version 2 = %v, want ErrUnknownVersion", err)
}
refusals := []struct {
c Composed
why string
}{
{Composed{Node: "ace", Module: "m", Resources: []json.RawMessage{file}}, "fingerprint"},
{Composed{Node: "", Module: "m", FP: fp, Resources: []json.RawMessage{file}}, "node name is empty"},
{Composed{Node: "ace", Module: "other", FP: fp, Resources: []json.RawMessage{file}}, `of "m"`},
{Composed{Node: "ace", Module: "m", FP: fp, Resources: []json.RawMessage{file}, Own: NodeOwn{Provisioner: "p"}}, "node's own"},
{Composed{Node: "ace", FP: fp, Resources: []json.RawMessage{}}, "names no fingerprint"},
{Composed{Node: "ace", Resources: []json.RawMessage{file}}, `of "m"`},
{Composed{Node: "ace", Module: "m", FP: fp, Resources: []json.RawMessage{json.RawMessage(`{"id":"m.f","type":"file","module":"m","path":"/etc/m","content":"x","image":"y"}`)}}, `"image"`},
{Composed{Node: "ace", Module: "m", FP: fp, Resources: []json.RawMessage{json.RawMessage(`{"id":"m.a","type":"action","module":"m"}`)}}, "action"},
}
for _, r := range refusals {
if _, err := EncodeComposed(r.c); err == nil || !strings.Contains(err.Error(), r.why) {
t.Errorf("EncodeComposed(%s/%s) = %v, want a refusal saying %q", r.c.Node, r.c.Module, err, r.why)
}
}
}
// 1943/AC2 (issue 1150): a module whose name holds a dot keys its have, its want and its composition escaped, and the
// parsers give its name back; a name with a wildcard, whitespace or an escape of its own is still refused.
func TestADottedModuleKeysEscaped(t *testing.T) {
fp := fp2of('f')
have, err := HaveKey("ace", "novox.be")
if err != nil || have != "ace.novox=2ebe" {
t.Fatalf("HaveKey = %q, %v", have, err)
}
if n, m, isNode, err := ParseHaveKey(have); err != nil || n != "ace" || m != "novox.be" || isNode {
t.Errorf("ParseHaveKey(%q) = %q, %q, %v, %v", have, n, m, isNode, err)
}
want, err := WantKey("ace", "novox.be")
if err != nil || want != "want.ace.novox=2ebe" {
t.Fatalf("WantKey = %q, %v", want, err)
}
if n, m, err := ParseWantKey(want); err != nil || n != "ace" || m != "novox.be" {
t.Errorf("ParseWantKey(%q) = %q, %q, %v", want, n, m, err)
}
comp, err := ComposedKey("ace", "novox.be", fp)
if err != nil || comp != "ace.novox=2ebe."+strings.Repeat("f", 64) {
t.Fatalf("ComposedKey = %q, %v", comp, err)
}
if n, m, back, _, err := ParseComposedKey(comp); err != nil || n != "ace" || m != "novox.be" || back != fp {
t.Errorf("ParseComposedKey(%q) = %q, %q, %q, %v", comp, n, m, back, err)
}
for _, bad := range []string{"m*", "m x", "m=2e", "m>", ".m", "m.", "a..b"} {
if _, err := HaveKey("ace", bad); err == nil {
t.Errorf("HaveKey(ace, %q) accepted", bad)
}
}
for _, bad := range []string{"ace.m=2", "ace.m=2E", "ace.m=41", "ace.=6eode"} {
if _, _, _, err := ParseHaveKey(bad); err == nil {
t.Errorf("ParseHaveKey(%q) accepted", bad)
}
}
}
// The node's own composition carries the filled paths of the files moved into it, the files of a module with no
// fingerprint of the new form: the provisioner fills them all the same (issue 1945, live on 2026-10-25: the paths were
// dropped and the node-engine wrote those files empty). It still holds no module's data.
func TestTheNodesOwnCompositionCarriesTheFilledPathsOfItsFiles(t *testing.T) {
filled := json.RawMessage(`{"id":"power.unit","type":"file","path":"/etc/systemd/system/p.service","mode":"0644","owner-at-runtime":"owned"}`)
own := Composed{Node: "ace", Resources: []json.RawMessage{filled}, Filled: []string{"/etc/systemd/system/p.service"},
Own: NodeOwn{Provisioner: "mesh-provisioner-client"}}
raw, err := EncodeComposed(own)
if err != nil {
t.Fatalf("the node's own composition with a filled file: %v", err)
}
back, err := DecodeComposed(raw)
if err != nil || len(back.Filled) != 1 {
t.Fatalf("read back: %v %v", back.Filled, err)
}
if _, err := EncodeComposed(Composed{Node: "ace", Data: true, Resources: []json.RawMessage{filled},
Filled: own.Filled}); err == nil {
t.Error("the node's own composition was let hold a module's data")
}
}
// A file the provisioner fills says no content: in a composition that does not say it is filled, the node-engine would
// write it empty. Such a composition is refused, in the node's own as in a module's (issue 1945).
func TestAFileWithNoContentThatIsNotFilledIsRefused(t *testing.T) {
fp := fp2of('a')
for _, c := range []Composed{
{Node: "ace", Resources: []json.RawMessage{json.RawMessage(
`{"id":"power.unit","type":"file","path":"/etc/p","mode":"0644","owner-at-runtime":"owned"}`)}},
{Node: "ace", Module: "m", FP: fp, Resources: []json.RawMessage{json.RawMessage(
`{"id":"m.f","type":"file","module":"m","path":"/etc/m","mode":"0644","owner-at-runtime":"owned"}`)}},
} {
if _, err := EncodeComposed(c); err == nil || !strings.Contains(err.Error(), "empty") {
t.Errorf("%s/%s: %v, want a refusal saying it would be written empty", c.Node, c.Module, err)
}
}
// A file that says its content, even an empty one, is written as it says.
if _, err := EncodeComposed(Composed{Node: "ace", Resources: []json.RawMessage{json.RawMessage(
`{"id":"x.f","type":"file","path":"/etc/x","mode":"0644","content":""}`)}}); err != nil {
t.Errorf("a file that says it is empty: %v", err)
}
}
+114 -24
View File
@@ -11,7 +11,8 @@
// `node`. A presence key is `<node>` for a node and `part.<part>.<instance>` for a part's instance, so a node may
// not be named `part`. A pulse key is `<bucket>.<key>`: the full name of the bucket watched, then that bucket's key,
// which may hold dots, so it splits at the first dot. A name is refused when it is empty or holds a dot, a wildcard,
// whitespace or a character a bus key refuses, and the error names it.
// whitespace or a character a bus key refuses, and the error names it; but a module's name may hold dots between its
// parts (`novox.be`), and keys with each dot escaped as EscapeName writes it (`novox=2ebe`, issue 1150).
//
// **Values** are JSON of under about 500 bytes, each carrying its version in `v`. A reader refuses a value whose
// version is missing or one it does not know, with an error that is ErrUnknownVersion (research 063 section 05
@@ -33,9 +34,12 @@
// **The release 4 layouts** (issues 1356 to 1359; ADR 0453 clauses 4, 6 and 16): the new fingerprint form (fingerprint2.go),
// the indexer's manifests by digest and the new fields of its modules and candidates (manifests.go, source.go), the
// builder's build keys (builder.go) and the checker's verdicts and kept verdicts (checker.go). A manifests value is the
// one exception to the size above: a manifest is up to about 21 KB, whole, byte for byte.
// one exception to the size above: a manifest is up to about 21 KB, whole, byte for byte. The bus module's have on the
// node holding mesh-broker is the other: its user list's handing carries a sealed box of every loaded user's hash, some
// kilobytes (handed.go).
//
// Standard library only, and this module's own asks package.
// Standard library only, and this module's own asks, declaration and seal packages (a composition holds declaration
// resources; the adoption handing is a sealed box, handed.go).
package delivery
import (
@@ -89,6 +93,10 @@ const (
// for which it had written a have key; the key is never deleted (issue 1369, ADR 0408 §4), so a reader can tell
// that the module stopped.
PhaseRemoved Phase = "removed"
// PhaseHeld is a module whose backup before an apply failed: it keeps running what it has, nothing of the new
// composition is applied, and the node waits for the answer in its want (novox/hq ADR 0539 clause 16). Detail
// carries the backup's own words.
PhaseHeld Phase = "held"
)
// Health is what the module's own check says.
@@ -164,13 +172,45 @@ func HaveKey(node, module string) (string, error) {
if err := checkName("node", node); err != nil {
return "", err
}
if err := checkName("module", module); err != nil {
part, err := moduleKeyPart(module)
if err != nil {
return "", err
}
if module == "node" {
return "", fmt.Errorf(`delivery: the module name "node" is refused: %s.node is the node's own key`, node)
}
return node + "." + module, nil
return node + "." + part, nil
}
var moduleNameOK = regexp.MustCompile(`^[-/_a-zA-Z0-9]+(\.[-/_a-zA-Z0-9]+)*$`)
// moduleKeyPart is a module's name as a key part (issue 1150): a name may hold a dot (`novox.be`), which a key part
// may not, so a dot is escaped as EscapeName writes it (`novox=2ebe`); a name of letters, digits, `-`, `/` and `_` keys
// as it is. A name that is empty, or holds a wildcard, whitespace, `=`, or an empty part between dots, is refused.
func moduleKeyPart(module string) (string, error) {
if module == "" {
return "", fmt.Errorf("delivery: the module name is empty")
}
if !moduleNameOK.MatchString(module) {
return "", fmt.Errorf("delivery: the module name %q holds a wildcard, whitespace, '=', an empty part between dots or a character a key refuses", module)
}
return EscapeName(module), nil
}
// parseModuleKeyPart is the inverse of moduleKeyPart; it refuses a part that is not the one moduleKeyPart writes for
// its name (an escape of a character that keys as it is, an upper-case escape, an escape of a refused character).
func parseModuleKeyPart(part string) (string, error) {
if err := checkName("module", part); err != nil {
return "", err
}
module, err := UnescapeName(part)
if err != nil {
return "", err
}
if back, err := moduleKeyPart(module); err != nil || back != part {
return "", fmt.Errorf("delivery: the module part %q is not a module name as a key writes it", part)
}
return module, nil
}
// HaveNodeKey is the key, in HaveBucket, of a node's own facts: `<node>.node`.
@@ -190,12 +230,13 @@ func ParseHaveKey(key string) (node, module string, isNode bool, err error) {
if err := checkName("node", node); err != nil {
return "", "", false, fmt.Errorf("delivery: %q is not a have key: %w", key, err)
}
if err := checkName("module", module); err != nil {
return "", "", false, fmt.Errorf("delivery: %q is not a have key: %w", key, err)
}
if module == "node" {
return node, "", true, nil
}
module, err = parseModuleKeyPart(module)
if err != nil {
return "", "", false, fmt.Errorf("delivery: %q is not a have key: %w", key, err)
}
return node, module, false, nil
}
@@ -282,24 +323,55 @@ func PulseTTL(moveAge time.Duration) (ttl time.Duration, stalled bool) {
// Have is a module's state on a node, written by that node's node-engine under HaveKey. FP is the module's
// fingerprint once the indexer exists (design 47 release 3); until then Build is the build the node was sent, and
// the view compares by build.
//
// Composed is the digest of the composition the node runs for the module (novox/hq ADR 0539 clause 9): the
// controller judges a move against it (clause 14), and the assigner reads it to know a want was applied.
type Have struct {
V int `json:"v"`
FP string `json:"fp,omitempty"`
Build string `json:"build,omitempty"`
Phase Phase `json:"phase"`
Health Health `json:"health"`
Detail string `json:"detail,omitempty"`
V int `json:"v"`
FP string `json:"fp,omitempty"`
Composed string `json:"composed,omitempty"`
Build string `json:"build,omitempty"`
Phase Phase `json:"phase"`
Health Health `json:"health"`
Detail string `json:"detail,omitempty"`
// UserList is what the node-engine of the node holding mesh-broker wrote of the bus's user list, in the have of the
// bus module (novox/hq ADR 0547 §7, §8). Nil on every other have.
UserList *UserListWritten `json:"user-list,omitempty"`
}
// UserListWritten is the bus's user list as the node-engine of the bus node wrote it to the bus module's file, applied
// alone and with no want (novox/hq ADR 0547 §8): where it was taken from, what that source holds, and what the file
// holds now. The bus reloads the file itself when its digest changes (the module's entrypoint), so Digest is the list
// the bus loads. Wanted differs from Digest only while a write failed, and Error says why.
type UserListWritten struct {
// Source is the key the list was taken from: the controller's composition of the bus module for this node
// (`<node>.<module>.<fp>`), or the grants part's `user-list` once it took over.
Source string `json:"source"`
Revision uint64 `json:"revision"`
// Wanted is the digest of the list the source holds, `sha256:<hex>`.
Wanted string `json:"wanted"`
// Digest is the digest of the file on the machine after the write, `sha256:<hex>`; empty when there is none.
Digest string `json:"digest,omitempty"`
Users int `json:"users"`
At time.Time `json:"at"`
Error string `json:"error,omitempty"`
// Handed is the hashes of the users of the list on the machine, sealed to the vault, for its adoption of the
// accounts it holds no value for (ADR 0547 clause 9, handed.go); nil while the vault's key is not known.
Handed *AccountsHanded `json:"handed,omitempty"`
}
// HaveNode is a node's own facts, written under HaveNodeKey. Each field is the node-engine's own JSON, carried whole:
// this package names the facts and does not type them.
//
// Composed is the digest of the node's own composition it runs (ADR 0539 clause 9).
type HaveNode struct {
V int `json:"v"`
Windows json.RawMessage `json:"windows,omitempty"`
Busy json.RawMessage `json:"busy,omitempty"`
Strays json.RawMessage `json:"strays,omitempty"`
Filters json.RawMessage `json:"filters,omitempty"`
Profile json.RawMessage `json:"profile,omitempty"`
V int `json:"v"`
Composed string `json:"composed,omitempty"`
Windows json.RawMessage `json:"windows,omitempty"`
Busy json.RawMessage `json:"busy,omitempty"`
Strays json.RawMessage `json:"strays,omitempty"`
Filters json.RawMessage `json:"filters,omitempty"`
Profile json.RawMessage `json:"profile,omitempty"`
}
// Presence is whether a node or a part's instance is online; the entry's own time says since when. The watchman
@@ -347,9 +419,14 @@ func decode(data []byte, into any) error {
func (h Have) check() error {
switch h.Phase {
case PhaseBackingUp, PhaseApplying, PhaseChecking, PhaseRunning, PhaseFailed, PhaseRemoved:
case PhaseBackingUp, PhaseApplying, PhaseChecking, PhaseRunning, PhaseFailed, PhaseRemoved, PhaseHeld:
default:
return fmt.Errorf("delivery: phase %q is not backing-up, applying, checking, running, failed or removed", h.Phase)
return fmt.Errorf("delivery: phase %q is not backing-up, applying, checking, running, failed, removed or held", h.Phase)
}
if h.Composed != "" {
if err := checkDigest(h.Composed, "the have's composition digest"); err != nil {
return err
}
}
switch h.Health {
case HealthOK, HealthFailed, HealthWaits, HealthUnknown:
@@ -420,11 +497,24 @@ func DecodePulse(data []byte) (Pulse, error) {
}
// SameHave says whether two haves are the same value, so a writer writes only when they differ.
func SameHave(a, b Have) bool { return a == b }
func SameHave(a, b Have) bool {
ua, ub := a.UserList, b.UserList
a.UserList, b.UserList = nil, nil
if a != b || (ua == nil) != (ub == nil) {
return false
}
if ua == nil {
return true
}
x, y := *ua, *ub
x.At, y.At = time.Time{}, time.Time{}
x.Handed, y.Handed = nil, nil
return x == y && ua.At.Equal(ub.At) && sameHanded(ua.Handed, ub.Handed)
}
// SameHaveNode says whether two node facts are the same value, comparing each carried JSON without its spacing.
func SameHaveNode(a, b HaveNode) bool {
return a.V == b.V && sameJSON(a.Windows, b.Windows) && sameJSON(a.Busy, b.Busy) &&
return a.V == b.V && a.Composed == b.Composed && sameJSON(a.Windows, b.Windows) && sameJSON(a.Busy, b.Busy) &&
sameJSON(a.Strays, b.Strays) && sameJSON(a.Filters, b.Filters) && sameJSON(a.Profile, b.Profile)
}
+88
View File
@@ -3,9 +3,12 @@ package delivery
import (
"encoding/json"
"errors"
"reflect"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/seal"
)
// 1131/AC1: the have key forms and parses back; bad names are refused, naming them.
@@ -149,6 +152,65 @@ func TestRoundTrip(t *testing.T) {
t.Fatalf("got %+v want %+v", out, in)
}
})
t.Run("have with the user list", func(t *testing.T) {
at := time.Date(2026, 10, 25, 21, 40, 0, 0, time.UTC)
in := Have{FP: "f1", Phase: PhaseRunning, Health: HealthOK, UserList: &UserListWritten{Source: "novox.nats.f1",
Revision: 7, Wanted: "sha256:ab", Digest: "sha256:ab", Users: 93, At: at}}
b, err := EncodeHave(in)
if err != nil {
t.Fatal(err)
}
hasV1(t, b)
out, err := DecodeHave(b)
if err != nil {
t.Fatal(err)
}
in.V = Version
if !reflect.DeepEqual(out, in) || !out.UserList.At.Equal(at) {
t.Fatalf("got %+v want %+v", out, in)
}
// And with the hashes handed to the vault (ADR 0547 clause 9): sealed to its key, opened by it alone, every
// listed user with a bcrypt hash once, and refused to another key.
hash := "$2a$11$" + strings.Repeat("a", 53)
list := []byte("{ user: \"controller\", password: \"" + hash + "\", permissions: {\n" +
"{ user: \"node.ace\", password: \"" + hash + "\", permissions: {\n" +
"{ user: \"controller\", password: \"" + hash + "\" }\n{ user: \"nkey\", password: \"plain\" }\n")
accounts := HandedAccountsOf(list)
if len(accounts) != 2 || accounts[0].User != "controller" || accounts[1].User != "node.ace" || accounts[1].Hash != hash {
t.Fatalf("handed accounts: %+v", accounts)
}
vault, err := seal.GenerateKeyPair()
if err != nil {
t.Fatal(err)
}
other, _ := seal.GenerateKeyPair()
handed, err := HandAccounts(accounts, vault.PublicKey, "sha256:ab", at)
if err != nil {
t.Fatal(err)
}
in.UserList.Handed = &handed
b, err = EncodeHave(in)
if err != nil {
t.Fatal(err)
}
if strings.Contains(string(b), "node.ace") || strings.Contains(string(b), hash) {
t.Fatalf("a user or a hash is in the clear: %s", b)
}
out, err = DecodeHave(b)
if err != nil || !SameHave(out, in) {
t.Fatalf("handed round trip: %v %+v", err, out.UserList)
}
opened, err := OpenHanded(*out.UserList.Handed, vault.PublicKey, vault.PrivateKey)
if err != nil || !reflect.DeepEqual(opened, accounts) {
t.Fatalf("opened %+v, %v", opened, err)
}
if _, err := OpenHanded(*out.UserList.Handed, other.PublicKey, other.PrivateKey); err == nil {
t.Fatal("a handing opened for another key")
}
if _, err := HandAccounts(accounts, "", "sha256:ab", at); err == nil {
t.Fatal("handed with no vault key")
}
})
t.Run("have node", func(t *testing.T) {
in := HaveNode{Windows: json.RawMessage(`{"a":1}`), Busy: json.RawMessage(`true`)}
b, err := EncodeHaveNode(in)
@@ -275,6 +337,32 @@ func TestSame(t *testing.T) {
if SameHave(a, b) {
t.Error("different haves same")
}
// The user list is compared by what it says, never by where it is held (ADR 0547 §8).
at := time.Date(2026, 10, 25, 21, 0, 0, 0, time.UTC)
ua, ub := a, a
ua.UserList = &UserListWritten{Source: "k", Revision: 1, Wanted: "sha256:a", Digest: "sha256:a", Users: 2, At: at}
ub.UserList = &UserListWritten{Source: "k", Revision: 1, Wanted: "sha256:a", Digest: "sha256:a", Users: 2, At: at.In(time.Local)}
if !SameHave(ua, ub) || SameHave(ua, a) {
t.Error("equal user lists differ, or one said and one not are the same")
}
ub.UserList.Revision = 2
if SameHave(ua, ub) {
t.Error("different user lists same")
}
// A handing is compared by what it says: one said and one not differ, as do two to other keys.
ub.UserList.Revision = 1
ua.UserList.Handed = &AccountsHanded{From: "sha256:a", To: "sha256:k", Users: 2, At: at}
if SameHave(ua, ub) {
t.Error("a handing said and one not are the same")
}
ub.UserList.Handed = &AccountsHanded{From: "sha256:a", To: "sha256:k", Users: 2, At: at.In(time.Local)}
if !SameHave(ua, ub) {
t.Error("equal handings differ")
}
ub.UserList.Handed.To = "sha256:j"
if SameHave(ua, ub) {
t.Error("handings to other keys same")
}
n := HaveNode{V: 1, Busy: json.RawMessage(`true`)}
if !SameHaveNode(n, HaveNode{V: 1, Busy: json.RawMessage(`true`)}) || SameHaveNode(n, HaveNode{V: 1}) {
t.Error("have node")
+92
View File
@@ -0,0 +1,92 @@
package delivery
import (
"encoding/json"
"fmt"
"reflect"
"strings"
)
// The controller's grants (novox/hq ADR 0528 clause 2, ADR 0429 §6 rule 6): when the controller has issued a module's
// bus credential for a fingerprint, it writes `<module>.<fingerprint hex>` in its bucket grants. The assigner reads
// it: a node's want moves to a fingerprint only once its grant exists. Only the controller writes the bucket.
//
// The names say Controller, since the catalogue's GrantKey (catalogue/streams.go) is a stream's grant pattern, unrelated.
const (
// ControllerGrantsBucket is the controller's bucket of issued credentials by fingerprint.
ControllerGrantsBucket = "mesh-controller_grants"
// ControllerGrantsHistory is the revisions the bucket keeps per key.
ControllerGrantsHistory = 5
)
// ControllerGrantKey is the key, in ControllerGrantsBucket, of a module's credential issued for a fingerprint:
// `<module>.<hex>`, the hex being the fingerprint's key part (FingerprintKeyPart).
func ControllerGrantKey(module, fingerprint string) (string, error) {
return BuilderBuildKey(module, fingerprint)
}
// ParseControllerGrantKey is the inverse of ControllerGrantKey; it refuses a want or held key, naming it.
func ParseControllerGrantKey(key string) (module, fingerprint string, err error) {
if module, fingerprint, err = ParseBuilderBuildKey(key); err != nil {
return "", "", fmt.Errorf("delivery: %q is not a grant key, <module>.<fingerprint hex>: %w", key, err)
}
return module, fingerprint, nil
}
// ControllerGrant is the value of a grant key: the bus accounts (their public keys) the module's processes were issued
// for that fingerprint, none for a module whose grants the tool runner's account carries, so a reissue for another
// account is a new revision. The entry's own time says when it was issued.
type ControllerGrant struct {
V int `json:"v"`
Accounts []string `json:"accounts,omitempty"`
}
func (g ControllerGrant) check() error {
for _, a := range g.Accounts {
if strings.TrimSpace(a) == "" || strings.ContainsAny(a, " \t\n") {
return fmt.Errorf("delivery: a grant names an empty or spaced account %q", a)
}
}
return nil
}
// EncodeControllerGrant is g as JSON at Version; it refuses an empty or spaced account.
func EncodeControllerGrant(g ControllerGrant) ([]byte, error) {
if err := g.check(); err != nil {
return nil, err
}
g.V = Version
return json.Marshal(g)
}
// DecodeControllerGrant reads a grant; it refuses an unknown version (ErrUnknownVersion) and an empty or spaced account.
func DecodeControllerGrant(data []byte) (ControllerGrant, error) {
var g ControllerGrant
if err := decode(data, &g); err != nil {
return ControllerGrant{}, err
}
if err := g.check(); err != nil {
return ControllerGrant{}, err
}
return g, nil
}
// SameControllerGrant says whether two grants are the same value.
func SameControllerGrant(a, b ControllerGrant) bool { return reflect.DeepEqual(a, b) }
// NamesNode says whether the grant names module's accounts on node: its own `<node>.<module>` or a process's
// `<node>.<module>.<process>`. A grant that names no account (a module that does not speak on the bus) names every
// node. The assigner wants a node only once the grant names its account (issue 1945), and a put waits for it (ADR
// 0546 clause 4).
func (g ControllerGrant) NamesNode(node, module string) bool {
if len(g.Accounts) == 0 {
return true
}
own := node + "." + module
for _, a := range g.Accounts {
if a == own || strings.HasPrefix(a, own+".") {
return true
}
}
return false
}
+125
View File
@@ -0,0 +1,125 @@
package delivery
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"regexp"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/seal"
)
// The adoption handing of novox/hq ADR 0547 clause 9 (issue 1945): the vault adopts, once and before `take-over`, the
// bcrypt hashes of the bus accounts it holds no value for (the controller's own, the node accounts, the enrolled
// devices) from the list the bus has loaded. The node-engine of the node holding mesh-broker is the one part that has
// that list, so it hands the hashes in the bus module's have, beside what it says of the list it wrote: sealed to the
// vault's own key, which it learns from the vault's answer to `node-key`, so that of every reader of the have bucket
// the vault alone can open them. The have says in the clear only how many users it hands, from which list and to which
// key; the users' names stay sealed, as the list itself is (ADR 0547 clause 7: it names every account).
//
// It hands every user of the loaded list; the vault takes from it only the users it holds no value for, and never
// replaces a hash it made from a value it holds, so the node-engine never needs to read the vault's state (which the
// grants part alone is to read, clause 9).
// AccountsHanded is the hashes of the loaded list's users, sealed to the vault, in UserListWritten.
type AccountsHanded struct {
// From is the digest of the list on the machine the hashes are taken from, `sha256:<hex>` (UserListWritten.Digest).
From string `json:"from"`
// To is the fingerprint of the vault's key the box is sealed to (KeyFingerprint).
To string `json:"to"`
// Users is how many users the box holds.
Users int `json:"users"`
// Box is the JSON of []HandedAccount, sealed to the vault's key (ADR 0206).
Box seal.Box `json:"box"`
At time.Time `json:"at"`
}
// HandedAccount is one user of an AccountsHanded box, opened: the bus user and the bcrypt hash the loaded list
// carries for it.
type HandedAccount struct {
User string `json:"user"`
Hash string `json:"hash"`
}
// KeyFingerprint is a public key's fingerprint as a handing names it: `sha256:<hex>` of its PEM, without the space
// around it.
func KeyFingerprint(publicPEM string) string {
sum := sha256.Sum256([]byte(strings.TrimSpace(publicPEM)))
return "sha256:" + hex.EncodeToString(sum[:])
}
// listedAccount is one user of a bus user list as the composer writes it (buscompose), with its password's hash.
var listedAccount = regexp.MustCompile(`\{\s*user:\s*"([^"]+)",\s*password:\s*"([^"]+)"`)
// HandedAccountsOf is every user of a bus user list that carries a bcrypt hash, in the list's order. A user listed
// twice is handed once, as the list first names it.
func HandedAccountsOf(list []byte) []HandedAccount {
var out []HandedAccount
seen := map[string]bool{}
for _, m := range listedAccount.FindAllSubmatch(list, -1) {
user, hash := string(m[1]), string(m[2])
if seen[user] || !IsBcrypt(hash) {
continue
}
seen[user] = true
out = append(out, HandedAccount{User: user, Hash: hash})
}
return out
}
// bcryptHash is the form of a bcrypt hash: version, cost, and 53 characters of salt and digest.
var bcryptHash = regexp.MustCompile(`^\$2[abxy]?\$[0-9]{2}\$[./A-Za-z0-9]{53}$`)
// IsBcrypt says whether s has the form of a bcrypt hash.
func IsBcrypt(s string) bool { return bcryptHash.MatchString(s) }
// HandAccounts seals the accounts to the vault's public key (SPKI PEM) as the handing of the list whose digest is from.
func HandAccounts(accounts []HandedAccount, vaultPEM, from string, at time.Time) (AccountsHanded, error) {
if strings.TrimSpace(vaultPEM) == "" {
return AccountsHanded{}, errors.New("delivery: no vault key to seal the handed accounts to")
}
body, err := json.Marshal(accounts)
if err != nil {
return AccountsHanded{}, err
}
box, err := seal.Seal(string(body), vaultPEM)
if err != nil {
return AccountsHanded{}, fmt.Errorf("delivery: the handed accounts could not be sealed to the vault's key: %w", err)
}
return AccountsHanded{From: from, To: KeyFingerprint(vaultPEM), Users: len(accounts), Box: box, At: at.UTC()}, nil
}
// OpenHanded opens a handing with the vault's private key (PEM), refusing one sealed to another key than publicPEM's
// and a box that does not hold as many users as it says.
func OpenHanded(h AccountsHanded, publicPEM, privatePEM string) ([]HandedAccount, error) {
if h.To != KeyFingerprint(publicPEM) {
return nil, fmt.Errorf("delivery: the handing is sealed to the key %s, not to this one (%s)", h.To,
KeyFingerprint(publicPEM))
}
plain, err := seal.Open(h.Box, privatePEM)
if err != nil {
return nil, fmt.Errorf("delivery: the handing does not open: %w", err)
}
var out []HandedAccount
if err := json.Unmarshal([]byte(plain), &out); err != nil {
return nil, fmt.Errorf("delivery: the handing holds no accounts: %w", err)
}
if len(out) != h.Users {
return nil, fmt.Errorf("delivery: the handing says %d users and holds %d", h.Users, len(out))
}
return out, nil
}
// sameHanded says whether two handings say the same: the same list, key, count and box, whatever zone their time is in.
func sameHanded(a, b *AccountsHanded) bool {
if a == nil || b == nil {
return a == b
}
x, y := *a, *b
x.At, y.At = time.Time{}, time.Time{}
return x == y && a.At.Equal(b.At)
}
+104
View File
@@ -0,0 +1,104 @@
package delivery
import (
"encoding/json"
"fmt"
"sort"
"time"
)
// Whether the artifact store still holds what a done key names (novox/hq issue 1937). A build module pushes a bundle or
// an archive as a bare blob (ADR 0498 §1), and the store's nightly collector deletes every blob no manifest names; only
// the controller's take-in and sweep hold one by its manifest. A done key the controller has not taken in can therefore
// name a blob that is gone, and reusing it sends a definition every machine fails on. So the builder asks before it
// reuses a done key, and the controller says when a take-in found a gone blob.
const (
// HeldVerb is the controller seat's verb the builder asks, as a module's invokes names it: references in, each
// answered held or gone.
HeldVerb = "seat:mesh-controller.held"
// ArtifactsGoneEvent is the controller seat's event said when a take-in is refused because the store does not hold
// an artifact the key names: as a consumer names it, `mesh-controller.artifacts-gone`.
ArtifactsGoneEvent = "artifacts-gone"
// ArtifactsGoneSubject is where it is published (the seat's event subject).
ArtifactsGoneSubject = "mesh.seat.mesh-controller.event." + ArtifactsGoneEvent
)
// HeldAnswer is the answer of HeldVerb: the references the store holds, and those it answered 404 for. A reference the
// store could not be asked about is in neither, and named in Unknown with why.
type HeldAnswer struct {
Store string `json:"store,omitempty"`
Held []string `json:"held"`
Gone []string `json:"gone"`
Unknown map[string]string `json:"unknown,omitempty"`
}
// DecodeHeldAnswer reads HeldVerb's answer: the verb answers {output, ok, answer}, the answer being HeldAnswer. A
// reference asked that the answer names in none of held, gone and unknown is an error: an answer that leaves one out
// would let a key with a gone blob be reused.
func DecodeHeldAnswer(raw []byte, asked []string) (HeldAnswer, error) {
var wrapped struct {
Output string `json:"output"`
OK bool `json:"ok"`
Answer json.RawMessage `json:"answer"`
}
if err := json.Unmarshal(raw, &wrapped); err != nil {
return HeldAnswer{}, fmt.Errorf("the answer of %s is not {output, ok, answer}: %w", HeldVerb, err)
}
if !wrapped.OK || len(wrapped.Answer) == 0 {
return HeldAnswer{}, fmt.Errorf("%s did not answer: %s", HeldVerb, wrapped.Output)
}
var a HeldAnswer
if err := json.Unmarshal(wrapped.Answer, &a); err != nil {
return HeldAnswer{}, fmt.Errorf("the answer of %s is not {held, gone}: %w", HeldVerb, err)
}
named := map[string]bool{}
for _, r := range a.Held {
named[r] = true
}
for _, r := range a.Gone {
named[r] = true
}
for r := range a.Unknown {
named[r] = true
}
for _, r := range asked {
if !named[r] {
return HeldAnswer{}, fmt.Errorf("the answer of %s names nothing of %s", HeldVerb, r)
}
}
return a, nil
}
// ArtifactsGone is the body of ArtifactsGoneEvent: the module whose builder's key the controller would not take in,
// the key and its fingerprint, and every reference of it the store does not hold.
type ArtifactsGone struct {
Event string `json:"event"`
Module string `json:"module"`
Key string `json:"key"`
Fingerprint string `json:"fingerprint"`
Gone []string `json:"gone"`
At time.Time `json:"at"`
}
// EncodeArtifactsGone is the event's body, the references sorted.
func EncodeArtifactsGone(e ArtifactsGone) ([]byte, error) {
e.Event = ArtifactsGoneEvent
e.Gone = append([]string(nil), e.Gone...)
sort.Strings(e.Gone)
if e.Module == "" || e.Fingerprint == "" || len(e.Gone) == 0 {
return nil, fmt.Errorf("delivery: %s names a module, a fingerprint and at least one gone reference", ArtifactsGoneEvent)
}
return json.Marshal(e)
}
// DecodeArtifactsGone reads the event's body, refusing one that names no module, fingerprint or reference.
func DecodeArtifactsGone(raw []byte) (ArtifactsGone, error) {
var e ArtifactsGone
if err := json.Unmarshal(raw, &e); err != nil {
return ArtifactsGone{}, fmt.Errorf("delivery: %s is not one: %w", ArtifactsGoneEvent, err)
}
if e.Event != ArtifactsGoneEvent || e.Module == "" || e.Fingerprint == "" || len(e.Gone) == 0 {
return ArtifactsGone{}, fmt.Errorf("delivery: %s names no module, fingerprint or gone reference: %s", ArtifactsGoneEvent, raw)
}
return e, nil
}
+84
View File
@@ -0,0 +1,84 @@
package delivery
import (
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
const (
refA = "artifact-store://node-tools/runtime/blobs/sha256:a46b5d51a30e8fe71107d75ceb46e3187022fe51f551d0498b8be817f22435cb"
refB = "artifact-store://mesh-host/host-arch/blobs/sha256:8a752f914f3523904a5442cae37ce3cb1ad297d6789b0f871585bce4d0c2ea00"
)
// 1937/AC1 (the contract): the controller's answer reads into held and gone, and an answer leaving a reference out is refused.
func TestDecodeHeldAnswer(t *testing.T) {
raw := []byte(`{"output":"…","ok":true,"answer":{"store":"s:5100","held":["` + refA + `"],"gone":["` + refB + `"]}}`)
a, err := DecodeHeldAnswer(raw, []string{refA, refB})
if err != nil {
t.Fatal(err)
}
if len(a.Held) != 1 || a.Held[0] != refA || len(a.Gone) != 1 || a.Gone[0] != refB {
t.Fatalf("read %+v", a)
}
if _, err := DecodeHeldAnswer(raw, []string{refA, refB, "artifact-store://x/y/blobs/sha256:00"}); err == nil ||
!strings.Contains(err.Error(), "names nothing of") {
t.Fatalf("an answer leaving a reference out was taken: %v", err)
}
if _, err := DecodeHeldAnswer([]byte(`{"output":"the store is down","ok":false}`), nil); err == nil ||
!strings.Contains(err.Error(), "the store is down") {
t.Fatalf("a failed verb was taken: %v", err)
}
}
// 1937/AC2 (the contract): the event's body names the module, the fingerprint and the gone references, sorted.
func TestArtifactsGoneRoundTrip(t *testing.T) {
raw, err := EncodeArtifactsGone(ArtifactsGone{Module: "node-tools", Key: "node-tools.75", Fingerprint: "src2:75",
Gone: []string{refB, refA}, At: time.Unix(0, 0).UTC()})
if err != nil {
t.Fatal(err)
}
e, err := DecodeArtifactsGone(raw)
if err != nil {
t.Fatal(err)
}
if e.Event != ArtifactsGoneEvent || e.Module != "node-tools" || len(e.Gone) != 2 || e.Gone[0] != refB {
t.Fatalf("read %+v", e)
}
if _, err := EncodeArtifactsGone(ArtifactsGone{Module: "m", Fingerprint: "src2:1"}); err == nil {
t.Fatal("an event naming no gone reference was made")
}
if _, err := DecodeArtifactsGone([]byte(`{"event":"artifacts-gone","module":"m"}`)); err == nil {
t.Fatal("an event naming no fingerprint was read")
}
}
// The verb and the event are on the controller seat's contract under the names the builder uses.
func TestHeldOnTheControllerSeat(t *testing.T) {
if HeldVerb != "seat:"+catalogue.ControllerSeatName+".held" {
t.Fatalf("HeldVerb %q", HeldVerb)
}
found := false
for _, v := range catalogue.ControllerVerbs {
if v.Name == "held" {
found = true
}
}
if !found {
t.Fatal("the controller seat declares no verb held")
}
for _, s := range catalogue.DefaultSeats() {
if s.Name != catalogue.ControllerSeatName {
continue
}
for _, e := range s.Emits {
if e == ArtifactsGoneEvent {
return
}
}
t.Fatalf("the controller seat emits %v, not %s", s.Emits, ArtifactsGoneEvent)
}
t.Fatal("no controller seat")
}

Some files were not shown because too many files have changed in this diff Show More