Author SHA1 Message Date
jschoubben 85d1707e82 selfcompose: the trunk-ask backoff test is a subtest of an existing test, no function added
Builds: design 47 G-22 A-487a, the test budget (module total back to main's count)
2026-11-19 01:24:02 +01:00
jschoubben cb6cd9d996 selfcompose: hear catalog-revision on the contract's subject, consume it in the manifest, one test
Subscribe to meshcatalog.SubjectCatalogRevisionAll (no string of its own); module.json consumes
mesh-catalog.catalog-revision so the node account is granted it; the two new tests are folded into
TestTrunkAskBackoffAndCatalogRevision and wait on channels, not time.

Builds: design 47 G-22 A-487a, A-487c, principle 3
2026-11-19 01:18:38 +01:00
jschoubben 2057c9d304 selfcompose: a refused catalog-revision subscription is said once and tried again, never refuses Watch
The subscription is a wake-up; the backoff bounds the asks without it. The refusal is said with its subject and error
(principle 3), the composer goes on, and the subscription is tried again on laterAgain's steps until it holds.

Builds: design 47 G-22 A-487a, principle 3
2026-11-19 00:59:17 +01:00
jschoubben 6ac8952541 selfcompose: ask a trunk build the catalog gave none of again on a backoff or a catalog-revision, not at every pass
A trunk build answered none waits laterAgain's step for its module (5 s growing to 5 min); the seat mesh-catalog's
event catalog-revision, heard through Env.Hear, ends the wait of every module at once; an answer starts the steps
again. A pass is woken at the step's end.

Builds: design 47 G-22 A-487a, A-461e, G-12 line 317
2026-11-19 00:57:21 +01:00
mesh-tools 6e89a5defd Merge pull request 'resolution: an own secret owned by the operator account or root (ADR 0680 §4 test rows)' (#237) from fix/adr0680-own-secret-owner-rows into main 2026-11-18 01:18:52 +00:00
jschoubben 4b11f66b4b resolution: an own secret's owner may be the operator account or root, as rows of the owner table (ADR 0680)
Builds: ADR 0680 §1, §4
2026-11-18 02:11:40 +01:00
mesh-tools 0439af3649 Merge pull request 'A-480..A-484a: a held provider's provides, the one place for a running build's manifest, the composition tool's deadlock fix' (#236) from fix/a480-held-provider-provides into main 2026-11-18 01:11:04 +00:00
jschoubben 5cc26e2659 selfcompose: the one place says its source at every caller, reads the have's build, answers HoldsData; the tool composition reads it outside the agent's lock (A-484, A-482)
Builds: design 47 G A-484, A-484a, A-482
2026-11-18 01:02:10 +01:00
jschoubben 91dc064f92 selfcompose: the running statement's manifest is the second source of the one place; a composition answers only whether the build keeps data (A-484a)
Builds: design 47 G A-484a, A-484
2026-11-18 00:49:53 +01:00
jschoubben 7b6468e2bb selfcompose: the manifest of the build a module runs here is answered in one place; every step reads it there (A-484)
Entry of the running build, else the composition this node runs of it, else the module's trunk manifest through the seat mesh-catalog's verb module;
said once per module and source; words naming the three where none answers. Read there: a held module's offer (frombus Scope.RunningManifest),
FormChanges (A-483), the store holder and the stand of a running module without an entry (A-461d, A-480a), and the tool composition.

Builds: design 47 G A-484, A-480, A-480a, A-483, A-461d, A-482
2026-11-18 00:46:49 +01:00
jschoubben 0cc1728a04 agent: a held have is judged again at a start from its wait's reason, not from its detail's opening words (A-453, §541)
Builds: design 47 G A-453, §541, A-484
2026-11-18 00:34:36 +01:00
jschoubben d1768241dc selfcompose: a running build with no entry keeps what its running composition keeps; with no data on either side the move goes on (A-483)
Builds: design 47 G A-483
2026-11-18 00:23:46 +01:00
jschoubben cdd8bd3cba selfcompose: what is read decides the trunk build's stand: the running event's manifest held, a running composition's build read as ranOf reads it (A-480a, A-482)
Builds: design 47 G A-480a, A-482
2026-11-17 23:58:13 +01:00
jschoubben 121a8090b3 composer: a take-in's templates digest is no difference between two manifests (A-481a)
Builds: design 47 G A-481a
2026-11-17 23:57:16 +01:00
jschoubben ba9d4787f9 selfcompose: a module counts as running here only when heard, run by the agent or its have says running or held; the trunk build stands only where no manifest is named (A-482, A-480a)
Both review findings of 7bfeb466 and 179a4a64 sit in runsWithoutEntry and are one change.

Builds: design 47 G A-482, A-480a
2026-11-17 23:46:03 +01:00
jschoubben 87cfae48b2 agent: the parts check counts the templates archives a composition was rendered from as fetched (A-481, A-481a)
compose fills Composed.Templates from the resolved manifest's TemplatesDigest; partsDifference adds them to the digests fetched.

Builds: design 47 G A-481, A-481a
2026-11-17 23:44:15 +01:00
jschoubben 179a4a6402 selfcompose: a module built by hand, with no running event, is held as running; its trunk build is asked (A-480a)
Builds: design 47 G A-480a (G-22-lead-answers-2.md)
2026-11-17 23:16:33 +01:00
jschoubben 7bfeb466bf selfcompose: a module that runs with no entry of its wanted or running build counts what its trunk build provides (A-480a)
Builds: design 47 G A-480a (G-22-lead-answers-2.md)
2026-11-17 23:09:15 +01:00
jschoubben e7803f8943 frombus: the A-480 rows join the held-rows test, no new test function
Builds: design 47 G A-480 (G-22-lead-answers-2.md)
2026-11-17 23:01:09 +01:00
jschoubben baeb87d195 frombus: a held module still provides what its running build provides (A-480)
A module the node holds keeps running what it runs, so a module that requires what it provides is no longer held for
"nothing provides": the held offer's provisions are the running composition's build's manifest, else the manifest the
module's running event names.

Builds: design 47 G A-480 (G-22-lead-answers-2.md)
2026-11-17 22:58:11 +01:00
jschoubben 9172eccbc4 selfcompose: only what the node needs of another node's placement is asked, beside the pass
The trunk build is asked, and a placed module laid, only for the store's
holder (a store seat's holding on that node) and the providers this node's
modules are bound to; never every module placed elsewhere. The asks run beside
the pass, three at a time, and an answer asks a pass; a module that waits is
only the one needing that address.

Builds: design 47 G-22 A-461, A-461d, A-461e
2026-11-17 16:52:28 +01:00
jschoubben 07ea73bd64 selfcompose: a stopped or build-less placed module is laid from its placement; an unobtained trunk build is asked again
lacksEntry holds for a stopped run and for one naming no build, and the loop
for another node's placed modules lays every one not yet laid where it holds.
A trunk build askTrunkBuilds could not get is said with its reason and asked
again on the next pass.

Builds: design 47 G-22 A-461d, A-461e
2026-11-17 16:12:51 +01:00
jschoubben d57e2325f6 composer: a store fetcher left out is held with its words, so its handed composition stays
Declared.Held carries the modules composeWithoutStoreFetchers leaves out
(A-452: each stays on the build it runs). The store-address comment and code
say A-461e: no address read without error holds only the fetchers, a fault
reading the inputs still refuses the node.

Builds: design 47 G-22 A-452, A-461, A-461e
2026-11-17 16:11:59 +01:00
jschoubben 43c812e98e selfcompose: a module placed on another node and not heard running is laid from its placed or trunk build
Being heard running is no condition: such a module counts as one whose
running build has no entry (wantedBuilds, askTrunkBuilds, layModules,
lacksEntry), so the store holder is found from the build its placement names or
its trunk build.

Builds: design 47 G-22 A-461d, A-461b
2026-11-17 16:06:01 +01:00
jschoubben e3d994fde1 composer: a module that fetches from an artifact store no node reaches waits alone, never the node
Typed resolution.NoStoreError from artifactsInto (image, source, tools bundle,
runtime program, templates archive), laid on its module by
composeWithoutStoreFetchers with A-452's "nothing provides" words and composed
again, instead of a whole-node refusal. Replaces storeHeld and
modulesNeedingTheStore, whose shape list missed bundles and held stand-ins.

Builds: design 47 G-22 A-452, A-461, A-461b, A-459b
2026-11-17 16:03:18 +01:00
mesh-tools eb89cb7da3 Merge pull request 'G-24: the node-engine's composer names the modules it blames' (#235) from fix/47-G-composer-blames into main 2026-11-17 12:15:37 +00:00
jschoubben ed7cefb5d8 composer: the search for culprits goes on past a hold that changes the problem (G-24 review r2)
A hold that changes the problem but leaves the machine not composing keeps that module held and blamed,
and the search goes on against the retry's problem within the same blame_tries, until the machine
composes or no hold changes it: every module the change refuses is blamed, never only the first.

Builds: design 47 G §721, §722, A-470c, A-470v
2026-11-17 13:10:51 +01:00
jschoubben f21b8c202d composer: a build that can never be made is blamed after the held composition is judged (G-24 H3 review r1)
The held composition's own blames and its search come first; the module whose build can never be made is
then blamed beside them. A held composition that still fails and blames nobody stays on the node's own
account, so the module is not blamed in its place.

Builds: design 47 G §721, §722, A-470j, A-470k
2026-11-17 13:05:38 +01:00
jschoubben b4bf772de4 composer: one search for culprits, inside the node-engine's composition, bounded by blame_tries (G-24 H2 review r1)
ComposeSent no longer runs the node-engine's own culprit search (fixed at eight modules), and the composer's
own loop, which reran that search at every try, is gone: ComposeSent holds the case's changed modules on the
machine one at a time, one composition each, at most blame_tries, and blames the first whose hold ends the
problem; past the bound the untried are named. Without blame_tries there is no search. A module the node
holds composing without the change is held as there, and two owners of one path are blamed outside the search.

Builds: design 47 G §721, A-468i, A-468l, A-469j, A-469k, A-470c
2026-11-17 13:03:15 +01:00
jschoubben c5f76cb974 composer: a build that can never be made is blamed, not a reason to judge no machine (G-24 H3)
A module awaited on a machine whose revision gives it no fingerprint (the case's none) is held as its
node-engine holds it, the machine judged from that composition, and the module blamed with the revision's
none words, so only its own pipelines fail. An awaited build with a fingerprint keeps the machine not judged.

Builds: design 47 G §721, §722, A-470j, A-470k
2026-11-17 12:04:20 +01:00
jschoubben 8700aabba3 composer: a machine that fails with no module held is searched for its culprit, bounded by blame_tries (G-24 H2)
The composer program has no settings, so the assigner bounds the search through the request: only the candidate's
changed modules placed on the machine are held, one at a time, the search ending at the first whose hold ends the
problem; past the bound the untried are named. A request with no blame_tries gets no search.

Builds: design 47 G §721, §733; A-468h, A-468i, A-469i, A-469j, A-469k
2026-11-17 11:21:49 +01:00
jschoubben 06781d4dbc composer: a machine's verdict names the modules it blames (G-24 H1)
The assigner fails only the composition steps of the modules a machine blames, so the composer names them: the modules
the change refuses that the node held neither in its read nor composing without the change, answered structured by
ComposeSent, and both owners of a path two modules own, from the resolution's pairs. Blamed is always written, so a
verdict blaming none is told from an old composer's. The two request-file tests are one, for the test budget.

Builds: design 47 G §721, §733; A-468h, A-468l, A-469h, A-469j
2026-11-17 11:19:54 +01:00
mesh-tools d0b208b668 Merge pull request 'node-engine: the golden node grant holds every verb the manifest invokes (A-461c)' (#234) from fix/47-G-node-engine-integration into main 2026-11-17 08:49:22 +00:00
jschoubben 85ad1eef74 nodegrant: the A-461c check of the golden grant runs in an existing test, no test added (test budget)
Builds: design 47 G-22 A-461c
2026-11-17 09:46:55 +01:00
jschoubben 51b89964c5 nodegrant: the golden grant is checked against the manifest's invokes and holds every invoked verb's call subject (A-461c)
Builds: design 47 G-22 A-461c, A-461b
2026-11-17 09:41:36 +01:00
jschoubben 1a6a380507 nodegrant: the node's grant holds the catalog verb module the composer asks (A-461b)
The composer asks the trunk build of another node's module through the seat mesh-catalog's verb module (declared in the manifest); the golden grant lacked its call subject, so the ask hung to the node's bound in every pass and held the integration tests.

Builds: design 47 G-22 A-461b
2026-11-17 09:39:03 +01:00
mesh-tools 0612916af0 Merge pull request 'Node-engine: holds carry their wait; upstream images are no part; the store's address from another node's placed or trunk build (A-453, A-454, A-454a, A-461, A-461a, A-461b)' (#233) from fix/47-G-hold-carries-its-wait into main 2026-11-17 00:23:00 +00:00
jschoubben 10a6824abe Compose: a fault of finding the store other than its listen refusal still refuses the node, tested (A-461b)
Builds: design 47 G A-461b
2026-11-17 01:22:03 +01:00
jschoubben bb285f9c1b Selfcompose, compose: the trunk build stands where a placement names none; the fallback is another node's only; only the store listen refusal holds fetchers (A-461b)
Builds: design 47 G A-461b
2026-11-17 01:19:21 +01:00
jschoubben cbed32524a Selfcompose: another node's module whose running build has no entry is laid from its placed build's entry (A-461, A-461a)
Builds: design 47 G A-461 (first sentence), A-461a
2026-11-17 01:12:37 +01:00
jschoubben b132653fc0 Compose: an artifact store with no known address holds the modules that fetch from it, not the node (A-461, second part)
Builds: design 47 G A-461 (second sentence; the first, laying another node's module from its want's entry, waits on a lead answer)
2026-11-17 01:10:49 +01:00
jschoubben 523ad97e28 Node-engine: a digest the want's parts name always counts as fetched (A-454a)
Builds: design 47 G A-454a
2026-11-17 01:06:57 +01:00
jschoubben 5b4484efd8 Node-engine: a hold after an interrupted backup waits as the others do (A-453)
Builds: design 47 G A-453 (§541)
2026-11-17 01:00:57 +01:00
jschoubben 0a43489a74 Node-engine: an upstream copy served through the registry seat is no part of a want (A-454)
Builds: design 47 G A-454
2026-11-17 00:49:44 +01:00
jschoubben dbb77efab1 Node-engine: only an image the mesh built is a part of a want (A-454)
A container's image counts as fetched for the parts check only when the composition
names the seat it came through; an upstream image its manifest pins by digest is no
part and is never 'fetched and not in the want's parts'.

Builds: design 47 G A-454 (§421's parts check, A-432)
2026-11-17 00:48:46 +01:00
jschoubben 1b9f9355b5 Node-engine: a hold of the composition carries its wait, step compose (A-453)
A module not composed for want of its entry (entry-asked, entry-missing) and a want
whose parts differ write their have with a Wait of step compose and that reason, as
the kept data's holds do; the hold for a failed backup carries none.

Builds: design 47 G A-453 (ADR 0539 §16, §541)
2026-11-17 00:39:39 +01:00
mesh-tools 72385fd84a Merge pull request 'Node-engine: an unanswered requirement costs its consumer, not the machine (A-452)' (#232) from fix/47-G-unanswered-requirement-holds-consumer into main 2026-11-16 23:28:45 +00:00
jschoubben 3bb895152f Resolution: an optional consumer of an unanswered requirement is left absent and listed (A-452, ADR 0330 §3, §7)
Builds: design 47 G A-452
2026-11-17 00:27:47 +01:00
jschoubben c6d893e141 Resolution: a requirement nothing in the mesh answers holds its consumers too; optional ones stay absent (A-452)
The mesh-wide refusal ("nothing in this mesh provides") holds its consumers as the
node-wide one does; a module that wants the name only optionally is left absent
(ADR 0330 §3), never held. A held why still names the nodes that offer nothing
here. The two tests of the first commit are folded as rows into one table test
that replaces two existing ones.

Builds: design 47 G A-452
2026-11-17 00:25:30 +01:00
jschoubben 310ff0cabc Resolution: an unanswered requirement holds its consumers, not the node (A-452)
A requirement nothing answers no longer refuses the whole composition: each
module that wants it is held with the "nothing provides" words, as a refused
claim's claimant is, and the node composes the rest. Held modules stay in the
set, so what they provide still counts. The node is refused when every module
is held or no module of the set wants the name. Tests that used an unanswered
requirement to make a node's own set fail now use an ambiguous one.

Builds: design 47 G A-452 (extends A-405, A-407; ADR 0636 §3)
2026-11-17 00:18:25 +01:00
mesh-tools 13fcfae051 Merge pull request 'Node-engine: judge a move's data forms from the entry kept of the running build; no endless handing again' (#231) from fix/47-G-engine-own-manifest into main 2026-11-16 19:00:07 +00:00
jschoubben f8f33e54a7 Node-engine: judge a move's data forms from the entry kept of the running build; no endless handing again
The running build's entry stays held (kept in the composition state) while the node runs it, though the catalog half dropped it; the agent no longer tells the composer it waits for a composition it holds, and the composer hands again when the entry answers.

Builds: design 47 G §296, §297, A-113, §614
2026-11-16 19:59:20 +01:00
mesh-tools 869d15e6d3 Merge pull request 'Design 47 G start: node-engine fixes placed by hand (A-226b/c, A-237b/c, manifest digest)' (#230) from fix/47-G-start-2-host into main 2026-11-16 14:15:10 +00:00
jschoubben 292aa54bbb node-engine: compose-candidate's chunked input is tested through HandActs.Call and past the inline bound, folded into the compose integration test; comments cite A-237c
Builds: design 47 G (G-22 lead answers 2) A-237c; A-237b
2026-11-16 14:49:39 +01:00
jschoubben f539cdb7ec node-engine: compose-candidate sends its composition input in chunks over the link's connection when asked with a call token (A-237b)
Builds: design 47 G (G-22 lead answers 2) A-237b; A-237, A-249
2026-11-16 14:38:31 +01:00
jschoubben 3b365ba7cc node-engine: a catalog revision's manifest digest is made the candidate's hex form (A-226b)
Builds: design 47 G (G-22 lead answers 2) A-226b
2026-11-16 14:26:03 +01:00
jschoubben 6eb41a17e1 node-engine: a run's compose-candidate key reads candidates.run-<id> by its group, with no heads compared (A-226c)
Builds: design 47 G (G-22 lead answers 2) A-226c; A-226b
2026-11-16 14:00:30 +01:00
jschoubben a1cd4a1a2f node-engine: compose-candidate reads a branch pipeline's catalog revision through mesh-catalog.revision and composes it as the candidate (A-226b)
Builds: design 47 G (G-22 lead answers 2) A-226b; ADR 0593 §7
2026-11-16 13:58:36 +01:00
jschoubben 7a30c0d54c node-engine: fold the three new tests into existing tests as cases, so the branch adds no test function (test budget)
Builds: design 47 G §524, §525, A-405, A-407, A-430, A-430a, A-432 (hq main 3723b2ea), ADR 0259 §8
2026-11-16 09:40:29 +01:00
jschoubben a9eb4372d9 Merge remote-tracking branch 'origin/main' into fix/47-G-start-2-host 2026-11-16 09:25:14 +01:00
jschoubben c025bb5f1e node-engine: at a start a hold for differing parts or a missing entry is judged again, not kept as a failed backup waiting for an answer (§524, §421, ADR 0568 §4) 2026-11-16 01:04:07 +01:00
jschoubben 5a3510b115 node-engine: a runtime of its own runs the tool runner's program, which the module's parts check leaves out (design 47 G §524, A-432) 2026-11-16 01:01:01 +01:00
jschoubben 7f892a757c node-engine: a held tool runner's kept program is not replaced by the tools bundles it serves, so novox's new indexer is run by it (ADR 0259 §8) 2026-11-16 00:50:33 +01:00
jschoubben 40ed0633e0 node-engine: a held tool runner keeps its running program, so a module of its own account is not held for want of a runtime that runs (ADR 0259 §8) 2026-11-16 00:34:50 +01:00
jschoubben 5bbde5db40 node-engine: the composer judges a changed want by its fingerprint, version, parts and keeping node, so a want that gains its parts is composed (design 47 G §525) 2026-11-16 00:28:41 +01:00
mesh-tools 21e452df28 Merge pull request 'Node-engine invokes the registries' versions verbs (design 47 G A-430a)' (#229) from fix/47-G-engine-invokes-versions into main 2026-11-15 16:08:13 +00:00
jschoubben dc32afdd7e Node-engine invokes the registries' versions verbs, so its grant admits resolving a want's version (design 47 G A-430a)
The grants part admits a seat verb's invokers by the manifest on the trunk; a node-engine that resolves a want's
version through oci-artifact-registry.versions and container-image-registry.versions is refused on the bus until
its manifest on main names both. Manifest only: a node-engine that does not resolve versions never calls them.
2026-11-15 17:04:37 +01:00
jschoubben 8a0c39280d Node-engine: a want naming its version is resolved to its digests through the registry seats (A-430a)
A want of a released build names the module's version and no parts. Each artifact of the build's entry is
resolved at that version through the verb versions of its kind's registry seat: oci-artifact-registry for a
bundle or an archive, container-image-registry for an image, under <module>/<artifact> or a package's own
name. An upstream copy, which is not numbered, takes the digest its manifest pins. partsMissing, buildOf and
KeptBuildOf (and the shadow's added modules) read those digests; a want with parts is read as before.

A lookup that fails is said in the journal, naming the module, the version and the seat, and holds the module
whole: nothing of it is composed. It is asked again after a minute. The manifest invokes both verbs.

Builds: design 47 G A-430, A-430a; lead answer A-430b
2026-11-15 16:52:59 +01:00
jschoubben a00cccfe0d Node-engine: a running module the assignment names is held, never removed, whatever composed nothing of it (A-410)
Builds: design 47 G A-410
2026-11-14 23:38:29 +01:00
jschoubben 592b4002bd Node-engine: a running module with no fingerprint is held in its answer, never removed (A-410)
Builds: design 47 G A-410
2026-11-14 23:18:36 +01:00
jschoubben 0adc5ab4e6 Node-engine: a running module without an entry resolves as the newest build its want names
Builds: design 47 G §421, lead answers A-405 and A-407
2026-11-14 22:47:33 +01:00
jschoubben 6079f634d6 Node-engine: say whether its seat is served and how long the first looks of what runs and of the accounts take (A-405)
Live on novox 2026-11-14 root-free answered 'judged no account yet' minutes after the start and nothing was logged:
the seat's serving and the first looks were silent.
2026-11-14 15:59:28 +01:00
jschoubben eaf011e971 Node-engine: a first start whose composing never ends is named by MESH_ENGINE_OWN_MODULE (A-404, live start recovery)
A-403 named the engine from the base's catalogue, but on novox the base stops at the first manifest it cannot read,
before mesh-host, so the engine stayed unnamed and its seat unserved.
2026-11-14 15:53:44 +01:00
jschoubben 1c3b2597ba Node-engine: named from the catalogue the base read, before the rest of the input (A-403)
Live on novox 2026-11-14 the G node-engine never ended a composing pass, so it never named its own module and never
served its seat node-engine: grants got no responders for root-free, judged novox not free, and the operator channel
held every Approve ask (the seed, hq#609) as no channel could prove who answered.
2026-11-14 15:48:45 +01:00
mesh-tools fd023f7473 Merge pull request 'Milestone 47 G: mesh-host's part of the one release' (#228) from feat/47-G-join12 into main 2026-11-13 13:09:29 +00:00
jschoubben 5caa760f5e A-342 engine-node-env: every process runs with its node's name as MESH_NODE
mesh/delivery merged: source.main moved to novox/mesh-host@fd023f7473bc
mesh/delivery-group group feat/47-G-join12 merged: every member is merged
mesh/check failed (2): mesh-delivery-checker: no published of the tagged version of go-meshconditions of me…; and 1 more | 5caa760f5eaa on fd023f7473bc
The apply gives each process the declaration's For as MESH_NODE beside its own env, so a process that reads MESH_NODE (mesh-sdk, the zfs follower's writer and its node-service-manager calls) names its node.

Builds: design 47 G A-342

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 09:20:46 +01:00
jschoubben d63aae1d55 Merge main into feat/47-G-join11: the node-engine's if-config, config-lines, config-json and stop at a failure (issue 1979)
The whole test's web manifest keeps G's entry form (no build artifact, the entry read by fingerprint) and takes main's resources under if-config and config-lines.

Builds: design 47 G §421; design 42 zfs-owns-zfs H1-H5

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 09:10:44 +01:00
mesh-tools f8acfaeed7 Merge pull request 'The node-engine stops a module's apply at its first failed resource, and reads if-config, config-lines and config-json (issue 1979)' (#227) from feature/1979-zfs-owns-zfs into main 2026-11-13 08:00:43 +00:00
jschoubben 0a9d1efe89 Merge main into feat/47-G-join11 (G takes in main before its push)
mesh/delivery-group group feat/47-G-join11 checking: 17 of 17 member(s) ready
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 0a9d1efe8918 on f8acfaeed79d | novox/mesh-host#226 in group feat/47-G-join11
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 02:16:56 +01:00
jschoubben 96a8690639 Fix 1979: mib-bytes writes the bytes as a whole number for every size
mesh/delivery merged: source.main moved to novox/mesh-host@f8acfaeed79d
mesh/delivery-group group feature/1979-zfs-owns-zfs merged: every member is merged
mesh/check being carried to 96a869063977 on f8acfaeed79d | from 0193949efe9f, mesh-catalog 117f933dacf0, mesh-delivery 7221a1e5cd7f, mesh-sdk e42b9491…
A size of a million MiB or more reached the file in exponent form and was refused.

Builds: design 42 zfs-owns-zfs §3, A-24
2026-11-13 01:32:36 +01:00
jschoubben 7868c9f365 Fix 1979: ${config-json} writes only the keys the module declares under config
A key a layer holds that the manifest does not declare (scrub-pool during a rename) is not a config value of the module.

Builds: design 42 zfs-owns-zfs §9, A-15, A-24, A-25
2026-11-13 01:32:27 +01:00
jschoubben 225f2abb1f H5 engine-integration: the node-engine whole applies if-config, config-lines and a module's stop at a failure
The existing node-engine integration test's module gains a file under if-config,
a template with config-lines from a list, and two files that fail and are left
under a value; the test asserts the file appears and goes with its value, the
lines follow the list, and the failing resource fails the module with the later
resource left. No integration test is added.

Builds: design 42 zfs-owns-zfs §5 (and §1 to §3), A-22
2026-11-13 00:24:20 +01:00
jschoubben 61806a7a92 H4 engine-states: the node record states the new words and forms
The node record's reads come from mesh-sdk's ManifestReads, so building with S1
and S2 states word:resources.if-config, form:config-lines and form:config-json;
the record test now asserts them.

Builds: design 42 zfs-owns-zfs §4, answer A-13
2026-11-13 00:22:48 +01:00
jschoubben 61c42f9fbc H3 engine-config-lines, H3b engine-config-json: a template writes lines from a config value and the config as JSON
`${config-lines:<key>:<prefix>[:mib-bytes]}` in a file's content writes one line
per entry (nothing for an unset value or empty list; a newline in an entry
refuses the composition); `${config-json}` writes the module's set config values
as one JSON object. Both join the sweep's table of fillers. Their rows join the
config table test (A-22).

Builds: design 42 zfs-owns-zfs §3, §9; answers A-15, A-24, A-22
2026-11-13 00:22:22 +01:00
jschoubben 86048f0ed9 H2b engine-found-handed: the found state of a unit passes to the record that stays
When one record of a unit goes while another still declares it, what the going
record found of the unit is kept for the staying one when that has no finding of
its own, so the module's leave gives back the state found before the mesh. The
preview test and the new case are one test (A-22).

Builds: design 42 zfs-owns-zfs §2, answers A-21, A-22
2026-11-13 00:20:59 +01:00
jschoubben 039415cdd9 H2 engine-if-config: a resource is declared only while a config value holds
The composer leaves out each resource whose `if-config` holds none of its
conditions against the module's config values (with defaults), and drops the key
from the rest, as the node-engine parses strictly. A resource that leaves is
removed by the existing orphan removal. One table test replaces the two tests of
setting_into_test.go.

Builds: design 42 zfs-owns-zfs §2, A-21 (declaration side), A-22
2026-11-13 00:19:49 +01:00
jschoubben fb492da7e7 H1 engine-stop-at-failure: a module's apply stops at its first failed resource
A failed resource of a module gates the rest of that module, which is reported
skipped with the failed resource named; the module's result names each one left.
A resource of no module, and other modules, go on as before. One table test
replaces two in apply_test.go; two fixtures adapted to the rule.

Builds: design 42 zfs-owns-zfs §1 (ADR 0679 §16 (a)), A-22
2026-11-13 00:19:49 +01:00
jschoubben a12417fc13 Remove the real sleep from the recovery integration test
mesh/delivery-group group feat/47-G-join11 checking: 7 of 7 member(s) ready
mesh/delivery closed: a newer head of the same pull request
mesh/check checking a12417fc13c7 on 0193949efe9f | novox/mesh-host#226 in group feat/47-G-join11
Builds: design 47 G (playbook 07 step 2: no real sleeping in a test)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 23:44:02 +01:00
jschoubben 8656e57eb9 Test 36: the recovery wiring is driven through the node agent by a revert want of db
Builds: design 47 G §296, §297, §554, §560, A-113, A-320

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 23:39:39 +01:00
jschoubben 650910dc9e R3-3: the node's own composer has no path without an entry call; its tests compose from the catalog half
Builds: design 47 G §628, §421, A-312, A-317

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 23:33:31 +01:00
jschoubben 321a0744d8 H9 test: entry-again takes a shadow's not-found entry in
Builds: design 47 G §698; lead answer A-319

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 23:13:57 +01:00
jschoubben 860ccd7e30 H9: an entry asked again by hand is asked again in the shadow's cache too, so a candidate waits no longer on a not-found it holds
Builds: design 47 G §698 (entry-again), §421 (A-308: the shadow reads the entries its wants name)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 23:12:41 +01:00
jschoubben 23799a30ed R3-1, R3-2, R3-3: the node-engine reads no build of the builder; the shadow builds from the entry it holds
The composer's Build is made from the catalog half's entry it holds and the parts of the want that names it (buildOf); the
inputs of the indexer's modules and the builder's builds, takeBuildDone, takeModule and the builder read are gone, and mesh-host
names no BuilderBuildsBucket. The shadow takes the entries of the builds it composes from (askBuildsEntries), builds a module a
candidate adds from its entry and the candidate's `added` parts (A-316), and holds, with the reason, a module whose entry
has no part given, whose fingerprint the verb module does not give, or for which it errors (A-312). Its composer is given the
entry call as the node's own is. Rows: a built module added with and without parts, the verb module answering no fingerprint
and erroring.

Builds: design 47 G §421, §628, A-308, A-312, A-316
2026-11-12 23:05:06 +01:00
jschoubben e558510a54 H5 (review r2, A-308, A-312): the shadow watches no indexer modules; it reads the entries its wants name and asks the verb module for a module a candidate adds
Builds: design 47 G §421, §628
2026-11-12 22:28:15 +01:00
jschoubben 1d409cd852 H3 (review r2): the entries test waits on the refusal the composer says until the ghost's reason is entry-missing
Builds: design 47 G §421, §614 (test only)
2026-11-12 22:24:38 +01:00
jschoubben 302dcc098f H4 (review r2, A-307): the node-engine's manifest names no uses; its invoke of seat:mesh-grants.withheld is the edge
Builds: design 47 G §635
2026-11-12 22:16:40 +01:00
jschoubben 3167f0104f H1 (review r2): the foundation host is the entry that claims the seat node-engine, not a module named mesh-host
Builds: design 47 G §527, §634
2026-11-12 22:16:27 +01:00
mesh-tools 0193949efe Merge pull request 'The own resolver's test reads the options from the catalog's template (issue 1748, ADR 0678)' (#225) from fix/1748-resolver-wait into main 2026-11-12 17:52:54 +00:00
jschoubben 8dc186944e fix-1748-host-test: the own resolver's test reads the options from the catalogue's template
mesh/delivery-group group fix/1748-resolver-wait merged: every member is merged
mesh/delivery merged: source.main moved to novox/mesh-host@0193949efe9f
mesh/check checking 8dc186944e7d on 0193949efe9f | novox/mesh-host#225 in group fix/1748-resolver-wait
TestTheCataloguesResolverComposesOnAMachine takes the options line from the systemd-resolved template beside it instead of the literal timeout:1 attempts:2 edns0, so it passes against the catalogue's main before and after the catalogue's change. Captured copies are left as they are.

Builds: design 50 resolver-wait §3 (and §2 of the main file, line 73)
2026-11-12 18:10:28 +01:00
jschoubben 905265a56b J25 A-293: a placed module's failed newest build said as refused in by-entries mode, on the builder's build-ended event
Reading the catalog half by fingerprint, the node-engine no longer reads the builder's builds, so #224's refused key was
never written there and a reader waited on a build that will never be done. The engine now hears the builder's
build.ended stream, read whole at start then followed (as the assigner hears it); the last trunk end of a placed module
that ended failed is written as the refused key with the event's words, a later done end deletes it, an end naming a
branch moves nothing. The manifest consumes mesh-delivery-builder.build.ended.**.

Builds: design 47 G A-293 (with §421, §628), ADR 0593 §6, issue 1966 U1/U8

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 17:20:30 +01:00
jschoubben 4ef07521eb Merge origin/main into the join: a placed module's failed build said as refused
Brings mesh-host #224 (issue 1966): newestFPs returns a placed module whose
newest build failed, composedValues writes its refused key at that fingerprint,
takeBuildDone starts the pass on a failed build. Conflict in newestFPs settled
by keeping G's by-entries branch first (design 47 G §421, §628: the node reads
neither the indexer's modules nor the builder's builds; it returns no failed),
and main's failed-build path in the other.

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 17:00:32 +01:00
jschoubben a9b9709911 J23 A-292: a node seat event the bus did not take is said again on the same link
After a restart, g14's engine restated its reads three seconds after its
first, bootstrapped statement, and the assigner never heard that event: a
say that failed past the bus's retry bound waited for the next link. Now a
say the bus did not take (no responders, a deadline) is said again on the
link until the bus takes it, off the caller, each failed attempt written to
the engine's journal with the event's name; only a lost link, or a refusal
the bus answered, keeps it for the next link.

Builds: design 47 G §590, A-292
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 16:57:36 +01:00
mesh-tools df1412081b Merge pull request 'The node-engine says a placed module's failed build as refused, so the provisioner stops waiting on it (issue 1966)' (#224) from fix/1966-unpublished-placed-module into main 2026-11-12 15:34:23 +00:00
jschoubben 8e993c759a H1 refused-at-failed-build: a placed module whose newest build failed is published as a refusal
mesh/delivery merged: source.main moved to novox/mesh-host@df1412081bed
mesh/delivery-group group fix/1966-unpublished-placed-module merged: every member is merged
mesh/check checking 8e993c759a5c on df1412081bed | novox/mesh-host#224 in group fix/1966-unpublished-placed-module
The node-engine writes a refused key, with the build's own error, at the indexer's
fingerprint in composed-<node> for a placed module whose newest build the builder
holds failed; a build queued or running stays unpublished (U1). A failed build of a
placed module's newest fingerprint also moves the pass (takeBuildDone), so the key is
written when it lands (U8).

Builds: design 47 fix-unpublished-placed-module U1, U8, ADR 0593 §6
2026-11-12 16:01:24 +01:00
jschoubben 0c37ac378c Join 11: the node seat's discovery test names the engine's own module, not a fixed one
H-17 made discovery read the node-engine's own name (§527) instead of the constant mesh-host,
but the integration test still gave its link no own name and expected mesh-host, so every
endpoint announced an empty module. It now gives the link an own name and expects that one.

Builds: design 47 G §634, A-149

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 14:13:34 +01:00
jschoubben d18bf8b3fd The node-engine's seat reader reads invokes in both forms and refuses a form it cannot read
Builds: design 47 G A-275, §558

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 11:07:38 +01:00
jschoubben bef467febe H-revert: the node agent starts a held revert on the live data when its want says revert live
A want with `revert` `live` starts the reverted build with no restore, no aside taken and no catch-up, and
applies as any want; without the field a revert with no restore point stays held.

Builds: design 47 G §710 (G-21-provider-and-revert-ask.md, "On the answer"), §300, §567

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 09:42:32 +01:00
jschoubben a1a328c43e Merge commit 'a3fea910' into feat/47-G-fix11-hrevert 2026-11-12 09:39:24 +01:00
jschoubben 77e6e48ecd AM-H1: the node-engine's composition names the withheld subjects, read through mesh-grants' withheld
The node-engine never reads the grants part's state: it asks the seat's verb
withheld in its own name, and says withheld unknown when nothing answers.

Builds: design 46 7 §6 (the composition reads them through the verb withheld; the node-engine's manifest gains mesh-grants in uses; "withheld unknown: the grants part does not answer"), lead answer A-239

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:26:10 +01:00
jschoubben 8734d5ee1a AM-D3 path: the node-engine takes the runner's reading and fingerprint into the node's record
The grants part decides a node's withheld subjects from the runner's own
reading tied to the runner that stated it; the merge into capabilities stays
for placement.

Builds: design 46 7 §6 (its node's node-engine takes both into the node's record), lead answer A-203 (runner-capabilities, runner-fingerprint)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:06:16 +01:00
jschoubben a3fea9106a H-find: the findings keep the keys the watchman raised them under
Each finding's id carries its token as a ref (A-14), so machine.<node>.network, .rewritten, .unreachable, .units and module.<module>.<node>.unhealthy and .kept
read as before.

Builds: design 47 G A-197

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:02:07 +01:00
jschoubben 4b215ba6ae H-find: the node-engine raises what only it knows of itself as its own findings
The have's failed phase (apply-failing), unhealthy health (module-unhealthy) and kept words (kept-as-it-runs), and each part of
its health report (machine-network, network-rewritten, network-unreachable, machine-units, relogin-needed,
directory-used-as-found, node-account-fallback), raised and cleared by the held writer where the engine knows the fact.

Builds: design 47 G A-197 (ADR 0596, A-192, A-149)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:00:42 +01:00
jschoubben 4021de1d42 HRUN: mesh-compose reads its pipeline input from MESH_RUN_INPUT
The composer answers the assigner's composition step: the request is read from the file
named by MESH_RUN_INPUT and the verdict is the last line of standard output, exit 0; the
compose-job path stays.

Builds: design 47 G §354, §63, §22; lead answer A-167

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:44:53 +01:00
jschoubben 9afb287b9d Merge branch 'feat/47-G-fix11-a1' into feat/47-G-fix11-hrun 2026-11-12 05:43:50 +01:00
jschoubben c2710f52dd MV1: move contracts and sdk under modules/, importers follow
The delivery repository's contracts and SDK live under modules/ as
mesh-delivery-contracts and mesh-delivery-sdk, their Go module paths
renamed to match the folders (the TypeScript names already did), and
every importer in mesh-delivery, mesh-catalog, mesh-host, mesh-tools
and mesh-sdk follows in the same group: a design as it should be, one
path per module, checked by the group's check building every importer.

Builds: design 47 G §427, §88

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:09:56 +01:00
jschoubben a3b646b3c9 H-24: the node-engine's hold tests are one test of cases, below main's count
1,726 test functions against main's 1,751.

Builds: design 47 G §419
2026-11-12 05:01:00 +01:00
jschoubben 732def66b0 H-21: the manifest invokes revision (none-yet) and withheld, and config view-waits-minutes
The long-form invokes entry is read by mesh-sdk's second reader release (09d6070e): packaging's manifest
tests pass with it placed and fail before it is joined.

Builds: design 47 G §558, §617, §633, §635
2026-11-12 05:00:14 +01:00
jschoubben ac3422f914 H-17: the node-engine's own name is read from its applied record or first composing
Discovery, the finding writer and the vault login names read it; no fixed name.

Builds: design 47 G §527, §634 (A-149)
2026-11-12 04:58:28 +01:00
jschoubben 4d656fe4e1 H-20: a tools bundle's words hold its module's want fingerprint
Builds: design 47 G §519, §618
2026-11-12 04:55:33 +01:00
jschoubben 94f6c3dba4 H-18: running carries healthy and is said again when it changes
Builds: design 47 G §520
2026-11-12 04:54:19 +01:00
jschoubben 7d06ef7997 Join 10: the composer command may import the catalog key codec where A-144 moved it
The codec moved from the mesh-delivery contracts into mesh-delivery's domain SDK (sdk/catalogkey), still a pure package
of the standard library and mesh-sdk; the command's import rule named the old place only.

Builds: design 47 G A-144

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 02:51:10 +01:00
jschoubben efee30e243 Fix 10: the compositions read the catalog key codec from mesh-delivery's sdk
Builds: design 47 G §614, §670 (ADR 0625 §4; lead answer A-144)
2026-11-12 02:26:55 +01:00
jschoubben e8bff57b55 Join 9: the node-engine's link names no seat list verb in its operatorask contract (A-126)
Builds: design 47 G A-126

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:51:54 +01:00
jschoubben c6df8fa20f G §683: a start composes the modules whose kept want and entry it holds before its other inputs are read
The node's own record comes from the node-engine's statement of itself, taken in memory with no bus.

Builds: design 47 G A-117, A-120, §683, A-95
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:12:06 +01:00
jschoubben 054ca5a6b3 G §614: the node-engine writes the finding entry-not-found for a fingerprint the catalog half holds none of
Builds: design 47 G A-117, §614, A-15, A-95
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:07:53 +01:00
jschoubben b3b13f2ae1 G §560: a cancelled recovery act holds the move with the hand act's why
Builds: design 47 G A-117, §560
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:06:39 +01:00
jschoubben 0f108fd8ac internal/apply: a test of the places a module is placed in follows no link on the machine it runs on
Builds: design 47 G A-95 (the owed hang of TestAModulesOwnPlacesAreStillPlaced on real mounts)
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:03:54 +01:00
jschoubben 0dd32d9722 A-101: the node-engine reads the router's record through mesh-sdk's operatorask, the bucket from the seat list
Builds: design 47 G A-101, §700, A-74
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:03:22 +01:00
jschoubben 31697e696d A-113: a revert whose restore fails starts the newer build again; a form change not known waits; holds say wait-held
Builds: design 47 G A-113, §297, §560, §541, §629
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:01:44 +01:00
jschoubben 6441ae7e96 gofmt cmd/mesh-host/keptartifact_test.go
Builds: design 47 G A-95 (the owed gofmt)
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:01:44 +01:00
jschoubben 5694f6c39b Merge branch 'feat/47-G-recover-host' into feat/47-G-join8
# Conflicts:
#	internal/selfcompose/entries.go
#	module.json
2026-11-12 00:27:57 +01:00
jschoubben ece79788a3 A-112: the node-engine reads recovery-wait-minutes from a file its manifest declares, 90 when missing
The manifest declares the file recovery-wait (content ${config:recovery-wait-minutes}); the node-engine reads it at its start and
does not call stdio.Config, which needs the runtime that launches a bundle.

Builds: design 47 G §633, A-112
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 00:21:10 +01:00
jschoubben dc1aee51d9 W13.5: the node agent asks mesh-recovery for a restore point before a form change and restores it at a revert
Before a move across a kept data's form change the node agent asks snapshot-kept and waits for the ended of its own act; at a revert it stops the newer build (Env.Stop), asks kept-points, asks restore-kept with the snapshot of the point of the form and fingerprint it reverts from, waits for restored, applies the reverted build, and calls the point's catch-up as the module's tool through the tool runner. The wait is in the have (ask-at, act), bounded by a bounds key of the state bounds (recovery-wait-minutes, 90), and each ending is read as the table of 560 says. The manifest consumes mesh-recovery's ended and restored and invokes its three verbs.

Builds: design 47 G §296, §297, §554, §555, §560, §630, §631, §632, §633, A-77, A-92, A-98, A-108
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 00:16:31 +01:00
jschoubben cf378f9a25 G §421, §683: the node-engine reads the entry its want names by fingerprint and keeps its entries and wants
The node's own composer reads the catalog half through meshcatalogclient's Entries cache over its own calls of
mesh-catalog's verb entry, and no longer watches or reads the indexer's modules, manifests and verdicts or the
builder's builds: a module's manifest is its entry's, each artifact's reference is made from the digest its want's
part names (artifact-store://<module>/<artifact>/blobs/<digest>, A-50). A want whose entry is asked or not found is
held, its have the phase held with a detail opening entry-asked: or entry-missing: naming the fingerprint. The
entries and wants it composes from are kept in composed-<node> (entry.<digits> and want.<module>), written before
the composition that uses them, read at start before anything is asked, a kept want standing until the wants'
live read answers, and dropped when nothing names them. The shadow keeps its reads (G §628).

The node grant gains the verb entry and the running stream's consumer calls; OverNATS hears the running stream
as the link's session does.

Builds: design 47 G §421, §683, §614, §628, A-47, A-50
2026-11-11 23:55:50 +01:00
jschoubben 2b916ac043 G A-98: the node agent gains Env.Stop, which stops a module's units and containers and removes nothing
Builds: design 47 G §297, A-98

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-11 23:51:09 +01:00
jschoubben 7c7b5e2d68 G A-92: the node-engine's manifest declares the state bounds, a lifetime per key, in its old section
Builds: design 47 G §560, §557, §627, A-77, A-92
(the seat definition: mesh-contracts 2ae0c679304b828a3973dd23ecd19861577d141d)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-11 23:48:16 +01:00
jschoubben 5173f2a31c W13.8: the node-engine fetches a kept build through the build node's kept-artifact, the node taken from the want's kept-on
A want that names a node in kept-on, for an artifact its parts name, is fetched through kept-artifact of that node's build seat and
read as an object on the bus, checked against the digest; with no kept-on it is fetched from the store, and the builder's builds
are read no more (the apply hook, the fetch and the object read from bc8fbc7, its node and fingerprint from the want).

Builds: design 47 G §271, §526, §421 (A-50)
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-11 23:39:36 +01:00
jschoubben 97d3df3e59 G §422, §698: the node-engine serves first-views, first-view-again and entry-again, and its manifest names the verbs it serves
first-views answers {first-views, entries, wants}, entry-again asks one entry again as a recorded hand act and
answers {asked, read}; first-view-again names no view the engine follows (it follows no branch) and refuses in
words. module.json's claim of node-engine lists every verb the engine serves.

Builds: design 47 G §422, §698
2026-11-11 23:27:28 +01:00
jschoubben 7703162c95 G §421, §683: the node-engine reads the entry its want names by fingerprint and keeps its entries and wants
The node's own composer reads the catalog half through meshcatalogclient's Entries cache over its own calls of
mesh-catalog's verb entry, and no longer watches or reads the indexer's modules, manifests and verdicts or the
builder's builds: a module's manifest is its entry's, each artifact's reference is made from the digest its want's
part names (artifact-store://<module>/<artifact>/blobs/<digest>, A-50). A want whose entry is asked or not found is
held, its have the phase held with a detail opening entry-asked: or entry-missing: naming the fingerprint. The
entries and wants it composes from are kept in composed-<node> (entry.<digits> and want.<module>), written before
the composition that uses them, read at start before anything is asked, a kept want standing until the wants'
live read answers, and dropped when nothing names them. The shadow keeps its reads (G §628).

The node grant gains the verb entry and the running stream's consumer calls; OverNATS hears the running stream
as the link's session does.

Builds: design 47 G §421, §683, §614, §628, A-47, A-50
2026-11-11 23:27:28 +01:00
jschoubben 38e13ef030 H-13: the node-engine's seat-list and held-module findings go through mesh-sdk's held writer
The seat list's five-minute finding and the finding of a held module are raised as they happen, so they use the held
writer (seeded at Start, Held() answers what stands) over the contract of the seat mesh-conditions; the keepers keep no
set or sweep of their own, and the writer is started before the seat list reader writes or clears any. The engine's own
finding stream stays its foundation stream mesh-host-findings.

Builds: design 47 G §529, A-29, A-30, A-37, A-38, A-40
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-11 22:37:21 +01:00
jschoubben 189ed10c36 The node-engine's compositions use the catalog key's codec meshcatalogkey, as meshcatalogclient's wrappers go
Builds: design 47 G §670

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:18:40 +01:00
jschoubben 75f5c64646 The node-engine says no capabilities-changed: manifest-reads-changed alone says a node's capabilities, as the contracts' K6 has the seat
The join with mesh-contracts' K6 (7d730a4a) removes the event and its body; H-3's second source of the same fact goes (haveState.capsSaid, capabilitiesToSay, sayCapabilities), the test asserts none is said.

Builds: design 47 G §590

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:13:24 +01:00
jschoubben 5d3b446988 H-8: a want's parts are checked by the digests the composition fetches
A want with parts is applied only when every fetched archive, process and container
digest is among the parts and every part is a fetched digest; otherwise the module
keeps running what it has and its have is held with detail parts-differ naming the
digests. A want with no parts is applied as before.

Builds: design 47 G §524, §379

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 15:47:04 +01:00
jschoubben ce01ec1f68 H-13: the node-engine's findings are written and said by mesh-sdk's finding writer
A finding is written in the node's state findings and followed by finding-written or finding-cleared on the engine's own
stream mesh-host-findings; the engine states every finding it observes at its start (the seat list's finding only while the
reader waits for the list, else not) and when the keeper asks, and answers an ask by a new statement. The node's grant in
the golden file gains the stream's publish, the ask stream's consumer, and the whole of its state findings.

Builds: design 47 G §383, §384, §390, §461, §485, §503

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 14:15:16 +01:00
jschoubben 602ed830e9 H-15: the apply guard and the yields read the node-engine's module from the entry claiming its seat
The module is the one whose catalog entry claims node-engine, else the one this node heard run the build its own
fingerprint names; no name is fixed in the guard or in the compose.

Builds: design 47 G §364, §527, §428

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 14:05:43 +01:00
jschoubben a073a001c5 H-7: the node record states the capability want-parts
Builds: design 47 G §525, §94, §192

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 14:00:21 +01:00
jschoubben 45e2b946c7 H-5: the seats a node passes on are decided from what every node said it runs, by SeatsToPassOn's new input
The node follows the node-engine seat's running events (an ordered read of the last message of each subject, then a durable
consumer), keeps what each node runs by `at`, acts on none until the first read ended, and hands the composer each node's
heard build with the manifest read by the event's digest; compose resolves the passes through catalogue.SeatsToPassOn
(heard, entries) with the trunk's entry for a node that said nothing.

Builds: design 47 G §199, §249, §459, §536, §556

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 13:58:33 +01:00
jschoubben b41a0d59b6 H-14: a module held for its claim is the finding module-held until it is composed or no longer wanted
Builds: design 47 main file (A claim problem holds its module), test 55

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 12:12:55 +01:00
jschoubben f3813a279e H-6: the seat verb process-builds reads the build each process was started with; the engine keeps no callers table
Builds: design 47 G §262, §245, §440, §447

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 12:09:56 +01:00
jschoubben 420087ab14 H-4: running says stopped, with no fingerprint, once the have no longer holds the module
Builds: design 47 G §365, §404

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 12:06:10 +01:00
jschoubben 26581d87ed H-3: the node-engine says composed, running and capabilities-changed on the seat's subjects
Builds: design 47 G §260, §227, §226, §177, §309, §404

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 12:05:18 +01:00
jschoubben 730860b274 H-2: every service and container is started with MESH_FINGERPRINT, the fingerprint of the want it applies
Builds: design 47 G §201, §205, §239

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 12:03:47 +01:00
jschoubben decbce4c54 H-1: the node-engine claims its seat, says its findings on its own stream and hears running
Builds: design 47 G §364, §387, §199, §368, §411, §438

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 12:03:13 +01:00
mesh-tools 14c22b4a45 Merge pull request 'The seat-list finding is read from the node's own state, so a restart clears it (issue 1966)' (#223) from fix/1966-seat-list-finding-from-the-bus into main 2026-11-11 01:34:11 +00:00
jschoubben e64c8ad0ca The node's golden grant reads its own findings, as the grants part composes it (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-host@14c22b4a4586
mesh/check checking e64c8ad0ca62 on 14c22b4a4586
mesh-grants gives the engine the owner's read of its state findings for its own node's keys (EngineFindingsGrants); the golden
grant held only the write, so the engine could not read the finding it clears at the first full read.

Builds: design 47 H-7, ADR 0636 §1 (the finding is cleared at the first full read), ADR 0596 §2 (the owner reads its state), issue 1966

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 02:23:10 +01:00
jschoubben 6304332ad5 The seat-list finding is read from the node's own state, not kept in memory (issue 1966)
A new node-engine process cleared nothing at its first full read of the seat list, because the Waiter remembered in memory whether
it had written the finding; ace and g14 kept the condition for ever. The Waiter now asks the node's state findings whether the
finding stands, clears it at the first full read and writes it at the bound only when it does not stand.

Builds: design 47 H-7, ADR 0636 §1 (the finding is cleared at the first full read), issue 1966

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 02:20:42 +01:00
mesh-tools 783a2d2207 Merge pull request 'The node-engine says its events once through a transport fault, and an unsaid one at the next link (ADR 0668; issue 1966; group fix/1966-call-made-once)' (#222) from fix/1966-call-made-once into main 2026-11-10 07:20:32 +00:00
jschoubben d581acdeda H4: the builder machine of the toolchain tests has the tool runner's seat-settle-delay (issue 1966; mesh-catalog's issue 1748, ADR 0664 §5)
mesh/delivery merged: source.main moved to novox/mesh-host@783a2d22073f
mesh/delivery-group group fix/1966-call-made-once merged: every member is merged
mesh/check checking d581acdedae3 on 783a2d22073f | novox/mesh-host#222 in group fix/1966-call-made-once
Since mesh-catalog 6b8e7fc6 the build modules ask the capability seat-settle-delay of the machine, so on
the stand-in builder without it mesh-build-go and mesh-build-node were left out as unhostable and both
toolchain tests failed against the catalogue's main, on mesh-host's trunk as on this group's head.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 05:57:04 +01:00
jschoubben 9cd03a7430 H3: the row of sayOnce feeds the old JetStream client's own no responders, nats.ErrNoStreamResponse (issue 1966, ADR 0668 §1; review round 1, finding 2)
mesh/delivery closed: a newer head of the same pull request
mesh/delivery-group group fix/1966-call-made-once rejected: composed together: the checker's verdict at the members' heads (c54b617d200d): mesh-delivery-checker
mesh/check failed: mesh-delivery-checker: no published of the tagged version of go-meshconditions of mesh-health-and-re… | 9cd03a7430cc on f50442ac7a2d
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 04:12:04 +01:00
jschoubben 5aa6d39a98 H2: an engine event not said is kept and said at the next link's opening (issue 1966, ADR 0668 §4)
A failed say was only a line in the engine's log; now it is said again when a link opens, beside the
statement made while no link was open (proved by a step of the node key test, with H1's publish).

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 02:57:34 +01:00
jschoubben 4685610c19 H1: the node-engine publishes its seat's events again through a transport fault, under one message id (issue 1966, ADR 0668)
One publish of manifest-reads-changed with a 10 s deadline was the likely loss behind note 155; the
stream takes the id once, so publishing again is safe.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 02:57:34 +01:00
mesh-tools f50442ac7a Merge pull request 'An unread seat list holds what is new, says so, and is read again by the operator's sessions (issue 1966; group fix/1966-seat-list-unread)' (#221) from fix/1966-seat-list-unread into main 2026-11-10 00:44:44 +00:00
jschoubben 5c353cfc7b S3: the read seat-list answers the operator's sessions and the healers, as its seat says
mesh/delivery-group group fix/1966-seat-list-unread merged: every member is merged
mesh/delivery merged: source.main moved to novox/mesh-host@f50442ac7a2d
mesh/check checking 5c353cfc7bf0 on f50442ac7a2d | novox/mesh-host#221 in group fix/1966-seat-list-unread
The node-engine seat's definition names "the operator and the healers" for
both seat-list verbs, and design 47 G keeps it; the read was opened to every
proved caller on a ruling the review withdrew. node-tools stays on both: every
console call is proved as its machine's tool runner (issue 627, issue 1966).

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 01:16:05 +01:00
jschoubben a484d5f960 S5: the whole node reads its seat list after a holder that re-served
The live case of ace, in process against a real bus: the stand-in of the seat
mesh-catalog answers as nothing serves it (no responders) while the node's
reader first asks, then serves; the same reader reads the list within the
bus's retry bound, with no event of the seat and no hand act (issue 1966,
ADR 0636 §8). The test of an unread list now reads the reason that names the
bootstrapped seats.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 00:48:14 +01:00
jschoubben 4af6c274b8 S4: an unread seat list holds a module before its build is read
On ace, a node-engine left on its bootstrapped seat list could not find the
artifact store's seat, refused avahi's archive reference and with it the whole
pass, so the seat-list hold never ran and every module said "reaches no
artifact store". While the list is unread, a placed module whose seats are not
all bootstrapped is now held as it runs before anything of its build is read
(frombus.Scope.Held), its reason names the bootstrapped seats the node holds,
and a pass refused whole says the unread list first (issue 1966).

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 00:40:53 +01:00
jschoubben 7bf7e29a32 S3: the seat-list verbs answer a proved operator's session
seat-list-again answered person.<name> and the healers, but every call through
the mesh MCP server is proved as its machine's tool runner (issue 627), so no
operator's session could make a node read its seat list again. node-tools now
stands for that session; the read seat-list is open to every proved caller, as
ADR 0636 §7 limits only the read again. The verb names come from the contract.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 00:32:44 +01:00
jschoubben 81bc0cc388 S1: keep the bus's "no responders" beside ErrNotAsked on an ask
A request that reached no one is asked again by mesh-sdk's busretry only when
it can see the bus's own error; the node-engine's ask dropped it, so the seat
list's first read, which fell into its holder's re-serve, was never asked
again (issue 1966).

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 00:29:54 +01:00
mesh-tools 9d8647aa0f Merge pull request 'The node-engine leaves an unchanged unit running across its own replacement, and says what changed when it restarts one (issue 1966)' (#220) from fix/1966-engine-restarts-unchanged into main 2026-11-09 22:09:40 +00:00
jschoubben aa649d85a0 The restart line names no part of a value; the new rows join existing tests (issue 1966, review round 1)
mesh/delivery merged: source.main moved to novox/mesh-host@9d8647aa0f56
mesh/check checking aa649d85a065 on 9d8647aa0f56
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-09 22:56:42 +01:00
jschoubben bcd815a6bb A tool runner is no longer restarted by a build of the engine: owned paths name the directory of every build, and a restart names its cause (issue 1966)
The tool runner's unit carries the paths modules own, for the holder of node-files. The engine's own
archive is unpacked into a place named for its build, so every engine replacement changed that word
and restarted the tool runner with nothing of it changed (novox 21:16, 22:25, 22:38 on 2026-11-09).
A resource unpacked into a place named for its build now owns the directory holding every build.
The restart line now names the unit's directive, word and key that differ, never a value.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-09 22:49:20 +01:00
mesh-tools 6e461bd24a Merge pull request 'A claim problem or a refused credential holds its module, not the whole pass (issue 1966, recovery)' (#219) from fix/1966-s1-partial-pass into main 2026-11-09 17:54:55 +00:00
jschoubben ef41fa2e27 The node-engine's live path holds a module whose claim or credential fails (issue 1966, review round 1)
mesh/delivery merged: source.main moved to novox/mesh-host@6e461bd24aa5
mesh/check being carried to ef41fa2e2796 on 6e461bd24aa5 | from 7c483da93a03
compose.CompositionsHolding returns the modules the declaration leaves out for a claim that cannot be held or a refused
credential; frombus composeEach holds each with those words, so the node-engine keeps what it handed of the module (no
stop of a running module), the candidate answer and the check's sandbox name it refused, and the rest composes.
CredentialsNeeded asks nothing for a held module. Resolve and checkClaims move to a test helper (production uses
ResolveHolding and holdClaims); a pass refused anyway names a held module's reason among the others.

Quick fix: Declared.Held read in the composer only. Thorough, chosen: the same held words reach every reader of the
composition (selfcompose, ComposeSent, the candidate answer) through composeEach's one hold path.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-09 13:04:55 +01:00
jschoubben 868ee6beb0 A claim problem or a refused credential holds its module, not the whole pass (issue 1966, S1 and S5)
The node-engine refused the whole pass when two modules claimed a seat held once, or when a consumer's credential was
refused. Now the composer leaves that module out of the declaration and composes the rest: the module is returned held
(Resolved.Held, resolution.ResolveHolding), reaches the declaration as left out through the filter and the declaration
#218 already keep (Rendering.NotPlaced), and Declared.LeftOut names it. The second claimant of a seat is the one held;
its claim is not this node's holding. The rest-of-mesh view and the network use the same, so a node with a held module
stays on the network. Resolve stays strict for callers that cannot leave a module out.

Quick fix: hold only the credential refusal (resolve.go planForNow), about 20 minutes. Thorough fix, chosen: every
claim problem is attributed to its module (claimsOf) and both kinds go through the one left-out path of ADR 0163 rule 6,
in the declaration, the filter and the network, so no per-kind refusal of a pass remains in the composer. Left for the
next step: a declared-and-not-made own secret (NotMadeError) and the Foreseeing-mode refusal still refuse the pass; the
frombus candidate path holds those by stub already.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-09 12:52:54 +01:00
mesh-tools 7c483da93a Merge pull request 'The filter keeps the openings of a module left out of the declaration that still runs (issue 1966, recovery)' (#218) from fix/1966-filter-keeps-openings into main 2026-11-09 09:49:15 +00:00
jschoubben 17a4ac620c A left-out module's openings keep its per-node narrowing (issue 1966, review round 1)
mesh/delivery merged: source.main moved to novox/mesh-host@7c483da93a03
mesh/check checking 17a4ac620c07 on 7c483da93a03
Rules skipped a left-out module's expose and reaches, so its listens
opened at the manifest's default source and an operator's narrowing for
the node was dropped (ADR 0046). Its exposure is now read like any
module's (exposureOf); where it cannot be read for that module, its
listens open no wider than the mesh (noWiderThanTheMesh), never at the
manifest's default.

Left for the thorough fix, composing a left-out module as what runs of
it (Carried, as a held module is): the filter also opens for a left-out
module that never ran on the node, and it opens the new manifest's
listens rather than what the old composition runs.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 10:43:48 +01:00
jschoubben 95f45b1a74 Keep a left-out module's openings in the filter: it still runs (issue 1966)
A module left out of the declaration for a setting its new definition no
longer composes (ADR 0163 rule 6) keeps running what it ran, but
resolutionCompose dropped it from the modules the filter is derived from,
so the filter closed the ports it still serves. On novox, 2026-11-09
07:46-09:24 UTC, the artifact store's want at build 6e1dfaa8 (listens on
5100, no container publishes 5000) met the node layer's ports
{"5000": 5100}: GivenPorts refused it, distribution was left out, and
5100 closed while the old registry container served there; every build
failed "dial tcp 10.10.0.1:5100: i/o timeout".

The filter is now derived from the composed modules and the left-out
ones; a left-out module's settings (exposure) are still not read.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 10:36:55 +01:00
mesh-tools 2019122379 Merge pull request 'Gate record of mesh-host's forge-made merges (issue 1966, recovery)' (#217) from fix/1966-forge-made-merge into main 2026-11-09 08:45:52 +00:00
jschoubben e504c12940 Record this repository's forge-made merges in the delivery gate, so the indexer takes them in (issue 1966, recovery)
mesh/delivery merged: source.main moved to novox/mesh-host@20191223794a
mesh/delivery-group group fix/1966-forge-made-merge merged: every member is merged
mesh/check checking e504c129401b on 20191223794a | novox/mesh-host#217 in group fix/1966-forge-made-merge
Merges made on the forge directly since 2026-11-09 07:04 UTC have no gate merge record,
and the indexer holds them; an empty merge through the gate covers this repository's head.
Second of its kind: the group's mesh-delivery member removes the need (the forge bridge
reads a forge-made merge's pull request from the forge).

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 09:17:02 +01:00
mesh-tools 25d677d20c Merge pull request 'The node-engine says why it composes a wanted module at no composition, and hands again one its agent lacks (issue 1966, recovery)' (#216) from fix/1966-views-page-time into main 2026-11-09 08:13:14 +00:00
jschoubben 6bde46cb00 The wait-for-the-composer step settles a want again without a real sleep and counts only ghost's own tells (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-host@25d677d20c93
mesh/delivery-group group fix/1966-views-page-time merged: every member is merged
mesh/check checking 6bde46cb0084 on 20191223794a | novox/mesh-host#216 in group fix/1966-views-page-time
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 09:09:40 +01:00
jschoubben 3ea81943f5 The wait-for-the-composer test settles the want again without a real sleep (issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 09:05:30 +01:00
jschoubben 3dc4307aec The node's words for a want it composes nothing of are said once per reason, and proved on the live paths (issue 1966)
The said-once keys no longer hold where the indexer's picture stands; a module composed from the manifest for want of a done
build is named in Pass.Unmade; the agent's wait is told once per want and again after the composition was there; the two tests
are folded into existing ones.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 09:04:45 +01:00
jschoubben 5f37a43b91 The node says why it composes nothing of a want, and hands again what its agent waits for (issue 1966)
Every pass names each wanted module it made no composition of, with why (Pass.Unmade, the verb compose, one journal line per
change); a want not taken is said; the agent says once per want that it waits for the composer and why, and tells it, so a
composition the composer believed handed is handed again.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 08:58:33 +01:00
mesh-tools 9b55bcb127 Merge pull request 'The node-engine fetches every artifact before it replaces anything, and a failed apply no longer claims the old composition runs (issue 1966, recovery)' (#215) from fix/1966-fetch-before-replace into main 2026-11-09 00:16:25 +00:00
jschoubben 9f6801a6c5 A module whose apply failed is no longer said to run what it ran (issue 1966, recovery)
mesh/delivery merged: source.main moved to novox/mesh-host@9b55bcb12743
On 2026-11-08 a failed apply of distribution's new build had already replaced
its store container, yet the node-engine still held the old build as running,
so the put back to it at 23:47 UTC was judged unchanged and applied nothing.
A failed apply now marks what runs of the module failed, and a want of the
composition that ran before is applied again.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-09 01:15:13 +01:00
jschoubben 22a37b758c Fetch every archive and process of an apply before anything is replaced (issue 1966, recovery)
On 2026-11-08 23:25 UTC the node-engine replaced distribution's store container
and only then fetched distribution's program from the store it had just cut off:
the module was left half applied and the artifact store unreachable from every
other machine. An apply now fetches and checks every archive and process first,
as it already pulls a module's images first; one that cannot be had gates its
module, so none of its resources is replaced and none of its orphans removed.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-09 01:14:00 +01:00
mesh-tools 25aa110920 Merge pull request 'Record forge-made merges in the delivery gate (issue 1966, recovery)' (#214) from fix/1966-gate-record-2 into main 2026-11-08 21:46:07 +00:00
jschoubben 71508cc962 Record the forge-made merge in the delivery gate, so the indexer takes it in (issue 1966, recovery)
mesh/delivery merged: source.main moved to novox/mesh-host@25aa110920cd
mesh/delivery-group group fix/1966-gate-record-2 merged: every member is merged
mesh/check checking 71508cc96297 on 25aa110920cd | novox/mesh-host#214 in group fix/1966-gate-record-2
mesh-host #213 (87bdc95a) was merged on the forge under recovery-fixes-merge-and-tell and has no gate merge record; the indexer holds its take-in. An empty merge through the gate covers it, and the forge-made wait is dropped (indexer_drop_pending).

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 19:23:06 +01:00
mesh-tools 87bdc95a3d Merge pull request 'The node-engine serves its seat's verbs from the bootstrapped definition until it holds the seat list (issue 1966, recovery)' (#213) from fix/1966-engine-seat-verbs-bootstrap into main 2026-11-08 17:34:51 +00:00
jschoubben 7b42c94a4d The node-engine serves its seat's verbs from the bootstrapped definition until it holds the seat list (issue 1966, recovery)
mesh/delivery merged: source.main moved to novox/mesh-host@87bdc95a3d65
mesh/check pass 7b42c94a4d78 on 87bdc95a3d65 | unit 1679 run, 333 kept, 31 skipped | integration 1692 run, 333 kept, 26 skipped | over bound: unit (sl…
The link serves the seat node-engine when it opens, before the node agent
starts the reader of the seat list; since ADR 0635 catalogue holds no seat
until then, so the engine registered no verb at all. Nobody answered
manifest-reads, compose, composition or the hand acts on novox, shanks and
now ace, and the assigner kept ace's old statement, holding its own
successor on seats the new engine states.

Quick and thorough fix are one here: the only seat-list read at start is
this one (the composer waits for the list, the reads statement restates).

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 18:34:20 +01:00
mesh-tools 6dd8e08a84 Merge pull request 'The node-engine passes seats with the SDK's one rule, by the build its node runs, and says the pass (ADR 0660, issue 1966, recovery)' (#212) from fix/1966-seat-pass-shared into main 2026-11-08 14:47:22 +00:00
jschoubben 60f413f9bc The node-engine reads each assignment by the build its node keeps and says a seat pass in its answer (issue 1966, ADR 0660)
mesh/delivery merged: source.main moved to novox/mesh-host@6dd8e08a84ed
mesh/delivery-group group fix/1966-seat-pass-shared merged: every member is merged
mesh/check checking 60f413f9bc6a on 6dd8e08a84ed | novox/mesh-host#212 in group fix/1966-seat-pass-shared
Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 15:44:00 +01:00
jschoubben 5108c28e22 The node-engine composes seat passes with the SDK's one rule instead of its own copy (issue 1966, ADR 0660)
Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 15:31:32 +01:00
mesh-tools 875b4ee5ce Merge pull request 'The node-engine states the seats of the list it holds, and again on every change (issue 1966, recovery)' (#211) from fix/1966-seat-reads-bootstrap into main 2026-11-08 03:45:15 +00:00
jschoubben 9ab85ef92e The node-engine states the seats of the seat list it holds, again at each change (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-host@875b4ee5ce38
mesh/delivery-group group fix/1966-seat-reads-bootstrap merged: every member is merged
mesh/check failed (2): mesh-delivery-checker: no published of the tagged version of go-sdk of mesh-delivery…; and 1 more | 9ab85ef92e6c on 875b4ee5ce38
manifestReads froze the statement at its first call, before the engine gave
catalogue even its bootstrapped seats (ADR 0635 empties mesh-sdk's table
until the list is read), so ace's engine stated no seat and its own next
build, naming mesh-catalog, was held on it.

The words and forms stay the build's, computed once; the seats are those of
the list held now. The reader of the seat list restates after each change:
the node record carries the new statement at once and the node seat's event
manifest-reads-changed says it (ADR 0618 §4); an unchanged statement writes
and says nothing; the verb manifest-reads answers when it was made.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 02:36:41 +01:00
mesh-tools 89475bb0cf Merge pull request 'The node-engine's bus watch never holds the client back (issue 1966, bootstrap recovery)' (#210) from fix/1966-watch-never-blocks into main 2026-11-07 23:40:48 +00:00
jschoubben 4f1e92160a Round 1: the pin tests merged (test budget), KVWatch's doc says what folds
mesh/delivery merged: source.main moved to novox/mesh-host@89475bb0cfec
The review of fe9533c1: a new test function raised a count already over the
budget, so two broker-pin tests are one; KVWatch no longer yields every
change, and its doc says so (issue 1966).

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 00:38:36 +01:00
jschoubben fe9533c186 The node-engine's bus watch never holds the client back: it keeps the newest update per key while its reader is busy (issue 1966)
Live fault 2026-11-08: node-engines on two machines logged only "nats: slow consumer, messages dropped" and applied nothing. The
client's watch sends into its channel from inside the subscription's callback; KVWatch forwarded that channel to its reader
blocking, and the reader (composing for minutes) stopped draining, so the subscription's pending limit overflowed.

Quick fix: a larger buffer on the watch's channel; it only moves the limit. Thorough fix (chosen): the forwarder always drains the
client's channel and, while its reader is busy, coalesces the pending updates by key (newest value and revision, first-seen
order, Caught kept in its place), so no reader's pace can back the client up, whatever the burst. A slow consumer error now
names its subscription and what it holds pending.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 00:26:33 +01:00
mesh-tools 70e70eda5a Merge pull request 'Registry milestone: mesh-host (issue 1966)' (#209) from feat/registry-milestone into main 2026-11-07 14:19:45 +00:00
jschoubben cb7a444a56 The check's composer says a node's refusal in the words of its composition with the lifted modules held (issue 1921, milestone 47/G)
mesh/delivery merged: source.main moved to novox/mesh-host@70e70eda5a24
mesh/delivery-group group feat/registry-milestone merged: every member is merged
mesh/check checking cb7a444a5634 on 70e70eda5a24 | novox/mesh-host#209 in group feat/registry-milestone
When a node refused also with the modules composed without their build held as its node-engine holds them, the
verdict kept the words of the composition from their unbuilt manifests, whose refusals the lift itself made (a tool
runner with no bundle, a resource whose artifact or template no build resolved). In the group check of c5d32074c945
that hid lab's real refusal on shanks among them. The node's refusal is now that of the held composition, named so.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 08:19:55 +01:00
jschoubben 53addd73c3 The check's composer composes a module at its build only where that build was made of the change's manifest (issue 1921, milestone 47/G)
mesh/delivery-group group feat/registry-milestone checking: 7 of 7 member(s) ready
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 53addd73c358 on e73aae3686a6 | novox/mesh-host#209 in group feat/registry-milestone
The running indexer could not read 17 of the group's manifests, so the running node-engine composed those modules at
their trunk builds and said nothing; the head composer laid those builds over the change's other manifests and ace
failed on "nothing provides artifact-store, wanted by mesh-build-go" (group check 8a80e8e86858). The input a node
answers a check with now names, per module taken in from its build, the content digest of the manifest that build was
made of (ComposedHow.Manifests); the composer compares it with the case's manifest, and, for an input from a
node-engine before it, compares what the two manifests say beyond what a take-in resolves (the lead's ruling, §17).
Each module lifted either way is composed from the case's manifest, its build awaited, and named in NotJudged with
the comparison that lifted it; one that cannot compose without its build is held as its stub. The digest is taken
over one JSON form because a request through a build module arrives compacted.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 07:24:16 +01:00
jschoubben b6c174a324 A consumer beside a held provider is told what it serves without the names it does not compose, never their placeholders (issue 1921, milestone 47/G)
The provider held on its machine is composed apart, so no module of the composed set serves the provision; its
manifest's served values reached the consumer's binding raw and the binding refused the consumer for a
${bound:route:name-web} it does not require (the group check of 8a80e8e86858 on novox, mesh-build-node beside a held
gitea). Each such key is now left out and said on the need, as a provider elsewhere leaves it out (ADR 0346).

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 07:19:58 +01:00
jschoubben 5b75e866d0 The composition verb says what holds each held module and the build it would be composed at (issue 1921, milestone 47/G H4)
mesh/delivery closed: a newer head of the same pull request
mesh/delivery-group group feat/registry-milestone checking: 6 of 7 member(s) ready
mesh/check failed (4): unit: git.novox.be/novox/mesh-catalog/checks/gitignored.TestNoProgramABuildLeavesIsC…; and 3 more | 5b75e866d079 on e73aae3686a6
The node's own composition named a held module's why only; it now also says whether its manifest's verdict or this
node-engine's decoder holds it, and at which build, as the input a check's composer reads says it. Said in the held
words, so the node-engine seat's answer keeps its shape.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 02:38:18 +01:00
jschoubben d0fb384f00 The check's composer composes a module the running indexer alone holds when the change's manifest is the check's to judge (issue 1921, milestone 47/G H2)
ADR 0578 §2 and §3: a case's accepted verdict, or none (the running checker drops the change's own), lifts the
input's hold on a manifest the composer reads whole; a case's refusal and bytes that do not decode stay held. It is
composed at the build the input carries (Input.Holds). Without that build the node is not judged and the verdict
names the module, why it was held and that its build is awaited; where it cannot compose without it, it is composed
held as its node-engine holds it, and said so. Never a silent pass (the lead's Q4 ruling). The issue 774 row now
gives the runner the refused verdict the checker gives a manifest the change does not bring.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 02:33:21 +01:00
jschoubben c114b786e8 The input a node answers a check with says what holds each held module and carries its build (issue 1921, milestone 47/G H3)
A module the running indexer refused stood in the input as its stub with no build, so a check's composer that
judges the change's manifest itself had nothing to compose it at. Each held module now says whether its verdict
or the decoder holds it; for a check's answer the node also takes in the build it would compose it at, when the
builder holds it done, beside the stub and never into what the node composes.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 02:33:21 +01:00
jschoubben e96a7ada50 The node-engine keeps no copy of a seat's verbs: the seat list comes from the bus (issue 1921, ADR 0635 §1)
Eight lists of seat verbs (checker, enrolment, artifact sweep, delivery, node-resolver, node-nfs-server,
node-mounts, mesh-config's input) and their schema helpers were read by nothing but one test, and drifted
from their seats (B0's verb input never reached the checker's copy). A guard row in the seats test finds
any list of the catalogue's verbs written in the node-engine's code.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 02:26:14 +01:00
jschoubben eec35a7c0c The sandbox composer composes against the seat list its request carries, and judges nothing without one (issue 1966)
The group try of 2026-11-06 10:07 failed its composition on all four
machines because mesh-compose, the node-engine's composer a check builds
at the head, loaded no seat list once the compiled one was deleted (ADR
0635 §1): the uplink's holder no longer stepped back from the resolver
file, and no holder of the registry seats was known. It now takes the
request's seats over the foundation's bootstrapped ones, as the
node-engine takes the bus's; without any it says each machine not judged
(ADR 0636 §1, ADR 0593 §9), never a refusal or pass made without them.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 12:01:00 +01:00
167 changed files with 14159 additions and 1915 deletions
+62 -4
View File
@@ -11,9 +11,12 @@
// checkout or `git archive` of the catalogue repository at a ref: the command fetches nothing). -node adds a machine to // checkout or `git archive` of the catalogue repository at a ref: the command fetches nothing). -node adds a machine to
// compose; -declarations answers each machine's declared paths with their owners and modes. // compose; -declarations answers each machine's declared paths with their owners and modes.
// //
// It opens no network connection, bus, store or file beyond its standard input, the -request file and the -catalogue // Run as a pipeline step it reads its input, a composer.Request at depth compose, from the file named by the environment
// directory, and reads // variable MESH_RUN_INPUT and answers the run's output as its composer.Verdict on the last line of standard output
// no environment. Exit codes: 0 the verdict is written (a machine that does not compose is a verdict, not an error); // (design 47 G A-167).
//
// It opens no network connection, bus, store or file beyond its standard input, the -request file, the file
// MESH_RUN_INPUT names and the -catalogue directory, and reads no other environment. Exit codes: 0 the verdict is written (a machine that does not compose is a verdict, not an error);
// 1 the request cannot be read or run, said on standard error; 2 the flags are wrong. // 1 the request cannot be read or run, said on standard error; 2 the flags are wrong.
package main package main
@@ -23,11 +26,17 @@ import (
"flag" "flag"
"fmt" "fmt"
"io" "io"
"maps"
"os" "os"
"path/filepath" "path/filepath"
"slices"
"sort" "sort"
"strings" "strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/seatdef"
"git.novox.be/novox/mesh-sdk/go/seatlist"
"github.com/novox/mesh-host/internal/composer" "github.com/novox/mesh-host/internal/composer"
) )
@@ -59,6 +68,11 @@ func run(args []string, stdin io.Reader, stdout, stderr io.Writer) int {
} }
var raw []byte var raw []byte
var err error var err error
// A pipeline step hands its input as a file named by MESH_RUN_INPUT (design 47 G A-167); -request, when given, wins.
runInput := os.Getenv("MESH_RUN_INPUT")
if *requestFile == "" && runInput != "" {
*requestFile = runInput
}
if *requestFile != "" { if *requestFile != "" {
raw, err = os.ReadFile(*requestFile) raw, err = os.ReadFile(*requestFile)
} else { } else {
@@ -105,7 +119,7 @@ func run(args []string, stdin io.Reader, stdout, stderr io.Writer) int {
if *declarations { if *declarations {
req.Declarations = true req.Declarations = true
} }
v, err := composer.Run(req) v, err := runWithSeats(req)
if err != nil { if err != nil {
fmt.Fprintln(stderr, "mesh-compose:", err) fmt.Fprintln(stderr, "mesh-compose:", err)
return 1 return 1
@@ -118,6 +132,50 @@ func run(args []string, stdin io.Reader, stdout, stderr io.Writer) int {
return 0 return 0
} }
// NoSeatList is what a machine reads, not judged, when the request carries no seat list (novox/hq ADR 0636 §1).
const NoSeatList = "the composer was handed no seat list, so it composes nothing (a reader of the seat list acts on nothing " +
"new until it has read the list, novox/hq ADR 0636 §1): the asker gives the seat definitions it judges against in the request's seats"
// runWithSeats composes req against the seat list the request carries (Request.Seats: the definitions the asker's seat list
// reader holds, the members' heads' laid over them in a check, ADR 0636 §4), over the foundation's bootstrapped ones
// (seatlist.Bootstrapped), as the node-engine composes against the list it read from the bus (ADR 0635 §1). The command
// compiles no seat list: without one in the request it reads every manifest and judges no machine, each said not judged
// with NoSeatList (ADR 0593 §9), never a refusal or a pass made without the seats' paths, deliveries and holders.
func runWithSeats(req composer.Request) (composer.Verdict, error) {
if len(req.Seats) == 0 {
machines := req.Machines
req.Machines = nil
v, err := composer.Run(req)
if err != nil || len(machines) == 0 {
return v, err
}
for i := range v.Cases {
v.Cases[i].Machines = map[string]composer.Machine{}
for _, node := range machines {
// Blamed is always written, blaming none here (design 47 G §721, A-469h).
v.Cases[i].Machines[node] = composer.Machine{NotJudged: NoSeatList, Blamed: []struct {
Module string `json:"module"`
Problem string `json:"problem"`
}{}}
}
}
return v, nil
}
defs := map[string]seatdef.Definition{}
for _, d := range seatlist.Bootstrapped() {
defs[d.Seat] = d
}
for _, raw := range req.Seats {
d, err := seatdef.Parse(raw)
if err != nil {
return composer.Verdict{}, fmt.Errorf("a seat definition of the request: %w", err)
}
defs[d.Seat] = d
}
catalogue.UseSeats(catalogue.SeatsFromDefinitions(slices.Collect(maps.Values(defs))))
return composer.Run(req)
}
// readCatalogue reads every module.json of a catalogue checkout by its module's name: the root's and each of // readCatalogue reads every module.json of a catalogue checkout by its module's name: the root's and each of
// modules/*/. A manifest whose name cannot be read is kept under its directory's name, so the reader's refusal is said. // modules/*/. A manifest whose name cannot be read is kept under its directory's name, so the reader's refusal is said.
func readCatalogue(dir string) (map[string]composer.Manifest, error) { func readCatalogue(dir string) (map[string]composer.Manifest, error) {
+176 -9
View File
@@ -3,8 +3,8 @@ package main
import ( import (
"bytes" "bytes"
"encoding/json" "encoding/json"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"github.com/novox/mesh-host/internal/buckets/bucketstest" "github.com/novox/mesh-host/internal/buckets/bucketstest"
"go/build" "go/build"
"os" "os"
@@ -17,6 +17,7 @@ import (
"testing" "testing"
"git.novox.be/novox/mesh-sdk/go/catalogue" "git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/seatlist/seatlisttest"
"github.com/novox/mesh-host/internal/beside" "github.com/novox/mesh-host/internal/beside"
"github.com/novox/mesh-host/internal/compose" "github.com/novox/mesh-host/internal/compose"
"github.com/novox/mesh-host/internal/composer" "github.com/novox/mesh-host/internal/composer"
@@ -49,11 +50,25 @@ func meshOf() compose.Input {
func request(t *testing.T, depth string) composer.Request { func request(t *testing.T, depth string) composer.Request {
t.Helper() t.Helper()
r := composer.Request{V: composer.Version, Depth: depth, HoldingsFromClaims: true, r := composer.Request{V: composer.Version, Depth: depth, HoldingsFromClaims: true,
Machines: []string{"anchor", "laptop", "ace"}, Machines: []string{"anchor", "laptop", "ace"}, Seats: seats(t),
Cases: []composer.Case{{Name: "trunk", Manifests: captured(t)}}} Cases: []composer.Case{{Name: "trunk", Manifests: captured(t)}}}
return withInput(t, r, meshOf()) return withInput(t, r, meshOf())
} }
// seats is the seat list a request carries, as the checker hands the one its reader holds (ADR 0635 §1, ADR 0636 §4).
func seats(t *testing.T) []json.RawMessage {
t.Helper()
var out []json.RawMessage
for _, d := range seatlisttest.Definitions() {
raw, err := json.Marshal(d)
if err != nil {
t.Fatal(err)
}
out = append(out, raw)
}
return out
}
// withInput is r with in as its input, as a caller puts it on the wire: the request's shape carries the input as JSON. // withInput is r with in as its input, as a caller puts it on the wire: the request's shape carries the input as JSON.
func withInput(t *testing.T, r composer.Request, in compose.Input) composer.Request { func withInput(t *testing.T, r composer.Request, in compose.Input) composer.Request {
t.Helper() t.Helper()
@@ -124,6 +139,134 @@ func TestTheCommandWritesOneVerdictPerManifestAndMachine(t *testing.T) {
t.Errorf("%s does not compose and says no reason", name) t.Errorf("%s does not compose and says no reason", name)
} }
} }
// Issue 1966, the group try of 2026-11-06 10:07: the composer judges against the seat list its request carries, the
// node-engine's from the bus (ADR 0635 §1). The uplink's holder steps back from the resolver file for the resolver's
// holder by the seat's definition; without the list nothing is judged (ADR 0636 §1), never "both declare the path".
resolverMachine := func(withSeats bool) composer.Machine {
r := composer.Request{V: composer.Version, Depth: composer.DepthCompose, HoldingsFromClaims: true,
Machines: []string{"laptop"}, Cases: []composer.Case{{Name: "trunk", Manifests: captured(t)}}}
if withSeats {
r.Seats = seats(t)
}
r = withInput(t, r, compose.Input{Nodes: []compose.Node{{Name: "laptop",
Assigned: []string{"networkmanager", "systemd-resolved", "pacman", "systemd", "dnsmasq"},
Capabilities: map[string]bool{"file-references": true, "privileged": true, "service-manager": true, "package-manager": true, "uplink-networkmanager": true}}}})
code, v, stderr := runJSON(t, nil, encode(t, r))
if code != 0 || len(v.Cases) != 1 {
t.Fatalf("exit %d: %s", code, stderr)
}
return v.Cases[0].Machines["laptop"]
}
if m := resolverMachine(true); strings.Contains(m.Reason+strings.Join(m.Problems, " "), "both declare") || m.NotJudged != "" {
t.Errorf("with the seat list, the uplink's and the resolver's holders on one machine: %+v, want the uplink's stepping back", m)
}
if m := resolverMachine(false); m.Composes || m.NotJudged != NoSeatList || m.Reason != "" || m.Blamed == nil {
t.Errorf("without a seat list: %+v, want not judged, saying why, blaming none", m)
}
// Design 47 G §721, §733 (the node-engine's composer): a machine's verdict names the modules it blames, each with its
// problem, and always writes blamed (A-469h). On a node composed as its node-engine composes it (ComposedHow), blamed is
// the modules the change refuses that the node held neither in its read nor composing without the change (A-469j).
file := func(module, path, content string) string {
return `{"module": "` + module + `", "version": "1", "resources": [{"id": "f", "type": "file", "path": "` + path +
`", "mode": "0600", "content": "` + content + `"}]}`
}
plain := func(module string) string { return `{"module": "` + module + `", "version": "1"}` }
secret := `N=${secret:repository}\\n`
search := map[string]string{"m1": plain("m1"), "m2": file("m2", "/var/lib/m2/env", secret), "m3": plain("m3")}
for _, row := range []struct {
name string
manifests map[string]string
assigned []string
how *compose.ComposedHow
tries int // the request's blame_tries
blamed []blame // each module blamed, its problem holding the words given
}{
// G-24 H-AC1: a fixture module whose manifest is refused on the machine is blamed by name.
{"a module the change refuses is blamed by name", map[string]string{"app": file("app", "/var/lib/app/env", `N=${secret:repository}\\n`)},
[]string{"app"}, &compose.ComposedHow{Node: "a", Builds: map[string]string{}}, 20, []blame{{Module: "app", Problem: "repository"}}},
// Held composing without the change (HeldBefore): left out of blamed.
{"a module held before the change is not blamed", map[string]string{"app": file("app", "/var/lib/app/env", `N=${secret:repository}\\n`)},
[]string{"app"}, &compose.ComposedHow{Node: "a", Builds: map[string]string{}, HeldBefore: map[string]string{"app": "held"}}, 0, []blame{}},
// G-24 H-AC2 (A-468l): two modules owning one path are both blamed, with that path.
{"two modules owning one path are both blamed", map[string]string{"one": file("one", "/etc/shared.conf", "x"), "two": file("two", "/etc/shared.conf", "y")},
[]string{"one", "two"}, &compose.ComposedHow{Node: "a", Builds: map[string]string{}}, 0,
[]blame{{Module: "one", Problem: `one and two both declare the path "/etc/shared.conf"`}, {Module: "two", Problem: `one and two both declare the path "/etc/shared.conf"`}}},
// The same, on a node whose input says not how it composes: nothing is held, both owners are blamed from the refusal.
{"two path owners, composed without how the node composes", map[string]string{"one": file("one", "/etc/shared.conf", "x"), "two": file("two", "/etc/shared.conf", "y")},
[]string{"one", "two"}, nil, 20,
[]blame{{Module: "one", Problem: `one and two both declare the path "/etc/shared.conf"`}, {Module: "two", Problem: `one and two both declare the path "/etc/shared.conf"`}}},
// G-24 H-AC5: a machine that fails with no module held is searched over the changed modules placed on it (here every
// one, the input naming no manifest its builds were made of, A-469k), in assignment order; the search ends at the
// first whose hold ends the problem, so m3 is never tried.
{"the search ends at the first module whose hold ends the problem", search, []string{"m1", "m2", "m3"}, &compose.ComposedHow{Node: "a"}, 20,
[]blame{{Module: "m2", Problem: "repository"}}},
// A module whose case manifest is the one its build was made of is not changed (A-469k), so never held: m2's
// problem is then no changed module's, and none is blamed.
{"a module its build was made of is not tried", search, []string{"m1", "m2", "m3"},
&compose.ComposedHow{Node: "a", Manifests: map[string]string{"m2": digest(t, search["m2"])}}, 20, []blame{}},
// A-470v: a hold that changes the problem but leaves the machine not composing keeps that module held and blamed,
// and the search goes on against the retry's problem: both modules the change refuses are blamed.
{"every module the change refuses is blamed", map[string]string{"m1": file("m1", "/var/lib/m1/env", secret), "m2": file("m2", "/var/lib/m2/env", secret), "m3": plain("m3")},
[]string{"m1", "m2", "m3"}, &compose.ComposedHow{Node: "a"}, 20, []blame{{Module: "m1", Problem: "repository"}, {Module: "m2", Problem: "repository"}}},
// G-24 H-AC4: past the request's blame_tries the modules not yet tried are named so.
{"past blame_tries the untried are named", search, []string{"m1", "m2", "m3"}, &compose.ComposedHow{Node: "a"}, 1,
[]blame{{Module: "m2", Problem: composer.NotTried}, {Module: "m3", Problem: composer.NotTried}}},
// A-468i: a request with no blame_tries gets no search; the machine fails on the node's own account (H-AC3).
{"a request with no blame_tries is not searched", search, []string{"m1", "m2", "m3"}, &compose.ComposedHow{Node: "a"}, 0, []blame{}},
// A-469j: one search, the composer's, inside the node-engine's composition (ComposeSent): an input that says not how
// its node composes is composed without one, so its failure is the node's own.
{"a node that says not how it composes is not searched", search, []string{"m1", "m2", "m3"}, nil, 20, []blame{}},
} {
t.Run(row.name, func(t *testing.T) {
r := composer.Request{V: composer.Version, Depth: composer.DepthCompose, Machines: []string{"a"}, Seats: seats(t), BlameTries: row.tries,
Cases: []composer.Case{{Name: "change", Manifests: map[string]composer.Manifest{}}}}
for name, raw := range row.manifests {
r.Cases[0].Manifests[name] = composer.Manifest{Raw: json.RawMessage(raw)}
}
r = withInput(t, r, compose.Input{Nodes: []compose.Node{{Name: "a", Assigned: row.assigned}}, ComposedHow: row.how})
var out, errOut bytes.Buffer
if code := run(nil, bytes.NewReader(encode(t, r)), &out, &errOut); code != 0 {
t.Fatalf("exit %d: %s", code, errOut.String())
}
if !strings.Contains(out.String(), `"blamed":`) {
t.Errorf("the verdict writes no blamed: %s", out.String())
}
var v composer.Verdict
if err := json.Unmarshal(out.Bytes(), &v); err != nil {
t.Fatal(err)
}
m := v.Cases[0].Machines["a"]
if held := row.how != nil && row.how.HeldBefore != nil; m.Composes != held {
t.Errorf("composes = %v, want %v: %+v", m.Composes, held, m)
}
if len(m.Blamed) != len(row.blamed) {
t.Fatalf("blamed %+v, want %+v (%+v)", m.Blamed, row.blamed, m)
}
for i, b := range row.blamed {
if m.Blamed[i].Module != b.Module || !strings.Contains(m.Blamed[i].Problem, b.Problem) {
t.Errorf("blamed %+v, want %+v", m.Blamed, row.blamed)
}
}
})
}
}
// digest is the content digest of a manifest (compose.ContentDigest).
func digest(t *testing.T, raw string) string {
t.Helper()
d, err := compose.ContentDigest([]byte(raw))
if err != nil {
t.Fatal(err)
}
return d
}
// blame is one module a machine's verdict blames (composer.Machine.Blamed).
type blame = struct {
Module string `json:"module"`
Problem string `json:"problem"`
} }
// 1621/AC3: over the captured catalogue and a mesh of three machines, the command's verdict equals, machine for // 1621/AC3: over the captured catalogue and a mesh of three machines, the command's verdict equals, machine for
@@ -171,6 +314,9 @@ func TestTheCommandAnswersWhatTheLinkedCompositionAnswers(t *testing.T) {
} }
} }
} }
// G-24 H-AC3 (§721): a machine that does not compose and blames no module (ace: its own sealing key) fails on the
// node's own account, blamed written empty.
want.Blamed = []blame{}
if !reflect.DeepEqual(got, want) { if !reflect.DeepEqual(got, want) {
t.Errorf("%s: %s: command %+v, linked %+v", depth, node, got, want) t.Errorf("%s: %s: command %+v, linked %+v", depth, node, got, want)
} }
@@ -196,7 +342,7 @@ func TestACatalogueCheckoutIsComposedForANode(t *testing.T) {
if err := os.WriteFile(filepath.Join(dir, "modules", "demo", "module.json"), []byte(manifest), 0o644); err != nil { if err := os.WriteFile(filepath.Join(dir, "modules", "demo", "module.json"), []byte(manifest), 0o644); err != nil {
t.Fatal(err) t.Fatal(err)
} }
in := encode(t, withInput(t, composer.Request{V: composer.Version}, compose.Input{Nodes: []compose.Node{{Name: "laptop", Assigned: []string{"demo"}}}})) in := encode(t, withInput(t, composer.Request{V: composer.Version, Seats: seats(t)}, compose.Input{Nodes: []compose.Node{{Name: "laptop", Assigned: []string{"demo"}}}}))
code, v, stderr := runJSON(t, []string{"-catalogue", dir, "-node", "laptop", "-declarations"}, in) code, v, stderr := runJSON(t, []string{"-catalogue", dir, "-node", "laptop", "-declarations"}, in)
if code != 0 { if code != 0 {
t.Fatalf("exit %d: %s", code, stderr) t.Fatalf("exit %d: %s", code, stderr)
@@ -209,8 +355,8 @@ func TestACatalogueCheckoutIsComposedForANode(t *testing.T) {
} }
} }
// A run step hands the command no standard input: the request is read from the file -request names, and a file that is // A run step hands the command no standard input: the request is read from the file -request names, or from the one
// not there is a request it cannot read. // MESH_RUN_INPUT names, and a file that is not there is a request it cannot read.
func TestTheRequestIsReadFromAFile(t *testing.T) { func TestTheRequestIsReadFromAFile(t *testing.T) {
file := filepath.Join(t.TempDir(), "request.json") file := filepath.Join(t.TempDir(), "request.json")
r := composer.Request{V: composer.Version, Cases: []composer.Case{{Name: "only", Manifests: map[string]composer.Manifest{ r := composer.Request{V: composer.Version, Cases: []composer.Case{{Name: "only", Manifests: map[string]composer.Manifest{
@@ -225,6 +371,27 @@ func TestTheRequestIsReadFromAFile(t *testing.T) {
if code, _, stderr := runJSON(t, []string{"-request", file + ".gone"}, nil); code != 1 || !strings.Contains(stderr, "cannot be read") { if code, _, stderr := runJSON(t, []string{"-request", file + ".gone"}, nil); code != 1 || !strings.Contains(stderr, "cannot be read") {
t.Errorf("a request file that is not there: exit %d, %q", code, stderr) t.Errorf("a request file that is not there: exit %d, %q", code, stderr)
} }
// A pipeline step names its input file in MESH_RUN_INPUT: the command reads the request there and the verdict is the
// last line of standard output (design 47 G A-167).
r.Depth = composer.DepthCompose
if err := os.WriteFile(file, encode(t, r), 0o600); err != nil {
t.Fatal(err)
}
t.Setenv("MESH_RUN_INPUT", file)
var out, errOut bytes.Buffer
if code := run(nil, bytes.NewReader(nil), &out, &errOut); code != 0 {
t.Fatalf("exit %d: %s", code, errOut.String())
}
lines := strings.Split(strings.TrimRight(out.String(), "\n"), "\n")
v = composer.Verdict{}
if err := json.Unmarshal([]byte(lines[len(lines)-1]), &v); err != nil || len(v.Cases) != 1 || !v.Cases[0].Manifests["demo"].Accepted {
t.Fatalf("last line is no verdict: %v\n%s", err, out.String())
}
t.Setenv("MESH_RUN_INPUT", file+".gone")
if code := run(nil, bytes.NewReader(nil), &out, &errOut); code != 1 || !strings.Contains(errOut.String(), "cannot be read") {
t.Errorf("a missing input file: exit %d, %q", code, errOut.String())
}
} }
// The exit codes: 1 for a request that cannot be read or run, 2 for wrong flags, each said on standard error. // The exit codes: 1 for a request that cannot be read or run, 2 for wrong flags, each said on standard error.
@@ -282,7 +449,7 @@ func ownPackages(t *testing.T) map[string][]string {
} }
// 1621/AC2 and AC4: the command imports, through every package of this repository it reaches, only the standard // 1621/AC2 and AC4: the command imports, through every package of this repository it reaches, only the standard
// library, mesh-sdk, the contracts modules' pure packages (seats' types, clients' codecs), the delivery SDK's composer (the command's own request and verdict types), the builder's builds package (a build's value and its keys), the private network's overlay package (mesh-wireguard's own pure package: names, graph and // library, mesh-sdk, the contracts modules' pure packages (seats' types, clients' codecs), the delivery SDK's composer (the command's own request and verdict types) and catalog key codec (a catalog entry's key, design 47 G A-144), the builder's builds package (a build's value and its keys), the private network's overlay package (mesh-wireguard's own pure package: names, graph and
// generator, no bus) and this repository's own packages, and none that reaches a bus, a store, a network service or // generator, no bus) and this repository's own packages, and none that reaches a bus, a store, a network service or
// another process. // another process.
func TestTheCommandImportsNoBusStoreNetworkOrProcess(t *testing.T) { func TestTheCommandImportsNoBusStoreNetworkOrProcess(t *testing.T) {
@@ -294,7 +461,7 @@ func TestTheCommandImportsNoBusStoreNetworkOrProcess(t *testing.T) {
for path, imports := range own { for path, imports := range own {
for _, imp := range imports { for _, imp := range imports {
std := !strings.Contains(strings.SplitN(imp, "/", 2)[0], ".") std := !strings.Contains(strings.SplitN(imp, "/", 2)[0], ".")
if !std && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-networking/modules/mesh-wireguard/go/overlay") && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-sdk/go") && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-contracts/") && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-delivery/contracts/") && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-delivery/sdk/composer") && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds") && !strings.HasPrefix(imp, "github.com/novox/mesh-host/") { if !std && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-networking/modules/mesh-wireguard/go/overlay") && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-sdk/go") && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-contracts/") && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/") && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk/composer") && imp != "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk/catalogkey" && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds") && !strings.HasPrefix(imp, "github.com/novox/mesh-host/") {
t.Errorf("%s imports %s, which is neither the standard library, mesh-sdk nor this repository", path, imp) t.Errorf("%s imports %s, which is neither the standard library, mesh-sdk nor this repository", path, imp)
} }
for _, f := range forbidden { for _, f := range forbidden {
+20
View File
@@ -1,12 +1,16 @@
package main package main
import ( import (
"encoding/json"
"errors" "errors"
"fmt" "fmt"
"os" "os"
"path/filepath" "path/filepath"
"regexp" "regexp"
"strings" "strings"
"github.com/novox/mesh-host/internal/agent"
"github.com/novox/mesh-host/internal/compositions"
) )
// appliedStore keeps every composition the agent applied, one file per module (the node's own as `node.json`), in the // appliedStore keeps every composition the agent applied, one file per module (the node's own as `node.json`), in the
@@ -105,3 +109,19 @@ func (s appliedStore) Drop(module string) error {
} }
return nil return nil
} }
// ownModuleApplied is the module of the composition the store holds at the fingerprint fp, the engine's own build's
// (design 47 G §527, A-149): "" when fp is none or no record holds it.
func ownModuleApplied(s appliedStore, fp string) string {
if fp == "" {
return ""
}
all, _ := s.Load()
for module, raw := range all {
var c compositions.Composed
if module != agent.NodeModule && json.Unmarshal(raw, &c) == nil && c.FP == fp && c.Module != "" {
return c.Module
}
}
return ""
}
+75 -10
View File
@@ -6,6 +6,7 @@ import (
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
"log"
"os" "os"
"path/filepath" "path/filepath"
"strings" "strings"
@@ -17,7 +18,9 @@ import (
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine" "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
occ "git.novox.be/novox/mesh-contracts/operator-and-conversation/go/clients/operatorchannelclient" occ "git.novox.be/novox/mesh-contracts/operator-and-conversation/go/clients/operatorchannelclient"
stdio "git.novox.be/novox/mesh-sdk/go" stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/buscalls"
"github.com/novox/mesh-host/internal/accounts" "github.com/novox/mesh-host/internal/accounts"
"github.com/novox/mesh-host/internal/agent"
"github.com/novox/mesh-host/internal/apply" "github.com/novox/mesh-host/internal/apply"
"github.com/novox/mesh-host/internal/link" "github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/selfcompose" "github.com/novox/mesh-host/internal/selfcompose"
@@ -284,7 +287,7 @@ const compositionReadFor = 20 * time.Second
// compositionFrom answers the verb composition from the node agent that runs now (issue 774); refused in words while // compositionFrom answers the verb composition from the node agent that runs now (issue 774); refused in words while
// none does. // none does.
func compositionFrom(current *agentNow) func(module, digest string, diff bool) (any, error) { func compositionFrom(current *agentNow, withheld func(ctx context.Context, module string) ([]link.WithheldSubject, string)) func(module, digest string, diff bool) (any, error) {
return func(module, digest string, diff bool) (any, error) { return func(module, digest string, diff bool) (any, error) {
ag := current.get() ag := current.get()
if ag == nil { if ag == nil {
@@ -292,7 +295,17 @@ func compositionFrom(current *agentNow) func(module, digest string, diff bool) (
} }
ctx, cancel := context.WithTimeout(context.Background(), compositionReadFor) ctx, cancel := context.WithTimeout(context.Background(), compositionReadFor)
defer cancel() defer cancel()
return ag.Composition(ctx, module, digest, diff) view, err := ag.Composition(ctx, module, digest, diff)
if err != nil {
return view, err
}
// And what the grants part withholds of it here, through its verb withheld (design 46 approval mark §6).
subjects, unknown := withheld(ctx, module)
for _, s := range subjects {
view.Withheld = append(view.Withheld, agent.WithheldView{Subject: s.Subject, Why: s.Why})
}
view.WithheldUnknown = unknown
return view, nil
} }
} }
@@ -367,9 +380,9 @@ func packagesFrom(node string, session *link.Session, composer *atomic.Pointer[s
// nec.AnswerBound, selfcompose candidateWithin). The answer is // nec.AnswerBound, selfcompose candidateWithin). The answer is
// the seat's shape (nodeengine.ComposeCandidateAnswer), checked as it is encoded; a node whose shadow does not run yet // the seat's shape (nodeengine.ComposeCandidateAnswer), checked as it is encoded; a node whose shadow does not run yet
// answers unready, which the checker takes as no answer. // answers unready, which the checker takes as no answer.
func composeCandidateFrom(node string, shadow *atomic.Pointer[selfcompose.Shadow]) func(candidate string, input bool) (any, error) { func composeCandidateFrom(node string, shadow *atomic.Pointer[selfcompose.Shadow]) func(candidate string, input bool, call string, line buscalls.ChunkLine) (any, error) {
return func(candidate string, input bool) (any, error) { return func(candidate string, input bool, call string, line buscalls.ChunkLine) (any, error) {
ask := nodeengine.ComposeCandidateInput{Candidate: candidate, Input: input} ask := nodeengine.ComposeCandidateInput{Candidate: candidate, Input: input, Call: call}
if _, err := nec.EncodeAsk(ask); err != nil { if _, err := nec.EncodeAsk(ask); err != nil {
return nil, err return nil, err
} }
@@ -383,10 +396,62 @@ func composeCandidateFrom(node string, shadow *atomic.Pointer[selfcompose.Shadow
defer cancel() defer cancel()
a = s.ComposeCandidate(ctx, ask) a = s.ComposeCandidate(ctx, ask)
} }
raw, err := nec.EncodeAnswer(a) return answerComposeCandidate(a, candidate, input, call, line)
if err != nil {
return nil, fmt.Errorf("this node's answer to %s does not fit the seat's shape: %w", candidate, err)
}
return json.RawMessage(raw), nil
} }
} }
// firstViewsFrom answers the seat's verb first-views from the composer of this run (design 47 G §698): the views the node-engine
// follows (none), the entries it keeps and the wants it composes from.
func firstViewsFrom(composer *atomic.Pointer[selfcompose.Composer]) func() (any, error) {
return func() (any, error) {
c := composer.Load()
if c == nil {
return nil, errors.New("this node composes none of its own modules now (it applies the controller's compositions, or its node agent is starting), so it keeps no entry")
}
views, entries, wants := c.FirstViews()
return map[string]any{"first-views": views, "entries": entries, "wants": wants}, nil
}
}
// entryAgainFrom asks one entry of the catalog half again, for the seat's verb entry-again (design 47 G §698).
func entryAgainFrom(composer *atomic.Pointer[selfcompose.Composer]) func(ctx context.Context, fingerprint string) error {
return func(ctx context.Context, fingerprint string) error {
c := composer.Load()
if c == nil {
return errors.New("this node composes none of its own modules now, so it keeps no entry to ask again")
}
return c.EntryAgain(ctx, fingerprint)
}
}
// answerComposeCandidate encodes the answer of compose-candidate. The verb is marked chunks (A-237b, A-237c): asked with input and a
// call token, the composition input goes in chunks over the link's connection, whatever its size, and the answer names them;
// asked with no token (the checker's, until G removes it, G-6 §321, A-237c) it stays inline in the answer.
func answerComposeCandidate(a nodeengine.ComposeCandidateAnswer, candidate string, input bool, call string, line buscalls.ChunkLine) (any, error) {
var chunked []byte
if input && call != "" && a.Input != nil {
if line == nil {
return nil, errors.New("this engine has no bus connection to send the composition input on")
}
body, err := json.Marshal(a.Input)
if err != nil {
return nil, fmt.Errorf("the composition input of %s cannot be encoded: %w", candidate, err)
}
chunked, a.Input = body, nil
a.Chunks = int64((len(body) + buscalls.SeatChunkBytes - 1) / buscalls.SeatChunkBytes)
}
raw, err := nec.EncodeAnswer(a)
if err != nil {
return nil, fmt.Errorf("this node's answer to %s does not fit the seat's shape: %w", candidate, err)
}
if chunked != nil {
send := buscalls.SeatChunks{Seat: nodeengine.Seat, Verb: nodeengine.VerbComposeCandidate, Call: call}
// The answer names the chunks first; the send runs after it and ends at a chunk the caller does not acknowledge.
go func() {
if err := send.Send(line, chunked, buscalls.SeatAckWithin, log.Printf); err != nil {
log.Printf("the send %s of the composition input of %s: %v", call, candidate, err)
}
}()
}
return json.RawMessage(raw), nil
}
+71
View File
@@ -0,0 +1,71 @@
package main
import (
"context"
"encoding/json"
"fmt"
"strings"
"time"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/nodebuildgo"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/nodebuildimage"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/nodebuildnode"
"github.com/novox/mesh-host/internal/apply"
)
// The artifact store's holder takes its own new build from the build node that kept it (design 47): its done build names the
// node and the digest, the node's build seat answers where the bytes are as an object on the bus, and the node-engine
// reads them and checks them against the digest. Never another module's files: the seat's verb is the way.
// keptAskWithin bounds one ask of a build seat; keptObjectWithin one read of the object.
var (
keptAskWithin = 30 * time.Second
keptObjectWithin = 10 * time.Minute
)
// keptBuildSeats are the three build seats, each of which serves the verb kept-artifact: the node that kept the build
// holds the seat of the language it was built in, and the others do not answer.
var keptBuildSeats = []string{nodebuildgo.Seat, nodebuildnode.Seat, nodebuildimage.Seat}
// keptArtifactFetch is the apply's fetch of what a module's done build kept on a build node.
func keptArtifactFetch(buildOf func(context.Context, string, string) (string, string, bool, error),
ask func(ctx context.Context, node, seat, verb string, args any, within time.Duration) (json.RawMessage, error),
object func(ctx context.Context, store, name string, limit int64) ([]byte, error)) apply.KeptFetch {
return func(ctx context.Context, module, digest string) ([]byte, bool, error) {
node, fp, kept, err := buildOf(ctx, module, digest)
if err != nil || !kept {
return nil, false, err
}
hex := strings.TrimPrefix(digest, "sha256:")
var notHeld []string
for _, seat := range keptBuildSeats {
raw, err := ask(ctx, node, seat, nodebuildgo.VerbKeptArtifact,
nodebuildgo.KeptArtifactInput{Module: module, Fp: fp, Digest: hex}, keptAskWithin)
if err != nil {
notHeld = append(notHeld, seat+": "+err.Error())
continue
}
var got nodebuildgo.KeptArtifactAnswer
if json.Unmarshal(raw, &got) != nil {
return nil, true, fmt.Errorf("%s of %s answered with something that is not an answer of kept-artifact", seat, node)
}
switch {
case got.Error != "":
return nil, true, fmt.Errorf("%s of %s could not give the kept build of %s: %s", seat, node, module, got.Error)
case got.Refused != "":
return nil, true, fmt.Errorf("%s of %s refuses the kept build of %s: %s %s", seat, node, module, got.Refused, got.Words)
case got.Store == "" || got.Object == "" || got.Digest != hex:
return nil, true, fmt.Errorf("%s of %s answered for %s without the store and the object of the digest %s", seat, node, module, hex)
}
octx, cancel := context.WithTimeout(ctx, keptObjectWithin)
body, err := object(octx, got.Store, got.Object, got.Size+1)
cancel()
if err != nil {
return nil, true, fmt.Errorf("the build of %s kept on %s could not be read from %s: %w", module, node, got.Store, err)
}
return body, true, nil
}
return nil, true, fmt.Errorf("%s keeps the build of %s for the artifact store and none of its build seats answers (%s)",
node, module, strings.Join(notHeld, "; "))
}
}
+75
View File
@@ -0,0 +1,75 @@
package main
import (
"context"
"encoding/json"
"errors"
"strings"
"testing"
"time"
)
// A module's done build that kept its artifact on a build node is fetched through that node's build seat (design 47).
func TestAKeptArtifactIsFetchedThroughTheBuildSeatOfItsNode(t *testing.T) {
hex := strings.Repeat("a", 64)
stored := func(string) (json.RawMessage, error) {
return json.RawMessage(`{"module":"distribution","fp":"f1","digest":"` + hex + `","size":5,"store":"kept-builds","object":"` + hex + `"}`), nil
}
for _, row := range []struct {
name string
kept bool
answer func(seat string) (json.RawMessage, error)
object error
wantKept bool
wantErr string
wantSeat string
}{
{"a build kept nothing is not asked", false, nil, nil, false, "", ""},
{"the seat of the language answers", true, func(seat string) (json.RawMessage, error) {
if seat != "node-build-node" {
return nil, errors.New("no responder")
}
return stored(seat)
}, nil, true, "", "node-build-node"},
{"no build seat of the node answers", true, func(string) (json.RawMessage, error) { return nil, errors.New("no responder") },
nil, true, "novox keeps the build of distribution", ""},
{"the seat refuses it", true, func(string) (json.RawMessage, error) {
return json.RawMessage(`{"module":"distribution","fp":"f1","refused":"not kept","words":"swept"}`), nil
}, nil, true, "refuses the kept build of distribution: not kept", ""},
{"the object cannot be read", true, stored, errors.New("no object"), true, "could not be read from kept-builds", "node-build-go"},
} {
t.Run(row.name, func(t *testing.T) {
var seats []string
fetch := keptArtifactFetch(
func(context.Context, string, string) (string, string, bool, error) {
return "novox", "f1", row.kept, nil
},
func(_ context.Context, node, seat, verb string, _ any, _ time.Duration) (json.RawMessage, error) {
if node != "novox" || verb != "kept-artifact" {
t.Errorf("asked %s of %s on %s", verb, seat, node)
}
seats = append(seats, seat)
return row.answer(seat)
},
func(_ context.Context, store, name string, limit int64) ([]byte, error) {
if store != "kept-builds" || name != hex || limit < 5 {
t.Errorf("read %s/%s up to %d", store, name, limit)
}
return []byte("bytes"), row.object
})
body, kept, err := fetch(context.Background(), "distribution", "sha256:"+hex)
if kept != row.wantKept {
t.Fatalf("kept = %v", kept)
}
if row.wantErr == "" && (err != nil || (kept && string(body) != "bytes")) {
t.Fatalf("got %q, %v", body, err)
}
if row.wantErr != "" && (err == nil || !strings.Contains(err.Error(), row.wantErr)) {
t.Fatalf("the error is %v, want %q", err, row.wantErr)
}
if !row.kept && len(seats) > 0 {
t.Errorf("a build that kept nothing asked %v", seats)
}
})
}
}
+7 -6
View File
@@ -87,15 +87,16 @@ func followLogins(ctx context.Context, session *link.Session, node string, hear
} }
} }
// loginSecretModule is the module the vault names this engine's login values under: own/<node>/mesh-host/<login> (ADR 0583 §3,
// insight).
const loginSecretModule = "mesh-host"
// fetchLogin asks the vault, through its existing hand-out of a machine's own secrets, for the value of login, sealed to this // fetchLogin asks the vault, through its existing hand-out of a machine's own secrets, for the value of login, sealed to this
// node's key, and opens it with the sealing key; no new verb is asked for it. // node's key, and opens it with the sealing key; no new verb is asked for it.
func fetchLogin(queue *link.Session, statePath string) func(context.Context, string) (string, error) { func fetchLogin(queue *link.Session, statePath string) func(context.Context, string) (string, error) {
unseal := sealOpener(statePath) unseal := sealOpener(statePath)
return func(ctx context.Context, login string) (string, error) { return func(ctx context.Context, login string) (string, error) {
// The vault names the login values under the engine's own module, which is §527's (§634).
own, err := queue.Own.Wait(ctx)
if err != nil {
return "", fmt.Errorf("the node-engine's own module is not named yet, so the vault's name of its login value is not known: %w", err)
}
raw, err := queue.AskVault(ctx, "own", map[string]any{}) raw, err := queue.AskVault(ctx, "own", map[string]any{})
if err != nil { if err != nil {
return "", err return "", err
@@ -111,7 +112,7 @@ func fetchLogin(queue *link.Session, statePath string) func(context.Context, str
return "", fmt.Errorf("the vault's own answer is not one: %w", err) return "", fmt.Errorf("the vault's own answer is not one: %w", err)
} }
for _, o := range a.Own { for _, o := range a.Own {
if o.Module != loginSecretModule || o.Name != login { if o.Module != own || o.Name != login {
continue continue
} }
sealed := string(o.Sealed) sealed := string(o.Sealed)
@@ -125,7 +126,7 @@ func fetchLogin(queue *link.Session, statePath string) func(context.Context, str
} }
return string(value), nil return string(value), nil
} }
return "", fmt.Errorf("the vault hands this node no value for %s (own/%s/%s/%s)", login, queueNode(queue), loginSecretModule, login) return "", fmt.Errorf("the vault hands this node no value for %s (own/%s/%s/%s)", login, queueNode(queue), own, login)
} }
} }
+51 -2
View File
@@ -16,6 +16,7 @@ import (
"errors" "errors"
"flag" "flag"
"fmt" "fmt"
nodeengine "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-contracts/identity-and-access/go/seats/meshvault" "git.novox.be/novox/mesh-contracts/identity-and-access/go/seats/meshvault"
"io" "io"
"net" "net"
@@ -35,6 +36,7 @@ import (
"github.com/novox/mesh-host/internal/agent" "github.com/novox/mesh-host/internal/agent"
"github.com/novox/mesh-host/internal/apply" "github.com/novox/mesh-host/internal/apply"
"github.com/novox/mesh-host/internal/bundle" "github.com/novox/mesh-host/internal/bundle"
"github.com/novox/mesh-host/internal/compositions"
"github.com/novox/mesh-host/internal/firewall" "github.com/novox/mesh-host/internal/firewall"
"github.com/novox/mesh-host/internal/identity" "github.com/novox/mesh-host/internal/identity"
"github.com/novox/mesh-host/internal/inventory" "github.com/novox/mesh-host/internal/inventory"
@@ -42,6 +44,7 @@ import (
"github.com/novox/mesh-host/internal/liveness" "github.com/novox/mesh-host/internal/liveness"
"github.com/novox/mesh-host/internal/network" "github.com/novox/mesh-host/internal/network"
"github.com/novox/mesh-host/internal/outward" "github.com/novox/mesh-host/internal/outward"
"github.com/novox/mesh-host/internal/processbuilds"
"github.com/novox/mesh-host/internal/profile" "github.com/novox/mesh-host/internal/profile"
"github.com/novox/mesh-host/internal/reachable" "github.com/novox/mesh-host/internal/reachable"
"github.com/novox/mesh-host/internal/store" "github.com/novox/mesh-host/internal/store"
@@ -985,7 +988,17 @@ func runLink(ctx context.Context, opts options) error {
// What this engine trusts a mesh signature from: the vault's key alone, once read from its signers (novox/hq ADR 0550 // What this engine trusts a mesh signature from: the vault's key alone, once read from its signers (novox/hq ADR 0550
// §2; issue 1126: the controller's key never, the controller ended by its unassignment). // §2; issue 1126: the controller's key never, the controller ended by its unassignment).
trust := newMeshTrust() trust := newMeshTrust()
own := link.NewOwnName()
// The engine's own module, first from its applied record of the build whose fingerprint is its own (design 47 G §527,
// §634, A-149); on a first start with no such record, from its first composing.
own.Set(ownModuleApplied(appliedStoreFor(opts.state), os.Getenv(compositions.FingerprintVar)))
// The live start's recovery (A-404): a first start whose composing never ends is named by its unit's environment, so it
// serves its seat (root-free); the name the assignments place on this node. Ended by the lookup from the bus after G.
if own.Get() == "" {
own.Set(os.Getenv("MESH_ENGINE_OWN_MODULE"))
}
queue := &link.Session{ queue := &link.Session{
Own: own,
Membership: membership, Membership: membership,
// And the public halves and certificate its record says (novox/hq ADR 0555 §3), read at each facts report. // And the public halves and certificate its record says (novox/hq ADR 0555 §3), read at each facts report.
Keys: recordKeys(opts.state, trust.Trusts), Keys: recordKeys(opts.state, trust.Trusts),
@@ -1014,10 +1027,23 @@ func runLink(ctx context.Context, opts options) error {
queue.Acts = engineActs(opts, mine.Node, queue, setuidSearch) queue.Acts = engineActs(opts, mine.Node, queue, setuidSearch)
go reconcileActs(ctx, queue.Acts) go reconcileActs(ctx, queue.Acts)
current := &agentNow{} current := &agentNow{}
queue.Acts.Composition = compositionFrom(current) queue.Acts.Composition = compositionFrom(current, queue.WithheldOf)
queue.Acts.Compose = composeFrom(current, &composerNow) queue.Acts.Compose = composeFrom(current, &composerNow)
queue.Acts.Packages = packagesFrom(mine.Node, queue, &composerNow) queue.Acts.Packages = packagesFrom(mine.Node, queue, &composerNow)
queue.Acts.FirstViews, queue.Acts.EntryAgain = firstViewsFrom(&composerNow), entryAgainFrom(&composerNow)
queue.Acts.SeatList, queue.Acts.SeatListAgain = seatListRead, seatListAgain queue.Acts.SeatList, queue.Acts.SeatListAgain = seatListRead, seatListAgain
queue.Acts.ProcessBuilds = func(module string) (any, error) {
d, err := declaredNow(opts.state, nil)
if errors.Is(err, store.ErrNothingDeclared) {
d, err = nil, nil
}
if err != nil {
return nodeengine.ProcessBuilds{Node: mine.Node, Error: "what runs on this node does not read: " + err.Error()}, nil
}
ctx, cancel := context.WithTimeout(context.Background(), lookBound)
defer cancel()
return processbuilds.Reader{Node: mine.Node, Run: processbuilds.Runner(opts.run())}.Read(ctx, d, module), nil
}
queue.Acts.ComposeCandidate = composeCandidateFrom(mine.Node, &shadowNow) queue.Acts.ComposeCandidate = composeCandidateFrom(mine.Node, &shadowNow)
setRunsHere(runsFromAgent(current)) setRunsHere(runsFromAgent(current))
factsOfNow := func(ctx context.Context, ag *agent.Agent) { factsOfNow := func(ctx context.Context, ag *agent.Agent) {
@@ -1236,6 +1262,7 @@ const (
// anyway. A statement the bus did not take is said at the next look. It reads; it never acts (ADR 0240 // anyway. A statement the bus did not take is said at the next look. It reads; it never acts (ADR 0240
// rule 6) — but for going on with a take act stopped at its bound (ADR 0328 §7), which runs beside it. // rule 6) — but for going on with a take act stopped at its bound (ADR 0328 §7), which runs beside it.
func judgeWhatRuns(ctx context.Context, j *liveness.Judge, queue *link.Session, say link.Announce, reapply, reconcile func(modules ...string)) { func judgeWhatRuns(ctx context.Context, j *liveness.Judge, queue *link.Session, say link.Announce, reapply, reconcile func(modules ...string)) {
say("the looks of what runs and of the accounts start (A-405)")
var missingBefore, inactiveBefore map[string]bool var missingBefore, inactiveBefore map[string]bool
ticker := time.NewTicker(liveness.LookEvery) ticker := time.NewTicker(liveness.LookEvery)
defer ticker.Stop() defer ticker.Stop()
@@ -1258,8 +1285,16 @@ func judgeWhatRuns(ctx context.Context, j *liveness.Judge, queue *link.Session,
case <-nudged: case <-nudged:
owed = true owed = true
} }
first := lastAccounts.Load() == nil
began := time.Now()
if first {
say("the first look of what runs begins (A-405)")
}
st, changed := j.Look(ctx) st, changed := j.Look(ctx)
owed = owed || changed owed = owed || changed
if first {
say(fmt.Sprintf("the first look of what runs took %s; units and accounts next (A-405)", time.Since(began).Round(time.Millisecond)))
}
var unitSt *units.Statement var unitSt *units.Statement
if u := unitJudge; u != nil { if u := unitJudge; u != nil {
us, unitsChanged := u.Look(ctx) us, unitsChanged := u.Look(ctx)
@@ -1273,9 +1308,15 @@ func judgeWhatRuns(ctx context.Context, j *liveness.Judge, queue *link.Session,
} }
var accountSt *accounts.Statement var accountSt *accounts.Statement
if a := accountJudge; a != nil { if a := accountJudge; a != nil {
if first {
say(fmt.Sprintf("the first look of the accounts begins at %s (A-405)", time.Since(began).Round(time.Millisecond)))
}
as, accountsChanged := a.Look(ctx) as, accountsChanged := a.Look(ctx)
accountSt = &as accountSt = &as
lastAccounts.Store(&as) lastAccounts.Store(&as)
if first {
say(fmt.Sprintf("the first look of the accounts ended at %s: %d accounts (A-405)", time.Since(began).Round(time.Millisecond), len(as.Accounts)))
}
owed = owed || accountsChanged owed = owed || accountsChanged
if accountsChanged { if accountsChanged {
for _, v := range as.Accounts { for _, v := range as.Accounts {
@@ -1381,7 +1422,15 @@ func writeHealthReport(ctx context.Context, queue *link.Session, h link.Health,
if c := machinesNow.Load(); c != nil { if c := machinesNow.Load(); c != nil {
machines.ByAddress, machines.Hub = c.Machines() machines.ByAddress, machines.Hub = c.Machines()
} }
_, err := queue.WriteNodeHealth(ctx, link.NodeHealthOf(queue.Membership.Node, h, peers, machines)) report := link.NodeHealthOf(queue.Membership.Node, h, peers, machines)
if queue.Fallback != nil && report.Fallback == nil {
report.Fallback = queue.Fallback() // node-account-fallback (ADR 0583 §6), as the written report carries it
}
_, err := queue.WriteNodeHealth(ctx, report)
// The findings the report says are raised by the engine here, at every look, written to the bus or not (design 47 G A-197).
if keeper := ownFindingsNow.Load(); keeper != nil {
keeper.Report(report, machines.ByAddress)
}
switch { switch {
case err != nil && err.Error() != *unwritten: case err != nil && err.Error() != *unwritten:
*unwritten = err.Error() *unwritten = err.Error()
+138 -9
View File
@@ -25,8 +25,10 @@ import (
"github.com/novox/mesh-host/internal/compose/frombus" "github.com/novox/mesh-host/internal/compose/frombus"
"github.com/novox/mesh-host/internal/link" "github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/liveness" "github.com/novox/mesh-host/internal/liveness"
"github.com/novox/mesh-host/internal/seatwait"
"github.com/novox/mesh-host/internal/selfcompose" "github.com/novox/mesh-host/internal/selfcompose"
"github.com/novox/mesh-host/internal/store" "github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
) )
// The node agent's wiring (novox/hq ADR 0429 §4, ADR 0539 §9): the buckets it watches, the haves it writes, the backup // The node agent's wiring (novox/hq ADR 0429 §4, ADR 0539 §9): the buckets it watches, the haves it writes, the backup
@@ -88,7 +90,7 @@ func (h sessionHaves) Read(ctx context.Context, module string) (agent.Have, bool
return agent.Have{}, false return agent.Have{}, false
} }
return agent.Have{Module: module, FP: got.Fp, Composed: got.Composed, Phase: string(got.Phase), Detail: got.Detail, return agent.Have{Module: module, FP: got.Fp, Composed: got.Composed, Phase: string(got.Phase), Detail: got.Detail,
Sealed: got.SealedAt}, true Sealed: got.SealedAt, Wait: got.Wait}, true
} }
// ReadErr is Read telling a have absent from one the bus could not read (agent.HavesReadErr). // ReadErr is Read telling a have absent from one the bus could not read (agent.HavesReadErr).
@@ -106,7 +108,7 @@ func (h sessionHaves) ReadErr(ctx context.Context, module string) (agent.Have, b
return agent.Have{}, false, err return agent.Have{}, false, err
} }
return agent.Have{Module: module, FP: got.Fp, Composed: got.Composed, Phase: string(got.Phase), Detail: got.Detail, return agent.Have{Module: module, FP: got.Fp, Composed: got.Composed, Phase: string(got.Phase), Detail: got.Detail,
Sealed: got.SealedAt}, true, nil Sealed: got.SealedAt, Wait: got.Wait}, true, nil
} }
func (h sessionHaves) Write(ctx context.Context, x agent.Have) { func (h sessionHaves) Write(ctx context.Context, x agent.Have) {
@@ -123,7 +125,7 @@ func (h sessionHaves) Write(ctx context.Context, x agent.Have) {
} }
h.session.NoteModule(ctx, link.ModuleFact{Module: x.Module, FP: x.FP, Composed: x.Composed, Phase: phase, h.session.NoteModule(ctx, link.ModuleFact{Module: x.Module, FP: x.FP, Composed: x.Composed, Phase: phase,
Failed: phase == string(nodeengineclient.PhaseFailed), Detail: shorten(x.Detail), Judgement: x.Judgement, Waits: x.Waits, Failed: phase == string(nodeengineclient.PhaseFailed), Detail: shorten(x.Detail), Judgement: x.Judgement, Waits: x.Waits,
BuildAlone: x.BuildAlone, Kept: x.Kept, Sealed: x.Sealed, PullWait: x.PullWait}) BuildAlone: x.BuildAlone, Kept: x.Kept, Sealed: x.Sealed, PullWait: x.PullWait, Wait: x.Wait})
} }
func (h sessionHaves) Removed(ctx context.Context, module string) { func (h sessionHaves) Removed(ctx context.Context, module string) {
@@ -447,6 +449,7 @@ func nodeAgent(ctx context.Context, opts options, signer ed25519.PublicKey, trus
} }
began := time.Now() began := time.Now()
node := session.Membership.Node node := session.Membership.Node
var keptFetch atomic.Pointer[apply.KeptFetch] // the composer's, set once it exists
composedBucket, err := compositions.BucketFor(node) composedBucket, err := compositions.BucketFor(node)
if err != nil { if err != nil {
cancel() cancel()
@@ -549,6 +552,9 @@ func nodeAgent(ctx context.Context, opts options, signer ed25519.PublicKey, trus
Backoff: link.Backoff, Backoff: link.Backoff,
Apply: func(ctx context.Context, c compositions.Composed, others []compositions.Composed) agent.Result { Apply: func(ctx context.Context, c compositions.Composed, others []compositions.Composed) agent.Result {
actx, done := applyContext(session.WithLink(ctx)) actx, done := applyContext(session.WithLink(ctx))
if kf := keptFetch.Load(); kf != nil {
actx = apply.WithKept(actx, *kf)
}
// While the grants part's list is followed, the controller's list in the bus module's composition is not // While the grants part's list is followed, the controller's list in the bus module's composition is not
// written: the apply keeps the list on the machine (ADR 0547 clause 8, never both). // written: the apply keeps the list on the machine (ADR 0547 clause 8, never both).
held, herr := users.Hold(c) held, herr := users.Hold(c)
@@ -573,6 +579,9 @@ func nodeAgent(ctx context.Context, opts options, signer ed25519.PublicKey, trus
sayFacts(ctx, ag) sayFacts(ctx, ag)
return res return res
}, },
Stop: func(ctx context.Context, module string) agent.Result {
return stopComposition(ctx, opts, module)
},
Remove: func(ctx context.Context, module string, others []compositions.Composed) agent.Result { Remove: func(ctx context.Context, module string, others []compositions.Composed) agent.Result {
actx, done := applyContext(session.WithLink(ctx)) actx, done := applyContext(session.WithLink(ctx))
res := removeComposition(actx, opts, module, others, ag.Placed(module), sched, say) res := removeComposition(actx, opts, module, others, ag.Placed(module), sched, say)
@@ -591,14 +600,33 @@ func nodeAgent(ctx context.Context, opts options, signer ed25519.PublicKey, trus
own := &agentNow{} own := &agentNow{}
// And the seat list it reads through the seat mesh-catalog, from its bootstrapped definitions until it has: the // And the seat list it reads through the seat mesh-catalog, from its bootstrapped definitions until it has: the
// composer hands nothing new meanwhile, and the node says it has waited five minutes (novox/hq ADR 0636 §1). // composer hands nothing new meanwhile, and the node says it has waited five minutes (novox/hq ADR 0636 §1).
seats, err := startSeatList(run, session, node, say) // The finding writer is started first, so it holds what an earlier run left before the reader writes or clears any.
if session.Own == nil {
session.Own = link.NewOwnName()
}
findings := &lateFindings{own: session.Own, make: func(module string) (*findingsOfTheNode, error) {
return newFindingsOfTheNode(session, node, module, say)
}}
findings.Start(run, say)
seats, err := startSeatList(run, session, node, findings, say)
if err != nil { if err != nil {
// No reader is no gate that lets everything through: the node acts on nothing new, as one that waits does. // No reader is no gate that lets everything through: the node acts on nothing new, as one that waits does.
say(err.Error() + "; the node composes nothing new until it is started again and reads the list (novox/hq ADR 0636 §1)") say(err.Error() + "; the node composes nothing new until it is started again and reads the list (novox/hq ADR 0636 §1)")
} }
// The marks of the haves run off the composition pass, each bounded (see marker). // The marks of the haves run off the composition pass, each bounded (see marker).
marks := newMarker(run) marks := newMarker(run)
heldFindings := &seatwait.HeldKeeper{Node: node, ObserverOf: findings.observer, Writer: findings, Ctx: run, Say: say}
// And what only this engine knows of itself, raised where it knows it (design 47 G A-197): its haves' failed phase, health
// and kept words as they are written, and each part of its health report at every look.
ownFindings := &seatwait.NodeKeeper{Node: node, ObserverOf: findings.observer, Writer: findings, Ctx: run, Say: say}
session.SetHaveSaid(ownFindings.Have, ownFindings.HaveGone)
ownFindingsNow.Store(ownFindings)
hooks := composerHooks{ hooks := composerHooks{
entryNotFound: func(fp string, notFound bool) {
if err := seatwait.SayEntryNotFound(run, findings, node, findings.observer(), fp, notFound, time.Now); err != nil {
say("the finding that the catalog half holds no entry of " + fp + " cannot be kept: " + err.Error())
}
},
seatList: seats, seatList: seats,
seatListFailed: err != nil, seatListFailed: err != nil,
running: func() []compositions.Composed { running: func() []compositions.Composed {
@@ -626,6 +654,11 @@ func nodeAgent(ctx context.Context, opts options, signer ed25519.PublicKey, trus
// A module held alone marks that module, never the others (issue 1126); the marking never holds the pass. // A module held alone marks that module, never the others (issue 1126); the marking never holds the pass.
marks.set("refused", func(ctx context.Context) { now.NotComposedEach(ctx, why, held) }) marks.set("refused", func(ctx context.Context) { now.NotComposedEach(ctx, why, held) })
} }
// And each module held says it as the finding `module-held`, cleared when it is composed again or no longer
// wanted here. A node refused whole (why) says nothing of its modules: their findings stand as they were.
if why == "" && run.Err() == nil {
marks.set("module-held", func(context.Context) { heldFindings.Reconcile(held) })
}
}, },
// And what it keeps of a module as it runs, for a manifest that does not say it yet: on the module's have (ADR 0601 §4). // And what it keeps of a module as it runs, for a manifest that does not say it yet: on the module's have (ADR 0601 §4).
kept: func(kept map[string][]string) { kept: func(kept map[string][]string) {
@@ -656,19 +689,35 @@ func nodeAgent(ctx context.Context, opts options, signer ed25519.PublicKey, trus
// composition bucket) is never applied. While the seat's assignment holds no key for this node it composes // composition bucket) is never applied. While the seat's assignment holds no key for this node it composes
// nothing and keeps running what it runs; a assignment it cannot watch stops the agent, said, and it begins again. // nothing and keeps running what it runs; a assignment it cannot watch stops the agent, said, and it begins again.
composer := selfComposer(session, node, hooks, say) composer := selfComposer(session, node, hooks, say)
machinesNow.Store(composer) // the machines a failing network part points at, by the assignments (issue 774) composer.AlsoEntryAgain = shadow.EntryAgain
composerNow.Store(composer) // what the verb compose asks a pass of, this run's alone composer.OnEngine = session.Own.Set // the first composing names the engine's own module on a first start (§527, §634)
machinesNow.Store(composer) // the machines a failing network part points at, by the assignments (issue 774)
composerNow.Store(composer) // what the verb compose asks a pass of, this run's alone
session.SetManifestOf(composer.ManifestOf) // the event running names the manifest of the build that runs
go followRunning(run, session, composer, say)
go followBuildEnds(run, session, composer, say)
apply.UseEngineModule(composer.EngineModule) // the guard of the engine's own trees fixes no name (§527)
env.Composed, env.SelfComposed, env.Policy, env.ComposedToday = composer, true, composer.Policy, composer.ComposedToday env.Composed, env.SelfComposed, env.Policy, env.ComposedToday = composer, true, composer.Policy, composer.ComposedToday
env.Recovery = recoveryEnv(session, composer)
env.WhyUnmade, env.Waiting, env.RunningManifest = composer.WhyUnmade, composer.Waiting, composer.RunningManifest
keptFetchFn := keptArtifactFetch(composer.KeptBuildOf, session.AskSeatOn, session.KeptObject)
keptFetch.Store(&keptFetchFn)
env.HoldsData = composer.HoldsData // what runs from the kept declaration says Data as its manifest does env.HoldsData = composer.HoldsData // what runs from the kept declaration says Data as its manifest does
session.NoteComposes(linked, nodeengineclient.ComposesOwn, "it composes its own modules, and applies no composition made elsewhere") session.NoteComposes(linked, nodeengineclient.ComposesOwn, "it composes its own modules, and applies no composition made elsewhere")
ag = agent.New(env) ag = agent.New(env)
own.set(ag) own.set(ag)
current.set(ag) current.set(ag)
// The node's own record is stated before the composer starts, from what the node knows of itself and with no bus (A-120).
sayFacts(run, nil)
err = ag.Run(run) err = ag.Run(run)
stopRun() stopRun()
cancel() cancel()
current.set(nil) current.set(nil)
composerNow.Store(nil) composerNow.Store(nil)
ownFindingsNow.Store(nil)
session.SetHaveSaid(nil, nil)
session.SetManifestOf(nil)
apply.UseEngineModule(nil)
shadowNow.Store(nil) shadowNow.Store(nil)
if ctx.Err() != nil { if ctx.Err() != nil {
return return
@@ -827,6 +876,30 @@ func removeComposition(ctx context.Context, opts options, module string, others
return agent.Result{} return agent.Result{}
} }
// stopComposition stops a module's units and containers from the node's record and removes nothing (design 47 G A-98,
// §297): what the module kept stays, so a revert restores it before the reverted build starts.
func stopComposition(ctx context.Context, opts options, module string) agent.Result {
applying.Lock()
defer applying.Unlock()
unlock, err := store.Lock(opts.state, nil)
if err != nil {
return agent.Result{Failed: true, Detail: shorten(err.Error())}
}
defer unlock()
known, err := store.Load(opts.state)
if err != nil {
return agent.Result{Failed: true, Detail: shorten(err.Error())}
}
built, err := system.For(builtFor)
if err != nil {
return agent.Result{Failed: true, Detail: shorten(err.Error())}
}
if _, err := apply.StopModule(ctx, built, known, module, opts.run()); err != nil {
return agent.Result{Failed: true, Detail: shorten(err.Error())}
}
return agent.Result{}
}
// busyModules are the modules whose last apply left a container running because it said busy. // busyModules are the modules whose last apply left a container running because it said busy.
func busyModules() []string { func busyModules() []string {
busyMu.Lock() busyMu.Lock()
@@ -845,6 +918,10 @@ func busyModules() []string {
// (writeHealthReport); nil on a node that does not compose its own, whose parts then name no target. // (writeHealthReport); nil on a node that does not compose its own, whose parts then name no target.
var machinesNow atomic.Pointer[selfcompose.Composer] var machinesNow atomic.Pointer[selfcompose.Composer]
// ownFindingsNow is the keeper of the findings this engine raises of itself in the run that composes the node's own now, for the
// health loop (writeHealthReport); nil while no run holds one.
var ownFindingsNow atomic.Pointer[seatwait.NodeKeeper]
// composerNow is the composer of the run that composes the node's own now, nil while the node applies the controller's // composerNow is the composer of the run that composes the node's own now, nil while the node applies the controller's
// compositions or no run is going: what the verb compose asks a pass of (issue 774). // compositions or no run is going: what the verb compose asks a pass of (issue 774).
var composerNow atomic.Pointer[selfcompose.Composer] var composerNow atomic.Pointer[selfcompose.Composer]
@@ -1052,7 +1129,9 @@ type composerHooks struct {
unseal func(string) ([]byte, error) unseal func(string) ([]byte, error)
refused func(why string, held map[string]string) refused func(why string, held map[string]string)
kept func(kept map[string][]string) kept func(kept map[string][]string)
held func(module string) (compositions.Seals, []string, error) // entryNotFound is selfcompose.Env's EntryNotFound.
entryNotFound func(fp string, notFound bool)
held func(module string) (compositions.Seals, []string, error)
// vaultAuthority and certify are selfcompose.Env's VaultAuthority and Certify. // vaultAuthority and certify are selfcompose.Env's VaultAuthority and Certify.
vaultAuthority func() string vaultAuthority func() string
certify func(ctx context.Context) error certify func(ctx context.Context) error
@@ -1075,13 +1154,15 @@ func selfComposer(session *link.Session, node string, hooks composerHooks, say f
// And every composition it makes in its composition state composed-<node>, a record the provisioner, the assigner // And every composition it makes in its composition state composed-<node>, a record the provisioner, the assigner
// and the view read and nothing applies (novox/hq ADR 0593 §6). // and the view read and nothing applies (novox/hq ADR 0593 §6).
env.ComposedState = link.EngineComposed{Session: session, Node: node} env.ComposedState = link.EngineComposed{Session: session, Node: node}
// And each composition it writes there said as the seat's event composed, which the assigner waits on.
env.SayComposed = session.SayComposed
return selfcompose.New(env) return selfcompose.New(env)
} }
// composerEnv is what the node's composers read of the bus, the vault and the agent: its own composer and its answer // composerEnv is what the node's composers read of the bus, the vault and the agent: its own composer and its answer
// to assignment candidates (selfcompose.Shadow) alike. // to assignment candidates (selfcompose.Shadow) alike.
func composerEnv(session *link.Session, node string, say func(string), hooks composerHooks) selfcompose.Env { func composerEnv(session *link.Session, node string, say func(string), hooks composerHooks) selfcompose.Env {
env := selfcompose.Env{Node: node, env := selfcompose.Env{Node: node, NodeRecord: session.OwnRecord,
Watch: func(ctx context.Context, bucket, pattern string) (<-chan agent.Update, error) { Watch: func(ctx context.Context, bucket, pattern string) (<-chan agent.Update, error) {
if bucket == buckets.Assignments || bucket == buckets.ProposalCandidates { if bucket == buckets.Assignments || bucket == buckets.ProposalCandidates {
// The seat's states are made when their holder starts: a node-engine that starts first waits for the // The seat's states are made when their holder starts: a node-engine that starts first waits for the
@@ -1109,12 +1190,60 @@ func composerEnv(session *link.Session, node string, say func(string), hooks com
Hear: session.Hear, Hear: session.Hear,
// And a moved module's templates archive, in the artifact store, as every archive is fetched (issue 1126). // And a moved module's templates archive, in the artifact store, as every archive is fetched (issue 1126).
Fetch: apply.Fetch, Fetch: apply.Fetch,
Running: hooks.running, Unseal: hooks.unseal, Refused: hooks.refused, Held: hooks.held, Kept: hooks.kept, Running: hooks.running, Unseal: hooks.unseal, Refused: hooks.refused, Held: hooks.held, Kept: hooks.kept, EntryNotFound: hooks.entryNotFound,
VaultAuthority: hooks.vaultAuthority, Certify: hooks.certify} VaultAuthority: hooks.vaultAuthority, Certify: hooks.certify}
if hooks.seatList != nil { if hooks.seatList != nil {
env.SeatList, env.SeatListChanged = hooks.seatList.reader, hooks.seatList.changed env.SeatList, env.SeatListChanged = hooks.seatList.reader, hooks.seatList.changed
} else if hooks.seatListFailed { } else if hooks.seatListFailed {
env.SeatList = selfcompose.Unread{} env.SeatList = selfcompose.Unread{}
} }
// And the shadow too: the catalog half read by the fingerprints its wants and runnings name, over its own calls of the seat mesh-catalog's
// verb entry on its link session (design 47 G §421): neither the indexer's nor the builder's states are read.
env.Entry = selfcompose.EntryCall(session.AskMeshSeat)
// And the registry seats' verb versions, to resolve a want that names its version to its artifacts' digests (A-430a).
env.Registry = selfcompose.EntryCall(session.AskMeshSeat)
return env return env
} }
// followRunning keeps the composer told what every node runs, from the seat node-engine's event running, for the run it is
// in: the stream or the link not being there is said and tried again after a backoff, never polled.
func followRunning(run context.Context, session *link.Session, composer *selfcompose.Composer, say func(string)) {
keepFollowing(run, "this node does not hear what the nodes run, and tries again: ", func(ctx context.Context) error {
return session.FollowRunning(ctx, composer.NoteRunning, composer.RunningRead)
}, composer.RunningLost, say)
}
// followBuildEnds keeps the composer told how the builder's builds ended, from its event build.ended, for the run it is in
// (design 47 G A-293): a placed module's newest build that ended failed is said as a refusal in the composition state.
func followBuildEnds(run context.Context, session *link.Session, composer *selfcompose.Composer, say func(string)) {
keepFollowing(run, "this node does not hear how the builder's builds end, so a failed build is not said as refused, and it tries again: ",
func(ctx context.Context) error {
return session.FollowBuildEnds(ctx, composer.NoteBuildEnded, composer.BuildEndsRead)
}, composer.BuildEndsLost, say)
}
// keepFollowing follows a stream with follow for the run, lost called each time it ends; why it ended is said once until it
// changes, and it is followed again after a backoff, never polled.
func keepFollowing(run context.Context, line string, follow func(context.Context) error, lost func(), say func(string)) {
said := ""
for try := 1; run.Err() == nil; try++ {
began := time.Now()
err := follow(run)
lost()
if run.Err() != nil {
return
}
if time.Since(began) > time.Minute {
try = 1
}
if why := fmt.Sprint(err); why != said {
say(line + why)
said = why
}
select {
case <-run.Done():
return
case <-time.After(link.Backoff(try)):
}
}
}
+286 -43
View File
@@ -7,19 +7,21 @@ import (
"crypto/ed25519" "crypto/ed25519"
"encoding/json" "encoding/json"
"errors" "errors"
"fmt"
"git.novox.be/novox/mesh-contracts/go/clients/meshmoduleassignerclient" "git.novox.be/novox/mesh-contracts/go/clients/meshmoduleassignerclient"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner" "git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
nodeengine "git.novox.be/novox/mesh-contracts/go/seats/nodeengine" nodeengine "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient" condseat "git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshconditions"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliveryclient" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshdeliveryclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshwants" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
builderbuilds "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants"
"github.com/novox/mesh-host/internal/buckets" "github.com/novox/mesh-host/internal/buckets"
"github.com/novox/mesh-host/internal/buckets/bucketstest" "github.com/novox/mesh-host/internal/buckets/bucketstest"
engcomp "github.com/novox/mesh-host/internal/compositions" engcomp "github.com/novox/mesh-host/internal/compositions"
"os" "os"
"path/filepath" "path/filepath"
"sort"
"strings" "strings"
"sync" "sync"
"sync/atomic" "sync/atomic"
@@ -27,6 +29,7 @@ import (
"time" "time"
nec "git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient" nec "git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient"
"git.novox.be/novox/mesh-sdk/go/buscalls"
"git.novox.be/novox/mesh-sdk/go/buscompose" "git.novox.be/novox/mesh-sdk/go/buscompose"
"git.novox.be/novox/mesh-sdk/go/declaration" "git.novox.be/novox/mesh-sdk/go/declaration"
"git.novox.be/novox/mesh-sdk/go/seatlist" "git.novox.be/novox/mesh-sdk/go/seatlist"
@@ -72,7 +75,59 @@ type itBed struct {
seatSilent atomic.Bool seatSilent atomic.Bool
// seatAsked counts the node's asks of the stand-in seat, answered or refused: the node reached the bus and asked. // seatAsked counts the node's asks of the stand-in seat, answered or refused: the node reached the bus and asked.
seatAsked atomic.Int32 seatAsked atomic.Int32
findings nats.KeyValue // seatAway makes the stand-in answer as the bus answers a subject nothing serves (no responders), as while the seat's
// holder re-serves (issue 1966).
seatAway atomic.Bool
findings nats.KeyValue
// entries are the entries the stand-in of the seat mesh-catalog answers its verb entry from, by fingerprint (design 47 G §421).
entries map[string]meshcatalog.CatalogEntry
}
// registerEntry makes the stand-in answer the entry of fp: module's manifest and its artifacts (refs `<module>:<name>`).
func (b *itBed) registerEntry(module, fp, manifest string, artifacts ...string) {
b.t.Helper()
var m map[string]any
if err := json.Unmarshal([]byte(manifest), &m); err != nil {
b.t.Fatal(err)
}
e := meshcatalog.CatalogEntry{Module: module, Fp: fp, Manifest: m, ManifestDigest: meshcatalogclient.ManifestDigest([]byte(manifest)),
Repository: "novox/" + module, Commit: strings.Repeat("1", 40)}
for _, a := range artifacts {
e.Artifacts = append(e.Artifacts, meshcatalog.Artifact{Ref: module + ":" + a, Kind: "archive", Language: "go"})
}
b.mu.Lock()
b.entries[fp] = e
b.mu.Unlock()
}
// running says the seat node-engine's event running of module on node at fp, as that node's engine does.
func (b *itBed) running(node, module, fp, manifestDigest string) {
b.t.Helper()
raw, _ := json.Marshal(nodeengine.Running{V: 1, Node: node, Module: module, Fingerprint: fp, Manifest: manifestDigest, At: time.Now().UTC()})
if _, err := b.js.Publish(nodeengine.SubjectRunning(node, module), raw); err != nil {
b.t.Fatal(err)
}
}
// standInEntry answers the seat mesh-catalog's verb entry from the registered entries, and as the holder refuses an unknown one.
func (b *itBed) standInEntry() {
sub, err := b.conn.Subscribe("mesh.seat.mesh-catalog.call.entry.*", func(m *nats.Msg) {
var in meshcatalog.EntryInput
_ = json.Unmarshal(m.Data, &in)
b.mu.Lock()
e, ok := b.entries[in.Fingerprint]
b.mu.Unlock()
if !ok {
_ = m.Respond([]byte(`{"error":"no entry of fingerprint ` + in.Fingerprint + ` is kept"}`))
return
}
out, _ := json.Marshal(map[string]any{"result": meshcatalog.EntryAnswer{Entry: e}})
_ = m.Respond(out)
})
if err != nil {
b.t.Fatal(err)
}
b.t.Cleanup(func() { _ = sub.Unsubscribe() })
} }
// aServer is an embedded bus with JetStream and two users: the test's, unrestricted, and the node's, granted exactly what // aServer is an embedded bus with JetStream and two users: the test's, unrestricted, and the node's, granted exactly what
@@ -125,7 +180,7 @@ func newItBedAs(t *testing.T, perms nodegrant.Permissions) *itBed {
builtFor = "arch" builtFor = "arch"
t.Cleanup(func() { builtFor = was }) t.Cleanup(func() { builtFor = was })
b := &itBed{t: t, dir: t.TempDir()} b := &itBed{t: t, dir: t.TempDir(), entries: map[string]meshcatalog.CatalogEntry{}}
b.conn, b.nodeConn = aServer(t, perms, func(err error) { b.conn, b.nodeConn = aServer(t, perms, func(err error) {
b.mu.Lock() b.mu.Lock()
b.refusals = append(b.refusals, err.Error()) b.refusals = append(b.refusals, err.Error())
@@ -154,7 +209,20 @@ func newItBedAs(t *testing.T, perms nodegrant.Permissions) *itBed {
if b.findings, err = b.js.CreateKeyValue(&nats.KeyValueConfig{Bucket: link.FindingsBucket, History: 1}); err != nil { if b.findings, err = b.js.CreateKeyValue(&nats.KeyValueConfig{Bucket: link.FindingsBucket, History: 1}); err != nil {
t.Fatal(err) t.Fatal(err)
} }
// The engine's own finding stream and the keeper's ask stream (design 47 G §383, §390), made as the bus module makes them.
for _, c := range []nats.StreamConfig{{Name: "MESH-HOST-FINDINGS", Subjects: []string{"mesh.stream.mesh-host.event.>"}},
{Name: condseat.StreamFindingsAsked, Subjects: []string{condseat.SubjectFindingsAsked}}} {
if _, err = b.js.AddStream(&c); err != nil {
t.Fatal(err)
}
}
// The stream the seat node-engine's event running is kept in, the last of each subject (design 47 G §199).
if _, err = b.js.AddStream(&nats.StreamConfig{Name: nodeengine.StreamRunning, Subjects: []string{nodeengine.SubjectRunningAll},
MaxMsgsPerSubject: 1}); err != nil {
t.Fatal(err)
}
b.standInSeat() b.standInSeat()
b.standInEntry()
b.opts = options{state: filepath.Join(b.dir, "state", "state.json"), timeout: 2 * time.Second} b.opts = options{state: filepath.Join(b.dir, "state", "state.json"), timeout: 2 * time.Second}
if err := os.MkdirAll(filepath.Dir(b.opts.state), 0o755); err != nil { if err := os.MkdirAll(filepath.Dir(b.opts.state), 0o755); err != nil {
t.Fatal(err) t.Fatal(err)
@@ -163,10 +231,19 @@ func newItBedAs(t *testing.T, perms nodegrant.Permissions) *itBed {
return b return b
} }
// itOwnModule is the node-engine's own module of the bed: what its applied record of its own build says (§527, §634).
const itOwnModule = "mesh-host"
func itOwn() *link.OwnName {
o := link.NewOwnName()
o.Set(itOwnModule)
return o
}
// start runs the node agent under a session on the bus. // start runs the node agent under a session on the bus.
func (b *itBed) start() (stop func()) { func (b *itBed) start() (stop func()) {
b.t.Helper() b.t.Helper()
b.session = &link.Session{Membership: link.Membership{Node: itNode}, Timeout: 2 * time.Second, b.session = &link.Session{Own: itOwn(), Membership: link.Membership{Node: itNode}, Timeout: 2 * time.Second,
Say: b.say, Keys: func() link.RecordKeys { return link.RecordKeys{} }} Say: b.say, Keys: func() link.RecordKeys { return link.RecordKeys{} }}
b.session.Attach(context.Background(), link.OverNATS{Conn: b.nodeConn, JS: b.nodeJS, Node: itNode}) b.session.Attach(context.Background(), link.OverNATS{Conn: b.nodeConn, JS: b.nodeJS, Node: itNode})
ctx, cancel := context.WithCancel(context.Background()) ctx, cancel := context.WithCancel(context.Background())
@@ -414,21 +491,26 @@ func h8(t *testing.T, b *itBed, checkBuild bool) (func(), h8state) {
} }
return kv return kv
} }
assignment, modules, manifests := kvOf(buckets.Assignments), kvOf(buckets.Modules), kvOf(buckets.Manifests) assignment, manifests := kvOf(buckets.Assignments), kvOf(buckets.Manifests)
resolved := kvOf(selfcompose.ResolvedBucket) resolved := kvOf(selfcompose.ResolvedBucket)
builds := kvOf(builderbuilds.BuilderBuildsBucket)
candidates := kvOf(buckets.ProposalCandidates) candidates := kvOf(buckets.ProposalCandidates)
packages := kvOf(link.PackagesBucket) packages := kvOf(link.PackagesBucket)
path := filepath.Join(b.dir, "web.conf") path := filepath.Join(b.dir, "web.conf")
man := `{"module":"web","version":"1","config":{"greeting":{"type":"string","description":"what the file says","default":"hello"}},` + // Beside the greeting: a file under `if-config`, a template with `config-lines` from a list, and two files under a
// value that makes the first of them fail (a file below a file), the second being left (design 42 zfs-owns-zfs §5).
more := func(name string) string { return filepath.Join(b.dir, "web-"+name+".conf") }
man := `{"module":"web","version":"1","config":{"greeting":{"type":"string","description":"what the file says","default":"hello"},` +
`"mods":{"type":"list","description":"lines","default":["a","b"]}},` +
`"resources":[{"id":"conf","type":"file","path":"` + path + `","mode":"0644","content":"greeting=${config:greeting}\n"},` + `"resources":[{"id":"conf","type":"file","path":"` + path + `","mode":"0644","content":"greeting=${config:greeting}\n"},` +
`{"id":"tools","type":"package","package":"web-tools"}],` + `{"id":"mods","type":"file","path":"` + more("mods") + `","mode":"0644","content":"${config-lines:mods:m=}\n"},` +
`"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]}}` `{"id":"salut","type":"file","path":"` + more("salut") + `","mode":"0644","content":"salut\n","if-config":["greeting=salut"]},` +
`{"id":"bad","type":"file","path":"` + path + `/x","mode":"0644","content":"x\n","if-config":["greeting=boom"]},` +
`{"id":"after","type":"file","path":"` + more("after") + `","mode":"0644","content":"after\n","if-config":["greeting=boom"]},` +
`{"id":"tools","type":"package","package":"web-tools"}]}`
digest := meshcatalogclient.ManifestDigest([]byte(man)) digest := meshcatalogclient.ManifestDigest([]byte(man))
mraw, _ := meshcatalogclient.EncodeManifest(meshcatalog.ManifestsByDigestValue{Module: "web", Manifest: man}) mraw, _ := meshcatalogclient.EncodeManifest(meshcatalog.ManifestsByDigestValue{Module: "web", Manifest: man})
mkey, _ := meshcatalogclient.ManifestsKey(digest) mkey, _ := meshcatalogclient.ManifestsKey(digest)
fp := fpIt("a") fp := fpIt("a")
modv, _ := meshcatalogclient.EncodeModule(meshcatalog.ModulesByModuleValue{Fp2: fp, Manifest: digest, Repository: "novox/web", Commit: strings.Repeat("1", 40)})
place, _ := meshmoduleassignerclient.EncodeAssignments(meshmoduleassigner.Assignments{Node: itNode, Modules: []meshmoduleassigner.PlacedModule{{Module: "web"}}}) place, _ := meshmoduleassignerclient.EncodeAssignments(meshmoduleassigner.Assignments{Node: itNode, Modules: []meshmoduleassigner.PlacedModule{{Module: "web"}}})
rec, _ := nec.EncodeHaveNode(nodeengine.HaveByNodeNodeValue{Record: &nodeengine.NodeRecord{Node: itNode, Capabilities: []string{"service-manager", "package-manager"}}}) rec, _ := nec.EncodeHaveNode(nodeengine.HaveByNodeNodeValue{Record: &nodeengine.NodeRecord{Node: itNode, Capabilities: []string{"service-manager", "package-manager"}}})
// it1 in one key per module and node (research 072 row 37), it2 in the form before: a node reads both. // it1 in one key per module and node (research 072 row 37), it2 in the form before: a node reads both.
@@ -442,30 +524,29 @@ func h8(t *testing.T, b *itBed, checkBuild bool) (func(), h8state) {
mdigest := meshcatalogclient.ManifestDigest([]byte(mailer)) mdigest := meshcatalogclient.ManifestDigest([]byte(mailer))
mlraw, _ := meshcatalogclient.EncodeManifest(meshcatalog.ManifestsByDigestValue{Module: "mailer", Manifest: mailer}) mlraw, _ := meshcatalogclient.EncodeManifest(meshcatalog.ManifestsByDigestValue{Module: "mailer", Manifest: mailer})
mlkey, _ := meshcatalogclient.ManifestsKey(mdigest) mlkey, _ := meshcatalogclient.ManifestsKey(mdigest)
mlmod, _ := meshcatalogclient.EncodeModule(meshcatalog.ModulesByModuleValue{Fp2: fpIt("b"), Manifest: mdigest, Repository: "novox/mailer", Commit: strings.Repeat("2", 40)})
places2, _ := meshmoduleassignerclient.AssignmentsValues(meshmoduleassigner.Assignments{Node: "it2", Modules: []meshmoduleassigner.PlacedModule{{Module: "mailer"}}}) places2, _ := meshmoduleassignerclient.AssignmentsValues(meshmoduleassigner.Assignments{Node: "it2", Modules: []meshmoduleassigner.PlacedModule{{Module: "mailer"}}})
rec2, _ := nec.EncodeHaveNode(nodeengine.HaveByNodeNodeValue{Record: &nodeengine.NodeRecord{Node: "it2", Capabilities: []string{"service-manager"}}}) rec2, _ := nec.EncodeHaveNode(nodeengine.HaveByNodeNodeValue{Record: &nodeengine.NodeRecord{Node: "it2", Capabilities: []string{"service-manager"}}})
pkey2, _ := meshmoduleassignerclient.NodeKey("it2") pkey2, _ := meshmoduleassignerclient.NodeKey("it2")
hkey2, _ := nec.HaveNodeKey("it2") hkey2, _ := nec.HaveNodeKey("it2")
buildKey, _ := builderbuilds.BuilderBuildKey("web", fp)
done, _ := builderbuilds.EncodeBuilderBuild(builderbuilds.BuilderBuild{State: builderbuilds.BuildDone, Manifest: digest,
Artifacts: []builderbuilds.BuiltArtifact{{Name: "files", Kind: "archive", Reference: "artifact-store://web/files/blobs/" + fpDigest("f"), Digest: fpDigest("f")}}})
firstBuild := done
if checkBuild {
firstBuild = []byte(`{"v":1,"state":"no such state"}`)
}
for _, w := range []struct { for _, w := range []struct {
kv nats.KeyValue kv nats.KeyValue
key string key string
value []byte value []byte
}{{manifests, mkey, mraw}, {modules, "web", modv}, {b.haves, hkey, rec}, }{{manifests, mkey, mraw}, {b.haves, hkey, rec},
{resolved, itNode + ".web.greeting", []byte(`{"value":"bonjour","source":"module"}`)}, {resolved, itNode + ".web.greeting", []byte(`{"value":"bonjour","source":"module"}`)},
{manifests, mlkey, mlraw}, {modules, "mailer", mlmod}, {b.haves, hkey2, rec2}, {assignment, pkey2, places2[pkey2]}, {assignment, "it2.mailer", places2["it2.mailer"]}, {manifests, mlkey, mlraw}, {b.haves, hkey2, rec2}, {assignment, pkey2, places2[pkey2]}, {assignment, "it2.mailer", places2["it2.mailer"]},
{resolved, "it2.mailer.domain", []byte(`{"value":"example.org","source":"module"}`)}, {builds, buildKey, firstBuild}} { {resolved, "it2.mailer.domain", []byte(`{"value":"example.org","source":"module"}`)}} {
if _, err := w.kv.Put(w.key, w.value); err != nil { if _, err := w.kv.Put(w.key, w.value); err != nil {
t.Fatal(err) t.Fatal(err)
} }
} }
// Design 47 G §421: the node reads the entry its want names, through the seat mesh-catalog's verb entry, and the part's digest
// from the want; what it hears another node run (the mailer on it2) is read by the same fingerprint.
if !checkBuild {
b.registerEntry("web", fp, man)
}
b.registerEntry("mailer", fpIt("b"), mailer)
b.running("it2", "mailer", fpIt("b"), mdigest)
want, _ := bucketstest.Want(meshwants.WantsByWantNodeModuleValue{Fp: fp, Self: true}) want, _ := bucketstest.Want(meshwants.WantsByWantNodeModuleValue{Fp: fp, Self: true})
wkey := meshwants.WantsByWantNodeModuleKey(itNode, "web") wkey := meshwants.WantsByWantNodeModuleKey(itNode, "web")
if _, err := b.wants.Put(wkey, want); err != nil { if _, err := b.wants.Put(wkey, want); err != nil {
@@ -486,17 +567,18 @@ func h8(t *testing.T, b *itBed, checkBuild bool) (func(), h8state) {
t.Fatal(err) t.Fatal(err)
} }
if checkBuild { if checkBuild {
// The build of the placed module is read through the node's session: one that does not decode is said, and nothing // The entry of the build its want names is asked of the catalog half through the node's session: one the half holds none
// is applied; a done one, then a config event, and the node composes. // of holds the want (entry-missing, naming the fingerprint) and nothing is applied; once the half holds it and it is
within(t, 5*time.Second, "the node did not read web's build key through its session", func() bool { // asked again by hand, then a config event, and the node composes.
b.mu.Lock() within(t, 5*time.Second, "the have does not say the entry of web's build is missing", func() bool {
defer b.mu.Unlock() h := b.have("web")
return strings.Contains(strings.Join(b.said, "\n"), "the builder's "+buildKey) return h.Phase == nec.PhaseHeld && strings.HasPrefix(h.Detail, agent.EntryMissing) && strings.Contains(h.Detail, fp)
}) })
if b.file("web") != "" { if b.file("web") != "" {
t.Fatal("composed with a build key that does not decode") t.Fatal("composed with an entry the catalog half does not hold")
} }
if _, err := builds.Put(buildKey, done); err != nil { b.registerEntry("web", fp, man)
if err := composerNow.Load().EntryAgain(context.Background(), fp); err != nil {
t.Fatal(err) t.Fatal(err)
} }
if _, err := resolved.Put(itNode+".web.greeting", []byte(`{"value":"bonjour","source":"module"}`)); err != nil { if _, err := resolved.Put(itNode+".web.greeting", []byte(`{"value":"bonjour","source":"module"}`)); err != nil {
@@ -533,6 +615,36 @@ func h8rest(t *testing.T, b *itBed, resolved, candidates nats.KeyValue, hkey str
t.Fatal(err) t.Fatal(err)
} }
within(t, 3*time.Second, "a changed config value was not composed and applied again", func() bool { return b.file("web") == "greeting=salut\n" }) within(t, 3*time.Second, "a changed config value was not composed and applied again", func() bool { return b.file("web") == "greeting=salut\n" })
// Design 42 zfs-owns-zfs §5: the file under `if-config` appears with its value and goes with it, the template's lines
// follow the list, and a failing resource stops its module's apply with the later resource left.
other := func(name string) string {
got, _ := os.ReadFile(filepath.Join(b.dir, "web-"+name+".conf"))
return string(got)
}
setGreeting := func(value string) {
if _, err := resolved.Put(itNode+".web.greeting", []byte(`{"value":"`+value+`","source":"module"}`)); err != nil {
t.Fatal(err)
}
}
within(t, 3*time.Second, "the file under if-config did not appear with its value, or the lines do not follow the list", func() bool {
return other("salut") == "salut\n" && other("mods") == "m=a\nm=b\n"
})
setGreeting("bonjour")
within(t, 3*time.Second, "the file under if-config did not go with its value", func() bool {
_, err := os.Stat(filepath.Join(b.dir, "web-salut.conf"))
return b.file("web") == "greeting=bonjour\n" && errors.Is(err, os.ErrNotExist)
})
setGreeting("boom")
within(t, 5*time.Second, "the failing resource did not fail its module", func() bool {
return b.have("web").Phase == nec.PhaseFailed
})
if got := other("after"); got != "" {
t.Errorf("a resource after the failed one was applied: %q", got)
}
setGreeting("salut")
within(t, 5*time.Second, "the module did not run again once its value held no failing resource", func() bool {
return b.file("web") == "greeting=salut\n" && b.have("web").Phase == nec.PhaseRunning
})
within(t, 2*time.Second, "the have does not name the revision", func() bool { within(t, 2*time.Second, "the have does not name the revision", func() bool {
h := b.have("web") h := b.have("web")
return h.Fp == fp && h.Composed != first && h.Phase == nec.PhaseRunning return h.Fp == fp && h.Composed != first && h.Phase == nec.PhaseRunning
@@ -591,7 +703,10 @@ func h8candidate(t *testing.T, b *itBed, fp, running string) {
t.Fatal(err) t.Fatal(err)
} }
path := filepath.Join(b.dir, "web.conf") path := filepath.Join(b.dir, "web.conf")
man := `{"module":"web","version":"2","config":{"greeting":{"type":"string","description":"what the file says","default":"hello"}},` + // A description past what one answer carries (900 KiB, and under the bus's 1 MiB for one value), so the composition input this
// node sends is over the inline bound (A-237b, A-237c).
pad := strings.Repeat("x", 930_000)
man := `{"module":"web","version":"2","config":{"greeting":{"type":"string","description":"what the file says ` + pad + `","default":"hello"}},` +
`"resources":[{"id":"conf","type":"file","path":"` + path + `","mode":"0640","content":"greeting=${config:greeting}\n"},` + `"resources":[{"id":"conf","type":"file","path":"` + path + `","mode":"0640","content":"greeting=${config:greeting}\n"},` +
`{"id":"tools","type":"package","package":"web-tools"}]}` `{"id":"tools","type":"package","package":"web-tools"}]}`
digest := meshcatalogclient.ManifestDigest([]byte(man)) digest := meshcatalogclient.ManifestDigest([]byte(man))
@@ -617,7 +732,43 @@ func h8candidate(t *testing.T, b *itBed, fp, running string) {
t.Fatal(err) t.Fatal(err)
} }
before, _ := b.composed.Status() before, _ := b.composed.Status()
got, err := composeCandidateFrom(itNode, &shadowNow)(key, false) // A-237c: asked with input and no call token (the checker's, until G removes it, G-6 §321) an input past what one answer carries
// is refused; asked with a token it is sent in chunks through the engine's call, whatever its size (A-237b).
if over, err := composeCandidateFrom(itNode, &shadowNow)(key, true, "", nil); err != nil {
t.Fatal(err)
} else if a, err := nec.DecodeAnswer(over.(json.RawMessage)); err != nil || a.Verdict != nec.Refused || !strings.Contains(a.Refused, "an answer on the bus carries") || a.Input != nil {
t.Errorf("an input over the inline bound asked with no token answered %+v (%v)", a, err)
}
line := &ackingLine{chunks: map[string][]byte{}, done: make(chan struct{})}
const call = "0123456789abcdef0123456789abcdef"
acts := &link.HandActs{Node: itNode, ComposeCandidate: composeCandidateFrom(itNode, &shadowNow)}
chunked, err := acts.Call(link.EngineCall{Verb: link.ActComposeCandidate, Line: line,
Args: map[string]any{"candidate": key, "input": true, "call": call}})
if err != nil {
t.Fatal(err)
}
ca, err := nec.DecodeAnswer(chunked.(json.RawMessage))
if err != nil || ca.Verdict != nec.Composed || ca.Chunks < 2 || ca.Input != nil {
t.Fatalf("the input asked with a token answered %+v (%v), want the chunks and no inline input", ca, err)
}
line.want = int(ca.Chunks)
select {
case <-line.done:
case <-time.After(10 * time.Second):
t.Fatalf("%d of %d chunks were sent", line.sent(), ca.Chunks)
}
var file []byte
for n := int64(1); n <= ca.Chunks; n++ {
var c buscalls.SeatChunk
if err := json.Unmarshal(line.chunks[buscalls.SeatChunks{Seat: nodeengine.Seat, Verb: nodeengine.VerbComposeCandidate, Call: call}.Chunk(n)], &c); err != nil {
t.Fatal(err)
}
file = append(file, c.Data...)
}
if len(file) <= 900<<10 || !json.Valid(file) {
t.Errorf("the chunks held %d bytes, want a JSON input over 900 KiB", len(file))
}
got, err := composeCandidateFrom(itNode, &shadowNow)(key, false, "", nil)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -741,7 +892,10 @@ func TestNatsTheBusServerIsTakenOverAndReplacedOnlyWhenItsBuildChangesIt(t *test
} }
folded, err := declaration.Parse([]byte(`{"declaration":1,"resources":[` + folded, err := declaration.Parse([]byte(`{"declaration":1,"resources":[` +
`{"id":"nats.server-conf","type":"file","path":"` + conf + `","mode":"0644","content":"listen: old\n"},` + `{"id":"nats.server-conf","type":"file","path":"` + conf + `","mode":"0644","content":"listen: old\n"},` +
`{"id":"nats.server","type":"container","name":"` + server + `","image":"` + image + `"}]}`)) `{"id":"nats.server","type":"container","name":"` + server + `","image":"` + image + `",` +
// As the node-engine starts every container since the fingerprint handover (47/G W13.1): the fingerprint of the
// want it applies, here the first want's, so that want changes the server's spec in nothing.
`"env":{"MESH_FINGERPRINT":"` + fpIt("e") + `"}}]}`))
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -770,7 +924,6 @@ func TestNatsTheBusServerIsTakenOverAndReplacedOnlyWhenItsBuildChangesIt(t *test
} }
assignment, modules, manifests := kvOf(buckets.Assignments), kvOf(buckets.Modules), kvOf(buckets.Manifests) assignment, modules, manifests := kvOf(buckets.Assignments), kvOf(buckets.Modules), kvOf(buckets.Manifests)
kvOf(selfcompose.ResolvedBucket) kvOf(selfcompose.ResolvedBucket)
builds := kvOf(builderbuilds.BuilderBuildsBucket)
kvOf(buckets.ProposalCandidates) kvOf(buckets.ProposalCandidates)
kvOf(link.PackagesBucket) kvOf(link.PackagesBucket)
// The bus module at a build, done: its manifest written and the indexer's module naming it at fp. // The bus module at a build, done: its manifest written and the indexer's module naming it at fp.
@@ -783,14 +936,10 @@ func TestNatsTheBusServerIsTakenOverAndReplacedOnlyWhenItsBuildChangesIt(t *test
mraw, _ := meshcatalogclient.EncodeManifest(meshcatalog.ManifestsByDigestValue{Module: "nats", Manifest: man}) mraw, _ := meshcatalogclient.EncodeManifest(meshcatalog.ManifestsByDigestValue{Module: "nats", Manifest: man})
mkey, _ := meshcatalogclient.ManifestsKey(digest) mkey, _ := meshcatalogclient.ManifestsKey(digest)
modv, _ := meshcatalogclient.EncodeModule(meshcatalog.ModulesByModuleValue{Fp2: fp, Manifest: digest, Repository: "novox/mesh-catalog", Commit: strings.Repeat("3", 40)}) modv, _ := meshcatalogclient.EncodeModule(meshcatalog.ModulesByModuleValue{Fp2: fp, Manifest: digest, Repository: "novox/mesh-catalog", Commit: strings.Repeat("3", 40)})
b.registerEntry("nats", fp, man) // the node reads the entry its want names (design 47 G §421)
if _, err := manifests.Put(mkey, mraw); err != nil { if _, err := manifests.Put(mkey, mraw); err != nil {
t.Fatal(err) t.Fatal(err)
} }
bkey, _ := builderbuilds.BuilderBuildKey("nats", fp)
done, _ := builderbuilds.EncodeBuilderBuild(builderbuilds.BuilderBuild{State: builderbuilds.BuildDone, Manifest: digest})
if _, err := builds.Put(bkey, done); err != nil {
t.Fatal(err)
}
if _, err := modules.Put("nats", modv); err != nil { if _, err := modules.Put("nats", modv); err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -918,6 +1067,10 @@ func (b *itBed) standInSeat() {
} }
_ = json.Unmarshal(m.Data, &q) _ = json.Unmarshal(m.Data, &q)
b.seatAsked.Add(1) b.seatAsked.Add(1)
if b.seatAway.Load() {
_ = m.RespondMsg(&nats.Msg{Header: nats.Header{"Status": []string{"503"}}}) // the bus's no responders
return
}
var result any var result any
switch { switch {
case b.seatSilent.Load(): case b.seatSilent.Load():
@@ -972,6 +1125,11 @@ func (c *fiveMinutes) pass() {
} }
} }
// findingSubject is the subject of one of the engine's finding events on node (views.FindingSubjects).
func findingSubject(event, node string) string {
return "mesh.stream." + itOwnModule + ".event." + event + "." + node
}
// ADR 0636 §1, §8, whole against a real bus and the stand-in of the seat: while the seat refuses, the node composes and // ADR 0636 §1, §8, whole against a real bus and the stand-in of the seat: while the seat refuses, the node composes and
// applies nothing new, and keeps running what it ran; after the bound it writes one finding in its state findings; when // applies nothing new, and keeps running what it ran; after the bound it writes one finding in its state findings; when
// the seat names its list the module is applied and the finding is cleared; a node started again with the seat gone // the seat names its list the module is applied and the finding is cleared; a node started again with the seat gone
@@ -1002,7 +1160,14 @@ func TestNatsANodeThatCannotReadTheSeatListActsOnNothingNew(t *testing.T) {
b.mu.Lock() b.mu.Lock()
defer b.mu.Unlock() defer b.mu.Unlock()
said := strings.Join(b.said, "\n") said := strings.Join(b.said, "\n")
return strings.Contains(said, selfcompose.WaitsForTheSeatListOwn) && strings.Contains(said, selfcompose.WaitsForTheSeatList) // web's reason names the bootstrapped seats the node holds (issue 1966).
var boot []string
for _, d := range seatlist.Bootstrapped() {
boot = append(boot, d.Seat)
}
sort.Strings(boot)
return strings.Contains(said, selfcompose.WaitsForTheSeatListOwn) &&
strings.Contains(said, "web: "+fmt.Sprintf(selfcompose.WaitsForTheSeatListHeld, len(boot), strings.Join(boot, ", ")))
}) })
if b.file("web") != "" { if b.file("web") != "" {
t.Fatalf("the node applied web before it read the seat list: %q", b.file("web")) t.Fatalf("the node applied web before it read the seat list: %q", b.file("web"))
@@ -1018,6 +1183,27 @@ func TestNatsANodeThatCannotReadTheSeatListActsOnNothingNew(t *testing.T) {
if b.file("web") != "" { if b.file("web") != "" {
t.Fatalf("the node applied web with its finding written: %q", b.file("web")) t.Fatalf("the node applied web with its finding written: %q", b.file("web"))
} }
// Design 47 G §383, §384, §390 (test 107): the key is followed by the event on the engine's own stream, the engine stated
// its findings at its start, and an ask of the keeper is answered by a new statement.
lastEvent := func(event string) (string, bool) {
m, err := b.js.GetLastMsg("MESH-HOST-FINDINGS", findingSubject(event, itNode))
if err != nil {
return "", false
}
return string(m.Data), true
}
within(t, 5*time.Second, "the node did not say finding-written after its key", func() bool {
got, ok := lastEvent("finding-written")
return ok && strings.Contains(got, `"kind":"seat-list-unread"`) && strings.Contains(got, `"observer":"mesh-host"`)
})
within(t, 5*time.Second, "the node did not state its findings at its start", func() bool { _, ok := lastEvent("findings-stated"); return ok })
if _, err := b.js.Publish(condseat.SubjectFindingsAsked, []byte(`{"ask":"a1","at":"2026-11-11T12:00:00Z","by":"test"}`)); err != nil {
t.Fatal(err)
}
within(t, 5*time.Second, "the node did not answer the keeper's ask with a statement naming the finding", func() bool {
got, ok := lastEvent("findings-stated")
return ok && strings.Contains(got, `"asked":"a1"`) && strings.Contains(got, `"kind":"seat-list-unread"`)
})
b.seatSilent.Store(false) b.seatSilent.Store(false)
// The hand act of the operator and the healers (the verb seat-list-again): a read now, as the seat's event would make. // The hand act of the operator and the healers (the verb seat-list-again): a read now, as the seat's event would make.
@@ -1029,6 +1215,15 @@ func TestNatsANodeThatCannotReadTheSeatListActsOnNothingNew(t *testing.T) {
_, err := b.findings.Get(finding) _, err := b.findings.Get(finding)
return errors.Is(err, nats.ErrKeyNotFound) return errors.Is(err, nats.ErrKeyNotFound)
}) })
within(t, 5*time.Second, "the node did not say finding-cleared for the seat list", func() bool {
for i := uint64(1); i < 40; i++ {
if m, err := b.js.GetMsg("MESH-HOST-FINDINGS", i); err == nil && m.Subject == findingSubject("finding-cleared", itNode) &&
strings.Contains(string(m.Data), `"kind":"seat-list-unread"`) {
return true
}
}
return false
})
// Started again with the seat refusing: nothing new is composed, and what ran is neither removed nor changed. // Started again with the seat refusing: nothing new is composed, and what ran is neither removed nor changed.
stop() stop()
@@ -1047,6 +1242,10 @@ func TestNatsANodeThatCannotReadTheSeatListActsOnNothingNew(t *testing.T) {
// The first read of the seat list ends the wait whether or not it changed what the reader held: Start returns at once, and // The first read of the seat list ends the wait whether or not it changed what the reader held: Start returns at once, and
// the node waits on the reader's Ready before it says the read is over (ADR 0636 §1). // the node waits on the reader's Ready before it says the read is over (ADR 0636 §1).
//
// Issue 1966, the live case of ace: the seat's holder re-serves while the node starts, so the node's reader is answered as
// nothing serves the seat (no responders); once the holder serves again the reader reads the list within the bus's retry
// bound, with no event of the seat and no hand act.
func TestNatsTheFirstReadOfTheSeatListEndsTheNodesWait(t *testing.T) { func TestNatsTheFirstReadOfTheSeatListEndsTheNodesWait(t *testing.T) {
clock := &fiveMinutes{} clock := &fiveMinutes{}
was := seatWaitClock was := seatWaitClock
@@ -1054,8 +1253,17 @@ func TestNatsTheFirstReadOfTheSeatListEndsTheNodesWait(t *testing.T) {
t.Cleanup(func() { seatWaitClock = was }) t.Cleanup(func() { seatWaitClock = was })
b := newItBedAs(t, nodegrant.For(itNode, "it2.mailer.domain")) b := newItBedAs(t, nodegrant.For(itNode, "it2.mailer.domain"))
b.seatAway.Store(true)
stop, _ := h8(t, b, false) stop, _ := h8(t, b, false)
defer stop() defer stop()
var st *seatListState
within(t, 5*time.Second, "the node started no reader of the seat list", func() bool { st = seatListNow.Load(); return st != nil })
asked := b.seatAsked.Load()
within(t, 10*time.Second, "the node's reader did not ask the seat while nothing served it", func() bool {
now := st.reader.Status()
return b.seatAsked.Load() > asked || (!now.Reading && !now.Heard)
})
b.seatAway.Store(false)
within(t, 30*time.Second, "the node did not end its wait at the first read of the seat list", func() bool { within(t, 30*time.Second, "the node did not end its wait at the first read of the seat list", func() bool {
st := seatListNow.Load() st := seatListNow.Load()
if st == nil { if st == nil {
@@ -1065,4 +1273,39 @@ func TestNatsTheFirstReadOfTheSeatListEndsTheNodesWait(t *testing.T) {
waiting, _ := st.waiter.Waiting() waiting, _ := st.waiter.Waiting()
return st.reader.Status().Heard && !waiting return st.reader.Status().Heard && !waiting
}) })
if seatListNow.Load() != st {
t.Error("the list was read by the reader of a run begun since, not by the one that met no responders")
}
}
// ackingLine is a bus connection that acknowledges every chunk published on it, as a caller does, and keeps the chunks.
type ackingLine struct {
mu sync.Mutex
onAck nats.MsgHandler
chunks map[string][]byte
done chan struct{}
want int
closed bool
}
func (l *ackingLine) Subscribe(_ string, cb nats.MsgHandler) (*nats.Subscription, error) {
l.mu.Lock()
l.onAck = cb
l.mu.Unlock()
return &nats.Subscription{}, nil
}
func (l *ackingLine) sent() int { l.mu.Lock(); defer l.mu.Unlock(); return len(l.chunks) }
func (l *ackingLine) Publish(subject string, data []byte) error {
l.mu.Lock()
l.chunks[subject] = data
cb := l.onAck
if l.want > 0 && len(l.chunks) >= l.want && !l.closed {
l.closed = true
close(l.done)
}
l.mu.Unlock()
go cb(&nats.Msg{Subject: strings.Replace(subject, ".chunk.", ".ack.", 1)})
return nil
} }
+77
View File
@@ -0,0 +1,77 @@
package main
import (
"context"
"os"
"path/filepath"
"strings"
"testing"
"time"
condseat "git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshconditions"
"github.com/novox/mesh-host/internal/link"
)
// Design 47 G §527, §634, A-149: the node-engine's own module is the module of its applied record at its own fingerprint,
// and its finding writer is made, observing as that module, once the name is known: at once from the record, else when the
// first composing names it. No name is fixed in code.
func TestTheEnginesOwnNameIsReadFromItsAppliedRecordOrItsFirstComposing(t *testing.T) {
fp := "src2:" + strings.Repeat("a", 64)
applied := appliedStoreFor(filepath.Join(t.TempDir(), "state", "state.json"))
for module, f := range map[string]string{"node": "", "the-engine": fp, "other": "src2:" + strings.Repeat("b", 64)} {
if err := applied.Keep(module, []byte(`{"module":"`+module+`","fp":"`+f+`"}`)); err != nil {
t.Fatal(err)
}
}
for _, tc := range []struct{ fp, want string }{{fp, "the-engine"}, {"", ""}, {"src2:" + strings.Repeat("c", 64), ""}} {
if got := ownModuleApplied(applied, tc.fp); got != tc.want {
t.Errorf("the own module at %q is %q, want %q", tc.fp, got, tc.want)
}
}
own := link.NewOwnName()
made := make(chan string, 1)
late := &lateFindings{own: own, make: func(module string) (*findingsOfTheNode, error) {
made <- module
return nil, context.Canceled // the writer itself is the SDK's: only when it is made is tested here
}}
ctx, stop := context.WithCancel(context.Background())
defer stop()
var said []string
late.Start(ctx, func(s string) { said = append(said, s) })
if err := late.Write(ctx, condseat.Finding{}); err == nil || late.Held() != nil {
t.Fatalf("a finding was taken before the engine's own module was known: %v", err)
}
if len(made) != 0 {
t.Fatal("the writer was made before a name")
}
own.Set("the-engine") // the first composing names it
select {
case m := <-made:
if m != "the-engine" || late.observer() != "the-engine" {
t.Fatalf("the writer was made as %q, observing as %q", m, late.observer())
}
case <-time.After(5 * time.Second):
t.Fatal("the writer was never made after the name was known")
}
}
// §633: the engine's bound for the first seat list is the file the manifest's `view-waits-minutes` feeds, five minutes
// when it holds nothing usable.
func TestTheViewWaitsBoundIsTheManifestsSetting(t *testing.T) {
was := viewWaitsFile
defer func() { viewWaitsFile = was }()
for content, want := range map[string]time.Duration{"2\n": 2 * time.Minute, "": 5 * time.Minute, "0": 5 * time.Minute, "x": 5 * time.Minute} {
viewWaitsFile = filepath.Join(t.TempDir(), "w")
if err := os.WriteFile(viewWaitsFile, []byte(content), 0o644); err != nil {
t.Fatal(err)
}
if got := viewWaitsBound(); got != want {
t.Errorf("the file %q gives %v, want %v", content, got, want)
}
}
viewWaitsFile = filepath.Join(t.TempDir(), "missing")
if got := viewWaitsBound(); got != 5*time.Minute {
t.Errorf("no file gives %v", got)
}
}
+5 -4
View File
@@ -88,11 +88,12 @@ func TestThePackagesGoInWhenTheApplyThatRestoresThePackageManagerWroteIt(t *test
conf := filepath.Join(dir, "pacman.conf") conf := filepath.Join(dir, "pacman.conf")
pacmanAnsweringOnce(t, conf) pacmanAnsweringOnce(t, conf)
opts := options{state: filepath.Join(dir, "state.json")} opts := options{state: filepath.Join(dir, "state.json")}
// The package first: within one pass it is asked about before the file is written, which is the order that // The file first, as the pacman module declares it: a module's apply stops at its first failed resource
// leaves it to a second pass. // (design 42 zfs-owns-zfs §1, answer A-26), so the file that makes the package manager answer comes before the
// package that needs it.
raw := []byte(`{"declaration":1,"resources":[` + raw := []byte(`{"declaration":1,"resources":[` +
`{"id":"pacman.base","type":"package","package":"base"},` + `{"id":"pacman.conf","type":"file","path":"` + conf + `","content":"[options]\n"},` +
`{"id":"pacman.conf","type":"file","path":"` + conf + `","content":"[options]\n"}]}`) `{"id":"pacman.base","type":"package","package":"base"}]}`)
report := applyAndKeep(context.Background(), opts, raw, nil, nil, nil) report := applyAndKeep(context.Background(), opts, raw, nil, nil, nil)
+89
View File
@@ -0,0 +1,89 @@
package main
import (
"context"
"encoding/json"
"os"
"strconv"
"strings"
"time"
"git.novox.be/novox/mesh-contracts/go/seats/meshrecovery"
"github.com/novox/mesh-host/internal/agent"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/selfcompose"
)
// recoveryEnv is what the node agent asks of the recovery module and the manifests to move a module's kept data (design 47 G §296,
// §297, §554, §560, §630, A-77, A-98, A-108): the seat's verbs asked in the node-engine's own name, the endings heard from the
// seat's events, the bound of a wait kept as a key of the state bounds, the catch-up called as the module's tool.
func recoveryEnv(session *link.Session, composer *selfcompose.Composer) *agent.Recovery {
ask := func(ctx context.Context, verb string, in, out any) error {
raw, err := session.AskMeshSeat(ctx, meshrecovery.Seat, verb, in)
if err != nil {
return err
}
return json.Unmarshal(raw, out)
}
return &agent.Recovery{
FormChanges: composer.FormChanges,
SnapshotKept: func(ctx context.Context, in meshrecovery.SnapshotKeptInput) (a meshrecovery.SnapshotKeptAnswer, err error) {
return a, ask(ctx, meshrecovery.VerbSnapshotKept, in, &a)
},
RestoreKept: func(ctx context.Context, in meshrecovery.RestoreKeptInput) (a meshrecovery.RestoreKeptAnswer, err error) {
return a, ask(ctx, meshrecovery.VerbRestoreKept, in, &a)
},
KeptPoints: func(ctx context.Context, in meshrecovery.KeptPointsInput) (a meshrecovery.KeptPointsAnswer, err error) {
return a, ask(ctx, meshrecovery.VerbKeptPoints, in, &a)
},
Hear: func(_ context.Context, kind, module, kept string) (<-chan agent.Ending, func(), error) {
heard, stop, err := session.HearRecoveryEnding(kind, module, kept)
if err != nil {
return nil, nil, err
}
out := make(chan agent.Ending, 64)
done := make(chan struct{})
go func() {
for {
select {
case ev := <-heard:
select {
case out <- agent.Ending(ev):
case <-done:
return
}
case <-done:
return
}
}
}()
return out, func() { stop(); close(done) }, nil
},
Bound: func(ctx context.Context, what, id, module string, since time.Time, d time.Duration) (<-chan struct{}, func(), error) {
return session.KeepBound(ctx, what, id, module, since, d)
},
CatchUp: func(ctx context.Context, module string, c meshrecovery.CatchUp) error {
return session.CallTool(ctx, module, c.Tool, c.Arguments)
},
WaitMinutes: recoveryWaitMinutes(),
}
}
// recoveryWaitFile is the file the manifest declares with the content ${config:recovery-wait-minutes} (design 47 G §633, A-112): the
// node-engine is no bundle under a runtime, so it reads the setting from its file at its start, as the builder does (A-71).
var recoveryWaitFile = "/var/lib/mesh-host/recovery-wait-minutes"
// recoveryWaitMinutes is the bound of the wait for the recovery module, in minutes: the file's number, the design's 90 when the
// file is missing, empty or not a positive number (design 47 G §554, A-112).
func recoveryWaitMinutes() int {
raw, err := os.ReadFile(recoveryWaitFile)
if err != nil {
return agent.RecoveryWaitMinutes
}
n, err := strconv.Atoi(strings.TrimSpace(string(raw)))
if err != nil || n <= 0 {
return agent.RecoveryWaitMinutes
}
return n
}
+301
View File
@@ -0,0 +1,301 @@
//go:build integration
package main
import (
"context"
"encoding/json"
"os"
"path/filepath"
"testing"
"time"
"fmt"
"git.novox.be/novox/mesh-contracts/go/clients/meshmoduleassignerclient"
nec "git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
"git.novox.be/novox/mesh-contracts/go/seats/meshrecovery"
nodeengine "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-host/internal/buckets"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/nodegrant"
"github.com/novox/mesh-host/internal/selfcompose"
"strings"
"sync"
"sync/atomic"
)
// Design 47 G §560, §630, A-77, A-92, A-108: the node agent's recovery step over a real bus. Its session asks the seat's verbs, hears
// the last `restored` already said and the next, keeps a wait's bound as a key of the state bounds whose time ends with the bus's
// marker, and calls the catch-up as the module's tool with its arguments. The recovery module is a stand-in that answers the verbs and
// says the events; the grants part composes the node's subjects from the manifest, so the test adds them to the node's grant.
func TestNatsTheRecoveryStepAsksHearsBoundsAndCatchesUpOverTheBus(t *testing.T) {
perms := nodegrant.For(itNode)
perms.Publish = append(perms.Publish, "$JS.API.>", "$KV.mesh-host_bounds.>", "mesh.seat.mesh-recovery.call.*.node~"+itNode,
"mesh.mod.db.tool.rebuild."+itNode)
perms.Subscribe = append(perms.Subscribe, "mesh.seat.mesh-recovery.event.>", "$KV.mesh-host_bounds.>", "_INBOX.>", "$JS.ACK.>", "$JS.FC.>")
b := newItBedAs(t, perms)
ctx := context.Background()
for _, s := range []struct{ stream, subject string }{{meshrecovery.StreamEnded, meshrecovery.SubjectEndedAll},
{meshrecovery.StreamRestored, meshrecovery.SubjectRestoredAll}} {
if _, err := b.js.AddStream(&nats.StreamConfig{Name: s.stream, Subjects: []string{s.subject}, MaxMsgsPerSubject: 1}); err != nil {
t.Fatal(err)
}
}
js, err := jetstream.New(b.conn)
if err != nil {
t.Fatal(err)
}
if _, err := js.CreateKeyValue(ctx, jetstream.KeyValueConfig{Bucket: link.BoundsBucket, History: 1, LimitMarkerTTL: time.Second}); err != nil {
t.Fatal(err)
}
say := func(act, outcome string) {
raw, _ := json.Marshal(meshrecovery.RestoredEvent{Act: act, Module: "db", Kept: "cache", Outcome: outcome, At: time.Now().UTC()})
if _, err := b.js.Publish(meshrecovery.SubjectRestored("db", "cache"), raw); err != nil {
t.Fatal(err)
}
}
var restoreAsked meshrecovery.RestoreKeptInput
// The kept points the stand-in answers: the one of the first checks, then (below) what the agent's revert finds, and none.
var pointsMu sync.Mutex
points := []meshrecovery.Point{{Module: "db", Kept: "cache", Form: 1, Fingerprint: "f2", Snapshot: "snap-1",
CatchUp: &meshrecovery.CatchUp{Tool: "rebuild", Arguments: map[string]any{"all": true}}}}
// The restore the agent's revert asks waits for the stand-in to say it restored: gated until released.
var gated atomic.Bool
release := make(chan struct{})
var restores atomic.Int32
for verb, answer := range map[string]func(m *nats.Msg) any{
meshrecovery.VerbKeptPoints: func(*nats.Msg) any {
pointsMu.Lock()
defer pointsMu.Unlock()
return meshrecovery.KeptPointsAnswer{Points: points}
},
meshrecovery.VerbRestoreKept: func(m *nats.Msg) any {
pointsMu.Lock()
_ = json.Unmarshal(m.Data, &restoreAsked)
pointsMu.Unlock()
act := fmt.Sprintf("r%d", restores.Add(1))
if gated.Load() {
go func() { <-release; say(act, meshrecovery.RestoredEventOutcomeDone) }()
} else {
say(act, meshrecovery.RestoredEventOutcomeDone)
}
return meshrecovery.RestoreKeptAnswer{Accepted: act}
},
} {
answer := answer
sub, err := b.conn.Subscribe("mesh.seat.mesh-recovery.call."+verb+".*", func(m *nats.Msg) {
out, _ := json.Marshal(map[string]any{"result": answer(m)})
_ = m.Respond(out)
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = sub.Unsubscribe() })
}
var toolArgs map[string]any
var toolCalls atomic.Int32
toolSub, err := b.conn.Subscribe(link.ToolSubject("db", "rebuild", itNode), func(m *nats.Msg) {
var args map[string]any
_ = json.Unmarshal(m.Data, &args)
pointsMu.Lock()
toolArgs = args
pointsMu.Unlock()
toolCalls.Add(1)
_ = m.Respond([]byte(`{"result":{}}`))
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = toolSub.Unsubscribe() })
// A-112: the bound is the number in the file the manifest declares, 90 where it is missing or not a number.
for content, want := range map[string]int{"45\n": 45, "": 90, "soon\n": 90, "0\n": 90} {
recoveryWaitFile = filepath.Join(t.TempDir(), "w")
if err := os.WriteFile(recoveryWaitFile, []byte(content), 0o644); err != nil {
t.Fatal(err)
}
if got := recoveryWaitMinutes(); got != want {
t.Errorf("the file %q gives %d minutes, want %d", content, got, want)
}
}
recoveryWaitFile = filepath.Join(t.TempDir(), "missing")
if got := recoveryWaitMinutes(); got != 90 {
t.Errorf("no file gives %d minutes, want 90", got)
}
say("old", meshrecovery.RestoredEventOutcomeFailed) // said before anyone listens: the last message of the subject
session := b.startSession()
r := recoveryEnv(session, nil)
events, stop, err := r.Hear(ctx, "restored", "db", "cache")
if err != nil {
t.Fatal(err)
}
defer stop()
if first := <-events; first.Act != "old" || first.Outcome != "failed" {
t.Fatalf("the first event heard is %+v, want the last one said before", first)
}
pts, err := r.KeptPoints(ctx, meshrecovery.KeptPointsInput{Module: "db", Kept: "cache"})
if err != nil || len(pts.Points) != 1 {
t.Fatalf("kept-points: %+v, %v", pts, err)
}
got, err := r.RestoreKept(ctx, meshrecovery.RestoreKeptInput{Module: "db", Kept: "cache", Form: 1, Fingerprint: "f2", Snapshot: pts.Points[0].Snapshot, Why: "w"})
pointsMu.Lock()
firstAsked := restoreAsked
pointsMu.Unlock()
if err != nil || got.Accepted != "r1" || firstAsked.Snapshot != "snap-1" {
t.Fatalf("restore-kept: %+v, %v, asked %+v", got, err, firstAsked)
}
if ev := <-events; ev.Act != "r1" || ev.Outcome != "done" {
t.Fatalf("the ending heard is %+v", ev)
}
err = r.CatchUp(ctx, "db", *pts.Points[0].CatchUp)
pointsMu.Lock()
gotArgs := toolArgs
pointsMu.Unlock()
if err != nil || gotArgs["all"] != true {
t.Fatalf("catch-up: %v, arguments %v", err, gotArgs)
}
expired, end, err := r.Bound(ctx, "recovery", "db.restored.cache", "db", time.Now(), time.Second)
if err != nil {
t.Fatal(err)
}
defer end()
select {
case <-expired:
case <-time.After(10 * time.Second):
t.Fatal("the bound's end, the bus's marker, was not heard")
}
// Design 47 G §296, §297, §554, §560, A-113: the node agent whole drives a revert want of db through the same recovery wiring
// (recoveryEnv over the session and the node's composer): db runs at a build whose cache is of form 2, and its want goes back to a
// build of form 1. With no point of form 1 taken before the newer build, its have says the restore point is missing, naming it,
// and the newer build runs again. With the point, the have is held (backing up) until the stand-in says restored, then the
// older build is applied and the point's catch-up tool is called.
dbManifest := func(form int, content string) string {
return `{"module":"db","version":"1","state":[{"name":"cache","form":` + fmt.Sprint(form) + `}],"resources":[{"id":"conf","type":"file","path":"` +
filepath.Join(b.dir, "db.conf") + `","mode":"0644","content":"` + content + `\n"}]}`
}
fpOld, fpNew := fpIt("a"), fpIt("b")
b.registerEntry("db", fpOld, dbManifest(1, "v1"))
b.registerEntry("db", fpNew, dbManifest(2, "v2"))
for _, name := range []string{buckets.Manifests, selfcompose.ResolvedBucket, buckets.ProposalCandidates, link.PackagesBucket} {
if _, err := b.js.CreateKeyValue(&nats.KeyValueConfig{Bucket: name, History: 5}); err != nil {
t.Fatal(err)
}
}
assignment, err := b.js.CreateKeyValue(&nats.KeyValueConfig{Bucket: buckets.Assignments, History: 5})
if err != nil {
t.Fatal(err)
}
places, _ := meshmoduleassignerclient.AssignmentsValues(meshmoduleassigner.Assignments{Node: itNode, Modules: []meshmoduleassigner.PlacedModule{{Module: "db"}}})
hkey, _ := nec.HaveNodeKey(itNode)
rec, _ := nec.EncodeHaveNode(nodeengine.HaveByNodeNodeValue{Record: &nodeengine.NodeRecord{Node: itNode, Capabilities: []string{"service-manager", "package-manager"}}})
if _, err := b.haves.Put(hkey, rec); err != nil {
t.Fatal(err)
}
for key, value := range places {
if _, err := assignment.Put(key, value); err != nil {
t.Fatal(err)
}
}
b.want("db", fpNew, "", "")
stopAgent := b.start()
defer stopAgent()
within(t, 20*time.Second, "db does not run its newer build", func() bool {
h := b.have("db")
return h.Fp == fpNew && h.Phase == nec.PhaseRunning && b.file("db") == "v2\n"
})
// The engine says what runs, as it does after each apply: the composer keeps the entry of the running build while the want moves on.
b.running(itNode, "db", fpNew, meshcatalogclient.ManifestDigest([]byte(dbManifest(2, "v2"))))
// The stand-in holds no point of form 1: the revert is held, the have names the point missing, the newer build runs.
pointsMu.Lock()
points = nil
pointsMu.Unlock()
b.want("db", fpOld, "", "")
within(t, 20*time.Second, "the have does not say the restore point is missing", func() bool {
h := b.have("db")
return h.Phase == nec.PhaseHeld && strings.Contains(h.Detail, "restore point missing") &&
strings.Contains(h.Detail, "db's cache has none of form 1 taken before "+fpNew)
})
if b.have("db").Fp != fpNew || b.file("db") != "v2\n" || restores.Load() != 1 {
t.Fatalf("the missing point left db %+v with %q, %d restore(s) asked", b.have("db"), b.file("db"), restores.Load())
}
// The stand-in now holds the point: the want answered "again" asks the restore, and the have waits for its ending.
pointsMu.Lock()
points = []meshrecovery.Point{{Module: "db", Kept: "cache", Form: 1, Fingerprint: fpNew, Snapshot: "snap-2", At: time.Now(),
CatchUp: &meshrecovery.CatchUp{Tool: "rebuild", Arguments: map[string]any{"all": "again"}}}}
pointsMu.Unlock()
gated.Store(true)
calls := toolCalls.Load()
b.want("db", fpOld, "", meshwants.WantsByWantNodeModuleValueBackupAgain)
within(t, 20*time.Second, "the restore of the revert is not asked", func() bool { return restores.Load() == 2 })
within(t, 20*time.Second, "the have does not wait for the restore", func() bool { return b.have("db").Phase == nec.PhaseBackingUp })
// The restored event is held until close(release): the older build must not be applied before the stand-in says restored.
if h := b.have("db"); h.Phase != nec.PhaseBackingUp || b.file("db") != "v2\n" || toolCalls.Load() != calls {
t.Fatalf("before the stand-in says restored the have is %+v, the file %q, %d catch-up call(s) more", h, b.file("db"), toolCalls.Load()-calls)
}
close(release)
within(t, 20*time.Second, "the older build is not applied after the restore", func() bool {
h := b.have("db")
return h.Fp == fpOld && h.Phase == nec.PhaseRunning && b.file("db") == "v1\n"
})
within(t, 20*time.Second, "the catch-up tool is not called after the restore", func() bool { return toolCalls.Load() == calls+1 })
pointsMu.Lock()
asked, gotArgs := restoreAsked, toolArgs
pointsMu.Unlock()
if asked.Snapshot != "snap-2" || asked.Fingerprint != fpNew || gotArgs["all"] != "again" {
t.Errorf("the revert asked %+v and called the catch-up with %v", asked, gotArgs)
}
stopAgent()
// Design 47 G A-293: over the same session, the builder's build ends the composer follows: the stream read whole first (the
// last end of each module and fingerprint, in the stream's order), then the read's end, then each end said after it.
if _, err := b.js.AddStream(&nats.StreamConfig{Name: link.BuildEndsStream, Subjects: []string{link.BuildEndsSubject}, MaxMsgsPerSubject: 1}); err != nil {
t.Fatal(err)
}
ended := func(fp, outcome, why string) {
raw, _ := json.Marshal(builds.BuildEnded{Module: "db", Fingerprint: fp, Outcome: outcome, Error: why})
if _, err := b.js.Publish("mesh.stream.mesh-delivery-builder.event."+builds.BuildEndedEventName("db", fp), raw); err != nil {
t.Fatal(err)
}
}
ended("f1", builds.BuildFailed, "first")
ended("f1", builds.BuildFailed, "no go")
ended("f0", builds.BuildDone, "")
heard := make(chan string, 8)
follow, stopFollow := context.WithCancel(ctx)
defer stopFollow()
go func() {
_ = session.FollowBuildEnds(follow, func(e builds.BuildEnded) { heard <- e.Fingerprint + ":" + e.Outcome + ":" + e.Error },
func() { heard <- "read" })
}()
for i, want := range []string{"f1:failed:no go", "f0:done:", "read", "f2:failed:later"} {
if want == "f2:failed:later" {
ended("f2", builds.BuildFailed, "later")
}
select {
case got := <-heard:
if got != want {
t.Fatalf("build end %d heard %q, want %q", i, got, want)
}
case <-time.After(10 * time.Second):
t.Fatalf("build end %d, %q, was not heard", i, want)
}
}
}
// startSession is a session on the node's own connection, with no node agent run on it.
func (b *itBed) startSession() *link.Session {
b.session = &link.Session{Own: itOwn(), Membership: link.Membership{Node: itNode}, Timeout: 2 * time.Second,
Say: b.say, Keys: func() link.RecordKeys { return link.RecordKeys{} }}
b.session.Attach(context.Background(), link.OverNATS{Conn: b.nodeConn, JS: b.nodeJS, Node: itNode})
return b.session
}
+128 -26
View File
@@ -5,11 +5,13 @@ import (
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
"os"
"strconv"
"strings" "strings"
"sync"
"sync/atomic" "sync/atomic"
"time" "time"
conditions "git.novox.be/novox/mesh-contracts/health-and-repair/go/clients/meshconditionsclient"
condseat "git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshconditions" condseat "git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshconditions"
"git.novox.be/novox/mesh-sdk/go/catalogue" "git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/seatdef" "git.novox.be/novox/mesh-sdk/go/seatdef"
@@ -103,46 +105,58 @@ func (realClock) AfterFunc(d time.Duration, f func()) seatwait.Timer {
return time.AfterFunc(d, f) return time.AfterFunc(d, f)
} }
// findingsOfTheNode writes the node's findings in its state `findings`, in the layout of the seat mesh-conditions. // nodeFindings is the node-engine's finding writer: mesh-sdk's held writer, for the engine's findings are raised as they
type findingsOfTheNode struct{ session *link.Session } // happen and not judged afresh (design 47 G A-29, A-40), with the contract of the seat mesh-conditions.
type nodeFindings = views.HeldWriter[condseat.Finding, condseat.FindingCleared, condseat.FindingsStated, condseat.FindingsAskedEvent]
func (w findingsOfTheNode) encode(f views.Finding) (string, []byte, error) { // findingsOfTheNode keeps the node's findings through mesh-sdk's held writer (design 47 G §383, §384, §461, A-29): each
key := conditions.Key(conditions.ScopeMachine, f.ID, f.Kind) // finding is written in the node's state `findings` and said as an event on the engine's own stream (a foundation part's
v := condseat.FindingsByKeyValue{V: 1, Key: key, Kind: f.Kind, Observer: f.Observer, Severity: conditions.Warning, // `mesh-host-findings`, §529, A-30), the engine says what it holds at its start and when the keeper asks, and the keeper
Subject: condseat.Subject{Scope: conditions.ScopeMachine, ID: f.ID, Machine: f.ID}, Summary: f.Summary, // reads no state of the engine's.
Headline: f.Headline, Explanation: f.Explanation, Source: "seat-list-unread", At: f.At.UTC()} type findingsOfTheNode struct {
if err := conditions.CheckFinding(v); err != nil { *nodeFindings
return "", nil, err
}
raw, err := json.Marshal(v)
return key, raw, err
} }
func (w findingsOfTheNode) Put(f views.Finding) error { // newFindingsOfTheNode makes the writer at once; it reads the state when it is started (A-40). Its events are published on
key, raw, err := w.encode(f) // the engine's own stream awaiting the bus's acknowledgement and its consumer of `findings-asked` is its own.
func newFindingsOfTheNode(session *link.Session, node, module string, say func(string)) (*findingsOfTheNode, error) {
writer, err := views.NewHeldWriter(views.WriterConfig[condseat.Finding, condseat.FindingCleared, condseat.FindingsStated, condseat.FindingsAskedEvent]{
Module: module,
Node: node,
State: link.FindingsState{Session: session},
Kinds: append([]string{seatwait.Kind, seatwait.HeldKind, seatwait.EntryNotFoundKind}, seatwait.NodeKinds...),
Asked: condseat.SubjectFindingsAsked,
Logf: func(format string, args ...any) { say(fmt.Sprintf(format, args...)) },
Publish: session.PublishEvent,
Listen: func(_ context.Context, subject string, handle func([]byte)) (func(), error) {
return session.ListenFrom(condseat.StreamFindingsAsked, subject, handle)
},
})
if err != nil { if err != nil {
return err return nil, err
} }
return w.session.PutFinding(key, raw) return &findingsOfTheNode{writer}, nil
} }
func (w findingsOfTheNode) Delete(f views.Finding) error { // Start seeds what the state holds, states the findings now and hears the keeper until ctx ends.
key, _, err := w.encode(f) func (w *findingsOfTheNode) Start(ctx context.Context) error {
if err != nil { go func() { <-ctx.Done(); w.Close() }()
return err return w.nodeFindings.Start(ctx)
}
return w.session.DeleteFinding(key)
} }
// startSeatList starts this run's reader: its bootstrapped seats are given to catalogue at once, and every change of what // startSeatList starts this run's reader: its bootstrapped seats are given to catalogue at once, and every change of what
// it holds again, saying so to the composer and to the waiter that writes the five-minute finding. // it holds again, saying so to the composer and to the waiter that writes the five-minute finding, and stating again what
func startSeatList(ctx context.Context, session *link.Session, node string, say func(string)) (*seatListState, error) { // the engine reads (link.Session.RestateReads, ADR 0618 §4): the seats it states are those of the list it holds.
func startSeatList(ctx context.Context, session *link.Session, node string, findings *lateFindings, say func(string)) (*seatListState, error) {
st := &seatListState{changed: make(chan struct{}, 1)} st := &seatListState{changed: make(chan struct{}, 1)}
// Start returns at once and reads the seat on a goroutine, which may announce a change before the waiter exists. // Start returns at once and reads the seat on a goroutine, which may announce a change before the waiter exists.
var announce atomic.Pointer[func()] var announce atomic.Pointer[func()]
reader, err := seatlist.Start(seatlist.Config{Bus: sessionBus{ctx, session}, Bootstrap: seatlist.Bootstrapped(), reader, err := seatlist.Start(seatlist.Config{Bus: sessionBus{ctx, session}, Bootstrap: seatlist.Bootstrapped(),
OnChange: func(seats []catalogue.Seat) { OnChange: func(seats []catalogue.Seat) {
catalogue.UseSeats(seats) catalogue.UseSeats(seats)
// What the engine reads changed with the list: its record and its event say it (ADR 0618 §4), off the
// reader's goroutine, which returns quickly.
go session.RestateReads(ctx)
if f := announce.Load(); f != nil { // the first read may end before the waiter exists if f := announce.Load(); f != nil { // the first read may end before the waiter exists
(*f)() (*f)()
} }
@@ -151,7 +165,7 @@ func startSeatList(ctx context.Context, session *link.Session, node string, say
return nil, fmt.Errorf("the node-engine cannot follow the seat list: %w", err) return nil, fmt.Errorf("the node-engine cannot follow the seat list: %w", err)
} }
st.reader = reader st.reader = reader
st.waiter = &seatwait.Waiter{Node: node, Observer: "node-engine", Reader: reader, Findings: findingsOfTheNode{session}, st.waiter = &seatwait.Waiter{Node: node, ObserverOf: findings.observer, Reader: reader, Writer: findings, Ctx: ctx, Bound: viewWaitsBound(),
Clock: seatWaitClock, Say: say} Clock: seatWaitClock, Say: say}
told := func() { told := func() {
st.waiter.Changed() st.waiter.Changed()
@@ -162,6 +176,7 @@ func startSeatList(ctx context.Context, session *link.Session, node string, say
} }
announce.Store(&told) announce.Store(&told)
catalogue.UseSeats(reader.Seats()) catalogue.UseSeats(reader.Seats())
session.RestateReads(ctx) // its first statement holds the bootstrapped seats at least (issue 1966)
told() told()
// The first read ends whether the seat answered or not, and a read that held what the reader held already says no // The first read ends whether the seat answered or not, and a read that held what the reader held already says no
// change: its end is told to the waiter and the composer, so a status that moved from reading to heard starts or ends // change: its end is told to the waiter and the composer, so a status that moved from reading to heard starts or ends
@@ -170,6 +185,7 @@ func startSeatList(ctx context.Context, session *link.Session, node string, say
select { select {
case <-reader.Ready(): case <-reader.Ready():
catalogue.UseSeats(reader.Seats()) catalogue.UseSeats(reader.Seats())
session.RestateReads(ctx)
told() told()
case <-ctx.Done(): case <-ctx.Done():
} }
@@ -177,3 +193,89 @@ func startSeatList(ctx context.Context, session *link.Session, node string, say
seatListNow.Store(st) seatListNow.Store(st)
return st, nil return st, nil
} }
// lateFindings is the node-engine's finding writer, made when the engine's own module is known (design 47 G §527, §634,
// A-149): at once from its applied record, or, on a first start with no such record, when the first composing names it. Till
// then a write or a clear is refused in words, never lost in silence.
type lateFindings struct {
own *link.OwnName
make func(module string) (*findingsOfTheNode, error)
mu sync.Mutex
w *findingsOfTheNode
}
// observer is the module the findings are observed as: the engine's own, "" while it is not known.
func (f *lateFindings) observer() string { return f.own.Get() }
// Start makes the writer and starts it as soon as the own name is known: in this call when it already is, else on a goroutine.
func (f *lateFindings) Start(ctx context.Context, say func(string)) {
begin := func(module string) {
w, err := f.make(module)
if err != nil { // a writer that cannot be made is a fault of its configuration: loud
say("the node-engine cannot make its finding writer: " + err.Error())
return
}
if err := w.Start(ctx); err != nil {
say("the node-engine's findings are kept in its state and not said as events until it is started again: " + err.Error())
}
f.mu.Lock()
f.w = w
f.mu.Unlock()
}
if name := f.own.Get(); name != "" {
begin(name)
return
}
go func() {
if name, err := f.own.Wait(ctx); err == nil {
begin(name)
}
}()
}
var errOwnNameUnknown = errors.New("the node-engine's own module is not yet named by its applied record or its first composing, so its finding writer is not made")
func (f *lateFindings) writer() *findingsOfTheNode {
f.mu.Lock()
defer f.mu.Unlock()
return f.w
}
func (f *lateFindings) Write(ctx context.Context, fi condseat.Finding) error {
if w := f.writer(); w != nil {
return w.Write(ctx, fi)
}
return errOwnNameUnknown
}
func (f *lateFindings) Clear(ctx context.Context, scope, id, kind, why string) error {
if w := f.writer(); w != nil {
return w.Clear(ctx, scope, id, kind, why)
}
return errOwnNameUnknown
}
func (f *lateFindings) Held() []condseat.Finding {
if w := f.writer(); w != nil {
return w.Held()
}
return nil
}
// viewWaitsFile is the file the manifest declares with the content ${config:view-waits-minutes} (design 47 G §633), read at
// the engine's start as recoveryWaitFile is.
var viewWaitsFile = "/var/lib/mesh-host/view-waits-minutes"
// viewWaitsBound is how long the reader of the seat list may wait before the engine says it: the file's minutes, the design's
// five (seatwait.Bound) when the file is missing, empty or not a positive number.
func viewWaitsBound() time.Duration {
raw, err := os.ReadFile(viewWaitsFile)
if err != nil {
return seatwait.Bound
}
n, err := strconv.Atoi(strings.TrimSpace(string(raw)))
if err != nil || n <= 0 {
return seatwait.Bound
}
return time.Duration(n) * time.Minute
}
+39
View File
@@ -26,3 +26,42 @@ func TestTheSettledEventIsTheManifestsConsume(t *testing.T) {
t.Errorf("the engine hears %s, its manifest consumes %v", settledEvent, m.Consumes) t.Errorf("the engine hears %s, its manifest consumes %v", settledEvent, m.Consumes)
} }
} }
// The manifest claims the seat node-engine, says its findings on its own stream and hears the keeper's ask and the
// node-engine seat's running (design 47 G §364, §387, §199).
func TestTheManifestClaimsItsSeatAndSaysItsFindings(t *testing.T) {
raw, err := os.ReadFile("../../module.json")
if err != nil {
t.Fatal(err)
}
var m struct {
Version string `json:"version"`
Consumes []string `json:"consumes"`
Claims []struct {
Name string `json:"name"`
Scope string `json:"scope"`
} `json:"claims"`
Streams []struct {
Name string `json:"name"`
Events []string `json:"events"`
} `json:"streams"`
}
if err := json.Unmarshal(raw, &m); err != nil {
t.Fatal(err)
}
if len(m.Claims) != 1 || m.Claims[0].Name != "node-engine" || m.Claims[0].Scope != "node" {
t.Errorf("claims = %+v, want the node seat node-engine", m.Claims)
}
if len(m.Streams) != 1 || m.Streams[0].Name != "mesh-host-findings" ||
!slices.Equal(m.Streams[0].Events, []string{"finding-written.*", "finding-cleared.*", "findings-stated.*"}) {
t.Errorf("streams = %+v, want mesh-host-findings with the three finding events", m.Streams)
}
for _, c := range []string{"mesh-conditions.findings-asked", "node-engine.running"} {
if !slices.Contains(m.Consumes, c) {
t.Errorf("consumes %v lacks %s", m.Consumes, c)
}
}
if m.Version != "0.1" {
t.Errorf("version = %q, want MAJOR.MINOR 0.1 (§368, §411)", m.Version)
}
}
+40
View File
@@ -0,0 +1,40 @@
package main
import (
"context"
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-host/internal/store"
)
// The node agent's Stop stops a module's container from the node's record, asks the machine nothing else and leaves the
// record, so the module's kept data and its record survive (design 47 G A-98).
func TestTheNodeAgentsStopStopsAModuleAndRemovesNothing(t *testing.T) {
was := builtFor
builtFor = "arch"
t.Cleanup(func() { builtFor = was })
path := filepath.Join(t.TempDir(), "state.json")
var s store.State
s.Record(store.Applied{ID: "app.box", Type: "container", Target: "app-box"})
s.Record(store.Applied{ID: "other.box", Type: "container", Target: "other-box"})
if err := store.Save(path, s); err != nil {
t.Fatal(err)
}
var asked []string
opts := options{state: path, runner: func(_ context.Context, name string, args ...string) (string, error) {
asked = append(asked, name+" "+strings.Join(args, " "))
return "", nil
}}
if res := stopComposition(context.Background(), opts, "app"); res.Failed {
t.Fatalf("stop failed: %s", res.Detail)
}
if !strings.Contains(strings.Join(asked, "\n"), "stop app-box") || strings.Contains(strings.Join(asked, "\n"), "other-box") ||
strings.Contains(strings.Join(asked, "\n"), " rm ") {
t.Fatalf("the machine was asked %v", asked)
}
if after, err := store.Load(path); err != nil || len(after.Resources) != 2 {
t.Fatalf("the record changed: %v, %v", after.Resources, err)
}
}
+1 -1
View File
@@ -12,7 +12,7 @@ import (
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner" "git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine" "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-contracts/identity-and-access/go/clients/meshvaultclient" "git.novox.be/novox/mesh-contracts/identity-and-access/go/clients/meshvaultclient"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"github.com/novox/mesh-host/internal/buckets" "github.com/novox/mesh-host/internal/buckets"
"os" "os"
"path/filepath" "path/filepath"
+1 -1
View File
@@ -8,7 +8,7 @@ import (
"git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient" "git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner" "git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine" "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
engcomp "github.com/novox/mesh-host/internal/compositions" engcomp "github.com/novox/mesh-host/internal/compositions"
"os" "os"
"path/filepath" "path/filepath"
+13 -9
View File
@@ -8,16 +8,19 @@ require (
git.novox.be/novox/mesh-contracts/identity-and-access/go/clients/meshnodeenrollerclient v0.0.0-00010101000000-000000000000 git.novox.be/novox/mesh-contracts/identity-and-access/go/clients/meshnodeenrollerclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/identity-and-access/go/clients/meshvaultclient v0.0.0-00010101000000-000000000000 git.novox.be/novox/mesh-contracts/identity-and-access/go/clients/meshvaultclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/provisioning/go/clients/meshprovisionerclient v0.0.0-00010101000000-000000000000 git.novox.be/novox/mesh-contracts/provisioning/go/clients/meshprovisionerclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/contracts/go/clients/containerimageregistryclient v0.0.0-00010101000000-000000000000 git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/containerimageregistryclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient v0.0.0-00010101000000-000000000000 git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliveryclient v0.0.0-00010101000000-000000000000 git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshdeliveryclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/contracts/go/seats/containerimageregistry v0.0.0-00010101000000-000000000000 git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/containerimageregistry v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog v0.0.0-00010101000000-000000000000 git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/contracts/go/seats/meshwants v0.0.0-00010101000000-000000000000 git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/contracts/go/seats/ociartifactregistry v0.0.0-00010101000000-000000000000 git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/nodebuildgo v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/nodebuildimage v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/nodebuildnode v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/ociartifactregistry v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds v0.0.0-00010101000000-000000000000 git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-grants/grantsstate v0.0.0-00010101000000-000000000000 git.novox.be/novox/mesh-delivery/modules/mesh-grants/grantsstate v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/sdk v0.0.0-00010101000000-000000000000 git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-networking/modules/mesh-wireguard/go v0.0.0-00010101000000-000000000000 git.novox.be/novox/mesh-networking/modules/mesh-wireguard/go v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-sdk/go v0.0.0-00010101000000-000000000000 git.novox.be/novox/mesh-sdk/go v0.0.0-00010101000000-000000000000
github.com/nats-io/nats-server/v2 v2.11.17 github.com/nats-io/nats-server/v2 v2.11.17
@@ -29,6 +32,7 @@ require (
require ( require (
git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient v0.0.0-00010101000000-000000000000 git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/go/seats/nodeengine v0.0.0-00010101000000-000000000000 git.novox.be/novox/mesh-contracts/go/seats/nodeengine v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/go/seats/meshrecovery v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/health-and-repair/go/clients/meshconditionsclient v0.0.0-00010101000000-000000000000 git.novox.be/novox/mesh-contracts/health-and-repair/go/clients/meshconditionsclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshconditions v0.0.0-00010101000000-000000000000 git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshconditions v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/identity-and-access/go/seats/meshvault v0.0.0-00010101000000-000000000000 git.novox.be/novox/mesh-contracts/identity-and-access/go/seats/meshvault v0.0.0-00010101000000-000000000000
@@ -45,6 +49,6 @@ require (
) )
require ( require (
git.novox.be/novox/mesh-delivery/contracts/go/seats/meshdelivery v0.0.0-00010101000000-000000000000 // indirect git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshdelivery v0.0.0-00010101000000-000000000000 // indirect
git.novox.be/novox/mesh-networking/modules/mesh-connectivity-contracts/go/seats/theprivatenetwork v0.0.0-00010101000000-000000000000 // indirect git.novox.be/novox/mesh-networking/modules/mesh-connectivity-contracts/go/seats/theprivatenetwork v0.0.0-00010101000000-000000000000 // indirect
) )
+125 -8
View File
@@ -19,7 +19,7 @@ import (
"fmt" "fmt"
"git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient" "git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient"
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine" "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshwants" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants"
"github.com/novox/mesh-host/internal/compositions" "github.com/novox/mesh-host/internal/compositions"
"slices" "slices"
"sort" "sort"
@@ -75,6 +75,8 @@ type Have struct {
// PullWait is, on a failed have, the pull from a store its apply failed on and waits on the holder of (issue 1966 // PullWait is, on a failed have, the pull from a store its apply failed on and waits on the holder of (issue 1966
// ENG-9, nodeengine.HaveByNodeModuleValue.PullWait): read through the have, as node_haves shows it. // ENG-9, nodeengine.HaveByNodeModuleValue.PullWait): read through the have, as node_haves shows it.
PullWait *nodeengine.PullWait PullWait *nodeengine.PullWait
// Wait is the hold or wait the have says, in the recovery wait with its ask-at and act (design 47 G §560).
Wait *nodeengine.Wait
} }
// Haves is the node's have bucket as the agent uses it. // Haves is the node's have bucket as the agent uses it.
@@ -125,12 +127,28 @@ type Env struct {
Wants Watcher Wants Watcher
Composed Watcher Composed Watcher
Haves Haves Haves Haves
// WhyUnmade is why the node's composer composed nothing of a module at its want's fingerprint, from its last pass; empty
// when it says nothing. Nil says nothing. Waiting tells the composer the agent waits for the composition of a module at
// a fingerprint it holds none of, so a composer that believes it handed it hands it again. Nil tells nobody.
WhyUnmade func(module string) string
Waiting func(module, fp string)
// RunningManifest is the one place that answers the manifest of the build a module runs here (design 47 G A-484): the source
// that answered with its build, or, where none did, the words naming the three it tried. The tool composition says it. Nil
// says nothing.
RunningManifest func(module string) (source, fp, words string)
// Apply applies one module's composition (scope NodeModule for the node's own), given the compositions the node's // Apply applies one module's composition (scope NodeModule for the node's own), given the compositions the node's
// other modules run so a path, unit or group they hold is not removed or taken. It is called with one module at a // other modules run so a path, unit or group they hold is not removed or taken. It is called with one module at a
// time. // time.
Apply func(ctx context.Context, c compositions.Composed, others []compositions.Composed) Result Apply func(ctx context.Context, c compositions.Composed, others []compositions.Composed) Result
// Remove removes the module's own resources and nothing of another's. // Remove removes the module's own resources and nothing of another's.
Remove func(ctx context.Context, module string, others []compositions.Composed) Result Remove func(ctx context.Context, module string, others []compositions.Composed) Result
// Stop stops the module's units and containers and removes nothing, so what it kept survives: a revert stops the newer
// build with it, restores the module's kept data, and then applies the reverted build (design 47 G A-98, §297). Nil
// stops nothing.
Stop func(ctx context.Context, module string) Result
// Recovery takes a module's kept data as a restore point before a form change moves it and restores it at a revert
// (design 47 G §296, §297, §554); nil moves none.
Recovery *Recovery
// Backup backs a module up before a new composition of it is applied; nil backs up nothing. // Backup backs a module up before a new composition of it is applied; nil backs up nothing.
Backup func(ctx context.Context, module string) Backup Backup func(ctx context.Context, module string) Backup
// Kept is the resources the node last applied, by module, from the declaration it kept before the switch (ADR 0539 // Kept is the resources the node last applied, by module, from the declaration it kept before the switch (ADR 0539
@@ -241,6 +259,8 @@ type Agent struct {
// prior is a module the node ran before this agent started, so it has something to back up. // prior is a module the node ran before this agent started, so it has something to back up.
prior map[string]bool prior map[string]bool
said map[string]string said map[string]string
// waited is, by module, the want (fingerprint and revision) the agent said it waits for the composer for.
waited map[string]string
// notComposed is the modules whose have says the node could not compose them: why, and the have before (NotComposed). // notComposed is the modules whose have says the node could not compose them: why, and the have before (NotComposed).
notComposed map[string]notComposedMark notComposed map[string]notComposedMark
// heldAlone is the modules the composer holds alone (NotComposedEach): what runs of each in the node's own // heldAlone is the modules the composer holds alone (NotComposedEach): what runs of each in the node's own
@@ -306,7 +326,7 @@ func New(env Env) *Agent {
env.Say = func(string) {} env.Say = func(string) {}
} }
return &Agent{env: env, wants: map[string]wantNow{}, comps: map[string]compNow{}, running: map[string]nodeengine.HaveByNodeModuleValue{}, applied: map[string]compNow{}, had: map[string]compNow{}, keptData: map[string]bool{}, return &Agent{env: env, wants: map[string]wantNow{}, comps: map[string]compNow{}, running: map[string]nodeengine.HaveByNodeModuleValue{}, applied: map[string]compNow{}, had: map[string]compNow{}, keptData: map[string]bool{},
attempt: map[string]attempt{}, checked: map[string]bool{}, prior: map[string]bool{}, said: map[string]string{}, notComposed: map[string]notComposedMark{}, attempt: map[string]attempt{}, checked: map[string]bool{}, prior: map[string]bool{}, said: map[string]string{}, waited: map[string]string{}, notComposed: map[string]notComposedMark{},
pulls: map[string]*pullWait{}, settledAt: map[string]time.Time{}, settledSeq: map[string]int{}, pulls: map[string]*pullWait{}, settledAt: map[string]time.Time{}, settledSeq: map[string]int{},
retry: make(chan []string, 8), reconcile: make(chan []string, 64), failedAgain: make(chan string, 8), tries: map[string]int{}, again: make(chan againAsk, 64), retry: make(chan []string, 8), reconcile: make(chan []string, 64), failedAgain: make(chan string, 8), tries: map[string]int{}, again: make(chan againAsk, 64),
transient: make(chan struct{}, 1), stopped: make(chan struct{}), quiet: make(chan chan struct{})} transient: make(chan struct{}, 1), stopped: make(chan struct{}), quiet: make(chan chan struct{})}
@@ -497,6 +517,9 @@ const NotComposedDetail = "the node cannot compose it, and keeps running what it
// composerMark says h is a mark NotComposed wrote: failed, its detail opening with NotComposedDetail. // composerMark says h is a mark NotComposed wrote: failed, its detail opening with NotComposedDetail.
func composerMark(h Have) bool { func composerMark(h Have) bool {
if h.Phase == string(nodeengineclient.PhaseHeld) && heldForEntry(h.Detail) {
return true
}
return h.Phase == string(nodeengineclient.PhaseFailed) && strings.HasPrefix(h.Detail, NotComposedDetail) return h.Phase == string(nodeengineclient.PhaseFailed) && strings.HasPrefix(h.Detail, NotComposedDetail)
} }
@@ -640,11 +663,26 @@ func (a *Agent) NotComposedEach(ctx context.Context, why string, held map[string
a.mu.Lock() a.mu.Lock()
a.notComposed[module] = mark a.notComposed[module] = mark
a.mu.Unlock() a.mu.Unlock()
phase, detail := string(nodeengineclient.PhaseFailed), NotComposedDetail+why
var wait *nodeengine.Wait
if heldForEntry(why) {
// The entry its want names is not held: the phase held, its detail opening entry-asked: or entry-missing: and
// naming the fingerprint (design 47 G section 421).
phase, detail = string(nodeengineclient.PhaseHeld), why
// The hold says what it waits on, as every hold but a failed backup does (design 47 G A-453, section 541).
wait = a.composeWait(strings.TrimSuffix(heldReason(why), ":"), why)
}
a.writeHave(ctx, Have{Module: module, FP: mark.prior.FP, Composed: mark.prior.Composed, Sealed: mark.prior.Sealed, a.writeHave(ctx, Have{Module: module, FP: mark.prior.FP, Composed: mark.prior.Composed, Sealed: mark.prior.Sealed,
Phase: string(nodeengineclient.PhaseFailed), Detail: NotComposedDetail + why}) Phase: phase, Detail: detail, Wait: wait})
} }
} }
// composeWait is the wait a hold of the composition carries in its have: step compose, with its reason and its words (design 47 G
// A-453, section 541). Only a hold for a failed backup carries none.
func (a *Agent) composeWait(reason, words string) *nodeengine.Wait {
return &nodeengine.Wait{Owner: nodeengine.Seat, Step: nodeengine.WaitStepCompose, Reason: reason, Since: a.now(), Words: words}
}
// writeHave writes a module's have (the node's own key for NodeModule); on a node composing its own, with the sealed // writeHave writes a module's have (the node's own key for NodeModule); on a node composing its own, with the sealed
// values of the composition it names when the caller gave none (Have.Sealed): those of the composition the agent applied or holds at that digest, else // values of the composition it names when the caller gave none (Have.Sealed): those of the composition the agent applied or holds at that digest, else
// those the module's last have said at that digest. // those the module's last have said at that digest.
@@ -1319,6 +1357,31 @@ func (a *Agent) HoldsNode() bool {
return ok && h.Phase != nodeengineclient.PhaseRemoved return ok && h.Phase != nodeengineclient.PhaseRemoved
} }
// waitsForComposer says once per want and reason that the agent waits for the node's own composition of module at fp, and
// why the composer made none (Env.WhyUnmade), and tells the composer (Env.Waiting) so it hands again what it believes it
// handed. Called without a.mu held.
func (a *Agent) waitsForComposer(module, fp string, rev uint64) {
why := "the composer has made none yet"
if a.env.WhyUnmade != nil {
if w := a.env.WhyUnmade(module); w != "" {
why = w
}
}
line := fmt.Sprintf("%s waits for the composer: no composition of it at %s: %s", module, fp, why)
// Said and told once per want (its fingerprint and revision), whatever the reason says; forgotten once composed.
at := fmt.Sprintf("%s@%d", fp, rev)
a.mu.Lock()
first := a.waited[module] != at
if first {
a.waited[module] = at
a.env.Say(line)
}
a.mu.Unlock()
if first && a.env.Waiting != nil {
a.env.Waiting(module, fp)
}
}
func (a *Agent) settleOne(ctx context.Context, module string) { func (a *Agent) settleOne(ctx context.Context, module string) {
a.mu.Lock() a.mu.Lock()
w := a.wants[module] w := a.wants[module]
@@ -1334,7 +1397,15 @@ func (a *Agent) settleOne(ctx context.Context, module string) {
comp, have := a.compositionFor(module, w.Want.Fp) comp, have := a.compositionFor(module, w.Want.Fp)
a.mu.Unlock() a.mu.Unlock()
if a.env.SelfComposed { if a.env.SelfComposed {
if have {
a.mu.Lock()
delete(a.waited, module)
a.mu.Unlock()
}
if !have || comp.Err != nil { if !have || comp.Err != nil {
if !have {
a.waitsForComposer(module, w.Want.Fp, w.Rev)
}
return // waits for the node's own composition of the want's build return // waits for the node's own composition of the want's build
} }
// The composition the node composed for the want's build is the one it applies (ADR 0547 §2). // The composition the node composed for the want's build is the one it applies (ADR 0547 §2).
@@ -1372,7 +1443,9 @@ func (a *Agent) settleOne(ctx context.Context, module string) {
} }
a.mu.Lock() a.mu.Lock()
r, runs := a.running[module] r, runs := a.running[module]
runsWant := runs && r.Fp == w.Want.Fp && r.Composed == w.Want.Composed && r.Phase != nodeengineclient.PhaseRemoved // A module whose last apply failed runs no composition whole, whatever it ran before (ranFailed).
runsWant := runs && r.Fp == w.Want.Fp && r.Composed == w.Want.Composed && r.Phase != nodeengineclient.PhaseRemoved &&
r.Phase != nodeengineclient.PhaseFailed
att, tried := a.attempt[module] att, tried := a.attempt[module]
same := tried && att.FP == w.Want.Fp && att.Composed == w.Want.Composed same := tried && att.FP == w.Want.Fp && att.Composed == w.Want.Composed
if same && att.Held { if same && att.Held {
@@ -1381,7 +1454,8 @@ func (a *Agent) settleOne(ctx context.Context, module string) {
// be swallowed by "it runs what its want names" (issue 1585, the second failure). The operator's answer, again or // be swallowed by "it runs what its want names" (issue 1585, the second failure). The operator's answer, again or
// skip, in a revision after the one the hold took, is acted on whatever runs; any other new revision of the want // skip, in a revision after the one the hold took, is acted on whatever runs; any other new revision of the want
// is the answer only for a move (ADR 0568 §4), as before. // is the answer only for a move (ADR 0568 §4), as before.
answered := w.Rev > att.AnsweredRev && (w.Want.Backup == meshwants.WantsByWantNodeModuleValueBackupAgain || w.Want.Backup == meshwants.WantsByWantNodeModuleValueBackupSkip) answered := w.Rev > att.AnsweredRev && (w.Want.Backup == meshwants.WantsByWantNodeModuleValueBackupAgain || w.Want.Backup == meshwants.WantsByWantNodeModuleValueBackupSkip ||
w.Want.Revert == meshwants.WantsByWantNodeModuleValueRevertLive)
if !answered && (runsWant || (att.AnsweredRev >= w.Rev && att.CompRev == comp.Rev)) { if !answered && (runsWant || (att.AnsweredRev >= w.Rev && att.CompRev == comp.Rev)) {
a.sayOnce("held:"+module, fmt.Sprintf("%s is held for a failed backup before %s: it waits for the operator's "+ a.sayOnce("held:"+module, fmt.Sprintf("%s is held for a failed backup before %s: it waits for the operator's "+
"answer, and its want (revision %d) carries none since the hold (revision %d)", module, w.Want.Fp, w.Rev, att.AnsweredRev)) "answer, and its want (revision %d) carries none since the hold (revision %d)", module, w.Want.Fp, w.Rev, att.AnsweredRev))
@@ -1469,7 +1543,7 @@ func (a *Agent) baseline(ctx context.Context, module string, w wantNow, comp com
a.mu.Unlock() a.mu.Unlock()
return false return false
} }
if h.Phase == string(nodeengineclient.PhaseHeld) { if h.Phase == string(nodeengineclient.PhaseHeld) && !judgedAgain(h) {
// Held for a failed backup, and the operator's answer is still to come: the hold is kept, nothing is // Held for a failed backup, and the operator's answer is still to come: the hold is kept, nothing is
// backed up again or applied, and the next revision of the want is the answer (ADR 0568 §4). Said, so // backed up again or applied, and the next revision of the want is the answer (ADR 0568 §4). Said, so
// why it holds is read from the journal and not guessed (issue 1585: this branch was silent). // why it holds is read from the journal and not guessed (issue 1585: this branch was silent).
@@ -1559,6 +1633,30 @@ func (a *Agent) apply(ctx context.Context, module string, w wantNow, comp compNo
ranBefore = ranBefore || a.prior[module] ranBefore = ranBefore || a.prior[module]
a.mu.Unlock() a.mu.Unlock()
// A want with parts is applied only when they and the digests its composition fetches agree (design 47 G §524): the
// module keeps running what it has, and the have says which digest differs.
if words := partsDifference(w.Want.Parts, comp.C.Resources, comp.C.Templates); words != "" {
a.mu.Lock()
prev, runs := a.running[module]
a.attempt[module] = attempt{FP: fp, Composed: dig, WantRev: w.Rev, CompRev: comp.Rev, Held: true, AnsweredRev: w.Rev}
a.mu.Unlock()
if !runs {
if h, found, _ := a.readHave(ctx, module); found {
prev = nodeengine.HaveByNodeModuleValue{Fp: h.FP, Composed: h.Composed}
}
}
a.env.Say(fmt.Sprintf("%s is held: %s", module, words))
a.writeHave(ctx, Have{Module: module, FP: prev.Fp, Composed: prev.Composed, Phase: string(nodeengineclient.PhaseHeld),
Detail: words, Wait: a.composeWait(strings.TrimSuffix(PartsDiffer, ":"), words)})
return
}
// A kept data on the bus whose form changes is taken as a restore point first, or restored at a revert (design 47 G §296, §297).
held, catchUps := a.keptStep(ctx, module, w, comp, ranBefore, others)
if held {
return
}
// A data module is backed up before a new composition of it moves (ADR 0429 §14), unless the want says to go on // A data module is backed up before a new composition of it moves (ADR 0429 §14), unless the want says to go on
// without one (ADR 0539 §16). // without one (ADR 0539 §16).
if comp.C.Data && ranBefore && a.env.Backup != nil && w.Want.Backup != meshwants.WantsByWantNodeModuleValueBackupSkip { if comp.C.Data && ranBefore && a.env.Backup != nil && w.Want.Backup != meshwants.WantsByWantNodeModuleValueBackupSkip {
@@ -1586,9 +1684,14 @@ func (a *Agent) apply(ctx context.Context, module string, w wantNow, comp compNo
a.mu.Lock() a.mu.Lock()
a.attempt[module] = attempt{FP: fp, Composed: dig, WantRev: w.Rev, CompRev: comp.Rev, Held: true, AnsweredRev: w.Rev} a.attempt[module] = attempt{FP: fp, Composed: dig, WantRev: w.Rev, CompRev: comp.Rev, Held: true, AnsweredRev: w.Rev}
a.mu.Unlock() a.mu.Unlock()
// It keeps running what it has: the have says what it holds and why it waits. // It keeps running what it has: the have says what it holds and why it waits. A backup interrupted is no failure of it, so
// its hold waits as kept.go's do, with the words as the reason; only a failed backup carries no wait (A-453).
var wait *nodeengine.Wait
if words == BackupInterrupted {
wait = a.composeWait(words, words)
}
a.writeHave(ctx, Have{Module: module, FP: prev.Fp, Composed: prev.Composed, Phase: string(nodeengineclient.PhaseHeld), a.writeHave(ctx, Have{Module: module, FP: prev.Fp, Composed: prev.Composed, Phase: string(nodeengineclient.PhaseHeld),
Detail: words}) Detail: words, Wait: wait})
return return
} }
} }
@@ -1598,6 +1701,9 @@ func (a *Agent) apply(ctx context.Context, module string, w wantNow, comp compNo
if !res.Failed || res.Transient { if !res.Failed || res.Transient {
delete(a.pulls, module) delete(a.pulls, module)
} }
if res.Failed {
a.ranFailed(module)
}
what := fmt.Sprintf("%s at %s", module, fp) what := fmt.Sprintf("%s at %s", module, fp)
if module == NodeModule { if module == NodeModule {
what = "the node's own composition" what = "the node's own composition"
@@ -1660,6 +1766,17 @@ func (a *Agent) apply(ctx context.Context, module string, w wantNow, comp compNo
a.env.Settled(module) a.env.Settled(module)
} }
a.writeHave(ctx, Have{Module: module, FP: fp, Composed: dig, Phase: string(nodeengineclient.PhaseRunning), Detail: res.Detail}) a.writeHave(ctx, Have{Module: module, FP: fp, Composed: dig, Phase: string(nodeengineclient.PhaseRunning), Detail: res.Detail})
a.catchUp(ctx, module, catchUps)
}
// ranFailed marks what runs of a module as failed once an apply of it failed (novox/hq issue 1966, 2026-11-08): a failed
// apply may have replaced some of its resources (distribution's store container) before it stopped, so what ran is no
// longer whole, and a want of it again (a put back) is applied rather than taken as running. Called with a.mu held.
func (a *Agent) ranFailed(module string) {
if r, ok := a.running[module]; ok && r.Phase != nodeengineclient.PhaseRemoved {
r.Phase = nodeengineclient.PhaseFailed
a.running[module] = r
}
} }
func (a *Agent) remove(ctx context.Context, module string, w wantNow) { func (a *Agent) remove(ctx context.Context, module string, w wantNow) {
+211 -11
View File
@@ -6,8 +6,8 @@ import (
"fmt" "fmt"
"git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient" "git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient"
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine" "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshwants" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants"
"github.com/novox/mesh-host/internal/buckets/bucketstest" "github.com/novox/mesh-host/internal/buckets/bucketstest"
"github.com/novox/mesh-host/internal/compositions" "github.com/novox/mesh-host/internal/compositions"
"github.com/novox/mesh-host/internal/judge" "github.com/novox/mesh-host/internal/judge"
@@ -322,6 +322,88 @@ func TestAWantAndItsCompositionApplyTheModuleAndTheHaveNamesBoth(t *testing.T) {
if got := b.appliedList(); len(got) != 1 { if got := b.appliedList(); len(got) != 1 {
t.Fatalf("a want naming another digest applied the composition: %v", got) t.Fatalf("a want naming another digest applied the composition: %v", got)
} }
// Design 47 G section 524: a want with parts is applied only when its parts and the digests the composition fetches
// agree; a want with no parts is applied as before.
d1, d2, d3 := "sha256:"+strings.Repeat("1", 64), "sha256:"+strings.Repeat("2", 64), "sha256:"+strings.Repeat("3", 64)
archive := func(id, digest string) json.RawMessage {
raw, _ := json.Marshal(map[string]any{"id": "web." + id, "type": "archive", "path": "/tmp/r6/web/" + id,
"source": "https://store/" + id, "digest": digest, "module": "web"})
return raw
}
for i, row := range []struct {
name string
parts map[string]string
applies bool
detail string
}{
{"agree", map[string]string{"web:a": d1, "web:b": d2}, true, ""},
{"no parts", nil, true, ""},
{"fetched outside", map[string]string{"web:a": d1}, false, PartsDiffer + " fetched and not in the want's parts: " + d2},
{"part not fetched", map[string]string{"web:a": d1, "web:b": d2, "web:c": d3}, false,
PartsDiffer + " in the want's parts and not fetched: " + d3},
} {
before := len(b.appliedList())
fpn := fpOf(string("5678"[i]))
ckn, crawn, dign := composition(t, "web", fpn, false, archive("a", d1), archive("b", d2))
b.comps.put(ckn, crawn)
wkn := meshwants.WantsByWantNodeModuleKey(node, "web")
wrawn, err := bucketstest.Want(meshwants.WantsByWantNodeModuleValue{Fp: fpn, Composed: dign, Parts: row.parts})
if err != nil {
t.Fatal(err)
}
b.wants.put(wkn, wrawn)
b.idle()
if got := len(b.appliedList()) - before; (got == 1) != row.applies {
t.Fatalf("parts %q: applied %d times, want applies=%v", row.name, got, row.applies)
}
if h := b.haves.last("web"); !row.applies && (h.Phase != string(nodeengineclient.PhaseHeld) || h.Detail != row.detail ||
h.Wait == nil || h.Wait.Step != nodeengine.WaitStepCompose || h.Wait.Reason != "parts-differ" || h.Wait.Words != row.detail) {
t.Fatalf("parts %q: the have is %s %q wait %+v, want held %q waiting on parts-differ (A-453)", row.name, h.Phase, h.Detail, h.Wait, row.detail)
}
}
// A container's digest is the one after its image's @.
container := json.RawMessage(`{"id":"web.c","type":"container","name":"web-c","image":"store/web@` + d3 + `","image-seat":"mesh-artifact-store","image-holder":"n1","module":"web"}`)
if got := partsDifference(map[string]string{"web:c": d3}, []json.RawMessage{container}, nil); got != "" {
t.Fatalf("a container whose digest is in the parts differs: %q", got)
}
// A-454: an image the manifest pins from outside, by its digest, is no part of the want's parts, and a mesh-built one still is.
upstream := json.RawMessage(`{"id":"web.u","type":"container","name":"web-u","image":"registry@` + d2 + `","module":"web"}`)
if got := partsDifference(map[string]string{"web:c": d3}, []json.RawMessage{container, upstream}, nil); got != "" {
t.Fatalf("an upstream image pinned by its digest is a part: %q", got)
}
// And neither is the registry seat's copy of an upstream image, though it is routed through the seat.
mirrored := json.RawMessage(`{"id":"web.m","type":"container","name":"web-m","image":"store:5000/upstream/docker.io/library/registry@` + d1 + `","image-seat":"container-image-registry","image-holder":"n1","module":"web"}`)
if got := partsDifference(map[string]string{"web:c": d3}, []json.RawMessage{container, mirrored}, nil); got != "" {
t.Fatalf("an upstream copy served through the registry seat is a part: %q", got)
}
if got := partsDifference(map[string]string{"web:c": d3}, []json.RawMessage{upstream}, nil); !strings.Contains(got, "in the want's parts and not fetched: "+d3) || strings.Contains(got, d2) {
t.Fatalf("with only an upstream image the mesh-built one is not missing, or the upstream one is named: %q", got)
}
// A-454a: a mesh-built image whose seat holder is not on record carries no image-seat; its digest is in the parts, so it is fetched.
noSeat := json.RawMessage(`{"id":"web.n","type":"container","name":"web-n","image":"store/web@` + d3 + `","module":"web"}`)
if got := partsDifference(map[string]string{"web:c": d3}, []json.RawMessage{noSeat}, nil); got != "" {
t.Fatalf("a mesh-built image with no image-seat, named by the parts, is not fetched: %q", got)
}
if got := partsDifference(map[string]string{"web:c": d3}, []json.RawMessage{noSeat, upstream}, nil); got != "" {
t.Fatalf("an upstream image beside it is a part: %q", got)
}
// A-481: a templates archive the composition was rendered from is fetched, one the want's parts do not name is a part
// the composition lacks, and the want's digest nothing read stays not fetched.
if got := partsDifference(map[string]string{"web:c": d3, "web:files": d1}, []json.RawMessage{container}, []string{d1}); got != "" {
t.Fatalf("a templates archive named by the parts and rendered from is fetched: %q", got)
}
if got := partsDifference(map[string]string{"web:c": d3, "web:files": d1}, []json.RawMessage{container}, nil); !strings.Contains(got, "in the want's parts and not fetched: "+d1) {
t.Fatalf("a templates archive nothing read stays not fetched: %q", got)
}
if got := partsDifference(map[string]string{"web:c": d3}, []json.RawMessage{container}, []string{d1}); !strings.Contains(got, "fetched and not in the want's parts: "+d1) {
t.Fatalf("a templates archive outside the parts differs: %q", got)
}
// A-432: a runtime of its own runs the tool runner's program, which the module's parts do not name.
ownRuntime := json.RawMessage(`{"id":"web.own-runtime","type":"process","name":"web-runtime","source":"store/node-tools","digest":"` + d1 + `","run":["./node-tools"],"module":"web"}`)
if got := partsDifference(map[string]string{"web:c": d3}, []json.RawMessage{container, ownRuntime}, nil); got != "" {
t.Fatalf("a runtime of its own differs from the module's parts: %q", got)
}
} }
// R6-A AC2: given a kept declaration and wants and compositions equal to it, when the agent starts, nothing is applied // R6-A AC2: given a kept declaration and wants and compositions equal to it, when the agent starts, nothing is applied
@@ -487,6 +569,45 @@ func TestAFailedApplyIsRetriedOnlyByAChangeOfEitherKey(t *testing.T) {
t.Errorf("a module running its want was applied again (%d applies)", n) t.Errorf("a module running its want was applied again (%d applies)", n)
} }
}) })
// Issue 1966 (2026-11-08 23:47 UTC): a failed apply of distribution's new build had replaced its store container, and
// the put back to the build that ran before applied nothing, judged as running. A module whose apply failed runs no
// composition whole: the want of what ran before is applied again.
t.Run("1966: after a failed apply, the composition that ran is applied again when wanted", func(t *testing.T) {
b := newBed(t)
stop := b.run()
defer stop()
fpA, fpB := fpOf("a"), fpOf("b")
ckA, crawA, digA := composition(t, "web", fpA, false, file("web", "x", "1"))
ckB, crawB, digB := composition(t, "web", fpB, false, file("web", "x", "2"))
b.comps.put(ckA, crawA)
b.comps.put(ckB, crawB)
wk, wrawA := wantFor(t, "web", fpA, digA, "")
_, wrawB := wantFor(t, "web", fpB, digB, "")
b.wants.put(wk, wrawA)
b.idle()
if n := len(b.appliedList()); n != 1 || b.haves.last("web").Phase != string(nodeengineclient.PhaseRunning) {
t.Fatalf("the first build was not applied and running (%d applies, have %+v)", n, b.haves.last("web"))
}
b.mu.Lock()
b.failWith = "distribution.program: the store does not answer"
b.mu.Unlock()
b.wants.put(wk, wrawB)
b.idle()
if n := len(b.appliedList()); n != 2 || b.haves.last("web").Phase != string(nodeengineclient.PhaseFailed) {
t.Fatalf("the second build's apply did not fail (%d applies, have %+v)", n, b.haves.last("web"))
}
b.mu.Lock()
b.failWith = ""
b.mu.Unlock()
b.wants.put(wk, wrawA) // the put back
b.idle()
if n := len(b.appliedList()); n != 3 {
t.Fatalf("the put back to the build that ran applied nothing (%d applies): a failed apply left it said as running", n)
}
if h := b.haves.last("web"); h.FP != fpA || h.Composed != digA || h.Phase != string(nodeengineclient.PhaseRunning) {
t.Errorf("after the put back the have is %+v, want the first build running", h)
}
})
// design 51 (ENG-8, ENG-11): a pull from a store that failed is tried again when the store's holder settles, once per // design 51 (ENG-8, ENG-11): a pull from a store that failed is tried again when the store's holder settles, once per
// settled, never on a clock; the have says whom it waits on and what triggered the last retry. // settled, never on a clock; the have says whom it waits on and what triggered the last retry.
t.Run("a pull from a store waits for its holder to settle", func(t *testing.T) { t.Run("a pull from a store waits for its holder to settle", func(t *testing.T) {
@@ -666,7 +787,8 @@ func TestADataModuleIsBackedUpFirstAndHeldWhenItFails(t *testing.T) {
}) })
eventually(t, "the have does not say held", func() bool { eventually(t, "the have does not say held", func() bool {
h := b.haves.last("db") h := b.haves.last("db")
return h.Phase == string(nodeengineclient.PhaseHeld) && strings.Contains(h.Detail, "repository is locked") && h.FP == fp1 // A hold for a failed backup carries no wait: the operator is asked about it (A-453).
return h.Phase == string(nodeengineclient.PhaseHeld) && strings.Contains(h.Detail, "repository is locked") && h.FP == fp1 && h.Wait == nil
}) })
if got := b.appliedList(); len(got) != 0 { if got := b.appliedList(); len(got) != 0 {
t.Fatalf("something was applied behind a failed backup: %v", got) t.Fatalf("something was applied behind a failed backup: %v", got)
@@ -725,8 +847,9 @@ func TestADataModuleIsBackedUpFirstAndHeldWhenItFails(t *testing.T) {
stopI := ib.run() stopI := ib.run()
<-began <-began
stopI() stopI()
if h := ib.haves.last("db"); h.Phase != string(nodeengineclient.PhaseHeld) || h.FP != fp1 || h.Composed != dig1 || h.Detail != BackupInterrupted { if h := ib.haves.last("db"); h.Phase != string(nodeengineclient.PhaseHeld) || h.FP != fp1 || h.Composed != dig1 || h.Detail != BackupInterrupted ||
t.Errorf("held after an interrupted backup: %+v", h) h.Wait == nil || h.Wait.Step != nodeengine.WaitStepCompose || h.Wait.Reason != BackupInterrupted {
t.Errorf("held after an interrupted backup, with its wait (A-453): %+v", h)
} }
// At a start: a backup the node agent's own stop interrupted (postgres on novox, 2026-10-31) is no failure, so it is // At a start: a backup the node agent's own stop interrupted (postgres on novox, 2026-10-31) is no failure, so it is
@@ -736,16 +859,28 @@ func TestADataModuleIsBackedUpFirstAndHeldWhenItFails(t *testing.T) {
detail, backup string detail, backup string
backups, applied int backups, applied int
phase string phase string
namesNothing bool // the have names no fp and no composition namesNothing bool // the have names no fp and no composition
reason string // the reason of the wait the have carries; empty: it carries none
}{ }{
{BackupInterrupted, meshwants.WantsByWantNodeModuleValueBackupAgain, 1, 1, nodeengineclient.PhaseRunning, false}, {BackupInterrupted, meshwants.WantsByWantNodeModuleValueBackupAgain, 1, 1, nodeengineclient.PhaseRunning, false, ""},
// novox's postgres: its held have, written before mesh-host #189, names nothing it runs. // novox's postgres: its held have, written before mesh-host #189, names nothing it runs.
{BackupInterrupted, "", 1, 1, nodeengineclient.PhaseRunning, true}, {BackupInterrupted, "", 1, 1, nodeengineclient.PhaseRunning, true, ""},
// A backup that failed, and the want carries the operator's answer, written before this start (nats on novox, // A backup that failed, and the want carries the operator's answer, written before this start (nats on novox,
// 2026-11-04, issue 1585): again backs it up and applies, skip applies without one; no answer keeps the hold. // 2026-11-04, issue 1585): again backs it up and applies, skip applies without one; no answer keeps the hold.
{"restic: the repository is locked", meshwants.WantsByWantNodeModuleValueBackupAgain, 1, 1, nodeengineclient.PhaseRunning, false}, {"restic: the repository is locked", meshwants.WantsByWantNodeModuleValueBackupAgain, 1, 1, nodeengineclient.PhaseRunning, false, ""},
{"restic: the repository is locked", meshwants.WantsByWantNodeModuleValueBackupSkip, 0, 1, nodeengineclient.PhaseRunning, false}, {"restic: the repository is locked", meshwants.WantsByWantNodeModuleValueBackupSkip, 0, 1, nodeengineclient.PhaseRunning, false, ""},
{"restic: the repository is locked", "", 0, 0, nodeengineclient.PhaseHeld, false}, {"restic: the repository is locked", "", 0, 0, nodeengineclient.PhaseHeld, false, ""},
// A hold the node-engine judges again (novox's indexer, 2026-11-16): its parts differed or its entry was not
// held, which no answer of the operator's releases; at a start it is judged again, here backed up and applied.
{PartsDiffer + " fetched and not in the want's parts: sha256:a03131c6", "", 1, 1, nodeengineclient.PhaseRunning, false, "parts-differ"},
{EntryMissing + " the catalog half holds no entry of the build", "", 1, 1, nodeengineclient.PhaseRunning, false, "entry-missing"},
// g14's feh, 2026-11-18: the kept step's hold opens "whether ..." and says its reason in its wait; the words decide nothing.
{"whether feh's kept data changes form is not known", "", 1, 1, nodeengineclient.PhaseRunning, false, "entry-missing"},
{EntryAsked + " the entry is asked", "", 1, 1, nodeengineclient.PhaseRunning, false, "entry-asked"},
// Opening words alone are no reason: a failed backup's detail that opens with them waits for the answer.
{PartsDiffer + " as the words of a failed backup", "", 0, 0, nodeengineclient.PhaseHeld, false, ""},
// A restore point not taken waits for the answer, its reason being its words.
{"the restore point of db's data was not taken", "", 0, 0, nodeengineclient.PhaseHeld, false, "the restore point of db's data was not taken"},
} { } {
r := newBed(t) r := newBed(t)
r.comps.before(ck1, craw1) r.comps.before(ck1, craw1)
@@ -753,6 +888,9 @@ func TestADataModuleIsBackedUpFirstAndHeldWhenItFails(t *testing.T) {
wk, wraw := wantFor(t, "db", fp2, dig2, row.backup) wk, wraw := wantFor(t, "db", fp2, dig2, row.backup)
r.wants.before(wk, wraw) r.wants.before(wk, wraw)
held := Have{Module: "db", FP: fp1, Composed: dig1, Phase: string(nodeengineclient.PhaseHeld), Detail: row.detail} held := Have{Module: "db", FP: fp1, Composed: dig1, Phase: string(nodeengineclient.PhaseHeld), Detail: row.detail}
if row.reason != "" {
held.Wait = &nodeengine.Wait{Owner: nodeengine.Seat, Step: nodeengine.WaitStepCompose, Reason: row.reason, Words: row.detail}
}
if row.namesNothing { if row.namesNothing {
held.FP, held.Composed = "", "" held.FP, held.Composed = "", ""
} }
@@ -847,6 +985,7 @@ func TestADataModuleIsBackedUpFirstAndHeldWhenItFails(t *testing.T) {
// A data module with a good backup is applied after it; one that never ran has nothing to back up. // A data module with a good backup is applied after it; one that never ran has nothing to back up.
func TestABackupDoneIsFollowedByTheApplyAndANewModuleIsNotBackedUp(t *testing.T) { func TestABackupDoneIsFollowedByTheApplyAndANewModuleIsNotBackedUp(t *testing.T) {
t.Run("a move across a kept data's form change is taken as a restore point and restored at a revert", testAMoveAcrossAFormChange)
b := newBed(t) b := newBed(t)
stop := b.run() stop := b.run()
defer stop() defer stop()
@@ -1517,6 +1656,11 @@ func TestTheNodesOwnCompositionIsAppliedWhenItDiffersAndARecordMoveWaitsForAYes(
history[key] = append(history[key], Update{Key: key, Value: raw, Revision: uint64(len(history[key]) + 1)}) history[key] = append(history[key], Update{Key: key, Value: raw, Revision: uint64(len(history[key]) + 1)})
} }
b.env.Held = func() map[string]string { return map[string]string{"mesh-build-go": "its binding is unset"} } b.env.Held = func() map[string]string { return map[string]string{"mesh-build-go": "its binding is unset"} }
// Issue 1966 (novox, 2026-11-09): a want the composer made no composition of is said once, with why, and the composer told.
var toldMu sync.Mutex
var told []string
b.env.WhyUnmade = func(string) string { return "no manifest in the composer's picture" }
b.env.Waiting = func(module, fp string) { toldMu.Lock(); told = append(told, module+"@"+fp); toldMu.Unlock() }
stop := b.run() stop := b.run()
defer stop() defer stop()
unsigned := func(module, fp string, rs ...json.RawMessage) (string, []byte, string) { unsigned := func(module, fp string, rs ...json.RawMessage) (string, []byte, string) {
@@ -1544,6 +1688,51 @@ func TestTheNodesOwnCompositionIsAppliedWhenItDiffersAndARecordMoveWaitsForAYes(
b.comps.put(ck, craw) b.comps.put(ck, craw)
want("web", fp, "") want("web", fp, "")
eventually(t, "the node's own composition was not applied", func() bool { return len(b.appliedList()) == 1 }) eventually(t, "the node's own composition was not applied", func() bool { return len(b.appliedList()) == 1 })
gk := meshwants.WantsByWantNodeModuleKey(node, "ghost")
graw, _ := bucketstest.Want(meshwants.WantsByWantNodeModuleValue{Fp: fpOf("9"), Self: true})
b.wants.put(gk, graw)
line := "ghost waits for the composer: no composition of it at " + fpOf("9") + ": no manifest in the composer's picture"
saidCount := func() (n int) {
b.mu.Lock()
defer b.mu.Unlock()
for _, l := range b.said {
if l == line {
n++
}
}
return n
}
eventually(t, "the wait for the composer was not said", func() bool { return saidCount() == 1 })
// Settled again by another want's change, it is neither said nor told again.
other, oraw, _ := unsigned("ghost", fpOf("8"), file("ghost", "x", "1"))
b.comps.put(other, oraw)
eventually(t, "ghost's other composition was not taken", func() bool {
b.ag.mu.Lock()
defer b.ag.mu.Unlock()
_, ok := b.ag.comps[other]
return ok
})
// What is settled after it: web's want moves on, so ghost's settle, queued before it, has run.
want("web", fp, "")
eventually(t, "the composer was not told once", func() bool { toldMu.Lock(); defer toldMu.Unlock(); return slices.Contains(told, "ghost@"+fpOf("9")) })
if _, err := b.ag.Composition(context.Background(), "ghost", "", false); err == nil || !strings.Contains(err.Error(), "waits for the composer: no manifest in the composer's picture") {
t.Errorf("the composition read of ghost says %v, want why the composer made none", err)
}
// Where the one place says what runs of it (A-484), the tool answers it though the want's build has no composition.
b.ag.env.RunningManifest = func(string) (string, string, string) {
_ = b.ag.Running() // the one place reads what the agent runs: the tool must not hold the agent's lock while it does
return "the module's trunk manifest in the catalog", "src2:t", "the manifest of the build ghost runs here is read from the module's trunk manifest in the catalog"
}
if view, err := b.ag.Composition(context.Background(), "ghost", "", false); err != nil || view.RunsManifest == nil ||
view.RunsManifest.Source != "the module's trunk manifest in the catalog" || !strings.Contains(view.Composition.Refused, "waits for the composer") {
t.Errorf("the composition read of ghost says %+v, %v, want what runs of it and why the composer made none", view.RunsManifest, err)
}
b.ag.env.RunningManifest = nil
toldMu.Lock()
if n := slices.Index(told, "ghost@"+fpOf("9")); saidCount() != 1 || n < 0 || slices.Contains(told[n+1:], "ghost@"+fpOf("9")) {
t.Errorf("said %d times, told %q; want once and ghost told once", saidCount(), told)
}
toldMu.Unlock()
eventually(t, "the have does not name the composed digest", func() bool { return b.haves.last("web").Composed == dig }) eventually(t, "the have does not name the composed digest", func() bool { return b.haves.last("web").Composed == dig })
// ADR 0601 §4: what the composer keeps of web as it runs is carried on its have, and goes once it keeps none. // ADR 0601 §4: what the composer keeps of web as it runs is carried on its have, and goes once it keeps none.
b.ag.KeptNow(context.Background(), map[string][]string{"web": {"web runs the tool runner's process here"}}) b.ag.KeptNow(context.Background(), map[string][]string{"web": {"web runs the tool runner's process here"}})
@@ -1649,6 +1838,17 @@ func TestTheNodesOwnCompositionIsAppliedWhenItDiffersAndARecordMoveWaitsForAYes(
if h := b.haves.last("web"); h.Phase != string(nodeengineclient.PhaseRunning) || h.Composed != dig { if h := b.haves.last("web"); h.Phase != string(nodeengineclient.PhaseRunning) || h.Composed != dig {
t.Errorf("web, which composes, is marked beside idle: %+v", h) t.Errorf("web, which composes, is marked beside idle: %+v", h)
} }
// Design 47 G §421: a module whose want's entry the catalog half holds none of is the phase held, its detail opening
// entry-missing: and naming the fingerprint; composed again, its have is what it runs.
// A-453: the hold carries its wait, step compose with the reason, as every hold but a failed backup does.
for _, row := range []struct{ opening, reason string }{{EntryMissing, "entry-missing"}, {EntryAsked, "entry-asked"}} {
words := row.opening + " the catalog half holds no entry of the build src2:x"
b.ag.NotComposedEach(context.Background(), "", map[string]string{"idle": words})
if h := b.haves.last("idle"); h.Phase != string(nodeengineclient.PhaseHeld) || h.Composed != idleDig || h.Detail != words ||
h.Wait == nil || h.Wait.Step != nodeengine.WaitStepCompose || h.Wait.Reason != row.reason || h.Wait.Words != words || h.Wait.Since.IsZero() {
t.Errorf("idle's have while its entry is %s: %+v", row.reason, h)
}
}
b.ag.NotComposedEach(context.Background(), "", nil) b.ag.NotComposedEach(context.Background(), "", nil)
if h := b.haves.last("idle"); h.Phase != string(nodeengineclient.PhaseRunning) || h.Composed != idleDig { if h := b.haves.last("idle"); h.Phase != string(nodeengineclient.PhaseRunning) || h.Composed != idleDig {
t.Errorf("idle's have once it composes again: %+v", h) t.Errorf("idle's have once it composes again: %+v", h)
+40 -1
View File
@@ -37,6 +37,9 @@ type CompositionView struct {
Revisions []RevisionView `json:"revisions,omitempty"` Revisions []RevisionView `json:"revisions,omitempty"`
// Runs is what runs of the module, and where that is known from; nil when nothing of it runs. // Runs is what runs of the module, and where that is known from; nil when nothing of it runs.
Runs *RunsView `json:"runs,omitempty"` Runs *RunsView `json:"runs,omitempty"`
// RunsManifest is the manifest of the build that runs of the module, as the one place reads it (design 47 G A-484): which
// source answered, or the words naming the three it tried. Said also where the want's build has no composition.
RunsManifest *RunsManifestView `json:"runs-manifest,omitempty"`
// Judged is the move from what runs to the composition read, as the agent judges it before an apply. // Judged is the move from what runs to the composition read, as the agent judges it before an apply.
Judged JudgedView `json:"judged"` Judged JudgedView `json:"judged"`
// Diff is every resource that differs from what runs, when asked. // Diff is every resource that differs from what runs, when asked.
@@ -44,6 +47,17 @@ type CompositionView struct {
// Own is, of the node's own composition (module node), its own fields beside what runs names (issue 774: ace's // Own is, of the node's own composition (module node), its own fields beside what runs names (issue 774: ace's
// own composition dropped the provisioner client's account, and nothing read could show it). // own composition dropped the provisioner client's account, and nothing read could show it).
Own *OwnView `json:"own,omitempty"` Own *OwnView `json:"own,omitempty"`
// Withheld are the module's marked subjects the grants part withholds from this node's runtime, each with why, read
// through the seat mesh-grants' verb withheld; WithheldUnknown says when that verb did not answer (design 46 approval
// mark §6, lead answer A-239).
Withheld []WithheldView `json:"withheld,omitempty"`
WithheldUnknown string `json:"withheld-unknown,omitempty"`
}
// WithheldView is one marked subject withheld, and why.
type WithheldView struct {
Subject string `json:"subject"`
Why string `json:"why"`
} }
// OwnView is the node's own fields of a composition (engcomp.NodeOwn) beside what runs of them, and what the node's // OwnView is the node's own fields of a composition (engcomp.NodeOwn) beside what runs of them, and what the node's
@@ -125,6 +139,14 @@ type RunsView struct {
Resources int `json:"resources"` Resources int `json:"resources"`
} }
// RunsManifestView is what the one place answers of the build a module runs here.
type RunsManifestView struct {
// Source is the source that answered; empty when none did.
Source string `json:"source,omitempty"`
FP string `json:"fp,omitempty"`
Words string `json:"words"`
}
// JudgedView is a move's judgement. // JudgedView is a move's judgement.
type JudgedView struct { type JudgedView struct {
Judgement string `json:"judgement"` Judgement string `json:"judgement"`
@@ -179,6 +201,11 @@ func (a *Agent) Composition(ctx context.Context, module, digest string, diff boo
} }
} }
a.mu.Unlock() a.mu.Unlock()
// Read outside the lock: the one place reads what the agent runs (Agent.Running takes it), and nothing it does is asked of the bus.
if a.env.RunningManifest != nil && module != NodeModule {
src, rfp, words := a.env.RunningManifest(module)
view.RunsManifest = &RunsManifestView{Source: src, FP: rfp, Words: words}
}
if module != NodeModule && fp == "" { if module != NodeModule && fp == "" {
return view, fmt.Errorf("%s has no want and no have on %s, so no build of it is composed here", module, a.env.Node) return view, fmt.Errorf("%s has no want and no have on %s, so no build of it is composed here", module, a.env.Node)
} }
@@ -209,7 +236,19 @@ func (a *Agent) Composition(ctx context.Context, module, digest string, diff boo
case digest != "" && read == nil: case digest != "" && read == nil:
return view, fmt.Errorf("no revision of %s the bucket keeps has the digest %s; revisions lists those it keeps", key, digest) return view, fmt.Errorf("no revision of %s the bucket keeps has the digest %s; revisions lists those it keeps", key, digest)
case read == nil && !haveLatest: case read == nil && !haveLatest:
return view, fmt.Errorf("the bucket holds no composition of %s at %s", module, fp) // What runs of the module is still said, through the one place (A-484): the composition read is the want's build, and a
// module held at another build has none of it.
words := fmt.Sprintf("the bucket holds no composition of %s at %s", module, fp)
if a.env.SelfComposed && a.env.WhyUnmade != nil && module != NodeModule {
if why := a.env.WhyUnmade(module); why != "" {
words = fmt.Sprintf("the bucket holds no composition of %s at %s; the agent waits for the composer: %s", module, fp, why)
}
}
if view.RunsManifest == nil {
return view, errors.New(words)
}
view.Composition = RevisionView{Key: key, Refused: words}
return view, nil
case read == nil: case read == nil:
read = &latest read = &latest
} }
+372
View File
@@ -0,0 +1,372 @@
package agent
import (
"context"
"fmt"
"time"
"git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient"
"git.novox.be/novox/mesh-contracts/go/seats/meshrecovery"
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-host/internal/compositions"
)
// A module's kept data on the bus, taken as a restore point before a form change moves it and restored at a revert (design 47 G
// §296, §297, §554, §555, §560, §630 to §632, the lead's answers A-77, A-98 and A-108): the node agent asks the seat
// `mesh-recovery`'s `snapshot-kept` and waits for the `ended` of its own act, or, at a revert, stops the newer build, asks
// `kept-points`, asks `restore-kept` with the snapshot of the point of the form it reverts from, waits for its `restored`, and
// after the reverted build's apply calls the point's catch-up as the module's tool.
// Ending is the ending event of an act of the recovery module as the node agent reads it: `ended` for a restore point taken,
// `restored` for one restored (design 47 G §560).
type Ending struct {
Act string
Outcome string
Error string
Snapshot string
// Why is the hand act's words for a cancel (design 47 G A-117, §560).
Why string
}
// RecoveryWaitMinutes is the default of the manifest's config key `recovery-wait-minutes`, the bound of the wait for an
// ending event, counted from the first ask (design 47 G §554, §633).
const RecoveryWaitMinutes = 90
// Recovery is what the node agent needs of the recovery module and the manifests to move kept data. Nil moves none.
type Recovery struct {
// FormChanges are the kept data whose form differs between the running build's manifest and the applied build's (the
// catalog half's entries by fingerprint); an error says it is not known.
FormChanges func(ctx context.Context, module, runningFp, appliedFp string) ([]catalogue.FormChange, error)
// The seat's verbs, asked in the node-engine's own name; an error is the ask not answered.
SnapshotKept func(ctx context.Context, in meshrecovery.SnapshotKeptInput) (meshrecovery.SnapshotKeptAnswer, error)
RestoreKept func(ctx context.Context, in meshrecovery.RestoreKeptInput) (meshrecovery.RestoreKeptAnswer, error)
KeptPoints func(ctx context.Context, in meshrecovery.KeptPointsInput) (meshrecovery.KeptPointsAnswer, error)
// Hear follows the ending events of kind "ended" or "restored" of one module's kept data: the last one said already first,
// then each after it, until stop is called. It is made before the verb is asked (design 47 G §560).
Hear func(ctx context.Context, kind, module, kept string) (events <-chan Ending, stop func(), err error)
// Bound writes the bound key `<what>.<id>` of a wait in the state bounds with its time to live, so its end is the bus's
// marker, heard as expired (design 47 G A-77, A-92); end deletes it. Nil keeps no bound.
Bound func(ctx context.Context, what, id, module string, since time.Time, d time.Duration) (expired <-chan struct{}, end func(), err error)
// CatchUp calls the module's tool on this node through the tool runner (design 47 G A-108).
CatchUp func(ctx context.Context, module string, c meshrecovery.CatchUp) error
// WaitMinutes is the bound in minutes, the config key recovery-wait-minutes; 0 reads as RecoveryWaitMinutes.
WaitMinutes int
// Now is the clock; nil reads the machine's.
Now func() time.Time
}
func (r *Recovery) now() time.Time {
if r.Now != nil {
return r.Now()
}
return time.Now()
}
func (r *Recovery) bound() time.Duration {
if r.WaitMinutes > 0 {
return time.Duration(r.WaitMinutes) * time.Minute
}
return RecoveryWaitMinutes * time.Minute
}
// KeptMove is one kept data a want moves across a change of its form, as the seat mesh-recovery's verbs name it.
type KeptMove struct {
// Name is the kept data's local name, Form the form the restore point holds (the older one), and Fingerprint the
// fingerprint the restore point is taken before (the newer build's) or reverted from.
Name string
Form int
Fingerprint string
// CatchUp is, when taking the point, the tool the newer manifest declares beside the kept data.
CatchUp *meshrecovery.CatchUp
}
// keptMoves are the restore points a move needs: a point to take before the newer build starts (the kept data's form rises,
// or the newer manifest declares it no more) and a point to restore before the older build starts (the kept data goes back to
// a lower form). A kept data the running build does not declare is neither: nothing was written before it.
func keptMoves(changes []catalogue.FormChange, runningFp, appliedFp string) (take, restore []KeptMove) {
for _, c := range changes {
switch {
case c.Running == nil:
case c.Applied == nil || c.Applied.Form > c.Running.Form:
m := KeptMove{Name: c.Name, Form: c.Running.Form, Fingerprint: appliedFp}
if c.Applied != nil && c.Applied.CatchUp != nil {
m.CatchUp = &meshrecovery.CatchUp{Tool: c.Applied.CatchUp.Tool, Arguments: c.Applied.CatchUp.Arguments}
}
take = append(take, m)
case c.Applied.Form < c.Running.Form:
restore = append(restore, KeptMove{Name: c.Name, Form: c.Applied.Form, Fingerprint: runningFp})
}
}
return take, restore
}
// recoveryHeld is how a wait for the recovery module ended when the move cannot go on: the words are the reason of the hold.
type recoveryHeld struct {
Words string
Interrupted bool
}
// askAndWait asks the verb until its act is accepted and ended, as the table of design 47 G §560 says: `busy` waits for the
// ending of the act it names and asks again, `failed` with `stopped` asks again within the same bound, a failure with any
// other error holds. The wait is written in the have with its first ask's time and the act, so a restart waits for that act
// and asks nothing again; past the bound the have says the wait, and the wait goes on (the watchman raises it).
func (a *Agent) askAndWait(ctx context.Context, h Have, kind string, k KeptMove, ask func() (accepted, busy, errWords string, err error)) (Ending, *recoveryHeld) {
r := a.env.Recovery
module := h.Module
step := nodeengine.WaitStepCompose
wait := nodeengine.Wait{Owner: nodeengine.Seat, Step: step, On: "mesh-recovery." + kind + "." + module + "." + k.Name,
Reason: "recovery waits", Bound: r.bound().String()}
// A restart: the have names the wait with its ask time and act.
last, ok := a.env.Haves.Read(ctx, module)
if ok && last.Wait != nil && last.Wait.On == wait.On && last.Wait.AskAt != nil {
wait.AskAt, wait.Act, wait.Since = last.Wait.AskAt, last.Wait.Act, last.Wait.Since
}
now := r.now()
if wait.AskAt == nil {
wait.AskAt, wait.Since = &now, now
}
events, stop, err := r.Hear(ctx, kind, module, k.Name)
if err != nil {
return Ending{}, &recoveryHeld{Words: "the end of the recovery module's act cannot be heard: " + err.Error()}
}
defer stop()
var expired <-chan struct{}
if r.Bound != nil {
bound := r.bound() - now.Sub(*wait.AskAt)
if bound < time.Second {
bound = time.Second
}
exp, end, berr := r.Bound(ctx, "recovery", module+"."+kind+"."+k.Name, module, *wait.AskAt, bound)
if berr != nil {
return Ending{}, &recoveryHeld{Words: "the bound of the wait for the recovery module cannot be written: " + berr.Error()}
}
defer end()
expired = exp
}
write := func() {
w := wait
h.Wait = &w
a.writeHave(ctx, h)
}
act, busyOn := wait.Act, ""
for {
if act == "" && busyOn == "" {
write() // the first ask's time is in the have before the ask
accepted, busy, words, err := ask()
switch {
case err != nil:
if ctx.Err() != nil {
return Ending{}, &recoveryHeld{Words: BackupInterrupted, Interrupted: true}
}
return Ending{}, &recoveryHeld{Words: "the recovery module could not be asked: " + err.Error()}
case words != "":
return Ending{}, &recoveryHeld{Words: words}
case accepted != "":
act, wait.Act = accepted, accepted
write()
case busy != "":
busyOn = busy
default:
return Ending{}, &recoveryHeld{Words: "the recovery module answered neither accepted nor busy"}
}
}
select {
case <-ctx.Done():
return Ending{}, &recoveryHeld{Words: BackupInterrupted, Interrupted: true}
case <-expired:
expired = nil
wait.Words = fmt.Sprintf("the recovery module has not ended its act within %s", r.bound())
write()
case ev, open := <-events:
if !open {
return Ending{}, &recoveryHeld{Words: "the end of the recovery module's act is no longer heard"}
}
switch {
case busyOn != "" && ev.Act == busyOn:
busyOn = ""
case act != "" && ev.Act == act:
switch {
case ev.Outcome == meshrecovery.EndedEventOutcomeDone:
return ev, nil
case ev.Error == "stopped":
act, wait.Act = "", ""
case ev.Error == "cancelled":
words := "cancelled"
if ev.Why != "" {
words += ": " + ev.Why
}
return ev, &recoveryHeld{Words: words}
default:
words := ev.Error
if words == "" {
words = "the recovery module's act failed"
}
return ev, &recoveryHeld{Words: words}
}
}
}
}
}
// keptStep takes the restore points a move of a module needs before it applies and restores the ones a revert needs: it
// answers whether the module is held (its have says why, and the running build is kept) and, after a restore, the catch-up
// of the point to call once the reverted build runs. A want that says to go on without a backup asks neither, and nor does one that answers a held revert live.
func (a *Agent) keptStep(ctx context.Context, module string, w wantNow, comp compNow, ranBefore bool, others []compositions.Composed) (held bool, catchUps []meshrecovery.CatchUp) {
r := a.env.Recovery
if !ranBefore || r == nil || r.FormChanges == nil || w.Want.Backup == meshwants.WantsByWantNodeModuleValueBackupSkip {
return false, nil
}
fp, dig := w.Want.Fp, w.Want.Composed
a.mu.Lock()
prev, runs := a.running[module]
a.mu.Unlock()
if !runs {
if h, found, _ := a.readHave(ctx, module); found {
prev = nodeengine.HaveByNodeModuleValue{Fp: h.FP, Composed: h.Composed}
}
}
if prev.Fp == "" || prev.Fp == fp {
return false, nil
}
changes, err := r.FormChanges(ctx, module, prev.Fp, fp)
if err != nil {
// Whether the move changes a form is not known: it waits for the entries and never moves without its restore point (design 47 G
// A-113). No attempt is kept, so the composer handing the composition again, or any settle, asks again.
words := fmt.Sprintf("whether %s's kept data changes form between %s and %s is not known, so it is not moved: %v", module, prev.Fp, fp, err)
a.sayOnce("kept:"+module+":"+fp, module+" waits: "+words)
since := r.now()
a.writeHave(ctx, Have{Module: module, FP: prev.Fp, Composed: prev.Composed, Phase: string(nodeengineclient.PhaseHeld), Detail: words,
Wait: &nodeengine.Wait{Owner: nodeengine.Seat, Step: nodeengine.WaitStepCompose, Reason: "entry-missing", Since: since, Words: words}})
// The composition of the want is held: the wait is for an entry, not for a composition, so the composer is not told it
// is waited for (that handed the same composition again every pass, and the move failed the same way again); it
// hands the composition again when the entry answers (Composer.entryChanged).
return true, nil
}
take, restore := keptMoves(changes, prev.Fp, fp)
if len(take)+len(restore) == 0 {
return false, nil
}
// The operator's answer "live" to a held revert, read from this want and from nothing else: the reverted build starts on the live
// data as it is, with no restore, no aside taken and no catch-up, and applies as any want (design 47 G section 710).
if len(restore) > 0 && w.Want.Revert == meshwants.WantsByWantNodeModuleValueRevertLive {
a.env.Say(fmt.Sprintf("%s starts %s on the live data as it is: its want (revision %d) carries the operator's answer %q to the held revert", module, fp, w.Rev, w.Want.Revert))
return false, nil
}
hold := func(words string, interrupted bool) {
if interrupted || ctx.Err() != nil {
words = BackupInterrupted
a.env.Say(fmt.Sprintf("%s waits: %s; it asks again at the next start", module, words))
} else {
a.env.Say(fmt.Sprintf("%s is held before %s: %s", module, fp, words))
}
a.mu.Lock()
a.attempt[module] = attempt{FP: fp, Composed: dig, WantRev: w.Rev, CompRev: comp.Rev, Held: true, AnsweredRev: w.Rev}
a.mu.Unlock()
since := r.now()
a.writeHave(ctx, Have{Module: module, FP: prev.Fp, Composed: prev.Composed, Phase: string(nodeengineclient.PhaseHeld), Detail: words,
Wait: &nodeengine.Wait{Owner: nodeengine.Seat, Step: nodeengine.WaitStepCompose, Reason: words, Since: since, Words: words}})
}
backing := Have{Module: module, FP: fp, Composed: dig, Phase: string(nodeengineclient.PhaseBackingUp)}
for _, m := range take {
why := fmt.Sprintf("form change of %s's %s to %s", module, m.Name, fp)
_, held := a.askAndWait(ctx, backing, "ended", m, func() (string, string, string, error) {
got, err := r.SnapshotKept(ctx, meshrecovery.SnapshotKeptInput{Module: module, Kept: m.Name, Form: int64(m.Form),
Fingerprint: m.Fingerprint, CatchUp: m.CatchUp, Why: why})
return got.Accepted, got.Busy, got.Error, err
})
if held != nil {
hold(fmt.Sprintf("the restore point of %s's %s (form %d) was not taken: %s", module, m.Name, m.Form, held.Words), held.Interrupted)
return true, nil
}
}
if len(restore) == 0 {
return false, nil
}
// A revert: the newer build stops first (it removes nothing), so the live data is not written while it is set aside.
stopped := false
if a.env.Stop != nil {
if res := a.env.Stop(ctx, module); res.Failed {
hold("the newer build could not be stopped before the restore: "+res.Detail, false)
return true, nil
}
stopped = true
}
// A revert that cannot restore starts the newer build again and holds the move, saying why: a module is never left stopped
// (design 47 G A-113).
holdRevert := func(words string, interrupted bool) {
if stopped {
words += "; " + a.startAgain(ctx, module, prev, others)
}
hold(words, interrupted)
}
for _, m := range restore {
pts, err := r.KeptPoints(ctx, meshrecovery.KeptPointsInput{Module: module, Kept: m.Name})
if err != nil {
holdRevert(fmt.Sprintf("restore point missing: the points of %s's %s cannot be read: %v", module, m.Name, err), false)
return true, nil
}
var point *meshrecovery.Point
for i := range pts.Points {
p := pts.Points[i]
if p.Module == module && p.Kept == m.Name && int(p.Form) == m.Form && p.Fingerprint == m.Fingerprint {
if point == nil || p.At.After(point.At) {
point = &pts.Points[i]
}
}
}
if point == nil {
holdRevert(fmt.Sprintf("restore point missing: %s's %s has none of form %d taken before %s", module, m.Name, m.Form, m.Fingerprint), false)
return true, nil
}
why := fmt.Sprintf("revert of %s's %s from %s", module, m.Name, m.Fingerprint)
_, held := a.askAndWait(ctx, backing, "restored", m, func() (string, string, string, error) {
got, err := r.RestoreKept(ctx, meshrecovery.RestoreKeptInput{Module: module, Kept: m.Name, Form: int64(m.Form),
Fingerprint: m.Fingerprint, Snapshot: point.Snapshot, Why: why})
return got.Accepted, got.Busy, got.Error, err
})
if held != nil {
holdRevert(fmt.Sprintf("%s's %s (form %d) is not restored: %s", module, m.Name, m.Form, held.Words), held.Interrupted)
return true, nil
}
if point.CatchUp != nil {
catchUps = append(catchUps, *point.CatchUp)
}
}
return false, catchUps
}
// startAgain starts the build a module ran before a revert stopped it, from the composition the agent applied or holds of its
// fingerprint, and says whether it did (design 47 G A-113).
func (a *Agent) startAgain(ctx context.Context, module string, prev nodeengine.HaveByNodeModuleValue, others []compositions.Composed) string {
a.mu.Lock()
c, ok := a.applied[module]
if !ok || c.C.Module == "" {
c, ok = a.compositionFor(module, prev.Fp)
}
a.mu.Unlock()
if !ok || a.env.Apply == nil {
return "the newer build could not be started again: its composition is not held"
}
res := a.env.Apply(context.WithoutCancel(ctx), c.C, others)
if res.Failed {
return "the newer build could not be started again: " + res.Detail
}
return "the newer build runs again"
}
// catchUp calls, once the reverted build of a module runs, the catch-up of each restore point restored, as the module's own
// tool through the tool runner (design 47 G A-108). A tool that fails is said: the build runs, and the tool can be called by hand.
func (a *Agent) catchUp(ctx context.Context, module string, calls []meshrecovery.CatchUp) {
if a.env.Recovery == nil || a.env.Recovery.CatchUp == nil {
return
}
for _, c := range calls {
if err := a.env.Recovery.CatchUp(ctx, module, c); err != nil {
a.env.Say(fmt.Sprintf("%s runs the reverted build, and its catch-up tool %s failed: %v", module, c.Tool, err))
continue
}
a.env.Say(fmt.Sprintf("%s's catch-up tool %s ran after the restore", module, c.Tool))
}
}
+326
View File
@@ -0,0 +1,326 @@
package agent
import (
"context"
"encoding/json"
"fmt"
"slices"
"strings"
"sync"
"testing"
"time"
"git.novox.be/novox/mesh-contracts/go/seats/meshrecovery"
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-host/internal/buckets/bucketstest"
"github.com/novox/mesh-host/internal/compositions"
)
// A recovery module that answers as scripted: each ask takes the next answer and says the events it carries after it.
type recoveryFake struct {
mu sync.Mutex
asks []string // verb and its input, in order
answers []fakeAnswer
points []meshrecovery.Point
events chan Ending
changes []catalogue.FormChange
calls []string // stop, ask, apply, catch-up, in order
catchUp []meshrecovery.CatchUp
expired chan struct{}
unknown error // FormChanges answers it
waited int // times the agent told the composer it waits for a composition
}
type fakeAnswer struct {
accepted, busy, errWords string
then []Ending
}
func (f *recoveryFake) next(verb string, in any) (string, string, string) {
f.mu.Lock()
defer f.mu.Unlock()
f.asks = append(f.asks, fmt.Sprintf("%s %s", verb, asJSON(in)))
f.calls = append(f.calls, verb)
if len(f.answers) == 0 {
return "", "", "no answer scripted"
}
a := f.answers[0]
f.answers = f.answers[1:]
for _, e := range a.then {
f.events <- e
}
return a.accepted, a.busy, a.errWords
}
func (f *recoveryFake) env() *Recovery {
return &Recovery{
FormChanges: func(context.Context, string, string, string) ([]catalogue.FormChange, error) {
return f.changes, f.unknown
},
SnapshotKept: func(_ context.Context, in meshrecovery.SnapshotKeptInput) (meshrecovery.SnapshotKeptAnswer, error) {
a, b, e := f.next("snapshot-kept", in)
return meshrecovery.SnapshotKeptAnswer{Accepted: a, Busy: b, Error: e}, nil
},
RestoreKept: func(_ context.Context, in meshrecovery.RestoreKeptInput) (meshrecovery.RestoreKeptAnswer, error) {
a, b, e := f.next("restore-kept", in)
return meshrecovery.RestoreKeptAnswer{Accepted: a, Busy: b, Error: e}, nil
},
KeptPoints: func(context.Context, meshrecovery.KeptPointsInput) (meshrecovery.KeptPointsAnswer, error) {
f.mu.Lock()
defer f.mu.Unlock()
f.calls = append(f.calls, "kept-points")
return meshrecovery.KeptPointsAnswer{Points: f.points}, nil
},
Hear: func(context.Context, string, string, string) (<-chan Ending, func(), error) {
return f.events, func() {}, nil
},
Bound: func(context.Context, string, string, string, time.Time, time.Duration) (<-chan struct{}, func(), error) {
return f.expired, func() {}, nil
},
CatchUp: func(_ context.Context, _ string, c meshrecovery.CatchUp) error {
f.mu.Lock()
defer f.mu.Unlock()
f.calls = append(f.calls, "catch-up")
f.catchUp = append(f.catchUp, c)
return nil
},
Now: time.Now,
}
}
func (f *recoveryFake) seen() []string {
f.mu.Lock()
defer f.mu.Unlock()
return slices.Clone(f.calls)
}
// moveDB runs a data module at fp "a", then wants it at fp "b", and answers the bed once the second want is settled or held.
func moveDB(t *testing.T, f *recoveryFake, ranFrom, to string, settle func(b *bed) bool, before ...func(b *bed, dig string)) *bed {
t.Helper()
b := newBed(t)
inner, applies := b.env.Apply, 0
b.env.Apply = func(ctx context.Context, c compositions.Composed, others []compositions.Composed) Result {
if applies++; applies >= 2 {
f.mu.Lock()
f.calls = append(f.calls, "apply")
f.mu.Unlock()
}
return inner(ctx, c, others)
}
f.events, f.expired = make(chan Ending, 16), make(chan struct{})
b.env.Recovery = f.env()
b.env.Waiting = func(string, string) { f.mu.Lock(); f.waited++; f.mu.Unlock() }
b.env.Stop = func(context.Context, string) Result {
f.mu.Lock()
f.calls = append(f.calls, "stop")
f.mu.Unlock()
return Result{}
}
stop := b.run()
t.Cleanup(stop)
fp1, fp2 := fpOf(ranFrom), fpOf(to)
ck, craw, dig := composition(t, "db", fp1, true, file("db", "x", "1"))
wk, wraw := wantFor(t, "db", fp1, dig, "")
b.comps.put(ck, craw)
b.wants.put(wk, wraw)
eventually(t, "the first build is not applied", func() bool { return len(b.appliedList()) == 1 })
b.idle()
ck2, craw2, dig2 := composition(t, "db", fp2, true, file("db", "x", "2"))
for _, h := range before {
h(b, dig)
}
wk2, wraw2 := wantFor(t, "db", fp2, dig2, "")
b.comps.put(ck2, craw2)
b.wants.put(wk2, wraw2)
eventually(t, "the move is neither applied nor held", func() bool { return settle(b) })
return b
}
// testAMoveAcrossAFormChange is design 47 G §296, §297, §554, §560, §630 to §632 and A-98, A-108 at the node agent.
func testAMoveAcrossAFormChange(t *testing.T) {
up := []catalogue.FormChange{{Name: "cache", Kind: catalogue.KeptKindState, Running: &catalogue.KeptLayout{Form: 1},
Applied: &catalogue.KeptLayout{Form: 2, CatchUp: &catalogue.CatchUp{Tool: "rebuild", Arguments: map[string]any{"all": true}}}}}
down := []catalogue.FormChange{{Name: "cache", Kind: catalogue.KeptKindState, Running: &catalogue.KeptLayout{Form: 2},
Applied: &catalogue.KeptLayout{Form: 1}}}
fp := func(c string) string { return fpOf(c) }
done := func(act string) Ending { return Ending{Act: act, Outcome: meshrecovery.EndedEventOutcomeDone} }
failed := func(act, why string) Ending {
e := Ending{Act: act, Outcome: meshrecovery.EndedEventOutcomeFailed, Error: why}
if why == "cancelled" {
e.Why = "the operator stopped it" // the hand act's words (design 47 G A-117)
}
return e
}
applied := func(b *bed) bool { return len(b.appliedList()) == 2 }
heldHave := func(b *bed) bool { return b.haves.last("db").Phase == "held" }
t.Run("a form that rises is taken as a restore point before the move, with the why and the catch-up, and then applied", func(t *testing.T) {
f := &recoveryFake{changes: up, answers: []fakeAnswer{{accepted: "a1", then: []Ending{done("a1")}}}}
moveDB(t, f, "a", "b", applied)
if got := f.seen(); !slices.Equal(got, []string{"snapshot-kept", "apply"}) {
t.Fatalf("calls %v", got)
}
want := fmt.Sprintf("snapshot-kept %s", asJSON(meshrecovery.SnapshotKeptInput{Module: "db", Kept: "cache", Form: 1, Fingerprint: fp("b"),
Why: "form change of db's cache to " + fp("b"), CatchUp: &meshrecovery.CatchUp{Tool: "rebuild", Arguments: map[string]any{"all": true}}}))
if f.asks[0] != want {
t.Fatalf("asked %s, want %s", f.asks[0], want)
}
})
t.Run("busy waits for the other act's ending and asks again, and a stopped act is asked again; another act's failure wakes nothing", func(t *testing.T) {
f := &recoveryFake{changes: up, answers: []fakeAnswer{
{busy: "x1", then: []Ending{failed("zz", "cap"), done("x1")}},
{accepted: "a2", then: []Ending{failed("a2", "stopped")}},
{accepted: "a3", then: []Ending{done("a3")}},
}}
moveDB(t, f, "a", "b", applied)
if got := f.seen(); !slices.Equal(got, []string{"snapshot-kept", "snapshot-kept", "snapshot-kept", "apply"}) {
t.Fatalf("calls %v", got)
}
})
for _, row := range []struct{ error, held string }{
{"cancelled", "cancelled: the operator stopped it"},
{"cap", "cap"},
{"restore point damaged", "restore point damaged"},
{"disk full", "disk full"},
} {
t.Run("an ending failed with "+row.error+" holds the move and asks nothing again", func(t *testing.T) {
f := &recoveryFake{changes: up, answers: []fakeAnswer{{accepted: "a1", then: []Ending{failed("a1", row.error)}}}}
b := moveDB(t, f, "a", "b", heldHave)
if h := b.haves.last("db"); h.FP != fp("a") || !contains(h.Detail, row.held) || h.Wait == nil || h.Wait.Reason == "" {
t.Fatalf("the have %+v", h)
}
if got := f.seen(); len(got) != 1 {
t.Fatalf("asked again: %v", got)
}
if n := len(b.appliedList()); n != 1 {
t.Fatalf("applied %d", n)
}
})
}
t.Run("the bound passed says the wait in the have and the wait goes on to the ending", func(t *testing.T) {
f := &recoveryFake{changes: up, answers: []fakeAnswer{{accepted: "a1"}}}
b := moveDB(t, f, "a", "b", func(b *bed) bool {
b.haves.mu.Lock()
defer b.haves.mu.Unlock()
for _, h := range b.haves.log {
if h.Wait != nil && h.Wait.Words != "" && h.Phase == "backing-up" {
return true
}
}
return false
}, func(*bed, string) { close(f.expired) })
f.events <- done("a1")
eventually(t, "the move went on after the ending", func() bool { return applied(b) })
})
t.Run("a revert stops the newer build, restores the point of the form it reverts from, applies, then calls its catch-up", func(t *testing.T) {
f := &recoveryFake{changes: down,
points: []meshrecovery.Point{
{Module: "db", Kept: "cache", Form: 1, Fingerprint: fp("z"), Snapshot: "other", At: time.Now()},
{Module: "db", Kept: "cache", Form: 1, Fingerprint: fp("a"), Snapshot: "snap-1", At: time.Now(), CatchUp: &meshrecovery.CatchUp{Tool: "rebuild"}}},
answers: []fakeAnswer{{accepted: "r1", then: []Ending{done("r1")}}}}
moveDB(t, f, "a", "b", func(b *bed) bool { return len(f.seen()) >= 5 })
if got := f.seen(); !slices.Equal(got, []string{"stop", "kept-points", "restore-kept", "apply", "catch-up"}) {
t.Fatalf("calls %v", got)
}
if f.catchUp[0].Tool != "rebuild" {
t.Fatalf("catch-up %+v", f.catchUp)
}
want := fmt.Sprintf("restore-kept %s", asJSON(meshrecovery.RestoreKeptInput{Module: "db", Kept: "cache", Form: 1, Fingerprint: fp("a"),
Snapshot: "snap-1", Why: "revert of db's cache from " + fp("a")}))
if f.asks[0] != want {
t.Fatalf("asked %s, want %s", f.asks[0], want)
}
})
t.Run("a revert with no point of its form holds as a restore point missing", func(t *testing.T) {
f := &recoveryFake{changes: down}
b := moveDB(t, f, "a", "b", heldHave)
if h := b.haves.last("db"); !contains(h.Detail, "restore point missing") {
t.Fatalf("the have %+v", h)
}
})
t.Run("a held revert, then the want with revert live, starts the older build with no restore, no aside and no catch-up (design 47 G section 710)", func(t *testing.T) {
f := &recoveryFake{changes: down}
b := moveDB(t, f, "a", "b", heldHave)
if h := b.haves.last("db"); !contains(h.Detail, "restore point missing") {
t.Fatalf("the have %+v", h)
}
before := len(b.appliedList())
_, _, dig2 := composition(t, "db", fpOf("b"), true, file("db", "x", "2"))
raw, err := bucketstest.Want(meshwants.WantsByWantNodeModuleValue{Fp: fpOf("b"), Composed: dig2, Revert: meshwants.WantsByWantNodeModuleValueRevertLive,
Words: "the operator answered \"live\" to the revert ask (asking 1)"})
if err != nil {
t.Fatal(err)
}
b.wants.put(meshwants.WantsByWantNodeModuleKey(node, "db"), raw)
eventually(t, "the older build runs on the answer", func() bool { h := b.haves.last("db"); return h.FP == fpOf("b") && h.Phase != "held" })
if got := f.seen(); !slices.Equal(got, []string{"stop", "kept-points", "apply", "apply"}) && len(b.appliedList()) <= before {
t.Fatalf("calls %v", got)
}
for _, c := range f.seen() {
if c == "restore-kept" || c == "snapshot-kept" || c == "catch-up" {
t.Fatalf("calls %v", f.seen())
}
}
if n := strings.Count(strings.Join(f.seen(), " "), "kept-points"); n != 1 {
t.Fatalf("the points were read again on the answer: %v", f.seen())
}
})
t.Run("a revert whose restore fails starts the newer build again and holds the move, saying why (A-113)", func(t *testing.T) {
f := &recoveryFake{changes: down,
points: []meshrecovery.Point{{Module: "db", Kept: "cache", Form: 1, Fingerprint: fp("a"), Snapshot: "snap-1", At: time.Now()}},
answers: []fakeAnswer{{accepted: "r1", then: []Ending{failed("r1", "restore point damaged")}}}}
b := moveDB(t, f, "a", "b", heldHave)
if got := f.seen(); !slices.Equal(got, []string{"stop", "kept-points", "restore-kept", "apply"}) {
t.Fatalf("calls %v", got)
}
if h := b.haves.last("db"); !contains(h.Detail, "restore point damaged") || !contains(h.Detail, "the newer build runs again") ||
h.Wait == nil || h.Wait.Step != nodeengine.WaitStepCompose {
t.Fatalf("the have %+v", h)
}
})
t.Run("a form change not known moves nothing and waits for the entries (A-113)", func(t *testing.T) {
f := &recoveryFake{unknown: fmt.Errorf("the entry is asked")}
b := moveDB(t, f, "a", "b", heldHave)
f.mu.Lock()
if f.waited != 0 {
t.Errorf("the composer was told the agent waits for a composition %d times; it waits for an entry and holds the composition", f.waited)
}
f.mu.Unlock()
if h := b.haves.last("db"); h.FP != fp("a") || !contains(h.Detail, "is not known") || h.Wait == nil || h.Wait.Reason != "entry-missing" {
t.Fatalf("the have %+v", h)
}
if n := len(b.appliedList()); n != 1 || len(f.seen()) != 0 {
t.Fatalf("applied %d, calls %v", n, f.seen())
}
})
t.Run("a restart waits for the act its have names and asks nothing again", func(t *testing.T) {
f := &recoveryFake{changes: up}
b := moveDB(t, f, "a", "b", applied, func(b *bed, dig string) {
at := time.Now().Add(-time.Minute)
b.haves.mu.Lock()
b.haves.stored["db"] = Have{Module: "db", FP: fp("a"), Phase: "backing-up", Wait: &nodeengine.Wait{
On: "mesh-recovery.ended.db.cache", AskAt: &at, Act: "a1", Since: at}}
b.haves.mu.Unlock()
f.events <- done("a1")
})
_ = b
if got := f.seen(); !slices.Equal(got, []string{"apply"}) {
t.Fatalf("calls %v", got)
}
})
}
func contains(s, sub string) bool { return strings.Contains(s, sub) }
func asJSON(v any) string { raw, _ := json.Marshal(v); return string(raw) }
+147
View File
@@ -0,0 +1,147 @@
package agent
import (
"encoding/json"
"fmt"
"sort"
"strings"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/containerimageregistryclient/reference"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
// PartsDiffer is the start of the detail of a have held because a want's parts and the digests its composition fetches
// disagree (design 47 G section 524).
const PartsDiffer = "parts-differ:"
// EntryAsked and EntryMissing open the detail of a have held for the entry the want names (design 47 G section 421 and
// section 614): the entry is asked of the catalog half and not yet answered, or the half answered that it holds none. Both
// are the phase held, and each names the fingerprint.
const (
EntryAsked = "entry-asked:"
EntryMissing = "entry-missing:"
)
// judgedAgain says a held have is a hold the node-engine judges again at a start, not one that waits for the operator's answer:
// its wait's reason (the structured field the have carries) is that its want's parts differ from its composition (§524) or its
// entry is not held (§421), whatever words its detail opens with. Only a failed backup or restore point waits for an answer (ADR
// 0568 §4); novox, 2026-11-16: the indexer held "parts-differ" was kept at a start as "held for a failed backup"; g14, 2026-11-18:
// feh held "whether ..." with the reason entry-missing was kept the same way.
func judgedAgain(h Have) bool {
if h.Wait == nil {
return false
}
switch h.Wait.Reason {
case strings.TrimSuffix(PartsDiffer, ":"), strings.TrimSuffix(EntryAsked, ":"), strings.TrimSuffix(EntryMissing, ":"):
return true
}
return false
}
// heldReason is the opening of a detail held for an entry: EntryAsked or EntryMissing.
func heldReason(why string) string {
if strings.HasPrefix(why, EntryAsked) {
return EntryAsked
}
return EntryMissing
}
// heldForEntry says whether why is a hold for the entry a want names.
func heldForEntry(why string) bool {
return strings.HasPrefix(why, EntryAsked) || strings.HasPrefix(why, EntryMissing)
}
// upstreamCopy is whether a routed image is the registry seat's copy of an upstream image, `<address>/upstream/<host>/<image>@<digest>`:
// the mesh mirrors it, but it built nothing, so it is no part of a want (design 47 G A-454).
func upstreamCopy(image string) bool {
_, path, ok := strings.Cut(image, "/")
if !ok {
return false
}
_, _, upstream := reference.InUpstream(path)
return upstream
}
// partsDifference checks a want's `parts` against the digests of the archive, process and container resources of the
// composition, which are the digests the apply fetches and checks (design 47 G section 524). A want with no `parts` (an
// assigner of before G, or a node whose record does not state `want-parts`) is applied as today: it returns "". With
// `parts`, the want is applied only when the digest of every resource fetched is among the values of `parts` and every
// value of `parts` is the digest of a resource fetched; otherwise it returns the detail naming each digest.
func partsDifference(parts map[string]string, resources []json.RawMessage, templates []string) string {
if len(parts) == 0 {
return ""
}
inParts := map[string]bool{}
for _, d := range parts {
inParts[d] = true
}
fetched := map[string]bool{}
// A templates archive the composition was rendered from was read from the artifact store and checked by its digest:
// a part fetched (design 47 G A-481, A-481a).
for _, d := range templates {
fetched[d] = true
}
for _, raw := range resources {
r, err := declaration.ParseResource(raw)
if err != nil {
continue
}
switch res := r.(type) {
case *declaration.Archive:
if res.Source != "" {
fetched[res.Digest] = true
}
case *declaration.Process:
// A runtime of its own runs the tool runner's program, which the tool runner's want checks (A-432).
if res.Source != "" && !strings.HasSuffix(res.ID, "."+catalogue.OwnRuntimeID()) {
fetched[res.Digest] = true
}
case *declaration.Container:
// A digest the want's parts name always counts as fetched, whatever marks the resource carries; one they do not
// name counts only when it is the mesh's own: an image the manifest pins from outside, or the registry seat's
// copy of an upstream image, is no part (design 47 G A-454, A-454a).
digest := ""
if strings.HasPrefix(res.Image, "sha256:") {
digest = res.Image
} else if _, d, ok := strings.Cut(res.Image, "@"); ok {
digest = d
}
if digest == "" {
continue
}
if !inParts[digest] && (res.ImageSeat == "" || upstreamCopy(res.Image)) {
continue
}
fetched[digest] = true
}
}
named := map[string]bool{}
for _, d := range parts {
named[d] = true
}
var outside, missing []string
for d := range fetched {
if !named[d] {
outside = append(outside, d)
}
}
for d := range named {
if !fetched[d] {
missing = append(missing, d)
}
}
if len(outside) == 0 && len(missing) == 0 {
return ""
}
sort.Strings(outside)
sort.Strings(missing)
var said []string
if len(outside) > 0 {
said = append(said, "fetched and not in the want's parts: "+strings.Join(outside, ", "))
}
if len(missing) > 0 {
said = append(said, "in the want's parts and not fetched: "+strings.Join(missing, ", "))
}
return fmt.Sprintf("%s %s", PartsDiffer, strings.Join(said, "; "))
}
+66 -8
View File
@@ -578,6 +578,17 @@ func applyScoped(
} }
if declaration.Type(orphan.Type) == declaration.TypeService { if declaration.Type(orphan.Type) == declaration.TypeService {
if by, held := heldUnits[unitKey(orphan.Scope, orphan.User, orphan.Target)]; held { if by, held := heldUnits[unitKey(orphan.Scope, orphan.User, orphan.Target)]; held {
// **What the going record found passes to the record that stays** (design 42 zfs-owns-zfs §2, answer
// A-21), as a file is handed to the resource still declaring its path (ADR 0223): the unit as the machine
// had it before the mesh is what the staying record gives back when it goes, not what the mesh made of
// it. Only where the staying record has no finding of its own, kept as an unfinished apply's is.
if orphan.Found != nil && declared[by] {
if rec, has := known.Find(by); !has || rec.Found == nil {
if _, pending := known.FoundFirst[by]; !pending {
known.KeepFound(by, origin, *orphan.Found)
}
}
}
known.Forget(orphan.ID) known.Forget(orphan.ID)
detail := "no longer declared; " + by + " still holds the unit, so it was left as it is" detail := "no longer declared; " + by + " still holds the unit, so it was left as it is"
report.Outcomes = append(report.Outcomes, Outcome{ report.Outcomes = append(report.Outcomes, Outcome{
@@ -708,10 +719,24 @@ func applyScoped(
ordered = append(afterWhatTheyRead(guard), afterWhatTheyRead(rest)...) ordered = append(afterWhatTheyRead(guard), afterWhatTheyRead(rest)...)
guardFirst = len(guard) guardFirst = len(guard)
} }
// Modules whose own step did not complete. What follows *within such a module* is not attempted;
// the rest of the machine is (novox/hq ADR 0136).
gated := map[string]bool{}
// Why each gated module was gated, said on every resource of it that was skipped.
gatedWhy := map[string]string{}
orphansRemoved := false orphansRemoved := false
removeOrphans := func() error { removeOrphans := func() error {
orphansRemoved = true orphansRemoved = true
for _, orphan := range orphans { for _, orphan := range orphans {
// **Nothing of a module whose image or artifact could not be had is removed either** (novox/hq issue 1966):
// a removal is as much a change of what it runs as a replacement, and the module keeps running what it ran.
if m := orphanModule(orphan); m != "" && gated[m] {
report.Outcomes = append(report.Outcomes, Outcome{
ID: orphan.ID, Type: orphan.Type, Target: orphan.Target, Action: "kept", Detail: gatedWhy[m],
})
log(fmt.Sprintf(" kept %s (%s): %s", orphan.ID, orphan.Target, gatedWhy[m]))
continue
}
if err := removeOrphan(orphan); err != nil { if err := removeOrphan(orphan); err != nil {
return err return err
} }
@@ -760,7 +785,7 @@ func applyScoped(
// change one apply late, and never misses it. // change one apply late, and never misses it.
// And one patience with the artifact store for the whole apply (novox/hq issue 291): a fetch it // And one patience with the artifact store for the whole apply (novox/hq issue 291): a fetch it
// does not answer during a maintenance window waits for the window instead of failing. // does not answer during a maintenance window waits for the window instead of failing.
in := inputs{declares: map[string]string{}, known: &known, windows: windows, away: newStoreAway(windows, log), in := inputs{node: d.For, declares: map[string]string{}, known: &known, windows: windows, away: newStoreAway(windows, log),
groups: wanted, handed: handed} groups: wanted, handed: handed}
// Where nothing is placed or mounted from, judged before anything is applied (novox/hq issue 339), and // Where nothing is placed or mounted from, judged before anything is applied (novox/hq issue 339), and
// which declared directories were already here: one this apply makes is the mesh's, one found may not be. // which declared directories were already here: one this apply makes is the mesh's, one found may not be.
@@ -797,11 +822,6 @@ func applyScoped(
var failures []*Error var failures []*Error
// Which resources applied without error in this run, for the move-out pass after the loop (issue 1024). // Which resources applied without error in this run, for the move-out pass after the loop (issue 1024).
appliedOK := map[string]bool{} appliedOK := map[string]bool{}
// Modules whose own step did not complete. What follows *within such a module* is not attempted;
// the rest of the machine is (novox/hq ADR 0136).
gated := map[string]bool{}
// Why each gated module was gated, said on every resource of it that was skipped.
gatedWhy := map[string]string{}
gatedModules := func() []string { gatedModules := func() []string {
out := make([]string, 0, len(gated)) out := make([]string, 0, len(gated))
for m := range gated { for m := range gated {
@@ -946,6 +966,11 @@ func applyScoped(
// follows in the module is skipped as it is for any step that did not complete. // follows in the module is skipped as it is for any step that did not complete.
in.pulled = map[string]bool{} in.pulled = map[string]bool{}
pullFirst(ctx, ordered, d, in.pulled, run, &failures, gated, gatedWhy, log) pullFirst(ctx, ordered, d, in.pulled, run, &failures, gated, gatedWhy, log)
// And every archive and process, fetched before anything is replaced (issue 1966). Its failures are counted with the
// others once the resources are through, as an identity's are: here, before the guard, they would read as the guard
// not being up, and nothing of the machine would be applied.
var fetchFailures []*Error
fetchFirst(ctx, ordered, d, &in, &fetchFailures, gated, gatedWhy, log)
for i, resource := range ordered { for i, resource := range ordered {
if !orphansRemoved && i == guardFirst { if !orphansRemoved && i == guardFirst {
// **Only a guard that is up may let the filter go.** Removing the derived filter's // **Only a guard that is up may let the filter go.** Removing the derived filter's
@@ -1148,6 +1173,16 @@ func applyScoped(
if p, ok := resource.(*declaration.Process); ok && p.RunOnce { if p, ok := resource.(*declaration.Process); ok && p.RunOnce {
gates = true gates = true
} }
// **A module's apply stops at its first failed resource** (design 42 zfs-owns-zfs §1, ADR 0679 §16 (a)): any
// other resource of a module that failed gates the rest of that module, which is reported skipped with this
// reason. What it applied before stays applied; other modules, and a resource of no module, go on as before.
if module, ours := d.ModuleOfID(resource.Identity()); ours && !gates {
failed.Gated = true
gated[module] = true
gatedWhy[module] = fmt.Sprintf("%s failed, so nothing more of this module was attempted", resource.Identity())
log(fmt.Sprintf(" gated %s.*: %s failed, so the rest of it was not attempted", module, resource.Identity()))
continue
}
if gates { if gates {
failed.Gated = true failed.Gated = true
// **A step that failed has not completed, whatever it did before** (novox/hq issue 467). A // **A step that failed has not completed, whatever it did before** (novox/hq issue 467). A
@@ -1304,6 +1339,7 @@ func applyScoped(
} }
failures = append(failures, identityFailures...) failures = append(failures, identityFailures...)
failures = append(failures, fetchFailures...)
// A directory or archive that says move-out keeps only what it knows (issue 1024, novox/hq ADR 0422). // A directory or archive that says move-out keeps only what it knows (issue 1024, novox/hq ADR 0422).
movedOut, leftIn, moveFailures := moveOutPass(d, known, appliedOK, log) movedOut, leftIn, moveFailures := moveOutPass(d, known, appliedOK, log)
@@ -1579,9 +1615,17 @@ func applyOne(ctx context.Context, sys system.System, r declaration.Resource, ru
case *declaration.User: case *declaration.User:
return applyUser(ctx, sys, res, run, previous, in.groups, in.handed) return applyUser(ctx, sys, res, run, previous, in.groups, in.handed)
case *declaration.Archive: case *declaration.Archive:
return applyArchive(ctx, res, previous, in.away, journalDir(in.windows)) held, err := in.artifact(res.Identity())
if err != nil {
return begin(r), err
}
return applyArchive(ctx, res, previous, held, in.away, journalDir(in.windows))
case *declaration.Process: case *declaration.Process:
return applyProcess(ctx, res, run, changed, previous, in.away) held, err := in.artifact(res.Identity())
if err != nil {
return begin(r), err
}
return applyProcess(ctx, withNode(res, in.node), run, changed, previous, held, in.away)
case *declaration.Action: case *declaration.Action:
return applyAction(ctx, res, run) return applyAction(ctx, res, run)
case *declaration.Network: case *declaration.Network:
@@ -2769,6 +2813,12 @@ type inputs struct {
// pulled is every image this apply found on the machine, or pulled, before any container was replaced (pullFirst): // pulled is every image this apply found on the machine, or pulled, before any container was replaced (pullFirst):
// applyContainer does not ask for it again. // applyContainer does not ask for it again.
pulled map[string]bool pulled map[string]bool
// fetched is every archive's and process's bytes this apply fetched before anything was replaced (fetchFirst), each
// taken by its applier once (artifact).
fetched map[string]fetchedArtifact
// node is the node's own name, the declaration's For: every process runs with it as MESH_NODE beside its own env
// (design 47 G A-342). Empty where the declaration names none (the first node's carried bundle).
node string
} }
// fileDigest is what a file the container reads holds, by digest. // fileDigest is what a file the container reads holds, by digest.
@@ -4354,6 +4404,14 @@ func ownerText(info os.FileInfo) string {
return fmt.Sprintf("%d:%d", uid, gid) return fmt.Sprintf("%d:%d", uid, gid)
} }
// orphanModule is the module a record going belongs to, by its id, else the module whose want it was applied for.
func orphanModule(a store.Applied) string {
if m, ok := declaration.ModuleGuess(a.ID); ok {
return m
}
return a.Module
}
// pullFirst finds on the machine, or pulls, the image of each container the declaration runs before any is replaced // pullFirst finds on the machine, or pulls, the image of each container the declaration runs before any is replaced
// (ENG-7) for a module of two containers or more. A container that is run once is left to its marker, and a runtime that // (ENG-7) for a module of two containers or more. A container that is run once is left to its marker, and a runtime that
// does not answer to applyContainer, which says so. The first image that cannot be had fails its container with the image named and gates its module, so none of // does not answer to applyContainer, which says so. The first image that cannot be had fails its container with the image named and gates its module, so none of
+80 -64
View File
@@ -182,77 +182,93 @@ func TestARenameToTheSamePathDoesNotDeleteTheNewFile(t *testing.T) {
} }
} }
func TestAFailedStepFailsTheApplyAndTheRestIsStillAttempted(t *testing.T) { // A failed resource fails the apply and names itself, and what else is attempted depends on whose it is
// The apply fails, names the resource, and carries what did happen — because the machine is // (novox/hq 04-ISSUES/011; design 42 zfs-owns-zfs §1, ADR 0679 §16 (a)): the resources of no module are all
// in whatever state the apply reached and the only honest thing to hand back is that list. // attempted, so one broken resource holds no machine hostage and every failure is counted, not just the first;
// // a module's apply stops at its first failed resource, reports each one left, and no other module is touched.
// **And everything is attempted.** It used to stop at the first failure, which made one func TestAFailedResourceFailsTheApplyAndWhatFollowsIsAttemptedByWhoseItIs(t *testing.T) {
// broken resource hold the whole machine hostage: a module declaring a package that does not
// exist meant every module after it was never applied, for ever
// (novox/hq 04-ISSUES/011). The case for stopping was that a later resource may depend on an
// earlier one — and it still may, and it then fails its own check and is reported, which is
// more information than not attempting it.
dir := t.TempDir() dir := t.TempDir()
blocker := filepath.Join(dir, "blocker") for _, name := range []string{"blocker", "one", "two"} {
if err := os.WriteFile(blocker, []byte("i am a file\n"), 0o644); err != nil { if err := os.WriteFile(filepath.Join(dir, name), []byte("i am a file\n"), 0o644); err != nil {
t.Fatal(err)
}
d := parse(t, `{"declaration":1,"resources":[
{"id":"fine","type":"file","path":"`+filepath.Join(dir, "fine.conf")+`","content":"a\n"},
{"id":"doomed","type":"directory","path":"`+blocker+`"},
{"id":"after","type":"file","path":"`+filepath.Join(dir, "after.conf")+`","content":"b\n"}
]}`)
_, _, err := Apply(context.Background(), archHost(t), d, store.State{}, store.OriginCarried, noServices, nil, nil)
if err == nil {
t.Fatal("an impossible resource did not fail the apply")
}
var applyErr *Error
if !errors.As(err, &applyErr) {
t.Fatalf("expected an apply error, got %T", err)
}
if applyErr.Resource != "doomed" {
t.Errorf("the failure names %q, not the resource that failed", applyErr.Resource)
}
// Everything that worked is in the report, before and after the failure.
if len(applyErr.Done.Outcomes) != 2 {
t.Errorf("the error does not carry what was applied: %+v", applyErr.Done.Outcomes)
}
if _, err := os.Stat(filepath.Join(dir, "after.conf")); err != nil {
t.Error("a resource after the failing one was never attempted, so one broken module " +
"still blocks every module after it")
}
}
func TestEveryFailureIsCountedNotJustTheFirst(t *testing.T) {
// "One thing failed" and "eleven things failed" are different machines, and the first line is
// what somebody reads.
dir := t.TempDir()
for _, name := range []string{"one", "two"} {
if err := os.WriteFile(filepath.Join(dir, name), []byte("a file\n"), 0o644); err != nil {
t.Fatal(err) t.Fatal(err)
} }
} }
d := parse(t, `{"declaration":1,"resources":[ file := func(id, name string) string {
{"id":"first","type":"directory","path":"`+filepath.Join(dir, "one")+`"}, return `{"id":"` + id + `","type":"file","path":"` + filepath.Join(dir, name) + `","content":"a\n"}`
{"id":"second","type":"directory","path":"`+filepath.Join(dir, "two")+`"}
]}`)
_, _, err := Apply(context.Background(), archHost(t), d, store.State{}, store.OriginCarried, noServices, nil, nil)
if err == nil {
t.Fatal("two impossible resources did not fail the apply")
} }
var applyErr *Error unwritable := func(id, name string) string { // a file below a file: it cannot be made
if !errors.As(err, &applyErr) { return `{"id":"` + id + `","type":"file","path":"` + filepath.Join(dir, name, "x.conf") + `","content":"a\n"}`
t.Fatalf("got %T", err)
} }
if applyErr.Others != 1 { doomed := func(id, name string) string {
t.Errorf("the failure says %d others also failed, and one did", applyErr.Others) return `{"id":"` + id + `","type":"directory","path":"` + filepath.Join(dir, name) + `"}`
} }
if !strings.Contains(err.Error(), "one other resource also failed") { exists := func(name string) bool {
t.Errorf("the message does not say others failed: %v", err) _, err := os.Stat(filepath.Join(dir, name))
return err == nil
}
for _, tc := range []struct {
name string
resources []string
failed string
others int
applied []string // files that must exist
left []string // files that must not, reported skipped
detail string // what the module's result says, when it is one module's
}{
{"of no module, the rest is attempted", []string{file("fine", "fine.conf"), doomed("doomed", "blocker"), file("after", "after.conf")},
"doomed", 0, []string{"fine.conf", "after.conf"}, nil, ""},
{"of no module, every failure is counted", []string{doomed("first", "one"), doomed("second", "two")},
"first", 1, nil, nil, ""},
{"of a module, it stops there and names each one left",
[]string{file("m.first", "first.conf"), unwritable("m.second", "blocker"), file("m.third", "third.conf"), file("m.fourth", "fourth.conf"), file("o.other", "other.conf")},
"m.second", 0, []string{"first.conf", "other.conf"}, []string{"third.conf", "fourth.conf"},
"m.second failed"},
} {
t.Run(tc.name, func(t *testing.T) {
d := parse(t, `{"declaration":1,"resources":[`+strings.Join(tc.resources, ",")+`]}`)
report, _, err := Apply(context.Background(), archHost(t), d, store.State{}, store.OriginCarried, noServices, nil, nil)
var applyErr *Error
if !errors.As(err, &applyErr) {
t.Fatalf("expected an apply error, got %v", err)
}
if applyErr.Resource != tc.failed || applyErr.Others != tc.others {
t.Errorf("the failure names %q with %d others, want %q with %d", applyErr.Resource, applyErr.Others, tc.failed, tc.others)
}
if tc.others == 1 && !strings.Contains(err.Error(), "one other resource also failed") {
t.Errorf("the message does not say others failed: %v", err)
}
for _, name := range tc.applied {
if !exists(name) {
t.Errorf("%s was not applied", name)
}
}
for _, name := range tc.left {
if exists(name) {
t.Errorf("%s was applied after its module failed", name)
}
}
if tc.detail == "" {
return
}
outcomes := applyErr.Done.Outcomes
skipped := 0
for _, o := range outcomes {
if o.Action == "skipped" {
skipped++
}
}
if skipped != len(tc.left) {
t.Errorf("%d resources reported skipped, want %d: %+v", skipped, len(tc.left), outcomes)
}
result := ModuleResults(d, report, err)["m"]
if !result.Failed || !strings.Contains(result.Detail, tc.detail) ||
!strings.Contains(result.Detail, "m.third") || !strings.Contains(result.Detail, "m.fourth") {
t.Errorf("the module's result does not name the failed resource and each one left: %+v", result)
}
if other := ModuleResults(d, report, err)["o"]; other.Failed {
t.Errorf("another module was failed by it: %+v", other)
}
})
} }
} }
+7 -4
View File
@@ -35,7 +35,7 @@ import (
// enough for a desktop theme and small enough to notice. // enough for a desktop theme and small enough to notice.
const maxArchive = 512 << 20 const maxArchive = 512 << 20
func applyArchive(ctx context.Context, r *declaration.Archive, previous store.Applied, away *storeAway, journals string) (Outcome, error) { func applyArchive(ctx context.Context, r *declaration.Archive, previous store.Applied, held []byte, away *storeAway, journals string) (Outcome, error) {
out := begin(r) out := begin(r)
out.Action = "unchanged" out.Action = "unchanged"
recorded := previous.Wrote != "" recorded := previous.Wrote != ""
@@ -44,9 +44,12 @@ func applyArchive(ctx context.Context, r *declaration.Archive, previous store.Ap
return out, err return out, err
} }
body, err := fetchMinding(ctx, r.Source, away) // What the apply fetched before anything was replaced (fetchFirst), else fetched now.
if err != nil { body, err := held, error(nil)
return out, err if body == nil {
if body, err = fetchMinding(ctx, r.Source, away); err != nil {
return out, err
}
} }
sum := sha256.Sum256(body) sum := sha256.Sum256(body)
got := "sha256:" + hex.EncodeToString(sum[:]) got := "sha256:" + hex.EncodeToString(sum[:])
+113
View File
@@ -0,0 +1,113 @@
package apply
import (
"context"
"crypto/sha256"
"encoding/hex"
"fmt"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
// **Every artifact an apply fetches is in hand before anything is replaced** (novox/hq issue 1966, 2026-11-08 23:25 UTC).
//
// An archive and a process are fetched from the artifact store on their own turn, and a container is replaced on its
// own: so a module whose container serves the store (distribution) had that container replaced, cut off from where it
// was reached, and then failed to fetch its program from the store it had just cut off, left half applied with the
// store unreachable from every other machine. pullFirst does this for a module's images (ENG-7); fetchFirst does it for
// its archives and processes, before any of the apply's resources is applied or any orphan removed.
//
// A fetch that fails (the store does not answer for the apply's patience, or what arrived is not what was declared)
// gates the resource's module, as a pull that fails does: nothing of the module is replaced or removed, the failure names
// the artifact, and the rest of the machine is applied. A resource of no module keeps its failure for its own turn.
// fetchedArtifact is one archive's or process's bytes, fetched and checked before the apply began, or why it was not.
type fetchedArtifact struct {
body []byte
err error
}
// artifactOf is the archive's or process's source and digest, and whether the resource is one the apply fetches.
func artifactOf(r declaration.Resource) (source, digest string, ok bool) {
switch res := r.(type) {
case *declaration.Archive:
return res.Source, res.Digest, res.Source != ""
case *declaration.Process:
return res.Source, res.Digest, res.Source != ""
}
return "", "", false
}
// fetchFirst fetches each archive and process the declaration applies, checks each against its declared digest, and
// keeps the bytes for its turn (in.fetched). It is called before the loop that applies, after pullFirst.
func fetchFirst(ctx context.Context, ordered []declaration.Resource, d *declaration.Declaration, in *inputs,
failures *[]*Error, gated map[string]bool, gatedWhy map[string]string, log func(string)) {
in.fetched = map[string]fetchedArtifact{}
bySource := map[string]fetchedArtifact{}
for _, resource := range ordered {
source, digest, isArtifact := artifactOf(resource)
if !isArtifact {
continue
}
module, ours := d.ModuleOfID(resource.Identity())
if ours && gated[module] {
continue
}
// On an adopted node, what a module not yet taken declares may be held as found and never fetched (hold.go): its
// own turn fetches it, as before.
if d.Adoption != nil {
if _, untaken := d.Adoption.UntakenModuleOf(resource.Identity()); untaken || runsIn(resource) != "" {
continue
}
}
// What the module's done build kept on a build node is fetched from there, not from the store (design 47).
cacheKey, kept := source, false
if kf := keptFetchFrom(ctx); kf != nil && ours {
cacheKey = "kept " + module + " " + digest
if _, seen := bySource[cacheKey]; seen {
kept = true
} else if body, isKept, err := kf(ctx, module, digest); isKept {
kept = true
bySource[cacheKey] = fetchedArtifact{body: body, err: err}
}
}
got, seen := bySource[cacheKey]
if !seen && !kept {
got.body, got.err = fetchMinding(ctx, source, in.away)
bySource[cacheKey] = got
}
if got.err == nil {
sum := sha256.Sum256(got.body)
if have := "sha256:" + hex.EncodeToString(sum[:]); have != digest {
got = fetchedArtifact{err: fmt.Errorf("%s was declared as %s and what arrived is %s; nothing was unpacked",
source, digest, have)}
}
}
if got.err == nil {
in.fetched[resource.Identity()] = got
continue
}
if !ours {
// Nothing to gate: the resource fails on its own turn with this error.
in.fetched[resource.Identity()] = got
continue
}
err := fmt.Errorf("not replacing anything of its module, which keeps running what it ran: %w", got.err)
*failures = append(*failures, &Error{Resource: resource.Identity(), Err: err})
log(fmt.Sprintf(" failed %s (%s): %v", resource.Identity(), resource.Target(), err))
gated[module] = true
gatedWhy[module] = "its artifact " + source + " could not be fetched, so nothing of it was replaced " +
"(novox/hq issue 1966)"
}
}
// artifact is the bytes fetchFirst holds for a resource, taken once so they are not kept past their turn; nil and no
// error when it holds none (an apply that did not fetch first: its applier fetches itself).
func (in inputs) artifact(id string) ([]byte, error) {
got, ok := in.fetched[id]
if !ok {
return nil, nil
}
delete(in.fetched, id)
return got.body, got.err
}
+1 -1
View File
@@ -95,7 +95,7 @@ func WriteFilledWith(
"yet taken (%s), so nothing was written", path, held.Why)} "yet taken (%s), so nothing was written", path, held.Why)}
} }
in := inputs{declares: map[string]string{}, known: &known, windows: windows, away: newStoreAway(windows, log), in := inputs{node: d.For, declares: map[string]string{}, known: &known, windows: windows, away: newStoreAway(windows, log),
groups: groupsWanted(d.Resources), handed: Handed{}} groups: groupsWanted(d.Resources), handed: Handed{}}
in.refused = refusedPlaces(d.Resources) in.refused = refusedPlaces(d.Resources)
in.agentHomes = rootNeverHomes(d) in.agentHomes = rootNeverHomes(d)
+85 -37
View File
@@ -283,7 +283,7 @@ func predecessor(t *testing.T) (dir, page string, m *machine) {
}} }}
} }
func TestAFoundFileOfAnUntakenModuleIsKeptAsItIs(t *testing.T) { func caseAFoundFileOfAnUntakenModuleIsKeptAsItIs(t *testing.T) {
dir, page, m := predecessor(t) dir, page, m := predecessor(t)
report, state := applyAdopted(t, adopted(t, untakenWeb, webResources(page)), store.State{}, m, dir) report, state := applyAdopted(t, adopted(t, untakenWeb, webResources(page)), store.State{}, m, dir)
@@ -318,7 +318,7 @@ func TestAFoundFileOfAnUntakenModuleIsKeptAsItIs(t *testing.T) {
} }
} }
func TestAFoundContainerOfAnUntakenModuleIsNotReplaced(t *testing.T) { func caseAFoundContainerOfAnUntakenModuleIsNotReplaced(t *testing.T) {
dir, page, m := predecessor(t) dir, page, m := predecessor(t)
report, state := applyAdopted(t, adopted(t, untakenWeb, webResources(page)), store.State{}, m, dir) report, state := applyAdopted(t, adopted(t, untakenWeb, webResources(page)), store.State{}, m, dir)
@@ -338,7 +338,7 @@ func TestAFoundContainerOfAnUntakenModuleIsNotReplaced(t *testing.T) {
} }
} }
func TestWhatIsNotFoundIsCreatedWhenAssigned(t *testing.T) { func caseWhatIsNotFoundIsCreatedWhenAssigned(t *testing.T) {
// Assigning prepares: what the module declares that is not there is made. // Assigning prepares: what the module declares that is not there is made.
dir := t.TempDir() dir := t.TempDir()
page := filepath.Join(dir, "index.html") page := filepath.Join(dir, "index.html")
@@ -355,7 +355,7 @@ func TestWhatIsNotFoundIsCreatedWhenAssigned(t *testing.T) {
} }
} }
func TestAFileThisHostWroteIsNotFound(t *testing.T) { func caseAFileThisHostWroteIsNotFound(t *testing.T) {
// Found means present with no record. A record of any origin — carried or declared, this life // Found means present with no record. A record of any origin — carried or declared, this life
// of the node or an earlier one — means this host wrote it. // of the node or an earlier one — means this host wrote it.
for _, origin := range []string{store.OriginCarried, store.OriginDeclared} { for _, origin := range []string{store.OriginCarried, store.OriginDeclared} {
@@ -372,7 +372,7 @@ func TestAFileThisHostWroteIsNotFound(t *testing.T) {
} }
} }
func TestAContainerAHostMadeIsNotFound(t *testing.T) { func caseAContainerAHostMadeIsNotFound(t *testing.T) {
dir, page, m := predecessor(t) dir, page, m := predecessor(t)
m.containers["hello-web"].spec = "some-spec" m.containers["hello-web"].spec = "some-spec"
report, _ := applyAdopted(t, adopted(t, untakenWeb, webResources(page)), store.State{}, m, dir) report, _ := applyAdopted(t, adopted(t, untakenWeb, webResources(page)), store.State{}, m, dir)
@@ -381,7 +381,7 @@ func TestAContainerAHostMadeIsNotFound(t *testing.T) {
} }
} }
func TestTheGenesisStoreAdoptedInPlaceIsNotFound(t *testing.T) { func caseTheGenesisStoreAdoptedInPlaceIsNotFound(t *testing.T) {
// ADR 0078: the foundation's store, raised from the bundle and recorded as carried, is adopted // ADR 0078: the foundation's store, raised from the bundle and recorded as carried, is adopted
// as a module by name. It is the mesh's own and must never read as a predecessor's. // as a module by name. It is the mesh's own and must never read as a predecessor's.
dir := t.TempDir() dir := t.TempDir()
@@ -398,7 +398,7 @@ func TestTheGenesisStoreAdoptedInPlaceIsNotFound(t *testing.T) {
} }
} }
func TestTakingAModuleReplacesWhatWasHeldAndTheOriginalSurvives(t *testing.T) { func caseTakingAModuleReplacesWhatWasHeldAndTheOriginalSurvives(t *testing.T) {
dir, page, m := predecessor(t) dir, page, m := predecessor(t)
_, state := applyAdopted(t, adopted(t, untakenWeb, webResources(page)), store.State{}, m, dir) _, state := applyAdopted(t, adopted(t, untakenWeb, webResources(page)), store.State{}, m, dir)
h, _ := state.HeldAt("hello-web.page") h, _ := state.HeldAt("hello-web.page")
@@ -426,7 +426,7 @@ func TestTakingAModuleReplacesWhatWasHeldAndTheOriginalSurvives(t *testing.T) {
} }
} }
func TestAHeldFileIsNeverRemovedWhenItsModuleIsUnassigned(t *testing.T) { func caseAHeldFileIsNeverRemovedWhenItsModuleIsUnassigned(t *testing.T) {
dir, page, m := predecessor(t) dir, page, m := predecessor(t)
_, state := applyAdopted(t, adopted(t, untakenWeb, webResources(page)), store.State{}, m, dir) _, state := applyAdopted(t, adopted(t, untakenWeb, webResources(page)), store.State{}, m, dir)
@@ -447,7 +447,7 @@ func TestAHeldFileIsNeverRemovedWhenItsModuleIsUnassigned(t *testing.T) {
} }
} }
func TestAHoldNoLongerDeclaredIsLetGoAndFoundAgainIfItsModuleReturns(t *testing.T) { func caseAHoldNoLongerDeclaredIsLetGoAndFoundAgainIfItsModuleReturns(t *testing.T) {
dir, page, m := predecessor(t) dir, page, m := predecessor(t)
_, state := applyAdopted(t, adopted(t, untakenWeb, webResources(page)), store.State{}, m, dir) _, state := applyAdopted(t, adopted(t, untakenWeb, webResources(page)), store.State{}, m, dir)
first, _ := state.HeldAt("hello-web.page") first, _ := state.HeldAt("hello-web.page")
@@ -482,7 +482,7 @@ func TestAHoldNoLongerDeclaredIsLetGoAndFoundAgainIfItsModuleReturns(t *testing.
} }
} }
func TestACarriedApplyLetsNoHoldGo(t *testing.T) { func caseACarriedApplyLetsNoHoldGo(t *testing.T) {
dir, page, m := predecessor(t) dir, page, m := predecessor(t)
_, state := applyAdopted(t, adopted(t, untakenWeb, webResources(page)), store.State{}, m, dir) _, state := applyAdopted(t, adopted(t, untakenWeb, webResources(page)), store.State{}, m, dir)
carried := parse(t, `{"declaration":1,"resources":[`+withConf(dir)+`]}`) carried := parse(t, `{"declaration":1,"resources":[`+withConf(dir)+`]}`)
@@ -496,7 +496,7 @@ func TestACarriedApplyLetsNoHoldGo(t *testing.T) {
} }
} }
func TestAHeldFileRewrittenIsReportedAndNotReverted(t *testing.T) { func caseAHeldFileRewrittenIsReportedAndNotReverted(t *testing.T) {
dir, page, m := predecessor(t) dir, page, m := predecessor(t)
d := adopted(t, untakenWeb, webResources(page)) d := adopted(t, untakenWeb, webResources(page))
_, state := applyAdopted(t, d, store.State{}, m, dir) _, state := applyAdopted(t, d, store.State{}, m, dir)
@@ -520,7 +520,7 @@ func TestAHeldFileRewrittenIsReportedAndNotReverted(t *testing.T) {
} }
} }
func TestAHeldContainerStoppedOrReplacedIsReportedAndNotRestarted(t *testing.T) { func caseAHeldContainerStoppedOrReplacedIsReportedAndNotRestarted(t *testing.T) {
for _, c := range []struct { for _, c := range []struct {
change func(*machine) change func(*machine)
want string want string
@@ -547,7 +547,7 @@ func TestAHeldContainerStoppedOrReplacedIsReportedAndNotRestarted(t *testing.T)
} }
} }
func TestAHeldFileThatVanishesIsNotCreated(t *testing.T) { func caseAHeldFileThatVanishesIsNotCreated(t *testing.T) {
dir, page, m := predecessor(t) dir, page, m := predecessor(t)
d := adopted(t, untakenWeb, webResources(page)) d := adopted(t, untakenWeb, webResources(page))
_, state := applyAdopted(t, d, store.State{}, m, dir) _, state := applyAdopted(t, d, store.State{}, m, dir)
@@ -563,7 +563,7 @@ func TestAHeldFileThatVanishesIsNotCreated(t *testing.T) {
} }
} }
func TestAConvergedNodeStillReplacesWhatItFinds(t *testing.T) { func caseAConvergedNodeStillReplacesWhatItFinds(t *testing.T) {
// No adoption, no holds: byte for byte what a converged node did before ADR 0100. // No adoption, no holds: byte for byte what a converged node did before ADR 0100.
dir, page, m := predecessor(t) dir, page, m := predecessor(t)
d := parse(t, `{"declaration":1,"resources":[`+webResources(page)+`]}`) d := parse(t, `{"declaration":1,"resources":[`+webResources(page)+`]}`)
@@ -585,7 +585,7 @@ func TestAConvergedNodeStillReplacesWhatItFinds(t *testing.T) {
} }
} }
func TestAFileWrittenOverIsKeptOnceAndOnlyWhenTheHostHasNoRecordOfIt(t *testing.T) { func caseAFileWrittenOverIsKeptOnceAndOnlyWhenTheHostHasNoRecordOfIt(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
conf := filepath.Join(dir, "nftables.conf") conf := filepath.Join(dir, "nftables.conf")
_ = os.WriteFile(conf, []byte("# the distribution's own\n"), 0o644) _ = os.WriteFile(conf, []byte("# the distribution's own\n"), 0o644)
@@ -621,7 +621,7 @@ func untaken(ids ...string) string {
return `{"taken":[],"untaken":{"hello-web":["` + strings.Join(ids, `","`) + `"]}}` return `{"taken":[],"untaken":{"hello-web":["` + strings.Join(ids, `","`) + `"]}}`
} }
func TestAFoundDirectoryOfAnUntakenModuleKeepsItsModeOwnerAndContents(t *testing.T) { func caseAFoundDirectoryOfAnUntakenModuleKeepsItsModeOwnerAndContents(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
data := filepath.Join(dir, "data") data := filepath.Join(dir, "data")
if err := os.Mkdir(data, 0o700); err != nil { if err := os.Mkdir(data, 0o700); err != nil {
@@ -652,7 +652,7 @@ func TestAFoundDirectoryOfAnUntakenModuleKeepsItsModeOwnerAndContents(t *testing
} }
} }
func TestADirectoryMadeInTheSameApplyIsNotFound(t *testing.T) { func caseADirectoryMadeInTheSameApplyIsNotFound(t *testing.T) {
// A file's parent is made as the file is written; what the mesh made is not found. // A file's parent is made as the file is written; what the mesh made is not found.
dir := t.TempDir() dir := t.TempDir()
data := filepath.Join(dir, "data") data := filepath.Join(dir, "data")
@@ -671,7 +671,7 @@ func TestADirectoryMadeInTheSameApplyIsNotFound(t *testing.T) {
} }
} }
func TestAFoundServiceOfAnUntakenModuleIsNeitherStartedNorEnabledNorRestarted(t *testing.T) { func caseAFoundServiceOfAnUntakenModuleIsNeitherStartedNorEnabledNorRestarted(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
conf := filepath.Join(dir, "hello.conf") conf := filepath.Join(dir, "hello.conf")
m := &machine{containers: map[string]*fakeContainer{}, m := &machine{containers: map[string]*fakeContainer{},
@@ -695,7 +695,7 @@ func TestAFoundServiceOfAnUntakenModuleIsNeitherStartedNorEnabledNorRestarted(t
} }
} }
func TestAHeldServiceIsStillReloadedButNeverRestarted(t *testing.T) { func caseAHeldServiceIsStillReloadedButNeverRestarted(t *testing.T) {
// A reload stops nothing (novox/hq ADR 0102); a restart would stop the predecessor's service. // A reload stops nothing (novox/hq ADR 0102); a restart would stop the predecessor's service.
dir := t.TempDir() dir := t.TempDir()
conf := filepath.Join(dir, "daemon.json") conf := filepath.Join(dir, "daemon.json")
@@ -716,7 +716,7 @@ func TestAHeldServiceIsStillReloadedButNeverRestarted(t *testing.T) {
} }
} }
func TestAUnitAPackageOnlyShipsIsNotFound(t *testing.T) { func caseAUnitAPackageOnlyShipsIsNotFound(t *testing.T) {
// The adoption bed found this: the private network's wg-quick@mesh0 is an instance of a unit // The adoption bed found this: the private network's wg-quick@mesh0 is an instance of a unit
// the tunnel package ships. Nothing had ever run it, yet it was held as a predecessor's, and // the tunnel package ships. Nothing had ever run it, yet it was held as a predecessor's, and
// the private network never came up. Found is what the machine runs. // the private network never came up. Found is what the machine runs.
@@ -738,7 +738,7 @@ func TestAUnitAPackageOnlyShipsIsNotFound(t *testing.T) {
} }
} }
func TestAServiceWhoseUnitIsNotThereIsAppliedAsUsual(t *testing.T) { func caseAServiceWhoseUnitIsNotThereIsAppliedAsUsual(t *testing.T) {
// No unit before the apply: nothing of a predecessor's to hold. // No unit before the apply: nothing of a predecessor's to hold.
dir := t.TempDir() dir := t.TempDir()
m := &machine{containers: map[string]*fakeContainer{}, units: map[string]*fakeUnit{}} m := &machine{containers: map[string]*fakeContainer{}, units: map[string]*fakeUnit{}}
@@ -750,7 +750,7 @@ func TestAServiceWhoseUnitIsNotThereIsAppliedAsUsual(t *testing.T) {
} }
} }
func TestAContainerThatWouldMountFoundDataIsNotCreated(t *testing.T) { func caseAContainerThatWouldMountFoundDataIsNotCreated(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
data := filepath.Join(dir, "predecessor-data") data := filepath.Join(dir, "predecessor-data")
if err := os.Mkdir(data, 0o700); err != nil { if err := os.Mkdir(data, 0o700); err != nil {
@@ -788,7 +788,7 @@ func TestAContainerThatWouldMountFoundDataIsNotCreated(t *testing.T) {
} }
} }
func TestAContainerMountingWhatTheMeshMadeIsCreated(t *testing.T) { func caseAContainerMountingWhatTheMeshMadeIsCreated(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
data := filepath.Join(dir, "data") data := filepath.Join(dir, "data")
m := &machine{containers: map[string]*fakeContainer{}} m := &machine{containers: map[string]*fakeContainer{}}
@@ -801,7 +801,7 @@ func TestAContainerMountingWhatTheMeshMadeIsCreated(t *testing.T) {
} }
} }
func TestARunOnceStepInAHeldContainerIsHeld(t *testing.T) { func caseARunOnceStepInAHeldContainerIsHeld(t *testing.T) {
dir, page, m := predecessor(t) dir, page, m := predecessor(t)
step := `{"id":"hello-web.migrate","type":"container","name":"hello-web-migrate","image":"` + pinned + `", step := `{"id":"hello-web.migrate","type":"container","name":"hello-web-migrate","image":"` + pinned + `",
"run-once":true,"network":"container:hello-web"}` "run-once":true,"network":"container:hello-web"}`
@@ -818,7 +818,7 @@ func TestARunOnceStepInAHeldContainerIsHeld(t *testing.T) {
} }
} }
func TestAnActionInAHeldContainerIsHeldUntilItsModuleIsTaken(t *testing.T) { func caseAnActionInAHeldContainerIsHeldUntilItsModuleIsTaken(t *testing.T) {
// An action cannot arrive over the link today, and a bundle cannot say a node is adopted; the // An action cannot arrive over the link today, and a bundle cannot say a node is adopted; the
// host holds one anyway, since what it would run in is the predecessor's. // host holds one anyway, since what it would run in is the predecessor's.
dir, page, m := predecessor(t) dir, page, m := predecessor(t)
@@ -850,7 +850,7 @@ func TestAnActionInAHeldContainerIsHeldUntilItsModuleIsTaken(t *testing.T) {
const sixtyFourZeros = "0000000000000000000000000000000000000000000000000000000000000000" const sixtyFourZeros = "0000000000000000000000000000000000000000000000000000000000000000"
func TestAnArchiveOverSomethingFoundIsNotUnpacked(t *testing.T) { func caseAnArchiveOverSomethingFoundIsNotUnpacked(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
at := filepath.Join(dir, "site") at := filepath.Join(dir, "site")
if err := os.Mkdir(at, 0o750); err != nil { if err := os.Mkdir(at, 0o750); err != nil {
@@ -874,7 +874,7 @@ func TestAnArchiveOverSomethingFoundIsNotUnpacked(t *testing.T) {
} }
} }
func TestAProcessWhoseUnitIsFoundIsNotWrittenOverOrRestarted(t *testing.T) { func caseAProcessWhoseUnitIsFoundIsNotWrittenOverOrRestarted(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
was := unitDir was := unitDir
unitDir = dir unitDir = dir
@@ -899,7 +899,7 @@ func TestAProcessWhoseUnitIsFoundIsNotWrittenOverOrRestarted(t *testing.T) {
} }
} }
func TestAUserFoundOnTheMachineKeepsItsShellAndGroups(t *testing.T) { func caseAUserFoundOnTheMachineKeepsItsShellAndGroups(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
m := &machine{containers: map[string]*fakeContainer{}, users: map[string]bool{"hello": true}} m := &machine{containers: map[string]*fakeContainer{}, users: map[string]bool{"hello": true}}
report, _ := applyAdopted(t, adopted(t, untaken("hello-web.user"), report, _ := applyAdopted(t, adopted(t, untaken("hello-web.user"),
@@ -915,7 +915,7 @@ func TestAUserFoundOnTheMachineKeepsItsShellAndGroups(t *testing.T) {
} }
} }
func TestMountingTheMachinesOwnPlumbingIsNotFoundData(t *testing.T) { func caseMountingTheMachinesOwnPlumbingIsNotFoundData(t *testing.T) {
// The runtime's socket, the kernel's filesystems and the clock are on every machine; a // The runtime's socket, the kernel's filesystems and the clock are on every machine; a
// container mounting them shares nothing a predecessor kept (novox/hq ADR 0103). // container mounting them shares nothing a predecessor kept (novox/hq ADR 0103).
dir := t.TempDir() dir := t.TempDir()
@@ -950,7 +950,7 @@ func (u *unreachable) run(_ context.Context, name string, args ...string) (strin
return "", nil return "", nil
} }
func TestARuntimeThatCannotAnswerNeverLetsTheMeshCreateOverAFoundContainer(t *testing.T) { func caseARuntimeThatCannotAnswerNeverLetsTheMeshCreateOverAFoundContainer(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
u := &unreachable{} u := &unreachable{}
d := adopted(t, untaken("hello-web.server"), d := adopted(t, untaken("hello-web.server"),
@@ -970,7 +970,7 @@ func TestARuntimeThatCannotAnswerNeverLetsTheMeshCreateOverAFoundContainer(t *te
} }
} }
func TestAHeldContainerStaysHeldWhenTheRuntimeCannotAnswer(t *testing.T) { func caseAHeldContainerStaysHeldWhenTheRuntimeCannotAnswer(t *testing.T) {
dir, page, m := predecessor(t) dir, page, m := predecessor(t)
d := adopted(t, untaken("hello-web.page", "hello-web.server"), webResources(page)) d := adopted(t, untaken("hello-web.page", "hello-web.server"), webResources(page))
_, state := applyAdopted(t, d, store.State{}, m, dir) _, state := applyAdopted(t, d, store.State{}, m, dir)
@@ -988,7 +988,7 @@ func TestAHeldContainerStaysHeldWhenTheRuntimeCannotAnswer(t *testing.T) {
} }
} }
func TestAVolumeTheRuntimeCannotBeAskedAboutStopsTheContainer(t *testing.T) { func caseAVolumeTheRuntimeCannotBeAskedAboutStopsTheContainer(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
run := func(_ context.Context, name string, args ...string) (string, error) { run := func(_ context.Context, name string, args ...string) (string, error) {
args = oldShape(name, args) args = oldShape(name, args)
@@ -1014,7 +1014,7 @@ func TestAVolumeTheRuntimeCannotBeAskedAboutStopsTheContainer(t *testing.T) {
} }
} }
func TestAUnitSomebodyInstalledIsHeldWhateverStateItIsIn(t *testing.T) { func caseAUnitSomebodyInstalledIsHeldWhateverStateItIsIn(t *testing.T) {
// A predecessor's unit under /etc, deliberately stopped and disabled: starting it would put // A predecessor's unit under /etc, deliberately stopped and disabled: starting it would put
// back a service somebody took down on purpose (novox/hq ADR 0103). // back a service somebody took down on purpose (novox/hq ADR 0103).
dir := t.TempDir() dir := t.TempDir()
@@ -1035,7 +1035,7 @@ func TestAUnitSomebodyInstalledIsHeldWhateverStateItIsIn(t *testing.T) {
} }
} }
func TestAPackagedUnitTheMachineUsesIsStillHeld(t *testing.T) { func caseAPackagedUnitTheMachineUsesIsStillHeld(t *testing.T) {
// The predecessor's own service from a package, running: not the mesh's to restart. // The predecessor's own service from a package, running: not the mesh's to restart.
dir := t.TempDir() dir := t.TempDir()
conf := filepath.Join(dir, "hello.conf") conf := filepath.Join(dir, "hello.conf")
@@ -1054,7 +1054,7 @@ func TestAPackagedUnitTheMachineUsesIsStillHeld(t *testing.T) {
} }
} }
func TestASecondOriginalAtTheSamePathIsKeptToo(t *testing.T) { func caseASecondOriginalAtTheSamePathIsKeptToo(t *testing.T) {
// Held, let go when the module was unassigned, rewritten by the predecessor, found again: // Held, let go when the module was unassigned, rewritten by the predecessor, found again:
// both originals are kept, and the report names the one it kept (novox/hq ADR 0100). // both originals are kept, and the report names the one it kept (novox/hq ADR 0100).
dir := t.TempDir() dir := t.TempDir()
@@ -1083,7 +1083,7 @@ func TestASecondOriginalAtTheSamePathIsKeptToo(t *testing.T) {
} }
} }
func TestAHoldLetGoAndFoundAgainKeepsBothOriginals(t *testing.T) { func caseAHoldLetGoAndFoundAgainKeepsBothOriginals(t *testing.T) {
dir, page, m := predecessor(t) dir, page, m := predecessor(t)
_, state := applyAdopted(t, adopted(t, untaken("hello-web.page", "hello-web.server"), webResources(page)), _, state := applyAdopted(t, adopted(t, untaken("hello-web.page", "hello-web.server"), webResources(page)),
store.State{}, m, dir) store.State{}, m, dir)
@@ -1111,7 +1111,7 @@ func TestAHoldLetGoAndFoundAgainKeepsBothOriginals(t *testing.T) {
} }
} }
func TestAPathTheMeshOnlySetAccessOnIsStillFound(t *testing.T) { func caseAPathTheMeshOnlySetAccessOnIsStillFound(t *testing.T) {
// An access record says the mesh set permissions on a path it does not own — which is what it // An access record says the mesh set permissions on a path it does not own — which is what it
// does to somebody else's directory. It is not a record of making it (novox/hq ADR 0103). // does to somebody else's directory. It is not a record of making it (novox/hq ADR 0103).
dir := t.TempDir() dir := t.TempDir()
@@ -1144,3 +1144,51 @@ func asideShape(args []string) []string {
} }
return args return args
} }
// TestHold runs the cases of this file as subtests of one test (the test budget: one test, a table of cases).
func TestHold(t *testing.T) {
for _, c := range []struct {
name string
run func(*testing.T)
}{
{"AFoundFileOfAnUntakenModuleIsKeptAsItIs", caseAFoundFileOfAnUntakenModuleIsKeptAsItIs},
{"AFoundContainerOfAnUntakenModuleIsNotReplaced", caseAFoundContainerOfAnUntakenModuleIsNotReplaced},
{"WhatIsNotFoundIsCreatedWhenAssigned", caseWhatIsNotFoundIsCreatedWhenAssigned},
{"AFileThisHostWroteIsNotFound", caseAFileThisHostWroteIsNotFound},
{"AContainerAHostMadeIsNotFound", caseAContainerAHostMadeIsNotFound},
{"TheGenesisStoreAdoptedInPlaceIsNotFound", caseTheGenesisStoreAdoptedInPlaceIsNotFound},
{"TakingAModuleReplacesWhatWasHeldAndTheOriginalSurvives", caseTakingAModuleReplacesWhatWasHeldAndTheOriginalSurvives},
{"AHeldFileIsNeverRemovedWhenItsModuleIsUnassigned", caseAHeldFileIsNeverRemovedWhenItsModuleIsUnassigned},
{"AHoldNoLongerDeclaredIsLetGoAndFoundAgainIfItsModuleReturns", caseAHoldNoLongerDeclaredIsLetGoAndFoundAgainIfItsModuleReturns},
{"ACarriedApplyLetsNoHoldGo", caseACarriedApplyLetsNoHoldGo},
{"AHeldFileRewrittenIsReportedAndNotReverted", caseAHeldFileRewrittenIsReportedAndNotReverted},
{"AHeldContainerStoppedOrReplacedIsReportedAndNotRestarted", caseAHeldContainerStoppedOrReplacedIsReportedAndNotRestarted},
{"AHeldFileThatVanishesIsNotCreated", caseAHeldFileThatVanishesIsNotCreated},
{"AConvergedNodeStillReplacesWhatItFinds", caseAConvergedNodeStillReplacesWhatItFinds},
{"AFileWrittenOverIsKeptOnceAndOnlyWhenTheHostHasNoRecordOfIt", caseAFileWrittenOverIsKeptOnceAndOnlyWhenTheHostHasNoRecordOfIt},
{"AFoundDirectoryOfAnUntakenModuleKeepsItsModeOwnerAndContents", caseAFoundDirectoryOfAnUntakenModuleKeepsItsModeOwnerAndContents},
{"ADirectoryMadeInTheSameApplyIsNotFound", caseADirectoryMadeInTheSameApplyIsNotFound},
{"AFoundServiceOfAnUntakenModuleIsNeitherStartedNorEnabledNorRestarted", caseAFoundServiceOfAnUntakenModuleIsNeitherStartedNorEnabledNorRestarted},
{"AHeldServiceIsStillReloadedButNeverRestarted", caseAHeldServiceIsStillReloadedButNeverRestarted},
{"AUnitAPackageOnlyShipsIsNotFound", caseAUnitAPackageOnlyShipsIsNotFound},
{"AServiceWhoseUnitIsNotThereIsAppliedAsUsual", caseAServiceWhoseUnitIsNotThereIsAppliedAsUsual},
{"AContainerThatWouldMountFoundDataIsNotCreated", caseAContainerThatWouldMountFoundDataIsNotCreated},
{"AContainerMountingWhatTheMeshMadeIsCreated", caseAContainerMountingWhatTheMeshMadeIsCreated},
{"ARunOnceStepInAHeldContainerIsHeld", caseARunOnceStepInAHeldContainerIsHeld},
{"AnActionInAHeldContainerIsHeldUntilItsModuleIsTaken", caseAnActionInAHeldContainerIsHeldUntilItsModuleIsTaken},
{"AnArchiveOverSomethingFoundIsNotUnpacked", caseAnArchiveOverSomethingFoundIsNotUnpacked},
{"AProcessWhoseUnitIsFoundIsNotWrittenOverOrRestarted", caseAProcessWhoseUnitIsFoundIsNotWrittenOverOrRestarted},
{"AUserFoundOnTheMachineKeepsItsShellAndGroups", caseAUserFoundOnTheMachineKeepsItsShellAndGroups},
{"MountingTheMachinesOwnPlumbingIsNotFoundData", caseMountingTheMachinesOwnPlumbingIsNotFoundData},
{"ARuntimeThatCannotAnswerNeverLetsTheMeshCreateOverAFoundContainer", caseARuntimeThatCannotAnswerNeverLetsTheMeshCreateOverAFoundContainer},
{"AHeldContainerStaysHeldWhenTheRuntimeCannotAnswer", caseAHeldContainerStaysHeldWhenTheRuntimeCannotAnswer},
{"AVolumeTheRuntimeCannotBeAskedAboutStopsTheContainer", caseAVolumeTheRuntimeCannotBeAskedAboutStopsTheContainer},
{"AUnitSomebodyInstalledIsHeldWhateverStateItIsIn", caseAUnitSomebodyInstalledIsHeldWhateverStateItIsIn},
{"APackagedUnitTheMachineUsesIsStillHeld", caseAPackagedUnitTheMachineUsesIsStillHeld},
{"ASecondOriginalAtTheSamePathIsKeptToo", caseASecondOriginalAtTheSamePathIsKeptToo},
{"AHoldLetGoAndFoundAgainKeepsBothOriginals", caseAHoldLetGoAndFoundAgainKeepsBothOriginals},
{"APathTheMeshOnlySetAccessOnIsStillFound", caseAPathTheMeshOnlySetAccessOnIsStillFound},
} {
t.Run(c.name, c.run)
}
}
+72 -13
View File
@@ -112,18 +112,77 @@ func TestTheRegistryMovesToTheRuntimesModuleWithoutLeavingTheList(t *testing.T)
} }
} }
// And the preview says the same before it happens. // A unit two records name, under opposite conditions, passes between them (design 42 zfs-owns-zfs §2, answer A-21): the
func TestThePlanForgetsARecordOfAUnitStillHeld(t *testing.T) { // going record is forgotten, the preview says so before it happens, and what it found of the unit before the mesh is what
path := filepath.Join(t.TempDir(), "daemon.json") // the staying record gives back when its module leaves, not what the mesh made of it.
var commands []string func TestARecordOfAUnitStillHeldIsForgottenAndPassesWhatItFound(t *testing.T) {
_, state, err := Apply(context.Background(), archHost(t), runtimeDecl(t, path, true), store.State{}, t.Run("the preview forgets it", func(t *testing.T) {
store.OriginDeclared, recordingServices(&commands), nil, nil) path := filepath.Join(t.TempDir(), "daemon.json")
if err != nil { var commands []string
t.Fatal(err) _, state, err := Apply(context.Background(), archHost(t), runtimeDecl(t, path, true), store.State{},
} store.OriginDeclared, recordingServices(&commands), nil, nil)
for _, step := range Plan(runtimeDecl(t, path, false), state, store.OriginDeclared) { if err != nil {
if step.ID == "networking.registry-trust-reload" && step.Verb != "forget" { t.Fatal(err)
t.Errorf("the plan would %s the network's record of a unit docker.runtime holds: %s", step.Verb, step.Why)
} }
} for _, step := range Plan(runtimeDecl(t, path, false), state, store.OriginDeclared) {
if step.ID == "networking.registry-trust-reload" && step.Verb != "forget" {
t.Errorf("the plan would %s the network's record of a unit docker.runtime holds: %s", step.Verb, step.Why)
}
}
})
t.Run("what it found passes to the record that stays", func(t *testing.T) {
// A unit found stopped and disabled, so the mesh's first record of it started it.
active, enabled := false, false
var commands []string
run := func(_ context.Context, name string, args ...string) (string, error) {
args = oldShape(name, args)
line := name + " " + strings.Join(args, " ")
commands = append(commands, line)
switch {
case strings.Contains(line, " start "):
active = true
case strings.Contains(line, " stop "):
active = false
case strings.Contains(line, " enable "):
enabled = true
case strings.Contains(line, " disable "):
enabled = false
case strings.Contains(line, "is-enabled"):
if enabled {
return "enabled", nil
}
return "disabled", nil
case strings.Contains(line, "show") && strings.Contains(line, "ActiveState"):
if active {
return "LoadState=loaded\nActiveState=active\nSubState=running", nil
}
return "LoadState=loaded\nActiveState=inactive\nSubState=dead", nil
}
return "", nil
}
unit := func(id, state, boot string) *declaration.Declaration {
return parse(t, `{"declaration":1,"resources":[{"id":"`+id+`","type":"service","unit":"z.service","state":"`+
state+`","boot":"`+boot+`"}]}`)
}
apply := func(d *declaration.Declaration, known store.State) store.State {
t.Helper()
_, known, err := Apply(context.Background(), archHost(t), d, known, store.OriginDeclared, run, nil, nil)
if err != nil {
t.Fatal(err)
}
return known
}
known := apply(unit("z.on", "running", "enabled"), store.State{})
known = apply(unit("z.off", "stopped", "disabled"), known) // the value changed: one record goes, the other stays
if rec, _ := known.Find("z.off"); rec.Found == nil || rec.Found.State != "stopped" || rec.Found.Boot != "disabled" {
t.Fatalf("the staying record did not receive what the going one found: %+v", rec.Found)
}
commands = nil
apply(somethingElse(t), known) // the module leaves
for _, c := range commands {
if strings.Contains(c, " start ") || strings.Contains(c, " enable ") {
t.Errorf("the unit was given back as the mesh made it, not as it was found: %q", c)
}
}
})
} }
+29
View File
@@ -0,0 +1,29 @@
package apply
import "context"
// The build the artifact store's holder keeps on a build node (novox/hq design 47, *The artifact store's holder takes its
// own new build even when its seat is the one broken*): the holder's program cannot be fetched from the store it serves, so
// its done build names the node that keeps it and the node-engine fetches it from there, through the build seat's verb.
// How is the caller's (a hook on the apply's context, as the files' link is); what apply does is use the bytes in place of
// the source's fetch, check them against the declared digest as every fetch is, and gate the module when they cannot be had.
// KeptFetch gives the bytes of the artifact a module's done build kept on a build node, by the artifact's digest
// ("sha256:" and 64 hex digits, as a declaration names it). kept is false when the module's build kept no such artifact:
// the artifact is then fetched from its source, as before. An error says the artifact is kept and could not be had.
type KeptFetch func(ctx context.Context, module, digest string) (body []byte, kept bool, err error)
type keptFetchKey struct{}
// WithKept is ctx carrying how a kept artifact is fetched.
func WithKept(ctx context.Context, f KeptFetch) context.Context {
if f == nil {
return ctx
}
return context.WithValue(ctx, keptFetchKey{}, f)
}
func keptFetchFrom(ctx context.Context) KeptFetch {
f, _ := ctx.Value(keptFetchKey{}).(KeptFetch)
return f
}
+7
View File
@@ -41,6 +41,7 @@ func ModuleResults(d *declaration.Declaration, report Report, err error) map[str
outcomes = append(append([]Outcome(nil), outcomes...), failed.Done.Outcomes...) outcomes = append(append([]Outcome(nil), outcomes...), failed.Done.Outcomes...)
} }
skipped := map[string]string{} skipped := map[string]string{}
left := map[string][]string{}
for _, o := range outcomes { for _, o := range outcomes {
module, ok := d.ModuleOfID(o.ID) module, ok := d.ModuleOfID(o.ID)
if !ok { if !ok {
@@ -48,6 +49,7 @@ func ModuleResults(d *declaration.Declaration, report Report, err error) map[str
} }
if o.Action == "skipped" { if o.Action == "skipped" {
skipped[module] = o.Detail skipped[module] = o.Detail
left[module] = append(left[module], o.ID)
continue continue
} }
attempted[o.ID] = true attempted[o.ID] = true
@@ -92,6 +94,11 @@ func ModuleResults(d *declaration.Declaration, report Report, err error) map[str
} }
} }
for module, why := range reasons { for module, why := range reasons {
// Each resource left unapplied is named (design 42 zfs-owns-zfs §1).
if ids := left[module]; len(ids) > 0 {
sort.Strings(ids)
why = append(why, "left unapplied: "+strings.Join(ids, ", "))
}
sort.Strings(why) sort.Strings(why)
out[module] = ModuleResult{Failed: true, Detail: strings.Join(why, "; ")} out[module] = ModuleResult{Failed: true, Detail: strings.Join(why, "; ")}
} }
+29 -4
View File
@@ -54,6 +54,7 @@ import (
"strconv" "strconv"
"strings" "strings"
"sync" "sync"
"sync/atomic"
"time" "time"
"git.novox.be/novox/mesh-sdk/go/declaration" "git.novox.be/novox/mesh-sdk/go/declaration"
@@ -303,8 +304,28 @@ func splitQuoted(s string) []string {
return out return out
} }
// engineModule is the module whose resources may place in the engine's own trees. // engineModuleOf answers the module whose resources may place in the engine's own trees: the module whose entry of the
const engineModule = "mesh-host" // catalog half claims the seat node-engine (design 47 G §527), asked of the node's composer; none until one is given.
var engineModuleOf atomic.Pointer[func() string]
// UseEngineModule gives the apply guard the function that answers the node-engine's own module; nil takes it away. The
// guard fixes no name (§364, §527).
func UseEngineModule(f func() string) {
if f == nil {
engineModuleOf.Store(nil)
return
}
engineModuleOf.Store(&f)
}
// isEngineModule is whether module is the node-engine's own.
func isEngineModule(module string) bool {
f := engineModuleOf.Load()
if f == nil || module == "" {
return false
}
return (*f)() == module
}
// passwdFile is the user database homes are read from. A variable so a test names its own. // passwdFile is the user database homes are read from. A variable so a test names its own.
var passwdFile = "/etc/passwd" var passwdFile = "/etc/passwd"
@@ -365,11 +386,15 @@ func below(path, dir string) bool {
// atOrBelow says whether path is dir or below it. // atOrBelow says whether path is dir or below it.
func atOrBelow(path, dir string) bool { return path == dir || below(path, dir) } func atOrBelow(path, dir string) bool { return path == dir || below(path, dir) }
// evalSymlinks follows the links of an existing path; a variable so a test judges paths whose machine it does not touch (a mount
// of the machine it runs on may not answer).
var evalSymlinks = filepath.EvalSymlinks
// resolved is a path with every link in it followed, as far as the path exists, and the rest as declared. // resolved is a path with every link in it followed, as far as the path exists, and the rest as declared.
func resolved(path string) string { func resolved(path string) string {
rest := "" rest := ""
for p := path; ; p = filepath.Dir(p) { for p := path; ; p = filepath.Dir(p) {
if real, err := filepath.EvalSymlinks(p); err == nil { if real, err := evalSymlinks(p); err == nil {
return filepath.Clean(filepath.Join(real, rest)) return filepath.Clean(filepath.Join(real, rest))
} }
if filepath.Dir(p) == p { if filepath.Dir(p) == p {
@@ -531,7 +556,7 @@ func refuseOne(path string, kind placing, module, owner string, declared map[str
return &PlacementRefusedError{Path: path, Why: "nothing is placed in or mounted from " + tree} return &PlacementRefusedError{Path: path, Why: "nothing is placed in or mounted from " + tree}
} }
} }
if module != engineModule { if !isEngineModule(module) {
for _, tree := range engineTrees { for _, tree := range engineTrees {
if atOrBelow(path, tree) { if atOrBelow(path, tree) {
return &PlacementRefusedError{Path: path, Why: tree + " is the node-engine's own, placed in by " + return &PlacementRefusedError{Path: path, Why: tree + " is the node-engine's own, placed in by " +
+25
View File
@@ -178,6 +178,11 @@ func TestAFoundDirectoryIsUsedAsFound(t *testing.T) {
// The module's own directories keep being placed: below /var/lib, below /etc as root's, a container's mount of // The module's own directories keep being placed: below /var/lib, below /etc as root's, a container's mount of
// the machine's docker socket. // the machine's docker socket.
func TestAModulesOwnPlacesAreStillPlaced(t *testing.T) { func TestAModulesOwnPlacesAreStillPlaced(t *testing.T) {
// The paths below are judged as written: no link of them is followed on the machine the test runs on, whose mounts of them
// (a share that does not answer) it does not depend on.
was := evalSymlinks
evalSymlinks = func(p string) (string, error) { return p, nil }
t.Cleanup(func() { evalSymlinks = was })
for _, path := range []string{"/var/lib/notes/data", "/etc/systemd/logind.conf.d", "/var/run/fail2ban", for _, path := range []string{"/var/lib/notes/data", "/etc/systemd/logind.conf.d", "/var/run/fail2ban",
"/srv/notes", "/mnt/plex/config", "/storage/media", "/var/lib/mesh/registry"} { "/srv/notes", "/mnt/plex/config", "/storage/media", "/var/lib/mesh/registry"} {
if err := refusePath(path, placingDirectory, "notes", ""); err != nil { if err := refusePath(path, placingDirectory, "notes", ""); err != nil {
@@ -193,4 +198,24 @@ func TestAModulesOwnPlacesAreStillPlaced(t *testing.T) {
if err := refusePath("/var/lib/notes/data", placingDirectory, "notes", "1001:2000"); err != nil { if err := refusePath("/var/lib/notes/data", placingDirectory, "notes", "1001:2000"); err != nil {
t.Errorf("an owner below /var/lib: %v", err) t.Errorf("an owner below /var/lib: %v", err)
} }
// Design 47 G §364, §527: the node-engine's own trees are placed in by the module the composer says claims its seat,
// whatever that module is called, and by no other; with none said, by nobody.
tree := engineTrees[0]
t.Cleanup(func() { UseEngineModule(nil) })
for _, c := range []struct {
engine func() string
module string
placed bool
}{
{func() string { return "the-engine" }, "the-engine", true},
{func() string { return "the-engine" }, "notes", false},
{func() string { return "the-engine" }, "", false},
{func() string { return "" }, "", false},
{nil, "the-engine", false},
} {
UseEngineModule(c.engine)
if err := refusePath(tree, placingDirectory, c.module, ""); (err == nil) != c.placed {
t.Errorf("%s for module %q: %v, want placed %v", tree, c.module, err, c.placed)
}
}
} }
+110 -4
View File
@@ -4,10 +4,13 @@ import (
"context" "context"
"crypto/sha256" "crypto/sha256"
"encoding/hex" "encoding/hex"
"encoding/json"
"fmt" "fmt"
"maps"
"os" "os"
"path/filepath" "path/filepath"
"runtime" "runtime"
"slices"
"strings" "strings"
"time" "time"
@@ -40,13 +43,16 @@ var daemonRoot = "/var/lib/mesh/daemons"
var unitDir = "/etc/systemd/system" var unitDir = "/etc/systemd/system"
func applyProcess(ctx context.Context, r *declaration.Process, run Runner, func applyProcess(ctx context.Context, r *declaration.Process, run Runner,
changed map[string]bool, previous store.Applied, away *storeAway) (Outcome, error) { changed map[string]bool, previous store.Applied, held []byte, away *storeAway) (Outcome, error) {
out := begin(r) out := begin(r)
out.Action = "unchanged" out.Action = "unchanged"
body, err := fetchMinding(ctx, r.Source, away) // What the apply fetched before anything was replaced (fetchFirst), else fetched now.
if err != nil { body, err := held, error(nil)
return out, err if body == nil {
if body, err = fetchMinding(ctx, r.Source, away); err != nil {
return out, err
}
} }
sum := sha256.Sum256(body) sum := sha256.Sum256(body)
got := "sha256:" + hex.EncodeToString(sum[:]) got := "sha256:" + hex.EncodeToString(sum[:])
@@ -257,6 +263,9 @@ func applyProcess(ctx context.Context, r *declaration.Process, run Runner,
} }
if because != "" { if because != "" {
out.Detail += ", restarted because " + because + " changed" out.Detail += ", restarted because " + because + " changed"
} else if what := unitDifference(previous.Wrote, want); what != "" {
// Said, so a restart nobody asked for names its cause (issue 1966): the file and the input of it, never a value.
out.Detail += ", restarted because " + what + " changed"
} }
if placing.Detail != "" { if placing.Detail != "" {
out.Detail += "; " + placing.Detail out.Detail += "; " + placing.Detail
@@ -265,6 +274,25 @@ func applyProcess(ctx context.Context, r *declaration.Process, run Runner,
return out, nil return out, nil
} }
// NodeEnv is the variable every process the node-engine runs reads its node's name from (design 47 G A-342), the one
// mesh-sdk reads (MESH_NODE).
const NodeEnv = "MESH_NODE"
// withNode is the process as it runs on node: its own env with NodeEnv beside it (design 47 G A-342). The validator
// refuses a resource that sets it, so the mesh's value is the only one; with no node named the process is as declared.
func withNode(r *declaration.Process, node string) *declaration.Process {
if node == "" {
return r
}
p := *r
p.Env = make(map[string]string, len(r.Env)+1)
for k, v := range r.Env {
p.Env[k] = v
}
p.Env[NodeEnv] = node
return &p
}
// unitFor is the unit the mesh writes for a daemon. // unitFor is the unit the mesh writes for a daemon.
// //
// **Generated whole and never edited in place**, the same rule as every other managed file: an // **Generated whole and never edited in place**, the same rule as every other managed file: an
@@ -550,3 +578,81 @@ func digestWritten(wrote string) string {
// DaemonRoot is where unpacked daemons live — and what a witness keeps beside them. // DaemonRoot is where unpacked daemons live — and what a witness keeps beside them.
func DaemonRoot() string { return daemonRoot } func DaemonRoot() string { return daemonRoot }
// unitDifference names what differs between what a process's record says was written and what is wanted now, both
// "sha256:<hex> <unit>": its bundle when the digest differs, and in the unit file each directive and each environment
// word that differs, with inside a word the keys of a JSON object (and, one level down, of its objects) that differ. Names only, never a value, so a secret in a word is never said. Empty
// when nothing differs or there was no record.
func unitDifference(was, want string) string {
if was == "" {
return ""
}
wasDigest, wasUnit, _ := strings.Cut(was, " ")
wantDigest, wantUnit, _ := strings.Cut(want, " ")
var names []string
if wasDigest != wantDigest {
names = append(names, "its bundle")
}
before, after := unitDirectives(wasUnit), unitDirectives(wantUnit)
keys := map[string]bool{}
for k := range before {
keys[k] = true
}
for k := range after {
keys[k] = true
}
for _, k := range slices.Sorted(maps.Keys(keys)) {
if before[k] != after[k] {
names = append(names, "its unit's "+k+valueDifference(before[k], after[k], 2))
}
}
return strings.Join(names, ", ")
}
// unitDirectives is a unit's directives by name: "Environment <word>" for an environment word, the directive's own
// name for the rest, each with its value as written (an environment word's value unescaped, as unitValue escapes it).
func unitDirectives(unit string) map[string]string {
out := map[string]string{}
for _, line := range strings.Split(unit, "\n") {
line = strings.TrimSpace(line)
if line == "" || strings.HasPrefix(line, "#") || strings.HasPrefix(line, "[") {
continue
}
key, value, _ := strings.Cut(line, "=")
if key == "Environment" && strings.HasPrefix(value, `"`) && strings.HasSuffix(value, `"`) {
word, rest, _ := strings.Cut(value[1:len(value)-1], "=")
rest = strings.NewReplacer(`\\`, `\`, `\"`, `"`, "%%", "%").Replace(rest)
out["Environment "+word] = rest
continue
}
out[key] += value
}
return out
}
// valueDifference is what differs inside one directive's value, as " (a, b)": the keys of a JSON object whose value
// differs (going down depth levels), else the names of the name=value entries of a comma-separated list that were
// added, removed or changed. Empty when the value says nothing more than that it differs.
func valueDifference(was, now string, depth int) string {
var a, b map[string]json.RawMessage
if depth > 0 && json.Unmarshal([]byte(was), &a) == nil && json.Unmarshal([]byte(now), &b) == nil && (a != nil || b != nil) {
all := map[string]bool{}
for k := range a {
all[k] = true
}
for k := range b {
all[k] = true
}
var names []string
for _, k := range slices.Sorted(maps.Keys(all)) {
if string(a[k]) == string(b[k]) {
continue
}
inner := valueDifference(string(a[k]), string(b[k]), depth-1)
names = append(names, k+inner)
}
return " (" + strings.Join(names, ", ") + ")"
}
// Anything else is a value: said as differing, never split, since a name before an "=" in it can be a secret.
return ""
}
+54
View File
@@ -319,11 +319,17 @@ func TestAProcessAppliedAgainIsUnchangedAndNotRestarted(t *testing.T) {
} }
d := declare(t, `{"id":"node-tools.runtime","type":"process","name":"node-tools","source":"`+serving(t, body)+ d := declare(t, `{"id":"node-tools.runtime","type":"process","name":"node-tools","source":"`+serving(t, body)+
`","digest":"`+digest+`","run":["node","main.js"],"env":{"MESH_TOOL_MODULES":"a=/x/index.js"}}`) `","digest":"`+digest+`","run":["node","main.js"],"env":{"MESH_TOOL_MODULES":"a=/x/index.js"}}`)
d.For = "anchor"
first, state, err := Apply(context.Background(), archHost(t), d, store.State{}, store.OriginDeclared, run, nil, nil) first, state, err := Apply(context.Background(), archHost(t), d, store.State{}, store.OriginDeclared, run, nil, nil)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
// Design 47 G A-342: the process runs with its node's name as MESH_NODE beside its own env.
if b, err := os.ReadFile(filepath.Join(units, "node-tools.service")); err != nil || !strings.Contains(string(b), `Environment="MESH_NODE=anchor"`) ||
!strings.Contains(string(b), `Environment="MESH_TOOL_MODULES=`) {
t.Errorf("the unit does not carry the node's name beside its env (%v):\n%s", err, b)
}
if o := outcomeOf(first, "node-tools.runtime"); o.Action != "created" { if o := outcomeOf(first, "node-tools.runtime"); o.Action != "created" {
t.Fatalf("first apply: %+v, want created", o) t.Fatalf("first apply: %+v, want created", o)
} }
@@ -357,6 +363,54 @@ func TestAProcessAppliedAgainIsUnchangedAndNotRestarted(t *testing.T) {
if o := outcomeOf(third, "node-tools.runtime"); o.Action != "unchanged" { if o := outcomeOf(third, "node-tools.runtime"); o.Action != "unchanged" {
t.Errorf("third apply: %+v, want unchanged", o) t.Errorf("third apply: %+v, want unchanged", o)
} }
// An input of its unit that differs restarts it, and the line says which (issue 1966).
changed := declare(t, `{"id":"node-tools.runtime","type":"process","name":"node-tools","source":"`+serving(t, body)+
`","digest":"`+digest+`","run":["node","main.js"],"env":{"MESH_TOOL_MODULES":"a=/x/index.js,b=/y/index.js"}}`)
changed.For = "anchor"
fourth, _, err := Apply(context.Background(), archHost(t), changed, state, store.OriginDeclared, run, nil, nil)
if err != nil {
t.Fatal(err)
}
if o := outcomeOf(fourth, "node-tools.runtime"); o.Action != "updated" || !strings.Contains(o.Detail, "its unit's Environment MESH_TOOL_MODULES") ||
strings.Contains(o.Detail, "/y/") {
t.Errorf("fourth apply: %+v, want updated and the word that changed named, no value", o)
}
for _, row := range []struct {
name, was, now, says string
}{
{"a word of a module inside a JSON word", `{"files":{"OWNED":"/a"},"mail":{"X":"1"}}`, `{"files":{"OWNED":"/b"},"mail":{"X":"1"}}`,
"Environment TOOL_ENV (files (OWNED))"},
{"a secret-shaped value", "c2VjcmV0MQ==", "c2VjcmV0Mg==", "Environment TOOL_VALUE"},
{"a URL with a password", "postgres://u:hunter1@h/db?sslmode=x", "postgres://u:hunter2@h/db?sslmode=x", "Environment TOOL_VALUE"},
{"nothing changed", "a=/x/1", "a=/x/1", ""},
} {
t.Run(row.name, func(t *testing.T) {
word := "TOOL_VALUE"
if strings.HasPrefix(row.was, "{") {
word = "TOOL_ENV"
}
r := aProcess()
mk := func(v string) string {
r.Env = map[string]string{word: v}
return "sha256:" + strings.Repeat("a", 64) + " " + unitFor(r)
}
got := unitDifference(mk(row.was), mk(row.now))
if row.says == "" {
if got != "" {
t.Fatalf("said %q for nothing changed", got)
}
return
}
for _, leak := range []string{"c2VjcmV0", "hunter", "sslmode", "postgres", "/b"} {
if strings.Contains(got, leak) {
t.Fatalf("said %q, which holds part of a value (%s)", got, leak)
}
}
if !strings.Contains(got, row.says) {
t.Fatalf("said %q, want it to name %q", got, row.says)
}
})
}
} }
// novox/hq ADR 0193: a bundle compiled to a binary runs itself — `./name` is that file in the // novox/hq ADR 0193: a bundle compiled to a binary runs itself — `./name` is that file in the
+3 -3
View File
@@ -118,10 +118,10 @@ func TestAFileIsJudgedByTheSameGuardAsADirectory(t *testing.T) {
if _, statErr := os.Stat(filepath.Join(home, "data", "notes.conf")); statErr == nil { if _, statErr := os.Stat(filepath.Join(home, "data", "notes.conf")); statErr == nil {
t.Error("a file was written below a directory the engine refused") t.Error("a file was written below a directory the engine refused")
} }
// A refused file has no outcome: it is a failure. Declared first, it is the failure the apply reports, // A refused file has no outcome: it is a failure. Declared first, it is the failure the apply reports, and the
// and the directory the other. // directory after it is left unapplied, as the rest of its module is (design 42 zfs-owns-zfs §1).
var applyErr *Error var applyErr *Error
if !errors.As(err, &applyErr) || applyErr.Resource != "notes.conf" || applyErr.Others != 1 { if !errors.As(err, &applyErr) || applyErr.Resource != "notes.conf" || applyErr.Others != 0 {
t.Errorf("the file below a refused directory is not among the failures: %v", err) t.Errorf("the file below a refused directory is not among the failures: %v", err)
} }
if o := outcomeOf(report, "notes.conf"); o.ID != "" { if o := outcomeOf(report, "notes.conf"); o.ID != "" {
+101
View File
@@ -3,6 +3,8 @@ package apply
import ( import (
"context" "context"
"errors" "errors"
"net/http"
"net/http/httptest"
"strings" "strings"
"testing" "testing"
@@ -104,6 +106,105 @@ func TestARunningContainerIsKeptWhenItsReplacementImageCannotBeObtained(t *testi
t.Errorf("%s: the failure does not name %s: %v", c.name, second, err) t.Errorf("%s: the failure does not name %s: %v", c.name, second, err)
} }
} }
// Issue 1966 (2026-11-08 23:25 UTC): a module whose container serves the artifact store had it replaced, and then
// could not fetch its program from the store it had cut off. Every archive and process is fetched before anything of
// the module is replaced or removed: with the store away, its container and its orphan stay as they were, and the
// failure names the program's source.
t.Run("an artifact that cannot be fetched replaces nothing of its module", func(t *testing.T) {
gone := httptest.NewServer(http.NotFoundHandler())
source := gone.URL + "/program.tar.gz"
gone.Close() // the store is cut off: refused
was := storeAwayAtMost
storeAwayAtMost = 0 // no patience: the refusal fails at once, without waiting
t.Cleanup(func() { storeAwayAtMost = was })
d := parseTrusted(t, `{"declaration":1,"resources":[
{"id":"distribution.store","type":"container","name":"store","image":"`+pinned+`","env":{"X":"new"}},
{"id":"distribution.program","type":"process","name":"program","source":"`+source+`",
"digest":"sha256:`+strings.Repeat("0", 64)+`","run":["program"]}
]}`)
known := store.State{}
known.Record(store.Applied{ID: "distribution.old", Type: string(declaration.TypeContainer), Target: "old",
Module: "distribution"})
var steps []string
removed := map[string]bool{}
run := func(ctx context.Context, name string, args ...string) (string, error) {
if name == "systemctl" {
steps = append(steps, "systemctl "+strings.Join(args, " "))
return "", nil
}
args = oldShape(name, args)
switch args[0] {
case "info":
return "27.0\n", nil
case "container":
if removed[args[len(args)-1]] {
return "", errors.New("no such container")
}
return "true\tsome-older-spec", nil
case "rm", "rename", "stop", "run":
steps = append(steps, args[0]+" "+strings.Join(args[1:], " "))
if args[0] == "rm" {
removed[args[len(args)-1]] = true
}
}
return "", nil
}
report, after, err := Apply(context.Background(), archHost(t), d, known, store.OriginCarried, run, nil, nil)
if len(steps) > 0 {
t.Errorf("the module was touched though its program could not be fetched: %q", steps)
}
if err == nil || !strings.Contains(err.Error(), source) {
t.Errorf("the failure does not name the program's source %s: %v", source, err)
}
if _, still := after.Find("distribution.old"); !still {
t.Error("the module's orphan was removed though its program could not be fetched")
}
if o := outcomeOf(report, "distribution.store"); o.Action != "skipped" {
t.Errorf("the container's outcome is %+v, want skipped", o)
}
})
// Design 47 (W13.8): the holder's own program is fetched from the build node that kept it, not from the store; when it
// cannot be had, or is not the declared digest, nothing of the module is replaced and the failure says so.
for _, row := range []struct {
name string
fetch KeptFetch
wantErr string
}{
{"the node that keeps it is away", func(context.Context, string, string) ([]byte, bool, error) {
return nil, true, errors.New("novox keeps it and does not answer")
}, "novox keeps it and does not answer"},
{"what the node kept is not the declared digest", func(context.Context, string, string) ([]byte, bool, error) {
return []byte("other"), true, nil
}, "what arrived is"},
} {
t.Run("a kept artifact: "+row.name, func(t *testing.T) {
gone := httptest.NewServer(http.NotFoundHandler())
source := gone.URL + "/program.tar.gz"
gone.Close()
was := storeAwayAtMost
storeAwayAtMost = 0
t.Cleanup(func() { storeAwayAtMost = was })
d := parseTrusted(t, `{"declaration":1,"resources":[
{"id":"distribution.program","type":"process","name":"program","source":"`+source+`",
"digest":"sha256:`+strings.Repeat("0", 64)+`","run":["program"]}
]}`)
var asked []string
fetch := func(ctx context.Context, module, digest string) ([]byte, bool, error) {
asked = append(asked, module+" "+digest)
return row.fetch(ctx, module, digest)
}
run := func(ctx context.Context, name string, args ...string) (string, error) { return "", nil }
_, _, err := Apply(WithKept(context.Background(), fetch), archHost(t), d, store.State{}, store.OriginCarried, run, nil, nil)
if err == nil || !strings.Contains(err.Error(), row.wantErr) {
t.Errorf("the failure is %v, want it to say %q", err, row.wantErr)
}
if len(asked) != 1 || asked[0] != "distribution sha256:"+strings.Repeat("0", 64) {
t.Errorf("the kept fetch was asked %q", asked)
}
})
}
} }
func TestAReplacementImageIsPresentBeforeTheRunningContainerIsRemoved(t *testing.T) { func TestAReplacementImageIsPresentBeforeTheRunningContainerIsRemoved(t *testing.T) {
+69
View File
@@ -0,0 +1,69 @@
package apply
import (
"context"
"fmt"
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
// StopModule stops the units and containers the node's record holds for a module and removes nothing: no unit, file,
// container, data or record, so what the module kept survives and a later Apply starts the build it is given (design 47
// G A-98, §297). Units come first, a timer before the service it starts, then containers; a unit the mesh never held
// (Stateless) is left as it is. It answers what it stopped, and the first failure.
func StopModule(ctx context.Context, sys system.System, known store.State, module string, run Runner) ([]string, error) {
var timers, services, containers []store.Applied
for _, r := range known.Resources {
if store.OwnerOf(r, declaration.ModuleGuess) != module {
continue
}
switch r.Type {
case string(declaration.TypeService):
if r.Stateless {
continue
}
if strings.HasSuffix(r.Target, ".timer") {
timers = append(timers, r)
} else {
services = append(services, r)
}
case string(declaration.TypeContainer):
containers = append(containers, r)
}
}
var stopped []string
for _, r := range append(timers, services...) {
manager := managerFor(r.Scope, r.User, run)
if state, err := sys.ServiceState(ctx, manager, r.Target); err != nil {
if strings.Contains(err.Error(), "does not exist on this machine") {
continue
}
return stopped, fmt.Errorf("stopping %s of %s: %w", r.Target, module, err)
} else if state == "stopped" {
continue
}
if err := sys.SetServiceState(ctx, manager, r.Target, "stopped"); err != nil {
return stopped, fmt.Errorf("stopping %s of %s: %w", r.Target, module, err)
}
stopped = append(stopped, r.Target)
}
if len(containers) == 0 {
return stopped, nil
}
cri, err := containerRuntime(ctx, run)
if err != nil {
return stopped, err
}
sort.Slice(containers, func(i, j int) bool { return containers[i].Target < containers[j].Target })
for _, r := range containers {
if _, err := run(ctx, cri, "stop", r.Target); err != nil {
return stopped, fmt.Errorf("stopping the container %s of %s: %w", r.Target, module, err)
}
stopped = append(stopped, r.Target)
}
return stopped, nil
}
+45
View File
@@ -0,0 +1,45 @@
package apply
import (
"context"
"reflect"
"testing"
"github.com/novox/mesh-host/internal/store"
)
// Stop stops a module's timer, then its service, then its container, and leaves every record, and another module's
// units, as they are (design 47 G A-98).
func TestStopModuleStopsAModulesUnitsAndContainersAndRemovesNothing(t *testing.T) {
m := unitsMachine(map[string]*fakeUnit{
"app.service": {active: "active", enabled: "enabled"},
"app.timer": {active: "active", enabled: "enabled"},
"other.service": {active: "active", enabled: "enabled"},
})
state := store.State{}
for _, r := range []store.Applied{
{ID: "app.run", Type: "service", Target: "app.service"},
{ID: "app.tick", Type: "service", Target: "app.timer"},
{ID: "app.c", Type: "container", Target: "app-box"},
{ID: "other.run", Type: "service", Target: "other.service"},
{ID: "other.c", Type: "container", Target: "other-box"},
} {
state.Record(r)
}
got, err := StopModule(context.Background(), archHost(t), state, "app", m.run)
if err != nil {
t.Fatal(err)
}
if want := []string{"app.timer", "app.service", "app-box"}; !reflect.DeepEqual(got, want) {
t.Fatalf("stopped %v, want %v, the timer first", got, want)
}
if m.units["app.service"].active != "inactive" || m.units["app.timer"].active != "inactive" || m.units["other.service"].active != "active" {
t.Fatalf("units are %+v", m.units)
}
if !m.did("docker stop app-box") || m.did("docker stop other-box") || m.did("docker rm") {
t.Fatalf("the machine was asked %v", m.asked)
}
if len(state.Resources) != 5 {
t.Fatalf("a record was removed: %d left", len(state.Resources))
}
}
+2 -2
View File
@@ -9,8 +9,8 @@ import (
vault "git.novox.be/novox/mesh-contracts/identity-and-access/go/seats/meshvault" vault "git.novox.be/novox/mesh-contracts/identity-and-access/go/seats/meshvault"
seat "git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner" seat "git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
catalog "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog" catalog "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
wants "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshwants" wants "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants"
"git.novox.be/novox/mesh-sdk/go/keys" "git.novox.be/novox/mesh-sdk/go/keys"
) )
+3 -3
View File
@@ -4,11 +4,11 @@ package bucketstest
import ( import (
"encoding/json" "encoding/json"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliveryclient" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshdeliveryclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner" "git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshwants" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants"
) )
// Version is the value version the holders write. // Version is the value version the holders write.
+84 -21
View File
@@ -8,7 +8,7 @@ import (
"sort" "sort"
"strings" "strings"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
engcomp "github.com/novox/mesh-host/internal/compositions" engcomp "github.com/novox/mesh-host/internal/compositions"
"git.novox.be/novox/mesh-networking/modules/mesh-wireguard/go/overlay" "git.novox.be/novox/mesh-networking/modules/mesh-wireguard/go/overlay"
@@ -28,7 +28,7 @@ import (
// writer folds them (issue 1126). Each module's Data says whether its manifest declares data. The compositions are unsigned // writer folds them (issue 1126). Each module's Data says whether its manifest declares data. The compositions are unsigned
// (ADR 0593 §6) and not yet by digest (Composed.ByDigest). // (ADR 0593 §6) and not yet by digest (Composed.ByDigest).
func Compositions(in Input, node string, fps map[string]string, mode Mode) (out map[string]engcomp.Composed, skipped []string, err error) { func Compositions(in Input, node string, fps map[string]string, mode Mode) (out map[string]engcomp.Composed, skipped []string, err error) {
out, skipped, _, err = compositions(in, node, fps, mode, false) out, skipped, _, _, err = compositions(in, node, fps, mode, false)
return out, skipped, err return out, skipped, err
} }
@@ -45,7 +45,7 @@ const BusUsersStandIn = "the bus's user list"
// ErrStoodIn, naming each, so a stand-in is never applied as if it were the value; but for the bus's user list // ErrStoodIn, naming each, so a stand-in is never applied as if it were the value; but for the bus's user list
// (BusUsersStandIn), which the bus node holds out of every apply of a composition it made itself. // (BusUsersStandIn), which the bus node holds out of every apply of a composition it made itself.
func CompositionsApplied(in Input, node string, fps map[string]string) (map[string]engcomp.Composed, []string, error) { func CompositionsApplied(in Input, node string, fps map[string]string) (map[string]engcomp.Composed, []string, error) {
out, skipped, _, err := compositions(in, node, fps, Reading, true) out, skipped, _, _, err := compositions(in, node, fps, Reading, true)
return out, skipped, err return out, skipped, err
} }
@@ -56,13 +56,22 @@ func CompositionsApplied(in Input, node string, fps map[string]string) (map[stri
// module the candidate adds, or one missing for a module the node runs. A module whose every value is held composes // module the candidate adds, or one missing for a module the node runs. A module whose every value is held composes
// as CompositionsApplied composes it: the modes differ only in an own secret never issued. // as CompositionsApplied composes it: the modes differ only in an own secret never issued.
func CompositionsShadow(in Input, node string, fps map[string]string) (out map[string]engcomp.Composed, skipped, stood []string, err error) { func CompositionsShadow(in Input, node string, fps map[string]string) (out map[string]engcomp.Composed, skipped, stood []string, err error) {
return compositions(in, node, fps, Foreseeing, false) out, skipped, stood, _, err = compositions(in, node, fps, Foreseeing, false)
return out, skipped, stood, err
} }
// CompositionsIn is the node's compositions in mode with what they stand in for answered (stood, sorted, the bus's user // CompositionsIn is the node's compositions in mode with what they stand in for answered (stood, sorted, the bus's user
// list left out) rather than refused: CompositionsShadow in Foreseeing, and in Reading what CompositionsApplied refuses // list left out) rather than refused: CompositionsShadow in Foreseeing, and in Reading what CompositionsApplied refuses
// before it is refused, so a caller that applies refuses each stand-in to the module it is for alone (issue 1126). // before it is refused, so a caller that applies refuses each stand-in to the module it is for alone (issue 1126).
func CompositionsIn(in Input, node string, fps map[string]string, mode Mode) (out map[string]engcomp.Composed, skipped, stood []string, err error) { func CompositionsIn(in Input, node string, fps map[string]string, mode Mode) (out map[string]engcomp.Composed, skipped, stood []string, err error) {
out, skipped, stood, _, err = compositions(in, node, fps, mode, false)
return out, skipped, stood, err
}
// CompositionsHolding is CompositionsIn with the modules the node leaves out as they run, each with why: one whose claim cannot be
// held or whose credential was refused (issue 1966). Their compositions are not in out, so a caller that applies keeps what it last
// applied of each and says it held, as for any module refused alone.
func CompositionsHolding(in Input, node string, fps map[string]string, mode Mode) (out map[string]engcomp.Composed, skipped, stood []string, held map[string]string, err error) {
return compositions(in, node, fps, mode, false) return compositions(in, node, fps, mode, false)
} }
@@ -75,7 +84,7 @@ func Folds(inOwn bool, fp string) bool {
return inOwn && !strings.HasPrefix(fp, meshcatalogclient.SourcePrefix2) return inOwn && !strings.HasPrefix(fp, meshcatalogclient.SourcePrefix2)
} }
func compositions(in Input, node string, fps map[string]string, mode Mode, applied bool) (out map[string]engcomp.Composed, skipped, stood []string, err error) { func compositions(in Input, node string, fps map[string]string, mode Mode, applied bool) (out map[string]engcomp.Composed, skipped, stood []string, heldBy map[string]string, err error) {
reading := in reading := in
reading.Nodes = append([]Node(nil), in.Nodes...) reading.Nodes = append([]Node(nil), in.Nodes...)
found := false found := false
@@ -99,7 +108,7 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
return module != "" && Folds(inOwn[module], fps[module]) return module != "" && Folds(inOwn[module], fps[module])
} }
if !found { if !found {
return nil, nil, nil, fmt.Errorf("compose: the mesh has no node %q", node) return nil, nil, nil, nil, fmt.Errorf("compose: the mesh has no node %q", node)
} }
// A folded module built by a build of the old form (Node.RunsOwn) is left to the node: composed from what runs of it // A folded module built by a build of the old form (Node.RunsOwn) is left to the node: composed from what runs of it
// (Carried), so what it provides and requires still shapes the rest and what the others read of it (its published // (Carried), so what it provides and requires still shapes the rest and what the others read of it (its published
@@ -152,7 +161,7 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
} }
kept := make(map[string]catalogue.Manifest, len(n.Kept)) kept := make(map[string]catalogue.Manifest, len(n.Kept))
for m, man := range n.Kept { for m, man := range n.Kept {
if man.Stubbed && man.RunsAs == "" { if man.Stubbed && man.RunsAs == "" && !keepsProgram(man) {
man.Bundles = runningBundles(n.RunsTools[m], reading.Catalogue[m]) man.Bundles = runningBundles(n.RunsTools[m], reading.Catalogue[m])
} }
kept[m] = man kept[m] = man
@@ -167,10 +176,10 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
} }
d, problems, err := ComposeIn(reading, node, mode) d, problems, err := ComposeIn(reading, node, mode)
if err != nil { if err != nil {
return nil, nil, nil, err return nil, nil, nil, nil, err
} }
if len(problems) > 0 { if len(problems) > 0 {
return nil, nil, nil, fmt.Errorf("compose: %s's declaration is refused by the node-engine's validator: %v", node, problems) return nil, nil, nil, nil, fmt.Errorf("compose: %s's declaration is refused by the node-engine's validator: %v", node, problems)
} }
for _, x := range d.StoodIn { for _, x := range d.StoodIn {
// The bus's user list is the bus node's own to follow, written alone and never from a composition the node // The bus's user list is the bus node's own to follow, written alone and never from a composition the node
@@ -186,11 +195,11 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
sort.Strings(stood) sort.Strings(stood)
stood = slices.Compact(stood) // an own secret is both stood in for and foreseen stood = slices.Compact(stood) // an own secret is both stood in for and foreseen
if applied && len(stood) > 0 { if applied && len(stood) > 0 {
return nil, nil, nil, fmt.Errorf("%w on %s: %s", ErrStoodIn, node, strings.Join(stood, ", ")) return nil, nil, nil, nil, fmt.Errorf("%w on %s: %s", ErrStoodIn, node, strings.Join(stood, ", "))
} }
body, err := d.Body() body, err := d.Body()
if err != nil { if err != nil {
return nil, nil, nil, err return nil, nil, nil, nil, err
} }
var envelope struct { var envelope struct {
Resources []json.RawMessage `json:"resources"` Resources []json.RawMessage `json:"resources"`
@@ -199,7 +208,7 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
Provisioner string `json:"provisioner_account"` Provisioner string `json:"provisioner_account"`
} }
if err := json.Unmarshal(body, &envelope); err != nil { if err := json.Unmarshal(body, &envelope); err != nil {
return nil, nil, nil, err return nil, nil, nil, nil, err
} }
var kept []json.RawMessage var kept []json.RawMessage
keptPaths := map[string]bool{} keptPaths := map[string]bool{}
@@ -213,7 +222,7 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
Path string `json:"path"` Path string `json:"path"`
} }
if err := json.Unmarshal(raw, &head); err != nil { if err := json.Unmarshal(raw, &head); err != nil {
return nil, nil, nil, err return nil, nil, nil, nil, err
} }
if isCarried(head.Module, head.ID) { if isCarried(head.Module, head.ID) {
continue // what the stub composed of it: the node's own carries what runs of it instead continue // what the stub composed of it: the node's own carries what runs of it instead
@@ -223,11 +232,11 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
// compositionsOf): the node's own composition is then the controller's, byte for byte. // compositionsOf): the node's own composition is then the controller's, byte for byte.
var fields map[string]json.RawMessage var fields map[string]json.RawMessage
if err := json.Unmarshal(raw, &fields); err != nil { if err := json.Unmarshal(raw, &fields); err != nil {
return nil, nil, nil, err return nil, nil, nil, nil, err
} }
delete(fields, "module") delete(fields, "module")
if raw, err = json.Marshal(fields); err != nil { if raw, err = json.Marshal(fields); err != nil {
return nil, nil, nil, err return nil, nil, nil, nil, err
} }
if head.Type == string(declaration.TypeFile) { if head.Type == string(declaration.TypeFile) {
foldedPaths[head.Path] = true foldedPaths[head.Path] = true
@@ -248,7 +257,7 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
if len(carried) > 0 { if len(carried) > 0 {
var err error var err error
if kept, foldedPaths, err = carry(kept, foldedPaths, runsOwn, isCarried); err != nil { if kept, foldedPaths, err = carry(kept, foldedPaths, runsOwn, isCarried); err != nil {
return nil, nil, nil, err return nil, nil, nil, nil, err
} }
for _, p := range runsOwn.Filled { for _, p := range runsOwn.Filled {
if foldedPaths[p] && !slices.Contains(envelope.Filled, p) { if foldedPaths[p] && !slices.Contains(envelope.Filled, p) {
@@ -270,12 +279,12 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
out, err = engcomp.Split(node, kept, fps, filled, out, err = engcomp.Split(node, kept, fps, filled,
engcomp.NodeOwn{Adoption: envelope.Adoption, Provisioner: envelope.Provisioner}) engcomp.NodeOwn{Adoption: envelope.Adoption, Provisioner: envelope.Provisioner})
if err != nil { if err != nil {
return nil, nil, nil, err return nil, nil, nil, nil, err
} }
if len(ownFilled) > 0 { if len(ownFilled) > 0 {
own, err := engcomp.NodeKey(node) own, err := engcomp.NodeKey(node)
if err != nil { if err != nil {
return nil, nil, nil, err return nil, nil, nil, nil, err
} }
c := out[own] c := out[own]
c.Filled = append(c.Filled, ownFilled...) c.Filled = append(c.Filled, ownFilled...)
@@ -288,6 +297,10 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
} }
if m, ok := in.Catalogue[c.Module]; ok && c.Module != "" { if m, ok := in.Catalogue[c.Module]; ok && c.Module != "" {
c.Data = len(m.DataItems()) > 0 c.Data = len(m.DataItems()) > 0
// The archive its files were rendered from, which the parts check counts as fetched (design 47 G A-481a).
if m.TemplatesDigest != "" {
c.Templates = []string{m.TemplatesDigest}
}
out[key] = c out[key] = c
} }
} }
@@ -295,7 +308,7 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
skipped = append(skipped, m) skipped = append(skipped, m)
} }
sort.Strings(skipped) sort.Strings(skipped)
return out, skipped, stood, nil return out, skipped, stood, d.Held, nil
} }
// RunsAnyOf says runs (a composition as it runs) holds a resource of module, by its module or the module its id names. // RunsAnyOf says runs (a composition as it runs) holds a resource of module, by its module or the module its id names.
@@ -379,8 +392,8 @@ func ToolsRunning(running []engcomp.Composed) map[string][]catalogue.Bundle {
} }
var own map[string]string var own map[string]string
for w, v := range words[module] { for w, v := range words[module] {
if w == catalogue.FilesOwnedPaths || w == catalogue.ReadsSecretPlaces { if w == catalogue.FilesOwnedPaths || w == catalogue.ReadsSecretPlaces || w == engcomp.FingerprintVar {
continue // the mesh's to say, composed again continue // the mesh's to say, composed again (the fingerprint is the want's, §519)
} }
if own == nil { if own == nil {
own = map[string]string{} own = map[string]string{}
@@ -480,6 +493,56 @@ func Stub(m catalogue.Manifest) catalogue.Manifest {
// run). What it provides, requires, listens on and grants is its manifest's. A module whose resources runs holds none // run). What it provides, requires, listens on and grants is its manifest's. A module whose resources runs holds none
// of is its Stub, as before. // of is its Stub, as before.
func Carried(m catalogue.Manifest, module string, runs *engcomp.Composed) catalogue.Manifest { func Carried(m catalogue.Manifest, module string, runs *engcomp.Composed) catalogue.Manifest {
stub := carried(m, module, runs)
if catalogue.IsToolRunner(m) && (len(stub.Bundles) != 1 || stub.Bundles[0].Binary == "") {
if b, ok := runningProgram(module, runs); ok {
stub.Bundles = []catalogue.Bundle{b}
}
}
return stub
}
// keepsProgram says a held tool runner carries the program it runs (Carried, runningProgram): its one bundle is that
// program, which the tools bundles it serves never replace, or the modules of their own account it runs are held for
// want of it (lead-47 2026-11-16: 40ed063 kept novox's, and the loop over what the runtime serves dropped it again).
func keepsProgram(m catalogue.Manifest) bool {
return catalogue.IsToolRunner(m) && len(m.Bundles) == 1 && m.Bundles[0].Binary != ""
}
// runningProgram is the program bundle of the tool runner as it runs on the node: its process's source, digest and binary.
// A held tool runner keeps it, so a module of its own account is run by the same program the node runs (novox/hq ADR 0259
// §8) and is not held for want of a runtime that runs (lead-47 2026-11-16: novox's tool runner was held for a build the
// catalog half held no entry of, and held the new indexer with it).
func runningProgram(module string, runs *engcomp.Composed) (catalogue.Bundle, bool) {
if runs == nil {
return catalogue.Bundle{}, false
}
for _, raw := range runs.Resources {
var r struct {
ID string `json:"id"`
Module string `json:"module"`
Type string `json:"type"`
Source string `json:"source"`
Digest string `json:"digest"`
Run []string `json:"run"`
}
if json.Unmarshal(raw, &r) != nil || r.Type != "process" || r.Source == "" || r.Digest == "" || len(r.Run) == 0 {
continue
}
owner := r.Module
if owner == "" {
owner, _ = declaration.ModuleGuess(r.ID)
}
binary, ok := strings.CutPrefix(r.Run[0], "./")
if owner != module || !ok || binary == "" || strings.Contains(binary, "/") {
continue
}
return catalogue.Bundle{Name: "program", Source: registryforms.Forms.Recorded(r.Source), Digest: r.Digest, Binary: binary}, true
}
return catalogue.Bundle{}, false
}
func carried(m catalogue.Manifest, module string, runs *engcomp.Composed) catalogue.Manifest {
if runs == nil { if runs == nil {
return Stub(m) return Stub(m)
} }
+69 -2
View File
@@ -2,8 +2,9 @@ package compose
import ( import (
"encoding/json" "encoding/json"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
engcomp "github.com/novox/mesh-host/internal/compositions" engcomp "github.com/novox/mesh-host/internal/compositions"
"slices"
"sort" "sort"
"strings" "strings"
"testing" "testing"
@@ -19,6 +20,7 @@ func fpFor(c byte) string { return meshcatalogclient.SourcePrefix2 + strings.Rep
// to: every resource once, each in its module's composition at the fingerprint given, the module's ownership said on // to: every resource once, each in its module's composition at the fingerprint given, the module's ownership said on
// it; a module given no fingerprint is named as skipped and composed nowhere. // it; a module given no fingerprint is named as skipped and composed nowhere.
func TestCompositionsAreTheDeclarationSplitByModule(t *testing.T) { func TestCompositionsAreTheDeclarationSplitByModule(t *testing.T) {
t.Run("a held tool runner keeps its program for the modules of their own account", aHeldToolRunnerKeepsItsProgramForTheModulesOfTheirOwnAccount)
in := aSmallMesh(t) in := aSmallMesh(t)
d, _, err := Compose(in, "laptop") d, _, err := Compose(in, "laptop")
if err != nil { if err != nil {
@@ -31,10 +33,19 @@ func TestCompositionsAreTheDeclarationSplitByModule(t *testing.T) {
sort.Strings(want) sort.Strings(want)
fps := map[string]string{"mesh-wireguard": fpFor('a'), "files": fpFor('b')} fps := map[string]string{"mesh-wireguard": fpFor('a'), "files": fpFor('b')}
// A-481a: the composition names the digest of the templates archive its module was rendered from.
templated := in.Catalogue["files"]
templated.TemplatesDigest = "sha256:" + strings.Repeat("e", 64)
in.Catalogue["files"] = templated
out, skipped, err := Compositions(in, "laptop", fps, Foreseeing) out, skipped, err := Compositions(in, "laptop", fps, Foreseeing)
if err != nil || len(skipped) != 0 { if err != nil || len(skipped) != 0 {
t.Fatalf("Compositions = %v, skipped %v", err, skipped) t.Fatalf("Compositions = %v, skipped %v", err, skipped)
} }
for _, c := range out {
if want := map[string][]string{"files": {templated.TemplatesDigest}}[c.Module]; !slices.Equal(c.Templates, want) {
t.Errorf("%s names the templates %v, want %v", c.Module, c.Templates, want)
}
}
var got []string var got []string
for key, c := range out { for key, c := range out {
node, module, fp, isNode, err := engcomp.ParseKey(key) node, module, fp, isNode, err := engcomp.ParseKey(key)
@@ -180,9 +191,65 @@ func TestCompositionsAreTheDeclarationSplitByModule(t *testing.T) {
} }
} }
} }
if env[catalogue.RuntimeToolModules] != served || env[catalogue.RuntimeToolEnv] != `{"lamp":{"LAMP_CONF":"/var/lib/lamp/conf"}}` { // §519: the words each bundle's module is given hold the fingerprint of the want it applies, where there is one.
words := `{"lamp":{"LAMP_CONF":"/var/lib/lamp/conf"}}`
if fp := row.fps["lamp"]; fp != "" {
words = `{"lamp":{"LAMP_CONF":"/var/lib/lamp/conf","MESH_FINGERPRINT":"` + fp + `"}}`
}
if env[catalogue.RuntimeToolModules] != served || env[catalogue.RuntimeToolEnv] != words {
t.Errorf("%s: the runtime serves %v with %v, want what runs of lamp, %s", row.name, t.Errorf("%s: the runtime serves %v with %v, want what runs of lamp, %s", row.name,
env[catalogue.RuntimeToolModules], env[catalogue.RuntimeToolEnv], served) env[catalogue.RuntimeToolModules], env[catalogue.RuntimeToolEnv], served)
} }
} }
} }
// lead-47 2026-11-16: a held tool runner keeps the program it runs (Carried, runningProgram) after what the runtime serves
// is laid on the stubs (Node.RunsTools), so a module of its own account on its node is run by that program, not held
// "the tool runner is not here to run it from" (novox, the new indexer).
func aHeldToolRunnerKeepsItsProgramForTheModulesOfTheirOwnAccount(t *testing.T) {
dg := "sha256:" + strings.Repeat("d", 64)
source := catalogue.ArtifactStoreScheme + "node-tools/runtime/blobs/" + dg
runtime := catalogue.Manifest{Module: "node-tools", Version: "1", ToolRunner: true,
OwnSecrets: catalogue.OwnSecrets{"broker": {Path: "/var/lib/mesh/node-tools/broker"}}}
own := catalogue.Manifest{Module: "mesh-self-check", Version: "1", RunsAs: "mesh-self-check",
OwnSecrets: catalogue.OwnSecrets{"broker": {Path: "/var/lib/mesh/mesh-self-check/broker", Taken: catalogue.TakenAtStart}},
Bundles: []catalogue.Bundle{{Name: "tools", Source: catalogue.ArtifactStoreScheme + "mesh-self-check/tools/blobs/" + dg,
Digest: dg, Language: "go", Loads: []string{"mesh-self-check-tools"}}}}
store := catalogue.Manifest{Module: "artifacts", Version: "1",
Claims: []catalogue.Claim{{Name: "container-image-registry", Scope: catalogue.ScopeMesh}},
Listens: []catalogue.Listening{{Name: "registry", Port: 5100, From: catalogue.FromMesh}}}
runs := engcomp.Composed{Node: "anchor", Module: "node-tools", Resources: []json.RawMessage{json.RawMessage(
`{"id":"node-tools.runtime","module":"node-tools","type":"process","name":"node-tools","source":"` + source +
`","digest":"` + dg + `","run":["./node-tools"],"env":{"MESH_TOOL_MODULES":""}}`)}}
in := aMesh(t)
in.Catalogue["node-tools"], in.Catalogue["artifacts"], in.Catalogue["mesh-self-check"] = runtime, store, own
anchor := nodeOf(t, &in, "anchor")
anchor.Assigned = append(anchor.Assigned, "artifacts", "node-tools", "mesh-self-check")
in.BusMinted = []string{"anchor.node-tools", "anchor.mesh-self-check"}
anchor.SealingKey = keyOf(t, "anchor")
anchor.Held = map[string]bool{"node-tools": true}
anchor.Kept = map[string]catalogue.Manifest{"node-tools": Carried(runtime, "node-tools", &runs)}
anchor.RunsTools = ToolsRunning([]engcomp.Composed{runs})
fps := map[string]string{overlay.Name: fpFor('a'), "artifacts": fpFor('5'), "node-tools": fpFor('e'), "mesh-self-check": fpFor('6')}
out, _, _, err := CompositionsIn(in, "anchor", fps, Foreseeing)
if err != nil {
t.Fatalf("a module of its own account beside a held tool runner: %v", err)
}
ran := false
for _, c := range out {
for _, raw := range c.Resources {
var r struct {
Type string `json:"type"`
Digest string `json:"digest"`
Run []string `json:"run"`
User string `json:"user"`
}
if json.Unmarshal(raw, &r) == nil && r.Type == "process" && r.User == "mesh-self-check" {
ran = r.Digest == dg && len(r.Run) == 1 && r.Run[0] == "./node-tools"
}
}
}
if !ran {
t.Errorf("mesh-self-check is not run by the program node-tools runs: %v", out)
}
}
+2 -2
View File
@@ -23,12 +23,12 @@ type CredentialKey struct {
// in neither list: the vault mints nothing for it (issue 1126). // in neither list: the vault mints nothing for it (issue 1126).
func CredentialsNeeded(in Input, node string) (asConsumer, asProvider []CredentialKey, err error) { func CredentialsNeeded(in Input, node string) (asConsumer, asProvider []CredentialKey, err error) {
c := newComposer(in) c := newComposer(in)
resolved, err := c.resolution(node) resolved, held, err := c.resolutionHolding(node)
if err != nil { if err != nil {
return nil, nil, err return nil, nil, err
} }
for _, n := range resolved.Needs { for _, n := range resolved.Needs {
if n.ByRecord || n.From == "" || credentialless(n) { if _, isHeld := held[n.For]; isHeld || n.ByRecord || n.From == "" || credentialless(n) {
continue continue
} }
asConsumer = append(asConsumer, CredentialKey{Provision: n.Name, Consumer: node, ConsumerModule: n.For, asConsumer = append(asConsumer, CredentialKey{Provision: n.Name, Consumer: node, ConsumerModule: n.For,
+3 -4
View File
@@ -201,10 +201,9 @@ func TestAGrantIsCreatedForEachCredentialMadeForAConsumer(t *testing.T) {
} }
// A consumer whose set does not resolve is skipped, not fatal. // A consumer whose set does not resolve is skipped, not fatal.
nodeOf(t, &in, "laptop").Assigned = append(nodeOf(t, &in, "laptop").Assigned, "files") nodeOf(t, &in, "laptop").Assigned = append(nodeOf(t, &in, "laptop").Assigned, "files")
in.Catalogue["files"] = catalogue.Manifest{Module: "files", Version: "1", Requires: []string{"s3-bucket"}, in.Catalogue["files"] = catalogue.Manifest{Module: "files", Version: "1", Requires: []string{"s3-bucket", "two-give-this"}}
Claims: []catalogue.Claim{{Name: "x", Scope: catalogue.ScopeNode}}} in.Catalogue["giver-a"] = catalogue.Manifest{Module: "giver-a", Version: "1", Provides: catalogue.Offers("two-give-this")}
in.Catalogue["twin"] = catalogue.Manifest{Module: "twin", Version: "1", Claims: []catalogue.Claim{{Name: "x", Scope: catalogue.ScopeNode}}} in.Catalogue["giver-b"] = catalogue.Manifest{Module: "giver-b", Version: "1", Provides: catalogue.Offers("two-give-this")}
nodeOf(t, &in, "laptop").Assigned = append(nodeOf(t, &in, "laptop").Assigned, "twin")
if got, err := Render(in, "anchor", Foreseeing); err != nil || len(got.Rendering.Grants) != 0 { if got, err := Render(in, "anchor", Foreseeing); err != nil || len(got.Rendering.Grants) != 0 {
t.Errorf("a consumer that does not resolve: %v %+v", err, got.Rendering.Grants) t.Errorf("a consumer that does not resolve: %v %+v", err, got.Rendering.Grants)
} }
+65 -4
View File
@@ -75,7 +75,7 @@ func Answer(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]st
return refuse(fmt.Errorf("frombus: the credentials of %s: %w", s.Node, err)) return refuse(fmt.Errorf("frombus: the credentials of %s: %w", s.Node, err))
} }
} }
composed, skipped, stood, err := composeEach(in, s.Node, fps, false, s.Running, failed, ranOf(ctx, r, mesh, s.Running, fps)) composed, skipped, stood, err := composeEach(in, s.Node, fps, false, s.Running, failed, s.RunningManifest, true)
if err != nil { if err != nil {
// Not one module's: the node's own (a value served elsewhere, the mesh's network), said once as the node's. // Not one module's: the node's own (a value served elsewhere, the mesh's network), said once as the node's.
if len(failed) == 0 { if len(failed) == 0 {
@@ -141,6 +141,45 @@ func Answer(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]st
a.Differs = slices.DeleteFunc(cmp.Differs, func(d nodeengine.CandidateDiffers) bool { return !notFailed(d.Module) }) a.Differs = slices.DeleteFunc(cmp.Differs, func(d nodeengine.CandidateDiffers) bool { return !notFailed(d.Module) })
cmp.Added = slices.DeleteFunc(cmp.Added, func(d nodeengine.CandidateAdded) bool { return !notFailed(d.Module) }) cmp.Added = slices.DeleteFunc(cmp.Added, func(d nodeengine.CandidateAdded) bool { return !notFailed(d.Module) })
skipped = slices.DeleteFunc(skipped, func(m string) bool { return !notFailed(m) }) skipped = slices.DeleteFunc(skipped, func(m string) bool { return !notFailed(m) })
// A module the node runs and an assignment names, but that no build can be given to compose at (what runs of it carries
// no fingerprint, it having started before the node-engine wrote them, and the declaration gives none), is held: kept as
// it runs, said as held with why, never removed and never stopped for it. Only a module no assignment names is removed
// (A-410).
for _, m := range skipped {
if !runs[m] {
continue
}
a.Removed = slices.DeleteFunc(a.Removed, func(x string) bool { return x == m })
a.Held = append(a.Held, nodeengine.CandidateRefusal{Module: m,
Why: "what runs of " + m + " has no fingerprint and none is given to compose it at, so it is kept as it runs",
Today: "running, kept as it runs"})
}
skipped = slices.DeleteFunc(skipped, func(m string) bool { return runs[m] })
// And whatever the candidate's assignment of this node names that runs and was composed to nothing for any other reason
// (the module is in no catalogue the composition reads, so it has no resources and is not even skipped): held the same
// way. Only a module no assignment names is removed (A-410).
placedHere := map[string]bool{}
if p, given := s.Assignments[s.Node]; given && p != nil {
for _, m := range p.Modules {
placedHere[m.Module] = true
}
} else if !given {
for _, m := range trunk[s.Node].Modules {
placedHere[m.Module] = true
}
}
a.Removed = slices.DeleteFunc(a.Removed, func(m string) bool {
if !placedHere[m] {
return false
}
a.Held = append(a.Held, nodeengine.CandidateRefusal{Module: m,
Why: "the assignment names " + m + " on this node, and the composition made nothing of it, so it is kept as it runs",
Today: "running, kept as it runs"})
return true
})
if len(a.Removed) == 0 {
a.Removed = nil
}
// What the proposal adds, by node and module: placed by a candidate and not by the trunk on that node. // What the proposal adds, by node and module: placed by a candidate and not by the trunk on that node.
added := map[string]bool{} added := map[string]bool{}
@@ -292,8 +331,12 @@ func standsInFor(x, node string) (string, string) {
// else the manifest it is composed from. Only the part of the offer that cannot compose is left out, by provision // else the manifest it is composed from. Only the part of the offer that cannot compose is left out, by provision
// (culprit): de-spiegel's routes, its names unset, leave route-proxy without them while mailu keeps its smtp account, // (culprit): de-spiegel's routes, its names unset, leave route-proxy without them while mailu keeps its smtp account,
// and the hold says so. // and the hold says so.
//
// search says a refusal is laid on a culprit (culprit). The node composing itself always searches; a composer's sandbox
// (ComposeSent) does not, since it runs the one search for culprits itself, bounded by its request (design 47 G §721;
// A-469j): there the first refusal no hold of the read answers is the node's (err).
func composeEach(in compose.Input, node string, fps map[string]string, applied bool, running []compositions.Composed, failed map[string]string, func composeEach(in compose.Input, node string, fps map[string]string, applied bool, running []compositions.Composed, failed map[string]string,
ran func(module string) (catalogue.Manifest, bool)) (out map[string]compositions.Composed, skipped, stood []string, err error) { ran func(module string) (catalogue.Manifest, bool), search bool) (out map[string]compositions.Composed, skipped, stood []string, err error) {
mode := compose.Foreseeing mode := compose.Foreseeing
if applied { if applied {
mode = compose.Reading mode = compose.Reading
@@ -317,7 +360,9 @@ func composeEach(in compose.Input, node string, fps map[string]string, applied b
} }
if ran != nil { if ran != nil {
if was, ok := ran(m); ok { if was, ok := ran(m); ok {
man.Contributes, man.ContributesMany, man.Listens = was.Contributes, was.ContributesMany, was.Listens // A held module keeps running what it runs, so it keeps offering what that build offers, whatever its version and
// whether the catalog lists the build: the module that requires it is no "nothing provides" (A-480, A-484).
man.Contributes, man.ContributesMany, man.Listens, man.Provides = was.Contributes, was.ContributesMany, was.Listens, was.Provides
} }
} }
src[m] = man src[m] = man
@@ -337,8 +382,12 @@ func composeEach(in compose.Input, node string, fps map[string]string, applied b
hold(m, failed[m], "") hold(m, failed[m], "")
} }
for { for {
out, skipped, stood, err = compose.CompositionsIn(in, node, fps, mode) var leftOut map[string]string
out, skipped, stood, leftOut, err = compose.CompositionsHolding(in, node, fps, mode)
if err != nil { if err != nil {
if !search {
return nil, nil, nil, err
}
module, leave := culprit(in, node, fps, mode, failed, err) module, leave := culprit(in, node, fps, mode, failed, err)
if module == "" { if module == "" {
return nil, nil, nil, firstReason(err, failed) return nil, nil, nil, firstReason(err, failed)
@@ -346,6 +395,18 @@ func composeEach(in compose.Input, node string, fps map[string]string, applied b
hold(module, err.Error(), leave) hold(module, err.Error(), leave)
continue continue
} }
// A module the node leaves out because its claim cannot be held or its credential was refused (issue 1966) is held with
// those words: its handed composition stays and the check names it refused; the rest composes.
heldNow := false
for _, m := range sortedKeys(leftOut) {
if _, done := failed[m]; !done {
hold(m, leftOut[m], "")
heldNow = true
}
}
if heldNow {
continue
}
again := false again := false
if applied && len(stood) > 0 { if applied && len(stood) > 0 {
placed := map[string]bool{} placed := map[string]bool{}
+81 -4
View File
@@ -1,16 +1,18 @@
package frombus package frombus
import ( import (
"bytes"
"context" "context"
"encoding/json" "encoding/json"
"fmt" "fmt"
"git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient" "git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner" "git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
nodeengine "git.novox.be/novox/mesh-contracts/go/seats/nodeengine" nodeengine "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
engcomp "github.com/novox/mesh-host/internal/compositions" engcomp "github.com/novox/mesh-host/internal/compositions"
"maps"
"os" "os"
"reflect" "reflect"
"slices" "slices"
@@ -192,6 +194,47 @@ func answerNamesEveryRefusalAndLeavesAnOldBuildToTheNode(t *testing.T) {
if a = Answer(ctx, b, Mesh{}, Scope{Node: "n1", Assignments: given(b), InOwn: []string{"spiegel"}}, fps, withMembership, "p-h1", 3); a.Verdict != nodeengine.CandidateAnswerVerdictSame { if a = Answer(ctx, b, Mesh{}, Scope{Node: "n1", Assignments: given(b), InOwn: []string{"spiegel"}}, fps, withMembership, "p-h1", 3); a.Verdict != nodeengine.CandidateAnswerVerdictSame {
t.Errorf("the node's own with its bus membership is answered %+v, want the same", a) t.Errorf("the node's own with its bus membership is answered %+v, want the same", a)
} }
// (C) A module the node runs and an assignment names, given no fingerprint to compose at (it started before the node
// wrote them): held as it runs, never removed (A-410). One no assignment names is removed.
noFp := maps.Clone(fps)
delete(noFp, "lemurs")
unmarked := make([]engcomp.Composed, 0, len(running))
for _, c := range running {
if c.Module == "lemurs" {
c.FP = ""
}
unmarked = append(unmarked, c)
}
unmarked = append(unmarked, engcomp.Composed{Node: "n1", Module: "gone", FP: fp("1")})
a = Answer(ctx, b, Mesh{}, Scope{Node: "n1", Assignments: given(b), InOwn: []string{"spiegel"}}, noFp, unmarked, "p-h1", 3)
if !reflect.DeepEqual(a.Removed, []string{"gone"}) {
t.Errorf("removed %v, want only the module no assignment names", a.Removed)
}
heldIt := false
for _, h := range a.Held {
heldIt = heldIt || h.Module == "lemurs" && h.Why != ""
}
if !heldIt || len(a.Added) != 0 {
t.Errorf("a running module with no fingerprint is answered held %+v added %+v, want held with why and not added", a.Held, a.Added)
}
// (D) One the assignment names whose manifest says its name alone (no resources: what stands in for a module whose entry
// is not held) composes to nothing, and is not even skipped: held too, not removed (A-410).
nameOnly := []byte(`{"module":"ghost","version":"0"}`)
ghostDigest := meshcatalogclient.ManifestDigest(nameOnly)
b.manifests[ghostDigest] = nameOnly
b.modules["ghost"] = meshcatalog.ModulesByModuleValue{Manifest: ghostDigest}
pGhost := b.assignments["n1"]
trunkMods := pGhost.Modules
pGhost.Modules = append(append([]meshmoduleassigner.PlacedModule{}, trunkMods...), meshmoduleassigner.PlacedModule{Module: "ghost"})
b.assignments["n1"] = pGhost
ghostRuns := append(append([]engcomp.Composed{}, unmarked...), engcomp.Composed{Node: "n1", Module: "ghost", FP: fp("2")})
a = Answer(ctx, b, Mesh{}, Scope{Node: "n1", Assignments: given(b), InOwn: []string{"spiegel"}}, noFp, ghostRuns, "p-h1", 3)
if !reflect.DeepEqual(a.Removed, []string{"gone"}) || !slices.ContainsFunc(a.Held, func(h nodeengine.CandidateRefusal) bool { return h.Module == "ghost" }) {
t.Errorf("a named module the composition made nothing of: removed %v held %+v, want it held and only gone removed (%.600s)", a.Removed, a.Held, fmt.Sprintf("%+v", a))
}
pGhost.Modules = trunkMods
b.assignments["n1"] = pGhost
delete(b.modules, "ghost")
// Without what runs, the fold composes spiegel from its manifest, as before, and the composition refuses it. // Without what runs, the fold composes spiegel from its manifest, as before, and the composition refuses it.
if _, _, held, err := Compose(ctx, b, Mesh{}, Scope{Node: "n1", InOwn: []string{"spiegel"}}, fps); err != nil || if _, _, held, err := Compose(ctx, b, Mesh{}, Scope{Node: "n1", InOwn: []string{"spiegel"}}, fps); err != nil ||
!strings.Contains(held["spiegel"], "spiegel has not been built for this mesh") { !strings.Contains(held["spiegel"], "spiegel has not been built for this mesh") {
@@ -218,6 +261,35 @@ func answerNamesEveryRefusalAndLeavesAnOldBuildToTheNode(t *testing.T) {
t.Errorf("spiegel in InOwn wanted at %s: %v, held %v, read as %s; want its build's image", sp.Fp2, err, t.Errorf("spiegel in InOwn wanted at %s: %v, held %v, read as %s; want its build's image", sp.Fp2, err,
read.ComposedHow, raw) read.ComposedHow, raw)
} }
// Issue 1921: the input a check is answered with names the manifest spiegel's build was made of (its content
// digest, the same for those bytes compacted inside a request), so the check's composer composes it at that
// build only where the change's manifest is that one.
var compact bytes.Buffer
_ = json.Compact(&compact, bb.manifests[sp.Manifest])
want, _ := compose.ContentDigest(bb.manifests[sp.Manifest])
if got, _ := compose.ContentDigest(compact.Bytes()); err == nil && (want == "" || read.ComposedHow.Manifests["spiegel"] != want || got != want) {
t.Errorf("the manifest spiegel's build was made of: %v, want %s (compacted %s)", read.ComposedHow.Manifests, want, got)
}
// Issue 1921: held by its manifest's verdict, spiegel is still held in what the node composes, and the input a check
// is answered with says what holds it and carries the build the node would compose it at, taken in while done; a
// node's own read takes in no build of it.
sp.Verdict, sp.Problems = meshcatalogclient.VerdictRefused, []string{"p"}
bb.modules["spiegel"] = sp
for _, done := range []bool{true, false} {
if !done {
bb.builds["spiegel "+sp.Fp2].State = builds.BuildBuilding
}
read, err = ComposingInput(ctx, bb, Mesh{}, Scope{Node: "n1"}, map[string]string{"lemurs": fp("e"), "spiegel": sp.Fp2})
hold := read.Holds["spiegel"]
raw, _ = json.Marshal(hold.Built)
if err != nil || !read.Catalogue["spiegel"].Stubbed || hold.By != compose.HeldByVerdict || hold.Build != sp.Fp2 ||
strings.Contains(string(raw), strings.Repeat("7", 64)) != done || (hold.Built == nil) == done {
t.Errorf("spiegel refused, its build done %v: %v, hold %+v (built %s)", done, err, hold, raw)
}
}
if own, err := Input(ctx, bb, Mesh{}, Scope{Node: "n1"}); err != nil || own.Holds["spiegel"] != (compose.Hold{By: compose.HeldByVerdict}) {
t.Errorf("a node's own read of spiegel refused: %v, hold %+v", err, own.Holds["spiegel"])
}
} }
// (C) Issue 1126, C5: what the others read of a module carried as it runs is what runs, never its stub's: the filter // (C) Issue 1126, C5: what the others read of a module carried as it runs is what runs, never its stub's: the filter
@@ -566,7 +638,10 @@ func answerNamesEveryRefusalAndLeavesAnOldBuildToTheNode(t *testing.T) {
siteRuns.FP = fp("7") siteRuns.FP = fp("7")
b.manifests[meshcatalogclient.ManifestDigest(site)] = site b.manifests[meshcatalogclient.ManifestDigest(site)] = site
b.builds["site "+fp("7")] = &builds.BuilderBuild{State: builds.BuildDone, Manifest: meshcatalogclient.ManifestDigest(site)} b.builds["site "+fp("7")] = &builds.BuilderBuild{State: builds.BuildDone, Manifest: meshcatalogclient.ManifestDigest(site)}
got, _, held, err = Compose(ctx, b, Mesh{}, Scope{Node: "n1", Running: []engcomp.Composed{siteRuns}}, fps) // The manifest of the build that runs is read in the one place (A-484), here the manifest of the build 7.
ranSite, _ := catalogue.Decode(site)
got, _, held, err = Compose(ctx, b, Mesh{}, Scope{Node: "n1", Running: []engcomp.Composed{siteRuns},
RunningManifest: func(module string) (catalogue.Manifest, bool) { return ranSite, module == "site" }}, fps)
if err != nil || held["site"] == "" || held["nft"] != "" { if err != nil || held["site"] == "" || held["nft"] != "" {
t.Errorf("site moved to 9090 in its new manifest: %v, held %v; want site held alone", err, held) t.Errorf("site moved to 9090 in its new manifest: %v, held %v; want site held alone", err, held)
} }
@@ -603,7 +678,9 @@ func otherNodeNotResolvedIsNamed(t *testing.T) {
given[n] = &p given[n] = &p
} }
a := Answer(ctx, b, Mesh{}, Scope{Node: "n1", Assignments: given}, map[string]string{"cons": "src2:" + strings.Repeat("1", 64)}, nil, "p-h1", 1) a := Answer(ctx, b, Mesh{}, Scope{Node: "n1", Assignments: given}, map[string]string{"cons": "src2:" + strings.Repeat("1", 64)}, nil, "p-h1", 1)
if a.Verdict != nodeengine.CandidateAnswerVerdictRefused || !strings.Contains(a.Refused, "prov on n2 lacks container-runtime") { // cons is held for the requirement nothing answers (A-452) and its why names n2's provider.
if a.Verdict != nodeengine.CandidateAnswerVerdictRefused || len(a.Refusals) != 1 || a.Refusals[0].Module != "cons" ||
!strings.Contains(a.Refusals[0].Why, "prov on n2 lacks container-runtime") {
t.Errorf("a refusal for want of n2's provider = %+v", a) t.Errorf("a refusal for want of n2's provider = %+v", a)
} }
} }
+293 -53
View File
@@ -45,9 +45,9 @@ import (
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner" "git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine" "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-contracts/identity-and-access/go/clients/meshvaultclient" "git.novox.be/novox/mesh-contracts/identity-and-access/go/clients/meshvaultclient"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
engcomp "github.com/novox/mesh-host/internal/compositions" engcomp "github.com/novox/mesh-host/internal/compositions"
"git.novox.be/novox/mesh-networking/modules/mesh-wireguard/go/overlay" "git.novox.be/novox/mesh-networking/modules/mesh-wireguard/go/overlay"
@@ -81,7 +81,7 @@ type Reader interface {
// and each it grants as a provider, with the copy sealed to it (the vault's `own` with the keys, ADR 0550 §5). A key // and each it grants as a provider, with the copy sealed to it (the vault's `own` with the keys, ADR 0550 §5). A key
// the vault holds no value for is left out; the composition then stands in for it, and Compose refuses it. // the vault holds no value for is left out; the composition then stands in for it, and Compose refuses it.
Credentials(ctx context.Context, node string, asConsumer, asProvider []compose.CredentialKey) ([]compose.Credential, []compose.Issued, error) Credentials(ctx context.Context, node string, asConsumer, asProvider []compose.CredentialKey) ([]compose.Credential, []compose.Issued, error)
// Build is a module's build at a fingerprint, the builder's key (builds.BuilderBuildsBucket, BuilderBuildKey); nil // Build is a module's build at a fingerprint, the builder's key (BuilderBuildKey); nil
// when the builder holds none. The composing node composes each of its modules from its build (issue 1126). // when the builder holds none. The composing node composes each of its modules from its build (issue 1126).
Build(ctx context.Context, module, fp string) (*builds.BuilderBuild, error) Build(ctx context.Context, module, fp string) (*builds.BuilderBuild, error)
// Archive is the archive at url, a kept reference routed to the artifact store as the composing node reaches it // Archive is the archive at url, a kept reference routed to the artifact store as the composing node reaches it
@@ -156,6 +156,17 @@ type Scope struct {
Assignments map[string]*meshmoduleassigner.Assignments Assignments map[string]*meshmoduleassigner.Assignments
InOwn []string InOwn []string
Running []engcomp.Composed Running []engcomp.Composed
// Heard is, by node and module, what each node said it runs, from the node-engine seat's event `running` kept by the
// composing node's own consumer (design 47 G §199): the passing of a seat reads it, and a module nothing was heard of
// counts by the trunk's entry. Never read from a have key, a want or a call.
Heard map[string]map[string]Heard
// Engine is told, at each composing of the node's input, the node-engine's own module: the one whose entry of the catalog half
// claims the seat node-engine, else the one of the node's heard modules that runs the build OwnFingerprint names (design 47 G
// §527); "" when neither shows it. Nil: told to nobody.
Engine func(module string)
// OwnFingerprint is the fingerprint this node-engine was started at (its MESH_FINGERPRINT); empty when it was started
// without one, as an engine of before G.
OwnFingerprint string
// HeldToday is, in an answer, the modules the node keeps today as they run ("node" for its own composition), each with // HeldToday is, in an answer, the modules the node keeps today as they run ("node" for its own composition), each with
// how ("held", "failed", "runs a build of the old form"): a candidate that cannot compose one, or holds one only // how ("held", "failed", "runs a build of the old form"): a candidate that cannot compose one, or holds one only
// behind one of them, keeps it as today, which is said under CandidateAnswer.Held and is no refusal (issue 1126). // behind one of them, keeps it as today, which is said under CandidateAnswer.Held and is no refusal (issue 1126).
@@ -167,6 +178,30 @@ type Scope struct {
// Kept is told, in words, each module composed as what runs shows because its manifest does not yet say it (the tool // Kept is told, in words, each module composed as what runs shows because its manifest does not yet say it (the tool
// runner, the core's credentials; ADR 0601 §4), so the node says it. Nil: said nowhere. // runner, the core's credentials; ADR 0601 §4), so the node says it. Nil: said nowhere.
Kept func(module, words string) Kept func(module, words string)
// Unmade is told, in words, each module placed on the node that is composed from the indexer's manifest and not from a
// build, with why: the indexer's modules hold no build of the new form for it, or the builder holds none done at that
// fingerprint. Nil: said nowhere.
Unmade func(module, why string)
// takeInHeld, set for the input a node answers a check with (ComposingInput), takes in the build of each module placed
// on the node that its manifest's verdict or its decoder holds, into compose.Input.Holds, never into what the node composes: the
// check's composer, judging the change's manifest itself, composes it at that build (issue 1921).
takeInHeld bool
// manifests, set with takeInHeld, is filled with the ContentDigest of the manifest each module placed on the node was
// built from, as it is taken in from that build (compose.ComposedHow.Manifests).
manifests map[string]string
// Holds is told, after Compose read the input, what holds each module it holds by its manifest's verdict or its
// decoder (compose.Input.Holds), with the build it would compose it at, so the node says why it holds it (issue 1921,
// the composition verb). Nil: said nowhere.
Holds func(map[string]compose.Hold)
// Held is the modules the node holds before anything of their builds is read, each with why (a node that has not read
// the seat list, novox/hq ADR 0636 §1): each is held as it runs, as a module refused alone is, so no take-in or
// artifact reference of it can refuse the node (issue 1966). Nil holds none.
Held map[string]string
// RunningManifest answers the manifest of the build a module runs on the composing node, from the one place that reads it
// (design 47 G A-484: its entry, else the composition the node runs of it, else the trunk's manifest): what a held module
// keeps offering the others (what it provides, contributes and listens on), since it keeps running what it runs. False
// where nothing is read; the manifest it is composed from is its offer then. Nil: none is read.
RunningManifest func(module string) (catalogue.Manifest, bool)
} }
// InOwn is, of the modules p places on its node, those whose resources the node's own composition it runs // InOwn is, of the modules p places on its node, those whose resources the node's own composition it runs
@@ -214,21 +249,23 @@ func Input(ctx context.Context, r Reader, mesh Mesh, s Scope) (compose.Input, er
// taken is a module's manifest as a reader takes it (novox/hq ADR 0578 §3): decoded leniently, with the indexer's verdict // taken is a module's manifest as a reader takes it (novox/hq ADR 0578 §3): decoded leniently, with the indexer's verdict
// from its modules value. A refused one, or bytes that do not decode, stands as its stub (unusable) and why says so; a // from its modules value. A refused one, or bytes that do not decode, stands as its stub (unusable) and why says so; a
// manifest not read past a field it set aside holds its name alone and is held the same. // manifest not read past a field it set aside holds its name alone and is held the same.
func taken(name string, raw []byte, m meshcatalog.ModulesByModuleValue) (man catalogue.Manifest, why string) { //
// by is what holds it (compose.HeldByVerdict or compose.HeldByDecoder), empty when nothing does.
func taken(name string, raw []byte, m meshcatalog.ModulesByModuleValue) (man catalogue.Manifest, why, by string) {
if v := meshcatalogclient.ModuleVerdict(name, m); v != nil { if v := meshcatalogclient.ModuleVerdict(name, m); v != nil {
if refused, _ := meshcatalogclient.TakeVerdict(v); refused { if refused, _ := meshcatalogclient.TakeVerdict(v); refused {
return unusable(name, raw), meshcatalogclient.VerdictWhy(*v) return unusable(name, raw), meshcatalogclient.VerdictWhy(*v), compose.HeldByVerdict
} }
} }
man, err := catalogue.Decode(raw) man, err := catalogue.Decode(raw)
switch { switch {
case err != nil: case err != nil:
return unusable(name, raw), fmt.Sprintf("the manifest of %s does not decode: %v", name, err) return unusable(name, raw), fmt.Sprintf("the manifest of %s does not decode: %v", name, err), compose.HeldByDecoder
case man.ReadNoFurther(): case man.ReadNoFurther():
return unusable(name, raw), fmt.Sprintf("the manifest of %s could not be read past a field this reader does not "+ return unusable(name, raw), fmt.Sprintf("the manifest of %s could not be read past a field this reader does not "+
"know: %s", name, man.SetAside()) "know: %s", name, man.SetAside()), compose.HeldByDecoder
} }
return man, "" return man, "", ""
} }
// input is Input; with refuse set, a module placed on the composing node whose build cannot be read or taken in, or whose // input is Input; with refuse set, a module placed on the composing node whose build cannot be read or taken in, or whose
@@ -239,6 +276,11 @@ func taken(name string, raw []byte, m meshcatalog.ModulesByModuleValue) (man cat
func input(ctx context.Context, r Reader, mesh Mesh, s Scope, refuse func(module string, err error), wanted map[string]string) (compose.Input, error) { func input(ctx context.Context, r Reader, mesh Mesh, s Scope, refuse func(module string, err error), wanted map[string]string) (compose.Input, error) {
b, err := readBase(ctx, r, mesh, s) b, err := readBase(ctx, r, mesh, s)
b.refuseOwn(refuse) b.refuseOwn(refuse)
// The engine is named from the catalogue the base read, before the rest of the input, so a node whose input fails
// still serves its seat (root-free): live on novox 2026-11-14 no pass ended, so the seat was never served (A-403).
if s.Engine != nil && len(b.in.Catalogue) > 0 {
s.Engine(EngineModule(b.in, s.Node, s.OwnFingerprint))
}
if err != nil { if err != nil {
return compose.Input{}, err return compose.Input{}, err
} }
@@ -374,9 +416,13 @@ func readBase(ctx context.Context, r Reader, mesh Mesh, s Scope) (base, error) {
// follows (issue 774: power's bar item refused every node's composition, 2026-10-30; ADR 0578 §3: one refused // follows (issue 774: power's bar item refused every node's composition, 2026-10-30; ADR 0578 §3: one refused
// manifest silenced disk-load for 45 hours). It stands in as its stub, so what it provides and requires still // manifest silenced disk-load for 45 hours). It stands in as its stub, so what it provides and requires still
// shapes the rest; it is held on the composing node where it is placed there, and elsewhere only its offer is read. // shapes the rest; it is held on the composing node where it is placed there, and elsewhere only its offer is read.
man, why := taken(name, raw, m) man, why, by := taken(name, raw, m)
if why != "" { if why != "" {
named(&in.Refused, name, why) named(&in.Refused, name, why)
if in.Holds == nil {
in.Holds = map[string]compose.Hold{}
}
in.Holds[name] = compose.Hold{By: by}
if ownPlaced[name] { if ownPlaced[name] {
b.ownRefused = append(b.ownRefused, [2]string{name, why}) b.ownRefused = append(b.ownRefused, [2]string{name, why})
} }
@@ -526,6 +572,7 @@ func (b base) finish(ctx context.Context, r Reader, build func(ctx context.Conte
if err := fromBuilds(ctx, r, build, &in, s, b.assignments[s.Node], b.modules, read, parse, refuse, wanted); err != nil { if err := fromBuilds(ctx, r, build, &in, s, b.assignments[s.Node], b.modules, read, parse, refuse, wanted); err != nil {
return compose.Input{}, err return compose.Input{}, err
} }
keepHeard(&in, s, read)
// Its own modules' secrets, sealed to its own key by the vault: a composition carries them sealed (ADR 0550 §5). // Its own modules' secrets, sealed to its own key by the vault: a composition carries them sealed (ADR 0550 §5).
// Asked only when a module placed on it declares one: a node none of whose modules has a secret composes // Asked only when a module placed on it declares one: a node none of whose modules has a secret composes
// whether or not the vault answers. // whether or not the vault answers.
@@ -556,6 +603,61 @@ func (b base) finish(ctx context.Context, r Reader, build func(ctx context.Conte
return in, nil return in, nil
} }
// Heard is one `running` event as the scope keeps it: the fingerprint of the build, the digest of its manifest, or Stopped.
type Heard struct {
Fingerprint string
Manifest string
Stopped bool
}
// EngineModule is the node-engine's own module (design 47 G §527): the module whose catalog entry claims the seat node-engine,
// which the validator makes one per revision (§508). With none claiming it, which only a revision of before G shows, as at a
// revert of G, the module this node heard run the build its own fingerprint names; "" when neither shows it.
func EngineModule(in compose.Input, node, ownFingerprint string) string {
for _, name := range sortedKeys(in.Catalogue) {
if resolution.ClaimsEngine(in.Catalogue[name]) {
return name
}
}
if ownFingerprint == "" {
return ""
}
for i := range in.Nodes {
if in.Nodes[i].Name != node {
continue
}
for _, module := range sortedKeys(in.Nodes[i].Heard) {
if in.Nodes[i].Heard[module].Fingerprint == ownFingerprint {
return module
}
}
}
return ""
}
// keepHeard puts, as each node's Heard, what it said it runs of each module it is assigned, with the manifest of the build
// read by the event's digest; a digest that cannot be read leaves the entry unread, and the passing says so (§459).
func keepHeard(in *compose.Input, s Scope, read func(module, digest string) (catalogue.Manifest, error)) {
for i := range in.Nodes {
n := &in.Nodes[i]
heard := s.Heard[n.Name]
if len(heard) == 0 {
continue
}
n.Heard = map[string]compose.Heard{}
for _, module := range sortedKeys(heard) {
h := heard[module]
got := compose.Heard{Fingerprint: h.Fingerprint, Stopped: h.Stopped}
if !h.Stopped && h.Manifest != "" {
if man, err := read(module, h.Manifest); err == nil {
got.Manifest, got.Read = man, true
}
}
n.Heard[module] = got
}
}
}
// Compose is the node's compositions over the bus (ADR 0547 §2): input for s, then composed at the fingerprints fps // Compose is the node's compositions over the bus (ADR 0547 §2): input for s, then composed at the fingerprints fps
// gives each module in the Reading mode, as the controller composes them, by the one rule its answer to a candidate // gives each module in the Reading mode, as the controller composes them, by the one rule its answer to a candidate
// follows too (composeEach, issue 1126): a module refused alone (its build unread or not taken in, its composition // follows too (composeEach, issue 1126): a module refused alone (its build unread or not taken in, its composition
@@ -564,7 +666,10 @@ func (b base) finish(ctx context.Context, r Reader, build func(ctx context.Conte
// node never applies a stand-in. Only a refusal that is no single module's (the node's own resources, a value served // node never applies a stand-in. Only a refusal that is no single module's (the node's own resources, a value served
// elsewhere, a stand-in for the node's own) refuses the node, err. // elsewhere, a stand-in for the node's own) refuses the node, err.
func Compose(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]string) (out map[string]engcomp.Composed, skipped []string, held map[string]string, err error) { func Compose(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]string) (out map[string]engcomp.Composed, skipped []string, held map[string]string, err error) {
held = map[string]string{} held = maps.Clone(s.Held)
if held == nil {
held = map[string]string{}
}
// A node refused whole says what it held on the way, since the refusal can be of a held module's stub (issue 774, // A node refused whole says what it held on the way, since the refusal can be of a held module's stub (issue 774,
// 2026-10-30 19:30: node-tools held, then "its build produced 0 bundle(s)" with its own refusal unsaid). // 2026-10-30 19:30: node-tools held, then "its build produced 0 bundle(s)" with its own refusal unsaid).
defer func() { defer func() {
@@ -580,6 +685,19 @@ func Compose(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]s
if err != nil { if err != nil {
return nil, nil, held, err return nil, nil, held, err
} }
if s.Engine != nil {
s.Engine(EngineModule(in, s.Node, s.OwnFingerprint))
}
if s.Holds != nil {
holds := map[string]compose.Hold{}
for name := range held {
if h, said := in.Holds[name]; said {
h.Build = fps[name]
holds[name] = h
}
}
s.Holds(holds)
}
// The pair credentials its composition reads, asked of the vault sealed to it (ADR 0550 §5). // The pair credentials its composition reads, asked of the vault sealed to it (ADR 0550 §5).
asConsumer, asProvider, err := compose.CredentialsNeeded(in, s.Node) asConsumer, asProvider, err := compose.CredentialsNeeded(in, s.Node)
if err != nil { if err != nil {
@@ -590,7 +708,7 @@ func Compose(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]s
return nil, nil, held, fmt.Errorf("frombus: the credentials of %s: %w", s.Node, err) return nil, nil, held, fmt.Errorf("frombus: the credentials of %s: %w", s.Node, err)
} }
} }
out, skipped, _, err = composeEach(in, s.Node, fps, true, s.Running, held, ranOf(ctx, r, mesh, s.Running, fps)) out, skipped, _, err = composeEach(in, s.Node, fps, true, s.Running, held, s.RunningManifest, true)
return out, skipped, held, err return out, skipped, held, err
} }
@@ -600,11 +718,12 @@ func Compose(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]s
// credential: the sandbox composes no pair value. // credential: the sandbox composes no pair value.
func ComposingInput(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]string) (compose.Input, error) { func ComposingInput(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]string) (compose.Input, error) {
held := map[string]string{} held := map[string]string{}
s.takeInHeld, s.manifests = true, map[string]string{}
in, err := input(ctx, r, mesh, s, func(module string, err error) { held[module] = err.Error() }, fps) in, err := input(ctx, r, mesh, s, func(module string, err error) { held[module] = err.Error() }, fps)
if err != nil { if err != nil {
return compose.Input{}, err return compose.Input{}, err
} }
c := &compose.ComposedHow{Node: s.Node, Builds: maps.Clone(fps), Held: held} c := &compose.ComposedHow{Node: s.Node, Builds: maps.Clone(fps), Held: held, Manifests: s.manifests}
for _, run := range s.Running { for _, run := range s.Running {
if _, isHeld := held[run.Module]; isHeld && run.Module != "" { if _, isHeld := held[run.Module]; isHeld && run.Module != "" {
c.Running = append(c.Running, run) c.Running = append(c.Running, run)
@@ -616,36 +735,146 @@ func ComposingInput(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[s
// ComposeSent composes node from an input ComposingInput read, through the path Compose takes (composeEach, the read's holds // ComposeSent composes node from an input ComposingInput read, through the path Compose takes (composeEach, the read's holds
// first): what a check's sandbox runs. False when the input says no way of composing node (an input from a node-engine // first): what a check's sandbox runs. False when the input says no way of composing node (an input from a node-engine
// before ComposedHow): the caller cannot compose it as the node does. // before ComposedHow): the caller cannot compose it as the node does. A node the change refuses is a *Refuses.
func ComposeSent(in compose.Input, node string) (map[string]engcomp.Composed, bool, error) { //
// It runs the one search for culprits (design 47 G §721; A-469j, A-468i): where the composition fails with no module held
// that the node held neither in the read nor before the change, and no path owned twice names its owners, each of
// s.Changed is held in turn, one composition each, and the first whose hold ends the problem (the try composes or fails
// otherwise) is blamed with it; one that leaves the machine not composing stays held, and the search goes on (A-470v). At most s.Tries tries; past them each module not yet tried is blamed NotTried. With no
// tries there is no search: the failure is the node's own (err).
func ComposeSent(in compose.Input, node string, s Search) (map[string]engcomp.Composed, bool, error) {
c := in.ComposedHow c := in.ComposedHow
if c == nil || c.Node != node { if c == nil || c.Node != node {
return nil, false, nil return nil, false, nil
} }
held := maps.Clone(c.Held) compose1 := func(also ...string) (map[string]engcomp.Composed, map[string]string, error) {
if held == nil { held := maps.Clone(c.Held)
held = map[string]string{} if held == nil {
held = map[string]string{}
}
// A module the node holds composing without the change is held as there: the candidate keeps it as the node does,
// which is no change and no search (issue 1126; design 47 G §721, HeldBefore left out of blamed).
for m, why := range c.HeldBefore {
if _, was := held[m]; !was {
held[m] = why
}
}
for _, m := range also {
held[m] = triedHeld
}
out, _, _, err := composeEach(in, node, c.Builds, false, c.Running, held, nil, false)
return out, held, err
} }
out, _, _, err := composeEach(in, node, c.Builds, false, c.Running, held, nil) ours := func(m string) bool {
if err != nil { _, was := c.Held[m]
return nil, true, err _, before := c.HeldBefore[m]
return !was && !before
} }
out, held, err := compose1()
// A module the sandbox held that the node-engine held neither in the read nor composing without the change is the change // A module the sandbox held that the node-engine held neither in the read nor composing without the change is the change
// refusing it: the node does not compose, said in that module's words, never passed with the rest composed (issue 774, // refusing it: the node does not compose, said in that module's words, never passed with the rest composed (issue 774,
// review of the sandbox input). // review of the sandbox input).
var refused []string var refused []string
blamed := map[string]string{}
for _, m := range sortedKeys(held) { for _, m := range sortedKeys(held) {
_, was := c.Held[m] if ours(m) {
if _, before := c.HeldBefore[m]; !was && !before {
refused = append(refused, m+": "+held[m]) refused = append(refused, m+": "+held[m])
blamed[m] = held[m]
} }
} }
if len(refused) > 0 { // Two modules owning one path are each blamed, with that path, outside the search (design 47 G §721; A-468l).
return nil, true, fmt.Errorf("on %s the change refuses %s", node, strings.Join(refused, "; ")) var refusal *resolution.Refusal
if err != nil && errors.As(err, &refusal) {
for _, p := range refusal.OwnedTwice {
words := fmt.Sprintf("%s and %s both declare the path %q", p.Modules[0], p.Modules[1], p.Path)
for _, owner := range p.Modules {
if ours(owner) {
blamed[owner] = words
}
}
}
} }
return out, true, nil // A hold that changes the problem but leaves the machine not composing keeps that module held and blamed, and the
// search goes on against the retry's problem, from the first changed module not held, within the same tries, until the
// machine composes or no hold changes it (A-470v): every module the change refuses is blamed, never only the first.
if err != nil && len(blamed) == 0 && s.Tries > 0 {
problem, tries := err.Error(), 0
var kept []string
search:
for {
tried := map[string]bool{} // against the problem of now
for _, mod := range s.Changed {
if slices.Contains(kept, mod) {
continue
}
if tries == s.Tries {
for _, untried := range s.Changed {
if _, done := blamed[untried]; !done && !tried[untried] {
blamed[untried] = NotTried
}
}
break search
}
tries++
tried[mod] = true
_, _, again := compose1(append(slices.Clone(kept), mod)...)
if again == nil {
blamed[mod] = problem
break search
}
if again.Error() != problem {
blamed[mod], problem = problem, again.Error()
kept = append(kept, mod)
continue search
}
}
break // no hold changes the problem
}
}
if len(blamed) == 0 {
return out, true, err
}
words := strings.Join(refused, "; ")
if err != nil {
words = err.Error()
}
e := &Refuses{Node: node, words: words}
for _, m := range sortedKeys(blamed) {
e.Blamed = append(e.Blamed, Blamed{Module: m, Problem: blamed[m]})
}
return nil, true, e
} }
// Search is the composer's one search for culprits in ComposeSent (design 47 G §721): the candidate's changed modules
// placed on the node, in assignment order, and the request's blame_tries.
type Search struct {
Changed []string
Tries int
}
// NotTried is the problem of a changed module the search for culprits did not hold, its bound reached (design 47 G §721).
const NotTried = "not tried: the search reached its bound"
// triedHeld is the words a module is held with while the search for culprits tries it.
const triedHeld = "held by the composer's search for the module the machine's problem is of"
// Refuses is ComposeSent's refusal of a node by the change, answered structured (design 47 G §721; A-469j): each module
// the sandbox held that its node-engine held neither in the read nor composing without the change (HeldBefore), and
// each other owner of a path one of them was held for, with its problem.
type Refuses struct {
Node string
Blamed []Blamed
words string
}
// Blamed is one module a refusal of the change blames, with its problem.
type Blamed struct {
Module string
Problem string
}
func (e *Refuses) Error() string { return fmt.Sprintf("on %s the change refuses %s", e.Node, e.words) }
// Plans is the provisioning plan of each module handed to the provisioner on the scope's node, composed from the bus // Plans is the provisioning plan of each module handed to the provisioner on the scope's node, composed from the bus
// alone (novox/hq ADR 0570 clause 2): the controller's handedPlans over the same input (mesh-control // alone (novox/hq ADR 0570 clause 2): the controller's handedPlans over the same input (mesh-control
// cmd/mesh-controller/compose_sdk.go), its rendering in the Foreseeing mode with the mesh's generators. A plan names // cmd/mesh-controller/compose_sdk.go), its rendering in the Foreseeing mode with the mesh's generators. A plan names
@@ -767,33 +996,6 @@ func parser(Mesh) func([]byte) (catalogue.Manifest, error) {
return catalogue.Decode return catalogue.Decode
} }
// ranOf is what a module last offered the others, read when it is held (composeEach, issue 1960): its manifest at the
// build its own composition runs at (running), where that is another build than the one it is composed at (fps) and
// the builder holds it done. False where nothing is known of it (it runs no composition of its own, it runs the build
// it is composed at, or its build is unread): the manifest it is composed from is its offer then. Only what it says to
// others is taken of it, which needs no take-in of its artifacts.
func ranOf(ctx context.Context, r Reader, mesh Mesh, running []engcomp.Composed, fps map[string]string) func(string) (catalogue.Manifest, bool) {
parse := parser(mesh)
return func(module string) (catalogue.Manifest, bool) {
for _, c := range running {
if c.Module != module || c.FP == "" || c.FP == fps[module] {
continue
}
b, err := r.Build(ctx, module, c.FP)
if err != nil || b == nil || b.State != builds.BuildDone || b.Manifest == "" {
return catalogue.Manifest{}, false
}
raw, err := r.Manifest(ctx, b.Manifest)
if err != nil {
return catalogue.Manifest{}, false
}
man, err := parse(raw)
return man, err == nil
}
return catalogue.Manifest{}, false
}
}
// fromBuilds puts in the built manifest of each module placed on the composing node, at the trunk's FP2, and of each // fromBuilds puts in the built manifest of each module placed on the composing node, at the trunk's FP2, and of each
// module it keeps at another build (Scope.KeptAt, into its node's Kept), read from the builder's done key and resolved // module it keeps at another build (Scope.KeptAt, into its node's Kept), read from the builder's done key and resolved
// by the one take-in rule (takein.Manifest), a templates archive read in the artifact store as the node reaches it // by the one take-in rule (takein.Manifest), a templates archive read in the artifact store as the node reaches it
@@ -901,7 +1103,14 @@ func fromBuilds(ctx context.Context, r Reader, build func(ctx context.Context, m
} }
} }
for _, pm := range p.Modules { for _, pm := range p.Modules {
if why, held := s.Held[pm.Module]; held && refuse != nil {
stub(pm.Module, errors.New(why)) // nothing of its build is read: it keeps what runs (Scope.Held)
continue
}
m, ok := modules[pm.Module] m, ok := modules[pm.Module]
if (!ok || m.Fp2 == "") && s.Unmade != nil && wanted[pm.Module] != "" {
s.Unmade(pm.Module, "the indexer's modules hold no build of the new form for it (fingerprint "+m.Fp2+"), so it is composed from the manifest")
}
if !ok || m.Fp2 == "" || keptAt[pm.Module] != "" || compose.Folds(slices.Contains(s.InOwn, pm.Module), wanted[pm.Module]) { if !ok || m.Fp2 == "" || keptAt[pm.Module] != "" || compose.Folds(slices.Contains(s.InOwn, pm.Module), wanted[pm.Module]) {
// A module it runs inside its own composition is composed there as before while it is folded: no build of the // A module it runs inside its own composition is composed there as before while it is folded: no build of the
// new form is what it runs (Scope.InOwn). Wanted at a build of the new form it is no longer folded (compose.Folds, // new form is what it runs (Scope.InOwn). Wanted at a build of the new form it is no longer folded (compose.Folds,
@@ -909,7 +1118,23 @@ func fromBuilds(ctx context.Context, r Reader, build func(ctx context.Context, m
continue continue
} }
if _, refused := in.Refused[pm.Module]; refused { if _, refused := in.Refused[pm.Module]; refused {
continue // held as its stub: no build of a refused manifest is taken in (ADR 0578 §3) // Held as its stub: no build of a refused manifest is composed (ADR 0578 §3). For a check, the build the node
// would compose it at is said beside it, and taken in when done, for the check's composer alone.
if hold, said := in.Holds[pm.Module]; s.takeInHeld && said {
hold.Build = m.Fp2
if w := wanted[pm.Module]; w != "" {
hold.Build = w
}
// Its stub names no build, so the builder is asked: a build not done, or not taken in, is awaited.
if b, err := build(ctx, pm.Module, hold.Build); err == nil && b != nil && b.State == builds.BuildDone {
if resolved, err := built(pm.Module, hold.Build, b); err == nil {
hold.Built = &resolved
}
}
in.Holds = maps.Clone(in.Holds)
in.Holds[pm.Module] = hold
}
continue
} }
if man, ok := in.Catalogue[pm.Module]; !ok || (man.Build == nil && !man.Templated()) { if man, ok := in.Catalogue[pm.Module]; !ok || (man.Build == nil && !man.Templated()) {
continue // nothing is built of it: its manifest is what it composes from continue // nothing is built of it: its manifest is what it composes from
@@ -931,6 +1156,13 @@ func fromBuilds(ctx context.Context, r Reader, build func(ctx context.Context, m
stub(pm.Module, fmt.Errorf("frombus: %s is wanted at %s, of which the builder holds no done build yet; it "+ stub(pm.Module, fmt.Errorf("frombus: %s is wanted at %s, of which the builder holds no done build yet; it "+
"keeps running what it runs", pm.Module, m.Fp2)) "keeps running what it runs", pm.Module, m.Fp2))
} }
if s.Unmade != nil && wanted[pm.Module] != "" {
state := "none"
if b != nil {
state = string(b.State)
}
s.Unmade(pm.Module, "the builder holds no done build of it at "+m.Fp2+" (build: "+state+"), so it is composed from the indexer's manifest, which names no artifacts")
}
continue // nothing of it runs: composed from the indexer's manifest, as before continue // nothing of it runs: composed from the indexer's manifest, as before
} }
resolved, err := built(pm.Module, m.Fp2, b) resolved, err := built(pm.Module, m.Fp2, b)
@@ -942,6 +1174,14 @@ func fromBuilds(ctx context.Context, r Reader, build func(ctx context.Context, m
continue continue
} }
in.Catalogue[pm.Module] = resolved in.Catalogue[pm.Module] = resolved
if s.manifests != nil {
// Unread or not JSON, it is left out: the check's composer then compares what the manifests say (issue 1921).
if raw, err := r.Manifest(ctx, b.Manifest); err == nil {
if digest, err := compose.ContentDigest(raw); err == nil {
s.manifests[pm.Module] = digest
}
}
}
} }
for i := range in.Nodes { for i := range in.Nodes {
if in.Nodes[i].Name != s.Node { if in.Nodes[i].Name != s.Node {
+33 -2
View File
@@ -15,10 +15,11 @@ import (
"git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient" "git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner" "git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
nodeengine "git.novox.be/novox/mesh-contracts/go/seats/nodeengine" nodeengine "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"github.com/novox/mesh-host/internal/compositions" "github.com/novox/mesh-host/internal/compositions"
"maps"
"os" "os"
"reflect" "reflect"
"slices" "slices"
@@ -575,6 +576,36 @@ func TestInput(t *testing.T) {
t.Errorf("the want's build is kept on the composing node alone: %v, %v", in.Nodes, err) t.Errorf("the want's build is kept on the composing node alone: %v, %v", in.Nodes, err)
} }
// Design 47 G §199 (test 56): what each node said it runs (the seat node-engine's event running) is that node's Heard,
// with the manifest read by the event's digest; a digest that cannot be read leaves the entry unread, and a stopped
// module has none.
in, err = Input(ctx, kept, Mesh{}, Scope{Node: "laptop", Heard: map[string]map[string]Heard{
"laptop": {"xterm": {Fingerprint: "f1", Manifest: digest}, "media": {Stopped: true}}, "anchor": {"xterm": {Fingerprint: "f2", Manifest: "gone"}}}})
if err != nil {
t.Fatal(err)
}
for _, n := range in.Nodes {
h := n.Heard["xterm"]
switch {
case n.Name == "laptop" && (!h.Read || h.Fingerprint != "f1" || h.Manifest.Module != "xterm" || !n.Heard["media"].Stopped):
t.Errorf("laptop's Heard holds %+v", n.Heard)
case n.Name == "anchor" && (h.Read || h.Fingerprint != "f2"):
t.Errorf("anchor's Heard holds %+v, want f2 unread", n.Heard)
}
}
// Design 47 G §527: the node-engine's own module is the one whose entry claims its seat; with none, the one this node heard
// run the build its own fingerprint names; with neither, none.
if got := EngineModule(in, "laptop", "f1"); got != "xterm" {
t.Errorf("by the own fingerprint: %q, want xterm", got)
}
if got := EngineModule(in, "laptop", "other"); got != "" {
t.Errorf("an unheard fingerprint names %q", got)
}
in.Catalogue = maps.Clone(in.Catalogue)
in.Catalogue["the-engine"] = catalogue.Manifest{Module: "the-engine", Claims: []catalogue.Claim{{Name: "node-engine"}}}
if got := EngineModule(in, "laptop", "f1"); got != "the-engine" {
t.Errorf("by the claim: %q, want the-engine", got)
}
// Issue 1126, the seed's refusals. restic and the provisioner's client as the catalogue holds them (mesh-catalog // Issue 1126, the seed's refusals. restic and the provisioner's client as the catalogue holds them (mesh-catalog
// c05aa5e6), restic moved with an own secret the vault leaves out (it hands it to the client, mesh-vault ownnode.go // c05aa5e6), restic moved with an own secret the vault leaves out (it hands it to the client, mesh-vault ownnode.go
// handed): composed from the indexer's manifest restic is not handed and stands in for that secret, as novox's, // handed): composed from the indexer's manifest restic is not handed and stands in for that secret, as novox's,
@@ -14,9 +14,9 @@ import (
"errors" "errors"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner" "git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine" "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"github.com/novox/mesh-host/internal/compose" "github.com/novox/mesh-host/internal/compose"
"github.com/novox/mesh-host/internal/compose/frombus" "github.com/novox/mesh-host/internal/compose/frombus"
"strings" "strings"
@@ -7,9 +7,9 @@ import (
"git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient" "git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner" "git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine" "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
engcomp "github.com/novox/mesh-host/internal/compositions" engcomp "github.com/novox/mesh-host/internal/compositions"
"reflect" "reflect"
"slices" "slices"
+84 -5
View File
@@ -7,9 +7,10 @@ import (
"fmt" "fmt"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner" "git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine" "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-host/internal/compose" "github.com/novox/mesh-host/internal/compose"
"github.com/novox/mesh-host/internal/compose/resolution" "github.com/novox/mesh-host/internal/compose/resolution"
engcomp "github.com/novox/mesh-host/internal/compositions" engcomp "github.com/novox/mesh-host/internal/compositions"
@@ -39,11 +40,17 @@ func aNode(t *testing.T, manifests ...string) (*bus, map[string]string) {
// refusal can be of the held module's stub (node-tools held, then "its build produced 0 bundle(s)", its own why unsaid). // refusal can be of the held module's stub (node-tools held, then "its build produced 0 bundle(s)", its own why unsaid).
func aNodeRefusedWholeSaysWhatItHeld(t *testing.T) { func aNodeRefusedWholeSaysWhatItHeld(t *testing.T) {
refused := `{"module":"power","version":"1","items":[{"collection":"contribution:node-bar:block","attributes":{"shows":"state"},"template":"files/a"}]}` refused := `{"module":"power","version":"1","items":[{"collection":"contribution:node-bar:block","attributes":{"shows":"state"},"template":"files/a"}]}`
needs := `{"module":"lamp","version":"1","requires":["nothing-provides-this"]}` needs := `{"module":"lamp","version":"1","requires":["two-provide-this"]}`
b, fps := aNode(t, refused, needs) giverA := `{"module":"giver-a","version":"1","provides":[{"name":"two-provide-this"}]}`
giverB := `{"module":"giver-b","version":"1","provides":[{"name":"two-provide-this"}]}`
b, fps := aNode(t, refused, needs, giverA, giverB)
// The two givers are in the catalogue and placed on no node, so lamp's requirement has two answers and nobody chose.
placed := b.assignments["n1"]
placed.Modules = placed.Modules[:2]
b.assignments["n1"] = placed
refuse(b, "power", `"shows" is none of them`) refuse(b, "power", `"shows" is none of them`)
_, _, held, err := Compose(context.Background(), b, Mesh{}, Scope{Node: "n1"}, fps) _, _, held, err := Compose(context.Background(), b, Mesh{}, Scope{Node: "n1"}, fps)
if err == nil || !strings.Contains(err.Error(), "nothing provides") || !strings.Contains(err.Error(), `held before: power: `) || if err == nil || !strings.Contains(err.Error(), "choose one") || !strings.Contains(err.Error(), `held before: power: `) ||
!strings.Contains(err.Error(), `"shows" is none of them`) { !strings.Contains(err.Error(), `"shows" is none of them`) {
t.Errorf("the node refused whole with %v (held %v), want lamp's refusal and power held before, with its why", err, held) t.Errorf("the node refused whole with %v (held %v), want lamp's refusal and power held before, with its why", err, held)
} }
@@ -300,3 +307,75 @@ func aHeldModuleOfItsOwnAccountNeverRefusesTheNode(t *testing.T) {
} }
} }
} }
// Issue 1966: two modules claiming one seat held once cost the second claimant, not the node: Compose holds it with the claim's
// words, composes the first, and (the node-engine keeping what it handed of a held module) leaves its composition out, whether or
// not it runs.
func TestASecondClaimantOfASeatIsHeldAndTheNodeComposesTheRest(t *testing.T) {
first := `{"module":"alpha","version":"1","claims":[{"name":"the-seat","scope":"node"}]}`
second := `{"module":"beta","version":"1","claims":[{"name":"the-seat","scope":"node"}]}`
for _, running := range []bool{false, true} {
b, fps := aNode(t, first, second)
scope := Scope{Node: "n1"}
if running {
scope.Running = []engcomp.Composed{{Node: "n1", Module: "beta", FP: fps["beta"], Resources: nil}}
}
got, _, held, err := Compose(context.Background(), b, Mesh{}, scope, fps)
if err != nil || len(held) != 1 || !strings.Contains(held["beta"], "alpha and beta both claim") {
t.Fatalf("running %v: err %v, held %v; want beta held with the claim's words", running, err, held)
}
for _, c := range got {
if c.Module == "beta" {
t.Errorf("running %v: the held claimant is composed: %+v", running, c)
}
}
// A check that adds the second claimant is refused for it, in the claim's words, and not judged to compose.
p := b.assignments["n1"]
ans := Answer(context.Background(), b, Mesh{}, Scope{Node: "n1", Assignments: map[string]*meshmoduleassigner.Assignments{"n1": &p}},
fps, scope.Running, "p-h1", 1)
if ans.Verdict != nodeengine.CandidateAnswerVerdictRefused || len(ans.Refusals) != 1 || ans.Refusals[0].Module != "beta" ||
!strings.Contains(ans.Refusals[0].Why, "both claim") {
t.Errorf("running %v: the answer is %+v, want beta refused with the claim's words", running, ans)
}
}
// A-480 rows, in this test to keep the module's count:
// A-480 (design 47 G): a module the node holds keeps running what it runs, and so keeps providing what its running build provides,
// whatever its catalogue entry: the module that requires it is not held for "nothing provides" (live, 2026-11-17 22:52: feh on g14
// held for "x11-display" while xorg runs there, held only for want of an entry of its wanted build). The offer is the running
// manifest of the build it runs, from the one place that reads it (A-484); a module neither running nor composable still gives
// "nothing provides".
const xorgRunning = `{"module":"xorg","version":"1","provides":[{"name":"x11-display"}]}`
feh := `{"module":"feh","version":"1","requires":["x11-display"]}`
nameOnly := `{"module":"xorg","version":"0"}`
for _, row := range []struct {
name string
scope func(fps map[string]string, b *bus) Scope
provided bool
}{
{"the one place answers the manifest of the build it runs (A-484)", func(fps map[string]string, b *bus) Scope {
man, _ := catalogue.Decode([]byte(xorgRunning))
return Scope{RunningManifest: func(module string) (catalogue.Manifest, bool) { return man, module == "xorg" }}
}, true},
{"the one place reads none", func(map[string]string, *bus) Scope {
return Scope{RunningManifest: func(string) (catalogue.Manifest, bool) { return catalogue.Manifest{}, false }}
}, false},
{"nothing of it runs", func(map[string]string, *bus) Scope { return Scope{} }, false},
} {
b, fps := aNode(t, nameOnly, feh)
scope := row.scope(fps, b)
scope.Node = "n1"
scope.Held = map[string]string{"xorg": "entry-missing: the catalog half holds no entry of the build"}
_, _, held, err := Compose(context.Background(), b, Mesh{}, scope, fps)
why, fehHeld := held["feh"]
if row.provided && (err != nil || fehHeld) {
t.Errorf("%s: err %v, feh held %q; want feh composed with xorg held as what runs", row.name, err, why)
}
if !row.provided && !(fehHeld && strings.Contains(why, `nothing provides "x11-display"`)) && (err == nil || !strings.Contains(err.Error(), "nothing provides")) {
t.Errorf("%s: err %v, held %v; want nothing provides", row.name, err, held)
}
if !strings.Contains(held["xorg"], "entry-missing") {
t.Errorf("%s: xorg's own hold is %q, want it said", row.name, held["xorg"])
}
}
}
+2 -2
View File
@@ -19,9 +19,9 @@ import (
"git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient" "git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner" "git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine" "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds" "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"github.com/novox/mesh-host/internal/buckets" "github.com/novox/mesh-host/internal/buckets"
"github.com/novox/mesh-host/internal/compose" "github.com/novox/mesh-host/internal/compose"
"github.com/novox/mesh-host/internal/compose/resolution" "github.com/novox/mesh-host/internal/compose/resolution"
+62
View File
@@ -16,6 +16,9 @@
package compose package compose
import ( import (
"bytes"
"crypto/sha256"
"encoding/hex"
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
@@ -114,11 +117,36 @@ type Input struct {
// (compose.Stub), so it is held: nothing of its own is composed, while what it provides and requires still shapes the // (compose.Stub), so it is held: nothing of its own is composed, while what it provides and requires still shapes the
// rest. A module not named is accepted, a missing verdict included. // rest. A module not named is accepted, a missing verdict included.
Refused map[string]string `json:",omitempty"` Refused map[string]string `json:",omitempty"`
// Holds is, by module Refused names, what holds it and the build its node would compose it at (issue 1921): a check's
// composer that judges the change's manifest itself (novox/hq ADR 0578 §2) composes a module refused by the running
// indexer alone from that build, and names one whose build it is not given. Nil from a node-engine before it.
Holds map[string]Hold `json:",omitempty"`
// SetAside is each module whose manifest has a field the reader's decoder does not know, by module, with what was set // SetAside is each module whose manifest has a field the reader's decoder does not know, by module, with what was set
// aside (catalogue.Manifest.SetAside): the rest of it is composed, and the reader names it (ADR 0578 §6). // aside (catalogue.Manifest.SetAside): the rest of it is composed, and the reader names it (ADR 0578 §6).
SetAside map[string]string `json:",omitempty"` SetAside map[string]string `json:",omitempty"`
} }
// What holds a module Refused names (Hold.By).
const (
// HeldByVerdict is a module the indexer's judge refused the manifest of (its verdict).
HeldByVerdict = "verdict"
// HeldByDecoder is a module whose bytes this reader's decoder does not decode, or cannot read past a field it does not
// know: no judge of the manifest overrules it.
HeldByDecoder = "decoder"
)
// Hold is what holds one module Refused names, and the build its node would compose it at (Input.Holds).
type Hold struct {
// By is HeldByVerdict or HeldByDecoder.
By string
// Build is the fingerprint the node would compose it at, its want's or else the indexer's newest; empty where it is not
// placed on the composing node or names no build.
Build string `json:",omitempty"`
// Built is its manifest as the node takes it in from that build, when the builder holds it done; nil while that build
// is awaited (or not asked: only a node's answer to a check asks it).
Built *catalogue.Manifest `json:",omitempty"`
}
// Issued is one pair credential made for a consumer of a provider (inv.Secret): ForProvider is its copy sealed to the // Issued is one pair credential made for a consumer of a provider (inv.Secret): ForProvider is its copy sealed to the
// provider, optional. // provider, optional.
type Issued struct { type Issued struct {
@@ -212,6 +240,9 @@ type Node struct {
// own resolution only, never to another node's, and a declaration carries the kept commit for the module. // own resolution only, never to another node's, and a declaration carries the kept commit for the module.
Kept map[string]catalogue.Manifest Kept map[string]catalogue.Manifest
KeptBuilds map[string]string KeptBuilds map[string]string
// Heard is, by module, what this node said it runs, from the node-engine seat's event `running` (design 47 G §199): the
// passing of a seat resolves the build each assignment runs from it. Absent for a module nothing was heard of.
Heard map[string]Heard
// Contract is the contract of its node-engine's newest statement (inv.HealthOf); Stated is whether it ever stated one. // Contract is the contract of its node-engine's newest statement (inv.HealthOf); Stated is whether it ever stated one.
Contract int Contract int
Stated bool Stated bool
@@ -258,6 +289,28 @@ type ComposedHow struct {
// HeldBefore is each module the node holds composing without the change (the candidate's base), held by the read or // HeldBefore is each module the node holds composing without the change (the candidate's base), held by the read or
// blamed later for a refusal: the change refuses none of them (issue 774, review round 1). // blamed later for a refusal: the change refuses none of them (issue 774, review round 1).
HeldBefore map[string]string HeldBefore map[string]string
// Manifests is, by module taken in from its build, the ContentDigest of the manifest that build was made of (its
// build record's manifest), filled in the input a node answers a check with: the check's composer composes a module
// at its build only where the change's manifest is that one (issue 1921). Nil from a node-engine before it.
Manifests map[string]string
}
// ContentDigest is the hex sha256 of a manifest's JSON in one form, its keys sorted and its spacing removed, its numbers
// as written: the same for the bytes its repository holds and for those bytes carried inside a request, which JSON
// encoding compacts on the way (a composition request through a build module), so the two are compared as one manifest.
func ContentDigest(raw []byte) (string, error) {
dec := json.NewDecoder(bytes.NewReader(raw))
dec.UseNumber()
var v any
if err := dec.Decode(&v); err != nil {
return "", fmt.Errorf("compose: the manifest is not JSON: %w", err)
}
canonical, err := json.Marshal(v)
if err != nil {
return "", err
}
sum := sha256.Sum256(canonical)
return hex.EncodeToString(sum[:]), nil
} }
// Place is where a node is, as its assignment says (inv.Overlay). // Place is where a node is, as its assignment says (inv.Overlay).
@@ -387,6 +440,15 @@ type Credential struct {
Why string Why string
} }
// Heard is what a node's `running` event for a module said: the build's fingerprint, or Stopped (no fingerprint, §365), with
// the manifest of the build's entry read by the event's manifest digest (Read false when it could not be read).
type Heard struct {
Fingerprint string
Stopped bool
Manifest catalogue.Manifest
Read bool
}
// Run is the build an assignment names: the commit it is of and the build made from it (chain.Ref). // Run is the build an assignment names: the commit it is of and the build made from it (chain.Ref).
type Run struct { type Run struct {
Commit string Commit string
+1 -1
View File
@@ -283,7 +283,7 @@ func (c *composer) onTheNetworkNow() []Node {
if len(n.Assigned) == 0 { if len(n.Assigned) == 0 {
continue continue
} }
got, err := resolution.Resolve(c.in.Catalogue, n.Assigned, got, _, err := resolution.ResolveHolding(c.in.Catalogue, n.Assigned,
catalogue.Node{Name: n.Name, Site: n.Place.Site, Capabilities: n.Capabilities}, catalogue.Node{Name: n.Name, Site: n.Place.Site, Capabilities: n.Capabilities},
catalogue.World{Unchecked: true, Holdings: c.in.Holdings}) catalogue.World{Unchecked: true, Holdings: c.in.Holdings})
if err != nil { if err != nil {
+7 -9
View File
@@ -164,9 +164,8 @@ func TestANodeNotOnThePrivateNetworkGetsNoAddressAndNoNetwork(t *testing.T) {
func TestARefusedNodeIsSaidWhenTheNetworkCannotBeComposed(t *testing.T) { func TestARefusedNodeIsSaidWhenTheNetworkCannotBeComposed(t *testing.T) {
in := aMesh(t) in := aMesh(t)
nodeOf(t, &in, "anchor").Place.Key = "" nodeOf(t, &in, "anchor").Place.Key = ""
one := catalogue.Manifest{Module: "rival-one", Version: "1", Claims: []catalogue.Claim{{Name: "the-seat", Scope: catalogue.ScopeNode}}} in.Catalogue["rival-one"] = catalogue.Manifest{Module: "rival-one", Version: "1", Resources: []map[string]any{{"id": "r", "type": "directory", "path": "/var/lib/the-same", "mode": "0700"}}}
two := catalogue.Manifest{Module: "rival-two", Version: "1", Claims: []catalogue.Claim{{Name: "the-seat", Scope: catalogue.ScopeNode}}} in.Catalogue["rival-two"] = catalogue.Manifest{Module: "rival-two", Version: "1", Resources: []map[string]any{{"id": "r", "type": "directory", "path": "/var/lib/the-same", "mode": "0700"}}}
in.Catalogue["rival-one"], in.Catalogue["rival-two"] = one, two
hub := nodeOf(t, &in, "anchor") hub := nodeOf(t, &in, "anchor")
hub.Assigned = append(hub.Assigned, "rival-one", "rival-two") // its own set is incoherent: the hub does not resolve hub.Assigned = append(hub.Assigned, "rival-one", "rival-two") // its own set is incoherent: the hub does not resolve
_, err := Generators(in) _, err := Generators(in)
@@ -293,14 +292,13 @@ func TestEachFieldOfTheInputReachesTheGenerator(t *testing.T) {
} }
laptop := nodeOf(t, &in, "laptop") laptop := nodeOf(t, &in, "laptop")
laptop.Assigned = append(laptop.Assigned, "holder-one", "holder-two") laptop.Assigned = append(laptop.Assigned, "holder-one", "holder-two")
// Without a key: a machine that does not resolve and holds a complete place keeps it on the network (issue 1833). // Two claimants and nobody on record: the second is held and the machine stays on the network (issue 1966).
laptop.Place.Key = "" if got, err := Resolve(in, "laptop"); err != nil || len(got.Held) != 1 || got.Held["holder-two"] == "" || addressesOf(in)["laptop"] != "10.77.0.2" {
if got := addressesOf(in); got["laptop"] != "" { t.Fatalf("two claimants, none on record: held %v, %v", got.Held, err)
t.Fatalf("a machine whose set claims one seat twice is on the network: %v", got)
} }
in.Holdings = []catalogue.Held{{Claim: "mesh-broker", Scope: catalogue.ScopeMesh, Node: "laptop", Module: "holder-one"}} in.Holdings = []catalogue.Held{{Claim: "mesh-broker", Scope: catalogue.ScopeMesh, Node: "laptop", Module: "holder-one"}}
if got := addressesOf(in); got["laptop"] != "10.77.0.2" { if got, err := Resolve(in, "laptop"); err != nil || len(got.Held) != 0 {
t.Errorf("a holder on record did not settle the seat: %v", got) t.Errorf("a holder on record did not settle the seat: held %v, %v", got.Held, err)
} }
}) })
} }
+52 -3
View File
@@ -2,6 +2,7 @@ package compose
import ( import (
"encoding/json" "encoding/json"
"errors"
"fmt" "fmt"
"sort" "sort"
"strings" "strings"
@@ -37,6 +38,9 @@ type Declared struct {
// ADR 0163, rule 6), sorted; LeftOutWhy is why each was. // ADR 0163, rule 6), sorted; LeftOutWhy is why each was.
LeftOut []string LeftOut []string
LeftOutWhy map[string]string LeftOutWhy map[string]string
// Held is, of those, each module left out because its claim cannot be held or its credential was refused, and why (issue
// 1966): the node keeps what runs of it and the composer says it held. Not on the wire.
Held map[string]string
// Not on the wire: // Not on the wire:
Received map[string]map[string][]catalogue.Contribution Received map[string]map[string][]catalogue.Contribution
@@ -60,7 +64,8 @@ type Declared struct {
ForeseenPorts []string ForeseenPorts []string
// CoreBoxesWanted are the core credentials whose box the send must ask the vault for and keep. // CoreBoxesWanted are the core credentials whose box the send must ask the vault for and keep.
CoreBoxesWanted []CoreBoxWanted CoreBoxesWanted []CoreBoxWanted
// Passes are the seats the mesh would pass on, for the caller to record, and Said the lines found while composing. // Passes are the seats composed as passed to their only claimant (ADR 0660), recorded by nobody here, and Said the lines found
// while composing, the passes among them.
Passes Passes Passes Passes
Said []string Said []string
} }
@@ -167,6 +172,34 @@ func ComposeIn(in Input, node string, mode Mode) (Declared, []string, error) {
return d, validate.Declaration(body), nil return d, validate.Declaration(body), nil
} }
// composeWithoutStoreFetchers (it answers the modules it left out, and why) is resolution.Compose where a module that fetches from the artifact store while the node reaches
// none (resolution.NoStoreError, whatever the shape: an image, a source, a tools bundle, a runtime program, a templates
// archive) is left out of the declaration with its words and the rest composes again; the node is never refused whole for it
// (design 47 G A-461, A-452). The rendering returned carries the modules so left out, for the filter and the rest to read.
func composeWithoutStoreFetchers(plan catalogue.Resolution, with catalogue.Rendering) (catalogue.Composed, catalogue.Rendering, map[string]string, error) {
var held map[string]string
for {
composed, err := resolution.Compose(plan, with)
var none *resolution.NoStoreError
if !errors.As(err, &none) {
return composed, with, held, err
}
if _, already := with.NotPlaced[none.Module]; already {
return catalogue.Composed{}, with, nil, err
}
left := make(map[string]string, len(with.NotPlaced)+1)
for m, why := range with.NotPlaced {
left[m] = why
}
left[none.Module] = none.Error()
with.NotPlaced = left
if held == nil {
held = map[string]string{}
}
held[none.Module] = none.Error()
}
}
// declaration is the controller's declarationWith over the plain Input. // declaration is the controller's declarationWith over the plain Input.
func (c *composer) declaration(nodeName string, mode Mode) (Declared, error) { func (c *composer) declaration(nodeName string, mode Mode) (Declared, error) {
resolved, err := c.planFor(nodeName) resolved, err := c.planFor(nodeName)
@@ -184,7 +217,7 @@ func (c *composer) declaration(nodeName string, mode Mode) (Declared, error) {
with := rendered.Rendering with := rendered.Rendering
with.Generators = gens with.Generators = gens
plan := resolved.Plan plan := resolved.Plan
composed, err := resolution.Compose(plan, with) composed, with, storeHeld, err := composeWithoutStoreFetchers(plan, with)
if err != nil { if err != nil {
return Declared{}, err return Declared{}, err
} }
@@ -198,7 +231,21 @@ func (c *composer) declaration(nodeName string, mode Mode) (Declared, error) {
if err != nil { if err != nil {
return Declared{}, err return Declared{}, err
} }
out := Declared{Resources: composed.Resources, Adoption: adoptionFor(plan, node, composed), // A module that fetches from a store no node reaches is held with its words, as a module whose claim cannot be held is: it
// stays on the build it runs, its composition kept (A-452, A-461).
held := resolved.Held
if len(storeHeld) > 0 {
held = make(map[string]string, len(resolved.Held)+len(storeHeld))
for m, why := range resolved.Held {
held[m] = why
}
for m, why := range storeHeld {
if _, said := held[m]; !said {
held[m] = why
}
}
}
out := Declared{Held: held, Resources: composed.Resources, Adoption: adoptionFor(plan, node, composed),
Received: composed.Received, Mesh: with.Mesh, BusUsers: with.BusUsers, Received: composed.Received, Mesh: with.Mesh, BusUsers: with.BusUsers,
LeftOut: sortedKeysOf(composed.LeftOut), LeftOutWhy: composed.LeftOut, LeftOut: sortedKeysOf(composed.LeftOut), LeftOutWhy: composed.LeftOut,
Withheld: rendered.Withheld, Unbound: rendered.Unbound, Withheld: rendered.Withheld, Unbound: rendered.Unbound,
@@ -228,6 +275,8 @@ func (c *composer) declaration(nodeName string, mode Mode) (Declared, error) {
return Declared{}, err return Declared{}, err
} }
out.Passes = passes out.Passes = passes
// A pass is composed, never recorded (ADR 0660 §4): the answer says it, with what two claimants leave unsettled.
out.Said = append(out.Said, passes.Lines()...)
for _, l := range left { for _, l := range left {
out.Said = append(out.Said, l.String()) out.Said = append(out.Said, l.String())
} }
+2 -18
View File
@@ -112,22 +112,6 @@ func TestAManifestTheNodeEngineRefusesComesBackAsProblemsAndNoError(t *testing.T
} }
} }
// 1298/AC3: a node whose own set does not compose returns an error and no declaration, told apart from an unreadable Input.
func TestANodeWhoseSetDoesNotComposeReturnsAnErrorAndNoDeclaration(t *testing.T) {
in := aMesh(t)
in.Catalogue["rival-one"] = catalogue.Manifest{Module: "rival-one", Version: "1", Claims: []catalogue.Claim{{Name: "the-seat", Scope: catalogue.ScopeNode}}}
in.Catalogue["rival-two"] = catalogue.Manifest{Module: "rival-two", Version: "1", Claims: []catalogue.Claim{{Name: "the-seat", Scope: catalogue.ScopeNode}}}
laptop := nodeOf(t, &in, "laptop")
laptop.Assigned = append(laptop.Assigned, "rival-one", "rival-two")
d, problems, err := Compose(in, "laptop")
if err == nil || !IsUnresolvable(err) || d.Resources != nil || problems != nil {
t.Fatalf("a set that does not compose: %v %v %v", d.Resources, problems, err)
}
if _, _, err := Compose(in, "nowhere"); err == nil || IsUnresolvable(err) {
t.Fatalf("a node the Input does not have: %v", err)
}
}
// 1298/AC4: composing the same Input twice gives byte-identical bodies, and the Input is unchanged. // 1298/AC4: composing the same Input twice gives byte-identical bodies, and the Input is unchanged.
func TestComposingTwiceGivesTheSameBytesAndChangesNothing(t *testing.T) { func TestComposingTwiceGivesTheSameBytesAndChangesNothing(t *testing.T) {
mk := func() Input { mk := func() Input {
@@ -392,8 +376,8 @@ func overlayName() string { return overlay.Name }
func TestANodeNotCountedAsOnTheNetworkIsSaidInTheResult(t *testing.T) { func TestANodeNotCountedAsOnTheNetworkIsSaidInTheResult(t *testing.T) {
in := aMesh(t) in := aMesh(t)
nodeOf(t, &in, "laptop").Place.Key = "" nodeOf(t, &in, "laptop").Place.Key = ""
in.Catalogue["rival-one"] = catalogue.Manifest{Module: "rival-one", Version: "1", Claims: []catalogue.Claim{{Name: "the-seat", Scope: catalogue.ScopeNode}}} in.Catalogue["rival-one"] = catalogue.Manifest{Module: "rival-one", Version: "1", Resources: []map[string]any{{"id": "r", "type": "directory", "path": "/var/lib/the-same", "mode": "0700"}}}
in.Catalogue["rival-two"] = catalogue.Manifest{Module: "rival-two", Version: "1", Claims: []catalogue.Claim{{Name: "the-seat", Scope: catalogue.ScopeNode}}} in.Catalogue["rival-two"] = catalogue.Manifest{Module: "rival-two", Version: "1", Resources: []map[string]any{{"id": "r", "type": "directory", "path": "/var/lib/the-same", "mode": "0700"}}}
laptop := nodeOf(t, &in, "laptop") laptop := nodeOf(t, &in, "laptop")
laptop.Assigned = append(laptop.Assigned, "rival-one", "rival-two") laptop.Assigned = append(laptop.Assigned, "rival-one", "rival-two")
said := func(lines []string) bool { said := func(lines []string) bool {
+8 -6
View File
@@ -11,12 +11,14 @@ import (
// fourOnTheNetwork is aMesh and two more machines on the private network at the site `there`: `ace`, dialled at // fourOnTheNetwork is aMesh and two more machines on the private network at the site `there`: `ace`, dialled at
// ace.example:51820, at 10.77.0.3, and `box`, behind no endpoint, at 10.77.0.4. A module `office` requires // ace.example:51820, at 10.77.0.3, and `box`, behind no endpoint, at 10.77.0.4. A module `office` requires
// `document-server`, which nothing in the catalogue provides: the shape of issue 1833 (nextcloud on the hub wanting a // `document-server`, which two modules of the catalogue provide and nobody chose: a refusal of the node's own set, as
// document server nobody offered). // issue 1833 was (nextcloud on the hub wanting a document server; an unanswered requirement now costs only its consumer, A-452).
func fourOnTheNetwork(t *testing.T) Input { func fourOnTheNetwork(t *testing.T) Input {
t.Helper() t.Helper()
in := aMesh(t) in := aMesh(t)
in.Catalogue["office"] = catalogue.Manifest{Module: "office", Version: "1", Requires: []string{"document-server"}} in.Catalogue["office"] = catalogue.Manifest{Module: "office", Version: "1", Requires: []string{"document-server"}}
in.Catalogue["docs-a"] = catalogue.Manifest{Module: "docs-a", Version: "1", Provides: catalogue.Offers("document-server")}
in.Catalogue["docs-b"] = catalogue.Manifest{Module: "docs-b", Version: "1", Provides: catalogue.Offers("document-server")}
caps := map[string]bool{"container-runtime": true, "wireguard": true, "systemd": true} caps := map[string]bool{"container-runtime": true, "wireguard": true, "systemd": true}
in.Nodes = append(in.Nodes, in.Nodes = append(in.Nodes,
Node{Name: "ace", Place: Place{Key: keyOf(t, "ace"), Endpoint: "ace.example:51820", Site: "there", Address: "10.77.0.3"}, Node{Name: "ace", Place: Place{Key: keyOf(t, "ace"), Endpoint: "ace.example:51820", Site: "there", Address: "10.77.0.3"},
@@ -26,7 +28,7 @@ func fourOnTheNetwork(t *testing.T) Input {
return in return in
} }
// unresolvableBy makes one machine's own set fail to resolve: it is also given `office`, whose requirement nothing provides. // unresolvableBy makes one machine's own set fail to resolve: it is also given `office`, whose requirement two modules provide.
func unresolvableBy(t *testing.T, in *Input, name string) { func unresolvableBy(t *testing.T, in *Input, name string) {
t.Helper() t.Helper()
n := nodeOf(t, in, name) n := nodeOf(t, in, name)
@@ -70,7 +72,7 @@ func TestOneMachineThatDoesNotResolveIsRefusedAlone(t *testing.T) {
if err == nil || !IsUnresolvable(err) { if err == nil || !IsUnresolvable(err) {
t.Fatalf("the machine that does not resolve is not refused as its own set failing: %v", err) t.Fatalf("the machine that does not resolve is not refused as its own set failing: %v", err)
} }
if !strings.Contains(err.Error(), `nothing provides "document-server", wanted by office`) { if !strings.Contains(err.Error(), `"document-server" is wanted by office and 2 modules provide it`) {
t.Errorf("the refusal does not name what could not be resolved: %v", err) t.Errorf("the refusal does not name what could not be resolved: %v", err)
} }
} }
@@ -147,8 +149,8 @@ func TestANetworkThatCannotBeComposedFromWhatRemainsStillFails(t *testing.T) {
// among the sources "from the mesh" -- and another machine still composes. // among the sources "from the mesh" -- and another machine still composes.
func TestAMachineThatDoesNotResolveKeepsItsNameAndItsAddressInTheMesh(t *testing.T) { func TestAMachineThatDoesNotResolveKeepsItsNameAndItsAddressInTheMesh(t *testing.T) {
in := fourOnTheNetwork(t) in := fourOnTheNetwork(t)
in.Catalogue["rival-one"] = catalogue.Manifest{Module: "rival-one", Version: "1", Claims: []catalogue.Claim{{Name: "the-seat", Scope: catalogue.ScopeNode}}} in.Catalogue["rival-one"] = catalogue.Manifest{Module: "rival-one", Version: "1", Resources: []map[string]any{{"id": "r", "type": "directory", "path": "/var/lib/the-same", "mode": "0700"}}}
in.Catalogue["rival-two"] = catalogue.Manifest{Module: "rival-two", Version: "1", Claims: []catalogue.Claim{{Name: "the-seat", Scope: catalogue.ScopeNode}}} in.Catalogue["rival-two"] = catalogue.Manifest{Module: "rival-two", Version: "1", Resources: []map[string]any{{"id": "r", "type": "directory", "path": "/var/lib/the-same", "mode": "0700"}}}
hub := nodeOf(t, &in, "anchor") hub := nodeOf(t, &in, "anchor")
hub.Assigned = append(hub.Assigned, "rival-one", "rival-two") hub.Assigned = append(hub.Assigned, "rival-one", "rival-two")
if _, err := Resolve(in, "anchor"); !IsUnresolvable(err) { if _, err := Resolve(in, "anchor"); !IsUnresolvable(err) {
+11
View File
@@ -114,6 +114,17 @@ func (c *composer) render(nodeName string, mode Mode) (Rendered, error) {
out.CoreBoxesWanted = wanted out.CoreBoxesWanted = wanted
} }
// A module whose claim cannot be held, or whose credential was refused, is left out of the declaration and run as it runs:
// the machine composes the rest (issue 1966, ADR 0636 §3).
for module, why := range resolved.Held {
if r.NotPlaced == nil {
r.NotPlaced = map[string]string{}
}
if _, said := r.NotPlaced[module]; !said {
r.NotPlaced[module] = why
}
}
// And a certificate for this machine's name inside the mesh, when anything on it asks. // And a certificate for this machine's name inside the mesh, when anything on it asks.
for _, m := range plan.Modules { for _, m := range plan.Modules {
if m.Certificate == nil { if m.Certificate == nil {
+17 -4
View File
@@ -92,7 +92,15 @@ func (c *composer) places(nodeName string, resolved Resolved) (Placed, error) {
r.MeshRange = c.overlayRange() r.MeshRange = c.overlayRange()
// The artifact store as this node reaches it now, with what the mesh has built. // The artifact store as this node reaches it now, with what the mesh has built.
// No address for any reason read without error (no holder, a holder laid by its name alone, a holder not on the private
// network, a holder opening no listen to the mesh, or two) holds only the modules that fetch from the store: they wait,
// with the words of an unanswered requirement, and the node composes the rest (design 47 G A-461, A-461e, A-452). A fault
// reading the inputs still refuses the node (A-461b).
store, err := c.artifactStoreAddress(nodeName) store, err := c.artifactStoreAddress(nodeName)
var noAddress *storeListenError
if errors.As(err, &noAddress) {
store, err = "", nil
}
if err != nil { if err != nil {
return out, err return out, err
} }
@@ -363,6 +371,11 @@ func (c *composer) storeHolders() map[string]catalogue.Manifest {
return out return out
} }
// storeListenError is the refusal of a holder that opens no listen to the mesh, or two: the store has no address (A-461).
type storeListenError struct{ said string }
func (e *storeListenError) Error() string { return e.said }
// storeListen is the store holder's listen every machine pulls from: its one listen open to the mesh. A holder opening // storeListen is the store holder's listen every machine pulls from: its one listen open to the mesh. A holder opening
// none, or more than one, says no address, by name: which of two ports is the store is no guess the composer makes. // none, or more than one, says no address, by name: which of two ports is the store is no guess the composer makes.
func storeListen(m catalogue.Manifest) (catalogue.Listening, error) { func storeListen(m catalogue.Manifest) (catalogue.Listening, error) {
@@ -376,15 +389,15 @@ func storeListen(m catalogue.Manifest) (catalogue.Listening, error) {
case 1: case 1:
return open[0], nil return open[0], nil
case 0: case 0:
return catalogue.Listening{}, fmt.Errorf("%s holds %s and opens no tcp listen to the mesh: the artifact store has no "+ return catalogue.Listening{}, &storeListenError{fmt.Sprintf("%s holds %s and opens no tcp listen to the mesh: the artifact store has no "+
"address (novox/hq ADR 0632 §1)", m.Module, StoreSeat) "address (novox/hq ADR 0632 §1)", m.Module, StoreSeat)}
} }
var said []string var said []string
for _, l := range open { for _, l := range open {
said = append(said, fmt.Sprintf("%s (%d)", l.Name, l.Port)) said = append(said, fmt.Sprintf("%s (%d)", l.Name, l.Port))
} }
return catalogue.Listening{}, fmt.Errorf("%s holds %s and opens %s to the mesh: the artifact store's address is its one "+ return catalogue.Listening{}, &storeListenError{fmt.Sprintf("%s holds %s and opens %s to the mesh: the artifact store's address is its one "+
"listen open to the mesh (novox/hq ADR 0632 §1)", m.Module, StoreSeat, strings.Join(said, ", ")) "listen open to the mesh (novox/hq ADR 0632 §1)", m.Module, StoreSeat, strings.Join(said, ", "))}
} }
// artifactStoreOnNetwork is the machine on this network that holds the store's seat, and the port that machine put the // artifactStoreOnNetwork is the machine on this network that holds the store's seat, and the port that machine put the
+53 -2
View File
@@ -2,6 +2,7 @@ package compose
import ( import (
"errors" "errors"
"slices"
"strings" "strings"
"testing" "testing"
@@ -351,8 +352,58 @@ func TestTheStoreIsFoundByTheImageSeatsHolderAndItsListen(t *testing.T) {
m := in.Catalogue["registry"] m := in.Catalogue["registry"]
m.Listens = append(m.Listens, catalogue.Listening{Name: "other", Port: 5300, From: catalogue.FromMesh}) m.Listens = append(m.Listens, catalogue.Listening{Name: "other", Port: 5300, From: catalogue.FromMesh})
in.Catalogue["registry"] = m in.Catalogue["registry"] = m
if _, err := Places(in, "laptop"); err == nil || !strings.Contains(err.Error(), "opens registry (5000), other (5300) to the mesh") { // A-461, A-452: a node that reaches no store address (a holder opening two listens to the mesh, so none is a guess, or a holder
t.Errorf("a holder opening two listens to the mesh: %v", err) // laid name-only, found by nobody) costs the modules that fetch from it, whatever the shape of the fetch (an image, a tools
// bundle of the runtime), and never the node: they wait with the words of an unanswered requirement, and the rest composes.
dg := "sha256:" + strings.Repeat("a", 64)
in.Catalogue["app"] = catalogue.Manifest{Module: "app", Version: "1",
Resources: []map[string]any{{"id": "c", "type": "container", "name": "app", "image": "artifact-store://app/c@" + dg}}}
in.Catalogue["node-tools"] = catalogue.Manifest{Module: "node-tools", Version: "1", ToolRunner: true,
OwnSecrets: catalogue.OwnSecrets{"broker": {Path: "/var/lib/node-tools/broker"}},
Bundles: []catalogue.Bundle{{Name: "runtime", Source: catalogue.ArtifactStoreScheme + "node-tools/runtime/blobs/" + dg,
Digest: dg, Language: "go", Binary: "node-tools"}}}
in.Catalogue["plain"] = catalogue.Manifest{Module: "plain", Version: "1",
Resources: []map[string]any{{"id": "state", "type": "directory", "path": "/var/lib/plain", "mode": "0700"}}}
laptop := nodeOf(t, &in, "laptop")
laptop.Assigned = append(laptop.Assigned, "app", "node-tools", "plain")
laptop.SealingKey = keyOf(t, "laptop")
in.BusMinted = []string{"laptop.node-tools"}
got, err := Render(in, "laptop", Reading)
if err != nil {
t.Fatalf("a holder opening two listens to the mesh refuses the whole node: %v", err)
}
if got.Rendering.ArtifactStore != "" {
t.Errorf("the store has an address: %q", got.Rendering.ArtifactStore)
}
d, _, err := Compose(in, "laptop")
if err != nil {
t.Fatalf("a node that reaches no store address is refused whole: %v", err)
}
for _, m := range []string{"app", "node-tools"} {
if why := d.LeftOutWhy[m]; !strings.Contains(why, "nothing provides") || !strings.Contains(why, "no artifact store") {
t.Errorf("%s, which fetches from the store, waits with %q", m, why)
}
// Held, not only left out: it stays on the build it runs and its handed composition is kept (A-452).
if why := d.Held[m]; !strings.Contains(why, "nothing provides") {
t.Errorf("%s, which fetches from the store, is held with %q", m, why)
}
}
if _, held := d.Held["plain"]; held {
t.Errorf("a module that fetches nothing from the store is held")
}
if why, held := d.LeftOutWhy["plain"]; held {
t.Errorf("a module that fetches nothing from the store waits: %q", why)
}
if !slices.ContainsFunc(d.Resources, func(r map[string]any) bool { return r["id"] == "plain.state" }) {
t.Errorf("the rest of the node was not composed: %v", d.Resources)
}
// A-461b: only the listen refusal holds the store's fetchers; any other fault of finding the store (here the holder's machine is
// on the network as first read but unknown when its assignment is read) still refuses the node.
broken := newComposer(in)
broken.onTheNetwork()
broken.in.Nodes = []Node{*nodeOf(t, &in, "laptop")}
if _, err := broken.places("laptop", Resolved{}); err == nil || !strings.Contains(err.Error(), "reading what anchor is assigned") {
t.Errorf("a fault of finding the store other than its listen refusal did not refuse the node: %v", err)
} }
} }
@@ -1,43 +0,0 @@
package resolution
import (
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// artifactSweepVerbs are the seat's verbs, with the controller's arguments so a caller changes only the seat it asks.
func artifactSweepVerbs() []mcat.Verb {
return []mcat.Verb{
{Name: "artifacts", Description: "Every artifact the mesh made that lives in its artifact store, each with its " +
"repository, digest, kind (image or archive) and state: kept, and why (a want names it, or one of the five " +
"newest builds of its module in the builder's log), eligible (made, kept for no reason, not yet let go), or " +
"unrecorded (in the store and named by nothing the mesh made: never removed). Reads only.",
Input: schema(map[string]string{
"repository": "only this repository's references, as <module>/<artifact> (the counts stay whole)",
"collected": "\"true\": list the collected references too (counted always)",
}, nil, "collected")},
{Name: "held", Description: "Whether the artifact store holds each reference given, asked with one HEAD each: " +
"held, gone (the store answered 404) or unknown with why. The builder asks it before it reuses a done key. " +
"Reads only.",
Input: listed(schema(map[string]string{
"references": "the artifacts' references, artifact-store://<module>/<artifact>/blobs/sha256:<hex>: a list, " +
"or one text separated by commas",
}, []string{"references"}), "references")},
{Name: "collect", Description: "The artifact store's sweep, now: what is eligible is let go of. Without confirm a " +
"dry run: what would be let go, nothing held or deleted. With confirm and why: every kept archive held first, " +
"then each eligible artifact let go of, oldest first, an index with the platform manifests no kept index " +
"names, each deletion recorded in the hand-act log. Bounded by most (500 by default, at most 5000) and 45 " +
"seconds; what is left is said. Never touches a digest the mesh did not make.",
Input: schema(map[string]string{
"confirm": "\"true\": let go of what is eligible (needs why); without it nothing is done",
"why": "with confirm: why, required and recorded",
"most": "let go of at most this many (500 by default, at most 5000)",
}, nil, "confirm")},
{Name: "mirrors", Description: "The bases the mesh copied into its artifact store, each kept (a kept build stood on " +
"it), eligible (only builds no longer kept stood on it) or unrecorded (no build the builder's log holds stood on " +
"it: never removed), by what the builder's log says each build stood on. Records nothing by hand.",
Input: schema(map[string]string{}, nil)},
{Name: "images", Description: "Every container image one machine's want and have name, each with the resources " +
"naming it: what the machine keeps when it prunes its images. Reads only.",
Input: schema(map[string]string{"node": "the machine's name"}, []string{"node"})},
}
}
+13 -2
View File
@@ -8,6 +8,17 @@ import (
"github.com/novox/mesh-host/internal/registryforms" "github.com/novox/mesh-host/internal/registryforms"
) )
// NoStoreError is a module that fetches from the artifact store while this node reaches none: it costs that module, never the
// node. The caller leaves the module out of the declaration, with these words (design 47 G A-461, A-452; ADR 0163 rule 6).
type NoStoreError struct {
Module string
said string
}
func (e *NoStoreError) Error() string {
return fmt.Sprintf("nothing provides %q, wanted by %s: %s", "the artifact store's address", e.Module, e.said)
}
// artifactsInto composes the artifact store's address into a resource's `image` and `source`. // artifactsInto composes the artifact store's address into a resource's `image` and `source`.
// //
// **Composed here, at the last moment before a machine, and stored nowhere.** A kept reference is // **Composed here, at the last moment before a machine, and stored nowhere.** A kept reference is
@@ -35,11 +46,11 @@ func artifactsInto(resource map[string]any, module string, with mcat.Rendering)
} }
if _, kept := registryforms.Forms.InArtifactStore(written); kept { if _, kept := registryforms.Forms.InArtifactStore(written); kept {
if with.ArtifactStore == "" { if with.ArtifactStore == "" {
return fmt.Errorf( return &NoStoreError{Module: module, said: fmt.Sprintf(
"%s's %v names %s, which is in the mesh's artifact store, and this mesh has no "+ "%s's %v names %s, which is in the mesh's artifact store, and this mesh has no "+
"artifact store on its network to fetch it from — nothing assigned holds the seat "+ "artifact store on its network to fetch it from — nothing assigned holds the seat "+
"container-image-registry, or the machine holding it is not on the private network", "container-image-registry, or the machine holding it is not on the private network",
module, resource["id"], written) module, resource["id"], written)}
} }
resource[key] = registryforms.Forms.Routed(written, with.ArtifactStore) resource[key] = registryforms.Forms.Routed(written, with.ArtifactStore)
imageSeatInto(resource, key, written, with) imageSeatInto(resource, key, written, with)
@@ -1,33 +0,0 @@
package resolution
import (
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// checkerVerbs are the checker seat's verbs: ask a try, read one, end one by hand. Optional until the checker serves them
// everywhere.
func checkerVerbs() []mcat.Verb {
return []mcat.Verb{
{Name: "try", Optional: true, Description: "Try a pushed head before its pull request is opened: the checker " +
"judges it as it judges a pull request's head, merged onto its trunk's head, its unit and integration " +
"levels through the build modules, and owes the forge nothing. Answers the try's id at once; try-read " +
"reads where it stands. A try of the same head that waits or runs is the answer to a second ask; at most " +
"4 wait or run at once, mesh-wide.",
Input: schema(map[string]string{"repository": "<owner>/<name>",
"head": "the full commit to try, 40 or 64 hex digits",
"trunk": "the trunk commit to merge it onto, or the trunk's branch for its head now; the trunk's head when absent",
"tests": "a go test -run pattern: only those tests run, and nothing is kept",
"level": "unit or integration: only this level; both when absent"},
[]string{"repository", "head"}),
Replaces: []string{"sh merge-check.sh", "docker run postgres"}},
{Name: "try-read", Optional: true, Description: "Where a try stands, as stored: queued, running, or its end " +
"(pass, fail, or error when it could not run), with what was said of it, the head and the trunk commit " +
"it was merged onto, the time it took and what failed first. Read-only.",
Input: schema(map[string]string{"id": "the try's id, try-<digits>"}, []string{"id"})},
{Name: "try-end", Optional: true, Description: "End a try that waits or runs, as error, by hand: say why (kept in " +
"what it says). Its place among the tries is free at once and the next try of its head asks anew with a new " +
"id. A try that ended already is refused.",
Input: schema(map[string]string{"id": "the try's id, try-<digits>", "why": "why it is ended, in words"},
[]string{"id", "why"})},
}
}
+116
View File
@@ -0,0 +1,116 @@
package resolution
import (
"encoding/json"
"fmt"
"math"
"regexp"
"slices"
"strconv"
"strings"
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// configLinesRef is `${config-lines:<key>:<prefix>}` and `${config-lines:<key>:<prefix>:mib-bytes}` in a file's
// content (design 42 zfs-owns-zfs §3, ADR 0679 §16 (c)): one line per entry of a list value, `<prefix><entry>`. The
// prefix is literal text with no `}`, `$` or newline. A placeholder that holds a whole line takes its newline with it.
var configLinesRef = regexp.MustCompile(`\$\{config-lines:([a-z0-9][a-z0-9_.-]*):([^}$\n]*?)(:mib-bytes)?\}(\n?)`)
// configJSONRef is `${config-json}` in a file's content (design 42 zfs-owns-zfs §9, answer A-15): the module's set
// config values as one JSON object.
var configJSONRef = regexp.MustCompile(`\$\{config-json\}`)
// configLinesInto writes each `${config-lines:…}` of a file's content from the layers: a line `<prefix><entry>` for each
// entry of a list, one for a set scalar, nothing for an unset value or an empty list. With `mib-bytes` an entry given in
// MiB is written in bytes. An entry holding a newline is refused, and the content is left as it was.
func configLinesInto(content string, layers []mcat.Layer, module string) (string, error) {
var refused error
filled := configLinesRef.ReplaceAllStringFunc(content, func(token string) string {
m := configLinesRef.FindStringSubmatch(token)
key, prefix, mib, newline := m[1], m[2], m[3] != "", m[4]
value, set := settingValue(layers, key)
if !set {
return ""
}
entries, isList := value.([]any)
if !isList {
entries = []any{value}
}
var lines []string
for _, e := range entries {
text := plainly(e)
if strings.ContainsAny(text, "\r\n") {
refused = fmt.Errorf("%s: an entry of the config value %q holds a newline, which would begin a line "+
"of its own in the file it is written into (design 42 §3)", module, key)
return token
}
if mib {
n, err := mibWhole(text)
if err != nil || n > 1<<44 {
refused = fmt.Errorf("%s: the config value %q holds %q, and mib-bytes writes a whole number of MiB "+
"in bytes (design 42 §3)", module, key, text)
return token
}
text = strconv.FormatUint(n<<20, 10)
}
lines = append(lines, prefix+text)
}
if len(lines) == 0 {
return ""
}
return strings.Join(lines, "\n") + newline
})
if refused != nil {
return content, refused
}
return filled, nil
}
// configJSONInto writes `${config-json}` in a file's content as the module's set config values in one JSON object
// (design 42 zfs-owns-zfs §9, answer A-15): only keys the module declares under `config` are written, and what no layer sets is absent, and no unset value refuses the module; a
// list and an object are written as JSON, a number, a boolean and a text as JSON values of their own type.
func configJSONInto(content string, layers []mcat.Layer, declared []string) (string, error) {
if !configJSONRef.MatchString(content) {
return content, nil
}
set := map[string]any{}
for _, layer := range layers {
for k, v := range layer.Values {
if slices.Contains(declared, k) {
set[k] = v
}
}
}
var out strings.Builder
enc := json.NewEncoder(&out)
enc.SetEscapeHTML(false)
enc.SetIndent("", " ")
if err := enc.Encode(set); err != nil {
return content, fmt.Errorf("the config values cannot be written as JSON: %w", err)
}
object := strings.TrimSuffix(out.String(), "\n")
return configJSONRef.ReplaceAllLiteralString(content, object), nil
}
// mibWhole is a size in MiB as a whole number: digits, or a whole number a program wrote in exponent form (`1e+06`),
// which is how a float of a million or more prints (design 42 §3).
func mibWhole(text string) (uint64, error) {
if n, err := strconv.ParseUint(text, 10, 64); err == nil {
return n, nil
}
f, err := strconv.ParseFloat(text, 64)
if err != nil || f < 0 || f != math.Trunc(f) || f > 1<<44 {
return 0, fmt.Errorf("not a whole number of MiB")
}
return uint64(f), nil
}
// declaredConfig is the keys a module's manifest declares under `config`.
func declaredConfig(m mcat.Manifest) []string {
keys := make([]string, 0, len(m.Settings))
for k := range m.Settings {
keys = append(keys, k)
}
return keys
}
+111 -38
View File
@@ -4,6 +4,8 @@ import (
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
"maps"
"slices"
"sort" "sort"
"strconv" "strconv"
"strings" "strings"
@@ -145,7 +147,7 @@ func resolutionCompose(r mcat.Resolution, with mcat.Rendering, owner map[string]
// Before placing, because a placement is a setting too. // Before placing, because a placement is a setting too.
left := LeftOut(r, with.Settings, with.Adopted) left := LeftOut(r, with.Settings, with.Adopted)
kept := make([]mcat.Manifest, 0, len(r.Modules)) kept := make([]mcat.Manifest, 0, len(r.Modules))
var stillBackedUp []mcat.Manifest var stillBackedUp, stillRunning []mcat.Manifest
for module, why := range with.NotPlaced { for module, why := range with.NotPlaced {
if _, already := left[module]; !already { if _, already := left[module]; !already {
left[module] = why left[module] = why
@@ -160,6 +162,10 @@ func resolutionCompose(r mcat.Resolution, with mcat.Rendering, owner map[string]
// the data it already holds on the machine is the reason to keep copying it. Only its data, // the data it already holds on the machine is the reason to keep copying it. Only its data,
// as the backup holder's lines are derived from it, and the directories they name. // as the backup holder's lines are derived from it, and the directories they name.
stillBackedUp = append(stillBackedUp, backupView(m)) stillBackedUp = append(stillBackedUp, backupView(m))
// **And what it runs is still let through** (issue 1966): its containers and services are left as they run, so
// the filter keeps opening what its listens say, at the machine ports this node holds for it, rather than
// closing a port that still serves (the artifact store's 5100 on the control-node, 98 minutes).
stillRunning = append(stillRunning, m)
continue continue
} }
kept = append(kept, m) kept = append(kept, m)
@@ -240,13 +246,22 @@ func resolutionCompose(r mcat.Resolution, with mcat.Rendering, owner map[string]
} }
if answered { if answered {
r.Needs[i].Serves = serves r.Needs[i].Serves = serves
continue
} }
// No module of the composed set serves it: its provider is held on this machine (composed apart, as what runs of
// it), and offers what its manifest serves. None of its own names is composed here, so a key that names one is
// left out and said on the need, as a provider elsewhere leaves it out (ServedLeftOut, novox/hq ADR 0346 §3),
// never handed to the consumer as a placeholder its binding then refuses (issue 1921, novox: "mesh-build-node has
// a file that says ${bound:route:name-web}, and mesh-build-node does not require route").
r.Needs[i].Serves, r.Needs[i].LeftOut = withoutOwnNames(r.Needs[i], r.Node)
} }
// Once, from every module's listens -- not per module. A module receiving only its own ports // Once, from every module's listens -- not per module. A module receiving only its own ports
// would write a rule set that closed every other module on the machine. Each module's per-node // would write a rule set that closed every other module on the machine. Each module's per-node
// exposure settings override its listens' source first (novox/hq ADR 0046). // exposure settings override its listens' source first (novox/hq ADR 0046).
rules, err := Rules(r, with) filtered := r
filtered.Modules = append(slices.Clone(r.Modules), stillRunning...)
rules, err := Rules(filtered, with)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -644,6 +659,9 @@ func resolutionCompose(r mcat.Resolution, with mcat.Rendering, owner map[string]
// parses strictly and knows no such field. // parses strictly and knows no such field.
absent := AbsentFor(r, m.Module) absent := AbsentFor(r, m.Module)
resources = withoutAbsent(resources, absent) resources = withoutAbsent(resources, absent)
// **A resource declared only while a config value holds** (design 42 zfs-owns-zfs §2): left out while none of
// its `if-config` conditions holds, and the key dropped from the rest.
resources = withoutUnconfigured(resources, WithDefaults(m, with.Settings[m.Module]))
// No container is given the mesh's names (novox/hq ADR 0148). It used to be: every // No container is given the mesh's names (novox/hq ADR 0148). It used to be: every
// container got the whole roster as `--add-host` entries at creation, and a name that // container got the whole roster as `--add-host` entries at creation, and a name that
@@ -891,7 +909,7 @@ func resolutionCompose(r mcat.Resolution, with mcat.Rendering, owner map[string]
// **An operator's value, from the assignment** (novox/hq ADR 0112, ADR 0155): what a // **An operator's value, from the assignment** (novox/hq ADR 0112, ADR 0155): what a
// definition may not carry because it is true of one installation only. Filled from // definition may not carry because it is true of one installation only. Filled from
// the same layers a mergeable file takes, and refused when no layer set it. // the same layers a mergeable file takes, and refused when no layer set it.
if err := unsetInConfigWords(settingInto(copied, WithDefaults(m, with.Settings[m.Module]), m.Module), m); err != nil { if err := unsetInConfigWords(settingInto(copied, WithDefaults(m, with.Settings[m.Module]), m.Module, declaredConfig(m)...), m); err != nil {
return nil, err return nil, err
} }
// **Placed before anything reads a path.** A pathless directory receives the path // **Placed before anything reads a path.** A pathless directory receives the path
@@ -1233,46 +1251,24 @@ func resolutionFiltersHere(r mcat.Resolution) string {
// is adopted or converged — the one loads it as a filter, the other declares it as openings. // is adopted or converged — the one loads it as a filter, the other declares it as openings.
func Rules(r mcat.Resolution, with mcat.Rendering) ([]mcat.Rule, error) { func Rules(r mcat.Resolution, with mcat.Rendering) ([]mcat.Rule, error) {
// A module whose settings cannot compose is left out of the declaration (novox/hq ADR 0163, // A module whose settings cannot compose is left out of the declaration (novox/hq ADR 0163,
// rule 6), and out of the filter with it: nothing of it is declared, so nothing of it is let // rule 6), and not out of the filter: what it runs is left running, so a port it serves stays
// through. // open (issue 1966). Its per-node exposure and reaches still narrow it (ADR 0046); where they
// cannot be read for it, its listens open no wider than the mesh, never at the manifest's
// default, which could reopen a port an operator had narrowed.
left := LeftOut(r, with.Settings, with.Adopted) left := LeftOut(r, with.Settings, with.Adopted)
for module, why := range with.NotPlaced {
if _, already := left[module]; !already {
left[module] = why
}
}
exposure := map[string]map[int]string{} exposure := map[string]map[int]string{}
for _, m := range r.Modules { for _, m := range r.Modules {
if _, isLeft := left[m.Module]; isLeft { e, err := exposureOf(m, with.Settings[m.Module])
continue
}
e, err := Exposure(m, with.Settings[m.Module])
if err != nil { if err != nil {
return nil, err if _, isLeft := left[m.Module]; !isLeft {
} return nil, err
// And how far each endpoint reaches, which says the same thing to the filter and more
// besides (novox/hq ADR 0138). Folded in here rather than beside: the filter has one
// question — from where — and a reach answers it, so giving it two inputs would let them
// disagree. Reaches refuses a port that both name, so this cannot silently prefer one.
reaches, err := Reaches(m, with.Settings[m.Module])
if err != nil {
return nil, err
}
// **Only for an endpoint the proxy does not serve.** A routed endpoint's port is how the
// proxy reaches it and nothing else (ADR 0045), so `public` there asks for a public name and
// says nothing about the port — opening it to the world as well would undo the arrangement
// the proxy exists for, and would silently reopen a port an operator had narrowed.
//
// Found by trying to express a real module: one whose routed name must be public and whose
// machine-side port must not be. Under one value for both, there was no way to say it.
routed := RoutedPorts(m)
for port, reach := range reaches {
if routed[port] {
continue
} }
source, ok := FilterSource(reach) e = noWiderThanTheMesh(m)
if !ok {
return nil, fmt.Errorf("%s: %q is not a reach the filter can read", m.Module, reach)
}
if e == nil {
e = map[int]string{}
}
e[port] = source
} }
if e != nil { if e != nil {
exposure[m.Module] = e exposure[m.Module] = e
@@ -1281,6 +1277,60 @@ func Rules(r mcat.Resolution, with mcat.Rendering) ([]mcat.Rule, error) {
return Filtering(r, with.Generators, with.Ports, exposure) return Filtering(r, with.Generators, with.Ports, exposure)
} }
// noWiderThanTheMesh is the exposure of a module left out whose own cannot be read: each of its
// listens from wider than the mesh narrowed to the mesh, a narrower one kept.
func noWiderThanTheMesh(m mcat.Manifest) map[int]string {
var e map[int]string
for _, l := range m.Listens {
if l.From == mcat.FromLocalNetwork || l.From == mcat.FromEverywhere {
if e == nil {
e = map[int]string{}
}
e[l.Port] = mcat.FromMesh
}
}
return e
}
// exposureOf is a module's per-node exposure, declared port to source: its expose setting, and
// the reaches of its endpoints the proxy does not serve.
func exposureOf(m mcat.Manifest, layers []mcat.Layer) (map[int]string, error) {
e, err := Exposure(m, layers)
if err != nil {
return nil, err
}
// And how far each endpoint reaches, which says the same thing to the filter and more
// besides (novox/hq ADR 0138). Folded in here rather than beside: the filter has one
// question — from where — and a reach answers it, so giving it two inputs would let them
// disagree. Reaches refuses a port that both name, so this cannot silently prefer one.
reaches, err := Reaches(m, layers)
if err != nil {
return nil, err
}
// **Only for an endpoint the proxy does not serve.** A routed endpoint's port is how the
// proxy reaches it and nothing else (ADR 0045), so `public` there asks for a public name and
// says nothing about the port — opening it to the world as well would undo the arrangement
// the proxy exists for, and would silently reopen a port an operator had narrowed.
//
// Found by trying to express a real module: one whose routed name must be public and whose
// machine-side port must not be. Under one value for both, there was no way to say it.
routed := RoutedPorts(m)
for port, reach := range reaches {
if routed[port] {
continue
}
source, ok := FilterSource(reach)
if !ok {
return nil, fmt.Errorf("%s: %q is not a reach the filter can read", m.Module, reach)
}
if e == nil {
e = map[int]string{}
}
e[port] = source
}
return e, nil
}
// provisionsAs is the account that reads what the mesh writes for this provider: the one secret // provisionsAs is the account that reads what the mesh writes for this provider: the one secret
// per consumer it must open to set that consumer's password (novox/hq issue 225). // per consumer it must open to set that consumer's password (novox/hq issue 225).
// //
@@ -2526,3 +2576,26 @@ func resolutionGeneratedHere(r mcat.Resolution, with mcat.Rendering) (map[string
} }
return generated, nil return generated, nil
} }
// withoutOwnNames is what a need's provider serves without each key whose value names one of the provider's own composed
// names (${bound:…}), and the need's left-out keys with each such key said: a provider not composed in this set
// composes none of its names here.
func withoutOwnNames(n mcat.Needed, node string) (map[string]any, []string) {
var asking []string
for _, key := range sortedAnyKeys(n.Serves) {
if text, ok := n.Serves[key].(string); ok && bound.MatchString(text) {
asking = append(asking, key)
}
}
if len(asking) == 0 {
return n.Serves, n.LeftOut
}
out := maps.Clone(n.Serves)
said := slices.Clone(n.LeftOut)
for _, key := range asking {
delete(out, key)
said = append(said, fmt.Sprintf("%s serves %s without %q: it names %s, and %s composes no names of %s, which it "+
"holds (novox/hq ADR 0346)", n.Module, n.Name, key, bound.FindString(n.Serves[key].(string)), orHere(node), n.Module))
}
return out, said
}
@@ -1,75 +0,0 @@
package resolution
import (
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliveryclient"
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// deliveryRemovedInR6 is the row of a verb release 6 removed from the delivery seat.
func deliveryRemovedInR6(name string) mcat.Verb {
return mcat.Verb{Name: name, Description: meshdeliveryclient.RemovedPrefix + meshdeliveryclient.RemovedVerbs()[name], Input: schema(nil, nil), Optional: true}
}
// deliveryVerbs are the delivery seat's tools: what reads the merges and checks, and the acts of a person and healer H2
// on a merge (ADR 0539 clause 17).
func deliveryVerbs() []mcat.Verb {
return []mcat.Verb{
{Name: "deliveries", Description: "Every delivery not final, and those that ended in the last day, one line " +
"each: its id (owner/repository@commit), its state, its group, what it waits for and since when. Narrowed " +
"by state, repository or group.",
Input: schema(map[string]string{"state": "one state, e.g. delivering or held",
"repository": "owner/repository", "group": "a group's id (its branch name)",
"all": "\"true\": the final ones of the last thirty days too"}, nil, "all")},
deliveryRemovedInR6("times"),
{Name: "show", Description: "One delivery or group whole: its delivery plan (what it builds, what each " +
"machine receives, what is not an ordinary send), every transition with when and why, the machine " +
"steps of its walk, its group and its order.",
Input: schema(map[string]string{"id": "a delivery's id, or a group's"}, []string{"id"})},
{Name: "groups", Description: "Every delivery group: its members in order, why each pair is ordered " +
"(declared, built by, version skew, engine before controller, by name), its composed check and its " +
"state, derived from its members.",
Input: schema(map[string]string{"all": "\"true\": the groups that ended too"}, nil, "all")},
deliveryRemovedInR6("what-if"),
{Name: "checks", Description: "What the mesh's checks said of a pull request's head or of one commit: each " +
"of the commit's mesh statuses (mesh/check, the one a main requires from R4e, hq's mesh/repo-check, " +
"mesh/delivery, …) with its state, " +
"description and when it was set; the merge check's full verdict as the controller said it — each " +
"layer's summary, the machine that ran it, when, its build id and its report; and whether the branch's " +
"protection would let it merge, every required status being success.",
Input: schema(map[string]string{"repository": "owner/repository",
"number": "a pull request's number: its head is read",
"commit": "a commit's sha, or the start of one, instead of a pull request"}, []string{"repository"}),
// Added after the seat's first holder shipped: optional until mesh-delivery serves it everywhere.
Optional: true},
{Name: "table", Description: "The state table every delivery runs by: each transition with its guard, " +
"each state's bound and what healer H2 may do once it has passed; and the machine steps' table.",
Input: schema(map[string]string{}, nil)},
{Name: "stalled", Description: "Every delivery held past its state's bound, with the transition the table " +
"lets healer H2 take for it, or none.",
Input: schema(map[string]string{}, nil)},
{Name: "recheck", Description: "Check a rejected or ready delivery again: it goes back to proposed and the " +
"controller is asked for its check. With why.",
Input: schema(map[string]string{"id": "the delivery's id", "why": "why, kept with the transition"},
[]string{"id", "why"})},
deliveryRemovedInR6("release"),
deliveryRemovedInR6("stop"),
{Name: "close", Description: "Healer H2's verb: take the one transition the table names for a delivery held " +
"past its bound. Refused for any other.",
Input: schema(map[string]string{"id": "the delivery's id", "why": "the condition it answers"},
[]string{"id", "why"})},
deliveryRemovedInR6("retire-history"),
{Name: "settle", Description: "Ask the operator to settle deliveries held because their merge reached the " +
"trunk without a passing check: one ask on the operator's channel for all of them, as an Approve is asked. " +
"The verb moves nothing; on the operator's answer each goes to merged, and whether it is on the machines " +
"is derived as for any merge (novox/hq ADR 0556). Refused for a delivery not held, or held for another " +
"reason, naming why.",
Input: map[string]any{"type": "object", "additionalProperties": false, "required": []string{"ids", "why"},
"properties": map[string]any{
"ids": map[string]any{"type": "array", "items": map[string]any{"type": "string"}, "minItems": 1,
"description": "the held deliveries' ids (owner/repository@commit)"},
"why": map[string]any{"type": "string", "description": "why they may be settled, shown to the operator"},
}},
// Added after the seat's release 6 holder shipped: optional until the gate serves it everywhere.
Optional: true},
}
}
@@ -1,30 +0,0 @@
package resolution
import (
"testing"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliveryclient"
)
// Each verb release 6 removed from the delivery seat is a row refusing it by name with the replacement the seat's client
// gives (the list is the contract's, not the SDK's: novox/hq ADR 0634 §2), and the rows are exactly those verbs.
func TestTheDeliverySeatRefusesTheRemovedVerbsByNameWithTheClientsReplacement(t *testing.T) {
removed := meshdeliveryclient.RemovedVerbs()
rows := 0
for _, v := range deliveryVerbs() {
want, isRemoved := removed[v.Name]
if !isRemoved {
continue
}
rows++
if v.Description != meshdeliveryclient.RemovedPrefix+want || !v.Optional {
t.Errorf("%s = %q (optional %v), want %q", v.Name, v.Description, v.Optional, meshdeliveryclient.RemovedPrefix+want)
}
}
if rows != len(removed) {
t.Errorf("%d rows refuse a removed verb, the client names %d", rows, len(removed))
}
if len(removed) != 5 {
t.Errorf("the removed verbs are %v", removed)
}
}
@@ -1,67 +0,0 @@
package resolution
import (
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// enrolmentDecision is the record that made the seat.
const enrolmentDecision = "novox/hq ADR 0587"
// enrolmentVerbs are the seat's verbs: what reads all it holds and does by hand all it does (ADR 0587 §8). Not optional:
// the seat is new with its holder, so no holder of before serves fewer.
func enrolmentVerbs() []mcat.Verb {
str := func(description string) map[string]any {
return map[string]any{"type": "string", "description": description}
}
return []mcat.Verb{
{Name: mcat.EnrolmentVerbIssue, Description: "Issue a one-time token for a new machine, on the operator's Approve: " +
"refused at once, naming every finding, for a name that is not a node name or that a node record, an assignments " +
"key or an open enrolment holds, and for every call before the controller's lease is handed away. Otherwise " +
"reserves the lowest free private-network address, asks the operator under the seat's own name and, on the " +
"warrant, adds the node to the assignments with its tunnel key, asks the vault for the token and says enrolment-issued. " +
"The token itself is read only through tokens.",
Input: schema(map[string]string{
"node": "the new machine's name in the mesh",
"why": "why it joins, shown to the operator with the ask",
"expires": "how long the token lasts, a duration such as 1h or 24h: one hour when not given, at most 24 hours",
// The operator's decision of 2026-11-02 01:05: the tunnel key is given at issue, so on the Approve the node
// enters the assignments with it and the hub peers it before redeem.
"overlay_key": "the new machine's private network (tunnel) public key, as `nox-mesh-host key` prints it",
}, []string{"node", "why", "overlay_key"})},
{Name: mcat.EnrolmentVerbRedeem, Description: "Spend a token: asked by a joining node-engine dialled as enrol.<node> " +
"alone, with the public halves of its keys and its identity key's proof over them. Spends the token, asks the " +
"vault for the node's account and certificate, writes the node into the assignments, and answers the account sealed " +
"to the node, its certificate and the vault's public half, signed by the vault. Asked again with the same keys " +
"it finishes what is not done and answers the same; with other keys it is refused.",
Input: map[string]any{"type": "object", "additionalProperties": false,
"required": []string{"node", "public_key", "sealing_key", "proof"},
"properties": map[string]any{
"node": str("the node the token was issued for"),
"public_key": str("the identity key's public half, base64"),
"overlay_key": str("the private network key's public half"),
"sealing_key": str("the sealing key's public half, which the account is sealed to"),
"serving_key": str("the serving key's public half, which the certificate binds"),
"profile": map[string]any{"type": "object", "description": "what the machine is, as its record says it"},
"tunnel": map[string]any{"type": "object", "description": "a tunnel found on the machine: everything but its private key"},
"proof": str("the identity key's signature over the redeem's proof bytes (delivery.RedeemProof), base64"),
}}},
{Name: mcat.EnrolmentVerbConfirm, Description: "Confirm an enrolment: asked by the node-engine dialled as its own " +
"node login alone, the runtime's proof of that caller being the confirmation. Says node-enrolled and ends the " +
"enrolment.",
Input: schema(map[string]string{}, nil)},
{Name: mcat.EnrolmentVerbRevoke, Description: "End an enrolment at any stage, with why, without an Approve: withdraws " +
"an open ask, deletes the node's key and its address, has the vault retire an account it already minted for " +
"it, records the act as a hand act and says enrolment-revoked.",
Input: schema(map[string]string{"node": "the node", "why": "why, recorded with the hand act"},
[]string{"node", "why"})},
{Name: mcat.EnrolmentVerbTokens, Description: "Read every open enrolment, or one: its stage, expiry, ask and revision, " +
"the bucket's fill against its cap. For a node at live, its " +
"token sealed to the operator key, asked of the vault at that moment and relayed as the vault sealed it. With " +
"history, the ended enrolments from the seat's events, as far back as the events are kept.",
Input: schema(map[string]string{"node": "one node", "history": "\"true\": the ended enrolments too"},
nil, "history")},
{Name: mcat.EnrolmentVerbAgain, Description: "Raise an issue's ask again when the operator's channel does not hold it, " +
"and nothing more.",
Input: schema(map[string]string{"node": "the node whose ask is raised again"}, []string{"node"})},
}
}
@@ -60,6 +60,59 @@ func TestTheModuleAskingForTheRuleSetGetsEveryModulesPorts(t *testing.T) {
if !strings.Contains(content, "dport 443 accept") { if !strings.Contains(content, "dport 443 accept") {
t.Fatalf("the rule set holds only the asking module's ports:\n%s", content) t.Fatalf("the rule set holds only the asking module's ports:\n%s", content)
} }
// Issue 1966, novox 2026-11-09 07:46-09:24 UTC: a module left out of the declaration for a setting its new definition
// no longer composes (ADR 0163 rule 6) keeps running what it ran, so the filter keeps letting it through. The
// artifact store's build 6e1dfaa8 listens on 5100 from a process and publishes no container port 5000, while its
// node layer still gave 5000 the machine port 5100: left out, and the filter closed 5100 for 98 minutes while the old
// registry container still served there; every build failed "dial tcp 10.10.0.1:5100: i/o timeout".
r.Modules = append(r.Modules, mcat.Manifest{Module: "distribution",
Listens: []mcat.Listening{{Name: "registry", Port: 5100, Protocol: "tcp", From: mcat.FromEverywhere}},
Resources: []map[string]any{{"id": "store", "type": "container", "name": "mesh-registry",
"image": "registry@sha256:" + strings.Repeat("a", 64), "ports": []any{"127.0.0.1:5001:5001"}}}})
with := mcat.Rendering{Mesh: []string{"198.51.100.2"}, OutwardLinks: []string{"eth0"}, TunnelInterface: "mesh0",
Settings: mcat.SettingsBy{"distribution": {{From: "novox", Placement: map[string]any{mcat.PortsSetting: map[string]any{"5000": 5100}}}}}}
if left := LeftOut(r, with.Settings, false); left["distribution"] == "" {
t.Fatalf("the stale port setting does not leave distribution out: %v", left)
}
if out, err = Declaration(r, with); err != nil {
t.Fatalf("declaration with a module left out: %v", err)
}
content = ""
for _, res := range out {
if res["path"] == "/etc/mesh/filter.nft" {
content, _ = res["content"].(string)
}
}
if !strings.Contains(content, "dport 5100 accept") || !strings.Contains(content, "dport 443 accept") {
t.Fatalf("the filter closes the port of a module left out of the declaration, which still runs:\n%s", content)
}
// Its per-node narrowing still holds (ADR 0046), and an exposure that cannot be read for it opens it no wider than
// the mesh, never at the manifest's default (review round 1).
for _, row := range []struct {
expose map[string]any
from string
}{
{map[string]any{"5100": mcat.FromMachine}, mcat.FromMachine},
{map[string]any{"9999": mcat.FromMachine}, mcat.FromMesh}, // a port it does not listen on: refused
} {
layer := with.Settings["distribution"][0]
layer.Placement = map[string]any{mcat.PortsSetting: map[string]any{"5000": 5100}, mcat.ExposeSetting: row.expose}
with.Settings = mcat.SettingsBy{"distribution": {layer}}
rules, err := Rules(r, with)
if err != nil {
t.Fatalf("expose %v: %v", row.expose, err)
}
var from []string
for _, rule := range rules {
if rule.Port == 5100 {
from = append(from, rule.From)
}
}
if len(from) != 1 || from[0] != row.from {
t.Errorf("expose %v: the left-out module's 5100 opens from %v, want %s", row.expose, from, row.from)
}
}
} }
// The host does not sort, so the order the mesh writes is the order a machine applies. // The host does not sort, so the order the mesh writes is the order a machine applies.
@@ -37,6 +37,11 @@ func TestWithNoHolderOnRecordTheSoleClaimantHoldsAndTwoAreRefused(t *testing.T)
if len(problems) != 1 || !strings.Contains(problems[0], "both claim") { if len(problems) != 1 || !strings.Contains(problems[0], "both claim") {
t.Fatalf("two claimants with nothing on record were not refused: %v", problems) t.Fatalf("two claimants with nothing on record were not refused: %v", problems)
} }
// Held for the module that cannot hold, which then holds nothing; the other still does (issue 1966).
held, holding := holdClaims([]mcat.Manifest{oldBroker(), newBroker()}, node, nil, nil)
if len(holding) != 1 || !strings.Contains(holding["new-broker"], "both claim") || len(held) != 1 || held[0].Module != "old-broker" {
t.Fatalf("the second claimant is not the one held: held=%v holding=%v", held, holding)
}
} }
// With a holder on record, the other eligible assignment is silent: not refused, and not holding. // With a holder on record, the other eligible assignment is silent: not refused, and not holding.
+80
View File
@@ -0,0 +1,80 @@
package resolution
import (
"fmt"
"strings"
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// withoutUnconfigured drops each resource whose `if-config` holds none of its conditions against the module's config
// values (design 42 zfs-owns-zfs §2, ADR 0679 §16 (b)), and the key itself from every resource that keeps it: the
// node-engine parses strictly and knows no such key. What leaves the declaration this way is removed as any resource
// that leaves it (ADR 0269 §2), by the node-engine's own orphan removal.
func withoutUnconfigured(resources []map[string]any, layers []mcat.Layer) []map[string]any {
out := make([]map[string]any, 0, len(resources))
for _, r := range resources {
conditions, has := r[mcat.IfConfigField]
if !has {
out = append(out, r)
continue
}
if !ifConfigHolds(conditions, layers) {
continue
}
kept := make(map[string]any, len(r))
for k, v := range r {
if k != mcat.IfConfigField {
kept[k] = v
}
}
out = append(out, kept)
}
return out
}
// ifConfigHolds is whether any condition of a list holds: `<key>` when the config value is set and is neither false,
// off, none, empty nor an empty list; `<key>=<value>` when the value equals it, or a list value holds it as an entry.
func ifConfigHolds(conditions any, layers []mcat.Layer) bool {
list, _ := conditions.([]any)
for _, c := range list {
key, want, equals := strings.Cut(fmt.Sprint(c), "=")
value, set := settingValue(layers, key)
if !set {
continue
}
if !equals {
if !isOff(value) {
return true
}
continue
}
if entries, isList := value.([]any); isList {
for _, e := range entries {
if plainly(e) == want {
return true
}
}
} else if plainly(value) == want {
return true
}
}
return false
}
// isOff is a config value that says "not here": false, off, none, an empty text or an empty list.
func isOff(value any) bool {
switch v := value.(type) {
case bool:
return !v
case []any:
return len(v) == 0
case nil:
return true
}
switch plainly(value) {
case "", "false", "off", "none":
return true
}
return false
}
@@ -6,36 +6,6 @@ import (
mcat "git.novox.be/novox/mesh-sdk/go/catalogue" mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
) )
// resolverVerbs is the contract every holder of node-resolver serves (novox/hq ADR 0247): what is routed
// where, route a set of domains, and take a route away. The same verbs are served on the machine itself to
// the modules there, so what a VPN pushed never crosses the bus to be routed; on the mesh they are the
// operator's way to read and correct the same.
func resolverVerbs() []mcat.Verb {
return []mcat.Verb{
{Name: "routes", Description: "What this machine's own resolver sends where: the mesh's resolvers, " +
"which answer every name not routed elsewhere, and each link given servers of its own with the " +
"domains routed to them. Also the resolver file's outside writes it kept, newest first: when, who " +
"wrote it as far as the file says, whether a module took it, and when the resolver's own file was " +
"put back.",
Input: schema(map[string]string{}, nil),
Replaces: []string{"resolvectl status", "resolvectl dns", "resolvectl domain"}},
{Name: "route", Description: "Send these domains, and every name under them, to these servers over " +
"this link — and only them: the link is never the machine's default route for names, and the " +
"mesh's own domain is refused. Replaces whatever the link was given before. A link that goes away " +
"takes its route with it.",
Input: schema(map[string]string{
"link": "the network link the servers are reached over, by name (a VPN's tunnel interface)",
"domains": "the domains to route there, separated by spaces or commas",
"servers": "the servers' addresses, separated by spaces or commas",
}, []string{"link", "domains", "servers"}),
Replaces: []string{"resolvectl dns", "resolvectl domain", "resolvectl default-route"}},
{Name: "unroute", Description: "Take one link's route away: its domains go back to the mesh's " +
"resolvers. Nothing changes when the link has none.",
Input: schema(map[string]string{"link": "the network link, by name"}, []string{"link"}),
Replaces: []string{"resolvectl revert"}},
}
}
// holdsSeat says whether a module claims a seat, by its current name. // holdsSeat says whether a module claims a seat, by its current name.
func holdsSeat(m mcat.Manifest, seat string) bool { func holdsSeat(m mcat.Manifest, seat string) bool {
for _, c := range m.Claims { for _, c := range m.Claims {
@@ -151,6 +151,16 @@ func TestTheCataloguesResolverHoldersWriteTheFileAndProvideSplitDNS(t *testing.T
func TestTheCataloguesResolverComposesOnAMachine(t *testing.T) { func TestTheCataloguesResolverComposesOnAMachine(t *testing.T) {
shelf := resolverShelf(t) shelf := resolverShelf(t)
shelf["systemd-resolved"] = catalogueManifest(t, "systemd-resolved") shelf["systemd-resolved"] = catalogueManifest(t, "systemd-resolved")
// The options line is the catalogue's own template's, never a literal of it here (design 50 resolver-wait §3).
options := ""
for _, l := range strings.Split(shelf["systemd-resolved"].Facts["resolvers"].Template, "\n") {
if strings.HasPrefix(l, "options ") {
options = l
}
}
if options == "" {
t.Fatal("the catalogue's resolver template has no options line")
}
got, err := Resolve(shelf, []string{"dnsmasq", "systemd-networkd", "systemd-resolved"}, splitDNSLaptop(), mcat.World{}) got, err := Resolve(shelf, []string{"dnsmasq", "systemd-networkd", "systemd-resolved"}, splitDNSLaptop(), mcat.World{})
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
@@ -166,7 +176,7 @@ func TestTheCataloguesResolverComposesOnAMachine(t *testing.T) {
ids := byID(out) ids := byID(out)
resolv, _ := ids["systemd-resolved.fact-resolvers"]["content"].(string) resolv, _ := ids["systemd-resolved.fact-resolvers"]["content"].(string)
kept, _ := ids["systemd-resolved.fact-kept"]["content"].(string) kept, _ := ids["systemd-resolved.fact-kept"]["content"].(string)
if !strings.Contains(resolv, "\nnameserver 10.42.0.2\noptions timeout:1 attempts:2 edns0\n") || resolv != kept { if !strings.Contains(resolv, "\nnameserver 10.42.0.2\n"+options+"\n") || resolv != kept {
t.Errorf("the resolver file does not name this machine's own resolver alone, or its copy differs:\n%s", resolv) t.Errorf("the resolver file does not name this machine's own resolver alone, or its copy differs:\n%s", resolv)
} }
if ids["systemd-networkd.fact-resolvers"] != nil { if ids["systemd-networkd.fact-resolvers"] != nil {
@@ -27,45 +27,53 @@ func mailSenderManifest(t *testing.T, own string) mcat.Manifest {
// 1170/AC1: a family owned by the module's own account lands at 0600, owned by that account — and a module's // 1170/AC1: a family owned by the module's own account lands at 0600, owned by that account — and a module's
// other own secrets (no per-secret owner declared) keep secrets-owner, unchanged. // other own secrets (no per-secret owner declared) keep secrets-owner, unchanged.
func TestAnOwnSecretsOwnerIsGivenToItsFiles(t *testing.T) { func TestAnOwnSecretsOwnerIsGivenToItsFiles(t *testing.T) {
m := mailSenderManifest(t, "mesh-mail-send") // ADR 0680 §1, §4: the owner is the account the manifest names: one the module makes, the operator account
m.OwnSecrets["smtp-relay-key"] = mcat.OwnSecret{Path: "/var/lib/mesh-mail/smtp-relay-key", Taken: mcat.TakenAtStart} // (`${machine:account}`, filled with the machine's account) or root.
r := mcat.Resolution{Node: "novox", Modules: []mcat.Manifest{m}} for _, c := range []struct{ name, owner, account, want string }{
out, err := Declaration(r, mcat.Rendering{ {"an account the module makes", "mesh-mail-send", "", "mesh-mail-send"},
Needed: map[string]map[string]string{"mesh-mail": {"mail-credential-smtp": "sealed-credential", "smtp-relay-key": "sealed-relay"}}, {"the operator account", "${machine:account}", "ops", "ops"},
}) {"root", "root", "ops", "root"},
if err != nil { } {
t.Fatalf("composing the declaration: %v", err) t.Run(c.name, func(t *testing.T) {
} m := mailSenderManifest(t, c.owner)
var family, plain map[string]any m.OwnSecrets["smtp-relay-key"] = mcat.OwnSecret{Path: "/var/lib/mesh-mail/smtp-relay-key", Taken: mcat.TakenAtStart}
for _, res := range out { r := mcat.Resolution{Node: "novox", Account: c.account, Modules: []mcat.Manifest{m}}
if res["type"] != "file" { out, err := Declaration(r, mcat.Rendering{
continue Needed: map[string]map[string]string{"mesh-mail": {"mail-credential-smtp": "sealed-credential", "smtp-relay-key": "sealed-relay"}},
} })
switch res["path"] { if err != nil {
case "/var/lib/mesh-mail/mail-credential-smtp.secret": t.Fatalf("composing the declaration: %v", err)
family = res }
case "/var/lib/mesh-mail/smtp-relay-key": var family, plain map[string]any
plain = res for _, res := range out {
} if res["type"] != "file" {
} continue
if family == nil { }
t.Fatalf("the family member was not composed at all: %v", out) switch res["path"] {
} case "/var/lib/mesh-mail/mail-credential-smtp.secret":
if got := family["owner"]; got != "mesh-mail-send" { family = res
t.Fatalf("the family member's owner: %v; want mesh-mail-send", got) case "/var/lib/mesh-mail/smtp-relay-key":
} plain = res
// review round 1, finding 7: asserted, not merely logged. ownedBy never sets "mode", so a sealed secret }
// file carries none here either — the host applies 0600 to every one regardless (novox/hq ADR 0086) — }
// and the owner clause must not have added one of its own that could override that. if family == nil {
if _, has := family["mode"]; has { t.Fatalf("the family member was not composed at all: %v", out)
t.Fatalf("the family member carries its own mode %v; the owner clause must not set one, so the "+ }
"host's 0600 for every sealed secret file stands", family["mode"]) if got := family["owner"]; got != c.want {
} t.Fatalf("the family member's owner: %v; want %s", got, c.want)
if plain == nil { }
t.Fatalf("the module's other own secret was not composed at all: %v", out) // ownedBy never sets "mode", so a sealed secret file carries none here either: the host applies
} // 0600 to every one regardless (novox/hq ADR 0086), and the owner clause must not override it.
if got := plain["owner"]; got != "mesh-mail" { if _, has := family["mode"]; has {
t.Fatalf("a secret with no per-secret owner kept %v; want the module's secrets-owner %q", got, "mesh-mail") t.Fatalf("the family member carries its own mode %v; the host's 0600 must stand", family["mode"])
}
if plain == nil {
t.Fatalf("the module's other own secret was not composed at all: %v", out)
}
if got := plain["owner"]; got != "mesh-mail" {
t.Fatalf("a secret with no per-secret owner kept %v; want the module's secrets-owner %q", got, "mesh-mail")
}
})
} }
} }
+11 -2
View File
@@ -7,6 +7,7 @@ import (
"sort" "sort"
"strings" "strings"
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
mcat "git.novox.be/novox/mesh-sdk/go/catalogue" mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
) )
@@ -56,8 +57,16 @@ func systemPath(path string) string {
return "" return ""
} }
// engineModule is the node-engine's own module, the one that places in engineTrees. // ClaimsEngine is whether a manifest claims the seat node-engine: the node-engine's own module is the one whose entry of
const engineModule = "mesh-host" // the catalog half does (design 47 G §527), never a name fixed in code.
func ClaimsEngine(m mcat.Manifest) bool {
for _, c := range m.Claims {
if c.Name == nodeengine.Seat {
return true
}
}
return false
}
// accessRef is how a module names one of its accesses: ${access:<id>}. // accessRef is how a module names one of its accesses: ${access:<id>}.
var accessRef = regexp.MustCompile(`\$\{access:([a-z0-9][a-z0-9-]*)\}`) var accessRef = regexp.MustCompile(`\$\{access:([a-z0-9][a-z0-9-]*)\}`)
+165 -33
View File
@@ -20,18 +20,35 @@ func heldKey(claim, kind string) string {
// //
// Every reason at once rather than the first, and each says what to do about it. A person // Every reason at once rather than the first, and each says what to do about it. A person
// resolving these fixes them in one pass or in four. // resolving these fixes them in one pass or in four.
type Refusal struct{ Problems []string } type Refusal struct {
Problems []string
// OwnedTwice is each path two modules both own among Problems, as a pair: a composer blames both owners with the
// path, never by the words (design 47 G §721; A-468l).
OwnedTwice []OwnedTwice
}
// OwnedTwice is one path two modules both declare as their own (checkResources), the module that declared it first
// first.
type OwnedTwice struct {
Path string
Modules [2]string
}
func (r *Refusal) Error() string { func (r *Refusal) Error() string {
return "these assignments cannot be applied:\n - " + strings.Join(r.Problems, "\n - ") return "these assignments cannot be applied:\n - " + strings.Join(r.Problems, "\n - ")
} }
// Resolve works out everything a node runs, from what was assigned to it. // ResolveHolding works out everything a node runs, from what was assigned to it. The catalogue is every module the mesh knows
// // about; assigned is what a person put on this node. What comes back is the closure -- assigned modules plus everything they
// The catalogue is every module the mesh knows about; assigned is what a person put on this node. // require -- or a refusal naming every reason it could not be closed. A claim problem costs the module that has it, not the
// What comes back is the closure — assigned modules plus everything they require — or a refusal // machine (issue 1966, ADR 0636 §3: a refused claim blocks that claimant only). The modules that cannot hold what they claim come back with why, each still in
// naming every reason it could not be closed. // the resolution's modules and none of them in its claims; the caller leaves them out of what it declares (LeftOut), so the
func Resolve(catalogue map[string]mcat.Manifest, assigned []string, node mcat.Node, world mcat.World) (mcat.Resolution, error) { // machine composes everything else. Every other refusal is Resolve's.
func ResolveHolding(catalogue map[string]mcat.Manifest, assigned []string, node mcat.Node, world mcat.World) (mcat.Resolution, map[string]string, error) {
return resolve(catalogue, assigned, node, world)
}
func resolve(catalogue map[string]mcat.Manifest, assigned []string, node mcat.Node, world mcat.World) (mcat.Resolution, map[string]string, error) {
elsewhere := world.Held elsewhere := world.Held
var problems []string var problems []string
@@ -176,6 +193,22 @@ func Resolve(catalogue map[string]mcat.Manifest, assigned []string, node mcat.No
absent[want] = true absent[want] = true
reported[want] = true reported[want] = true
} }
// A requirement nothing answers costs the modules that want it, not the machine (A-452, ADR 0636 §3): said once per name,
// each consumer is held after the walk, when the set is known.
// The value is the remedy the mesh-wide refusal adds, empty for a requirement nothing on the node answers.
unanswered := map[string]string{}
var unansweredOrder []string
holdWhenUnanswered := func(want, remedy string) {
if !wantedHard(want) && wantedOptionally(want) {
// Only optional consumers: left absent, never held (ADR 0330 §3).
leaveAbsent(want)
return
}
if _, ok := unanswered[want]; !ok {
unansweredOrder = append(unansweredOrder, want)
}
unanswered[want] = remedy
}
for { for {
if len(queue) == 0 { if len(queue) == 0 {
@@ -368,9 +401,8 @@ func Resolve(catalogue map[string]mcat.Manifest, assigned []string, node mcat.No
} else if len(answers) > 1 { } else if len(answers) > 1 {
remedy = "— assign one of these to a node: " + strings.Join(answers, ", ") remedy = "— assign one of these to a node: " + strings.Join(answers, ", ")
} }
problems = append(problems, fmt.Sprintf( // Not answered on the node nor in the rest of the mesh: costs its consumers only (A-452).
"nothing in this mesh provides %q, wanted by %s %s", holdWhenUnanswered(want, remedy)
want, because[want], remedy))
case len(where) == 1: case len(where) == 1:
if c, pinned := world.Pinned[want]; pinned && !chosenMatches(c, where[0]) { if c, pinned := world.Pinned[want]; pinned && !chosenMatches(c, where[0]) {
// One provider, and it is not the one this machine was told to use. Silently // One provider, and it is not the one this machine was told to use. Silently
@@ -478,8 +510,7 @@ func Resolve(catalogue map[string]mcat.Manifest, assigned []string, node mcat.No
continue continue
} }
reported[want] = true reported[want] = true
problems = append(problems, fmt.Sprintf( holdWhenUnanswered(want, "")
"nothing provides %q, wanted by %s", want, because[want]))
continue continue
case 1: case 1:
// No choice to make, so none is made. This is the case that lets `install i3` bring // No choice to make, so none is made. This is the case that lets `install i3` bring
@@ -586,16 +617,19 @@ func Resolve(catalogue map[string]mcat.Manifest, assigned []string, node mcat.No
resolution := mcat.Resolution{Node: node.Name, At: node.At, PublicDomain: node.PublicDomain, resolution := mcat.Resolution{Node: node.Name, At: node.At, PublicDomain: node.PublicDomain,
Account: node.Account, AccountHome: node.AccountHome, AgentAccount: node.AgentAccount, Account: node.Account, AccountHome: node.AccountHome, AgentAccount: node.AgentAccount,
AgentAccountHome: node.AgentAccountHome, Capabilities: node.Capabilities, AgentAccountHome: node.AgentAccountHome, Capabilities: node.Capabilities,
Because: because, Needs: needs, Unhostable: unhostable, Kept: kept, Because: because, Needs: needs, Unhostable: unhostable, Kept: kept}
Absent: absentOf(absent, order, catalogue)}
for _, n := range providersFirst(order, catalogue) { for _, n := range providersFirst(order, catalogue) {
resolution.Modules = append(resolution.Modules, catalogue[n]) resolution.Modules = append(resolution.Modules, catalogue[n])
} }
problems = append(problems, checkCapabilities(resolution.Modules, node)...) problems = append(problems, checkCapabilities(resolution.Modules, node)...)
claims, claimProblems := checkClaims(resolution.Modules, node, elsewhere, world.Holdings) claims, held := holdClaims(resolution.Modules, node, elsewhere, world.Holdings)
problems = append(problems, claimProblems...) held, unheld := holdUnanswered(resolution.Modules, unansweredOrder, unanswered, because, held, absent)
problems = append(problems, checkResources(resolution.Modules)...) // Listed after the holding: an optional consumer of a requirement nothing answers is left absent there (ADR 0330 §3, §7).
resolution.Absent = absentOf(absent, order, catalogue)
problems = append(problems, unheld...)
owned, twice := resourceProblems(resolution.Modules)
problems = append(problems, owned...)
resolution.Claims = claims resolution.Claims = claims
// Judged over the closure — what this node will actually run — so a holder pulled in by a // Judged over the closure — what this node will actually run — so a holder pulled in by a
@@ -615,10 +649,14 @@ func Resolve(catalogue map[string]mcat.Manifest, assigned []string, node mcat.No
} }
if len(problems) > 0 { if len(problems) > 0 {
// A pass refused anyway says every reason at once, a held module's among them.
for _, why := range held {
problems = append(problems, why)
}
sort.Strings(problems) sort.Strings(problems)
return mcat.Resolution{}, &Refusal{Problems: problems} return mcat.Resolution{}, nil, &Refusal{Problems: problems, OwnedTwice: twice}
} }
return resolution, nil return resolution, held, nil
} }
// isModule reports whether a name is a module in its own right rather than only something // isModule reports whether a name is a module in its own right rather than only something
@@ -672,12 +710,81 @@ func checkCapabilities(modules []mcat.Manifest, node mcat.Node) []string {
return problems return problems
} }
// checkClaims refuses two modules holding one singular thing. // unansweredWords is why a consumer is held for a requirement nothing answers, naming the provision.
// func unansweredWords(want, consumer, remedy string) string {
// Within this node's own set, and against what is already held elsewhere for the wider scopes. A if remedy == "" {
// claim at mesh scope is the same idea as the mesh's one hub, said once instead of hard-coded. return fmt.Sprintf("nothing provides %q, wanted by %s", want, consumer)
func checkClaims(modules []mcat.Manifest, node mcat.Node, elsewhere []mcat.Held, holdings []mcat.Held) ([]mcat.Held, []string) { }
return fmt.Sprintf("nothing in this mesh provides %q, wanted by %s %s", want, consumer, remedy)
}
// holdUnanswered holds each module of the set that wants a requirement nothing answers, with the words "nothing provides" naming the
// provision (A-452): it stays in the set, so what it provides still counts for the others. A requirement none of the set's modules
// wants (an assigned name the catalogue does not know) has no consumer to hold and comes back as a problem of the node; so does a
// set whose every module is held, since then nothing could be composed.
func holdUnanswered(modules []mcat.Manifest, unanswered []string, remedies map[string]string, because map[string]string, held map[string]string, absent map[string]bool) (map[string]string, []string) {
var problems []string var problems []string
for _, want := range unanswered {
found := false
for _, m := range modules {
wants := false
for _, w := range m.Wants() {
if w == want {
wants = true
break
}
}
if !wants {
continue
}
found = true
if m.IsOptional(want) {
// Optional for this module: not held, left absent and listed (ADR 0330 §3, §7).
absent[want] = true
continue
}
if held == nil {
held = map[string]string{}
}
why := unansweredWords(want, m.Module, remedies[want])
if prior, ok := held[m.Module]; ok {
why = prior + "; " + why
}
held[m.Module] = why
}
if !found {
problems = append(problems, unansweredWords(want, because[want], remedies[want]))
}
}
if len(unanswered) > 0 && len(held) >= len(modules) {
problems = append(problems, "no module of the node could be composed: every one is held")
}
return held, problems
}
// holdClaims refuses two modules holding one singular thing, within this node's own set and against what is held elsewhere for
// the wider scopes: the claims that hold, and for each module that cannot hold what it claims, why (every reason, joined).
func holdClaims(modules []mcat.Manifest, node mcat.Node, elsewhere []mcat.Held, holdings []mcat.Held) ([]mcat.Held, map[string]string) {
held, problems := claimsOf(modules, node, elsewhere, holdings)
var out map[string]string
for _, p := range problems {
if out == nil {
out = map[string]string{}
}
if prior, ok := out[p.module]; ok {
out[p.module] = prior + "; " + p.why
} else {
out[p.module] = p.why
}
}
return held, out
}
// claimProblem is a refusal of one module's claim.
type claimProblem struct{ module, why string }
func claimsOf(modules []mcat.Manifest, node mcat.Node, elsewhere []mcat.Held, holdings []mcat.Held) ([]mcat.Held, []claimProblem) {
var problems []claimProblem
var held []mcat.Held var held []mcat.Held
// onRecord is every recorded holder of a seat, if a handover ever named one: one for most seats, // onRecord is every recorded holder of a seat, if a handover ever named one: one for most seats,
@@ -717,9 +824,9 @@ func checkClaims(modules []mcat.Manifest, node mcat.Node, elsewhere []mcat.Held,
// disagrees with the mesh's definition of the role, and it is refused, named, rather // disagrees with the mesh's definition of the role, and it is refused, named, rather
// than letting two machines answer for what the mesh has one of. // than letting two machines answer for what the mesh has one of.
if s, known := mcat.SeatNamed(c.Name); known && !s.Replicated && len(recs) > 1 { if s, known := mcat.SeatNamed(c.Name); known && !s.Replicated && len(recs) > 1 {
problems = append(problems, fmt.Sprintf( problems = append(problems, claimProblem{m.Module, fmt.Sprintf(
"%q is on record as held by %d assignments, and it is held once per %s — "+ "%q is on record as held by %d assignments, and it is held once per %s — "+
"`seat %s --to <node>/<module>` records one", c.Name, len(recs), scope, c.Name)) "`seat %s --to <node>/<module>` records one", c.Name, len(recs), scope, c.Name)})
continue continue
} }
if !recordedHere(c.Name, scope, m.Module) { if !recordedHere(c.Name, scope, m.Module) {
@@ -734,9 +841,9 @@ func checkClaims(modules []mcat.Manifest, node mcat.Node, elsewhere []mcat.Held,
// one seat, compared by the seat they resolve to and not by how each spelled it. // one seat, compared by the seat they resolve to and not by how each spelled it.
seat := heldKey(c.Name, c.Kind) seat := heldKey(c.Name, c.Kind)
if other, taken := byScope[scope][seat]; taken { if other, taken := byScope[scope][seat]; taken {
problems = append(problems, fmt.Sprintf( problems = append(problems, claimProblem{m.Module, fmt.Sprintf(
"%s and %s both claim %q, and only one thing may hold it per %s", "%s and %s both claim %q, and only one thing may hold it per %s",
other, m.Module, seat, scope)) other, m.Module, seat, scope)})
continue continue
} }
byScope[scope][seat] = m.Module byScope[scope][seat] = m.Module
@@ -762,20 +869,34 @@ func checkClaims(modules []mcat.Manifest, node mcat.Node, elsewhere []mcat.Held,
// Both claim and nobody is on record, or this refusal could not have happened. // Both claim and nobody is on record, or this refusal could not have happened.
// The remedy is the handover that records the holder (novox/hq ADR 0131, // The remedy is the handover that records the holder (novox/hq ADR 0131,
// 04-ISSUES/170), so it is named here rather than left to be found. // 04-ISSUES/170), so it is named here rather than left to be found.
problems = append(problems, fmt.Sprintf( problems = append(problems, claimProblem{h.Module, fmt.Sprintf(
"%s on %s claims %q, which %s on %s already holds — one per mesh. Nothing is "+ "%s on %s claims %q, which %s on %s already holds — one per mesh. Nothing is "+
"on record for it; `seat %s --to %s/%s` records the holder, and the other "+ "on record for it; `seat %s --to %s/%s` records the holder, and the other "+
"assignment then stands beside it, eligible and silent", "assignment then stands beside it, eligible and silent",
h.Module, node.Name, h.Claim, e.Module, e.Node, h.Claim, e.Node, e.Module)) h.Module, node.Name, h.Claim, e.Module, e.Node, h.Claim, e.Node, e.Module)})
case mcat.ScopeSite: case mcat.ScopeSite:
if node.Site != "" && node.Site == e.Site { if node.Site != "" && node.Site == e.Site {
problems = append(problems, fmt.Sprintf( problems = append(problems, claimProblem{h.Module, fmt.Sprintf(
"%s on %s claims %q, which %s on %s already holds at %s — one per site", "%s on %s claims %q, which %s on %s already holds at %s — one per site",
h.Module, node.Name, h.Claim, e.Module, e.Node, node.Site)) h.Module, node.Name, h.Claim, e.Module, e.Node, node.Site)})
} }
} }
} }
} }
// A module refused here holds nothing of what it claims: its claims are not this node's holdings.
refused := map[string]bool{}
for _, p := range problems {
refused[p.module] = true
}
if len(refused) > 0 {
kept := held[:0]
for _, h := range held {
if !refused[h.Module] {
kept = append(kept, h)
}
}
held = kept
}
return held, problems return held, problems
} }
@@ -802,6 +923,13 @@ func canonicalSeat(name string) string {
// resolves with no refusal. What is refused is the contradiction — a path one module owns and // resolves with no refusal. What is refused is the contradiction — a path one module owns and
// another merely accesses — because shared data is the operator's and nobody's to own. // another merely accesses — because shared data is the operator's and nobody's to own.
func checkResources(modules []mcat.Manifest) []string { func checkResources(modules []mcat.Manifest) []string {
problems, _ := resourceProblems(modules)
return problems
}
// resourceProblems is checkResources' problems, with each path two modules both own as a pair beside its words.
func resourceProblems(modules []mcat.Manifest) ([]string, []OwnedTwice) {
var twice []OwnedTwice
// Judged as the machine composes them: where the node-resolver holder writes the resolver file, the // Judged as the machine composes them: where the node-resolver holder writes the resolver file, the
// uplink holder's rendering of it is not composed, so it is not a second owner (novox/hq ADR 0247). // uplink holder's rendering of it is not composed, so it is not a second owner (novox/hq ADR 0247).
modules = steppedBack(modules) modules = steppedBack(modules)
@@ -879,6 +1007,9 @@ func checkResources(modules []mcat.Manifest) []string {
if other, taken := owner[key]; taken && other != m.Module { if other, taken := owner[key]; taken && other != m.Module {
problems = append(problems, fmt.Sprintf( problems = append(problems, fmt.Sprintf(
"%s and %s both declare the %s %q", other, m.Module, field, value)) "%s and %s both declare the %s %q", other, m.Module, field, value))
if field == "path" {
twice = append(twice, OwnedTwice{Path: value, Modules: [2]string{other, m.Module}})
}
} }
owner[key] = m.Module owner[key] = m.Module
if field == "path" { if field == "path" {
@@ -899,6 +1030,7 @@ func checkResources(modules []mcat.Manifest) []string {
if other, taken := owner[key]; taken && other != m.Module { if other, taken := owner[key]; taken && other != m.Module {
problems = append(problems, fmt.Sprintf( problems = append(problems, fmt.Sprintf(
"%s and %s both declare the path %q", other, m.Module, fact.Path)) "%s and %s both declare the path %q", other, m.Module, fact.Path))
twice = append(twice, OwnedTwice{Path: fact.Path, Modules: [2]string{other, m.Module}})
} }
owner[key] = m.Module owner[key] = m.Module
ownedPath[fact.Path] = m.Module ownedPath[fact.Path] = m.Module
@@ -934,7 +1066,7 @@ func checkResources(modules []mcat.Manifest) []string {
} }
} }
} }
return problems return problems, twice
} }
// meshNetwork is what to assign to a machine that needs to reach another one. // meshNetwork is what to assign to a machine that needs to reach another one.
+69 -25
View File
@@ -53,31 +53,75 @@ func TestARequirementWithOneAnswerIsTakenSilently(t *testing.T) {
} }
} }
func TestARequirementWithSeveralAnswersIsRefusedAndNamed(t *testing.T) { // A requirement nothing answers costs the modules that want it, not the machine (A-452; ace, 2026-11-16 23:11: ten unanswered
// The mesh does not pick. A default would be a choice made for somebody who finds out later, // requirements refused the whole composition), whether nothing on the node or nothing in the whole mesh answers it; one that
// and naming the candidates is the whole remedy. // only optional consumers want is left absent (ADR 0330 §3); several answers are still the node's refusal, naming them (the
_, err := Resolve(shelf( // mesh does not pick: a default would be a choice made for somebody who finds out later); and a node whose every module is held,
mod("editor", nil, []string{"shell"}, nil), // or an assigned name nobody wants, is still refused.
mod("bash", []string{"shell"}, nil, nil), func TestARequirementNoOneAnswersHoldsItsConsumersAndSeveralAnswersRefuseTheNode(t *testing.T) {
mod("zsh", []string{"shell"}, nil, nil), optional := mod("viewer", nil, []string{"codec"}, nil)
mod("fish", []string{"shell"}, nil, nil), optional.OptionalRequires = map[string]bool{"codec": true}
), []string{"editor"}, workstation(), mcat.World{}) hard := mod("player", nil, []string{"codec"}, nil)
meshDB := mcat.Manifest{Module: "postgres", Provides: []mcat.Offer{{Name: "postgres-database", Scope: mcat.ScopeMesh}}}
if err == nil { for _, c := range []struct {
t.Fatal("a requirement with three answers was resolved without asking") name string
} shelf map[string]mcat.Manifest
for _, want := range []string{"bash", "fish", "zsh", "choose one"} { assigned []string
if !strings.Contains(err.Error(), want) { held map[string]string // module -> words in its why
t.Errorf("the refusal does not mention %q: %v", want, err) refused []string // words of the node's refusal
} absent bool
} }{
} {name: "on the node", assigned: []string{"ca-trust", "baserow", "sheet-viewer", "editor"},
shelf: shelf(mod("ca-trust", nil, []string{"internal-acme-ca"}, nil),
func TestARequirementWithNoAnswerIsRefused(t *testing.T) { mod("baserow", []string{"spreadsheet"}, []string{"route"}, nil),
_, err := Resolve(shelf(mod("i3", nil, []string{"xorg"}, nil)), []string{"i3"}, workstation(), mcat.World{}) mod("sheet-viewer", nil, []string{"spreadsheet"}, nil), mod("editor", nil, nil, nil)),
held: map[string]string{"ca-trust": `nothing provides "internal-acme-ca"`, "baserow": `nothing provides "route"`}},
if err == nil || !strings.Contains(err.Error(), "nothing provides") { {name: "in the whole mesh", assigned: []string{"baserow", "editor"},
t.Fatalf("a requirement nothing satisfies gave %v", err) shelf: shelf(meshDB, mod("baserow", nil, []string{"postgres-database"}, nil), mod("editor", nil, nil, nil)),
held: map[string]string{"baserow": `nothing in this mesh provides "postgres-database", wanted by baserow`}},
{name: "wanted hard by one and optionally by another", assigned: []string{"player", "viewer"},
shelf: shelf(hard, optional), held: map[string]string{"player": `nothing provides "codec"`}, absent: true},
{name: "every module held", assigned: []string{"ca-trust"},
shelf: shelf(mod("ca-trust", nil, []string{"internal-acme-ca"}, nil)),
refused: []string{"nothing provides", "every one is held"}},
{name: "nobody wants it", assigned: []string{"ghost"}, shelf: shelf(mod("editor", nil, nil, nil)),
refused: []string{"nothing provides"}},
{name: "several answers", assigned: []string{"editor"},
shelf: shelf(mod("editor", nil, []string{"shell"}, nil), mod("bash", []string{"shell"}, nil, nil),
mod("zsh", []string{"shell"}, nil, nil), mod("fish", []string{"shell"}, nil, nil)),
refused: []string{"bash", "fish", "zsh", "choose one"}},
} {
t.Run(c.name, func(t *testing.T) {
got, held, err := ResolveHolding(c.shelf, c.assigned, workstation(), mcat.World{})
if c.refused != nil {
if err == nil {
t.Fatal("the node was composed, not refused")
}
for _, w := range c.refused {
if !strings.Contains(err.Error(), w) {
t.Errorf("the refusal does not mention %q: %v", w, err)
}
}
return
}
if err != nil {
t.Fatalf("the node was refused whole: %v", err)
}
if len(held) != len(c.held) {
t.Errorf("held %v, want modules %v", held, c.held)
}
for m, w := range c.held {
if !strings.Contains(held[m], w) {
t.Errorf("%s is held with %q, want %q", m, held[m], w)
}
}
if _, heldToo := held["viewer"]; c.absent && (heldToo || len(got.Absent) != 1 || got.Absent[0] != (mcat.Absent{Consumer: "viewer", Requirement: "codec"})) {
t.Errorf("the optional consumer was held or not listed absent: held %v absent %v", held, got.Absent)
}
if len(got.Modules) != len(c.assigned) {
t.Errorf("the node does not keep every module of its set: %v", names(got))
}
})
} }
} }
+13 -4
View File
@@ -310,16 +310,25 @@ func TestAModuleOfItsOwnAccountIsServedByARuntimeOfItsOwn(t *testing.T) {
} }
gob, img := builder("gob", "node-build-go"), builder("img", "node-build-image") gob, img := builder("gob", "node-build-go"), builder("img", "node-build-image")
with.Needed["gob"], with.Needed["img"] = map[string]string{"broker": "own"}, map[string]string{"broker": "own"} with.Needed["gob"], with.Needed["img"] = map[string]string{"broker": "own"}, map[string]string{"broker": "own"}
// The node-engine's module is the one whose entry claims its seat (design 47 G §527), whatever it is called; with none in
// the resolution its settings are not found and the default stands.
engine := mcat.Manifest{Module: "mesh-host", Claims: []mcat.Claim{{Name: "node-engine"}}}
for _, c := range []struct { for _, c := range []struct {
settings mcat.SettingsBy settings mcat.SettingsBy
share string share string
noEngine bool
}{ }{
{nil, "50"}, {nil, "50", false},
{mcat.SettingsBy{"mesh-host": {{From: "anchor", Values: map[string]any{"build-cpu-share": float64(75)}}}}, "75"}, {mcat.SettingsBy{"mesh-host": {{From: "anchor", Values: map[string]any{"build-cpu-share": float64(75)}}}}, "75", false},
{mcat.SettingsBy{"mesh-host": {{From: "anchor", Values: map[string]any{"build-cpu-share": float64(0)}}}}, "50"}, {mcat.SettingsBy{"mesh-host": {{From: "anchor", Values: map[string]any{"build-cpu-share": float64(0)}}}}, "50", false},
{mcat.SettingsBy{"mesh-host": {{From: "anchor", Values: map[string]any{"build-cpu-share": float64(75)}}}}, "50", true},
} { } {
with.Settings = c.settings with.Settings = c.settings
runtimes, err := resolutionOwnRuntimes(mcat.Resolution{Node: "anchor", Modules: []mcat.Manifest{gob, img, telegram, goRuntime}}, with) modules := []mcat.Manifest{gob, img, telegram, goRuntime, engine}
if c.noEngine {
modules = modules[:4]
}
runtimes, err := resolutionOwnRuntimes(mcat.Resolution{Node: "anchor", Modules: modules}, with)
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -29,6 +29,10 @@ var appliedThrough = map[string]string{
// A variable rather than a constant only so a test can hold both behaviours; nothing else sets it. // A variable rather than a constant only so a test can hold both behaviours; nothing else sets it.
var enforceSeatDependencies = true var enforceSeatDependencies = true
// nodeEngineSeat is the seat the node-engine's own module claims (design 47 G §508, §527, §634): the
// foundation's host is the entry that claims it, never a module named here.
const nodeEngineSeat = "node-engine"
// foundationModules are the pieces genesis lays before any module exists (novox/hq ADR 0207 §5): // foundationModules are the pieces genesis lays before any module exists (novox/hq ADR 0207 §5):
// the host and the private network. Registered as modules so they can be assigned, but what they // the host and the private network. Registered as modules so they can be assigned, but what they
// declare is the installation's, not a module's, so it is never judged. The third piece, the // declare is the installation's, not a module's, so it is never judged. The third piece, the
@@ -38,7 +42,6 @@ var enforceSeatDependencies = true
// The private network's module is overlay.Name, written out because the overlay package composes // The private network's module is overlay.Name, written out because the overlay package composes
// on top of this one; its resources are computed, which exempts it by the rule below as well. // on top of this one; its resources are computed, which exempts it by the rule below as well.
var foundationModules = map[string]bool{ var foundationModules = map[string]bool{
"mesh-host": true,
"mesh-wireguard": true, "mesh-wireguard": true,
} }
@@ -46,7 +49,7 @@ var foundationModules = map[string]bool{
// module whose resources are computed is the mesh's by construction — the private network's peer // module whose resources are computed is the mesh's by construction — the private network's peer
// list and its tools are the controller's, written per node — so it counts whatever its name. // list and its tools are the controller's, written per node — so it counts whatever its name.
func isFoundation(m mcat.Manifest) bool { func isFoundation(m mcat.Manifest) bool {
return foundationModules[m.Module] || m.Computed != "" return foundationModules[m.Module] || m.ClaimsSeat(nodeEngineSeat) || m.Computed != ""
} }
// DependsOn is every seat a module needs held on its node: derived from the resource types it // DependsOn is every seat a module needs held on its node: derived from the resource types it
@@ -95,3 +95,17 @@ func TestWithTheSwitchFlippedAnUnmetDependencyRefusesTheNode(t *testing.T) {
t.Errorf("the first pass refused an unmet dependency: %v", err) t.Errorf("the first pass refused an unmet dependency: %v", err)
} }
} }
func TestTheFoundationHostIsTheEntryThatClaimsTheNodeEngineSeat(t *testing.T) {
svc := res("service", "x")
// Whatever its name, the entry claiming the seat is the foundation's: nothing to depend on.
engine := withResources(mod("the-engine", nil, nil, nil, mcat.Claim{Name: "node-engine"}), svc)
if got := DependsOn(nil, engine); len(got) != 0 {
t.Errorf("the entry claiming node-engine depends on %v, want nothing (§527)", got)
}
// A module named as the old code named the engine, claiming no such seat, is an ordinary module.
named := withResources(mod("mesh-host", nil, nil, nil), svc)
if got := DependsOn(nil, named); !reflect.DeepEqual(got, []string{mcat.ServiceManagerSeat}) {
t.Errorf("a module merely named mesh-host depends on %v, want the service manager seat", got)
}
}
-74
View File
@@ -1,74 +0,0 @@
package resolution
import (
"fmt"
"sort"
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// SeatsToPassOn reads the mesh for seats held once whose holder on record has left (novox/hq ADR 0449 §4).
//
// **A holder on record that is assigned on no machine holds nothing, and a seat that answers nothing is
// the day a delivery removed its module.** Of such a seat, exactly one assignment in the mesh able to hold
// it (it claims the seat at its scope and CanHold accepts it) takes it over: that is a pass, and the caller
// records it and writes the hand act. With no such assignment the record stays; with two or more nothing is
// recorded and the answer says so, naming both and the `seat --to` that settles it. A holder still assigned
// anywhere changes nothing: the other claimants stand beside it, eligible and silent (ADR 0131). Only mesh
// seats held once are read: a replicated seat's holders are added by an act (ADR 0223), a kinded bench is held
// by its claims, and a seat with several holders on record is a store that disagrees with the mesh (refused
// by the resolver).
func SeatsToPassOn(holdings []mcat.Held, assignments []mcat.Assignment) ([]mcat.SeatPass, []string) {
assigned := map[string]bool{}
for _, a := range assignments {
assigned[a.Manifest.Module] = true
}
bySeat := map[string][]mcat.Held{}
var order []string
for _, h := range holdings {
s, ok := mcat.SeatNamed(h.Claim)
if !ok || s.Replicated || s.Scope != mcat.ScopeMesh || h.Scope != mcat.ScopeMesh || mcat.KindedBenches[s.Name] {
continue
}
if _, seen := bySeat[s.Name]; !seen {
order = append(order, s.Name)
}
bySeat[s.Name] = append(bySeat[s.Name], h)
}
sort.Strings(order)
var passes []mcat.SeatPass
var said []string
for _, name := range order {
recs := bySeat[name]
if len(recs) != 1 || assigned[recs[0].Module] {
continue
}
seat, _ := mcat.SeatNamed(name)
var claimants []mcat.Held
for _, a := range assignments {
if mcat.CanHold(a.Manifest, seat) != nil {
continue
}
claimants = append(claimants, mcat.Held{Claim: seat.Name, Scope: seat.Scope, Node: a.Node, Module: a.Manifest.Module})
}
sort.Slice(claimants, func(i, j int) bool {
if claimants[i].Node != claimants[j].Node {
return claimants[i].Node < claimants[j].Node
}
return claimants[i].Module < claimants[j].Module
})
switch len(claimants) {
case 0:
case 1:
passes = append(passes, mcat.SeatPass{Seat: seat, From: recs[0], To: claimants[0]})
default:
said = append(said, fmt.Sprintf(
"%s on %s is on record as the holder of %q and is assigned nowhere, and %s on %s and %s on %s both claim it "+
"— nothing is recorded; `seat %s --to <node>/<module>` records the one that holds it",
recs[0].Module, recs[0].Node, seat.Name, claimants[0].Module, claimants[0].Node,
claimants[1].Module, claimants[1].Node, seat.Name))
}
}
return passes, said
}
@@ -1,80 +0,0 @@
package resolution
import (
"strings"
"testing"
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
func onRecord(node, module string) []mcat.Held {
return []mcat.Held{{Claim: "mesh-broker", Scope: mcat.ScopeMesh, Node: node, Module: module}}
}
// 1317/AC1
func TestASeatWhoseHolderIsAssignedNowhereIsPassedToItsOnlyClaimant(t *testing.T) {
busSeatDelivering(t, "mesh-bus")
passes, said := SeatsToPassOn(onRecord("novox", "old-broker"),
[]mcat.Assignment{{Node: "novox", Manifest: newBroker()}})
if len(said) != 0 || len(passes) != 1 {
t.Fatalf("the only claimant was not given the seat: passes=%v said=%v", passes, said)
}
p := passes[0]
if p.Seat.Name != "mesh-broker" || p.From.Module != "old-broker" || p.From.Node != "novox" ||
p.To.Module != "new-broker" || p.To.Node != "novox" {
t.Fatalf("the pass does not name the seat, the former holder and the new one: %+v", p)
}
}
// 1317/AC2
func TestWithTwoClaimantsAndTheHolderGoneNothingIsPassedAndTheRefusalNamesBoth(t *testing.T) {
busSeatDelivering(t, "mesh-bus")
passes, said := SeatsToPassOn(onRecord("novox", "gone"), []mcat.Assignment{
{Node: "novox", Manifest: newBroker()}, {Node: "ace", Manifest: oldBroker()}})
if len(passes) != 0 || len(said) != 1 {
t.Fatalf("two claimants: passes=%v said=%v", passes, said)
}
for _, want := range []string{"old-broker", "new-broker", "seat mesh-broker --to"} {
if !strings.Contains(said[0], want) {
t.Fatalf("the refusal does not say %q: %s", want, said[0])
}
}
}
// 1317/AC3
func TestAHolderStillAssignedSomewhereChangesNothing(t *testing.T) {
busSeatDelivering(t, "mesh-bus")
// Assigned on another machine than the record names: still assigned somewhere.
passes, said := SeatsToPassOn(onRecord("novox", "old-broker"), []mcat.Assignment{
{Node: "ace", Manifest: oldBroker()}, {Node: "novox", Manifest: newBroker()}})
if len(passes) != 0 || len(said) != 0 {
t.Fatalf("a holder still assigned lost its seat: passes=%v said=%v", passes, said)
}
}
// 1317/AC4
func TestWithNoClaimantTheRecordStays(t *testing.T) {
busSeatDelivering(t, "mesh-bus")
passes, said := SeatsToPassOn(onRecord("novox", "old-broker"), []mcat.Assignment{
{Node: "novox", Manifest: mcat.Manifest{Module: "other"}}})
if len(passes) != 0 || len(said) != 0 {
t.Fatalf("with no claimant: passes=%v said=%v", passes, said)
}
// A claimant that cannot hold the seat (it does not provide what the seat delivers) is no claimant.
bad := mcat.Manifest{Module: "no-bus", Claims: []mcat.Claim{{Name: "mesh-broker", Scope: mcat.ScopeMesh}}}
passes, said = SeatsToPassOn(onRecord("novox", "old-broker"), []mcat.Assignment{{Node: "novox", Manifest: bad}})
if len(passes) != 0 || len(said) != 0 {
t.Fatalf("a module that cannot hold the seat was passed it: passes=%v said=%v", passes, said)
}
}
// A replicated seat is never passed: its holders are added by an act (ADR 0223).
func TestAReplicatedSeatIsNeverPassed(t *testing.T) {
record := []mcat.Held{{Claim: "mesh-dns-resolver", Scope: mcat.ScopeMesh, Node: "novox", Module: "gone"}}
resolver := mcat.Manifest{Module: "resolver", Provides: []mcat.Offer{{Name: "wildcard-resolution", Scope: mcat.ScopeMesh}},
Claims: []mcat.Claim{{Name: "mesh-dns-resolver", Scope: mcat.ScopeMesh}}}
passes, _ := SeatsToPassOn(record, []mcat.Assignment{{Node: "novox", Manifest: resolver}})
if len(passes) != 0 {
t.Fatalf("a replicated seat was passed: %v", passes)
}
}
+16 -1
View File
@@ -26,7 +26,7 @@ func declaredPaths(m mcat.Manifest) map[string]bool {
} }
for _, r := range m.Resources { for _, r := range m.Resources {
if p, ok := r["path"].(string); ok && strings.HasPrefix(p, "/") { if p, ok := r["path"].(string); ok && strings.HasPrefix(p, "/") {
out[path.Clean(p)] = true out[path.Clean(versionedPlaceOwner(r, p))] = true
} }
} }
for _, f := range m.Facts { for _, f := range m.Facts {
@@ -123,3 +123,18 @@ func seatPathsOnMachine(modules []mcat.Manifest) []string {
} }
return problems return problems
} }
// versionedPlaceOwner is the path a resource owns: its own, unless the resource is an archive unpacked into a place named
// for its build (the artifact's digest, short: `.../versions/<version>`, catalogue `${version}`), whose owner is the
// directory holding every such place, because the place changes with each build and the owned paths are an input of the
// tool runner's unit (the holder of node-files reads them at its start): a build of the engine itself renamed its place
// and restarted the tool runner, with nothing of the tool runner changed (issue 1966).
func versionedPlaceOwner(r map[string]any, p string) string {
digest, _ := r["digest"].(string)
version := strings.TrimPrefix(digest, "sha256:")
base := path.Base(path.Clean(p))
if r["type"] == "archive" && len(base) >= 8 && strings.HasPrefix(version, base) {
return path.Dir(path.Clean(p))
}
return p
}
@@ -28,6 +28,26 @@ func TestAGeneratedResourceOnASeatsPathIsRefused(t *testing.T) {
t.Errorf("the refusal does not name %s: %v", want, err) t.Errorf("the refusal does not name %s: %v", want, err)
} }
} }
// And the paths a module owns do not change with its build (issue 1966): the holder of node-files is told them in a
// word of the tool runner's unit, so a place named for each build restarted the tool runner with each build.
digest := "sha256:" + strings.Repeat("ab12", 16)
for _, row := range []struct {
name string
res map[string]any
owns string
}{
{"a build's own place", map[string]any{"id": "next", "type": "archive", "digest": digest, "path": "/usr/lib/x/versions/ab12ab12ab12"}, "/usr/lib/x/versions"},
{"another build's place", map[string]any{"id": "next", "type": "archive", "digest": "sha256:" + strings.Repeat("cd34", 16), "path": "/usr/lib/x/versions/cd34cd34cd34"}, "/usr/lib/x/versions"},
{"a fixed place", map[string]any{"id": "app", "type": "archive", "digest": digest, "path": "/opt/app"}, "/opt/app"},
{"a file", map[string]any{"id": "f", "type": "file", "path": "/etc/f"}, "/etc/f"},
} {
t.Run(row.name, func(t *testing.T) {
got := DeclaredPaths(mcat.Manifest{Module: "m", Resources: []map[string]any{row.res}})
if len(got) != 1 || got[0] != row.owns {
t.Fatalf("owns %v, want [%s]", got, row.owns)
}
})
}
} }
// The stepping back comes from the seat, and is of a rendered file only: an uplink holder declaring the // The stepping back comes from the seat, and is of a rendered file only: an uplink holder declaring the
-15
View File
@@ -4,21 +4,6 @@ import (
mcat "git.novox.be/novox/mesh-sdk/go/catalogue" mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
) )
// configInput is the schema of mesh-config's verb: a value of the key's own type (a number for a number, never its
// text), and a limit that is a number; every other argument a string.
func configInput() map[string]any {
in := schema(map[string]string{"action": "list (when not given), set, clear or history",
"key": "the config key", "value": "for set: the value, of the key's type",
"module": "a module: its own value, with node", "node": "a node",
"why": "for set and clear: why", "by": "who gave the value at the controller's terminal",
"limit": "for history: at most this many, newest"}, nil)
props := in["properties"].(map[string]any)
props["value"] = map[string]any{"description": "for set: the value, of the key's type: a number, a string, " +
"a boolean, or for a trusted key an object or a list"}
props["limit"] = map[string]any{"type": "number", "description": "for history: at most this many, newest"}
return in
}
func providesAt(m mcat.Manifest, provision, scope string) bool { func providesAt(m mcat.Manifest, provision, scope string) bool {
for _, o := range m.Provides { for _, o := range m.Provides {
if o.Name == provision && o.At() == scope { if o.Name == provision && o.At() == scope {
+94
View File
@@ -1,6 +1,12 @@
package resolution package resolution
import ( import (
"go/ast"
"go/parser"
"go/token"
"io/fs"
"path/filepath"
"strconv"
"strings" "strings"
"testing" "testing"
@@ -70,4 +76,92 @@ func TestTheHolderIsTheModuleNotTheMachine(t *testing.T) {
t.Fatalf("the holder was not told apart from a neighbour: %+v", holder) t.Fatalf("the holder was not told apart from a neighbour: %+v", holder)
} }
// A seat's verbs are its definition's, read from the bus (novox/hq ADR 0635 §1): the node-engine keeps no copy of
// them, so no copy can drift from the seat as the seven it kept did (issue 1921). A copy is a list of the catalogue's
// verbs written in the code; the guard proves it finds one, then finds none in the node-engine.
t.Run("no seat's verbs are copied into the node-engine", func(t *testing.T) {
copied := `package x
import mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
func someVerbs() []mcat.Verb { return []mcat.Verb{{Name: "routes"}} }
`
if got := verbCopies(t, "copied.go", copied); len(got) == 0 {
t.Fatal("the guard finds no copy in a file that holds one, so it proves nothing")
}
var found []string
root := filepath.Join("..", "..", "..")
err := filepath.WalkDir(root, func(p string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() && (d.Name() == "testdata" || strings.HasPrefix(d.Name(), ".")) && p != root {
return filepath.SkipDir
}
if d.IsDir() || !strings.HasSuffix(p, ".go") || strings.HasSuffix(p, "_test.go") {
return nil
}
found = append(found, verbCopies(t, p, nil)...)
return nil
})
if err != nil {
t.Fatal(err)
}
if len(found) > 0 {
t.Errorf("the node-engine copies a seat's verbs instead of reading the seat list from the bus: %v", found)
}
})
}
// verbCopies is where one Go file writes the catalogue's verbs itself: a function answering a list of them, or a verb
// written as a literal.
func verbCopies(t *testing.T, name string, src any) []string {
t.Helper()
fset := token.NewFileSet()
f, err := parser.ParseFile(fset, name, src, parser.SkipObjectResolution)
if err != nil {
t.Fatal(err)
}
alias := ""
for _, imp := range f.Imports {
if p, _ := strconv.Unquote(imp.Path.Value); p == "git.novox.be/novox/mesh-sdk/go/catalogue" {
alias = "catalogue"
if imp.Name != nil {
alias = imp.Name.Name
}
}
}
if alias == "" {
return nil
}
isVerb := func(e ast.Expr) bool {
if a, ok := e.(*ast.ArrayType); ok {
e = a.Elt
}
sel, ok := e.(*ast.SelectorExpr)
if !ok || sel.Sel.Name != "Verb" {
return false
}
x, ok := sel.X.(*ast.Ident)
return ok && x.Name == alias
}
var out []string
ast.Inspect(f, func(n ast.Node) bool {
switch n := n.(type) {
case *ast.FuncDecl:
if n.Type.Results != nil {
for _, r := range n.Type.Results.List {
if isVerb(r.Type) {
out = append(out, fset.Position(n.Pos()).String()+" "+n.Name.Name)
}
}
}
case *ast.CompositeLit:
if n.Type != nil && isVerb(n.Type) {
out = append(out, fset.Position(n.Pos()).String())
}
}
return true
})
return out
} }
@@ -1,6 +1,7 @@
package resolution package resolution
import ( import (
"slices"
"strings" "strings"
"testing" "testing"
@@ -75,6 +76,29 @@ func TestAConsumerBesideTheProviderIsServedItsName(t *testing.T) {
if strings.Contains(binding, "${bound:") { if strings.Contains(binding, "${bound:") {
t.Fatalf("a placeholder reached the consumer:\n%s", binding) t.Fatalf("a placeholder reached the consumer:\n%s", binding)
} }
// Issue 1921 (the group check of 8a80e8e86858, novox): the provider held on this machine (composed apart, as what runs
// of it) is not in the composed set, so none of its own names is composed here. The consumer beside it is told
// what it serves without the keys that name them, each said on the need, never the placeholder: its binding refused
// the consumer with "has a file that says ${bound:route:name-web}, and lab does not require route".
held, err := Resolve(shelf(minimalRouteProxy(), m, consumer), []string{"route-proxy", "forge", "lab"}, node, mcat.World{})
if err != nil {
t.Fatal(err)
}
held.Modules = slices.DeleteFunc(slices.Clone(held.Modules), func(m mcat.Manifest) bool { return m.Module == "forge" })
out, err = Declaration(held, mcat.Rendering{})
if err != nil {
t.Fatalf("a consumer beside a held provider: %v", err)
}
for _, r := range out {
if c, _ := r["content"].(string); r["path"] == "/var/lib/lab/package-registry.json" && (strings.Contains(c, "${bound:") || !strings.Contains(c, "npm-path")) {
t.Fatalf("the binding beside a held provider:\n%s", c)
}
}
for _, n := range held.Needs {
if n.Name == "npm-package-registry" && (len(n.LeftOut) != 1 || !strings.Contains(n.LeftOut[0], `"public-root"`)) {
t.Fatalf("the need beside a held provider says not what was left out: %+v", n)
}
}
} }
// From another machine, the provider's served values carry what that machine composed, and a key it // From another machine, the provider's served values carry what that machine composed, and a key it
+10 -2
View File
@@ -65,8 +65,9 @@ func unsetInConfigWords(err error, m mcat.Manifest) error {
// The last layer setting a key wins, which is the node's over the mesh's over the module's own // The last layer setting a key wins, which is the node's over the mesh's over the module's own
// default (novox/hq ADR 0262) — the same order settle applies to a mergeable file. The caller lays // default (novox/hq ADR 0262) — the same order settle applies to a mergeable file. The caller lays
// the defaults under the layers with WithDefaults. A value that is not a string is written the way a program would read // the defaults under the layers with WithDefaults. A value that is not a string is written the way a program would read
// it (a number without a trailing .000000, a boolean as true/false). // it (a number without a trailing .000000, a boolean as true/false). declared are the keys the module's manifest
func settingInto(resource map[string]any, layers []mcat.Layer, module string) error { // declares under `config`, the only ones `${config-json}` writes (design 42 §9, answer A-15).
func settingInto(resource map[string]any, layers []mcat.Layer, module string, declared ...string) error {
if fmt.Sprint(resource["type"]) == "service" { if fmt.Sprint(resource["type"]) == "service" {
return settingIntoUnit(resource, layers, module) return settingIntoUnit(resource, layers, module)
} }
@@ -83,6 +84,13 @@ func settingInto(resource map[string]any, layers []mcat.Layer, module string) er
if !ok { if !ok {
return nil return nil
} }
content, err := configLinesInto(content, layers, module)
if err != nil {
return err
}
if content, err = configJSONInto(content, layers, declared); err != nil {
return err
}
for _, key := range settingsUsed(content) { for _, key := range settingsUsed(content) {
value, set := settingValue(layers, key) value, set := settingValue(layers, key)
if !set { if !set {
+134 -25
View File
@@ -1,42 +1,151 @@
package resolution package resolution
import ( import (
"errors"
"fmt"
"strings" "strings"
"testing" "testing"
mcat "git.novox.be/novox/mesh-sdk/go/catalogue" mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
) )
// An operator's value reaches a file from the assignment's settings, the node's layer over the // A config value reaches a resource from the assignment's layers, the node's over the mesh's (novox/hq ADR 0112,
// mesh's (novox/hq ADR 0112, ADR 0155), and a value nothing set is refused by name. // ADR 0155), and a value nothing set is refused by name; and a resource under `if-config` is declared only while a
func TestASettingReachesAFileFromTheLayers(t *testing.T) { // config value holds (design 42 zfs-owns-zfs §2, ADR 0679 §16 (b)). One table of the two, each row its layers, what it
file := map[string]any{"id": "env", "type": "file", "path": "/x/mail.env", // does to a resource and what comes of it.
"content": "DOMAIN=${setting:domain}\nSITENAME=${setting:sitename}\nWORKERS=${setting:workers}\n"} func TestAConfigValueReachesAResourceAndDecidesWhetherItIsDeclared(t *testing.T) {
layers := []mcat.Layer{ filled := func(content string) func([]mcat.Layer) (string, error) {
{From: "the mesh", Values: map[string]any{"domain": "example.tld", "sitename": "Mesh", "workers": float64(4)}}, return func(layers []mcat.Layer) (string, error) {
{From: "this node", Values: map[string]any{"sitename": "This one"}}, file := map[string]any{"id": "env", "type": "file", "path": "/x/mail.env", "content": content}
err := settingInto(file, layers, "mail", "n", "b", "s", "l", "o")
return file["content"].(string), err
}
} }
if err := settingInto(file, layers, "mail"); err != nil { // declared is the ids of what is left of a resource under each condition, and whether the key is gone from it.
t.Fatal(err) declared := func(conditions ...any) func([]mcat.Layer) (string, error) {
return func(layers []mcat.Layer) (string, error) {
kept := withoutUnconfigured([]map[string]any{
{"id": "plain", "type": "file"}, {"id": "cond", "type": "file", mcat.IfConfigField: conditions}}, layers)
var ids []string
for _, r := range kept {
if _, has := r[mcat.IfConfigField]; has {
return "", errKeyKept
}
ids = append(ids, r["id"].(string))
}
return strings.Join(ids, ","), nil
}
} }
if file["content"] != "DOMAIN=example.tld\nSITENAME=This one\nWORKERS=4\n" { values := func(kv map[string]any) []mcat.Layer { return []mcat.Layer{{From: "the mesh", Values: kv}} }
t.Fatalf("filled as %q", file["content"]) for _, row := range []struct {
name string
layers []mcat.Layer
do func([]mcat.Layer) (string, error)
want string // what comes of it
errs []string // or what the refusal says
}{
{"the node's layer over the mesh's", []mcat.Layer{
{From: "the mesh", Values: map[string]any{"domain": "example.tld", "sitename": "Mesh", "workers": float64(4)}},
{From: "this node", Values: map[string]any{"sitename": "This one"}}},
filled("DOMAIN=${setting:domain}\nSITENAME=${setting:sitename}\nWORKERS=${setting:workers}\n"),
"DOMAIN=example.tld\nSITENAME=This one\nWORKERS=4\n", nil},
{"a value nothing set is refused, the file left as it was", values(map[string]any{"other": "x"}),
filled("DOMAIN=${setting:domain}\n"), "DOMAIN=${setting:domain}\n",
[]string{"${setting:domain}", "settings set mail", "set today: other"}},
{"config-lines: a list of two writes two lines in order", values(map[string]any{"k": []any{"a", "b"}}),
filled("h\n${config-lines:k:options zfs x=}\nt\n"), "h\noptions zfs x=a\noptions zfs x=b\nt\n", nil},
{"config-lines: mib-bytes writes a million MiB as a whole number", values(map[string]any{"k": float64(1000000)}),
filled("${config-lines:k:zfs_arc_max=:mib-bytes}\n"), "zfs_arc_max=1048576000000\n", nil},
{"config-lines: mib-bytes writes a million MiB given as text as a whole number", values(map[string]any{"k": "1000000"}),
filled("${config-lines:k:zfs_arc_max=:mib-bytes}\n"), "zfs_arc_max=1048576000000\n", nil},
{"config-lines: mib-bytes writes a size printed in exponent form as a whole number", values(map[string]any{"k": "1e+06"}),
filled("${config-lines:k:zfs_arc_max=:mib-bytes}\n"), "zfs_arc_max=1048576000000\n", nil},
{"config-lines: mib-bytes refuses a fraction", values(map[string]any{"k": "1.5"}),
filled("${config-lines:k:zfs_arc_max=:mib-bytes}\n"), "${config-lines:k:zfs_arc_max=:mib-bytes}\n", []string{"mib-bytes"}},
{"config-lines: an empty list writes nothing", values(map[string]any{"k": []any{}}),
filled("h\n${config-lines:k:p=}\nt\n"), "h\nt\n", nil},
{"config-lines: an unset value writes nothing", nil, filled("h\n${config-lines:k:p=}\nt\n"), "h\nt\n", nil},
{"config-lines: a set scalar writes one line", values(map[string]any{"k": "v"}),
filled("${config-lines:k:p=}\n"), "p=v\n", nil},
{"config-lines: mib-bytes writes MiB in bytes", values(map[string]any{"k": float64(64)}),
filled("${config-lines:k:zfs_arc_max=:mib-bytes}\n"), "zfs_arc_max=67108864\n", nil},
{"config-lines: an entry with a newline is refused, the file left as it was", values(map[string]any{"k": []any{"a\nb"}}),
filled("${config-lines:k:p=}\n"), "${config-lines:k:p=}\n", []string{"newline"}},
{"config-json: the set values only, each as its own type", values(map[string]any{"n": float64(64), "b": true, "s": "x",
"l": []any{"a"}, "o": map[string]any{"f": float64(1)}}), filled("${config-json}\n"),
"{\n \"b\": true,\n \"l\": [\n \"a\"\n ],\n \"n\": 64,\n \"o\": {\n \"f\": 1\n },\n \"s\": \"x\"\n}\n", nil},
{"config-json: a key the module does not declare is not written", values(map[string]any{"s": "x", "scrub-pool": "tank"}),
filled("${config-json}\n"), "{\n \"s\": \"x\"\n}\n", nil},
{"config-json: nothing set is an empty object, not a refusal", nil, filled("${config-json}\n"), "{}\n", nil},
{"if-config: a key never set leaves it out", nil, declared("k"), "plain", nil},
{"if-config: a key set to text declares it", values(map[string]any{"k": "x"}), declared("k"), "plain,cond", nil},
{"if-config: a key set to false, off, none, empty or an empty list leaves it out", values(map[string]any{"k": false}),
declared("k"), "plain", nil},
{"if-config: off", values(map[string]any{"k": "off"}), declared("k"), "plain", nil},
{"if-config: none", values(map[string]any{"k": "none"}), declared("k"), "plain", nil},
{"if-config: empty text", values(map[string]any{"k": ""}), declared("k"), "plain", nil},
{"if-config: an empty list", values(map[string]any{"k": []any{}}), declared("k"), "plain", nil},
{"if-config: a list of entries", values(map[string]any{"k": []any{"a"}}), declared("k"), "plain,cond", nil},
{"if-config: a value equal to the condition", values(map[string]any{"k": "on"}), declared("k=on"), "plain,cond", nil},
{"if-config: a value other than the condition", values(map[string]any{"k": "off"}), declared("k=on"), "plain", nil},
{"if-config: a list holding the entry", values(map[string]any{"k": []any{"a", "b"}}), declared("k=b"), "plain,cond", nil},
{"if-config: a list without the entry", values(map[string]any{"k": []any{"a"}}), declared("k=b"), "plain", nil},
{"if-config: any of the conditions", values(map[string]any{"j": "x"}), declared("k", "j"), "plain,cond", nil},
{"if-config: a value from a default layer", []mcat.Layer{{Default: true, Values: map[string]any{"k": "on"}}},
declared("k=on"), "plain,cond", nil},
{"if-config: a number equal to the condition", values(map[string]any{"k": float64(64)}), declared("k=64"), "plain,cond", nil},
} {
t.Run(row.name, func(t *testing.T) {
got, err := row.do(row.layers)
if len(row.errs) == 0 && err != nil {
t.Fatal(err)
}
if len(row.errs) > 0 && err == nil {
t.Fatalf("a refusal was expected, got %q", got)
}
for _, want := range row.errs {
if !strings.Contains(err.Error(), want) {
t.Fatalf("the refusal lacks %q: %v", want, err)
}
}
if got != row.want {
t.Fatalf("got %q, want %q", got, row.want)
}
})
} }
} }
func TestASettingNothingSetIsRefusedByName(t *testing.T) { // errKeyKept is what a row of the table above says when `if-config` reached the node-engine, which parses strictly.
file := map[string]any{"id": "env", "type": "file", "content": "DOMAIN=${setting:domain}\n"} var errKeyKept = errors.New("the if-config key was kept in a declared resource")
err := settingInto(file, []mcat.Layer{{From: "the mesh", Values: map[string]any{"other": "x"}}}, "mail")
if err == nil { // The composition takes a resource out while its condition does not hold, and a value going from set to unset takes it
t.Fatal("a setting nothing set was written as something") // out of the next composition, which the node-engine removes as it removes any resource that left (design 42 §2).
} func TestAConfigConditionedResourceIsComposedWhileItsValueHolds(t *testing.T) {
for _, want := range []string{"${setting:domain}", "settings set mail", "set today: other"} { m := mustParse(t, `{"module":"zfs","version":"1","config":{"zed":{"type":"string","default":"off"},"mods":{"type":"list"}},"resources":[
if !strings.Contains(err.Error(), want) { {"id":"opts","type":"file","path":"/etc/modprobe.d/zfs.conf","content":"${config-lines:mods:options zfs }\n${config-json}\n"},
t.Fatalf("the refusal lacks %q: %v", want, err) {"id":"conf","type":"file","path":"/etc/zfs/mesh.conf","content":"x\n"},
{"id":"zed","type":"file","path":"/etc/zfs/zed.rc","content":"y\n","if-config":["zed=on"]}]}`)
for _, row := range []struct {
layers []mcat.Layer
zed bool
}{{nil, false}, {[]mcat.Layer{{From: "this node", Values: map[string]any{"zed": "on", "mods": []any{"a=1"}}}}, true}} {
out, err := Declaration(mcat.Resolution{Node: "workstation", Modules: []mcat.Manifest{m}},
mcat.Rendering{Settings: mcat.SettingsBy{"zfs": row.layers}})
if err != nil {
t.Fatal(err)
}
if fileNamed(out, "zfs.conf") == nil {
t.Error("the unconditioned file was left out")
}
if opts := fileNamed(out, "zfs.opts"); opts == nil || (row.zed != strings.HasPrefix(fmt.Sprint(opts["content"]), "options zfs a=1\n")) {
t.Errorf("the template's lines do not follow the list: %v", opts)
}
zed := fileNamed(out, "zfs.zed")
if (zed != nil) != row.zed {
t.Errorf("with %v the conditioned file is declared: %v, want %v", row.layers, zed != nil, row.zed)
}
if _, has := zed[mcat.IfConfigField]; has {
t.Errorf("%s reached the node-engine, which parses strictly", mcat.IfConfigField)
} }
} }
// Left as it was: the literal placeholder must never reach a machine.
if file["content"] != "DOMAIN=${setting:domain}\n" {
t.Fatalf("content was changed on refusal: %q", file["content"])
}
} }
+1 -1
View File
@@ -458,7 +458,7 @@ func JudgeSettings(m mcat.Manifest, layers []mcat.Layer, adopted bool) error {
for k, v := range settled { for k, v := range settled {
copied[k] = v copied[k] = v
} }
if err := unsetInConfigWords(settingInto(copied, WithDefaults(m, layers), m.Module), m); err != nil { if err := unsetInConfigWords(settingInto(copied, WithDefaults(m, layers), m.Module, declaredConfig(m)...), m); err != nil {
return err return err
} }
} }
-78
View File
@@ -1,78 +0,0 @@
package resolution
import (
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// nfsServerVerbs is the contract every holder of node-nfs-server serves (novox/hq ADR 0263).
func nfsServerVerbs() []mcat.Verb {
return []mcat.Verb{
{Name: "exports", Description: "Every share this machine exports: its name, its path, read-write or " +
"read-only, the owner every client is mapped to (uid and gid), each node it is exported to with that " +
"node's private address and access (only the nodes the server grants it to and that ask for it), " +
"what is granted and not asked for or asked for and not granted, and whether the kernel holds it " +
"now. Also the exports found that are not the mesh's: a dataset's sharenfs property, a line in " +
"/etc/exports.",
Input: schema(map[string]string{}, nil),
Replaces: []string{"exportfs -v", "cat /etc/exports", "zfs get sharenfs"}},
{Name: "clients", Description: "Which machines have mounted which share now, as the NFS server " +
"knows its clients.",
Input: schema(map[string]string{}, nil),
Replaces: []string{"ss -tn sport = :2049", "cat /proc/fs/nfsd/clients/*/info"}},
{Name: "test", Description: "Whether this machine exports one share for the mesh now, and whether its " +
"NFS service is up; with an address, also whether the share is exported to that address: a node's " +
"own private address, when the server grants it the share and the node asks for it. What a machine " +
"mounting the share asks before it mounts.",
Input: schema(map[string]string{
"share": "the share, by its name",
"address": "a node's private address, to ask whether the share is exported to it (optional)",
}, []string{"share"}),
Replaces: []string{"showmount -e"}},
{Name: "reload", Description: "Have the kernel read this machine's export files again now (exportfs " +
"-ra) and answer the shares as it then holds them. The module's own process writes its export " +
"file; this is for after a change made outside it. Changes no shared path.",
Input: schema(map[string]string{}, nil),
Replaces: []string{"exportfs -ra"}},
{Name: "adopt", Description: "Take over an export a person made by hand: clear a dataset's sharenfs " +
"property for a path the mesh's own export now serves — only when that export is live. Without " +
"confirm, says what it would do and changes nothing.",
Input: schema(map[string]string{
"path": "the shared path whose hand-made export is taken over",
"confirm": "\"true\": change it (needs why); anything else is a dry run",
"why": "why, for the record",
}, []string{"path"}, "confirm"),
Replaces: []string{"zfs set sharenfs=off"}},
}
}
// mountsVerbs is the contract every holder of node-mounts serves (novox/hq ADR 0263).
func mountsVerbs() []mcat.Verb {
return []mcat.Verb{
{Name: "list", Description: "Every mount point on this machine: its fstab line, the mesh's mount and " +
"automount units for it, its state (armed, mounted, absent, unreachable, failed) and since when, " +
"and which settings asked for it. A password in a mount's options is never shown.",
Input: schema(map[string]string{}, nil),
Replaces: []string{"cat /etc/fstab", "findmnt", "systemctl list-units --type=mount"}},
{Name: "test", Description: "Whether one share's or occasional source's server answers from this " +
"machine now, without touching its mount point.",
Input: schema(map[string]string{"name": "the share or source, by its name"}, []string{"name"}),
Replaces: []string{"showmount -e", "ping"}},
{Name: "mount", Description: "Mount one share or occasional source now, rather than at its first " +
"access.",
Input: schema(map[string]string{"name": "the share or source, by its name"}, []string{"name"}),
Replaces: []string{"mount"}},
{Name: "unmount", Description: "Release one share or occasional source now; its automount stays " +
"armed, so the next access mounts it again.",
Input: schema(map[string]string{"name": "the share or source, by its name"}, []string{"name"}),
Replaces: []string{"umount"}},
{Name: "adopt", Description: "Take over a mount a person wrote by hand: comment out the /etc/fstab " +
"line for a mount point the mesh's own units now serve, keeping a copy of the file — only when " +
"the mesh's automount for it is armed. Without confirm, says what it would do and changes nothing.",
Input: schema(map[string]string{
"mountpoint": "the mount point whose fstab line is taken over",
"confirm": "\"true\": change it (needs why); anything else is a dry run",
"why": "why, for the record",
}, []string{"mountpoint"}, "confirm"),
Replaces: []string{"sed -i /etc/fstab"}},
}
}
@@ -0,0 +1,35 @@
package resolution
import (
"sort"
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// Resolve is ResolveHolding for a test that wants a claim problem as a refusal of the whole set, as before issue 1966.
func Resolve(catalogue map[string]mcat.Manifest, assigned []string, node mcat.Node, world mcat.World) (mcat.Resolution, error) {
got, held, err := ResolveHolding(catalogue, assigned, node, world)
if err != nil {
return mcat.Resolution{}, err
}
if len(held) > 0 {
var problems []string
for _, why := range held {
problems = append(problems, why)
}
sort.Strings(problems)
return mcat.Resolution{}, &Refusal{Problems: problems}
}
return got, nil
}
// checkClaims is holdClaims with the problems as a list.
func checkClaims(modules []mcat.Manifest, node mcat.Node, elsewhere []mcat.Held, holdings []mcat.Held) ([]mcat.Held, []string) {
held, why := holdClaims(modules, node, elsewhere, holdings)
var problems []string
for _, w := range why {
problems = append(problems, w)
}
sort.Strings(problems)
return held, problems
}
@@ -10,10 +10,11 @@ import (
// builderNode is a machine that can run a build module: the package manager and user namespaces it asks of // builderNode is a machine that can run a build module: the package manager and user namespaces it asks of
// the machine, and the forge account (mesh-scope, provided by the forge) and the artifact store the build // the machine, and the forge account (mesh-scope, provided by the forge) and the artifact store the build
// modules publish to (ADR 0513, provided by distribution) stood in for, and node-tools, the tool runner they pull their // modules publish to (ADR 0513, provided by distribution) stood in for, and node-tools, the tool runner they pull their
// asks through (mesh-catalog issue 1911; mesh-tools' module, not in the catalogue). // asks through (mesh-catalog issue 1911; mesh-tools' module, not in the catalogue), with the tool runner's
// seat-settle-delay the build modules ask of it since issue 1748 (ADR 0664 §5).
func builderNode() mcat.Node { func builderNode() mcat.Node {
return mcat.Node{Name: "builder", Site: "house", return mcat.Node{Name: "builder", Site: "house",
Capabilities: map[string]bool{"package-manager": true, "user-namespaces": true}} Capabilities: map[string]bool{"package-manager": true, "user-namespaces": true, "seat-settle-delay": true}}
} }
func toolchainShelf(t *testing.T) map[string]mcat.Manifest { func toolchainShelf(t *testing.T) map[string]mcat.Manifest {
@@ -25,6 +25,10 @@ type filler struct {
// or the sweep refuses the placeholder it would have filled. // or the sweep refuses the placeholder it would have filled.
var fillers = []filler{ var fillers = []filler{
{namespace: "setting", patterns: []*regexp.Regexp{settingRef}, fields: map[string][]string{"file": {"content"}, "service": {"unit"}, "directory": {"owner"}, "container": {"groups"}}}, {namespace: "setting", patterns: []*regexp.Regexp{settingRef}, fields: map[string][]string{"file": {"content"}, "service": {"unit"}, "directory": {"owner"}, "container": {"groups"}}},
// A file's content only: lines from a config value's entries, and the module's config as one JSON object
// (design 42 zfs-owns-zfs §3, §9; config_lines.go).
{namespace: "config-lines", patterns: []*regexp.Regexp{configLinesRef}, fields: map[string][]string{"file": {"content"}}},
{namespace: "config-json", patterns: []*regexp.Regexp{configJSONRef}, fields: map[string][]string{"file": {"content"}}},
{namespace: "dir", patterns: []*regexp.Regexp{dirRef}, fields: map[string][]string{"*": {"path", "content", "volumes", "env", "env-file"}}}, {namespace: "dir", patterns: []*regexp.Regexp{dirRef}, fields: map[string][]string{"*": {"path", "content", "volumes", "env", "env-file"}}},
{namespace: "access", patterns: []*regexp.Regexp{accessRef}, fields: map[string][]string{"*": {"path", "content", "volumes", "env", "env-file"}}}, {namespace: "access", patterns: []*regexp.Regexp{accessRef}, fields: map[string][]string{"*": {"path", "content", "volumes", "env", "env-file"}}},
{namespace: "secret", patterns: []*regexp.Regexp{placeholder}, fields: map[string][]string{"file": {"content"}}, {namespace: "secret", patterns: []*regexp.Regexp{placeholder}, fields: map[string][]string{"file": {"content"}},

Some files were not shown because too many files have changed in this diff Show More