Author SHA1 Message Date
jschoubben 5caa760f5e A-342 engine-node-env: every process runs with its node's name as MESH_NODE
mesh/delivery merged: source.main moved to novox/mesh-host@fd023f7473bc
mesh/delivery-group group feat/47-G-join12 merged: every member is merged
mesh/check failed (2): mesh-delivery-checker: no published of the tagged version of go-meshconditions of me…; and 1 more | 5caa760f5eaa on fd023f7473bc
The apply gives each process the declaration's For as MESH_NODE beside its own env, so a process that reads MESH_NODE (mesh-sdk, the zfs follower's writer and its node-service-manager calls) names its node.

Builds: design 47 G A-342

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 09:20:46 +01:00
jschoubben d63aae1d55 Merge main into feat/47-G-join11: the node-engine's if-config, config-lines, config-json and stop at a failure (issue 1979)
The whole test's web manifest keeps G's entry form (no build artifact, the entry read by fingerprint) and takes main's resources under if-config and config-lines.

Builds: design 47 G §421; design 42 zfs-owns-zfs H1-H5

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 09:10:44 +01:00
mesh-tools f8acfaeed7 Merge pull request 'The node-engine stops a module's apply at its first failed resource, and reads if-config, config-lines and config-json (issue 1979)' (#227) from feature/1979-zfs-owns-zfs into main 2026-11-13 08:00:43 +00:00
jschoubben 0a9d1efe89 Merge main into feat/47-G-join11 (G takes in main before its push)
mesh/delivery-group group feat/47-G-join11 checking: 17 of 17 member(s) ready
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 0a9d1efe8918 on f8acfaeed79d | novox/mesh-host#226 in group feat/47-G-join11
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 02:16:56 +01:00
jschoubben 96a8690639 Fix 1979: mib-bytes writes the bytes as a whole number for every size
mesh/delivery merged: source.main moved to novox/mesh-host@f8acfaeed79d
mesh/delivery-group group feature/1979-zfs-owns-zfs merged: every member is merged
mesh/check being carried to 96a869063977 on f8acfaeed79d | from 0193949efe9f, mesh-catalog 117f933dacf0, mesh-delivery 7221a1e5cd7f, mesh-sdk e42b9491…
A size of a million MiB or more reached the file in exponent form and was refused.

Builds: design 42 zfs-owns-zfs §3, A-24
2026-11-13 01:32:36 +01:00
jschoubben 7868c9f365 Fix 1979: ${config-json} writes only the keys the module declares under config
A key a layer holds that the manifest does not declare (scrub-pool during a rename) is not a config value of the module.

Builds: design 42 zfs-owns-zfs §9, A-15, A-24, A-25
2026-11-13 01:32:27 +01:00
jschoubben 225f2abb1f H5 engine-integration: the node-engine whole applies if-config, config-lines and a module's stop at a failure
The existing node-engine integration test's module gains a file under if-config,
a template with config-lines from a list, and two files that fail and are left
under a value; the test asserts the file appears and goes with its value, the
lines follow the list, and the failing resource fails the module with the later
resource left. No integration test is added.

Builds: design 42 zfs-owns-zfs §5 (and §1 to §3), A-22
2026-11-13 00:24:20 +01:00
jschoubben 61806a7a92 H4 engine-states: the node record states the new words and forms
The node record's reads come from mesh-sdk's ManifestReads, so building with S1
and S2 states word:resources.if-config, form:config-lines and form:config-json;
the record test now asserts them.

Builds: design 42 zfs-owns-zfs §4, answer A-13
2026-11-13 00:22:48 +01:00
jschoubben 61c42f9fbc H3 engine-config-lines, H3b engine-config-json: a template writes lines from a config value and the config as JSON
`${config-lines:<key>:<prefix>[:mib-bytes]}` in a file's content writes one line
per entry (nothing for an unset value or empty list; a newline in an entry
refuses the composition); `${config-json}` writes the module's set config values
as one JSON object. Both join the sweep's table of fillers. Their rows join the
config table test (A-22).

Builds: design 42 zfs-owns-zfs §3, §9; answers A-15, A-24, A-22
2026-11-13 00:22:22 +01:00
jschoubben 86048f0ed9 H2b engine-found-handed: the found state of a unit passes to the record that stays
When one record of a unit goes while another still declares it, what the going
record found of the unit is kept for the staying one when that has no finding of
its own, so the module's leave gives back the state found before the mesh. The
preview test and the new case are one test (A-22).

Builds: design 42 zfs-owns-zfs §2, answers A-21, A-22
2026-11-13 00:20:59 +01:00
jschoubben 039415cdd9 H2 engine-if-config: a resource is declared only while a config value holds
The composer leaves out each resource whose `if-config` holds none of its
conditions against the module's config values (with defaults), and drops the key
from the rest, as the node-engine parses strictly. A resource that leaves is
removed by the existing orphan removal. One table test replaces the two tests of
setting_into_test.go.

Builds: design 42 zfs-owns-zfs §2, A-21 (declaration side), A-22
2026-11-13 00:19:49 +01:00
jschoubben fb492da7e7 H1 engine-stop-at-failure: a module's apply stops at its first failed resource
A failed resource of a module gates the rest of that module, which is reported
skipped with the failed resource named; the module's result names each one left.
A resource of no module, and other modules, go on as before. One table test
replaces two in apply_test.go; two fixtures adapted to the rule.

Builds: design 42 zfs-owns-zfs §1 (ADR 0679 §16 (a)), A-22
2026-11-13 00:19:49 +01:00
jschoubben a12417fc13 Remove the real sleep from the recovery integration test
mesh/delivery-group group feat/47-G-join11 checking: 7 of 7 member(s) ready
mesh/delivery closed: a newer head of the same pull request
mesh/check checking a12417fc13c7 on 0193949efe9f | novox/mesh-host#226 in group feat/47-G-join11
Builds: design 47 G (playbook 07 step 2: no real sleeping in a test)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 23:44:02 +01:00
jschoubben 8656e57eb9 Test 36: the recovery wiring is driven through the node agent by a revert want of db
Builds: design 47 G §296, §297, §554, §560, A-113, A-320

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 23:39:39 +01:00
jschoubben 650910dc9e R3-3: the node's own composer has no path without an entry call; its tests compose from the catalog half
Builds: design 47 G §628, §421, A-312, A-317

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 23:33:31 +01:00
jschoubben 321a0744d8 H9 test: entry-again takes a shadow's not-found entry in
Builds: design 47 G §698; lead answer A-319

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 23:13:57 +01:00
jschoubben 860ccd7e30 H9: an entry asked again by hand is asked again in the shadow's cache too, so a candidate waits no longer on a not-found it holds
Builds: design 47 G §698 (entry-again), §421 (A-308: the shadow reads the entries its wants name)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 23:12:41 +01:00
jschoubben 23799a30ed R3-1, R3-2, R3-3: the node-engine reads no build of the builder; the shadow builds from the entry it holds
The composer's Build is made from the catalog half's entry it holds and the parts of the want that names it (buildOf); the
inputs of the indexer's modules and the builder's builds, takeBuildDone, takeModule and the builder read are gone, and mesh-host
names no BuilderBuildsBucket. The shadow takes the entries of the builds it composes from (askBuildsEntries), builds a module a
candidate adds from its entry and the candidate's `added` parts (A-316), and holds, with the reason, a module whose entry
has no part given, whose fingerprint the verb module does not give, or for which it errors (A-312). Its composer is given the
entry call as the node's own is. Rows: a built module added with and without parts, the verb module answering no fingerprint
and erroring.

Builds: design 47 G §421, §628, A-308, A-312, A-316
2026-11-12 23:05:06 +01:00
jschoubben e558510a54 H5 (review r2, A-308, A-312): the shadow watches no indexer modules; it reads the entries its wants name and asks the verb module for a module a candidate adds
Builds: design 47 G §421, §628
2026-11-12 22:28:15 +01:00
jschoubben 1d409cd852 H3 (review r2): the entries test waits on the refusal the composer says until the ghost's reason is entry-missing
Builds: design 47 G §421, §614 (test only)
2026-11-12 22:24:38 +01:00
jschoubben 302dcc098f H4 (review r2, A-307): the node-engine's manifest names no uses; its invoke of seat:mesh-grants.withheld is the edge
Builds: design 47 G §635
2026-11-12 22:16:40 +01:00
jschoubben 3167f0104f H1 (review r2): the foundation host is the entry that claims the seat node-engine, not a module named mesh-host
Builds: design 47 G §527, §634
2026-11-12 22:16:27 +01:00
mesh-tools 0193949efe Merge pull request 'The own resolver's test reads the options from the catalog's template (issue 1748, ADR 0678)' (#225) from fix/1748-resolver-wait into main 2026-11-12 17:52:54 +00:00
jschoubben 8dc186944e fix-1748-host-test: the own resolver's test reads the options from the catalogue's template
mesh/delivery-group group fix/1748-resolver-wait merged: every member is merged
mesh/delivery merged: source.main moved to novox/mesh-host@0193949efe9f
mesh/check checking 8dc186944e7d on 0193949efe9f | novox/mesh-host#225 in group fix/1748-resolver-wait
TestTheCataloguesResolverComposesOnAMachine takes the options line from the systemd-resolved template beside it instead of the literal timeout:1 attempts:2 edns0, so it passes against the catalogue's main before and after the catalogue's change. Captured copies are left as they are.

Builds: design 50 resolver-wait §3 (and §2 of the main file, line 73)
2026-11-12 18:10:28 +01:00
jschoubben 905265a56b J25 A-293: a placed module's failed newest build said as refused in by-entries mode, on the builder's build-ended event
Reading the catalog half by fingerprint, the node-engine no longer reads the builder's builds, so #224's refused key was
never written there and a reader waited on a build that will never be done. The engine now hears the builder's
build.ended stream, read whole at start then followed (as the assigner hears it); the last trunk end of a placed module
that ended failed is written as the refused key with the event's words, a later done end deletes it, an end naming a
branch moves nothing. The manifest consumes mesh-delivery-builder.build.ended.**.

Builds: design 47 G A-293 (with §421, §628), ADR 0593 §6, issue 1966 U1/U8

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 17:20:30 +01:00
jschoubben 4ef07521eb Merge origin/main into the join: a placed module's failed build said as refused
Brings mesh-host #224 (issue 1966): newestFPs returns a placed module whose
newest build failed, composedValues writes its refused key at that fingerprint,
takeBuildDone starts the pass on a failed build. Conflict in newestFPs settled
by keeping G's by-entries branch first (design 47 G §421, §628: the node reads
neither the indexer's modules nor the builder's builds; it returns no failed),
and main's failed-build path in the other.

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 17:00:32 +01:00
jschoubben a9b9709911 J23 A-292: a node seat event the bus did not take is said again on the same link
After a restart, g14's engine restated its reads three seconds after its
first, bootstrapped statement, and the assigner never heard that event: a
say that failed past the bus's retry bound waited for the next link. Now a
say the bus did not take (no responders, a deadline) is said again on the
link until the bus takes it, off the caller, each failed attempt written to
the engine's journal with the event's name; only a lost link, or a refusal
the bus answered, keeps it for the next link.

Builds: design 47 G §590, A-292
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 16:57:36 +01:00
mesh-tools df1412081b Merge pull request 'The node-engine says a placed module's failed build as refused, so the provisioner stops waiting on it (issue 1966)' (#224) from fix/1966-unpublished-placed-module into main 2026-11-12 15:34:23 +00:00
jschoubben 8e993c759a H1 refused-at-failed-build: a placed module whose newest build failed is published as a refusal
mesh/delivery merged: source.main moved to novox/mesh-host@df1412081bed
mesh/delivery-group group fix/1966-unpublished-placed-module merged: every member is merged
mesh/check checking 8e993c759a5c on df1412081bed | novox/mesh-host#224 in group fix/1966-unpublished-placed-module
The node-engine writes a refused key, with the build's own error, at the indexer's
fingerprint in composed-<node> for a placed module whose newest build the builder
holds failed; a build queued or running stays unpublished (U1). A failed build of a
placed module's newest fingerprint also moves the pass (takeBuildDone), so the key is
written when it lands (U8).

Builds: design 47 fix-unpublished-placed-module U1, U8, ADR 0593 §6
2026-11-12 16:01:24 +01:00
jschoubben 0c37ac378c Join 11: the node seat's discovery test names the engine's own module, not a fixed one
H-17 made discovery read the node-engine's own name (§527) instead of the constant mesh-host,
but the integration test still gave its link no own name and expected mesh-host, so every
endpoint announced an empty module. It now gives the link an own name and expects that one.

Builds: design 47 G §634, A-149

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 14:13:34 +01:00
jschoubben d18bf8b3fd The node-engine's seat reader reads invokes in both forms and refuses a form it cannot read
Builds: design 47 G A-275, §558

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 11:07:38 +01:00
jschoubben bef467febe H-revert: the node agent starts a held revert on the live data when its want says revert live
A want with `revert` `live` starts the reverted build with no restore, no aside taken and no catch-up, and
applies as any want; without the field a revert with no restore point stays held.

Builds: design 47 G §710 (G-21-provider-and-revert-ask.md, "On the answer"), §300, §567

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 09:42:32 +01:00
jschoubben a1a328c43e Merge commit 'a3fea910' into feat/47-G-fix11-hrevert 2026-11-12 09:39:24 +01:00
jschoubben 77e6e48ecd AM-H1: the node-engine's composition names the withheld subjects, read through mesh-grants' withheld
The node-engine never reads the grants part's state: it asks the seat's verb
withheld in its own name, and says withheld unknown when nothing answers.

Builds: design 46 7 §6 (the composition reads them through the verb withheld; the node-engine's manifest gains mesh-grants in uses; "withheld unknown: the grants part does not answer"), lead answer A-239

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:26:10 +01:00
jschoubben 8734d5ee1a AM-D3 path: the node-engine takes the runner's reading and fingerprint into the node's record
The grants part decides a node's withheld subjects from the runner's own
reading tied to the runner that stated it; the merge into capabilities stays
for placement.

Builds: design 46 7 §6 (its node's node-engine takes both into the node's record), lead answer A-203 (runner-capabilities, runner-fingerprint)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:06:16 +01:00
jschoubben a3fea9106a H-find: the findings keep the keys the watchman raised them under
Each finding's id carries its token as a ref (A-14), so machine.<node>.network, .rewritten, .unreachable, .units and module.<module>.<node>.unhealthy and .kept
read as before.

Builds: design 47 G A-197

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:02:07 +01:00
jschoubben 4b215ba6ae H-find: the node-engine raises what only it knows of itself as its own findings
The have's failed phase (apply-failing), unhealthy health (module-unhealthy) and kept words (kept-as-it-runs), and each part of
its health report (machine-network, network-rewritten, network-unreachable, machine-units, relogin-needed,
directory-used-as-found, node-account-fallback), raised and cleared by the held writer where the engine knows the fact.

Builds: design 47 G A-197 (ADR 0596, A-192, A-149)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:00:42 +01:00
jschoubben 4021de1d42 HRUN: mesh-compose reads its pipeline input from MESH_RUN_INPUT
The composer answers the assigner's composition step: the request is read from the file
named by MESH_RUN_INPUT and the verdict is the last line of standard output, exit 0; the
compose-job path stays.

Builds: design 47 G §354, §63, §22; lead answer A-167

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:44:53 +01:00
jschoubben 9afb287b9d Merge branch 'feat/47-G-fix11-a1' into feat/47-G-fix11-hrun 2026-11-12 05:43:50 +01:00
jschoubben c2710f52dd MV1: move contracts and sdk under modules/, importers follow
The delivery repository's contracts and SDK live under modules/ as
mesh-delivery-contracts and mesh-delivery-sdk, their Go module paths
renamed to match the folders (the TypeScript names already did), and
every importer in mesh-delivery, mesh-catalog, mesh-host, mesh-tools
and mesh-sdk follows in the same group: a design as it should be, one
path per module, checked by the group's check building every importer.

Builds: design 47 G §427, §88

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:09:56 +01:00
jschoubben a3b646b3c9 H-24: the node-engine's hold tests are one test of cases, below main's count
1,726 test functions against main's 1,751.

Builds: design 47 G §419
2026-11-12 05:01:00 +01:00
jschoubben 732def66b0 H-21: the manifest invokes revision (none-yet) and withheld, and config view-waits-minutes
The long-form invokes entry is read by mesh-sdk's second reader release (09d6070e): packaging's manifest
tests pass with it placed and fail before it is joined.

Builds: design 47 G §558, §617, §633, §635
2026-11-12 05:00:14 +01:00
jschoubben ac3422f914 H-17: the node-engine's own name is read from its applied record or first composing
Discovery, the finding writer and the vault login names read it; no fixed name.

Builds: design 47 G §527, §634 (A-149)
2026-11-12 04:58:28 +01:00
jschoubben 4d656fe4e1 H-20: a tools bundle's words hold its module's want fingerprint
Builds: design 47 G §519, §618
2026-11-12 04:55:33 +01:00
jschoubben 94f6c3dba4 H-18: running carries healthy and is said again when it changes
Builds: design 47 G §520
2026-11-12 04:54:19 +01:00
jschoubben 7d06ef7997 Join 10: the composer command may import the catalog key codec where A-144 moved it
The codec moved from the mesh-delivery contracts into mesh-delivery's domain SDK (sdk/catalogkey), still a pure package
of the standard library and mesh-sdk; the command's import rule named the old place only.

Builds: design 47 G A-144

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 02:51:10 +01:00
jschoubben efee30e243 Fix 10: the compositions read the catalog key codec from mesh-delivery's sdk
Builds: design 47 G §614, §670 (ADR 0625 §4; lead answer A-144)
2026-11-12 02:26:55 +01:00
jschoubben e8bff57b55 Join 9: the node-engine's link names no seat list verb in its operatorask contract (A-126)
Builds: design 47 G A-126

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:51:54 +01:00
jschoubben c6df8fa20f G §683: a start composes the modules whose kept want and entry it holds before its other inputs are read
The node's own record comes from the node-engine's statement of itself, taken in memory with no bus.

Builds: design 47 G A-117, A-120, §683, A-95
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:12:06 +01:00
jschoubben 054ca5a6b3 G §614: the node-engine writes the finding entry-not-found for a fingerprint the catalog half holds none of
Builds: design 47 G A-117, §614, A-15, A-95
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:07:53 +01:00
jschoubben b3b13f2ae1 G §560: a cancelled recovery act holds the move with the hand act's why
Builds: design 47 G A-117, §560
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:06:39 +01:00
jschoubben 0f108fd8ac internal/apply: a test of the places a module is placed in follows no link on the machine it runs on
Builds: design 47 G A-95 (the owed hang of TestAModulesOwnPlacesAreStillPlaced on real mounts)
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:03:54 +01:00
jschoubben 0dd32d9722 A-101: the node-engine reads the router's record through mesh-sdk's operatorask, the bucket from the seat list
Builds: design 47 G A-101, §700, A-74
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:03:22 +01:00
jschoubben 31697e696d A-113: a revert whose restore fails starts the newer build again; a form change not known waits; holds say wait-held
Builds: design 47 G A-113, §297, §560, §541, §629
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:01:44 +01:00
jschoubben 6441ae7e96 gofmt cmd/mesh-host/keptartifact_test.go
Builds: design 47 G A-95 (the owed gofmt)
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:01:44 +01:00
jschoubben 5694f6c39b Merge branch 'feat/47-G-recover-host' into feat/47-G-join8
# Conflicts:
#	internal/selfcompose/entries.go
#	module.json
2026-11-12 00:27:57 +01:00
jschoubben ece79788a3 A-112: the node-engine reads recovery-wait-minutes from a file its manifest declares, 90 when missing
The manifest declares the file recovery-wait (content ${config:recovery-wait-minutes}); the node-engine reads it at its start and
does not call stdio.Config, which needs the runtime that launches a bundle.

Builds: design 47 G §633, A-112
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 00:21:10 +01:00
jschoubben dc1aee51d9 W13.5: the node agent asks mesh-recovery for a restore point before a form change and restores it at a revert
Before a move across a kept data's form change the node agent asks snapshot-kept and waits for the ended of its own act; at a revert it stops the newer build (Env.Stop), asks kept-points, asks restore-kept with the snapshot of the point of the form and fingerprint it reverts from, waits for restored, applies the reverted build, and calls the point's catch-up as the module's tool through the tool runner. The wait is in the have (ask-at, act), bounded by a bounds key of the state bounds (recovery-wait-minutes, 90), and each ending is read as the table of 560 says. The manifest consumes mesh-recovery's ended and restored and invokes its three verbs.

Builds: design 47 G §296, §297, §554, §555, §560, §630, §631, §632, §633, A-77, A-92, A-98, A-108
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 00:16:31 +01:00
jschoubben cf378f9a25 G §421, §683: the node-engine reads the entry its want names by fingerprint and keeps its entries and wants
The node's own composer reads the catalog half through meshcatalogclient's Entries cache over its own calls of
mesh-catalog's verb entry, and no longer watches or reads the indexer's modules, manifests and verdicts or the
builder's builds: a module's manifest is its entry's, each artifact's reference is made from the digest its want's
part names (artifact-store://<module>/<artifact>/blobs/<digest>, A-50). A want whose entry is asked or not found is
held, its have the phase held with a detail opening entry-asked: or entry-missing: naming the fingerprint. The
entries and wants it composes from are kept in composed-<node> (entry.<digits> and want.<module>), written before
the composition that uses them, read at start before anything is asked, a kept want standing until the wants'
live read answers, and dropped when nothing names them. The shadow keeps its reads (G §628).

The node grant gains the verb entry and the running stream's consumer calls; OverNATS hears the running stream
as the link's session does.

Builds: design 47 G §421, §683, §614, §628, A-47, A-50
2026-11-11 23:55:50 +01:00
jschoubben 2b916ac043 G A-98: the node agent gains Env.Stop, which stops a module's units and containers and removes nothing
Builds: design 47 G §297, A-98

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-11 23:51:09 +01:00
jschoubben 7c7b5e2d68 G A-92: the node-engine's manifest declares the state bounds, a lifetime per key, in its old section
Builds: design 47 G §560, §557, §627, A-77, A-92
(the seat definition: mesh-contracts 2ae0c679304b828a3973dd23ecd19861577d141d)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-11 23:48:16 +01:00
jschoubben 5173f2a31c W13.8: the node-engine fetches a kept build through the build node's kept-artifact, the node taken from the want's kept-on
A want that names a node in kept-on, for an artifact its parts name, is fetched through kept-artifact of that node's build seat and
read as an object on the bus, checked against the digest; with no kept-on it is fetched from the store, and the builder's builds
are read no more (the apply hook, the fetch and the object read from bc8fbc7, its node and fingerprint from the want).

Builds: design 47 G §271, §526, §421 (A-50)
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-11 23:39:36 +01:00
jschoubben 97d3df3e59 G §422, §698: the node-engine serves first-views, first-view-again and entry-again, and its manifest names the verbs it serves
first-views answers {first-views, entries, wants}, entry-again asks one entry again as a recorded hand act and
answers {asked, read}; first-view-again names no view the engine follows (it follows no branch) and refuses in
words. module.json's claim of node-engine lists every verb the engine serves.

Builds: design 47 G §422, §698
2026-11-11 23:27:28 +01:00
jschoubben 7703162c95 G §421, §683: the node-engine reads the entry its want names by fingerprint and keeps its entries and wants
The node's own composer reads the catalog half through meshcatalogclient's Entries cache over its own calls of
mesh-catalog's verb entry, and no longer watches or reads the indexer's modules, manifests and verdicts or the
builder's builds: a module's manifest is its entry's, each artifact's reference is made from the digest its want's
part names (artifact-store://<module>/<artifact>/blobs/<digest>, A-50). A want whose entry is asked or not found is
held, its have the phase held with a detail opening entry-asked: or entry-missing: naming the fingerprint. The
entries and wants it composes from are kept in composed-<node> (entry.<digits> and want.<module>), written before
the composition that uses them, read at start before anything is asked, a kept want standing until the wants'
live read answers, and dropped when nothing names them. The shadow keeps its reads (G §628).

The node grant gains the verb entry and the running stream's consumer calls; OverNATS hears the running stream
as the link's session does.

Builds: design 47 G §421, §683, §614, §628, A-47, A-50
2026-11-11 23:27:28 +01:00
jschoubben 38e13ef030 H-13: the node-engine's seat-list and held-module findings go through mesh-sdk's held writer
The seat list's five-minute finding and the finding of a held module are raised as they happen, so they use the held
writer (seeded at Start, Held() answers what stands) over the contract of the seat mesh-conditions; the keepers keep no
set or sweep of their own, and the writer is started before the seat list reader writes or clears any. The engine's own
finding stream stays its foundation stream mesh-host-findings.

Builds: design 47 G §529, A-29, A-30, A-37, A-38, A-40
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-11 22:37:21 +01:00
jschoubben 189ed10c36 The node-engine's compositions use the catalog key's codec meshcatalogkey, as meshcatalogclient's wrappers go
Builds: design 47 G §670

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:18:40 +01:00
jschoubben 75f5c64646 The node-engine says no capabilities-changed: manifest-reads-changed alone says a node's capabilities, as the contracts' K6 has the seat
The join with mesh-contracts' K6 (7d730a4a) removes the event and its body; H-3's second source of the same fact goes (haveState.capsSaid, capabilitiesToSay, sayCapabilities), the test asserts none is said.

Builds: design 47 G §590

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:13:24 +01:00
jschoubben 5d3b446988 H-8: a want's parts are checked by the digests the composition fetches
A want with parts is applied only when every fetched archive, process and container
digest is among the parts and every part is a fetched digest; otherwise the module
keeps running what it has and its have is held with detail parts-differ naming the
digests. A want with no parts is applied as before.

Builds: design 47 G §524, §379

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 15:47:04 +01:00
jschoubben ce01ec1f68 H-13: the node-engine's findings are written and said by mesh-sdk's finding writer
A finding is written in the node's state findings and followed by finding-written or finding-cleared on the engine's own
stream mesh-host-findings; the engine states every finding it observes at its start (the seat list's finding only while the
reader waits for the list, else not) and when the keeper asks, and answers an ask by a new statement. The node's grant in
the golden file gains the stream's publish, the ask stream's consumer, and the whole of its state findings.

Builds: design 47 G §383, §384, §390, §461, §485, §503

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 14:15:16 +01:00
jschoubben 602ed830e9 H-15: the apply guard and the yields read the node-engine's module from the entry claiming its seat
The module is the one whose catalog entry claims node-engine, else the one this node heard run the build its own
fingerprint names; no name is fixed in the guard or in the compose.

Builds: design 47 G §364, §527, §428

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 14:05:43 +01:00
jschoubben a073a001c5 H-7: the node record states the capability want-parts
Builds: design 47 G §525, §94, §192

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 14:00:21 +01:00
jschoubben 45e2b946c7 H-5: the seats a node passes on are decided from what every node said it runs, by SeatsToPassOn's new input
The node follows the node-engine seat's running events (an ordered read of the last message of each subject, then a durable
consumer), keeps what each node runs by `at`, acts on none until the first read ended, and hands the composer each node's
heard build with the manifest read by the event's digest; compose resolves the passes through catalogue.SeatsToPassOn
(heard, entries) with the trunk's entry for a node that said nothing.

Builds: design 47 G §199, §249, §459, §536, §556

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 13:58:33 +01:00
jschoubben b41a0d59b6 H-14: a module held for its claim is the finding module-held until it is composed or no longer wanted
Builds: design 47 main file (A claim problem holds its module), test 55

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 12:12:55 +01:00
jschoubben f3813a279e H-6: the seat verb process-builds reads the build each process was started with; the engine keeps no callers table
Builds: design 47 G §262, §245, §440, §447

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 12:09:56 +01:00
jschoubben 420087ab14 H-4: running says stopped, with no fingerprint, once the have no longer holds the module
Builds: design 47 G §365, §404

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 12:06:10 +01:00
jschoubben 26581d87ed H-3: the node-engine says composed, running and capabilities-changed on the seat's subjects
Builds: design 47 G §260, §227, §226, §177, §309, §404

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 12:05:18 +01:00
jschoubben 730860b274 H-2: every service and container is started with MESH_FINGERPRINT, the fingerprint of the want it applies
Builds: design 47 G §201, §205, §239

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 12:03:47 +01:00
jschoubben decbce4c54 H-1: the node-engine claims its seat, says its findings on its own stream and hears running
Builds: design 47 G §364, §387, §199, §368, §411, §438

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 12:03:13 +01:00
mesh-tools 14c22b4a45 Merge pull request 'The seat-list finding is read from the node's own state, so a restart clears it (issue 1966)' (#223) from fix/1966-seat-list-finding-from-the-bus into main 2026-11-11 01:34:11 +00:00
jschoubben e64c8ad0ca The node's golden grant reads its own findings, as the grants part composes it (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-host@14c22b4a4586
mesh/check checking e64c8ad0ca62 on 14c22b4a4586
mesh-grants gives the engine the owner's read of its state findings for its own node's keys (EngineFindingsGrants); the golden
grant held only the write, so the engine could not read the finding it clears at the first full read.

Builds: design 47 H-7, ADR 0636 §1 (the finding is cleared at the first full read), ADR 0596 §2 (the owner reads its state), issue 1966

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 02:23:10 +01:00
jschoubben 6304332ad5 The seat-list finding is read from the node's own state, not kept in memory (issue 1966)
A new node-engine process cleared nothing at its first full read of the seat list, because the Waiter remembered in memory whether
it had written the finding; ace and g14 kept the condition for ever. The Waiter now asks the node's state findings whether the
finding stands, clears it at the first full read and writes it at the bound only when it does not stand.

Builds: design 47 H-7, ADR 0636 §1 (the finding is cleared at the first full read), issue 1966

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 02:20:42 +01:00
mesh-tools 783a2d2207 Merge pull request 'The node-engine says its events once through a transport fault, and an unsaid one at the next link (ADR 0668; issue 1966; group fix/1966-call-made-once)' (#222) from fix/1966-call-made-once into main 2026-11-10 07:20:32 +00:00
jschoubben d581acdeda H4: the builder machine of the toolchain tests has the tool runner's seat-settle-delay (issue 1966; mesh-catalog's issue 1748, ADR 0664 §5)
mesh/delivery merged: source.main moved to novox/mesh-host@783a2d22073f
mesh/delivery-group group fix/1966-call-made-once merged: every member is merged
mesh/check checking d581acdedae3 on 783a2d22073f | novox/mesh-host#222 in group fix/1966-call-made-once
Since mesh-catalog 6b8e7fc6 the build modules ask the capability seat-settle-delay of the machine, so on
the stand-in builder without it mesh-build-go and mesh-build-node were left out as unhostable and both
toolchain tests failed against the catalogue's main, on mesh-host's trunk as on this group's head.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 05:57:04 +01:00
jschoubben 9cd03a7430 H3: the row of sayOnce feeds the old JetStream client's own no responders, nats.ErrNoStreamResponse (issue 1966, ADR 0668 §1; review round 1, finding 2)
mesh/delivery closed: a newer head of the same pull request
mesh/delivery-group group fix/1966-call-made-once rejected: composed together: the checker's verdict at the members' heads (c54b617d200d): mesh-delivery-checker
mesh/check failed: mesh-delivery-checker: no published of the tagged version of go-meshconditions of mesh-health-and-re… | 9cd03a7430cc on f50442ac7a2d
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 04:12:04 +01:00
jschoubben 5aa6d39a98 H2: an engine event not said is kept and said at the next link's opening (issue 1966, ADR 0668 §4)
A failed say was only a line in the engine's log; now it is said again when a link opens, beside the
statement made while no link was open (proved by a step of the node key test, with H1's publish).

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 02:57:34 +01:00
jschoubben 4685610c19 H1: the node-engine publishes its seat's events again through a transport fault, under one message id (issue 1966, ADR 0668)
One publish of manifest-reads-changed with a 10 s deadline was the likely loss behind note 155; the
stream takes the id once, so publishing again is safe.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 02:57:34 +01:00
mesh-tools f50442ac7a Merge pull request 'An unread seat list holds what is new, says so, and is read again by the operator's sessions (issue 1966; group fix/1966-seat-list-unread)' (#221) from fix/1966-seat-list-unread into main 2026-11-10 00:44:44 +00:00
jschoubben 5c353cfc7b S3: the read seat-list answers the operator's sessions and the healers, as its seat says
mesh/delivery-group group fix/1966-seat-list-unread merged: every member is merged
mesh/delivery merged: source.main moved to novox/mesh-host@f50442ac7a2d
mesh/check checking 5c353cfc7bf0 on f50442ac7a2d | novox/mesh-host#221 in group fix/1966-seat-list-unread
The node-engine seat's definition names "the operator and the healers" for
both seat-list verbs, and design 47 G keeps it; the read was opened to every
proved caller on a ruling the review withdrew. node-tools stays on both: every
console call is proved as its machine's tool runner (issue 627, issue 1966).

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 01:16:05 +01:00
jschoubben a484d5f960 S5: the whole node reads its seat list after a holder that re-served
The live case of ace, in process against a real bus: the stand-in of the seat
mesh-catalog answers as nothing serves it (no responders) while the node's
reader first asks, then serves; the same reader reads the list within the
bus's retry bound, with no event of the seat and no hand act (issue 1966,
ADR 0636 §8). The test of an unread list now reads the reason that names the
bootstrapped seats.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 00:48:14 +01:00
jschoubben 4af6c274b8 S4: an unread seat list holds a module before its build is read
On ace, a node-engine left on its bootstrapped seat list could not find the
artifact store's seat, refused avahi's archive reference and with it the whole
pass, so the seat-list hold never ran and every module said "reaches no
artifact store". While the list is unread, a placed module whose seats are not
all bootstrapped is now held as it runs before anything of its build is read
(frombus.Scope.Held), its reason names the bootstrapped seats the node holds,
and a pass refused whole says the unread list first (issue 1966).

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 00:40:53 +01:00
jschoubben 7bf7e29a32 S3: the seat-list verbs answer a proved operator's session
seat-list-again answered person.<name> and the healers, but every call through
the mesh MCP server is proved as its machine's tool runner (issue 627), so no
operator's session could make a node read its seat list again. node-tools now
stands for that session; the read seat-list is open to every proved caller, as
ADR 0636 §7 limits only the read again. The verb names come from the contract.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 00:32:44 +01:00
jschoubben 81bc0cc388 S1: keep the bus's "no responders" beside ErrNotAsked on an ask
A request that reached no one is asked again by mesh-sdk's busretry only when
it can see the bus's own error; the node-engine's ask dropped it, so the seat
list's first read, which fell into its holder's re-serve, was never asked
again (issue 1966).

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 00:29:54 +01:00
mesh-tools 9d8647aa0f Merge pull request 'The node-engine leaves an unchanged unit running across its own replacement, and says what changed when it restarts one (issue 1966)' (#220) from fix/1966-engine-restarts-unchanged into main 2026-11-09 22:09:40 +00:00
jschoubben aa649d85a0 The restart line names no part of a value; the new rows join existing tests (issue 1966, review round 1)
mesh/delivery merged: source.main moved to novox/mesh-host@9d8647aa0f56
mesh/check checking aa649d85a065 on 9d8647aa0f56
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-09 22:56:42 +01:00
jschoubben bcd815a6bb A tool runner is no longer restarted by a build of the engine: owned paths name the directory of every build, and a restart names its cause (issue 1966)
The tool runner's unit carries the paths modules own, for the holder of node-files. The engine's own
archive is unpacked into a place named for its build, so every engine replacement changed that word
and restarted the tool runner with nothing of it changed (novox 21:16, 22:25, 22:38 on 2026-11-09).
A resource unpacked into a place named for its build now owns the directory holding every build.
The restart line now names the unit's directive, word and key that differ, never a value.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-09 22:49:20 +01:00
mesh-tools 6e461bd24a Merge pull request 'A claim problem or a refused credential holds its module, not the whole pass (issue 1966, recovery)' (#219) from fix/1966-s1-partial-pass into main 2026-11-09 17:54:55 +00:00
jschoubben ef41fa2e27 The node-engine's live path holds a module whose claim or credential fails (issue 1966, review round 1)
mesh/delivery merged: source.main moved to novox/mesh-host@6e461bd24aa5
mesh/check being carried to ef41fa2e2796 on 6e461bd24aa5 | from 7c483da93a03
compose.CompositionsHolding returns the modules the declaration leaves out for a claim that cannot be held or a refused
credential; frombus composeEach holds each with those words, so the node-engine keeps what it handed of the module (no
stop of a running module), the candidate answer and the check's sandbox name it refused, and the rest composes.
CredentialsNeeded asks nothing for a held module. Resolve and checkClaims move to a test helper (production uses
ResolveHolding and holdClaims); a pass refused anyway names a held module's reason among the others.

Quick fix: Declared.Held read in the composer only. Thorough, chosen: the same held words reach every reader of the
composition (selfcompose, ComposeSent, the candidate answer) through composeEach's one hold path.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-09 13:04:55 +01:00
jschoubben 868ee6beb0 A claim problem or a refused credential holds its module, not the whole pass (issue 1966, S1 and S5)
The node-engine refused the whole pass when two modules claimed a seat held once, or when a consumer's credential was
refused. Now the composer leaves that module out of the declaration and composes the rest: the module is returned held
(Resolved.Held, resolution.ResolveHolding), reaches the declaration as left out through the filter and the declaration
#218 already keep (Rendering.NotPlaced), and Declared.LeftOut names it. The second claimant of a seat is the one held;
its claim is not this node's holding. The rest-of-mesh view and the network use the same, so a node with a held module
stays on the network. Resolve stays strict for callers that cannot leave a module out.

Quick fix: hold only the credential refusal (resolve.go planForNow), about 20 minutes. Thorough fix, chosen: every
claim problem is attributed to its module (claimsOf) and both kinds go through the one left-out path of ADR 0163 rule 6,
in the declaration, the filter and the network, so no per-kind refusal of a pass remains in the composer. Left for the
next step: a declared-and-not-made own secret (NotMadeError) and the Foreseeing-mode refusal still refuse the pass; the
frombus candidate path holds those by stub already.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-09 12:52:54 +01:00
mesh-tools 7c483da93a Merge pull request 'The filter keeps the openings of a module left out of the declaration that still runs (issue 1966, recovery)' (#218) from fix/1966-filter-keeps-openings into main 2026-11-09 09:49:15 +00:00
jschoubben 17a4ac620c A left-out module's openings keep its per-node narrowing (issue 1966, review round 1)
mesh/delivery merged: source.main moved to novox/mesh-host@7c483da93a03
mesh/check checking 17a4ac620c07 on 7c483da93a03
Rules skipped a left-out module's expose and reaches, so its listens
opened at the manifest's default source and an operator's narrowing for
the node was dropped (ADR 0046). Its exposure is now read like any
module's (exposureOf); where it cannot be read for that module, its
listens open no wider than the mesh (noWiderThanTheMesh), never at the
manifest's default.

Left for the thorough fix, composing a left-out module as what runs of
it (Carried, as a held module is): the filter also opens for a left-out
module that never ran on the node, and it opens the new manifest's
listens rather than what the old composition runs.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 10:43:48 +01:00
jschoubben 95f45b1a74 Keep a left-out module's openings in the filter: it still runs (issue 1966)
A module left out of the declaration for a setting its new definition no
longer composes (ADR 0163 rule 6) keeps running what it ran, but
resolutionCompose dropped it from the modules the filter is derived from,
so the filter closed the ports it still serves. On novox, 2026-11-09
07:46-09:24 UTC, the artifact store's want at build 6e1dfaa8 (listens on
5100, no container publishes 5000) met the node layer's ports
{"5000": 5100}: GivenPorts refused it, distribution was left out, and
5100 closed while the old registry container served there; every build
failed "dial tcp 10.10.0.1:5100: i/o timeout".

The filter is now derived from the composed modules and the left-out
ones; a left-out module's settings (exposure) are still not read.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 10:36:55 +01:00
mesh-tools 2019122379 Merge pull request 'Gate record of mesh-host's forge-made merges (issue 1966, recovery)' (#217) from fix/1966-forge-made-merge into main 2026-11-09 08:45:52 +00:00
jschoubben e504c12940 Record this repository's forge-made merges in the delivery gate, so the indexer takes them in (issue 1966, recovery)
mesh/delivery merged: source.main moved to novox/mesh-host@20191223794a
mesh/delivery-group group fix/1966-forge-made-merge merged: every member is merged
mesh/check checking e504c129401b on 20191223794a | novox/mesh-host#217 in group fix/1966-forge-made-merge
Merges made on the forge directly since 2026-11-09 07:04 UTC have no gate merge record,
and the indexer holds them; an empty merge through the gate covers this repository's head.
Second of its kind: the group's mesh-delivery member removes the need (the forge bridge
reads a forge-made merge's pull request from the forge).

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 09:17:02 +01:00
mesh-tools 25d677d20c Merge pull request 'The node-engine says why it composes a wanted module at no composition, and hands again one its agent lacks (issue 1966, recovery)' (#216) from fix/1966-views-page-time into main 2026-11-09 08:13:14 +00:00
jschoubben 6bde46cb00 The wait-for-the-composer step settles a want again without a real sleep and counts only ghost's own tells (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-host@25d677d20c93
mesh/delivery-group group fix/1966-views-page-time merged: every member is merged
mesh/check checking 6bde46cb0084 on 20191223794a | novox/mesh-host#216 in group fix/1966-views-page-time
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 09:09:40 +01:00
jschoubben 3ea81943f5 The wait-for-the-composer test settles the want again without a real sleep (issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 09:05:30 +01:00
jschoubben 3dc4307aec The node's words for a want it composes nothing of are said once per reason, and proved on the live paths (issue 1966)
The said-once keys no longer hold where the indexer's picture stands; a module composed from the manifest for want of a done
build is named in Pass.Unmade; the agent's wait is told once per want and again after the composition was there; the two tests
are folded into existing ones.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 09:04:45 +01:00
jschoubben 5f37a43b91 The node says why it composes nothing of a want, and hands again what its agent waits for (issue 1966)
Every pass names each wanted module it made no composition of, with why (Pass.Unmade, the verb compose, one journal line per
change); a want not taken is said; the agent says once per want that it waits for the composer and why, and tells it, so a
composition the composer believed handed is handed again.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 08:58:33 +01:00
mesh-tools 9b55bcb127 Merge pull request 'The node-engine fetches every artifact before it replaces anything, and a failed apply no longer claims the old composition runs (issue 1966, recovery)' (#215) from fix/1966-fetch-before-replace into main 2026-11-09 00:16:25 +00:00
jschoubben 9f6801a6c5 A module whose apply failed is no longer said to run what it ran (issue 1966, recovery)
mesh/delivery merged: source.main moved to novox/mesh-host@9b55bcb12743
On 2026-11-08 a failed apply of distribution's new build had already replaced
its store container, yet the node-engine still held the old build as running,
so the put back to it at 23:47 UTC was judged unchanged and applied nothing.
A failed apply now marks what runs of the module failed, and a want of the
composition that ran before is applied again.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-09 01:15:13 +01:00
jschoubben 22a37b758c Fetch every archive and process of an apply before anything is replaced (issue 1966, recovery)
On 2026-11-08 23:25 UTC the node-engine replaced distribution's store container
and only then fetched distribution's program from the store it had just cut off:
the module was left half applied and the artifact store unreachable from every
other machine. An apply now fetches and checks every archive and process first,
as it already pulls a module's images first; one that cannot be had gates its
module, so none of its resources is replaced and none of its orphans removed.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-09 01:14:00 +01:00
mesh-tools 25aa110920 Merge pull request 'Record forge-made merges in the delivery gate (issue 1966, recovery)' (#214) from fix/1966-gate-record-2 into main 2026-11-08 21:46:07 +00:00
jschoubben 71508cc962 Record the forge-made merge in the delivery gate, so the indexer takes it in (issue 1966, recovery)
mesh/delivery merged: source.main moved to novox/mesh-host@25aa110920cd
mesh/delivery-group group fix/1966-gate-record-2 merged: every member is merged
mesh/check checking 71508cc96297 on 25aa110920cd | novox/mesh-host#214 in group fix/1966-gate-record-2
mesh-host #213 (87bdc95a) was merged on the forge under recovery-fixes-merge-and-tell and has no gate merge record; the indexer holds its take-in. An empty merge through the gate covers it, and the forge-made wait is dropped (indexer_drop_pending).

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 19:23:06 +01:00
mesh-tools 87bdc95a3d Merge pull request 'The node-engine serves its seat's verbs from the bootstrapped definition until it holds the seat list (issue 1966, recovery)' (#213) from fix/1966-engine-seat-verbs-bootstrap into main 2026-11-08 17:34:51 +00:00
jschoubben 7b42c94a4d The node-engine serves its seat's verbs from the bootstrapped definition until it holds the seat list (issue 1966, recovery)
mesh/delivery merged: source.main moved to novox/mesh-host@87bdc95a3d65
mesh/check pass 7b42c94a4d78 on 87bdc95a3d65 | unit 1679 run, 333 kept, 31 skipped | integration 1692 run, 333 kept, 26 skipped | over bound: unit (sl…
The link serves the seat node-engine when it opens, before the node agent
starts the reader of the seat list; since ADR 0635 catalogue holds no seat
until then, so the engine registered no verb at all. Nobody answered
manifest-reads, compose, composition or the hand acts on novox, shanks and
now ace, and the assigner kept ace's old statement, holding its own
successor on seats the new engine states.

Quick and thorough fix are one here: the only seat-list read at start is
this one (the composer waits for the list, the reads statement restates).

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 18:34:20 +01:00
mesh-tools 6dd8e08a84 Merge pull request 'The node-engine passes seats with the SDK's one rule, by the build its node runs, and says the pass (ADR 0660, issue 1966, recovery)' (#212) from fix/1966-seat-pass-shared into main 2026-11-08 14:47:22 +00:00
jschoubben 60f413f9bc The node-engine reads each assignment by the build its node keeps and says a seat pass in its answer (issue 1966, ADR 0660)
mesh/delivery merged: source.main moved to novox/mesh-host@6dd8e08a84ed
mesh/delivery-group group fix/1966-seat-pass-shared merged: every member is merged
mesh/check checking 60f413f9bc6a on 6dd8e08a84ed | novox/mesh-host#212 in group fix/1966-seat-pass-shared
Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 15:44:00 +01:00
jschoubben 5108c28e22 The node-engine composes seat passes with the SDK's one rule instead of its own copy (issue 1966, ADR 0660)
Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 15:31:32 +01:00
mesh-tools 875b4ee5ce Merge pull request 'The node-engine states the seats of the list it holds, and again on every change (issue 1966, recovery)' (#211) from fix/1966-seat-reads-bootstrap into main 2026-11-08 03:45:15 +00:00
jschoubben 9ab85ef92e The node-engine states the seats of the seat list it holds, again at each change (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-host@875b4ee5ce38
mesh/delivery-group group fix/1966-seat-reads-bootstrap merged: every member is merged
mesh/check failed (2): mesh-delivery-checker: no published of the tagged version of go-sdk of mesh-delivery…; and 1 more | 9ab85ef92e6c on 875b4ee5ce38
manifestReads froze the statement at its first call, before the engine gave
catalogue even its bootstrapped seats (ADR 0635 empties mesh-sdk's table
until the list is read), so ace's engine stated no seat and its own next
build, naming mesh-catalog, was held on it.

The words and forms stay the build's, computed once; the seats are those of
the list held now. The reader of the seat list restates after each change:
the node record carries the new statement at once and the node seat's event
manifest-reads-changed says it (ADR 0618 §4); an unchanged statement writes
and says nothing; the verb manifest-reads answers when it was made.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 02:36:41 +01:00
mesh-tools 89475bb0cf Merge pull request 'The node-engine's bus watch never holds the client back (issue 1966, bootstrap recovery)' (#210) from fix/1966-watch-never-blocks into main 2026-11-07 23:40:48 +00:00
jschoubben 4f1e92160a Round 1: the pin tests merged (test budget), KVWatch's doc says what folds
mesh/delivery merged: source.main moved to novox/mesh-host@89475bb0cfec
The review of fe9533c1: a new test function raised a count already over the
budget, so two broker-pin tests are one; KVWatch no longer yields every
change, and its doc says so (issue 1966).

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 00:38:36 +01:00
jschoubben fe9533c186 The node-engine's bus watch never holds the client back: it keeps the newest update per key while its reader is busy (issue 1966)
Live fault 2026-11-08: node-engines on two machines logged only "nats: slow consumer, messages dropped" and applied nothing. The
client's watch sends into its channel from inside the subscription's callback; KVWatch forwarded that channel to its reader
blocking, and the reader (composing for minutes) stopped draining, so the subscription's pending limit overflowed.

Quick fix: a larger buffer on the watch's channel; it only moves the limit. Thorough fix (chosen): the forwarder always drains the
client's channel and, while its reader is busy, coalesces the pending updates by key (newest value and revision, first-seen
order, Caught kept in its place), so no reader's pace can back the client up, whatever the burst. A slow consumer error now
names its subscription and what it holds pending.

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 00:26:33 +01:00
mesh-tools 70e70eda5a Merge pull request 'Registry milestone: mesh-host (issue 1966)' (#209) from feat/registry-milestone into main 2026-11-07 14:19:45 +00:00
jschoubben cb7a444a56 The check's composer says a node's refusal in the words of its composition with the lifted modules held (issue 1921, milestone 47/G)
mesh/delivery merged: source.main moved to novox/mesh-host@70e70eda5a24
mesh/delivery-group group feat/registry-milestone merged: every member is merged
mesh/check checking cb7a444a5634 on 70e70eda5a24 | novox/mesh-host#209 in group feat/registry-milestone
When a node refused also with the modules composed without their build held as its node-engine holds them, the
verdict kept the words of the composition from their unbuilt manifests, whose refusals the lift itself made (a tool
runner with no bundle, a resource whose artifact or template no build resolved). In the group check of c5d32074c945
that hid lab's real refusal on shanks among them. The node's refusal is now that of the held composition, named so.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 08:19:55 +01:00
jschoubben 53addd73c3 The check's composer composes a module at its build only where that build was made of the change's manifest (issue 1921, milestone 47/G)
mesh/delivery-group group feat/registry-milestone checking: 7 of 7 member(s) ready
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 53addd73c358 on e73aae3686a6 | novox/mesh-host#209 in group feat/registry-milestone
The running indexer could not read 17 of the group's manifests, so the running node-engine composed those modules at
their trunk builds and said nothing; the head composer laid those builds over the change's other manifests and ace
failed on "nothing provides artifact-store, wanted by mesh-build-go" (group check 8a80e8e86858). The input a node
answers a check with now names, per module taken in from its build, the content digest of the manifest that build was
made of (ComposedHow.Manifests); the composer compares it with the case's manifest, and, for an input from a
node-engine before it, compares what the two manifests say beyond what a take-in resolves (the lead's ruling, §17).
Each module lifted either way is composed from the case's manifest, its build awaited, and named in NotJudged with
the comparison that lifted it; one that cannot compose without its build is held as its stub. The digest is taken
over one JSON form because a request through a build module arrives compacted.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 07:24:16 +01:00
jschoubben b6c174a324 A consumer beside a held provider is told what it serves without the names it does not compose, never their placeholders (issue 1921, milestone 47/G)
The provider held on its machine is composed apart, so no module of the composed set serves the provision; its
manifest's served values reached the consumer's binding raw and the binding refused the consumer for a
${bound:route:name-web} it does not require (the group check of 8a80e8e86858 on novox, mesh-build-node beside a held
gitea). Each such key is now left out and said on the need, as a provider elsewhere leaves it out (ADR 0346).

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 07:19:58 +01:00
jschoubben 5b75e866d0 The composition verb says what holds each held module and the build it would be composed at (issue 1921, milestone 47/G H4)
mesh/delivery closed: a newer head of the same pull request
mesh/delivery-group group feat/registry-milestone checking: 6 of 7 member(s) ready
mesh/check failed (4): unit: git.novox.be/novox/mesh-catalog/checks/gitignored.TestNoProgramABuildLeavesIsC…; and 3 more | 5b75e866d079 on e73aae3686a6
The node's own composition named a held module's why only; it now also says whether its manifest's verdict or this
node-engine's decoder holds it, and at which build, as the input a check's composer reads says it. Said in the held
words, so the node-engine seat's answer keeps its shape.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 02:38:18 +01:00
jschoubben d0fb384f00 The check's composer composes a module the running indexer alone holds when the change's manifest is the check's to judge (issue 1921, milestone 47/G H2)
ADR 0578 §2 and §3: a case's accepted verdict, or none (the running checker drops the change's own), lifts the
input's hold on a manifest the composer reads whole; a case's refusal and bytes that do not decode stay held. It is
composed at the build the input carries (Input.Holds). Without that build the node is not judged and the verdict
names the module, why it was held and that its build is awaited; where it cannot compose without it, it is composed
held as its node-engine holds it, and said so. Never a silent pass (the lead's Q4 ruling). The issue 774 row now
gives the runner the refused verdict the checker gives a manifest the change does not bring.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 02:33:21 +01:00
jschoubben c114b786e8 The input a node answers a check with says what holds each held module and carries its build (issue 1921, milestone 47/G H3)
A module the running indexer refused stood in the input as its stub with no build, so a check's composer that
judges the change's manifest itself had nothing to compose it at. Each held module now says whether its verdict
or the decoder holds it; for a check's answer the node also takes in the build it would compose it at, when the
builder holds it done, beside the stub and never into what the node composes.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 02:33:21 +01:00
jschoubben e96a7ada50 The node-engine keeps no copy of a seat's verbs: the seat list comes from the bus (issue 1921, ADR 0635 §1)
Eight lists of seat verbs (checker, enrolment, artifact sweep, delivery, node-resolver, node-nfs-server,
node-mounts, mesh-config's input) and their schema helpers were read by nothing but one test, and drifted
from their seats (B0's verb input never reached the checker's copy). A guard row in the seats test finds
any list of the catalogue's verbs written in the node-engine's code.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 02:26:14 +01:00
jschoubben eec35a7c0c The sandbox composer composes against the seat list its request carries, and judges nothing without one (issue 1966)
The group try of 2026-11-06 10:07 failed its composition on all four
machines because mesh-compose, the node-engine's composer a check builds
at the head, loaded no seat list once the compiled one was deleted (ADR
0635 §1): the uplink's holder no longer stepped back from the resolver
file, and no holder of the registry seats was known. It now takes the
request's seats over the foundation's bootstrapped ones, as the
node-engine takes the bus's; without any it says each machine not judged
(ADR 0636 §1, ADR 0593 §9), never a refusal or pass made without them.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 12:01:00 +01:00
158 changed files with 10944 additions and 1682 deletions
+58 -4
View File
@@ -11,9 +11,12 @@
// checkout or `git archive` of the catalogue repository at a ref: the command fetches nothing). -node adds a machine to
// compose; -declarations answers each machine's declared paths with their owners and modes.
//
// It opens no network connection, bus, store or file beyond its standard input, the -request file and the -catalogue
// directory, and reads
// no environment. Exit codes: 0 the verdict is written (a machine that does not compose is a verdict, not an error);
// Run as a pipeline step it reads its input, a composer.Request at depth compose, from the file named by the environment
// variable MESH_RUN_INPUT and answers the run's output as its composer.Verdict on the last line of standard output
// (design 47 G A-167).
//
// It opens no network connection, bus, store or file beyond its standard input, the -request file, the file
// MESH_RUN_INPUT names and the -catalogue directory, and reads no other environment. Exit codes: 0 the verdict is written (a machine that does not compose is a verdict, not an error);
// 1 the request cannot be read or run, said on standard error; 2 the flags are wrong.
package main
@@ -23,11 +26,17 @@ import (
"flag"
"fmt"
"io"
"maps"
"os"
"path/filepath"
"slices"
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/seatdef"
"git.novox.be/novox/mesh-sdk/go/seatlist"
"github.com/novox/mesh-host/internal/composer"
)
@@ -59,6 +68,11 @@ func run(args []string, stdin io.Reader, stdout, stderr io.Writer) int {
}
var raw []byte
var err error
// A pipeline step hands its input as a file named by MESH_RUN_INPUT (design 47 G A-167); -request, when given, wins.
runInput := os.Getenv("MESH_RUN_INPUT")
if *requestFile == "" && runInput != "" {
*requestFile = runInput
}
if *requestFile != "" {
raw, err = os.ReadFile(*requestFile)
} else {
@@ -105,7 +119,7 @@ func run(args []string, stdin io.Reader, stdout, stderr io.Writer) int {
if *declarations {
req.Declarations = true
}
v, err := composer.Run(req)
v, err := runWithSeats(req)
if err != nil {
fmt.Fprintln(stderr, "mesh-compose:", err)
return 1
@@ -118,6 +132,46 @@ func run(args []string, stdin io.Reader, stdout, stderr io.Writer) int {
return 0
}
// NoSeatList is what a machine reads, not judged, when the request carries no seat list (novox/hq ADR 0636 §1).
const NoSeatList = "the composer was handed no seat list, so it composes nothing (a reader of the seat list acts on nothing " +
"new until it has read the list, novox/hq ADR 0636 §1): the asker gives the seat definitions it judges against in the request's seats"
// runWithSeats composes req against the seat list the request carries (Request.Seats: the definitions the asker's seat list
// reader holds, the members' heads' laid over them in a check, ADR 0636 §4), over the foundation's bootstrapped ones
// (seatlist.Bootstrapped), as the node-engine composes against the list it read from the bus (ADR 0635 §1). The command
// compiles no seat list: without one in the request it reads every manifest and judges no machine, each said not judged
// with NoSeatList (ADR 0593 §9), never a refusal or a pass made without the seats' paths, deliveries and holders.
func runWithSeats(req composer.Request) (composer.Verdict, error) {
if len(req.Seats) == 0 {
machines := req.Machines
req.Machines = nil
v, err := composer.Run(req)
if err != nil || len(machines) == 0 {
return v, err
}
for i := range v.Cases {
v.Cases[i].Machines = map[string]composer.Machine{}
for _, node := range machines {
v.Cases[i].Machines[node] = composer.Machine{NotJudged: NoSeatList}
}
}
return v, nil
}
defs := map[string]seatdef.Definition{}
for _, d := range seatlist.Bootstrapped() {
defs[d.Seat] = d
}
for _, raw := range req.Seats {
d, err := seatdef.Parse(raw)
if err != nil {
return composer.Verdict{}, fmt.Errorf("a seat definition of the request: %w", err)
}
defs[d.Seat] = d
}
catalogue.UseSeats(catalogue.SeatsFromDefinitions(slices.Collect(maps.Values(defs))))
return composer.Run(req)
}
// readCatalogue reads every module.json of a catalogue checkout by its module's name: the root's and each of
// modules/*/. A manifest whose name cannot be read is kept under its directory's name, so the reader's refusal is said.
func readCatalogue(dir string) (map[string]composer.Manifest, error) {
+72 -7
View File
@@ -3,8 +3,8 @@ package main
import (
"bytes"
"encoding/json"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"github.com/novox/mesh-host/internal/buckets/bucketstest"
"go/build"
"os"
@@ -17,6 +17,7 @@ import (
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/seatlist/seatlisttest"
"github.com/novox/mesh-host/internal/beside"
"github.com/novox/mesh-host/internal/compose"
"github.com/novox/mesh-host/internal/composer"
@@ -49,11 +50,25 @@ func meshOf() compose.Input {
func request(t *testing.T, depth string) composer.Request {
t.Helper()
r := composer.Request{V: composer.Version, Depth: depth, HoldingsFromClaims: true,
Machines: []string{"anchor", "laptop", "ace"},
Cases: []composer.Case{{Name: "trunk", Manifests: captured(t)}}}
Machines: []string{"anchor", "laptop", "ace"}, Seats: seats(t),
Cases: []composer.Case{{Name: "trunk", Manifests: captured(t)}}}
return withInput(t, r, meshOf())
}
// seats is the seat list a request carries, as the checker hands the one its reader holds (ADR 0635 §1, ADR 0636 §4).
func seats(t *testing.T) []json.RawMessage {
t.Helper()
var out []json.RawMessage
for _, d := range seatlisttest.Definitions() {
raw, err := json.Marshal(d)
if err != nil {
t.Fatal(err)
}
out = append(out, raw)
}
return out
}
// withInput is r with in as its input, as a caller puts it on the wire: the request's shape carries the input as JSON.
func withInput(t *testing.T, r composer.Request, in compose.Input) composer.Request {
t.Helper()
@@ -124,6 +139,31 @@ func TestTheCommandWritesOneVerdictPerManifestAndMachine(t *testing.T) {
t.Errorf("%s does not compose and says no reason", name)
}
}
// Issue 1966, the group try of 2026-11-06 10:07: the composer judges against the seat list its request carries, the
// node-engine's from the bus (ADR 0635 §1). The uplink's holder steps back from the resolver file for the resolver's
// holder by the seat's definition; without the list nothing is judged (ADR 0636 §1), never "both declare the path".
resolverMachine := func(withSeats bool) composer.Machine {
r := composer.Request{V: composer.Version, Depth: composer.DepthCompose, HoldingsFromClaims: true,
Machines: []string{"laptop"}, Cases: []composer.Case{{Name: "trunk", Manifests: captured(t)}}}
if withSeats {
r.Seats = seats(t)
}
r = withInput(t, r, compose.Input{Nodes: []compose.Node{{Name: "laptop",
Assigned: []string{"networkmanager", "systemd-resolved", "pacman", "systemd", "dnsmasq"},
Capabilities: map[string]bool{"file-references": true, "privileged": true, "service-manager": true, "package-manager": true, "uplink-networkmanager": true}}}})
code, v, stderr := runJSON(t, nil, encode(t, r))
if code != 0 || len(v.Cases) != 1 {
t.Fatalf("exit %d: %s", code, stderr)
}
return v.Cases[0].Machines["laptop"]
}
if m := resolverMachine(true); strings.Contains(m.Reason+strings.Join(m.Problems, " "), "both declare") || m.NotJudged != "" {
t.Errorf("with the seat list, the uplink's and the resolver's holders on one machine: %+v, want the uplink's stepping back", m)
}
if m := resolverMachine(false); m.Composes || m.NotJudged != NoSeatList || m.Reason != "" {
t.Errorf("without a seat list: %+v, want not judged, saying why", m)
}
}
// 1621/AC3: over the captured catalogue and a mesh of three machines, the command's verdict equals, machine for
@@ -196,7 +236,7 @@ func TestACatalogueCheckoutIsComposedForANode(t *testing.T) {
if err := os.WriteFile(filepath.Join(dir, "modules", "demo", "module.json"), []byte(manifest), 0o644); err != nil {
t.Fatal(err)
}
in := encode(t, withInput(t, composer.Request{V: composer.Version}, compose.Input{Nodes: []compose.Node{{Name: "laptop", Assigned: []string{"demo"}}}}))
in := encode(t, withInput(t, composer.Request{V: composer.Version, Seats: seats(t)}, compose.Input{Nodes: []compose.Node{{Name: "laptop", Assigned: []string{"demo"}}}}))
code, v, stderr := runJSON(t, []string{"-catalogue", dir, "-node", "laptop", "-declarations"}, in)
if code != 0 {
t.Fatalf("exit %d: %s", code, stderr)
@@ -227,6 +267,31 @@ func TestTheRequestIsReadFromAFile(t *testing.T) {
}
}
// A pipeline step names its input file in MESH_RUN_INPUT: the command reads the request there and the verdict is the last
// line of standard output (design 47 G A-167).
func TestThePipelineStepReadsMESH_RUN_INPUT(t *testing.T) {
file := filepath.Join(t.TempDir(), "input.json")
r := composer.Request{V: composer.Version, Depth: composer.DepthCompose, Cases: []composer.Case{{Name: "only", Manifests: map[string]composer.Manifest{
"demo": {Raw: json.RawMessage(`{"module":"demo","version":"1"}`)}}}}}
if err := os.WriteFile(file, encode(t, r), 0o600); err != nil {
t.Fatal(err)
}
t.Setenv("MESH_RUN_INPUT", file)
var out, errOut bytes.Buffer
if code := run(nil, bytes.NewReader(nil), &out, &errOut); code != 0 {
t.Fatalf("exit %d: %s", code, errOut.String())
}
lines := strings.Split(strings.TrimRight(out.String(), "\n"), "\n")
var v composer.Verdict
if err := json.Unmarshal([]byte(lines[len(lines)-1]), &v); err != nil || len(v.Cases) != 1 || !v.Cases[0].Manifests["demo"].Accepted {
t.Fatalf("last line is no verdict: %v\n%s", err, out.String())
}
t.Setenv("MESH_RUN_INPUT", file+".gone")
if code := run(nil, bytes.NewReader(nil), &out, &errOut); code != 1 || !strings.Contains(errOut.String(), "cannot be read") {
t.Errorf("a missing input file: exit %d, %q", code, errOut.String())
}
}
// The exit codes: 1 for a request that cannot be read or run, 2 for wrong flags, each said on standard error.
func TestTheExitCodesSayWhatFailed(t *testing.T) {
unfilled := encode(t, composer.Request{V: composer.Version, Cases: []composer.Case{{Name: "c", Manifests: map[string]composer.Manifest{"x": {Digest: "abc"}}}}})
@@ -282,7 +347,7 @@ func ownPackages(t *testing.T) map[string][]string {
}
// 1621/AC2 and AC4: the command imports, through every package of this repository it reaches, only the standard
// library, mesh-sdk, the contracts modules' pure packages (seats' types, clients' codecs), the delivery SDK's composer (the command's own request and verdict types), the builder's builds package (a build's value and its keys), the private network's overlay package (mesh-wireguard's own pure package: names, graph and
// library, mesh-sdk, the contracts modules' pure packages (seats' types, clients' codecs), the delivery SDK's composer (the command's own request and verdict types) and catalog key codec (a catalog entry's key, design 47 G A-144), the builder's builds package (a build's value and its keys), the private network's overlay package (mesh-wireguard's own pure package: names, graph and
// generator, no bus) and this repository's own packages, and none that reaches a bus, a store, a network service or
// another process.
func TestTheCommandImportsNoBusStoreNetworkOrProcess(t *testing.T) {
@@ -294,7 +359,7 @@ func TestTheCommandImportsNoBusStoreNetworkOrProcess(t *testing.T) {
for path, imports := range own {
for _, imp := range imports {
std := !strings.Contains(strings.SplitN(imp, "/", 2)[0], ".")
if !std && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-networking/modules/mesh-wireguard/go/overlay") && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-sdk/go") && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-contracts/") && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-delivery/contracts/") && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-delivery/sdk/composer") && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds") && !strings.HasPrefix(imp, "github.com/novox/mesh-host/") {
if !std && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-networking/modules/mesh-wireguard/go/overlay") && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-sdk/go") && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-contracts/") && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/") && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk/composer") && imp != "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk/catalogkey" && !strings.HasPrefix(imp, "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds") && !strings.HasPrefix(imp, "github.com/novox/mesh-host/") {
t.Errorf("%s imports %s, which is neither the standard library, mesh-sdk nor this repository", path, imp)
}
for _, f := range forbidden {
+20
View File
@@ -1,12 +1,16 @@
package main
import (
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"regexp"
"strings"
"github.com/novox/mesh-host/internal/agent"
"github.com/novox/mesh-host/internal/compositions"
)
// appliedStore keeps every composition the agent applied, one file per module (the node's own as `node.json`), in the
@@ -105,3 +109,19 @@ func (s appliedStore) Drop(module string) error {
}
return nil
}
// ownModuleApplied is the module of the composition the store holds at the fingerprint fp, the engine's own build's
// (design 47 G §527, A-149): "" when fp is none or no record holds it.
func ownModuleApplied(s appliedStore, fp string) string {
if fp == "" {
return ""
}
all, _ := s.Load()
for module, raw := range all {
var c compositions.Composed
if module != agent.NodeModule && json.Unmarshal(raw, &c) == nil && c.FP == fp && c.Module != "" {
return c.Module
}
}
return ""
}
+37 -2
View File
@@ -18,6 +18,7 @@ import (
occ "git.novox.be/novox/mesh-contracts/operator-and-conversation/go/clients/operatorchannelclient"
stdio "git.novox.be/novox/mesh-sdk/go"
"github.com/novox/mesh-host/internal/accounts"
"github.com/novox/mesh-host/internal/agent"
"github.com/novox/mesh-host/internal/apply"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/selfcompose"
@@ -284,7 +285,7 @@ const compositionReadFor = 20 * time.Second
// compositionFrom answers the verb composition from the node agent that runs now (issue 774); refused in words while
// none does.
func compositionFrom(current *agentNow) func(module, digest string, diff bool) (any, error) {
func compositionFrom(current *agentNow, withheld func(ctx context.Context, module string) ([]link.WithheldSubject, string)) func(module, digest string, diff bool) (any, error) {
return func(module, digest string, diff bool) (any, error) {
ag := current.get()
if ag == nil {
@@ -292,7 +293,17 @@ func compositionFrom(current *agentNow) func(module, digest string, diff bool) (
}
ctx, cancel := context.WithTimeout(context.Background(), compositionReadFor)
defer cancel()
return ag.Composition(ctx, module, digest, diff)
view, err := ag.Composition(ctx, module, digest, diff)
if err != nil {
return view, err
}
// And what the grants part withholds of it here, through its verb withheld (design 46 approval mark §6).
subjects, unknown := withheld(ctx, module)
for _, s := range subjects {
view.Withheld = append(view.Withheld, agent.WithheldView{Subject: s.Subject, Why: s.Why})
}
view.WithheldUnknown = unknown
return view, nil
}
}
@@ -390,3 +401,27 @@ func composeCandidateFrom(node string, shadow *atomic.Pointer[selfcompose.Shadow
return json.RawMessage(raw), nil
}
}
// firstViewsFrom answers the seat's verb first-views from the composer of this run (design 47 G §698): the views the node-engine
// follows (none), the entries it keeps and the wants it composes from.
func firstViewsFrom(composer *atomic.Pointer[selfcompose.Composer]) func() (any, error) {
return func() (any, error) {
c := composer.Load()
if c == nil {
return nil, errors.New("this node composes none of its own modules now (it applies the controller's compositions, or its node agent is starting), so it keeps no entry")
}
views, entries, wants := c.FirstViews()
return map[string]any{"first-views": views, "entries": entries, "wants": wants}, nil
}
}
// entryAgainFrom asks one entry of the catalog half again, for the seat's verb entry-again (design 47 G §698).
func entryAgainFrom(composer *atomic.Pointer[selfcompose.Composer]) func(ctx context.Context, fingerprint string) error {
return func(ctx context.Context, fingerprint string) error {
c := composer.Load()
if c == nil {
return errors.New("this node composes none of its own modules now, so it keeps no entry to ask again")
}
return c.EntryAgain(ctx, fingerprint)
}
}
+71
View File
@@ -0,0 +1,71 @@
package main
import (
"context"
"encoding/json"
"fmt"
"strings"
"time"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/nodebuildgo"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/nodebuildimage"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/nodebuildnode"
"github.com/novox/mesh-host/internal/apply"
)
// The artifact store's holder takes its own new build from the build node that kept it (design 47): its done build names the
// node and the digest, the node's build seat answers where the bytes are as an object on the bus, and the node-engine
// reads them and checks them against the digest. Never another module's files: the seat's verb is the way.
// keptAskWithin bounds one ask of a build seat; keptObjectWithin one read of the object.
var (
keptAskWithin = 30 * time.Second
keptObjectWithin = 10 * time.Minute
)
// keptBuildSeats are the three build seats, each of which serves the verb kept-artifact: the node that kept the build
// holds the seat of the language it was built in, and the others do not answer.
var keptBuildSeats = []string{nodebuildgo.Seat, nodebuildnode.Seat, nodebuildimage.Seat}
// keptArtifactFetch is the apply's fetch of what a module's done build kept on a build node.
func keptArtifactFetch(buildOf func(context.Context, string, string) (string, string, bool, error),
ask func(ctx context.Context, node, seat, verb string, args any, within time.Duration) (json.RawMessage, error),
object func(ctx context.Context, store, name string, limit int64) ([]byte, error)) apply.KeptFetch {
return func(ctx context.Context, module, digest string) ([]byte, bool, error) {
node, fp, kept, err := buildOf(ctx, module, digest)
if err != nil || !kept {
return nil, false, err
}
hex := strings.TrimPrefix(digest, "sha256:")
var notHeld []string
for _, seat := range keptBuildSeats {
raw, err := ask(ctx, node, seat, nodebuildgo.VerbKeptArtifact,
nodebuildgo.KeptArtifactInput{Module: module, Fp: fp, Digest: hex}, keptAskWithin)
if err != nil {
notHeld = append(notHeld, seat+": "+err.Error())
continue
}
var got nodebuildgo.KeptArtifactAnswer
if json.Unmarshal(raw, &got) != nil {
return nil, true, fmt.Errorf("%s of %s answered with something that is not an answer of kept-artifact", seat, node)
}
switch {
case got.Error != "":
return nil, true, fmt.Errorf("%s of %s could not give the kept build of %s: %s", seat, node, module, got.Error)
case got.Refused != "":
return nil, true, fmt.Errorf("%s of %s refuses the kept build of %s: %s %s", seat, node, module, got.Refused, got.Words)
case got.Store == "" || got.Object == "" || got.Digest != hex:
return nil, true, fmt.Errorf("%s of %s answered for %s without the store and the object of the digest %s", seat, node, module, hex)
}
octx, cancel := context.WithTimeout(ctx, keptObjectWithin)
body, err := object(octx, got.Store, got.Object, got.Size+1)
cancel()
if err != nil {
return nil, true, fmt.Errorf("the build of %s kept on %s could not be read from %s: %w", module, node, got.Store, err)
}
return body, true, nil
}
return nil, true, fmt.Errorf("%s keeps the build of %s for the artifact store and none of its build seats answers (%s)",
node, module, strings.Join(notHeld, "; "))
}
}
+75
View File
@@ -0,0 +1,75 @@
package main
import (
"context"
"encoding/json"
"errors"
"strings"
"testing"
"time"
)
// A module's done build that kept its artifact on a build node is fetched through that node's build seat (design 47).
func TestAKeptArtifactIsFetchedThroughTheBuildSeatOfItsNode(t *testing.T) {
hex := strings.Repeat("a", 64)
stored := func(string) (json.RawMessage, error) {
return json.RawMessage(`{"module":"distribution","fp":"f1","digest":"` + hex + `","size":5,"store":"kept-builds","object":"` + hex + `"}`), nil
}
for _, row := range []struct {
name string
kept bool
answer func(seat string) (json.RawMessage, error)
object error
wantKept bool
wantErr string
wantSeat string
}{
{"a build kept nothing is not asked", false, nil, nil, false, "", ""},
{"the seat of the language answers", true, func(seat string) (json.RawMessage, error) {
if seat != "node-build-node" {
return nil, errors.New("no responder")
}
return stored(seat)
}, nil, true, "", "node-build-node"},
{"no build seat of the node answers", true, func(string) (json.RawMessage, error) { return nil, errors.New("no responder") },
nil, true, "novox keeps the build of distribution", ""},
{"the seat refuses it", true, func(string) (json.RawMessage, error) {
return json.RawMessage(`{"module":"distribution","fp":"f1","refused":"not kept","words":"swept"}`), nil
}, nil, true, "refuses the kept build of distribution: not kept", ""},
{"the object cannot be read", true, stored, errors.New("no object"), true, "could not be read from kept-builds", "node-build-go"},
} {
t.Run(row.name, func(t *testing.T) {
var seats []string
fetch := keptArtifactFetch(
func(context.Context, string, string) (string, string, bool, error) {
return "novox", "f1", row.kept, nil
},
func(_ context.Context, node, seat, verb string, _ any, _ time.Duration) (json.RawMessage, error) {
if node != "novox" || verb != "kept-artifact" {
t.Errorf("asked %s of %s on %s", verb, seat, node)
}
seats = append(seats, seat)
return row.answer(seat)
},
func(_ context.Context, store, name string, limit int64) ([]byte, error) {
if store != "kept-builds" || name != hex || limit < 5 {
t.Errorf("read %s/%s up to %d", store, name, limit)
}
return []byte("bytes"), row.object
})
body, kept, err := fetch(context.Background(), "distribution", "sha256:"+hex)
if kept != row.wantKept {
t.Fatalf("kept = %v", kept)
}
if row.wantErr == "" && (err != nil || (kept && string(body) != "bytes")) {
t.Fatalf("got %q, %v", body, err)
}
if row.wantErr != "" && (err == nil || !strings.Contains(err.Error(), row.wantErr)) {
t.Fatalf("the error is %v, want %q", err, row.wantErr)
}
if !row.kept && len(seats) > 0 {
t.Errorf("a build that kept nothing asked %v", seats)
}
})
}
}
+7 -6
View File
@@ -87,15 +87,16 @@ func followLogins(ctx context.Context, session *link.Session, node string, hear
}
}
// loginSecretModule is the module the vault names this engine's login values under: own/<node>/mesh-host/<login> (ADR 0583 §3,
// insight).
const loginSecretModule = "mesh-host"
// fetchLogin asks the vault, through its existing hand-out of a machine's own secrets, for the value of login, sealed to this
// node's key, and opens it with the sealing key; no new verb is asked for it.
func fetchLogin(queue *link.Session, statePath string) func(context.Context, string) (string, error) {
unseal := sealOpener(statePath)
return func(ctx context.Context, login string) (string, error) {
// The vault names the login values under the engine's own module, which is §527's (§634).
own, err := queue.Own.Wait(ctx)
if err != nil {
return "", fmt.Errorf("the node-engine's own module is not named yet, so the vault's name of its login value is not known: %w", err)
}
raw, err := queue.AskVault(ctx, "own", map[string]any{})
if err != nil {
return "", err
@@ -111,7 +112,7 @@ func fetchLogin(queue *link.Session, statePath string) func(context.Context, str
return "", fmt.Errorf("the vault's own answer is not one: %w", err)
}
for _, o := range a.Own {
if o.Module != loginSecretModule || o.Name != login {
if o.Module != own || o.Name != login {
continue
}
sealed := string(o.Sealed)
@@ -125,7 +126,7 @@ func fetchLogin(queue *link.Session, statePath string) func(context.Context, str
}
return string(value), nil
}
return "", fmt.Errorf("the vault hands this node no value for %s (own/%s/%s/%s)", login, queueNode(queue), loginSecretModule, login)
return "", fmt.Errorf("the vault hands this node no value for %s (own/%s/%s/%s)", login, queueNode(queue), own, login)
}
}
+31 -2
View File
@@ -16,6 +16,7 @@ import (
"errors"
"flag"
"fmt"
nodeengine "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-contracts/identity-and-access/go/seats/meshvault"
"io"
"net"
@@ -35,6 +36,7 @@ import (
"github.com/novox/mesh-host/internal/agent"
"github.com/novox/mesh-host/internal/apply"
"github.com/novox/mesh-host/internal/bundle"
"github.com/novox/mesh-host/internal/compositions"
"github.com/novox/mesh-host/internal/firewall"
"github.com/novox/mesh-host/internal/identity"
"github.com/novox/mesh-host/internal/inventory"
@@ -42,6 +44,7 @@ import (
"github.com/novox/mesh-host/internal/liveness"
"github.com/novox/mesh-host/internal/network"
"github.com/novox/mesh-host/internal/outward"
"github.com/novox/mesh-host/internal/processbuilds"
"github.com/novox/mesh-host/internal/profile"
"github.com/novox/mesh-host/internal/reachable"
"github.com/novox/mesh-host/internal/store"
@@ -985,7 +988,12 @@ func runLink(ctx context.Context, opts options) error {
// What this engine trusts a mesh signature from: the vault's key alone, once read from its signers (novox/hq ADR 0550
// §2; issue 1126: the controller's key never, the controller ended by its unassignment).
trust := newMeshTrust()
own := link.NewOwnName()
// The engine's own module, first from its applied record of the build whose fingerprint is its own (design 47 G §527,
// §634, A-149); on a first start with no such record, from its first composing.
own.Set(ownModuleApplied(appliedStoreFor(opts.state), os.Getenv(compositions.FingerprintVar)))
queue := &link.Session{
Own: own,
Membership: membership,
// And the public halves and certificate its record says (novox/hq ADR 0555 §3), read at each facts report.
Keys: recordKeys(opts.state, trust.Trusts),
@@ -1014,10 +1022,23 @@ func runLink(ctx context.Context, opts options) error {
queue.Acts = engineActs(opts, mine.Node, queue, setuidSearch)
go reconcileActs(ctx, queue.Acts)
current := &agentNow{}
queue.Acts.Composition = compositionFrom(current)
queue.Acts.Composition = compositionFrom(current, queue.WithheldOf)
queue.Acts.Compose = composeFrom(current, &composerNow)
queue.Acts.Packages = packagesFrom(mine.Node, queue, &composerNow)
queue.Acts.FirstViews, queue.Acts.EntryAgain = firstViewsFrom(&composerNow), entryAgainFrom(&composerNow)
queue.Acts.SeatList, queue.Acts.SeatListAgain = seatListRead, seatListAgain
queue.Acts.ProcessBuilds = func(module string) (any, error) {
d, err := declaredNow(opts.state, nil)
if errors.Is(err, store.ErrNothingDeclared) {
d, err = nil, nil
}
if err != nil {
return nodeengine.ProcessBuilds{Node: mine.Node, Error: "what runs on this node does not read: " + err.Error()}, nil
}
ctx, cancel := context.WithTimeout(context.Background(), lookBound)
defer cancel()
return processbuilds.Reader{Node: mine.Node, Run: processbuilds.Runner(opts.run())}.Read(ctx, d, module), nil
}
queue.Acts.ComposeCandidate = composeCandidateFrom(mine.Node, &shadowNow)
setRunsHere(runsFromAgent(current))
factsOfNow := func(ctx context.Context, ag *agent.Agent) {
@@ -1381,7 +1402,15 @@ func writeHealthReport(ctx context.Context, queue *link.Session, h link.Health,
if c := machinesNow.Load(); c != nil {
machines.ByAddress, machines.Hub = c.Machines()
}
_, err := queue.WriteNodeHealth(ctx, link.NodeHealthOf(queue.Membership.Node, h, peers, machines))
report := link.NodeHealthOf(queue.Membership.Node, h, peers, machines)
if queue.Fallback != nil && report.Fallback == nil {
report.Fallback = queue.Fallback() // node-account-fallback (ADR 0583 §6), as the written report carries it
}
_, err := queue.WriteNodeHealth(ctx, report)
// The findings the report says are raised by the engine here, at every look, written to the bus or not (design 47 G A-197).
if keeper := ownFindingsNow.Load(); keeper != nil {
keeper.Report(report, machines.ByAddress)
}
switch {
case err != nil && err.Error() != *unwritten:
*unwritten = err.Error()
+136 -9
View File
@@ -25,8 +25,10 @@ import (
"github.com/novox/mesh-host/internal/compose/frombus"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/liveness"
"github.com/novox/mesh-host/internal/seatwait"
"github.com/novox/mesh-host/internal/selfcompose"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
// The node agent's wiring (novox/hq ADR 0429 §4, ADR 0539 §9): the buckets it watches, the haves it writes, the backup
@@ -88,7 +90,7 @@ func (h sessionHaves) Read(ctx context.Context, module string) (agent.Have, bool
return agent.Have{}, false
}
return agent.Have{Module: module, FP: got.Fp, Composed: got.Composed, Phase: string(got.Phase), Detail: got.Detail,
Sealed: got.SealedAt}, true
Sealed: got.SealedAt, Wait: got.Wait}, true
}
// ReadErr is Read telling a have absent from one the bus could not read (agent.HavesReadErr).
@@ -106,7 +108,7 @@ func (h sessionHaves) ReadErr(ctx context.Context, module string) (agent.Have, b
return agent.Have{}, false, err
}
return agent.Have{Module: module, FP: got.Fp, Composed: got.Composed, Phase: string(got.Phase), Detail: got.Detail,
Sealed: got.SealedAt}, true, nil
Sealed: got.SealedAt, Wait: got.Wait}, true, nil
}
func (h sessionHaves) Write(ctx context.Context, x agent.Have) {
@@ -123,7 +125,7 @@ func (h sessionHaves) Write(ctx context.Context, x agent.Have) {
}
h.session.NoteModule(ctx, link.ModuleFact{Module: x.Module, FP: x.FP, Composed: x.Composed, Phase: phase,
Failed: phase == string(nodeengineclient.PhaseFailed), Detail: shorten(x.Detail), Judgement: x.Judgement, Waits: x.Waits,
BuildAlone: x.BuildAlone, Kept: x.Kept, Sealed: x.Sealed, PullWait: x.PullWait})
BuildAlone: x.BuildAlone, Kept: x.Kept, Sealed: x.Sealed, PullWait: x.PullWait, Wait: x.Wait})
}
func (h sessionHaves) Removed(ctx context.Context, module string) {
@@ -447,6 +449,7 @@ func nodeAgent(ctx context.Context, opts options, signer ed25519.PublicKey, trus
}
began := time.Now()
node := session.Membership.Node
var keptFetch atomic.Pointer[apply.KeptFetch] // the composer's, set once it exists
composedBucket, err := compositions.BucketFor(node)
if err != nil {
cancel()
@@ -549,6 +552,9 @@ func nodeAgent(ctx context.Context, opts options, signer ed25519.PublicKey, trus
Backoff: link.Backoff,
Apply: func(ctx context.Context, c compositions.Composed, others []compositions.Composed) agent.Result {
actx, done := applyContext(session.WithLink(ctx))
if kf := keptFetch.Load(); kf != nil {
actx = apply.WithKept(actx, *kf)
}
// While the grants part's list is followed, the controller's list in the bus module's composition is not
// written: the apply keeps the list on the machine (ADR 0547 clause 8, never both).
held, herr := users.Hold(c)
@@ -573,6 +579,9 @@ func nodeAgent(ctx context.Context, opts options, signer ed25519.PublicKey, trus
sayFacts(ctx, ag)
return res
},
Stop: func(ctx context.Context, module string) agent.Result {
return stopComposition(ctx, opts, module)
},
Remove: func(ctx context.Context, module string, others []compositions.Composed) agent.Result {
actx, done := applyContext(session.WithLink(ctx))
res := removeComposition(actx, opts, module, others, ag.Placed(module), sched, say)
@@ -591,14 +600,33 @@ func nodeAgent(ctx context.Context, opts options, signer ed25519.PublicKey, trus
own := &agentNow{}
// And the seat list it reads through the seat mesh-catalog, from its bootstrapped definitions until it has: the
// composer hands nothing new meanwhile, and the node says it has waited five minutes (novox/hq ADR 0636 §1).
seats, err := startSeatList(run, session, node, say)
// The finding writer is started first, so it holds what an earlier run left before the reader writes or clears any.
if session.Own == nil {
session.Own = link.NewOwnName()
}
findings := &lateFindings{own: session.Own, make: func(module string) (*findingsOfTheNode, error) {
return newFindingsOfTheNode(session, node, module, say)
}}
findings.Start(run, say)
seats, err := startSeatList(run, session, node, findings, say)
if err != nil {
// No reader is no gate that lets everything through: the node acts on nothing new, as one that waits does.
say(err.Error() + "; the node composes nothing new until it is started again and reads the list (novox/hq ADR 0636 §1)")
}
// The marks of the haves run off the composition pass, each bounded (see marker).
marks := newMarker(run)
heldFindings := &seatwait.HeldKeeper{Node: node, ObserverOf: findings.observer, Writer: findings, Ctx: run, Say: say}
// And what only this engine knows of itself, raised where it knows it (design 47 G A-197): its haves' failed phase, health
// and kept words as they are written, and each part of its health report at every look.
ownFindings := &seatwait.NodeKeeper{Node: node, ObserverOf: findings.observer, Writer: findings, Ctx: run, Say: say}
session.SetHaveSaid(ownFindings.Have, ownFindings.HaveGone)
ownFindingsNow.Store(ownFindings)
hooks := composerHooks{
entryNotFound: func(fp string, notFound bool) {
if err := seatwait.SayEntryNotFound(run, findings, node, findings.observer(), fp, notFound, time.Now); err != nil {
say("the finding that the catalog half holds no entry of " + fp + " cannot be kept: " + err.Error())
}
},
seatList: seats,
seatListFailed: err != nil,
running: func() []compositions.Composed {
@@ -626,6 +654,11 @@ func nodeAgent(ctx context.Context, opts options, signer ed25519.PublicKey, trus
// A module held alone marks that module, never the others (issue 1126); the marking never holds the pass.
marks.set("refused", func(ctx context.Context) { now.NotComposedEach(ctx, why, held) })
}
// And each module held says it as the finding `module-held`, cleared when it is composed again or no longer
// wanted here. A node refused whole (why) says nothing of its modules: their findings stand as they were.
if why == "" && run.Err() == nil {
marks.set("module-held", func(context.Context) { heldFindings.Reconcile(held) })
}
},
// And what it keeps of a module as it runs, for a manifest that does not say it yet: on the module's have (ADR 0601 §4).
kept: func(kept map[string][]string) {
@@ -656,19 +689,35 @@ func nodeAgent(ctx context.Context, opts options, signer ed25519.PublicKey, trus
// composition bucket) is never applied. While the seat's assignment holds no key for this node it composes
// nothing and keeps running what it runs; a assignment it cannot watch stops the agent, said, and it begins again.
composer := selfComposer(session, node, hooks, say)
machinesNow.Store(composer) // the machines a failing network part points at, by the assignments (issue 774)
composerNow.Store(composer) // what the verb compose asks a pass of, this run's alone
composer.AlsoEntryAgain = shadow.EntryAgain
composer.OnEngine = session.Own.Set // the first composing names the engine's own module on a first start (§527, §634)
machinesNow.Store(composer) // the machines a failing network part points at, by the assignments (issue 774)
composerNow.Store(composer) // what the verb compose asks a pass of, this run's alone
session.SetManifestOf(composer.ManifestOf) // the event running names the manifest of the build that runs
go followRunning(run, session, composer, say)
go followBuildEnds(run, session, composer, say)
apply.UseEngineModule(composer.EngineModule) // the guard of the engine's own trees fixes no name (§527)
env.Composed, env.SelfComposed, env.Policy, env.ComposedToday = composer, true, composer.Policy, composer.ComposedToday
env.Recovery = recoveryEnv(session, composer)
env.WhyUnmade, env.Waiting = composer.WhyUnmade, composer.Waiting
keptFetchFn := keptArtifactFetch(composer.KeptBuildOf, session.AskSeatOn, session.KeptObject)
keptFetch.Store(&keptFetchFn)
env.HoldsData = composer.HoldsData // what runs from the kept declaration says Data as its manifest does
session.NoteComposes(linked, nodeengineclient.ComposesOwn, "it composes its own modules, and applies no composition made elsewhere")
ag = agent.New(env)
own.set(ag)
current.set(ag)
// The node's own record is stated before the composer starts, from what the node knows of itself and with no bus (A-120).
sayFacts(run, nil)
err = ag.Run(run)
stopRun()
cancel()
current.set(nil)
composerNow.Store(nil)
ownFindingsNow.Store(nil)
session.SetHaveSaid(nil, nil)
session.SetManifestOf(nil)
apply.UseEngineModule(nil)
shadowNow.Store(nil)
if ctx.Err() != nil {
return
@@ -827,6 +876,30 @@ func removeComposition(ctx context.Context, opts options, module string, others
return agent.Result{}
}
// stopComposition stops a module's units and containers from the node's record and removes nothing (design 47 G A-98,
// §297): what the module kept stays, so a revert restores it before the reverted build starts.
func stopComposition(ctx context.Context, opts options, module string) agent.Result {
applying.Lock()
defer applying.Unlock()
unlock, err := store.Lock(opts.state, nil)
if err != nil {
return agent.Result{Failed: true, Detail: shorten(err.Error())}
}
defer unlock()
known, err := store.Load(opts.state)
if err != nil {
return agent.Result{Failed: true, Detail: shorten(err.Error())}
}
built, err := system.For(builtFor)
if err != nil {
return agent.Result{Failed: true, Detail: shorten(err.Error())}
}
if _, err := apply.StopModule(ctx, built, known, module, opts.run()); err != nil {
return agent.Result{Failed: true, Detail: shorten(err.Error())}
}
return agent.Result{}
}
// busyModules are the modules whose last apply left a container running because it said busy.
func busyModules() []string {
busyMu.Lock()
@@ -845,6 +918,10 @@ func busyModules() []string {
// (writeHealthReport); nil on a node that does not compose its own, whose parts then name no target.
var machinesNow atomic.Pointer[selfcompose.Composer]
// ownFindingsNow is the keeper of the findings this engine raises of itself in the run that composes the node's own now, for the
// health loop (writeHealthReport); nil while no run holds one.
var ownFindingsNow atomic.Pointer[seatwait.NodeKeeper]
// composerNow is the composer of the run that composes the node's own now, nil while the node applies the controller's
// compositions or no run is going: what the verb compose asks a pass of (issue 774).
var composerNow atomic.Pointer[selfcompose.Composer]
@@ -1052,7 +1129,9 @@ type composerHooks struct {
unseal func(string) ([]byte, error)
refused func(why string, held map[string]string)
kept func(kept map[string][]string)
held func(module string) (compositions.Seals, []string, error)
// entryNotFound is selfcompose.Env's EntryNotFound.
entryNotFound func(fp string, notFound bool)
held func(module string) (compositions.Seals, []string, error)
// vaultAuthority and certify are selfcompose.Env's VaultAuthority and Certify.
vaultAuthority func() string
certify func(ctx context.Context) error
@@ -1075,13 +1154,15 @@ func selfComposer(session *link.Session, node string, hooks composerHooks, say f
// And every composition it makes in its composition state composed-<node>, a record the provisioner, the assigner
// and the view read and nothing applies (novox/hq ADR 0593 §6).
env.ComposedState = link.EngineComposed{Session: session, Node: node}
// And each composition it writes there said as the seat's event composed, which the assigner waits on.
env.SayComposed = session.SayComposed
return selfcompose.New(env)
}
// composerEnv is what the node's composers read of the bus, the vault and the agent: its own composer and its answer
// to assignment candidates (selfcompose.Shadow) alike.
func composerEnv(session *link.Session, node string, say func(string), hooks composerHooks) selfcompose.Env {
env := selfcompose.Env{Node: node,
env := selfcompose.Env{Node: node, NodeRecord: session.OwnRecord,
Watch: func(ctx context.Context, bucket, pattern string) (<-chan agent.Update, error) {
if bucket == buckets.Assignments || bucket == buckets.ProposalCandidates {
// The seat's states are made when their holder starts: a node-engine that starts first waits for the
@@ -1109,12 +1190,58 @@ func composerEnv(session *link.Session, node string, say func(string), hooks com
Hear: session.Hear,
// And a moved module's templates archive, in the artifact store, as every archive is fetched (issue 1126).
Fetch: apply.Fetch,
Running: hooks.running, Unseal: hooks.unseal, Refused: hooks.refused, Held: hooks.held, Kept: hooks.kept,
Running: hooks.running, Unseal: hooks.unseal, Refused: hooks.refused, Held: hooks.held, Kept: hooks.kept, EntryNotFound: hooks.entryNotFound,
VaultAuthority: hooks.vaultAuthority, Certify: hooks.certify}
if hooks.seatList != nil {
env.SeatList, env.SeatListChanged = hooks.seatList.reader, hooks.seatList.changed
} else if hooks.seatListFailed {
env.SeatList = selfcompose.Unread{}
}
// And the shadow too: the catalog half read by the fingerprints its wants and runnings name, over its own calls of the seat mesh-catalog's
// verb entry on its link session (design 47 G §421): neither the indexer's nor the builder's states are read.
env.Entry = selfcompose.EntryCall(session.AskMeshSeat)
return env
}
// followRunning keeps the composer told what every node runs, from the seat node-engine's event running, for the run it is
// in: the stream or the link not being there is said and tried again after a backoff, never polled.
func followRunning(run context.Context, session *link.Session, composer *selfcompose.Composer, say func(string)) {
keepFollowing(run, "this node does not hear what the nodes run, and tries again: ", func(ctx context.Context) error {
return session.FollowRunning(ctx, composer.NoteRunning, composer.RunningRead)
}, composer.RunningLost, say)
}
// followBuildEnds keeps the composer told how the builder's builds ended, from its event build.ended, for the run it is in
// (design 47 G A-293): a placed module's newest build that ended failed is said as a refusal in the composition state.
func followBuildEnds(run context.Context, session *link.Session, composer *selfcompose.Composer, say func(string)) {
keepFollowing(run, "this node does not hear how the builder's builds end, so a failed build is not said as refused, and it tries again: ",
func(ctx context.Context) error {
return session.FollowBuildEnds(ctx, composer.NoteBuildEnded, composer.BuildEndsRead)
}, composer.BuildEndsLost, say)
}
// keepFollowing follows a stream with follow for the run, lost called each time it ends; why it ended is said once until it
// changes, and it is followed again after a backoff, never polled.
func keepFollowing(run context.Context, line string, follow func(context.Context) error, lost func(), say func(string)) {
said := ""
for try := 1; run.Err() == nil; try++ {
began := time.Now()
err := follow(run)
lost()
if run.Err() != nil {
return
}
if time.Since(began) > time.Minute {
try = 1
}
if why := fmt.Sprint(err); why != said {
say(line + why)
said = why
}
select {
case <-run.Done():
return
case <-time.After(link.Backoff(try)):
}
}
}
+212 -41
View File
@@ -7,19 +7,21 @@ import (
"crypto/ed25519"
"encoding/json"
"errors"
"fmt"
"git.novox.be/novox/mesh-contracts/go/clients/meshmoduleassignerclient"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
nodeengine "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliveryclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshwants"
builderbuilds "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds"
condseat "git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshconditions"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshdeliveryclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants"
"github.com/novox/mesh-host/internal/buckets"
"github.com/novox/mesh-host/internal/buckets/bucketstest"
engcomp "github.com/novox/mesh-host/internal/compositions"
"os"
"path/filepath"
"sort"
"strings"
"sync"
"sync/atomic"
@@ -72,7 +74,59 @@ type itBed struct {
seatSilent atomic.Bool
// seatAsked counts the node's asks of the stand-in seat, answered or refused: the node reached the bus and asked.
seatAsked atomic.Int32
findings nats.KeyValue
// seatAway makes the stand-in answer as the bus answers a subject nothing serves (no responders), as while the seat's
// holder re-serves (issue 1966).
seatAway atomic.Bool
findings nats.KeyValue
// entries are the entries the stand-in of the seat mesh-catalog answers its verb entry from, by fingerprint (design 47 G §421).
entries map[string]meshcatalog.CatalogEntry
}
// registerEntry makes the stand-in answer the entry of fp: module's manifest and its artifacts (refs `<module>:<name>`).
func (b *itBed) registerEntry(module, fp, manifest string, artifacts ...string) {
b.t.Helper()
var m map[string]any
if err := json.Unmarshal([]byte(manifest), &m); err != nil {
b.t.Fatal(err)
}
e := meshcatalog.CatalogEntry{Module: module, Fp: fp, Manifest: m, ManifestDigest: meshcatalogclient.ManifestDigest([]byte(manifest)),
Repository: "novox/" + module, Commit: strings.Repeat("1", 40)}
for _, a := range artifacts {
e.Artifacts = append(e.Artifacts, meshcatalog.Artifact{Ref: module + ":" + a, Kind: "archive", Language: "go"})
}
b.mu.Lock()
b.entries[fp] = e
b.mu.Unlock()
}
// running says the seat node-engine's event running of module on node at fp, as that node's engine does.
func (b *itBed) running(node, module, fp, manifestDigest string) {
b.t.Helper()
raw, _ := json.Marshal(nodeengine.Running{V: 1, Node: node, Module: module, Fingerprint: fp, Manifest: manifestDigest, At: time.Now().UTC()})
if _, err := b.js.Publish(nodeengine.SubjectRunning(node, module), raw); err != nil {
b.t.Fatal(err)
}
}
// standInEntry answers the seat mesh-catalog's verb entry from the registered entries, and as the holder refuses an unknown one.
func (b *itBed) standInEntry() {
sub, err := b.conn.Subscribe("mesh.seat.mesh-catalog.call.entry.*", func(m *nats.Msg) {
var in meshcatalog.EntryInput
_ = json.Unmarshal(m.Data, &in)
b.mu.Lock()
e, ok := b.entries[in.Fingerprint]
b.mu.Unlock()
if !ok {
_ = m.Respond([]byte(`{"error":"no entry of fingerprint ` + in.Fingerprint + ` is kept"}`))
return
}
out, _ := json.Marshal(map[string]any{"result": meshcatalog.EntryAnswer{Entry: e}})
_ = m.Respond(out)
})
if err != nil {
b.t.Fatal(err)
}
b.t.Cleanup(func() { _ = sub.Unsubscribe() })
}
// aServer is an embedded bus with JetStream and two users: the test's, unrestricted, and the node's, granted exactly what
@@ -125,7 +179,7 @@ func newItBedAs(t *testing.T, perms nodegrant.Permissions) *itBed {
builtFor = "arch"
t.Cleanup(func() { builtFor = was })
b := &itBed{t: t, dir: t.TempDir()}
b := &itBed{t: t, dir: t.TempDir(), entries: map[string]meshcatalog.CatalogEntry{}}
b.conn, b.nodeConn = aServer(t, perms, func(err error) {
b.mu.Lock()
b.refusals = append(b.refusals, err.Error())
@@ -154,7 +208,20 @@ func newItBedAs(t *testing.T, perms nodegrant.Permissions) *itBed {
if b.findings, err = b.js.CreateKeyValue(&nats.KeyValueConfig{Bucket: link.FindingsBucket, History: 1}); err != nil {
t.Fatal(err)
}
// The engine's own finding stream and the keeper's ask stream (design 47 G §383, §390), made as the bus module makes them.
for _, c := range []nats.StreamConfig{{Name: "MESH-HOST-FINDINGS", Subjects: []string{"mesh.stream.mesh-host.event.>"}},
{Name: condseat.StreamFindingsAsked, Subjects: []string{condseat.SubjectFindingsAsked}}} {
if _, err = b.js.AddStream(&c); err != nil {
t.Fatal(err)
}
}
// The stream the seat node-engine's event running is kept in, the last of each subject (design 47 G §199).
if _, err = b.js.AddStream(&nats.StreamConfig{Name: nodeengine.StreamRunning, Subjects: []string{nodeengine.SubjectRunningAll},
MaxMsgsPerSubject: 1}); err != nil {
t.Fatal(err)
}
b.standInSeat()
b.standInEntry()
b.opts = options{state: filepath.Join(b.dir, "state", "state.json"), timeout: 2 * time.Second}
if err := os.MkdirAll(filepath.Dir(b.opts.state), 0o755); err != nil {
t.Fatal(err)
@@ -163,10 +230,19 @@ func newItBedAs(t *testing.T, perms nodegrant.Permissions) *itBed {
return b
}
// itOwnModule is the node-engine's own module of the bed: what its applied record of its own build says (§527, §634).
const itOwnModule = "mesh-host"
func itOwn() *link.OwnName {
o := link.NewOwnName()
o.Set(itOwnModule)
return o
}
// start runs the node agent under a session on the bus.
func (b *itBed) start() (stop func()) {
b.t.Helper()
b.session = &link.Session{Membership: link.Membership{Node: itNode}, Timeout: 2 * time.Second,
b.session = &link.Session{Own: itOwn(), Membership: link.Membership{Node: itNode}, Timeout: 2 * time.Second,
Say: b.say, Keys: func() link.RecordKeys { return link.RecordKeys{} }}
b.session.Attach(context.Background(), link.OverNATS{Conn: b.nodeConn, JS: b.nodeJS, Node: itNode})
ctx, cancel := context.WithCancel(context.Background())
@@ -414,21 +490,26 @@ func h8(t *testing.T, b *itBed, checkBuild bool) (func(), h8state) {
}
return kv
}
assignment, modules, manifests := kvOf(buckets.Assignments), kvOf(buckets.Modules), kvOf(buckets.Manifests)
assignment, manifests := kvOf(buckets.Assignments), kvOf(buckets.Manifests)
resolved := kvOf(selfcompose.ResolvedBucket)
builds := kvOf(builderbuilds.BuilderBuildsBucket)
candidates := kvOf(buckets.ProposalCandidates)
packages := kvOf(link.PackagesBucket)
path := filepath.Join(b.dir, "web.conf")
man := `{"module":"web","version":"1","config":{"greeting":{"type":"string","description":"what the file says","default":"hello"}},` +
// Beside the greeting: a file under `if-config`, a template with `config-lines` from a list, and two files under a
// value that makes the first of them fail (a file below a file), the second being left (design 42 zfs-owns-zfs §5).
more := func(name string) string { return filepath.Join(b.dir, "web-"+name+".conf") }
man := `{"module":"web","version":"1","config":{"greeting":{"type":"string","description":"what the file says","default":"hello"},` +
`"mods":{"type":"list","description":"lines","default":["a","b"]}},` +
`"resources":[{"id":"conf","type":"file","path":"` + path + `","mode":"0644","content":"greeting=${config:greeting}\n"},` +
`{"id":"tools","type":"package","package":"web-tools"}],` +
`"build":{"artifacts":[{"name":"files","kind":"archive","from":"files"}]}}`
`{"id":"mods","type":"file","path":"` + more("mods") + `","mode":"0644","content":"${config-lines:mods:m=}\n"},` +
`{"id":"salut","type":"file","path":"` + more("salut") + `","mode":"0644","content":"salut\n","if-config":["greeting=salut"]},` +
`{"id":"bad","type":"file","path":"` + path + `/x","mode":"0644","content":"x\n","if-config":["greeting=boom"]},` +
`{"id":"after","type":"file","path":"` + more("after") + `","mode":"0644","content":"after\n","if-config":["greeting=boom"]},` +
`{"id":"tools","type":"package","package":"web-tools"}]}`
digest := meshcatalogclient.ManifestDigest([]byte(man))
mraw, _ := meshcatalogclient.EncodeManifest(meshcatalog.ManifestsByDigestValue{Module: "web", Manifest: man})
mkey, _ := meshcatalogclient.ManifestsKey(digest)
fp := fpIt("a")
modv, _ := meshcatalogclient.EncodeModule(meshcatalog.ModulesByModuleValue{Fp2: fp, Manifest: digest, Repository: "novox/web", Commit: strings.Repeat("1", 40)})
place, _ := meshmoduleassignerclient.EncodeAssignments(meshmoduleassigner.Assignments{Node: itNode, Modules: []meshmoduleassigner.PlacedModule{{Module: "web"}}})
rec, _ := nec.EncodeHaveNode(nodeengine.HaveByNodeNodeValue{Record: &nodeengine.NodeRecord{Node: itNode, Capabilities: []string{"service-manager", "package-manager"}}})
// it1 in one key per module and node (research 072 row 37), it2 in the form before: a node reads both.
@@ -442,30 +523,29 @@ func h8(t *testing.T, b *itBed, checkBuild bool) (func(), h8state) {
mdigest := meshcatalogclient.ManifestDigest([]byte(mailer))
mlraw, _ := meshcatalogclient.EncodeManifest(meshcatalog.ManifestsByDigestValue{Module: "mailer", Manifest: mailer})
mlkey, _ := meshcatalogclient.ManifestsKey(mdigest)
mlmod, _ := meshcatalogclient.EncodeModule(meshcatalog.ModulesByModuleValue{Fp2: fpIt("b"), Manifest: mdigest, Repository: "novox/mailer", Commit: strings.Repeat("2", 40)})
places2, _ := meshmoduleassignerclient.AssignmentsValues(meshmoduleassigner.Assignments{Node: "it2", Modules: []meshmoduleassigner.PlacedModule{{Module: "mailer"}}})
rec2, _ := nec.EncodeHaveNode(nodeengine.HaveByNodeNodeValue{Record: &nodeengine.NodeRecord{Node: "it2", Capabilities: []string{"service-manager"}}})
pkey2, _ := meshmoduleassignerclient.NodeKey("it2")
hkey2, _ := nec.HaveNodeKey("it2")
buildKey, _ := builderbuilds.BuilderBuildKey("web", fp)
done, _ := builderbuilds.EncodeBuilderBuild(builderbuilds.BuilderBuild{State: builderbuilds.BuildDone, Manifest: digest,
Artifacts: []builderbuilds.BuiltArtifact{{Name: "files", Kind: "archive", Reference: "artifact-store://web/files/blobs/" + fpDigest("f"), Digest: fpDigest("f")}}})
firstBuild := done
if checkBuild {
firstBuild = []byte(`{"v":1,"state":"no such state"}`)
}
for _, w := range []struct {
kv nats.KeyValue
key string
value []byte
}{{manifests, mkey, mraw}, {modules, "web", modv}, {b.haves, hkey, rec},
}{{manifests, mkey, mraw}, {b.haves, hkey, rec},
{resolved, itNode + ".web.greeting", []byte(`{"value":"bonjour","source":"module"}`)},
{manifests, mlkey, mlraw}, {modules, "mailer", mlmod}, {b.haves, hkey2, rec2}, {assignment, pkey2, places2[pkey2]}, {assignment, "it2.mailer", places2["it2.mailer"]},
{resolved, "it2.mailer.domain", []byte(`{"value":"example.org","source":"module"}`)}, {builds, buildKey, firstBuild}} {
{manifests, mlkey, mlraw}, {b.haves, hkey2, rec2}, {assignment, pkey2, places2[pkey2]}, {assignment, "it2.mailer", places2["it2.mailer"]},
{resolved, "it2.mailer.domain", []byte(`{"value":"example.org","source":"module"}`)}} {
if _, err := w.kv.Put(w.key, w.value); err != nil {
t.Fatal(err)
}
}
// Design 47 G §421: the node reads the entry its want names, through the seat mesh-catalog's verb entry, and the part's digest
// from the want; what it hears another node run (the mailer on it2) is read by the same fingerprint.
if !checkBuild {
b.registerEntry("web", fp, man)
}
b.registerEntry("mailer", fpIt("b"), mailer)
b.running("it2", "mailer", fpIt("b"), mdigest)
want, _ := bucketstest.Want(meshwants.WantsByWantNodeModuleValue{Fp: fp, Self: true})
wkey := meshwants.WantsByWantNodeModuleKey(itNode, "web")
if _, err := b.wants.Put(wkey, want); err != nil {
@@ -486,17 +566,18 @@ func h8(t *testing.T, b *itBed, checkBuild bool) (func(), h8state) {
t.Fatal(err)
}
if checkBuild {
// The build of the placed module is read through the node's session: one that does not decode is said, and nothing
// is applied; a done one, then a config event, and the node composes.
within(t, 5*time.Second, "the node did not read web's build key through its session", func() bool {
b.mu.Lock()
defer b.mu.Unlock()
return strings.Contains(strings.Join(b.said, "\n"), "the builder's "+buildKey)
// The entry of the build its want names is asked of the catalog half through the node's session: one the half holds none
// of holds the want (entry-missing, naming the fingerprint) and nothing is applied; once the half holds it and it is
// asked again by hand, then a config event, and the node composes.
within(t, 5*time.Second, "the have does not say the entry of web's build is missing", func() bool {
h := b.have("web")
return h.Phase == nec.PhaseHeld && strings.HasPrefix(h.Detail, agent.EntryMissing) && strings.Contains(h.Detail, fp)
})
if b.file("web") != "" {
t.Fatal("composed with a build key that does not decode")
t.Fatal("composed with an entry the catalog half does not hold")
}
if _, err := builds.Put(buildKey, done); err != nil {
b.registerEntry("web", fp, man)
if err := composerNow.Load().EntryAgain(context.Background(), fp); err != nil {
t.Fatal(err)
}
if _, err := resolved.Put(itNode+".web.greeting", []byte(`{"value":"bonjour","source":"module"}`)); err != nil {
@@ -533,6 +614,36 @@ func h8rest(t *testing.T, b *itBed, resolved, candidates nats.KeyValue, hkey str
t.Fatal(err)
}
within(t, 3*time.Second, "a changed config value was not composed and applied again", func() bool { return b.file("web") == "greeting=salut\n" })
// Design 42 zfs-owns-zfs §5: the file under `if-config` appears with its value and goes with it, the template's lines
// follow the list, and a failing resource stops its module's apply with the later resource left.
other := func(name string) string {
got, _ := os.ReadFile(filepath.Join(b.dir, "web-"+name+".conf"))
return string(got)
}
setGreeting := func(value string) {
if _, err := resolved.Put(itNode+".web.greeting", []byte(`{"value":"`+value+`","source":"module"}`)); err != nil {
t.Fatal(err)
}
}
within(t, 3*time.Second, "the file under if-config did not appear with its value, or the lines do not follow the list", func() bool {
return other("salut") == "salut\n" && other("mods") == "m=a\nm=b\n"
})
setGreeting("bonjour")
within(t, 3*time.Second, "the file under if-config did not go with its value", func() bool {
_, err := os.Stat(filepath.Join(b.dir, "web-salut.conf"))
return b.file("web") == "greeting=bonjour\n" && errors.Is(err, os.ErrNotExist)
})
setGreeting("boom")
within(t, 5*time.Second, "the failing resource did not fail its module", func() bool {
return b.have("web").Phase == nec.PhaseFailed
})
if got := other("after"); got != "" {
t.Errorf("a resource after the failed one was applied: %q", got)
}
setGreeting("salut")
within(t, 5*time.Second, "the module did not run again once its value held no failing resource", func() bool {
return b.file("web") == "greeting=salut\n" && b.have("web").Phase == nec.PhaseRunning
})
within(t, 2*time.Second, "the have does not name the revision", func() bool {
h := b.have("web")
return h.Fp == fp && h.Composed != first && h.Phase == nec.PhaseRunning
@@ -741,7 +852,10 @@ func TestNatsTheBusServerIsTakenOverAndReplacedOnlyWhenItsBuildChangesIt(t *test
}
folded, err := declaration.Parse([]byte(`{"declaration":1,"resources":[` +
`{"id":"nats.server-conf","type":"file","path":"` + conf + `","mode":"0644","content":"listen: old\n"},` +
`{"id":"nats.server","type":"container","name":"` + server + `","image":"` + image + `"}]}`))
`{"id":"nats.server","type":"container","name":"` + server + `","image":"` + image + `",` +
// As the node-engine starts every container since the fingerprint handover (47/G W13.1): the fingerprint of the
// want it applies, here the first want's, so that want changes the server's spec in nothing.
`"env":{"MESH_FINGERPRINT":"` + fpIt("e") + `"}}]}`))
if err != nil {
t.Fatal(err)
}
@@ -770,7 +884,6 @@ func TestNatsTheBusServerIsTakenOverAndReplacedOnlyWhenItsBuildChangesIt(t *test
}
assignment, modules, manifests := kvOf(buckets.Assignments), kvOf(buckets.Modules), kvOf(buckets.Manifests)
kvOf(selfcompose.ResolvedBucket)
builds := kvOf(builderbuilds.BuilderBuildsBucket)
kvOf(buckets.ProposalCandidates)
kvOf(link.PackagesBucket)
// The bus module at a build, done: its manifest written and the indexer's module naming it at fp.
@@ -783,14 +896,10 @@ func TestNatsTheBusServerIsTakenOverAndReplacedOnlyWhenItsBuildChangesIt(t *test
mraw, _ := meshcatalogclient.EncodeManifest(meshcatalog.ManifestsByDigestValue{Module: "nats", Manifest: man})
mkey, _ := meshcatalogclient.ManifestsKey(digest)
modv, _ := meshcatalogclient.EncodeModule(meshcatalog.ModulesByModuleValue{Fp2: fp, Manifest: digest, Repository: "novox/mesh-catalog", Commit: strings.Repeat("3", 40)})
b.registerEntry("nats", fp, man) // the node reads the entry its want names (design 47 G §421)
if _, err := manifests.Put(mkey, mraw); err != nil {
t.Fatal(err)
}
bkey, _ := builderbuilds.BuilderBuildKey("nats", fp)
done, _ := builderbuilds.EncodeBuilderBuild(builderbuilds.BuilderBuild{State: builderbuilds.BuildDone, Manifest: digest})
if _, err := builds.Put(bkey, done); err != nil {
t.Fatal(err)
}
if _, err := modules.Put("nats", modv); err != nil {
t.Fatal(err)
}
@@ -918,6 +1027,10 @@ func (b *itBed) standInSeat() {
}
_ = json.Unmarshal(m.Data, &q)
b.seatAsked.Add(1)
if b.seatAway.Load() {
_ = m.RespondMsg(&nats.Msg{Header: nats.Header{"Status": []string{"503"}}}) // the bus's no responders
return
}
var result any
switch {
case b.seatSilent.Load():
@@ -972,6 +1085,11 @@ func (c *fiveMinutes) pass() {
}
}
// findingSubject is the subject of one of the engine's finding events on node (views.FindingSubjects).
func findingSubject(event, node string) string {
return "mesh.stream." + itOwnModule + ".event." + event + "." + node
}
// ADR 0636 §1, §8, whole against a real bus and the stand-in of the seat: while the seat refuses, the node composes and
// applies nothing new, and keeps running what it ran; after the bound it writes one finding in its state findings; when
// the seat names its list the module is applied and the finding is cleared; a node started again with the seat gone
@@ -1002,7 +1120,14 @@ func TestNatsANodeThatCannotReadTheSeatListActsOnNothingNew(t *testing.T) {
b.mu.Lock()
defer b.mu.Unlock()
said := strings.Join(b.said, "\n")
return strings.Contains(said, selfcompose.WaitsForTheSeatListOwn) && strings.Contains(said, selfcompose.WaitsForTheSeatList)
// web's reason names the bootstrapped seats the node holds (issue 1966).
var boot []string
for _, d := range seatlist.Bootstrapped() {
boot = append(boot, d.Seat)
}
sort.Strings(boot)
return strings.Contains(said, selfcompose.WaitsForTheSeatListOwn) &&
strings.Contains(said, "web: "+fmt.Sprintf(selfcompose.WaitsForTheSeatListHeld, len(boot), strings.Join(boot, ", ")))
})
if b.file("web") != "" {
t.Fatalf("the node applied web before it read the seat list: %q", b.file("web"))
@@ -1018,6 +1143,27 @@ func TestNatsANodeThatCannotReadTheSeatListActsOnNothingNew(t *testing.T) {
if b.file("web") != "" {
t.Fatalf("the node applied web with its finding written: %q", b.file("web"))
}
// Design 47 G §383, §384, §390 (test 107): the key is followed by the event on the engine's own stream, the engine stated
// its findings at its start, and an ask of the keeper is answered by a new statement.
lastEvent := func(event string) (string, bool) {
m, err := b.js.GetLastMsg("MESH-HOST-FINDINGS", findingSubject(event, itNode))
if err != nil {
return "", false
}
return string(m.Data), true
}
within(t, 5*time.Second, "the node did not say finding-written after its key", func() bool {
got, ok := lastEvent("finding-written")
return ok && strings.Contains(got, `"kind":"seat-list-unread"`) && strings.Contains(got, `"observer":"mesh-host"`)
})
within(t, 5*time.Second, "the node did not state its findings at its start", func() bool { _, ok := lastEvent("findings-stated"); return ok })
if _, err := b.js.Publish(condseat.SubjectFindingsAsked, []byte(`{"ask":"a1","at":"2026-11-11T12:00:00Z","by":"test"}`)); err != nil {
t.Fatal(err)
}
within(t, 5*time.Second, "the node did not answer the keeper's ask with a statement naming the finding", func() bool {
got, ok := lastEvent("findings-stated")
return ok && strings.Contains(got, `"asked":"a1"`) && strings.Contains(got, `"kind":"seat-list-unread"`)
})
b.seatSilent.Store(false)
// The hand act of the operator and the healers (the verb seat-list-again): a read now, as the seat's event would make.
@@ -1029,6 +1175,15 @@ func TestNatsANodeThatCannotReadTheSeatListActsOnNothingNew(t *testing.T) {
_, err := b.findings.Get(finding)
return errors.Is(err, nats.ErrKeyNotFound)
})
within(t, 5*time.Second, "the node did not say finding-cleared for the seat list", func() bool {
for i := uint64(1); i < 40; i++ {
if m, err := b.js.GetMsg("MESH-HOST-FINDINGS", i); err == nil && m.Subject == findingSubject("finding-cleared", itNode) &&
strings.Contains(string(m.Data), `"kind":"seat-list-unread"`) {
return true
}
}
return false
})
// Started again with the seat refusing: nothing new is composed, and what ran is neither removed nor changed.
stop()
@@ -1047,6 +1202,10 @@ func TestNatsANodeThatCannotReadTheSeatListActsOnNothingNew(t *testing.T) {
// The first read of the seat list ends the wait whether or not it changed what the reader held: Start returns at once, and
// the node waits on the reader's Ready before it says the read is over (ADR 0636 §1).
//
// Issue 1966, the live case of ace: the seat's holder re-serves while the node starts, so the node's reader is answered as
// nothing serves the seat (no responders); once the holder serves again the reader reads the list within the bus's retry
// bound, with no event of the seat and no hand act.
func TestNatsTheFirstReadOfTheSeatListEndsTheNodesWait(t *testing.T) {
clock := &fiveMinutes{}
was := seatWaitClock
@@ -1054,8 +1213,17 @@ func TestNatsTheFirstReadOfTheSeatListEndsTheNodesWait(t *testing.T) {
t.Cleanup(func() { seatWaitClock = was })
b := newItBedAs(t, nodegrant.For(itNode, "it2.mailer.domain"))
b.seatAway.Store(true)
stop, _ := h8(t, b, false)
defer stop()
var st *seatListState
within(t, 5*time.Second, "the node started no reader of the seat list", func() bool { st = seatListNow.Load(); return st != nil })
asked := b.seatAsked.Load()
within(t, 10*time.Second, "the node's reader did not ask the seat while nothing served it", func() bool {
now := st.reader.Status()
return b.seatAsked.Load() > asked || (!now.Reading && !now.Heard)
})
b.seatAway.Store(false)
within(t, 30*time.Second, "the node did not end its wait at the first read of the seat list", func() bool {
st := seatListNow.Load()
if st == nil {
@@ -1065,4 +1233,7 @@ func TestNatsTheFirstReadOfTheSeatListEndsTheNodesWait(t *testing.T) {
waiting, _ := st.waiter.Waiting()
return st.reader.Status().Heard && !waiting
})
if seatListNow.Load() != st {
t.Error("the list was read by the reader of a run begun since, not by the one that met no responders")
}
}
+77
View File
@@ -0,0 +1,77 @@
package main
import (
"context"
"os"
"path/filepath"
"strings"
"testing"
"time"
condseat "git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshconditions"
"github.com/novox/mesh-host/internal/link"
)
// Design 47 G §527, §634, A-149: the node-engine's own module is the module of its applied record at its own fingerprint,
// and its finding writer is made, observing as that module, once the name is known: at once from the record, else when the
// first composing names it. No name is fixed in code.
func TestTheEnginesOwnNameIsReadFromItsAppliedRecordOrItsFirstComposing(t *testing.T) {
fp := "src2:" + strings.Repeat("a", 64)
applied := appliedStoreFor(filepath.Join(t.TempDir(), "state", "state.json"))
for module, f := range map[string]string{"node": "", "the-engine": fp, "other": "src2:" + strings.Repeat("b", 64)} {
if err := applied.Keep(module, []byte(`{"module":"`+module+`","fp":"`+f+`"}`)); err != nil {
t.Fatal(err)
}
}
for _, tc := range []struct{ fp, want string }{{fp, "the-engine"}, {"", ""}, {"src2:" + strings.Repeat("c", 64), ""}} {
if got := ownModuleApplied(applied, tc.fp); got != tc.want {
t.Errorf("the own module at %q is %q, want %q", tc.fp, got, tc.want)
}
}
own := link.NewOwnName()
made := make(chan string, 1)
late := &lateFindings{own: own, make: func(module string) (*findingsOfTheNode, error) {
made <- module
return nil, context.Canceled // the writer itself is the SDK's: only when it is made is tested here
}}
ctx, stop := context.WithCancel(context.Background())
defer stop()
var said []string
late.Start(ctx, func(s string) { said = append(said, s) })
if err := late.Write(ctx, condseat.Finding{}); err == nil || late.Held() != nil {
t.Fatalf("a finding was taken before the engine's own module was known: %v", err)
}
if len(made) != 0 {
t.Fatal("the writer was made before a name")
}
own.Set("the-engine") // the first composing names it
select {
case m := <-made:
if m != "the-engine" || late.observer() != "the-engine" {
t.Fatalf("the writer was made as %q, observing as %q", m, late.observer())
}
case <-time.After(5 * time.Second):
t.Fatal("the writer was never made after the name was known")
}
}
// §633: the engine's bound for the first seat list is the file the manifest's `view-waits-minutes` feeds, five minutes
// when it holds nothing usable.
func TestTheViewWaitsBoundIsTheManifestsSetting(t *testing.T) {
was := viewWaitsFile
defer func() { viewWaitsFile = was }()
for content, want := range map[string]time.Duration{"2\n": 2 * time.Minute, "": 5 * time.Minute, "0": 5 * time.Minute, "x": 5 * time.Minute} {
viewWaitsFile = filepath.Join(t.TempDir(), "w")
if err := os.WriteFile(viewWaitsFile, []byte(content), 0o644); err != nil {
t.Fatal(err)
}
if got := viewWaitsBound(); got != want {
t.Errorf("the file %q gives %v, want %v", content, got, want)
}
}
viewWaitsFile = filepath.Join(t.TempDir(), "missing")
if got := viewWaitsBound(); got != 5*time.Minute {
t.Errorf("no file gives %v", got)
}
}
+5 -4
View File
@@ -88,11 +88,12 @@ func TestThePackagesGoInWhenTheApplyThatRestoresThePackageManagerWroteIt(t *test
conf := filepath.Join(dir, "pacman.conf")
pacmanAnsweringOnce(t, conf)
opts := options{state: filepath.Join(dir, "state.json")}
// The package first: within one pass it is asked about before the file is written, which is the order that
// leaves it to a second pass.
// The file first, as the pacman module declares it: a module's apply stops at its first failed resource
// (design 42 zfs-owns-zfs §1, answer A-26), so the file that makes the package manager answer comes before the
// package that needs it.
raw := []byte(`{"declaration":1,"resources":[` +
`{"id":"pacman.base","type":"package","package":"base"},` +
`{"id":"pacman.conf","type":"file","path":"` + conf + `","content":"[options]\n"}]}`)
`{"id":"pacman.conf","type":"file","path":"` + conf + `","content":"[options]\n"},` +
`{"id":"pacman.base","type":"package","package":"base"}]}`)
report := applyAndKeep(context.Background(), opts, raw, nil, nil, nil)
+89
View File
@@ -0,0 +1,89 @@
package main
import (
"context"
"encoding/json"
"os"
"strconv"
"strings"
"time"
"git.novox.be/novox/mesh-contracts/go/seats/meshrecovery"
"github.com/novox/mesh-host/internal/agent"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/selfcompose"
)
// recoveryEnv is what the node agent asks of the recovery module and the manifests to move a module's kept data (design 47 G §296,
// §297, §554, §560, §630, A-77, A-98, A-108): the seat's verbs asked in the node-engine's own name, the endings heard from the
// seat's events, the bound of a wait kept as a key of the state bounds, the catch-up called as the module's tool.
func recoveryEnv(session *link.Session, composer *selfcompose.Composer) *agent.Recovery {
ask := func(ctx context.Context, verb string, in, out any) error {
raw, err := session.AskMeshSeat(ctx, meshrecovery.Seat, verb, in)
if err != nil {
return err
}
return json.Unmarshal(raw, out)
}
return &agent.Recovery{
FormChanges: composer.FormChanges,
SnapshotKept: func(ctx context.Context, in meshrecovery.SnapshotKeptInput) (a meshrecovery.SnapshotKeptAnswer, err error) {
return a, ask(ctx, meshrecovery.VerbSnapshotKept, in, &a)
},
RestoreKept: func(ctx context.Context, in meshrecovery.RestoreKeptInput) (a meshrecovery.RestoreKeptAnswer, err error) {
return a, ask(ctx, meshrecovery.VerbRestoreKept, in, &a)
},
KeptPoints: func(ctx context.Context, in meshrecovery.KeptPointsInput) (a meshrecovery.KeptPointsAnswer, err error) {
return a, ask(ctx, meshrecovery.VerbKeptPoints, in, &a)
},
Hear: func(_ context.Context, kind, module, kept string) (<-chan agent.Ending, func(), error) {
heard, stop, err := session.HearRecoveryEnding(kind, module, kept)
if err != nil {
return nil, nil, err
}
out := make(chan agent.Ending, 64)
done := make(chan struct{})
go func() {
for {
select {
case ev := <-heard:
select {
case out <- agent.Ending(ev):
case <-done:
return
}
case <-done:
return
}
}
}()
return out, func() { stop(); close(done) }, nil
},
Bound: func(ctx context.Context, what, id, module string, since time.Time, d time.Duration) (<-chan struct{}, func(), error) {
return session.KeepBound(ctx, what, id, module, since, d)
},
CatchUp: func(ctx context.Context, module string, c meshrecovery.CatchUp) error {
return session.CallTool(ctx, module, c.Tool, c.Arguments)
},
WaitMinutes: recoveryWaitMinutes(),
}
}
// recoveryWaitFile is the file the manifest declares with the content ${config:recovery-wait-minutes} (design 47 G §633, A-112): the
// node-engine is no bundle under a runtime, so it reads the setting from its file at its start, as the builder does (A-71).
var recoveryWaitFile = "/var/lib/mesh-host/recovery-wait-minutes"
// recoveryWaitMinutes is the bound of the wait for the recovery module, in minutes: the file's number, the design's 90 when the
// file is missing, empty or not a positive number (design 47 G §554, A-112).
func recoveryWaitMinutes() int {
raw, err := os.ReadFile(recoveryWaitFile)
if err != nil {
return agent.RecoveryWaitMinutes
}
n, err := strconv.Atoi(strings.TrimSpace(string(raw)))
if err != nil || n <= 0 {
return agent.RecoveryWaitMinutes
}
return n
}
+301
View File
@@ -0,0 +1,301 @@
//go:build integration
package main
import (
"context"
"encoding/json"
"os"
"path/filepath"
"testing"
"time"
"fmt"
"git.novox.be/novox/mesh-contracts/go/clients/meshmoduleassignerclient"
nec "git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
"git.novox.be/novox/mesh-contracts/go/seats/meshrecovery"
nodeengine "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"github.com/novox/mesh-host/internal/buckets"
"github.com/novox/mesh-host/internal/link"
"github.com/novox/mesh-host/internal/nodegrant"
"github.com/novox/mesh-host/internal/selfcompose"
"strings"
"sync"
"sync/atomic"
)
// Design 47 G §560, §630, A-77, A-92, A-108: the node agent's recovery step over a real bus. Its session asks the seat's verbs, hears
// the last `restored` already said and the next, keeps a wait's bound as a key of the state bounds whose time ends with the bus's
// marker, and calls the catch-up as the module's tool with its arguments. The recovery module is a stand-in that answers the verbs and
// says the events; the grants part composes the node's subjects from the manifest, so the test adds them to the node's grant.
func TestNatsTheRecoveryStepAsksHearsBoundsAndCatchesUpOverTheBus(t *testing.T) {
perms := nodegrant.For(itNode)
perms.Publish = append(perms.Publish, "$JS.API.>", "$KV.mesh-host_bounds.>", "mesh.seat.mesh-recovery.call.*.node~"+itNode,
"mesh.mod.db.tool.rebuild."+itNode)
perms.Subscribe = append(perms.Subscribe, "mesh.seat.mesh-recovery.event.>", "$KV.mesh-host_bounds.>", "_INBOX.>", "$JS.ACK.>", "$JS.FC.>")
b := newItBedAs(t, perms)
ctx := context.Background()
for _, s := range []struct{ stream, subject string }{{meshrecovery.StreamEnded, meshrecovery.SubjectEndedAll},
{meshrecovery.StreamRestored, meshrecovery.SubjectRestoredAll}} {
if _, err := b.js.AddStream(&nats.StreamConfig{Name: s.stream, Subjects: []string{s.subject}, MaxMsgsPerSubject: 1}); err != nil {
t.Fatal(err)
}
}
js, err := jetstream.New(b.conn)
if err != nil {
t.Fatal(err)
}
if _, err := js.CreateKeyValue(ctx, jetstream.KeyValueConfig{Bucket: link.BoundsBucket, History: 1, LimitMarkerTTL: time.Second}); err != nil {
t.Fatal(err)
}
say := func(act, outcome string) {
raw, _ := json.Marshal(meshrecovery.RestoredEvent{Act: act, Module: "db", Kept: "cache", Outcome: outcome, At: time.Now().UTC()})
if _, err := b.js.Publish(meshrecovery.SubjectRestored("db", "cache"), raw); err != nil {
t.Fatal(err)
}
}
var restoreAsked meshrecovery.RestoreKeptInput
// The kept points the stand-in answers: the one of the first checks, then (below) what the agent's revert finds, and none.
var pointsMu sync.Mutex
points := []meshrecovery.Point{{Module: "db", Kept: "cache", Form: 1, Fingerprint: "f2", Snapshot: "snap-1",
CatchUp: &meshrecovery.CatchUp{Tool: "rebuild", Arguments: map[string]any{"all": true}}}}
// The restore the agent's revert asks waits for the stand-in to say it restored: gated until released.
var gated atomic.Bool
release := make(chan struct{})
var restores atomic.Int32
for verb, answer := range map[string]func(m *nats.Msg) any{
meshrecovery.VerbKeptPoints: func(*nats.Msg) any {
pointsMu.Lock()
defer pointsMu.Unlock()
return meshrecovery.KeptPointsAnswer{Points: points}
},
meshrecovery.VerbRestoreKept: func(m *nats.Msg) any {
pointsMu.Lock()
_ = json.Unmarshal(m.Data, &restoreAsked)
pointsMu.Unlock()
act := fmt.Sprintf("r%d", restores.Add(1))
if gated.Load() {
go func() { <-release; say(act, meshrecovery.RestoredEventOutcomeDone) }()
} else {
say(act, meshrecovery.RestoredEventOutcomeDone)
}
return meshrecovery.RestoreKeptAnswer{Accepted: act}
},
} {
answer := answer
sub, err := b.conn.Subscribe("mesh.seat.mesh-recovery.call."+verb+".*", func(m *nats.Msg) {
out, _ := json.Marshal(map[string]any{"result": answer(m)})
_ = m.Respond(out)
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = sub.Unsubscribe() })
}
var toolArgs map[string]any
var toolCalls atomic.Int32
toolSub, err := b.conn.Subscribe(link.ToolSubject("db", "rebuild", itNode), func(m *nats.Msg) {
var args map[string]any
_ = json.Unmarshal(m.Data, &args)
pointsMu.Lock()
toolArgs = args
pointsMu.Unlock()
toolCalls.Add(1)
_ = m.Respond([]byte(`{"result":{}}`))
})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = toolSub.Unsubscribe() })
// A-112: the bound is the number in the file the manifest declares, 90 where it is missing or not a number.
for content, want := range map[string]int{"45\n": 45, "": 90, "soon\n": 90, "0\n": 90} {
recoveryWaitFile = filepath.Join(t.TempDir(), "w")
if err := os.WriteFile(recoveryWaitFile, []byte(content), 0o644); err != nil {
t.Fatal(err)
}
if got := recoveryWaitMinutes(); got != want {
t.Errorf("the file %q gives %d minutes, want %d", content, got, want)
}
}
recoveryWaitFile = filepath.Join(t.TempDir(), "missing")
if got := recoveryWaitMinutes(); got != 90 {
t.Errorf("no file gives %d minutes, want 90", got)
}
say("old", meshrecovery.RestoredEventOutcomeFailed) // said before anyone listens: the last message of the subject
session := b.startSession()
r := recoveryEnv(session, nil)
events, stop, err := r.Hear(ctx, "restored", "db", "cache")
if err != nil {
t.Fatal(err)
}
defer stop()
if first := <-events; first.Act != "old" || first.Outcome != "failed" {
t.Fatalf("the first event heard is %+v, want the last one said before", first)
}
pts, err := r.KeptPoints(ctx, meshrecovery.KeptPointsInput{Module: "db", Kept: "cache"})
if err != nil || len(pts.Points) != 1 {
t.Fatalf("kept-points: %+v, %v", pts, err)
}
got, err := r.RestoreKept(ctx, meshrecovery.RestoreKeptInput{Module: "db", Kept: "cache", Form: 1, Fingerprint: "f2", Snapshot: pts.Points[0].Snapshot, Why: "w"})
pointsMu.Lock()
firstAsked := restoreAsked
pointsMu.Unlock()
if err != nil || got.Accepted != "r1" || firstAsked.Snapshot != "snap-1" {
t.Fatalf("restore-kept: %+v, %v, asked %+v", got, err, firstAsked)
}
if ev := <-events; ev.Act != "r1" || ev.Outcome != "done" {
t.Fatalf("the ending heard is %+v", ev)
}
err = r.CatchUp(ctx, "db", *pts.Points[0].CatchUp)
pointsMu.Lock()
gotArgs := toolArgs
pointsMu.Unlock()
if err != nil || gotArgs["all"] != true {
t.Fatalf("catch-up: %v, arguments %v", err, gotArgs)
}
expired, end, err := r.Bound(ctx, "recovery", "db.restored.cache", "db", time.Now(), time.Second)
if err != nil {
t.Fatal(err)
}
defer end()
select {
case <-expired:
case <-time.After(10 * time.Second):
t.Fatal("the bound's end, the bus's marker, was not heard")
}
// Design 47 G §296, §297, §554, §560, A-113: the node agent whole drives a revert want of db through the same recovery wiring
// (recoveryEnv over the session and the node's composer): db runs at a build whose cache is of form 2, and its want goes back to a
// build of form 1. With no point of form 1 taken before the newer build, its have says the restore point is missing, naming it,
// and the newer build runs again. With the point, the have is held (backing up) until the stand-in says restored, then the
// older build is applied and the point's catch-up tool is called.
dbManifest := func(form int, content string) string {
return `{"module":"db","version":"1","state":[{"name":"cache","form":` + fmt.Sprint(form) + `}],"resources":[{"id":"conf","type":"file","path":"` +
filepath.Join(b.dir, "db.conf") + `","mode":"0644","content":"` + content + `\n"}]}`
}
fpOld, fpNew := fpIt("a"), fpIt("b")
b.registerEntry("db", fpOld, dbManifest(1, "v1"))
b.registerEntry("db", fpNew, dbManifest(2, "v2"))
for _, name := range []string{buckets.Manifests, selfcompose.ResolvedBucket, buckets.ProposalCandidates, link.PackagesBucket} {
if _, err := b.js.CreateKeyValue(&nats.KeyValueConfig{Bucket: name, History: 5}); err != nil {
t.Fatal(err)
}
}
assignment, err := b.js.CreateKeyValue(&nats.KeyValueConfig{Bucket: buckets.Assignments, History: 5})
if err != nil {
t.Fatal(err)
}
places, _ := meshmoduleassignerclient.AssignmentsValues(meshmoduleassigner.Assignments{Node: itNode, Modules: []meshmoduleassigner.PlacedModule{{Module: "db"}}})
hkey, _ := nec.HaveNodeKey(itNode)
rec, _ := nec.EncodeHaveNode(nodeengine.HaveByNodeNodeValue{Record: &nodeengine.NodeRecord{Node: itNode, Capabilities: []string{"service-manager", "package-manager"}}})
if _, err := b.haves.Put(hkey, rec); err != nil {
t.Fatal(err)
}
for key, value := range places {
if _, err := assignment.Put(key, value); err != nil {
t.Fatal(err)
}
}
b.want("db", fpNew, "", "")
stopAgent := b.start()
defer stopAgent()
within(t, 20*time.Second, "db does not run its newer build", func() bool {
h := b.have("db")
return h.Fp == fpNew && h.Phase == nec.PhaseRunning && b.file("db") == "v2\n"
})
// The engine says what runs, as it does after each apply: the composer keeps the entry of the running build while the want moves on.
b.running(itNode, "db", fpNew, meshcatalogclient.ManifestDigest([]byte(dbManifest(2, "v2"))))
// The stand-in holds no point of form 1: the revert is held, the have names the point missing, the newer build runs.
pointsMu.Lock()
points = nil
pointsMu.Unlock()
b.want("db", fpOld, "", "")
within(t, 20*time.Second, "the have does not say the restore point is missing", func() bool {
h := b.have("db")
return h.Phase == nec.PhaseHeld && strings.Contains(h.Detail, "restore point missing") &&
strings.Contains(h.Detail, "db's cache has none of form 1 taken before "+fpNew)
})
if b.have("db").Fp != fpNew || b.file("db") != "v2\n" || restores.Load() != 1 {
t.Fatalf("the missing point left db %+v with %q, %d restore(s) asked", b.have("db"), b.file("db"), restores.Load())
}
// The stand-in now holds the point: the want answered "again" asks the restore, and the have waits for its ending.
pointsMu.Lock()
points = []meshrecovery.Point{{Module: "db", Kept: "cache", Form: 1, Fingerprint: fpNew, Snapshot: "snap-2", At: time.Now(),
CatchUp: &meshrecovery.CatchUp{Tool: "rebuild", Arguments: map[string]any{"all": "again"}}}}
pointsMu.Unlock()
gated.Store(true)
calls := toolCalls.Load()
b.want("db", fpOld, "", meshwants.WantsByWantNodeModuleValueBackupAgain)
within(t, 20*time.Second, "the restore of the revert is not asked", func() bool { return restores.Load() == 2 })
within(t, 20*time.Second, "the have does not wait for the restore", func() bool { return b.have("db").Phase == nec.PhaseBackingUp })
// The restored event is held until close(release): the older build must not be applied before the stand-in says restored.
if h := b.have("db"); h.Phase != nec.PhaseBackingUp || b.file("db") != "v2\n" || toolCalls.Load() != calls {
t.Fatalf("before the stand-in says restored the have is %+v, the file %q, %d catch-up call(s) more", h, b.file("db"), toolCalls.Load()-calls)
}
close(release)
within(t, 20*time.Second, "the older build is not applied after the restore", func() bool {
h := b.have("db")
return h.Fp == fpOld && h.Phase == nec.PhaseRunning && b.file("db") == "v1\n"
})
within(t, 20*time.Second, "the catch-up tool is not called after the restore", func() bool { return toolCalls.Load() == calls+1 })
pointsMu.Lock()
asked, gotArgs := restoreAsked, toolArgs
pointsMu.Unlock()
if asked.Snapshot != "snap-2" || asked.Fingerprint != fpNew || gotArgs["all"] != "again" {
t.Errorf("the revert asked %+v and called the catch-up with %v", asked, gotArgs)
}
stopAgent()
// Design 47 G A-293: over the same session, the builder's build ends the composer follows: the stream read whole first (the
// last end of each module and fingerprint, in the stream's order), then the read's end, then each end said after it.
if _, err := b.js.AddStream(&nats.StreamConfig{Name: link.BuildEndsStream, Subjects: []string{link.BuildEndsSubject}, MaxMsgsPerSubject: 1}); err != nil {
t.Fatal(err)
}
ended := func(fp, outcome, why string) {
raw, _ := json.Marshal(builds.BuildEnded{Module: "db", Fingerprint: fp, Outcome: outcome, Error: why})
if _, err := b.js.Publish("mesh.stream.mesh-delivery-builder.event."+builds.BuildEndedEventName("db", fp), raw); err != nil {
t.Fatal(err)
}
}
ended("f1", builds.BuildFailed, "first")
ended("f1", builds.BuildFailed, "no go")
ended("f0", builds.BuildDone, "")
heard := make(chan string, 8)
follow, stopFollow := context.WithCancel(ctx)
defer stopFollow()
go func() {
_ = session.FollowBuildEnds(follow, func(e builds.BuildEnded) { heard <- e.Fingerprint + ":" + e.Outcome + ":" + e.Error },
func() { heard <- "read" })
}()
for i, want := range []string{"f1:failed:no go", "f0:done:", "read", "f2:failed:later"} {
if want == "f2:failed:later" {
ended("f2", builds.BuildFailed, "later")
}
select {
case got := <-heard:
if got != want {
t.Fatalf("build end %d heard %q, want %q", i, got, want)
}
case <-time.After(10 * time.Second):
t.Fatalf("build end %d, %q, was not heard", i, want)
}
}
}
// startSession is a session on the node's own connection, with no node agent run on it.
func (b *itBed) startSession() *link.Session {
b.session = &link.Session{Own: itOwn(), Membership: link.Membership{Node: itNode}, Timeout: 2 * time.Second,
Say: b.say, Keys: func() link.RecordKeys { return link.RecordKeys{} }}
b.session.Attach(context.Background(), link.OverNATS{Conn: b.nodeConn, JS: b.nodeJS, Node: itNode})
return b.session
}
+128 -26
View File
@@ -5,11 +5,13 @@ import (
"encoding/json"
"errors"
"fmt"
"os"
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
conditions "git.novox.be/novox/mesh-contracts/health-and-repair/go/clients/meshconditionsclient"
condseat "git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshconditions"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/seatdef"
@@ -103,46 +105,58 @@ func (realClock) AfterFunc(d time.Duration, f func()) seatwait.Timer {
return time.AfterFunc(d, f)
}
// findingsOfTheNode writes the node's findings in its state `findings`, in the layout of the seat mesh-conditions.
type findingsOfTheNode struct{ session *link.Session }
// nodeFindings is the node-engine's finding writer: mesh-sdk's held writer, for the engine's findings are raised as they
// happen and not judged afresh (design 47 G A-29, A-40), with the contract of the seat mesh-conditions.
type nodeFindings = views.HeldWriter[condseat.Finding, condseat.FindingCleared, condseat.FindingsStated, condseat.FindingsAskedEvent]
func (w findingsOfTheNode) encode(f views.Finding) (string, []byte, error) {
key := conditions.Key(conditions.ScopeMachine, f.ID, f.Kind)
v := condseat.FindingsByKeyValue{V: 1, Key: key, Kind: f.Kind, Observer: f.Observer, Severity: conditions.Warning,
Subject: condseat.Subject{Scope: conditions.ScopeMachine, ID: f.ID, Machine: f.ID}, Summary: f.Summary,
Headline: f.Headline, Explanation: f.Explanation, Source: "seat-list-unread", At: f.At.UTC()}
if err := conditions.CheckFinding(v); err != nil {
return "", nil, err
}
raw, err := json.Marshal(v)
return key, raw, err
// findingsOfTheNode keeps the node's findings through mesh-sdk's held writer (design 47 G §383, §384, §461, A-29): each
// finding is written in the node's state `findings` and said as an event on the engine's own stream (a foundation part's
// `mesh-host-findings`, §529, A-30), the engine says what it holds at its start and when the keeper asks, and the keeper
// reads no state of the engine's.
type findingsOfTheNode struct {
*nodeFindings
}
func (w findingsOfTheNode) Put(f views.Finding) error {
key, raw, err := w.encode(f)
// newFindingsOfTheNode makes the writer at once; it reads the state when it is started (A-40). Its events are published on
// the engine's own stream awaiting the bus's acknowledgement and its consumer of `findings-asked` is its own.
func newFindingsOfTheNode(session *link.Session, node, module string, say func(string)) (*findingsOfTheNode, error) {
writer, err := views.NewHeldWriter(views.WriterConfig[condseat.Finding, condseat.FindingCleared, condseat.FindingsStated, condseat.FindingsAskedEvent]{
Module: module,
Node: node,
State: link.FindingsState{Session: session},
Kinds: append([]string{seatwait.Kind, seatwait.HeldKind, seatwait.EntryNotFoundKind}, seatwait.NodeKinds...),
Asked: condseat.SubjectFindingsAsked,
Logf: func(format string, args ...any) { say(fmt.Sprintf(format, args...)) },
Publish: session.PublishEvent,
Listen: func(_ context.Context, subject string, handle func([]byte)) (func(), error) {
return session.ListenFrom(condseat.StreamFindingsAsked, subject, handle)
},
})
if err != nil {
return err
return nil, err
}
return w.session.PutFinding(key, raw)
return &findingsOfTheNode{writer}, nil
}
func (w findingsOfTheNode) Delete(f views.Finding) error {
key, _, err := w.encode(f)
if err != nil {
return err
}
return w.session.DeleteFinding(key)
// Start seeds what the state holds, states the findings now and hears the keeper until ctx ends.
func (w *findingsOfTheNode) Start(ctx context.Context) error {
go func() { <-ctx.Done(); w.Close() }()
return w.nodeFindings.Start(ctx)
}
// startSeatList starts this run's reader: its bootstrapped seats are given to catalogue at once, and every change of what
// it holds again, saying so to the composer and to the waiter that writes the five-minute finding.
func startSeatList(ctx context.Context, session *link.Session, node string, say func(string)) (*seatListState, error) {
// it holds again, saying so to the composer and to the waiter that writes the five-minute finding, and stating again what
// the engine reads (link.Session.RestateReads, ADR 0618 §4): the seats it states are those of the list it holds.
func startSeatList(ctx context.Context, session *link.Session, node string, findings *lateFindings, say func(string)) (*seatListState, error) {
st := &seatListState{changed: make(chan struct{}, 1)}
// Start returns at once and reads the seat on a goroutine, which may announce a change before the waiter exists.
var announce atomic.Pointer[func()]
reader, err := seatlist.Start(seatlist.Config{Bus: sessionBus{ctx, session}, Bootstrap: seatlist.Bootstrapped(),
OnChange: func(seats []catalogue.Seat) {
catalogue.UseSeats(seats)
// What the engine reads changed with the list: its record and its event say it (ADR 0618 §4), off the
// reader's goroutine, which returns quickly.
go session.RestateReads(ctx)
if f := announce.Load(); f != nil { // the first read may end before the waiter exists
(*f)()
}
@@ -151,7 +165,7 @@ func startSeatList(ctx context.Context, session *link.Session, node string, say
return nil, fmt.Errorf("the node-engine cannot follow the seat list: %w", err)
}
st.reader = reader
st.waiter = &seatwait.Waiter{Node: node, Observer: "node-engine", Reader: reader, Findings: findingsOfTheNode{session},
st.waiter = &seatwait.Waiter{Node: node, ObserverOf: findings.observer, Reader: reader, Writer: findings, Ctx: ctx, Bound: viewWaitsBound(),
Clock: seatWaitClock, Say: say}
told := func() {
st.waiter.Changed()
@@ -162,6 +176,7 @@ func startSeatList(ctx context.Context, session *link.Session, node string, say
}
announce.Store(&told)
catalogue.UseSeats(reader.Seats())
session.RestateReads(ctx) // its first statement holds the bootstrapped seats at least (issue 1966)
told()
// The first read ends whether the seat answered or not, and a read that held what the reader held already says no
// change: its end is told to the waiter and the composer, so a status that moved from reading to heard starts or ends
@@ -170,6 +185,7 @@ func startSeatList(ctx context.Context, session *link.Session, node string, say
select {
case <-reader.Ready():
catalogue.UseSeats(reader.Seats())
session.RestateReads(ctx)
told()
case <-ctx.Done():
}
@@ -177,3 +193,89 @@ func startSeatList(ctx context.Context, session *link.Session, node string, say
seatListNow.Store(st)
return st, nil
}
// lateFindings is the node-engine's finding writer, made when the engine's own module is known (design 47 G §527, §634,
// A-149): at once from its applied record, or, on a first start with no such record, when the first composing names it. Till
// then a write or a clear is refused in words, never lost in silence.
type lateFindings struct {
own *link.OwnName
make func(module string) (*findingsOfTheNode, error)
mu sync.Mutex
w *findingsOfTheNode
}
// observer is the module the findings are observed as: the engine's own, "" while it is not known.
func (f *lateFindings) observer() string { return f.own.Get() }
// Start makes the writer and starts it as soon as the own name is known: in this call when it already is, else on a goroutine.
func (f *lateFindings) Start(ctx context.Context, say func(string)) {
begin := func(module string) {
w, err := f.make(module)
if err != nil { // a writer that cannot be made is a fault of its configuration: loud
say("the node-engine cannot make its finding writer: " + err.Error())
return
}
if err := w.Start(ctx); err != nil {
say("the node-engine's findings are kept in its state and not said as events until it is started again: " + err.Error())
}
f.mu.Lock()
f.w = w
f.mu.Unlock()
}
if name := f.own.Get(); name != "" {
begin(name)
return
}
go func() {
if name, err := f.own.Wait(ctx); err == nil {
begin(name)
}
}()
}
var errOwnNameUnknown = errors.New("the node-engine's own module is not yet named by its applied record or its first composing, so its finding writer is not made")
func (f *lateFindings) writer() *findingsOfTheNode {
f.mu.Lock()
defer f.mu.Unlock()
return f.w
}
func (f *lateFindings) Write(ctx context.Context, fi condseat.Finding) error {
if w := f.writer(); w != nil {
return w.Write(ctx, fi)
}
return errOwnNameUnknown
}
func (f *lateFindings) Clear(ctx context.Context, scope, id, kind, why string) error {
if w := f.writer(); w != nil {
return w.Clear(ctx, scope, id, kind, why)
}
return errOwnNameUnknown
}
func (f *lateFindings) Held() []condseat.Finding {
if w := f.writer(); w != nil {
return w.Held()
}
return nil
}
// viewWaitsFile is the file the manifest declares with the content ${config:view-waits-minutes} (design 47 G §633), read at
// the engine's start as recoveryWaitFile is.
var viewWaitsFile = "/var/lib/mesh-host/view-waits-minutes"
// viewWaitsBound is how long the reader of the seat list may wait before the engine says it: the file's minutes, the design's
// five (seatwait.Bound) when the file is missing, empty or not a positive number.
func viewWaitsBound() time.Duration {
raw, err := os.ReadFile(viewWaitsFile)
if err != nil {
return seatwait.Bound
}
n, err := strconv.Atoi(strings.TrimSpace(string(raw)))
if err != nil || n <= 0 {
return seatwait.Bound
}
return time.Duration(n) * time.Minute
}
+39
View File
@@ -26,3 +26,42 @@ func TestTheSettledEventIsTheManifestsConsume(t *testing.T) {
t.Errorf("the engine hears %s, its manifest consumes %v", settledEvent, m.Consumes)
}
}
// The manifest claims the seat node-engine, says its findings on its own stream and hears the keeper's ask and the
// node-engine seat's running (design 47 G §364, §387, §199).
func TestTheManifestClaimsItsSeatAndSaysItsFindings(t *testing.T) {
raw, err := os.ReadFile("../../module.json")
if err != nil {
t.Fatal(err)
}
var m struct {
Version string `json:"version"`
Consumes []string `json:"consumes"`
Claims []struct {
Name string `json:"name"`
Scope string `json:"scope"`
} `json:"claims"`
Streams []struct {
Name string `json:"name"`
Events []string `json:"events"`
} `json:"streams"`
}
if err := json.Unmarshal(raw, &m); err != nil {
t.Fatal(err)
}
if len(m.Claims) != 1 || m.Claims[0].Name != "node-engine" || m.Claims[0].Scope != "node" {
t.Errorf("claims = %+v, want the node seat node-engine", m.Claims)
}
if len(m.Streams) != 1 || m.Streams[0].Name != "mesh-host-findings" ||
!slices.Equal(m.Streams[0].Events, []string{"finding-written.*", "finding-cleared.*", "findings-stated.*"}) {
t.Errorf("streams = %+v, want mesh-host-findings with the three finding events", m.Streams)
}
for _, c := range []string{"mesh-conditions.findings-asked", "node-engine.running"} {
if !slices.Contains(m.Consumes, c) {
t.Errorf("consumes %v lacks %s", m.Consumes, c)
}
}
if m.Version != "0.1" {
t.Errorf("version = %q, want MAJOR.MINOR 0.1 (§368, §411)", m.Version)
}
}
+40
View File
@@ -0,0 +1,40 @@
package main
import (
"context"
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-host/internal/store"
)
// The node agent's Stop stops a module's container from the node's record, asks the machine nothing else and leaves the
// record, so the module's kept data and its record survive (design 47 G A-98).
func TestTheNodeAgentsStopStopsAModuleAndRemovesNothing(t *testing.T) {
was := builtFor
builtFor = "arch"
t.Cleanup(func() { builtFor = was })
path := filepath.Join(t.TempDir(), "state.json")
var s store.State
s.Record(store.Applied{ID: "app.box", Type: "container", Target: "app-box"})
s.Record(store.Applied{ID: "other.box", Type: "container", Target: "other-box"})
if err := store.Save(path, s); err != nil {
t.Fatal(err)
}
var asked []string
opts := options{state: path, runner: func(_ context.Context, name string, args ...string) (string, error) {
asked = append(asked, name+" "+strings.Join(args, " "))
return "", nil
}}
if res := stopComposition(context.Background(), opts, "app"); res.Failed {
t.Fatalf("stop failed: %s", res.Detail)
}
if !strings.Contains(strings.Join(asked, "\n"), "stop app-box") || strings.Contains(strings.Join(asked, "\n"), "other-box") ||
strings.Contains(strings.Join(asked, "\n"), " rm ") {
t.Fatalf("the machine was asked %v", asked)
}
if after, err := store.Load(path); err != nil || len(after.Resources) != 2 {
t.Fatalf("the record changed: %v, %v", after.Resources, err)
}
}
+1 -1
View File
@@ -12,7 +12,7 @@ import (
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-contracts/identity-and-access/go/clients/meshvaultclient"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"github.com/novox/mesh-host/internal/buckets"
"os"
"path/filepath"
+1 -1
View File
@@ -8,7 +8,7 @@ import (
"git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
engcomp "github.com/novox/mesh-host/internal/compositions"
"os"
"path/filepath"
+13 -9
View File
@@ -8,16 +8,19 @@ require (
git.novox.be/novox/mesh-contracts/identity-and-access/go/clients/meshnodeenrollerclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/identity-and-access/go/clients/meshvaultclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/provisioning/go/clients/meshprovisionerclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/contracts/go/clients/containerimageregistryclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliveryclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/contracts/go/seats/containerimageregistry v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/contracts/go/seats/meshwants v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/contracts/go/seats/ociartifactregistry v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/containerimageregistryclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshdeliveryclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/containerimageregistry v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/nodebuildgo v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/nodebuildimage v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/nodebuildnode v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/ociartifactregistry v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-grants/grantsstate v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/sdk v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-networking/modules/mesh-wireguard/go v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-sdk/go v0.0.0-00010101000000-000000000000
github.com/nats-io/nats-server/v2 v2.11.17
@@ -29,6 +32,7 @@ require (
require (
git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/go/seats/nodeengine v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/go/seats/meshrecovery v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/health-and-repair/go/clients/meshconditionsclient v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/health-and-repair/go/seats/meshconditions v0.0.0-00010101000000-000000000000
git.novox.be/novox/mesh-contracts/identity-and-access/go/seats/meshvault v0.0.0-00010101000000-000000000000
@@ -45,6 +49,6 @@ require (
)
require (
git.novox.be/novox/mesh-delivery/contracts/go/seats/meshdelivery v0.0.0-00010101000000-000000000000 // indirect
git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshdelivery v0.0.0-00010101000000-000000000000 // indirect
git.novox.be/novox/mesh-networking/modules/mesh-connectivity-contracts/go/seats/theprivatenetwork v0.0.0-00010101000000-000000000000 // indirect
)
+104 -5
View File
@@ -19,7 +19,7 @@ import (
"fmt"
"git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient"
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshwants"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants"
"github.com/novox/mesh-host/internal/compositions"
"slices"
"sort"
@@ -75,6 +75,8 @@ type Have struct {
// PullWait is, on a failed have, the pull from a store its apply failed on and waits on the holder of (issue 1966
// ENG-9, nodeengine.HaveByNodeModuleValue.PullWait): read through the have, as node_haves shows it.
PullWait *nodeengine.PullWait
// Wait is the hold or wait the have says, in the recovery wait with its ask-at and act (design 47 G §560).
Wait *nodeengine.Wait
}
// Haves is the node's have bucket as the agent uses it.
@@ -125,12 +127,24 @@ type Env struct {
Wants Watcher
Composed Watcher
Haves Haves
// WhyUnmade is why the node's composer composed nothing of a module at its want's fingerprint, from its last pass; empty
// when it says nothing. Nil says nothing. Waiting tells the composer the agent waits for the composition of a module at
// a fingerprint it holds none of, so a composer that believes it handed it hands it again. Nil tells nobody.
WhyUnmade func(module string) string
Waiting func(module, fp string)
// Apply applies one module's composition (scope NodeModule for the node's own), given the compositions the node's
// other modules run so a path, unit or group they hold is not removed or taken. It is called with one module at a
// time.
Apply func(ctx context.Context, c compositions.Composed, others []compositions.Composed) Result
// Remove removes the module's own resources and nothing of another's.
Remove func(ctx context.Context, module string, others []compositions.Composed) Result
// Stop stops the module's units and containers and removes nothing, so what it kept survives: a revert stops the newer
// build with it, restores the module's kept data, and then applies the reverted build (design 47 G A-98, §297). Nil
// stops nothing.
Stop func(ctx context.Context, module string) Result
// Recovery takes a module's kept data as a restore point before a form change moves it and restores it at a revert
// (design 47 G §296, §297, §554); nil moves none.
Recovery *Recovery
// Backup backs a module up before a new composition of it is applied; nil backs up nothing.
Backup func(ctx context.Context, module string) Backup
// Kept is the resources the node last applied, by module, from the declaration it kept before the switch (ADR 0539
@@ -241,6 +255,8 @@ type Agent struct {
// prior is a module the node ran before this agent started, so it has something to back up.
prior map[string]bool
said map[string]string
// waited is, by module, the want (fingerprint and revision) the agent said it waits for the composer for.
waited map[string]string
// notComposed is the modules whose have says the node could not compose them: why, and the have before (NotComposed).
notComposed map[string]notComposedMark
// heldAlone is the modules the composer holds alone (NotComposedEach): what runs of each in the node's own
@@ -306,7 +322,7 @@ func New(env Env) *Agent {
env.Say = func(string) {}
}
return &Agent{env: env, wants: map[string]wantNow{}, comps: map[string]compNow{}, running: map[string]nodeengine.HaveByNodeModuleValue{}, applied: map[string]compNow{}, had: map[string]compNow{}, keptData: map[string]bool{},
attempt: map[string]attempt{}, checked: map[string]bool{}, prior: map[string]bool{}, said: map[string]string{}, notComposed: map[string]notComposedMark{},
attempt: map[string]attempt{}, checked: map[string]bool{}, prior: map[string]bool{}, said: map[string]string{}, waited: map[string]string{}, notComposed: map[string]notComposedMark{},
pulls: map[string]*pullWait{}, settledAt: map[string]time.Time{}, settledSeq: map[string]int{},
retry: make(chan []string, 8), reconcile: make(chan []string, 64), failedAgain: make(chan string, 8), tries: map[string]int{}, again: make(chan againAsk, 64),
transient: make(chan struct{}, 1), stopped: make(chan struct{}), quiet: make(chan chan struct{})}
@@ -497,6 +513,9 @@ const NotComposedDetail = "the node cannot compose it, and keeps running what it
// composerMark says h is a mark NotComposed wrote: failed, its detail opening with NotComposedDetail.
func composerMark(h Have) bool {
if h.Phase == string(nodeengineclient.PhaseHeld) && heldForEntry(h.Detail) {
return true
}
return h.Phase == string(nodeengineclient.PhaseFailed) && strings.HasPrefix(h.Detail, NotComposedDetail)
}
@@ -640,8 +659,14 @@ func (a *Agent) NotComposedEach(ctx context.Context, why string, held map[string
a.mu.Lock()
a.notComposed[module] = mark
a.mu.Unlock()
phase, detail := string(nodeengineclient.PhaseFailed), NotComposedDetail+why
if heldForEntry(why) {
// The entry its want names is not held: the phase held, its detail opening entry-asked: or entry-missing: and
// naming the fingerprint (design 47 G section 421).
phase, detail = string(nodeengineclient.PhaseHeld), why
}
a.writeHave(ctx, Have{Module: module, FP: mark.prior.FP, Composed: mark.prior.Composed, Sealed: mark.prior.Sealed,
Phase: string(nodeengineclient.PhaseFailed), Detail: NotComposedDetail + why})
Phase: phase, Detail: detail})
}
}
@@ -1319,6 +1344,31 @@ func (a *Agent) HoldsNode() bool {
return ok && h.Phase != nodeengineclient.PhaseRemoved
}
// waitsForComposer says once per want and reason that the agent waits for the node's own composition of module at fp, and
// why the composer made none (Env.WhyUnmade), and tells the composer (Env.Waiting) so it hands again what it believes it
// handed. Called without a.mu held.
func (a *Agent) waitsForComposer(module, fp string, rev uint64) {
why := "the composer has made none yet"
if a.env.WhyUnmade != nil {
if w := a.env.WhyUnmade(module); w != "" {
why = w
}
}
line := fmt.Sprintf("%s waits for the composer: no composition of it at %s: %s", module, fp, why)
// Said and told once per want (its fingerprint and revision), whatever the reason says; forgotten once composed.
at := fmt.Sprintf("%s@%d", fp, rev)
a.mu.Lock()
first := a.waited[module] != at
if first {
a.waited[module] = at
a.env.Say(line)
}
a.mu.Unlock()
if first && a.env.Waiting != nil {
a.env.Waiting(module, fp)
}
}
func (a *Agent) settleOne(ctx context.Context, module string) {
a.mu.Lock()
w := a.wants[module]
@@ -1334,7 +1384,15 @@ func (a *Agent) settleOne(ctx context.Context, module string) {
comp, have := a.compositionFor(module, w.Want.Fp)
a.mu.Unlock()
if a.env.SelfComposed {
if have {
a.mu.Lock()
delete(a.waited, module)
a.mu.Unlock()
}
if !have || comp.Err != nil {
if !have {
a.waitsForComposer(module, w.Want.Fp, w.Rev)
}
return // waits for the node's own composition of the want's build
}
// The composition the node composed for the want's build is the one it applies (ADR 0547 §2).
@@ -1372,7 +1430,9 @@ func (a *Agent) settleOne(ctx context.Context, module string) {
}
a.mu.Lock()
r, runs := a.running[module]
runsWant := runs && r.Fp == w.Want.Fp && r.Composed == w.Want.Composed && r.Phase != nodeengineclient.PhaseRemoved
// A module whose last apply failed runs no composition whole, whatever it ran before (ranFailed).
runsWant := runs && r.Fp == w.Want.Fp && r.Composed == w.Want.Composed && r.Phase != nodeengineclient.PhaseRemoved &&
r.Phase != nodeengineclient.PhaseFailed
att, tried := a.attempt[module]
same := tried && att.FP == w.Want.Fp && att.Composed == w.Want.Composed
if same && att.Held {
@@ -1381,7 +1441,8 @@ func (a *Agent) settleOne(ctx context.Context, module string) {
// be swallowed by "it runs what its want names" (issue 1585, the second failure). The operator's answer, again or
// skip, in a revision after the one the hold took, is acted on whatever runs; any other new revision of the want
// is the answer only for a move (ADR 0568 §4), as before.
answered := w.Rev > att.AnsweredRev && (w.Want.Backup == meshwants.WantsByWantNodeModuleValueBackupAgain || w.Want.Backup == meshwants.WantsByWantNodeModuleValueBackupSkip)
answered := w.Rev > att.AnsweredRev && (w.Want.Backup == meshwants.WantsByWantNodeModuleValueBackupAgain || w.Want.Backup == meshwants.WantsByWantNodeModuleValueBackupSkip ||
w.Want.Revert == meshwants.WantsByWantNodeModuleValueRevertLive)
if !answered && (runsWant || (att.AnsweredRev >= w.Rev && att.CompRev == comp.Rev)) {
a.sayOnce("held:"+module, fmt.Sprintf("%s is held for a failed backup before %s: it waits for the operator's "+
"answer, and its want (revision %d) carries none since the hold (revision %d)", module, w.Want.Fp, w.Rev, att.AnsweredRev))
@@ -1559,6 +1620,30 @@ func (a *Agent) apply(ctx context.Context, module string, w wantNow, comp compNo
ranBefore = ranBefore || a.prior[module]
a.mu.Unlock()
// A want with parts is applied only when they and the digests its composition fetches agree (design 47 G §524): the
// module keeps running what it has, and the have says which digest differs.
if words := partsDifference(w.Want.Parts, comp.C.Resources); words != "" {
a.mu.Lock()
prev, runs := a.running[module]
a.attempt[module] = attempt{FP: fp, Composed: dig, WantRev: w.Rev, CompRev: comp.Rev, Held: true, AnsweredRev: w.Rev}
a.mu.Unlock()
if !runs {
if h, found, _ := a.readHave(ctx, module); found {
prev = nodeengine.HaveByNodeModuleValue{Fp: h.FP, Composed: h.Composed}
}
}
a.env.Say(fmt.Sprintf("%s is held: %s", module, words))
a.writeHave(ctx, Have{Module: module, FP: prev.Fp, Composed: prev.Composed, Phase: string(nodeengineclient.PhaseHeld),
Detail: words})
return
}
// A kept data on the bus whose form changes is taken as a restore point first, or restored at a revert (design 47 G §296, §297).
held, catchUps := a.keptStep(ctx, module, w, comp, ranBefore, others)
if held {
return
}
// A data module is backed up before a new composition of it moves (ADR 0429 §14), unless the want says to go on
// without one (ADR 0539 §16).
if comp.C.Data && ranBefore && a.env.Backup != nil && w.Want.Backup != meshwants.WantsByWantNodeModuleValueBackupSkip {
@@ -1598,6 +1683,9 @@ func (a *Agent) apply(ctx context.Context, module string, w wantNow, comp compNo
if !res.Failed || res.Transient {
delete(a.pulls, module)
}
if res.Failed {
a.ranFailed(module)
}
what := fmt.Sprintf("%s at %s", module, fp)
if module == NodeModule {
what = "the node's own composition"
@@ -1660,6 +1748,17 @@ func (a *Agent) apply(ctx context.Context, module string, w wantNow, comp compNo
a.env.Settled(module)
}
a.writeHave(ctx, Have{Module: module, FP: fp, Composed: dig, Phase: string(nodeengineclient.PhaseRunning), Detail: res.Detail})
a.catchUp(ctx, module, catchUps)
}
// ranFailed marks what runs of a module as failed once an apply of it failed (novox/hq issue 1966, 2026-11-08): a failed
// apply may have replaced some of its resources (distribution's store container) before it stopped, so what ran is no
// longer whole, and a want of it again (a put back) is applied rather than taken as running. Called with a.mu held.
func (a *Agent) ranFailed(module string) {
if r, ok := a.running[module]; ok && r.Phase != nodeengineclient.PhaseRemoved {
r.Phase = nodeengineclient.PhaseFailed
a.running[module] = r
}
}
func (a *Agent) remove(ctx context.Context, module string, w wantNow) {
+132 -2
View File
@@ -6,8 +6,8 @@ import (
"fmt"
"git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient"
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshwants"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants"
"github.com/novox/mesh-host/internal/buckets/bucketstest"
"github.com/novox/mesh-host/internal/compositions"
"github.com/novox/mesh-host/internal/judge"
@@ -322,6 +322,50 @@ func TestAWantAndItsCompositionApplyTheModuleAndTheHaveNamesBoth(t *testing.T) {
if got := b.appliedList(); len(got) != 1 {
t.Fatalf("a want naming another digest applied the composition: %v", got)
}
// Design 47 G section 524: a want with parts is applied only when its parts and the digests the composition fetches
// agree; a want with no parts is applied as before.
d1, d2, d3 := "sha256:"+strings.Repeat("1", 64), "sha256:"+strings.Repeat("2", 64), "sha256:"+strings.Repeat("3", 64)
archive := func(id, digest string) json.RawMessage {
raw, _ := json.Marshal(map[string]any{"id": "web." + id, "type": "archive", "path": "/tmp/r6/web/" + id,
"source": "https://store/" + id, "digest": digest, "module": "web"})
return raw
}
for i, row := range []struct {
name string
parts map[string]string
applies bool
detail string
}{
{"agree", map[string]string{"web:a": d1, "web:b": d2}, true, ""},
{"no parts", nil, true, ""},
{"fetched outside", map[string]string{"web:a": d1}, false, PartsDiffer + " fetched and not in the want's parts: " + d2},
{"part not fetched", map[string]string{"web:a": d1, "web:b": d2, "web:c": d3}, false,
PartsDiffer + " in the want's parts and not fetched: " + d3},
} {
before := len(b.appliedList())
fpn := fpOf(string("5678"[i]))
ckn, crawn, dign := composition(t, "web", fpn, false, archive("a", d1), archive("b", d2))
b.comps.put(ckn, crawn)
wkn := meshwants.WantsByWantNodeModuleKey(node, "web")
wrawn, err := bucketstest.Want(meshwants.WantsByWantNodeModuleValue{Fp: fpn, Composed: dign, Parts: row.parts})
if err != nil {
t.Fatal(err)
}
b.wants.put(wkn, wrawn)
b.idle()
if got := len(b.appliedList()) - before; (got == 1) != row.applies {
t.Fatalf("parts %q: applied %d times, want applies=%v", row.name, got, row.applies)
}
if h := b.haves.last("web"); !row.applies && (h.Phase != string(nodeengineclient.PhaseHeld) || h.Detail != row.detail) {
t.Fatalf("parts %q: the have is %s %q, want held %q", row.name, h.Phase, h.Detail, row.detail)
}
}
// A container's digest is the one after its image's @.
container := json.RawMessage(`{"id":"web.c","type":"container","name":"web-c","image":"store/web@` + d3 + `","module":"web"}`)
if got := partsDifference(map[string]string{"web:c": d3}, []json.RawMessage{container}); got != "" {
t.Fatalf("a container whose digest is in the parts differs: %q", got)
}
}
// R6-A AC2: given a kept declaration and wants and compositions equal to it, when the agent starts, nothing is applied
@@ -487,6 +531,45 @@ func TestAFailedApplyIsRetriedOnlyByAChangeOfEitherKey(t *testing.T) {
t.Errorf("a module running its want was applied again (%d applies)", n)
}
})
// Issue 1966 (2026-11-08 23:47 UTC): a failed apply of distribution's new build had replaced its store container, and
// the put back to the build that ran before applied nothing, judged as running. A module whose apply failed runs no
// composition whole: the want of what ran before is applied again.
t.Run("1966: after a failed apply, the composition that ran is applied again when wanted", func(t *testing.T) {
b := newBed(t)
stop := b.run()
defer stop()
fpA, fpB := fpOf("a"), fpOf("b")
ckA, crawA, digA := composition(t, "web", fpA, false, file("web", "x", "1"))
ckB, crawB, digB := composition(t, "web", fpB, false, file("web", "x", "2"))
b.comps.put(ckA, crawA)
b.comps.put(ckB, crawB)
wk, wrawA := wantFor(t, "web", fpA, digA, "")
_, wrawB := wantFor(t, "web", fpB, digB, "")
b.wants.put(wk, wrawA)
b.idle()
if n := len(b.appliedList()); n != 1 || b.haves.last("web").Phase != string(nodeengineclient.PhaseRunning) {
t.Fatalf("the first build was not applied and running (%d applies, have %+v)", n, b.haves.last("web"))
}
b.mu.Lock()
b.failWith = "distribution.program: the store does not answer"
b.mu.Unlock()
b.wants.put(wk, wrawB)
b.idle()
if n := len(b.appliedList()); n != 2 || b.haves.last("web").Phase != string(nodeengineclient.PhaseFailed) {
t.Fatalf("the second build's apply did not fail (%d applies, have %+v)", n, b.haves.last("web"))
}
b.mu.Lock()
b.failWith = ""
b.mu.Unlock()
b.wants.put(wk, wrawA) // the put back
b.idle()
if n := len(b.appliedList()); n != 3 {
t.Fatalf("the put back to the build that ran applied nothing (%d applies): a failed apply left it said as running", n)
}
if h := b.haves.last("web"); h.FP != fpA || h.Composed != digA || h.Phase != string(nodeengineclient.PhaseRunning) {
t.Errorf("after the put back the have is %+v, want the first build running", h)
}
})
// design 51 (ENG-8, ENG-11): a pull from a store that failed is tried again when the store's holder settles, once per
// settled, never on a clock; the have says whom it waits on and what triggered the last retry.
t.Run("a pull from a store waits for its holder to settle", func(t *testing.T) {
@@ -847,6 +930,7 @@ func TestADataModuleIsBackedUpFirstAndHeldWhenItFails(t *testing.T) {
// A data module with a good backup is applied after it; one that never ran has nothing to back up.
func TestABackupDoneIsFollowedByTheApplyAndANewModuleIsNotBackedUp(t *testing.T) {
t.Run("a move across a kept data's form change is taken as a restore point and restored at a revert", testAMoveAcrossAFormChange)
b := newBed(t)
stop := b.run()
defer stop()
@@ -1517,6 +1601,11 @@ func TestTheNodesOwnCompositionIsAppliedWhenItDiffersAndARecordMoveWaitsForAYes(
history[key] = append(history[key], Update{Key: key, Value: raw, Revision: uint64(len(history[key]) + 1)})
}
b.env.Held = func() map[string]string { return map[string]string{"mesh-build-go": "its binding is unset"} }
// Issue 1966 (novox, 2026-11-09): a want the composer made no composition of is said once, with why, and the composer told.
var toldMu sync.Mutex
var told []string
b.env.WhyUnmade = func(string) string { return "no manifest in the composer's picture" }
b.env.Waiting = func(module, fp string) { toldMu.Lock(); told = append(told, module+"@"+fp); toldMu.Unlock() }
stop := b.run()
defer stop()
unsigned := func(module, fp string, rs ...json.RawMessage) (string, []byte, string) {
@@ -1544,6 +1633,41 @@ func TestTheNodesOwnCompositionIsAppliedWhenItDiffersAndARecordMoveWaitsForAYes(
b.comps.put(ck, craw)
want("web", fp, "")
eventually(t, "the node's own composition was not applied", func() bool { return len(b.appliedList()) == 1 })
gk := meshwants.WantsByWantNodeModuleKey(node, "ghost")
graw, _ := bucketstest.Want(meshwants.WantsByWantNodeModuleValue{Fp: fpOf("9"), Self: true})
b.wants.put(gk, graw)
line := "ghost waits for the composer: no composition of it at " + fpOf("9") + ": no manifest in the composer's picture"
saidCount := func() (n int) {
b.mu.Lock()
defer b.mu.Unlock()
for _, l := range b.said {
if l == line {
n++
}
}
return n
}
eventually(t, "the wait for the composer was not said", func() bool { return saidCount() == 1 })
// Settled again by another want's change, it is neither said nor told again.
other, oraw, _ := unsigned("ghost", fpOf("8"), file("ghost", "x", "1"))
b.comps.put(other, oraw)
eventually(t, "ghost's other composition was not taken", func() bool {
b.ag.mu.Lock()
defer b.ag.mu.Unlock()
_, ok := b.ag.comps[other]
return ok
})
// What is settled after it: web's want moves on, so ghost's settle, queued before it, has run.
want("web", fp, "")
eventually(t, "the composer was not told once", func() bool { toldMu.Lock(); defer toldMu.Unlock(); return slices.Contains(told, "ghost@"+fpOf("9")) })
if _, err := b.ag.Composition(context.Background(), "ghost", "", false); err == nil || !strings.Contains(err.Error(), "waits for the composer: no manifest in the composer's picture") {
t.Errorf("the composition read of ghost says %v, want why the composer made none", err)
}
toldMu.Lock()
if n := slices.Index(told, "ghost@"+fpOf("9")); saidCount() != 1 || n < 0 || slices.Contains(told[n+1:], "ghost@"+fpOf("9")) {
t.Errorf("said %d times, told %q; want once and ghost told once", saidCount(), told)
}
toldMu.Unlock()
eventually(t, "the have does not name the composed digest", func() bool { return b.haves.last("web").Composed == dig })
// ADR 0601 §4: what the composer keeps of web as it runs is carried on its have, and goes once it keeps none.
b.ag.KeptNow(context.Background(), map[string][]string{"web": {"web runs the tool runner's process here"}})
@@ -1649,6 +1773,12 @@ func TestTheNodesOwnCompositionIsAppliedWhenItDiffersAndARecordMoveWaitsForAYes(
if h := b.haves.last("web"); h.Phase != string(nodeengineclient.PhaseRunning) || h.Composed != dig {
t.Errorf("web, which composes, is marked beside idle: %+v", h)
}
// Design 47 G §421: a module whose want's entry the catalog half holds none of is the phase held, its detail opening
// entry-missing: and naming the fingerprint; composed again, its have is what it runs.
b.ag.NotComposedEach(context.Background(), "", map[string]string{"idle": EntryMissing + " the catalog half holds no entry of the build src2:x"})
if h := b.haves.last("idle"); h.Phase != string(nodeengineclient.PhaseHeld) || h.Composed != idleDig || h.Detail != EntryMissing+" the catalog half holds no entry of the build src2:x" {
t.Errorf("idle's have while its entry is missing: %+v", h)
}
b.ag.NotComposedEach(context.Background(), "", nil)
if h := b.haves.last("idle"); h.Phase != string(nodeengineclient.PhaseRunning) || h.Composed != idleDig {
t.Errorf("idle's have once it composes again: %+v", h)
+16
View File
@@ -44,6 +44,17 @@ type CompositionView struct {
// Own is, of the node's own composition (module node), its own fields beside what runs names (issue 774: ace's
// own composition dropped the provisioner client's account, and nothing read could show it).
Own *OwnView `json:"own,omitempty"`
// Withheld are the module's marked subjects the grants part withholds from this node's runtime, each with why, read
// through the seat mesh-grants' verb withheld; WithheldUnknown says when that verb did not answer (design 46 approval
// mark §6, lead answer A-239).
Withheld []WithheldView `json:"withheld,omitempty"`
WithheldUnknown string `json:"withheld-unknown,omitempty"`
}
// WithheldView is one marked subject withheld, and why.
type WithheldView struct {
Subject string `json:"subject"`
Why string `json:"why"`
}
// OwnView is the node's own fields of a composition (engcomp.NodeOwn) beside what runs of them, and what the node's
@@ -209,6 +220,11 @@ func (a *Agent) Composition(ctx context.Context, module, digest string, diff boo
case digest != "" && read == nil:
return view, fmt.Errorf("no revision of %s the bucket keeps has the digest %s; revisions lists those it keeps", key, digest)
case read == nil && !haveLatest:
if a.env.SelfComposed && a.env.WhyUnmade != nil && module != NodeModule {
if why := a.env.WhyUnmade(module); why != "" {
return view, fmt.Errorf("the bucket holds no composition of %s at %s; the agent waits for the composer: %s", module, fp, why)
}
}
return view, fmt.Errorf("the bucket holds no composition of %s at %s", module, fp)
case read == nil:
read = &latest
+372
View File
@@ -0,0 +1,372 @@
package agent
import (
"context"
"fmt"
"time"
"git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient"
"git.novox.be/novox/mesh-contracts/go/seats/meshrecovery"
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-host/internal/compositions"
)
// A module's kept data on the bus, taken as a restore point before a form change moves it and restored at a revert (design 47 G
// §296, §297, §554, §555, §560, §630 to §632, the lead's answers A-77, A-98 and A-108): the node agent asks the seat
// `mesh-recovery`'s `snapshot-kept` and waits for the `ended` of its own act, or, at a revert, stops the newer build, asks
// `kept-points`, asks `restore-kept` with the snapshot of the point of the form it reverts from, waits for its `restored`, and
// after the reverted build's apply calls the point's catch-up as the module's tool.
// Ending is the ending event of an act of the recovery module as the node agent reads it: `ended` for a restore point taken,
// `restored` for one restored (design 47 G §560).
type Ending struct {
Act string
Outcome string
Error string
Snapshot string
// Why is the hand act's words for a cancel (design 47 G A-117, §560).
Why string
}
// RecoveryWaitMinutes is the default of the manifest's config key `recovery-wait-minutes`, the bound of the wait for an
// ending event, counted from the first ask (design 47 G §554, §633).
const RecoveryWaitMinutes = 90
// Recovery is what the node agent needs of the recovery module and the manifests to move kept data. Nil moves none.
type Recovery struct {
// FormChanges are the kept data whose form differs between the running build's manifest and the applied build's (the
// catalog half's entries by fingerprint); an error says it is not known.
FormChanges func(ctx context.Context, module, runningFp, appliedFp string) ([]catalogue.FormChange, error)
// The seat's verbs, asked in the node-engine's own name; an error is the ask not answered.
SnapshotKept func(ctx context.Context, in meshrecovery.SnapshotKeptInput) (meshrecovery.SnapshotKeptAnswer, error)
RestoreKept func(ctx context.Context, in meshrecovery.RestoreKeptInput) (meshrecovery.RestoreKeptAnswer, error)
KeptPoints func(ctx context.Context, in meshrecovery.KeptPointsInput) (meshrecovery.KeptPointsAnswer, error)
// Hear follows the ending events of kind "ended" or "restored" of one module's kept data: the last one said already first,
// then each after it, until stop is called. It is made before the verb is asked (design 47 G §560).
Hear func(ctx context.Context, kind, module, kept string) (events <-chan Ending, stop func(), err error)
// Bound writes the bound key `<what>.<id>` of a wait in the state bounds with its time to live, so its end is the bus's
// marker, heard as expired (design 47 G A-77, A-92); end deletes it. Nil keeps no bound.
Bound func(ctx context.Context, what, id, module string, since time.Time, d time.Duration) (expired <-chan struct{}, end func(), err error)
// CatchUp calls the module's tool on this node through the tool runner (design 47 G A-108).
CatchUp func(ctx context.Context, module string, c meshrecovery.CatchUp) error
// WaitMinutes is the bound in minutes, the config key recovery-wait-minutes; 0 reads as RecoveryWaitMinutes.
WaitMinutes int
// Now is the clock; nil reads the machine's.
Now func() time.Time
}
func (r *Recovery) now() time.Time {
if r.Now != nil {
return r.Now()
}
return time.Now()
}
func (r *Recovery) bound() time.Duration {
if r.WaitMinutes > 0 {
return time.Duration(r.WaitMinutes) * time.Minute
}
return RecoveryWaitMinutes * time.Minute
}
// KeptMove is one kept data a want moves across a change of its form, as the seat mesh-recovery's verbs name it.
type KeptMove struct {
// Name is the kept data's local name, Form the form the restore point holds (the older one), and Fingerprint the
// fingerprint the restore point is taken before (the newer build's) or reverted from.
Name string
Form int
Fingerprint string
// CatchUp is, when taking the point, the tool the newer manifest declares beside the kept data.
CatchUp *meshrecovery.CatchUp
}
// keptMoves are the restore points a move needs: a point to take before the newer build starts (the kept data's form rises,
// or the newer manifest declares it no more) and a point to restore before the older build starts (the kept data goes back to
// a lower form). A kept data the running build does not declare is neither: nothing was written before it.
func keptMoves(changes []catalogue.FormChange, runningFp, appliedFp string) (take, restore []KeptMove) {
for _, c := range changes {
switch {
case c.Running == nil:
case c.Applied == nil || c.Applied.Form > c.Running.Form:
m := KeptMove{Name: c.Name, Form: c.Running.Form, Fingerprint: appliedFp}
if c.Applied != nil && c.Applied.CatchUp != nil {
m.CatchUp = &meshrecovery.CatchUp{Tool: c.Applied.CatchUp.Tool, Arguments: c.Applied.CatchUp.Arguments}
}
take = append(take, m)
case c.Applied.Form < c.Running.Form:
restore = append(restore, KeptMove{Name: c.Name, Form: c.Applied.Form, Fingerprint: runningFp})
}
}
return take, restore
}
// recoveryHeld is how a wait for the recovery module ended when the move cannot go on: the words are the reason of the hold.
type recoveryHeld struct {
Words string
Interrupted bool
}
// askAndWait asks the verb until its act is accepted and ended, as the table of design 47 G §560 says: `busy` waits for the
// ending of the act it names and asks again, `failed` with `stopped` asks again within the same bound, a failure with any
// other error holds. The wait is written in the have with its first ask's time and the act, so a restart waits for that act
// and asks nothing again; past the bound the have says the wait, and the wait goes on (the watchman raises it).
func (a *Agent) askAndWait(ctx context.Context, h Have, kind string, k KeptMove, ask func() (accepted, busy, errWords string, err error)) (Ending, *recoveryHeld) {
r := a.env.Recovery
module := h.Module
step := nodeengine.WaitStepCompose
wait := nodeengine.Wait{Owner: nodeengine.Seat, Step: step, On: "mesh-recovery." + kind + "." + module + "." + k.Name,
Reason: "recovery waits", Bound: r.bound().String()}
// A restart: the have names the wait with its ask time and act.
last, ok := a.env.Haves.Read(ctx, module)
if ok && last.Wait != nil && last.Wait.On == wait.On && last.Wait.AskAt != nil {
wait.AskAt, wait.Act, wait.Since = last.Wait.AskAt, last.Wait.Act, last.Wait.Since
}
now := r.now()
if wait.AskAt == nil {
wait.AskAt, wait.Since = &now, now
}
events, stop, err := r.Hear(ctx, kind, module, k.Name)
if err != nil {
return Ending{}, &recoveryHeld{Words: "the end of the recovery module's act cannot be heard: " + err.Error()}
}
defer stop()
var expired <-chan struct{}
if r.Bound != nil {
bound := r.bound() - now.Sub(*wait.AskAt)
if bound < time.Second {
bound = time.Second
}
exp, end, berr := r.Bound(ctx, "recovery", module+"."+kind+"."+k.Name, module, *wait.AskAt, bound)
if berr != nil {
return Ending{}, &recoveryHeld{Words: "the bound of the wait for the recovery module cannot be written: " + berr.Error()}
}
defer end()
expired = exp
}
write := func() {
w := wait
h.Wait = &w
a.writeHave(ctx, h)
}
act, busyOn := wait.Act, ""
for {
if act == "" && busyOn == "" {
write() // the first ask's time is in the have before the ask
accepted, busy, words, err := ask()
switch {
case err != nil:
if ctx.Err() != nil {
return Ending{}, &recoveryHeld{Words: BackupInterrupted, Interrupted: true}
}
return Ending{}, &recoveryHeld{Words: "the recovery module could not be asked: " + err.Error()}
case words != "":
return Ending{}, &recoveryHeld{Words: words}
case accepted != "":
act, wait.Act = accepted, accepted
write()
case busy != "":
busyOn = busy
default:
return Ending{}, &recoveryHeld{Words: "the recovery module answered neither accepted nor busy"}
}
}
select {
case <-ctx.Done():
return Ending{}, &recoveryHeld{Words: BackupInterrupted, Interrupted: true}
case <-expired:
expired = nil
wait.Words = fmt.Sprintf("the recovery module has not ended its act within %s", r.bound())
write()
case ev, open := <-events:
if !open {
return Ending{}, &recoveryHeld{Words: "the end of the recovery module's act is no longer heard"}
}
switch {
case busyOn != "" && ev.Act == busyOn:
busyOn = ""
case act != "" && ev.Act == act:
switch {
case ev.Outcome == meshrecovery.EndedEventOutcomeDone:
return ev, nil
case ev.Error == "stopped":
act, wait.Act = "", ""
case ev.Error == "cancelled":
words := "cancelled"
if ev.Why != "" {
words += ": " + ev.Why
}
return ev, &recoveryHeld{Words: words}
default:
words := ev.Error
if words == "" {
words = "the recovery module's act failed"
}
return ev, &recoveryHeld{Words: words}
}
}
}
}
}
// keptStep takes the restore points a move of a module needs before it applies and restores the ones a revert needs: it
// answers whether the module is held (its have says why, and the running build is kept) and, after a restore, the catch-up
// of the point to call once the reverted build runs. A want that says to go on without a backup asks neither, and nor does one that answers a held revert live.
func (a *Agent) keptStep(ctx context.Context, module string, w wantNow, comp compNow, ranBefore bool, others []compositions.Composed) (held bool, catchUps []meshrecovery.CatchUp) {
r := a.env.Recovery
if !ranBefore || r == nil || r.FormChanges == nil || w.Want.Backup == meshwants.WantsByWantNodeModuleValueBackupSkip {
return false, nil
}
fp, dig := w.Want.Fp, w.Want.Composed
a.mu.Lock()
prev, runs := a.running[module]
a.mu.Unlock()
if !runs {
if h, found, _ := a.readHave(ctx, module); found {
prev = nodeengine.HaveByNodeModuleValue{Fp: h.FP, Composed: h.Composed}
}
}
if prev.Fp == "" || prev.Fp == fp {
return false, nil
}
changes, err := r.FormChanges(ctx, module, prev.Fp, fp)
if err != nil {
// Whether the move changes a form is not known: it waits for the entries and never moves without its restore point (design 47 G
// A-113). No attempt is kept, so the composer handing the composition again, or any settle, asks again.
words := fmt.Sprintf("whether %s's kept data changes form between %s and %s is not known, so it is not moved: %v", module, prev.Fp, fp, err)
a.sayOnce("kept:"+module+":"+fp, module+" waits: "+words)
since := r.now()
a.writeHave(ctx, Have{Module: module, FP: prev.Fp, Composed: prev.Composed, Phase: string(nodeengineclient.PhaseHeld), Detail: words,
Wait: &nodeengine.Wait{Owner: nodeengine.Seat, Step: nodeengine.WaitStepCompose, Reason: "entry-missing", Since: since, Words: words}})
if a.env.Waiting != nil {
a.env.Waiting(module, fp)
}
return true, nil
}
take, restore := keptMoves(changes, prev.Fp, fp)
if len(take)+len(restore) == 0 {
return false, nil
}
// The operator's answer "live" to a held revert, read from this want and from nothing else: the reverted build starts on the live
// data as it is, with no restore, no aside taken and no catch-up, and applies as any want (design 47 G section 710).
if len(restore) > 0 && w.Want.Revert == meshwants.WantsByWantNodeModuleValueRevertLive {
a.env.Say(fmt.Sprintf("%s starts %s on the live data as it is: its want (revision %d) carries the operator's answer %q to the held revert", module, fp, w.Rev, w.Want.Revert))
return false, nil
}
hold := func(words string, interrupted bool) {
if interrupted || ctx.Err() != nil {
words = BackupInterrupted
a.env.Say(fmt.Sprintf("%s waits: %s; it asks again at the next start", module, words))
} else {
a.env.Say(fmt.Sprintf("%s is held before %s: %s", module, fp, words))
}
a.mu.Lock()
a.attempt[module] = attempt{FP: fp, Composed: dig, WantRev: w.Rev, CompRev: comp.Rev, Held: true, AnsweredRev: w.Rev}
a.mu.Unlock()
since := r.now()
a.writeHave(ctx, Have{Module: module, FP: prev.Fp, Composed: prev.Composed, Phase: string(nodeengineclient.PhaseHeld), Detail: words,
Wait: &nodeengine.Wait{Owner: nodeengine.Seat, Step: nodeengine.WaitStepCompose, Reason: words, Since: since, Words: words}})
}
backing := Have{Module: module, FP: fp, Composed: dig, Phase: string(nodeengineclient.PhaseBackingUp)}
for _, m := range take {
why := fmt.Sprintf("form change of %s's %s to %s", module, m.Name, fp)
_, held := a.askAndWait(ctx, backing, "ended", m, func() (string, string, string, error) {
got, err := r.SnapshotKept(ctx, meshrecovery.SnapshotKeptInput{Module: module, Kept: m.Name, Form: int64(m.Form),
Fingerprint: m.Fingerprint, CatchUp: m.CatchUp, Why: why})
return got.Accepted, got.Busy, got.Error, err
})
if held != nil {
hold(fmt.Sprintf("the restore point of %s's %s (form %d) was not taken: %s", module, m.Name, m.Form, held.Words), held.Interrupted)
return true, nil
}
}
if len(restore) == 0 {
return false, nil
}
// A revert: the newer build stops first (it removes nothing), so the live data is not written while it is set aside.
stopped := false
if a.env.Stop != nil {
if res := a.env.Stop(ctx, module); res.Failed {
hold("the newer build could not be stopped before the restore: "+res.Detail, false)
return true, nil
}
stopped = true
}
// A revert that cannot restore starts the newer build again and holds the move, saying why: a module is never left stopped
// (design 47 G A-113).
holdRevert := func(words string, interrupted bool) {
if stopped {
words += "; " + a.startAgain(ctx, module, prev, others)
}
hold(words, interrupted)
}
for _, m := range restore {
pts, err := r.KeptPoints(ctx, meshrecovery.KeptPointsInput{Module: module, Kept: m.Name})
if err != nil {
holdRevert(fmt.Sprintf("restore point missing: the points of %s's %s cannot be read: %v", module, m.Name, err), false)
return true, nil
}
var point *meshrecovery.Point
for i := range pts.Points {
p := pts.Points[i]
if p.Module == module && p.Kept == m.Name && int(p.Form) == m.Form && p.Fingerprint == m.Fingerprint {
if point == nil || p.At.After(point.At) {
point = &pts.Points[i]
}
}
}
if point == nil {
holdRevert(fmt.Sprintf("restore point missing: %s's %s has none of form %d taken before %s", module, m.Name, m.Form, m.Fingerprint), false)
return true, nil
}
why := fmt.Sprintf("revert of %s's %s from %s", module, m.Name, m.Fingerprint)
_, held := a.askAndWait(ctx, backing, "restored", m, func() (string, string, string, error) {
got, err := r.RestoreKept(ctx, meshrecovery.RestoreKeptInput{Module: module, Kept: m.Name, Form: int64(m.Form),
Fingerprint: m.Fingerprint, Snapshot: point.Snapshot, Why: why})
return got.Accepted, got.Busy, got.Error, err
})
if held != nil {
holdRevert(fmt.Sprintf("%s's %s (form %d) is not restored: %s", module, m.Name, m.Form, held.Words), held.Interrupted)
return true, nil
}
if point.CatchUp != nil {
catchUps = append(catchUps, *point.CatchUp)
}
}
return false, catchUps
}
// startAgain starts the build a module ran before a revert stopped it, from the composition the agent applied or holds of its
// fingerprint, and says whether it did (design 47 G A-113).
func (a *Agent) startAgain(ctx context.Context, module string, prev nodeengine.HaveByNodeModuleValue, others []compositions.Composed) string {
a.mu.Lock()
c, ok := a.applied[module]
if !ok || c.C.Module == "" {
c, ok = a.compositionFor(module, prev.Fp)
}
a.mu.Unlock()
if !ok || a.env.Apply == nil {
return "the newer build could not be started again: its composition is not held"
}
res := a.env.Apply(context.WithoutCancel(ctx), c.C, others)
if res.Failed {
return "the newer build could not be started again: " + res.Detail
}
return "the newer build runs again"
}
// catchUp calls, once the reverted build of a module runs, the catch-up of each restore point restored, as the module's own
// tool through the tool runner (design 47 G A-108). A tool that fails is said: the build runs, and the tool can be called by hand.
func (a *Agent) catchUp(ctx context.Context, module string, calls []meshrecovery.CatchUp) {
if a.env.Recovery == nil || a.env.Recovery.CatchUp == nil {
return
}
for _, c := range calls {
if err := a.env.Recovery.CatchUp(ctx, module, c); err != nil {
a.env.Say(fmt.Sprintf("%s runs the reverted build, and its catch-up tool %s failed: %v", module, c.Tool, err))
continue
}
a.env.Say(fmt.Sprintf("%s's catch-up tool %s ran after the restore", module, c.Tool))
}
}
+319
View File
@@ -0,0 +1,319 @@
package agent
import (
"context"
"encoding/json"
"fmt"
"slices"
"strings"
"sync"
"testing"
"time"
"git.novox.be/novox/mesh-contracts/go/seats/meshrecovery"
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"github.com/novox/mesh-host/internal/buckets/bucketstest"
"github.com/novox/mesh-host/internal/compositions"
)
// A recovery module that answers as scripted: each ask takes the next answer and says the events it carries after it.
type recoveryFake struct {
mu sync.Mutex
asks []string // verb and its input, in order
answers []fakeAnswer
points []meshrecovery.Point
events chan Ending
changes []catalogue.FormChange
calls []string // stop, ask, apply, catch-up, in order
catchUp []meshrecovery.CatchUp
expired chan struct{}
unknown error // FormChanges answers it
}
type fakeAnswer struct {
accepted, busy, errWords string
then []Ending
}
func (f *recoveryFake) next(verb string, in any) (string, string, string) {
f.mu.Lock()
defer f.mu.Unlock()
f.asks = append(f.asks, fmt.Sprintf("%s %s", verb, asJSON(in)))
f.calls = append(f.calls, verb)
if len(f.answers) == 0 {
return "", "", "no answer scripted"
}
a := f.answers[0]
f.answers = f.answers[1:]
for _, e := range a.then {
f.events <- e
}
return a.accepted, a.busy, a.errWords
}
func (f *recoveryFake) env() *Recovery {
return &Recovery{
FormChanges: func(context.Context, string, string, string) ([]catalogue.FormChange, error) {
return f.changes, f.unknown
},
SnapshotKept: func(_ context.Context, in meshrecovery.SnapshotKeptInput) (meshrecovery.SnapshotKeptAnswer, error) {
a, b, e := f.next("snapshot-kept", in)
return meshrecovery.SnapshotKeptAnswer{Accepted: a, Busy: b, Error: e}, nil
},
RestoreKept: func(_ context.Context, in meshrecovery.RestoreKeptInput) (meshrecovery.RestoreKeptAnswer, error) {
a, b, e := f.next("restore-kept", in)
return meshrecovery.RestoreKeptAnswer{Accepted: a, Busy: b, Error: e}, nil
},
KeptPoints: func(context.Context, meshrecovery.KeptPointsInput) (meshrecovery.KeptPointsAnswer, error) {
f.mu.Lock()
defer f.mu.Unlock()
f.calls = append(f.calls, "kept-points")
return meshrecovery.KeptPointsAnswer{Points: f.points}, nil
},
Hear: func(context.Context, string, string, string) (<-chan Ending, func(), error) {
return f.events, func() {}, nil
},
Bound: func(context.Context, string, string, string, time.Time, time.Duration) (<-chan struct{}, func(), error) {
return f.expired, func() {}, nil
},
CatchUp: func(_ context.Context, _ string, c meshrecovery.CatchUp) error {
f.mu.Lock()
defer f.mu.Unlock()
f.calls = append(f.calls, "catch-up")
f.catchUp = append(f.catchUp, c)
return nil
},
Now: time.Now,
}
}
func (f *recoveryFake) seen() []string {
f.mu.Lock()
defer f.mu.Unlock()
return slices.Clone(f.calls)
}
// moveDB runs a data module at fp "a", then wants it at fp "b", and answers the bed once the second want is settled or held.
func moveDB(t *testing.T, f *recoveryFake, ranFrom, to string, settle func(b *bed) bool, before ...func(b *bed, dig string)) *bed {
t.Helper()
b := newBed(t)
inner, applies := b.env.Apply, 0
b.env.Apply = func(ctx context.Context, c compositions.Composed, others []compositions.Composed) Result {
if applies++; applies >= 2 {
f.mu.Lock()
f.calls = append(f.calls, "apply")
f.mu.Unlock()
}
return inner(ctx, c, others)
}
f.events, f.expired = make(chan Ending, 16), make(chan struct{})
b.env.Recovery = f.env()
b.env.Stop = func(context.Context, string) Result {
f.mu.Lock()
f.calls = append(f.calls, "stop")
f.mu.Unlock()
return Result{}
}
stop := b.run()
t.Cleanup(stop)
fp1, fp2 := fpOf(ranFrom), fpOf(to)
ck, craw, dig := composition(t, "db", fp1, true, file("db", "x", "1"))
wk, wraw := wantFor(t, "db", fp1, dig, "")
b.comps.put(ck, craw)
b.wants.put(wk, wraw)
eventually(t, "the first build is not applied", func() bool { return len(b.appliedList()) == 1 })
b.idle()
ck2, craw2, dig2 := composition(t, "db", fp2, true, file("db", "x", "2"))
for _, h := range before {
h(b, dig)
}
wk2, wraw2 := wantFor(t, "db", fp2, dig2, "")
b.comps.put(ck2, craw2)
b.wants.put(wk2, wraw2)
eventually(t, "the move is neither applied nor held", func() bool { return settle(b) })
return b
}
// testAMoveAcrossAFormChange is design 47 G §296, §297, §554, §560, §630 to §632 and A-98, A-108 at the node agent.
func testAMoveAcrossAFormChange(t *testing.T) {
up := []catalogue.FormChange{{Name: "cache", Kind: catalogue.KeptKindState, Running: &catalogue.KeptLayout{Form: 1},
Applied: &catalogue.KeptLayout{Form: 2, CatchUp: &catalogue.CatchUp{Tool: "rebuild", Arguments: map[string]any{"all": true}}}}}
down := []catalogue.FormChange{{Name: "cache", Kind: catalogue.KeptKindState, Running: &catalogue.KeptLayout{Form: 2},
Applied: &catalogue.KeptLayout{Form: 1}}}
fp := func(c string) string { return fpOf(c) }
done := func(act string) Ending { return Ending{Act: act, Outcome: meshrecovery.EndedEventOutcomeDone} }
failed := func(act, why string) Ending {
e := Ending{Act: act, Outcome: meshrecovery.EndedEventOutcomeFailed, Error: why}
if why == "cancelled" {
e.Why = "the operator stopped it" // the hand act's words (design 47 G A-117)
}
return e
}
applied := func(b *bed) bool { return len(b.appliedList()) == 2 }
heldHave := func(b *bed) bool { return b.haves.last("db").Phase == "held" }
t.Run("a form that rises is taken as a restore point before the move, with the why and the catch-up, and then applied", func(t *testing.T) {
f := &recoveryFake{changes: up, answers: []fakeAnswer{{accepted: "a1", then: []Ending{done("a1")}}}}
moveDB(t, f, "a", "b", applied)
if got := f.seen(); !slices.Equal(got, []string{"snapshot-kept", "apply"}) {
t.Fatalf("calls %v", got)
}
want := fmt.Sprintf("snapshot-kept %s", asJSON(meshrecovery.SnapshotKeptInput{Module: "db", Kept: "cache", Form: 1, Fingerprint: fp("b"),
Why: "form change of db's cache to " + fp("b"), CatchUp: &meshrecovery.CatchUp{Tool: "rebuild", Arguments: map[string]any{"all": true}}}))
if f.asks[0] != want {
t.Fatalf("asked %s, want %s", f.asks[0], want)
}
})
t.Run("busy waits for the other act's ending and asks again, and a stopped act is asked again; another act's failure wakes nothing", func(t *testing.T) {
f := &recoveryFake{changes: up, answers: []fakeAnswer{
{busy: "x1", then: []Ending{failed("zz", "cap"), done("x1")}},
{accepted: "a2", then: []Ending{failed("a2", "stopped")}},
{accepted: "a3", then: []Ending{done("a3")}},
}}
moveDB(t, f, "a", "b", applied)
if got := f.seen(); !slices.Equal(got, []string{"snapshot-kept", "snapshot-kept", "snapshot-kept", "apply"}) {
t.Fatalf("calls %v", got)
}
})
for _, row := range []struct{ error, held string }{
{"cancelled", "cancelled: the operator stopped it"},
{"cap", "cap"},
{"restore point damaged", "restore point damaged"},
{"disk full", "disk full"},
} {
t.Run("an ending failed with "+row.error+" holds the move and asks nothing again", func(t *testing.T) {
f := &recoveryFake{changes: up, answers: []fakeAnswer{{accepted: "a1", then: []Ending{failed("a1", row.error)}}}}
b := moveDB(t, f, "a", "b", heldHave)
if h := b.haves.last("db"); h.FP != fp("a") || !contains(h.Detail, row.held) || h.Wait == nil || h.Wait.Reason == "" {
t.Fatalf("the have %+v", h)
}
if got := f.seen(); len(got) != 1 {
t.Fatalf("asked again: %v", got)
}
if n := len(b.appliedList()); n != 1 {
t.Fatalf("applied %d", n)
}
})
}
t.Run("the bound passed says the wait in the have and the wait goes on to the ending", func(t *testing.T) {
f := &recoveryFake{changes: up, answers: []fakeAnswer{{accepted: "a1"}}}
b := moveDB(t, f, "a", "b", func(b *bed) bool {
b.haves.mu.Lock()
defer b.haves.mu.Unlock()
for _, h := range b.haves.log {
if h.Wait != nil && h.Wait.Words != "" && h.Phase == "backing-up" {
return true
}
}
return false
}, func(*bed, string) { close(f.expired) })
f.events <- done("a1")
eventually(t, "the move went on after the ending", func() bool { return applied(b) })
})
t.Run("a revert stops the newer build, restores the point of the form it reverts from, applies, then calls its catch-up", func(t *testing.T) {
f := &recoveryFake{changes: down,
points: []meshrecovery.Point{
{Module: "db", Kept: "cache", Form: 1, Fingerprint: fp("z"), Snapshot: "other", At: time.Now()},
{Module: "db", Kept: "cache", Form: 1, Fingerprint: fp("a"), Snapshot: "snap-1", At: time.Now(), CatchUp: &meshrecovery.CatchUp{Tool: "rebuild"}}},
answers: []fakeAnswer{{accepted: "r1", then: []Ending{done("r1")}}}}
moveDB(t, f, "a", "b", func(b *bed) bool { return len(f.seen()) >= 5 })
if got := f.seen(); !slices.Equal(got, []string{"stop", "kept-points", "restore-kept", "apply", "catch-up"}) {
t.Fatalf("calls %v", got)
}
if f.catchUp[0].Tool != "rebuild" {
t.Fatalf("catch-up %+v", f.catchUp)
}
want := fmt.Sprintf("restore-kept %s", asJSON(meshrecovery.RestoreKeptInput{Module: "db", Kept: "cache", Form: 1, Fingerprint: fp("a"),
Snapshot: "snap-1", Why: "revert of db's cache from " + fp("a")}))
if f.asks[0] != want {
t.Fatalf("asked %s, want %s", f.asks[0], want)
}
})
t.Run("a revert with no point of its form holds as a restore point missing", func(t *testing.T) {
f := &recoveryFake{changes: down}
b := moveDB(t, f, "a", "b", heldHave)
if h := b.haves.last("db"); !contains(h.Detail, "restore point missing") {
t.Fatalf("the have %+v", h)
}
})
t.Run("a held revert, then the want with revert live, starts the older build with no restore, no aside and no catch-up (design 47 G section 710)", func(t *testing.T) {
f := &recoveryFake{changes: down}
b := moveDB(t, f, "a", "b", heldHave)
if h := b.haves.last("db"); !contains(h.Detail, "restore point missing") {
t.Fatalf("the have %+v", h)
}
before := len(b.appliedList())
_, _, dig2 := composition(t, "db", fpOf("b"), true, file("db", "x", "2"))
raw, err := bucketstest.Want(meshwants.WantsByWantNodeModuleValue{Fp: fpOf("b"), Composed: dig2, Revert: meshwants.WantsByWantNodeModuleValueRevertLive,
Words: "the operator answered \"live\" to the revert ask (asking 1)"})
if err != nil {
t.Fatal(err)
}
b.wants.put(meshwants.WantsByWantNodeModuleKey(node, "db"), raw)
eventually(t, "the older build runs on the answer", func() bool { h := b.haves.last("db"); return h.FP == fpOf("b") && h.Phase != "held" })
if got := f.seen(); !slices.Equal(got, []string{"stop", "kept-points", "apply", "apply"}) && len(b.appliedList()) <= before {
t.Fatalf("calls %v", got)
}
for _, c := range f.seen() {
if c == "restore-kept" || c == "snapshot-kept" || c == "catch-up" {
t.Fatalf("calls %v", f.seen())
}
}
if n := strings.Count(strings.Join(f.seen(), " "), "kept-points"); n != 1 {
t.Fatalf("the points were read again on the answer: %v", f.seen())
}
})
t.Run("a revert whose restore fails starts the newer build again and holds the move, saying why (A-113)", func(t *testing.T) {
f := &recoveryFake{changes: down,
points: []meshrecovery.Point{{Module: "db", Kept: "cache", Form: 1, Fingerprint: fp("a"), Snapshot: "snap-1", At: time.Now()}},
answers: []fakeAnswer{{accepted: "r1", then: []Ending{failed("r1", "restore point damaged")}}}}
b := moveDB(t, f, "a", "b", heldHave)
if got := f.seen(); !slices.Equal(got, []string{"stop", "kept-points", "restore-kept", "apply"}) {
t.Fatalf("calls %v", got)
}
if h := b.haves.last("db"); !contains(h.Detail, "restore point damaged") || !contains(h.Detail, "the newer build runs again") ||
h.Wait == nil || h.Wait.Step != nodeengine.WaitStepCompose {
t.Fatalf("the have %+v", h)
}
})
t.Run("a form change not known moves nothing and waits for the entries (A-113)", func(t *testing.T) {
f := &recoveryFake{unknown: fmt.Errorf("the entry is asked")}
b := moveDB(t, f, "a", "b", heldHave)
if h := b.haves.last("db"); h.FP != fp("a") || !contains(h.Detail, "is not known") || h.Wait == nil || h.Wait.Reason != "entry-missing" {
t.Fatalf("the have %+v", h)
}
if n := len(b.appliedList()); n != 1 || len(f.seen()) != 0 {
t.Fatalf("applied %d, calls %v", n, f.seen())
}
})
t.Run("a restart waits for the act its have names and asks nothing again", func(t *testing.T) {
f := &recoveryFake{changes: up}
b := moveDB(t, f, "a", "b", applied, func(b *bed, dig string) {
at := time.Now().Add(-time.Minute)
b.haves.mu.Lock()
b.haves.stored["db"] = Have{Module: "db", FP: fp("a"), Phase: "backing-up", Wait: &nodeengine.Wait{
On: "mesh-recovery.ended.db.cache", AskAt: &at, Act: "a1", Since: at}}
b.haves.mu.Unlock()
f.events <- done("a1")
})
_ = b
if got := f.seen(); !slices.Equal(got, []string{"apply"}) {
t.Fatalf("calls %v", got)
}
})
}
func contains(s, sub string) bool { return strings.Contains(s, sub) }
func asJSON(v any) string { raw, _ := json.Marshal(v); return string(raw) }
+89
View File
@@ -0,0 +1,89 @@
package agent
import (
"encoding/json"
"fmt"
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
// PartsDiffer is the start of the detail of a have held because a want's parts and the digests its composition fetches
// disagree (design 47 G section 524).
const PartsDiffer = "parts-differ:"
// EntryAsked and EntryMissing open the detail of a have held for the entry the want names (design 47 G section 421 and
// section 614): the entry is asked of the catalog half and not yet answered, or the half answered that it holds none. Both
// are the phase held, and each names the fingerprint.
const (
EntryAsked = "entry-asked:"
EntryMissing = "entry-missing:"
)
// heldForEntry says whether why is a hold for the entry a want names.
func heldForEntry(why string) bool {
return strings.HasPrefix(why, EntryAsked) || strings.HasPrefix(why, EntryMissing)
}
// partsDifference checks a want's `parts` against the digests of the archive, process and container resources of the
// composition, which are the digests the apply fetches and checks (design 47 G section 524). A want with no `parts` (an
// assigner of before G, or a node whose record does not state `want-parts`) is applied as today: it returns "". With
// `parts`, the want is applied only when the digest of every resource fetched is among the values of `parts` and every
// value of `parts` is the digest of a resource fetched; otherwise it returns the detail naming each digest.
func partsDifference(parts map[string]string, resources []json.RawMessage) string {
if len(parts) == 0 {
return ""
}
fetched := map[string]bool{}
for _, raw := range resources {
r, err := declaration.ParseResource(raw)
if err != nil {
continue
}
switch res := r.(type) {
case *declaration.Archive:
if res.Source != "" {
fetched[res.Digest] = true
}
case *declaration.Process:
if res.Source != "" {
fetched[res.Digest] = true
}
case *declaration.Container:
if strings.HasPrefix(res.Image, "sha256:") {
fetched[res.Image] = true
} else if _, digest, ok := strings.Cut(res.Image, "@"); ok {
fetched[digest] = true
}
}
}
named := map[string]bool{}
for _, d := range parts {
named[d] = true
}
var outside, missing []string
for d := range fetched {
if !named[d] {
outside = append(outside, d)
}
}
for d := range named {
if !fetched[d] {
missing = append(missing, d)
}
}
if len(outside) == 0 && len(missing) == 0 {
return ""
}
sort.Strings(outside)
sort.Strings(missing)
var said []string
if len(outside) > 0 {
said = append(said, "fetched and not in the want's parts: "+strings.Join(outside, ", "))
}
if len(missing) > 0 {
said = append(said, "in the want's parts and not fetched: "+strings.Join(missing, ", "))
}
return fmt.Sprintf("%s %s", PartsDiffer, strings.Join(said, "; "))
}
+66 -8
View File
@@ -578,6 +578,17 @@ func applyScoped(
}
if declaration.Type(orphan.Type) == declaration.TypeService {
if by, held := heldUnits[unitKey(orphan.Scope, orphan.User, orphan.Target)]; held {
// **What the going record found passes to the record that stays** (design 42 zfs-owns-zfs §2, answer
// A-21), as a file is handed to the resource still declaring its path (ADR 0223): the unit as the machine
// had it before the mesh is what the staying record gives back when it goes, not what the mesh made of
// it. Only where the staying record has no finding of its own, kept as an unfinished apply's is.
if orphan.Found != nil && declared[by] {
if rec, has := known.Find(by); !has || rec.Found == nil {
if _, pending := known.FoundFirst[by]; !pending {
known.KeepFound(by, origin, *orphan.Found)
}
}
}
known.Forget(orphan.ID)
detail := "no longer declared; " + by + " still holds the unit, so it was left as it is"
report.Outcomes = append(report.Outcomes, Outcome{
@@ -708,10 +719,24 @@ func applyScoped(
ordered = append(afterWhatTheyRead(guard), afterWhatTheyRead(rest)...)
guardFirst = len(guard)
}
// Modules whose own step did not complete. What follows *within such a module* is not attempted;
// the rest of the machine is (novox/hq ADR 0136).
gated := map[string]bool{}
// Why each gated module was gated, said on every resource of it that was skipped.
gatedWhy := map[string]string{}
orphansRemoved := false
removeOrphans := func() error {
orphansRemoved = true
for _, orphan := range orphans {
// **Nothing of a module whose image or artifact could not be had is removed either** (novox/hq issue 1966):
// a removal is as much a change of what it runs as a replacement, and the module keeps running what it ran.
if m := orphanModule(orphan); m != "" && gated[m] {
report.Outcomes = append(report.Outcomes, Outcome{
ID: orphan.ID, Type: orphan.Type, Target: orphan.Target, Action: "kept", Detail: gatedWhy[m],
})
log(fmt.Sprintf(" kept %s (%s): %s", orphan.ID, orphan.Target, gatedWhy[m]))
continue
}
if err := removeOrphan(orphan); err != nil {
return err
}
@@ -760,7 +785,7 @@ func applyScoped(
// change one apply late, and never misses it.
// And one patience with the artifact store for the whole apply (novox/hq issue 291): a fetch it
// does not answer during a maintenance window waits for the window instead of failing.
in := inputs{declares: map[string]string{}, known: &known, windows: windows, away: newStoreAway(windows, log),
in := inputs{node: d.For, declares: map[string]string{}, known: &known, windows: windows, away: newStoreAway(windows, log),
groups: wanted, handed: handed}
// Where nothing is placed or mounted from, judged before anything is applied (novox/hq issue 339), and
// which declared directories were already here: one this apply makes is the mesh's, one found may not be.
@@ -797,11 +822,6 @@ func applyScoped(
var failures []*Error
// Which resources applied without error in this run, for the move-out pass after the loop (issue 1024).
appliedOK := map[string]bool{}
// Modules whose own step did not complete. What follows *within such a module* is not attempted;
// the rest of the machine is (novox/hq ADR 0136).
gated := map[string]bool{}
// Why each gated module was gated, said on every resource of it that was skipped.
gatedWhy := map[string]string{}
gatedModules := func() []string {
out := make([]string, 0, len(gated))
for m := range gated {
@@ -946,6 +966,11 @@ func applyScoped(
// follows in the module is skipped as it is for any step that did not complete.
in.pulled = map[string]bool{}
pullFirst(ctx, ordered, d, in.pulled, run, &failures, gated, gatedWhy, log)
// And every archive and process, fetched before anything is replaced (issue 1966). Its failures are counted with the
// others once the resources are through, as an identity's are: here, before the guard, they would read as the guard
// not being up, and nothing of the machine would be applied.
var fetchFailures []*Error
fetchFirst(ctx, ordered, d, &in, &fetchFailures, gated, gatedWhy, log)
for i, resource := range ordered {
if !orphansRemoved && i == guardFirst {
// **Only a guard that is up may let the filter go.** Removing the derived filter's
@@ -1148,6 +1173,16 @@ func applyScoped(
if p, ok := resource.(*declaration.Process); ok && p.RunOnce {
gates = true
}
// **A module's apply stops at its first failed resource** (design 42 zfs-owns-zfs §1, ADR 0679 §16 (a)): any
// other resource of a module that failed gates the rest of that module, which is reported skipped with this
// reason. What it applied before stays applied; other modules, and a resource of no module, go on as before.
if module, ours := d.ModuleOfID(resource.Identity()); ours && !gates {
failed.Gated = true
gated[module] = true
gatedWhy[module] = fmt.Sprintf("%s failed, so nothing more of this module was attempted", resource.Identity())
log(fmt.Sprintf(" gated %s.*: %s failed, so the rest of it was not attempted", module, resource.Identity()))
continue
}
if gates {
failed.Gated = true
// **A step that failed has not completed, whatever it did before** (novox/hq issue 467). A
@@ -1304,6 +1339,7 @@ func applyScoped(
}
failures = append(failures, identityFailures...)
failures = append(failures, fetchFailures...)
// A directory or archive that says move-out keeps only what it knows (issue 1024, novox/hq ADR 0422).
movedOut, leftIn, moveFailures := moveOutPass(d, known, appliedOK, log)
@@ -1579,9 +1615,17 @@ func applyOne(ctx context.Context, sys system.System, r declaration.Resource, ru
case *declaration.User:
return applyUser(ctx, sys, res, run, previous, in.groups, in.handed)
case *declaration.Archive:
return applyArchive(ctx, res, previous, in.away, journalDir(in.windows))
held, err := in.artifact(res.Identity())
if err != nil {
return begin(r), err
}
return applyArchive(ctx, res, previous, held, in.away, journalDir(in.windows))
case *declaration.Process:
return applyProcess(ctx, res, run, changed, previous, in.away)
held, err := in.artifact(res.Identity())
if err != nil {
return begin(r), err
}
return applyProcess(ctx, withNode(res, in.node), run, changed, previous, held, in.away)
case *declaration.Action:
return applyAction(ctx, res, run)
case *declaration.Network:
@@ -2769,6 +2813,12 @@ type inputs struct {
// pulled is every image this apply found on the machine, or pulled, before any container was replaced (pullFirst):
// applyContainer does not ask for it again.
pulled map[string]bool
// fetched is every archive's and process's bytes this apply fetched before anything was replaced (fetchFirst), each
// taken by its applier once (artifact).
fetched map[string]fetchedArtifact
// node is the node's own name, the declaration's For: every process runs with it as MESH_NODE beside its own env
// (design 47 G A-342). Empty where the declaration names none (the first node's carried bundle).
node string
}
// fileDigest is what a file the container reads holds, by digest.
@@ -4354,6 +4404,14 @@ func ownerText(info os.FileInfo) string {
return fmt.Sprintf("%d:%d", uid, gid)
}
// orphanModule is the module a record going belongs to, by its id, else the module whose want it was applied for.
func orphanModule(a store.Applied) string {
if m, ok := declaration.ModuleGuess(a.ID); ok {
return m
}
return a.Module
}
// pullFirst finds on the machine, or pulls, the image of each container the declaration runs before any is replaced
// (ENG-7) for a module of two containers or more. A container that is run once is left to its marker, and a runtime that
// does not answer to applyContainer, which says so. The first image that cannot be had fails its container with the image named and gates its module, so none of
+80 -64
View File
@@ -182,77 +182,93 @@ func TestARenameToTheSamePathDoesNotDeleteTheNewFile(t *testing.T) {
}
}
func TestAFailedStepFailsTheApplyAndTheRestIsStillAttempted(t *testing.T) {
// The apply fails, names the resource, and carries what did happen — because the machine is
// in whatever state the apply reached and the only honest thing to hand back is that list.
//
// **And everything is attempted.** It used to stop at the first failure, which made one
// broken resource hold the whole machine hostage: a module declaring a package that does not
// exist meant every module after it was never applied, for ever
// (novox/hq 04-ISSUES/011). The case for stopping was that a later resource may depend on an
// earlier one — and it still may, and it then fails its own check and is reported, which is
// more information than not attempting it.
// A failed resource fails the apply and names itself, and what else is attempted depends on whose it is
// (novox/hq 04-ISSUES/011; design 42 zfs-owns-zfs §1, ADR 0679 §16 (a)): the resources of no module are all
// attempted, so one broken resource holds no machine hostage and every failure is counted, not just the first;
// a module's apply stops at its first failed resource, reports each one left, and no other module is touched.
func TestAFailedResourceFailsTheApplyAndWhatFollowsIsAttemptedByWhoseItIs(t *testing.T) {
dir := t.TempDir()
blocker := filepath.Join(dir, "blocker")
if err := os.WriteFile(blocker, []byte("i am a file\n"), 0o644); err != nil {
t.Fatal(err)
}
d := parse(t, `{"declaration":1,"resources":[
{"id":"fine","type":"file","path":"`+filepath.Join(dir, "fine.conf")+`","content":"a\n"},
{"id":"doomed","type":"directory","path":"`+blocker+`"},
{"id":"after","type":"file","path":"`+filepath.Join(dir, "after.conf")+`","content":"b\n"}
]}`)
_, _, err := Apply(context.Background(), archHost(t), d, store.State{}, store.OriginCarried, noServices, nil, nil)
if err == nil {
t.Fatal("an impossible resource did not fail the apply")
}
var applyErr *Error
if !errors.As(err, &applyErr) {
t.Fatalf("expected an apply error, got %T", err)
}
if applyErr.Resource != "doomed" {
t.Errorf("the failure names %q, not the resource that failed", applyErr.Resource)
}
// Everything that worked is in the report, before and after the failure.
if len(applyErr.Done.Outcomes) != 2 {
t.Errorf("the error does not carry what was applied: %+v", applyErr.Done.Outcomes)
}
if _, err := os.Stat(filepath.Join(dir, "after.conf")); err != nil {
t.Error("a resource after the failing one was never attempted, so one broken module " +
"still blocks every module after it")
}
}
func TestEveryFailureIsCountedNotJustTheFirst(t *testing.T) {
// "One thing failed" and "eleven things failed" are different machines, and the first line is
// what somebody reads.
dir := t.TempDir()
for _, name := range []string{"one", "two"} {
if err := os.WriteFile(filepath.Join(dir, name), []byte("a file\n"), 0o644); err != nil {
for _, name := range []string{"blocker", "one", "two"} {
if err := os.WriteFile(filepath.Join(dir, name), []byte("i am a file\n"), 0o644); err != nil {
t.Fatal(err)
}
}
d := parse(t, `{"declaration":1,"resources":[
{"id":"first","type":"directory","path":"`+filepath.Join(dir, "one")+`"},
{"id":"second","type":"directory","path":"`+filepath.Join(dir, "two")+`"}
]}`)
_, _, err := Apply(context.Background(), archHost(t), d, store.State{}, store.OriginCarried, noServices, nil, nil)
if err == nil {
t.Fatal("two impossible resources did not fail the apply")
file := func(id, name string) string {
return `{"id":"` + id + `","type":"file","path":"` + filepath.Join(dir, name) + `","content":"a\n"}`
}
var applyErr *Error
if !errors.As(err, &applyErr) {
t.Fatalf("got %T", err)
unwritable := func(id, name string) string { // a file below a file: it cannot be made
return `{"id":"` + id + `","type":"file","path":"` + filepath.Join(dir, name, "x.conf") + `","content":"a\n"}`
}
if applyErr.Others != 1 {
t.Errorf("the failure says %d others also failed, and one did", applyErr.Others)
doomed := func(id, name string) string {
return `{"id":"` + id + `","type":"directory","path":"` + filepath.Join(dir, name) + `"}`
}
if !strings.Contains(err.Error(), "one other resource also failed") {
t.Errorf("the message does not say others failed: %v", err)
exists := func(name string) bool {
_, err := os.Stat(filepath.Join(dir, name))
return err == nil
}
for _, tc := range []struct {
name string
resources []string
failed string
others int
applied []string // files that must exist
left []string // files that must not, reported skipped
detail string // what the module's result says, when it is one module's
}{
{"of no module, the rest is attempted", []string{file("fine", "fine.conf"), doomed("doomed", "blocker"), file("after", "after.conf")},
"doomed", 0, []string{"fine.conf", "after.conf"}, nil, ""},
{"of no module, every failure is counted", []string{doomed("first", "one"), doomed("second", "two")},
"first", 1, nil, nil, ""},
{"of a module, it stops there and names each one left",
[]string{file("m.first", "first.conf"), unwritable("m.second", "blocker"), file("m.third", "third.conf"), file("m.fourth", "fourth.conf"), file("o.other", "other.conf")},
"m.second", 0, []string{"first.conf", "other.conf"}, []string{"third.conf", "fourth.conf"},
"m.second failed"},
} {
t.Run(tc.name, func(t *testing.T) {
d := parse(t, `{"declaration":1,"resources":[`+strings.Join(tc.resources, ",")+`]}`)
report, _, err := Apply(context.Background(), archHost(t), d, store.State{}, store.OriginCarried, noServices, nil, nil)
var applyErr *Error
if !errors.As(err, &applyErr) {
t.Fatalf("expected an apply error, got %v", err)
}
if applyErr.Resource != tc.failed || applyErr.Others != tc.others {
t.Errorf("the failure names %q with %d others, want %q with %d", applyErr.Resource, applyErr.Others, tc.failed, tc.others)
}
if tc.others == 1 && !strings.Contains(err.Error(), "one other resource also failed") {
t.Errorf("the message does not say others failed: %v", err)
}
for _, name := range tc.applied {
if !exists(name) {
t.Errorf("%s was not applied", name)
}
}
for _, name := range tc.left {
if exists(name) {
t.Errorf("%s was applied after its module failed", name)
}
}
if tc.detail == "" {
return
}
outcomes := applyErr.Done.Outcomes
skipped := 0
for _, o := range outcomes {
if o.Action == "skipped" {
skipped++
}
}
if skipped != len(tc.left) {
t.Errorf("%d resources reported skipped, want %d: %+v", skipped, len(tc.left), outcomes)
}
result := ModuleResults(d, report, err)["m"]
if !result.Failed || !strings.Contains(result.Detail, tc.detail) ||
!strings.Contains(result.Detail, "m.third") || !strings.Contains(result.Detail, "m.fourth") {
t.Errorf("the module's result does not name the failed resource and each one left: %+v", result)
}
if other := ModuleResults(d, report, err)["o"]; other.Failed {
t.Errorf("another module was failed by it: %+v", other)
}
})
}
}
+7 -4
View File
@@ -35,7 +35,7 @@ import (
// enough for a desktop theme and small enough to notice.
const maxArchive = 512 << 20
func applyArchive(ctx context.Context, r *declaration.Archive, previous store.Applied, away *storeAway, journals string) (Outcome, error) {
func applyArchive(ctx context.Context, r *declaration.Archive, previous store.Applied, held []byte, away *storeAway, journals string) (Outcome, error) {
out := begin(r)
out.Action = "unchanged"
recorded := previous.Wrote != ""
@@ -44,9 +44,12 @@ func applyArchive(ctx context.Context, r *declaration.Archive, previous store.Ap
return out, err
}
body, err := fetchMinding(ctx, r.Source, away)
if err != nil {
return out, err
// What the apply fetched before anything was replaced (fetchFirst), else fetched now.
body, err := held, error(nil)
if body == nil {
if body, err = fetchMinding(ctx, r.Source, away); err != nil {
return out, err
}
}
sum := sha256.Sum256(body)
got := "sha256:" + hex.EncodeToString(sum[:])
+113
View File
@@ -0,0 +1,113 @@
package apply
import (
"context"
"crypto/sha256"
"encoding/hex"
"fmt"
"git.novox.be/novox/mesh-sdk/go/declaration"
)
// **Every artifact an apply fetches is in hand before anything is replaced** (novox/hq issue 1966, 2026-11-08 23:25 UTC).
//
// An archive and a process are fetched from the artifact store on their own turn, and a container is replaced on its
// own: so a module whose container serves the store (distribution) had that container replaced, cut off from where it
// was reached, and then failed to fetch its program from the store it had just cut off, left half applied with the
// store unreachable from every other machine. pullFirst does this for a module's images (ENG-7); fetchFirst does it for
// its archives and processes, before any of the apply's resources is applied or any orphan removed.
//
// A fetch that fails (the store does not answer for the apply's patience, or what arrived is not what was declared)
// gates the resource's module, as a pull that fails does: nothing of the module is replaced or removed, the failure names
// the artifact, and the rest of the machine is applied. A resource of no module keeps its failure for its own turn.
// fetchedArtifact is one archive's or process's bytes, fetched and checked before the apply began, or why it was not.
type fetchedArtifact struct {
body []byte
err error
}
// artifactOf is the archive's or process's source and digest, and whether the resource is one the apply fetches.
func artifactOf(r declaration.Resource) (source, digest string, ok bool) {
switch res := r.(type) {
case *declaration.Archive:
return res.Source, res.Digest, res.Source != ""
case *declaration.Process:
return res.Source, res.Digest, res.Source != ""
}
return "", "", false
}
// fetchFirst fetches each archive and process the declaration applies, checks each against its declared digest, and
// keeps the bytes for its turn (in.fetched). It is called before the loop that applies, after pullFirst.
func fetchFirst(ctx context.Context, ordered []declaration.Resource, d *declaration.Declaration, in *inputs,
failures *[]*Error, gated map[string]bool, gatedWhy map[string]string, log func(string)) {
in.fetched = map[string]fetchedArtifact{}
bySource := map[string]fetchedArtifact{}
for _, resource := range ordered {
source, digest, isArtifact := artifactOf(resource)
if !isArtifact {
continue
}
module, ours := d.ModuleOfID(resource.Identity())
if ours && gated[module] {
continue
}
// On an adopted node, what a module not yet taken declares may be held as found and never fetched (hold.go): its
// own turn fetches it, as before.
if d.Adoption != nil {
if _, untaken := d.Adoption.UntakenModuleOf(resource.Identity()); untaken || runsIn(resource) != "" {
continue
}
}
// What the module's done build kept on a build node is fetched from there, not from the store (design 47).
cacheKey, kept := source, false
if kf := keptFetchFrom(ctx); kf != nil && ours {
cacheKey = "kept " + module + " " + digest
if _, seen := bySource[cacheKey]; seen {
kept = true
} else if body, isKept, err := kf(ctx, module, digest); isKept {
kept = true
bySource[cacheKey] = fetchedArtifact{body: body, err: err}
}
}
got, seen := bySource[cacheKey]
if !seen && !kept {
got.body, got.err = fetchMinding(ctx, source, in.away)
bySource[cacheKey] = got
}
if got.err == nil {
sum := sha256.Sum256(got.body)
if have := "sha256:" + hex.EncodeToString(sum[:]); have != digest {
got = fetchedArtifact{err: fmt.Errorf("%s was declared as %s and what arrived is %s; nothing was unpacked",
source, digest, have)}
}
}
if got.err == nil {
in.fetched[resource.Identity()] = got
continue
}
if !ours {
// Nothing to gate: the resource fails on its own turn with this error.
in.fetched[resource.Identity()] = got
continue
}
err := fmt.Errorf("not replacing anything of its module, which keeps running what it ran: %w", got.err)
*failures = append(*failures, &Error{Resource: resource.Identity(), Err: err})
log(fmt.Sprintf(" failed %s (%s): %v", resource.Identity(), resource.Target(), err))
gated[module] = true
gatedWhy[module] = "its artifact " + source + " could not be fetched, so nothing of it was replaced " +
"(novox/hq issue 1966)"
}
}
// artifact is the bytes fetchFirst holds for a resource, taken once so they are not kept past their turn; nil and no
// error when it holds none (an apply that did not fetch first: its applier fetches itself).
func (in inputs) artifact(id string) ([]byte, error) {
got, ok := in.fetched[id]
if !ok {
return nil, nil
}
delete(in.fetched, id)
return got.body, got.err
}
+1 -1
View File
@@ -95,7 +95,7 @@ func WriteFilledWith(
"yet taken (%s), so nothing was written", path, held.Why)}
}
in := inputs{declares: map[string]string{}, known: &known, windows: windows, away: newStoreAway(windows, log),
in := inputs{node: d.For, declares: map[string]string{}, known: &known, windows: windows, away: newStoreAway(windows, log),
groups: groupsWanted(d.Resources), handed: Handed{}}
in.refused = refusedPlaces(d.Resources)
in.agentHomes = rootNeverHomes(d)
+85 -37
View File
@@ -283,7 +283,7 @@ func predecessor(t *testing.T) (dir, page string, m *machine) {
}}
}
func TestAFoundFileOfAnUntakenModuleIsKeptAsItIs(t *testing.T) {
func caseAFoundFileOfAnUntakenModuleIsKeptAsItIs(t *testing.T) {
dir, page, m := predecessor(t)
report, state := applyAdopted(t, adopted(t, untakenWeb, webResources(page)), store.State{}, m, dir)
@@ -318,7 +318,7 @@ func TestAFoundFileOfAnUntakenModuleIsKeptAsItIs(t *testing.T) {
}
}
func TestAFoundContainerOfAnUntakenModuleIsNotReplaced(t *testing.T) {
func caseAFoundContainerOfAnUntakenModuleIsNotReplaced(t *testing.T) {
dir, page, m := predecessor(t)
report, state := applyAdopted(t, adopted(t, untakenWeb, webResources(page)), store.State{}, m, dir)
@@ -338,7 +338,7 @@ func TestAFoundContainerOfAnUntakenModuleIsNotReplaced(t *testing.T) {
}
}
func TestWhatIsNotFoundIsCreatedWhenAssigned(t *testing.T) {
func caseWhatIsNotFoundIsCreatedWhenAssigned(t *testing.T) {
// Assigning prepares: what the module declares that is not there is made.
dir := t.TempDir()
page := filepath.Join(dir, "index.html")
@@ -355,7 +355,7 @@ func TestWhatIsNotFoundIsCreatedWhenAssigned(t *testing.T) {
}
}
func TestAFileThisHostWroteIsNotFound(t *testing.T) {
func caseAFileThisHostWroteIsNotFound(t *testing.T) {
// Found means present with no record. A record of any origin — carried or declared, this life
// of the node or an earlier one — means this host wrote it.
for _, origin := range []string{store.OriginCarried, store.OriginDeclared} {
@@ -372,7 +372,7 @@ func TestAFileThisHostWroteIsNotFound(t *testing.T) {
}
}
func TestAContainerAHostMadeIsNotFound(t *testing.T) {
func caseAContainerAHostMadeIsNotFound(t *testing.T) {
dir, page, m := predecessor(t)
m.containers["hello-web"].spec = "some-spec"
report, _ := applyAdopted(t, adopted(t, untakenWeb, webResources(page)), store.State{}, m, dir)
@@ -381,7 +381,7 @@ func TestAContainerAHostMadeIsNotFound(t *testing.T) {
}
}
func TestTheGenesisStoreAdoptedInPlaceIsNotFound(t *testing.T) {
func caseTheGenesisStoreAdoptedInPlaceIsNotFound(t *testing.T) {
// ADR 0078: the foundation's store, raised from the bundle and recorded as carried, is adopted
// as a module by name. It is the mesh's own and must never read as a predecessor's.
dir := t.TempDir()
@@ -398,7 +398,7 @@ func TestTheGenesisStoreAdoptedInPlaceIsNotFound(t *testing.T) {
}
}
func TestTakingAModuleReplacesWhatWasHeldAndTheOriginalSurvives(t *testing.T) {
func caseTakingAModuleReplacesWhatWasHeldAndTheOriginalSurvives(t *testing.T) {
dir, page, m := predecessor(t)
_, state := applyAdopted(t, adopted(t, untakenWeb, webResources(page)), store.State{}, m, dir)
h, _ := state.HeldAt("hello-web.page")
@@ -426,7 +426,7 @@ func TestTakingAModuleReplacesWhatWasHeldAndTheOriginalSurvives(t *testing.T) {
}
}
func TestAHeldFileIsNeverRemovedWhenItsModuleIsUnassigned(t *testing.T) {
func caseAHeldFileIsNeverRemovedWhenItsModuleIsUnassigned(t *testing.T) {
dir, page, m := predecessor(t)
_, state := applyAdopted(t, adopted(t, untakenWeb, webResources(page)), store.State{}, m, dir)
@@ -447,7 +447,7 @@ func TestAHeldFileIsNeverRemovedWhenItsModuleIsUnassigned(t *testing.T) {
}
}
func TestAHoldNoLongerDeclaredIsLetGoAndFoundAgainIfItsModuleReturns(t *testing.T) {
func caseAHoldNoLongerDeclaredIsLetGoAndFoundAgainIfItsModuleReturns(t *testing.T) {
dir, page, m := predecessor(t)
_, state := applyAdopted(t, adopted(t, untakenWeb, webResources(page)), store.State{}, m, dir)
first, _ := state.HeldAt("hello-web.page")
@@ -482,7 +482,7 @@ func TestAHoldNoLongerDeclaredIsLetGoAndFoundAgainIfItsModuleReturns(t *testing.
}
}
func TestACarriedApplyLetsNoHoldGo(t *testing.T) {
func caseACarriedApplyLetsNoHoldGo(t *testing.T) {
dir, page, m := predecessor(t)
_, state := applyAdopted(t, adopted(t, untakenWeb, webResources(page)), store.State{}, m, dir)
carried := parse(t, `{"declaration":1,"resources":[`+withConf(dir)+`]}`)
@@ -496,7 +496,7 @@ func TestACarriedApplyLetsNoHoldGo(t *testing.T) {
}
}
func TestAHeldFileRewrittenIsReportedAndNotReverted(t *testing.T) {
func caseAHeldFileRewrittenIsReportedAndNotReverted(t *testing.T) {
dir, page, m := predecessor(t)
d := adopted(t, untakenWeb, webResources(page))
_, state := applyAdopted(t, d, store.State{}, m, dir)
@@ -520,7 +520,7 @@ func TestAHeldFileRewrittenIsReportedAndNotReverted(t *testing.T) {
}
}
func TestAHeldContainerStoppedOrReplacedIsReportedAndNotRestarted(t *testing.T) {
func caseAHeldContainerStoppedOrReplacedIsReportedAndNotRestarted(t *testing.T) {
for _, c := range []struct {
change func(*machine)
want string
@@ -547,7 +547,7 @@ func TestAHeldContainerStoppedOrReplacedIsReportedAndNotRestarted(t *testing.T)
}
}
func TestAHeldFileThatVanishesIsNotCreated(t *testing.T) {
func caseAHeldFileThatVanishesIsNotCreated(t *testing.T) {
dir, page, m := predecessor(t)
d := adopted(t, untakenWeb, webResources(page))
_, state := applyAdopted(t, d, store.State{}, m, dir)
@@ -563,7 +563,7 @@ func TestAHeldFileThatVanishesIsNotCreated(t *testing.T) {
}
}
func TestAConvergedNodeStillReplacesWhatItFinds(t *testing.T) {
func caseAConvergedNodeStillReplacesWhatItFinds(t *testing.T) {
// No adoption, no holds: byte for byte what a converged node did before ADR 0100.
dir, page, m := predecessor(t)
d := parse(t, `{"declaration":1,"resources":[`+webResources(page)+`]}`)
@@ -585,7 +585,7 @@ func TestAConvergedNodeStillReplacesWhatItFinds(t *testing.T) {
}
}
func TestAFileWrittenOverIsKeptOnceAndOnlyWhenTheHostHasNoRecordOfIt(t *testing.T) {
func caseAFileWrittenOverIsKeptOnceAndOnlyWhenTheHostHasNoRecordOfIt(t *testing.T) {
dir := t.TempDir()
conf := filepath.Join(dir, "nftables.conf")
_ = os.WriteFile(conf, []byte("# the distribution's own\n"), 0o644)
@@ -621,7 +621,7 @@ func untaken(ids ...string) string {
return `{"taken":[],"untaken":{"hello-web":["` + strings.Join(ids, `","`) + `"]}}`
}
func TestAFoundDirectoryOfAnUntakenModuleKeepsItsModeOwnerAndContents(t *testing.T) {
func caseAFoundDirectoryOfAnUntakenModuleKeepsItsModeOwnerAndContents(t *testing.T) {
dir := t.TempDir()
data := filepath.Join(dir, "data")
if err := os.Mkdir(data, 0o700); err != nil {
@@ -652,7 +652,7 @@ func TestAFoundDirectoryOfAnUntakenModuleKeepsItsModeOwnerAndContents(t *testing
}
}
func TestADirectoryMadeInTheSameApplyIsNotFound(t *testing.T) {
func caseADirectoryMadeInTheSameApplyIsNotFound(t *testing.T) {
// A file's parent is made as the file is written; what the mesh made is not found.
dir := t.TempDir()
data := filepath.Join(dir, "data")
@@ -671,7 +671,7 @@ func TestADirectoryMadeInTheSameApplyIsNotFound(t *testing.T) {
}
}
func TestAFoundServiceOfAnUntakenModuleIsNeitherStartedNorEnabledNorRestarted(t *testing.T) {
func caseAFoundServiceOfAnUntakenModuleIsNeitherStartedNorEnabledNorRestarted(t *testing.T) {
dir := t.TempDir()
conf := filepath.Join(dir, "hello.conf")
m := &machine{containers: map[string]*fakeContainer{},
@@ -695,7 +695,7 @@ func TestAFoundServiceOfAnUntakenModuleIsNeitherStartedNorEnabledNorRestarted(t
}
}
func TestAHeldServiceIsStillReloadedButNeverRestarted(t *testing.T) {
func caseAHeldServiceIsStillReloadedButNeverRestarted(t *testing.T) {
// A reload stops nothing (novox/hq ADR 0102); a restart would stop the predecessor's service.
dir := t.TempDir()
conf := filepath.Join(dir, "daemon.json")
@@ -716,7 +716,7 @@ func TestAHeldServiceIsStillReloadedButNeverRestarted(t *testing.T) {
}
}
func TestAUnitAPackageOnlyShipsIsNotFound(t *testing.T) {
func caseAUnitAPackageOnlyShipsIsNotFound(t *testing.T) {
// The adoption bed found this: the private network's wg-quick@mesh0 is an instance of a unit
// the tunnel package ships. Nothing had ever run it, yet it was held as a predecessor's, and
// the private network never came up. Found is what the machine runs.
@@ -738,7 +738,7 @@ func TestAUnitAPackageOnlyShipsIsNotFound(t *testing.T) {
}
}
func TestAServiceWhoseUnitIsNotThereIsAppliedAsUsual(t *testing.T) {
func caseAServiceWhoseUnitIsNotThereIsAppliedAsUsual(t *testing.T) {
// No unit before the apply: nothing of a predecessor's to hold.
dir := t.TempDir()
m := &machine{containers: map[string]*fakeContainer{}, units: map[string]*fakeUnit{}}
@@ -750,7 +750,7 @@ func TestAServiceWhoseUnitIsNotThereIsAppliedAsUsual(t *testing.T) {
}
}
func TestAContainerThatWouldMountFoundDataIsNotCreated(t *testing.T) {
func caseAContainerThatWouldMountFoundDataIsNotCreated(t *testing.T) {
dir := t.TempDir()
data := filepath.Join(dir, "predecessor-data")
if err := os.Mkdir(data, 0o700); err != nil {
@@ -788,7 +788,7 @@ func TestAContainerThatWouldMountFoundDataIsNotCreated(t *testing.T) {
}
}
func TestAContainerMountingWhatTheMeshMadeIsCreated(t *testing.T) {
func caseAContainerMountingWhatTheMeshMadeIsCreated(t *testing.T) {
dir := t.TempDir()
data := filepath.Join(dir, "data")
m := &machine{containers: map[string]*fakeContainer{}}
@@ -801,7 +801,7 @@ func TestAContainerMountingWhatTheMeshMadeIsCreated(t *testing.T) {
}
}
func TestARunOnceStepInAHeldContainerIsHeld(t *testing.T) {
func caseARunOnceStepInAHeldContainerIsHeld(t *testing.T) {
dir, page, m := predecessor(t)
step := `{"id":"hello-web.migrate","type":"container","name":"hello-web-migrate","image":"` + pinned + `",
"run-once":true,"network":"container:hello-web"}`
@@ -818,7 +818,7 @@ func TestARunOnceStepInAHeldContainerIsHeld(t *testing.T) {
}
}
func TestAnActionInAHeldContainerIsHeldUntilItsModuleIsTaken(t *testing.T) {
func caseAnActionInAHeldContainerIsHeldUntilItsModuleIsTaken(t *testing.T) {
// An action cannot arrive over the link today, and a bundle cannot say a node is adopted; the
// host holds one anyway, since what it would run in is the predecessor's.
dir, page, m := predecessor(t)
@@ -850,7 +850,7 @@ func TestAnActionInAHeldContainerIsHeldUntilItsModuleIsTaken(t *testing.T) {
const sixtyFourZeros = "0000000000000000000000000000000000000000000000000000000000000000"
func TestAnArchiveOverSomethingFoundIsNotUnpacked(t *testing.T) {
func caseAnArchiveOverSomethingFoundIsNotUnpacked(t *testing.T) {
dir := t.TempDir()
at := filepath.Join(dir, "site")
if err := os.Mkdir(at, 0o750); err != nil {
@@ -874,7 +874,7 @@ func TestAnArchiveOverSomethingFoundIsNotUnpacked(t *testing.T) {
}
}
func TestAProcessWhoseUnitIsFoundIsNotWrittenOverOrRestarted(t *testing.T) {
func caseAProcessWhoseUnitIsFoundIsNotWrittenOverOrRestarted(t *testing.T) {
dir := t.TempDir()
was := unitDir
unitDir = dir
@@ -899,7 +899,7 @@ func TestAProcessWhoseUnitIsFoundIsNotWrittenOverOrRestarted(t *testing.T) {
}
}
func TestAUserFoundOnTheMachineKeepsItsShellAndGroups(t *testing.T) {
func caseAUserFoundOnTheMachineKeepsItsShellAndGroups(t *testing.T) {
dir := t.TempDir()
m := &machine{containers: map[string]*fakeContainer{}, users: map[string]bool{"hello": true}}
report, _ := applyAdopted(t, adopted(t, untaken("hello-web.user"),
@@ -915,7 +915,7 @@ func TestAUserFoundOnTheMachineKeepsItsShellAndGroups(t *testing.T) {
}
}
func TestMountingTheMachinesOwnPlumbingIsNotFoundData(t *testing.T) {
func caseMountingTheMachinesOwnPlumbingIsNotFoundData(t *testing.T) {
// The runtime's socket, the kernel's filesystems and the clock are on every machine; a
// container mounting them shares nothing a predecessor kept (novox/hq ADR 0103).
dir := t.TempDir()
@@ -950,7 +950,7 @@ func (u *unreachable) run(_ context.Context, name string, args ...string) (strin
return "", nil
}
func TestARuntimeThatCannotAnswerNeverLetsTheMeshCreateOverAFoundContainer(t *testing.T) {
func caseARuntimeThatCannotAnswerNeverLetsTheMeshCreateOverAFoundContainer(t *testing.T) {
dir := t.TempDir()
u := &unreachable{}
d := adopted(t, untaken("hello-web.server"),
@@ -970,7 +970,7 @@ func TestARuntimeThatCannotAnswerNeverLetsTheMeshCreateOverAFoundContainer(t *te
}
}
func TestAHeldContainerStaysHeldWhenTheRuntimeCannotAnswer(t *testing.T) {
func caseAHeldContainerStaysHeldWhenTheRuntimeCannotAnswer(t *testing.T) {
dir, page, m := predecessor(t)
d := adopted(t, untaken("hello-web.page", "hello-web.server"), webResources(page))
_, state := applyAdopted(t, d, store.State{}, m, dir)
@@ -988,7 +988,7 @@ func TestAHeldContainerStaysHeldWhenTheRuntimeCannotAnswer(t *testing.T) {
}
}
func TestAVolumeTheRuntimeCannotBeAskedAboutStopsTheContainer(t *testing.T) {
func caseAVolumeTheRuntimeCannotBeAskedAboutStopsTheContainer(t *testing.T) {
dir := t.TempDir()
run := func(_ context.Context, name string, args ...string) (string, error) {
args = oldShape(name, args)
@@ -1014,7 +1014,7 @@ func TestAVolumeTheRuntimeCannotBeAskedAboutStopsTheContainer(t *testing.T) {
}
}
func TestAUnitSomebodyInstalledIsHeldWhateverStateItIsIn(t *testing.T) {
func caseAUnitSomebodyInstalledIsHeldWhateverStateItIsIn(t *testing.T) {
// A predecessor's unit under /etc, deliberately stopped and disabled: starting it would put
// back a service somebody took down on purpose (novox/hq ADR 0103).
dir := t.TempDir()
@@ -1035,7 +1035,7 @@ func TestAUnitSomebodyInstalledIsHeldWhateverStateItIsIn(t *testing.T) {
}
}
func TestAPackagedUnitTheMachineUsesIsStillHeld(t *testing.T) {
func caseAPackagedUnitTheMachineUsesIsStillHeld(t *testing.T) {
// The predecessor's own service from a package, running: not the mesh's to restart.
dir := t.TempDir()
conf := filepath.Join(dir, "hello.conf")
@@ -1054,7 +1054,7 @@ func TestAPackagedUnitTheMachineUsesIsStillHeld(t *testing.T) {
}
}
func TestASecondOriginalAtTheSamePathIsKeptToo(t *testing.T) {
func caseASecondOriginalAtTheSamePathIsKeptToo(t *testing.T) {
// Held, let go when the module was unassigned, rewritten by the predecessor, found again:
// both originals are kept, and the report names the one it kept (novox/hq ADR 0100).
dir := t.TempDir()
@@ -1083,7 +1083,7 @@ func TestASecondOriginalAtTheSamePathIsKeptToo(t *testing.T) {
}
}
func TestAHoldLetGoAndFoundAgainKeepsBothOriginals(t *testing.T) {
func caseAHoldLetGoAndFoundAgainKeepsBothOriginals(t *testing.T) {
dir, page, m := predecessor(t)
_, state := applyAdopted(t, adopted(t, untaken("hello-web.page", "hello-web.server"), webResources(page)),
store.State{}, m, dir)
@@ -1111,7 +1111,7 @@ func TestAHoldLetGoAndFoundAgainKeepsBothOriginals(t *testing.T) {
}
}
func TestAPathTheMeshOnlySetAccessOnIsStillFound(t *testing.T) {
func caseAPathTheMeshOnlySetAccessOnIsStillFound(t *testing.T) {
// An access record says the mesh set permissions on a path it does not own — which is what it
// does to somebody else's directory. It is not a record of making it (novox/hq ADR 0103).
dir := t.TempDir()
@@ -1144,3 +1144,51 @@ func asideShape(args []string) []string {
}
return args
}
// TestHold runs the cases of this file as subtests of one test (the test budget: one test, a table of cases).
func TestHold(t *testing.T) {
for _, c := range []struct {
name string
run func(*testing.T)
}{
{"AFoundFileOfAnUntakenModuleIsKeptAsItIs", caseAFoundFileOfAnUntakenModuleIsKeptAsItIs},
{"AFoundContainerOfAnUntakenModuleIsNotReplaced", caseAFoundContainerOfAnUntakenModuleIsNotReplaced},
{"WhatIsNotFoundIsCreatedWhenAssigned", caseWhatIsNotFoundIsCreatedWhenAssigned},
{"AFileThisHostWroteIsNotFound", caseAFileThisHostWroteIsNotFound},
{"AContainerAHostMadeIsNotFound", caseAContainerAHostMadeIsNotFound},
{"TheGenesisStoreAdoptedInPlaceIsNotFound", caseTheGenesisStoreAdoptedInPlaceIsNotFound},
{"TakingAModuleReplacesWhatWasHeldAndTheOriginalSurvives", caseTakingAModuleReplacesWhatWasHeldAndTheOriginalSurvives},
{"AHeldFileIsNeverRemovedWhenItsModuleIsUnassigned", caseAHeldFileIsNeverRemovedWhenItsModuleIsUnassigned},
{"AHoldNoLongerDeclaredIsLetGoAndFoundAgainIfItsModuleReturns", caseAHoldNoLongerDeclaredIsLetGoAndFoundAgainIfItsModuleReturns},
{"ACarriedApplyLetsNoHoldGo", caseACarriedApplyLetsNoHoldGo},
{"AHeldFileRewrittenIsReportedAndNotReverted", caseAHeldFileRewrittenIsReportedAndNotReverted},
{"AHeldContainerStoppedOrReplacedIsReportedAndNotRestarted", caseAHeldContainerStoppedOrReplacedIsReportedAndNotRestarted},
{"AHeldFileThatVanishesIsNotCreated", caseAHeldFileThatVanishesIsNotCreated},
{"AConvergedNodeStillReplacesWhatItFinds", caseAConvergedNodeStillReplacesWhatItFinds},
{"AFileWrittenOverIsKeptOnceAndOnlyWhenTheHostHasNoRecordOfIt", caseAFileWrittenOverIsKeptOnceAndOnlyWhenTheHostHasNoRecordOfIt},
{"AFoundDirectoryOfAnUntakenModuleKeepsItsModeOwnerAndContents", caseAFoundDirectoryOfAnUntakenModuleKeepsItsModeOwnerAndContents},
{"ADirectoryMadeInTheSameApplyIsNotFound", caseADirectoryMadeInTheSameApplyIsNotFound},
{"AFoundServiceOfAnUntakenModuleIsNeitherStartedNorEnabledNorRestarted", caseAFoundServiceOfAnUntakenModuleIsNeitherStartedNorEnabledNorRestarted},
{"AHeldServiceIsStillReloadedButNeverRestarted", caseAHeldServiceIsStillReloadedButNeverRestarted},
{"AUnitAPackageOnlyShipsIsNotFound", caseAUnitAPackageOnlyShipsIsNotFound},
{"AServiceWhoseUnitIsNotThereIsAppliedAsUsual", caseAServiceWhoseUnitIsNotThereIsAppliedAsUsual},
{"AContainerThatWouldMountFoundDataIsNotCreated", caseAContainerThatWouldMountFoundDataIsNotCreated},
{"AContainerMountingWhatTheMeshMadeIsCreated", caseAContainerMountingWhatTheMeshMadeIsCreated},
{"ARunOnceStepInAHeldContainerIsHeld", caseARunOnceStepInAHeldContainerIsHeld},
{"AnActionInAHeldContainerIsHeldUntilItsModuleIsTaken", caseAnActionInAHeldContainerIsHeldUntilItsModuleIsTaken},
{"AnArchiveOverSomethingFoundIsNotUnpacked", caseAnArchiveOverSomethingFoundIsNotUnpacked},
{"AProcessWhoseUnitIsFoundIsNotWrittenOverOrRestarted", caseAProcessWhoseUnitIsFoundIsNotWrittenOverOrRestarted},
{"AUserFoundOnTheMachineKeepsItsShellAndGroups", caseAUserFoundOnTheMachineKeepsItsShellAndGroups},
{"MountingTheMachinesOwnPlumbingIsNotFoundData", caseMountingTheMachinesOwnPlumbingIsNotFoundData},
{"ARuntimeThatCannotAnswerNeverLetsTheMeshCreateOverAFoundContainer", caseARuntimeThatCannotAnswerNeverLetsTheMeshCreateOverAFoundContainer},
{"AHeldContainerStaysHeldWhenTheRuntimeCannotAnswer", caseAHeldContainerStaysHeldWhenTheRuntimeCannotAnswer},
{"AVolumeTheRuntimeCannotBeAskedAboutStopsTheContainer", caseAVolumeTheRuntimeCannotBeAskedAboutStopsTheContainer},
{"AUnitSomebodyInstalledIsHeldWhateverStateItIsIn", caseAUnitSomebodyInstalledIsHeldWhateverStateItIsIn},
{"APackagedUnitTheMachineUsesIsStillHeld", caseAPackagedUnitTheMachineUsesIsStillHeld},
{"ASecondOriginalAtTheSamePathIsKeptToo", caseASecondOriginalAtTheSamePathIsKeptToo},
{"AHoldLetGoAndFoundAgainKeepsBothOriginals", caseAHoldLetGoAndFoundAgainKeepsBothOriginals},
{"APathTheMeshOnlySetAccessOnIsStillFound", caseAPathTheMeshOnlySetAccessOnIsStillFound},
} {
t.Run(c.name, c.run)
}
}
+72 -13
View File
@@ -112,18 +112,77 @@ func TestTheRegistryMovesToTheRuntimesModuleWithoutLeavingTheList(t *testing.T)
}
}
// And the preview says the same before it happens.
func TestThePlanForgetsARecordOfAUnitStillHeld(t *testing.T) {
path := filepath.Join(t.TempDir(), "daemon.json")
var commands []string
_, state, err := Apply(context.Background(), archHost(t), runtimeDecl(t, path, true), store.State{},
store.OriginDeclared, recordingServices(&commands), nil, nil)
if err != nil {
t.Fatal(err)
}
for _, step := range Plan(runtimeDecl(t, path, false), state, store.OriginDeclared) {
if step.ID == "networking.registry-trust-reload" && step.Verb != "forget" {
t.Errorf("the plan would %s the network's record of a unit docker.runtime holds: %s", step.Verb, step.Why)
// A unit two records name, under opposite conditions, passes between them (design 42 zfs-owns-zfs §2, answer A-21): the
// going record is forgotten, the preview says so before it happens, and what it found of the unit before the mesh is what
// the staying record gives back when its module leaves, not what the mesh made of it.
func TestARecordOfAUnitStillHeldIsForgottenAndPassesWhatItFound(t *testing.T) {
t.Run("the preview forgets it", func(t *testing.T) {
path := filepath.Join(t.TempDir(), "daemon.json")
var commands []string
_, state, err := Apply(context.Background(), archHost(t), runtimeDecl(t, path, true), store.State{},
store.OriginDeclared, recordingServices(&commands), nil, nil)
if err != nil {
t.Fatal(err)
}
}
for _, step := range Plan(runtimeDecl(t, path, false), state, store.OriginDeclared) {
if step.ID == "networking.registry-trust-reload" && step.Verb != "forget" {
t.Errorf("the plan would %s the network's record of a unit docker.runtime holds: %s", step.Verb, step.Why)
}
}
})
t.Run("what it found passes to the record that stays", func(t *testing.T) {
// A unit found stopped and disabled, so the mesh's first record of it started it.
active, enabled := false, false
var commands []string
run := func(_ context.Context, name string, args ...string) (string, error) {
args = oldShape(name, args)
line := name + " " + strings.Join(args, " ")
commands = append(commands, line)
switch {
case strings.Contains(line, " start "):
active = true
case strings.Contains(line, " stop "):
active = false
case strings.Contains(line, " enable "):
enabled = true
case strings.Contains(line, " disable "):
enabled = false
case strings.Contains(line, "is-enabled"):
if enabled {
return "enabled", nil
}
return "disabled", nil
case strings.Contains(line, "show") && strings.Contains(line, "ActiveState"):
if active {
return "LoadState=loaded\nActiveState=active\nSubState=running", nil
}
return "LoadState=loaded\nActiveState=inactive\nSubState=dead", nil
}
return "", nil
}
unit := func(id, state, boot string) *declaration.Declaration {
return parse(t, `{"declaration":1,"resources":[{"id":"`+id+`","type":"service","unit":"z.service","state":"`+
state+`","boot":"`+boot+`"}]}`)
}
apply := func(d *declaration.Declaration, known store.State) store.State {
t.Helper()
_, known, err := Apply(context.Background(), archHost(t), d, known, store.OriginDeclared, run, nil, nil)
if err != nil {
t.Fatal(err)
}
return known
}
known := apply(unit("z.on", "running", "enabled"), store.State{})
known = apply(unit("z.off", "stopped", "disabled"), known) // the value changed: one record goes, the other stays
if rec, _ := known.Find("z.off"); rec.Found == nil || rec.Found.State != "stopped" || rec.Found.Boot != "disabled" {
t.Fatalf("the staying record did not receive what the going one found: %+v", rec.Found)
}
commands = nil
apply(somethingElse(t), known) // the module leaves
for _, c := range commands {
if strings.Contains(c, " start ") || strings.Contains(c, " enable ") {
t.Errorf("the unit was given back as the mesh made it, not as it was found: %q", c)
}
}
})
}
+29
View File
@@ -0,0 +1,29 @@
package apply
import "context"
// The build the artifact store's holder keeps on a build node (novox/hq design 47, *The artifact store's holder takes its
// own new build even when its seat is the one broken*): the holder's program cannot be fetched from the store it serves, so
// its done build names the node that keeps it and the node-engine fetches it from there, through the build seat's verb.
// How is the caller's (a hook on the apply's context, as the files' link is); what apply does is use the bytes in place of
// the source's fetch, check them against the declared digest as every fetch is, and gate the module when they cannot be had.
// KeptFetch gives the bytes of the artifact a module's done build kept on a build node, by the artifact's digest
// ("sha256:" and 64 hex digits, as a declaration names it). kept is false when the module's build kept no such artifact:
// the artifact is then fetched from its source, as before. An error says the artifact is kept and could not be had.
type KeptFetch func(ctx context.Context, module, digest string) (body []byte, kept bool, err error)
type keptFetchKey struct{}
// WithKept is ctx carrying how a kept artifact is fetched.
func WithKept(ctx context.Context, f KeptFetch) context.Context {
if f == nil {
return ctx
}
return context.WithValue(ctx, keptFetchKey{}, f)
}
func keptFetchFrom(ctx context.Context) KeptFetch {
f, _ := ctx.Value(keptFetchKey{}).(KeptFetch)
return f
}
+7
View File
@@ -41,6 +41,7 @@ func ModuleResults(d *declaration.Declaration, report Report, err error) map[str
outcomes = append(append([]Outcome(nil), outcomes...), failed.Done.Outcomes...)
}
skipped := map[string]string{}
left := map[string][]string{}
for _, o := range outcomes {
module, ok := d.ModuleOfID(o.ID)
if !ok {
@@ -48,6 +49,7 @@ func ModuleResults(d *declaration.Declaration, report Report, err error) map[str
}
if o.Action == "skipped" {
skipped[module] = o.Detail
left[module] = append(left[module], o.ID)
continue
}
attempted[o.ID] = true
@@ -92,6 +94,11 @@ func ModuleResults(d *declaration.Declaration, report Report, err error) map[str
}
}
for module, why := range reasons {
// Each resource left unapplied is named (design 42 zfs-owns-zfs §1).
if ids := left[module]; len(ids) > 0 {
sort.Strings(ids)
why = append(why, "left unapplied: "+strings.Join(ids, ", "))
}
sort.Strings(why)
out[module] = ModuleResult{Failed: true, Detail: strings.Join(why, "; ")}
}
+29 -4
View File
@@ -54,6 +54,7 @@ import (
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
"git.novox.be/novox/mesh-sdk/go/declaration"
@@ -303,8 +304,28 @@ func splitQuoted(s string) []string {
return out
}
// engineModule is the module whose resources may place in the engine's own trees.
const engineModule = "mesh-host"
// engineModuleOf answers the module whose resources may place in the engine's own trees: the module whose entry of the
// catalog half claims the seat node-engine (design 47 G §527), asked of the node's composer; none until one is given.
var engineModuleOf atomic.Pointer[func() string]
// UseEngineModule gives the apply guard the function that answers the node-engine's own module; nil takes it away. The
// guard fixes no name (§364, §527).
func UseEngineModule(f func() string) {
if f == nil {
engineModuleOf.Store(nil)
return
}
engineModuleOf.Store(&f)
}
// isEngineModule is whether module is the node-engine's own.
func isEngineModule(module string) bool {
f := engineModuleOf.Load()
if f == nil || module == "" {
return false
}
return (*f)() == module
}
// passwdFile is the user database homes are read from. A variable so a test names its own.
var passwdFile = "/etc/passwd"
@@ -365,11 +386,15 @@ func below(path, dir string) bool {
// atOrBelow says whether path is dir or below it.
func atOrBelow(path, dir string) bool { return path == dir || below(path, dir) }
// evalSymlinks follows the links of an existing path; a variable so a test judges paths whose machine it does not touch (a mount
// of the machine it runs on may not answer).
var evalSymlinks = filepath.EvalSymlinks
// resolved is a path with every link in it followed, as far as the path exists, and the rest as declared.
func resolved(path string) string {
rest := ""
for p := path; ; p = filepath.Dir(p) {
if real, err := filepath.EvalSymlinks(p); err == nil {
if real, err := evalSymlinks(p); err == nil {
return filepath.Clean(filepath.Join(real, rest))
}
if filepath.Dir(p) == p {
@@ -531,7 +556,7 @@ func refuseOne(path string, kind placing, module, owner string, declared map[str
return &PlacementRefusedError{Path: path, Why: "nothing is placed in or mounted from " + tree}
}
}
if module != engineModule {
if !isEngineModule(module) {
for _, tree := range engineTrees {
if atOrBelow(path, tree) {
return &PlacementRefusedError{Path: path, Why: tree + " is the node-engine's own, placed in by " +
+25
View File
@@ -178,6 +178,11 @@ func TestAFoundDirectoryIsUsedAsFound(t *testing.T) {
// The module's own directories keep being placed: below /var/lib, below /etc as root's, a container's mount of
// the machine's docker socket.
func TestAModulesOwnPlacesAreStillPlaced(t *testing.T) {
// The paths below are judged as written: no link of them is followed on the machine the test runs on, whose mounts of them
// (a share that does not answer) it does not depend on.
was := evalSymlinks
evalSymlinks = func(p string) (string, error) { return p, nil }
t.Cleanup(func() { evalSymlinks = was })
for _, path := range []string{"/var/lib/notes/data", "/etc/systemd/logind.conf.d", "/var/run/fail2ban",
"/srv/notes", "/mnt/plex/config", "/storage/media", "/var/lib/mesh/registry"} {
if err := refusePath(path, placingDirectory, "notes", ""); err != nil {
@@ -193,4 +198,24 @@ func TestAModulesOwnPlacesAreStillPlaced(t *testing.T) {
if err := refusePath("/var/lib/notes/data", placingDirectory, "notes", "1001:2000"); err != nil {
t.Errorf("an owner below /var/lib: %v", err)
}
// Design 47 G §364, §527: the node-engine's own trees are placed in by the module the composer says claims its seat,
// whatever that module is called, and by no other; with none said, by nobody.
tree := engineTrees[0]
t.Cleanup(func() { UseEngineModule(nil) })
for _, c := range []struct {
engine func() string
module string
placed bool
}{
{func() string { return "the-engine" }, "the-engine", true},
{func() string { return "the-engine" }, "notes", false},
{func() string { return "the-engine" }, "", false},
{func() string { return "" }, "", false},
{nil, "the-engine", false},
} {
UseEngineModule(c.engine)
if err := refusePath(tree, placingDirectory, c.module, ""); (err == nil) != c.placed {
t.Errorf("%s for module %q: %v, want placed %v", tree, c.module, err, c.placed)
}
}
}
+110 -4
View File
@@ -4,10 +4,13 @@ import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"maps"
"os"
"path/filepath"
"runtime"
"slices"
"strings"
"time"
@@ -40,13 +43,16 @@ var daemonRoot = "/var/lib/mesh/daemons"
var unitDir = "/etc/systemd/system"
func applyProcess(ctx context.Context, r *declaration.Process, run Runner,
changed map[string]bool, previous store.Applied, away *storeAway) (Outcome, error) {
changed map[string]bool, previous store.Applied, held []byte, away *storeAway) (Outcome, error) {
out := begin(r)
out.Action = "unchanged"
body, err := fetchMinding(ctx, r.Source, away)
if err != nil {
return out, err
// What the apply fetched before anything was replaced (fetchFirst), else fetched now.
body, err := held, error(nil)
if body == nil {
if body, err = fetchMinding(ctx, r.Source, away); err != nil {
return out, err
}
}
sum := sha256.Sum256(body)
got := "sha256:" + hex.EncodeToString(sum[:])
@@ -257,6 +263,9 @@ func applyProcess(ctx context.Context, r *declaration.Process, run Runner,
}
if because != "" {
out.Detail += ", restarted because " + because + " changed"
} else if what := unitDifference(previous.Wrote, want); what != "" {
// Said, so a restart nobody asked for names its cause (issue 1966): the file and the input of it, never a value.
out.Detail += ", restarted because " + what + " changed"
}
if placing.Detail != "" {
out.Detail += "; " + placing.Detail
@@ -265,6 +274,25 @@ func applyProcess(ctx context.Context, r *declaration.Process, run Runner,
return out, nil
}
// NodeEnv is the variable every process the node-engine runs reads its node's name from (design 47 G A-342), the one
// mesh-sdk reads (MESH_NODE).
const NodeEnv = "MESH_NODE"
// withNode is the process as it runs on node: its own env with NodeEnv beside it (design 47 G A-342). The validator
// refuses a resource that sets it, so the mesh's value is the only one; with no node named the process is as declared.
func withNode(r *declaration.Process, node string) *declaration.Process {
if node == "" {
return r
}
p := *r
p.Env = make(map[string]string, len(r.Env)+1)
for k, v := range r.Env {
p.Env[k] = v
}
p.Env[NodeEnv] = node
return &p
}
// unitFor is the unit the mesh writes for a daemon.
//
// **Generated whole and never edited in place**, the same rule as every other managed file: an
@@ -550,3 +578,81 @@ func digestWritten(wrote string) string {
// DaemonRoot is where unpacked daemons live — and what a witness keeps beside them.
func DaemonRoot() string { return daemonRoot }
// unitDifference names what differs between what a process's record says was written and what is wanted now, both
// "sha256:<hex> <unit>": its bundle when the digest differs, and in the unit file each directive and each environment
// word that differs, with inside a word the keys of a JSON object (and, one level down, of its objects) that differ. Names only, never a value, so a secret in a word is never said. Empty
// when nothing differs or there was no record.
func unitDifference(was, want string) string {
if was == "" {
return ""
}
wasDigest, wasUnit, _ := strings.Cut(was, " ")
wantDigest, wantUnit, _ := strings.Cut(want, " ")
var names []string
if wasDigest != wantDigest {
names = append(names, "its bundle")
}
before, after := unitDirectives(wasUnit), unitDirectives(wantUnit)
keys := map[string]bool{}
for k := range before {
keys[k] = true
}
for k := range after {
keys[k] = true
}
for _, k := range slices.Sorted(maps.Keys(keys)) {
if before[k] != after[k] {
names = append(names, "its unit's "+k+valueDifference(before[k], after[k], 2))
}
}
return strings.Join(names, ", ")
}
// unitDirectives is a unit's directives by name: "Environment <word>" for an environment word, the directive's own
// name for the rest, each with its value as written (an environment word's value unescaped, as unitValue escapes it).
func unitDirectives(unit string) map[string]string {
out := map[string]string{}
for _, line := range strings.Split(unit, "\n") {
line = strings.TrimSpace(line)
if line == "" || strings.HasPrefix(line, "#") || strings.HasPrefix(line, "[") {
continue
}
key, value, _ := strings.Cut(line, "=")
if key == "Environment" && strings.HasPrefix(value, `"`) && strings.HasSuffix(value, `"`) {
word, rest, _ := strings.Cut(value[1:len(value)-1], "=")
rest = strings.NewReplacer(`\\`, `\`, `\"`, `"`, "%%", "%").Replace(rest)
out["Environment "+word] = rest
continue
}
out[key] += value
}
return out
}
// valueDifference is what differs inside one directive's value, as " (a, b)": the keys of a JSON object whose value
// differs (going down depth levels), else the names of the name=value entries of a comma-separated list that were
// added, removed or changed. Empty when the value says nothing more than that it differs.
func valueDifference(was, now string, depth int) string {
var a, b map[string]json.RawMessage
if depth > 0 && json.Unmarshal([]byte(was), &a) == nil && json.Unmarshal([]byte(now), &b) == nil && (a != nil || b != nil) {
all := map[string]bool{}
for k := range a {
all[k] = true
}
for k := range b {
all[k] = true
}
var names []string
for _, k := range slices.Sorted(maps.Keys(all)) {
if string(a[k]) == string(b[k]) {
continue
}
inner := valueDifference(string(a[k]), string(b[k]), depth-1)
names = append(names, k+inner)
}
return " (" + strings.Join(names, ", ") + ")"
}
// Anything else is a value: said as differing, never split, since a name before an "=" in it can be a secret.
return ""
}
+54
View File
@@ -319,11 +319,17 @@ func TestAProcessAppliedAgainIsUnchangedAndNotRestarted(t *testing.T) {
}
d := declare(t, `{"id":"node-tools.runtime","type":"process","name":"node-tools","source":"`+serving(t, body)+
`","digest":"`+digest+`","run":["node","main.js"],"env":{"MESH_TOOL_MODULES":"a=/x/index.js"}}`)
d.For = "anchor"
first, state, err := Apply(context.Background(), archHost(t), d, store.State{}, store.OriginDeclared, run, nil, nil)
if err != nil {
t.Fatal(err)
}
// Design 47 G A-342: the process runs with its node's name as MESH_NODE beside its own env.
if b, err := os.ReadFile(filepath.Join(units, "node-tools.service")); err != nil || !strings.Contains(string(b), `Environment="MESH_NODE=anchor"`) ||
!strings.Contains(string(b), `Environment="MESH_TOOL_MODULES=`) {
t.Errorf("the unit does not carry the node's name beside its env (%v):\n%s", err, b)
}
if o := outcomeOf(first, "node-tools.runtime"); o.Action != "created" {
t.Fatalf("first apply: %+v, want created", o)
}
@@ -357,6 +363,54 @@ func TestAProcessAppliedAgainIsUnchangedAndNotRestarted(t *testing.T) {
if o := outcomeOf(third, "node-tools.runtime"); o.Action != "unchanged" {
t.Errorf("third apply: %+v, want unchanged", o)
}
// An input of its unit that differs restarts it, and the line says which (issue 1966).
changed := declare(t, `{"id":"node-tools.runtime","type":"process","name":"node-tools","source":"`+serving(t, body)+
`","digest":"`+digest+`","run":["node","main.js"],"env":{"MESH_TOOL_MODULES":"a=/x/index.js,b=/y/index.js"}}`)
changed.For = "anchor"
fourth, _, err := Apply(context.Background(), archHost(t), changed, state, store.OriginDeclared, run, nil, nil)
if err != nil {
t.Fatal(err)
}
if o := outcomeOf(fourth, "node-tools.runtime"); o.Action != "updated" || !strings.Contains(o.Detail, "its unit's Environment MESH_TOOL_MODULES") ||
strings.Contains(o.Detail, "/y/") {
t.Errorf("fourth apply: %+v, want updated and the word that changed named, no value", o)
}
for _, row := range []struct {
name, was, now, says string
}{
{"a word of a module inside a JSON word", `{"files":{"OWNED":"/a"},"mail":{"X":"1"}}`, `{"files":{"OWNED":"/b"},"mail":{"X":"1"}}`,
"Environment TOOL_ENV (files (OWNED))"},
{"a secret-shaped value", "c2VjcmV0MQ==", "c2VjcmV0Mg==", "Environment TOOL_VALUE"},
{"a URL with a password", "postgres://u:hunter1@h/db?sslmode=x", "postgres://u:hunter2@h/db?sslmode=x", "Environment TOOL_VALUE"},
{"nothing changed", "a=/x/1", "a=/x/1", ""},
} {
t.Run(row.name, func(t *testing.T) {
word := "TOOL_VALUE"
if strings.HasPrefix(row.was, "{") {
word = "TOOL_ENV"
}
r := aProcess()
mk := func(v string) string {
r.Env = map[string]string{word: v}
return "sha256:" + strings.Repeat("a", 64) + " " + unitFor(r)
}
got := unitDifference(mk(row.was), mk(row.now))
if row.says == "" {
if got != "" {
t.Fatalf("said %q for nothing changed", got)
}
return
}
for _, leak := range []string{"c2VjcmV0", "hunter", "sslmode", "postgres", "/b"} {
if strings.Contains(got, leak) {
t.Fatalf("said %q, which holds part of a value (%s)", got, leak)
}
}
if !strings.Contains(got, row.says) {
t.Fatalf("said %q, want it to name %q", got, row.says)
}
})
}
}
// novox/hq ADR 0193: a bundle compiled to a binary runs itself — `./name` is that file in the
+3 -3
View File
@@ -118,10 +118,10 @@ func TestAFileIsJudgedByTheSameGuardAsADirectory(t *testing.T) {
if _, statErr := os.Stat(filepath.Join(home, "data", "notes.conf")); statErr == nil {
t.Error("a file was written below a directory the engine refused")
}
// A refused file has no outcome: it is a failure. Declared first, it is the failure the apply reports,
// and the directory the other.
// A refused file has no outcome: it is a failure. Declared first, it is the failure the apply reports, and the
// directory after it is left unapplied, as the rest of its module is (design 42 zfs-owns-zfs §1).
var applyErr *Error
if !errors.As(err, &applyErr) || applyErr.Resource != "notes.conf" || applyErr.Others != 1 {
if !errors.As(err, &applyErr) || applyErr.Resource != "notes.conf" || applyErr.Others != 0 {
t.Errorf("the file below a refused directory is not among the failures: %v", err)
}
if o := outcomeOf(report, "notes.conf"); o.ID != "" {
+101
View File
@@ -3,6 +3,8 @@ package apply
import (
"context"
"errors"
"net/http"
"net/http/httptest"
"strings"
"testing"
@@ -104,6 +106,105 @@ func TestARunningContainerIsKeptWhenItsReplacementImageCannotBeObtained(t *testi
t.Errorf("%s: the failure does not name %s: %v", c.name, second, err)
}
}
// Issue 1966 (2026-11-08 23:25 UTC): a module whose container serves the artifact store had it replaced, and then
// could not fetch its program from the store it had cut off. Every archive and process is fetched before anything of
// the module is replaced or removed: with the store away, its container and its orphan stay as they were, and the
// failure names the program's source.
t.Run("an artifact that cannot be fetched replaces nothing of its module", func(t *testing.T) {
gone := httptest.NewServer(http.NotFoundHandler())
source := gone.URL + "/program.tar.gz"
gone.Close() // the store is cut off: refused
was := storeAwayAtMost
storeAwayAtMost = 0 // no patience: the refusal fails at once, without waiting
t.Cleanup(func() { storeAwayAtMost = was })
d := parseTrusted(t, `{"declaration":1,"resources":[
{"id":"distribution.store","type":"container","name":"store","image":"`+pinned+`","env":{"X":"new"}},
{"id":"distribution.program","type":"process","name":"program","source":"`+source+`",
"digest":"sha256:`+strings.Repeat("0", 64)+`","run":["program"]}
]}`)
known := store.State{}
known.Record(store.Applied{ID: "distribution.old", Type: string(declaration.TypeContainer), Target: "old",
Module: "distribution"})
var steps []string
removed := map[string]bool{}
run := func(ctx context.Context, name string, args ...string) (string, error) {
if name == "systemctl" {
steps = append(steps, "systemctl "+strings.Join(args, " "))
return "", nil
}
args = oldShape(name, args)
switch args[0] {
case "info":
return "27.0\n", nil
case "container":
if removed[args[len(args)-1]] {
return "", errors.New("no such container")
}
return "true\tsome-older-spec", nil
case "rm", "rename", "stop", "run":
steps = append(steps, args[0]+" "+strings.Join(args[1:], " "))
if args[0] == "rm" {
removed[args[len(args)-1]] = true
}
}
return "", nil
}
report, after, err := Apply(context.Background(), archHost(t), d, known, store.OriginCarried, run, nil, nil)
if len(steps) > 0 {
t.Errorf("the module was touched though its program could not be fetched: %q", steps)
}
if err == nil || !strings.Contains(err.Error(), source) {
t.Errorf("the failure does not name the program's source %s: %v", source, err)
}
if _, still := after.Find("distribution.old"); !still {
t.Error("the module's orphan was removed though its program could not be fetched")
}
if o := outcomeOf(report, "distribution.store"); o.Action != "skipped" {
t.Errorf("the container's outcome is %+v, want skipped", o)
}
})
// Design 47 (W13.8): the holder's own program is fetched from the build node that kept it, not from the store; when it
// cannot be had, or is not the declared digest, nothing of the module is replaced and the failure says so.
for _, row := range []struct {
name string
fetch KeptFetch
wantErr string
}{
{"the node that keeps it is away", func(context.Context, string, string) ([]byte, bool, error) {
return nil, true, errors.New("novox keeps it and does not answer")
}, "novox keeps it and does not answer"},
{"what the node kept is not the declared digest", func(context.Context, string, string) ([]byte, bool, error) {
return []byte("other"), true, nil
}, "what arrived is"},
} {
t.Run("a kept artifact: "+row.name, func(t *testing.T) {
gone := httptest.NewServer(http.NotFoundHandler())
source := gone.URL + "/program.tar.gz"
gone.Close()
was := storeAwayAtMost
storeAwayAtMost = 0
t.Cleanup(func() { storeAwayAtMost = was })
d := parseTrusted(t, `{"declaration":1,"resources":[
{"id":"distribution.program","type":"process","name":"program","source":"`+source+`",
"digest":"sha256:`+strings.Repeat("0", 64)+`","run":["program"]}
]}`)
var asked []string
fetch := func(ctx context.Context, module, digest string) ([]byte, bool, error) {
asked = append(asked, module+" "+digest)
return row.fetch(ctx, module, digest)
}
run := func(ctx context.Context, name string, args ...string) (string, error) { return "", nil }
_, _, err := Apply(WithKept(context.Background(), fetch), archHost(t), d, store.State{}, store.OriginCarried, run, nil, nil)
if err == nil || !strings.Contains(err.Error(), row.wantErr) {
t.Errorf("the failure is %v, want it to say %q", err, row.wantErr)
}
if len(asked) != 1 || asked[0] != "distribution sha256:"+strings.Repeat("0", 64) {
t.Errorf("the kept fetch was asked %q", asked)
}
})
}
}
func TestAReplacementImageIsPresentBeforeTheRunningContainerIsRemoved(t *testing.T) {
+69
View File
@@ -0,0 +1,69 @@
package apply
import (
"context"
"fmt"
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/declaration"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
// StopModule stops the units and containers the node's record holds for a module and removes nothing: no unit, file,
// container, data or record, so what the module kept survives and a later Apply starts the build it is given (design 47
// G A-98, §297). Units come first, a timer before the service it starts, then containers; a unit the mesh never held
// (Stateless) is left as it is. It answers what it stopped, and the first failure.
func StopModule(ctx context.Context, sys system.System, known store.State, module string, run Runner) ([]string, error) {
var timers, services, containers []store.Applied
for _, r := range known.Resources {
if store.OwnerOf(r, declaration.ModuleGuess) != module {
continue
}
switch r.Type {
case string(declaration.TypeService):
if r.Stateless {
continue
}
if strings.HasSuffix(r.Target, ".timer") {
timers = append(timers, r)
} else {
services = append(services, r)
}
case string(declaration.TypeContainer):
containers = append(containers, r)
}
}
var stopped []string
for _, r := range append(timers, services...) {
manager := managerFor(r.Scope, r.User, run)
if state, err := sys.ServiceState(ctx, manager, r.Target); err != nil {
if strings.Contains(err.Error(), "does not exist on this machine") {
continue
}
return stopped, fmt.Errorf("stopping %s of %s: %w", r.Target, module, err)
} else if state == "stopped" {
continue
}
if err := sys.SetServiceState(ctx, manager, r.Target, "stopped"); err != nil {
return stopped, fmt.Errorf("stopping %s of %s: %w", r.Target, module, err)
}
stopped = append(stopped, r.Target)
}
if len(containers) == 0 {
return stopped, nil
}
cri, err := containerRuntime(ctx, run)
if err != nil {
return stopped, err
}
sort.Slice(containers, func(i, j int) bool { return containers[i].Target < containers[j].Target })
for _, r := range containers {
if _, err := run(ctx, cri, "stop", r.Target); err != nil {
return stopped, fmt.Errorf("stopping the container %s of %s: %w", r.Target, module, err)
}
stopped = append(stopped, r.Target)
}
return stopped, nil
}
+45
View File
@@ -0,0 +1,45 @@
package apply
import (
"context"
"reflect"
"testing"
"github.com/novox/mesh-host/internal/store"
)
// Stop stops a module's timer, then its service, then its container, and leaves every record, and another module's
// units, as they are (design 47 G A-98).
func TestStopModuleStopsAModulesUnitsAndContainersAndRemovesNothing(t *testing.T) {
m := unitsMachine(map[string]*fakeUnit{
"app.service": {active: "active", enabled: "enabled"},
"app.timer": {active: "active", enabled: "enabled"},
"other.service": {active: "active", enabled: "enabled"},
})
state := store.State{}
for _, r := range []store.Applied{
{ID: "app.run", Type: "service", Target: "app.service"},
{ID: "app.tick", Type: "service", Target: "app.timer"},
{ID: "app.c", Type: "container", Target: "app-box"},
{ID: "other.run", Type: "service", Target: "other.service"},
{ID: "other.c", Type: "container", Target: "other-box"},
} {
state.Record(r)
}
got, err := StopModule(context.Background(), archHost(t), state, "app", m.run)
if err != nil {
t.Fatal(err)
}
if want := []string{"app.timer", "app.service", "app-box"}; !reflect.DeepEqual(got, want) {
t.Fatalf("stopped %v, want %v, the timer first", got, want)
}
if m.units["app.service"].active != "inactive" || m.units["app.timer"].active != "inactive" || m.units["other.service"].active != "active" {
t.Fatalf("units are %+v", m.units)
}
if !m.did("docker stop app-box") || m.did("docker stop other-box") || m.did("docker rm") {
t.Fatalf("the machine was asked %v", m.asked)
}
if len(state.Resources) != 5 {
t.Fatalf("a record was removed: %d left", len(state.Resources))
}
}
+2 -2
View File
@@ -9,8 +9,8 @@ import (
vault "git.novox.be/novox/mesh-contracts/identity-and-access/go/seats/meshvault"
seat "git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
catalog "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
wants "git.novox.be/novox/mesh-delivery/contracts/go/seats/meshwants"
catalog "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
wants "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants"
"git.novox.be/novox/mesh-sdk/go/keys"
)
+3 -3
View File
@@ -4,11 +4,11 @@ package bucketstest
import (
"encoding/json"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliveryclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshdeliveryclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshwants"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshwants"
)
// Version is the value version the holders write.
+29 -20
View File
@@ -8,7 +8,7 @@ import (
"sort"
"strings"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
engcomp "github.com/novox/mesh-host/internal/compositions"
"git.novox.be/novox/mesh-networking/modules/mesh-wireguard/go/overlay"
@@ -28,7 +28,7 @@ import (
// writer folds them (issue 1126). Each module's Data says whether its manifest declares data. The compositions are unsigned
// (ADR 0593 §6) and not yet by digest (Composed.ByDigest).
func Compositions(in Input, node string, fps map[string]string, mode Mode) (out map[string]engcomp.Composed, skipped []string, err error) {
out, skipped, _, err = compositions(in, node, fps, mode, false)
out, skipped, _, _, err = compositions(in, node, fps, mode, false)
return out, skipped, err
}
@@ -45,7 +45,7 @@ const BusUsersStandIn = "the bus's user list"
// ErrStoodIn, naming each, so a stand-in is never applied as if it were the value; but for the bus's user list
// (BusUsersStandIn), which the bus node holds out of every apply of a composition it made itself.
func CompositionsApplied(in Input, node string, fps map[string]string) (map[string]engcomp.Composed, []string, error) {
out, skipped, _, err := compositions(in, node, fps, Reading, true)
out, skipped, _, _, err := compositions(in, node, fps, Reading, true)
return out, skipped, err
}
@@ -56,13 +56,22 @@ func CompositionsApplied(in Input, node string, fps map[string]string) (map[stri
// module the candidate adds, or one missing for a module the node runs. A module whose every value is held composes
// as CompositionsApplied composes it: the modes differ only in an own secret never issued.
func CompositionsShadow(in Input, node string, fps map[string]string) (out map[string]engcomp.Composed, skipped, stood []string, err error) {
return compositions(in, node, fps, Foreseeing, false)
out, skipped, stood, _, err = compositions(in, node, fps, Foreseeing, false)
return out, skipped, stood, err
}
// CompositionsIn is the node's compositions in mode with what they stand in for answered (stood, sorted, the bus's user
// list left out) rather than refused: CompositionsShadow in Foreseeing, and in Reading what CompositionsApplied refuses
// before it is refused, so a caller that applies refuses each stand-in to the module it is for alone (issue 1126).
func CompositionsIn(in Input, node string, fps map[string]string, mode Mode) (out map[string]engcomp.Composed, skipped, stood []string, err error) {
out, skipped, stood, _, err = compositions(in, node, fps, mode, false)
return out, skipped, stood, err
}
// CompositionsHolding is CompositionsIn with the modules the node leaves out as they run, each with why: one whose claim cannot be
// held or whose credential was refused (issue 1966). Their compositions are not in out, so a caller that applies keeps what it last
// applied of each and says it held, as for any module refused alone.
func CompositionsHolding(in Input, node string, fps map[string]string, mode Mode) (out map[string]engcomp.Composed, skipped, stood []string, held map[string]string, err error) {
return compositions(in, node, fps, mode, false)
}
@@ -75,7 +84,7 @@ func Folds(inOwn bool, fp string) bool {
return inOwn && !strings.HasPrefix(fp, meshcatalogclient.SourcePrefix2)
}
func compositions(in Input, node string, fps map[string]string, mode Mode, applied bool) (out map[string]engcomp.Composed, skipped, stood []string, err error) {
func compositions(in Input, node string, fps map[string]string, mode Mode, applied bool) (out map[string]engcomp.Composed, skipped, stood []string, heldBy map[string]string, err error) {
reading := in
reading.Nodes = append([]Node(nil), in.Nodes...)
found := false
@@ -99,7 +108,7 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
return module != "" && Folds(inOwn[module], fps[module])
}
if !found {
return nil, nil, nil, fmt.Errorf("compose: the mesh has no node %q", node)
return nil, nil, nil, nil, fmt.Errorf("compose: the mesh has no node %q", node)
}
// A folded module built by a build of the old form (Node.RunsOwn) is left to the node: composed from what runs of it
// (Carried), so what it provides and requires still shapes the rest and what the others read of it (its published
@@ -167,10 +176,10 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
}
d, problems, err := ComposeIn(reading, node, mode)
if err != nil {
return nil, nil, nil, err
return nil, nil, nil, nil, err
}
if len(problems) > 0 {
return nil, nil, nil, fmt.Errorf("compose: %s's declaration is refused by the node-engine's validator: %v", node, problems)
return nil, nil, nil, nil, fmt.Errorf("compose: %s's declaration is refused by the node-engine's validator: %v", node, problems)
}
for _, x := range d.StoodIn {
// The bus's user list is the bus node's own to follow, written alone and never from a composition the node
@@ -186,11 +195,11 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
sort.Strings(stood)
stood = slices.Compact(stood) // an own secret is both stood in for and foreseen
if applied && len(stood) > 0 {
return nil, nil, nil, fmt.Errorf("%w on %s: %s", ErrStoodIn, node, strings.Join(stood, ", "))
return nil, nil, nil, nil, fmt.Errorf("%w on %s: %s", ErrStoodIn, node, strings.Join(stood, ", "))
}
body, err := d.Body()
if err != nil {
return nil, nil, nil, err
return nil, nil, nil, nil, err
}
var envelope struct {
Resources []json.RawMessage `json:"resources"`
@@ -199,7 +208,7 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
Provisioner string `json:"provisioner_account"`
}
if err := json.Unmarshal(body, &envelope); err != nil {
return nil, nil, nil, err
return nil, nil, nil, nil, err
}
var kept []json.RawMessage
keptPaths := map[string]bool{}
@@ -213,7 +222,7 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
Path string `json:"path"`
}
if err := json.Unmarshal(raw, &head); err != nil {
return nil, nil, nil, err
return nil, nil, nil, nil, err
}
if isCarried(head.Module, head.ID) {
continue // what the stub composed of it: the node's own carries what runs of it instead
@@ -223,11 +232,11 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
// compositionsOf): the node's own composition is then the controller's, byte for byte.
var fields map[string]json.RawMessage
if err := json.Unmarshal(raw, &fields); err != nil {
return nil, nil, nil, err
return nil, nil, nil, nil, err
}
delete(fields, "module")
if raw, err = json.Marshal(fields); err != nil {
return nil, nil, nil, err
return nil, nil, nil, nil, err
}
if head.Type == string(declaration.TypeFile) {
foldedPaths[head.Path] = true
@@ -248,7 +257,7 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
if len(carried) > 0 {
var err error
if kept, foldedPaths, err = carry(kept, foldedPaths, runsOwn, isCarried); err != nil {
return nil, nil, nil, err
return nil, nil, nil, nil, err
}
for _, p := range runsOwn.Filled {
if foldedPaths[p] && !slices.Contains(envelope.Filled, p) {
@@ -270,12 +279,12 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
out, err = engcomp.Split(node, kept, fps, filled,
engcomp.NodeOwn{Adoption: envelope.Adoption, Provisioner: envelope.Provisioner})
if err != nil {
return nil, nil, nil, err
return nil, nil, nil, nil, err
}
if len(ownFilled) > 0 {
own, err := engcomp.NodeKey(node)
if err != nil {
return nil, nil, nil, err
return nil, nil, nil, nil, err
}
c := out[own]
c.Filled = append(c.Filled, ownFilled...)
@@ -295,7 +304,7 @@ func compositions(in Input, node string, fps map[string]string, mode Mode, appli
skipped = append(skipped, m)
}
sort.Strings(skipped)
return out, skipped, stood, nil
return out, skipped, stood, d.Held, nil
}
// RunsAnyOf says runs (a composition as it runs) holds a resource of module, by its module or the module its id names.
@@ -379,8 +388,8 @@ func ToolsRunning(running []engcomp.Composed) map[string][]catalogue.Bundle {
}
var own map[string]string
for w, v := range words[module] {
if w == catalogue.FilesOwnedPaths || w == catalogue.ReadsSecretPlaces {
continue // the mesh's to say, composed again
if w == catalogue.FilesOwnedPaths || w == catalogue.ReadsSecretPlaces || w == engcomp.FingerprintVar {
continue // the mesh's to say, composed again (the fingerprint is the want's, §519)
}
if own == nil {
own = map[string]string{}
+7 -2
View File
@@ -2,7 +2,7 @@ package compose
import (
"encoding/json"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
engcomp "github.com/novox/mesh-host/internal/compositions"
"sort"
"strings"
@@ -180,7 +180,12 @@ func TestCompositionsAreTheDeclarationSplitByModule(t *testing.T) {
}
}
}
if env[catalogue.RuntimeToolModules] != served || env[catalogue.RuntimeToolEnv] != `{"lamp":{"LAMP_CONF":"/var/lib/lamp/conf"}}` {
// §519: the words each bundle's module is given hold the fingerprint of the want it applies, where there is one.
words := `{"lamp":{"LAMP_CONF":"/var/lib/lamp/conf"}}`
if fp := row.fps["lamp"]; fp != "" {
words = `{"lamp":{"LAMP_CONF":"/var/lib/lamp/conf","MESH_FINGERPRINT":"` + fp + `"}}`
}
if env[catalogue.RuntimeToolModules] != served || env[catalogue.RuntimeToolEnv] != words {
t.Errorf("%s: the runtime serves %v with %v, want what runs of lamp, %s", row.name,
env[catalogue.RuntimeToolModules], env[catalogue.RuntimeToolEnv], served)
}
+2 -2
View File
@@ -23,12 +23,12 @@ type CredentialKey struct {
// in neither list: the vault mints nothing for it (issue 1126).
func CredentialsNeeded(in Input, node string) (asConsumer, asProvider []CredentialKey, err error) {
c := newComposer(in)
resolved, err := c.resolution(node)
resolved, held, err := c.resolutionHolding(node)
if err != nil {
return nil, nil, err
}
for _, n := range resolved.Needs {
if n.ByRecord || n.From == "" || credentialless(n) {
if _, isHeld := held[n.For]; isHeld || n.ByRecord || n.From == "" || credentialless(n) {
continue
}
asConsumer = append(asConsumer, CredentialKey{Provision: n.Name, Consumer: node, ConsumerModule: n.For,
+1 -4
View File
@@ -201,10 +201,7 @@ func TestAGrantIsCreatedForEachCredentialMadeForAConsumer(t *testing.T) {
}
// A consumer whose set does not resolve is skipped, not fatal.
nodeOf(t, &in, "laptop").Assigned = append(nodeOf(t, &in, "laptop").Assigned, "files")
in.Catalogue["files"] = catalogue.Manifest{Module: "files", Version: "1", Requires: []string{"s3-bucket"},
Claims: []catalogue.Claim{{Name: "x", Scope: catalogue.ScopeNode}}}
in.Catalogue["twin"] = catalogue.Manifest{Module: "twin", Version: "1", Claims: []catalogue.Claim{{Name: "x", Scope: catalogue.ScopeNode}}}
nodeOf(t, &in, "laptop").Assigned = append(nodeOf(t, &in, "laptop").Assigned, "twin")
in.Catalogue["files"] = catalogue.Manifest{Module: "files", Version: "1", Requires: []string{"s3-bucket", "nothing-gives-this"}}
if got, err := Render(in, "anchor", Foreseeing); err != nil || len(got.Rendering.Grants) != 0 {
t.Errorf("a consumer that does not resolve: %v %+v", err, got.Rendering.Grants)
}
+14 -1
View File
@@ -337,7 +337,8 @@ func composeEach(in compose.Input, node string, fps map[string]string, applied b
hold(m, failed[m], "")
}
for {
out, skipped, stood, err = compose.CompositionsIn(in, node, fps, mode)
var leftOut map[string]string
out, skipped, stood, leftOut, err = compose.CompositionsHolding(in, node, fps, mode)
if err != nil {
module, leave := culprit(in, node, fps, mode, failed, err)
if module == "" {
@@ -346,6 +347,18 @@ func composeEach(in compose.Input, node string, fps map[string]string, applied b
hold(module, err.Error(), leave)
continue
}
// A module the node leaves out because its claim cannot be held or its credential was refused (issue 1966) is held with
// those words: its handed composition stays and the check names it refused; the rest composes.
heldNow := false
for _, m := range sortedKeys(leftOut) {
if _, done := failed[m]; !done {
hold(m, leftOut[m], "")
heldNow = true
}
}
if heldNow {
continue
}
again := false
if applied && len(stood) > 0 {
placed := map[string]bool{}
+32 -2
View File
@@ -1,15 +1,16 @@
package frombus
import (
"bytes"
"context"
"encoding/json"
"fmt"
"git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
nodeengine "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
engcomp "github.com/novox/mesh-host/internal/compositions"
"os"
"reflect"
@@ -218,6 +219,35 @@ func answerNamesEveryRefusalAndLeavesAnOldBuildToTheNode(t *testing.T) {
t.Errorf("spiegel in InOwn wanted at %s: %v, held %v, read as %s; want its build's image", sp.Fp2, err,
read.ComposedHow, raw)
}
// Issue 1921: the input a check is answered with names the manifest spiegel's build was made of (its content
// digest, the same for those bytes compacted inside a request), so the check's composer composes it at that
// build only where the change's manifest is that one.
var compact bytes.Buffer
_ = json.Compact(&compact, bb.manifests[sp.Manifest])
want, _ := compose.ContentDigest(bb.manifests[sp.Manifest])
if got, _ := compose.ContentDigest(compact.Bytes()); err == nil && (want == "" || read.ComposedHow.Manifests["spiegel"] != want || got != want) {
t.Errorf("the manifest spiegel's build was made of: %v, want %s (compacted %s)", read.ComposedHow.Manifests, want, got)
}
// Issue 1921: held by its manifest's verdict, spiegel is still held in what the node composes, and the input a check
// is answered with says what holds it and carries the build the node would compose it at, taken in while done; a
// node's own read takes in no build of it.
sp.Verdict, sp.Problems = meshcatalogclient.VerdictRefused, []string{"p"}
bb.modules["spiegel"] = sp
for _, done := range []bool{true, false} {
if !done {
bb.builds["spiegel "+sp.Fp2].State = builds.BuildBuilding
}
read, err = ComposingInput(ctx, bb, Mesh{}, Scope{Node: "n1"}, map[string]string{"lemurs": fp("e"), "spiegel": sp.Fp2})
hold := read.Holds["spiegel"]
raw, _ = json.Marshal(hold.Built)
if err != nil || !read.Catalogue["spiegel"].Stubbed || hold.By != compose.HeldByVerdict || hold.Build != sp.Fp2 ||
strings.Contains(string(raw), strings.Repeat("7", 64)) != done || (hold.Built == nil) == done {
t.Errorf("spiegel refused, its build done %v: %v, hold %+v (built %s)", done, err, hold, raw)
}
}
if own, err := Input(ctx, bb, Mesh{}, Scope{Node: "n1"}); err != nil || own.Holds["spiegel"] != (compose.Hold{By: compose.HeldByVerdict}) {
t.Errorf("a node's own read of spiegel refused: %v, hold %+v", err, own.Holds["spiegel"])
}
}
// (C) Issue 1126, C5: what the others read of a module carried as it runs is what runs, never its stub's: the filter
+159 -12
View File
@@ -45,9 +45,9 @@ import (
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-contracts/identity-and-access/go/clients/meshvaultclient"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
engcomp "github.com/novox/mesh-host/internal/compositions"
"git.novox.be/novox/mesh-networking/modules/mesh-wireguard/go/overlay"
@@ -81,7 +81,7 @@ type Reader interface {
// and each it grants as a provider, with the copy sealed to it (the vault's `own` with the keys, ADR 0550 §5). A key
// the vault holds no value for is left out; the composition then stands in for it, and Compose refuses it.
Credentials(ctx context.Context, node string, asConsumer, asProvider []compose.CredentialKey) ([]compose.Credential, []compose.Issued, error)
// Build is a module's build at a fingerprint, the builder's key (builds.BuilderBuildsBucket, BuilderBuildKey); nil
// Build is a module's build at a fingerprint, the builder's key (BuilderBuildKey); nil
// when the builder holds none. The composing node composes each of its modules from its build (issue 1126).
Build(ctx context.Context, module, fp string) (*builds.BuilderBuild, error)
// Archive is the archive at url, a kept reference routed to the artifact store as the composing node reaches it
@@ -156,6 +156,17 @@ type Scope struct {
Assignments map[string]*meshmoduleassigner.Assignments
InOwn []string
Running []engcomp.Composed
// Heard is, by node and module, what each node said it runs, from the node-engine seat's event `running` kept by the
// composing node's own consumer (design 47 G §199): the passing of a seat reads it, and a module nothing was heard of
// counts by the trunk's entry. Never read from a have key, a want or a call.
Heard map[string]map[string]Heard
// Engine is told, at each composing of the node's input, the node-engine's own module: the one whose entry of the catalog half
// claims the seat node-engine, else the one of the node's heard modules that runs the build OwnFingerprint names (design 47 G
// §527); "" when neither shows it. Nil: told to nobody.
Engine func(module string)
// OwnFingerprint is the fingerprint this node-engine was started at (its MESH_FINGERPRINT); empty when it was started
// without one, as an engine of before G.
OwnFingerprint string
// HeldToday is, in an answer, the modules the node keeps today as they run ("node" for its own composition), each with
// how ("held", "failed", "runs a build of the old form"): a candidate that cannot compose one, or holds one only
// behind one of them, keeps it as today, which is said under CandidateAnswer.Held and is no refusal (issue 1126).
@@ -167,6 +178,25 @@ type Scope struct {
// Kept is told, in words, each module composed as what runs shows because its manifest does not yet say it (the tool
// runner, the core's credentials; ADR 0601 §4), so the node says it. Nil: said nowhere.
Kept func(module, words string)
// Unmade is told, in words, each module placed on the node that is composed from the indexer's manifest and not from a
// build, with why: the indexer's modules hold no build of the new form for it, or the builder holds none done at that
// fingerprint. Nil: said nowhere.
Unmade func(module, why string)
// takeInHeld, set for the input a node answers a check with (ComposingInput), takes in the build of each module placed
// on the node that its manifest's verdict or its decoder holds, into compose.Input.Holds, never into what the node composes: the
// check's composer, judging the change's manifest itself, composes it at that build (issue 1921).
takeInHeld bool
// manifests, set with takeInHeld, is filled with the ContentDigest of the manifest each module placed on the node was
// built from, as it is taken in from that build (compose.ComposedHow.Manifests).
manifests map[string]string
// Holds is told, after Compose read the input, what holds each module it holds by its manifest's verdict or its
// decoder (compose.Input.Holds), with the build it would compose it at, so the node says why it holds it (issue 1921,
// the composition verb). Nil: said nowhere.
Holds func(map[string]compose.Hold)
// Held is the modules the node holds before anything of their builds is read, each with why (a node that has not read
// the seat list, novox/hq ADR 0636 §1): each is held as it runs, as a module refused alone is, so no take-in or
// artifact reference of it can refuse the node (issue 1966). Nil holds none.
Held map[string]string
}
// InOwn is, of the modules p places on its node, those whose resources the node's own composition it runs
@@ -214,21 +244,23 @@ func Input(ctx context.Context, r Reader, mesh Mesh, s Scope) (compose.Input, er
// taken is a module's manifest as a reader takes it (novox/hq ADR 0578 §3): decoded leniently, with the indexer's verdict
// from its modules value. A refused one, or bytes that do not decode, stands as its stub (unusable) and why says so; a
// manifest not read past a field it set aside holds its name alone and is held the same.
func taken(name string, raw []byte, m meshcatalog.ModulesByModuleValue) (man catalogue.Manifest, why string) {
//
// by is what holds it (compose.HeldByVerdict or compose.HeldByDecoder), empty when nothing does.
func taken(name string, raw []byte, m meshcatalog.ModulesByModuleValue) (man catalogue.Manifest, why, by string) {
if v := meshcatalogclient.ModuleVerdict(name, m); v != nil {
if refused, _ := meshcatalogclient.TakeVerdict(v); refused {
return unusable(name, raw), meshcatalogclient.VerdictWhy(*v)
return unusable(name, raw), meshcatalogclient.VerdictWhy(*v), compose.HeldByVerdict
}
}
man, err := catalogue.Decode(raw)
switch {
case err != nil:
return unusable(name, raw), fmt.Sprintf("the manifest of %s does not decode: %v", name, err)
return unusable(name, raw), fmt.Sprintf("the manifest of %s does not decode: %v", name, err), compose.HeldByDecoder
case man.ReadNoFurther():
return unusable(name, raw), fmt.Sprintf("the manifest of %s could not be read past a field this reader does not "+
"know: %s", name, man.SetAside())
"know: %s", name, man.SetAside()), compose.HeldByDecoder
}
return man, ""
return man, "", ""
}
// input is Input; with refuse set, a module placed on the composing node whose build cannot be read or taken in, or whose
@@ -374,9 +406,13 @@ func readBase(ctx context.Context, r Reader, mesh Mesh, s Scope) (base, error) {
// follows (issue 774: power's bar item refused every node's composition, 2026-10-30; ADR 0578 §3: one refused
// manifest silenced disk-load for 45 hours). It stands in as its stub, so what it provides and requires still
// shapes the rest; it is held on the composing node where it is placed there, and elsewhere only its offer is read.
man, why := taken(name, raw, m)
man, why, by := taken(name, raw, m)
if why != "" {
named(&in.Refused, name, why)
if in.Holds == nil {
in.Holds = map[string]compose.Hold{}
}
in.Holds[name] = compose.Hold{By: by}
if ownPlaced[name] {
b.ownRefused = append(b.ownRefused, [2]string{name, why})
}
@@ -526,6 +562,7 @@ func (b base) finish(ctx context.Context, r Reader, build func(ctx context.Conte
if err := fromBuilds(ctx, r, build, &in, s, b.assignments[s.Node], b.modules, read, parse, refuse, wanted); err != nil {
return compose.Input{}, err
}
keepHeard(&in, s, read)
// Its own modules' secrets, sealed to its own key by the vault: a composition carries them sealed (ADR 0550 §5).
// Asked only when a module placed on it declares one: a node none of whose modules has a secret composes
// whether or not the vault answers.
@@ -556,6 +593,61 @@ func (b base) finish(ctx context.Context, r Reader, build func(ctx context.Conte
return in, nil
}
// Heard is one `running` event as the scope keeps it: the fingerprint of the build, the digest of its manifest, or Stopped.
type Heard struct {
Fingerprint string
Manifest string
Stopped bool
}
// EngineModule is the node-engine's own module (design 47 G §527): the module whose catalog entry claims the seat node-engine,
// which the validator makes one per revision (§508). With none claiming it, which only a revision of before G shows, as at a
// revert of G, the module this node heard run the build its own fingerprint names; "" when neither shows it.
func EngineModule(in compose.Input, node, ownFingerprint string) string {
for _, name := range sortedKeys(in.Catalogue) {
if resolution.ClaimsEngine(in.Catalogue[name]) {
return name
}
}
if ownFingerprint == "" {
return ""
}
for i := range in.Nodes {
if in.Nodes[i].Name != node {
continue
}
for _, module := range sortedKeys(in.Nodes[i].Heard) {
if in.Nodes[i].Heard[module].Fingerprint == ownFingerprint {
return module
}
}
}
return ""
}
// keepHeard puts, as each node's Heard, what it said it runs of each module it is assigned, with the manifest of the build
// read by the event's digest; a digest that cannot be read leaves the entry unread, and the passing says so (§459).
func keepHeard(in *compose.Input, s Scope, read func(module, digest string) (catalogue.Manifest, error)) {
for i := range in.Nodes {
n := &in.Nodes[i]
heard := s.Heard[n.Name]
if len(heard) == 0 {
continue
}
n.Heard = map[string]compose.Heard{}
for _, module := range sortedKeys(heard) {
h := heard[module]
got := compose.Heard{Fingerprint: h.Fingerprint, Stopped: h.Stopped}
if !h.Stopped && h.Manifest != "" {
if man, err := read(module, h.Manifest); err == nil {
got.Manifest, got.Read = man, true
}
}
n.Heard[module] = got
}
}
}
// Compose is the node's compositions over the bus (ADR 0547 §2): input for s, then composed at the fingerprints fps
// gives each module in the Reading mode, as the controller composes them, by the one rule its answer to a candidate
// follows too (composeEach, issue 1126): a module refused alone (its build unread or not taken in, its composition
@@ -564,7 +656,10 @@ func (b base) finish(ctx context.Context, r Reader, build func(ctx context.Conte
// node never applies a stand-in. Only a refusal that is no single module's (the node's own resources, a value served
// elsewhere, a stand-in for the node's own) refuses the node, err.
func Compose(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]string) (out map[string]engcomp.Composed, skipped []string, held map[string]string, err error) {
held = map[string]string{}
held = maps.Clone(s.Held)
if held == nil {
held = map[string]string{}
}
// A node refused whole says what it held on the way, since the refusal can be of a held module's stub (issue 774,
// 2026-10-30 19:30: node-tools held, then "its build produced 0 bundle(s)" with its own refusal unsaid).
defer func() {
@@ -580,6 +675,19 @@ func Compose(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]s
if err != nil {
return nil, nil, held, err
}
if s.Engine != nil {
s.Engine(EngineModule(in, s.Node, s.OwnFingerprint))
}
if s.Holds != nil {
holds := map[string]compose.Hold{}
for name := range held {
if h, said := in.Holds[name]; said {
h.Build = fps[name]
holds[name] = h
}
}
s.Holds(holds)
}
// The pair credentials its composition reads, asked of the vault sealed to it (ADR 0550 §5).
asConsumer, asProvider, err := compose.CredentialsNeeded(in, s.Node)
if err != nil {
@@ -600,11 +708,12 @@ func Compose(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]s
// credential: the sandbox composes no pair value.
func ComposingInput(ctx context.Context, r Reader, mesh Mesh, s Scope, fps map[string]string) (compose.Input, error) {
held := map[string]string{}
s.takeInHeld, s.manifests = true, map[string]string{}
in, err := input(ctx, r, mesh, s, func(module string, err error) { held[module] = err.Error() }, fps)
if err != nil {
return compose.Input{}, err
}
c := &compose.ComposedHow{Node: s.Node, Builds: maps.Clone(fps), Held: held}
c := &compose.ComposedHow{Node: s.Node, Builds: maps.Clone(fps), Held: held, Manifests: s.manifests}
for _, run := range s.Running {
if _, isHeld := held[run.Module]; isHeld && run.Module != "" {
c.Running = append(c.Running, run)
@@ -901,7 +1010,14 @@ func fromBuilds(ctx context.Context, r Reader, build func(ctx context.Context, m
}
}
for _, pm := range p.Modules {
if why, held := s.Held[pm.Module]; held && refuse != nil {
stub(pm.Module, errors.New(why)) // nothing of its build is read: it keeps what runs (Scope.Held)
continue
}
m, ok := modules[pm.Module]
if (!ok || m.Fp2 == "") && s.Unmade != nil && wanted[pm.Module] != "" {
s.Unmade(pm.Module, "the indexer's modules hold no build of the new form for it (fingerprint "+m.Fp2+"), so it is composed from the manifest")
}
if !ok || m.Fp2 == "" || keptAt[pm.Module] != "" || compose.Folds(slices.Contains(s.InOwn, pm.Module), wanted[pm.Module]) {
// A module it runs inside its own composition is composed there as before while it is folded: no build of the
// new form is what it runs (Scope.InOwn). Wanted at a build of the new form it is no longer folded (compose.Folds,
@@ -909,7 +1025,23 @@ func fromBuilds(ctx context.Context, r Reader, build func(ctx context.Context, m
continue
}
if _, refused := in.Refused[pm.Module]; refused {
continue // held as its stub: no build of a refused manifest is taken in (ADR 0578 §3)
// Held as its stub: no build of a refused manifest is composed (ADR 0578 §3). For a check, the build the node
// would compose it at is said beside it, and taken in when done, for the check's composer alone.
if hold, said := in.Holds[pm.Module]; s.takeInHeld && said {
hold.Build = m.Fp2
if w := wanted[pm.Module]; w != "" {
hold.Build = w
}
// Its stub names no build, so the builder is asked: a build not done, or not taken in, is awaited.
if b, err := build(ctx, pm.Module, hold.Build); err == nil && b != nil && b.State == builds.BuildDone {
if resolved, err := built(pm.Module, hold.Build, b); err == nil {
hold.Built = &resolved
}
}
in.Holds = maps.Clone(in.Holds)
in.Holds[pm.Module] = hold
}
continue
}
if man, ok := in.Catalogue[pm.Module]; !ok || (man.Build == nil && !man.Templated()) {
continue // nothing is built of it: its manifest is what it composes from
@@ -931,6 +1063,13 @@ func fromBuilds(ctx context.Context, r Reader, build func(ctx context.Context, m
stub(pm.Module, fmt.Errorf("frombus: %s is wanted at %s, of which the builder holds no done build yet; it "+
"keeps running what it runs", pm.Module, m.Fp2))
}
if s.Unmade != nil && wanted[pm.Module] != "" {
state := "none"
if b != nil {
state = string(b.State)
}
s.Unmade(pm.Module, "the builder holds no done build of it at "+m.Fp2+" (build: "+state+"), so it is composed from the indexer's manifest, which names no artifacts")
}
continue // nothing of it runs: composed from the indexer's manifest, as before
}
resolved, err := built(pm.Module, m.Fp2, b)
@@ -942,6 +1081,14 @@ func fromBuilds(ctx context.Context, r Reader, build func(ctx context.Context, m
continue
}
in.Catalogue[pm.Module] = resolved
if s.manifests != nil {
// Unread or not JSON, it is left out: the check's composer then compares what the manifests say (issue 1921).
if raw, err := r.Manifest(ctx, b.Manifest); err == nil {
if digest, err := compose.ContentDigest(raw); err == nil {
s.manifests[pm.Module] = digest
}
}
}
}
for i := range in.Nodes {
if in.Nodes[i].Name != s.Node {
+33 -2
View File
@@ -15,10 +15,11 @@ import (
"git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
nodeengine "git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"github.com/novox/mesh-host/internal/compositions"
"maps"
"os"
"reflect"
"slices"
@@ -575,6 +576,36 @@ func TestInput(t *testing.T) {
t.Errorf("the want's build is kept on the composing node alone: %v, %v", in.Nodes, err)
}
// Design 47 G §199 (test 56): what each node said it runs (the seat node-engine's event running) is that node's Heard,
// with the manifest read by the event's digest; a digest that cannot be read leaves the entry unread, and a stopped
// module has none.
in, err = Input(ctx, kept, Mesh{}, Scope{Node: "laptop", Heard: map[string]map[string]Heard{
"laptop": {"xterm": {Fingerprint: "f1", Manifest: digest}, "media": {Stopped: true}}, "anchor": {"xterm": {Fingerprint: "f2", Manifest: "gone"}}}})
if err != nil {
t.Fatal(err)
}
for _, n := range in.Nodes {
h := n.Heard["xterm"]
switch {
case n.Name == "laptop" && (!h.Read || h.Fingerprint != "f1" || h.Manifest.Module != "xterm" || !n.Heard["media"].Stopped):
t.Errorf("laptop's Heard holds %+v", n.Heard)
case n.Name == "anchor" && (h.Read || h.Fingerprint != "f2"):
t.Errorf("anchor's Heard holds %+v, want f2 unread", n.Heard)
}
}
// Design 47 G §527: the node-engine's own module is the one whose entry claims its seat; with none, the one this node heard
// run the build its own fingerprint names; with neither, none.
if got := EngineModule(in, "laptop", "f1"); got != "xterm" {
t.Errorf("by the own fingerprint: %q, want xterm", got)
}
if got := EngineModule(in, "laptop", "other"); got != "" {
t.Errorf("an unheard fingerprint names %q", got)
}
in.Catalogue = maps.Clone(in.Catalogue)
in.Catalogue["the-engine"] = catalogue.Manifest{Module: "the-engine", Claims: []catalogue.Claim{{Name: "node-engine"}}}
if got := EngineModule(in, "laptop", "f1"); got != "the-engine" {
t.Errorf("by the claim: %q, want the-engine", got)
}
// Issue 1126, the seed's refusals. restic and the provisioner's client as the catalogue holds them (mesh-catalog
// c05aa5e6), restic moved with an own secret the vault leaves out (it hands it to the client, mesh-vault ownnode.go
// handed): composed from the indexer's manifest restic is not handed and stands in for that secret, as novox's,
@@ -14,9 +14,9 @@ import (
"errors"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"github.com/novox/mesh-host/internal/compose"
"github.com/novox/mesh-host/internal/compose/frombus"
"strings"
@@ -7,9 +7,9 @@ import (
"git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
engcomp "github.com/novox/mesh-host/internal/compositions"
"reflect"
"slices"
+34 -2
View File
@@ -7,9 +7,9 @@ import (
"fmt"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"github.com/novox/mesh-host/internal/compose"
"github.com/novox/mesh-host/internal/compose/resolution"
engcomp "github.com/novox/mesh-host/internal/compositions"
@@ -300,3 +300,35 @@ func aHeldModuleOfItsOwnAccountNeverRefusesTheNode(t *testing.T) {
}
}
}
// Issue 1966: two modules claiming one seat held once cost the second claimant, not the node: Compose holds it with the claim's
// words, composes the first, and (the node-engine keeping what it handed of a held module) leaves its composition out, whether or
// not it runs.
func TestASecondClaimantOfASeatIsHeldAndTheNodeComposesTheRest(t *testing.T) {
first := `{"module":"alpha","version":"1","claims":[{"name":"the-seat","scope":"node"}]}`
second := `{"module":"beta","version":"1","claims":[{"name":"the-seat","scope":"node"}]}`
for _, running := range []bool{false, true} {
b, fps := aNode(t, first, second)
scope := Scope{Node: "n1"}
if running {
scope.Running = []engcomp.Composed{{Node: "n1", Module: "beta", FP: fps["beta"], Resources: nil}}
}
got, _, held, err := Compose(context.Background(), b, Mesh{}, scope, fps)
if err != nil || len(held) != 1 || !strings.Contains(held["beta"], "alpha and beta both claim") {
t.Fatalf("running %v: err %v, held %v; want beta held with the claim's words", running, err, held)
}
for _, c := range got {
if c.Module == "beta" {
t.Errorf("running %v: the held claimant is composed: %+v", running, c)
}
}
// A check that adds the second claimant is refused for it, in the claim's words, and not judged to compose.
p := b.assignments["n1"]
ans := Answer(context.Background(), b, Mesh{}, Scope{Node: "n1", Assignments: map[string]*meshmoduleassigner.Assignments{"n1": &p}},
fps, scope.Running, "p-h1", 1)
if ans.Verdict != nodeengine.CandidateAnswerVerdictRefused || len(ans.Refusals) != 1 || ans.Refusals[0].Module != "beta" ||
!strings.Contains(ans.Refusals[0].Why, "both claim") {
t.Errorf("running %v: the answer is %+v, want beta refused with the claim's words", running, ans)
}
}
}
+2 -2
View File
@@ -19,9 +19,9 @@ import (
"git.novox.be/novox/mesh-contracts/go/clients/nodeengineclient"
"git.novox.be/novox/mesh-contracts/go/seats/meshmoduleassigner"
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-builder/builds"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/clients/meshcatalogclient"
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"github.com/novox/mesh-host/internal/buckets"
"github.com/novox/mesh-host/internal/compose"
"github.com/novox/mesh-host/internal/compose/resolution"
+62
View File
@@ -16,6 +16,9 @@
package compose
import (
"bytes"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
@@ -114,11 +117,36 @@ type Input struct {
// (compose.Stub), so it is held: nothing of its own is composed, while what it provides and requires still shapes the
// rest. A module not named is accepted, a missing verdict included.
Refused map[string]string `json:",omitempty"`
// Holds is, by module Refused names, what holds it and the build its node would compose it at (issue 1921): a check's
// composer that judges the change's manifest itself (novox/hq ADR 0578 §2) composes a module refused by the running
// indexer alone from that build, and names one whose build it is not given. Nil from a node-engine before it.
Holds map[string]Hold `json:",omitempty"`
// SetAside is each module whose manifest has a field the reader's decoder does not know, by module, with what was set
// aside (catalogue.Manifest.SetAside): the rest of it is composed, and the reader names it (ADR 0578 §6).
SetAside map[string]string `json:",omitempty"`
}
// What holds a module Refused names (Hold.By).
const (
// HeldByVerdict is a module the indexer's judge refused the manifest of (its verdict).
HeldByVerdict = "verdict"
// HeldByDecoder is a module whose bytes this reader's decoder does not decode, or cannot read past a field it does not
// know: no judge of the manifest overrules it.
HeldByDecoder = "decoder"
)
// Hold is what holds one module Refused names, and the build its node would compose it at (Input.Holds).
type Hold struct {
// By is HeldByVerdict or HeldByDecoder.
By string
// Build is the fingerprint the node would compose it at, its want's or else the indexer's newest; empty where it is not
// placed on the composing node or names no build.
Build string `json:",omitempty"`
// Built is its manifest as the node takes it in from that build, when the builder holds it done; nil while that build
// is awaited (or not asked: only a node's answer to a check asks it).
Built *catalogue.Manifest `json:",omitempty"`
}
// Issued is one pair credential made for a consumer of a provider (inv.Secret): ForProvider is its copy sealed to the
// provider, optional.
type Issued struct {
@@ -212,6 +240,9 @@ type Node struct {
// own resolution only, never to another node's, and a declaration carries the kept commit for the module.
Kept map[string]catalogue.Manifest
KeptBuilds map[string]string
// Heard is, by module, what this node said it runs, from the node-engine seat's event `running` (design 47 G §199): the
// passing of a seat resolves the build each assignment runs from it. Absent for a module nothing was heard of.
Heard map[string]Heard
// Contract is the contract of its node-engine's newest statement (inv.HealthOf); Stated is whether it ever stated one.
Contract int
Stated bool
@@ -258,6 +289,28 @@ type ComposedHow struct {
// HeldBefore is each module the node holds composing without the change (the candidate's base), held by the read or
// blamed later for a refusal: the change refuses none of them (issue 774, review round 1).
HeldBefore map[string]string
// Manifests is, by module taken in from its build, the ContentDigest of the manifest that build was made of (its
// build record's manifest), filled in the input a node answers a check with: the check's composer composes a module
// at its build only where the change's manifest is that one (issue 1921). Nil from a node-engine before it.
Manifests map[string]string
}
// ContentDigest is the hex sha256 of a manifest's JSON in one form, its keys sorted and its spacing removed, its numbers
// as written: the same for the bytes its repository holds and for those bytes carried inside a request, which JSON
// encoding compacts on the way (a composition request through a build module), so the two are compared as one manifest.
func ContentDigest(raw []byte) (string, error) {
dec := json.NewDecoder(bytes.NewReader(raw))
dec.UseNumber()
var v any
if err := dec.Decode(&v); err != nil {
return "", fmt.Errorf("compose: the manifest is not JSON: %w", err)
}
canonical, err := json.Marshal(v)
if err != nil {
return "", err
}
sum := sha256.Sum256(canonical)
return hex.EncodeToString(sum[:]), nil
}
// Place is where a node is, as its assignment says (inv.Overlay).
@@ -387,6 +440,15 @@ type Credential struct {
Why string
}
// Heard is what a node's `running` event for a module said: the build's fingerprint, or Stopped (no fingerprint, §365), with
// the manifest of the build's entry read by the event's manifest digest (Read false when it could not be read).
type Heard struct {
Fingerprint string
Stopped bool
Manifest catalogue.Manifest
Read bool
}
// Run is the build an assignment names: the commit it is of and the build made from it (chain.Ref).
type Run struct {
Commit string
+1 -1
View File
@@ -283,7 +283,7 @@ func (c *composer) onTheNetworkNow() []Node {
if len(n.Assigned) == 0 {
continue
}
got, err := resolution.Resolve(c.in.Catalogue, n.Assigned,
got, _, err := resolution.ResolveHolding(c.in.Catalogue, n.Assigned,
catalogue.Node{Name: n.Name, Site: n.Place.Site, Capabilities: n.Capabilities},
catalogue.World{Unchecked: true, Holdings: c.in.Holdings})
if err != nil {
+7 -9
View File
@@ -164,9 +164,8 @@ func TestANodeNotOnThePrivateNetworkGetsNoAddressAndNoNetwork(t *testing.T) {
func TestARefusedNodeIsSaidWhenTheNetworkCannotBeComposed(t *testing.T) {
in := aMesh(t)
nodeOf(t, &in, "anchor").Place.Key = ""
one := catalogue.Manifest{Module: "rival-one", Version: "1", Claims: []catalogue.Claim{{Name: "the-seat", Scope: catalogue.ScopeNode}}}
two := catalogue.Manifest{Module: "rival-two", Version: "1", Claims: []catalogue.Claim{{Name: "the-seat", Scope: catalogue.ScopeNode}}}
in.Catalogue["rival-one"], in.Catalogue["rival-two"] = one, two
in.Catalogue["rival-one"] = catalogue.Manifest{Module: "rival-one", Version: "1", Resources: []map[string]any{{"id": "r", "type": "directory", "path": "/var/lib/the-same", "mode": "0700"}}}
in.Catalogue["rival-two"] = catalogue.Manifest{Module: "rival-two", Version: "1", Resources: []map[string]any{{"id": "r", "type": "directory", "path": "/var/lib/the-same", "mode": "0700"}}}
hub := nodeOf(t, &in, "anchor")
hub.Assigned = append(hub.Assigned, "rival-one", "rival-two") // its own set is incoherent: the hub does not resolve
_, err := Generators(in)
@@ -293,14 +292,13 @@ func TestEachFieldOfTheInputReachesTheGenerator(t *testing.T) {
}
laptop := nodeOf(t, &in, "laptop")
laptop.Assigned = append(laptop.Assigned, "holder-one", "holder-two")
// Without a key: a machine that does not resolve and holds a complete place keeps it on the network (issue 1833).
laptop.Place.Key = ""
if got := addressesOf(in); got["laptop"] != "" {
t.Fatalf("a machine whose set claims one seat twice is on the network: %v", got)
// Two claimants and nobody on record: the second is held and the machine stays on the network (issue 1966).
if got, err := Resolve(in, "laptop"); err != nil || len(got.Held) != 1 || got.Held["holder-two"] == "" || addressesOf(in)["laptop"] != "10.77.0.2" {
t.Fatalf("two claimants, none on record: held %v, %v", got.Held, err)
}
in.Holdings = []catalogue.Held{{Claim: "mesh-broker", Scope: catalogue.ScopeMesh, Node: "laptop", Module: "holder-one"}}
if got := addressesOf(in); got["laptop"] != "10.77.0.2" {
t.Errorf("a holder on record did not settle the seat: %v", got)
if got, err := Resolve(in, "laptop"); err != nil || len(got.Held) != 0 {
t.Errorf("a holder on record did not settle the seat: held %v, %v", got.Held, err)
}
})
}
+8 -2
View File
@@ -37,6 +37,9 @@ type Declared struct {
// ADR 0163, rule 6), sorted; LeftOutWhy is why each was.
LeftOut []string
LeftOutWhy map[string]string
// Held is, of those, each module left out because its claim cannot be held or its credential was refused, and why (issue
// 1966): the node keeps what runs of it and the composer says it held. Not on the wire.
Held map[string]string
// Not on the wire:
Received map[string]map[string][]catalogue.Contribution
@@ -60,7 +63,8 @@ type Declared struct {
ForeseenPorts []string
// CoreBoxesWanted are the core credentials whose box the send must ask the vault for and keep.
CoreBoxesWanted []CoreBoxWanted
// Passes are the seats the mesh would pass on, for the caller to record, and Said the lines found while composing.
// Passes are the seats composed as passed to their only claimant (ADR 0660), recorded by nobody here, and Said the lines found
// while composing, the passes among them.
Passes Passes
Said []string
}
@@ -198,7 +202,7 @@ func (c *composer) declaration(nodeName string, mode Mode) (Declared, error) {
if err != nil {
return Declared{}, err
}
out := Declared{Resources: composed.Resources, Adoption: adoptionFor(plan, node, composed),
out := Declared{Held: resolved.Held, Resources: composed.Resources, Adoption: adoptionFor(plan, node, composed),
Received: composed.Received, Mesh: with.Mesh, BusUsers: with.BusUsers,
LeftOut: sortedKeysOf(composed.LeftOut), LeftOutWhy: composed.LeftOut,
Withheld: rendered.Withheld, Unbound: rendered.Unbound,
@@ -228,6 +232,8 @@ func (c *composer) declaration(nodeName string, mode Mode) (Declared, error) {
return Declared{}, err
}
out.Passes = passes
// A pass is composed, never recorded (ADR 0660 §4): the answer says it, with what two claimants leave unsettled.
out.Said = append(out.Said, passes.Lines()...)
for _, l := range left {
out.Said = append(out.Said, l.String())
}
+2 -18
View File
@@ -112,22 +112,6 @@ func TestAManifestTheNodeEngineRefusesComesBackAsProblemsAndNoError(t *testing.T
}
}
// 1298/AC3: a node whose own set does not compose returns an error and no declaration, told apart from an unreadable Input.
func TestANodeWhoseSetDoesNotComposeReturnsAnErrorAndNoDeclaration(t *testing.T) {
in := aMesh(t)
in.Catalogue["rival-one"] = catalogue.Manifest{Module: "rival-one", Version: "1", Claims: []catalogue.Claim{{Name: "the-seat", Scope: catalogue.ScopeNode}}}
in.Catalogue["rival-two"] = catalogue.Manifest{Module: "rival-two", Version: "1", Claims: []catalogue.Claim{{Name: "the-seat", Scope: catalogue.ScopeNode}}}
laptop := nodeOf(t, &in, "laptop")
laptop.Assigned = append(laptop.Assigned, "rival-one", "rival-two")
d, problems, err := Compose(in, "laptop")
if err == nil || !IsUnresolvable(err) || d.Resources != nil || problems != nil {
t.Fatalf("a set that does not compose: %v %v %v", d.Resources, problems, err)
}
if _, _, err := Compose(in, "nowhere"); err == nil || IsUnresolvable(err) {
t.Fatalf("a node the Input does not have: %v", err)
}
}
// 1298/AC4: composing the same Input twice gives byte-identical bodies, and the Input is unchanged.
func TestComposingTwiceGivesTheSameBytesAndChangesNothing(t *testing.T) {
mk := func() Input {
@@ -392,8 +376,8 @@ func overlayName() string { return overlay.Name }
func TestANodeNotCountedAsOnTheNetworkIsSaidInTheResult(t *testing.T) {
in := aMesh(t)
nodeOf(t, &in, "laptop").Place.Key = ""
in.Catalogue["rival-one"] = catalogue.Manifest{Module: "rival-one", Version: "1", Claims: []catalogue.Claim{{Name: "the-seat", Scope: catalogue.ScopeNode}}}
in.Catalogue["rival-two"] = catalogue.Manifest{Module: "rival-two", Version: "1", Claims: []catalogue.Claim{{Name: "the-seat", Scope: catalogue.ScopeNode}}}
in.Catalogue["rival-one"] = catalogue.Manifest{Module: "rival-one", Version: "1", Resources: []map[string]any{{"id": "r", "type": "directory", "path": "/var/lib/the-same", "mode": "0700"}}}
in.Catalogue["rival-two"] = catalogue.Manifest{Module: "rival-two", Version: "1", Resources: []map[string]any{{"id": "r", "type": "directory", "path": "/var/lib/the-same", "mode": "0700"}}}
laptop := nodeOf(t, &in, "laptop")
laptop.Assigned = append(laptop.Assigned, "rival-one", "rival-two")
said := func(lines []string) bool {
+2 -2
View File
@@ -147,8 +147,8 @@ func TestANetworkThatCannotBeComposedFromWhatRemainsStillFails(t *testing.T) {
// among the sources "from the mesh" -- and another machine still composes.
func TestAMachineThatDoesNotResolveKeepsItsNameAndItsAddressInTheMesh(t *testing.T) {
in := fourOnTheNetwork(t)
in.Catalogue["rival-one"] = catalogue.Manifest{Module: "rival-one", Version: "1", Claims: []catalogue.Claim{{Name: "the-seat", Scope: catalogue.ScopeNode}}}
in.Catalogue["rival-two"] = catalogue.Manifest{Module: "rival-two", Version: "1", Claims: []catalogue.Claim{{Name: "the-seat", Scope: catalogue.ScopeNode}}}
in.Catalogue["rival-one"] = catalogue.Manifest{Module: "rival-one", Version: "1", Resources: []map[string]any{{"id": "r", "type": "directory", "path": "/var/lib/the-same", "mode": "0700"}}}
in.Catalogue["rival-two"] = catalogue.Manifest{Module: "rival-two", Version: "1", Resources: []map[string]any{{"id": "r", "type": "directory", "path": "/var/lib/the-same", "mode": "0700"}}}
hub := nodeOf(t, &in, "anchor")
hub.Assigned = append(hub.Assigned, "rival-one", "rival-two")
if _, err := Resolve(in, "anchor"); !IsUnresolvable(err) {
+11
View File
@@ -114,6 +114,17 @@ func (c *composer) render(nodeName string, mode Mode) (Rendered, error) {
out.CoreBoxesWanted = wanted
}
// A module whose claim cannot be held, or whose credential was refused, is left out of the declaration and run as it runs:
// the machine composes the rest (issue 1966, ADR 0636 §3).
for module, why := range resolved.Held {
if r.NotPlaced == nil {
r.NotPlaced = map[string]string{}
}
if _, said := r.NotPlaced[module]; !said {
r.NotPlaced[module] = why
}
}
// And a certificate for this machine's name inside the mesh, when anything on it asks.
for _, m := range plan.Modules {
if m.Certificate == nil {
@@ -1,43 +0,0 @@
package resolution
import (
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// artifactSweepVerbs are the seat's verbs, with the controller's arguments so a caller changes only the seat it asks.
func artifactSweepVerbs() []mcat.Verb {
return []mcat.Verb{
{Name: "artifacts", Description: "Every artifact the mesh made that lives in its artifact store, each with its " +
"repository, digest, kind (image or archive) and state: kept, and why (a want names it, or one of the five " +
"newest builds of its module in the builder's log), eligible (made, kept for no reason, not yet let go), or " +
"unrecorded (in the store and named by nothing the mesh made: never removed). Reads only.",
Input: schema(map[string]string{
"repository": "only this repository's references, as <module>/<artifact> (the counts stay whole)",
"collected": "\"true\": list the collected references too (counted always)",
}, nil, "collected")},
{Name: "held", Description: "Whether the artifact store holds each reference given, asked with one HEAD each: " +
"held, gone (the store answered 404) or unknown with why. The builder asks it before it reuses a done key. " +
"Reads only.",
Input: listed(schema(map[string]string{
"references": "the artifacts' references, artifact-store://<module>/<artifact>/blobs/sha256:<hex>: a list, " +
"or one text separated by commas",
}, []string{"references"}), "references")},
{Name: "collect", Description: "The artifact store's sweep, now: what is eligible is let go of. Without confirm a " +
"dry run: what would be let go, nothing held or deleted. With confirm and why: every kept archive held first, " +
"then each eligible artifact let go of, oldest first, an index with the platform manifests no kept index " +
"names, each deletion recorded in the hand-act log. Bounded by most (500 by default, at most 5000) and 45 " +
"seconds; what is left is said. Never touches a digest the mesh did not make.",
Input: schema(map[string]string{
"confirm": "\"true\": let go of what is eligible (needs why); without it nothing is done",
"why": "with confirm: why, required and recorded",
"most": "let go of at most this many (500 by default, at most 5000)",
}, nil, "confirm")},
{Name: "mirrors", Description: "The bases the mesh copied into its artifact store, each kept (a kept build stood on " +
"it), eligible (only builds no longer kept stood on it) or unrecorded (no build the builder's log holds stood on " +
"it: never removed), by what the builder's log says each build stood on. Records nothing by hand.",
Input: schema(map[string]string{}, nil)},
{Name: "images", Description: "Every container image one machine's want and have name, each with the resources " +
"naming it: what the machine keeps when it prunes its images. Reads only.",
Input: schema(map[string]string{"node": "the machine's name"}, []string{"node"})},
}
}
@@ -1,33 +0,0 @@
package resolution
import (
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// checkerVerbs are the checker seat's verbs: ask a try, read one, end one by hand. Optional until the checker serves them
// everywhere.
func checkerVerbs() []mcat.Verb {
return []mcat.Verb{
{Name: "try", Optional: true, Description: "Try a pushed head before its pull request is opened: the checker " +
"judges it as it judges a pull request's head, merged onto its trunk's head, its unit and integration " +
"levels through the build modules, and owes the forge nothing. Answers the try's id at once; try-read " +
"reads where it stands. A try of the same head that waits or runs is the answer to a second ask; at most " +
"4 wait or run at once, mesh-wide.",
Input: schema(map[string]string{"repository": "<owner>/<name>",
"head": "the full commit to try, 40 or 64 hex digits",
"trunk": "the trunk commit to merge it onto, or the trunk's branch for its head now; the trunk's head when absent",
"tests": "a go test -run pattern: only those tests run, and nothing is kept",
"level": "unit or integration: only this level; both when absent"},
[]string{"repository", "head"}),
Replaces: []string{"sh merge-check.sh", "docker run postgres"}},
{Name: "try-read", Optional: true, Description: "Where a try stands, as stored: queued, running, or its end " +
"(pass, fail, or error when it could not run), with what was said of it, the head and the trunk commit " +
"it was merged onto, the time it took and what failed first. Read-only.",
Input: schema(map[string]string{"id": "the try's id, try-<digits>"}, []string{"id"})},
{Name: "try-end", Optional: true, Description: "End a try that waits or runs, as error, by hand: say why (kept in " +
"what it says). Its place among the tries is free at once and the next try of its head asks anew with a new " +
"id. A try that ended already is refused.",
Input: schema(map[string]string{"id": "the try's id, try-<digits>", "why": "why it is ended, in words"},
[]string{"id", "why"})},
}
}
+116
View File
@@ -0,0 +1,116 @@
package resolution
import (
"encoding/json"
"fmt"
"math"
"regexp"
"slices"
"strconv"
"strings"
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// configLinesRef is `${config-lines:<key>:<prefix>}` and `${config-lines:<key>:<prefix>:mib-bytes}` in a file's
// content (design 42 zfs-owns-zfs §3, ADR 0679 §16 (c)): one line per entry of a list value, `<prefix><entry>`. The
// prefix is literal text with no `}`, `$` or newline. A placeholder that holds a whole line takes its newline with it.
var configLinesRef = regexp.MustCompile(`\$\{config-lines:([a-z0-9][a-z0-9_.-]*):([^}$\n]*?)(:mib-bytes)?\}(\n?)`)
// configJSONRef is `${config-json}` in a file's content (design 42 zfs-owns-zfs §9, answer A-15): the module's set
// config values as one JSON object.
var configJSONRef = regexp.MustCompile(`\$\{config-json\}`)
// configLinesInto writes each `${config-lines:…}` of a file's content from the layers: a line `<prefix><entry>` for each
// entry of a list, one for a set scalar, nothing for an unset value or an empty list. With `mib-bytes` an entry given in
// MiB is written in bytes. An entry holding a newline is refused, and the content is left as it was.
func configLinesInto(content string, layers []mcat.Layer, module string) (string, error) {
var refused error
filled := configLinesRef.ReplaceAllStringFunc(content, func(token string) string {
m := configLinesRef.FindStringSubmatch(token)
key, prefix, mib, newline := m[1], m[2], m[3] != "", m[4]
value, set := settingValue(layers, key)
if !set {
return ""
}
entries, isList := value.([]any)
if !isList {
entries = []any{value}
}
var lines []string
for _, e := range entries {
text := plainly(e)
if strings.ContainsAny(text, "\r\n") {
refused = fmt.Errorf("%s: an entry of the config value %q holds a newline, which would begin a line "+
"of its own in the file it is written into (design 42 §3)", module, key)
return token
}
if mib {
n, err := mibWhole(text)
if err != nil || n > 1<<44 {
refused = fmt.Errorf("%s: the config value %q holds %q, and mib-bytes writes a whole number of MiB "+
"in bytes (design 42 §3)", module, key, text)
return token
}
text = strconv.FormatUint(n<<20, 10)
}
lines = append(lines, prefix+text)
}
if len(lines) == 0 {
return ""
}
return strings.Join(lines, "\n") + newline
})
if refused != nil {
return content, refused
}
return filled, nil
}
// configJSONInto writes `${config-json}` in a file's content as the module's set config values in one JSON object
// (design 42 zfs-owns-zfs §9, answer A-15): only keys the module declares under `config` are written, and what no layer sets is absent, and no unset value refuses the module; a
// list and an object are written as JSON, a number, a boolean and a text as JSON values of their own type.
func configJSONInto(content string, layers []mcat.Layer, declared []string) (string, error) {
if !configJSONRef.MatchString(content) {
return content, nil
}
set := map[string]any{}
for _, layer := range layers {
for k, v := range layer.Values {
if slices.Contains(declared, k) {
set[k] = v
}
}
}
var out strings.Builder
enc := json.NewEncoder(&out)
enc.SetEscapeHTML(false)
enc.SetIndent("", " ")
if err := enc.Encode(set); err != nil {
return content, fmt.Errorf("the config values cannot be written as JSON: %w", err)
}
object := strings.TrimSuffix(out.String(), "\n")
return configJSONRef.ReplaceAllLiteralString(content, object), nil
}
// mibWhole is a size in MiB as a whole number: digits, or a whole number a program wrote in exponent form (`1e+06`),
// which is how a float of a million or more prints (design 42 §3).
func mibWhole(text string) (uint64, error) {
if n, err := strconv.ParseUint(text, 10, 64); err == nil {
return n, nil
}
f, err := strconv.ParseFloat(text, 64)
if err != nil || f < 0 || f != math.Trunc(f) || f > 1<<44 {
return 0, fmt.Errorf("not a whole number of MiB")
}
return uint64(f), nil
}
// declaredConfig is the keys a module's manifest declares under `config`.
func declaredConfig(m mcat.Manifest) []string {
keys := make([]string, 0, len(m.Settings))
for k := range m.Settings {
keys = append(keys, k)
}
return keys
}
+111 -38
View File
@@ -4,6 +4,8 @@ import (
"encoding/json"
"errors"
"fmt"
"maps"
"slices"
"sort"
"strconv"
"strings"
@@ -145,7 +147,7 @@ func resolutionCompose(r mcat.Resolution, with mcat.Rendering, owner map[string]
// Before placing, because a placement is a setting too.
left := LeftOut(r, with.Settings, with.Adopted)
kept := make([]mcat.Manifest, 0, len(r.Modules))
var stillBackedUp []mcat.Manifest
var stillBackedUp, stillRunning []mcat.Manifest
for module, why := range with.NotPlaced {
if _, already := left[module]; !already {
left[module] = why
@@ -160,6 +162,10 @@ func resolutionCompose(r mcat.Resolution, with mcat.Rendering, owner map[string]
// the data it already holds on the machine is the reason to keep copying it. Only its data,
// as the backup holder's lines are derived from it, and the directories they name.
stillBackedUp = append(stillBackedUp, backupView(m))
// **And what it runs is still let through** (issue 1966): its containers and services are left as they run, so
// the filter keeps opening what its listens say, at the machine ports this node holds for it, rather than
// closing a port that still serves (the artifact store's 5100 on the control-node, 98 minutes).
stillRunning = append(stillRunning, m)
continue
}
kept = append(kept, m)
@@ -240,13 +246,22 @@ func resolutionCompose(r mcat.Resolution, with mcat.Rendering, owner map[string]
}
if answered {
r.Needs[i].Serves = serves
continue
}
// No module of the composed set serves it: its provider is held on this machine (composed apart, as what runs of
// it), and offers what its manifest serves. None of its own names is composed here, so a key that names one is
// left out and said on the need, as a provider elsewhere leaves it out (ServedLeftOut, novox/hq ADR 0346 §3),
// never handed to the consumer as a placeholder its binding then refuses (issue 1921, novox: "mesh-build-node has
// a file that says ${bound:route:name-web}, and mesh-build-node does not require route").
r.Needs[i].Serves, r.Needs[i].LeftOut = withoutOwnNames(r.Needs[i], r.Node)
}
// Once, from every module's listens -- not per module. A module receiving only its own ports
// would write a rule set that closed every other module on the machine. Each module's per-node
// exposure settings override its listens' source first (novox/hq ADR 0046).
rules, err := Rules(r, with)
filtered := r
filtered.Modules = append(slices.Clone(r.Modules), stillRunning...)
rules, err := Rules(filtered, with)
if err != nil {
return nil, err
}
@@ -644,6 +659,9 @@ func resolutionCompose(r mcat.Resolution, with mcat.Rendering, owner map[string]
// parses strictly and knows no such field.
absent := AbsentFor(r, m.Module)
resources = withoutAbsent(resources, absent)
// **A resource declared only while a config value holds** (design 42 zfs-owns-zfs §2): left out while none of
// its `if-config` conditions holds, and the key dropped from the rest.
resources = withoutUnconfigured(resources, WithDefaults(m, with.Settings[m.Module]))
// No container is given the mesh's names (novox/hq ADR 0148). It used to be: every
// container got the whole roster as `--add-host` entries at creation, and a name that
@@ -891,7 +909,7 @@ func resolutionCompose(r mcat.Resolution, with mcat.Rendering, owner map[string]
// **An operator's value, from the assignment** (novox/hq ADR 0112, ADR 0155): what a
// definition may not carry because it is true of one installation only. Filled from
// the same layers a mergeable file takes, and refused when no layer set it.
if err := unsetInConfigWords(settingInto(copied, WithDefaults(m, with.Settings[m.Module]), m.Module), m); err != nil {
if err := unsetInConfigWords(settingInto(copied, WithDefaults(m, with.Settings[m.Module]), m.Module, declaredConfig(m)...), m); err != nil {
return nil, err
}
// **Placed before anything reads a path.** A pathless directory receives the path
@@ -1233,46 +1251,24 @@ func resolutionFiltersHere(r mcat.Resolution) string {
// is adopted or converged — the one loads it as a filter, the other declares it as openings.
func Rules(r mcat.Resolution, with mcat.Rendering) ([]mcat.Rule, error) {
// A module whose settings cannot compose is left out of the declaration (novox/hq ADR 0163,
// rule 6), and out of the filter with it: nothing of it is declared, so nothing of it is let
// through.
// rule 6), and not out of the filter: what it runs is left running, so a port it serves stays
// open (issue 1966). Its per-node exposure and reaches still narrow it (ADR 0046); where they
// cannot be read for it, its listens open no wider than the mesh, never at the manifest's
// default, which could reopen a port an operator had narrowed.
left := LeftOut(r, with.Settings, with.Adopted)
for module, why := range with.NotPlaced {
if _, already := left[module]; !already {
left[module] = why
}
}
exposure := map[string]map[int]string{}
for _, m := range r.Modules {
if _, isLeft := left[m.Module]; isLeft {
continue
}
e, err := Exposure(m, with.Settings[m.Module])
e, err := exposureOf(m, with.Settings[m.Module])
if err != nil {
return nil, err
}
// And how far each endpoint reaches, which says the same thing to the filter and more
// besides (novox/hq ADR 0138). Folded in here rather than beside: the filter has one
// question — from where — and a reach answers it, so giving it two inputs would let them
// disagree. Reaches refuses a port that both name, so this cannot silently prefer one.
reaches, err := Reaches(m, with.Settings[m.Module])
if err != nil {
return nil, err
}
// **Only for an endpoint the proxy does not serve.** A routed endpoint's port is how the
// proxy reaches it and nothing else (ADR 0045), so `public` there asks for a public name and
// says nothing about the port — opening it to the world as well would undo the arrangement
// the proxy exists for, and would silently reopen a port an operator had narrowed.
//
// Found by trying to express a real module: one whose routed name must be public and whose
// machine-side port must not be. Under one value for both, there was no way to say it.
routed := RoutedPorts(m)
for port, reach := range reaches {
if routed[port] {
continue
if _, isLeft := left[m.Module]; !isLeft {
return nil, err
}
source, ok := FilterSource(reach)
if !ok {
return nil, fmt.Errorf("%s: %q is not a reach the filter can read", m.Module, reach)
}
if e == nil {
e = map[int]string{}
}
e[port] = source
e = noWiderThanTheMesh(m)
}
if e != nil {
exposure[m.Module] = e
@@ -1281,6 +1277,60 @@ func Rules(r mcat.Resolution, with mcat.Rendering) ([]mcat.Rule, error) {
return Filtering(r, with.Generators, with.Ports, exposure)
}
// noWiderThanTheMesh is the exposure of a module left out whose own cannot be read: each of its
// listens from wider than the mesh narrowed to the mesh, a narrower one kept.
func noWiderThanTheMesh(m mcat.Manifest) map[int]string {
var e map[int]string
for _, l := range m.Listens {
if l.From == mcat.FromLocalNetwork || l.From == mcat.FromEverywhere {
if e == nil {
e = map[int]string{}
}
e[l.Port] = mcat.FromMesh
}
}
return e
}
// exposureOf is a module's per-node exposure, declared port to source: its expose setting, and
// the reaches of its endpoints the proxy does not serve.
func exposureOf(m mcat.Manifest, layers []mcat.Layer) (map[int]string, error) {
e, err := Exposure(m, layers)
if err != nil {
return nil, err
}
// And how far each endpoint reaches, which says the same thing to the filter and more
// besides (novox/hq ADR 0138). Folded in here rather than beside: the filter has one
// question — from where — and a reach answers it, so giving it two inputs would let them
// disagree. Reaches refuses a port that both name, so this cannot silently prefer one.
reaches, err := Reaches(m, layers)
if err != nil {
return nil, err
}
// **Only for an endpoint the proxy does not serve.** A routed endpoint's port is how the
// proxy reaches it and nothing else (ADR 0045), so `public` there asks for a public name and
// says nothing about the port — opening it to the world as well would undo the arrangement
// the proxy exists for, and would silently reopen a port an operator had narrowed.
//
// Found by trying to express a real module: one whose routed name must be public and whose
// machine-side port must not be. Under one value for both, there was no way to say it.
routed := RoutedPorts(m)
for port, reach := range reaches {
if routed[port] {
continue
}
source, ok := FilterSource(reach)
if !ok {
return nil, fmt.Errorf("%s: %q is not a reach the filter can read", m.Module, reach)
}
if e == nil {
e = map[int]string{}
}
e[port] = source
}
return e, nil
}
// provisionsAs is the account that reads what the mesh writes for this provider: the one secret
// per consumer it must open to set that consumer's password (novox/hq issue 225).
//
@@ -2526,3 +2576,26 @@ func resolutionGeneratedHere(r mcat.Resolution, with mcat.Rendering) (map[string
}
return generated, nil
}
// withoutOwnNames is what a need's provider serves without each key whose value names one of the provider's own composed
// names (${bound:…}), and the need's left-out keys with each such key said: a provider not composed in this set
// composes none of its names here.
func withoutOwnNames(n mcat.Needed, node string) (map[string]any, []string) {
var asking []string
for _, key := range sortedAnyKeys(n.Serves) {
if text, ok := n.Serves[key].(string); ok && bound.MatchString(text) {
asking = append(asking, key)
}
}
if len(asking) == 0 {
return n.Serves, n.LeftOut
}
out := maps.Clone(n.Serves)
said := slices.Clone(n.LeftOut)
for _, key := range asking {
delete(out, key)
said = append(said, fmt.Sprintf("%s serves %s without %q: it names %s, and %s composes no names of %s, which it "+
"holds (novox/hq ADR 0346)", n.Module, n.Name, key, bound.FindString(n.Serves[key].(string)), orHere(node), n.Module))
}
return out, said
}
@@ -1,75 +0,0 @@
package resolution
import (
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliveryclient"
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// deliveryRemovedInR6 is the row of a verb release 6 removed from the delivery seat.
func deliveryRemovedInR6(name string) mcat.Verb {
return mcat.Verb{Name: name, Description: meshdeliveryclient.RemovedPrefix + meshdeliveryclient.RemovedVerbs()[name], Input: schema(nil, nil), Optional: true}
}
// deliveryVerbs are the delivery seat's tools: what reads the merges and checks, and the acts of a person and healer H2
// on a merge (ADR 0539 clause 17).
func deliveryVerbs() []mcat.Verb {
return []mcat.Verb{
{Name: "deliveries", Description: "Every delivery not final, and those that ended in the last day, one line " +
"each: its id (owner/repository@commit), its state, its group, what it waits for and since when. Narrowed " +
"by state, repository or group.",
Input: schema(map[string]string{"state": "one state, e.g. delivering or held",
"repository": "owner/repository", "group": "a group's id (its branch name)",
"all": "\"true\": the final ones of the last thirty days too"}, nil, "all")},
deliveryRemovedInR6("times"),
{Name: "show", Description: "One delivery or group whole: its delivery plan (what it builds, what each " +
"machine receives, what is not an ordinary send), every transition with when and why, the machine " +
"steps of its walk, its group and its order.",
Input: schema(map[string]string{"id": "a delivery's id, or a group's"}, []string{"id"})},
{Name: "groups", Description: "Every delivery group: its members in order, why each pair is ordered " +
"(declared, built by, version skew, engine before controller, by name), its composed check and its " +
"state, derived from its members.",
Input: schema(map[string]string{"all": "\"true\": the groups that ended too"}, nil, "all")},
deliveryRemovedInR6("what-if"),
{Name: "checks", Description: "What the mesh's checks said of a pull request's head or of one commit: each " +
"of the commit's mesh statuses (mesh/check, the one a main requires from R4e, hq's mesh/repo-check, " +
"mesh/delivery, …) with its state, " +
"description and when it was set; the merge check's full verdict as the controller said it — each " +
"layer's summary, the machine that ran it, when, its build id and its report; and whether the branch's " +
"protection would let it merge, every required status being success.",
Input: schema(map[string]string{"repository": "owner/repository",
"number": "a pull request's number: its head is read",
"commit": "a commit's sha, or the start of one, instead of a pull request"}, []string{"repository"}),
// Added after the seat's first holder shipped: optional until mesh-delivery serves it everywhere.
Optional: true},
{Name: "table", Description: "The state table every delivery runs by: each transition with its guard, " +
"each state's bound and what healer H2 may do once it has passed; and the machine steps' table.",
Input: schema(map[string]string{}, nil)},
{Name: "stalled", Description: "Every delivery held past its state's bound, with the transition the table " +
"lets healer H2 take for it, or none.",
Input: schema(map[string]string{}, nil)},
{Name: "recheck", Description: "Check a rejected or ready delivery again: it goes back to proposed and the " +
"controller is asked for its check. With why.",
Input: schema(map[string]string{"id": "the delivery's id", "why": "why, kept with the transition"},
[]string{"id", "why"})},
deliveryRemovedInR6("release"),
deliveryRemovedInR6("stop"),
{Name: "close", Description: "Healer H2's verb: take the one transition the table names for a delivery held " +
"past its bound. Refused for any other.",
Input: schema(map[string]string{"id": "the delivery's id", "why": "the condition it answers"},
[]string{"id", "why"})},
deliveryRemovedInR6("retire-history"),
{Name: "settle", Description: "Ask the operator to settle deliveries held because their merge reached the " +
"trunk without a passing check: one ask on the operator's channel for all of them, as an Approve is asked. " +
"The verb moves nothing; on the operator's answer each goes to merged, and whether it is on the machines " +
"is derived as for any merge (novox/hq ADR 0556). Refused for a delivery not held, or held for another " +
"reason, naming why.",
Input: map[string]any{"type": "object", "additionalProperties": false, "required": []string{"ids", "why"},
"properties": map[string]any{
"ids": map[string]any{"type": "array", "items": map[string]any{"type": "string"}, "minItems": 1,
"description": "the held deliveries' ids (owner/repository@commit)"},
"why": map[string]any{"type": "string", "description": "why they may be settled, shown to the operator"},
}},
// Added after the seat's release 6 holder shipped: optional until the gate serves it everywhere.
Optional: true},
}
}
@@ -1,30 +0,0 @@
package resolution
import (
"testing"
"git.novox.be/novox/mesh-delivery/contracts/go/clients/meshdeliveryclient"
)
// Each verb release 6 removed from the delivery seat is a row refusing it by name with the replacement the seat's client
// gives (the list is the contract's, not the SDK's: novox/hq ADR 0634 §2), and the rows are exactly those verbs.
func TestTheDeliverySeatRefusesTheRemovedVerbsByNameWithTheClientsReplacement(t *testing.T) {
removed := meshdeliveryclient.RemovedVerbs()
rows := 0
for _, v := range deliveryVerbs() {
want, isRemoved := removed[v.Name]
if !isRemoved {
continue
}
rows++
if v.Description != meshdeliveryclient.RemovedPrefix+want || !v.Optional {
t.Errorf("%s = %q (optional %v), want %q", v.Name, v.Description, v.Optional, meshdeliveryclient.RemovedPrefix+want)
}
}
if rows != len(removed) {
t.Errorf("%d rows refuse a removed verb, the client names %d", rows, len(removed))
}
if len(removed) != 5 {
t.Errorf("the removed verbs are %v", removed)
}
}
@@ -1,67 +0,0 @@
package resolution
import (
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// enrolmentDecision is the record that made the seat.
const enrolmentDecision = "novox/hq ADR 0587"
// enrolmentVerbs are the seat's verbs: what reads all it holds and does by hand all it does (ADR 0587 §8). Not optional:
// the seat is new with its holder, so no holder of before serves fewer.
func enrolmentVerbs() []mcat.Verb {
str := func(description string) map[string]any {
return map[string]any{"type": "string", "description": description}
}
return []mcat.Verb{
{Name: mcat.EnrolmentVerbIssue, Description: "Issue a one-time token for a new machine, on the operator's Approve: " +
"refused at once, naming every finding, for a name that is not a node name or that a node record, an assignments " +
"key or an open enrolment holds, and for every call before the controller's lease is handed away. Otherwise " +
"reserves the lowest free private-network address, asks the operator under the seat's own name and, on the " +
"warrant, adds the node to the assignments with its tunnel key, asks the vault for the token and says enrolment-issued. " +
"The token itself is read only through tokens.",
Input: schema(map[string]string{
"node": "the new machine's name in the mesh",
"why": "why it joins, shown to the operator with the ask",
"expires": "how long the token lasts, a duration such as 1h or 24h: one hour when not given, at most 24 hours",
// The operator's decision of 2026-11-02 01:05: the tunnel key is given at issue, so on the Approve the node
// enters the assignments with it and the hub peers it before redeem.
"overlay_key": "the new machine's private network (tunnel) public key, as `nox-mesh-host key` prints it",
}, []string{"node", "why", "overlay_key"})},
{Name: mcat.EnrolmentVerbRedeem, Description: "Spend a token: asked by a joining node-engine dialled as enrol.<node> " +
"alone, with the public halves of its keys and its identity key's proof over them. Spends the token, asks the " +
"vault for the node's account and certificate, writes the node into the assignments, and answers the account sealed " +
"to the node, its certificate and the vault's public half, signed by the vault. Asked again with the same keys " +
"it finishes what is not done and answers the same; with other keys it is refused.",
Input: map[string]any{"type": "object", "additionalProperties": false,
"required": []string{"node", "public_key", "sealing_key", "proof"},
"properties": map[string]any{
"node": str("the node the token was issued for"),
"public_key": str("the identity key's public half, base64"),
"overlay_key": str("the private network key's public half"),
"sealing_key": str("the sealing key's public half, which the account is sealed to"),
"serving_key": str("the serving key's public half, which the certificate binds"),
"profile": map[string]any{"type": "object", "description": "what the machine is, as its record says it"},
"tunnel": map[string]any{"type": "object", "description": "a tunnel found on the machine: everything but its private key"},
"proof": str("the identity key's signature over the redeem's proof bytes (delivery.RedeemProof), base64"),
}}},
{Name: mcat.EnrolmentVerbConfirm, Description: "Confirm an enrolment: asked by the node-engine dialled as its own " +
"node login alone, the runtime's proof of that caller being the confirmation. Says node-enrolled and ends the " +
"enrolment.",
Input: schema(map[string]string{}, nil)},
{Name: mcat.EnrolmentVerbRevoke, Description: "End an enrolment at any stage, with why, without an Approve: withdraws " +
"an open ask, deletes the node's key and its address, has the vault retire an account it already minted for " +
"it, records the act as a hand act and says enrolment-revoked.",
Input: schema(map[string]string{"node": "the node", "why": "why, recorded with the hand act"},
[]string{"node", "why"})},
{Name: mcat.EnrolmentVerbTokens, Description: "Read every open enrolment, or one: its stage, expiry, ask and revision, " +
"the bucket's fill against its cap. For a node at live, its " +
"token sealed to the operator key, asked of the vault at that moment and relayed as the vault sealed it. With " +
"history, the ended enrolments from the seat's events, as far back as the events are kept.",
Input: schema(map[string]string{"node": "one node", "history": "\"true\": the ended enrolments too"},
nil, "history")},
{Name: mcat.EnrolmentVerbAgain, Description: "Raise an issue's ask again when the operator's channel does not hold it, " +
"and nothing more.",
Input: schema(map[string]string{"node": "the node whose ask is raised again"}, []string{"node"})},
}
}
@@ -60,6 +60,59 @@ func TestTheModuleAskingForTheRuleSetGetsEveryModulesPorts(t *testing.T) {
if !strings.Contains(content, "dport 443 accept") {
t.Fatalf("the rule set holds only the asking module's ports:\n%s", content)
}
// Issue 1966, novox 2026-11-09 07:46-09:24 UTC: a module left out of the declaration for a setting its new definition
// no longer composes (ADR 0163 rule 6) keeps running what it ran, so the filter keeps letting it through. The
// artifact store's build 6e1dfaa8 listens on 5100 from a process and publishes no container port 5000, while its
// node layer still gave 5000 the machine port 5100: left out, and the filter closed 5100 for 98 minutes while the old
// registry container still served there; every build failed "dial tcp 10.10.0.1:5100: i/o timeout".
r.Modules = append(r.Modules, mcat.Manifest{Module: "distribution",
Listens: []mcat.Listening{{Name: "registry", Port: 5100, Protocol: "tcp", From: mcat.FromEverywhere}},
Resources: []map[string]any{{"id": "store", "type": "container", "name": "mesh-registry",
"image": "registry@sha256:" + strings.Repeat("a", 64), "ports": []any{"127.0.0.1:5001:5001"}}}})
with := mcat.Rendering{Mesh: []string{"198.51.100.2"}, OutwardLinks: []string{"eth0"}, TunnelInterface: "mesh0",
Settings: mcat.SettingsBy{"distribution": {{From: "novox", Placement: map[string]any{mcat.PortsSetting: map[string]any{"5000": 5100}}}}}}
if left := LeftOut(r, with.Settings, false); left["distribution"] == "" {
t.Fatalf("the stale port setting does not leave distribution out: %v", left)
}
if out, err = Declaration(r, with); err != nil {
t.Fatalf("declaration with a module left out: %v", err)
}
content = ""
for _, res := range out {
if res["path"] == "/etc/mesh/filter.nft" {
content, _ = res["content"].(string)
}
}
if !strings.Contains(content, "dport 5100 accept") || !strings.Contains(content, "dport 443 accept") {
t.Fatalf("the filter closes the port of a module left out of the declaration, which still runs:\n%s", content)
}
// Its per-node narrowing still holds (ADR 0046), and an exposure that cannot be read for it opens it no wider than
// the mesh, never at the manifest's default (review round 1).
for _, row := range []struct {
expose map[string]any
from string
}{
{map[string]any{"5100": mcat.FromMachine}, mcat.FromMachine},
{map[string]any{"9999": mcat.FromMachine}, mcat.FromMesh}, // a port it does not listen on: refused
} {
layer := with.Settings["distribution"][0]
layer.Placement = map[string]any{mcat.PortsSetting: map[string]any{"5000": 5100}, mcat.ExposeSetting: row.expose}
with.Settings = mcat.SettingsBy{"distribution": {layer}}
rules, err := Rules(r, with)
if err != nil {
t.Fatalf("expose %v: %v", row.expose, err)
}
var from []string
for _, rule := range rules {
if rule.Port == 5100 {
from = append(from, rule.From)
}
}
if len(from) != 1 || from[0] != row.from {
t.Errorf("expose %v: the left-out module's 5100 opens from %v, want %s", row.expose, from, row.from)
}
}
}
// The host does not sort, so the order the mesh writes is the order a machine applies.
@@ -37,6 +37,11 @@ func TestWithNoHolderOnRecordTheSoleClaimantHoldsAndTwoAreRefused(t *testing.T)
if len(problems) != 1 || !strings.Contains(problems[0], "both claim") {
t.Fatalf("two claimants with nothing on record were not refused: %v", problems)
}
// Held for the module that cannot hold, which then holds nothing; the other still does (issue 1966).
held, holding := holdClaims([]mcat.Manifest{oldBroker(), newBroker()}, node, nil, nil)
if len(holding) != 1 || !strings.Contains(holding["new-broker"], "both claim") || len(held) != 1 || held[0].Module != "old-broker" {
t.Fatalf("the second claimant is not the one held: held=%v holding=%v", held, holding)
}
}
// With a holder on record, the other eligible assignment is silent: not refused, and not holding.
+80
View File
@@ -0,0 +1,80 @@
package resolution
import (
"fmt"
"strings"
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// withoutUnconfigured drops each resource whose `if-config` holds none of its conditions against the module's config
// values (design 42 zfs-owns-zfs §2, ADR 0679 §16 (b)), and the key itself from every resource that keeps it: the
// node-engine parses strictly and knows no such key. What leaves the declaration this way is removed as any resource
// that leaves it (ADR 0269 §2), by the node-engine's own orphan removal.
func withoutUnconfigured(resources []map[string]any, layers []mcat.Layer) []map[string]any {
out := make([]map[string]any, 0, len(resources))
for _, r := range resources {
conditions, has := r[mcat.IfConfigField]
if !has {
out = append(out, r)
continue
}
if !ifConfigHolds(conditions, layers) {
continue
}
kept := make(map[string]any, len(r))
for k, v := range r {
if k != mcat.IfConfigField {
kept[k] = v
}
}
out = append(out, kept)
}
return out
}
// ifConfigHolds is whether any condition of a list holds: `<key>` when the config value is set and is neither false,
// off, none, empty nor an empty list; `<key>=<value>` when the value equals it, or a list value holds it as an entry.
func ifConfigHolds(conditions any, layers []mcat.Layer) bool {
list, _ := conditions.([]any)
for _, c := range list {
key, want, equals := strings.Cut(fmt.Sprint(c), "=")
value, set := settingValue(layers, key)
if !set {
continue
}
if !equals {
if !isOff(value) {
return true
}
continue
}
if entries, isList := value.([]any); isList {
for _, e := range entries {
if plainly(e) == want {
return true
}
}
} else if plainly(value) == want {
return true
}
}
return false
}
// isOff is a config value that says "not here": false, off, none, an empty text or an empty list.
func isOff(value any) bool {
switch v := value.(type) {
case bool:
return !v
case []any:
return len(v) == 0
case nil:
return true
}
switch plainly(value) {
case "", "false", "off", "none":
return true
}
return false
}
@@ -6,36 +6,6 @@ import (
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// resolverVerbs is the contract every holder of node-resolver serves (novox/hq ADR 0247): what is routed
// where, route a set of domains, and take a route away. The same verbs are served on the machine itself to
// the modules there, so what a VPN pushed never crosses the bus to be routed; on the mesh they are the
// operator's way to read and correct the same.
func resolverVerbs() []mcat.Verb {
return []mcat.Verb{
{Name: "routes", Description: "What this machine's own resolver sends where: the mesh's resolvers, " +
"which answer every name not routed elsewhere, and each link given servers of its own with the " +
"domains routed to them. Also the resolver file's outside writes it kept, newest first: when, who " +
"wrote it as far as the file says, whether a module took it, and when the resolver's own file was " +
"put back.",
Input: schema(map[string]string{}, nil),
Replaces: []string{"resolvectl status", "resolvectl dns", "resolvectl domain"}},
{Name: "route", Description: "Send these domains, and every name under them, to these servers over " +
"this link — and only them: the link is never the machine's default route for names, and the " +
"mesh's own domain is refused. Replaces whatever the link was given before. A link that goes away " +
"takes its route with it.",
Input: schema(map[string]string{
"link": "the network link the servers are reached over, by name (a VPN's tunnel interface)",
"domains": "the domains to route there, separated by spaces or commas",
"servers": "the servers' addresses, separated by spaces or commas",
}, []string{"link", "domains", "servers"}),
Replaces: []string{"resolvectl dns", "resolvectl domain", "resolvectl default-route"}},
{Name: "unroute", Description: "Take one link's route away: its domains go back to the mesh's " +
"resolvers. Nothing changes when the link has none.",
Input: schema(map[string]string{"link": "the network link, by name"}, []string{"link"}),
Replaces: []string{"resolvectl revert"}},
}
}
// holdsSeat says whether a module claims a seat, by its current name.
func holdsSeat(m mcat.Manifest, seat string) bool {
for _, c := range m.Claims {
@@ -151,6 +151,16 @@ func TestTheCataloguesResolverHoldersWriteTheFileAndProvideSplitDNS(t *testing.T
func TestTheCataloguesResolverComposesOnAMachine(t *testing.T) {
shelf := resolverShelf(t)
shelf["systemd-resolved"] = catalogueManifest(t, "systemd-resolved")
// The options line is the catalogue's own template's, never a literal of it here (design 50 resolver-wait §3).
options := ""
for _, l := range strings.Split(shelf["systemd-resolved"].Facts["resolvers"].Template, "\n") {
if strings.HasPrefix(l, "options ") {
options = l
}
}
if options == "" {
t.Fatal("the catalogue's resolver template has no options line")
}
got, err := Resolve(shelf, []string{"dnsmasq", "systemd-networkd", "systemd-resolved"}, splitDNSLaptop(), mcat.World{})
if err != nil {
t.Fatal(err)
@@ -166,7 +176,7 @@ func TestTheCataloguesResolverComposesOnAMachine(t *testing.T) {
ids := byID(out)
resolv, _ := ids["systemd-resolved.fact-resolvers"]["content"].(string)
kept, _ := ids["systemd-resolved.fact-kept"]["content"].(string)
if !strings.Contains(resolv, "\nnameserver 10.42.0.2\noptions timeout:1 attempts:2 edns0\n") || resolv != kept {
if !strings.Contains(resolv, "\nnameserver 10.42.0.2\n"+options+"\n") || resolv != kept {
t.Errorf("the resolver file does not name this machine's own resolver alone, or its copy differs:\n%s", resolv)
}
if ids["systemd-networkd.fact-resolvers"] != nil {
+11 -2
View File
@@ -7,6 +7,7 @@ import (
"sort"
"strings"
"git.novox.be/novox/mesh-contracts/go/seats/nodeengine"
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
@@ -56,8 +57,16 @@ func systemPath(path string) string {
return ""
}
// engineModule is the node-engine's own module, the one that places in engineTrees.
const engineModule = "mesh-host"
// ClaimsEngine is whether a manifest claims the seat node-engine: the node-engine's own module is the one whose entry of
// the catalog half does (design 47 G §527), never a name fixed in code.
func ClaimsEngine(m mcat.Manifest) bool {
for _, c := range m.Claims {
if c.Name == nodeengine.Seat {
return true
}
}
return false
}
// accessRef is how a module names one of its accesses: ${access:<id>}.
var accessRef = regexp.MustCompile(`\$\{access:([a-z0-9][a-z0-9-]*)\}`)
+63 -24
View File
@@ -26,12 +26,17 @@ func (r *Refusal) Error() string {
return "these assignments cannot be applied:\n - " + strings.Join(r.Problems, "\n - ")
}
// Resolve works out everything a node runs, from what was assigned to it.
//
// The catalogue is every module the mesh knows about; assigned is what a person put on this node.
// What comes back is the closure — assigned modules plus everything they require — or a refusal
// naming every reason it could not be closed.
func Resolve(catalogue map[string]mcat.Manifest, assigned []string, node mcat.Node, world mcat.World) (mcat.Resolution, error) {
// ResolveHolding works out everything a node runs, from what was assigned to it. The catalogue is every module the mesh knows
// about; assigned is what a person put on this node. What comes back is the closure -- assigned modules plus everything they
// require -- or a refusal naming every reason it could not be closed. A claim problem costs the module that has it, not the
// machine (issue 1966, ADR 0636 §3: a refused claim blocks that claimant only). The modules that cannot hold what they claim come back with why, each still in
// the resolution's modules and none of them in its claims; the caller leaves them out of what it declares (LeftOut), so the
// machine composes everything else. Every other refusal is Resolve's.
func ResolveHolding(catalogue map[string]mcat.Manifest, assigned []string, node mcat.Node, world mcat.World) (mcat.Resolution, map[string]string, error) {
return resolve(catalogue, assigned, node, world)
}
func resolve(catalogue map[string]mcat.Manifest, assigned []string, node mcat.Node, world mcat.World) (mcat.Resolution, map[string]string, error) {
elsewhere := world.Held
var problems []string
@@ -593,8 +598,7 @@ func Resolve(catalogue map[string]mcat.Manifest, assigned []string, node mcat.No
}
problems = append(problems, checkCapabilities(resolution.Modules, node)...)
claims, claimProblems := checkClaims(resolution.Modules, node, elsewhere, world.Holdings)
problems = append(problems, claimProblems...)
claims, held := holdClaims(resolution.Modules, node, elsewhere, world.Holdings)
problems = append(problems, checkResources(resolution.Modules)...)
resolution.Claims = claims
@@ -615,10 +619,14 @@ func Resolve(catalogue map[string]mcat.Manifest, assigned []string, node mcat.No
}
if len(problems) > 0 {
// A pass refused anyway says every reason at once, a held module's among them.
for _, why := range held {
problems = append(problems, why)
}
sort.Strings(problems)
return mcat.Resolution{}, &Refusal{Problems: problems}
return mcat.Resolution{}, nil, &Refusal{Problems: problems}
}
return resolution, nil
return resolution, held, nil
}
// isModule reports whether a name is a module in its own right rather than only something
@@ -672,12 +680,29 @@ func checkCapabilities(modules []mcat.Manifest, node mcat.Node) []string {
return problems
}
// checkClaims refuses two modules holding one singular thing.
//
// Within this node's own set, and against what is already held elsewhere for the wider scopes. A
// claim at mesh scope is the same idea as the mesh's one hub, said once instead of hard-coded.
func checkClaims(modules []mcat.Manifest, node mcat.Node, elsewhere []mcat.Held, holdings []mcat.Held) ([]mcat.Held, []string) {
var problems []string
// holdClaims refuses two modules holding one singular thing, within this node's own set and against what is held elsewhere for
// the wider scopes: the claims that hold, and for each module that cannot hold what it claims, why (every reason, joined).
func holdClaims(modules []mcat.Manifest, node mcat.Node, elsewhere []mcat.Held, holdings []mcat.Held) ([]mcat.Held, map[string]string) {
held, problems := claimsOf(modules, node, elsewhere, holdings)
var out map[string]string
for _, p := range problems {
if out == nil {
out = map[string]string{}
}
if prior, ok := out[p.module]; ok {
out[p.module] = prior + "; " + p.why
} else {
out[p.module] = p.why
}
}
return held, out
}
// claimProblem is a refusal of one module's claim.
type claimProblem struct{ module, why string }
func claimsOf(modules []mcat.Manifest, node mcat.Node, elsewhere []mcat.Held, holdings []mcat.Held) ([]mcat.Held, []claimProblem) {
var problems []claimProblem
var held []mcat.Held
// onRecord is every recorded holder of a seat, if a handover ever named one: one for most seats,
@@ -717,9 +742,9 @@ func checkClaims(modules []mcat.Manifest, node mcat.Node, elsewhere []mcat.Held,
// disagrees with the mesh's definition of the role, and it is refused, named, rather
// than letting two machines answer for what the mesh has one of.
if s, known := mcat.SeatNamed(c.Name); known && !s.Replicated && len(recs) > 1 {
problems = append(problems, fmt.Sprintf(
problems = append(problems, claimProblem{m.Module, fmt.Sprintf(
"%q is on record as held by %d assignments, and it is held once per %s — "+
"`seat %s --to <node>/<module>` records one", c.Name, len(recs), scope, c.Name))
"`seat %s --to <node>/<module>` records one", c.Name, len(recs), scope, c.Name)})
continue
}
if !recordedHere(c.Name, scope, m.Module) {
@@ -734,9 +759,9 @@ func checkClaims(modules []mcat.Manifest, node mcat.Node, elsewhere []mcat.Held,
// one seat, compared by the seat they resolve to and not by how each spelled it.
seat := heldKey(c.Name, c.Kind)
if other, taken := byScope[scope][seat]; taken {
problems = append(problems, fmt.Sprintf(
problems = append(problems, claimProblem{m.Module, fmt.Sprintf(
"%s and %s both claim %q, and only one thing may hold it per %s",
other, m.Module, seat, scope))
other, m.Module, seat, scope)})
continue
}
byScope[scope][seat] = m.Module
@@ -762,20 +787,34 @@ func checkClaims(modules []mcat.Manifest, node mcat.Node, elsewhere []mcat.Held,
// Both claim and nobody is on record, or this refusal could not have happened.
// The remedy is the handover that records the holder (novox/hq ADR 0131,
// 04-ISSUES/170), so it is named here rather than left to be found.
problems = append(problems, fmt.Sprintf(
problems = append(problems, claimProblem{h.Module, fmt.Sprintf(
"%s on %s claims %q, which %s on %s already holds — one per mesh. Nothing is "+
"on record for it; `seat %s --to %s/%s` records the holder, and the other "+
"assignment then stands beside it, eligible and silent",
h.Module, node.Name, h.Claim, e.Module, e.Node, h.Claim, e.Node, e.Module))
h.Module, node.Name, h.Claim, e.Module, e.Node, h.Claim, e.Node, e.Module)})
case mcat.ScopeSite:
if node.Site != "" && node.Site == e.Site {
problems = append(problems, fmt.Sprintf(
problems = append(problems, claimProblem{h.Module, fmt.Sprintf(
"%s on %s claims %q, which %s on %s already holds at %s — one per site",
h.Module, node.Name, h.Claim, e.Module, e.Node, node.Site))
h.Module, node.Name, h.Claim, e.Module, e.Node, node.Site)})
}
}
}
}
// A module refused here holds nothing of what it claims: its claims are not this node's holdings.
refused := map[string]bool{}
for _, p := range problems {
refused[p.module] = true
}
if len(refused) > 0 {
kept := held[:0]
for _, h := range held {
if !refused[h.Module] {
kept = append(kept, h)
}
}
held = kept
}
return held, problems
}
+13 -4
View File
@@ -310,16 +310,25 @@ func TestAModuleOfItsOwnAccountIsServedByARuntimeOfItsOwn(t *testing.T) {
}
gob, img := builder("gob", "node-build-go"), builder("img", "node-build-image")
with.Needed["gob"], with.Needed["img"] = map[string]string{"broker": "own"}, map[string]string{"broker": "own"}
// The node-engine's module is the one whose entry claims its seat (design 47 G §527), whatever it is called; with none in
// the resolution its settings are not found and the default stands.
engine := mcat.Manifest{Module: "mesh-host", Claims: []mcat.Claim{{Name: "node-engine"}}}
for _, c := range []struct {
settings mcat.SettingsBy
share string
noEngine bool
}{
{nil, "50"},
{mcat.SettingsBy{"mesh-host": {{From: "anchor", Values: map[string]any{"build-cpu-share": float64(75)}}}}, "75"},
{mcat.SettingsBy{"mesh-host": {{From: "anchor", Values: map[string]any{"build-cpu-share": float64(0)}}}}, "50"},
{nil, "50", false},
{mcat.SettingsBy{"mesh-host": {{From: "anchor", Values: map[string]any{"build-cpu-share": float64(75)}}}}, "75", false},
{mcat.SettingsBy{"mesh-host": {{From: "anchor", Values: map[string]any{"build-cpu-share": float64(0)}}}}, "50", false},
{mcat.SettingsBy{"mesh-host": {{From: "anchor", Values: map[string]any{"build-cpu-share": float64(75)}}}}, "50", true},
} {
with.Settings = c.settings
runtimes, err := resolutionOwnRuntimes(mcat.Resolution{Node: "anchor", Modules: []mcat.Manifest{gob, img, telegram, goRuntime}}, with)
modules := []mcat.Manifest{gob, img, telegram, goRuntime, engine}
if c.noEngine {
modules = modules[:4]
}
runtimes, err := resolutionOwnRuntimes(mcat.Resolution{Node: "anchor", Modules: modules}, with)
if err != nil {
t.Fatal(err)
}
@@ -29,6 +29,10 @@ var appliedThrough = map[string]string{
// A variable rather than a constant only so a test can hold both behaviours; nothing else sets it.
var enforceSeatDependencies = true
// nodeEngineSeat is the seat the node-engine's own module claims (design 47 G §508, §527, §634): the
// foundation's host is the entry that claims it, never a module named here.
const nodeEngineSeat = "node-engine"
// foundationModules are the pieces genesis lays before any module exists (novox/hq ADR 0207 §5):
// the host and the private network. Registered as modules so they can be assigned, but what they
// declare is the installation's, not a module's, so it is never judged. The third piece, the
@@ -38,7 +42,6 @@ var enforceSeatDependencies = true
// The private network's module is overlay.Name, written out because the overlay package composes
// on top of this one; its resources are computed, which exempts it by the rule below as well.
var foundationModules = map[string]bool{
"mesh-host": true,
"mesh-wireguard": true,
}
@@ -46,7 +49,7 @@ var foundationModules = map[string]bool{
// module whose resources are computed is the mesh's by construction — the private network's peer
// list and its tools are the controller's, written per node — so it counts whatever its name.
func isFoundation(m mcat.Manifest) bool {
return foundationModules[m.Module] || m.Computed != ""
return foundationModules[m.Module] || m.ClaimsSeat(nodeEngineSeat) || m.Computed != ""
}
// DependsOn is every seat a module needs held on its node: derived from the resource types it
@@ -95,3 +95,17 @@ func TestWithTheSwitchFlippedAnUnmetDependencyRefusesTheNode(t *testing.T) {
t.Errorf("the first pass refused an unmet dependency: %v", err)
}
}
func TestTheFoundationHostIsTheEntryThatClaimsTheNodeEngineSeat(t *testing.T) {
svc := res("service", "x")
// Whatever its name, the entry claiming the seat is the foundation's: nothing to depend on.
engine := withResources(mod("the-engine", nil, nil, nil, mcat.Claim{Name: "node-engine"}), svc)
if got := DependsOn(nil, engine); len(got) != 0 {
t.Errorf("the entry claiming node-engine depends on %v, want nothing (§527)", got)
}
// A module named as the old code named the engine, claiming no such seat, is an ordinary module.
named := withResources(mod("mesh-host", nil, nil, nil), svc)
if got := DependsOn(nil, named); !reflect.DeepEqual(got, []string{mcat.ServiceManagerSeat}) {
t.Errorf("a module merely named mesh-host depends on %v, want the service manager seat", got)
}
}
-74
View File
@@ -1,74 +0,0 @@
package resolution
import (
"fmt"
"sort"
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// SeatsToPassOn reads the mesh for seats held once whose holder on record has left (novox/hq ADR 0449 §4).
//
// **A holder on record that is assigned on no machine holds nothing, and a seat that answers nothing is
// the day a delivery removed its module.** Of such a seat, exactly one assignment in the mesh able to hold
// it (it claims the seat at its scope and CanHold accepts it) takes it over: that is a pass, and the caller
// records it and writes the hand act. With no such assignment the record stays; with two or more nothing is
// recorded and the answer says so, naming both and the `seat --to` that settles it. A holder still assigned
// anywhere changes nothing: the other claimants stand beside it, eligible and silent (ADR 0131). Only mesh
// seats held once are read: a replicated seat's holders are added by an act (ADR 0223), a kinded bench is held
// by its claims, and a seat with several holders on record is a store that disagrees with the mesh (refused
// by the resolver).
func SeatsToPassOn(holdings []mcat.Held, assignments []mcat.Assignment) ([]mcat.SeatPass, []string) {
assigned := map[string]bool{}
for _, a := range assignments {
assigned[a.Manifest.Module] = true
}
bySeat := map[string][]mcat.Held{}
var order []string
for _, h := range holdings {
s, ok := mcat.SeatNamed(h.Claim)
if !ok || s.Replicated || s.Scope != mcat.ScopeMesh || h.Scope != mcat.ScopeMesh || mcat.KindedBenches[s.Name] {
continue
}
if _, seen := bySeat[s.Name]; !seen {
order = append(order, s.Name)
}
bySeat[s.Name] = append(bySeat[s.Name], h)
}
sort.Strings(order)
var passes []mcat.SeatPass
var said []string
for _, name := range order {
recs := bySeat[name]
if len(recs) != 1 || assigned[recs[0].Module] {
continue
}
seat, _ := mcat.SeatNamed(name)
var claimants []mcat.Held
for _, a := range assignments {
if mcat.CanHold(a.Manifest, seat) != nil {
continue
}
claimants = append(claimants, mcat.Held{Claim: seat.Name, Scope: seat.Scope, Node: a.Node, Module: a.Manifest.Module})
}
sort.Slice(claimants, func(i, j int) bool {
if claimants[i].Node != claimants[j].Node {
return claimants[i].Node < claimants[j].Node
}
return claimants[i].Module < claimants[j].Module
})
switch len(claimants) {
case 0:
case 1:
passes = append(passes, mcat.SeatPass{Seat: seat, From: recs[0], To: claimants[0]})
default:
said = append(said, fmt.Sprintf(
"%s on %s is on record as the holder of %q and is assigned nowhere, and %s on %s and %s on %s both claim it "+
"— nothing is recorded; `seat %s --to <node>/<module>` records the one that holds it",
recs[0].Module, recs[0].Node, seat.Name, claimants[0].Module, claimants[0].Node,
claimants[1].Module, claimants[1].Node, seat.Name))
}
}
return passes, said
}
@@ -1,80 +0,0 @@
package resolution
import (
"strings"
"testing"
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
func onRecord(node, module string) []mcat.Held {
return []mcat.Held{{Claim: "mesh-broker", Scope: mcat.ScopeMesh, Node: node, Module: module}}
}
// 1317/AC1
func TestASeatWhoseHolderIsAssignedNowhereIsPassedToItsOnlyClaimant(t *testing.T) {
busSeatDelivering(t, "mesh-bus")
passes, said := SeatsToPassOn(onRecord("novox", "old-broker"),
[]mcat.Assignment{{Node: "novox", Manifest: newBroker()}})
if len(said) != 0 || len(passes) != 1 {
t.Fatalf("the only claimant was not given the seat: passes=%v said=%v", passes, said)
}
p := passes[0]
if p.Seat.Name != "mesh-broker" || p.From.Module != "old-broker" || p.From.Node != "novox" ||
p.To.Module != "new-broker" || p.To.Node != "novox" {
t.Fatalf("the pass does not name the seat, the former holder and the new one: %+v", p)
}
}
// 1317/AC2
func TestWithTwoClaimantsAndTheHolderGoneNothingIsPassedAndTheRefusalNamesBoth(t *testing.T) {
busSeatDelivering(t, "mesh-bus")
passes, said := SeatsToPassOn(onRecord("novox", "gone"), []mcat.Assignment{
{Node: "novox", Manifest: newBroker()}, {Node: "ace", Manifest: oldBroker()}})
if len(passes) != 0 || len(said) != 1 {
t.Fatalf("two claimants: passes=%v said=%v", passes, said)
}
for _, want := range []string{"old-broker", "new-broker", "seat mesh-broker --to"} {
if !strings.Contains(said[0], want) {
t.Fatalf("the refusal does not say %q: %s", want, said[0])
}
}
}
// 1317/AC3
func TestAHolderStillAssignedSomewhereChangesNothing(t *testing.T) {
busSeatDelivering(t, "mesh-bus")
// Assigned on another machine than the record names: still assigned somewhere.
passes, said := SeatsToPassOn(onRecord("novox", "old-broker"), []mcat.Assignment{
{Node: "ace", Manifest: oldBroker()}, {Node: "novox", Manifest: newBroker()}})
if len(passes) != 0 || len(said) != 0 {
t.Fatalf("a holder still assigned lost its seat: passes=%v said=%v", passes, said)
}
}
// 1317/AC4
func TestWithNoClaimantTheRecordStays(t *testing.T) {
busSeatDelivering(t, "mesh-bus")
passes, said := SeatsToPassOn(onRecord("novox", "old-broker"), []mcat.Assignment{
{Node: "novox", Manifest: mcat.Manifest{Module: "other"}}})
if len(passes) != 0 || len(said) != 0 {
t.Fatalf("with no claimant: passes=%v said=%v", passes, said)
}
// A claimant that cannot hold the seat (it does not provide what the seat delivers) is no claimant.
bad := mcat.Manifest{Module: "no-bus", Claims: []mcat.Claim{{Name: "mesh-broker", Scope: mcat.ScopeMesh}}}
passes, said = SeatsToPassOn(onRecord("novox", "old-broker"), []mcat.Assignment{{Node: "novox", Manifest: bad}})
if len(passes) != 0 || len(said) != 0 {
t.Fatalf("a module that cannot hold the seat was passed it: passes=%v said=%v", passes, said)
}
}
// A replicated seat is never passed: its holders are added by an act (ADR 0223).
func TestAReplicatedSeatIsNeverPassed(t *testing.T) {
record := []mcat.Held{{Claim: "mesh-dns-resolver", Scope: mcat.ScopeMesh, Node: "novox", Module: "gone"}}
resolver := mcat.Manifest{Module: "resolver", Provides: []mcat.Offer{{Name: "wildcard-resolution", Scope: mcat.ScopeMesh}},
Claims: []mcat.Claim{{Name: "mesh-dns-resolver", Scope: mcat.ScopeMesh}}}
passes, _ := SeatsToPassOn(record, []mcat.Assignment{{Node: "novox", Manifest: resolver}})
if len(passes) != 0 {
t.Fatalf("a replicated seat was passed: %v", passes)
}
}
+16 -1
View File
@@ -26,7 +26,7 @@ func declaredPaths(m mcat.Manifest) map[string]bool {
}
for _, r := range m.Resources {
if p, ok := r["path"].(string); ok && strings.HasPrefix(p, "/") {
out[path.Clean(p)] = true
out[path.Clean(versionedPlaceOwner(r, p))] = true
}
}
for _, f := range m.Facts {
@@ -123,3 +123,18 @@ func seatPathsOnMachine(modules []mcat.Manifest) []string {
}
return problems
}
// versionedPlaceOwner is the path a resource owns: its own, unless the resource is an archive unpacked into a place named
// for its build (the artifact's digest, short: `.../versions/<version>`, catalogue `${version}`), whose owner is the
// directory holding every such place, because the place changes with each build and the owned paths are an input of the
// tool runner's unit (the holder of node-files reads them at its start): a build of the engine itself renamed its place
// and restarted the tool runner, with nothing of the tool runner changed (issue 1966).
func versionedPlaceOwner(r map[string]any, p string) string {
digest, _ := r["digest"].(string)
version := strings.TrimPrefix(digest, "sha256:")
base := path.Base(path.Clean(p))
if r["type"] == "archive" && len(base) >= 8 && strings.HasPrefix(version, base) {
return path.Dir(path.Clean(p))
}
return p
}
@@ -28,6 +28,26 @@ func TestAGeneratedResourceOnASeatsPathIsRefused(t *testing.T) {
t.Errorf("the refusal does not name %s: %v", want, err)
}
}
// And the paths a module owns do not change with its build (issue 1966): the holder of node-files is told them in a
// word of the tool runner's unit, so a place named for each build restarted the tool runner with each build.
digest := "sha256:" + strings.Repeat("ab12", 16)
for _, row := range []struct {
name string
res map[string]any
owns string
}{
{"a build's own place", map[string]any{"id": "next", "type": "archive", "digest": digest, "path": "/usr/lib/x/versions/ab12ab12ab12"}, "/usr/lib/x/versions"},
{"another build's place", map[string]any{"id": "next", "type": "archive", "digest": "sha256:" + strings.Repeat("cd34", 16), "path": "/usr/lib/x/versions/cd34cd34cd34"}, "/usr/lib/x/versions"},
{"a fixed place", map[string]any{"id": "app", "type": "archive", "digest": digest, "path": "/opt/app"}, "/opt/app"},
{"a file", map[string]any{"id": "f", "type": "file", "path": "/etc/f"}, "/etc/f"},
} {
t.Run(row.name, func(t *testing.T) {
got := DeclaredPaths(mcat.Manifest{Module: "m", Resources: []map[string]any{row.res}})
if len(got) != 1 || got[0] != row.owns {
t.Fatalf("owns %v, want [%s]", got, row.owns)
}
})
}
}
// The stepping back comes from the seat, and is of a rendered file only: an uplink holder declaring the
-15
View File
@@ -4,21 +4,6 @@ import (
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// configInput is the schema of mesh-config's verb: a value of the key's own type (a number for a number, never its
// text), and a limit that is a number; every other argument a string.
func configInput() map[string]any {
in := schema(map[string]string{"action": "list (when not given), set, clear or history",
"key": "the config key", "value": "for set: the value, of the key's type",
"module": "a module: its own value, with node", "node": "a node",
"why": "for set and clear: why", "by": "who gave the value at the controller's terminal",
"limit": "for history: at most this many, newest"}, nil)
props := in["properties"].(map[string]any)
props["value"] = map[string]any{"description": "for set: the value, of the key's type: a number, a string, " +
"a boolean, or for a trusted key an object or a list"}
props["limit"] = map[string]any{"type": "number", "description": "for history: at most this many, newest"}
return in
}
func providesAt(m mcat.Manifest, provision, scope string) bool {
for _, o := range m.Provides {
if o.Name == provision && o.At() == scope {
+94
View File
@@ -1,6 +1,12 @@
package resolution
import (
"go/ast"
"go/parser"
"go/token"
"io/fs"
"path/filepath"
"strconv"
"strings"
"testing"
@@ -70,4 +76,92 @@ func TestTheHolderIsTheModuleNotTheMachine(t *testing.T) {
t.Fatalf("the holder was not told apart from a neighbour: %+v", holder)
}
// A seat's verbs are its definition's, read from the bus (novox/hq ADR 0635 §1): the node-engine keeps no copy of
// them, so no copy can drift from the seat as the seven it kept did (issue 1921). A copy is a list of the catalogue's
// verbs written in the code; the guard proves it finds one, then finds none in the node-engine.
t.Run("no seat's verbs are copied into the node-engine", func(t *testing.T) {
copied := `package x
import mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
func someVerbs() []mcat.Verb { return []mcat.Verb{{Name: "routes"}} }
`
if got := verbCopies(t, "copied.go", copied); len(got) == 0 {
t.Fatal("the guard finds no copy in a file that holds one, so it proves nothing")
}
var found []string
root := filepath.Join("..", "..", "..")
err := filepath.WalkDir(root, func(p string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() && (d.Name() == "testdata" || strings.HasPrefix(d.Name(), ".")) && p != root {
return filepath.SkipDir
}
if d.IsDir() || !strings.HasSuffix(p, ".go") || strings.HasSuffix(p, "_test.go") {
return nil
}
found = append(found, verbCopies(t, p, nil)...)
return nil
})
if err != nil {
t.Fatal(err)
}
if len(found) > 0 {
t.Errorf("the node-engine copies a seat's verbs instead of reading the seat list from the bus: %v", found)
}
})
}
// verbCopies is where one Go file writes the catalogue's verbs itself: a function answering a list of them, or a verb
// written as a literal.
func verbCopies(t *testing.T, name string, src any) []string {
t.Helper()
fset := token.NewFileSet()
f, err := parser.ParseFile(fset, name, src, parser.SkipObjectResolution)
if err != nil {
t.Fatal(err)
}
alias := ""
for _, imp := range f.Imports {
if p, _ := strconv.Unquote(imp.Path.Value); p == "git.novox.be/novox/mesh-sdk/go/catalogue" {
alias = "catalogue"
if imp.Name != nil {
alias = imp.Name.Name
}
}
}
if alias == "" {
return nil
}
isVerb := func(e ast.Expr) bool {
if a, ok := e.(*ast.ArrayType); ok {
e = a.Elt
}
sel, ok := e.(*ast.SelectorExpr)
if !ok || sel.Sel.Name != "Verb" {
return false
}
x, ok := sel.X.(*ast.Ident)
return ok && x.Name == alias
}
var out []string
ast.Inspect(f, func(n ast.Node) bool {
switch n := n.(type) {
case *ast.FuncDecl:
if n.Type.Results != nil {
for _, r := range n.Type.Results.List {
if isVerb(r.Type) {
out = append(out, fset.Position(n.Pos()).String()+" "+n.Name.Name)
}
}
}
case *ast.CompositeLit:
if n.Type != nil && isVerb(n.Type) {
out = append(out, fset.Position(n.Pos()).String())
}
}
return true
})
return out
}
@@ -1,6 +1,7 @@
package resolution
import (
"slices"
"strings"
"testing"
@@ -75,6 +76,29 @@ func TestAConsumerBesideTheProviderIsServedItsName(t *testing.T) {
if strings.Contains(binding, "${bound:") {
t.Fatalf("a placeholder reached the consumer:\n%s", binding)
}
// Issue 1921 (the group check of 8a80e8e86858, novox): the provider held on this machine (composed apart, as what runs
// of it) is not in the composed set, so none of its own names is composed here. The consumer beside it is told
// what it serves without the keys that name them, each said on the need, never the placeholder: its binding refused
// the consumer with "has a file that says ${bound:route:name-web}, and lab does not require route".
held, err := Resolve(shelf(minimalRouteProxy(), m, consumer), []string{"route-proxy", "forge", "lab"}, node, mcat.World{})
if err != nil {
t.Fatal(err)
}
held.Modules = slices.DeleteFunc(slices.Clone(held.Modules), func(m mcat.Manifest) bool { return m.Module == "forge" })
out, err = Declaration(held, mcat.Rendering{})
if err != nil {
t.Fatalf("a consumer beside a held provider: %v", err)
}
for _, r := range out {
if c, _ := r["content"].(string); r["path"] == "/var/lib/lab/package-registry.json" && (strings.Contains(c, "${bound:") || !strings.Contains(c, "npm-path")) {
t.Fatalf("the binding beside a held provider:\n%s", c)
}
}
for _, n := range held.Needs {
if n.Name == "npm-package-registry" && (len(n.LeftOut) != 1 || !strings.Contains(n.LeftOut[0], `"public-root"`)) {
t.Fatalf("the need beside a held provider says not what was left out: %+v", n)
}
}
}
// From another machine, the provider's served values carry what that machine composed, and a key it
+10 -2
View File
@@ -65,8 +65,9 @@ func unsetInConfigWords(err error, m mcat.Manifest) error {
// The last layer setting a key wins, which is the node's over the mesh's over the module's own
// default (novox/hq ADR 0262) — the same order settle applies to a mergeable file. The caller lays
// the defaults under the layers with WithDefaults. A value that is not a string is written the way a program would read
// it (a number without a trailing .000000, a boolean as true/false).
func settingInto(resource map[string]any, layers []mcat.Layer, module string) error {
// it (a number without a trailing .000000, a boolean as true/false). declared are the keys the module's manifest
// declares under `config`, the only ones `${config-json}` writes (design 42 §9, answer A-15).
func settingInto(resource map[string]any, layers []mcat.Layer, module string, declared ...string) error {
if fmt.Sprint(resource["type"]) == "service" {
return settingIntoUnit(resource, layers, module)
}
@@ -83,6 +84,13 @@ func settingInto(resource map[string]any, layers []mcat.Layer, module string) er
if !ok {
return nil
}
content, err := configLinesInto(content, layers, module)
if err != nil {
return err
}
if content, err = configJSONInto(content, layers, declared); err != nil {
return err
}
for _, key := range settingsUsed(content) {
value, set := settingValue(layers, key)
if !set {
+134 -25
View File
@@ -1,42 +1,151 @@
package resolution
import (
"errors"
"fmt"
"strings"
"testing"
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// An operator's value reaches a file from the assignment's settings, the node's layer over the
// mesh's (novox/hq ADR 0112, ADR 0155), and a value nothing set is refused by name.
func TestASettingReachesAFileFromTheLayers(t *testing.T) {
file := map[string]any{"id": "env", "type": "file", "path": "/x/mail.env",
"content": "DOMAIN=${setting:domain}\nSITENAME=${setting:sitename}\nWORKERS=${setting:workers}\n"}
layers := []mcat.Layer{
{From: "the mesh", Values: map[string]any{"domain": "example.tld", "sitename": "Mesh", "workers": float64(4)}},
{From: "this node", Values: map[string]any{"sitename": "This one"}},
// A config value reaches a resource from the assignment's layers, the node's over the mesh's (novox/hq ADR 0112,
// ADR 0155), and a value nothing set is refused by name; and a resource under `if-config` is declared only while a
// config value holds (design 42 zfs-owns-zfs §2, ADR 0679 §16 (b)). One table of the two, each row its layers, what it
// does to a resource and what comes of it.
func TestAConfigValueReachesAResourceAndDecidesWhetherItIsDeclared(t *testing.T) {
filled := func(content string) func([]mcat.Layer) (string, error) {
return func(layers []mcat.Layer) (string, error) {
file := map[string]any{"id": "env", "type": "file", "path": "/x/mail.env", "content": content}
err := settingInto(file, layers, "mail", "n", "b", "s", "l", "o")
return file["content"].(string), err
}
}
if err := settingInto(file, layers, "mail"); err != nil {
t.Fatal(err)
// declared is the ids of what is left of a resource under each condition, and whether the key is gone from it.
declared := func(conditions ...any) func([]mcat.Layer) (string, error) {
return func(layers []mcat.Layer) (string, error) {
kept := withoutUnconfigured([]map[string]any{
{"id": "plain", "type": "file"}, {"id": "cond", "type": "file", mcat.IfConfigField: conditions}}, layers)
var ids []string
for _, r := range kept {
if _, has := r[mcat.IfConfigField]; has {
return "", errKeyKept
}
ids = append(ids, r["id"].(string))
}
return strings.Join(ids, ","), nil
}
}
if file["content"] != "DOMAIN=example.tld\nSITENAME=This one\nWORKERS=4\n" {
t.Fatalf("filled as %q", file["content"])
values := func(kv map[string]any) []mcat.Layer { return []mcat.Layer{{From: "the mesh", Values: kv}} }
for _, row := range []struct {
name string
layers []mcat.Layer
do func([]mcat.Layer) (string, error)
want string // what comes of it
errs []string // or what the refusal says
}{
{"the node's layer over the mesh's", []mcat.Layer{
{From: "the mesh", Values: map[string]any{"domain": "example.tld", "sitename": "Mesh", "workers": float64(4)}},
{From: "this node", Values: map[string]any{"sitename": "This one"}}},
filled("DOMAIN=${setting:domain}\nSITENAME=${setting:sitename}\nWORKERS=${setting:workers}\n"),
"DOMAIN=example.tld\nSITENAME=This one\nWORKERS=4\n", nil},
{"a value nothing set is refused, the file left as it was", values(map[string]any{"other": "x"}),
filled("DOMAIN=${setting:domain}\n"), "DOMAIN=${setting:domain}\n",
[]string{"${setting:domain}", "settings set mail", "set today: other"}},
{"config-lines: a list of two writes two lines in order", values(map[string]any{"k": []any{"a", "b"}}),
filled("h\n${config-lines:k:options zfs x=}\nt\n"), "h\noptions zfs x=a\noptions zfs x=b\nt\n", nil},
{"config-lines: mib-bytes writes a million MiB as a whole number", values(map[string]any{"k": float64(1000000)}),
filled("${config-lines:k:zfs_arc_max=:mib-bytes}\n"), "zfs_arc_max=1048576000000\n", nil},
{"config-lines: mib-bytes writes a million MiB given as text as a whole number", values(map[string]any{"k": "1000000"}),
filled("${config-lines:k:zfs_arc_max=:mib-bytes}\n"), "zfs_arc_max=1048576000000\n", nil},
{"config-lines: mib-bytes writes a size printed in exponent form as a whole number", values(map[string]any{"k": "1e+06"}),
filled("${config-lines:k:zfs_arc_max=:mib-bytes}\n"), "zfs_arc_max=1048576000000\n", nil},
{"config-lines: mib-bytes refuses a fraction", values(map[string]any{"k": "1.5"}),
filled("${config-lines:k:zfs_arc_max=:mib-bytes}\n"), "${config-lines:k:zfs_arc_max=:mib-bytes}\n", []string{"mib-bytes"}},
{"config-lines: an empty list writes nothing", values(map[string]any{"k": []any{}}),
filled("h\n${config-lines:k:p=}\nt\n"), "h\nt\n", nil},
{"config-lines: an unset value writes nothing", nil, filled("h\n${config-lines:k:p=}\nt\n"), "h\nt\n", nil},
{"config-lines: a set scalar writes one line", values(map[string]any{"k": "v"}),
filled("${config-lines:k:p=}\n"), "p=v\n", nil},
{"config-lines: mib-bytes writes MiB in bytes", values(map[string]any{"k": float64(64)}),
filled("${config-lines:k:zfs_arc_max=:mib-bytes}\n"), "zfs_arc_max=67108864\n", nil},
{"config-lines: an entry with a newline is refused, the file left as it was", values(map[string]any{"k": []any{"a\nb"}}),
filled("${config-lines:k:p=}\n"), "${config-lines:k:p=}\n", []string{"newline"}},
{"config-json: the set values only, each as its own type", values(map[string]any{"n": float64(64), "b": true, "s": "x",
"l": []any{"a"}, "o": map[string]any{"f": float64(1)}}), filled("${config-json}\n"),
"{\n \"b\": true,\n \"l\": [\n \"a\"\n ],\n \"n\": 64,\n \"o\": {\n \"f\": 1\n },\n \"s\": \"x\"\n}\n", nil},
{"config-json: a key the module does not declare is not written", values(map[string]any{"s": "x", "scrub-pool": "tank"}),
filled("${config-json}\n"), "{\n \"s\": \"x\"\n}\n", nil},
{"config-json: nothing set is an empty object, not a refusal", nil, filled("${config-json}\n"), "{}\n", nil},
{"if-config: a key never set leaves it out", nil, declared("k"), "plain", nil},
{"if-config: a key set to text declares it", values(map[string]any{"k": "x"}), declared("k"), "plain,cond", nil},
{"if-config: a key set to false, off, none, empty or an empty list leaves it out", values(map[string]any{"k": false}),
declared("k"), "plain", nil},
{"if-config: off", values(map[string]any{"k": "off"}), declared("k"), "plain", nil},
{"if-config: none", values(map[string]any{"k": "none"}), declared("k"), "plain", nil},
{"if-config: empty text", values(map[string]any{"k": ""}), declared("k"), "plain", nil},
{"if-config: an empty list", values(map[string]any{"k": []any{}}), declared("k"), "plain", nil},
{"if-config: a list of entries", values(map[string]any{"k": []any{"a"}}), declared("k"), "plain,cond", nil},
{"if-config: a value equal to the condition", values(map[string]any{"k": "on"}), declared("k=on"), "plain,cond", nil},
{"if-config: a value other than the condition", values(map[string]any{"k": "off"}), declared("k=on"), "plain", nil},
{"if-config: a list holding the entry", values(map[string]any{"k": []any{"a", "b"}}), declared("k=b"), "plain,cond", nil},
{"if-config: a list without the entry", values(map[string]any{"k": []any{"a"}}), declared("k=b"), "plain", nil},
{"if-config: any of the conditions", values(map[string]any{"j": "x"}), declared("k", "j"), "plain,cond", nil},
{"if-config: a value from a default layer", []mcat.Layer{{Default: true, Values: map[string]any{"k": "on"}}},
declared("k=on"), "plain,cond", nil},
{"if-config: a number equal to the condition", values(map[string]any{"k": float64(64)}), declared("k=64"), "plain,cond", nil},
} {
t.Run(row.name, func(t *testing.T) {
got, err := row.do(row.layers)
if len(row.errs) == 0 && err != nil {
t.Fatal(err)
}
if len(row.errs) > 0 && err == nil {
t.Fatalf("a refusal was expected, got %q", got)
}
for _, want := range row.errs {
if !strings.Contains(err.Error(), want) {
t.Fatalf("the refusal lacks %q: %v", want, err)
}
}
if got != row.want {
t.Fatalf("got %q, want %q", got, row.want)
}
})
}
}
func TestASettingNothingSetIsRefusedByName(t *testing.T) {
file := map[string]any{"id": "env", "type": "file", "content": "DOMAIN=${setting:domain}\n"}
err := settingInto(file, []mcat.Layer{{From: "the mesh", Values: map[string]any{"other": "x"}}}, "mail")
if err == nil {
t.Fatal("a setting nothing set was written as something")
}
for _, want := range []string{"${setting:domain}", "settings set mail", "set today: other"} {
if !strings.Contains(err.Error(), want) {
t.Fatalf("the refusal lacks %q: %v", want, err)
// errKeyKept is what a row of the table above says when `if-config` reached the node-engine, which parses strictly.
var errKeyKept = errors.New("the if-config key was kept in a declared resource")
// The composition takes a resource out while its condition does not hold, and a value going from set to unset takes it
// out of the next composition, which the node-engine removes as it removes any resource that left (design 42 §2).
func TestAConfigConditionedResourceIsComposedWhileItsValueHolds(t *testing.T) {
m := mustParse(t, `{"module":"zfs","version":"1","config":{"zed":{"type":"string","default":"off"},"mods":{"type":"list"}},"resources":[
{"id":"opts","type":"file","path":"/etc/modprobe.d/zfs.conf","content":"${config-lines:mods:options zfs }\n${config-json}\n"},
{"id":"conf","type":"file","path":"/etc/zfs/mesh.conf","content":"x\n"},
{"id":"zed","type":"file","path":"/etc/zfs/zed.rc","content":"y\n","if-config":["zed=on"]}]}`)
for _, row := range []struct {
layers []mcat.Layer
zed bool
}{{nil, false}, {[]mcat.Layer{{From: "this node", Values: map[string]any{"zed": "on", "mods": []any{"a=1"}}}}, true}} {
out, err := Declaration(mcat.Resolution{Node: "workstation", Modules: []mcat.Manifest{m}},
mcat.Rendering{Settings: mcat.SettingsBy{"zfs": row.layers}})
if err != nil {
t.Fatal(err)
}
if fileNamed(out, "zfs.conf") == nil {
t.Error("the unconditioned file was left out")
}
if opts := fileNamed(out, "zfs.opts"); opts == nil || (row.zed != strings.HasPrefix(fmt.Sprint(opts["content"]), "options zfs a=1\n")) {
t.Errorf("the template's lines do not follow the list: %v", opts)
}
zed := fileNamed(out, "zfs.zed")
if (zed != nil) != row.zed {
t.Errorf("with %v the conditioned file is declared: %v, want %v", row.layers, zed != nil, row.zed)
}
if _, has := zed[mcat.IfConfigField]; has {
t.Errorf("%s reached the node-engine, which parses strictly", mcat.IfConfigField)
}
}
// Left as it was: the literal placeholder must never reach a machine.
if file["content"] != "DOMAIN=${setting:domain}\n" {
t.Fatalf("content was changed on refusal: %q", file["content"])
}
}
+1 -1
View File
@@ -458,7 +458,7 @@ func JudgeSettings(m mcat.Manifest, layers []mcat.Layer, adopted bool) error {
for k, v := range settled {
copied[k] = v
}
if err := unsetInConfigWords(settingInto(copied, WithDefaults(m, layers), m.Module), m); err != nil {
if err := unsetInConfigWords(settingInto(copied, WithDefaults(m, layers), m.Module, declaredConfig(m)...), m); err != nil {
return err
}
}
-78
View File
@@ -1,78 +0,0 @@
package resolution
import (
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// nfsServerVerbs is the contract every holder of node-nfs-server serves (novox/hq ADR 0263).
func nfsServerVerbs() []mcat.Verb {
return []mcat.Verb{
{Name: "exports", Description: "Every share this machine exports: its name, its path, read-write or " +
"read-only, the owner every client is mapped to (uid and gid), each node it is exported to with that " +
"node's private address and access (only the nodes the server grants it to and that ask for it), " +
"what is granted and not asked for or asked for and not granted, and whether the kernel holds it " +
"now. Also the exports found that are not the mesh's: a dataset's sharenfs property, a line in " +
"/etc/exports.",
Input: schema(map[string]string{}, nil),
Replaces: []string{"exportfs -v", "cat /etc/exports", "zfs get sharenfs"}},
{Name: "clients", Description: "Which machines have mounted which share now, as the NFS server " +
"knows its clients.",
Input: schema(map[string]string{}, nil),
Replaces: []string{"ss -tn sport = :2049", "cat /proc/fs/nfsd/clients/*/info"}},
{Name: "test", Description: "Whether this machine exports one share for the mesh now, and whether its " +
"NFS service is up; with an address, also whether the share is exported to that address: a node's " +
"own private address, when the server grants it the share and the node asks for it. What a machine " +
"mounting the share asks before it mounts.",
Input: schema(map[string]string{
"share": "the share, by its name",
"address": "a node's private address, to ask whether the share is exported to it (optional)",
}, []string{"share"}),
Replaces: []string{"showmount -e"}},
{Name: "reload", Description: "Have the kernel read this machine's export files again now (exportfs " +
"-ra) and answer the shares as it then holds them. The module's own process writes its export " +
"file; this is for after a change made outside it. Changes no shared path.",
Input: schema(map[string]string{}, nil),
Replaces: []string{"exportfs -ra"}},
{Name: "adopt", Description: "Take over an export a person made by hand: clear a dataset's sharenfs " +
"property for a path the mesh's own export now serves — only when that export is live. Without " +
"confirm, says what it would do and changes nothing.",
Input: schema(map[string]string{
"path": "the shared path whose hand-made export is taken over",
"confirm": "\"true\": change it (needs why); anything else is a dry run",
"why": "why, for the record",
}, []string{"path"}, "confirm"),
Replaces: []string{"zfs set sharenfs=off"}},
}
}
// mountsVerbs is the contract every holder of node-mounts serves (novox/hq ADR 0263).
func mountsVerbs() []mcat.Verb {
return []mcat.Verb{
{Name: "list", Description: "Every mount point on this machine: its fstab line, the mesh's mount and " +
"automount units for it, its state (armed, mounted, absent, unreachable, failed) and since when, " +
"and which settings asked for it. A password in a mount's options is never shown.",
Input: schema(map[string]string{}, nil),
Replaces: []string{"cat /etc/fstab", "findmnt", "systemctl list-units --type=mount"}},
{Name: "test", Description: "Whether one share's or occasional source's server answers from this " +
"machine now, without touching its mount point.",
Input: schema(map[string]string{"name": "the share or source, by its name"}, []string{"name"}),
Replaces: []string{"showmount -e", "ping"}},
{Name: "mount", Description: "Mount one share or occasional source now, rather than at its first " +
"access.",
Input: schema(map[string]string{"name": "the share or source, by its name"}, []string{"name"}),
Replaces: []string{"mount"}},
{Name: "unmount", Description: "Release one share or occasional source now; its automount stays " +
"armed, so the next access mounts it again.",
Input: schema(map[string]string{"name": "the share or source, by its name"}, []string{"name"}),
Replaces: []string{"umount"}},
{Name: "adopt", Description: "Take over a mount a person wrote by hand: comment out the /etc/fstab " +
"line for a mount point the mesh's own units now serve, keeping a copy of the file — only when " +
"the mesh's automount for it is armed. Without confirm, says what it would do and changes nothing.",
Input: schema(map[string]string{
"mountpoint": "the mount point whose fstab line is taken over",
"confirm": "\"true\": change it (needs why); anything else is a dry run",
"why": "why, for the record",
}, []string{"mountpoint"}, "confirm"),
Replaces: []string{"sed -i /etc/fstab"}},
}
}
@@ -0,0 +1,35 @@
package resolution
import (
"sort"
mcat "git.novox.be/novox/mesh-sdk/go/catalogue"
)
// Resolve is ResolveHolding for a test that wants a claim problem as a refusal of the whole set, as before issue 1966.
func Resolve(catalogue map[string]mcat.Manifest, assigned []string, node mcat.Node, world mcat.World) (mcat.Resolution, error) {
got, held, err := ResolveHolding(catalogue, assigned, node, world)
if err != nil {
return mcat.Resolution{}, err
}
if len(held) > 0 {
var problems []string
for _, why := range held {
problems = append(problems, why)
}
sort.Strings(problems)
return mcat.Resolution{}, &Refusal{Problems: problems}
}
return got, nil
}
// checkClaims is holdClaims with the problems as a list.
func checkClaims(modules []mcat.Manifest, node mcat.Node, elsewhere []mcat.Held, holdings []mcat.Held) ([]mcat.Held, []string) {
held, why := holdClaims(modules, node, elsewhere, holdings)
var problems []string
for _, w := range why {
problems = append(problems, w)
}
sort.Strings(problems)
return held, problems
}
@@ -10,10 +10,11 @@ import (
// builderNode is a machine that can run a build module: the package manager and user namespaces it asks of
// the machine, and the forge account (mesh-scope, provided by the forge) and the artifact store the build
// modules publish to (ADR 0513, provided by distribution) stood in for, and node-tools, the tool runner they pull their
// asks through (mesh-catalog issue 1911; mesh-tools' module, not in the catalogue).
// asks through (mesh-catalog issue 1911; mesh-tools' module, not in the catalogue), with the tool runner's
// seat-settle-delay the build modules ask of it since issue 1748 (ADR 0664 §5).
func builderNode() mcat.Node {
return mcat.Node{Name: "builder", Site: "house",
Capabilities: map[string]bool{"package-manager": true, "user-namespaces": true}}
Capabilities: map[string]bool{"package-manager": true, "user-namespaces": true, "seat-settle-delay": true}}
}
func toolchainShelf(t *testing.T) map[string]mcat.Manifest {
@@ -25,6 +25,10 @@ type filler struct {
// or the sweep refuses the placeholder it would have filled.
var fillers = []filler{
{namespace: "setting", patterns: []*regexp.Regexp{settingRef}, fields: map[string][]string{"file": {"content"}, "service": {"unit"}, "directory": {"owner"}, "container": {"groups"}}},
// A file's content only: lines from a config value's entries, and the module's config as one JSON object
// (design 42 zfs-owns-zfs §3, §9; config_lines.go).
{namespace: "config-lines", patterns: []*regexp.Regexp{configLinesRef}, fields: map[string][]string{"file": {"content"}}},
{namespace: "config-json", patterns: []*regexp.Regexp{configJSONRef}, fields: map[string][]string{"file": {"content"}}},
{namespace: "dir", patterns: []*regexp.Regexp{dirRef}, fields: map[string][]string{"*": {"path", "content", "volumes", "env", "env-file"}}},
{namespace: "access", patterns: []*regexp.Regexp{accessRef}, fields: map[string][]string{"*": {"path", "content", "volumes", "env", "env-file"}}},
{namespace: "secret", patterns: []*regexp.Regexp{placeholder}, fields: map[string][]string{"file": {"content"}},
-59
View File
@@ -1,59 +0,0 @@
package resolution
import (
"maps"
"slices"
)
// schema is a JSON schema for an object of string properties, which is every argument the verbs
// above take. Kept small on purpose: a schema an agent cannot read is a tool it cannot call. The
// switches are the properties that are "true" or "false" and nothing else, said in the schema as an
// enum so a caller sees it and the verb can refuse any other word rather than read it as false.
//
// **The schema is the whole of what a verb takes** (novox/hq issue 244): an argument it does not
// name is refused when the verb is called, never passed over.
// withObject adds to a verb's input s the property name, an object of the string fields given, each required.
func withObject(s map[string]any, name, description string, fields map[string]string) map[string]any {
inner := schema(fields, slices.Sorted(maps.Keys(fields)))
inner["description"], inner["additionalProperties"] = description, false
s["properties"].(map[string]any)[name] = inner
return s
}
func schema(properties map[string]string, required []string, switches ...string) map[string]any {
isSwitch := map[string]bool{}
for _, s := range switches {
if _, declared := properties[s]; !declared {
panic("a switch that is not a property: " + s)
}
isSwitch[s] = true
}
props := map[string]any{}
for name, description := range properties {
p := map[string]any{"type": "string", "description": description}
if isSwitch[name] {
p["enum"] = []string{"true", "false"}
}
props[name] = p
}
out := map[string]any{"type": "object", "properties": props}
if len(required) > 0 {
out["required"] = required
}
return out
}
// listed makes the named properties of a schema lists of text: a caller gives them as a JSON array (or, as
// any argument, one text separated by commas).
func listed(in map[string]any, names ...string) map[string]any {
props, _ := in["properties"].(map[string]any)
for _, n := range names {
p, _ := props[n].(map[string]any)
if p == nil {
panic("a list that is not a property: " + n)
}
props[n] = map[string]any{"type": "array", "items": map[string]any{"type": "string"},
"description": p["description"]}
}
return in
}
+3 -3
View File
@@ -21,14 +21,14 @@ func resolutionYieldsOf(r mcat.Resolution, m mcat.Manifest, with mcat.Rendering)
if !yields(m) {
return nil, nil
}
engine := mcat.Manifest{Module: engineModule}
var engine mcat.Manifest // the module claiming the seat node-engine; none in the resolution leaves the default
for _, e := range r.Modules {
if e.Module == engineModule {
if ClaimsEngine(e) {
engine = e
}
}
share := mcat.BuildCPUDefaultPercent
for _, s := range Effective(engine, with.Settings[engineModule]) {
for _, s := range Effective(engine, with.Settings[engine.Module]) {
if s.Key == mcat.BuildCPUShareSetting {
if n, ok := percentOf(s.Value); ok {
share = n
+23 -12
View File
@@ -22,6 +22,10 @@ type Resolved struct {
// NotGiven are the words of each credential only a person gives that nobody has: the consumer still resolves without it
// (novox/hq ADR 0337 clause 4).
NotGiven []string
// Held are the modules this machine leaves out of its declaration and runs as they run, and why: one whose claim cannot be
// held (two claim a seat held once) or whose credential was refused (issue 1966, ADR 0636 §3). The rest of the machine
// composes; the declaration says each as left out (Declared.LeftOut), so a condition names it.
Held map[string]string
}
// Resolve works out everything a node should run, from what was assigned to it (the controller's planFor, plan.go): the
@@ -77,7 +81,7 @@ func (r Resolved) copied() Resolved {
// planForNow is planFor worked out.
func (c *composer) planForNow(nodeName string) (Resolved, error) {
resolved, err := c.resolution(nodeName)
resolved, held, err := c.resolutionHolding(nodeName)
if err != nil {
return Resolved{}, err
}
@@ -87,6 +91,7 @@ func (c *composer) planForNow(nodeName string) (Resolved, error) {
}
shelf := c.shelfFor(node)
var out Resolved
out.Held = held
// The credential for each thing this node takes from elsewhere.
for i, n := range resolved.Needs {
@@ -129,8 +134,13 @@ func (c *composer) planForNow(nodeName string) (Resolved, error) {
case cred.State == CredentialRefused:
// Said rather than skipped. A machine that resolves cleanly and receives no credential is one that will fail to
// authenticate at some later, less obvious moment.
return Resolved{}, fmt.Errorf("%s on %s needs %s from %s and no credential could be made for it: %s",
// Of that module only: it is held, the rest of the machine composes (issue 1966).
if out.Held == nil {
out.Held = map[string]string{}
}
out.Held[n.For] = fmt.Sprintf("%s on %s needs %s from %s and no credential could be made for it: %s",
n.For, nodeName, n.Name, n.From, cred.Why)
continue
case cred.State == CredentialNotGiven, outside && !(held && cred.State == CredentialHeld):
// Not given, and the rest of the consumer runs (ADR 0337 clause 4): no credential file, so the module can tell
// it was not given; nothing the mesh made stands in for it.
@@ -249,18 +259,19 @@ func movedProvider(m catalogue.Manifest) bool {
return m.Moved || slices.ContainsFunc(m.Claims, func(c catalogue.Claim) bool { return c.Name == "mesh-vault" })
}
// resolution works out what one node should run, from what was assigned to it: the controller's planFor up to and
// including the resolution (plan.go planFor), and no further. A failure to compose the node's own modules is wrapped as
// notResolvable; every other failure is returned as it is.
func (c *composer) resolution(nodeName string) (catalogue.Resolution, error) {
// resolutionHolding works out what one node should run, from what was assigned to it: the controller's planFor up to and
// including the resolution (plan.go planFor), and no further. A module whose claim cannot be held is returned beside the
// resolution, with why, and not refused with the machine (issue 1966). A failure to compose the node's own modules is
// wrapped as notResolvable; every other failure is returned as it is.
func (c *composer) resolutionHolding(nodeName string) (catalogue.Resolution, map[string]string, error) {
node, err := c.in.node(nodeName)
if err != nil {
return catalogue.Resolution{}, err
return catalogue.Resolution{}, nil, err
}
shelf := c.shelfFor(node)
world, _, left, err := c.theRestOfTheMesh(nodeName)
if err != nil {
return catalogue.Resolution{}, err
return catalogue.Resolution{}, nil, err
}
world.Pinned = node.Pins
// Where each of its consumers of a provision that keeps data was last sent (novox/hq ADR 0232).
@@ -268,14 +279,14 @@ func (c *composer) resolution(nodeName string) (catalogue.Resolution, error) {
onNetwork, err := c.whereEveryoneIs()
if err != nil {
return catalogue.Resolution{}, err
return catalogue.Resolution{}, nil, err
}
// What this mesh can answer with a record rather than a machine, and which record each of this node's modules was
// put on.
world.Licences, world.Using = c.licencesFor(nodeName)
resolved, err := resolution.Resolve(shelf, node.Assigned,
resolved, held, err := resolution.ResolveHolding(shelf, node.Assigned,
catalogue.Node{Name: nodeName, Site: node.Place.Site, Capabilities: node.Capabilities,
At: onNetwork[nodeName], PublicDomain: node.PublicDomain,
Account: node.Account, AccountHome: node.AccountHome,
@@ -295,9 +306,9 @@ func (c *composer) resolution(nodeName string) (catalogue.Resolution, error) {
err, strings.Join(c.unhostable, "; "))
}
// The node's own set does not compose: the only failure a mesh-wide gatherer may pass over.
return catalogue.Resolution{}, notResolvable{err}
return catalogue.Resolution{}, nil, notResolvable{err}
}
return resolved, nil
return resolved, held, nil
}
// shelfFor is the catalogue as one node composes its own set: the Input's, with each recorded or core module the node keeps at the
+40 -16
View File
@@ -2,6 +2,7 @@ package compose
import (
"errors"
"slices"
"strings"
"testing"
@@ -223,21 +224,38 @@ func TestACredentialOnlyAPersonGivesIsNotMadeAndTheConsumerStillResolves(t *test
}
}
// 1362/AC3: a credential the send would refuse, a given value sealed to a key a machine no longer has, refuses the resolution with
// today's words, and is not a node whose set does not compose.
func TestACredentialTheSendWouldRefuseRefusesTheResolutionInTheSendsWords(t *testing.T) {
// 1362/AC3, issue 1966: a fault of one module's own, a credential the send would refuse (a given value sealed to a key a machine no
// longer has) or a second claimant of a seat held once, costs that module and not the machine: it is held with the send's words, its
// declaration says it left out, and every other module of the machine composes. Not a node whose set does not compose.
func TestAModuleWhoseCredentialOrClaimFailsIsHeldAndTheMachineComposes(t *testing.T) {
why := "files's \"s3-bucket\" credential from anchor was accepted from a person, and a sealing key at one end has changed since. " +
"The mesh cannot re-seal a value it does not hold: accept it again with `secret accept laptop files s3-bucket --provider anchor`"
in := theObjectsMesh(t, objects())
in.Credentials = []Credential{{Provision: "s3-bucket", Consumer: "laptop", ConsumerModule: "files", Provider: "anchor",
State: CredentialRefused, Why: why}}
_, err := Resolve(in, "laptop")
want := "files on laptop needs s3-bucket from anchor and no credential could be made for it: " + why
if err == nil || err.Error() != want {
t.Fatalf("got %v, want %q", err, want)
}
if IsUnresolvable(err) {
t.Error("a refused credential is a node whose set does not compose")
for _, c := range []struct{ name, module, said string }{
{"a refused credential", "files", "files on laptop needs s3-bucket from anchor and no credential could be made for it: " + why},
{"a second claimant", "holder-two", "holder-one and holder-two both claim"},
} {
t.Run(c.name, func(t *testing.T) {
withSeat(t)
in := theObjectsMesh(t, objects())
in.Credentials = []Credential{{Provision: "s3-bucket", Consumer: "laptop", ConsumerModule: "files", Provider: "anchor",
State: CredentialRefused, Why: why}}
if c.module == "holder-two" {
in = aMesh(t)
for _, name := range []string{"holder-one", "holder-two"} {
in.Catalogue[name] = aBrokerClaiming(name)
}
laptop := nodeOf(t, &in, "laptop")
laptop.Assigned = append(laptop.Assigned, "holder-one", "holder-two")
}
got, err := Resolve(in, "laptop")
if err != nil || IsUnresolvable(err) || len(got.Held) != 1 || !strings.Contains(got.Held[c.module], c.said) || len(got.Plan.Modules) == 0 {
t.Fatalf("held %v, %v; want %s held with %q", got.Held, err, c.module, c.said)
}
d, _, err := Compose(in, "laptop")
if err != nil || !slices.Contains(d.LeftOut, c.module) || !strings.Contains(d.LeftOutWhy[c.module], c.said) || len(d.Resources) == 0 {
t.Errorf("declared: left out %v, %d resources, %v", d.LeftOut, len(d.Resources), err)
}
})
}
}
@@ -277,13 +295,19 @@ func TestARecordAnsweredNeedGetsItsKeyAndTheManagerItsPublicKey(t *testing.T) {
// 1362/AC5: a node whose own set does not compose returns an error callers can tell from an unreadable Input.
func TestANodeWhoseOwnSetDoesNotComposeIsToldApartFromAnUnreadableInput(t *testing.T) {
in := aMesh(t)
in.Catalogue["rival-one"] = catalogue.Manifest{Module: "rival-one", Version: "1", Claims: []catalogue.Claim{{Name: "the-seat", Scope: catalogue.ScopeNode}}}
in.Catalogue["rival-two"] = catalogue.Manifest{Module: "rival-two", Version: "1", Claims: []catalogue.Claim{{Name: "the-seat", Scope: catalogue.ScopeNode}}}
in.Catalogue["rival-one"] = catalogue.Manifest{Module: "rival-one", Version: "1", Requires: []string{"nothing-gives-this"}}
laptop := nodeOf(t, &in, "laptop")
laptop.Assigned = append(laptop.Assigned, "rival-one", "rival-two")
laptop.Assigned = append(laptop.Assigned, "rival-one")
if _, err := Resolve(in, "laptop"); err == nil || !IsUnresolvable(err) {
t.Fatalf("a set that does not compose: %v", err)
}
// 1298/AC3: and its declaration is an error and nothing else.
if d, problems, err := Compose(in, "laptop"); err == nil || !IsUnresolvable(err) || d.Resources != nil || problems != nil {
t.Fatalf("a set that does not compose: %v %v %v", d.Resources, problems, err)
}
if _, _, err := Compose(in, "nowhere"); err == nil || IsUnresolvable(err) {
t.Fatalf("a node the Input does not have: %v", err)
}
if _, err := Resolve(in, "nowhere"); err == nil || IsUnresolvable(err) || !errors.Is(err, ErrNoSuchNode) {
t.Fatalf("a node the mesh does not have: %v", err)
}
+73 -21
View File
@@ -25,9 +25,10 @@ func unresolvable(err error) bool {
return errors.As(err, &n)
}
// Passes is what a composition found that the mesh must record, and did not: each seat held once whose holder on record
// is assigned nowhere that exactly one claimant takes over (novox/hq ADR 0449 clause 4), and the lines that say what was
// found. The caller records a pass and writes its hand act; this package does neither.
// Passes is what a composition found and composed with: each seat held once whose holder on record has left it (assigned
// nowhere, or none of its assignments' manifests claims the seat) that exactly one claimant takes over (novox/hq ADR 0449
// clause 4, widened by ADR 0660), and the lines that say what was found. A pass is composed and said, never recorded: neither
// this package nor its caller writes the holdings (ADR 0660 clause 4).
type Passes struct {
// Passes are the seats to pass, in the order they were found.
Passes []catalogue.SeatPass
@@ -92,8 +93,8 @@ func (c *composer) theRestOfTheMeshNow(exclude string) (catalogue.World, Passes,
pins map[string]catalogue.Chosen
}
var others []candidate
// Every machine's assignments, before the holdings are read by any resolution (novox/hq ADR 0449 clause 4): a seat
// held once whose holder on record is assigned on no machine passes to its only claimant here.
// Every machine's assignments, before the holdings are read by any resolution (novox/hq ADR 0449 clause 4, ADR 0660): a
// seat held once whose holder on record has left it passes to its only claimant here, composed and not recorded.
everyAssigned := map[string][]string{}
for _, n := range nodes {
if len(n.Assigned) > 0 {
@@ -101,7 +102,7 @@ func (c *composer) theRestOfTheMeshNow(exclude string) (catalogue.World, Passes,
}
}
var passes Passes
holdings, passes = passSeatsOn(holdings, assignmentsOf(shelf, everyAssigned))
holdings, passes = passSeatsOn(holdings, c.assignmentsOf(everyAssigned), c.heard, c.entries(shelf))
for _, n := range nodes {
if n.Name == exclude {
continue
@@ -121,7 +122,7 @@ func (c *composer) theRestOfTheMeshNow(exclude string) (catalogue.World, Passes,
var firstHeld []catalogue.Held
c.unhostable = nil
for _, o := range others {
got, err := resolution.Resolve(shelf, o.assigned, o.node, catalogue.World{Unchecked: true, Holdings: holdings})
got, _, err := resolution.ResolveHolding(shelf, o.assigned, o.node, catalogue.World{Unchecked: true, Holdings: holdings})
if err != nil {
// Said, not skipped: a machine dropped here offers nothing and holds nothing.
left = append(left, LeftOut{Node: o.node.Name, Err: err})
@@ -161,7 +162,7 @@ func (c *composer) theRestOfTheMeshNow(exclude string) (catalogue.World, Passes,
// Each machine is resolved with its own pins, as its plan is.
theirs := world
theirs.Pinned = o.pins
got, err := resolution.Resolve(shelf, o.assigned, o.node, theirs)
got, _, err := resolution.ResolveHolding(shelf, o.assigned, o.node, theirs)
if err != nil {
// Said only for the whole-mesh view: with one machine excluded, the view is partial by design.
if exclude == "" {
@@ -175,11 +176,12 @@ func (c *composer) theRestOfTheMeshNow(exclude string) (catalogue.World, Passes,
return world, passes, left, nil
}
// passSeatsOn finds, in the composition that finds it, the new holder of each seat held once whose holder on record is
// assigned nowhere and which has exactly one claimant able to hold it (novox/hq ADR 0449 clause 4), and answers the
// holdings as they are after it, with the passes. Nothing is recorded.
func passSeatsOn(holdings []catalogue.Held, assignments []catalogue.Assignment) ([]catalogue.Held, Passes) {
passes, said := resolution.SeatsToPassOn(holdings, assignments)
// passSeatsOn finds, in the composition that finds it, the new holder of each seat held once whose holder on record has left
// it and which has exactly one claimant able to hold it (catalogue.SeatsToPassOn, novox/hq ADR 0660, design 47 G §199, §459), and
// answers the holdings as they are after it, with the passes. Nothing is recorded.
func passSeatsOn(holdings []catalogue.Held, assignments []catalogue.Assignment, heard func(node, module string) (catalogue.Heard, bool),
entries catalogue.EntryReader) ([]catalogue.Held, Passes) {
passes, said := catalogue.SeatsToPassOn(holdings, assignments, heard, entries)
out := Passes{Said: said}
for _, p := range passes {
out.Passes = append(out.Passes, p)
@@ -196,9 +198,10 @@ func passSeatsOn(holdings []catalogue.Held, assignments []catalogue.Assignment)
return holdings, out
}
// assignmentsOf are the assignments of every machine, as the passing of a seat reads them. A module assigned and
// missing from the shelf is still assigned: an assignment with no claims, so its holder is not taken for gone.
func assignmentsOf(shelf map[string]catalogue.Manifest, assigned map[string][]string) []catalogue.Assignment {
// assignmentsOf are the assignments of every machine, as the passing of a seat reads them (§459): a node, a module and no
// placement branch, this composer's view of the catalog half being the trunk's alone. The build each runs is resolved by
// catalogue.SeatsToPassOn from c.heard and c.entries.
func (c *composer) assignmentsOf(assigned map[string][]string) []catalogue.Assignment {
var out []catalogue.Assignment
names := make([]string, 0, len(assigned))
for node := range assigned {
@@ -207,16 +210,65 @@ func assignmentsOf(shelf map[string]catalogue.Manifest, assigned map[string][]st
sort.Strings(names)
for _, node := range names {
for _, module := range assigned[node] {
m, ok := shelf[module]
if !ok {
m = catalogue.Manifest{Module: module}
}
out = append(out, catalogue.Assignment{Node: node, Manifest: m})
out = append(out, catalogue.Assignment{Node: node, Module: module})
}
}
return out
}
// heard is what the node-engine seat's `running` said of a module on a node (Node.Heard): a fingerprint or stopped. Nothing
// is heard of a module no event named, and the passing then takes the entry of the module on the trunk (§199).
func (c *composer) heard(node, module string) (catalogue.Heard, bool) {
for _, n := range c.in.Nodes {
if n.Name == node {
h, ok := n.Heard[module]
return catalogue.Heard{Fingerprint: h.Fingerprint, Stopped: h.Stopped}, ok
}
}
return catalogue.Heard{}, false
}
// entries are the composer's reader of the catalog half for the passing of a seat: an entry by the fingerprint a node said it
// runs, read through the digest its event named, and a module's entry on the trunk, the shelf.
func (c *composer) entries(shelf map[string]catalogue.Manifest) catalogue.EntryReader {
byFingerprint := map[string]catalogue.Manifest{}
for _, n := range c.in.Nodes {
for _, h := range n.Heard {
if h.Read && h.Fingerprint != "" {
byFingerprint[h.Fingerprint] = h.Manifest
}
}
}
return entryReader{byFingerprint: byFingerprint, shelf: shelf}
}
type entryReader struct {
byFingerprint map[string]catalogue.Manifest
shelf map[string]catalogue.Manifest
}
func (r entryReader) ByFingerprint(fingerprint string) (catalogue.Manifest, bool) {
m, ok := r.byFingerprint[fingerprint]
return m, ok
}
// OnBranch answers the trunk's entry whatever the branch: this composer's shelf is the trunk's alone.
func (r entryReader) OnBranch(module, _ string) (catalogue.Manifest, bool) {
m, ok := r.shelf[module]
return m, ok
}
// Lines are what the passes say, one each, then what the finding itself says: a pass is composed so and recorded by nobody
// (ADR 0660 clause 4).
func (p Passes) Lines() []string {
var out []string
for _, x := range p.Passes {
out = append(out, fmt.Sprintf("%s: %s on %s is on record as its holder and holds it no more; composed as held by %s on %s, not recorded",
x.Seat.Name, x.From.Module, x.From.Node, x.To.Module, x.To.Node))
}
return append(out, p.Said...)
}
// servedOnNode is what a module on a node tells a consumer of one of its provisions, with THAT node's ports on it: the
// port the node was given over the one the mesh assigned over the manifest's own, settled with the node's settings
// layers. The one derivation for every reader of a provider's address.

Some files were not shown because too many files have changed in this diff Show More