Commit Graph
1229 Commits
Author SHA1 Message Date
mesh-tools 869d15e6d3 Merge pull request 'Design 47 G start: node-engine fixes placed by hand (A-226b/c, A-237b/c, manifest digest)' (#230) from fix/47-G-start-2-host into main 2026-11-16 14:15:10 +00:00
jschoubben 292aa54bbb node-engine: compose-candidate's chunked input is tested through HandActs.Call and past the inline bound, folded into the compose integration test; comments cite A-237c
Builds: design 47 G (G-22 lead answers 2) A-237c; A-237b
2026-11-16 14:49:39 +01:00
jschoubben f539cdb7ec node-engine: compose-candidate sends its composition input in chunks over the link's connection when asked with a call token (A-237b)
Builds: design 47 G (G-22 lead answers 2) A-237b; A-237, A-249
2026-11-16 14:38:31 +01:00
jschoubben 3b365ba7cc node-engine: a catalog revision's manifest digest is made the candidate's hex form (A-226b)
Builds: design 47 G (G-22 lead answers 2) A-226b
2026-11-16 14:26:03 +01:00
jschoubben 6eb41a17e1 node-engine: a run's compose-candidate key reads candidates.run-<id> by its group, with no heads compared (A-226c)
Builds: design 47 G (G-22 lead answers 2) A-226c; A-226b
2026-11-16 14:00:30 +01:00
jschoubben a1cd4a1a2f node-engine: compose-candidate reads a branch pipeline's catalog revision through mesh-catalog.revision and composes it as the candidate (A-226b)
Builds: design 47 G (G-22 lead answers 2) A-226b; ADR 0593 §7
2026-11-16 13:58:36 +01:00
jschoubben 7a30c0d54c node-engine: fold the three new tests into existing tests as cases, so the branch adds no test function (test budget)
Builds: design 47 G §524, §525, A-405, A-407, A-430, A-430a, A-432 (hq main 3723b2ea), ADR 0259 §8
2026-11-16 09:40:29 +01:00
jschoubben a9eb4372d9 Merge remote-tracking branch 'origin/main' into fix/47-G-start-2-host 2026-11-16 09:25:14 +01:00
jschoubben c025bb5f1e node-engine: at a start a hold for differing parts or a missing entry is judged again, not kept as a failed backup waiting for an answer (§524, §421, ADR 0568 §4) 2026-11-16 01:04:07 +01:00
jschoubben 5a3510b115 node-engine: a runtime of its own runs the tool runner's program, which the module's parts check leaves out (design 47 G §524, A-432) 2026-11-16 01:01:01 +01:00
jschoubben 7f892a757c node-engine: a held tool runner's kept program is not replaced by the tools bundles it serves, so novox's new indexer is run by it (ADR 0259 §8) 2026-11-16 00:50:33 +01:00
jschoubben 40ed0633e0 node-engine: a held tool runner keeps its running program, so a module of its own account is not held for want of a runtime that runs (ADR 0259 §8) 2026-11-16 00:34:50 +01:00
jschoubben 5bbde5db40 node-engine: the composer judges a changed want by its fingerprint, version, parts and keeping node, so a want that gains its parts is composed (design 47 G §525) 2026-11-16 00:28:41 +01:00
mesh-tools 21e452df28 Merge pull request 'Node-engine invokes the registries' versions verbs (design 47 G A-430a)' (#229) from fix/47-G-engine-invokes-versions into main 2026-11-15 16:08:13 +00:00
jschoubben dc32afdd7e Node-engine invokes the registries' versions verbs, so its grant admits resolving a want's version (design 47 G A-430a)
The grants part admits a seat verb's invokers by the manifest on the trunk; a node-engine that resolves a want's
version through oci-artifact-registry.versions and container-image-registry.versions is refused on the bus until
its manifest on main names both. Manifest only: a node-engine that does not resolve versions never calls them.
2026-11-15 17:04:37 +01:00
jschoubben 8a0c39280d Node-engine: a want naming its version is resolved to its digests through the registry seats (A-430a)
A want of a released build names the module's version and no parts. Each artifact of the build's entry is
resolved at that version through the verb versions of its kind's registry seat: oci-artifact-registry for a
bundle or an archive, container-image-registry for an image, under <module>/<artifact> or a package's own
name. An upstream copy, which is not numbered, takes the digest its manifest pins. partsMissing, buildOf and
KeptBuildOf (and the shadow's added modules) read those digests; a want with parts is read as before.

A lookup that fails is said in the journal, naming the module, the version and the seat, and holds the module
whole: nothing of it is composed. It is asked again after a minute. The manifest invokes both verbs.

Builds: design 47 G A-430, A-430a; lead answer A-430b
2026-11-15 16:52:59 +01:00
jschoubben a00cccfe0d Node-engine: a running module the assignment names is held, never removed, whatever composed nothing of it (A-410)
Builds: design 47 G A-410
2026-11-14 23:38:29 +01:00
jschoubben 592b4002bd Node-engine: a running module with no fingerprint is held in its answer, never removed (A-410)
Builds: design 47 G A-410
2026-11-14 23:18:36 +01:00
jschoubben 0adc5ab4e6 Node-engine: a running module without an entry resolves as the newest build its want names
Builds: design 47 G §421, lead answers A-405 and A-407
2026-11-14 22:47:33 +01:00
jschoubben 6079f634d6 Node-engine: say whether its seat is served and how long the first looks of what runs and of the accounts take (A-405)
Live on novox 2026-11-14 root-free answered 'judged no account yet' minutes after the start and nothing was logged:
the seat's serving and the first looks were silent.
2026-11-14 15:59:28 +01:00
jschoubben eaf011e971 Node-engine: a first start whose composing never ends is named by MESH_ENGINE_OWN_MODULE (A-404, live start recovery)
A-403 named the engine from the base's catalogue, but on novox the base stops at the first manifest it cannot read,
before mesh-host, so the engine stayed unnamed and its seat unserved.
2026-11-14 15:53:44 +01:00
jschoubben 1c3b2597ba Node-engine: named from the catalogue the base read, before the rest of the input (A-403)
Live on novox 2026-11-14 the G node-engine never ended a composing pass, so it never named its own module and never
served its seat node-engine: grants got no responders for root-free, judged novox not free, and the operator channel
held every Approve ask (the seed, hq#609) as no channel could prove who answered.
2026-11-14 15:48:45 +01:00
mesh-tools fd023f7473 Merge pull request 'Milestone 47 G: mesh-host's part of the one release' (#228) from feat/47-G-join12 into main 2026-11-13 13:09:29 +00:00
jschoubben 5caa760f5e A-342 engine-node-env: every process runs with its node's name as MESH_NODE
mesh/delivery merged: source.main moved to novox/mesh-host@fd023f7473bc
mesh/delivery-group group feat/47-G-join12 merged: every member is merged
mesh/check failed (2): mesh-delivery-checker: no published of the tagged version of go-meshconditions of me…; and 1 more | 5caa760f5eaa on fd023f7473bc
The apply gives each process the declaration's For as MESH_NODE beside its own env, so a process that reads MESH_NODE (mesh-sdk, the zfs follower's writer and its node-service-manager calls) names its node.

Builds: design 47 G A-342

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 09:20:46 +01:00
jschoubben d63aae1d55 Merge main into feat/47-G-join11: the node-engine's if-config, config-lines, config-json and stop at a failure (issue 1979)
The whole test's web manifest keeps G's entry form (no build artifact, the entry read by fingerprint) and takes main's resources under if-config and config-lines.

Builds: design 47 G §421; design 42 zfs-owns-zfs H1-H5

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 09:10:44 +01:00
mesh-tools f8acfaeed7 Merge pull request 'The node-engine stops a module's apply at its first failed resource, and reads if-config, config-lines and config-json (issue 1979)' (#227) from feature/1979-zfs-owns-zfs into main 2026-11-13 08:00:43 +00:00
jschoubben 0a9d1efe89 Merge main into feat/47-G-join11 (G takes in main before its push)
mesh/delivery-group group feat/47-G-join11 checking: 17 of 17 member(s) ready
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 0a9d1efe8918 on f8acfaeed79d | novox/mesh-host#226 in group feat/47-G-join11
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 02:16:56 +01:00
jschoubben 96a8690639 Fix 1979: mib-bytes writes the bytes as a whole number for every size
mesh/delivery merged: source.main moved to novox/mesh-host@f8acfaeed79d
mesh/delivery-group group feature/1979-zfs-owns-zfs merged: every member is merged
mesh/check being carried to 96a869063977 on f8acfaeed79d | from 0193949efe9f, mesh-catalog 117f933dacf0, mesh-delivery 7221a1e5cd7f, mesh-sdk e42b9491…
A size of a million MiB or more reached the file in exponent form and was refused.

Builds: design 42 zfs-owns-zfs §3, A-24
2026-11-13 01:32:36 +01:00
jschoubben 7868c9f365 Fix 1979: ${config-json} writes only the keys the module declares under config
A key a layer holds that the manifest does not declare (scrub-pool during a rename) is not a config value of the module.

Builds: design 42 zfs-owns-zfs §9, A-15, A-24, A-25
2026-11-13 01:32:27 +01:00
jschoubben 225f2abb1f H5 engine-integration: the node-engine whole applies if-config, config-lines and a module's stop at a failure
The existing node-engine integration test's module gains a file under if-config,
a template with config-lines from a list, and two files that fail and are left
under a value; the test asserts the file appears and goes with its value, the
lines follow the list, and the failing resource fails the module with the later
resource left. No integration test is added.

Builds: design 42 zfs-owns-zfs §5 (and §1 to §3), A-22
2026-11-13 00:24:20 +01:00
jschoubben 61806a7a92 H4 engine-states: the node record states the new words and forms
The node record's reads come from mesh-sdk's ManifestReads, so building with S1
and S2 states word:resources.if-config, form:config-lines and form:config-json;
the record test now asserts them.

Builds: design 42 zfs-owns-zfs §4, answer A-13
2026-11-13 00:22:48 +01:00
jschoubben 61c42f9fbc H3 engine-config-lines, H3b engine-config-json: a template writes lines from a config value and the config as JSON
`${config-lines:<key>:<prefix>[:mib-bytes]}` in a file's content writes one line
per entry (nothing for an unset value or empty list; a newline in an entry
refuses the composition); `${config-json}` writes the module's set config values
as one JSON object. Both join the sweep's table of fillers. Their rows join the
config table test (A-22).

Builds: design 42 zfs-owns-zfs §3, §9; answers A-15, A-24, A-22
2026-11-13 00:22:22 +01:00
jschoubben 86048f0ed9 H2b engine-found-handed: the found state of a unit passes to the record that stays
When one record of a unit goes while another still declares it, what the going
record found of the unit is kept for the staying one when that has no finding of
its own, so the module's leave gives back the state found before the mesh. The
preview test and the new case are one test (A-22).

Builds: design 42 zfs-owns-zfs §2, answers A-21, A-22
2026-11-13 00:20:59 +01:00
jschoubben 039415cdd9 H2 engine-if-config: a resource is declared only while a config value holds
The composer leaves out each resource whose `if-config` holds none of its
conditions against the module's config values (with defaults), and drops the key
from the rest, as the node-engine parses strictly. A resource that leaves is
removed by the existing orphan removal. One table test replaces the two tests of
setting_into_test.go.

Builds: design 42 zfs-owns-zfs §2, A-21 (declaration side), A-22
2026-11-13 00:19:49 +01:00
jschoubben fb492da7e7 H1 engine-stop-at-failure: a module's apply stops at its first failed resource
A failed resource of a module gates the rest of that module, which is reported
skipped with the failed resource named; the module's result names each one left.
A resource of no module, and other modules, go on as before. One table test
replaces two in apply_test.go; two fixtures adapted to the rule.

Builds: design 42 zfs-owns-zfs §1 (ADR 0679 §16 (a)), A-22
2026-11-13 00:19:49 +01:00
jschoubben a12417fc13 Remove the real sleep from the recovery integration test
mesh/delivery-group group feat/47-G-join11 checking: 7 of 7 member(s) ready
mesh/delivery closed: a newer head of the same pull request
mesh/check checking a12417fc13c7 on 0193949efe9f | novox/mesh-host#226 in group feat/47-G-join11
Builds: design 47 G (playbook 07 step 2: no real sleeping in a test)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 23:44:02 +01:00
jschoubben 8656e57eb9 Test 36: the recovery wiring is driven through the node agent by a revert want of db
Builds: design 47 G §296, §297, §554, §560, A-113, A-320

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 23:39:39 +01:00
jschoubben 650910dc9e R3-3: the node's own composer has no path without an entry call; its tests compose from the catalog half
Builds: design 47 G §628, §421, A-312, A-317

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 23:33:31 +01:00
jschoubben 321a0744d8 H9 test: entry-again takes a shadow's not-found entry in
Builds: design 47 G §698; lead answer A-319

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 23:13:57 +01:00
jschoubben 860ccd7e30 H9: an entry asked again by hand is asked again in the shadow's cache too, so a candidate waits no longer on a not-found it holds
Builds: design 47 G §698 (entry-again), §421 (A-308: the shadow reads the entries its wants name)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 23:12:41 +01:00
jschoubben 23799a30ed R3-1, R3-2, R3-3: the node-engine reads no build of the builder; the shadow builds from the entry it holds
The composer's Build is made from the catalog half's entry it holds and the parts of the want that names it (buildOf); the
inputs of the indexer's modules and the builder's builds, takeBuildDone, takeModule and the builder read are gone, and mesh-host
names no BuilderBuildsBucket. The shadow takes the entries of the builds it composes from (askBuildsEntries), builds a module a
candidate adds from its entry and the candidate's `added` parts (A-316), and holds, with the reason, a module whose entry
has no part given, whose fingerprint the verb module does not give, or for which it errors (A-312). Its composer is given the
entry call as the node's own is. Rows: a built module added with and without parts, the verb module answering no fingerprint
and erroring.

Builds: design 47 G §421, §628, A-308, A-312, A-316
2026-11-12 23:05:06 +01:00
jschoubben e558510a54 H5 (review r2, A-308, A-312): the shadow watches no indexer modules; it reads the entries its wants name and asks the verb module for a module a candidate adds
Builds: design 47 G §421, §628
2026-11-12 22:28:15 +01:00
jschoubben 1d409cd852 H3 (review r2): the entries test waits on the refusal the composer says until the ghost's reason is entry-missing
Builds: design 47 G §421, §614 (test only)
2026-11-12 22:24:38 +01:00
jschoubben 302dcc098f H4 (review r2, A-307): the node-engine's manifest names no uses; its invoke of seat:mesh-grants.withheld is the edge
Builds: design 47 G §635
2026-11-12 22:16:40 +01:00
jschoubben 3167f0104f H1 (review r2): the foundation host is the entry that claims the seat node-engine, not a module named mesh-host
Builds: design 47 G §527, §634
2026-11-12 22:16:27 +01:00
mesh-tools 0193949efe Merge pull request 'The own resolver's test reads the options from the catalog's template (issue 1748, ADR 0678)' (#225) from fix/1748-resolver-wait into main 2026-11-12 17:52:54 +00:00
jschoubben 8dc186944e fix-1748-host-test: the own resolver's test reads the options from the catalogue's template
mesh/delivery-group group fix/1748-resolver-wait merged: every member is merged
mesh/delivery merged: source.main moved to novox/mesh-host@0193949efe9f
mesh/check checking 8dc186944e7d on 0193949efe9f | novox/mesh-host#225 in group fix/1748-resolver-wait
TestTheCataloguesResolverComposesOnAMachine takes the options line from the systemd-resolved template beside it instead of the literal timeout:1 attempts:2 edns0, so it passes against the catalogue's main before and after the catalogue's change. Captured copies are left as they are.

Builds: design 50 resolver-wait §3 (and §2 of the main file, line 73)
2026-11-12 18:10:28 +01:00
jschoubben 905265a56b J25 A-293: a placed module's failed newest build said as refused in by-entries mode, on the builder's build-ended event
Reading the catalog half by fingerprint, the node-engine no longer reads the builder's builds, so #224's refused key was
never written there and a reader waited on a build that will never be done. The engine now hears the builder's
build.ended stream, read whole at start then followed (as the assigner hears it); the last trunk end of a placed module
that ended failed is written as the refused key with the event's words, a later done end deletes it, an end naming a
branch moves nothing. The manifest consumes mesh-delivery-builder.build.ended.**.

Builds: design 47 G A-293 (with §421, §628), ADR 0593 §6, issue 1966 U1/U8

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 17:20:30 +01:00
jschoubben 4ef07521eb Merge origin/main into the join: a placed module's failed build said as refused
Brings mesh-host #224 (issue 1966): newestFPs returns a placed module whose
newest build failed, composedValues writes its refused key at that fingerprint,
takeBuildDone starts the pass on a failed build. Conflict in newestFPs settled
by keeping G's by-entries branch first (design 47 G §421, §628: the node reads
neither the indexer's modules nor the builder's builds; it returns no failed),
and main's failed-build path in the other.

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 17:00:32 +01:00
jschoubben a9b9709911 J23 A-292: a node seat event the bus did not take is said again on the same link
After a restart, g14's engine restated its reads three seconds after its
first, bootstrapped statement, and the assigner never heard that event: a
say that failed past the bus's retry bound waited for the next link. Now a
say the bus did not take (no responders, a deadline) is said again on the
link until the bus takes it, off the caller, each failed attempt written to
the engine's journal with the event's name; only a lost link, or a refusal
the bus answered, keeps it for the next link.

Builds: design 47 G §590, A-292
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 16:57:36 +01:00