Commit Graph
337 Commits
Author SHA1 Message Date
mesh-tools 75be8acbad Merge pull request 'Build seats yield: the decoder, buildcpu and NodeLimits (issue 1806, ADR 0553/0554, change A)' (#94) from fix/build-cpu-yields into main 2026-10-26 14:22:47 +00:00
jschoubben fd7d93333e Keep a failed toolchain probe a loud refusal; only the command directory is fixed (issue 734, review round 1)
mesh/delivery merged: source.main moved to novox/mesh-sdk@d103ed88ab17
mesh/delivery-group group fix/daemon-dir-kept merged: every member is merged
mesh/check checking fd7d93333e6b on 21252194d530 | novox/mesh-sdk#93 in group fix/daemon-dir-kept
Declining an ask whose probe failed would turn a truly broken toolchain into a
wait of up to the decline bound that then fails with an empty "needs a node
with" reason. With the node-engine no longer deleting a running build and
commands started in a directory that exists, the getwd cause is gone, so the
guard's refusal stays as it was. The decline refactor goes with it.
2026-10-26 15:11:04 +01:00
jschoubben 36061b5c56 Start commands in a directory that exists, and decline an ask whose toolchain could not be read (issue 734)
A build module runs in the directory its build was unpacked in. When the
node-engine replaced that build, every command the module started inherited a
deleted working directory, and the toolchain probe's failure became a final
refusal of the check that asked it (three checks on ace on 2026-10-26).

forgekey: ExecRunnerClean starts its command in the HOME it is given, or the
temporary directory; the other runners keep the process's directory while it
exists and use the temporary directory once it is gone.

buildstep: a guard whose probe failed answers a Refusal with Again set; the
puller declines such an ask (delivery.StepDeclined) so its asker puts it again
for another node or a later pull, instead of ending it refused. A version
mismatch stays a refusal.
2026-10-26 15:01:57 +01:00
jschoubben 17b7cc480d Merge main into fix/build-cpu-yields
mesh/delivery merged: source.main moved to novox/mesh-sdk@75be8acbad4d
mesh/delivery-group group fix/build-cpu-yields merged: every member is merged
mesh/check checking 17b7cc480d6d on d103ed88ab17 | novox/mesh-sdk#94 in group fix/build-cpu-yields
2026-10-26 14:46:01 +01:00
mesh-tools 21252194d5 Merge pull request 'A check ask can name needs and packages, and a node that lacks a need declines it (issue 890, ADR 0552)' (#91) from feat/890-placement into main 2026-10-26 13:40:35 +00:00
mesh-tools 545a0c9cb4 Merge pull request 'An asker of the build seats hears when its ask is given up, and the seat's users may read their own dead letters (issue 734)' (#92) from fix/734-lost-ask into main 2026-10-26 13:40:30 +00:00
jschoubben 527d87613e Without the composer's share a step has no thread limit; with it, the share over the module's own capacity (ADR 0553 round 2)
Review round 2: change A must not slow builds before any slice exists, so a
module given no share sets no GOMAXPROCS. The slice's quota bounds the total;
threads only size one step, from the node's share over the module's own
capacity (shanks: 24 x 50% / 3 = 4). The share is the node-engine's own config.
2026-10-26 14:39:39 +01:00
jschoubben 36c92861d2 Merge main into fix/build-cpu-yields 2026-10-26 14:22:26 +01:00
jschoubben 852ba81c5a Build seats yield to one shared slice per node; the composer half waits for the engines (ADR 0553, issue 1806)
Review round 1: the decoder type ships before any composer can emit it, so a
routine SDK bump never sends yields to an engine that refuses them; the
composer half moves to a later change. Every yielding process of a node goes
into one top-level slice, meshyield.slice, so the three build modules share one
quota instead of taking one each; the share is one node value, node-tools'
build-cpu-share, and each step's threads divide it by every build slot of the
node (words from the composer, a conservative fallback without them).
2026-10-26 14:20:05 +01:00
jschoubben 3203358c4c buildrun: an asker keeps whether it hears DEAD_LETTERS, and its health tool answers from it (issue 734, review round 1)
mesh/delivery merged: source.main moved to novox/mesh-sdk@545a0c9cb43b
mesh/delivery-group group fix/734-lost-ask merged: every member is merged
mesh/check being carried to 3203358c4cb5 on 2aae6f5580f2 | from mesh-delivery c8b26735c8cf | novox/mesh-sdk#92 in group fix/734-lost-ask
A deaf asker must be loud: SayHearing keeps each Hearing in the asker's state hearing, and HearingHealth answers the
module's health tool from it, unhealthy with why while DEAD_LETTERS is not heard.
2026-10-26 14:18:41 +01:00
jschoubben 9b01c94a9e buildrun: the asker hears DEAD_LETTERS again on its next reconnect or ask, in the background, and says whether it hears; buscompose grants only what a pulled ordered consumer needs (issue 734, review round 1)
An asker started before its grant was composed was refused once and stayed deaf until it started again, silently.
DeadLetterHearing tries in the background and again on an event (Again: a reconnect, an ask waited for while deaf), and
tells each change of Hearing so the module's health tool can say it. DeadLetterReads drops the flow control a pull
consumer does not use and deletes one consumer by name.
2026-10-26 14:16:58 +01:00
jschoubben 33395f68b3 Merge main (issue 1890's step records) into feat/890-placement: a decline begins its step with what it is
mesh/delivery merged: source.main moved to novox/mesh-sdk@21252194d530
mesh/delivery-group group feat/890-placement merged: every member is merged
mesh/check checking 33395f68b309 on 2aae6f5580f2 | novox/mesh-sdk#91 in group feat/890-placement
2026-10-26 14:05:30 +01:00
jschoubben 45564e69a6 A placed check ask names the packages it tests, so the first ask's other packages are not run or counted again (issue 890, ADR 0552 §4; review round 1) 2026-10-26 14:01:05 +01:00
jschoubben d96b9fd660 buildrun: an ask the build seat gave up on ends its asker's wait as failed with why, for every asker of the build seats (issue 734)
The checker's try and the builder's lanes waited for ever on an ask given up during a bus reload: no build node writes a
step for it. LostAskWaiter hands the asker a failed step on the node bus when DEAD_LETTERS keeps the ask's copy
(HearDeadLetters, granted by DeadLetterReads), and under NotTakenBound when no node took the ask within the bound.
2026-10-26 14:00:46 +01:00
jschoubben 6b84973142 Build steps: an ask key past the step bound is ended, at start and on the module's round (issue 1890 review 2)
mesh/delivery merged: source.main moved to novox/mesh-sdk@2aae6f5580f2
mesh/delivery-group group fix/1890-left-running-steps merged: every member is merged
mesh/check pass 6b8497314297 on 2aae6f5580f2 | carried from b4cd95e7c4a4, mesh-catalog a4f5df797b68 | unit 3132 run, 3022 kept, 7 skipped | integratio…
2026-10-26 13:51:54 +01:00
jschoubben e7f0dc612e buscompose: a user of a seat pulled by kind reads its seat's dead letters, so an asker hears an ask given up (issue 734)
A check ask given up during a bus reload left its try running for ever: the asker had no grant to learn that
DEAD_LETTERS holds it. DeadLetterReads grants an ordered consumer on DEAD_LETTERS filtered to the seat stream's copies,
and nothing else of the stream.
2026-10-26 13:51:35 +01:00
jschoubben 76deab6c67 gofmt the cap's test (issue 1806) 2026-10-26 13:49:09 +01:00
jschoubben e66c6fea62 Build steps: test the ask key with a real ask id (issue 1890) 2026-10-26 13:43:05 +01:00
jschoubben 6b1c7a5613 Build steps: a restart leaves pulled asks to the stream, ends steps by compare-and-set, and step_end refuses a step this process started (issue 1890 review 1) 2026-10-26 13:42:50 +01:00
jschoubben 04902ff24a Build seats yield: idle priority and a per-node processor cap on their runtimes, threads per step within it (issue 1806)
The operator's desktop slows while it builds: the Go build runtime took 1650% of
shanks' 24 processors with nothing holding it back. A build seat's holder now
runs its runtime at idle priority under its node's cpu-cap share (declaration
Yields, composed from Seat.Yields and the module's cpu-cap setting), and
buildcpu/buildstep give each step its part of that cap in threads.
2026-10-26 13:37:00 +01:00
jschoubben bb71310337 Let a check ask name what its node needs, decline it where that is lacking, and leave unmet packages to such a node (issue 890)
A check ask's packages may need something only some build nodes have (mesh-build-node's integration tests read the
Node build module's installed toolchain). An ask now names its Needs; a holder lacking one ends the ask's step refused
as declined and drops the message for its asker to put again, skipping that worker until its next pull wait; a level's
packages whose declared needs the node lacks are left untested and named unmet, for the asker to place.
2026-10-26 13:35:12 +01:00
jschoubben 18e0a01c54 Build steps: EndStaleAtStart waits for the runtime and ends the left-running steps (issue 1890) 2026-10-26 13:25:58 +01:00
jschoubben ddafe1b1aa Build steps: a step records what it is, a restart ends the steps it left running, a tool ends one (issue 1890) 2026-10-26 13:25:13 +01:00
mesh-tools b4cd95e7c4 Merge pull request 'Take-over of the bus's user list by the grants part: sealed list layout, followed-list have, handed hashes, named readers (issue 1945)' (#89) from feat/1945-user-list-switch into main 2026-10-26 12:21:39 +00:00
jschoubben c2a410bb63 The have's user list says why the bus node does not watch the grants part's key (review of the switch, issue 1945)
mesh/delivery merged: source.main moved to novox/mesh-sdk@b4cd95e7c4a4
mesh/delivery-group group feat/1945-user-list-switch merged: every member is merged
mesh/check checking c2a410bb6376 on 89c5838176b2 | novox/mesh-sdk#89 in group feat/1945-user-list-switch
2026-10-26 12:25:27 +01:00
jschoubben af66adc180 Bring in the adoption handing (fix/1945-adopt-hashes 17478994): the have's user list carries Handed beside Follows and Key; a state that names readers is not Readable to a composer that knows no readers (review of the switch, issue 1945) 2026-10-26 12:23:07 +01:00
jschoubben 9c61918ac3 delivery: a turn never holds the check of heads that wait in it, so two approved merges in one turn never hold each other's carry (ADR 0548 §3; issue 568, review round 1)
mesh/delivery merged: source.main moved to novox/mesh-sdk@89c5838176b2
mesh/delivery-group group feat/548-one-check merged: every member is merged
mesh/check checking 9c61918ac36f on 85fa8511c4b3 | novox/mesh-sdk#88 in group feat/548-one-check
2026-10-26 12:12:53 +01:00
jschoubben 83ee7769d8 Make the grants part's state in the switch's test as the part makes a declared state, by its stream (issue 1945) 2026-10-26 12:00:29 +01:00
jschoubben 175a5567b8 The bus node follows the grants part's sealed user list, and only the grants part reads the vault's hashes (ADR 0547 clauses 7, 9; issue 1945)
The grants part writes its list as an object of its own bucket, sealed to the bus node's key, and names it in
user-list.sealed; the node-engine of the node holding mesh-broker is granted that one key and an object by name, the
grants part its bucket (the controller's copy carries the same grants, byte for byte, golden regenerated there).
A state may name its readers (announce, NamedReaders): mesh-vault.bus-accounts is read by mesh-grants alone.
2026-10-26 11:53:11 +01:00
jschoubben 1747899419 delivery: the bus node hands the loaded list's hashes to the vault, sealed, in the bus module's have (ADR 0547 clause 9, issue 1945)
The vault holds no value for the controller's own account, the node accounts and the enrolled devices, so it cannot
hash them; the one part that has their hashes is the node-engine of the bus node, in the list the bus loaded. The have
of the bus module now carries that list's hashes sealed to the vault's key (UserListWritten.Handed), which every reader
of the have bucket sees as a count, a list digest and a key fingerprint, and only the vault opens. HandedAccountsOf
reads them from a composed list, HandAccounts seals, OpenHanded opens and refuses a box sealed to another key.
2026-10-26 11:28:38 +01:00
jschoubben f5c233a928 delivery: a verdict names the siblings it read and the commits it was carried from, and a composition its fingerprint set, so a trunk or sibling move is carried, not judged again (ADR 0548 §5, §6; issue 568 S2) 2026-10-26 11:25:07 +01:00
jschoubben 0abe480dba delivery: the gate's merge turns, so the checker holds new checks off a repository an approved merge waits on (ADR 0548 §3, §4; issue 568 S1) 2026-10-26 11:24:06 +01:00
jschoubben 79649c3fa9 Run every test step of a check's level after a failure, and let a verdict name every failed test (issue 1921)
mesh/delivery merged: source.main moved to novox/mesh-sdk@85fa8511c4b3
mesh/delivery-group group fix/1921-every-failure-r merged: every member is merged
mesh/check checking 79649c3fa9f4 on c3544b6a17a2 | novox/mesh-sdk#87 in group fix/1921-every-failure-r
A check ask stopped its level at the first test step that did not pass,
so the packages after the changed ones never ran and a check found one
fault per round. Every step now runs; the level still fails, and no
later level runs (ADR 0484 clause 13). The verdict gains Failures, every
failed test with its first line, at most MaxFailures and a count.
2026-10-26 10:57:02 +01:00
jschoubben 136fa8b2a8 Say how a declared bucket and module stream are created, so the grants part makes the same stream the controller does
mesh/delivery merged: source.main moved to novox/mesh-sdk@c3544b6a17a2
mesh/delivery-group group fix/1790-grants-create-buckets merged: every member is merged
mesh/check checking 136fa8b2a896 on 095d41fa3e81 | novox/mesh-sdk#86 in group fix/1790-grants-create-buckets
BucketStreamConfig and ModuleStreamConfig are the stream configurations the controller's
CreateOrUpdateKeyValue (BucketConfig) and StreamConfigFor assert at its start, pure, for the
grants part's create of a bucket or stream a manifest declares and the bus lacks. A log already
has Log.Config. Never used for an update.

Issue: 1790
2026-10-26 09:04:43 +01:00
jschoubben 8f6ee2fd18 Let the grants part create the streams manifests declare, so a new module state no longer waits for a controller start
The copied bus composer grants mesh-grants' principal $JS.API.STREAM.CREATE.<stream> and
$JS.API.STREAM.INFO.<stream> for each bucket (KV_<module>_<state>), log (LOG_<module>_<log>) and
module stream a placed module's manifest declares, each by its own name: NATS wildcards are whole
tokens, so nothing narrower than an exact name exists and nothing wider is granted. The writers
table lets the grants part create, never update or delete. The operator allowed this one grant
on 2026-10-26 08:54, beside the memberships grant of 22:31 (ADR 0547 clause 1 and 5).

The expected lists were written again by the controller's own internal/broker with the same
change (mesh-control fix/1790-grants-create-buckets); a fixture now places mesh-grants beside the
builder's state, log and stream.

Issue: 1790
2026-10-26 09:01:32 +01:00
jschoubben 095d41fa3e Merge pull request 'buscompose: the controller's bus composer in the SDK, byte-equal to the controller's, with the mesh-grants grant (ADR 0547)' (#85) from fix/1945-memberships into main
Reviewed-on: #85
2026-10-25 22:43:57 +00:00
jschoubben b7b5c6ef27 The copied composer grants the grants part reading every membership, and its memberships are held to the controller's too (ADR 0547 §5, review of mesh-grants)
mesh/delivery merged: source.main moved to novox/mesh-sdk@095d41fa3e81
mesh/delivery-group group fix/1945-memberships merged: every member is merged
mesh/check failed: integration: github.com/novox/mesh-controller/cmd/mesh-controller.TestTheDeclarationsStreamGoesOnceEveryNodeRunsTheNewEngine: with …
The grants part reads each module's last membership before it publishes, so it needs the direct get on every
membership subject, as the controller's composer now grants (fix/1945-mesh-grants-publishes-memberships). The equality
test compares every membership the fixtures issue with the controller's MembershipFor, written by its own code.
2026-10-25 22:50:21 +01:00
mesh-tools 7ac4493ce9 Merge pull request 'process: open a part's buckets as essential (stop, naming the grant) or optional (log what is not heard, keep running) (issue 1945)' (#84) from fix/1945-ask-again-and-why into main 2026-10-25 21:43:08 +00:00
jschoubben 881da7f599 The copied bus composer grants the grants part publishing memberships, as the controller's does (ADR 0547 §5)
The same line as mesh-controller f67d1553; the fixtures name a mesh-grants module and the lists were written again by
the controller's own composer on that branch, so the copy is held byte-equal to it.
2026-10-25 22:33:20 +01:00
jschoubben e0d1de0475 process: a part opens what it cannot work without as essential, the rest as optional (issue 1945)
mesh/delivery merged: source.main moved to novox/mesh-sdk@7ac4493ce9ab
mesh/delivery-group group fix/1945-ask-again-and-why merged: every member is merged
mesh/check pass e0d1de047585 on 3db051be57ca | unit 1642 run, 752 kept, 2 skipped | integration 1710 run, 803 kept, 2 skipped | over bound: unit, inte…
The delivery builder exited twice on 2026-10-25 for a bucket the bus
refused at start, though it could build without it: a grant reaches the
bus with a delivery, so a refusal at start is often only a grant not
there yet. EssentialBucket stops a start naming the bucket and its grant;
OptionalBucket and NotWatched say what is not heard and go on. Here so
the seven delivery parts take one copy, not seven.
2026-10-25 22:12:20 +01:00
jschoubben b5f8604d03 Copy the controller's bus composer into mesh-sdk as buscompose, held byte for byte to the controller's lists (issue 1377, ADR 0547 §5)
The grants part composes the bus's user list with the composer the controller uses (ADR 0547 §5), copied rather than
moved: the controller is not changed and keeps its own until it loses the lease. Left out: the bus client, the broker's
address from the environment and the client's two methods; the lease's two subjects are copied in.

One table test holds the copy to four meshes' lists and the server golden, written by the controller's own
internal/broker from the same fixtures (testdata/controller-user-lists.json).
2026-10-25 21:54:58 +01:00
jschoubben 0be0ffd615 Compare the have's user list by what it says, so a have is not written again for the same list (ADR 0547 §8)
mesh/delivery merged: source.main moved to novox/mesh-sdk@3db051be57ca
mesh/delivery-group group fix/1945-user-list-follows merged: every member is merged
mesh/check pass 0be0ffd6159e on db26f7e8b6f5 | unit 1467 run, 1696 kept, 7 skipped | integration 1471 run, 1740 kept, 6 skipped | over bound: unit, in…
2026-10-25 21:47:30 +01:00
jschoubben f63fa86c74 Say in the bus module's have the user list its node wrote alone, so the bridge of ADR 0547 §8 reports what the bus loads (issue 1945) 2026-10-25 21:42:13 +01:00
jschoubben d4ae4b6790 Say the put's tools, not a seat's verbs, in the put's words (ADR 0546)
mesh/delivery merged: source.main moved to novox/mesh-sdk@db26f7e8b6f5
mesh/check pass d4ae4b6790b8 on db26f7e8b6f5 | unit 1102 run, 680 kept, 2 skipped | integration 1103 run, 722 kept, 2 skipped | over bound: unit (slow…
2026-10-25 21:08:10 +01:00
jschoubben 2f6741b7f8 No seat and no compose verb for the put: the controller takes no change, so the assigner serves the put as its own tools from compositions the controller already writes (ADR 0546) 2026-10-25 21:07:58 +01:00
jschoubben 20a6dc4920 A grant says whether it names a module's accounts on a node, for the assigner and the compose verb (ADR 0546) 2026-10-25 20:50:51 +01:00
jschoubben c4f37c279a The assigner's seat with put, release and puts, the controller's compose verb, a put's key and value, and the view answers no option that authorises (ADR 0546) 2026-10-25 20:49:29 +01:00
jschoubben 518dae5ec9 Pull as not paused when the pause cannot be read, and say so once, instead of ending the puller silently
mesh/delivery held for a person: merged without a passing check: only a person decides what follows
mesh/delivery-group group fix/steps-bucket-ages held: a member is held: novox/mesh-sdk@518dae5ec9c8 (merged without a passing check: only a person decides what…
mesh/check checking 518dae5ec9c8 on 7f89cb51de15 | novox/mesh-sdk#81 in group fix/steps-bucket-ages
A Load error ended Pull, and the modules run it in a goroutine whose error nobody reads: every
build node would stop pulling when the new pauses state could not be read (or when Pull reached
restore before Serve). A node kept from building is held by capacity 0, not by a pause.
2026-10-25 18:56:17 +01:00
jschoubben e8f8ef63fa Keep a build node's pause in a pauses state of its own, so the steps state can drop its old keys without ending a pause
The steps state of the build modules filled to the bus's 64 MiB cap (27,370 keys of mesh-build-go,
none ever removed) and refused a passed check's end. Its keys get an age in the catalogue; the
pause key lived among them and would then have ended by itself. Puller.PausePut writes it to
delivery.PausesState instead; Put stays the fallback for a module not yet moved.
2026-10-25 18:51:04 +01:00
jschoubben a8fbabbfc3 Fold the puller-wake and Served tests into existing tests to stay within the test budget (issue 1887)
mesh/delivery merged: source.main moved to novox/mesh-sdk@4f4d5ca9c956
mesh/delivery-group group fix/puller-wakes-on-capacity merged: every member is merged
mesh/check pass a8fbabbfc373 on 4f4d5ca9c956 | unit 1102 run, 679 kept, 2 skipped | integration 1103 run, 721 kept, 2 skipped | over bound: unit, inte…
2026-10-25 17:44:03 +01:00