Commit Graph
3303 Commits
Author SHA1 Message Date
jschoubben 22b92b827d BM publish version: the build modules' bundle asks name the ask's version, as the SDK now refuses one without
A bundle publishes under the version its ask reserved, and the delivery SDK's
publish refuses an empty one loudly; the tests' bundle asks carry a version, a
bundle asked with none is shown refused, and an archive still publishes by its
digest alone.

Builds: design 47 H2 H-A-373, H-A-373 addendum
2026-11-22 01:50:30 +01:00
jschoubben d3d9ff5183 M1: the build modules and the registry say their class and name every store's cap; detected-steps is pruned
Each of the four is core. Every state that named no cap names one, from its
live fill read 2026-11-22 01:4x CET (mesh-broker show, nats_streams on novox):
mesh-build-go steps 16.0 MiB (16,783,695 bytes) -> 128 MiB; mesh-build-node
steps 2.35 MiB (2,463,833 bytes) -> 16 MiB; every other state under 2 MiB ->
8 MiB (largest: distribution holds 1,844,287 bytes, mesh-build-go
detected-steps 868,554, distribution transfers 846,359, findings 668,161, idle
401,736, mesh-build-go pauses 254,366). The Go build module's finding-events is
8 MiB for 7 days. The objects stores already name their caps; none keeps a log.
The Go and Node build modules prune their state detected-steps (each key an
artifact's steps as detected, detected again when gone) at their own writes,
past prune-mark 70 % down to prune-target 50 %, and serve
<module>_state_prune_now and <module>_state_keep; no other state of the four is
prunable (each is truth: running steps, pauses, keys, toolchains, findings,
holds, transfers, windows, mirrors, taken asks).

Builds: design 47 H2 §812, §820; H-A-411 (1), (2), (3), H-A-437
2026-11-22 01:39:49 +01:00
jschoubben 1697b7041c BM4: each build module keeps a run's test output in its outputs objects
The Go build module's level answers its text in output (test2json's Output
lines), the Node build module's what its commands wrote; the delivery SDK's step
run keeps it as <node>.<ask> in the module's outputs objects, which each module
asks for in requires (1 GiB, kept 7 days) and reads through the verb output at
form 2. Each module's kit test runs the SDK's SeatVerbs.

Builds: design 47 H2 §791, §820; H-A-412 (1), H-A-412 second addendum (a), H-A-421, H-A-434
2026-11-22 01:32:40 +01:00
jschoubben 46990669de BM4: the bus module's keeper keeps an objects store's objects for its max-age-days
The objects template takes max-age-days (absent keeps an object for good): the
keeper plans it as the object store's max age, which the bus applies as the
store's TTL, so a build module's outputs objects go at their seven days.

Builds: design 47 H2 §820 (outputs: 7 days, the store's max-age), §791; H-A-434
2026-11-22 01:32:03 +01:00
jschoubben e335e859d5 BM3: the Node build module installs a module's npm tree and publishes it as <module>:node-modules
An install ask's body runs npm ci from the lock with no install script and no
link, packs the tree alone under modules/<module>/node_modules, publishes it to
the artifact store as the archive <module>:node-modules under the ask's version
and answers its reference, which the step's end carries. The package.json npm
ci needs is written from the lock's root entry, a lock without one refused. The
integration level's setup bound in code goes: half of the level's bound, which
each level's script gets as MESH_TEST_LEVEL_BOUND.

Builds: design 47 H2 §800; H-A-399 (2), H-A-403, H-A-427
2026-11-22 01:24:44 +01:00
jschoubben 1d8839640e BM4b: the build modules keep an oversized end in steps-objects, read by the verb answer
The delivery SDK replaced the output chunks with the verb answer over the
steps state's objects and output at form 2: each build module keeps an end
over the steps state's value with KeepAnswerIn, serves AnswerTool and
OutputTool over its steps state, and its steps name their outputs objects.

Builds: design 47 H2 §791, H-A-412 addendum, H-A-412 second addendum (b), H-A-399 (1)
2026-11-22 01:19:00 +01:00
jschoubben 7519191861 BM5: the steps read tools list an ask's steps under its asker, so they are never empty while asks run
Each build module's steps read tool listed only `<node>.<id>` keys and so
skipped every ask's step, kept as `<asker>.<node>.<id>`: it now reads every key
of its node, newest id first. The Go build module's test of capacity 0 also
holds the seat's current at form 2: no ask taken, why not capacity 0, then
paused.

Builds: design 47 H2 §801; H-A-399 (1)
2026-11-22 01:10:54 +01:00
jschoubben c63b4ae31d BM2: a test that cannot run says NOT RUN; the level's bound and the install archive reach the tests
The Go build module's level result carries not-run (the SDK reader's NOT RUN
skips), its tests get MESH_TEST_LEVEL_BOUND, the level's bound, from which
mesh-sdk's helpers take the setup bound; a test call naming an install archive
reads it from the artifact store by its reference and unpacks it into the
workspace's tree before go test. check-needs is read from the module's
manifest, on the Go artifact whose path holds the package, and the marker
file goes: the Node build module's manifest declares it on its bundle.

Builds: design 47 H2 §800; H-A-294, H-A-350 DMC-6, H-A-368, H-A-399 (1) (2) (3)
2026-11-22 01:07:45 +01:00
jschoubben 681de09f04 RG2: the registry's list pages by module and cursor; the sweep reads it page by page
oci-artifact-registry's list filters by name or module while it reads and
answers at most 768 KB a page with capped, total and the cursor next, in
order of the names; container-image-registry's answers whole as its form
says. The registry takes a removal for retention whoever asks it: the
refusal "not the sweep's" and its read of the sweep seat's holder go. The
sweep reads the list from its cursor until no page is left.

Builds: design 47 H2 §813 (7), §811, §779; H-A-350 DMC-10, H-A-399
2026-11-22 00:49:42 +01:00
jschoubben cb5de6fd31 MR1: the recovery module's whole test waits its start by the setup bound
The module's start is the test's setup: its two waits on the bus take
testbus.SetupBound (half of MESH_TEST_LEVEL_BOUND) and say a starved setup
in the one line testbus.Starved writes; its 30 s values go.

Builds: design 47 H2 §800; H-A-327, H-A-368
2026-11-22 00:43:38 +01:00
jschoubben 5d2d0bade8 SC1: the self-check raises over-budget from the gate's times
gate_times answers per class its newest delivery (last) with whether it was
over its budget; the self-check's probe D-budget reads gate_times and raises
over-budget with the class as its id, naming the source revision and its
longest phase, and a newest delivery within its budget clears it.

Builds: design 47 H2 §810; H-A-396
2026-11-22 00:37:03 +01:00
jschoubben 872a02b378 RG1: a repeated publish ask reads the store's held version in its new form
The merge of main's H-A-387/H-A-393 fix gave Store.Held a reference and the
digest another publish holds; a repeat answers ended when the reference is set.

Builds: design 47 H2 §775; H-A-381
2026-11-22 00:33:39 +01:00
jschoubben 83012ad97b Merge remote-tracking branch 'origin/main' into feat/47-H2 2026-11-22 00:31:45 +01:00
jschoubben 607a1d76ea GT2: a delivery record keeps its delivery time; gate_times judges it
The head entry keeps merge-called from its opening and the pipeline's phases
from its end; a record whose every mover runs ends with delivery-time (from
merge-called to the last running of its movers) and class (core when a mover's
catalog entry, read once at its entry's opening, says core). gate_times answers
each class's budget (budget-core 10m, budget-leaf 3m), the last seven days'
median and worst, and every delivery over its budget with its longest phase.

Builds: design 47 H2 §810, §785; H-A-394
2026-11-22 00:30:48 +01:00
jschoubben 41610e903f BM1: the build modules give a step met away again by again-delays
Each build module declares again-delays (15s,60s) and gives its puller the
delays steprun.AgainDelays reads from it; a publish's no responders is a
transport fault by steprun.PublishFault with no seat call to learn why, so
the modules no longer invoke the assigner's assignments.

Builds: design 47 H2 §799; H-A-350 DMC-7, H-A-390
2026-11-22 00:27:04 +01:00
jschoubben eac56f9990 GT1: the gate's whole test over a real bus reads the mover's entry
The test bus serves the ledger's query the movers index answers, and the
take-in's mover is looked for in its own entry, the head counting it.

Builds: design 47 H2 §809; H-A-391
2026-11-22 00:22:05 +01:00
jschoubben e866a64be3 GT4: the gate watches a source-only registration's branch as any
The gate reads the registrations page by page from the cursor each page
answers, so a source-only registration (no module) past the first page is
watched as any branch; its merge's take-in with no movers ends the record
nothing to deliver as before.

Builds: design 47 H2 §815, §806, §779
2026-11-22 00:21:06 +01:00
jschoubben 6f744d6602 GT1: the gate's ledger keeps one entry per module and revision
A record's head entry record/<branch.source> keeps the record-wide fields and
the count of its movers; each mover, the catalog half's too, has its entry
mover/<branch.source.module> with its pipeline, version and step outcomes per
node, found through the index movers on its record's key. An entry over
64 KiB is refused with record-too-large, cleared when written within it or
at the record's end; the ledger is capped at 1 GiB.

Builds: design 47 H2 §809, §820; H-A-385, H-A-391
2026-11-22 00:19:16 +01:00
jschoubben 7765947476 The artifact registry refuses a held version published at another digest (H-A-393)
While the reserved digest is still empty, a second publish of the same
version with another digest passed the reservation check and the store's
Put moved the version's tag. Publish now refuses it in its held branch,
before the reservation is asked, naming both digests, with the publish's
finding and refused said as any refusal; the same digest is taken as before.
The store's Held answers the digest it holds at the version when it differs.

Builds: design 47 H H-A-393, design 47 G A-490j
2026-11-22 00:18:20 +01:00
jschoubben 410451a2a2 GT3: the gate's deliveries filters while it reads and pages at 768 KB
deliveries takes module, outcome, since and the cursor next beside state,
repository and group, applies them while it reads, and answers at most
768 KB a page with capped, total and the next page's opaque cursor.

Builds: design 47 H2 §811, §779
2026-11-22 00:14:55 +01:00
jschoubben 69940d1019 The artifact registry takes a publish whose reservation names no digest yet (H-A-387)
The builder records a build's digests only at its end, while the Go build
module publishes during the build step with the reserved version (H-A-373),
so the distribution holder's strict digest check refused every archive-only
publish with "reserved ... for another digest". Recorded now takes an
artifact whose reserved digest is empty, as A-490j decided for every holder,
still refuses a different recorded digest, and its refusal names the case:
the digest reserved and the one published, or no reservation of the version.

Builds: design 47 H H-A-387, design 47 G A-490j
2026-11-22 00:14:30 +01:00
jschoubben 34f55bdf79 RG1: the bounded steps' finding test joins their one test function
Builds: design 47 H2 §819
2026-11-22 00:06:07 +01:00
jschoubben babcd1bb31 RG1: the registry takes a publish ask once by its id
A publish carrying an ask id is kept in the state taken for ask-taken-bound
(10m): a repeat while the act runs joins it, a repeat after it ended answers
the act's state read afresh (the transfer's lacking parts, a held version
ended, a transfer gone refused), never publishing twice.

Builds: design 47 H2 §775, §820; H-A-359, H-A-381
2026-11-22 00:05:58 +01:00
jschoubben d545299a71 RG3: the registry purges a deleted chunk's leftovers
Every delete of a chunk in the incoming object store is followed by the
runtime's purge of what it left (busstate.Objects.Purge), so the store's kept
descriptions equal its live objects; a failed purge is said.

Builds: design 47 H2 §813 (8); H-A-227, H-A-366
2026-11-22 00:02:56 +01:00
jschoubben 086745edd5 Merge remote-tracking branch 'origin/main' into feat/47-H2 2026-11-21 23:53:45 +01:00
jschoubben b5c05b326a Go build module: a bundle's and an archive's publish carry the version the ask reserved
Every bundle and archive was published with an empty version, a by-digest
publish, so the builder marked none against its reservation: the same kind
as the Node build module's bundles.

Builds: design 47 H H-A-373
2026-11-21 23:45:30 +01:00
jschoubben ebf4d9896b Node build module: a bundle's publish carries the version the ask reserved
Every bundle was published with an empty version, a by-digest publish, so
the builder never saw package-published and the bundle was not marked
against its reservation.

Builds: design 47 H H-A-372
2026-11-21 23:38:37 +01:00
jschoubben 02371b32de Go build module: a module directory that chooses no Go module is an empty scope
The builder sends a module's directory with every lane, and an archive is
packed by the Go build seat, so the fetch's refusal of a directory without
go.mod left every archive-only module without Go code unbuildable. Such a
directory is now an empty scope (the empty scope file written); a Go bundle
or package without go.mod still fails at its build step, naming it.

Builds: design 47 H H-A-371
2026-11-21 23:37:51 +01:00
jschoubben 0bc03bba35 §780 readers: take the form the tolerant seat decoders answer
mesh-contracts' K1 regenerated every seat package to answer (value, form, error); the catalog's four
callers take the form.

Builds: design 47 H2 §780; H-A-350 DMC-4
2026-11-21 23:20:11 +01:00
jschoubben 0206ec825e VA1: the vault takes a module's own secret through own-put and answers it through get
The forge module needs a place for its token that only its own processes read: own-put keeps it as
module/<module>/<name>, named by the caller's account, its newest generation only.

Builds: design 47 H2 §822, §808; H-A-345, H-A-349 DQ-7, H-A-360
2026-11-21 23:14:43 +01:00
jschoubben 1cbe0b1294 N1: nats_connections carries each connection's five busiest subscriptions
Also fixes the trunk race in the request test: the watch hands its handler over a channel.

Builds: design 47 H2 §813 (5); H-A-356
2026-11-21 23:07:54 +01:00
jschoubben 51c27203de N3: the keeper's approve-level bend names the research that ends it
Builds: design 47 H2 §813 (11)
2026-11-21 23:06:52 +01:00
jschoubben 1ad170705e N2: the keeper withdraws a refused drop's or recreate's ask on operator-channel
A refused request leaves nothing for the operator to answer, so its ask is cancelled in the same pass.

Builds: design 47 H2 §813 (6)
2026-11-21 23:06:43 +01:00
jschoubben c5a593eb05 N1: the bus module's nats_consumers lists consumers by delivered-rate or created
The flow per consumer is measured within one call, so a busy bus says with what.

Builds: design 47 H2 §813 (5), §819
2026-11-21 23:04:31 +01:00
mesh-tools 32e1ed6c55 Merge pull request 'The Node build module's integration tests supply the SDK the skipped fetch would place (design 47 G §571)' (#732) from fix/47-H1-build-node-tests-under-load into main 2026-11-21 21:44:31 +00:00
jschoubben 6527ebd077 mesh-build-node: the real-toolchain tests' fixture holds the SDK where the fetch places it
Since the toolchain lock pins no SDK (design 47 G §571), only the fetch puts
@novox/mesh-sdk in a module's node_modules. The three tests that build with
the real tsc and esbuild skip the fetch, so they read the SDK from an older
installed toolchain that still carried one. Once a build node installed the
current lock (shanks 19:26, then g14), they failed with TS2307 (main.137,
main.155). The fixture now lays a minimal SDK of its own where the fetch
would, so the tests no longer depend on which toolchains a node still keeps.

Builds: design 47 G §571
2026-11-21 22:25:38 +01:00
jschoubben 31abcad907 nftables: test that a start waits for the watch's expiry; refuse a stale node-engine package
Review round 2 (1): the start's wait for the expiries the watch handed over
had no test that failed without it; the start test gains a store whose
deletes wait, and without the wait the key is said expired twice. The note:
a node-engine package naming no verb converged refuses the start loudly
instead of asking seat:node-engine.undefined.

Builds: design 33 control-verbs §10, §11, §14; design 47 H H-A-312
2026-11-21 22:05:47 +01:00
jschoubben f589b5af5c nftables: hold the verbs until the start ends; until with seconds; the seat's own forms
Review round 1 (1)-(3): an until without seconds was accepted, which the
node-engine's RFC 3339 read refuses, withholding every later compose; the
verbs answered before started was said and before the watch's expiries
ended; and the holder kept its own copies of the seat's names, key forms
and value forms, which had drifted from H-A-324. The holder now takes them
from the seat's generated package, the tools' inputs are checked against
its input types, and every seat verb waits for the start's end.

Builds: design 33 control-verbs §2, §3, §11, §12, §14; design 47 H H-A-324, H-A-330
2026-11-21 21:42:45 +01:00
jschoubben 38ea9ce25c nftables: allow names the outward link a kept allowance opens
An allowance with no interface never opens an outward link; one named in
interfaces does, and allow now says so: a kept allowance whose interface is
in the outward links of the converged answer asked for the call answers
with outward, that link. The holder reads no shape of the loaded table to
tell an outward link from the tunnel; the event carries no outward.

Builds: design 33 control-verbs §4, §14; design 47 H H-A-324, H-A-325
2026-11-21 21:19:03 +01:00
jschoubben 09d5758343 nftables: ask converged through the node-engine seat's generated package
The holder's ask of the node-engine's converged takes the seat's name, the
verb's name and the answer's form from the node-engine seat's TypeScript
package, as the holder says its own event through its seat's package.

Builds: design 33 control-verbs §10, §12; design 47 H H-A-316, H-A-321
2026-11-21 21:04:16 +01:00
jschoubben 83482a4513 nftables: allowances per machine, refused unproven or adopted, written in the hand-act log
The security and seams reviews of the packet filter's control verbs: the
allowances state is one bucket per machine whose keys carry the node, so a
holder lists and writes its own node's only; allow and disallow refuse an
unproven caller, an adopted node and a converged that does not answer within
10 s, and are written in the hand-act log before the state is; allow refuses a
port the loaded mesh table accepts from given sources only unless a source is
given, and more than the setting allowances-max; allowed answers none on an
adopted node and asks converged again while none answered; rules marks loaded
only on an equal at and lists loaded allowance comments nobody keeps. The event
is said through the seat's generated package; the claim says it, so emits
stays without it.

Builds: design 33 control-verbs §1-§11, §14; ADR 0683; design 47 H H-A-314 (3), (5), (6), H-A-315 (1)-(3), (5), (7), H-A-316, H-A-317, H-A-318, H-A-319
2026-11-21 20:52:34 +01:00
jschoubben 7bd1e74086 nftables: serve allow, disallow and allowed, keeping allowances as its own state
The operator, 2026-11-21 20:35: a firewall module has tools to control the
firewall beside the provisioning from declarations. The holder keeps one
allowance per protocol, port and interface in its state allowances, says every
change as allowances-changed, removes an allowance at its until by a timer on
its own state (and every expired one at its start, before started), and rules
marks each allowance loaded by its comment in the mesh's table. It writes no
rule: the node-engine renders the allowances.

Builds: ADR 0683; design 33 control-verbs §1-§7, §9; design 47 H H-A-304..310, H-A-312 (2)
2026-11-21 20:40:35 +01:00
mesh-tools 553bfb0ccf Merge pull request 'The forge's whole-seat test ends NOT RUN when its npm tooling is missing (H-A-294)' (#730) from fix/47-H1-gitea-whole-test-not-run into main 2026-11-21 18:30:10 +00:00
jschoubben e2571a6468 gitea: the seat git's whole test says NOT RUN where node, npm or its npm packages are missing
The Go integration level fetches the forge module's tree without node_modules and
without network, so TestTheSeatGitWholeOverARealBus failed every pipeline carrying
the forge. It now checks its needs first and skips by name, as the Node build
module's realMachine does; with its needs present it runs whole as before.

Issue 1960 (H1).
Builds: design 47 H H-A-294
2026-11-21 19:26:31 +01:00
jschoubben 94bcc14a82 zfs: the keeper's socket test no longer races, and a refused peer is told so
The test switched the peer's uid in a plain variable the keeper's serving
goroutines read, a data race that fails it under -race; it is an atomic now.
A peer the keeper refuses could also get "broken pipe" instead of the
refusal, when the keeper closed before the request was written: the client
now reads the refusal that waits on the socket. A request larger than the
socket's buffer makes that case happen every time.

Builds: A-37 (only root opens the keeper's socket; a non-root peer is told so); design 47 H section 10 rows 165 and 167 (hq branch design/47-H1-answers-59, 81fa1ee4)
2026-11-21 19:26:14 +01:00
jschoubben 4fb3ab6df5 mesh-build-node: say a placed release outside the package file's range (H-A-276)
The latest release is placed whatever the range a package.json names
(ADR 0573 §2); one outside that range is now said in the journal, for
the SDK and every other mesh package, so a module built against a
release its range excludes is visible. A range of a form not read is
said as such.

Builds: design 47 H H-A-267, H-A-276
2026-11-21 15:47:46 +01:00
jschoubben a5976cf299 mesh-build-node: hold the member's-build precedence over the registry in a test (H-A-267)
A fetch naming a member's build of a mesh package the registry lacks
lays the build and waits on nothing; without the skip in
placeMeshPackages it waits for the registry's release.

Builds: design 47 H H-A-267
2026-11-21 15:45:10 +01:00
jschoubben 820840076f nats: the whole test waits for the first fill pass, not a first view's answer (H-A-268)
The keeper never marks a first view listing no module as answered
(A-367), so the subtest waited for what cannot come and hid every later
subtest. It waits for the first fill pass and takes it as the first.

Builds: design 47 H H-A-268
2026-11-21 15:37:46 +01:00
jschoubben 83802dc4a6 mesh-build-node: place every mesh package a module names at its latest release (H-A-267)
A trunk build put only the SDK into a module's node_modules; any other
@novox/mesh-* package its package.json names was left out of the install
and only a group or a member's build brought it, so the forge module's
build failed with "Cannot find module" for its contracts package. Each
such package is now placed at the latest release the registry holds, as
the SDK is; a group's packed package or a member's build still takes
precedence.

Builds: design 47 H H-A-267
2026-11-21 15:35:26 +01:00
jschoubben 72872b045a Design 47 H1: a build's end comes from the builder's log; an unread end or view holds the pass loudly (H-A-258, H-A-260)
A done build whose history read failed was left out of the view for the whole process, so later passes judged from an
incomplete keep set, and every build cost one history read. A build's end is now taken from the build-ended entries of
the builder's log the pass already reads once, and only a build the log lacks is read from its history. While any done
key's end is unread, or the view is not yet taken in, no pass judges: it ends with the finding artifact-builds-unread,
written once and cleared by the next pass that judges. Every change heard while a watch's handover is gathered now goes
into that handover, so a build finished between the handover and its taking in is not lost by the swap (H-A-257).

Builds: design 47 H §H-A-255 (1), §H-A-257, §H-A-258, §H-A-260
2026-11-21 12:06:18 +01:00