mesh-catalog #495 gave read on every repository to the tools' account alone. The build modules
fetch over ssh as nox-builder (the forge-account provider's builders account, ADR 0466 §3), so
ace's and g14's fetch of jschoubben/car-hunter and jschoubben/snake still failed; the machines'
agents (nox-agent, ADR 0410 §1) could not read them either.
- gitea: readall.ts grants every account of one list, meshAccounts in tools-account.ts (mesh-tools,
nox-builder, nox-agent; a Go test holds the two Go names to internal/forgeaccount's constants),
on the same start pass, forge events and tools. gitea_unreadable_repos answers per account and
names the accounts the forge does not hold yet; gitea_grant_read takes accounts as well as
repositories. An absent account is asked nothing; the start pass runs again ten minutes apart
until every account was there once.
- mesh-build-go, mesh-build-node, mesh-build-image: pin mesh-sdk ff81bdf8 (fix/fetch-reads-all),
whose forgekey.Said keeps git's words past the bare "error:" line: the mirror's line said only
"git -C: error:". Their own git reasons (ls-files, ls-remote, rev-parse, repack) go through it too.
buildah writes STEP lines on stdout and its error on stderr, so the last line of both hid every image build's cause.
mesh-build-image's reason is now stderr's last lines (600 bytes) with the exit status, and the journal gets both tails;
mesh-build-node keeps stderr's end and stdout's start apart; mesh-build-go's cause skips go build's '# package' line.
Change A rebuilds the three build modules, so their runtimes restart once; it
changes nothing else about their builds until change B gives them the share.
Review round 1: a cap per module let three build modules take three shares
of the node. The share is now one node value held by the node-engine's
yielding slice; each step divides it by every build slot of the node, from the
composer's words, or assumes every build seat at its own capacity without them.
The operator's desktop slows while it builds (shanks at 92%, the Go build
runtime at 1650% of 24 processors). The build modules declare cpu-cap
(default shanks=50,g14=50,ace=75), which the composer turns into the runtime's
quota at idle priority; each step's go, node and buildah commands get
GOMAXPROCS and UV_THREADPOOL_SIZE for their part of it, and the status tools
show the limits and what the kernel holds the runtime to.
mesh-build-node's integration tests read the Node toolchain installed only where the node holds node-build-node, and
skipped on ace: its package now declares node-build-node beside its tests; the integration digest names it, a node
lacking it leaves the package unmet for the checker to place, and declines a placed ask. Re-pins mesh-sdk feat/890-placement.
mesh/check failed: ace: the check of novox/mesh-catalog#492 refused: the installed go version could not be read: `go version` with GOTOOLCHAIN=local f…
mesh/check checking 15a5fe09b62e on c51a46b802d3 | novox/mesh-catalog#488 in group fix/1921-every-failure-r
Pins mesh-sdk fix/1921-every-failure-r, whose check ask no longer stops a level at its first failing test step. A captured fixture with a failing test in each of two packages proves the build module's test step names both.
mesh/check failed: integration: meshbuildnode/cmd/mesh-build-node.TestAFixtureBundleBuiltTwiceHasOneDigest: no toolchain of this lock is installed und…
The review found a cut read took an arbitrary 1000 keys, so the newest steps could be missing. A step id is
base-36 milliseconds plus random digits, so its string order is its start order; the newest max(limit*4, 100)
keys are read now, 1000 at most. The toolchains read logs when it drops keys past its cap.
On a steps state of 1164 keys / 20 MB (mesh-build-node: 10034 keys) the read tools listed every key and
then read each value one round trip at a time, filtering by node only after the Get, so they timed out
at 30 s. Now the keys are filtered by the node's prefix before any Get, the values are read 16 at a
time, at most 1000 a call (the answer says it was cut), and one log line gives rows read and time.
build_*_toolchains reads its values the same parallel way. The kind: a read tool that reads a whole
state key by key; every read tool of the three modules is covered.
mesh/delivery-group group fix/steps-bucket-ages held: a member is held: novox/mesh-sdk@518dae5ec9c8 (merged without a passing check: only a person decides what…
mesh/check pass 4e82a824be7a on cdbe2945cf6e | unit 1994 run, 2322 kept, 5 skipped | integration 2016 run, 2341 kept, 9 skipped | over bound: unit, in…
Without it in Kept, stdio.KeptState refuses the state before the bus (mesh-sdk go/keeps.go
refusedUnannounced), so the pause could be neither read nor written.
mesh-build-go's steps bucket filled to the bus's 64 MiB cap (27,370 keys, none ever removed) and
refused the end of a passed check, so the check read failed; mesh-build-node's (17.8 MB) and
mesh-build-image's grow the same way. A step's key is per step: its reader reads its end within
the step's bound, and a check's verdict is kept by the checker, so two days is ample. The node's
pause key lived in the same bucket and would have ended by itself; it moves to a pauses state
without an age (mesh-sdk fix/steps-bucket-ages, Puller.PausePut). No node is paused today, so
nothing is carried over.
g14's Go builder sat idle at capacity 2 because nothing woke its parked
puller. mesh-build-go, mesh-build-node and mesh-build-image now pin mesh-sdk
b9911dc1, whose puller watches the config value capacity as mesh-config
resolved it and wakes on each change, and hand it that watch.
mesh/delivery-group group fix/1901-package-cause held: a member is held: novox/mesh-catalog@155124b59af4 (merged without a passing check: only a person decides…
mesh/check pass 155124b59af4 on f27f4c8cd58c | unit 2015 run, 2883 kept, 5 skipped | integration 2037 run, 2978 kept, 9 skipped | over bound: unit, in…
An indented log line of another running test that held 'panic:' won over the
real timeout panic, since the lines were trimmed before matching. Go prints
the runtime's lines at column 0; a test's log is indented.
mesh-controller's try was worded 'PASS' again because the race detector
reported after the tests passed and the program exited 66. The cause is now
the race: its access, the frame that made it and where its goroutine was
created, instead of PASS or testing's generic 'race detected during execution
of test'. Tests from captured go test -race -json (1901/AC4).
A try of mesh-controller kept 'PASS' as its only failure: the package's test
program stopped after printing PASS and the package-level failure took its
first line. Now a failure that is not one test's assertion names the tests that
were running and carries the panic line (a timeout's with its running tests),
a signal or fatal error line, a build's first error, or words saying the
program exited during which tests or after they passed; go test's summary lines
are never kept. Tests from captured go test -json of eight real failures
(1901/AC4).
mesh/check failed: shanks: novox/mesh-catalog#473: modules/mesh-build-go/go.mod names mesh-sdk 19c1d35939bc, which is neither on mesh-sdk's trunk nor …
mesh/check failed: mesh-build-go: the check of novox/mesh-catalog#470: the ask mvlnph6fd84a7b's step on shanks did not end within 30m0s of its start |…
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1792875397682979170 (novox/mesh-catalog at 6fb0ceee into main,…
mesh/delivery covered: a later merge that contains it was delivered: novox/mesh-catalog@753e8fc202c3 (merged as 4b1bed4b into main, walk plan-17927949853…
mesh/check failed: shanks: the check of novox/mesh-catalog#465 failed: the unit level: packages is a list of package patterns | 70ef99595aa9 on e52024…
mesh/check failed: integration: mesh-delivery-gate/cmd/mesh-d… | c32474ee40f4 on 3bec357968da | judged by the candidate's mesh-build-go at 2e667ac15ce7
mesh/check failed: unit: git.novox.be/novox/mesh-catalog/chec… | d6b9cef9d005 on 3bec357968da | judged by the candidate's mesh-build-go at 769f9d906be2
mesh/check failed: unit: git.novox.be/novox/mesh-catalog/chec… | b60077850dfb on 3bec357968da | judged by the candidate's mesh-build-go at 47bcc1b5810e
mesh-tools
jschoubben