Commit Graph
3 Commits
Author SHA1 Message Date
jschoubben f311fa36dc The setuid walk is paced before each file too, so a large directory is not read blind (review of issue 433)
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
A file's own details (lstat) may be read from the disk as a directory is,
so a directory of tens of thousands of files was read without a look and
the header's promise that no burst is longer than PaceLook did not hold.
The walk now asks the pacer before each file as well; the 250 ms gate
makes that a clock read per file. A test with one directory of twenty
files fails without it. The clock is read once after a run of pauses,
and the constants test says that it guards the constants only.
2026-10-11 01:55:39 +02:00
jschoubben 30242197fa The paced walk keeps a tenth of the time, and reads the pressure since its last look (review of issue 433)
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
The review of #79 found that pacing alone brought back issue 361's
fault. The control-node's own I/O pressure sits near the 10 % bound with
no walk running, so a walk that paused 30 s for every 250 ms it read
would read 0.8 % of the time: a 16-minute walk would take some 32 hours,
outlive the age a search judges for, and be begun again at an engine
restart, leaving the machine unjudged.

A run of pauses is now at most nine times as long as the walk read since
the one before, so the walk keeps at least a tenth of the time whatever
the pressure, and takes at most ten times as long as unpaced: about
2.7 hours for the 16-minute walk, inside the 12 hours the controller
waits quietly. A test ties the bound to rootsearch.Quiet.

The pressure is the share of the time since the last look in which some
task waited on I/O, from PSI's cumulative counter, not its ten-second
average, which the kernel moves only every two seconds. A pause is
500 ms, then the pressure is looked at again.

The engine's walker is built in one place, engineWalker, and a test
fails if it is not paced by /proc/pressure/io. What the walk cannot read
is said through the search's own channel (the cache's Say), as the rest
of the search is, with what it means for the disk.
2026-10-11 01:53:25 +02:00
jschoubben 8f7db549ae The setuid walk paces itself by the machine's I/O pressure (issue 433)
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
The walk for setuid programs is put in the idle I/O class so it yields
to everything else, but only a scheduler that schedules by class (BFQ)
honours that. The control-node's disks run mq-deadline, which does not,
so the walk read at full priority. On 2026-10-11 the walk started at
01:27, right after the engine applied a new controller, and ran sixteen
minutes; it kept two spinning disks saturated, the bus whose store
shares them stalled for half a minute twice, and the controller lost
its lease and restarted each time.

The walk now also paces itself by PSI, which reads the same whatever
the scheduler: before it reads a directory, while some task has waited
on I/O for more than 10 % of the last ten seconds, it pauses two
seconds at a time. It looks at most every 250 ms, pauses at most 30 s
in a row and then reads one more directory, so a machine that is never
quiet is walked slowly but still judged. Its own reads count in the
pressure, so on an otherwise idle machine it holds the disk below the
bound. Pressure it cannot read pauses nothing, and is said once.

A paced walk finds what an unpaced one finds; a walk ended while it
pauses is an incomplete walk, never one that found nothing.
2026-10-11 01:47:43 +02:00