Commit Graph
538 Commits
Author SHA1 Message Date
jschoubben 281837847e Put busybox's account removal back, test the sample, and read a held lock as kept only for a running walk (hq ADR 0328 review)
mesh/merge-gate head 281837847e03 merged into trunk 303d4037e84e — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
mesh/repo-check head 281837847e03 merged into trunk 303d4037e84e — pass: its merge-check.sh passed
mesh/delivery delivered
An earlier change dropped alpine's RemoveAccount and RemoveGroup, so alpine made no
module identity at all. The sample beneath a root in shape had no test of its own.
And an apply that found the library's lock held read the library as kept even when
what held it was a take act about to be refused for a missing disk; it now does so
only while a take walk is running, and otherwise waits and judges as always.
2026-10-12 22:21:42 +02:00
jschoubben c7551162d6 Give no account a home, keep old data's sub-paths for the warrant, and never hand out setgid (hq ADR 0328 review)
usermod --gid regroups every file under an account's home, and accounts were made
with / as home: they now have /nonexistent, and an older account's home is moved
there (without its files) before its group changes. A sub-path asked for later that
holds data not in the group waits for the operator as the root would. The take act
gives the group to hardlinked files an account owns (the *arr apps' downloads) and
leaves root's; a file's setuid or setgid bit the regroup takes is never put back, so
no setgid file hands out the library's group; each is counted. An apply no longer
waits for a running take walk.
2026-10-12 22:18:16 +02:00
jschoubben 2cffa33bcc Keep a root without asking only when its data beneath is the library's already (hq ADR 0328 §7)
A root in the group with setgid was kept without a warrant even when the data beneath
it was not in the group, which §7 says the operator is asked about. A root is now
kept without asking only when it is empty, or it and a bounded sample beneath it are
all in the group with group write (and setgid on folders). On busybox, an account the
mesh made is given its added groups with addgroup and delgroup; a change of its
primary group is refused, saying why.
2026-10-12 22:14:10 +02:00
jschoubben 2a2be687f3 Keep libraries under groups of their own, and give modules users in them (hq ADR 0328, issue 567)
ADR 0328 supersedes ADR 0325: shared data is a library with one home, kept usable by
its group rather than by ACL entries. The engine now makes each library's group and
each module's user in its libraries' groups; keeps a library's root and requested
sub-paths in its group with group write and setgid on its home, making missing
sub-paths as the root is; waits for the operator's word before touching a root that
holds data, and on the take act walks it by descriptor (no link, same filesystem,
bounded and resumed, files with other names left); holds the containers of a library
whose disk or mount is not there; and puts the tool runner's account in the group.
The ACL keeping, mask repair and watch are gone; an access is again as before ADR 0325.
2026-10-12 22:11:09 +02:00
mesh-admin 303d4037e8 Merge pull request 'Give the controller the contract it can fill: tool runners by name, the take act's access id, wait reasons and ACL support (hq ADR 0325)' (#94) from feat/325-the-contract-the-controller-needs into main 2026-10-12 17:57:41 +00:00
jschoubben 5c8e768ef5 Read an account by its name, never as a number getent takes for a uid (hq ADR 0325)
mesh/merge-gate head 5c8e768ef520 merged into trunk ef26dba2753a — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
mesh/repo-check head 5c8e768ef520 merged into trunk ef26dba2753a — pass: its merge-check.sh passed
mesh/delivery delivered
getent passwd reads an all-digit key as a uid, so a tool runner's account named
"1000" resolved to whoever has uid 1000 and the entry went to them. Only an answer
naming the account asked for is taken.
2026-10-12 19:51:09 +02:00
jschoubben aad33dfe5c Give the controller the contract it can fill (hq ADR 0325, issue 567)
The controller cannot know the operator's account's uid, names an access by its
declared id (which holds a slash), and should not parse words. A tool runner's
account may now come by name alone and is read here (refused when absent or root's,
its uid said in the access's listing); the take act matches the declared id exactly;
an access wait says its reason (keep or disk) and, for a filesystem that refuses
ACLs, whether the take act can enable them.
2026-10-12 19:51:09 +02:00
mesh-admin ef26dba275 Merge pull request 'Give every module an identity and keep shared folders through ACL entries (hq ADR 0325, issue 567)' (#93) from feat/325-identities-and-kept-folders into main 2026-10-12 17:50:59 +00:00
jschoubben d12bba0109 Give the kept-folder tests' folders an unprivileged owner when they run as root (hq ADR 0325)
mesh/merge-gate head d12bba01090d merged into trunk dd9ab7927d96 — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
mesh/repo-check head d12bba01090d merged into trunk dd9ab7927d96 — pass: its merge-check.sh passed
mesh/delivery delivered
The build seat runs merge-check as root, so every folder a test made was root's
data, which is never kept (§12), and 28 tests failed for that alone. A test's
folders now stand for an operator's: run as root, they are given to nobody before
they are judged. The root rule is unchanged.
2026-10-12 19:34:59 +02:00
jschoubben c85c2804a0 Apply an older controller's accesses exactly as before (hq ADR 0325)
mesh/merge-gate head c85c2804a010 merged into trunk dd9ab7927d96 — pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes wi…
mesh/repo-check head c85c2804a010 merged into trunk dd9ab7927d96 — fail: its merge-check.sh failed: --- FAIL: TestAnAccessTheMeshMayNotKeepYetIsSaidAsAWait…
mesh/delivery superseded: a newer head of the same pull request
The engine delivers before the controller half, and today's controller sends no
identities, users or taken. Keeping now starts only for an access whose folder has
users, or a declaration with identities; every other access is mounted when present
and refused when absent as on main, with nothing made, kept, held or judged. A
leftover whose walk could not run says why, not always that its disk is away.
2026-10-12 19:24:24 +02:00
jschoubben 2ff40109ac Read a filesystem that could not be named as unknown, never as a move (hq ADR 0325 sixth review)
A name read that timed out gave an empty name, which differed from the recorded one
and so reset the record: a folder kept while empty asked again, a warranted one was
walked again. An unnamed filesystem now leaves the folder exactly as it is, the
access is said not known for that look, and it resumes once it is named.
2026-10-12 19:20:53 +02:00
jschoubben 46c16e18fe Name a filesystem by its UUID, its kernel id or its source, and keep nothing that has no name (hq ADR 0325 fifth review)
NFS and FUSE report a zero f_fsid, so another export or another FUSE mount at the
same point was named alike and kept without a warrant; xfs's f_fsid follows the
device number. A filesystem is now named by its UUID (its device matched in
/dev/disk/by-uuid), else a nonzero f_fsid, else mountinfo's source; one that cannot
be named is never kept or taken. Entries left on a filesystem not back within 30 days
are forgotten, said once, and the naming read is shared per mount.
2026-10-12 19:14:42 +02:00
jschoubben f00f9c2a2f Name a filesystem by its kernel id, never its device name (hq ADR 0325 fourth review)
An ext4 or xfs library mounted by UUID can come back from a reboot as another device
(/dev/sdb1 as /dev/sdc1); naming the filesystem by mountinfo's source read that as a
move, stripped the warranted library's entries and asked for a new Approve. A
filesystem is now named by statfs's f_fsid (from the UUID, or the ZFS dataset), its
type and its mount point. Entries left on the filesystem a folder is on now are
merged back or held, never walked away, and a TakenOn naming a mount point is not
taken.
2026-10-12 19:10:18 +02:00
jschoubben 21cd036232 Format the third review's tests (hq ADR 0325) 2026-10-12 19:06:57 +02:00
jschoubben f0adb177f0 Leave no leftover for a folder kept only while empty, and name filesystems by more than their mount point (hq ADR 0325 third review)
A folder kept because it was empty, then covered by its disk, was said as unhealthy
leftover entries for ever: its entries sit on a hidden empty folder and grant nothing,
so they are now forgotten, and only entries walked onto data under a warrant are kept
to be taken out. A warrant and a kept folder now name their filesystem by mount point,
source and type, so another pool at the same point is a move; records naming a mount
point are read as its name. A record with no access ids is said under a path-free name.
2026-10-12 19:06:34 +02:00
jschoubben d2d9fbbd7d Hold what mounts a missing disk, and say entries the mesh could not take out (hq ADR 0325 re-review)
A container was still created over an unmounted mount point, so its writes landed on
the filesystem below; a folder moved to another disk was kept on the old warrant; and
entries left by a drop or a move that could not run lived only in a log line. Every
container mounting an access whose disk is away now waits, the warrant holds only for
the filesystem it was carried out on (the wait names the new one, path-free), entries
left on a filesystem are recorded per mount and taken out when it is back, and every
leftover is said as unhealthy until it is. A hung stat is shared per path, and a
widened mask takes from other entries only the bits it adds.
2026-10-12 19:03:02 +02:00
jschoubben 2026ef36f3 Judge a kept folder before every walk, and keep nothing a disk mounted over (hq ADR 0325 review)
A walk run while a dataset was unmounted wrote onto root's empty mount point, or
"dropped" entries from it and forgot the ones on the data; a folder kept while empty
below an unnamed mount was then swept whole once the disk mounted, with no warrant.
Every walk is now judged as an apply is, a changed mount judges the folder afresh, and
systemd .mount units and mounts seen are read for the first-sight rule. Widening a
mask caps the entries it limited, so the owning group gains nothing. The watch lets
go of removed folders, re-arms on lost notice, reads users under its lock and ends
with its context. A drop that cannot read an ACL is counted, a broken folder's words
carry no path, a grant's group that could not be left is tried again, and a folder's
stat is bounded.
2026-10-12 18:55:47 +02:00
jschoubben 51e97a8c14 Give every module an identity and keep shared folders through ACL entries (hq ADR 0325, issue 567)
A module's containers ran as whatever their image chose and shared folders were
made writable by putting accounts in the folder's group (ADR 0312), which ADR 0325
supersedes. The engine now makes each declared module identity as a system account
with the controller's numbers, runs a container declaring `identity` as it, and keeps
each placed shared folder usable through one POSIX ACL entry per user, set through
the kernel's xattrs on descriptors it opened itself. A folder holding data is kept
only after the operator's warrant (the take act); a missing folder is made where its
data lives and waits where its disk is not; root's data is never kept. A watch and a
bounded sweep put back what an app narrows. The group grant is removed, and what it
put an account in is taken back.

Reuses the descriptor walks, the create act, the missing-path wait and the warrant-
backed act message from the paused issue 567 branch.
2026-10-12 18:43:04 +02:00
mesh-admin dd9ab7927d Merge pull request 'Genesis trusts its registry before it pushes to it (issue 92)' (#91) from fix/92-genesis-trusts-its-registry into main 2026-10-12 14:36:16 +00:00
jochen 0818e4c29a Hold the trust-before-push order by a test, and refuse early what would stop it (issue 92)
mesh/merge-gate pass: the change touches no module of the mesh's graph
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
The fix is an order, and nothing failed if the trust call moved after the
push or went: steps 7 and 8 are now one function a test drives against a
runtime that refuses an untrusted push. Preflight now refuses, before
anything is raised, a runtime file that is no JSON object, a docker.service
that is not loaded and active, and a dockerd started with
--insecure-registry, which would refuse the reload. The comment names the
order the docker module's takeover rests on.
2026-10-12 16:29:14 +02:00
jochen 41e640d668 Have genesis trust its registry before pushing to it (issue 92)
mesh/merge-gate pass: the change touches no module of the mesh's graph
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
Genesis pushed the control plane's image to the mesh's plain-HTTP registry
before anything told the container runtime to trust it; the docker module
that writes that trust is applied long after. Genesis now writes the trust
itself at the registry step, with the same into-json writer and a reload,
for the address given and the node's mesh name on the node's registry
port, and refuses before any push unless the runtime says it trusts each.
The docker module later finds the mesh name there and takes it over
without a change.
2026-10-12 16:13:11 +02:00
mesh-admin a1f9856b39 Merge pull request 'A package manager that does not answer fails only the packages; the rest of the declaration applies (issue 233)' (#90) from fix/233-a-failed-probe-fails-only-what-needs-the-tool-2 into main 2026-10-12 14:11:11 +00:00
jochen 74ed7cbe0a Say a re-apply whose result changed, refuse whole when the package manager is not there, and install the packages in the apply that made it answer (issue 233 review)
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/233-a-failed-probe-fails-only-what-needs-the-tool-2 delivered: every member is delivered
2026-10-12 15:39:07 +02:00
mesh-admin 36704b5471 Merge pull request 'Give a mesh directory its mode and owner only through a descriptor (issue 358)' (#87) from fix/358-descriptor-only-directory-mode into main 2026-10-12 13:13:13 +00:00
jochen 6e44a27a63 Fail only the packages when the package manager does not answer, and apply the rest, so a later declaration can restore what an earlier one removed (issue 233)
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery-group group fix/233-a-failed-probe-fails-only-what-needs-the-tool-2 ready: every member ready, and composed together they pass
mesh/delivery superseded: a newer head of the same pull request
2026-10-12 15:12:42 +02:00
mesh-admin dce0eef1ca Merge pull request 'Judge a node's directories read-only before the walk refuses any (issue 358)' (#88) from fix/358-judge-directories-read-only into main 2026-10-12 12:48:55 +00:00
jschoubben e3c3bb37d0 Give a mesh directory its mode and owner only through a descriptor, and gate what the walk refuses (issue 358)
mesh/repo-check pass: its merge-check.sh passed
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/delivery delivered
A mesh directory outside every home was chmodded and chowned by path, so a
parent turned into a link between two applies carried root's chmod and chown
to the link's target. It is now made and changed through the descriptor of
the walk from / that follows a link only root alone could have made.

A directory that walk refuses is a refusal like the placement guard's
(review 47): judged before anything is applied, and again when the
directory is applied, used as found or not; refused at either point it gates
its module and refuses what is below it, so no file is written into the
link's target and the report does not say everything was attempted.
2026-10-12 14:00:53 +02:00
jschoubben b92fb68116 Judge a node's directories read-only before the walk refuses any (issue 358)
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
mesh/delivery-group group fix/358-judge-directories-read-only delivered: every member is delivered
Once a mesh directory's mode and owner go only through a walk that follows no
link another account could make, a directory behind such a link is refused
and its module gated: on the control-node that could stop the controller.
The operator is to be told which directories that walk would refuse on every
node before it lands. So the engine judges the declaration it keeps with the
same walk, making and changing nothing: on a signed ask from the controller
(mesh.node.<node>.ask.judge-directories, held to the hand-over's checks) and
as `nox-mesh-host judge-directories` at the machine.
2026-10-12 13:57:52 +02:00
mesh-admin 3f7f797168 Merge pull request 'A read-write access grants the path's own group, and the path is never changed (hq ADR 0312, issue 537)' (#86) from feat/537-a-read-write-access-grants-the-paths-group into main 2026-10-12 11:22:56 +00:00
jschoubben c433e76de2 Judge a granted group by number, grant nothing a group cannot write, and keep the look bounded (hq ADR 0312, issue 537 review)
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
The engine is built without cgo, so a check by name missed a root group a name
service holds or one number carrying another name: the root groups are resolved
to numbers (now with kmem and shadow) and a number named nothing is refused for
containers too. A group without write is granted nothing (§2). The look reads
each path off the lock within five seconds, so a dead network mount cannot stop
every health statement. A grant's account may not start with a dash. A group a
record leaves for another asker is handed to it, so it is still taken back when
the last asker goes (§5). The scheduler skips paths issue 339's guard refuses.
2026-10-12 12:53:42 +02:00
jschoubben f4efc3cad0 Grant a read-write access's path group to what runs the module's code, so a module can write shared data without the mesh changing it (hq ADR 0312, issue 537)
The engine reads the group owning the accessed path at apply and gives it to the
containers and the tools' account an access's new `grant` names: --group-add for a
container (recreated with it), an account group for the account (recorded, taken
back as ADR 0252 takes back). The path is never changed; an access whose group
cannot write is stated unhealthy for the operator to decide; group 0 and the
root-granting groups are refused by name. The tool runner restarts through the
restart-on the controller gives it, because the access reports updated exactly
when the account's groups moved. Health says GrantContract (4), which tells the
controller it may send `grant`.
2026-10-12 12:23:38 +02:00
mesh-admin a76baa4b47 Merge pull request 'Carry the controller's grant to ask the delivery's owner for its groups in the installer's first user list (hq ADR 0306, issue 525)' (#85) from feat/525-grant-delivery-groups into main 2026-10-12 10:14:32 +00:00
jschoubben 90ce9e9d9f Carry the controller's grant to ask the delivery's owner for its groups in the installer's first user list (hq ADR 0306, issue 525)
mesh/delivery delivered
mesh/repo-check pass: its merge-check.sh passed
mesh/merge-gate pass: the change touches no module of the mesh's graph
The controller is to ask mesh-delivery whether a merge's delivery
group still has a member to merge, so that a merge in no waiting group
is cut at once (ADR 0304, ADR 0306); its bus grant names the verb, and
the installer's first user list must too, as for times (212a3cc).
2026-10-12 12:09:18 +02:00
mesh-admin df5de3bd9f Merge pull request 'Say an unhealthy resource's restart count as what it counts (issue 458)' (#84) from fix/458-restarts-say-what-they-count into main 2026-10-12 09:03:47 +00:00
jschoubben a18d09db75 Say an unhealthy resource's restart count as the engine's own since it began watching
mesh/delivery delivered
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery-group group fix/458-restarts-say-what-they-count delivered: every member is delivered
Beside the state it read as restarts since then (novox/hq issue 458).
2026-10-12 10:47:07 +02:00
mesh-admin 3333a05a74 Merge pull request 'A refused directory gates its module, and a file meets the placement guard (issue 495)' (#83) from fix/495-a-refused-directory-gates-its-module into main 2026-10-11 17:00:22 +00:00
jschoubben 4695dfc0a2 Say both a gated module and a machine stop, and count skipped apart (review of #83)
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
A module-less action that stopped the apply after a module was gated was reported as
everything else attempted. The failure now names the gated modules and the stop, counts
skipped resources apart from applied ones, and the tests cover an archive below a refused
path, a gated module tried again by the next apply, and a refused file among the failures.
2026-10-11 18:51:04 +02:00
jschoubben 7233ddbe35 Gate a module behind its refused directory and judge a file by the placement guard (issue 495)
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
On ace the engine refused Docker's data root as not placed, said nothing was touched, then
wrote a file into it and ran the module's step. A refused directory now gates the rest of
its module as a failed step does, a file is refused in a forbidden tree or below a refused
path, and the failure names the modules whose rest was not attempted.
2026-10-11 18:19:25 +02:00
jschoubben 1acfed4add Test that a refused directory gates its module and a file meets the same guard (issue 495)
Both fail on main: the shape seen on ace, a refused directory, a file below it and a
step after it, writes the file, runs the step and reports that everything was attempted.
2026-10-11 18:17:59 +02:00
mesh-admin 6defec8f68 Merge pull request 'Refuse a declaration composed from an older assignment generation than the machine applied (issue 234)' (#82) from fix/234-refuse-a-declaration-from-an-older-generation into main 2026-10-11 16:00:01 +00:00
mesh-admin 2833caf3d6 Merge pull request 'A failed run-once process runs again on every apply until it passes (hq issue 467)' (#81) from fix/467-a-failed-step-runs-again into main 2026-10-11 09:30:49 +00:00
jschoubben 20d5af9b2d Refuse a declaration composed from an older assignment generation than the machine applied (hq issue 234)
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
A newer sequence carrying an older view of the assignments undeclared four
assigned modules on 2026-10-04; the sequence cannot tell it, the generation
can. The engine keeps the highest generation it applied, refuses an older one
before reading the machine unless the declaration is a gate's put-back, and
says the refusal in its report so the controller can name the sender. A
declaration without a generation is applied as before, and every report says
it reads one, so the controller sends it only to engines that do.
2026-10-11 11:22:30 +02:00
jschoubben 0ec96a048e Clear a step's success only when it was started and failed, and say updated when it passes again (novox/hq issue 467)
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
A bundle that could not be fetched refused nothing; clearing for it would rerun
every step on a node once the artifact store came back from an outage.
2026-10-11 11:12:15 +02:00
jschoubben c957f1ec5e Run a failed run-once process again on every apply until it passes (novox/hq issue 467)
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
A failure recorded nothing, so the record of the step's last success stayed and
the next apply skipped it, opening ADR 0136's gate on what it had just refused.
The record is kept, for removal, with what it was run as cleared.
2026-10-11 11:07:17 +02:00
mesh-admin 85c51ebb59 Merge pull request 'A refused line's request is read before it is answered, so a busy line reads busy (issue 434)' (#80) from fix/434-a-busy-line-is-answered-busy-whatever-the-load into main 2026-10-11 00:55:47 +00:00
jschoubben f9805b2113 Bound what a refusal costs the engine, and let the busy tests fail instead of hang
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
A refusal waiting for its request parsed it into a 64 KiB reader for up to 5 s,
on a socket every account may open. It now drains at most 4 KiB through a small
buffer for at most 1 s and never past the accept-time deadline, only 32 wait at
once, and an unknown account is refused before it takes a slot. The busy tests
wait for the first line with a timeout and say what came of it (novox/hq issue 434).
2026-10-11 02:47:11 +02:00
jschoubben 70cb613614 Read a refused line's request before answering it, so a busy line is read as busy
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
The engine answered busy and closed before the client's request had landed;
the client's write then broke and it never read the refusal waiting for it.
Under the whole suite's load this failed the busy test (novox/hq issue 434).
2026-10-11 02:35:20 +02:00
mesh-admin bd5cc69804 Merge pull request 'The setuid walk paces itself by the machine's I/O pressure (issue 433)' (#79) from fix/433-the-setuid-walk-yields-to-a-busy-disk into main 2026-10-11 00:03:46 +00:00
jschoubben f311fa36dc The setuid walk is paced before each file too, so a large directory is not read blind (review of issue 433)
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
A file's own details (lstat) may be read from the disk as a directory is,
so a directory of tens of thousands of files was read without a look and
the header's promise that no burst is longer than PaceLook did not hold.
The walk now asks the pacer before each file as well; the 250 ms gate
makes that a clock read per file. A test with one directory of twenty
files fails without it. The clock is read once after a run of pauses,
and the constants test says that it guards the constants only.
2026-10-11 01:55:39 +02:00
jschoubben 30242197fa The paced walk keeps a tenth of the time, and reads the pressure since its last look (review of issue 433)
mesh/merge-gate pass: builds mesh-host → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
The review of #79 found that pacing alone brought back issue 361's
fault. The control-node's own I/O pressure sits near the 10 % bound with
no walk running, so a walk that paused 30 s for every 250 ms it read
would read 0.8 % of the time: a 16-minute walk would take some 32 hours,
outlive the age a search judges for, and be begun again at an engine
restart, leaving the machine unjudged.

A run of pauses is now at most nine times as long as the walk read since
the one before, so the walk keeps at least a tenth of the time whatever
the pressure, and takes at most ten times as long as unpaced: about
2.7 hours for the 16-minute walk, inside the 12 hours the controller
waits quietly. A test ties the bound to rootsearch.Quiet.

The pressure is the share of the time since the last look in which some
task waited on I/O, from PSI's cumulative counter, not its ten-second
average, which the kernel moves only every two seconds. A pause is
500 ms, then the pressure is looked at again.

The engine's walker is built in one place, engineWalker, and a test
fails if it is not paced by /proc/pressure/io. What the walk cannot read
is said through the search's own channel (the cache's Say), as the rest
of the search is, with what it means for the disk.
2026-10-11 01:53:25 +02:00