mesh-reads: a node's files and accounts read as an account that holds no secret (hq ADR 0310, issue 418) #244
Closed
mesh-admin
wants to merge 2 commits from
fix/418-node-reads into main
pull from: fix/418-node-reads
merge into: :main
:main
:fix/build-node-step-ends
:fix/1981-sign-in-page-names
:fix/nats-stream-purge
:fix/47-G-start-2
:feat/47-G-join11
:fix/1966-gate-record-catalog
:pre/registry-milestone
:fix/1748-retake-and-resolver
:pin/sdk1
:fix/check-every-failure
:feat/l5-l7
:feat/l5-l7-a3
:feat/judge-once-pkgs
:fix/turn-handed-on-approve
:fix/checker-full-failure
:revert/774-l3
:fix/check-names-every-conflict
:fix/check-speed
:probe/1126-seed-10
:probe/1126-seed-9
:probe/1126-seed-8
:probe/1126-seed-7
:probe/1126-seed-6
:probe/1126-seed-5
:feat/1126-seed-4
:revert/1126-seed-2
:feat/774-vault-mints
:fix/1901-node-test-cause
:fix/1945-adopt-hashes
:fix/1945-vault-hashing
:feat/try-speed
:fix/1921-every-failure
:feat/r4g-finish-composer
:feat/r4g-finish-1935
:fix/1901
:fix/forge-requires-no-status
:feat/mail-list
:feat/modules-name-no-node
:fix/r4e-r2
:fix/r4e-k2
:fix/r4e-k1
:feat/r4e-leaf
:feat/r4e-buildmods
:feat/r4e-gate
:fix/1106-sdk-member-merges
:feat/r4c5-checker
:fix/xterm-config-defaults
:fix/1165-fill-retry
:feat/r4d-automx-go
:feat/r4d-n8n
:fix/1275-group-merges-whole-or-not
:fix/1129-no-drift-pass
:fix/270-nats-emits-bus-faults
:fix/claude-code-tips-preference
:fix/628-modules-declare-every-package-they-need
:fix/613-home-network-shares
:feat/bus-websocket-listener
:fix/340-tool-servers-only-from-the-terminal
:feat/the-store-collects-nightly-again
:feat/dbus-module
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
agent-ace
agent-g14
agent-novox
agent-shanks
jschoubben
mesh-tools
miel.truyen
nox-agent
Clear assignees
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: novox/mesh-catalog#244
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
What. A new module,
modules/mesh-reads, which holdsnode-reads(mesh-controller #257) and serves its six verbs as a Go bundle:stat: each component of one or more paths (at most 200), from / down. For each: type, owner and group (name and number), octal mode with the setuid, setgid and sticky bits, the ls permissions, a link's target, and whether it is a mount point, with its file system and source (from/proc/self/mountinfo). A link is not followed unlessfollowis "true". A component the account cannot reach stops the walk and says why.readlink: the target as written, where the path leads, and whether that exists.list: the entries of one directory with type, owner, group, mode, size and link. At most 2000 entries; a longer list is cut and says so.read: text only. 64 KiB by default, 256 KiB at most, cut at a line and said so.fromandlineschoose lines; a negativefromreads from the end. The file is opened non-blocking and judged by what was opened, so a fifo answers rather than waits. A binary file, a directory or a device is refused.search: a fixed string, never a pattern. By default 100 matches, 4 directories deep; at most 500 and 8 when asked. Also bounded at 5000 files, 4 MiB per file and 20 s. Links are not followed and binary files are skipped. The answer says which bound it met.account: an account by name or number, with its group and every group it is in; a group with its members; or, with neither, every account and group. Read from/etc/passwdand/etc/group. Never a password field.How it is bounded (ADR 0310 §3–§5).
runs-as: mesh-readsis an account the module makes:/usr/bin/nologin, home/var/lib/mesh-reads, no groups. The tool runner serves it in a runtime of its own (ADR 0259 §8), never as the operator's account or the agent's account (ADR 0266, refused by the controller). The node's file modes are the first bound. It refuses every read if it finds itself running as root.list,readandsearch:MESH_SECRET_PLACESfor its node. Without that list it does not start.refuse.go):.ssh,.gnupg,.aws,.kube, the agent's.credentials.json,.netrc,.pgpass,.git-credentials,id_*,*.key,ssh_host_*_key,/etc/shadow,/etc/ssl/private,/etc/wireguard, the network manager's connections.statandreadlinkstill say its owner and mode and mark itsecret-place, because they read no content.outward.CheckSecret(mesh-sdk #23). A line shaped like a secret is withheld and its number said.Tests (
go test -race):TestEveryReadIsRefusedAsRootTestItDoesNotStartWithoutTheNodesSecretPlacesTestSecretPlacesAreRefusedByNameBeforeAnyRead: vault, own-secret directory, the agent's credentials,.ssh,.pgpass, an ssh host key,/etc/shadowand*.key, each readable by mode and each refused. A link to the module's broker, a listed vault directory and a search that skips them. A public key and a certificate are not refused.TestAFileTheAccountMayNotReadIsRefusedByItsMode: a file of mode 0000, and a path through a directory the account cannot search. Skipped as root.TestAReadIsBounded: the default cut, more than the maximum refused, lines 10–11, the last two lines, a line past the end.TestOnlyATextFileIsRead: binary, directory, and a fifo answered within 5 s.TestALineShapedLikeASecretIsWithheld: in a read and in a search.TestASearchIsBounded: matches, the maximum matches and depth, depth 3 against a file 5 deep, case, the time bound, and a pattern's characters taken literally.TestStatSaysEachComponent: owner, setgid mode, a link not followed and then followed, an NFS mount point, a missing path, a relative path, 201 paths.TestReadlinkSaysTheTargetAndWhereItLeads,TestListSaysEveryEntry(including the 2000 cut),TestAccountAndGroupLookup,TestNoAnswerIsLargerThanTheBound,TestItServesTheSeatsVerbs.TestTheAccountHoldsNoSecretsGroupAndNoLogin, on the manifest.By hand, over MCP on stdio with the built binary on g14:
/proc/loadavgwas read./var/lib/mesh/bundleswas walked from / with its owners and modes.~/.claude/.credentials.jsonwas refused by name.account jochengave his ten groups.MESH_GATE=<controller built from mesh-controller #257> MESH_CHECK_MODULES=mesh-reads sh merge-check.shpasses: words, agent-home, paths, every manifest (69 without health, as kept), deps, and the module's tests.Order. mesh-sdk #23, then mesh-controller #257, which must be live, then this. Until #257 runs,
mesh/merge-gatehere reads the claim with a controller that has nonode-readsand refuses it. After merge: assignmesh-readsto novox first. The survey of issue 358's 123 directories needs it there. Then ace, shanks and g14.Limits I chose.
/etc/passwdand/etc/grouponly, and the answer says so. A directory service or systemd's dynamic accounts are not covered.Not here. Moving messenger onto the SDK's rule (it would cut into #178), the gate check that every node holds
node-reads, and the approval mark onexecute. Each follows under issue 418.Part of issue 418. Not merged.
Delivery
novox/mesh-catalog@05c0396f6be0— stopped since 2026-10-12T12:33:00ZDelivery plan — builds new: modules/mesh-reads, sent nowhere; no bus step
Group
fix/418-node-reads, in order: novox/mesh-controller@63a99d4323 → novox/mesh-sdk@67bfa22132Transitions
The commit's note under
refs/notes/mesh-plankeeps every transition:git log --notes=mesh-plan.Merge check at
05c0396fmesh/merge-gate: FAIL — a manifest the change touches fails the module check: modules/mesh-reads/module.json: this manifest cannot be used: (modules: new:modules/mesh-reads)mesh/repo-check: FAIL — its merge-check.sh failed: files asking for a setting without saying whether it is trusted: 25Delivery plan — builds new: modules/mesh-reads, sent nowhere; no bus step
Run by the build seat on g14 as
build-1791806740426423506(builds --log build-1791806740426423506).Verdict: request changes at
05c0396f6bIndependent review for issue 418 (ADR 0310). In
modules/mesh-reads,go vet ./...is clean andgo test -race -count=1 ./...passes. With a controller built from mesh-controller #257 head 4ead5a3,MESH_GATE=… MESH_CHECK_MODULES=mesh-reads sh merge-check.shpasses, so the redmesh/merge-gateandmesh/repo-checkcome from the running controller lacking the seat.What holds:
nologin, has its own home and no groups, and every read is refused as root.O_NONBLOCKandfstaton what was opened)./proc/self/environandcmdlineare refused as binary.EvalSymlinks, including/proc/self/fd/Nthat leads into one.accountnever answers a password field.Required 1: the name check and the open are two lookups (TOCTOU).
refusedReadjudgespathandEvalSymlinks(path). Thenos.OpenFile(path, …)resolves the path again and follows links. I tested a link flipped in a loop between a harmless file and a file inside a secret place.Readanswered the secret place's content 3744 times in 20000 calls.searchFilehas the same gap:WalkDirjudges the entry by itslstattype, then the open follows a link that was swapped in.Why this matters: the name rule is the only bound for what
mesh-readsmay read by mode. That includes its own bus credential,/var/lib/mesh-reads/broker, which it owns.fs.protected_symlinks=1closes this in/tmp, but not in any non-sticky directory another account can write.Fix: judge what was opened, not the name.
O_RDONLY|O_NONBLOCK|O_NOCTTY|O_CLOEXEC. Insearch, also useO_NOFOLLOW.fstatthe descriptor and require a regular file.readlink("/proc/self/fd/<n>")and runRefusedon that real path. If the path ends in(deleted)or is not absolute, refuse it.Lstatbefore opening, so a device is never opened at all. Opening/dev/ptmxor a tape or watchdog device has side effects even when the read is then refused.Required 2: a private key's body leaks a line. I generated an RSA-2048 PKCS#8 key into
server.pem(the name is not refused).readwithheld 27 of its 28 lines; one base64 line of the key passedCheckSecret. Fix: inreadand insearch, withhold every line from a-----BEGIN …PRIVATE KEY-----(or anyBEGINthe rule refuses) through itsEND. Better: refuse a file that contains such a block. Add a test with a generated key.Required 3: shell history by name. History files are where tokens are typed. Add
.bash_history,.zsh_history,.histfile,.python_history,.psql_history,.mysql_history,.lesshst,.viminfoand.node_repl_historytoknownNames. On g14 they are mode 0600 inside a 0700 home, but the name rule is the second bound for a node where a home is 0755.Required 4: the stray binary. The first commit carries a 5.1 MB built
mesh-reads(Go 1.27.1, not stripped). I extracted it:/home/jochen/projects/novox/mesh-catalog-418/…,/home/jochen/go/pkg/mod/…).This repository merges with merge commits, so the binary would stay in main's history for good: about 5 MB of bloat and a disclosure of local paths. It is not a secret leak. Since this PR needs another round anyway, the clean way out is a new branch from current main (for example
fix/418-node-reads-2). Squash the two commits (and the fixes above) so the binary never appears, open it as a new PR, and close this one. Also addmodules/*/mesh-*(or/modules/mesh-reads/mesh-reads) to.gitignore.Should, not blocking:
searchcan hang on a hard NFS mount, becauseSearchTimeis checked only between entries. Stay on the root's file system by default (st_dev, likegrep -rwith-xdevthinking), and skip/procand/sysunless the root is inside them./proc/*/{environ,mem,fd,map_files,root,cwd}(only the account's own processes are reachable, but/proc/self/fd/Nreaches a deleted file today; I tested it),/etc/rancher,/etc/kubernetes/admin.conf, and browser profile directories (.mozilla,.config/google-chrome,.config/chromium).fs.protected_hardlinks=1limits who can make one, and the file mode still bounds the read. Say this limit in the README.go.modto the SDK's merged main commit. 4dc5f29 stays reachable through the merge commit, so this is tidiness, not a break.Merge order: mesh-sdk #23, then mesh-controller #257 and its rollout done, then the replacement for this PR, assigned to novox first.
Replaced by #247: a fresh branch from main with one commit, so the stray binary never appears in history, plus review 55's fixes. Closing.
Pull request closed