jschoubben
ab1811d003
mesh/merge-gate head ab1811d00357 merged into trunk c79337579d6b — pass: builds new: modules/kernel, sent nowhere; no bus step; every machine composes with…
mesh/repo-check head ab1811d00357 merged into trunk c79337579d6b — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
A restart ran restore then apply, so every change passed back through the found mode and rotated the journal again. Reload applies the new value directly; restore also drops its state; no rotation while the machine boots; the unit is sandboxed.
53 lines
4.4 KiB
Markdown
53 lines
4.4 KiB
Markdown
# kernel
|
|
|
|
The kernel's run-time settings on a machine (novox/hq to-be 42 Phase 1, row 11). Today it holds one:
|
|
**transparent huge pages**, set per machine. The rest of the row (the kernel package, microcode, boot
|
|
entries) is not built yet.
|
|
|
|
## Why (novox/hq issue 746)
|
|
|
|
With transparent huge pages `always`, the page cache of a file mapped by a program is collapsed into
|
|
2 MiB folios (khugepaged), and a small write into such a folio dirties all of it: writeback then writes
|
|
2 MiB. journald writes its journal files through mmap, and every journal reader (fail2ban's journal
|
|
jails among them) maps them too. Measured on the control node on 2026-10-14: journald's cgroup had
|
|
written 2.5 TB in 12 days while the journal grew by about 25-30 KB/s, fail2ban's cgroup 63 GB in 4 h
|
|
while its own files did not grow, both holding hundreds of MiB of file pages in huge folios; the bursts
|
|
saturated the mirrored spinning root disk that holds the bus store and the store, and the controller
|
|
lost its lease four times in 30 minutes. With `madvise`, only memory a program asks to have in huge pages
|
|
(`madvise(MADV_HUGEPAGE)`) gets them; a database or a JVM that wants them still asks.
|
|
|
|
## What it owns
|
|
|
|
| | what | notes |
|
|
|---|---|---|
|
|
| setting | `transparent-huge-pages` | `always`, `madvise` or `never`. Default `always`, what the distribution's kernel boots with, so assigning the module changes nothing until a machine sets another value |
|
|
| file | `${dir:state}/huge-pages` | the script, written from the setting: `apply` writes the mode to `/sys/kernel/mm/transparent_hugepage/enabled` and reads it back (a value that does not take fails the unit); `restore` gives back the mode found before the module first set it, kept in `${dir:state}/found` |
|
|
| file | `/etc/systemd/system/mesh-kernel-huge-pages.service` | oneshot, early in boot (`sysinit.target`, before the services that fill the page cache); reloading it runs `apply`, stopping it runs `restore`, which also removes `found`. Sandboxed: the file system read-only but for the module's state (`/sys` stays writable), no home, no network, no new privileges; the journal's rotation goes over its local socket, which a private network leaves reachable |
|
|
| service | `mesh-kernel-huge-pages.service` | running and enabled; **reloaded** when the script (so the setting) changes, so a new value is set directly and never passes back through the found one; restarted only when the unit changes; its health is the unit's |
|
|
|
|
The per-size controls (`hugepages-<n>kB/enabled`) are left alone: on the machines read, 2048 kB says
|
|
`inherit` (it follows the mode) and every other size `never`. `kernel_huge_pages` shows each one.
|
|
|
|
**A folio already in the page cache keeps its size.** A new mode applies to pages faulted or collapsed
|
|
from then on. The active journal files would keep their 2 MiB folios until archived, so `apply` rotates
|
|
the journal (`journalctl --rotate`) whenever it moves the mode away from `always`: new files are mapped
|
|
under the new mode. A failed rotation is said in the unit's journal and does not fail the setting. No
|
|
rotation is made while the machine is still starting (`systemctl is-system-running` says `initializing`
|
|
or `starting`): nothing is in huge pages yet, and the persistent journal opens after this unit.
|
|
|
|
## Tools
|
|
|
|
| tool | | answers |
|
|
|---|---|---|
|
|
| `kernel_huge_pages` | r | the mode, defrag and shared-memory mode; each per-size control and what `inherit` resolves to; huge pages by kind from `/proc/meminfo` (MiB); the units holding the most file pages in huge folios (`memory.stat` `file_thp`) with their dirty and writeback bytes, khugepaged's collapses and the bytes their cgroup wrote (`io.stat`, the most any one device saw: a mirror and its members each count the same write) |
|
|
|
|
## Tests
|
|
|
|
`go test ./...`: the tool over a stand-in tree holding the control node's own files of 2026-10-14; the
|
|
manifest's shape (the default, the order, the unit and its sandbox, reload, restart and health); and the script as the mesh
|
|
writes it, run with `sh` against a stand-in directory: it sets and reads back, keeps what it found once,
|
|
rotates the journal once and not for `always` nor while the machine starts, goes straight to a new value on a
|
|
reload without rewriting what it found, gives back what it found and then keeps nothing, refuses a value the kernel
|
|
does not take before writing anything, fails when the write does not take, and says when the kernel has
|
|
no transparent huge pages.
|