jschoubben
3a84044ece
With THP 'always' journald's mmapped journal files sit in 2 MiB folios, so each small journal write dirties and writes back 2 MiB; on the control node that saturated the root mirror holding the bus store and the store, and the controller lost its lease. The module sets the mode from a per-machine setting (default always, the kernel's own, so assigning it changes nothing until a machine sets madvise), verifies it took, rotates the journal once so new files are mapped under the new mode, gives back what it found when stopped, and serves kernel_huge_pages to measure file_thp per unit.
3.7 KiB
kernel
The kernel's run-time settings on a machine (novox/hq to-be 42 Phase 1, row 11). Today it holds one: transparent huge pages, set per machine. The rest of the row (the kernel package, microcode, boot entries) is not built yet.
Why (novox/hq issue 746)
With transparent huge pages always, the page cache of a file mapped by a program is collapsed into
2 MiB folios (khugepaged), and a small write into such a folio dirties all of it: writeback then writes
2 MiB. journald writes its journal files through mmap, and every journal reader (fail2ban's journal
jails among them) maps them too. Measured on the control node on 2026-10-14: journald's cgroup had
written 2.5 TB in 12 days while the journal grew by about 25-30 KB/s, fail2ban's cgroup 63 GB in 4 h
while its own files did not grow, both holding hundreds of MiB of file pages in huge folios; the bursts
saturated the mirrored spinning root disk that holds the bus store and the store, and the controller
lost its lease four times in 30 minutes. With madvise, only memory a program asks to have in huge pages
(madvise(MADV_HUGEPAGE)) gets them; a database or a JVM that wants them still asks.
What it owns
| what | notes | |
|---|---|---|
| setting | transparent-huge-pages |
always, madvise or never. Default always, what the distribution's kernel boots with, so assigning the module changes nothing until a machine sets another value |
| file | ${dir:state}/huge-pages |
the script, written from the setting: apply writes the mode to /sys/kernel/mm/transparent_hugepage/enabled and reads it back (a value that does not take fails the unit); restore gives back the mode found before the module first set it, kept in ${dir:state}/found |
| file | /etc/systemd/system/mesh-kernel-huge-pages.service |
oneshot, early in boot (sysinit.target, before the services that fill the page cache); stopping it runs restore |
| service | mesh-kernel-huge-pages.service |
running and enabled, restarted when the script (so the setting) or the unit changes; its health is the unit's |
The per-size controls (hugepages-<n>kB/enabled) are left alone: on the machines read, 2048 kB says
inherit (it follows the mode) and every other size never. kernel_huge_pages shows each one.
A folio already in the page cache keeps its size. A new mode applies to pages faulted or collapsed
from then on. The active journal files would keep their 2 MiB folios until archived, so apply rotates
the journal (journalctl --rotate) whenever it moves the mode away from always: new files are mapped
under the new mode. A failed rotation is said in the unit's journal and does not fail the setting. At
boot the kernel starts at always, so a machine set to madvise rotates its journal once per boot.
Tools
| tool | answers | |
|---|---|---|
kernel_huge_pages |
r | the mode, defrag and shared-memory mode; each per-size control and what inherit resolves to; huge pages by kind from /proc/meminfo (MiB); the units holding the most file pages in huge folios (memory.stat file_thp) with their dirty and writeback bytes, khugepaged's collapses and the bytes their cgroup wrote (io.stat, the most any one device saw: a mirror and its members each count the same write) |
Tests
go test ./...: the tool over a stand-in tree holding the control node's own files of 2026-10-14; the
manifest's shape (the default, the order, the unit, restart and health); and the script as the mesh
writes it, run with sh against a stand-in directory: it sets and reads back, keeps what it found once,
rotates the journal once and not for always, gives back what it found, refuses a value the kernel
does not take before writing anything, fails when the write does not take, and says when the kernel has
no transparent huge pages.