issue 1841/S16: kernel_boot answers the loader, entries, kernels, booted entry and the fallback

ADR 0508 §9: the operator sees whether every boot menu lists the fallback kernel, and the room left on /boot.

modules/kernel/README.md

@@ -1,8 +1,9 @@
 # kernel
 
-The kernel's run-time settings on a machine (novox/hq to-be 42 Phase 1, row 11). Today it holds one:
-**transparent huge pages**, set per machine. The rest of the row (the kernel package, microcode, boot
-entries) is not built yet.
+The kernel's run-time settings on a machine (novox/hq to-be 42 Phase 1, row 11). It holds two things:
+**transparent huge pages**, set per machine, and **the fallback kernel** (`linux-lts` with its headers) and
+its place on the boot menu (novox/hq ADR 0508 §9 and §10). The default kernel's package, and microcode, are
+not built yet.
 
 ## Why (novox/hq issue 746)
 
@@ -35,16 +36,34 @@ under the new mode. A failed rotation is said in the unit's journal and does not
 rotation is made while the machine is still starting (`systemctl is-system-running` says `initializing`
 or `starting`): nothing is in huge pages yet, and the persistent journal opens after this unit.
 
+## The fallback kernel (novox/hq ADR 0508, issue 1841)
+
+Every machine keeps a second, older kernel to start from, so a kernel upgrade that does not boot is not the
+end of the machine. The module declares the packages `linux-lts` and `linux-lts-headers` (so every DKMS module is
+built for the fallback kernel too) after its boot-entries service, so the entry is in place before the kernel is.
+
+| | what | notes |
+|---|---|---|
+| file | `${dir:state}/boot-entries` | the script. It reads from the machine which loader booted it: the EFI variable `LoaderEntrySelected` present means systemd-boot, a `/boot/grub/grub.cfg` without it means GRUB, neither fails the unit loudly |
+| | systemd-boot | writes `/boot/loader/entries/mesh-linux-lts.conf` from `/proc/cmdline` without `initrd=` and `BOOT_IMAGE=`, with the microcode images present in `/boot` and the fallback initramfs as initrd lines; `loader.conf` and the default entry are never touched |
+| | GRUB | runs `grub-mkconfig -o /boot/grub/grub.cfg` only when the set of `/boot/vmlinuz-*` differs from the kernels the menu names; the default is left as it is |
+| | room | while `/boot/vmlinuz-linux-lts` is not there yet, less than 96 MiB free on `/boot` fails the unit and says so (the fallback kernel writes about 72 MB, issue 1836) |
+| | a kernel-copying hook | a hook in `/etc/pacman.d/hooks` that the mesh did not write, triggered by kernel images and copying to one fixed file name in `/boot`, would put the fallback kernel in the default kernel's place: it is moved into `${dir:state}/taken-over/` and named in the unit's journal. Each kernel is then placed under its own package's name by the distribution's hooks |
+| file | `/etc/systemd/system/mesh-kernel-boot-entries.service` | oneshot, sandboxed: the file system read-only but for `/boot`, `/etc/pacman.d/hooks` and the module's state; no home, no network |
+| file | `/etc/pacman.d/hooks/90-mesh-kernel-boot-entries.hook` | runs the script after a kernel image is installed, upgraded or removed |
+| packages | `linux-lts`, `linux-lts-headers` | declared after the service and the hook |
+
 ## Tools
 
 | tool | | answers |
 |---|---|---|
+| `kernel_boot` | r | the loader (`systemd-boot`, `grub` or `none`), the menu's entries, the kernels in `/boot`, the entry booted (systemd-boot) or the kernel image booted (GRUB), whether the fallback kernel is on the menu with its kernel in `/boot`, and the room free on `/boot` in KiB |
 | `kernel_huge_pages` | r | the mode, defrag and shared-memory mode; each per-size control and what `inherit` resolves to; huge pages by kind from `/proc/meminfo` (MiB); the units holding the most file pages in huge folios (`memory.stat` `file_thp`) with their dirty and writeback bytes, khugepaged's collapses and the bytes their cgroup wrote (`io.stat`, the most any one device saw: a mirror and its members each count the same write) |
 
 ## Tests
 
 `go test ./...`: the tool over a stand-in tree holding the control node's own files of 2026-10-14; the
-manifest's shape (the default, the order, the unit and its sandbox, reload, restart and health); and the script as the mesh
+manifest's shape (the default, the order, the unit and its sandbox, reload, restart and health); the boot-entries script run with `sh` over stand-in roots (a systemd-boot entries directory, a GRUB menu, a hook that copies kernels to one fixed name): the entry from the booted command line, a menu generated again only when the kernels changed, no loader, no room, the taken-over hook; `kernel_boot` over the same roots; and the huge-pages script as the mesh
 writes it, run with `sh` against a stand-in directory: it sets and reads back, keeps what it found once,
 rotates the journal once and not for `always` nor while the machine starts, goes straight to a new value on a
 reload without rewriting what it found, gives back what it found and then keeps nothing, refuses a value the kernel