mesh-catalog/modules/sensors/README.md

sensors

The condition of a machine's own hardware: temperatures with the thresholds the hardware gives for itself, fans, each drive's health, and the kernel's counts of memory and PCIe errors (novox/hq ADR 0256). It reads, and changes nothing on the hardware. Every flag is measured against the hardware's own threshold, never one the mesh chose. A class the machine does not have is said as not present, never as an error, so the module can be assigned to every machine: a laptop, a desktop, a server, a virtual machine.

Why it exists, and why here

The operator set the boundary on 2026-10-08. What hardware a machine has is a node-engine fact. A module holding a generic role reads and provides it. A vendor module adds only vendor-specific extras.

Before this module, only the laptop's vendor module read a temperature, and no module read a drive's health. The machines' 20 drives, 15 of them spinning, were watched by nothing. power keeps the battery and the power draw. disk-load keeps how busy the disks are. This module keeps everything else the kernel says about the hardware's condition.

It holds no seat. A seat is a role that callers reach without knowing the module, and nothing asks for this role yet. A new seat would also be a controller change first (ADR 0246). When a caller needs to ask by role, such as the bar's temperature blocks or a vendor module's fan policy, these tools become the seat's verbs. The full reasoning is in ADR 0256.

What runs

resource	what	as
smartmontools	the package: smartctl and smartd
/etc/smartd.conf	every drive (DEVICESCAN), every check, never waking a drive in standby, no mail	written whole
smartd.service drop-in	runs on a virtual machine too, stays up with no drive to watch (-q never), and checks each drive hourly (-i 3600)	written whole
smartd	watches every drive between looks, and logs what it finds to the journal	root, the package's unit
watch (sensors watch)	each drive's health with /usr/bin/smartctl --json -a -n standby,2,3: a spinning drive hourly, any other every 10 min; every 30 s which fans turn; one file, /run/mesh-sensors/readings.json (0644, in a 0755 directory)	root, the node-engine's process
the tools	read /sys, /proc and the watch's file, once per call	the operator account, in the tool runner

The tools need no root, because the watch has already read what only root may read.

The drives never hold up the fans. The watch reads the drives beside its looks at the fans, one pass at a time. Each smartctl is bounded by 20 s, plus 2 s for a child that keeps its output open. The file is written every 30 s whatever a drive does, so a file older than five minutes means the watch is hung or gone, and the health check says so.

A drive is never woken, and is read seldom enough to sleep.

• smartctl's -n standby,2,3 stops with status 2 and the drive's power mode when the drive is in standby or asleep. The watch then keeps the reading before and says the drive was asleep. A drive is asleep only on status 2 with a parsed power mode (or smartctl's own "STANDBY (OS)").
• It stops with status 3 when the drive cannot say (a USB bridge that does not pass CHECK POWER MODE on). Such a drive is not read, so that reading it never wakes it, and it is said as not read.
• Reading SMART from an awake drive can restart its standby timer, so a drive read more often than its timer is set for never spins down. No mesh tool reads a drive's standby timer, so whether the servers' drives spin down is not known. A spinning drive is therefore read hourly, by the watch and by smartd. A drive whose timer is set longer than an hour may still be kept awake.
• smartd has no per-drive equivalent of status 3. A drive behind such a bridge may be woken by smartd.

Tools

All six only read.

tool	what	replaces
sensors_summary	one line per finding, flagged CRITICAL, HIGH, FAILING or STOPPED; then one line per class: temperatures with the hottest, fans, drives, memory and PCIe errors	sensors, smartctl -H
sensors_temperatures	every thermal zone with its trip points, and every hwmon temperature with its label, its kind (cpu, gpu, drive, board, network), the drive it sits in, and its own maximum and critical values	sensors
sensors_fans	every fan's RPM, its chip's minimum and alarm, and whether the watch saw it turn since boot	sensors
sensors_drives	each drive: model, serial, size, whether it spins, the SMART verdict; for NVMe the critical warning bits, spare, endurance used and media errors; for SATA the reallocated, pending and unreadable sectors, the error log and the last self-test	smartctl -a, nvme smart-log
sensors_faults	EDAC memory errors per controller and module, PCIe AER errors per device, and how often an Intel CPU was slowed for heat	edac-util
sensors_health	the module's health check: healthy or not, with why

The flags

flag	when
temperature HIGH	at or above its own maximum, or a zone's hot trip (without one, its lowest passive trip)
temperature CRITICAL	at or above its own critical or emergency value, or a zone's critical trip
drive FAILING	SMART's overall verdict failed (SATA), a pre-failure attribute is failing now, or an NVMe critical warning bit other than temperature is set
drive HIGH	an old-age attribute failing now; an attribute that failed in the past; the NVMe temperature bit alone; reallocated, pending or unreadable sectors; NVMe media errors or 90 % of rated endurance used; errors in the error log; a last self-test that did not pass
fan STOPPED	it ran since boot, reads 0 now, and the CPU, the board or a sensor on its own chip is at or above its own maximum
fan note	under its chip's minimum, or its chip's alarm bit set: said, never flagged
memory HIGH / CRITICAL	corrected / uncorrectable errors since boot
PCIe HIGH	fatal or non-fatal errors since boot

What is no threshold, and no reading.

• A threshold given as the format's largest value is no threshold. The laptop's NVMe drive gives 65261.85 °C as the maximum of two of its sensors.
• A threshold of 0 is no threshold either, and neither is anything outside −40 to 200 °C. The reading is then said without a flag.
• A CPU sensor that gives no threshold, such as AMD's Tctl, is never flagged.
• 0 °C is a reading. 127 and 127.5 °C are not: they are what a board chip's input with nothing on it answers (a phantom AUXTIN). Such a sensor is said as not read.

Why a fan at 0, a minimum or an alarm is not enough. Laptop fans and GPU fans in zero-fan mode stop on purpose when the machine is cool, under the minimum their chip gives. A board chip latches an alarm bit, and a header with nothing on it reads 0. Only a fan that ran and stopped while what it cools is hot is a fault. A hot drive or network adapter says nothing of a fan.

Health

The module declares its health under ADR 0240:

• smartd has the unit kind: the unit's own readiness, which the module does not judge itself.
• The watch has the tool kind: sensors_health, every 60 s, unhealthy after 2 looks in a row, with a 60 s grace.

sensors_health is unhealthy when:

• a CPU, GPU, drive or zone temperature is at or above its own critical value. A board chip's (Super I/O) critical value is shown CRITICAL by the tools and never makes the module unhealthy: its limits are whatever the firmware left;
• a drive says it is failing (the FAILING row above);
• a fan is STOPPED (the row above);
• memory reports an uncorrectable error;
• the watch has written no readings since the machine started, or none for five minutes.

Its why is one plain sentence per finding, for the operator. For example: "The drive sdb (WDC WUH722222ALE6L4) says it is failing: its overall SMART health check failed." HIGH readings and fan notes raise nothing. A drive whose health was not read (asleep since boot, or behind a bridge) is said in why and does not make the check unhealthy.

The condition this raises is module.sensors.<machine>.unhealthy. ADR 0256 records what this costs:

• the condition names the module, not the machine;
• a standing hardware fault holds this module's own new builds at their gate on that machine.

The machine judging its own hardware, as it judges its network, is the follow-up.

Tests

go test ./.... The tests read a tree standing in for /sys, /proc and /run, and smartctl answers as smartmontools 7.5 writes them (its standby answer as its source sets it). There are three machines:

• a laptop, as the laptop showed itself on 2026-10-08;
• a server with ECC memory, three SATA drives read by drivetemp and a board chip with an empty fan header;
• a virtual machine with no sensor at all.

They cover:

• thresholds taken from the hardware only, and a zone's hwmon copy left out;
• HIGH at the maximum and CRITICAL at the critical value; a zone's passive trip as its high;
• 0 °C as a reading, a phantom 127 °C as not read, and a board chip's critical value raising nothing;
• every class not present on the virtual machine, which is healthy;
• a failing SATA drive; an NVMe critical warning; the NVMe temperature bit alone as HIGH; NVMe without smart_status; an old-age attribute failing now and a past failure as HIGH;
• a missing, stale or earlier-boot reading, and the watch's staleness making the module unhealthy;
• a fan at rest, a fan that stopped while the board is hot, a hot drive saying nothing of a fan, a hot sensor on the fan's own chip, an empty header, a zero-fan GPU under its minimum, and a latched alarm;
• memory and PCIe errors, and CPU throttling;
• the watch reading as root without waking a drive: smartctl's real standby answer, an open failure that is not sleep, and a bridge without a power mode;
• a spinning drive read hourly and a solid-state one every 10 min;
• a smartctl that hangs with a child holding its output, bounded, and the fans looked at and written while a drive hangs;
• a restart keeping what the last watch read; the readings directory given its mode;
• a watch that is not root asking through sudo -n, and saying what stops it;
• the health answer in the shape the node-engine reads;
• the manifest: the tools listed are the tools served, all read-only, no seat claimed, no machine named, and the two health checks.

Follow-ups

• The machine's hardware judged by the node-engine, raising a machine condition, instead of this module's health (ADR 0256).
• The module-health condition carrying the tool check's why as its plain words (ADR 0253).
• Temperature blocks for the bar, once there is a generic way for a module to provide bar blocks.
• The laptop vendor module's zephyrus_thermals keeps only what is the vendor's.