jochen
66a816c3ac
The second review of #129 found a read stuck in the kernel kept the file fresh and the module healthy, smartd as a second reader could wake or keep awake the drives the watch spares, and a board chip's maximum could stop a fan. Read each drive in its own goroutine from a root reader of the module's own, mark each read in flight, and say a read past its bound or a drive not looked at for twice its cadence; drop smartd; tell smartctl each drive's kind; take a SAS drive's standby; say a drive asleep since when.
87 lines
4.4 KiB
Go
87 lines
4.4 KiB
Go
package main
|
|
|
|
import (
|
|
stdio "git.novox.be/novox/mesh-sdk/go"
|
|
)
|
|
|
|
// Tools is the module's tools over one machine. Every one only reads, and none needs root: what only root
|
|
// may read, the drive reader has read (drives.go, watch.go).
|
|
func Tools(m *Machine) []stdio.Tool {
|
|
return []stdio.Tool{
|
|
{
|
|
Name: "sensors_summary",
|
|
Description: "The condition of this machine's hardware in a few lines: one per finding, flagged CRITICAL, HIGH, FAILING or " +
|
|
"STOPPED against the hardware's own thresholds (never the mesh's), then one per class — temperatures with the hottest, " +
|
|
"fans, drives' health, memory and PCIe errors — saying what was read and what the machine does not have. Replaces " +
|
|
"sensors and smartctl -H. (r)",
|
|
Run: func(map[string]any) (any, error) {
|
|
l := m.LookNow()
|
|
h := l.Health()
|
|
return map[string]any{"said": l.Said(), "healthy": h.Healthy}, nil
|
|
},
|
|
},
|
|
{
|
|
Name: "sensors_temperatures",
|
|
Description: "Every temperature the machine reports, in °C: each thermal zone with its trip points, and each hwmon sensor " +
|
|
"(CPU, GPU, drives, board, network adapters) with its label and its own maximum and critical values. HIGH at its own " +
|
|
"maximum (or a zone's hot or passive trip), CRITICAL at its own critical value; a sensor that gives no threshold is " +
|
|
"never flagged, and one that gives no reading is said as not read. Replaces sensors. (r)",
|
|
Run: func(map[string]any) (any, error) {
|
|
ts := m.Temperatures()
|
|
return map[string]any{"temperatures": ts, "count": len(ts)}, nil
|
|
},
|
|
},
|
|
{
|
|
Name: "sensors_fans",
|
|
Description: "Every fan the machine reports: RPM, the chip's own minimum and alarm where it has them (said in a note, never " +
|
|
"flagged: a zero-fan GPU is under its minimum by design and a board chip latches its alarm), and whether the " +
|
|
"module's fan watch saw it turning since the machine started. STOPPED only when it ran and reads 0 while a CPU " +
|
|
"sensor, a thermal zone or a sensor on its own non-board chip is at or above its own maximum. " +
|
|
"Replaces sensors. (r)",
|
|
Run: func(map[string]any) (any, error) {
|
|
l := m.LookNow()
|
|
return map[string]any{"fans": l.Fans, "count": len(l.Fans)}, nil
|
|
},
|
|
},
|
|
{
|
|
Name: "sensors_drives",
|
|
Description: "Each drive's health, as the module's drive reader last read it as root with smartctl (a spinning drive " +
|
|
"hourly, any other every 10 minutes; a drive in standby is not read, and said as asleep since when): model, serial, size, whether it spins, the overall SMART verdict; for NVMe the critical " +
|
|
"warning bits, spare, endurance used and media errors; for SATA the reallocated, pending and unreadable sectors, the " +
|
|
"error log and the last self-test. FAILING when the drive says so, HIGH for the early signs. Replaces smartctl -a " +
|
|
"and nvme smart-log. (r)",
|
|
Run: func(map[string]any) (any, error) {
|
|
ds, note := m.Drives(m.ReadReadings(DrivesPath))
|
|
out := map[string]any{"drives": ds, "count": len(ds)}
|
|
if note != "" {
|
|
out["note"] = note
|
|
}
|
|
return out, nil
|
|
},
|
|
},
|
|
{
|
|
Name: "sensors_faults",
|
|
Description: "The kernel's counts of hardware errors since the machine started: corrected and uncorrectable memory " +
|
|
"errors per memory controller and module (EDAC, where the memory has ECC), PCIe errors per device that counted one " +
|
|
"(AER), and how often an Intel CPU was slowed for heat. A machine without a class has it as not present. Replaces " +
|
|
"edac-util and reading /sys by hand. (r)",
|
|
Run: func(map[string]any) (any, error) {
|
|
return m.Faults(), nil
|
|
},
|
|
},
|
|
{
|
|
Name: "sensors_health",
|
|
Description: "Whether the hardware is failing, as the module's health check (novox/hq ADR 0256): unhealthy when a " +
|
|
"CPU, GPU, drive or zone temperature is at or above its own critical value (a board chip's is shown, never judged), " +
|
|
"a drive says it is failing, a fan that ran reads 0 while a CPU sensor, a thermal zone or its own (non-board) chip is at its own maximum, " +
|
|
"memory reports an uncorrectable error, or the module's own processes are stale (a file five minutes old, a drive's " +
|
|
"read stuck, a drive not looked at for twice its cadence); with " +
|
|
"why, one sentence per finding. HIGH readings, a fan's alarm bit and a fan under its minimum do not make it " +
|
|
"unhealthy. (r)",
|
|
Run: func(map[string]any) (any, error) {
|
|
return m.LookNow().Health(), nil
|
|
},
|
|
},
|
|
}
|
|
}
|