A one-shot at rest satisfies a stopped declaration, so the kernel's boot cleanup applies (issue 1126) #162

2026-10-28T13:30:28Z

mesh-tools commented

2026-10-28 13:30:28 +00:00

Issue 1126 notes 73-75. Group fix/ace-tun's kernel delivery failed on ace: 'kernel.modules-cleanup failed: linux-modules-cleanup.service was asked to be stopped and is running'. The unit (from kernel-modules-hook) is Type=oneshot without RemainAfterExit, declared stopped and enabled at boot, and ace's systemd read it inactive/dead, never run.

Cause: ServiceState reads a non-lingering one-shot at rest with exit 0 as "running" (the firewall's one-shot fix). That is right for a one-shot declared running, but a stopped declaration could then never be satisfied: systemctl stop does nothing and the read back says running again.

Fix: the service apply reads the unit through declaredState. Only when the declaration says stopped and ServiceState says running does it ask the new arch.ServiceAtRest, and a one-shot at rest then counts as stopped. ServiceState itself is unchanged, so the hold, take-over and every running declaration read as before. Round 1's "start timestamp 0 means never ran" was dropped: systemd also shows 0 for a finished one-shot it unloaded (measured on systemd 262.1), so it could have looped.

Quick vs thorough: quick was declaring the cleanup "running", which passes only through the misreading; thorough fixes the reading for every stopped one-shot on every node.
Recovery: once delivered, the node-engine's restart on each node applies a failed kernel have again; no new kernel build is needed.
Tests: arch table (5 rows), apply row (fails on main). Unit count 1241 (was 1242). TestNatsANodeThePlacementNamesComposesAndAppliesItsOwn passes (25.8 s).

Reviewed-by: rev-kcleanup@shanks at 1a021052fe

Issue 1126 notes 73-75. Group fix/ace-tun's kernel delivery failed on ace: 'kernel.modules-cleanup failed: linux-modules-cleanup.service was asked to be stopped and is running'. The unit (from kernel-modules-hook) is Type=oneshot without RemainAfterExit, declared stopped and enabled at boot, and ace's systemd read it inactive/dead, never run. Cause: ServiceState reads a non-lingering one-shot at rest with exit 0 as "running" (the firewall's one-shot fix). That is right for a one-shot declared running, but a stopped declaration could then never be satisfied: `systemctl stop` does nothing and the read back says running again. Fix: the service apply reads the unit through declaredState. Only when the declaration says stopped and ServiceState says running does it ask the new arch.ServiceAtRest, and a one-shot at rest then counts as stopped. ServiceState itself is unchanged, so the hold, take-over and every running declaration read as before. Round 1's "start timestamp 0 means never ran" was dropped: systemd also shows 0 for a finished one-shot it unloaded (measured on systemd 262.1), so it could have looped. Quick vs thorough: quick was declaring the cleanup "running", which passes only through the misreading; thorough fixes the reading for every stopped one-shot on every node. Recovery: once delivered, the node-engine's restart on each node applies a failed kernel have again; no new kernel build is needed. Tests: arch table (5 rows), apply row (fails on main). Unit count 1241 (was 1242). TestNatsANodeThePlacementNamesComposesAndAppliesItsOwn passes (25.8 s). Reviewed-by: rev-kcleanup@shanks at 1a021052fe07b8cb4a0a74f4e9d3f2ad58c2efe5

mesh-tools added 2 commits 2026-10-28 13:30:28 +00:00

Read a one-shot that never ran as stopped, and one at rest as stopped to a stopped declaration (issue 1126) cf738cdb90

The kernel module's boot-time cleanup, linux-modules-cleanup.service (Type=oneshot,
declared stopped and enabled at boot), failed its delivery on ace with 'was asked to
be stopped and is running': ServiceState read every inactive one-shot with
ExecMainStatus=0 as running, and systemd says 0 for one that never ran too. Stopping
an inactive unit changes nothing, so the read back said running again; after a boot
that ran it, the same declaration would fail every apply.

ServiceState now reads ExecMainStartTimestampMonotonic: 0 means never ran, read as
stopped (a package's one-shot declared running is then started, and the hold no
longer counts it as in use). The service apply reads a one-shot at rest as stopped
to a declaration that it be stopped (ServiceAtRest), whatever its last run did.

Quick fix weighed: declare the cleanup running in the catalogue. Thorough chosen:
the reading was wrong for every one-shot, and this is its second fault of the kind
(the firewall's was the first). Unit tests: three arch one-shot tests merged into one
table (7 rows), one apply test (2 rows); 1263 to 1262.

Drop the never-ran reading: a start timestamp of 0 does not mean a one-shot never ran (issue 1126)

mesh/delivery merged: source.main moved to novox/mesh-host@b4fd2cc210bf

Details

mesh/check being carried to 1a021052fe07 on b4fd2cc210bf | from 7a2af2376405

1a021052fe

Review round 1 measured it on systemd 262.1: a finished one-shot nothing references is
unloaded, and loaded again it reads inactive, ExecMainStatus=0 and a start timestamp of
0, as one that never ran does. The previous commit's rule read such a unit stopped, so
a one-shot declared running would be started, read stopped and fail on every apply, and
the hold, the take-over and the giving back would judge it wrongly.

ServiceState keeps main's reading. ServiceAtRest and declaredState stay: they alone
satisfy a stopped declaration for a one-shot at rest, which is the kernel cleanup's
case. A one-shot declared running that must be seen to have run lingers
(RemainAfterExit) or is enabled at boot. Unit tests: the arch table has 5 rows, the
apply test 1; 1262 in all.

mesh-tools commented

2026-10-28 13:30:34 +00:00

Delivery novox/mesh-host@1a021052fe07 — merged since 2026-10-28T13:34:23Z

Source — source.main moved to novox/mesh-host@b4fd2cc210. Whether it reached the machines is mesh-delivery-view's, derived from source.main, the wants and the haves.

Transitions

2026-10-28 13:30 (new) → proposed (announced): novox/mesh-host#162's head announced
2026-10-28 13:30 proposed → ready (accepted): a verdict names this commit and the trunk's head as its trunk commit, the gate passed or warned, and the repository's own check did not fail; or the check gates no merge (check-gates-merge false)
2026-10-28 13:34 ready → merged (merged): on the trunk its modules follow: merged there

The commit's note under refs/notes/mesh-plan keeps every transition of this change up to its merge: git log --notes=mesh-plan.

**Delivery** `novox/mesh-host@1a021052fe07` — **merged** since 2026-10-28T13:34:23Z **Source** — source.main moved to novox/mesh-host@b4fd2cc210bf. Whether it reached the machines is `mesh-delivery-view`'s, derived from `source.main`, the wants and the haves. **Transitions** - 2026-10-28 13:30 (new) → proposed (announced): novox/mesh-host#162's head announced - 2026-10-28 13:30 proposed → ready (accepted): a verdict names this commit and the trunk's head as its trunk commit, the gate passed or warned, and the repository's own check did not fail; or the check gates no merge (check-gates-merge false) - 2026-10-28 13:34 ready → merged (merged): on the trunk its modules follow: merged there The commit's note under `refs/notes/mesh-plan` keeps every transition of this change up to its merge: `git log --notes=mesh-plan`.

mesh-tools merged commit b4fd2cc210 into main

2026-10-28 13:34:21 +00:00

mesh-tools deleted branch fix/ace-tun-cleanup

2026-10-28 13:34:21 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: novox/mesh-host#162