A service the mesh asked to run is still running a moment later (hq ADR 0184) #75

2026-10-02T16:16:44Z

mesh-admin commented

2026-10-02 16:16:44 +00:00

The host already read a unit back after acting on it, with comments in three places saying why. The read raced the failure: a service manager returns when it has started the process, and a daemon that refuses the configuration the mesh just wrote exits a fraction of a second later. fail2ban took 221 ms on the control node; the apply reported "restarted" onto a dead daemon, and both public machines kept no bans at all with every check passing.

The host now looks a second time, after that moment. A unit still coming up reads as running at both looks and is accepted. A service asked to be stopped is not waited on. No command for this reaches a machine: a module declaring how to test its configuration was weighed and refused, because the link carries no actions.

Tests in internal/apply/stayed_running_test.go; the first one fails against main, which is how I know it catches the real fault. Full suite green.

The host already read a unit back after acting on it, with comments in three places saying why. The read raced the failure: a service manager returns when it has started the process, and a daemon that refuses the configuration the mesh just wrote exits a fraction of a second later. fail2ban took 221 ms on the control node; the apply reported "restarted" onto a dead daemon, and both public machines kept no bans at all with every check passing. The host now looks a second time, after that moment. A unit still coming up reads as running at both looks and is accepted. A service asked to be stopped is not waited on. No command for this reaches a machine: a module declaring how to test its configuration was weighed and refused, because the link carries no actions. Tests in `internal/apply/stayed_running_test.go`; the first one fails against main, which is how I know it catches the real fault. Full suite green.

mesh-admin added 1 commit 2026-10-02 16:16:44 +00:00

A service the mesh asked to run is still running a moment later (hq ADR 0184) 286865dfa7

The read-back raced the failure: a service manager returns when it has started the process, and a
daemon that refuses its configuration exits a fraction of a second later, so one look saw it alive.
fail2ban took 221ms on the control node and the apply reported "restarted" onto a dead daemon while
both public machines kept no bans at all. The host looks again, after that moment. A unit still
starting is accepted at both looks; a service asked to stop is not waited on.

mesh-admin merged commit b840ce0a77 into main

2026-10-02 16:25:26 +00:00

mesh-admin deleted branch fix/a-service-asked-to-run-is-still-running

2026-10-02 16:25:26 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: novox/mesh-host#75