A service is restarted only after every file it names is written (hq issue 260) #25

2026-10-05T19:56:02Z

mesh-admin commented

2026-10-05 19:56:02 +00:00

Fixes novox/hq issue 260 (04-ISSUES/260-the-resolver-started-before-its-zones-file-existed).

Cause

The apply loop takes resources in declared order, and a service acts on restart-on / reload-on at its own place, using changed, which only knows what moved earlier in this apply. The resolver's service is declared before its zones file, because a module's facts are appended after its resources. So the service was restarted for config while fact-zones did not exist yet, and the daemon failed its first start on every machine.

The same ordering also caused a quieter fault: a change to a file declared after its service was never acted on. changed is edge-triggered within one apply, so by the time the file moved, the service had already been passed.

Fix

afterWhatTheyRead (internal/apply/apply.go) orders the declaration before it is applied. Each service is placed right after the last resource it names under restart-on or reload-on that is declared after it. The ids are matched whole, so a name from another module works. A name not in the declaration is ignored, as it already is when the restart is decided.

Nothing else moves. Files, directories, containers, processes, actions and run-once steps keep their declared places, so module gating and directory-before-file order are unchanged.
On an adopted node, the guard part and the rest are ordered separately, so a guard service never leaves the part applied first.
A service waits only for resources declared after it, so no cycle can form.
A reload stays a reload: docker.service with reload-on only moves; how it is acted on is unchanged (ADR 0102).
If a named resource fails, it is not in changed, as before. The service is still applied after it and restarts only for what did change.

Containers and processes are left as they were. A container's spec already folds in what it reads, so one declared ahead of its file sees the change one apply late but never misses it. Moving a run-once step would risk the gate it holds over its module.

Test

internal/apply/restart_order_test.go:

TestAServiceIsRestartedOnlyOnceEveryFileItNamesIsWritten: config, then the service (restart-on: [config, fact-zones]), then the zones file, with a daemon that cannot start without the zones file. Checks one restart after both files, with both named in the outcome. A second apply that changes only the zones file must restart the service.
TestAServiceIsOrderedAfterWhatItNamesAndNothingElseMoves: covers restart-on, reload-on from another module, an undeclared name, and checks that nothing else moves.

With the reordering turned off, both tests fail. The first fails with the issue's own message: restarting resolver.service: starting it again: cannot read .../zones.conf. With the fix, gofmt -l is clean, and go vet ./... and go test ./... pass.

Fixes novox/hq issue 260 (04-ISSUES/260-the-resolver-started-before-its-zones-file-existed). ## Cause The apply loop takes resources in declared order, and a service acts on `restart-on` / `reload-on` at its own place, using `changed`, which only knows what moved earlier in this apply. The resolver's service is declared before its zones file, because a module's facts are appended after its resources. So the service was restarted for `config` while `fact-zones` did not exist yet, and the daemon failed its first start on every machine. The same ordering also caused a quieter fault: a change to a file declared after its service was never acted on. `changed` is edge-triggered within one apply, so by the time the file moved, the service had already been passed. ## Fix `afterWhatTheyRead` (internal/apply/apply.go) orders the declaration before it is applied. Each service is placed right after the last resource it names under `restart-on` or `reload-on` that is declared after it. The ids are matched whole, so a name from another module works. A name not in the declaration is ignored, as it already is when the restart is decided. - Nothing else moves. Files, directories, containers, processes, actions and run-once steps keep their declared places, so module gating and directory-before-file order are unchanged. - On an adopted node, the guard part and the rest are ordered separately, so a guard service never leaves the part applied first. - A service waits only for resources declared after it, so no cycle can form. - A reload stays a reload: `docker.service` with `reload-on` only moves; how it is acted on is unchanged (ADR 0102). - If a named resource fails, it is not in `changed`, as before. The service is still applied after it and restarts only for what did change. Containers and processes are left as they were. A container's spec already folds in what it reads, so one declared ahead of its file sees the change one apply late but never misses it. Moving a run-once step would risk the gate it holds over its module. ## Test `internal/apply/restart_order_test.go`: - `TestAServiceIsRestartedOnlyOnceEveryFileItNamesIsWritten`: config, then the service (`restart-on: [config, fact-zones]`), then the zones file, with a daemon that cannot start without the zones file. Checks one restart after both files, with both named in the outcome. A second apply that changes only the zones file must restart the service. - `TestAServiceIsOrderedAfterWhatItNamesAndNothingElseMoves`: covers restart-on, reload-on from another module, an undeclared name, and checks that nothing else moves. With the reordering turned off, both tests fail. The first fails with the issue's own message: `restarting resolver.service: starting it again: cannot read .../zones.conf`. With the fix, `gofmt -l` is clean, and `go vet ./...` and `go test ./...` pass.

mesh-admin added 1 commit 2026-10-05 19:56:03 +00:00

Restart a service only after every file it names is written (hq issue 260) 101f0c0d61

A service was restarted where it was declared, so one declared ahead of a
file in its restart-on was restarted before that file existed (the
resolver's zones file, a fact appended after its module's resources), and
a later change to such a file was never acted on. Each service is now
applied right after the last resource it names under restart-on or
reload-on; nothing else moves.

mesh-admin merged commit a566add815 into main

2026-10-05 19:57:22 +00:00

mesh-admin deleted branch fix/a-service-restarts-after-every-file-it-reads

2026-10-05 19:57:22 +00:00

mesh-admin commented

2026-10-06 23:10:48 +00:00

Delivery novox/mesh-host@a566add81590 — held since 2026-10-06T23:05:28Z

Held for a person: merged without a passing check: only a person decides that it goes on — mesh-delivery.release with why lets it go on.

Delivery plan — builds mesh-host → ace, g14, novox, shanks; no bus step

tier 0: mesh-host
ace: receives mesh-host
g14: receives mesh-host
novox: receives mesh-host
shanks: receives mesh-host

Transitions

2026-10-06 23:05 (new) → proposed (announced): novox/mesh-host#25 merged; its head was not heard by this owner, so it is made from the forge's word
2026-10-06 23:05 proposed → held (merged-unchecked): merged without a passing check — a verdict known with it is taken first: only a person decides that it goes on

The commit's note under refs/notes/mesh-plan keeps every transition: git log --notes=mesh-plan.

**Delivery** `novox/mesh-host@a566add81590` — **held** since 2026-10-06T23:05:28Z Held for a person: merged without a passing check: only a person decides that it goes on — `mesh-delivery.release` with why lets it go on. **Delivery plan** — builds mesh-host → ace, g14, novox, shanks; no bus step - tier 0: mesh-host - ace: receives mesh-host - g14: receives mesh-host - novox: receives mesh-host - shanks: receives mesh-host **Transitions** - 2026-10-06 23:05 (new) → proposed (announced): novox/mesh-host#25 merged; its head was not heard by this owner, so it is made from the forge's word - 2026-10-06 23:05 proposed → held (merged-unchecked): merged without a passing check — a verdict known with it is taken first: only a person decides that it goes on The commit's note under `refs/notes/mesh-plan` keeps every transition: `git log --notes=mesh-plan`.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: novox/mesh-host#25