From 68018c7c1ba4246a7c375f56ff5dae773681f9ba Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 9 Oct 2026 01:35:27 +0200 Subject: [PATCH 1/5] ADR 0269 (proposed): what the mesh's own declaration stopped is started again when it goes Unassigning the zfs module would leave the home server's pool with no scrub schedule: ADR 0118 never starts a unit on the way out, so the distribution's timer the module held stopped would stay stopped. The host now records what it declared beside what it found, and gives back only what the mesh's own declaration stopped. --- ...n-stopped-is-started-again-when-it-goes.md | 90 +++++++++++++++++++ 1 file changed, 90 insertions(+) create mode 100644 02-DECISIONS/0269-what-the-meshs-own-declaration-stopped-is-started-again-when-it-goes.md diff --git a/02-DECISIONS/0269-what-the-meshs-own-declaration-stopped-is-started-again-when-it-goes.md b/02-DECISIONS/0269-what-the-meshs-own-declaration-stopped-is-started-again-when-it-goes.md new file mode 100644 index 00000000..9b79485d --- /dev/null +++ b/02-DECISIONS/0269-what-the-meshs-own-declaration-stopped-is-started-again-when-it-goes.md @@ -0,0 +1,90 @@ +--- +topic: what runs on it +status: proposed +date: 2026-10-09 +deciders: jochen +reconstructed: false +extends: 02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md +--- + +# 269. What the mesh's own declaration stopped is started again when it goes + +## Context + +[ADR 0118](0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md) gives a unit back as the +host found it, with one rule written against the direction of the rest: **never started on the way +out.** A unit the mesh stopped is not started again when its declaration goes, because "starting +something is a decision, and the operator makes it". When 0118 was written, the units the mesh stopped +were its own: a packet filter it had started, a guard it had loaded. Stopping a unit the machine ran +before the mesh came was not a case that existed. + +It exists now. The `zfs` module owns the scrub schedule of the home server's pool. The machine already +had one: the distribution's `zfs-scrub-weekly@.timer`, enabled by hand, found running and enabled. +To leave exactly one schedule, the module declares that timer stopped and disabled, in the same +declaration that starts its own. Unassigning the module removes the module's own timer, which the mesh +made, and gives the distribution's back under 0118. That keeps it stopped and disabled. So the pool +would be left with **no scrub schedule at all**, and nothing would say so. The second review of +mesh-catalog #147 found this on 2026-10-09. + +The host already knows half of what it needs. It records what it **found** (the unit's state and boot +before the mesh touched it) and keeps that from the first apply on. It did not record what the mesh +**declared**. So at removal it could not tell a unit that the mesh's own declaration stopped from one +that somebody stopped since. + +## Considered Options + +1. **Keep 0118 as it is, and have the module say a required step before unassigning** (re-enable the + distribution's timer by hand), with a health warning. Rejected: a step a person must remember is the + failure mode of the record ("failure must be loud", ADR 0227). The warning would come from a module + that is going away, so it would be said once and then never again. +2. **Start again, on removal, every unit found running.** Rejected: a unit somebody else stopped after + the mesh, an operator or another module, would be started behind their back. That is what 0118 + refused, for good reason. +3. **Start again only what the mesh's own declaration stopped** — chosen. The host records what it + declared beside what it found. At removal, a unit found running that the declaration held stopped is + started again, and one found enabled that the declaration held disabled is enabled again. + +## Decision + +**The host records, for every service with a state, what the mesh declared: running or stopped, and +enabled or disabled where the declaration says. When the service's declaration goes, a unit that was +found running and declared stopped is started again, and one found enabled and declared disabled is +enabled again. The report says so as `restored`, and the plan says it before the apply does.** + +0118's other rules stand: + +- a unit found running and never stopped by the mesh is left running; +- a unit the mesh started is stopped again; +- a unit whose file the mesh wrote is the mesh's own; +- a unit with an unknown history is left alone. A record written before the host kept what it declared + says nothing about who stopped the unit, so nothing is started. + +"Never started on the way out" now reads **never started on the way out, but for what the mesh's own +declaration stopped**. Starting something is still the operator's decision. Here the operator made it +before the mesh came, and the mesh's stopping it was the mesh's doing, which goes with the mesh. + +## Consequences + +- Unassigning the `zfs` module gives the home server back its distribution scrub timer, running and + enabled. A module may take over a machine's unit by holding it stopped, and its removal undoes that. +- A unit that is moved to another unit name (ADR 0118's "moved") is given back by the same rule. +- **Harder:** a unit the mesh stopped and that should *stay* stopped after the module goes (one that + was found running by accident) is now started again. Saying so needs a field on the resource, which + no module needs yet. Until one does, the plan's `restore … started … again` step is where it is seen. + +## How it is checked + +| Rule | Checked by | +|---|---| +| A unit found running and enabled, declared stopped and disabled, is started and enabled again when undeclared, and the report says `restored` | mesh-host `internal/apply/giveback_test.go`, `TestAUnitTheMeshStoppedAndDisabledIsGivenBackRunningAndEnabled` | +| A unit found stopped stays stopped; one never stopped by the mesh is left as it is | `TestOnlyWhatTheMeshsDeclarationStoppedIsStartedAgain` | +| A record without what was declared starts nothing | `TestARecordWithoutWhatWasDeclaredStartsNothing` | +| The plan says it before the apply does | `TestThePlanSaysAStoppedUnitIsStartedAgain` | +| 0118's existing rules | mesh-host `internal/apply/found_test.go`, unchanged and passing | + +## References + +- [ADR 0118](0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md), which this widens. +- mesh-host: the host's record (`store.Applied.Declared`), `startUnitBack` in `internal/apply/apply.go`, + and the plan's step in `internal/apply/plan.go`. +- mesh-catalog #147 (the `zfs` module's scrub schedule) and its second review, 2026-10-09. From 2e2be955e9a8002a2d0ea6aa61f318ac0286b568 Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 9 Oct 2026 01:54:42 +0200 Subject: [PATCH 2/5] ADR 0269: say that a unit stopped by hand while held cannot be told apart, that a restored Persistent timer catches up at once, and what is left alone Third review: a person's stopping a unit the mesh also held stopped leaves no trace, so it is started again on removal; the home server's restored weekly timer would start a scrub of about 46 hours at once. A masked or since-changed unit is left alone, and a failed restore waits. --- ...n-stopped-is-started-again-when-it-goes.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/02-DECISIONS/0269-what-the-meshs-own-declaration-stopped-is-started-again-when-it-goes.md b/02-DECISIONS/0269-what-the-meshs-own-declaration-stopped-is-started-again-when-it-goes.md index 9b79485d..4101019d 100644 --- a/02-DECISIONS/0269-what-the-meshs-own-declaration-stopped-is-started-again-when-it-goes.md +++ b/02-DECISIONS/0269-what-the-meshs-own-declaration-stopped-is-started-again-when-it-goes.md @@ -51,6 +51,12 @@ enabled or disabled where the declaration says. When the service's declaration g found running and declared stopped is started again, and one found enabled and declared disabled is enabled again. The report says so as `restored`, and the plan says it before the apply does.** +**Only a unit exactly as the mesh held it.** At removal the unit must still be the stopped and disabled +the declaration held it to. A masked unit is somebody's word that it must not run, and is left as it is. +So is a unit somebody started or enabled since. In both cases the report says why. A restore that fails +(the start refused, or the unit not running after it) puts back what that attempt changed. It is reported +as *waiting*, never as an error that makes an apply fatal, and the next apply tries again. + 0118's other rules stand: - a unit found running and never stopped by the mesh is left running; @@ -68,6 +74,16 @@ before the mesh came, and the mesh's stopping it was the mesh's doing, which goe - Unassigning the `zfs` module gives the home server back its distribution scrub timer, running and enabled. A module may take over a machine's unit by holding it stopped, and its removal undoes that. - A unit that is moved to another unit name (ADR 0118's "moved") is given back by the same rule. +- **A unit a person stopped by hand while the mesh held it cannot be told apart, and is started again.** + While the declaration holds a unit stopped and disabled, a person who also stops or disables it by hand + leaves no trace: the unit is in the state the mesh holds it to either way. At removal it is started and + enabled again, as it was found before the mesh. A person who wants it to stay off after the module goes + must disable it again afterwards, or mask it, which removal respects. +- **A restored timer catches up at once.** A timer with `Persistent=true` that was stopped past its + scheduled time fires as soon as it is started again. On the home server the distribution's weekly scrub + timer is such a timer, and a scrub of its pool took about 46 hours (2026-10-05 00:08 to 2026-10-06 + 22:06). Unassigning the `zfs` module therefore starts a scrub of about 46 hours, within the hour of + the timer's random delay. Unassign it when that load is acceptable. - **Harder:** a unit the mesh stopped and that should *stay* stopped after the module goes (one that was found running by accident) is now started again. Saying so needs a field on the resource, which no module needs yet. Until one does, the plan's `restore … started … again` step is where it is seen. @@ -79,6 +95,9 @@ before the mesh came, and the mesh's stopping it was the mesh's doing, which goe | A unit found running and enabled, declared stopped and disabled, is started and enabled again when undeclared, and the report says `restored` | mesh-host `internal/apply/giveback_test.go`, `TestAUnitTheMeshStoppedAndDisabledIsGivenBackRunningAndEnabled` | | A unit found stopped stays stopped; one never stopped by the mesh is left as it is | `TestOnlyWhatTheMeshsDeclarationStoppedIsStartedAgain` | | A record without what was declared starts nothing | `TestARecordWithoutWhatWasDeclaredStartsNothing` | +| A masked unit, or one started or enabled since, is left as it is, and the report says why | `TestAMaskedOrChangedUnitIsLeftAloneOnRemoval` | +| A failed restore waits and is never fatal, and the record stays for the next apply | `TestAFailedRestoreWaitsAndIsNeverFatal` | +| Every systemctl call that names a unit passes `--` before it | mesh-host `internal/system/dashdash_test.go` | | The plan says it before the apply does | `TestThePlanSaysAStoppedUnitIsStartedAgain` | | 0118's existing rules | mesh-host `internal/apply/found_test.go`, unchanged and passing | From 61ab74c70eca4ddc3ad107eae6dd217d075c990e Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 9 Oct 2026 01:54:51 +0200 Subject: [PATCH 3/5] ADR 0269: the catch-up happens when the timer's last run is more than a week back --- ...-own-declaration-stopped-is-started-again-when-it-goes.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/02-DECISIONS/0269-what-the-meshs-own-declaration-stopped-is-started-again-when-it-goes.md b/02-DECISIONS/0269-what-the-meshs-own-declaration-stopped-is-started-again-when-it-goes.md index 4101019d..9de5b485 100644 --- a/02-DECISIONS/0269-what-the-meshs-own-declaration-stopped-is-started-again-when-it-goes.md +++ b/02-DECISIONS/0269-what-the-meshs-own-declaration-stopped-is-started-again-when-it-goes.md @@ -82,8 +82,9 @@ before the mesh came, and the mesh's stopping it was the mesh's doing, which goe - **A restored timer catches up at once.** A timer with `Persistent=true` that was stopped past its scheduled time fires as soon as it is started again. On the home server the distribution's weekly scrub timer is such a timer, and a scrub of its pool took about 46 hours (2026-10-05 00:08 to 2026-10-06 - 22:06). Unassigning the `zfs` module therefore starts a scrub of about 46 hours, within the hour of - the timer's random delay. Unassign it when that load is acceptable. + 22:06). Unassigning the `zfs` module after the timer's last run is more than a week back therefore + starts a scrub of about 46 hours, within the hour of the timer's random delay. Unassign it when that + load is acceptable. - **Harder:** a unit the mesh stopped and that should *stay* stopped after the module goes (one that was found running by accident) is now started again. Saying so needs a field on the resource, which no module needs yet. Until one does, the plan's `restore … started … again` step is where it is seen. From 1a813ba31341fc5d76122537d14571d2aee0533d Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 9 Oct 2026 02:03:32 +0200 Subject: [PATCH 4/5] ADR 0269: the -- check covers the calls in internal/system that name a service resource's unit --- ...shs-own-declaration-stopped-is-started-again-when-it-goes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/02-DECISIONS/0269-what-the-meshs-own-declaration-stopped-is-started-again-when-it-goes.md b/02-DECISIONS/0269-what-the-meshs-own-declaration-stopped-is-started-again-when-it-goes.md index 9de5b485..577dbfe6 100644 --- a/02-DECISIONS/0269-what-the-meshs-own-declaration-stopped-is-started-again-when-it-goes.md +++ b/02-DECISIONS/0269-what-the-meshs-own-declaration-stopped-is-started-again-when-it-goes.md @@ -98,7 +98,7 @@ before the mesh came, and the mesh's stopping it was the mesh's doing, which goe | A record without what was declared starts nothing | `TestARecordWithoutWhatWasDeclaredStartsNothing` | | A masked unit, or one started or enabled since, is left as it is, and the report says why | `TestAMaskedOrChangedUnitIsLeftAloneOnRemoval` | | A failed restore waits and is never fatal, and the record stays for the next apply | `TestAFailedRestoreWaitsAndIsNeverFatal` | -| Every systemctl call that names a unit passes `--` before it | mesh-host `internal/system/dashdash_test.go` | +| Every call in `internal/system` that names a service resource's unit passes `--` before it | mesh-host `internal/system/dashdash_test.go` | | The plan says it before the apply does | `TestThePlanSaysAStoppedUnitIsStartedAgain` | | 0118's existing rules | mesh-host `internal/apply/found_test.go`, unchanged and passing | From a11b14f7ae78f9f5dcfcb0f86a7a2b0669d8a7ab Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 9 Oct 2026 02:17:32 +0200 Subject: [PATCH 5/5] ADR 0269 accepted: note the narrowing on ADR 0118, and give it a home in the node-engine's design The operator accepted ADR 0269 on 2026-10-09. ADR 0118 gets the dated note the README asks for where a later record narrows its mechanism. The node-engine's to-be design names the decision and says what it changes, which the cycle check requires of an accepted record. --- ...ng-gives-a-unit-back-the-state-it-was-found-in.md | 6 ++++++ ...laration-stopped-is-started-again-when-it-goes.md | 2 +- 03-DESIGN/01-to-be/05-the-node-host.md | 12 +++++++++++- 3 files changed, 18 insertions(+), 2 deletions(-) diff --git a/02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md b/02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md index c5958f13..7e11f52b 100644 --- a/02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md +++ b/02-DECISIONS/0118-undeclaring-gives-a-unit-back-the-state-it-was-found-in.md @@ -80,6 +80,12 @@ records the state it first found the unit in, and undeclaring returns the unit t manager: running before the mesh arrived, running after it leaves. - **A unit the mesh started is stopped again**, and one it enabled is disabled again — the packet filter a converge loaded, which returning to adopted unloads. +> **Narrowed, not replaced — 2026-10-09, by [ADR 0269](0269-what-the-meshs-own-declaration-stopped-is-started-again-when-it-goes.md).** +> "Never started on the way out" has one exception. The host now records what the mesh declared beside +> what it found. A unit found running or enabled that the mesh's own declaration held stopped or disabled +> is started or enabled again when that declaration goes, but only while it is still exactly as the mesh +> held it: a masked unit, or one changed since, is left as it is. The rest of this record stands. + - **Never started on the way out.** A unit the mesh stopped is not started again when its declaration goes; starting something is a decision, and the operator makes it. - **Unknown is left alone.** A record written before the host kept what it found says nothing diff --git a/02-DECISIONS/0269-what-the-meshs-own-declaration-stopped-is-started-again-when-it-goes.md b/02-DECISIONS/0269-what-the-meshs-own-declaration-stopped-is-started-again-when-it-goes.md index 577dbfe6..a2b4ae7a 100644 --- a/02-DECISIONS/0269-what-the-meshs-own-declaration-stopped-is-started-again-when-it-goes.md +++ b/02-DECISIONS/0269-what-the-meshs-own-declaration-stopped-is-started-again-when-it-goes.md @@ -1,6 +1,6 @@ --- topic: what runs on it -status: proposed +status: accepted date: 2026-10-09 deciders: jochen reconstructed: false diff --git a/03-DESIGN/01-to-be/05-the-node-host.md b/03-DESIGN/01-to-be/05-the-node-host.md index fc95481e..567ccd96 100644 --- a/03-DESIGN/01-to-be/05-the-node-host.md +++ b/03-DESIGN/01-to-be/05-the-node-host.md @@ -2,8 +2,9 @@ layer: to-be status: in-progress code: [mesh-host] -updated: 2026-10-02 +updated: 2026-10-09 decisions: + - 02-DECISIONS/0269-what-the-meshs-own-declaration-stopped-is-started-again-when-it-goes.md - 02-DECISIONS/0184-a-service-the-mesh-asked-to-run-is-still-running-a-moment-later.md - 02-DECISIONS/0168-a-converged-machine-is-filtered-by-the-mesh-alone.md - 02-DECISIONS/0163-taking-a-module-over-is-a-comparison.md @@ -186,6 +187,15 @@ the firewall it was found with is in force and who retired it. It retires that f converged apply, not once, records *found inactive* apart from *disabled by the mesh*, and says when the step was skipped. *How it is checked:* ADR 0168's table. +**What the mesh's own declaration stopped goes back with it** — revision, 2026-10-09 +([ADR 0269](../../02-DECISIONS/0269-what-the-meshs-own-declaration-stopped-is-started-again-when-it-goes.md)). The +node-engine records, for a service with a state, what the mesh declared beside what it found. When the +declaration goes, a unit found running or enabled that the declaration held stopped or disabled is +started or enabled again, but only while it is still exactly as the mesh held it. A masked unit, or one +changed since, is left as it is, and the report says why. A restore that fails waits for the next apply +and is never fatal. Every call that names a service's unit passes `--` before it. *How it is checked:* +ADR 0269's table. + **Found reaches every kind that can touch what the machine has** ([ADR 0103](../../02-DECISIONS/0103-what-an-adopted-node-holds-and-what-its-guard-refuses.md)). For a module not yet taken, a directory present with no record keeps its mode and owner, a unit present with no record keeps its state and boot setting, a