novox/mesh-catalog
9
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

restic: the backup retention is a preference, today's 14/8/6 its default (hq issue 416, first half) #200

Merged
mesh-admin merged 2 commits from fix/416-retention-and-log-rotation-are-preferences into main 2026-10-11 02:24:36 +00:00
Conversation 3 Commits 2 Files Changed 4
+251 -6
mesh-admin commented 2026-10-11 01:05:33 +00:00
Contributor
Copy Link
Copy Source

Fixes the backup half of novox/hq issue 416 (backup retention and the container runtime's log rotation cannot become preferences yet). The log-rotation half needs a decision and is not in this pull request; see the note on 416.

Cause

The retention was a Go literal in modules/restic/cmd/restic-backups/backups.go (Keep = {14, 8, 6}), so no setting could reach it. The controller's existing mechanism (ADR 0262 preferences filling ${setting:} into a file, as zfs's scrub schedule does) supports it; no controller change is needed.

Change

  • module.json: three preferences, keep-daily 14, keep-weekly 8, keep-monthly 6 (today's values), and a new file ${dir:state}/retention.conf (daily=, weekly=, monthly=) that asks for them. The file is trusted: true: lowering a count deletes restore points, so the value is set at the terminal alone (issue 339).
  • retention.go: the night reads that file. Each value must be a count from 1 to 9999 with no leading zero; an unknown, repeated or missing key, or a line that is not key=value, is refused. A refused file means nothing is thinned, every restore point is kept, and the module says which setting is wrong in its journal. No raw value ever reaches restic's arguments (issue 346).
  • A missing file (a machine placed by the older definition) keeps today's 14/8/6.

Tests (written first)

retention_test.go: against main with stubs, three of the four fail (TestANightThinsByTheRetentionTheMeshWrote, TestARetentionThatIsNotACountThinsNothingAndSaysWhich, TestTheManifestsDefaultsAreTodaysRetention); TestWithoutTheFileANightKeepsTodaysRetention passes on main by design. The manifest test fills the file with the declared defaults and checks that it parses to exactly today's retention.

Run here: go vet and go test -race of modules/restic; merge-check.sh with MESH_GATE set to a controller built from mesh-controller main 11ffab8 and MESH_CHECK_CHANGED set to this diff (exit 0, restic: ok, no untrusted-setting warning); the words check (cd checks/words && go run . ../.., clean).

What changes on a machine with the defaults

mesh-controller.plan cannot show an unmerged branch, so this is stated, not measured. restic runs on ace, g14, novox and shanks. On each:

  • a new file, <restic state>/retention.conf, holding daily=14, weekly=8, monthly=6;
  • the restic tools bundle is rebuilt and restarts with the new binary.

The nightly restic forget line is byte for byte the same (14/8/6), so no restore point is kept or dropped differently. No other file, unit or module changes.

Not touched: modules/mesh-delivery, modules/mesh-cli (open #198, #199).

Fixes the backup half of novox/hq issue 416 (backup retention and the container runtime's log rotation cannot become preferences yet). The log-rotation half needs a decision and is not in this pull request; see the note on 416. ## Cause The retention was a Go literal in `modules/restic/cmd/restic-backups/backups.go` (`Keep = {14, 8, 6}`), so no setting could reach it. The controller's existing mechanism (ADR 0262 preferences filling `${setting:}` into a file, as zfs's scrub schedule does) supports it; no controller change is needed. ## Change - `module.json`: three preferences, `keep-daily` 14, `keep-weekly` 8, `keep-monthly` 6 (today's values), and a new file `${dir:state}/retention.conf` (`daily=`, `weekly=`, `monthly=`) that asks for them. The file is `trusted: true`: lowering a count deletes restore points, so the value is set at the terminal alone (issue 339). - `retention.go`: the night reads that file. Each value must be a count from 1 to 9999 with no leading zero; an unknown, repeated or missing key, or a line that is not key=value, is refused. A refused file means **nothing is thinned, every restore point is kept**, and the module says which setting is wrong in its journal. No raw value ever reaches restic's arguments (issue 346). - A missing file (a machine placed by the older definition) keeps today's 14/8/6. ## Tests (written first) `retention_test.go`: against main with stubs, three of the four fail (`TestANightThinsByTheRetentionTheMeshWrote`, `TestARetentionThatIsNotACountThinsNothingAndSaysWhich`, `TestTheManifestsDefaultsAreTodaysRetention`); `TestWithoutTheFileANightKeepsTodaysRetention` passes on main by design. The manifest test fills the file with the declared defaults and checks that it parses to exactly today's retention. Run here: `go vet` and `go test -race` of modules/restic; `merge-check.sh` with `MESH_GATE` set to a controller built from mesh-controller main 11ffab8 and `MESH_CHECK_CHANGED` set to this diff (exit 0, `restic: ok`, no untrusted-setting warning); the words check (`cd checks/words && go run . ../..`, clean). ## What changes on a machine with the defaults `mesh-controller.plan` cannot show an unmerged branch, so this is stated, not measured. restic runs on ace, g14, novox and shanks. On each: - a new file, `<restic state>/retention.conf`, holding `daily=14`, `weekly=8`, `monthly=6`; - the restic tools bundle is rebuilt and restarts with the new binary. The nightly `restic forget` line is byte for byte the same (14/8/6), so no restore point is kept or dropped differently. No other file, unit or module changes. Not touched: modules/mesh-delivery, modules/mesh-cli (open #198, #199).
mesh-admin added 1 commit 2026-10-11 01:05:34 +00:00
restic: take the retention from the machine's settings, so a disk can keep more or fewer restore points without a new build (novox/hq issue 416)
mesh/merge-gate pass: builds restic → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
Details
mesh/repo-check pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-check-typescript.sh (typescript)
Details
mesh/delivery superseded: a newer head of the same pull request
Details
39d5b25f95 
The keep-daily, keep-weekly and keep-monthly preferences default to
today's 14, 8 and 6. The night reads them from a file in its state and
refuses anything but a count from 1 to 9999: it then thins nothing and
says which setting is wrong, never handing a raw value to restic.
mesh-admin commented 2026-10-11 01:05:49 +00:00
Author
Contributor
Copy Link
Copy Source

Delivery novox/mesh-catalog@ecf6d7803f8c — delivered since 2026-10-11T02:30:23Z

Delivery plan — builds restic → ace, g14, novox, shanks; no bus step

  • tier 0: restic
  • ace: receives restic
  • g14: receives restic
  • novox: receives restic
  • shanks: receives restic
  • restic keeps data (ADR 0233): what it holds is backed up before it moves

Machines

  • restic on ace: passed — healthy 3 times over 2m30s
  • restic on the rest running it: sent

Transitions

  • 2026-10-11 01:10 (new) → proposed (announced): novox/mesh-catalog#200's head announced
  • 2026-10-11 01:12 proposed → checked (checked): the verdict names this commit
  • 2026-10-11 01:12 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail
  • 2026-10-11 02:26 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move
  • 2026-10-11 02:26 published → delivering (go): its walk started: let go by this owner in its group's order, by a person, or on the controller's own path — not a failed walk that carried a later commit for it, after which it waits for a walk of its own commit
  • 2026-10-11 02:30 delivering → delivered (done): its walk is done, carrying its own commit: every machine of its deploy plan passed, or was left as its policy says

The commit's note under refs/notes/mesh-plan keeps every transition: git log --notes=mesh-plan.

<!-- mesh-delivery:view --> **Delivery** `novox/mesh-catalog@ecf6d7803f8c` — **delivered** since 2026-10-11T02:30:23Z **Delivery plan** — builds restic → ace, g14, novox, shanks; no bus step - tier 0: restic - ace: receives restic - g14: receives restic - novox: receives restic - shanks: receives restic - restic keeps data (ADR 0233): what it holds is backed up before it moves **Machines** - restic on ace: passed — healthy 3 times over 2m30s - restic on the rest running it: sent **Transitions** - 2026-10-11 01:10 (new) → proposed (announced): novox/mesh-catalog#200's head announced - 2026-10-11 01:12 proposed → checked (checked): the verdict names this commit - 2026-10-11 01:12 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail - 2026-10-11 02:26 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move - 2026-10-11 02:26 published → delivering (go): its walk started: let go by this owner in its group's order, by a person, or on the controller's own path — not a failed walk that carried a later commit for it, after which it waits for a walk of its own commit - 2026-10-11 02:30 delivering → delivered (done): its walk is done, carrying its own commit: every machine of its deploy plan passed, or was left as its policy says The commit's note under `refs/notes/mesh-plan` keeps every transition: `git log --notes=mesh-plan`.
mesh-admin commented 2026-10-11 01:07:22 +00:00
Author
Contributor
Copy Link
Copy Source

Merge check at 39d5b25f

  • mesh/merge-gate: PASS — every machine composes with the change as it did without (4 of 4 compose) (modules: restic)
  • mesh/repo-check: PASS — its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-check-typescript.sh (typescript)

Delivery plan — builds restic → ace, g14, novox, shanks; no bus step

  • tier 0: restic
  • ace: receives restic
  • g14: receives restic
  • novox: receives restic
  • shanks: receives restic
  • restic keeps data (ADR 0233): what it holds is backed up before it moves

Run by the build seat on ace as build-1791680748004148785 (builds --log build-1791680748004148785).

--- the gate: restic
restic: ok, keeps repository (rebuildable), state (cache)
long-running resources without health: 0
files asking for a setting without saying whether it is trusted: 0
1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here
judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-11T01:00:25Z says they run
composing every machine as the mesh is
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
composing every machine with the change
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose)
judged by controller development build against the facts of 2026-10-11T01:00:25Z

modules the change moves:
  - restic (changed)

machines:
  - a machine (f35): composes; the change adds 1 resource(s) and removes 0 (+file:restic.retention)
  - holds mesh-dns-resolver (idt): composes; the change adds 1 resource(s) and removes 0 (+file:restic.retention)
  - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes; the change adds 1 resource(s) and removes 0 (+file:restic.retention)
  - a machine (yvbpsm): composes; the change adds 1 resource(s) and removes 0 (+file:restic.retention)

a merge would rebuild 1 module(s) in 1 tier(s): restic
--- its merge-check.sh (go toolchain)
ok  	git.novox.be/novox/mesh-catalog/checks/words	0.002s
words: 12 retired words and 70 daemons' lines, none in the 39460 strings of 655 files
ok  	git.novox.be/novox/mesh-catalog/checks/agenthome	0.011s
agent-home: no archive and no file written into under the agent's home
ok  	git.novox.be/novox/mesh-catalog/checks/paths	0.005s
paths: no module names another's state on its machine (3 known, listed to go)
long-running resources without health: 69, as kept
testing modules/restic
ok  	restic/cmd/restic-backups	1.044s
--- its merge-check-typescript.sh (typescript toolchain)
**Merge check** at `39d5b25f` - `mesh/merge-gate`: **PASS** — every machine composes with the change as it did without (4 of 4 compose) (modules: restic) - `mesh/repo-check`: **PASS** — its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-check-typescript.sh (typescript) **Delivery plan** — builds restic → ace, g14, novox, shanks; no bus step - tier 0: restic - ace: receives restic - g14: receives restic - novox: receives restic - shanks: receives restic - restic keeps data (ADR 0233): what it holds is backed up before it moves Run by the build seat on ace as `build-1791680748004148785` (`builds --log build-1791680748004148785`). ``` --- the gate: restic restic: ok, keeps repository (rebuildable), state (cache) long-running resources without health: 0 files asking for a setting without saying whether it is trusted: 0 1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-11T01:00:25Z says they run composing every machine as the mesh is the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued composing every machine with the change the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose) judged by controller development build against the facts of 2026-10-11T01:00:25Z modules the change moves: - restic (changed) machines: - a machine (f35): composes; the change adds 1 resource(s) and removes 0 (+file:restic.retention) - holds mesh-dns-resolver (idt): composes; the change adds 1 resource(s) and removes 0 (+file:restic.retention) - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes; the change adds 1 resource(s) and removes 0 (+file:restic.retention) - a machine (yvbpsm): composes; the change adds 1 resource(s) and removes 0 (+file:restic.retention) a merge would rebuild 1 module(s) in 1 tier(s): restic --- its merge-check.sh (go toolchain) ok git.novox.be/novox/mesh-catalog/checks/words 0.002s words: 12 retired words and 70 daemons' lines, none in the 39460 strings of 655 files ok git.novox.be/novox/mesh-catalog/checks/agenthome 0.011s agent-home: no archive and no file written into under the agent's home ok git.novox.be/novox/mesh-catalog/checks/paths 0.005s paths: no module names another's state on its machine (3 known, listed to go) long-running resources without health: 69, as kept testing modules/restic ok restic/cmd/restic-backups 1.044s --- its merge-check-typescript.sh (typescript toolchain) ```
jschoubben added 1 commit 2026-10-11 01:09:47 +00:00
restic: say in each retention setting that lowering it deletes restore points for good, so the operator sees the cost before setting it (novox/hq issue 416)
mesh/merge-gate pass: builds restic → ace, g14, novox, shanks; no bus step; every machine composes with the change as it did without (4 of 4 compose)
Details
mesh/repo-check pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-check-typescript.sh (typescript)
Details
mesh/delivery delivered
Details
ecf6d7803f
mesh-admin commented 2026-10-11 01:12:01 +00:00
Author
Contributor
Copy Link
Copy Source

Merge check at ecf6d780

  • mesh/merge-gate: PASS — every machine composes with the change as it did without (4 of 4 compose) (modules: restic)
  • mesh/repo-check: PASS — its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-check-typescript.sh (typescript)

Delivery plan — builds restic → ace, g14, novox, shanks; no bus step

  • tier 0: restic
  • ace: receives restic
  • g14: receives restic
  • novox: receives restic
  • shanks: receives restic
  • restic keeps data (ADR 0233): what it holds is backed up before it moves

Run by the build seat on shanks as build-1791681002021917746 (builds --log build-1791681002021917746).

--- the gate: restic
restic: ok, keeps repository (rebuildable), state (cache)
long-running resources without health: 0
files asking for a setting without saying whether it is trusted: 0
1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here
judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-11T01:09:19Z says they run
composing every machine as the mesh is
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
composing every machine with the change
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose)
judged by controller development build against the facts of 2026-10-11T01:09:19Z

modules the change moves:
  - restic (changed)

machines:
  - a machine (f35): composes; the change adds 1 resource(s) and removes 0 (+file:restic.retention)
  - holds mesh-dns-resolver (idt): composes; the change adds 1 resource(s) and removes 0 (+file:restic.retention)
  - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes; the change adds 1 resource(s) and removes 0 (+file:restic.retention)
  - a machine (yvbpsm): composes; the change adds 1 resource(s) and removes 0 (+file:restic.retention)

a merge would rebuild 1 module(s) in 1 tier(s): restic
--- its merge-check.sh (go toolchain)
ok  	git.novox.be/novox/mesh-catalog/checks/words	0.003s
words: 12 retired words and 70 daemons' lines, none in the 39460 strings of 655 files
ok  	git.novox.be/novox/mesh-catalog/checks/agenthome	0.013s
agent-home: no archive and no file written into under the agent's home
ok  	git.novox.be/novox/mesh-catalog/checks/paths	0.006s
paths: no module names another's state on its machine (3 known, listed to go)
long-running resources without health: 69, as kept
testing modules/restic
ok  	restic/cmd/restic-backups	1.088s
--- its merge-check-typescript.sh (typescript toolchain)
**Merge check** at `ecf6d780` - `mesh/merge-gate`: **PASS** — every machine composes with the change as it did without (4 of 4 compose) (modules: restic) - `mesh/repo-check`: **PASS** — its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-check-typescript.sh (typescript) **Delivery plan** — builds restic → ace, g14, novox, shanks; no bus step - tier 0: restic - ace: receives restic - g14: receives restic - novox: receives restic - shanks: receives restic - restic keeps data (ADR 0233): what it holds is backed up before it moves Run by the build seat on shanks as `build-1791681002021917746` (`builds --log build-1791681002021917746`). ``` --- the gate: restic restic: ok, keeps repository (rebuildable), state (cache) long-running resources without health: 0 files asking for a setting without saying whether it is trusted: 0 1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-11T01:09:19Z says they run composing every machine as the mesh is the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued composing every machine with the change the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose) judged by controller development build against the facts of 2026-10-11T01:09:19Z modules the change moves: - restic (changed) machines: - a machine (f35): composes; the change adds 1 resource(s) and removes 0 (+file:restic.retention) - holds mesh-dns-resolver (idt): composes; the change adds 1 resource(s) and removes 0 (+file:restic.retention) - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes; the change adds 1 resource(s) and removes 0 (+file:restic.retention) - a machine (yvbpsm): composes; the change adds 1 resource(s) and removes 0 (+file:restic.retention) a merge would rebuild 1 module(s) in 1 tier(s): restic --- its merge-check.sh (go toolchain) ok git.novox.be/novox/mesh-catalog/checks/words 0.003s words: 12 retired words and 70 daemons' lines, none in the 39460 strings of 655 files ok git.novox.be/novox/mesh-catalog/checks/agenthome 0.013s agent-home: no archive and no file written into under the agent's home ok git.novox.be/novox/mesh-catalog/checks/paths 0.006s paths: no module names another's state on its machine (3 known, listed to go) long-running resources without health: 69, as kept testing modules/restic ok restic/cmd/restic-backups 1.088s --- its merge-check-typescript.sh (typescript toolchain) ```
mesh-admin merged commit 33ee89794e into main 2026-10-11 02:24:36 +00:00
mesh-admin deleted branch fix/416-retention-and-log-rotation-are-preferences 2026-10-11 02:24:36 +00:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
mesh/ready
Reviewed and ready: mesh-delivery merges at the marked head once its checks pass (hq ADR 0364)
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
agent-ace agent-g14 agent-novox agent-shanks jschoubben mesh-tools miel.truyen nox-agent
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: novox/mesh-catalog#200