novox/mesh-host
18
0
Fork 0
Code Issues Pull Requests 1 Releases Wiki Activity

Act at the engine's start on a backup answer the want already carries, so an answer written before a restart is never swallowed (issue 1585)

Browse Source 
nats on novox, 2026-11-04: the operator answered "Back up again", the
engine restarted, and baseline() took the hold with AnsweredRev = the
want's current revision, so the answer in that very revision was never
acted on and compose refused "held for a failed backup".

Quick fix: answer once more (a new want revision). Thorough fix, chosen:
at baseline, a held module whose want carries again or skip is applied
(again backs it up first, skip goes on without one); a hold with no
answer stays held. The cost said in the code: an answer already acted on
before a stop, whose backup failed again, is tried once more per start.

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
This commit is contained in:
jschoubben
2026-11-04 10:36:48 +01:00
parent ee0f46b78f
commit 768f637bd0
2 changed files with 20 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/agent/agent.go
+13
View File
@@ -1375,6 +1375,19 @@ func (a *Agent) baseline(ctx context.Context, module string, w wantNow, comp com
a.mu.Unlock()
return false
}
if h.Phase == string(delivery.PhaseHeld) && (w.Want.Backup == delivery.BackupAgain || w.Want.Backup == delivery.BackupSkip) {
// Held for a failed backup, and the want already carries the operator's answer, written before this engine
// started (nats on novox, 2026-11-04: "Back up again" answered, then the engine restarted and took the
// answer for the hold's own revision, so it was never acted on; issue 1585). The answer is acted on now:
// again backs it up and applies (a.prior makes the apply back it up first), skip applies without one.
// An answer the engine already acted on before its stop, and whose backup failed once more, is so acted
// on once more at this start: one backup more, never one skipped.
a.mu.Lock()
a.sayOnce("answered:"+module, fmt.Sprintf("%s is held for a failed backup, and its want carries the operator's "+
"answer %q: acted on at this start", module, w.Want.Backup))
a.mu.Unlock()
return false
}
if h.Phase == string(delivery.PhaseHeld) {
// Held for a failed backup, and the operator's answer is still to come: the hold is kept, nothing is
// backed up again or applied, and the next revision of the want is the answer (ADR 0568 §4).
internal/agent/agent_test.go
+7 -2
View File
@@ -595,7 +595,8 @@ func TestADataModuleIsBackedUpFirstAndHeldWhenItFails(t *testing.T) {
}
// At a start: a backup the node agent's own stop interrupted (postgres on novox, 2026-10-31) is no failure, so it is
// backed up again and applied while the want asks a backup; a backup that failed stays held for the operator.
// backed up again and applied while the want asks a backup; a backup that failed stays held for the operator's
// answer, and an answer the want already carries is acted on.
for _, row := range []struct {
detail, backup string
backups, applied int
@@ -605,7 +606,11 @@ func TestADataModuleIsBackedUpFirstAndHeldWhenItFails(t *testing.T) {
{BackupInterrupted, delivery.BackupAgain, 1, 1, delivery.PhaseRunning, false},
// novox's postgres: its held have, written before mesh-host #189, names nothing it runs.
{BackupInterrupted, "", 1, 1, delivery.PhaseRunning, true},
{"restic: the repository is locked", delivery.BackupAgain, 0, 0, delivery.PhaseHeld, false},
// A backup that failed, and the want carries the operator's answer, written before this start (nats on novox,
// 2026-11-04, issue 1585): again backs it up and applies, skip applies without one; no answer keeps the hold.
{"restic: the repository is locked", delivery.BackupAgain, 1, 1, delivery.PhaseRunning, false},
{"restic: the repository is locked", delivery.BackupSkip, 0, 1, delivery.PhaseRunning, false},
{"restic: the repository is locked", "", 0, 0, delivery.PhaseHeld, false},
} {
r := newBed(t)
r.comps.before(ck1, craw1)