novox/mesh-host
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

A failed action stops what follows; nothing else does

Browse Source 
The previous commit continued past every failure, and the lab found the
cost immediately: the bootstrap's store-readiness gate failed, the apply
carried on and started the broker and control plane against a machine
that was not ready, and the database still initialising was shut down.

An action is the only shape whose purpose is to make something true
before the next thing needs it — which is why it is the only one with a
verify. The bootstrap is a row of them. Everything else is independent
state, and stopping there is what made one broken module hold a whole
machine hostage.

The report says which happened: "these things failed" and "these things
failed and the rest was never tried" are different machines.
This commit is contained in:
jschoubben
2026-08-30 20:07:42 +02:00
parent aec37bf89e
commit 08e91065b4
2 changed files with 71 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/apply/apply.go
+36 -6
View File
@@ -80,6 +80,11 @@ type Error struct {
// because "one thing failed" and "eleven things failed" are different machines and the first
// line is what somebody reads.
Others int
// Gated is true when what failed was an action, so nothing after it was attempted. A person
// reading a report needs to know the difference between "these things failed" and "these
// things failed and the rest was never tried".
Gated bool
}
func (e *Error) Error() string {
@@ -89,9 +94,14 @@ func (e *Error) Error() string {
} else if e.Others > 1 {
also = fmt.Sprintf(", and %d other resources also failed", e.Others)
}
return fmt.Sprintf("applying %q: %v%s\n\n%d resource(s) were applied and remain; "+
"everything was attempted, so what is not listed as failed was done.",
e.Resource, e.Err, also, len(e.Done.Outcomes))
rest := "everything was attempted, so what is not listed as failed was done."
if e.Gated {
// An action is a gate: it exists to make something true before the next thing needs it.
rest = "this is an action, so nothing after it was attempted — the machine is in " +
"whatever state that left it."
}
return fmt.Sprintf("applying %q: %v%s\n\n%d resource(s) were applied and remain; %s",
e.Resource, e.Err, also, len(e.Done.Outcomes), rest)
}
func (e *Error) Unwrap() error { return e.Err }
@@ -161,10 +171,30 @@ func Apply(
was, _ := known.Find(resource.Identity())
outcome, err := applyOne(ctx, sys, resource, run, changed, was, unseal)
if err != nil {
failures = append(failures, &Error{
Resource: resource.Identity(), Err: err, Done: report,
})
failed := &Error{Resource: resource.Identity(), Err: err, Done: report}
failures = append(failures, failed)
log(fmt.Sprintf(" failed %s (%s): %v", resource.Identity(), outcome.Target, err))
// **A failed action stops what follows. Nothing else does.**
//
// An action is the only shape whose purpose is to make something true *before* the
// next thing needs it — which is why it is the only one with a `verify`. The
// bootstrap is a row of them: the store answers, then its databases exist, then their
// schemas, then the broker. Carrying on past one that did not happen means starting
// things against a machine that is not ready, and on a small machine that is how a
// database still initialising gets its memory taken away and shuts down. Observed,
// in the lab, caused by an earlier version of this loop.
//
// Everything else is independent state. A package that will not install has nothing
// to do with a file on the other side of the declaration, and stopping there is what
// made one broken module hold a whole machine hostage
// (novox/hq 04-ISSUES/011).
if resource.Kind() == declaration.TypeAction {
failed.Done = report
failed.Others = len(failures) - 1
failed.Gated = true
return report, known, failed
}
continue
}