Judge a send by when a fault began, not when it was last raised (hq issue 348)

On 2026-10-09 the control node's resolver refused from 10:57:57 UTC. A
node-engine restarted by the 10:59:34 send said its names undecided, that
statement cleared the network condition, the next look raised it again
after the send, and the gate put back two builds for a fault older than
them.

- A condition keeps First across a reopening; the gate reads Began.
- An undecided network statement (unknown, starting) clears nothing.
- D10 counts what a release's tier names as rolling, so a walked
  node-engine is not core-behind on its own first machine.
- D2 raises a resolver that refuses every try at once: a refusal is an
  answer, not a loaded resolver (issue 277).
This commit is contained in:
2026-10-09 15:00:56 +02:00
committed by jochen
parent c58516f819
commit 077ddf0eb8
7 changed files with 370 additions and 17 deletions
+32 -1
View File
@@ -98,8 +98,9 @@ func judgeNetworks(ctx context.Context, inv *inventory.Inventory, k *conditions.
problems = append(problems, err.Error())
}
}
undecided := undecidedMachines(f)
for _, c := range open {
if !slices.Contains(networkKinds, c.Kind) || said[c.Key] {
if !slices.Contains(networkKinds, c.Kind) || said[c.Key] || undecided[c.Subject.Machine] {
continue
}
why := "no machine says it any more"
@@ -116,6 +117,36 @@ func judgeNetworks(ctx context.Context, inv *inventory.Inventory, k *conditions.
return nil
}
// undecidedMachines is every machine whose newest statement has a part not yet judged: starting, or
// unknown — one look failed and a second decides (novox/hq issue 348). Such a statement does not say a
// fault is gone, so an open network condition about that machine is kept until a statement decides.
//
// On 2026-10-09 the control node's resolver refused every question from 10:58 to 11:18 UTC. A build of
// the node-engine sent at 10:59:34 restarted it; its first statement after the restart judged the names
// once (unknown, "one look failed; a second decides"), and that statement cleared the control node's
// network condition while its last evidence still said "connection refused". The second look raised it
// again forty seconds later — after the send — and the gate failed the build for a fault from before it.
// Pure.
func undecidedMachines(f networkFacts) map[string]bool {
out := map[string]bool{}
for m, h := range f.healths {
if h.Network == nil {
continue
}
if h.Network.State == link.StateUnknown || h.Network.State == link.StateStarting {
out[m] = true
continue
}
for _, p := range h.Network.Parts {
if p.State == link.StateUnknown || p.State == link.StateStarting {
out[m] = true
break
}
}
}
return out
}
// pointed is one machine's failing part that points at another machine.
type pointed struct {
from string