jschoubben
275ddd76e8
Once chain_moved records the kind, a plan is a key of the rollouts bucket under its id and the open batch is the same key in state assembling. A save refuses a copy another save moved since (the plan's own Revision, kept in its value, counts saves) and keeps the merges the record holds, so a copy read before a merge moved cannot lay it back. The merges a batch or walk answers are the record's held field, with the columns the table had; a merge no plan answers stays, flagged, in the record that held it last, or in the record kept for those. A merge moved to a plan is written to the receiving record first and then taken from the giving one; one found in both is read from the one it entered last and repaired at the next cut. Plans and merges are read from a view kept by one watch of the bucket. A bus that does not answer, a lease lost and a key moved are now not-now like the store being away, so a merge whose plan could not be written is held and handed over again, never acknowledged. The start moves release_plan and batched_merge into the bucket, one kind, counted against both tables.
65 lines
2.6 KiB
Go
65 lines
2.6 KiB
Go
package inventory
|
|
|
|
import (
|
|
"context"
|
|
"errors"
|
|
"io"
|
|
"net"
|
|
|
|
"github.com/jackc/pgx/v5/pgconn"
|
|
"github.com/nats-io/nats.go"
|
|
"github.com/nats-io/nats.go/jetstream"
|
|
|
|
"github.com/novox/mesh-controller/internal/chain"
|
|
)
|
|
|
|
// Unreachable says whether an error means the store could not be asked right now, rather than
|
|
// that it answered no.
|
|
//
|
|
// The difference decides whether something worth keeping is kept or lost. The store is recreated
|
|
// when the foundation is adopted (ADR 0078), and for those seconds every write fails; a caller
|
|
// that treats that like a refusal throws away what it was writing — a node's report of the apply
|
|
// that caused the restart was lost exactly so, and the mesh never heard from the node again
|
|
// (novox/hq issue 082).
|
|
//
|
|
// Unreachable is the connection failing and the server saying "not now". The connection failing
|
|
// is a network error, a connection that ended mid-conversation (a store killed rather than
|
|
// stopped gives a bare unexpected EOF), a timeout, or anything pgx marks safe to retry. The server
|
|
// saying "not now" is a connection exception (class 08) or it shutting down or starting up (57P01,
|
|
// 57P02, 57P03). Everything else is an answer, and asking again gets the same one: a wrong
|
|
// password, a database that does not exist, a statement cancelled, a constraint. Those are
|
|
// checked first, even inside a failed connection, because a connection that failed on a wrong
|
|
// password would otherwise read as a network fault and be asked again for ever.
|
|
func Unreachable(err error) bool {
|
|
if err == nil {
|
|
return false
|
|
}
|
|
// The chain on the bus (novox/hq ADR 0408): a bus that does not answer, a lease this controller no longer holds, a key
|
|
// somebody else wrote first — each is "not now", as the store being away is.
|
|
if errors.Is(err, chain.ErrNotHeld) || errors.Is(err, chain.ErrMoved) || errors.Is(err, context.DeadlineExceeded) ||
|
|
errors.Is(err, nats.ErrTimeout) || errors.Is(err, nats.ErrNoResponders) || errors.Is(err, nats.ErrConnectionClosed) ||
|
|
errors.Is(err, nats.ErrDisconnected) || errors.Is(err, nats.ErrNoServers) || errors.Is(err, jetstream.ErrNoStreamResponse) {
|
|
return true
|
|
}
|
|
var pg *pgconn.PgError
|
|
if errors.As(err, &pg) {
|
|
switch pg.Code {
|
|
case "57P01", "57P02", "57P03":
|
|
return true
|
|
}
|
|
return len(pg.Code) == 5 && pg.Code[:2] == "08"
|
|
}
|
|
if errors.Is(err, io.ErrUnexpectedEOF) || errors.Is(err, io.EOF) {
|
|
return true
|
|
}
|
|
if pgconn.Timeout(err) || pgconn.SafeToRetry(err) {
|
|
return true
|
|
}
|
|
var network net.Error
|
|
if errors.As(err, &network) {
|
|
return true
|
|
}
|
|
var connect *pgconn.ConnectError
|
|
return errors.As(err, &connect)
|
|
}
|