When the store does not come up, say what it said
The bootstrap's readiness wait failed on a loaded machine and reported `docker exited 1:` with nothing after the colon. The file's own comment already records this failing three times before and being fixed by running it again — "the worst kind, because it teaches people to run things twice". Raising the timeout a second time would treat the symptom. What makes a retry the only available response is a timeout that reports nothing, so the wait now prints what pg_isready says and the store's own last lines before giving up.
This commit is contained in:
@@ -21,6 +21,11 @@
|
|||||||
// failed that way three times in the lab -- a flaky bootstrap that a second run always fixed,
|
// failed that way three times in the lab -- a flaky bootstrap that a second run always fixed,
|
||||||
// which is the worst kind because it teaches people to run things twice.
|
// which is the worst kind because it teaches people to run things twice.
|
||||||
//
|
//
|
||||||
|
// It failed a fourth time on 2026-08-30, on a loaded machine, and the report was `docker exited
|
||||||
|
// 1:` with nothing after the colon. Raising the timeout again would treat the symptom; what makes
|
||||||
|
// a retry the only available response is a timeout that reports nothing. So the wait now says
|
||||||
|
// what it saw before giving up.
|
||||||
|
//
|
||||||
// The store's data is a NAMED VOLUME, not a directory on the machine. A directory the host
|
// The store's data is a NAMED VOLUME, not a directory on the machine. A directory the host
|
||||||
// creates is owned by root, and the database runs as somebody else inside the container — so it
|
// creates is owned by root, and the database runs as somebody else inside the container — so it
|
||||||
// could not write, and the container crash-looped. A named volume lets the image set up its own
|
// could not write, and the container crash-looped. A named volume lets the image set up its own
|
||||||
@@ -57,7 +62,7 @@
|
|||||||
"id": "store-ready",
|
"id": "store-ready",
|
||||||
"type": "action",
|
"type": "action",
|
||||||
"in": "mesh-store",
|
"in": "mesh-store",
|
||||||
"command": ["sh", "-c", "for i in $(seq 1 180); do pg_isready -U postgres >/dev/null 2>&1 && exit 0; sleep 1; done; exit 1"],
|
"command": ["sh", "-c", "for i in $(seq 1 180); do pg_isready -U postgres >/dev/null 2>&1 && exit 0; sleep 1; done; echo 'the store did not answer within 180s; its own last words follow'; pg_isready -U postgres; tail -n 20 /var/lib/postgresql/data/log/*.log 2>/dev/null; exit 1"],
|
||||||
"verify": ["pg_isready", "-U", "postgres"]
|
"verify": ["pg_isready", "-U", "postgres"]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
|
|||||||
Reference in New Issue
Block a user