Files
hq/04-ISSUES/163-a-delivered-host-stood-aside-on-every-push-and-reported-nothing/00-report.md
T
jschoubben a4384f13d3 Issue 163: a delivered host stood aside on every push and reported nothing
Asked whether a newer host was delivered using the link-time stamp, which
every delivered host carries as 'development build' now that the version
comes from where the binary sits. Never matched, so it stood aside on
every push for ever; standing aside cancels the report, so the mesh never
heard from it. Read as healthy throughout.

The three-minute push wait made it invisible: a wait long enough to
absorb a whole apply is long enough to hide that the machine never
answered.
2026-09-30 13:47:28 +02:00

2.6 KiB

status, opened, located-in, fixed-by, amended-design
status opened located-in fixed-by amended-design
resolved 2026-09-30
mesh-host cmd/mesh-host/main.go (the successor check after an apply)
mesh-host PR 58 — the check asks with the running version, read from the binary's path, not the link-time stamp

163 — A delivered host stood aside on every push, and reported nothing

What was observed

2026-09-30, rolling the mesh-built host onto the last two machines.

Every push to a machine running a delivered host produced, in order:

host 093231796eb0 is delivered; standing aside so the launcher runs it
applied 333 resource(s)
applied, and could not tell the mesh: reporting: context canceled
nox-mesh-host-launch: the host exited cleanly; starting it again
nox-mesh-host-launch: running /usr/lib/nox-mesh-host/versions/093231796eb0/nox-mesh-host

— for the version it was already running. It restarted itself on every push, for ever, and the mesh never received a single report from it: node show kept the version from before the crossover, and the operator's push waited its full three minutes for an answer that was never coming.

Read as healthy throughout: unit active, bus link up, "hearing what this node should be".

Why

After an apply the host asks whether a newer host has been delivered than the one running, and the question was asked with the link-time version stamp. Since issue 161 a delivered host's version comes from where it sits and its stamp is development build — so the comparison never matched the newest delivered version, and "a newer host is waiting" was always true.

Standing aside cancels the context the report is published with, so the report was lost on every one of those applies. Two faults from one wrong argument.

The change that moved the version to the path was applied to the report and to the known-good record, and not here. Half a change, and the half left behind was the one that decides whether to exit.

Why the three-minute wait made it invisible

The push's --wait timing out read as slow. It was not slow: the report was never going to arrive. The operator put it exactly: if you don't get a response in five seconds, something is wrong. A wait long enough to absorb a machine's whole apply is a wait long enough to hide that the machine never answered.

How it is checked

A machine running a delivered host is pushed a declaration that delivers nothing new; it applies, reports, and does not stand aside. A machine running a delivered host is pushed a genuinely newer version; it stands aside once, and the next push it does not.