It reported "N machines run an older host than another" by comparing
versions as strings. A host reports its version as a commit, and commits
have no order. On the live mesh it named the three machines running the
NEWER host as the ones behind — ced54d4 sorts above 04a27ca and that is
all it means.
The code even carried a caveat saying versions compare as strings and that
this "is enough for the timestamps and commits this mesh uses". That was
the error, written down and not noticed: enough for timestamps, meaningless
for commits, and the mesh reports commits.
It now reports the split and claims no ordering, which is more useful as
well as more honest — the reader sees who is on which side, and that is
what decides whether a field can be sent. Ordering is left with the host,
which would have to report something ordered for anybody to have it.
This is issue 145 arriving by my own door an hour after I closed it: a
report that confidently says the opposite of the truth is worse than one
that says less.
The front door for "something is wrong" at the level of the mesh's design or governance.
Diagnosis happens here, where the whole mesh is in view; the fix lands in the owning code
repository.
What belongs here
Belongs here
Belongs in the knowledge base
The design permits a failure to be silent
How to fix one occurrence of it
A documented rule is enforced by nothing
A command that works around it
A stated invariant is false in practice
A node-specific quirk
The owner is unknown and finding it needs the whole mesh in view
Symptom → fix, once the answer is known
The knowledge base already holds the operational record and is indexed on symptoms. This
folder is not a second copy of it. An issue here is a question HQ must answer; an entry
there is an incident someone must clear. An issue whose answer is a general lesson belongs in
both.
Structure
NNN-short-name/
00-report.md the symptom as observed, with the evidence; status in frontmatter
01-diagnosis.md the investigation trail, dated, including what was ruled out
Frontmatter, on 00-report.md
---status:open | diagnosing | located | resolved | wontfixopened:YYYY-MM-DDlocated-in:[]# owning repo(s) or module(s), filled by diagnosisfixed-by:# pull request or commit reference, filled at resolutionamended-design:# design doc path, when the root cause was a design gap---
Rules
Anyone may open an issue. No localisation is required to report one.