Issue 348: a resolver that stopped answering ran on unnoticed, and builds sent after it were blamed #249

Merged
mesh-admin merged 4 commits from issues/348-the-resolver-stopped-answering-and-nothing-restarted-it into main 2026-10-09 13:25:22 +00:00
Contributor

The 2026-10-09 outage on the control node (10:57:57-11:18:12 UTC): symptom, timeline, what is measured and what is guessed, the reproduction of dnsmasq's bind-dynamic closing its listeners on a bridge teardown, and the fixes in review (mesh-controller #179, mesh-catalog #161).

Open for the operator: the dnsmasq watch restarts on health, which departs from ADR 0240 rule 6; why the second resolver did not serve the route proxy is still a guess (its container's resolver file, readable once docker_resolv_conf is delivered); a mesh-lab replay is owed before the issue resolves.

records, index, words and cycle pass locally. Not merged.

The 2026-10-09 outage on the control node (10:57:57-11:18:12 UTC): symptom, timeline, what is measured and what is guessed, the reproduction of dnsmasq's bind-dynamic closing its listeners on a bridge teardown, and the fixes in review (mesh-controller #179, mesh-catalog #161). Open for the operator: the dnsmasq watch restarts on health, which departs from ADR 0240 rule 6; why the second resolver did not serve the route proxy is still a guess (its container's resolver file, readable once docker_resolv_conf is delivered); a mesh-lab replay is owed before the issue resolves. records, index, words and cycle pass locally. Not merged.
jschoubben force-pushed issues/348-the-resolver-stopped-answering-and-nothing-restarted-it from 8d75a37de5 to d6f889d054 2026-10-09 11:51:47 +00:00 Compare
Author
Contributor

Delivery novox/hq@fbd00a16ca40 — delivered since 2026-10-09T13:25:45Z

Delivery plan — builds nothing: the change touches no module of the mesh's graph

Transitions

  • 2026-10-09 13:06 (new) → proposed (announced): novox/hq#249's head announced
  • 2026-10-09 13:06 proposed → checked (checked): the verdict names this commit
  • 2026-10-09 13:06 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail
  • 2026-10-09 13:25 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move
  • 2026-10-09 13:25 published → delivered (done): nothing for a walk to move

The commit's note under refs/notes/mesh-plan keeps every transition: git log --notes=mesh-plan.

<!-- mesh-delivery:view --> **Delivery** `novox/hq@fbd00a16ca40` — **delivered** since 2026-10-09T13:25:45Z **Delivery plan** — builds nothing: the change touches no module of the mesh's graph **Transitions** - 2026-10-09 13:06 (new) → proposed (announced): novox/hq#249's head announced - 2026-10-09 13:06 proposed → checked (checked): the verdict names this commit - 2026-10-09 13:06 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail - 2026-10-09 13:25 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move - 2026-10-09 13:25 published → delivered (done): nothing for a walk to move The commit's note under `refs/notes/mesh-plan` keeps every transition: `git log --notes=mesh-plan`.
jschoubben added 2 commits 2026-10-09 12:12:44 +00:00
Records the 2026-10-09 outage on the control node: what was measured
(the unit ran, the private address refused; a restarted node-engine's
undecided statement cleared the network condition and its reopening
dated the fault after the send), what the reproduction showed of
bind-dynamic, what is still guessed, and the fixes in review.
Issue 349: a merge acted on late superseded the merge after it, and built the older commit
mesh/merge-gate pass: the change touches no module of the mesh's graph
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
3acc1ec219
Plans of a branch superseded each other by when they were made; the
catch-up's late plan of an older merge took over a newer one, once a
security fix to the forge's module. Recorded with the fix in review.
jschoubben force-pushed issues/348-the-resolver-stopped-answering-and-nothing-restarted-it from 381d3fe477 to 3acc1ec219 2026-10-09 12:12:44 +00:00 Compare
jschoubben added 1 commit 2026-10-09 12:22:20 +00:00
Issue 348: the manifest test takes either binding until the catalogue merges
mesh/merge-gate pass: the change touches no module of the mesh's graph
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery superseded: a newer head of the same pull request
17b64270b0
jschoubben added 1 commit 2026-10-09 13:05:44 +00:00
Issues 348 and 349: the review's answers — gaps, parts, D2 unchanged, the newest merge in any state
mesh/merge-gate pass: the change touches no module of the mesh's graph
mesh/repo-check pass: its merge-check.sh passed
mesh/delivery delivered
fbd00a16ca
mesh-admin merged commit 80b02dc9a0 into main 2026-10-09 13:25:22 +00:00
mesh-admin deleted branch issues/348-the-resolver-stopped-answering-and-nothing-restarted-it 2026-10-09 13:25:23 +00:00
Sign in to join this conversation.
No Reviewers
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: novox/hq#249