From fbd00a16ca40d39eed9f80eecd7f52d5597e97f7 Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 9 Oct 2026 15:05:42 +0200 Subject: [PATCH] =?UTF-8?q?Issues=20348=20and=20349:=20the=20review's=20an?= =?UTF-8?q?swers=20=E2=80=94=20gaps,=20parts,=20D2=20unchanged,=20the=20ne?= =?UTF-8?q?west=20merge=20in=20any=20state?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../01-diagnosis.md | 44 +++++++++++-------- .../00-report.md | 21 ++++++--- 2 files changed, 41 insertions(+), 24 deletions(-) diff --git a/04-ISSUES/348-a-resolver-that-stopped-answering-ran-on-unnoticed-and-builds-sent-after-it-were-blamed/01-diagnosis.md b/04-ISSUES/348-a-resolver-that-stopped-answering-ran-on-unnoticed-and-builds-sent-after-it-were-blamed/01-diagnosis.md index 91f3aaa4..8a6fd1c5 100644 --- a/04-ISSUES/348-a-resolver-that-stopped-answering-ran-on-unnoticed-and-builds-sent-after-it-were-blamed/01-diagnosis.md +++ b/04-ISSUES/348-a-resolver-that-stopped-answering-ran-on-unnoticed-and-builds-sent-after-it-were-blamed/01-diagnosis.md @@ -126,50 +126,58 @@ waits for the private address (above). TypeScript tools were never served on any machine). Its new long-running process, the watch, runs as root and every 15 seconds asks the machine's own name — from the roster — on every address the configuration listens on, over UDP and TCP, and checks the answer is that machine's address. After two failing looks in -a row it restarts the unit, at most once every two minutes and three times an hour, and the looks after it +a row it restarts the unit — only one that runs: a unit stopped or disabled on purpose, still starting +(waiting at boot for the private address) or failed is the service manager's or a person's, and is held +and said — without waiting for the start, at most once every two minutes and three times an hour (a +budget it reads back when it restarts itself), and the looks after it say whether the restart helped. Its tool `dnsmasq_health` is the watch's declared check (ADR 0240): it asks again and reads what the watch did, so a restart that helped leaves the module healthy and the operator hears nothing, and one that did not becomes `module.dnsmasq..unhealthy`, saying "restarted at …, which did not help". The unit itself is judged by its unit (`health: unit`). The bundle emits `restarted` -on the bus beside the names it answers. The repair is the module's own, as the identity provider repairs +and, once the looks decide it, `restart-judged` (helped or not) on the bus beside the names it answers, +each remembered as said only once emitted. The repair is the module's own, as the identity provider repairs its administrator (ADR 0224 §5), because a resolver that stops answering can cut its machine off from the bus by name, and then nothing outside the machine reaches it. **This departs from ADR 0240 rule 6** ("nothing restarts on health"), which left a restart on health to a healer's own record under ADR 0231. That record is not written; whether this module-local repair stands, or becomes a row of the healer registry, is the operator's decision. -**mesh-controller — a fault is judged by when it began.** +**mesh-controller — a fault is judged by whether it was there at the send.** -- A condition keeps `first`, when the fault was first raised, across a reopening within `ReopenWithin` - (and across a controller restart, from the history); `Began()` reads it. The gate compares `Began()`, - not the raised time, with the send. -- An undecided network statement — a part `unknown` or `starting`, as a node-engine just started says — - clears no network condition about that machine. The next statement decides. +- A condition reopened within `ReopenWithin` keeps when the fault was first raised (`first`) and its + gaps, each from a clearing to the reopening after it, across a controller restart too (from the + history). The gate asks `OpenAt(send)`: the fault began at or before the send and the send fell in none + of its gaps. So a fault there at the send that flapped after it is not the send's, and a machine that + had recovered before the send and broke after it fails the gate, as it must (review of the controller's + pull request: a silent pass is worse than a false blame). +- An undecided part of a network statement — `unknown` or `starting`, as a node-engine just started says + — holds the open network conditions whose newest evidence names that part, and only those. A statement + undecided as a whole holds every part. The next statement decides. - D10 counts what a release's tier names as rolling out, so a walked node-engine is not `core-behind` on its own first machine. -- D2 raises a resolver at once when every try of every question it asked was refused: a refusal is the - machine's answer that nothing listens there, not a loaded resolver. The detection is then at most one - self-check interval (five minutes). The node-engine's own network check stays the fast one (48 seconds - on the day). +- D2 is unchanged: a resolver that answered nothing, refused or silent, is held for the next run, because + a few hundred milliseconds of refusals is a resolver restarting as well as one that stopped. D2 says it + five to ten minutes after; the node-engine's own network check is the fast one (48 seconds on the day). **mesh-catalog — the tools that were missing.** - `netcheck_dns` asks one named resolver directly (`server`, `tcp`), and `netcheck_listening` lists what listens on the machine (`ss -lunt`). netcheck is assigned on the home server and the laptop only; assigning it on the control node and the workstation is the operator's. -- `docker_events` says how far back the runtime's memory reaches, and says plainly when the window asked - reaches further back, instead of answering an empty list. +- `docker_events` reads the runtime's whole buffer of events: the window is complete when the buffer is + not full (it then holds every event since the runtime started) or reaches past the window's start, and + otherwise says where the history begins, instead of answering an empty list. - `docker_resolv_conf` reads a container's resolver file as the container sees it. ## How it is checked | Rule | Check | |---|---| -| A reopened fault from before a send fails no gate | controller `TestReplay348` (the day's statements: raised, an undecided restart, raised again; the gate on a send between), `TestAReopenedFaultFromBeforeTheSendIsNotRaisedSinceIt`, conditions `TestAReopeningKeepsWhenTheFaultBegan` — the first two fail on the commit before the fix | -| An undecided statement clears nothing | `TestReplay348`, `TestAnUndecidedStatementKeepsTheMachinesNetworkConditions` | +| A fault there at the send fails no gate; one recovered at the send and back after it does | controller `TestReplay348` (the day's statements: raised, an undecided restart, raised again; the gate on a send between), `TestAFaultThatFlappedAfterTheSendIsNotTheSendsAndOneThatRecoveredBeforeItIs`, conditions `TestAReopeningKeepsWhenTheFaultBegan`, `TestAFaultKeepsItsNewestGapsAndForgetsWhatWasBefore` | +| An undecided part holds only what names it | `TestReplay348`, `TestAnUndecidedPartHoldsOnlyWhatNamesIt` | | A release's modules are rolling | `TestAReleaseRollsOutWhatItsTierNames` | -| A refusing resolver is raised at once | `TestAResolverThatRefusesEveryTryIsSaidAtOnce` — fails on the commit before the fix | -| The resolver is asked a real question, restarted, and said when it did not help | dnsmasq `TestALookFindsAResolverThatStoppedAnsweringOnOneAddress`, `TestTheWatchRestartsOnASecondFailingLookWithinItsBudget`, `TestTheWatchRestartsAResolverThatStoppedAnsweringAndHealthSaysWhetherItHelped` | +| The resolver is asked a real question, restarted, and said when it did not help | dnsmasq `TestALookFindsAResolverThatStoppedAnsweringOnOneAddress`, `TestTheWatchRestartsOnASecondFailingLookWithinItsBudget`, `TestTheWatchRestartsAResolverThatStoppedAnsweringAndHealthSaysWhetherItHelped`, `TestTheWatchRestartsOnlyAUnitThatRuns`, `TestTheWatchLoopKeepsItsBudgetAcrossItsOwnRestart`, `TestHealthSaysWhenTheWatchIsNotLookingOrTheUnitIsNotRunning`, `TestTheAnnouncerSaysWhatChangedAndRemembersOnlyWhatWasSaid` | +| Every changed condition is tested | the controller's and the modules' changes were mutated one at a time (28 in the controller, 31 in the modules); each failed a test | | live | after delivery: `dnsmasq_health` on both resolvers healthy; a restart by the watch said as `restarted` on the bus | A replay in mesh-lab's register is still owed before this core issue resolves (ADR 0237). diff --git a/04-ISSUES/349-a-merge-acted-on-late-superseded-the-merge-after-it-and-built-the-older-commit/00-report.md b/04-ISSUES/349-a-merge-acted-on-late-superseded-the-merge-after-it-and-built-the-older-commit/00-report.md index 29602fb9..3f2ff2ff 100644 --- a/04-ISSUES/349-a-merge-acted-on-late-superseded-the-merge-after-it-and-built-the-older-commit/00-report.md +++ b/04-ISSUES/349-a-merge-acted-on-late-superseded-the-merge-after-it-and-built-the-older-commit/00-report.md @@ -39,17 +39,26 @@ left it, so a later merge's commit contains every earlier one's, and the forge's In mesh-controller, in the same pull request as issue 348 (PR 179): -- A plan keeps when its merge was made (`merged_at`, migration 0086; null for a release and for a plan kept - before it). +- A plan keeps when its merge was made, to the nanosecond (`merged_at`, migration 0086; null for a release + and for a plan kept before it). - `supersededBy` orders two plans of a branch by their merge times where both are known, and by when they were made only where one is not. -- A merge older than an open plan of its branch is planned at that plan's commit, which contains it, and - that plan is superseded by it as by any newer plan. What both merges moved is built once, from the newest - commit of the branch. +- A merge older than the newest planned merge of its branch — that plan in any state, open or already done + — is planned at that merge's commit, which contains it, with that merge's time. A module the newer merge + already looked at then reads as history and is not built again; what only the older merge moved, and its + dependents, are built from the newest commit. Comparing with open plans alone left a gap the review + found: a late merge arriving after the newer plan was done rebuilt its dependents at the older commit. + That gap is closed. +- Not covered: a newer merge that moved nothing the mesh builds made no plan, so it is not known here; a + late older merge after it is then planned at its own commit, as before. Its modules are those the newer + merge did not change, so what it builds from the older commit is what the newer one holds too, unless a + dependent's build reads shared code the newer merge changed. ## How it is checked `TestTwoMergesActedOnInReverseOrderBuildTheNewerCommit` replays the day's case: the later merge acted on first, the earlier one second; one plan is left open, at the later commit, building what both moved. -`TestTheBranchOrderIsTheMerges` holds the order and the lookup. Both fail on the commit before the fix. +`TestALateMergeAfterTheNewerPlanEndedBuildsTheNewerCommit` acts on the late merge after the newer plan is +done. `TestTheBranchOrderIsTheMerges` holds the order and the lookup, to the nanosecond. The first and the +last fail on the commit before the fix, and every condition of the change was mutated and caught. A replay in mesh-lab's register is still owed before this core issue resolves (ADR 0237).