Issues 348 and 349: the review's answers — gaps, parts, D2 unchanged, the newest merge in any state

04-ISSUES/348-a-resolver-that-stopped-answering-ran-on-unnoticed-and-builds-sent-after-it-were-blamed/01-diagnosis.md

@@ -126,50 +126,58 @@ waits for the private address (above).
 TypeScript tools were never served on any machine). Its new long-running process, the watch, runs as root
 and every 15 seconds asks the machine's own name — from the roster — on every address the configuration
 listens on, over UDP and TCP, and checks the answer is that machine's address. After two failing looks in
-a row it restarts the unit, at most once every two minutes and three times an hour, and the looks after it
+a row it restarts the unit — only one that runs: a unit stopped or disabled on purpose, still starting
+(waiting at boot for the private address) or failed is the service manager's or a person's, and is held
+and said — without waiting for the start, at most once every two minutes and three times an hour (a
+budget it reads back when it restarts itself), and the looks after it
 say whether the restart helped. Its tool `dnsmasq_health` is the watch's declared check (ADR 0240): it asks
 again and reads what the watch did, so a restart that helped leaves the module healthy and the operator
 hears nothing, and one that did not becomes `module.dnsmasq.<machine>.unhealthy`, saying "restarted at …,
 which did not help". The unit itself is judged by its unit (`health: unit`). The bundle emits `restarted`
-on the bus beside the names it answers. The repair is the module's own, as the identity provider repairs
+and, once the looks decide it, `restart-judged` (helped or not) on the bus beside the names it answers,
+each remembered as said only once emitted. The repair is the module's own, as the identity provider repairs
 its administrator (ADR 0224 §5), because a resolver that stops answering can cut its machine off from the
 bus by name, and then nothing outside the machine reaches it. **This departs from ADR 0240 rule 6**
 ("nothing restarts on health"), which left a restart on health to a healer's own record under ADR 0231.
 That record is not written; whether this module-local repair stands, or becomes a row of the healer
 registry, is the operator's decision.
 
-**mesh-controller — a fault is judged by when it began.**
+**mesh-controller — a fault is judged by whether it was there at the send.**
 
-- A condition keeps `first`, when the fault was first raised, across a reopening within `ReopenWithin`
-  (and across a controller restart, from the history); `Began()` reads it. The gate compares `Began()`,
-  not the raised time, with the send.
-- An undecided network statement — a part `unknown` or `starting`, as a node-engine just started says —
-  clears no network condition about that machine. The next statement decides.
+- A condition reopened within `ReopenWithin` keeps when the fault was first raised (`first`) and its
+  gaps, each from a clearing to the reopening after it, across a controller restart too (from the
+  history). The gate asks `OpenAt(send)`: the fault began at or before the send and the send fell in none
+  of its gaps. So a fault there at the send that flapped after it is not the send's, and a machine that
+  had recovered before the send and broke after it fails the gate, as it must (review of the controller's
+  pull request: a silent pass is worse than a false blame).
+- An undecided part of a network statement — `unknown` or `starting`, as a node-engine just started says
+  — holds the open network conditions whose newest evidence names that part, and only those. A statement
+  undecided as a whole holds every part. The next statement decides.
 - D10 counts what a release's tier names as rolling out, so a walked node-engine is not `core-behind` on
   its own first machine.
-- D2 raises a resolver at once when every try of every question it asked was refused: a refusal is the
-  machine's answer that nothing listens there, not a loaded resolver. The detection is then at most one
-  self-check interval (five minutes). The node-engine's own network check stays the fast one (48 seconds
-  on the day).
+- D2 is unchanged: a resolver that answered nothing, refused or silent, is held for the next run, because
+  a few hundred milliseconds of refusals is a resolver restarting as well as one that stopped. D2 says it
+  five to ten minutes after; the node-engine's own network check is the fast one (48 seconds on the day).
 
 **mesh-catalog — the tools that were missing.**
 
 - `netcheck_dns` asks one named resolver directly (`server`, `tcp`), and `netcheck_listening` lists what
   listens on the machine (`ss -lunt`). netcheck is assigned on the home server and the laptop only;
   assigning it on the control node and the workstation is the operator's.
-- `docker_events` says how far back the runtime's memory reaches, and says plainly when the window asked
-  reaches further back, instead of answering an empty list.
+- `docker_events` reads the runtime's whole buffer of events: the window is complete when the buffer is
+  not full (it then holds every event since the runtime started) or reaches past the window's start, and
+  otherwise says where the history begins, instead of answering an empty list.
 - `docker_resolv_conf` reads a container's resolver file as the container sees it.
 
 ## How it is checked
 
 | Rule | Check |
 |---|---|
-| A reopened fault from before a send fails no gate | controller `TestReplay348` (the day's statements: raised, an undecided restart, raised again; the gate on a send between), `TestAReopenedFaultFromBeforeTheSendIsNotRaisedSinceIt`, conditions `TestAReopeningKeepsWhenTheFaultBegan` — the first two fail on the commit before the fix |
-| An undecided statement clears nothing | `TestReplay348`, `TestAnUndecidedStatementKeepsTheMachinesNetworkConditions` |
+| A fault there at the send fails no gate; one recovered at the send and back after it does | controller `TestReplay348` (the day's statements: raised, an undecided restart, raised again; the gate on a send between), `TestAFaultThatFlappedAfterTheSendIsNotTheSendsAndOneThatRecoveredBeforeItIs`, conditions `TestAReopeningKeepsWhenTheFaultBegan`, `TestAFaultKeepsItsNewestGapsAndForgetsWhatWasBefore` |
+| An undecided part holds only what names it | `TestReplay348`, `TestAnUndecidedPartHoldsOnlyWhatNamesIt` |
 | A release's modules are rolling | `TestAReleaseRollsOutWhatItsTierNames` |
-| A refusing resolver is raised at once | `TestAResolverThatRefusesEveryTryIsSaidAtOnce` — fails on the commit before the fix |
-| The resolver is asked a real question, restarted, and said when it did not help | dnsmasq `TestALookFindsAResolverThatStoppedAnsweringOnOneAddress`, `TestTheWatchRestartsOnASecondFailingLookWithinItsBudget`, `TestTheWatchRestartsAResolverThatStoppedAnsweringAndHealthSaysWhetherItHelped` |
+| The resolver is asked a real question, restarted, and said when it did not help | dnsmasq `TestALookFindsAResolverThatStoppedAnsweringOnOneAddress`, `TestTheWatchRestartsOnASecondFailingLookWithinItsBudget`, `TestTheWatchRestartsAResolverThatStoppedAnsweringAndHealthSaysWhetherItHelped`, `TestTheWatchRestartsOnlyAUnitThatRuns`, `TestTheWatchLoopKeepsItsBudgetAcrossItsOwnRestart`, `TestHealthSaysWhenTheWatchIsNotLookingOrTheUnitIsNotRunning`, `TestTheAnnouncerSaysWhatChangedAndRemembersOnlyWhatWasSaid` |
+| Every changed condition is tested | the controller's and the modules' changes were mutated one at a time (28 in the controller, 31 in the modules); each failed a test |
 | live | after delivery: `dnsmasq_health` on both resolvers healthy; a restart by the watch said as `restarted` on the bus |
 
 A replay in mesh-lab's register is still owed before this core issue resolves (ADR 0237).

04-ISSUES/349-a-merge-acted-on-late-superseded-the-merge-after-it-and-built-the-older-commit/00-report.md

@@ -39,17 +39,26 @@ left it, so a later merge's commit contains every earlier one's, and the forge's
 
 In mesh-controller, in the same pull request as issue 348 (PR 179):
 
-- A plan keeps when its merge was made (`merged_at`, migration 0086; null for a release and for a plan kept
-  before it).
+- A plan keeps when its merge was made, to the nanosecond (`merged_at`, migration 0086; null for a release
+  and for a plan kept before it).
 - `supersededBy` orders two plans of a branch by their merge times where both are known, and by when they
   were made only where one is not.
-- A merge older than an open plan of its branch is planned at that plan's commit, which contains it, and
-  that plan is superseded by it as by any newer plan. What both merges moved is built once, from the newest
-  commit of the branch.
+- A merge older than the newest planned merge of its branch — that plan in any state, open or already done
+  — is planned at that merge's commit, which contains it, with that merge's time. A module the newer merge
+  already looked at then reads as history and is not built again; what only the older merge moved, and its
+  dependents, are built from the newest commit. Comparing with open plans alone left a gap the review
+  found: a late merge arriving after the newer plan was done rebuilt its dependents at the older commit.
+  That gap is closed.
+- Not covered: a newer merge that moved nothing the mesh builds made no plan, so it is not known here; a
+  late older merge after it is then planned at its own commit, as before. Its modules are those the newer
+  merge did not change, so what it builds from the older commit is what the newer one holds too, unless a
+  dependent's build reads shared code the newer merge changed.
 
 ## How it is checked
 
 `TestTwoMergesActedOnInReverseOrderBuildTheNewerCommit` replays the day's case: the later merge acted on
 first, the earlier one second; one plan is left open, at the later commit, building what both moved.
-`TestTheBranchOrderIsTheMerges` holds the order and the lookup. Both fail on the commit before the fix.
+`TestALateMergeAfterTheNewerPlanEndedBuildsTheNewerCommit` acts on the late merge after the newer plan is
+done. `TestTheBranchOrderIsTheMerges` holds the order and the lookup, to the nanosecond. The first and the
+last fail on the commit before the fix, and every condition of the change was mutated and caught.
 A replay in mesh-lab's register is still owed before this core issue resolves (ADR 0237).