Files
hq/04-ISSUES/187-the-mesh-tells-nobody-when-it-stops-working/00-report.md
T

3.9 KiB

status, opened, located-in, fixed-by, amended-design
status opened located-in fixed-by amended-design
open 2026-10-01

187 — The mesh tells nobody when it stops working

What was observed

One day, 2026-10-01, and five faults, each found by a person reading a container's log hours after it began, and each invisible to every surface the mesh offers:

  • The controller's receive loop was blocked for twenty-four minutes by a merge handler, then for nineteen minutes by a publish the bus had refused (184, 185). Throughout, status answered and read as quiet, builds listed what it had, the console answered every tool. Nothing said the controller has taken nothing in since 13:17.
  • The build machine dropped twenty-six of forty-three asks (186). Nothing counts asks against builds; the queue read as empty; the loss was inferred two hours later from a wave that would not finish.
  • The controller's own grant refused every membership it published (183), and then every module on the new runtime was refused its one read of the stream. Both were one Publish Violation line each in the bus's log, which nothing in the mesh reads.
  • The bus dropped the machines' heartbeats as a slow consumer, twice, and said so to the controller's log only.

In every case the designed fallback held — runtimes served the derived shape, a push later carried what an earlier one had not — which is why the mesh kept working and why nobody was told.

One more the same evening: the laptop applied a declaration and logged applied, and could not tell the mesh: reporting: context canceled. The report was not retried; the mesh went on believing the machine's previous state until the next push, and nothing on either side counted the loss.

Why this is here

The repository's own rule is that a rule states how it is checked, and every record here does. But the checks are tests and status, both asked by a person. The mesh has no account of its own liveness: whether the controller is hearing, whether the queue is moving, whether the bus is refusing what the mesh composed, how old each machine's last report is. A fault that leaves the fallbacks standing is a fault nobody learns about until it compounds, and today three of them compounded into an evening of reading logs. This is the design permitting a failure to be silent, which is the first line of what belongs here.

What a decision would settle

  • What the mesh observes about itself. At least: the receive loop's last message taken and its age; asks against builds, with the oldest unbuilt ask's age; publishes the bus refused and subscriptions it dropped, read from the bus rather than from a log; each machine's last report and heartbeat age; a module whose runtime says it serves the derived shape.
  • Where it says so. As events on the bus under the controller's seat, so a log viewer and a notifier are consumers and not special cases; and in status, which must go red for any of them rather than listing only machines that are behind.
  • Who is told. A channel a person actually reads — the mesh already has modules that send mail and messages — chosen once, with a rule for what interrupts a person and what waits for status.
  • What is not a monitor. Nothing here is a dashboard product the mesh adopts; it is the mesh stating facts about itself, the way ADR 0134 made it state what it did.

How this would be checked: a controller test where the loop is held and status goes red naming the age; a test where an ask is unbuilt past a bound and builds says so; live, the next fault of today's kinds reaches a person before a person reaches the log.