Report a provider that keeps failing a consumer in status (hq ADR 0224) #70

2026-10-05T22:14:09Z

mesh-admin commented

2026-10-05 22:14:09 +00:00

On 2026-10-05 the identity provider failed every consumer ~31,000 times in a day and status called the mesh well (hq issue 179).

Every module that receives contributions (a provider) is granted provisioner.failing / provisioner.recovered (Manifest.EmitsAll), so no manifest can forget them.
The controller follows both from any module (mesh.mod.*.event.provisioner.{failing,recovered} — the one wildcard on its list), reads the emitter from the subject, keeps the newest failing word per provider module/machine/consumer (migration 0065, provider_standing), clears it on recovery; a recovery is held while the store is away.
status names each (and is not all-well), status --json carries failing, node show lists those on the machine; an unassigned provider's word is ignored; a standing not repeated for 30 min is shown as gone quiet.
Golden composed.conf updated (controller subscribe line).

Tests: link (fake + live NATS), inventory (live Postgres), cmd status/node show (live Postgres). go test -p 1 ./... green with MESH_TEST_POSTGRES + MESH_TEST_NATS before rebase. After rebasing onto main, internal/catalogue fails on the sibling mesh-catalog main's hosts/node-hosts-file claims — that's main-vs-main drift from #69, not this change.

Merge order: this, then mesh-host (genesis user list), then mesh-catalog (providers announce). Pairs with hq ADR 0224.

On 2026-10-05 the identity provider failed every consumer ~31,000 times in a day and `status` called the mesh well (hq issue 179). - Every module that `receives` contributions (a provider) is granted `provisioner.failing` / `provisioner.recovered` (`Manifest.EmitsAll`), so no manifest can forget them. - The controller follows both from any module (`mesh.mod.*.event.provisioner.{failing,recovered}` — the one wildcard on its list), reads the emitter from the subject, keeps the newest failing word per provider module/machine/consumer (migration 0065, `provider_standing`), clears it on recovery; a recovery is held while the store is away. - `status` names each (and is not all-well), `status --json` carries `failing`, `node show` lists those on the machine; an unassigned provider's word is ignored; a standing not repeated for 30 min is shown as gone quiet. - Golden composed.conf updated (controller subscribe line). Tests: link (fake + live NATS), inventory (live Postgres), cmd status/node show (live Postgres). `go test -p 1 ./...` green with MESH_TEST_POSTGRES + MESH_TEST_NATS before rebase. After rebasing onto main, `internal/catalogue` fails on the sibling mesh-catalog main's `hosts`/`node-hosts-file` claims — that's main-vs-main drift from #69, not this change. Merge order: this, then mesh-host (genesis user list), then mesh-catalog (providers announce). Pairs with hq ADR 0224.

mesh-admin added 1 commit 2026-10-05 22:14:09 +00:00

Report a provider that keeps failing a consumer in status (hq ADR 0224) 8d9d33ae85

The identity provider failed every consumer for a day and status called the
mesh well (hq issue 179). The controller now follows every provider's
provisioner.failing/recovered, keeps the newest failing word per provider,
machine and consumer (migration 0065), and status, its JSON and node show
name it until it recovers. Every module that receives contributions is
granted the two events, so no manifest can forget them.

mesh-admin merged commit 6fdcfad8d3 into main

2026-10-05 22:20:46 +00:00

mesh-admin deleted branch feat/a-provider-failing-a-consumer-is-reported

2026-10-05 22:20:46 +00:00

mesh-admin commented

2026-10-06 23:09:55 +00:00

Delivery novox/mesh-controller@6fdcfad8d3e1 — held since 2026-10-06T23:05:28Z

Held for a person: merged without a passing check: only a person decides that it goes on — mesh-delivery.release with why lets it go on.

Delivery plan — builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step

tier 0: mesh-controller
tier 1: build-agent
tier 2: route-proxy
ace: receives build-agent, route-proxy
g14: receives build-agent
novox: receives mesh-controller, build-agent, route-proxy
shanks: receives build-agent
build-agent keeps data (ADR 0233): what it holds is backed up before it moves
route-proxy provides route: its consumers are sent again after it
route-proxy keeps data (ADR 0233): what it holds is backed up before it moves

Transitions

2026-10-06 23:05 (new) → proposed (announced): novox/mesh-controller#70 merged; its head was not heard by this owner, so it is made from the forge's word
2026-10-06 23:05 proposed → held (merged-unchecked): merged without a passing check — a verdict known with it is taken first: only a person decides that it goes on

The commit's note under refs/notes/mesh-plan keeps every transition: git log --notes=mesh-plan.

**Delivery** `novox/mesh-controller@6fdcfad8d3e1` — **held** since 2026-10-06T23:05:28Z Held for a person: merged without a passing check: only a person decides that it goes on — `mesh-delivery.release` with why lets it go on. **Delivery plan** — builds build-agent, mesh-controller, route-proxy → ace, g14, novox, shanks; no bus step - tier 0: mesh-controller - tier 1: build-agent - tier 2: route-proxy - ace: receives build-agent, route-proxy - g14: receives build-agent - novox: receives mesh-controller, build-agent, route-proxy - shanks: receives build-agent - build-agent keeps data (ADR 0233): what it holds is backed up before it moves - route-proxy provides route: its consumers are sent again after it - route-proxy keeps data (ADR 0233): what it holds is backed up before it moves **Transitions** - 2026-10-06 23:05 (new) → proposed (announced): novox/mesh-controller#70 merged; its head was not heard by this owner, so it is made from the forge's word - 2026-10-06 23:05 proposed → held (merged-unchecked): merged without a passing check — a verdict known with it is taken first: only a person decides that it goes on The commit's note under `refs/notes/mesh-plan` keeps every transition: `git log --notes=mesh-plan`.

This repo is archived. You cannot comment on pull requests.

1 Participants

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: novox/mesh-controller#70