ADR 0236, to-be 45: a build is judged on its first machine and put back by something other than itself
Phase 4 of to-be 45, started by the operator: the core's health as probes, a gate on every plan's first machine, the rollback there once per build, the witness's contract with the host, the bus as a step a person starts, and — with those in place — rolling out as the default, keeping record where a module says why. 47 pushes by hand in one day are what it ends.
This commit is contained in:
@@ -4,6 +4,7 @@ status: in-progress
|
||||
code: [mesh-controller, mesh-host, mesh-tools, mesh-catalog, mesh-sdk, mesh-lab]
|
||||
updated: 2026-10-06
|
||||
decisions:
|
||||
- 02-DECISIONS/0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md
|
||||
- 02-DECISIONS/0234-the-mesh-holds-a-conversation-with-its-operator.md
|
||||
- 02-DECISIONS/0233-a-module-declares-the-data-it-holds-and-the-mesh-protects-and-watches-it-from-that.md
|
||||
- 02-DECISIONS/0227-the-core-holds-nine-rules-each-checked-and-is-built-to-them-in-six-phases.md
|
||||
@@ -224,7 +225,9 @@ itself — an unanswered probe is never a pass.
|
||||
| D12 | every consumer of a provision that keeps its data is bound where it was last sent, or moves by a pin (`binding-kept`, `binding-moving`, `binding-moved`) | ADR 0232 |
|
||||
| D13 | every item of data a machine declares is measured, is there, holds what it held, is written where it says it is, is backed up within its bound or sits on healthy redundant storage, and is no empty replacement of a copy kept elsewhere; an irreplaceable or valuable item a machine no longer declares is retired, not forgotten. Each machine's `node-backup` holder is asked `backed-up`; every provider of kept consumer data `provisioner_retirement` (with each held consumer's size) — see §2 for the kinds | ADR 0233, issue 273 |
|
||||
| DW | the watchdogs of §3 ran within three of their intervals: the watchers are watched, and raise S10 when the self-check stops | rule 6 |
|
||||
| H-* | the health probes of §8, run for every core component on every machine | rule 8 |
|
||||
| H-* | the health probes of §8, run for every core component on every machine: H-controller, H-engine, H-tools, H-bus (as built, [ADR 0236](../../02-DECISIONS/0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md)) | rule 8 |
|
||||
| DG | every build that failed its gate, and every core build a witness put back, is said — `build.<module>.<machine>.rolled-back`, `core.<component>.<machine>.<outcome>`, `rollback-failed` — until a newer build passes or the reports stop saying it (ADR 0236) | rule 8 |
|
||||
| DB | a bus upgrade a person started is said as `bus-maintenance` while it runs, and ends healthy within its bound or is said `bus-upgrade-failed`, with its snapshot (ADR 0236) | rule 8 |
|
||||
|
||||
- **`doctor`** answers the last run's verdict at once: per probe, pass, fail or failed-to-run, and age.
|
||||
**`doctor run`** runs now under a call id. **`doctor probes`** lists the registry; **`doctor
|
||||
@@ -403,12 +406,44 @@ the other machines follow. "Reported applied" is not enough.
|
||||
verdict, time to verdict, rolled back or not. Read through `plans`.
|
||||
- **A rolled-back build is not retried** by the same plan. A newer merge makes a new plan.
|
||||
|
||||
**As built** ([ADR 0236](../../02-DECISIONS/0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md)):
|
||||
|
||||
- **Every release plan is gated, not only the core's.** A module is judged by its own health: it reported
|
||||
applied, no witness put it back, no condition was raised since its send about the machine or about the
|
||||
module there, and its tools are served on that machine where it has tools. Three healthy judgings at
|
||||
least forty seconds apart and two minutes after the send, within ten minutes of it. A policy of
|
||||
*together* is not gated.
|
||||
- **The rollback is the ordinary path once per build.** The build is marked failed at its gate (never
|
||||
registered again, `plans retry` refuses it); the module's registered build goes back to the build the
|
||||
first machine ran before, still kept by ADR 0189; that machine is sent it. The verdict is written before
|
||||
the send. Said as `build.<module>.<machine>.rolled-back` (warning) or `core.<component>.<machine>.
|
||||
rolled-back` (urgent), `rollback-failed` (urgent, the operator's) when nothing could be put back, and as
|
||||
the event `rolled-back`.
|
||||
- **The witness, as built on the host** (mesh-host `internal/witness/contract.go`, the controller's half
|
||||
`internal/lease/witness.go`): the controller's is the lease alone — the holder on this machine, taken
|
||||
since the start, renewed within fifteen seconds, within sixty seconds of the start — and the node tools'
|
||||
is their answer to the services protocol's ping within five seconds, within sixty. The controller's
|
||||
"status in bound, doctor ran once" is the controller's own word in the lease's value, which the gate
|
||||
reads and the host does not: a controller that holds the lease and never becomes ready is put back by
|
||||
the gate sending the previous build. A witness says its verdict in `rollbacks` on every report while it
|
||||
stands; the controller raises `core.<component>.<machine>.<outcome>` from it — urgent for rolled-back,
|
||||
not-reversible, restore-failed and halted — and clears it with the first report without it.
|
||||
- **A new controller that passes its gate sends the bus's machine the user list it composes**, when that
|
||||
changed and nothing held back would go with it.
|
||||
- **The default policy is to roll** (ADR 0236 §4); `record` stays where a module says why, keeps
|
||||
irreplaceable data, or is the bus.
|
||||
|
||||
**The bus is a planned step.** A bus upgrade is a maintenance step a person starts through the
|
||||
controller: the streams are snapshotted; a `bus-maintenance` condition is open for the step's duration;
|
||||
the bus is replaced; afterwards D6, D7 and the round trip must pass, or the step is reported failed and
|
||||
the snapshot is the way back. A step whose new version cannot be reverted (the bus's 2.10 → 2.11 is one)
|
||||
says so before it starts and runs only on a person's explicit word, recorded as a hand act. Whether the
|
||||
bus becomes a cluster that can be upgraded live is left to its own effort. *2026-10-06:* the snapshot
|
||||
bus becomes a cluster that can be upgraded live is left to its own effort.
|
||||
|
||||
As built (ADR 0236): the verb is `bus upgrade --why … --reversible|--irreversible`; the snapshot is the
|
||||
bus machine's backup holder backing the bus module up now ([ADR 0235](../../02-DECISIONS/0235-the-bus-is-backed-up-by-its-own-snapshot-of-each-stream.md)),
|
||||
or one a person took and names with `--snapshot-taken`; the step's bound is fifteen minutes; a push
|
||||
naming the bus's machine is refused while a new bus build waits for it. *2026-10-06:* the snapshot
|
||||
and the way back from it exist — the bus image's own snapshot program, the same one the night's backup
|
||||
runs, and a restore that builds a new store beside the live one for a person to swap in
|
||||
([ADR 0235](../../02-DECISIONS/0235-the-bus-is-backed-up-by-its-own-snapshot-of-each-stream.md),
|
||||
@@ -547,7 +582,7 @@ warning, not compared), and SMART read under an array.
|
||||
|
||||
| Repository | Delivers |
|
||||
|---|---|
|
||||
| mesh-controller | the health probes of §8; the gate on a core component's first machine; the rollout record in the plan; the bus maintenance step as a verb |
|
||||
| mesh-controller | the health probes of §8; the gate on a core component's first machine — as built, on every plan's first machine (ADR 0236); the rollout record in the plan; the bus maintenance step as a verb; the default upgrade policy `roll` (ADR 0236) |
|
||||
| mesh-host | keeping the previous controller and node tools builds; restoring one when its health is not met in bound, watching the lease bucket for the controller |
|
||||
| mesh-tools | answering the health ping |
|
||||
| mesh-lab | R3, R7, R8 |
|
||||
@@ -557,6 +592,21 @@ one that starts and does nothing, one that crashes, one that cannot reach the bu
|
||||
back with no hand, the mesh ends on the previous build, and a condition and a message say so. Live: the
|
||||
next three core rollouts each record a health verdict.
|
||||
|
||||
**Built 2026-10-06** ([ADR 0236](../../02-DECISIONS/0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md),
|
||||
the operator's "start phase 4"): in mesh-controller (migration 0073), the probes H-controller, H-engine,
|
||||
H-tools, H-bus, DG and DB; the gate on every plan's first machine, its record in the plan (`plans <id>`)
|
||||
and in the store; the rollback by the ordinary path, once per build; the witnesses' verdicts as
|
||||
conditions; the grants the witness reads; `bus` and `bus upgrade`; the default policy `roll`, `upgrade`
|
||||
listing every module's policy and where it comes from; the user list carried after a controller passes;
|
||||
a module deleted at its source forgotten, not built. In mesh-host, the witness (keeping the previous
|
||||
controller and node tools, restoring one not healthy in bound, the launcher's for the node-engine) and the
|
||||
installer's grants. In mesh-catalog, the modules that keep `record` say why, and the forge's announcer
|
||||
says which files a merge deleted. On branches, not yet merged. **Not yet:** R3, R7, R8 on a lab mesh, and
|
||||
so the *done when*; container state in the node-engine's report, without which a container that
|
||||
crash-loops after its compose applied is seen only through what it breaks; composing `not-reversible` for
|
||||
a build whose migration cannot be undone; the gate for a build asked by hand outside a plan; the next
|
||||
three core rollouts' verdicts.
|
||||
|
||||
### Phase 5 — Checks before merge, and the replays
|
||||
|
||||
| Repository | Delivers |
|
||||
|
||||
Reference in New Issue
Block a user