diff --git a/01-RESEARCH/008-delivery-coordinator/00-overview.md b/01-RESEARCH/008-delivery-coordinator/00-overview.md index d862506..9117f4c 100644 --- a/01-RESEARCH/008-delivery-coordinator/00-overview.md +++ b/01-RESEARCH/008-delivery-coordinator/00-overview.md @@ -1,12 +1,14 @@ --- -status: active +status: graduated initiated: 2026-08-23 touches: - 03-DESIGN/00-as-is/04-delivery.md - 02-DECISIONS/0014-build-publish-and-deploy-are-three-silos.md - 02-DECISIONS/0013-an-artifact-is-build-output.md - 01-RESEARCH/006-mesh-from-scratch/code-skeleton.md -became: [] +became: + - 02-DECISIONS/0058-delivery-ends-in-a-declaration.md + - 02-DECISIONS/0063-delivery-is-reconciliation-not-a-pipeline.md --- # 008 — The coordinator: a change checked in becomes a deployed state @@ -41,26 +43,53 @@ Research 006 adds a requirement the current design does not have: the coordinato **before the mesh is self-hosting**, when source and artifacts come from outside, and keep working across the transition to self-hosted providers. -## Answered since +## What it became + +*Closed 2026-08-28.* All six questions are answered, by two records, and the second exists +because the first was honest about what it did not fix. **Does the coordinator dispatch stages, or converge nodes on a declaration?** — *Converge.* [ADR 0058](../../02-DECISIONS/0058-delivery-ends-in-a-declaration.md): a pipeline ends when the -declaration is updated, and deploy stops being once-per-node. The host applies it and reads back, -so the reporter is the applier — which is what this effort was circling when it asked what a -*deployed state* is. +declaration is updated, and the host applies it and reads back — so the reporter is the applier. -**Does the three-silo split survive?** — *Yes, with the third redefined.* Build and publish are -unchanged; deploy becomes one write rather than a fan-out. +**Does the three-silo split survive?** — *Yes, with the third redefined.* The cardinality +observation holds; the third silo is not a stage any more. -**What is a deployed state?** — *Partly.* "The declaration is updated, and here is which nodes -have applied it" is the answer 0058 gives, and it makes *outstanding* a first-class result rather -than a stall. What it does not answer is what a node reports and how, which waits on the link. +**How does a change become a pipeline, reliably?** — *It does not become a pipeline at all.* +[ADR 0063](../../02-DECISIONS/0063-delivery-is-reconciliation-not-a-pipeline.md) applies 0058's +move one level up: the control plane holds what source exists and what has been built, and builds +the difference. **An event makes it fast; nothing makes it necessary.** The failures this effort +catalogued — a truncated commit list, a broken path match — become latency rather than silence. -**Explicitly NOT answered, and 0058 says so:** *how a change becomes a pipeline, reliably.* -Detection remains the fragile input — *a merge that created no pipeline, and nothing said so* is -upstream of everything 0058 changed and is untouched by it. +**What is a deployed state?** — *Two comparisons, not an event.* Does every node's reported state +match what is declared, and is what is declared built from current source? A milestone can be +claimed by something that did not check; a comparison cannot. -## The questions +**What produces a verdict, and what is it about?** — *An artifact, and it gates eligibility.* The +mesh must not converge onto something broken, so an artifact may be declared only once the lab +has judged it fit. Sharper than the question expected: a verdict is a property an artifact has, +not a report about a run. + +**How does delivery work before self-hosting?** — *It mostly stops being a question.* A reconciler +reads source and writes artifacts; where those live is a binding, external first and internal +later. The transition looked hard because a pipeline's stages name their targets. + +## What this effort was right about + +Its first question — *what is a deployed state, and how does the mesh know it is in one* — was +marked "everything follows from this", and everything did. Both records above are answers to it: +0058 makes the applier the reporter, and 0063 makes currency a comparison. The effort put the +load-bearing question first. + +## What is NOT closed by this + +[ADR 0063](../../02-DECISIONS/0063-delivery-is-reconciliation-not-a-pipeline.md) names four costs +and one of them is a real risk rather than a trade: **a reconciler that cannot reach its target +retries forever, and without something that notices, the failure is silence** — which is the +fault this effort exists to catalogue, reintroduced in a new place. That belongs to observability +and it is not designed. + +## The questions (all answered above) | Question | Why it matters | |---|---| diff --git a/02-DECISIONS/0063-delivery-is-reconciliation-not-a-pipeline.md b/02-DECISIONS/0063-delivery-is-reconciliation-not-a-pipeline.md new file mode 100644 index 0000000..16e78cb --- /dev/null +++ b/02-DECISIONS/0063-delivery-is-reconciliation-not-a-pipeline.md @@ -0,0 +1,120 @@ +--- +status: proposed +date: 2026-08-28 +deciders: jochen +reconstructed: false +extends: 0058-delivery-ends-in-a-declaration.md +--- + +# 63. Delivery is reconciliation, not a pipeline + +## Context + +[ADR 0058](0058-delivery-ends-in-a-declaration.md) stopped *deploy* being a stage that pushes to +nodes: the control plane says what should be true, the host reconciles, and the thing that +reports is the thing that did the work. It fixed the third silo and left the first two as they +were — **and it said plainly what it did not fix:** + +> **Detection stays the fragile input, and this does not fix it.** *A merge that created no +> pipeline, and nothing said so* is upstream of everything here and is untouched. + +That is not a defect in the detector. It is what happens when a system's correctness depends on +an **event arriving**. The as-is records the ways it has failed — a webhook truncating its commit +list on a large merge, a forge address whose port broke module-path matching — and the shape is +always the same: *nothing happened, and nothing said so.* + +**The same move that fixed deploy fixes this, applied one level up.** This record is not a +better detector. It is the removal of detection as a load-bearing mechanism. + +**And this is not the current coordinator repaired.** The existing pipeline is a state machine +over stages; what follows is not that with better inputs. The old system's value here is as a +catalogue of the ways this can fail, and it has been used for exactly that. + +## Decision + +> **The control plane holds what source exists and what has been built from it, and builds the +> difference.** + +A change becomes a build because **source is ahead of artifacts** — a comparison, answerable at +any moment — rather than because a message arrived. + +**An event makes it fast. Nothing makes it necessary.** A push notification is an optimisation +that lowers latency; a missed one costs latency and cannot cost correctness. That is the same +property the host's drift timer has, and it is the whole point of both. + +So the mesh is one idea at two layers: + +| | reconciles | against | +|---|---|---| +| **the control plane** | artifacts | source | +| **the host** | machine state | declarations | + +**What disappears:** the pipeline as a state machine. There is no stage list something can be +omitted from — which is how a verify stage was built and never scheduled — and no run to lose. + +## What this answers + +[Research 008](../01-RESEARCH/008-delivery-coordinator/00-overview.md) asked six questions. Two +were answered by [ADR 0058](0058-delivery-ends-in-a-declaration.md); this answers the rest. + +**What is a deployed state?** Not an event — **two comparisons**, both answerable on demand: does +every node's reported state match what is declared, and is what is declared built from current +source? A milestone can be claimed by something that did not check. A comparison cannot. + +**What produces a verdict, and what is it about?** **An artifact, and it gates eligibility.** The +mesh must not converge onto something broken, so an artifact may be declared only once something +has judged it fit. The lab is what judges. This is sharper than the question expected: a verdict +is not a report about a run, it is a property an artifact does or does not have. + +**How does delivery work before self-hosting?** It mostly stops being a question. A reconciler +needs to read source and write artifacts; where those live is a **binding**, external at first +and internal later. A pipeline has stages that name their targets, which is why the transition +looked hard. + +## What survives from ADR 0014 + +[ADR 0014](0014-build-publish-and-deploy-are-three-silos.md) is a decision about **cardinality**, +and the cardinality observation is right and unchanged: building is per module, publishing is per +module, and what happens on nodes is per node. What changes is that those are no longer three +**silos of a job**. They are steps of reconciling one artifact, and the third is not a step at all +any more ([ADR 0058](0058-delivery-ends-in-a-declaration.md)). + +## What this costs + +Named because each is a way this can go wrong, and a decision that lists none has not been +examined. + +- **"Is this artifact current?" must be answerable without building it.** A commit recorded + against each artifact does it, and that record becomes load-bearing: wrong, and the mesh either + rebuilds forever or never rebuilds at all. +- **Rebuild storms.** One change to a shared library makes everything out of date at once. The + ordering that today's *levels* provide has to come from the module graph + ([research 011](../01-RESEARCH/011-the-module-graph/00-overview.md)), which is designed and not + built. +- **The run identity people actually use is lost.** *Did my change go out?* is answerable today + by opening a pipeline. With convergence there is no run to open, and **something has to replace + that** — a query over the two comparisons above — or this will be worse to live with than what + it replaces, whatever its properties. +- **A loop that will not converge is harder to debug than a job that failed.** A failed job stops + and names its step. A reconciler that cannot reach its target retries forever, and without + something that notices *this has been trying for an hour*, the failure is silence — which is + the fault this record is removing, reintroduced in a new place. **This is the real risk and it + is not solved here.** + +## Consequences + +- **Detection stops being correctness and becomes latency.** The specific faults the as-is + records — a truncated commit list, a broken path match — become slow rather than silent. +- **The coordinator is not ported.** What replaces it is a comparison and a build, and the + existing implementation informs it only as a list of things that went wrong. +- **Two things must be cheap that are not yet designed**: reading what source exists, and reading + what has been built. Both are queries against providers the mesh will host, and both are on the + path of everything above. +- **Research 008 can close**, which it could not before this. + +## References + +- [ADR 0058](0058-delivery-ends-in-a-declaration.md) — the same move, one level down. +- [ADR 0014](0014-build-publish-and-deploy-are-three-silos.md) — the cardinality that survives. +- [ADR 0035](0035-a-picture-is-read-from-what-runs.md) — why a comparison beats a claim. +- [`00-as-is/04`](../03-DESIGN/00-as-is/04-delivery.md) — the pitfalls this is designed against.