novox/hq
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Delivery is reconciliation, not a pipeline; research 008 closes

Browse Source 
Jochen: don't rebuild the current coordinator, use it as a pitfall list. That
reframed the last open question rather than answering it.

0058 stopped deploy being a stage that pushes to nodes, and said plainly what
it did not fix: detection. A merge that created no pipeline, and nothing said
so. That is not a defect in the detector -- it is what happens when correctness
depends on an event ARRIVING.

0063 applies 0058's move one level up. The control plane holds what source
exists and what has been built from it, and builds the difference. A change
becomes a build because source is ahead of artifacts, which is a comparison
answerable at any moment. An event makes it fast; nothing makes it necessary,
so a missed webhook costs latency and cannot cost correctness.

The mesh becomes one idea at two layers: the control plane reconciles artifacts
against source, the host reconciles machine state against declarations. The
pipeline as a state machine disappears, and with it the stage list that a
verify step was once omitted from.

That reframing answered the three questions still open in 008, so it graduates
with all six closed. A deployed state is two comparisons rather than an event.
A verdict is about an ARTIFACT and gates whether it may be declared -- sharper
than the question expected. And "before self-hosting" mostly dissolves, because
a reconciler needs source and artifacts as bindings where a pipeline's stages
name their targets.

Four costs recorded, and one is a real risk rather than a trade: a reconciler
that cannot reach its target retries forever, and without something noticing,
the failure is silence -- the exact fault this removes, reintroduced elsewhere.
Also named: the run identity people actually use is lost, and "did my change go
out?" needs a replacement or this will be worse to live with than what it
replaces, whatever its properties.
This commit is contained in:
jschoubben
2026-08-28 02:59:42 +02:00
parent f728c3fd98
commit 4ab8a0507f
2 changed files with 164 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
01-RESEARCH/008-delivery-coordinator/00-overview.md
+44 -15
View File
@@ -1,12 +1,14 @@
---
status: active
status: graduated
initiated: 2026-08-23
touches:
- 03-DESIGN/00-as-is/04-delivery.md
- 02-DECISIONS/0014-build-publish-and-deploy-are-three-silos.md
- 02-DECISIONS/0013-an-artifact-is-build-output.md
- 01-RESEARCH/006-mesh-from-scratch/code-skeleton.md
became: []
became:
- 02-DECISIONS/0058-delivery-ends-in-a-declaration.md
- 02-DECISIONS/0063-delivery-is-reconciliation-not-a-pipeline.md
---
# 008 — The coordinator: a change checked in becomes a deployed state
@@ -41,26 +43,53 @@ Research 006 adds a requirement the current design does not have: the coordinato
**before the mesh is self-hosting**, when source and artifacts come from outside, and keep
working across the transition to self-hosted providers.
## Answered since
## What it became
*Closed 2026-08-28.* All six questions are answered, by two records, and the second exists
because the first was honest about what it did not fix.
**Does the coordinator dispatch stages, or converge nodes on a declaration?** — *Converge.*
[ADR 0058](../../02-DECISIONS/0058-delivery-ends-in-a-declaration.md): a pipeline ends when the
declaration is updated, and deploy stops being once-per-node. The host applies it and reads back,
so the reporter is the applier — which is what this effort was circling when it asked what a
*deployed state* is.
declaration is updated, and the host applies it and reads back — so the reporter is the applier.
**Does the three-silo split survive?** — *Yes, with the third redefined.* Build and publish are
unchanged; deploy becomes one write rather than a fan-out.
**Does the three-silo split survive?** — *Yes, with the third redefined.* The cardinality
observation holds; the third silo is not a stage any more.
**What is a deployed state?** — *Partly.* "The declaration is updated, and here is which nodes
have applied it" is the answer 0058 gives, and it makes *outstanding* a first-class result rather
than a stall. What it does not answer is what a node reports and how, which waits on the link.
**How does a change become a pipeline, reliably?** — *It does not become a pipeline at all.*
[ADR 0063](../../02-DECISIONS/0063-delivery-is-reconciliation-not-a-pipeline.md) applies 0058's
move one level up: the control plane holds what source exists and what has been built, and builds
the difference. **An event makes it fast; nothing makes it necessary.** The failures this effort
catalogued — a truncated commit list, a broken path match — become latency rather than silence.
**Explicitly NOT answered, and 0058 says so:** *how a change becomes a pipeline, reliably.*
Detection remains the fragile input — *a merge that created no pipeline, and nothing said so* is
upstream of everything 0058 changed and is untouched by it.
**What is a deployed state?** — *Two comparisons, not an event.* Does every node's reported state
match what is declared, and is what is declared built from current source? A milestone can be
claimed by something that did not check; a comparison cannot.
## The questions
**What produces a verdict, and what is it about?** — *An artifact, and it gates eligibility.* The
mesh must not converge onto something broken, so an artifact may be declared only once the lab
has judged it fit. Sharper than the question expected: a verdict is a property an artifact has,
not a report about a run.
**How does delivery work before self-hosting?** — *It mostly stops being a question.* A reconciler
reads source and writes artifacts; where those live is a binding, external first and internal
later. The transition looked hard because a pipeline's stages name their targets.
## What this effort was right about
Its first question — *what is a deployed state, and how does the mesh know it is in one* — was
marked "everything follows from this", and everything did. Both records above are answers to it:
0058 makes the applier the reporter, and 0063 makes currency a comparison. The effort put the
load-bearing question first.
## What is NOT closed by this
[ADR 0063](../../02-DECISIONS/0063-delivery-is-reconciliation-not-a-pipeline.md) names four costs
and one of them is a real risk rather than a trade: **a reconciler that cannot reach its target
retries forever, and without something that notices, the failure is silence** — which is the
fault this effort exists to catalogue, reintroduced in a new place. That belongs to observability
and it is not designed.
## The questions (all answered above)
| Question | Why it matters |
|---|---|
02-DECISIONS/0063-delivery-is-reconciliation-not-a-pipeline.md
+120
View File
@@ -0,0 +1,120 @@
---
status: proposed
date: 2026-08-28
deciders: jochen
reconstructed: false
extends: 0058-delivery-ends-in-a-declaration.md
---
# 63. Delivery is reconciliation, not a pipeline
## Context
[ADR 0058](0058-delivery-ends-in-a-declaration.md) stopped *deploy* being a stage that pushes to
nodes: the control plane says what should be true, the host reconciles, and the thing that
reports is the thing that did the work. It fixed the third silo and left the first two as they
were — **and it said plainly what it did not fix:**
> **Detection stays the fragile input, and this does not fix it.** *A merge that created no
> pipeline, and nothing said so* is upstream of everything here and is untouched.
That is not a defect in the detector. It is what happens when a system's correctness depends on
an **event arriving**. The as-is records the ways it has failed — a webhook truncating its commit
list on a large merge, a forge address whose port broke module-path matching — and the shape is
always the same: *nothing happened, and nothing said so.*
**The same move that fixed deploy fixes this, applied one level up.** This record is not a
better detector. It is the removal of detection as a load-bearing mechanism.
**And this is not the current coordinator repaired.** The existing pipeline is a state machine
over stages; what follows is not that with better inputs. The old system's value here is as a
catalogue of the ways this can fail, and it has been used for exactly that.
## Decision
> **The control plane holds what source exists and what has been built from it, and builds the
> difference.**
A change becomes a build because **source is ahead of artifacts** — a comparison, answerable at
any moment — rather than because a message arrived.
**An event makes it fast. Nothing makes it necessary.** A push notification is an optimisation
that lowers latency; a missed one costs latency and cannot cost correctness. That is the same
property the host's drift timer has, and it is the whole point of both.
So the mesh is one idea at two layers:
| | reconciles | against |
|---|---|---|
| **the control plane** | artifacts | source |
| **the host** | machine state | declarations |
**What disappears:** the pipeline as a state machine. There is no stage list something can be
omitted from — which is how a verify stage was built and never scheduled — and no run to lose.
## What this answers
[Research 008](../01-RESEARCH/008-delivery-coordinator/00-overview.md) asked six questions. Two
were answered by [ADR 0058](0058-delivery-ends-in-a-declaration.md); this answers the rest.
**What is a deployed state?** Not an event — **two comparisons**, both answerable on demand: does
every node's reported state match what is declared, and is what is declared built from current
source? A milestone can be claimed by something that did not check. A comparison cannot.
**What produces a verdict, and what is it about?** **An artifact, and it gates eligibility.** The
mesh must not converge onto something broken, so an artifact may be declared only once something
has judged it fit. The lab is what judges. This is sharper than the question expected: a verdict
is not a report about a run, it is a property an artifact does or does not have.
**How does delivery work before self-hosting?** It mostly stops being a question. A reconciler
needs to read source and write artifacts; where those live is a **binding**, external at first
and internal later. A pipeline has stages that name their targets, which is why the transition
looked hard.
## What survives from ADR 0014
[ADR 0014](0014-build-publish-and-deploy-are-three-silos.md) is a decision about **cardinality**,
and the cardinality observation is right and unchanged: building is per module, publishing is per
module, and what happens on nodes is per node. What changes is that those are no longer three
**silos of a job**. They are steps of reconciling one artifact, and the third is not a step at all
any more ([ADR 0058](0058-delivery-ends-in-a-declaration.md)).
## What this costs
Named because each is a way this can go wrong, and a decision that lists none has not been
examined.
- **"Is this artifact current?" must be answerable without building it.** A commit recorded
against each artifact does it, and that record becomes load-bearing: wrong, and the mesh either
rebuilds forever or never rebuilds at all.
- **Rebuild storms.** One change to a shared library makes everything out of date at once. The
ordering that today's *levels* provide has to come from the module graph
([research 011](../01-RESEARCH/011-the-module-graph/00-overview.md)), which is designed and not
built.
- **The run identity people actually use is lost.** *Did my change go out?* is answerable today
by opening a pipeline. With convergence there is no run to open, and **something has to replace
that** — a query over the two comparisons above — or this will be worse to live with than what
it replaces, whatever its properties.
- **A loop that will not converge is harder to debug than a job that failed.** A failed job stops
and names its step. A reconciler that cannot reach its target retries forever, and without
something that notices *this has been trying for an hour*, the failure is silence — which is
the fault this record is removing, reintroduced in a new place. **This is the real risk and it
is not solved here.**
## Consequences
- **Detection stops being correctness and becomes latency.** The specific faults the as-is
records — a truncated commit list, a broken path match — become slow rather than silent.
- **The coordinator is not ported.** What replaces it is a comparison and a build, and the
existing implementation informs it only as a list of things that went wrong.
- **Two things must be cheap that are not yet designed**: reading what source exists, and reading
what has been built. Both are queries against providers the mesh will host, and both are on the
path of everything above.
- **Research 008 can close**, which it could not before this.
## References
- [ADR 0058](0058-delivery-ends-in-a-declaration.md) — the same move, one level down.
- [ADR 0014](0014-build-publish-and-deploy-are-three-silos.md) — the cardinality that survives.
- [ADR 0035](0035-a-picture-is-read-from-what-runs.md) — why a comparison beats a claim.
- [`00-as-is/04`](../03-DESIGN/00-as-is/04-delivery.md) — the pitfalls this is designed against.