jschoubben
c0b35652d0
papa-hq reads 01 research -> 03 decision -> 02 design. The order is a scar, not a choice: 02-DESIGN existed from its initial commit, and when adr/ was finally promoted on 2026-07-13 it took the next free number rather than its place in the sequence. By then design was too settled to renumber. hal-hq was three commits old, so it is not. adr/ becomes 02-DECISIONS and 02-DESIGN becomes 03-DESIGN, and following the folder numbers now walks the process in the order it happens: research produces a decision, the decision authorises a design. 00-GENESIS becomes 00-META, matching papa's rename from the same restructure. Every path reference rewritten across documents, frontmatter, playbooks and skills. All links resolve; all 58 frontmatter blocks parse and their path fields still point at files that exist.
3.5 KiB
status, date, deciders, reconstructed
| status | date | deciders | reconstructed |
|---|---|---|---|
| accepted | 2026-08-04 | jochen | true |
14. Build, publish and deploy are three silos with different cardinality
Reconstructed after the fact from the evidence cited below.
Context
Delivery had been treated as one pipeline that a module passes through. It is not: its stages run a different number of times.
- Compiling happens once per module feature, on the build node.
- Packaging and uploading happens once per module feature, on the build node.
- Installing, configuring, starting and verifying happens once per module feature per node.
Conflating them is what made earlier versions slow and hard to reason about. Work that should happen once was being repeated per node, and the fan-out point was implicit rather than a boundary anything could observe.
The split had been declared before it was real. Packaging still happened inside the build, which meant the boundary existed in the documentation and not in the code.
Considered options
- One pipeline, stages that know their own cardinality. Rejected — it is what existed. Cardinality is then a property of each stage's implementation, and nothing can reason about the pipeline as a whole.
- Two silos: build-and-publish, then deploy. Rejected. It leaves packaging inside build, so build must know every module, every feature, and how each composes its artifact — exactly the coupling the split exists to remove. A failed upload then retries by re-sending a stale package instead of re-packaging.
- Three silos, with an explicit handover between each. Chosen.
Decision
Delivery is three silos, and the boundaries are real:
| Silo | Runs | Where |
|---|---|---|
| build | once per module feature | the build node |
| publish | once per module feature | the build node |
| deploy | once per module feature per node | every assigned node |
Commands and events are addressed per feature, not per module.
Build compiles and hands over a staged tree — not a package. Publish applies the module's packaging rules, packages that tree, and uploads it. Publishing to a package registry is publishing, so a module whose artifact is a package publishes in the publish silo, not the build one.
Modules are resolved into dependency levels, and a level completes before the next begins, so a module always builds against its dependencies' freshly published versions.
Consequences
- Work that should happen once happens once. The fan-out point is explicit and observable.
- A failed upload retries by re-packaging, because packaging belongs to the stage that uploads.
- The handover is a staged tree in a known location rather than the build's working directory, which is reference-counted and cannot be assumed to still exist when a later stage runs.
- The build node is now the only node that has already passed through two silos when the fan-out happens. Anything tracking a node's stage must account for both pre-fan-out stages; code that knew only about the first parked the build node forever while every other node deployed cleanly.
- A recovery mechanism that knows a subset of the stages it guards is worse than none — it reports success over a stall it cannot see.
References
publish owns packaging — the silos were not actually split(#677), 2026-08-04.- Knowledge base:
pipeline/three-silos— including the note that the older architecture documents claimed otherwise and were stale until 2026-08-06. - The build-node stage-tracking failure was observed on pipeline #5557.