2026-10-04 01:30:48 +00:00
3 changed files with 89 additions and 0 deletions
@@ -107,6 +107,16 @@ unreferenced.
 - The store is briefly unavailable each night, for as long as collection takes. Everything that
  pulls from it retries; nothing in the mesh treats a momentary store as a failure
  ([ADR 0185](0185-a-control-plane-behind-its-seats-row-serves-what-it-can.md)).
 - **An apply arriving during the window reopens it**, because the host's rule for a container it
  finds stopped is to replace it, and `while-stopped` is the first thing that makes a stopped
  container intentional. Found by reading this before it merged, recorded as
  [issue 224](../04-ISSUES/224-an-apply-reopens-a-maintenance-window-by-recreating-what-it-held-still/00-report.md)
  rather than fixed here: the two candidate fixes — the window takes the apply lock, or the apply
  learns which containers are held — are each a decision with its own cost, and neither belongs
  inside this record. Nothing is worse than it was; the store has never collected at all.
 - **The sweep is bounded**: at most two hundred artifacts and sixty seconds per build, stopping at
  the first refusal, because it runs inside somebody's build. What is left over is offered again
  next time. The store stops growing from the first sweep; it does not empty in one.
 ## How this is checked
@@ -94,6 +94,15 @@ a literal in a consumer's definition is not merely redundant — it is the one t
 disagree with what the provider will actually create. The three object-store consumers lose their
 hand-written bucket names in this change.
 **5. A consumer that keeps several holders of one provision may not be served a derived value.**
 Each holder gets its own login, `…_<local>` ([ADR 0094](0094-a-module-may-hold-several-secrets-from-one-provider.md)),
 and a provider derives from the login — so it would make one resource per holder, while the
 consumer's side has one binding and one `${bound:<provision>:<key>}`, both derived from the
 un-suffixed identity. That is this record's own failure one case to the side, and just as quiet:
 the consumer would authenticate and be refused on every object. Refused at resolution, naming
 both ends. Lifting it means giving the consumer's side a local dimension, which is a decision and
 not an omission.
 ## Consequences
 - One more thing a definition may say, and one less thing a module may be wrong about. The
@@ -105,6 +114,10 @@ hand-written bucket names in this change.
 - A provider that already serves consumers keeps serving them: the derived value equals what the
  code derived, so no bucket, database or login changes name. This is a change of **who says it**,
  not of **what is said**.
 - A refusal here fails **that machine's push**, naming the definition, and nothing else. That is
  deliberate and is the opposite of a module quietly left out: a definition that transcribes
  somebody else's rule is wrong everywhere, not just here, and the loud failure is in front of
  whoever can fix it.
 - The mesh now holds a rule in another system's alphabet — one rule, `dns`, stated once. A second
  alphabet is a decision, not an addition: the cost of each is that the mesh must be right about
  somebody else's naming, and that cost is only worth paying where the mesh already mints the name.
@@ -118,6 +131,8 @@ hand-written bucket names in this change.
  consumer — one test asserting the three agree, because agreeing is the whole point.
 - Two consumers of one provider on one machine get two different derived values, and neither gets
  the other's.
 - A consumer with several holders of a deriving provider is refused, with both ends named — the
  test asserts the refusal, not merely that something failed.
 - A catalogue-wide test refuses a consumer definition that writes a literal where its provider
  derives: the provider's `serves` names the key, so the catalogue can say which definitions
  transcribe one.
@@ -0,0 +1,64 @@
 ---
 status: open
 opened: 2026-10-04
 located-in: [mesh-host internal/apply/apply.go, mesh-host internal/apply/schedule.go]
 fixed-by:
 amended-design:
 ---
 # 224 — An apply arriving during a maintenance window reopens it, by recreating the container the window is holding still
 ## What was observed
 Reviewing [ADR 0189](../../02-DECISIONS/0189-the-store-keeps-what-the-records-name.md)'s
 `while-stopped` before merging it, 2026-10-04. Found by reading, not by running.
 A scheduled step may hold its module's containers still while it runs. The host stops them, runs
 the step, starts them again. Nothing tells the **apply** that a window is open, and the apply's
 rule for a container it finds stopped is to replace it:
 ```
 case existed && (before.Spec == want || legacy) && before.Running && len(reasons) == 0:
        out.Action = "unchanged"
 case existed:
        rm -f
 ```
 `before.Running` is false for a container a window is holding, so the second branch takes it:
 the container is removed and recreated, **running**, in the middle of the step that required it
 to be still.
 ## Why it matters
 For the store, which is what the field was built for, the chain is: a push lands at 03:30 → the
 apply recreates the registry → the registry accepts an upload from a build running at the same
 time → `garbage-collect`, already past its mark phase, sweeps the blob that upload just wrote.
 The image is then in the store with a layer missing, and the build that made it reported success.
 Two things have to coincide, so it is not likely. It is also not rare enough to leave unsaid: the
 mesh pushes on every merge, at any hour, and a collection over a store this size is minutes rather
 than seconds.
 **The general shape is the one that matters.** `while-stopped` is the first thing in the mesh that
 makes a container's stopped state *intentional*. Everything else in the host reads "stopped" as
 "broken, fix it", which is right everywhere else and wrong here. Any future use of the field
 inherits this.
 ## What this is not
 Not a regression. The store has never collected anything, so nothing is worse than it was; this
 is a hole in something new rather than something that broke.
 ## Open questions
 - **Should a window take the apply lock?** The daemon already serialises applies with `applying`
  and, across processes, with `store.Lock`. A window that held it would make the race impossible.
  The cost is that a push arriving mid-window waits for minutes, and a push that waits is what
  [issue 185](../185-a-refused-membership-publish-stops-the-controller/00-report.md)'s
  family of outages looked like from outside.
 - **Or should the apply learn that a container is held?** Narrower: the scheduler says which
  containers a window currently holds, and `applyContainer` reports those unchanged instead of
  recreating them. Nothing blocks, and the apply tells the truth for the minutes it matters —
  at the cost of a second source for "is this container meant to be running".
 - Either way: should the *report* say a window is open, so a machine that looks half-stopped at
  03:31 reads as working rather than broken?