Merge pull request 'ADRs 0087 and 0088; issues 031, 035 and 054 resolved; issue 041 surveyed; issue 073 opened' (#60) from feat/migration-blockers into main

2026-09-21 13:49:09 +02:00
parent a211aecdfa ea42b00d0a
commit 1db93ffc64
11 changed files with 226 additions and 12 deletions
@@ -0,0 +1,63 @@
+---
+topic: what runs on it
+status: accepted
+date: 2026-09-21
+deciders: jochen
+reconstructed: false
+extends: 02-DECISIONS/0010-delivery.md
+---
+
+# 87. A seeded file is created once, and what grows in it is not the mesh's
+
+## Context
+
+A declaration is complete for what the host owns, and the host reconciles what is declared
+([ADR 0010](0010-delivery.md)): a file with this content, held to it. That is the only thing a
+manifest could say about a file, and it is the wrong thing for a file a module needs to **exist
+before first start** and something else then legitimately writes into — an access list a
+provisioner appends consumers to and the program persists back, a bootstrap configuration a
+program rewrites. Every reconcile restored the seed behind the running program, erased what had
+grown in it, and reported success
+([issue 035](../04-ISSUES/035-reconciling-a-seed-file-wipes-what-grew-in-it/00-report.md)).
+
+A run-once step ([ADR 0052](0052-a-step-that-runs-once-before-a-container.md)) can write a seed
+only if absent, and that closed the instance for a broker whose seed is a program's job. It left
+the general case: a plain file the mesh writes and never overwrites.
+
+## Considered Options
+
+1. **Two owners never share a file: the provisioner owns it, and first-start ordering is solved
+   another way.** Rejected as the only answer — some software refuses to start without the file,
+   and a module that must ship a program merely to write an empty file has been made to write a
+   program to say one word.
+2. **A create-once semantic on a file.** Adopted.
+
+## Decision
+
+A file resource may say `create-once`. The host writes it when it is absent and, when it is
+present, leaves it entirely alone — content, mode and owner — and reports it as **kept**, not
+corrected. What is in the file then is somebody else's work the mesh asked for. The mesh removes
+nothing it did not create ([ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md)); it now
+also does not overwrite what it created once and handed over.
+
+On the security question ADR 0010 asks of every new resource behaviour: this **narrows** what a
+declaration can do to a machine. A create-once file gives a compromised control plane one fewer
+way to change a machine repeatedly — it can seed, once, and never again.
+
+## Consequences
+
+A module says which of its files are seeds, and the difference is visible in the manifest rather
+than in whether the file happened to be revisited. A later change to a seed's declared content
+does not reach a machine that already has the file; that is the meaning of a seed, and a module
+that needs the new content ships it as a run-once step that migrates the existing file.
+
+## How it is checked
+
+The host's apply tests: a seed is created, grown into by hand, reconciled, and the growth survives
+with the outcome `kept`. The vault bed declares one on a real node, grows it, pushes again, and
+reads it back.
+
+## References
+
+- [issue 035](../04-ISSUES/035-reconciling-a-seed-file-wipes-what-grew-in-it/00-report.md)
+- [ADR 0010](0010-delivery.md), [ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md), [ADR 0052](0052-a-step-that-runs-once-before-a-container.md)
@@ -0,0 +1,60 @@
+---
+topic: the mesh
+status: accepted
+date: 2026-09-21
+deciders: jochen
+reconstructed: false
+extends: 02-DECISIONS/0078-the-store-and-broker-are-modules.md
+---
+
+# 88. The foundation filters before anything listens
+
+## Context
+
+Adopting the store and broker as ordinary modules
+([ADR 0078](0078-the-store-and-broker-are-modules.md)) needs them reachable by consumers across
+the mesh, so genesis raises them bound to every interface. The packet filter that decides who may
+reach them is a module too, installed a dozen steps later. Between the two, a control-node facing
+the network has its store and its bus open to anyone who can reach the machine, for the length
+of the install ([issue 054](../04-ISSUES/054-the-adopted-store-and-broker-are-open-before-the-filter/00-report.md)).
+The design's rule — what a port is reachable from is decided by the filter — is enforced by
+nothing for that window.
+
+## Considered Options
+
+1. **Accept the window**: the machine is mid-bootstrap and the exposure matches what the
+   pre-adoption modules had in steady state. Rejected — the whole point of deriving the filter
+   was to stop accepting that.
+2. **Bind narrowly at genesis and widen once the filter exists.** Rejected — a bind change is a
+   recreate of the mesh's store during install, and adoption in place needs the same spec.
+3. **The foundation carries a filter of its own, applied before the store.** Adopted.
+
+## Decision
+
+The foundation bundle installs the packet filter and loads a base ruleset **before the store and
+broker are raised**: drop by default; keep loopback, replies, ping and ssh; keep the mesh's own
+ports a node must reach before it is on the private network — the bus it enrols over and the
+registry it pulls from; and let the container runtime's own networks through the forward chain so
+containers keep working. It is written into the **same table** the filter module later derives, so
+that module replaces it wholesale the moment it can compute one from what the mesh knows, and
+nothing of the base survives to contradict it.
+
+## Consequences
+
+From its first resource a machine being made into a mesh refuses what it will refuse when
+finished; the window closes. What got harder: the base ruleset is static and names two ports the
+mesh's derived one also names — a change to which ports the foundation needs is now made in two
+places, and the bundle's own test says which.
+
+## How it is checked
+
+The installer's bundle test asserts the filter and its load precede the store and broker and that
+the rules name ssh, the bus and the registry and not the store's or broker's client ports. The
+genesis bed probes the machine from outside throughout the install: the store's port is never
+reachable, while the bus becomes reachable.
+
+## References
+
+- [issue 054](../04-ISSUES/054-the-adopted-store-and-broker-are-open-before-the-filter/00-report.md), issue 047
+- [ADR 0078](0078-the-store-and-broker-are-modules.md)
+- [`03-DESIGN/01-to-be/07-the-foundation.md`](../03-DESIGN/01-to-be/07-the-foundation.md)
@@ -86,6 +86,7 @@ python3 00-META/checks/index.py            fail if stale
 - **0003** — [An agent is a persistent employee, not an instance of a pool](0003-agents-are-persistent-employees.md)
 - **0077** — [The parts are named controller, foundation, node — not control plane, substrate, master](0077-the-controller-and-the-foundation.md)
 - **0083** — [One push leaves the mesh consistent](0083-one-push-leaves-the-mesh-consistent.md)
+- **0088** — [The foundation filters before anything listens](0088-the-foundation-filters-before-anything-listens.md)

 ### Its tiers, from the bottom up

@@ -138,6 +139,7 @@ python3 00-META/checks/index.py            fail if stale
 - **0055** — [Model access is answered by a licence, or by a node that hosts the model](0055-model-access-is-answered-by-a-licence-or-a-node.md)
 - **0084** — [Which provider serves a consumer, when the mesh runs more than one](0084-which-provider-serves-a-consumer.md)
 - **0085** — [A secret is a provision, and the vault is the module that provides it](0085-a-secret-is-a-provision.md)
+- **0087** — [A seeded file is created once, and what grows in it is not the mesh's](0087-a-seeded-file-is-created-once.md)

 ### How it is built

@@ -2,7 +2,7 @@
 layer: to-be
 status: in-progress
 code: [mesh-host]
-updated: 2026-08-31
+updated: 2026-09-21
 decisions:
  - 02-DECISIONS/0019-how-this-repository-works.md
  - 02-DECISIONS/0004-a-node-and-how-it-joins.md
@@ -367,4 +367,17 @@ of the reason to manage a machine.
  silently incomplete.

 **A partial host does archives and refuses users**: an archive needs a filesystem and a way to
-fetch; a user needs a user database the host is allowed to write.
+fetch; a user needs a user database the host is allowed to write.
+
+## A machine becomes the last thing it was told
+
+Every declaration is complete, so applying an old one is never wrong, only wasted — and under a
+flurry of pushes a machine spent minutes becoming things the mesh had moved past
+([issue 031](../../04-ISSUES/031-a-machine-becomes-each-thing-it-was-told-in-turn/00-report.md)).
+So the host looks at what is already waiting before it applies anything: it holds a small window
+of unacknowledged declarations, applies the newest, and sets the rest aside — each **reported as
+superseded**, naming the one applied instead, because silence would read as a machine that
+ignored an instruction and "applied" would be a lie. Applying stays one at a time; only seeing
+is not. **Checked** by the link's unit tests on the drain, and by the genesis bed's settle wait,
+which counts on a node catching up to the newest declaration rather than the oldest.
+
@@ -13,6 +13,7 @@ code:
  - mesh-lab test/integration/mesh.test.ts (a bare machine becomes a mesh)
 updated: 2026-09-21
 decisions:
+  - 02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md
  - 02-DECISIONS/0004-a-node-and-how-it-joins.md
  - 02-DECISIONS/0078-the-store-and-broker-are-modules.md
  - 02-DECISIONS/0079-the-foundation-seats-are-named-after-their-servers.md
@@ -157,6 +158,20 @@ a real machine with a network; the sealed case is the lab, and the lab places im
 Reproducibility comes from pinning the identity of a thing rather than carrying its bytes, which
 is what keeps the bundle small enough for a person to read and check.

+## Filtered from its first resource
+
+The bundle installs the packet filter and loads a base ruleset before the store and broker come
+up ([ADR 0088](../../02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md)): drop by
+default, keep loopback, replies, ping, ssh, the bus and the registry, and the container runtime's
+networks through the forward chain. It is written into the same table the filter module derives,
+so that module replaces it wholesale once it can. Until it does, the machine admits nothing else —
+not the overlay hub's port, which is derived from the hub's endpoint — so the filter module is
+assigned to the control-node before a hub is placed there, as genesis does; a lab bed that raises
+the foundation without genesis must do the same, and says so by waiting for the hub's port in the
+ruleset the machine loaded. **Checked** by the installer's bundle test (order and rules) and by
+the genesis bed, which probes the machine from outside for the length of the install: the store's
+port never answers, the bus's does.
+
 ## Raising it

 The order, from [research 011](../../01-RESEARCH/011-the-module-graph/worked-provider.md):
@@ -5,8 +5,9 @@ code:
  - mesh-controller cmd/mesh-builder
  - mesh-controller internal/builder
  - mesh-catalog modules/builder
-updated: 2026-09-15
+updated: 2026-09-21
 decisions:
+  - 02-DECISIONS/0087-a-seeded-file-is-created-once.md
  - 02-DECISIONS/0040-what-a-module-is.md
  - 02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md
  - 02-DECISIONS/0069-a-module-is-a-repository-and-a-path.md
@@ -206,6 +207,10 @@ disagrees with it.
 | `service` | an **existing** unit put into a state | for software shipping its own unit |
 | `action` | a command to run | ❌ **refused** — [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) |

+A `file` may also say `create-once`: written when absent, left alone when present, reported as
+kept — a seed a program then owns ([ADR 0087](../../02-DECISIONS/0087-a-seeded-file-is-created-once.md)).
+Checked by the host's apply tests and by the vault bed, which grows into one and pushes again.
+
 **The two at the bottom are the interesting rows.** `action` is refused outright: the link may not
 carry a command, so a module needing something done ships a program that reconciles — which is what
 a run-once `process` is. `service` installs no unit by design, which is right for software that
@@ -1,9 +1,9 @@
 ---
-status: open
+status: resolved
 opened: 2026-09-02
 located-in: [mesh-host]
-fixed-by:
-amended-design:
+fixed-by: mesh-host feat/migration-blockers (internal/link/run.go: the host prefetches a window of declarations, applies the newest and sets the rest aside, each reported as superseded); mesh-controller (the link records a superseded report as a word that the node is there)
+amended-design: 03-DESIGN/01-to-be/05-the-node-host.md
 ---

 # 031 — A machine becomes each thing it was told, in turn
@@ -1,9 +1,9 @@
 ---
-status: open
+status: resolved
 opened: 2026-09-02
 located-in: []
-fixed-by: partly — mesh-host 24e9ae4 (a run-once step writes a seed only if absent, ADR 0052); mesh-catalog ec1e718 (mosquitto). A file resource itself has no create-once semantic, and no bed asserts a seed's grown content survives a re-apply
-amended-design:
+fixed-by: ADR 0087; mesh-host feat/migration-blockers (a file resource may say create-once; kept, never corrected); proven by the apply tests and the vault bed
+amended-design: 03-DESIGN/01-to-be/18-building-a-module.md
 ---

 # 035 — Reconciling a seed file wipes what grew in it
@@ -73,3 +73,21 @@ refuses the undeclared shape. The controller, the instance this was opened on, r
 its credentials from files. The 35 other containers carry a declared reason; converting each where
 its software accepts a path remains per-module work and is not this issue's.

+## The declared exceptions, surveyed 2026-09-21
+
+Of the 35 containers marked when the rule landed, a survey of each image's own configuration
+loader (read at the pinned digest where it was cached) found: 24 variables convertible now, 8 not
+convertible (baserow, mssql, only-office, umami, mailu's initial admin), 9 read by the code of
+Novox's own applications outside the mesh repositories (amqp-email-forwarder, de-spiegel,
+invoicing, photos), 4 unverifiable (keycloak's admin, letta, mailu's other containers), and 2
+dead deliveries that nothing read. Converted and proven in a bed: amqp-ping, minio, mongodb,
+mesh-catalog and model-usage; the two dead deliveries removed. One image, mongodb's, drops to its
+own user before it reads the file, so its manifest names that user as the owner of its secrets — the
+first use of the owner field outside the controller, and the check any converted image needs: run it
+with a file its user cannot read. The 25 that remain carry the
+surveyed reason on the container. Gitea, nextcloud, mailu-admin, influxdb, step-ca and n8n are
+convertible and wait for a bed that exercises their credential
+([issue 073](../073-beds-carry-copies-of-catalogue-manifests/00-report.md)); icecast, keycloak's
+database, searxng and nextcloud's object store convert through a generated configuration file
+rather than a variable.
+
@@ -1,9 +1,9 @@
 ---
-status: open
+status: resolved
 opened: 2026-09-16
 located-in: []
-fixed-by:
-amended-design:
+fixed-by: ADR 0088; mesh-host feat/migration-blockers (the foundation bundle installs nftables and loads a base ruleset before the store); proven by the bundle test and the genesis bed probing from outside during the install
+amended-design: 03-DESIGN/01-to-be/07-the-foundation.md
 ---

 # 054 — The adopted store and broker are open before the packet filter exists
@@ -0,0 +1,38 @@
+---
+status: located
+opened: 2026-09-21
+located-in: [mesh-lab test/integration]
+fixed-by:
+amended-design:
+---
+
+# Beds carry copies of catalogue manifests, so a catalogue change is proven nowhere
+
+## Symptom, as observed
+
+The per-module beds (`assigned-catalogue-small`, `assigned-catalogue-apps`, `assigned-grafana`,
+`assigned-model-usage`, `assigned-redis`, and others) build the manifests they install inline, as
+JSON in the test, rather than reading `modules/<name>/module.json` from the catalogue checkout the
+lab is pointed at. The copies were taken when each bed was written and have not moved since.
+
+Converting six modules to file-delivered secrets ([ADR 0086](../../02-DECISIONS/0086-a-secret-reaches-a-process-as-a-file.md))
+therefore changed nothing any of those beds run. Worse: the copies still deliver secrets through
+an env-file without the declared exception, which the catalogue engine now refuses, so the beds
+would fail against a current controller for a reason that has nothing to do with what they test.
+
+## Why it matters beyond this instance
+
+- It is [issue 072](../072-the-controllers-manifest-exists-twice/00-report.md) again, one copy per
+  bed: a manifest with two sources of truth, and the one the proof runs against is the stale one.
+- "Proven in the lab" is the standard playbook 06 sets for a module. A bed that installs a copy
+  proves the copy.
+- The genesis bed and the vault bed already read the catalogue (`--catalog` and a
+  `catalogueManifest()` helper that swaps the build artifact for the image the lab built); the
+  pattern exists and is one helper away for the rest.
+
+## What would close it
+
+Every bed that installs a catalogue module reads its manifest from the catalogue checkout,
+rewriting only what the lab must (the built artifact's image, a lab-local address). The inline
+copies go. Until then, each bed's copy is updated by hand alongside the catalogue, which is how
+the six conversions were proven.