From ee67f69ef478d0df7658357781a59ec47eca5af9 Mon Sep 17 00:00:00 2001 From: jochen Date: Mon, 21 Sep 2026 12:11:50 +0200 Subject: [PATCH 1/5] ADRs 0087 and 0088; issues 031, 035 and 054 resolved; designs 05, 07 and 18 amended A seeded file is created once (0087); the foundation filters before anything listens (0088); a machine becomes the last thing it was told (design 05). Each says how it is checked. --- .../0087-a-seeded-file-is-created-once.md | 63 +++++++++++++++++++ ...ndation-filters-before-anything-listens.md | 60 ++++++++++++++++++ 02-DECISIONS/README.md | 2 + 03-DESIGN/01-to-be/05-the-node-host.md | 17 ++++- 03-DESIGN/01-to-be/07-the-foundation.md | 11 ++++ 03-DESIGN/01-to-be/18-building-a-module.md | 7 ++- .../00-report.md | 6 +- .../00-report.md | 6 +- .../00-report.md | 6 +- 9 files changed, 166 insertions(+), 12 deletions(-) create mode 100644 02-DECISIONS/0087-a-seeded-file-is-created-once.md create mode 100644 02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md diff --git a/02-DECISIONS/0087-a-seeded-file-is-created-once.md b/02-DECISIONS/0087-a-seeded-file-is-created-once.md new file mode 100644 index 0000000..0d67892 --- /dev/null +++ b/02-DECISIONS/0087-a-seeded-file-is-created-once.md @@ -0,0 +1,63 @@ +--- +topic: what runs on it +status: accepted +date: 2026-09-21 +deciders: jochen +reconstructed: false +extends: 02-DECISIONS/0010-delivery.md +--- + +# 87. A seeded file is created once, and what grows in it is not the mesh's + +## Context + +A declaration is complete for what the host owns, and the host reconciles what is declared +([ADR 0010](0010-delivery.md)): a file with this content, held to it. That is the only thing a +manifest could say about a file, and it is the wrong thing for a file a module needs to **exist +before first start** and something else then legitimately writes into — an access list a +provisioner appends consumers to and the program persists back, a bootstrap configuration a +program rewrites. Every reconcile restored the seed behind the running program, erased what had +grown in it, and reported success +([issue 035](../04-ISSUES/035-reconciling-a-seed-file-wipes-what-grew-in-it/00-report.md)). + +A run-once step ([ADR 0052](0052-a-step-that-runs-once-before-a-container.md)) can write a seed +only if absent, and that closed the instance for a broker whose seed is a program's job. It left +the general case: a plain file the mesh writes and never overwrites. + +## Considered Options + +1. **Two owners never share a file: the provisioner owns it, and first-start ordering is solved + another way.** Rejected as the only answer — some software refuses to start without the file, + and a module that must ship a program merely to write an empty file has been made to write a + program to say one word. +2. **A create-once semantic on a file.** Adopted. + +## Decision + +A file resource may say `create-once`. The host writes it when it is absent and, when it is +present, leaves it entirely alone — content, mode and owner — and reports it as **kept**, not +corrected. What is in the file then is somebody else's work the mesh asked for. The mesh removes +nothing it did not create ([ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md)); it now +also does not overwrite what it created once and handed over. + +On the security question ADR 0010 asks of every new resource behaviour: this **narrows** what a +declaration can do to a machine. A create-once file gives a compromised control plane one fewer +way to change a machine repeatedly — it can seed, once, and never again. + +## Consequences + +A module says which of its files are seeds, and the difference is visible in the manifest rather +than in whether the file happened to be revisited. A later change to a seed's declared content +does not reach a machine that already has the file; that is the meaning of a seed, and a module +that needs the new content ships it as a run-once step that migrates the existing file. + +## How it is checked + +The host's apply tests: a seed is created, grown into by hand, reconciled, and the growth survives +with the outcome `kept`. The vault bed declares one on a real node, grows it, pushes again, and +reads it back. + +## References + +- [issue 035](../04-ISSUES/035-reconciling-a-seed-file-wipes-what-grew-in-it/00-report.md) +- [ADR 0010](0010-delivery.md), [ADR 0030](0030-data-outlives-the-mesh-that-declared-it.md), [ADR 0052](0052-a-step-that-runs-once-before-a-container.md) diff --git a/02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md b/02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md new file mode 100644 index 0000000..d46e5c7 --- /dev/null +++ b/02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md @@ -0,0 +1,60 @@ +--- +topic: the mesh +status: accepted +date: 2026-09-21 +deciders: jochen +reconstructed: false +extends: 02-DECISIONS/0078-the-store-and-broker-are-modules.md +--- + +# 88. The foundation filters before anything listens + +## Context + +Adopting the store and broker as ordinary modules +([ADR 0078](0078-the-store-and-broker-are-modules.md)) needs them reachable by consumers across +the mesh, so genesis raises them bound to every interface. The packet filter that decides who may +reach them is a module too, installed a dozen steps later. Between the two, a control-node facing +the network has its store and its bus open to anyone who can reach the machine, for the length +of the install ([issue 054](../04-ISSUES/054-the-adopted-store-and-broker-are-open-before-the-filter/00-report.md)). +The design's rule — what a port is reachable from is decided by the filter — is enforced by +nothing for that window. + +## Considered Options + +1. **Accept the window**: the machine is mid-bootstrap and the exposure matches what the + pre-adoption modules had in steady state. Rejected — the whole point of deriving the filter + was to stop accepting that. +2. **Bind narrowly at genesis and widen once the filter exists.** Rejected — a bind change is a + recreate of the mesh's store during install, and adoption in place needs the same spec. +3. **The foundation carries a filter of its own, applied before the store.** Adopted. + +## Decision + +The foundation bundle installs the packet filter and loads a base ruleset **before the store and +broker are raised**: drop by default; keep loopback, replies, ping and ssh; keep the mesh's own +ports a node must reach before it is on the private network — the bus it enrols over and the +registry it pulls from; and let the container runtime's own networks through the forward chain so +containers keep working. It is written into the **same table** the filter module later derives, so +that module replaces it wholesale the moment it can compute one from what the mesh knows, and +nothing of the base survives to contradict it. + +## Consequences + +From its first resource a machine being made into a mesh refuses what it will refuse when +finished; the window closes. What got harder: the base ruleset is static and names two ports the +mesh's derived one also names — a change to which ports the foundation needs is now made in two +places, and the bundle's own test says which. + +## How it is checked + +The installer's bundle test asserts the filter and its load precede the store and broker and that +the rules name ssh, the bus and the registry and not the store's or broker's client ports. The +genesis bed probes the machine from outside throughout the install: the store's port is never +reachable, while the bus becomes reachable. + +## References + +- [issue 054](../04-ISSUES/054-the-adopted-store-and-broker-are-open-before-the-filter/00-report.md), issue 047 +- [ADR 0078](0078-the-store-and-broker-are-modules.md) +- [`03-DESIGN/01-to-be/07-the-foundation.md`](../03-DESIGN/01-to-be/07-the-foundation.md) diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index 0a51db1..1bdb3ba 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -86,6 +86,7 @@ python3 00-META/checks/index.py fail if stale - **0003** — [An agent is a persistent employee, not an instance of a pool](0003-agents-are-persistent-employees.md) - **0077** — [The parts are named controller, foundation, node — not control plane, substrate, master](0077-the-controller-and-the-foundation.md) - **0083** — [One push leaves the mesh consistent](0083-one-push-leaves-the-mesh-consistent.md) +- **0088** — [The foundation filters before anything listens](0088-the-foundation-filters-before-anything-listens.md) ### Its tiers, from the bottom up @@ -138,6 +139,7 @@ python3 00-META/checks/index.py fail if stale - **0055** — [Model access is answered by a licence, or by a node that hosts the model](0055-model-access-is-answered-by-a-licence-or-a-node.md) - **0084** — [Which provider serves a consumer, when the mesh runs more than one](0084-which-provider-serves-a-consumer.md) - **0085** — [A secret is a provision, and the vault is the module that provides it](0085-a-secret-is-a-provision.md) +- **0087** — [A seeded file is created once, and what grows in it is not the mesh's](0087-a-seeded-file-is-created-once.md) ### How it is built diff --git a/03-DESIGN/01-to-be/05-the-node-host.md b/03-DESIGN/01-to-be/05-the-node-host.md index 42c7654..7c0aecf 100644 --- a/03-DESIGN/01-to-be/05-the-node-host.md +++ b/03-DESIGN/01-to-be/05-the-node-host.md @@ -2,7 +2,7 @@ layer: to-be status: in-progress code: [mesh-host] -updated: 2026-08-31 +updated: 2026-09-21 decisions: - 02-DECISIONS/0019-how-this-repository-works.md - 02-DECISIONS/0004-a-node-and-how-it-joins.md @@ -367,4 +367,17 @@ of the reason to manage a machine. silently incomplete. **A partial host does archives and refuses users**: an archive needs a filesystem and a way to -fetch; a user needs a user database the host is allowed to write. \ No newline at end of file +fetch; a user needs a user database the host is allowed to write. + +## A machine becomes the last thing it was told + +Every declaration is complete, so applying an old one is never wrong, only wasted — and under a +flurry of pushes a machine spent minutes becoming things the mesh had moved past +([issue 031](../../04-ISSUES/031-a-machine-becomes-each-thing-it-was-told-in-turn/00-report.md)). +So the host looks at what is already waiting before it applies anything: it holds a small window +of unacknowledged declarations, applies the newest, and sets the rest aside — each **reported as +superseded**, naming the one applied instead, because silence would read as a machine that +ignored an instruction and "applied" would be a lie. Applying stays one at a time; only seeing +is not. **Checked** by the link's unit tests on the drain, and by the genesis bed's settle wait, +which counts on a node catching up to the newest declaration rather than the oldest. + diff --git a/03-DESIGN/01-to-be/07-the-foundation.md b/03-DESIGN/01-to-be/07-the-foundation.md index 818c523..c786d6f 100644 --- a/03-DESIGN/01-to-be/07-the-foundation.md +++ b/03-DESIGN/01-to-be/07-the-foundation.md @@ -13,6 +13,7 @@ code: - mesh-lab test/integration/mesh.test.ts (a bare machine becomes a mesh) updated: 2026-09-21 decisions: + - 02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md - 02-DECISIONS/0004-a-node-and-how-it-joins.md - 02-DECISIONS/0078-the-store-and-broker-are-modules.md - 02-DECISIONS/0079-the-foundation-seats-are-named-after-their-servers.md @@ -157,6 +158,16 @@ a real machine with a network; the sealed case is the lab, and the lab places im Reproducibility comes from pinning the identity of a thing rather than carrying its bytes, which is what keeps the bundle small enough for a person to read and check. +## Filtered from its first resource + +The bundle installs the packet filter and loads a base ruleset before the store and broker come +up ([ADR 0088](../../02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md)): drop by +default, keep loopback, replies, ping, ssh, the bus and the registry, and the container runtime's +networks through the forward chain. It is written into the same table the filter module derives, +so that module replaces it wholesale once it can. **Checked** by the installer's bundle test +(order and rules) and by the genesis bed, which probes the machine from outside for the length of +the install: the store's port never answers, the bus's does. + ## Raising it The order, from [research 011](../../01-RESEARCH/011-the-module-graph/worked-provider.md): diff --git a/03-DESIGN/01-to-be/18-building-a-module.md b/03-DESIGN/01-to-be/18-building-a-module.md index f5f3121..2cb0ab9 100644 --- a/03-DESIGN/01-to-be/18-building-a-module.md +++ b/03-DESIGN/01-to-be/18-building-a-module.md @@ -5,8 +5,9 @@ code: - mesh-controller cmd/mesh-builder - mesh-controller internal/builder - mesh-catalog modules/builder -updated: 2026-09-15 +updated: 2026-09-21 decisions: + - 02-DECISIONS/0087-a-seeded-file-is-created-once.md - 02-DECISIONS/0040-what-a-module-is.md - 02-DECISIONS/0039-what-the-sdk-holds-and-refuses.md - 02-DECISIONS/0069-a-module-is-a-repository-and-a-path.md @@ -206,6 +207,10 @@ disagrees with it. | `service` | an **existing** unit put into a state | for software shipping its own unit | | `action` | a command to run | ❌ **refused** — [ADR 0005](../../02-DECISIONS/0005-the-node-host.md) | +A `file` may also say `create-once`: written when absent, left alone when present, reported as +kept — a seed a program then owns ([ADR 0087](../../02-DECISIONS/0087-a-seeded-file-is-created-once.md)). +Checked by the host's apply tests and by the vault bed, which grows into one and pushes again. + **The two at the bottom are the interesting rows.** `action` is refused outright: the link may not carry a command, so a module needing something done ships a program that reconciles — which is what a run-once `process` is. `service` installs no unit by design, which is right for software that diff --git a/04-ISSUES/031-a-machine-becomes-each-thing-it-was-told-in-turn/00-report.md b/04-ISSUES/031-a-machine-becomes-each-thing-it-was-told-in-turn/00-report.md index 97ee87c..f6c3e0c 100644 --- a/04-ISSUES/031-a-machine-becomes-each-thing-it-was-told-in-turn/00-report.md +++ b/04-ISSUES/031-a-machine-becomes-each-thing-it-was-told-in-turn/00-report.md @@ -1,9 +1,9 @@ --- -status: open +status: resolved opened: 2026-09-02 located-in: [mesh-host] -fixed-by: -amended-design: +fixed-by: mesh-host feat/migration-blockers (internal/link/run.go: the host prefetches a window of declarations, applies the newest and sets the rest aside, each reported as superseded); mesh-controller (the link records a superseded report as a word that the node is there) +amended-design: 03-DESIGN/01-to-be/05-the-node-host.md --- # 031 — A machine becomes each thing it was told, in turn diff --git a/04-ISSUES/035-reconciling-a-seed-file-wipes-what-grew-in-it/00-report.md b/04-ISSUES/035-reconciling-a-seed-file-wipes-what-grew-in-it/00-report.md index a7631ee..c216574 100644 --- a/04-ISSUES/035-reconciling-a-seed-file-wipes-what-grew-in-it/00-report.md +++ b/04-ISSUES/035-reconciling-a-seed-file-wipes-what-grew-in-it/00-report.md @@ -1,9 +1,9 @@ --- -status: open +status: resolved opened: 2026-09-02 located-in: [] -fixed-by: partly — mesh-host 24e9ae4 (a run-once step writes a seed only if absent, ADR 0052); mesh-catalog ec1e718 (mosquitto). A file resource itself has no create-once semantic, and no bed asserts a seed's grown content survives a re-apply -amended-design: +fixed-by: ADR 0087; mesh-host feat/migration-blockers (a file resource may say create-once; kept, never corrected); proven by the apply tests and the vault bed +amended-design: 03-DESIGN/01-to-be/18-building-a-module.md --- # 035 — Reconciling a seed file wipes what grew in it diff --git a/04-ISSUES/054-the-adopted-store-and-broker-are-open-before-the-filter/00-report.md b/04-ISSUES/054-the-adopted-store-and-broker-are-open-before-the-filter/00-report.md index d2bc8d1..1a8e0e0 100644 --- a/04-ISSUES/054-the-adopted-store-and-broker-are-open-before-the-filter/00-report.md +++ b/04-ISSUES/054-the-adopted-store-and-broker-are-open-before-the-filter/00-report.md @@ -1,9 +1,9 @@ --- -status: open +status: resolved opened: 2026-09-16 located-in: [] -fixed-by: -amended-design: +fixed-by: ADR 0088; mesh-host feat/migration-blockers (the foundation bundle installs nftables and loads a base ruleset before the store); proven by the bundle test and the genesis bed probing from outside during the install +amended-design: 03-DESIGN/01-to-be/07-the-foundation.md --- # 054 — The adopted store and broker are open before the packet filter exists From 5fa35362ac5531cec39f832aff643cd8da30b287 Mon Sep 17 00:00:00 2001 From: jochen Date: Mon, 21 Sep 2026 12:31:04 +0200 Subject: [PATCH 2/5] Issue 073: beds carry copies of catalogue manifests --- .../00-report.md | 38 +++++++++++++++++++ 1 file changed, 38 insertions(+) create mode 100644 04-ISSUES/073-beds-carry-copies-of-catalogue-manifests/00-report.md diff --git a/04-ISSUES/073-beds-carry-copies-of-catalogue-manifests/00-report.md b/04-ISSUES/073-beds-carry-copies-of-catalogue-manifests/00-report.md new file mode 100644 index 0000000..78d676b --- /dev/null +++ b/04-ISSUES/073-beds-carry-copies-of-catalogue-manifests/00-report.md @@ -0,0 +1,38 @@ +--- +status: located +opened: 2026-09-21 +located-in: [mesh-lab test/integration] +fixed-by: +amended-design: +--- + +# Beds carry copies of catalogue manifests, so a catalogue change is proven nowhere + +## Symptom, as observed + +The per-module beds (`assigned-catalogue-small`, `assigned-catalogue-apps`, `assigned-grafana`, +`assigned-model-usage`, `assigned-redis`, and others) build the manifests they install inline, as +JSON in the test, rather than reading `modules//module.json` from the catalogue checkout the +lab is pointed at. The copies were taken when each bed was written and have not moved since. + +Converting six modules to file-delivered secrets ([ADR 0086](../../02-DECISIONS/0086-a-secret-reaches-a-process-as-a-file.md)) +therefore changed nothing any of those beds run. Worse: the copies still deliver secrets through +an env-file without the declared exception, which the catalogue engine now refuses, so the beds +would fail against a current controller for a reason that has nothing to do with what they test. + +## Why it matters beyond this instance + +- It is [issue 072](../072-the-controllers-manifest-exists-twice/00-report.md) again, one copy per + bed: a manifest with two sources of truth, and the one the proof runs against is the stale one. +- "Proven in the lab" is the standard playbook 06 sets for a module. A bed that installs a copy + proves the copy. +- The genesis bed and the vault bed already read the catalogue (`--catalog` and a + `catalogueManifest()` helper that swaps the build artifact for the image the lab built); the + pattern exists and is one helper away for the rest. + +## What would close it + +Every bed that installs a catalogue module reads its manifest from the catalogue checkout, +rewriting only what the lab must (the built artifact's image, a lab-local address). The inline +copies go. Until then, each bed's copy is updated by hand alongside the catalogue, which is how +the six conversions were proven. From a438cbdac3d43e01fb1ffd77caaf8e55cd5b13c0 Mon Sep 17 00:00:00 2001 From: jochen Date: Mon, 21 Sep 2026 12:53:14 +0200 Subject: [PATCH 3/5] Issue 041: the declared exceptions, surveyed and partly converted --- .../00-report.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/04-ISSUES/041-a-sealed-credential-ends-up-in-the-process-environment/00-report.md b/04-ISSUES/041-a-sealed-credential-ends-up-in-the-process-environment/00-report.md index 58fa584..526bec7 100644 --- a/04-ISSUES/041-a-sealed-credential-ends-up-in-the-process-environment/00-report.md +++ b/04-ISSUES/041-a-sealed-credential-ends-up-in-the-process-environment/00-report.md @@ -73,3 +73,18 @@ refuses the undeclared shape. The controller, the instance this was opened on, r its credentials from files. The 35 other containers carry a declared reason; converting each where its software accepts a path remains per-module work and is not this issue's. +## The declared exceptions, surveyed 2026-09-21 + +Of the 35 containers marked when the rule landed, a survey of each image's own configuration +loader (read at the pinned digest where it was cached) found: 24 variables convertible now, 8 not +convertible (baserow, mssql, only-office, umami, mailu's initial admin), 9 read by the code of +Novox's own applications outside the mesh repositories (amqp-email-forwarder, de-spiegel, +invoicing, photos), 4 unverifiable (keycloak's admin, letta, mailu's other containers), and 2 +dead deliveries that nothing read. Converted and proven in a bed: amqp-ping, minio, mongodb, +mesh-catalog and model-usage; the two dead deliveries removed. The 25 that remain carry the +surveyed reason on the container. Gitea, nextcloud, mailu-admin, influxdb, step-ca and n8n are +convertible and wait for a bed that exercises their credential +([issue 073](../073-beds-carry-copies-of-catalogue-manifests/00-report.md)); icecast, keycloak's +database, searxng and nextcloud's object store convert through a generated configuration file +rather than a variable. + From 198e4db0a340b65dc9e38c53f023c3451600039a Mon Sep 17 00:00:00 2001 From: jochen Date: Mon, 21 Sep 2026 13:30:26 +0200 Subject: [PATCH 4/5] Design 07: the base ruleset closes the hub's port until the filter module derives one --- 03-DESIGN/01-to-be/07-the-foundation.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/03-DESIGN/01-to-be/07-the-foundation.md b/03-DESIGN/01-to-be/07-the-foundation.md index c786d6f..995c5bb 100644 --- a/03-DESIGN/01-to-be/07-the-foundation.md +++ b/03-DESIGN/01-to-be/07-the-foundation.md @@ -164,9 +164,13 @@ The bundle installs the packet filter and loads a base ruleset before the store up ([ADR 0088](../../02-DECISIONS/0088-the-foundation-filters-before-anything-listens.md)): drop by default, keep loopback, replies, ping, ssh, the bus and the registry, and the container runtime's networks through the forward chain. It is written into the same table the filter module derives, -so that module replaces it wholesale once it can. **Checked** by the installer's bundle test -(order and rules) and by the genesis bed, which probes the machine from outside for the length of -the install: the store's port never answers, the bus's does. +so that module replaces it wholesale once it can. Until it does, the machine admits nothing else — +not the overlay hub's port, which is derived from the hub's endpoint — so the filter module is +assigned to the control-node before a hub is placed there, as genesis does; a lab bed that raises +the foundation without genesis must do the same, and says so by waiting for the hub's port in the +ruleset the machine loaded. **Checked** by the installer's bundle test (order and rules) and by +the genesis bed, which probes the machine from outside for the length of the install: the store's +port never answers, the bus's does. ## Raising it From ea42b00d0a499e975f6013da79298f9ed5e8ff84 Mon Sep 17 00:00:00 2001 From: jochen Date: Mon, 21 Sep 2026 13:47:39 +0200 Subject: [PATCH 5/5] Issue 041: mongodb names its secrets' owner; the three conversions are proven --- .../00-report.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/04-ISSUES/041-a-sealed-credential-ends-up-in-the-process-environment/00-report.md b/04-ISSUES/041-a-sealed-credential-ends-up-in-the-process-environment/00-report.md index 526bec7..a845e43 100644 --- a/04-ISSUES/041-a-sealed-credential-ends-up-in-the-process-environment/00-report.md +++ b/04-ISSUES/041-a-sealed-credential-ends-up-in-the-process-environment/00-report.md @@ -81,7 +81,10 @@ convertible (baserow, mssql, only-office, umami, mailu's initial admin), 9 read Novox's own applications outside the mesh repositories (amqp-email-forwarder, de-spiegel, invoicing, photos), 4 unverifiable (keycloak's admin, letta, mailu's other containers), and 2 dead deliveries that nothing read. Converted and proven in a bed: amqp-ping, minio, mongodb, -mesh-catalog and model-usage; the two dead deliveries removed. The 25 that remain carry the +mesh-catalog and model-usage; the two dead deliveries removed. One image, mongodb's, drops to its +own user before it reads the file, so its manifest names that user as the owner of its secrets — the +first use of the owner field outside the controller, and the check any converted image needs: run it +with a file its user cannot read. The 25 that remain carry the surveyed reason on the container. Gitea, nextcloud, mailu-admin, influxdb, step-ca and n8n are convertible and wait for a bed that exercises their credential ([issue 073](../073-beds-carry-copies-of-catalogue-manifests/00-report.md)); icecast, keycloak's