From ab027691e17d9681be384e4b17958e6dff876868 Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 9 Oct 2026 00:28:55 +0200 Subject: [PATCH 1/4] Issue 339: name the merge order against mesh-host #61, record used-as-found as a behaviour change, and the review's follow-ups --- .../00-report.md | 6 +- .../01-diagnosis.md | 56 ++++++++++++++++++- 2 files changed, 56 insertions(+), 6 deletions(-) diff --git a/04-ISSUES/339-a-verb-can-place-a-directory-at-the-machines-own/00-report.md b/04-ISSUES/339-a-verb-can-place-a-directory-at-the-machines-own/00-report.md index f3ddb4c5..b196a271 100644 --- a/04-ISSUES/339-a-verb-can-place-a-directory-at-the-machines-own/00-report.md +++ b/04-ISSUES/339-a-verb-can-place-a-directory-at-the-machines-own/00-report.md @@ -75,9 +75,9 @@ pushing. that holds one. A `settings set` or `settings clear` line through `command` is refused. The layer is unchanged after every refusal. At the terminal the same placement is taken, but the machine's own trees (`/`, `/etc`, `/usr`, `/var/lib`, `/home`, the node-engine's state and the rest) are refused from - anywhere. A line break, carriage return or NUL in any setting is refused, keys included. PEM blocks - alone may hold lines, because a provider serves its authority's root as a setting. Each of these tests - fails on the commit before the fix. + anywhere. A line break, carriage return, any other line end or NUL in any setting is refused, keys + included. Only step-ca's `root` may hold lines, as certificates that decode and parse (since the review's + follow-up; at first any PEM block was taken). Each of these tests fails on the commit before the fix. - **In the node-engine's tests, through `Apply`.** These are refused, each as a failed resource with its reason: - a directory, access or mount source that is one of the machine's roots or holds one; diff --git a/04-ISSUES/339-a-verb-can-place-a-directory-at-the-machines-own/01-diagnosis.md b/04-ISSUES/339-a-verb-can-place-a-directory-at-the-machines-own/01-diagnosis.md index 66c3ae05..8fd8ea14 100644 --- a/04-ISSUES/339-a-verb-can-place-a-directory-at-the-machines-own/01-diagnosis.md +++ b/04-ISSUES/339-a-verb-can-place-a-directory-at-the-machines-own/01-diagnosis.md @@ -80,10 +80,60 @@ that the mesh did not record making at that path, is **used as found**: its owne changed, the outcome says so, and it is never removed. That is ADR 0182 applied beyond the home. The record of every directory on a running machine already names its path, so nothing applied today changes. -### The two pull requests are independent +### The order the pull requests merge in -The controller's tests do not compose with the node-engine, and the engine's guard needs nothing from the -controller. Either can merge first. Each closes the hole on its own side. +*Corrected 2026-10-09, after review.* The first version of this section said either pull request could merge +first. That is true of the two pull requests for this issue: the controller's tests do not compose with +the node-engine, and the engine's guard needs nothing from the controller. It is not true of the node-engine +pull request for ADR 0266's agent account (`mesh-host` #61), which carries a guard of its own over the same +appliers and the same files. The two conflict, so the order is: + +1. The node-engine's fix for this issue (`mesh-host` #63) merges first. +2. `mesh-host` #61 is then rebased onto it. It drops its own copy of the guard and adds its rules for below a + home (an archive, or a file written into, below the home of an account that never becomes root) to the + guard #63 placed, `internal/apply/placement_guard.go`. + +The controller's fix (`mesh-controller` #168) merged on 2026-10-09. The controller's pull request for ADR 0266 +(`mesh-controller` #164) is rebased onto it in the same way. + +### A behaviour change: a directory found on a machine is used as found + +Before this fix, the node-engine gave every declared directory the declared owner and mode, whether the mesh +had made it or not. Now a directory that was there before the mesh first applied it, at that path, keeps its +owner and mode, and is never removed. That is the safe default. It is also a new way for a module to fail +quietly: a module whose data directory was found with another owner cannot write it. Its container fails, +and nothing says why, except one line in the apply's outcome. + +No directory on a running machine is affected today, because each already has a record of the mesh +applying it at its path. The first to be affected would be: + +- a module newly assigned to a machine where its directory already exists; +- a placement moved onto a directory that already exists. + +Until the node-engine's follow-up (below) lands, the only trace is that one outcome line. + +### The review's follow-ups (2026-10-09) + +- **`mesh-controller`:** + - The one setting that may hold lines is narrowed to step-ca's `root`, holding certificates that + `encoding/pem` decodes (label CERTIFICATE, no headers) and `x509.ParseCertificate` parses. + - Vertical tab, form feed, NEL and the Unicode line and paragraph separators are refused alongside line + breaks, carriage returns and NUL. + - The spool, `/opt`, the container runtimes' data and any `.ssh` are added to the paths no placement or + access may name. +- **`mesh-host`, a directory used as found is said:** + - a WARNING line in every apply's log; + - its module's resource of kind `directory`, stated unhealthy on every look, with its owner, its mode and + what was declared. The controller raises that as the module's condition, naming the directory. + - `mesh-host hand-over `, run by a person at the machine, hands it to the mesh. The next apply gives + it the declared owner and mode. + + An unhealthy resource holds a walk's gate for its module, so a delivery of a module with a directory + used as found waits until someone hands it over. That is intended: such a module may not be able to + write its data. +- **`mesh-host`, the guard:** the same paths as the controller are added. The container runtime's data root + is read from its configuration. An account's `.ssh` is refused as an access or a mount, and as a + directory anywhere but below its own account's home. ### Step 3: what the live layers hold From 5fbd89c0621ff24a8ec728a06fd9de23eb721469 Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 9 Oct 2026 00:55:54 +0200 Subject: [PATCH 2/4] Issue 339: the gate reads a directory used as found as a wait for a person; the trust anchors are the terminal's --- .../01-diagnosis.md | 34 +++++++++++++++---- 1 file changed, 27 insertions(+), 7 deletions(-) diff --git a/04-ISSUES/339-a-verb-can-place-a-directory-at-the-machines-own/01-diagnosis.md b/04-ISSUES/339-a-verb-can-place-a-directory-at-the-machines-own/01-diagnosis.md index 8fd8ea14..eda8292b 100644 --- a/04-ISSUES/339-a-verb-can-place-a-directory-at-the-machines-own/01-diagnosis.md +++ b/04-ISSUES/339-a-verb-can-place-a-directory-at-the-machines-own/01-diagnosis.md @@ -119,20 +119,40 @@ Until the node-engine's follow-up (below) lands, the only trace is that one outc `encoding/pem` decodes (label CERTIFICATE, no headers) and `x509.ParseCertificate` parses. - Vertical tab, form feed, NEL and the Unicode line and paragraph separators are refused alongside line breaks, carriage returns and NUL. - - The spool, `/opt`, the container runtimes' data and any `.ssh` are added to the paths no placement or - access may name. + - The spool, `/opt`, the container runtimes' data (`/var/lib/docker`, `/var/lib/containers`, + `/var/lib/containerd`) and any `.ssh` are added to the paths no placement or access may name. + - **The mesh's trust anchors are the terminal's too**, refused through every verb route exactly as + `places` and `accesses` are: + - step-ca's `root`, `roots` and `path`: the root its consumers are handed, and where they fetch the roots + and the ACME directory; + - the identity provider's `issuer`: the issuer every OIDC consumer checks a login's token against. + + Before this, any caller of the settings verb could replace the internal authority's trust anchor or the + issuer. The catalogue was read on 2026-10-09 for every served fact and every setting a file asks for, and + these are the ones a consumer trusts. They are named in the controller (`TerminalKeys`), because no manifest + field says "this is trusted" yet. - **`mesh-host`, a directory used as found is said:** - a WARNING line in every apply's log; - its module's resource of kind `directory`, stated unhealthy on every look, with its owner, its mode and what was declared. The controller raises that as the module's condition, naming the directory. - `mesh-host hand-over `, run by a person at the machine, hands it to the mesh. The next apply gives - it the declared owner and mode. + it the declared owner and mode. A hand-over names the resource it was for, so it lapses if another + declaration later points another resource at the same path. - An unhealthy resource holds a walk's gate for its module, so a delivery of a module with a directory - used as found waits until someone hands it over. That is intended: such a module may not be able to - write its data. + **The gate reads it as a wait for a person**, the way it reads a wait for a new login + ([ADR 0254](../../02-DECISIONS/0254-a-wait-for-a-persons-new-login-is-not-a-gates-failure-and-each-module-of-a-send-keeps-its-own-pass.md)). + - The node-engine keeps when it first found the directory (`found_at`) across applies. + - A directory found before the send being judged is no fault of that build. The module's gate passes, and + its verdict carries the wait: "uses … as found and waits for a person to hand it over". An urgent fix to + that module still reaches every machine. + - A directory found by that very send was brought by it, so the gate holds the module and, at its bound, + puts it back. - **`mesh-host`, the guard:** the same paths as the controller are added. The container runtime's data root - is read from its configuration. An account's `.ssh` is refused as an access or a mount, and as a + is read from its configuration: `data-root` in dockerd's `daemon.json`; `--data-root` (or the older `-g` and + `--graph`) on its unit's `ExecStart` and in the unit's drop-ins; and `graphroot` in podman's + `storage.conf`. `/var/lib/containerd` is refused as well. Two stores are not read. A containerd configured + elsewhere is not a runtime this mesh runs. Podman's rootless stores sit below each account's home, where + only that account's directories are placed anyway. An account's `.ssh` is refused as an access or a mount, and as a directory anywhere but below its own account's home. ### Step 3: what the live layers hold From 5d054b722dede7039df67cfbcb4e8cd08545458b Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 9 Oct 2026 01:24:14 +0200 Subject: [PATCH 3/4] Issue 339: the terminal's settings are derived from what a module serves and which files it trusts; a found directory is its own condition the gate never reads as a fault --- .../01-diagnosis.md | 59 +++++++++++++------ 1 file changed, 41 insertions(+), 18 deletions(-) diff --git a/04-ISSUES/339-a-verb-can-place-a-directory-at-the-machines-own/01-diagnosis.md b/04-ISSUES/339-a-verb-can-place-a-directory-at-the-machines-own/01-diagnosis.md index eda8292b..76d8720f 100644 --- a/04-ISSUES/339-a-verb-can-place-a-directory-at-the-machines-own/01-diagnosis.md +++ b/04-ISSUES/339-a-verb-can-place-a-directory-at-the-machines-own/01-diagnosis.md @@ -121,16 +121,24 @@ Until the node-engine's follow-up (below) lands, the only trace is that one outc breaks, carriage returns and NUL. - The spool, `/opt`, the container runtimes' data (`/var/lib/docker`, `/var/lib/containers`, `/var/lib/containerd`) and any `.ssh` are added to the paths no placement or access may name. - - **The mesh's trust anchors are the terminal's too**, refused through every verb route exactly as - `places` and `accesses` are: - - step-ca's `root`, `roots` and `path`: the root its consumers are handed, and where they fetch the roots - and the ACME directory; - - the identity provider's `issuer`: the issuer every OIDC consumer checks a login's token against. + - **What a provider serves, and what a trusted file asks for, is the terminal's too.** A setting overrides any + key a provider serves (`Settle`), and what is served is what every consumer connects to and believes: + a database's or a registry's port, an object store's scheme, an identity provider's issuer and token path, + a model's name. Through the settings verb, any caller could point every consumer at a listener of its own + and collect what they present. The third review found this; the first fix had listed two modules by hand. + The controller now derives the keys a verb may not change from the module's manifest (`TerminalKeys`): + - `places` and `accesses`; + - every key under the module's `serves`, and every setting a served value asks for (`${setting:…}`); + - every setting a file marked `trusted` asks for. `trusted` is a new manifest field on a file: true for a + file that root or a consumer trusts (a logind drop-in, an env file that says which uid a container runs + as, a script run as root), false otherwise. The controller takes it out before the declaration, because + the node-engine parses strictly. - Before this, any caller of the settings verb could replace the internal authority's trust anchor or the - issuer. The catalogue was read on 2026-10-09 for every served fact and every setting a file asks for, and - these are the ones a consumer trusts. They are named in the controller (`TerminalKeys`), because no manifest - field says "this is trusted" yet. + `module check` names every file that asks for a setting without saying `trusted`, counts them in a line the + catalogue's merge check can keep, and refuses them from 2026-10-30. On 2026-10-09 that was 5 files in the + catalogue and 13 in the media catalogue. Their `trusted` is set in a catalogue change that can land only + once a controller that takes the field out of the declaration runs: an older one would pass it to the + node-engine, which refuses an unknown field. - **`mesh-host`, a directory used as found is said:** - a WARNING line in every apply's log; - its module's resource of kind `directory`, stated unhealthy on every look, with its owner, its mode and @@ -141,16 +149,31 @@ Until the node-engine's follow-up (below) lands, the only trace is that one outc **The gate reads it as a wait for a person**, the way it reads a wait for a new login ([ADR 0254](../../02-DECISIONS/0254-a-wait-for-a-persons-new-login-is-not-a-gates-failure-and-each-module-of-a-send-keeps-its-own-pass.md)). - - The node-engine keeps when it first found the directory (`found_at`) across applies. - - A directory found before the send being judged is no fault of that build. The module's gate passes, and - its verdict carries the wait: "uses … as found and waits for a person to hand it over". An urgent fix to - that module still reaches every machine. - - A directory found by that very send was brought by it, so the gate holds the module and, at its bound, - puts it back. + - **Its own condition kind**, `directory-used-as-found`, raised when every unhealthy resource of the module + on that machine is a directory used as found. It is the operator's to resolve, a warning however long it + stays open, never urgent, and cleared on the first statement that no longer says it. If anything else of + the module is unhealthy beside it, the module's ordinary unhealthy condition is raised instead, with the + directory among its resources. + - **The gate never reads that condition as a fault of the build**, in both places it reads conditions: + the module's own (`judgeHealth`) and the machine's (`aboutTheMachine`). This is the same exemption the + relogin kind has, and it holds even when the condition was raised after the send. + - **From the health statement**, the node-engine keeps when it first found the directory (`found_at`) + across applies. A directory found before the send being judged is no fault of that build: the module's + gate passes, and its verdict carries the wait ("uses … as found and waits for a person to hand it over"). + An urgent fix to that module still reaches every machine. A test holds this with the condition open and + raised after the send. + - **A directory found by that very send** was brought by it, so the gate holds the module and, at its + bound, puts it back. + - **Not done yet:** found before or after the send is judged by the node-engine's clock against the + controller's. A large clock skew between the two could misjudge it. Comparing by the declaration's + sequence instead is left for later. + - **`mesh-host`, the guard:** the same paths as the controller are added. The container runtime's data root - is read from its configuration: `data-root` in dockerd's `daemon.json`; `--data-root` (or the older `-g` and - `--graph`) on its unit's `ExecStart` and in the unit's drop-ins; and `graphroot` in podman's - `storage.conf`. `/var/lib/containerd` is refused as well. Two stores are not read. A containerd configured + is asked of the running runtimes first (`docker info`, `podman info`), and read from their configuration + beside that: `data-root` in dockerd's `daemon.json` or a `--config-file`; `--data-root` (or the older `-g` + and `--graph`) on the unit's `ExecStart`, continued over lines, quoted, or through `Environment=` and + `EnvironmentFile=`, in the unit and its drop-ins; and `graphroot` in podman's `storage.conf`, as a basic or + a literal string. The asking and the extra forms are in a further pull request stacked on the first. `/var/lib/containerd` is refused as well. Two stores are not read. A containerd configured elsewhere is not a runtime this mesh runs. Podman's rootless stores sit below each account's home, where only that account's directories are placed anyway. An account's `.ssh` is refused as an access or a mount, and as a directory anywhere but below its own account's home. From 1d9d53591230e8bd7fe4eb6536e90b4ed83da1b3 Mon Sep 17 00:00:00 2001 From: jochen Date: Fri, 9 Oct 2026 01:42:51 +0200 Subject: [PATCH 4/4] Issue 339: an unmarked file counts as trusted, and the rollback hazard of the trusted field and its fix --- .../01-diagnosis.md | 25 ++++++++++++------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/04-ISSUES/339-a-verb-can-place-a-directory-at-the-machines-own/01-diagnosis.md b/04-ISSUES/339-a-verb-can-place-a-directory-at-the-machines-own/01-diagnosis.md index 76d8720f..77a53f6c 100644 --- a/04-ISSUES/339-a-verb-can-place-a-directory-at-the-machines-own/01-diagnosis.md +++ b/04-ISSUES/339-a-verb-can-place-a-directory-at-the-machines-own/01-diagnosis.md @@ -129,16 +129,23 @@ Until the node-engine's follow-up (below) lands, the only trace is that one outc The controller now derives the keys a verb may not change from the module's manifest (`TerminalKeys`): - `places` and `accesses`; - every key under the module's `serves`, and every setting a served value asks for (`${setting:…}`); - - every setting a file marked `trusted` asks for. `trusted` is a new manifest field on a file: true for a - file that root or a consumer trusts (a logind drop-in, an env file that says which uid a container runs - as, a script run as root), false otherwise. The controller takes it out before the declaration, because - the node-engine parses strictly. + - every setting a file asks for, **unless the file says `"trusted": false`**. An unmarked file counts as + trusted (fail closed, the fourth review): a file root or a consumer trusts — a logind drop-in, an env file + that says which uid a container runs as, a script run as root — must never be changed through a verb, + and saying nothing must not be the unsafe answer. `"trusted": false` is the opt-out, for a file nothing + but its own person reads. The controller takes the field out before the declaration. - `module check` names every file that asks for a setting without saying `trusted`, counts them in a line the - catalogue's merge check can keep, and refuses them from 2026-10-30. On 2026-10-09 that was 5 files in the - catalogue and 13 in the media catalogue. Their `trusted` is set in a catalogue change that can land only - once a controller that takes the field out of the declaration runs: an older one would pass it to the - node-engine, which refuses an unknown field. + `module check` lists and counts the unmarked files, so an author can opt out a harmless one. It never + refuses them, because counting them as trusted is already the safe reading; the refusal date the third + review's fix had is gone. On 2026-10-09 the catalogue had 9 unmarked files and the media catalogue 13. + Two are opted out in a catalogue change: the notifier's look and a person's folder names. The other 20 + stay trusted. + + **A rollback hazard, and its fix.** The node-engine parses a declaration strictly, so a `trusted` it did not + know would cost a node its whole declaration. The controller takes it out, but an older controller, or + one rolled back to, would pass it on. So the node-engine now accepts `trusted` on a file and does nothing + with it (in `mesh-host` #65), and the catalogue change that sets it waits until the controller with this + fix is live. - **`mesh-host`, a directory used as found is said:** - a WARNING line in every apply's log; - its module's resource of kind `directory`, stated unhealthy on every look, with its owner, its mode and