Merge pull request 'ADR 0147: a module anchors the mesh's authority, and issue 146: the foundation cannot be raised' (#184) from decision/0147-a-module-anchors-the-meshs-authority into main

Raising a first node hits them in order: the bundle's bus image named for a
registry that is gone; the certificate made by openssl in an image that has
none; enrolment dialling TLS at a bus that speaks first, then refusing its own
token for the empty bus. Each was right until the bus changed and nothing has
raised a foundation since.

The fourth is not a patch: the installer carries the bus's first user list and
the controller composes the rest through the bus being a module, and at genesis
there is no module — so the first node cannot be let onto the bus it just
raised.

02-DECISIONS/0147-a-module-anchors-the-meshs-authority.md

@@ -0,0 +1,139 @@
+---
+topic: what runs on it
+status: accepted
+date: 2026-09-29
+deciders: jochen
+reconstructed: false
+extends: 02-DECISIONS/0098-a-fact-a-provider-makes-at-first-start-is-fetched-from-it.md
+---
+
+# 147. A module anchors the mesh's authority on a machine, and takes it away again
+
+## Context
+
+The mesh runs its own certificate authority and every internal name is served with a certificate
+from it. No machine trusts it. On an enrolled, adopted workstation — on the private network,
+resolving through the mesh's resolver — every internal HTTPS name fails verification with
+*unable to get local issuer certificate*
+([issue 129](../04-ISSUES/129-nothing-makes-a-machine-trust-the-meshs-authority/00-report.md)).
+The certificates are genuine; nothing on the machine has ever been told what issued them.
+
+The authority's only consumer today is a proxy, which fetches the root into a directory of its own
+and hands it to one program ([ADR 0098](0098-a-fact-a-provider-makes-at-first-start-is-fetched-from-it.md)).
+That is enough for the proxy and for nothing else: a browser, `git` over HTTPS, `curl`, a package
+manager and every module that calls another module by an internal name read the machine's trust
+store, which holds the predecessor's authority and a developer tool's local root, and nothing of
+the mesh's.
+
+The predecessor wrote its root into every machine it set up. Removing it was deliberate — an
+honest failure beats a name that verifies for the wrong reason — and it leaves the mesh with no
+answer at all until this one lands. It is also what keeps the predecessor alive on the machines
+that still speak TLS to a mesh name.
+
+**What makes this a decision rather than a patch** is where the knowledge goes. Two mechanisms in
+the mesh already write things onto a machine because it is on the private network: `/etc/hosts`
+and the registry's plaintext trust ([ADR 0082](0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md)).
+Following that precedent, the controller would inject an anchor into every such machine's
+declaration, and issue 129 proposed exactly that. It would work. It would also put *where this
+operating system keeps trust anchors* and *which command refreshes its bundles* into the control
+plane, for a fact the control plane does not have (the root does not exist until the authority has
+run) and a machine that may have no reason to verify a mesh name at all.
+
+## Considered Options
+
+1. **The controller injects the anchor into every machine on the private network**, the
+   `/etc/hosts` and insecure-registry shape. Rejected: being on the network is what makes the
+   registry reachable, and that is why network presence is the right trigger *there* — the trust
+   and the reachability are the same fact. Trusting an authority is not the same fact as being
+   able to reach it, and the anchor's path and the bundle refresh are a property of the machine's
+   operating system, which is the host's half of the mesh, not the controller's.
+2. **A new host primitive — a `trust-anchor` resource type.** Rejected for now, not on principle.
+   The host's vocabulary should grow when a shape cannot be said with what exists, and this one
+   can: a file and a service already express it, as the packet filter proves
+   ([ADR 0140](0140-the-filter-constrains-what-arrives-from-outside.md), whose module writes a
+   unit file and a service and nothing else). The primitive becomes right the moment a second
+   operating system is in play, because the anchor directory and the refresh command are exactly
+   the difference `internal/system` exists to hold. Until then it would be a vocabulary word with
+   one speaker.
+3. **A module that requires the authority, fetches its root, installs it as an anchor and
+   refreshes the machine's bundles — and removes both when it is no longer assigned.** Adopted.
+
+## Decision
+
+**A machine trusts the mesh's authority because a module put its root there, and stops trusting it
+when that module is taken away.**
+
+1. **The module requires `internal-acme-ca`** and reads the provider's bound address and the path
+   it serves its root at. It requires nothing else and provides nothing: it is a consumer of the
+   authority like any other.
+2. **It fetches the root over the mesh's own network, without prior trust**, because there is no
+   prior trust to have — this is the module that establishes it — and the network is what
+   authenticates the fetch ([ADR 0098](0098-a-fact-a-provider-makes-at-first-start-is-fetched-from-it.md),
+   the same reasoning that lets the proxy fetch it). What it accepts is checked: a body that is
+   not a certificate fails, and the failure is the module's, not a later handshake's.
+3. **It installs the root where this machine's TLS clients look, and refreshes the extracted
+   bundles** — the command that does the refresh is an ordinary part of the unit that places the
+   anchor, not a new thing the mesh can be asked to do.
+4. **Removal is symmetric and is the same unit's business.** Undeclared, the host stops the unit;
+   stopping it removes the anchor and refreshes the bundles again. A machine that leaves the mesh
+   stops trusting the mesh, without anybody remembering to go and look.
+5. **It is an ordinary assignment.** No machine is given it automatically. A machine that verifies
+   a mesh name is assigned it, and a machine that does not is not — which is the same statement
+   the mesh already makes about every other module, and is why this is not the controller's
+   business.
+
+**One operating system, said out loud.** The anchor directory and the refresh command in the
+module today are Arch's. On a machine that is not Arch the unit fails, visibly, rather than
+writing a file nothing reads. That is the accurate failure, and it is the signal that option 2
+above has become right.
+
+## How this is checked
+
+- **The verification that could not succeed before.** On a machine holding the module, a plain
+  client fetches an internal HTTPS name with no `-k` and no bundle argument and verifies. On a
+  machine without it, the same fetch fails with *unable to get local issuer certificate*. Both
+  halves, because only the pair distinguishes "the anchor works" from "something else already
+  trusted it".
+- **The removal half, in the same bed:** unassign the module, refetch, and the failure returns.
+  Checking only the arrival is how a trust store fills up with authorities nobody can account for.
+- **What is deliberately not checked here:** that the authority issues, that a name resolves, that
+  the proxy serves. Those have their own beds, and this module's bed passing for those reasons is
+  the failure mode this record is most exposed to — which is why the negative half is not optional.
+
+**What this bed is dialled at, and why it is the authority itself.** The authority serves its own
+API with a certificate it issued, so the handshake under test needs nothing else in the mesh to be
+right. A trust bed that reached for a routed name through the proxy would be passing or failing for
+the proxy's reasons and the resolver's.
+
+**Written, and not yet run** *(2026-09-29)*. The bed is `trust-anchor` in the lab, and it cannot
+execute: raising a foundation fails before any module is reached, in both bundles that exist
+([issue 146](../04-ISSUES/146-the-foundation-cannot-be-raised-on-the-bus-the-mesh-runs-on/00-report.md)).
+So what stands behind this record today is the rendering — the script the machine would run names
+the authority it was bound to, checked in the control plane's own test suite — and **not** a machine
+that verified anything. That is a weaker thing than the paragraph above describes, and it stays
+written this way until the bed runs.
+
+## Consequences
+
+The predecessor's authority can be retired from a machine once this module is assigned to it,
+which is the first time that has been true. `git` over HTTPS to the mesh's forge starts working,
+so the ssh-only clone URL stops being a rule. A module on any machine can call another module's
+internal name and verify it.
+
+What got harder: one more module to assign to a machine that needs it, and the machine's trust
+store now changes when an assignment changes — which is the point, and is also a thing an operator
+can be surprised by. The fetch without prior trust is the same exposure ADR 0098 accepted, now on
+every machine that holds the module rather than only where a proxy runs: anything that can stand
+in the middle of the mesh's own network at the moment of the fetch can be believed. The mesh
+already treats that network as the thing it authenticates.
+
+## References
+
+- [issue 129](../04-ISSUES/129-nothing-makes-a-machine-trust-the-meshs-authority/00-report.md) —
+  the symptom and the evidence.
+- [ADR 0098](0098-a-fact-a-provider-makes-at-first-start-is-fetched-from-it.md) — a fact made at
+  first start is fetched from its provider; this extends it from one program to the machine.
+- [ADR 0082](0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md) — the precedent
+  this deliberately does not follow, and why it is right where it is.
+- [ADR 0005](0005-the-node-host.md) — the host is where one operating system's difference lives.
+- [`03-DESIGN/01-to-be/08-connectivity.md`](../03-DESIGN/01-to-be/08-connectivity.md).

02-DECISIONS/README.md

@@ -225,7 +225,9 @@ python3 00-META/checks/index.py            fail if stale
 - **0141** — [The host delivers its own successor, and versions live side by side](0141-the-host-delivers-its-own-successor.md)
 - **0143** — [A consumer verifies the grant it is given](0143-a-consumer-verifies-the-grant-it-is-given.md)  *(superseded)*
 - **0144** — [Anything on a machine may call anything on it, and that is the whole of "local"](0144-anything-on-a-machine-may-call-anything-on-it.md)
-- **0145** — [A module checks what the mesh claims is reachable, and it checks itself](0145-a-module-checks-what-the-mesh-claims-is-reachable.md)
+- **0145** — [A module checks what the mesh claims is reachable, and it checks itself](0145-a-module-checks-what-the-mesh-claims-is-reachable.md)  *(superseded)*
+- **0146** — [Connectivity is checked by name, per hosting form, with a valid certificate](0146-connectivity-is-checked-by-name-per-hosting-form.md)
+- **0147** — [A module anchors the mesh's authority on a machine, and takes it away again](0147-a-module-anchors-the-meshs-authority.md)
 
 ### How it is built
 

03-DESIGN/01-to-be/08-connectivity.md

@@ -7,8 +7,9 @@ code:
   - mesh-controller internal/identity/authority.go
   - mesh-host internal/identity/serving.go
   - mesh-host internal/apply (the service that reflects a rule set)
-updated: 2026-09-28
+updated: 2026-09-29
 decisions:
+  - 02-DECISIONS/0147-a-module-anchors-the-meshs-authority.md
   - 02-DECISIONS/0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md
   - 02-DECISIONS/0140-the-filter-constrains-what-arrives-from-outside.md
   - 02-DECISIONS/0104-a-provision-may-be-answered-by-an-adapter-to-the-predecessor.md
@@ -710,6 +711,22 @@ step, so when the authority moves the root is fetched again and the proxy is rec
 *How it is checked:* the route-forwarding bed installs the authority, the proxy and a consumer
 from the catalogue and asserts the routed name is served.
 
+**And a machine trusts that authority because a module put its root in its trust store**
+([ADR 0147](../../02-DECISIONS/0147-a-module-anchors-the-meshs-authority.md)). The proxy's fetch
+answers for the proxy and for nothing else: a browser, `git` over HTTPS, a package manager and
+every module calling another by an internal name read the machine's own trust store, and the mesh
+had never written anything there. A module requiring the authority does the whole of it — fetch
+the root over the mesh network, place it where this machine's TLS clients look, refresh the
+extracted bundles — and stopping it, which is what being unassigned does, takes the anchor away
+and refreshes them again. Not the controller's business, because being on the private network is
+what makes the authority *reachable* and is not the same fact as having a reason to *verify* a
+mesh name; and because where anchors live and which command refreshes them is one operating
+system's difference, which is the host's half of the mesh
+([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)).
+*How it is checked:* on a machine holding the module a plain client verifies an internal HTTPS
+name with no bundle argument, and on one without it the same fetch fails to find an issuer — both
+halves, because only the pair tells the anchor apart from something that already trusted it.
+
 ### What was built
 
 *2026-08-31.*

04-ISSUES/129-nothing-makes-a-machine-trust-the-meshs-authority/00-report.md

@@ -1,7 +1,7 @@
 ---
-status: open
+status: located
 opened: 2026-09-26
-located-in: [mesh-controller, mesh-catalog step-ca]
+located-in: [mesh-catalog ca-trust]
 ---
 
 # 129 — nothing makes a machine trust the mesh's own certificate authority

04-ISSUES/129-nothing-makes-a-machine-trust-the-meshs-authority/01-diagnosis.md

@@ -0,0 +1,32 @@
+# Diagnosis
+
+*2026-09-29.*
+
+## What was ruled out
+
+**That something already carries the root and it is only misplaced.** It does not. The authority
+serves its root at a path beside its ACME directory, and the one thing that fetches it — the route
+proxy — puts it in a directory of its own and hands it to one program. Nothing has ever written
+into a machine's trust store. Measured on three converged machines: the anchors present are the
+predecessor's authority and a developer tool's local root, and on the machines where the
+predecessor's was deliberately removed, every internal name fails verification.
+
+**That the private network could carry it, the way it carries the registry's trust.** That is what
+the report proposed, and it was rejected on consideration rather than on difficulty
+([ADR 0147](../../02-DECISIONS/0147-a-module-anchors-the-meshs-authority.md), option 1): being on
+the network is what makes the registry *reachable* and is therefore the right trigger there, while
+trusting an authority is a separate fact from being able to reach it. The anchor's directory and
+the command that refreshes the extracted bundles are also one operating system's difference, which
+is the host's half of the mesh and not the controller's.
+
+**That it needs a new host resource type.** It does not, today. A file and a service say the whole
+of it, which the packet filter already proves. The primitive becomes the right answer when a second
+operating system is in play, and not before.
+
+## Where it belongs
+
+A module in the catalogue: it requires `internal-acme-ca`, fetches the root over the mesh's own
+network, installs it as a trust anchor, refreshes the machine's bundles, and — because being
+unassigned stops its unit, and stopping the unit is what undoes it — takes both away again.
+
+The owner is therefore `mesh-catalog`, module `ca-trust`, and nothing in the control plane.

04-ISSUES/146-the-foundation-cannot-be-raised-on-the-bus-the-mesh-runs-on/00-report.md

@@ -0,0 +1,73 @@
+---
+status: located
+opened: 2026-09-29
+located-in: [mesh-host examples + internal/link, mesh-controller internal/broker]
+---
+
+# 146 — the foundation cannot be raised on the bus the mesh runs on
+
+## What was observed
+
+Raising a first node in the lab, to check a module against a real mesh, fails before any module is
+reached. Two separate faults, in the two bundles that exist:
+
+**The older bundle raises a control plane that cannot start.** It brings up the previous broker,
+and the control plane it then starts says, once every few seconds, for ever:
+
+```
+mesh-controller: this control plane has no MESH_BUS_NATS, so it cannot reach the mesh's bus
+```
+
+That is the control plane being right. The mesh moved to one bus
+([ADR 0131](../../02-DECISIONS/0131-everything-on-the-mesh-speaks-to-the-broker-seat.md)) and the
+bundle did not. Every bed that raises a foundation raises this one, so every bed is in this state.
+
+**The newer bundle, written for the new bus, stops one step earlier.** Its certificate step asks a
+container to make the broker's certificate:
+
+```
+docker run --rm --entrypoint sh -v <the broker's tls volume>:/tls <the bus image> \
+  -c "test -f /tls/tls.crt || (openssl req -x509 ... )"
+...
+failed bus-certificate: running the action: docker exited 127
+```
+
+127 is *command not found*. The bus's image has a shell and no `openssl`; the previous broker's
+image had both, which is why the step worked when it was written against that one. Substituting the
+store's image — the only other image the bundle carries — does not help: it has no `openssl`
+either. So the step as written cannot succeed with anything the bundle names, and the fault is not
+one image's: **the bundle asks for a certificate to be made by a tool it never says must be there.**
+
+Measured 2026-09-29 on a fresh lab machine, both bundles, from bare.
+
+## Why this is here and not a note in the knowledge base
+
+The mesh's own foundation is the one thing it cannot raise. Nothing reports that: the bundles are
+files in a repository, nothing applies them but a person raising a node, and the last thing that
+did was the hand-driven cut-over
+([ADR 0131](../../02-DECISIONS/0131-everything-on-the-mesh-speaks-to-the-broker-seat.md), whose
+work was done on the machines rather than from a bundle). So the state where the mesh cannot make
+another one of itself is reachable, and was reached, without anything saying so.
+
+It is also load-bearing for everything else: a lab bed proves a claim by raising a mesh, so while
+this holds, **no bed can run**, and every "checked in the lab" written from now on is a promise
+against a suite nobody can execute.
+
+## What would have prevented it
+
+- **Something raising the foundation on a schedule, from the bundle, as it is written** — the
+  bundle is the mesh's own installer and nothing installs from it. A bed that raises a first node
+  is exactly that check, and it is the bed that cannot run.
+- **A step naming what it needs.** The certificate step names an image and assumes a program inside
+  it. An action that said which tool it requires would have failed at the declaration rather than
+  at 127 on a machine.
+
+## Evidence to carry into diagnosis
+
+- `mesh-host examples/foundation-first-node.lock` — the previous broker, no `MESH_BUS_NATS`.
+- `mesh-host examples/foundation-first-node-nats.lock` — the new bus; `bus-certificate` and its
+  `verify` both run `openssl` in the bus's image.
+- The bus image the bundle pins has `sh` and no `openssl`; the store's image likewise.
+- The lab rewrites a bundle's registry-prefixed third-party references to upstream ones for a
+  machine with an uplink (`test/integration/harness.ts`); the new bundle's bus reference needed
+  that rule added, which is done and is not this issue.

04-ISSUES/146-the-foundation-cannot-be-raised-on-the-bus-the-mesh-runs-on/01-diagnosis.md

@@ -0,0 +1,96 @@
+# Diagnosis
+
+*2026-09-29, by raising a first node in the lab over and over and writing down each thing it hit.*
+
+Not one fault. **Four, stacked**, each hidden behind the one before it, and every one of them the
+same shape: a step that was right while the mesh ran on the previous broker and was never asked a
+question again after the bus changed. Nothing had raised a foundation since, so nothing said so.
+
+## 1 — the bundle's bus image is named for a registry that is gone *(fixed)*
+
+The newer bundle pins `<a lab registry>/nats@…`, which resolves nowhere outside the lab that
+raised that registry. The lab already rewrites the store's and the previous broker's references to
+upstream ones for a machine with an uplink; the bus had no such rule because no bed had ever tried
+to raise this bundle. Added (`mesh-lab test/integration/harness.ts`). The digest is the bundle's
+own — what the registry served was a copy, so the same digest resolves upstream, and this is a
+prefix being removed rather than a reference being replaced.
+
+## 2 — the bus's certificate was made by a tool the bus does not have *(fixed)*
+
+```
+failed bus-certificate … docker exited 127
+```
+
+The step ran `openssl` inside the broker's image. The previous broker's image carried it; the bus's
+does not — it is Alpine with a shell and no `openssl` — and neither does any other image the bundle
+names, so there was nothing to substitute. **The program that needs the certificate now makes it**:
+`mesh-controller broker certificate --into <dir>`, with `--check` as the step's verify. The
+controller is already on the machine at that point (the schema step ran it) and needs nothing from
+the image it writes into. Self-signed, as before and on purpose — a host pins this server's exact
+certificate ([ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)) and at that moment
+there is no authority to ask. Idempotent, because a second certificate is one every host that
+pinned the first no longer believes. It runs `--user 0:0`: the volume is root's, and the control
+plane's image runs as nobody, which is right for the long-lived server and wrong for a one-shot
+writing into a fresh volume.
+
+## 3 — enrolment dialled TLS at a bus that speaks first *(fixed)*
+
+```
+mesh-host: cannot reach the broker at …:5671: tls: first record does not look like a TLS handshake
+```
+
+Enrolment opened a raw TLS connection to check the pinned certificate before saying anything. NATS
+speaks its own protocol and upgrades afterwards, so the handshake met a plaintext greeting. The pin
+was never the problem: the same pinned configuration is handed to the client that presents the
+token, and the verification runs inside *that* handshake — so what
+[ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md) requires still holds, and holds
+better, because the one-time secret is sent only after the certificate has been checked. The raw
+dial is gone from the enrolment path and kept only as what its tests always proved: that a wrong
+certificate is refused before a byte of application data is sent.
+
+**Then, immediately behind it:**
+
+```
+mesh-host: this token is for the "" bus, and the mesh's bus is nats
+```
+
+The enrolment left the transport empty and meant *whatever the mesh runs today*, which was true
+while two buses existed and became a refusal the moment one did. The host knows which bus the mesh
+runs; it says so now.
+
+## 4 — a first node cannot be let onto its own bus *(open, and this is the real one)*
+
+```
+mesh-host: cannot reach the bus at …:5671 as anchor: nats: Authorization Violation
+```
+
+The bus's user list is a file beside its configuration. The installer carries the first one — the
+controller's own account at a bootstrap password — and **the controller composes every user after
+that** (design 25 §6; the controller's own test asserts the carried list matches what it would
+derive). On the running mesh that composition reaches the bus because the bus is a *module*, with
+the list delivered to it the way anything is delivered to a module.
+
+At genesis there is no module. The foundation's bus is raised by the installer, the control plane is
+given no way to write beside its configuration — it mounts the certificate and nothing else — and so
+the account a joining node needs cannot come into existence. **The first node cannot join the mesh
+it just raised.**
+
+That is not a line to fix in a bundle. It is the open half of the mesh delivering its own components
+([ADR 0142](../../02-DECISIONS/0142-the-mesh-delivers-its-own-components-as-binaries.md)) and of the
+bus becoming a module: either the installer's bus is raised as the module the mesh will go on
+managing, or genesis carries a user list that includes the first node's enrolment and the controller
+takes over from there. Both are decisions, not patches, and both belong to the genesis step that was
+deliberately left until last.
+
+## Where it belongs
+
+`mesh-host` (the bundle and the enrolment path) and `mesh-controller` (the certificate command, and
+the composition that cannot reach the bus at genesis). Three of the four are fixed on branches; the
+fourth is the genesis work.
+
+## What it cost, for the next person
+
+Every lab bed still names `foundation-first-node.lock` in its own instructions, and that bundle
+raises the previous broker with a control plane that refuses to start without `MESH_BUS_NATS`. Until
+the fourth fault is answered and the two bundles become one, a bed runs with `MESH_LAB_BUNDLE`
+pointing at the NATS bundle by hand, and stops at the enrolment.