ADR 0147: a module anchors the mesh's authority on a machine, and takes it away again

Issue 129: every internal HTTPS name fails verification on every machine, because nothing has ever written the mesh's root into a trust store. The report proposed the controller inject it the way the private network writes the registry's trust; this record rejects that — reachability and trust are not the same fact, and where anchors live is the host's difference, not the controller's. A module requiring internal-acme-ca does the whole of it, and being unassigned undoes it.
2026-09-29 15:02:37 +02:00
parent ced547dae9
commit 6c14d313b8
5 changed files with 181 additions and 4 deletions
@@ -0,0 +1,126 @@
 ---
 topic: what runs on it
 status: accepted
 date: 2026-09-29
 deciders: jochen
 reconstructed: false
 extends: 02-DECISIONS/0098-a-fact-a-provider-makes-at-first-start-is-fetched-from-it.md
 ---
 # 147. A module anchors the mesh's authority on a machine, and takes it away again
 ## Context
 The mesh runs its own certificate authority and every internal name is served with a certificate
 from it. No machine trusts it. On an enrolled, adopted workstation — on the private network,
 resolving through the mesh's resolver — every internal HTTPS name fails verification with
 *unable to get local issuer certificate*
 ([issue 129](../04-ISSUES/129-nothing-makes-a-machine-trust-the-meshs-authority/00-report.md)).
 The certificates are genuine; nothing on the machine has ever been told what issued them.
 The authority's only consumer today is a proxy, which fetches the root into a directory of its own
 and hands it to one program ([ADR 0098](0098-a-fact-a-provider-makes-at-first-start-is-fetched-from-it.md)).
 That is enough for the proxy and for nothing else: a browser, `git` over HTTPS, `curl`, a package
 manager and every module that calls another module by an internal name read the machine's trust
 store, which holds the predecessor's authority and a developer tool's local root, and nothing of
 the mesh's.
 The predecessor wrote its root into every machine it set up. Removing it was deliberate — an
 honest failure beats a name that verifies for the wrong reason — and it leaves the mesh with no
 answer at all until this one lands. It is also what keeps the predecessor alive on the machines
 that still speak TLS to a mesh name.
 **What makes this a decision rather than a patch** is where the knowledge goes. Two mechanisms in
 the mesh already write things onto a machine because it is on the private network: `/etc/hosts`
 and the registry's plaintext trust ([ADR 0082](0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md)).
 Following that precedent, the controller would inject an anchor into every such machine's
 declaration, and issue 129 proposed exactly that. It would work. It would also put *where this
 operating system keeps trust anchors* and *which command refreshes its bundles* into the control
 plane, for a fact the control plane does not have (the root does not exist until the authority has
 run) and a machine that may have no reason to verify a mesh name at all.
 ## Considered Options
 1. **The controller injects the anchor into every machine on the private network**, the
   `/etc/hosts` and insecure-registry shape. Rejected: being on the network is what makes the
   registry reachable, and that is why network presence is the right trigger *there* — the trust
   and the reachability are the same fact. Trusting an authority is not the same fact as being
   able to reach it, and the anchor's path and the bundle refresh are a property of the machine's
   operating system, which is the host's half of the mesh, not the controller's.
 2. **A new host primitive — a `trust-anchor` resource type.** Rejected for now, not on principle.
   The host's vocabulary should grow when a shape cannot be said with what exists, and this one
   can: a file and a service already express it, as the packet filter proves
   ([ADR 0140](0140-the-filter-constrains-what-arrives-from-outside.md), whose module writes a
   unit file and a service and nothing else). The primitive becomes right the moment a second
   operating system is in play, because the anchor directory and the refresh command are exactly
   the difference `internal/system` exists to hold. Until then it would be a vocabulary word with
   one speaker.
 3. **A module that requires the authority, fetches its root, installs it as an anchor and
   refreshes the machine's bundles — and removes both when it is no longer assigned.** Adopted.
 ## Decision
 **A machine trusts the mesh's authority because a module put its root there, and stops trusting it
 when that module is taken away.**
 1. **The module requires `internal-acme-ca`** and reads the provider's bound address and the path
   it serves its root at. It requires nothing else and provides nothing: it is a consumer of the
   authority like any other.
 2. **It fetches the root over the mesh's own network, without prior trust**, because there is no
   prior trust to have — this is the module that establishes it — and the network is what
   authenticates the fetch ([ADR 0098](0098-a-fact-a-provider-makes-at-first-start-is-fetched-from-it.md),
   the same reasoning that lets the proxy fetch it). What it accepts is checked: a body that is
   not a certificate fails, and the failure is the module's, not a later handshake's.
 3. **It installs the root where this machine's TLS clients look, and refreshes the extracted
   bundles** — the command that does the refresh is an ordinary part of the unit that places the
   anchor, not a new thing the mesh can be asked to do.
 4. **Removal is symmetric and is the same unit's business.** Undeclared, the host stops the unit;
   stopping it removes the anchor and refreshes the bundles again. A machine that leaves the mesh
   stops trusting the mesh, without anybody remembering to go and look.
 5. **It is an ordinary assignment.** No machine is given it automatically. A machine that verifies
   a mesh name is assigned it, and a machine that does not is not — which is the same statement
   the mesh already makes about every other module, and is why this is not the controller's
   business.
 **One operating system, said out loud.** The anchor directory and the refresh command in the
 module today are Arch's. On a machine that is not Arch the unit fails, visibly, rather than
 writing a file nothing reads. That is the accurate failure, and it is the signal that option 2
 above has become right.
 ## How this is checked
 - **The verification that could not succeed before.** On a machine holding the module, a plain
  client fetches an internal HTTPS name with no `-k` and no bundle argument and verifies. On a
  machine without it, the same fetch fails with *unable to get local issuer certificate*. Both
  halves, because only the pair distinguishes "the anchor works" from "something else already
  trusted it".
 - **The removal half, in the same bed:** unassign the module, refetch, and the failure returns.
  Checking only the arrival is how a trust store fills up with authorities nobody can account for.
 - **What is deliberately not checked here:** that the authority issues, that a name resolves, that
  the proxy serves. Those have their own beds, and this module's bed passing for those reasons is
  the failure mode this record is most exposed to — which is why the negative half is not optional.
 ## Consequences
 The predecessor's authority can be retired from a machine once this module is assigned to it,
 which is the first time that has been true. `git` over HTTPS to the mesh's forge starts working,
 so the ssh-only clone URL stops being a rule. A module on any machine can call another module's
 internal name and verify it.
 What got harder: one more module to assign to a machine that needs it, and the machine's trust
 store now changes when an assignment changes — which is the point, and is also a thing an operator
 can be surprised by. The fetch without prior trust is the same exposure ADR 0098 accepted, now on
 every machine that holds the module rather than only where a proxy runs: anything that can stand
 in the middle of the mesh's own network at the moment of the fetch can be believed. The mesh
 already treats that network as the thing it authenticates.
 ## References
 - [issue 129](../04-ISSUES/129-nothing-makes-a-machine-trust-the-meshs-authority/00-report.md) —
  the symptom and the evidence.
 - [ADR 0098](0098-a-fact-a-provider-makes-at-first-start-is-fetched-from-it.md) — a fact made at
  first start is fetched from its provider; this extends it from one program to the machine.
 - [ADR 0082](0082-the-registry-is-reached-by-name-and-trusted-by-the-overlay.md) — the precedent
  this deliberately does not follow, and why it is right where it is.
 - [ADR 0005](0005-the-node-host.md) — the host is where one operating system's difference lives.
 - [`03-DESIGN/01-to-be/08-connectivity.md`](../03-DESIGN/01-to-be/08-connectivity.md).
@@ -225,7 +225,9 @@ python3 00-META/checks/index.py            fail if stale
 - **0141** — [The host delivers its own successor, and versions live side by side](0141-the-host-delivers-its-own-successor.md)
 - **0143** — [A consumer verifies the grant it is given](0143-a-consumer-verifies-the-grant-it-is-given.md)  *(superseded)*
 - **0144** — [Anything on a machine may call anything on it, and that is the whole of "local"](0144-anything-on-a-machine-may-call-anything-on-it.md)
- **0145** — [A module checks what the mesh claims is reachable, and it checks itself](0145-a-module-checks-what-the-mesh-claims-is-reachable.md)
+- **0145** — [A module checks what the mesh claims is reachable, and it checks itself](0145-a-module-checks-what-the-mesh-claims-is-reachable.md)  *(superseded)*
 - **0146** — [Connectivity is checked by name, per hosting form, with a valid certificate](0146-connectivity-is-checked-by-name-per-hosting-form.md)
 - **0147** — [A module anchors the mesh's authority on a machine, and takes it away again](0147-a-module-anchors-the-meshs-authority.md)
 ### How it is built
@@ -7,8 +7,9 @@ code:
  - mesh-controller internal/identity/authority.go
  - mesh-host internal/identity/serving.go
  - mesh-host internal/apply (the service that reflects a rule set)
-updated: 2026-09-28
+updated: 2026-09-29
 decisions:
  - 02-DECISIONS/0147-a-module-anchors-the-meshs-authority.md
  - 02-DECISIONS/0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md
  - 02-DECISIONS/0140-the-filter-constrains-what-arrives-from-outside.md
  - 02-DECISIONS/0104-a-provision-may-be-answered-by-an-adapter-to-the-predecessor.md
@@ -710,6 +711,22 @@ step, so when the authority moves the root is fetched again and the proxy is rec
 *How it is checked:* the route-forwarding bed installs the authority, the proxy and a consumer
 from the catalogue and asserts the routed name is served.
 **And a machine trusts that authority because a module put its root in its trust store**
 ([ADR 0147](../../02-DECISIONS/0147-a-module-anchors-the-meshs-authority.md)). The proxy's fetch
 answers for the proxy and for nothing else: a browser, `git` over HTTPS, a package manager and
 every module calling another by an internal name read the machine's own trust store, and the mesh
 had never written anything there. A module requiring the authority does the whole of it — fetch
 the root over the mesh network, place it where this machine's TLS clients look, refresh the
 extracted bundles — and stopping it, which is what being unassigned does, takes the anchor away
 and refreshes them again. Not the controller's business, because being on the private network is
 what makes the authority *reachable* and is not the same fact as having a reason to *verify* a
 mesh name; and because where anchors live and which command refreshes them is one operating
 system's difference, which is the host's half of the mesh
 ([ADR 0005](../../02-DECISIONS/0005-the-node-host.md)).
 *How it is checked:* on a machine holding the module a plain client verifies an internal HTTPS
 name with no bundle argument, and on one without it the same fetch fails to find an issuer — both
 halves, because only the pair tells the anchor apart from something that already trusted it.
 ### What was built
 *2026-08-31.*
@@ -1,7 +1,7 @@
 ---
-status: open
+status: located
 opened: 2026-09-26
-located-in: [mesh-controller, mesh-catalog step-ca]
+located-in: [mesh-catalog ca-trust]
 ---
 # 129 — nothing makes a machine trust the mesh's own certificate authority
@@ -0,0 +1,32 @@
 # Diagnosis
 *2026-09-29.*
 ## What was ruled out
 **That something already carries the root and it is only misplaced.** It does not. The authority
 serves its root at a path beside its ACME directory, and the one thing that fetches it — the route
 proxy — puts it in a directory of its own and hands it to one program. Nothing has ever written
 into a machine's trust store. Measured on three converged machines: the anchors present are the
 predecessor's authority and a developer tool's local root, and on the machines where the
 predecessor's was deliberately removed, every internal name fails verification.
 **That the private network could carry it, the way it carries the registry's trust.** That is what
 the report proposed, and it was rejected on consideration rather than on difficulty
 ([ADR 0147](../../02-DECISIONS/0147-a-module-anchors-the-meshs-authority.md), option 1): being on
 the network is what makes the registry *reachable* and is therefore the right trigger there, while
 trusting an authority is a separate fact from being able to reach it. The anchor's directory and
 the command that refreshes the extracted bundles are also one operating system's difference, which
 is the host's half of the mesh and not the controller's.
 **That it needs a new host resource type.** It does not, today. A file and a service say the whole
 of it, which the packet filter already proves. The primitive becomes the right answer when a second
 operating system is in play, and not before.
 ## Where it belongs
 A module in the catalogue: it requires `internal-acme-ca`, fetches the root over the mesh's own
 network, installs it as a trust anchor, refreshes the machine's bundles, and — because being
 unassigned stops its unit, and stopping the unit is what undoes it — takes both away again.
 The owner is therefore `mesh-catalog`, module `ca-trust`, and nothing in the control plane.