ADR 0146: connectivity is checked by name, per hosting form, with a valid certificate

0145's core stands — a module checks what the mesh claims, from where the callers
are — and everything it said about what to dial was wrong.

A raw port is not how anything in this mesh is reached. A real caller resolves a
name, the proxy answers, the proxy reaches the service; dialling a port tests the
last hop of a four-hop path and skips the three where most connectivity lives.

So each hosting form gets its own endpoint, route and name:
connect-docker.<node>.internal for a container, connect-process for the mesh's own
code in a unit it writes, connect-unit for a unit a package ships — and the same
under each public domain. Every machine checks every machine, by name, over TLS,
verifying the certificate against the authority that should have issued it. The
names are the instrument: a failure reads as connect-docker.g14.internal did not
answer.

No name is written anywhere: the machines come from the roster fact, the labels are
the module's, and a machine that joins is checked by the others on the next push.

Five things this needs that the mesh does not have, named rather than assumed —
a module every machine has (ScopeNode is exclusivity, not obligation), a container
running a bundle, a unit a package ships, the public domain in the roster fact, and
issue 129, until which every internal name will correctly fail verification.

02-DECISIONS/0145-a-module-checks-what-the-mesh-claims-is-reachable.md

@@ -0,0 +1,120 @@
+---
+topic: what runs on it
+status: superseded
+date: 2026-09-29
+deciders: jochen
+reconstructed: false
+extends: 02-DECISIONS/0144-anything-on-a-machine-may-call-anything-on-it.md
+superseded-by: 02-DECISIONS/0146-connectivity-is-checked-by-name-per-hosting-form.md
+---
+
+# 145. A module checks what the mesh claims is reachable, and it checks itself
+
+## Context
+
+The mesh asserts three things are callable ([ADR 0144](0144-anything-on-a-machine-may-call-anything-on-it.md)):
+what runs on the same machine, another machine's service exposed to the private network, and another
+machine's service exposed publicly. It has never checked any of them.
+
+[Issue 145](../04-ISSUES/145-a-machine-reads-healthy-while-its-modules-cannot-reach-each-other/00-report.md):
+the first of the three was broken for eleven hours and the mesh answered *all heard from, every module
+current with its source* throughout. Every check it makes is about the relationship between the mesh and
+a machine — applied, current, containers running — and none about whether anything can reach anything.
+
+**A first answer was drafted and withdrawn.** [ADR 0143](0143-a-consumer-verifies-the-grant-it-is-given.md)
+put the check inside the host, verifying each grant from the consumer's network position. It was
+superseded because the difference it worked so hard to reproduce — whether a caller sat in a container —
+was the bug itself. What survives from it is the part that was right: a check run from the wrong place
+proves nothing, and the mesh's own reports are not evidence about the network.
+
+**The mesh already has the shape for this and it is a module.** A module can declare a container that
+runs on a cadence ([ADR 0053](0053-a-step-that-runs-on-a-schedule.md), and three modules already use
+`*/5 * * * *`), can be given the mesh's roster as a rendered fact — every machine's name, address and
+this node's own identity, the same mechanism the resolver and the operator's ssh configuration use — and
+can emit what it found on the bus. Nothing new is needed to build this except the module.
+
+**What it must not check is the trap.** The obvious probe target is ssh: present on every machine, never
+closed by design. Dialling it would have passed throughout the outage, because ssh is admitted
+unconditionally and the thing that broke was a service exposed to the private network. A checker whose
+probe is unconditionally open measures the one path that cannot fail, which is the failure this whole
+sequence keeps producing — a check that reads as verification and verifies nothing.
+
+## Considered Options
+
+1. **The host verifies each grant** ([ADR 0143](0143-a-consumer-verifies-the-grant-it-is-given.md)).
+   Superseded. It needed the host to act from another network position, which is machinery that exists
+   only while local calls are filtered wrongly.
+2. **The control plane dials every node.** Rejected: it sits on one machine and reaches the others by a
+   path no ordinary caller uses. It would have passed throughout the outage.
+3. **Probe an existing service.** Rejected for the target problem above: the services guaranteed on every
+   machine are the ones that are never closed, so they cannot fail the way the mesh fails.
+4. **A module on every machine that serves its own probe and dials the others'.** Adopted.
+
+## Decision
+
+**A module runs on every machine, serves an endpoint of its own, and dials every other machine's.** The
+probe is the module's own endpoint, declared reachable over the private network — so the thing being
+dialled is admitted by exactly the rule that governs every other internally-exposed service, and fails
+when that rule is wrong. A second endpoint, declared public, does the same for the public path where a
+machine has one.
+
+**It checks the three cases the mesh claims, by name:**
+
+- its **own machine**, by dialling its own machine's address — the case that broke, and the only one that
+  distinguishes a caller on the machine from a caller in one of its containers;
+- **each other machine over the private network**;
+- **each machine's public path**, where one is recorded.
+
+**It resolves before it dials, and says which failed.** A name that does not resolve and a port that does
+not answer are different faults with different owners, and a checker that reports one sentence for both
+sends a reader to the wrong place.
+
+**It runs where the callers run.** The module's own code in its own container, on the cadence the mesh
+already has, from the same position as every other module on that machine. It is not the host and not the
+control plane, and that is the whole point.
+
+**It says what it found and nothing else.** It emits results; it repairs nothing, opens nothing and holds
+no credential beyond its own. A checker that fixes things is a second control plane.
+
+**One failure is not a fault.** A machine rebooting is ordinary. A path is reported broken after it has
+failed on consecutive runs, and the count travels with the result so a reader can tell "briefly away"
+from "never worked" — the one thing [ADR 0143](0143-a-consumer-verifies-the-grant-it-is-given.md) got
+right and worth keeping.
+
+## Consequences
+
+- **The mesh gains the ability to be wrong out loud about the network.** Eleven hours becomes two runs.
+- **It is a module, so it is assigned, built, pushed and reported on like everything else** — no new host
+  capability, no new vocabulary, nothing in the control plane that has to know about checking.
+- **Its own endpoint is the instrument.** That is what makes it able to fail; it also means the checker
+  must be assigned to a machine before that machine can be checked, and a machine without it is
+  unchecked rather than healthy.
+- **It cannot check what it cannot be told.** The roster gives it machines; it does not give it every
+  module's endpoints, so this checks the paths the mesh claims and not every grant in the mesh. That is
+  the honest scope of a first one, and the difference is worth saying rather than growing quietly.
+- **What got harder:** one more module on every machine, and a module whose whole purpose is to fail
+  visibly when something else is wrong. Its own failures will be read as the mesh's, which is the cost of
+  an instrument.
+
+## How it is checked
+
+- **It catches the measured outage.** A bed closes the path from a container to a service exposed to the
+  private network on its own machine — issue 145's shape — and the checker reports its own machine
+  unreachable while every other path still reads reachable. This fails against a probe on a port that is
+  never closed, which is the wrong target this record exists to name.
+- **A machine rebooting is not a fault**: one failed run reports nothing, the count rises and falls.
+- **A name that does not resolve is reported as that**, not as a port that did not answer.
+- **It reports and does not act**: asserted by giving it a broken path and checking nothing on the machine
+  changed.
+- **A machine without the module reads unchecked**, never healthy — asserted on what the mesh says about
+  a machine it is not assigned to.
+
+## References
+
+- [ADR 0144](0144-anything-on-a-machine-may-call-anything-on-it.md) — the three things that must be callable
+- [ADR 0143](0143-a-consumer-verifies-the-grant-it-is-given.md) — superseded; what survives is that the
+  position matters
+- [ADR 0053](0053-a-step-that-runs-on-a-schedule.md) — the cadence
+- [ADR 0138](0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md) — internal and public,
+  which the probe endpoints declare
+- [issue 145](../04-ISSUES/145-a-machine-reads-healthy-while-its-modules-cannot-reach-each-other/00-report.md)

02-DECISIONS/0146-connectivity-is-checked-by-name-per-hosting-form.md

@@ -0,0 +1,125 @@
+---
+topic: what runs on it
+status: accepted
+date: 2026-09-29
+deciders: jochen
+reconstructed: false
+extends: 02-DECISIONS/0145-a-module-checks-what-the-mesh-claims-is-reachable.md
+supersedes: 02-DECISIONS/0145-a-module-checks-what-the-mesh-claims-is-reachable.md
+---
+
+# 146. Connectivity is checked by name, per hosting form, with a valid certificate
+
+## Context
+
+[ADR 0145](0145-a-module-checks-what-the-mesh-claims-is-reachable.md) decided that a module checks what
+the mesh claims is reachable, from where the callers are, because the mesh reported four machines healthy
+for eleven hours while a module could not reach its database
+([issue 145](../04-ISSUES/145-a-machine-reads-healthy-while-its-modules-cannot-reach-each-other/00-report.md)).
+That decision stands. What it got wrong is everything about *what* is dialled.
+
+It dialled a raw port on each machine's address. Three things are wrong with that:
+
+- **A raw port is not how anything in this mesh is reached.** A real caller resolves a name, the proxy
+  answers it, and the proxy reaches the service. A check that dials a port tests the last hop of a path
+  with four hops in it, and the three it skips — resolution, the proxy, the certificate — are where most
+  of the mesh's connectivity actually lives.
+- **It tested one hosting form.** A module is software that delivers services, and it may deliver them
+  from a container, from a unit the mesh writes for its own code, or from a unit a package ships. Those
+  are three different paths to the same machine, and the outage that produced this was two of them
+  disagreeing. A probe served one way measures one way.
+- **It said nothing about certificates.** An internal name that resolves, routes and answers over TLS
+  that nothing can verify is not a working path; it is a working path for whoever holds the proxy's
+  trust and nobody else.
+
+## Decision
+
+**Each hosting form gets its own endpoint, its own route and therefore its own name.** On every machine:
+
+| name | what serves it |
+|---|---|
+| `connect-docker.<node>.internal` | a container |
+| `connect-process.<node>.internal` | the mesh's own code, in a unit the mesh writes |
+| `connect-unit.<node>.internal` | a unit a package ships |
+
+and the same set under each machine's public domain where it has one — `connect-docker.<domain>` and its
+siblings. The names are the instrument: a failure reads as *`connect-docker.g14.internal` did not answer*,
+which says which machine and which hosting form without anybody interpreting anything.
+
+**Every machine checks every machine, by name, over TLS, verifying the certificate.** Not a port, not an
+address: resolve the name, connect, complete the handshake, check the certificate against the authority
+that should have issued it — the mesh's own for an internal name, a public one for a public name. That is
+the whole path a real caller takes, and each step failing is reported as itself.
+
+**No name is written anywhere.** The machines come from the roster the mesh already renders as a fact, and
+the labels are the module's. A machine that joins appears in every other machine's roster on the next
+push, and they begin checking it without an edit.
+
+**And the module arrives on a machine because the machine exists, not because somebody assigned it.** A
+machine that joins and does not have it is worse than unchecked: every other machine is already dialling
+its names, so it reads as broken everywhere until someone notices. This is the part the mesh cannot
+currently express — see below — and it is the part that makes the rest safe.
+
+**What survives from 0145**, unchanged: it reports and repairs nothing; one failure is not a fault and a
+path is broken after consecutive runs with the count travelling with the result; findings are said on the
+bus, because a finding in a file on the machine is what this exists to end; and the bus is the one path
+that cannot report its own failure, so an emit that does not land is written locally and nowhere else.
+
+## What this needs that the mesh does not have
+
+Named here rather than assumed, because each is a decision of its own and this record is not the place to
+make them:
+
+1. **A module that every machine has.** `ScopeNode` means *at most one holder per node* — an exclusivity
+   rule, not an obligation — and nothing assigns a module at enrolment. Today the resolver, the packet
+   filter, ssh and intrusion prevention are each assigned per machine by hand, which is the same gap
+   wearing different clothes.
+2. **A container running a module's own bundle.** A `process` runs the mesh's own compiled code with no
+   image; a `container` needs an image of the module's own, which means a Dockerfile — the thing the
+   `bundle` artifact exists to abolish. Nothing in the catalogue runs a bundle in a container, so
+   `connect-docker` has no shape yet.
+3. **A unit a package ships, for `connect-unit`.** The `service` resource puts an existing unit into a
+   state and deliberately installs none, so this form needs a package that serves a port — and naming a
+   program the machine may not have is
+   [issue 136](../04-ISSUES/136-a-module-may-name-a-program-the-machine-does-not-have/00-report.md).
+4. **A machine's public domain in the roster fact.** The fact carries each machine's name, mesh name,
+   address and operator account. The public names cannot be composed without the domain.
+5. **Something that installs the mesh's own root on a machine.** This is
+   [issue 129](../04-ISSUES/129-nothing-makes-a-machine-trust-the-meshs-authority/00-report.md), open
+   since before any of this. Until it is closed, every internal name will fail certificate verification
+   from every machine — correctly, because nothing can verify it. That is the checker working, and it is
+   worth saying in advance so the first run is not read as the checker being broken.
+
+## Consequences
+
+- **A failure names the machine and the hosting form.** That is the whole gain over a port: eleven hours
+  became two runs under 0145, and under this it also becomes one line that says where to look.
+- **The checker surfaces issue 129 immediately**, and will report every internal name unverifiable until
+  it is fixed. A reader must be told that before the first run rather than after.
+- **Five things must be built before this is what it says it is**, and until they are, what exists is a
+  port dial from one position — useful, and not this.
+- **What got harder:** a module with three hosting forms of the same trivial service is a strange thing to
+  read. It is justified only because those three forms are how the mesh actually runs software, and a
+  checker that tested one of them would keep the class of outage it exists to catch.
+
+## How it is checked
+
+- **A name per hosting form answers from every machine**, asserted by name and not by port.
+- **A certificate that does not verify is reported as that**, distinctly from a name that does not resolve
+  and a port that does not answer — three faults, three owners.
+- **A machine that joins is checked by every other machine without an edit**, asserted by adding one to a
+  bed and looking at what the others dial on their next run.
+- **A machine that joins has the module**, which is gap 1 above and is the assertion that cannot be
+  written yet.
+- **The measured outage is still caught**: the path from a container to a service on its own machine is
+  closed and `connect-docker.<that node>.internal` fails from that machine while the others still pass.
+
+## References
+
+- [ADR 0145](0145-a-module-checks-what-the-mesh-claims-is-reachable.md) — superseded; its core stands
+- [ADR 0138](0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md) — the two reaches these
+  names come from
+- [ADR 0066](0066-public-routing-is-name-agnostic.md) — a label plus a domain, which is why no name is written
+- [issue 129](../04-ISSUES/129-nothing-makes-a-machine-trust-the-meshs-authority/00-report.md) — what the
+  internal names will fail on until it is closed
+- [issue 145](../04-ISSUES/145-a-machine-reads-healthy-while-its-modules-cannot-reach-each-other/00-report.md)

02-DECISIONS/README.md

@@ -225,6 +225,7 @@ python3 00-META/checks/index.py            fail if stale
 - **0141** — [The host delivers its own successor, and versions live side by side](0141-the-host-delivers-its-own-successor.md)
 - **0143** — [A consumer verifies the grant it is given](0143-a-consumer-verifies-the-grant-it-is-given.md)  *(superseded)*
 - **0144** — [Anything on a machine may call anything on it, and that is the whole of "local"](0144-anything-on-a-machine-may-call-anything-on-it.md)
+- **0145** — [A module checks what the mesh claims is reachable, and it checks itself](0145-a-module-checks-what-the-mesh-claims-is-reachable.md)
 
 ### How it is built
 

04-ISSUES/145-a-machine-reads-healthy-while-its-modules-cannot-reach-each-other/00-report.md

@@ -90,9 +90,17 @@ same machine, another machine's service exposed to the private network, and anot
 exposed publicly. The filter had the second and third and expressed the first as a list of addresses that
 no container could match.
 
-**The question this issue is actually about is still open.** Nothing in the mesh would have said a grant
-had stopped working, and nothing does now. That is not answered by a configuration fix, and it should not
-be answered by a mechanism adopted as a remedy for one.
+**And then the question this issue is actually about was answered on its own terms.**
+[ADR 0145](../../02-DECISIONS/0145-a-module-checks-what-the-mesh-claims-is-reachable.md): a module on
+every machine serves an endpoint of its own and dials every other machine's, from the position the
+callers are in. Its probe is its own endpoint declared reachable over the private network, so it is
+admitted by exactly the rule that governs every internally-exposed service and fails when that rule is
+wrong — where a probe on a service every machine has would have passed for all eleven hours, because the
+services every machine has are the ones never closed.
+
+Adopted on its merits rather than as the remedy for a configuration error, which is what 0143 was and
+why it went. The module is written and merged; it is not yet assigned, so every machine currently reads
+unchecked.
 
 ## Open questions