ADR 0146: connectivity is checked by name, per hosting form, with a valid certificate
0145's core stands — a module checks what the mesh claims, from where the callers are — and everything it said about what to dial was wrong. A raw port is not how anything in this mesh is reached. A real caller resolves a name, the proxy answers, the proxy reaches the service; dialling a port tests the last hop of a four-hop path and skips the three where most connectivity lives. So each hosting form gets its own endpoint, route and name: connect-docker.<node>.internal for a container, connect-process for the mesh's own code in a unit it writes, connect-unit for a unit a package ships — and the same under each public domain. Every machine checks every machine, by name, over TLS, verifying the certificate against the authority that should have issued it. The names are the instrument: a failure reads as connect-docker.g14.internal did not answer. No name is written anywhere: the machines come from the roster fact, the labels are the module's, and a machine that joins is checked by the others on the next push. Five things this needs that the mesh does not have, named rather than assumed — a module every machine has (ScopeNode is exclusivity, not obligation), a container running a bundle, a unit a package ships, the public domain in the roster fact, and issue 129, until which every internal name will correctly fail verification.
This commit is contained in:
@@ -1,10 +1,11 @@
|
||||
---
|
||||
topic: what runs on it
|
||||
status: accepted
|
||||
status: superseded
|
||||
date: 2026-09-29
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0144-anything-on-a-machine-may-call-anything-on-it.md
|
||||
superseded-by: 02-DECISIONS/0146-connectivity-is-checked-by-name-per-hosting-form.md
|
||||
---
|
||||
|
||||
# 145. A module checks what the mesh claims is reachable, and it checks itself
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
---
|
||||
topic: what runs on it
|
||||
status: accepted
|
||||
date: 2026-09-29
|
||||
deciders: jochen
|
||||
reconstructed: false
|
||||
extends: 02-DECISIONS/0145-a-module-checks-what-the-mesh-claims-is-reachable.md
|
||||
supersedes: 02-DECISIONS/0145-a-module-checks-what-the-mesh-claims-is-reachable.md
|
||||
---
|
||||
|
||||
# 146. Connectivity is checked by name, per hosting form, with a valid certificate
|
||||
|
||||
## Context
|
||||
|
||||
[ADR 0145](0145-a-module-checks-what-the-mesh-claims-is-reachable.md) decided that a module checks what
|
||||
the mesh claims is reachable, from where the callers are, because the mesh reported four machines healthy
|
||||
for eleven hours while a module could not reach its database
|
||||
([issue 145](../04-ISSUES/145-a-machine-reads-healthy-while-its-modules-cannot-reach-each-other/00-report.md)).
|
||||
That decision stands. What it got wrong is everything about *what* is dialled.
|
||||
|
||||
It dialled a raw port on each machine's address. Three things are wrong with that:
|
||||
|
||||
- **A raw port is not how anything in this mesh is reached.** A real caller resolves a name, the proxy
|
||||
answers it, and the proxy reaches the service. A check that dials a port tests the last hop of a path
|
||||
with four hops in it, and the three it skips — resolution, the proxy, the certificate — are where most
|
||||
of the mesh's connectivity actually lives.
|
||||
- **It tested one hosting form.** A module is software that delivers services, and it may deliver them
|
||||
from a container, from a unit the mesh writes for its own code, or from a unit a package ships. Those
|
||||
are three different paths to the same machine, and the outage that produced this was two of them
|
||||
disagreeing. A probe served one way measures one way.
|
||||
- **It said nothing about certificates.** An internal name that resolves, routes and answers over TLS
|
||||
that nothing can verify is not a working path; it is a working path for whoever holds the proxy's
|
||||
trust and nobody else.
|
||||
|
||||
## Decision
|
||||
|
||||
**Each hosting form gets its own endpoint, its own route and therefore its own name.** On every machine:
|
||||
|
||||
| name | what serves it |
|
||||
|---|---|
|
||||
| `connect-docker.<node>.internal` | a container |
|
||||
| `connect-process.<node>.internal` | the mesh's own code, in a unit the mesh writes |
|
||||
| `connect-unit.<node>.internal` | a unit a package ships |
|
||||
|
||||
and the same set under each machine's public domain where it has one — `connect-docker.<domain>` and its
|
||||
siblings. The names are the instrument: a failure reads as *`connect-docker.g14.internal` did not answer*,
|
||||
which says which machine and which hosting form without anybody interpreting anything.
|
||||
|
||||
**Every machine checks every machine, by name, over TLS, verifying the certificate.** Not a port, not an
|
||||
address: resolve the name, connect, complete the handshake, check the certificate against the authority
|
||||
that should have issued it — the mesh's own for an internal name, a public one for a public name. That is
|
||||
the whole path a real caller takes, and each step failing is reported as itself.
|
||||
|
||||
**No name is written anywhere.** The machines come from the roster the mesh already renders as a fact, and
|
||||
the labels are the module's. A machine that joins appears in every other machine's roster on the next
|
||||
push, and they begin checking it without an edit.
|
||||
|
||||
**And the module arrives on a machine because the machine exists, not because somebody assigned it.** A
|
||||
machine that joins and does not have it is worse than unchecked: every other machine is already dialling
|
||||
its names, so it reads as broken everywhere until someone notices. This is the part the mesh cannot
|
||||
currently express — see below — and it is the part that makes the rest safe.
|
||||
|
||||
**What survives from 0145**, unchanged: it reports and repairs nothing; one failure is not a fault and a
|
||||
path is broken after consecutive runs with the count travelling with the result; findings are said on the
|
||||
bus, because a finding in a file on the machine is what this exists to end; and the bus is the one path
|
||||
that cannot report its own failure, so an emit that does not land is written locally and nowhere else.
|
||||
|
||||
## What this needs that the mesh does not have
|
||||
|
||||
Named here rather than assumed, because each is a decision of its own and this record is not the place to
|
||||
make them:
|
||||
|
||||
1. **A module that every machine has.** `ScopeNode` means *at most one holder per node* — an exclusivity
|
||||
rule, not an obligation — and nothing assigns a module at enrolment. Today the resolver, the packet
|
||||
filter, ssh and intrusion prevention are each assigned per machine by hand, which is the same gap
|
||||
wearing different clothes.
|
||||
2. **A container running a module's own bundle.** A `process` runs the mesh's own compiled code with no
|
||||
image; a `container` needs an image of the module's own, which means a Dockerfile — the thing the
|
||||
`bundle` artifact exists to abolish. Nothing in the catalogue runs a bundle in a container, so
|
||||
`connect-docker` has no shape yet.
|
||||
3. **A unit a package ships, for `connect-unit`.** The `service` resource puts an existing unit into a
|
||||
state and deliberately installs none, so this form needs a package that serves a port — and naming a
|
||||
program the machine may not have is
|
||||
[issue 136](../04-ISSUES/136-a-module-may-name-a-program-the-machine-does-not-have/00-report.md).
|
||||
4. **A machine's public domain in the roster fact.** The fact carries each machine's name, mesh name,
|
||||
address and operator account. The public names cannot be composed without the domain.
|
||||
5. **Something that installs the mesh's own root on a machine.** This is
|
||||
[issue 129](../04-ISSUES/129-nothing-makes-a-machine-trust-the-meshs-authority/00-report.md), open
|
||||
since before any of this. Until it is closed, every internal name will fail certificate verification
|
||||
from every machine — correctly, because nothing can verify it. That is the checker working, and it is
|
||||
worth saying in advance so the first run is not read as the checker being broken.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **A failure names the machine and the hosting form.** That is the whole gain over a port: eleven hours
|
||||
became two runs under 0145, and under this it also becomes one line that says where to look.
|
||||
- **The checker surfaces issue 129 immediately**, and will report every internal name unverifiable until
|
||||
it is fixed. A reader must be told that before the first run rather than after.
|
||||
- **Five things must be built before this is what it says it is**, and until they are, what exists is a
|
||||
port dial from one position — useful, and not this.
|
||||
- **What got harder:** a module with three hosting forms of the same trivial service is a strange thing to
|
||||
read. It is justified only because those three forms are how the mesh actually runs software, and a
|
||||
checker that tested one of them would keep the class of outage it exists to catch.
|
||||
|
||||
## How it is checked
|
||||
|
||||
- **A name per hosting form answers from every machine**, asserted by name and not by port.
|
||||
- **A certificate that does not verify is reported as that**, distinctly from a name that does not resolve
|
||||
and a port that does not answer — three faults, three owners.
|
||||
- **A machine that joins is checked by every other machine without an edit**, asserted by adding one to a
|
||||
bed and looking at what the others dial on their next run.
|
||||
- **A machine that joins has the module**, which is gap 1 above and is the assertion that cannot be
|
||||
written yet.
|
||||
- **The measured outage is still caught**: the path from a container to a service on its own machine is
|
||||
closed and `connect-docker.<that node>.internal` fails from that machine while the others still pass.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR 0145](0145-a-module-checks-what-the-mesh-claims-is-reachable.md) — superseded; its core stands
|
||||
- [ADR 0138](0138-an-assignment-binds-an-endpoint-and-says-how-far-it-reaches.md) — the two reaches these
|
||||
names come from
|
||||
- [ADR 0066](0066-public-routing-is-name-agnostic.md) — a label plus a domain, which is why no name is written
|
||||
- [issue 129](../04-ISSUES/129-nothing-makes-a-machine-trust-the-meshs-authority/00-report.md) — what the
|
||||
internal names will fail on until it is closed
|
||||
- [issue 145](../04-ISSUES/145-a-machine-reads-healthy-while-its-modules-cannot-reach-each-other/00-report.md)
|
||||
Reference in New Issue
Block a user