hq/03-DESIGN/01-to-be/08-connectivity.md

---
layer: to-be
status: designed
code: []
updated: 2026-08-27
decisions:
  - 02-DECISIONS/0037-the-node-host.md
  - 02-DECISIONS/0036-a-node-and-how-it-joins.md
  - 02-DECISIONS/0049-connectivity.md
  - 02-DECISIONS/0049-connectivity.md
  - 02-DECISIONS/0036-a-node-and-how-it-joins.md
  - 02-DECISIONS/0049-connectivity.md
  - 02-DECISIONS/0048-the-substrate-and-the-control-plane.md
---

# Connectivity

One of [the control plane's](06-the-control-plane.md) ten contexts, and the one with the most
moving parts: **overlay, resolution, exposure, filtering, certificates.**

It is written as a whole because the five are one design. They share inputs, they must agree, and
every one of them today is computed in a different place by a different module from a different
copy of the same facts.

## Why it is control-plane work

Apply [the test](06-the-control-plane.md) — *everything that needs to know about more than one
node* — to each responsibility:

| | needs to know | whose |
|---|---|---|
| **overlay** — who peers with whom, at what address | **every node**, and which of them can be dialled | control plane |
| **resolution** — which name is which node | **every node** | control plane |
| **exposure** — which public name reaches which container | **which node is publicly reachable** ([ADR 0049](../../02-DECISIONS/0049-connectivity.md)) | control plane |
| **filtering** — which port is open, to whom | what is assigned here, and the overlay's shape | control plane decides, host applies |
| **certificates** — who may present which name | which name belongs to which node | control plane |

**Not one of the five can be answered by a machine on its own.** That is the whole reason this is
a context rather than a set of node-local modules — and it is exactly what the current
arrangement gets wrong, by computing all five on the node from a direct database connection.

## The shape: decided centrally, delivered as files

Every one of the five resolves the same way, and it is worth stating once rather than five times:

> **The connectivity context computes the configuration. It arrives over the link as `file`
> resources. The service reads files and knows nothing about the mesh.**

This costs **no new host vocabulary**. `file`, `directory`, `service` and `container` already
exist; WireGuard, the resolver and the proxy are all *a container or a package, plus files*.

It is also what removes the last two upward dependencies.
[Research 006](../../01-RESEARCH/006-mesh-from-scratch/host-size.md) counted exactly two modules
opening a direct connection to the control plane's database — `wireguard` and `traefik` — and
they are the reason every node permanently holds a credential to it
([ADR 0036](../../02-DECISIONS/0036-a-node-and-how-it-joins.md)). Both are connectivity
modules. **Closing this context closes that set.**

## The order it comes up in

The one thing to get right, because everything else depends on it:

```
0  the node has an underlay address     the machine's own — DHCP, or a provider gave it one
1  the node dials the mesh              OVER THE UNDERLAY, at the address in its token
2  it proves itself, and is proved to   the link exists      (ADR 0039, ADR 0051)
3  the mesh grants it an identity       and an overlay address
4  the overlay comes up                 peer graph delivered as files
5  names resolve                        resolver config delivered as files
6  filtering is applied                 derived from what is assigned here
7  routes and certificates              once this node has something to expose
```

**Step 1 runs on the underlay and never on the overlay.** This is the circularity that must not
be created: the overlay is configured by the mesh, so a link that required the overlay could
never be established on a new node. The link stays on the underlay permanently — it is
outbound-only and carries its own identity, so it needs nothing the overlay provides.

**Nothing before step 3 can resolve a mesh name**, which is why the token carries an *address*
([ADR 0036](../../02-DECISIONS/0036-a-node-and-how-it-joins.md)). Today this is
patched with an `/etc/hosts` floor written underneath the resolver; under this design there is
nothing to patch.

## 1 — The overlay

**What is decided:** the peer graph. For every node: its overlay address, which peers it holds,
which of those it may dial, and which must dial it.

**Inputs, all declared:**

- **reachability** — an endpoint, or none
  ([ADR 0049](../../02-DECISIONS/0049-connectivity.md)). Not
  inferred from the address shape, which is wrong for carrier-grade NAT, wrong for IPv6, and
  wrong for a routable address behind a closed firewall.
- **site** — where the machine physically is, or nothing if it roams.
- **role** — hub or not, **declared**. Today it is inferred from an address prefix, which means a
  renumbering is an outage and nothing can be asked which node is the hub.

**Keys.** Each node generates its own keypair. **The private key never leaves the machine**; the
public key is published to the mesh. This is already true and it is already right — it is
[ADR 0036](../../02-DECISIONS/0036-a-node-and-how-it-joins.md)'s *a node holds its own
identity* applied to the overlay, and it means the control plane computes a graph it cannot
itself impersonate.

**Shape: a hub, with direct peering between co-located nodes.**

| | |
|---|---|
| two nodes at the same site | peer **directly**, host-routed, with a keepalive |
| everything else | routes through the **hub** |
| a node with no site — it roams | **hub only** |

**Roaming is hub-only deliberately, and the reason is a property of WireGuard rather than a
preference: there is no failover.** A more specific route to a dead endpoint blackholes; it does
not fall back to the general one. So a node whose location changes gets exactly one path, because
two paths would mean one of them silently swallowing traffic.

**What the host receives:** an interface configuration and a peer list, as files. It does not
compute them, and after this it holds no credential to the mesh's database.

## 2 — Resolution

**Two name spaces, and they do not mix:**

| | resolves to | certified by |
|---|---|---|
| **internal names** | overlay addresses | the **mesh CA** |
| **public names** | whatever the outside world must reach | a **public authority** |

A node's mesh name is its overlay address. Its public name, if it has one, is a separate fact
used by things outside the mesh — and the separation carries two lessons that were learned
expensively enough to be worth restating:

- **Mesh names are not multicast names.** A name resolved by local multicast discovery introduces
  a delay and a failure mode that appears on one node and not others — the worst shape a fault
  can have.
- **A node must not pin its own public name locally.** The duplicate record breaks resolution of
  that name for everything else that needs it.

**What the host receives:** the resolver's configuration, as files, listing every peer's internal
name and overlay address.

**What goes away:** the `/etc/hosts` floor. It exists because a node had to reach the mesh
database before its own DNS existed; with [ADR 0036](../../02-DECISIONS/0036-a-node-and-how-it-joins.md)
nothing needs a name before the link, and a fallback nothing needs is a path nothing tests.

## 3 — Exposure

Settled by [ADR 0049](../../02-DECISIONS/0049-connectivity.md); summarised here because
this is where it belongs.

**A route is a grant.** A module that must be reachable declares it needs one; the proxy provides
it and hands back the public name. Ordinary
[ADR 0044](../../02-DECISIONS/0044-modules-and-the-graph.md)
vocabulary — the mirror of a database grant, where the consumer supplies a target and receives a
name rather than supplying nothing and receiving credentials.

**A workload on an unreachable node is proxied by a reachable one, across the overlay.** Which is
the case is a mesh-level fact, which is the fourth reason exposure is control-plane work.

## 4 — Filtering

**Derived from what is assigned here, and from the overlay's shape** — a node's open ports are a
consequence of what runs on it and who must reach it, not an independent declaration to keep in
step by hand.

**A rule names its source** ([ADR 0049](../../02-DECISIONS/0049-connectivity.md)).
A rule with no source is open, and must say so rather than appear to restrict something. `scope:`
is removed rather than implemented: five manifests carry it today, it is referenced by no code,
and it is the clearest instance in the repository of *an unenforced rule is indistinguishable
from a wrong one, and costs more, because people believe it.*

**Unknown keys are refused** — the discipline the host's declaration parser already has
([ADR 0037](../../02-DECISIONS/0037-the-node-host.md)), and
the one manifests lack. `scope:` survived because nothing rejected it.

## 5 — Certificates

**Two authorities, kept separate on purpose.**

| | issued by | for |
|---|---|---|
| **public names** | a public ACME authority | anything outside the mesh reaches |
| **internal names** | the **mesh CA** | node-to-node, over the overlay |

**The split is not collapsed, including in the lab.** A single-CA lab would hide any bug living
in the split, so the lab runs its own ACME issuer on its public segment and keeps the mesh CA
unchanged ([research 004](../../01-RESEARCH/004-lab-network/00-overview.md)).

**Public issuance requires genuine public reachability.** The HTTP-01 challenge must be answered
at the name being certified, so issuance happens through a publicly reachable node regardless of
where the workload runs — the same asymmetry as exposure, for the same reason.

**The issuer must be configurable.** Today it is not: the proxy sets no `caServer` and therefore
defaults to the public authority's *production* endpoint. Two consequences, and the second is
worse than the lab problem that found it — every certificate experiment on a real node consumes
production issuance quota, and a retry loop can exhaust it for a week.

**The mesh CA is not a bootstrap concern.** A joining node verifies the control plane against the
fingerprint in its token ([ADR 0036](../../02-DECISIONS/0036-a-node-and-how-it-joins.md)),
so nothing needs the CA before membership. It certifies internal names afterwards, and that is
all it does.

## What this removes

The list is worth having in one place, because it is most of the argument:

- **The last two direct database connections from nodes** — `wireguard` and `traefik`, the only
  two, both connectivity.
- **Therefore the database credential on every node**, and the object-store credential beside it.
  [ADR 0036](../../02-DECISIONS/0036-a-node-and-how-it-joins.md)'s central claim becomes
  true rather than aspirational.
- **The `/etc/hosts` floor**, and the bootstrap circularity it patched.
- **Hub election by address prefix**, and the silent no-hub failure when nobody knew the
  convention.
- **The RFC1918 inference**, and the lab substitution that existed to satisfy it.
- **`scope:`**, and the class of manifest key that means nothing.

## Open

- ~~**What happens when the hub is down.**~~ **Resolved** by
  [ADR 0048](../../02-DECISIONS/0048-the-substrate-and-the-control-plane.md), together with `06`'s
  matching question — they were one question. Nothing takes over. WireGuard has no failover, the
  hub is declared rather than elected, and non-co-located paths stop while co-located direct peers
  and every already-assigned workload keep running. The recovery path is restore, and its deadline
  is certificate renewal.
- **Renumbering the overlay.** Made *possible* by declaring the hub rather than inferring it from
  an address, but no procedure exists, and a graph delivered node by node has an ordering problem
  while it is half-applied.
- **Revoking a route** when a module is unassigned ([ADR 0049](../../02-DECISIONS/0049-connectivity.md)).
  A stale public name pointing at nothing fails more visibly than a stale grant.
- **IPv6.** [ADR 0049](../../02-DECISIONS/0049-connectivity.md) makes
  it expressible; nothing here says the overlay or the resolver handle it.
- **Reporting declared-versus-observed.** ADR 0050 makes the disagreement detectable and does not
  say who looks or what they are told.