novox/hq
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Resolve the ingress gap: a route is a grant

Browse Source 
ADR 0048 named ingress as an unclosed hole -- nothing said what terminates
TLS, how a public name reaches a container, or which tier owned it. Resolving
it needed no new concepts, which is why it survived: nobody had applied the
rules already written to it.

Ingress is not substrate. The control plane does not need a route to start,
and no node needs one to reach it -- the node dials out and has no listening
control surface. It grants itself a route afterwards, like a bucket.

A route is an instantiation edge under ADR 0044. The direction mirrors a
database -- the consumer supplies a target and receives a name rather than
credentials -- but it is the same edge.

The substantive finding is that exposure is three facts at two scopes: name
resolution and certificate issuance need to know which node is publicly
reachable, and only the proxy mapping is a single machine's business. That is
why it belongs to the connectivity context, and why Traefik doing all three on
the node is wrong.

Which matters beyond tidiness: research 006 counted traefik as one of two
modules opening a direct Postgres connection, reading nodes and mesh_ca. That
violates 0037, 0045 and 0039 at once, and is why every node permanently holds
a credential to the control plane's database. Deriving the config centrally and
delivering it as `file` resources removes it, costs zero new host vocabulary,
and closes the set 0039 identified -- wireguard was the other.

Left open deliberately: the mesh's internal CA is the other thing traefik
reads, and it belongs to the link's mutual authority, not to exposure.
Conflating the two is what made the gap hard to see.

Also fixes an inconsistency from the previous commit: 06 still claimed the
virtual host was raised from the bundle.

Proposed, not accepted -- for review.
This commit is contained in:
jschoubben
2026-08-27 00:22:52 +02:00
parent 4d19e93900
commit 8d9282d86b
4 changed files with 162 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
02-DECISIONS/0048-the-substrate-is-named.md
+5 -7
View File
@@ -51,14 +51,12 @@ correction applies — a role is a legitimate abstraction, but the product belon
|---|---|---|
| **the forge** | **Gitea** | a hosted workload — the mesh builds from it but does not need it to run |
| **the coordinator** | the mesh's own pipeline | tier 2 — part of the control plane, not a product |
| **ingress** — *exposure*, *certificates* | **Traefik** | see below |
| **ingress** — *exposure*, *certificates* | **Traefik** | not substrate — [ADR 0049](0049-a-route-is-a-grant.md) |
**Ingress is a real gap rather than a naming one, and this record does not close it.** The
connectivity context lists *exposure* and *certificates* among its responsibilities, and no
design document says what terminates TLS, how a route reaches a container, or which tier that
belongs to. Traefik is what does it today. Whether it is substrate turns on the same test — can
the control plane grant itself a route? — and nobody has applied the test. **Named here so the
gap is visible; left open because naming it is not answering it.**
**Ingress was a real gap rather than a naming one**, and it is closed by
[ADR 0049](0049-a-route-is-a-grant.md): applying the same test shows it is **not** substrate, and
a route is an ordinary grant. Recorded here because finding it was the point — naming the
products is what made the unnamed role visible.
**Identity is deliberately absent.** Whether an identity provider is substrate at all depends on
whether the control plane delegates authentication, which is undecided
02-DECISIONS/0049-a-route-is-a-grant.md
+147
View File
@@ -0,0 +1,147 @@
---
status: proposed
date: 2026-08-27
deciders: jochen
reconstructed: false
extends: 0044-a-module-declares-presence-instantiation-and-exclusion.md
---
# 49. A route is a grant
## Context
[ADR 0048](0048-the-substrate-is-named.md) named the substrate's products and, in doing so,
found a hole rather than a naming problem: the `connectivity` context lists *exposure* and
*certificates* among its responsibilities, and **no document says what terminates TLS, how a
public name reaches a container, or which tier owns any of it.**
Traefik is what does it today, and how it does it is the problem.
[Research 006](../01-RESEARCH/006-mesh-from-scratch/host-size.md) counted it as one of only two
modules that **open a direct Postgres connection to the control plane's database** — reading
`nodes` and `mesh_ca` and computing its own configuration from them.
That is three violations in one module:
- **[ADR 0037](0037-the-host-applies-it-does-not-decide.md)** — it decides, on the node, from
mesh-wide knowledge.
- **[ADR 0045](0045-a-context-owns-its-store.md)** — it reads another context's tables directly.
- **[ADR 0039](0039-the-link-is-the-security-boundary.md)** — it is the reason every node
permanently holds a credential to the control plane's database.
So exposure was never designed; it was accreted, and it is one of the two things standing
between the current arrangement and the security boundary ADR 0039 describes.
## Decision
### Ingress is not substrate
The test from [`07-the-substrate.md`](../03-DESIGN/01-to-be/07-the-substrate.md), applied
honestly:
| | |
|---|---|
| Does the control plane need a route to **start**? | **No.** It listens locally. |
| Does a **node** need one to reach it? | **No** — the node dials out over AMQP, and has no listening control surface at all ([ADR 0039](0039-the-link-is-the-security-boundary.md)). |
| Does anything need one **before the control plane runs**? | **No.** |
**So ingress is an ordinary provider module**, provisioned like anything else once a mesh exists.
The strongest objection deserves stating rather than dodging: the control plane's `api` is the
one interface every surface speaks to, so eventually it *does* want a public name. But *wanting
one later* is not *needing one to start* — that distinction is the entire substrate test, and
ingress is on the ordinary side of it. At bootstrap the first node's surface is reached locally,
and the mesh grants itself a route afterwards, the same way it grants itself a bucket.
### A route is an instantiation edge
A module that must be reachable declares it needs a **route**, and the proxy module provides
one. This is [ADR 0044](0044-a-module-declares-presence-instantiation-and-exclusion.md)'s
instantiation edge with no extension: a provider makes something for a consumer and hands back
an identity.
The direction is worth noticing, because it is the mirror of a database and could be mistaken
for a different kind of thing:
| | a database grant | a route grant |
|---|---|---|
| consumer supplies | nothing | where to send traffic |
| consumer receives | credentials | **the public name it is reachable at** |
Both are still *the provider made something and told the consumer how to use it*, which is what
the edge means. Nothing new is required.
### Exposure is three facts at two scopes, and that is why it is a context
The reason this cannot simply live on the node:
| The fact | Scope | Whose |
|---|---|---|
| the public name resolves to an address | **mesh** — which node is publicly reachable | the **control plane**, `connectivity` |
| a certificate valid for that name exists | **mesh** — issued once, for a name, used on one node | the **control plane**, `connectivity` |
| the proxy maps that name to that container | **node** | the **host**, applying a declaration |
Two of the three need to know about more than one node, which is exactly
[the control plane's definition](../03-DESIGN/01-to-be/06-the-control-plane.md). The third is a
single machine's business. The line falls where the tier rule already puts it, and the current
arrangement is wrong precisely because Traefik does all three on the node.
### The proxy reads files; it does not read the mesh
The connectivity context computes the proxy's configuration and the certificate, and they arrive
over the link as **`file` resources**.
**This costs zero new host vocabulary.** `file` already exists — it was one of the first three
shapes built. The proxy becomes a `container` with `file` configuration, which the host already
knows how to apply and read back.
And it removes a database credential from every node, which is half of what
[ADR 0039](0039-the-link-is-the-security-boundary.md) is for. The same move was already made for
`wireguard`; this applies it to the other offender.
### A node without a public address is routed through one that has
Most nodes sit behind a connection with no forwarded port
([research 004](../01-RESEARCH/004-lab-network/00-overview.md)), so exposure cannot assume the
workload's node is reachable. Two cases, and the mesh must handle both because the difference is
invisible until it matters:
- **the node is publicly reachable** — the proxy runs there and the route is direct;
- **it is not** — a publicly reachable node proxies to it across the overlay.
Which case applies is a mesh-level fact, which is the fourth reason exposure is control-plane
work. Research 004 already records the hard variant — *a node that is publicly named but sits
behind NAT* — as the case the lab exists to get right.
## Consequences
- **Traefik's upward dependency is removed, and the module mostly disappears.** It computed its
own configuration; now it is an image plus files somebody else derived. Of the 426 lines
research 006 counted, what survives is a declaration.
- **One of two remaining database credentials leaves the node.** `wireguard` was the other, and
it is already handled — so this closes the set that ADR 0039 identified.
- **Certificate issuance becomes a control-plane responsibility with a real constraint**: an
ACME challenge can only be answered at a publicly reachable address, so issuance happens
through a public node regardless of where the workload runs. The lab already runs its own
issuer, so this is testable ([research 004](../01-RESEARCH/004-lab-network/00-overview.md)).
- **The proxy is a presence edge for anything exposed.** A module with a route needs the proxy on
its node — ordinary ADR 0044 vocabulary, no special case.
- **This does not settle the mesh's internal CA.** `mesh_ca` is the *other* thing Traefik reads,
and it belongs to a different question: ADR 0039 requires the control plane to prove it is the
mesh, which needs something a joining node can verify before it trusts anything. **Internal
identity and public exposure are two certificate stories and this record only closes the
second.** Conflating them is what made the gap hard to see.
- **Nothing says how a route is revoked** when a module is unassigned. The grant model implies
it — removing a consumer drops what it was granted
([ADR 0045](0045-a-context-owns-its-store.md)) — but a stale public name pointing at nothing is
a more visible failure than a stale database, and it is not designed.
## References
- [ADR 0044](0044-a-module-declares-presence-instantiation-and-exclusion.md) — the edge a route is.
- [ADR 0037](0037-the-host-applies-it-does-not-decide.md) — why the proxy may not decide.
- [ADR 0039](0039-the-link-is-the-security-boundary.md) — the credential this removes.
- [ADR 0048](0048-the-substrate-is-named.md) — which named this gap and left it open.
- [Research 006](../01-RESEARCH/006-mesh-from-scratch/host-size.md) — the count and the two
offenders.
- [Research 004](../01-RESEARCH/004-lab-network/00-overview.md) — the topology and the lab's
own issuer.
03-DESIGN/01-to-be/06-the-control-plane.md
+8 -3
View File
@@ -49,7 +49,7 @@ Ten contexts and one interface, from the skeleton
| **record** | the event log every other context integrates through |
| **inventory** | nodes, modules, assignments, versions |
| **config** | settings, secrets, and deriving them onto nodes |
| **connectivity** | overlay, resolution, exposure, filtering, certificates |
| **connectivity** | overlay, resolution, exposure, filtering, certificates — it *decides* routes; the proxy on a node applies them ([ADR 0049](../../02-DECISIONS/0049-a-route-is-a-grant.md)) |
| **provisioning** | resource grants between modules |
| **delivery** | source to artifact to node |
| **observability** | health, logs, metrics, alerts |
@@ -84,11 +84,16 @@ own.** It needs a PostgreSQL database, an AMQP virtual host, and a bucket — th
module needs, granted the same way.
That is the circularity the tiers exist to resolve rather than hide: the control plane cannot
provision its own database, because it is not running yet. So its store and its virtual host are
raised from the bundle the host carries, before there is a control plane to ask
provision its own database, because it is not running yet. So its **store** is raised from the
bundle the host carries, before there is a control plane to ask
([ADR 0038](../../02-DECISIONS/0038-a-node-joins-by-linking-first.md),
[research 011](../../01-RESEARCH/011-the-module-graph/worked-provider.md)).
Its virtual host and its bucket are **not** in the bundle — by the time they are wanted there is
a control plane to grant them. Whether the bus must come first is
[open](07-the-substrate.md#open), and it turns on whether these contexts talk to each other over
it.
## Where it runs
**On nodes, like anything else.** It is not a place outside the mesh; it is modules the mesh
03-DESIGN/01-to-be/07-the-substrate.md
+2
View File
@@ -10,6 +10,7 @@ decisions:
- 02-DECISIONS/0046-the-installer-fetches-what-it-pins.md
- 02-DECISIONS/0047-the-bundle-may-carry-actions-the-link-may-not.md
- 02-DECISIONS/0048-the-substrate-is-named.md
- 02-DECISIONS/0049-a-route-is-a-grant.md
---
# The substrate
@@ -36,6 +37,7 @@ The test, applied:
| an object store — **MinIO** | artifacts and blobs it delivers | no — it needs a bucket to hold them | **substrate** |
| an image registry — **the OCI registry** | images it delivers to nodes | no — it needs a repository | **substrate** |
| an identity provider | only if it delegates authentication | — | **conditional, below** |
| ingress — **Traefik** | not to start; only to be reached by name | — it grants itself one afterwards | **not substrate** ([ADR 0049](../../02-DECISIONS/0049-a-route-is-a-grant.md)) |
| anything else the mesh hosts | no | — | not substrate |
**The role and the product are both written**, here and everywhere