diff --git a/02-DECISIONS/0048-the-substrate-is-named.md b/02-DECISIONS/0048-the-substrate-is-named.md index 1652214..632a3ba 100644 --- a/02-DECISIONS/0048-the-substrate-is-named.md +++ b/02-DECISIONS/0048-the-substrate-is-named.md @@ -51,14 +51,12 @@ correction applies — a role is a legitimate abstraction, but the product belon |---|---|---| | **the forge** | **Gitea** | a hosted workload — the mesh builds from it but does not need it to run | | **the coordinator** | the mesh's own pipeline | tier 2 — part of the control plane, not a product | -| **ingress** — *exposure*, *certificates* | **Traefik** | see below | +| **ingress** — *exposure*, *certificates* | **Traefik** | not substrate — [ADR 0049](0049-a-route-is-a-grant.md) | -**Ingress is a real gap rather than a naming one, and this record does not close it.** The -connectivity context lists *exposure* and *certificates* among its responsibilities, and no -design document says what terminates TLS, how a route reaches a container, or which tier that -belongs to. Traefik is what does it today. Whether it is substrate turns on the same test — can -the control plane grant itself a route? — and nobody has applied the test. **Named here so the -gap is visible; left open because naming it is not answering it.** +**Ingress was a real gap rather than a naming one**, and it is closed by +[ADR 0049](0049-a-route-is-a-grant.md): applying the same test shows it is **not** substrate, and +a route is an ordinary grant. Recorded here because finding it was the point — naming the +products is what made the unnamed role visible. **Identity is deliberately absent.** Whether an identity provider is substrate at all depends on whether the control plane delegates authentication, which is undecided diff --git a/02-DECISIONS/0049-a-route-is-a-grant.md b/02-DECISIONS/0049-a-route-is-a-grant.md new file mode 100644 index 0000000..05871da --- /dev/null +++ b/02-DECISIONS/0049-a-route-is-a-grant.md @@ -0,0 +1,147 @@ +--- +status: proposed +date: 2026-08-27 +deciders: jochen +reconstructed: false +extends: 0044-a-module-declares-presence-instantiation-and-exclusion.md +--- + +# 49. A route is a grant + +## Context + +[ADR 0048](0048-the-substrate-is-named.md) named the substrate's products and, in doing so, +found a hole rather than a naming problem: the `connectivity` context lists *exposure* and +*certificates* among its responsibilities, and **no document says what terminates TLS, how a +public name reaches a container, or which tier owns any of it.** + +Traefik is what does it today, and how it does it is the problem. +[Research 006](../01-RESEARCH/006-mesh-from-scratch/host-size.md) counted it as one of only two +modules that **open a direct Postgres connection to the control plane's database** — reading +`nodes` and `mesh_ca` and computing its own configuration from them. + +That is three violations in one module: + +- **[ADR 0037](0037-the-host-applies-it-does-not-decide.md)** — it decides, on the node, from + mesh-wide knowledge. +- **[ADR 0045](0045-a-context-owns-its-store.md)** — it reads another context's tables directly. +- **[ADR 0039](0039-the-link-is-the-security-boundary.md)** — it is the reason every node + permanently holds a credential to the control plane's database. + +So exposure was never designed; it was accreted, and it is one of the two things standing +between the current arrangement and the security boundary ADR 0039 describes. + +## Decision + +### Ingress is not substrate + +The test from [`07-the-substrate.md`](../03-DESIGN/01-to-be/07-the-substrate.md), applied +honestly: + +| | | +|---|---| +| Does the control plane need a route to **start**? | **No.** It listens locally. | +| Does a **node** need one to reach it? | **No** — the node dials out over AMQP, and has no listening control surface at all ([ADR 0039](0039-the-link-is-the-security-boundary.md)). | +| Does anything need one **before the control plane runs**? | **No.** | + +**So ingress is an ordinary provider module**, provisioned like anything else once a mesh exists. + +The strongest objection deserves stating rather than dodging: the control plane's `api` is the +one interface every surface speaks to, so eventually it *does* want a public name. But *wanting +one later* is not *needing one to start* — that distinction is the entire substrate test, and +ingress is on the ordinary side of it. At bootstrap the first node's surface is reached locally, +and the mesh grants itself a route afterwards, the same way it grants itself a bucket. + +### A route is an instantiation edge + +A module that must be reachable declares it needs a **route**, and the proxy module provides +one. This is [ADR 0044](0044-a-module-declares-presence-instantiation-and-exclusion.md)'s +instantiation edge with no extension: a provider makes something for a consumer and hands back +an identity. + +The direction is worth noticing, because it is the mirror of a database and could be mistaken +for a different kind of thing: + +| | a database grant | a route grant | +|---|---|---| +| consumer supplies | nothing | where to send traffic | +| consumer receives | credentials | **the public name it is reachable at** | + +Both are still *the provider made something and told the consumer how to use it*, which is what +the edge means. Nothing new is required. + +### Exposure is three facts at two scopes, and that is why it is a context + +The reason this cannot simply live on the node: + +| The fact | Scope | Whose | +|---|---|---| +| the public name resolves to an address | **mesh** — which node is publicly reachable | the **control plane**, `connectivity` | +| a certificate valid for that name exists | **mesh** — issued once, for a name, used on one node | the **control plane**, `connectivity` | +| the proxy maps that name to that container | **node** | the **host**, applying a declaration | + +Two of the three need to know about more than one node, which is exactly +[the control plane's definition](../03-DESIGN/01-to-be/06-the-control-plane.md). The third is a +single machine's business. The line falls where the tier rule already puts it, and the current +arrangement is wrong precisely because Traefik does all three on the node. + +### The proxy reads files; it does not read the mesh + +The connectivity context computes the proxy's configuration and the certificate, and they arrive +over the link as **`file` resources**. + +**This costs zero new host vocabulary.** `file` already exists — it was one of the first three +shapes built. The proxy becomes a `container` with `file` configuration, which the host already +knows how to apply and read back. + +And it removes a database credential from every node, which is half of what +[ADR 0039](0039-the-link-is-the-security-boundary.md) is for. The same move was already made for +`wireguard`; this applies it to the other offender. + +### A node without a public address is routed through one that has + +Most nodes sit behind a connection with no forwarded port +([research 004](../01-RESEARCH/004-lab-network/00-overview.md)), so exposure cannot assume the +workload's node is reachable. Two cases, and the mesh must handle both because the difference is +invisible until it matters: + +- **the node is publicly reachable** — the proxy runs there and the route is direct; +- **it is not** — a publicly reachable node proxies to it across the overlay. + +Which case applies is a mesh-level fact, which is the fourth reason exposure is control-plane +work. Research 004 already records the hard variant — *a node that is publicly named but sits +behind NAT* — as the case the lab exists to get right. + +## Consequences + +- **Traefik's upward dependency is removed, and the module mostly disappears.** It computed its + own configuration; now it is an image plus files somebody else derived. Of the 426 lines + research 006 counted, what survives is a declaration. +- **One of two remaining database credentials leaves the node.** `wireguard` was the other, and + it is already handled — so this closes the set that ADR 0039 identified. +- **Certificate issuance becomes a control-plane responsibility with a real constraint**: an + ACME challenge can only be answered at a publicly reachable address, so issuance happens + through a public node regardless of where the workload runs. The lab already runs its own + issuer, so this is testable ([research 004](../01-RESEARCH/004-lab-network/00-overview.md)). +- **The proxy is a presence edge for anything exposed.** A module with a route needs the proxy on + its node — ordinary ADR 0044 vocabulary, no special case. +- **This does not settle the mesh's internal CA.** `mesh_ca` is the *other* thing Traefik reads, + and it belongs to a different question: ADR 0039 requires the control plane to prove it is the + mesh, which needs something a joining node can verify before it trusts anything. **Internal + identity and public exposure are two certificate stories and this record only closes the + second.** Conflating them is what made the gap hard to see. +- **Nothing says how a route is revoked** when a module is unassigned. The grant model implies + it — removing a consumer drops what it was granted + ([ADR 0045](0045-a-context-owns-its-store.md)) — but a stale public name pointing at nothing is + a more visible failure than a stale database, and it is not designed. + +## References + +- [ADR 0044](0044-a-module-declares-presence-instantiation-and-exclusion.md) — the edge a route is. +- [ADR 0037](0037-the-host-applies-it-does-not-decide.md) — why the proxy may not decide. +- [ADR 0039](0039-the-link-is-the-security-boundary.md) — the credential this removes. +- [ADR 0048](0048-the-substrate-is-named.md) — which named this gap and left it open. +- [Research 006](../01-RESEARCH/006-mesh-from-scratch/host-size.md) — the count and the two + offenders. +- [Research 004](../01-RESEARCH/004-lab-network/00-overview.md) — the topology and the lab's + own issuer. diff --git a/03-DESIGN/01-to-be/06-the-control-plane.md b/03-DESIGN/01-to-be/06-the-control-plane.md index 5a4ff82..4b7b66f 100644 --- a/03-DESIGN/01-to-be/06-the-control-plane.md +++ b/03-DESIGN/01-to-be/06-the-control-plane.md @@ -49,7 +49,7 @@ Ten contexts and one interface, from the skeleton | **record** | the event log every other context integrates through | | **inventory** | nodes, modules, assignments, versions | | **config** | settings, secrets, and deriving them onto nodes | -| **connectivity** | overlay, resolution, exposure, filtering, certificates | +| **connectivity** | overlay, resolution, exposure, filtering, certificates — it *decides* routes; the proxy on a node applies them ([ADR 0049](../../02-DECISIONS/0049-a-route-is-a-grant.md)) | | **provisioning** | resource grants between modules | | **delivery** | source to artifact to node | | **observability** | health, logs, metrics, alerts | @@ -84,11 +84,16 @@ own.** It needs a PostgreSQL database, an AMQP virtual host, and a bucket — th module needs, granted the same way. That is the circularity the tiers exist to resolve rather than hide: the control plane cannot -provision its own database, because it is not running yet. So its store and its virtual host are -raised from the bundle the host carries, before there is a control plane to ask +provision its own database, because it is not running yet. So its **store** is raised from the +bundle the host carries, before there is a control plane to ask ([ADR 0038](../../02-DECISIONS/0038-a-node-joins-by-linking-first.md), [research 011](../../01-RESEARCH/011-the-module-graph/worked-provider.md)). +Its virtual host and its bucket are **not** in the bundle — by the time they are wanted there is +a control plane to grant them. Whether the bus must come first is +[open](07-the-substrate.md#open), and it turns on whether these contexts talk to each other over +it. + ## Where it runs **On nodes, like anything else.** It is not a place outside the mesh; it is modules the mesh diff --git a/03-DESIGN/01-to-be/07-the-substrate.md b/03-DESIGN/01-to-be/07-the-substrate.md index ef61c03..216ae21 100644 --- a/03-DESIGN/01-to-be/07-the-substrate.md +++ b/03-DESIGN/01-to-be/07-the-substrate.md @@ -10,6 +10,7 @@ decisions: - 02-DECISIONS/0046-the-installer-fetches-what-it-pins.md - 02-DECISIONS/0047-the-bundle-may-carry-actions-the-link-may-not.md - 02-DECISIONS/0048-the-substrate-is-named.md + - 02-DECISIONS/0049-a-route-is-a-grant.md --- # The substrate @@ -36,6 +37,7 @@ The test, applied: | an object store — **MinIO** | artifacts and blobs it delivers | no — it needs a bucket to hold them | **substrate** | | an image registry — **the OCI registry** | images it delivers to nodes | no — it needs a repository | **substrate** | | an identity provider | only if it delegates authentication | — | **conditional, below** | +| ingress — **Traefik** | not to start; only to be reached by name | — it grants itself one afterwards | **not substrate** ([ADR 0049](../../02-DECISIONS/0049-a-route-is-a-grant.md)) | | anything else the mesh hosts | no | — | not substrate | **The role and the product are both written**, here and everywhere