112 lines
8.6 KiB
Markdown
112 lines
8.6 KiB
Markdown
# route-proxy — the mesh's public ingress
|
|
|
|
The shipping form of the reference reverse proxy (novox/hq ADR 0007, novox/hq 08-connectivity §3
|
|
"Exposure"). It is the `traefik` replacement: the module that **provides `route`**.
|
|
|
|
## What it is
|
|
|
|
**A route is a provision** (issue 774, research 072 row 34; the operator, 2026-10-31: "there is only 1 path, the other
|
|
is simply deprecated/invalid"). A module that must be reachable declares `requires: ["route"]` and contributes the
|
|
name it wants and the port it listens on; this module provides `route`. The controller composes each consumer's plan,
|
|
and the seat `mesh-provisioner` keeps it in its state `plans`: the plan's entry for the route names this proxy as
|
|
its provider and carries what the consumer contributes, composed (`given`: its names joined to the node's domains, its
|
|
endpoint's port, its policy), and where the consumer's machine is on the private network (`consumer-at`). The proxy
|
|
follows the plans on the bus and forwards by the `Host` header of each request to `http://<consumer-at>:<port>`, or
|
|
over loopback for a consumer on its own machine.
|
|
|
|
The controller does not write the proxy's configuration. It composes the facts into the plans and the proxy turns them
|
|
into whatever it runs. That boundary is why swapping this proxy for another costs nothing to the modules that publish
|
|
through it. **There is no received file and no second source:** a route no plan names is not served, and a module
|
|
declaring `receives.route` is refused by the catalogue check (mesh-catalog `checks/credentialform`), but for the
|
|
migration scaffolding still listed there.
|
|
|
|
| field | value | why |
|
|
|---|---|---|
|
|
| `provides` | `route` (scope `mesh`, answered by the helper) | a module on any node may require it |
|
|
| `serves.route` | `{}` | a route hands back a name, not a credential — the consumer already knows the name it asked for |
|
|
| `reads` | `mesh-provisioner.plans`, `mesh-provisioner.provisions` | the plans are where the routes are; the helper reads both |
|
|
| `listens` | `80` and `443`, both `from: anywhere` | the one machine with a public opening; the firewall opens these for free (ADR 0045) |
|
|
|
|
Until the bus answers, the proxy routes nothing and says so on its log; it refuses to start without its broker
|
|
credential (`MESH_BROKER_FILE`). Who is inside the private network, and so may be served an internal-only name, is
|
|
still read from its membership (novox/hq ADR 0167).
|
|
|
|
## Which issuer: Let's Encrypt, stated here (novox/hq ADR 0226)
|
|
|
|
**The public issuer is this module's own fact, not a provision.** Until 2026-10-06 it came from a
|
|
second module, `public-acme`, that ran nothing and offered `acme-ca` pointing at Let's Encrypt; every
|
|
node running a proxy was assigned it too, and nothing else ever consumed it. It is retired: the proxy
|
|
names the production directory in its own `acme.env`, and the only issuer it is still bound to is the
|
|
mesh's own (`internal-acme-ca`, from `step-ca`).
|
|
|
|
**`acme.env` is byte for byte what the binding rendered.** The proxy keeps each authority's account
|
|
and certificates in a directory under `/var/lib/route-proxy/acme` named after a digest of the
|
|
directory URL — `https://acme-v02.api.letsencrypt.org:443/directory`, port included, as the binding
|
|
spelled it — and of the root bundle the `trust` container copies in. Any change to either is a new
|
|
authority to the proxy: a new account, and every routed name ordered again against Let's Encrypt's
|
|
rate limits. So the URL keeps the `:443`, `ACME_ROOTS_PATH` stays empty, and the `trust` artifact's
|
|
pinned image is not moved without a decision; mesh-controller's
|
|
`TestRouteProxyKeepsItsPublicAccountDirectory` holds all three.
|
|
|
|
## How it ships the Go proxy
|
|
|
|
The proxy is a Go program, unlike the TypeScript tool-runtime modules. Its source sits in this module, at
|
|
`cmd/route-proxy`, in the module's own Go module (`go.mod`, with a `vendor/` directory), the contract written as
|
|
something that runs. No image is built for it (novox/hq ADR 0480): the `server` artifact is a Go bundle compiled
|
|
for alpine, an `archive` resource unpacks it onto the machine, and the `route-proxy` container runs the pinned
|
|
`trust` image (the same alpine digest whose system roots the public account directory is named after) with the
|
|
archive mounted read-only at `/opt/mesh` and `/opt/mesh/route-proxy` as its program. A new build of the program
|
|
restarts the container, because the archive is in its `restart-on`.
|
|
|
|
In the lab, a scenario stocks the same alpine image and serves it by digest.
|
|
|
|
## Certificates, and the endpoint (resolves 04-ISSUES/004)
|
|
|
|
TLS is opt-in and terminates here for public names, via ACME (autocert). A proxy with no
|
|
`TLS_LISTEN` serves plain HTTP; with it, it obtains a publicly-trusted certificate for exactly the
|
|
names the mesh routes here (`HostPolicy` = only routed names, so a scan cannot spend the account's
|
|
issuance quota) and answers the HTTP-01 challenge on `:80` at the name being certified.
|
|
|
|
**`ACME_DIRECTORY` defaults to Let's Encrypt *staging*, not production.** This closes
|
|
novox/hq 04-ISSUES/004 (certificate issuance targets production):
|
|
the issuer is a per-node choice, and the safe endpoint is the default. Production issuance is
|
|
rate-limited per domain and per account and does not replenish quickly; defaulting to production
|
|
would leave the safe path depending on somebody remembering to opt out of it, on exactly the work
|
|
most likely to iterate. A staging certificate is trusted by no browser, so the mistake announces
|
|
itself on the first request rather than a fortnight later at the rate limit.
|
|
|
|
**The module states production.** The binary's staging default stands for anything that runs it
|
|
without the module — the lab, a developer's machine — and the module's `acme.env` names production
|
|
for every node it is assigned to, because a node assigned the public proxy is one that serves public
|
|
traffic.
|
|
|
|
| env | default | meaning |
|
|
|---|---|---|
|
|
| `MESH_BROKER_FILE` | *(required)* | the proxy's broker credential: the plans it routes from, and its membership |
|
|
| `LISTEN` | `:80` | HTTP, and the ACME HTTP-01 challenge |
|
|
| `TLS_LISTEN` | `:443` | HTTPS; unset to serve plain HTTP only |
|
|
| `ACME_CACHE` | `/acme` | where issued certificates persist; required when `TLS_LISTEN` is set, so a restart does not re-order |
|
|
| `ACME_DIRECTORY` | Let's Encrypt **staging** in the binary; production in the module's `acme.env` | the issuer |
|
|
| `ACME_CA_BUNDLE` | *(unset)* | a file of roots to trust for the issuer's own API — set only for a private/lab authority whose API certificate the world does not yet trust |
|
|
|
|
## Answered by the provider helper (issue 792)
|
|
|
|
A small Go program beside the proxy (`cmd/route-provider`) runs the SDK's provider helper for `route`, with no credential,
|
|
and the offer says `"answered-by": "helper"`: a handed consumer (gitea's origins file) is filled by its machine's client
|
|
from this module's entry, which carries the public names the controller composed for it. The module is **not** marked
|
|
moved: its own files, `acme.env` and the broker secret, are composed as before, and the proxy itself is unchanged.
|
|
|
|
## What it says on the bus, and its tools (issue 774)
|
|
|
|
The proxy keeps two states, announced readable (`conn.Keeps`, novox/hq ADR 0495), each keyed `<node>.<name>`:
|
|
|
|
| state | one key per | holds | written when |
|
|
|---|---|---|---|
|
|
| `routes` | served name | the rules it is served by (path, module, node, target, refusal, redirect, authentication); its certificate (authority, issuer, not-after, last renewal, outcome, or why it failed); the last upstream answer (status, time, error, path) | a route arrives, changes or leaves; an authority stores a certificate; a handshake fails to find one or hands out one not yet recorded; an upstream answer differs from the last one written, or the same one is a minute old |
|
|
| `unrouted` | name answered "no route for", in a request or a TLS handshake, under a domain the proxy serves | when it was last asked, how often since the start, whether it is an internal-only name asked from outside; never who asked (the journal keeps that, for fail2ban) | first asked, at most five new names a minute, then at most once a minute; deleted when the name is served; kept a week. Every other refusal is only counted, in one key per proxy (`<node>._other`: the last minute's count and the total), written once a minute. A refused write is dropped, never retried |
|
|
|
|
Nothing is written on a timer. The provider helper's binary serves the module's tools from those states, one call each:
|
|
`route_proxy_routes` (every served name with its target, certificate expiry and last answer), `route_proxy_route`
|
|
(one name in full, or that it is not served and when it was last asked for) and `route_proxy_unrouted` (the names
|
|
refused, newest first).
|