diff --git a/02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md b/02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md index 08c5935..f128413 100644 --- a/02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md +++ b/02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md @@ -11,6 +11,14 @@ extends: 0191-the-meshs-resolver-holds-only-the-meshs-own-names.md # 194. The mesh has one resolver, and every node asks it for the mesh's names +> **Narrowed, not replaced — 2026-10-03.** How a node asks is decided again by +> [ADR 0196](0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md): every +> node and container asks `mesh-resolver` first and a public resolver only when it is silent. There is +> no `systemd-resolved` module and no runtime `dns` naming `mesh-resolver`, and step 2 of the migration +> reads as 0196 states it. Option 2 below was rejected for a laptop with its tunnel down resolving +> nothing; a public resolver listed second answers exactly then. The one resolver, its placement and +> the retirement of every per-node copy stand. + ## Context **Every node runs its own resolver and holds its own copy of the mesh's names.** On the production diff --git a/02-DECISIONS/0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md b/02-DECISIONS/0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md new file mode 100644 index 0000000..e07bc6a --- /dev/null +++ b/02-DECISIONS/0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md @@ -0,0 +1,98 @@ +--- +topic: the tiers +status: accepted +date: 2026-10-03 +deciders: jochen +reconstructed: false +supersedes-in-part: + - 0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md +--- + +# 196. A node asks the mesh's resolver first, and a public one only when it is silent + +## Context + +**[ADR 0194](0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md) gave the +mesh one resolver and had each node ask it for the mesh's names only.** Because `resolv.conf` cannot +route by domain, that needed a stub on every node — a `systemd-resolved` module — and a separate +`dns` for the container runtime, which cannot use a loopback stub. It rejected the simpler shape, +every node sending every query to the mesh's resolver, on the grounds that *"a laptop whose tunnel is +down could resolve nothing at all."* + +**That is true only of a `resolv.conf` naming the mesh's resolver alone.** The C library asks the +servers it lists in order and moves to the next when one does not answer within its timeout. A public +resolver listed second is asked exactly when the mesh's is unreachable — the anchor down, the tunnel +down, a laptop behind a captive portal that has not let the tunnel up — and never otherwise. An answer +from the first, including "no such name", is final, so `.internal` is never asked of a public resolver +while the mesh's answers. + +**And the container runtime copies a machine's resolvers into its containers when they are not +loopback addresses.** With the mesh's resolver and a public one listed, every container gets both, as +they are, with nothing configured for the runtime. + +## Considered Options + +**1. Keep ADR 0194's stub.** Public names never touch the mesh, and a node with the anchor down +resolves public names at full speed. It costs a module and a running service on every node, a second +configuration for containers, and the one asymmetry ADR 0194 had to state — containers' public names +through the mesh, nodes' not. + +**2. Every node asks the mesh's resolver for everything, with a public resolver as the silent +fallback.** One server answers every node and every container; nothing on a node routes, holds names, +or runs. Chosen. + +## Decision + +**A node's `/etc/resolv.conf` names the mesh's resolver first and a public resolver second, with a +short timeout and a single attempt.** It is written by the module holding `node-resolver-config` — the +existing `resolv-conf` — which now names `mesh-resolver`'s address instead of the machine's own. The +mesh's resolver answers the mesh's names from what it holds and forwards every other name, giving the +public answer ([ADR 0191](0191-the-meshs-resolver-holds-only-the-meshs-own-names.md) is unchanged: no +public name gets a private answer). + +**Containers take the same two resolvers from their machine.** The container runtime's own `dns` +setting is not written; the runtime copies the machine's non-loopback resolvers into every container. + +**This replaces, from ADR 0194:** the asking side as a stub (*"So the asking side is a +`systemd-resolved` module"*), the container runtime's `dns` naming `mesh-resolver`, and step 2 of the +migration as written. There is no `systemd-resolved` module. Everything else in ADR 0194 stands — one +`mesh-resolver`, on the node every tunnel converges on, holding each node's internal domain, the +retirement of `node-dns-resolver` and every per-node copy, and a LAN's resolver not being the mesh's. + +**The migration, as it now reads:** + +1. `mesh-resolver` is assigned and answers on the private network. +2. Each node's `resolv-conf` names `mesh-resolver` first and a public resolver second. +3. A LAN whose router points at a node's resolver is pointed at its router. +4. `node-dns-resolver` is unassigned from every node, and the hosts region is withdrawn. + +## Consequences + +- **Every name a node or container asks goes through the anchor while it is up.** A public lookup + takes a few milliseconds longer than asking a public resolver directly, and the mesh's resolver sees + every name its nodes look up. It is the operator's own server. +- **With the anchor unreachable, each lookup waits out one timeout, then resolves publicly.** + `.internal` names fail then — as `.internal` traffic does, every tunnel going through the anchor. +- **A LAN is unaffected by this choice.** Devices that are not members never read a node's + `resolv.conf`; they get their resolver from their router, which step 3 points at itself. +- **Nothing new runs on a node.** No stub, no module, no per-node configuration for containers. +- **The runtime's `dns` key goes with `node-dns-resolver`.** The dnsmasq module wrote it into the + runtime's configuration; unassigning that module in step 4 withdraws it, and the runtime reads the + change only when it next starts — with `live-restore` on, that restart keeps every container + running. + +**How each is checked:** + +- **Order:** each node's `/etc/resolv.conf` lists `mesh-resolver`'s private address first and a public + resolver second, and nothing else. +- **Fallback:** with `mesh-resolver` unreachable from a node, a public name still resolves there, after + the timeout. +- **Containers:** a container started on a node lists the same two resolvers. +- **A LAN:** the router's DHCP DNS option names the router, not a node. + +## References + +- [ADR 0194](0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md) — the one + resolver; this record replaces how nodes and containers ask it. +- [ADR 0191](0191-the-meshs-resolver-holds-only-the-meshs-own-names.md) — what the resolver holds. +- [Connectivity §2](../03-DESIGN/01-to-be/08-connectivity.md), amended alongside this record. diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index 0ee7e90..04ff2e7 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -224,6 +224,7 @@ python3 00-META/checks/index.py fail if stale - **0151** — [A route's internal name is composed under the node that serves it](0151-a-routes-internal-name-is-composed-under-the-node-that-serves-it.md) - **0191** — [The mesh's resolver holds only the mesh's own names; a public name resolves publicly](0191-the-meshs-resolver-holds-only-the-meshs-own-names.md) - **0194** — [The mesh has one resolver, and every node asks it for the mesh's names](0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md) +- **0196** — [A node asks the mesh's resolver first, and a public one only when it is silent](0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md) ### What runs on them, and how it gets there diff --git a/03-DESIGN/01-to-be/08-connectivity.md b/03-DESIGN/01-to-be/08-connectivity.md index 8249359..98de8d2 100644 --- a/03-DESIGN/01-to-be/08-connectivity.md +++ b/03-DESIGN/01-to-be/08-connectivity.md @@ -10,6 +10,7 @@ code: - mesh-host internal/apply (the service that reflects a rule set) updated: 2026-10-03 decisions: + - 02-DECISIONS/0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md - 02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md - 02-DECISIONS/0191-the-meshs-resolver-holds-only-the-meshs-own-names.md - 02-DECISIONS/0180-the-found-front-end-is-uninstalled-once-a-machine-is-converged.md @@ -292,8 +293,9 @@ expensively enough to be worth restating: that name for everything else that needs it. **What the host receives:** what to ask, not what to answer. The mesh has **one resolver**, holding -every node's internal domain; a node routes the mesh's suffix to it and every other name to public -resolvers ([ADR 0194](../../02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md)). +every node's internal domain; a node asks it first and a public resolver only when it is silent +([ADR 0194](../../02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md), +[ADR 0196](../../02-DECISIONS/0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md)). **What goes away:** the `/etc/hosts` floor. It exists because a node had to reach the mesh database before its own DNS existed; with [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md) @@ -355,21 +357,25 @@ machine — declared or not — is what a nameserver in `resolv.conf` would be f *2026-10-03.* **The mesh's names live in one place: the module holding `mesh-resolver`**, a mesh-scoped seat of capacity one, placed on the node every tunnel converges on. It holds one wildcard per node — -`.internal` and everything under it — and listens on the private network only. Every node's -`node-resolver-config` routes the mesh's suffix to it and leaves every other name with public -resolvers; plain `resolv.conf` cannot route by domain, so the asking side is a stub that can — a -`systemd-resolved` module claiming `node-resolver-config` in place of `resolv-conf`, routing the -suffix to `mesh-resolver`. The container runtime cannot use a loopback stub, so its `dns` names -`mesh-resolver`, which forwards public names for containers — the one place a public name passes -through the mesh -([ADR 0194](../../02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md)). +`.internal` and everything under it — and listens on the private network only. It answers the +mesh's names from what it holds and forwards every other name, giving the public answer. + +**Every node asks it for everything, and a public resolver only when it is silent.** The module +holding `node-resolver-config` writes `/etc/resolv.conf` naming `mesh-resolver` first and a public +resolver second, with a short timeout and one attempt: the C library moves to the second only when the +first does not answer — the anchor or the tunnel down, a captive portal holding the tunnel back — so +public names keep resolving then, and `.internal` is never asked of a public resolver while the mesh's +answers. Containers take the same two from their machine, the runtime copying non-loopback resolvers +into every container, so the runtime is given no `dns` of its own +([ADR 0196](../../02-DECISIONS/0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md), +replacing ADR 0194's per-node `systemd-resolved` stub). **No node holds a copy.** The per-node resolver, its zones file and the mesh's region of `/etc/hosts` go: every resolution fault found on 2026-10-03 was a copy disagreeing with the truth — a hosts file read once at start, an operator's old line beside the mesh's, a node's resolver lent to a LAN. No -member's resolver answers a LAN; a router pointing at one is moved first. *Checked by `resolvectl` on -each node (the tunnel's link, `mesh-resolver`, the suffix as routing domain), by no node but the -holder answering DNS on a private or LAN address, and by the router's DHCP DNS option naming no node.* +member's resolver answers a LAN; a router pointing at one is moved first. *Checked by each node's +`/etc/resolv.conf` naming `mesh-resolver` then a public resolver, by no node but the holder answering +DNS on any address, and by the router's DHCP DNS option naming the router.* *What follows describes the per-node resolver this replaces — how it was built and why the roles were split. The split stands; the serving role's scope is what moved.*