Merge pull request 'ADR 0196: a node asks the mesh's resolver first, and a public one only when it is silent' (#330) from decision/0196-nodes-ask-the-mesh-resolver-with-a-public-fallback into main

2026-10-03 20:01:05 +00:00
parent 577ddf0089 f5d518d256
commit 50ddaf8408
4 changed files with 126 additions and 13 deletions
@@ -11,6 +11,14 @@ extends: 0191-the-meshs-resolver-holds-only-the-meshs-own-names.md

 # 194. The mesh has one resolver, and every node asks it for the mesh's names

+> **Narrowed, not replaced — 2026-10-03.** How a node asks is decided again by
+> [ADR 0196](0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md): every
+> node and container asks `mesh-resolver` first and a public resolver only when it is silent. There is
+> no `systemd-resolved` module and no runtime `dns` naming `mesh-resolver`, and step 2 of the migration
+> reads as 0196 states it. Option 2 below was rejected for a laptop with its tunnel down resolving
+> nothing; a public resolver listed second answers exactly then. The one resolver, its placement and
+> the retirement of every per-node copy stand.
+
 ## Context

 **Every node runs its own resolver and holds its own copy of the mesh's names.** On the production
@@ -0,0 +1,98 @@
+---
+topic: the tiers
+status: accepted
+date: 2026-10-03
+deciders: jochen
+reconstructed: false
+supersedes-in-part:
+  - 0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md
+---
+
+# 196. A node asks the mesh's resolver first, and a public one only when it is silent
+
+## Context
+
+**[ADR 0194](0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md) gave the
+mesh one resolver and had each node ask it for the mesh's names only.** Because `resolv.conf` cannot
+route by domain, that needed a stub on every node — a `systemd-resolved` module — and a separate
+`dns` for the container runtime, which cannot use a loopback stub. It rejected the simpler shape,
+every node sending every query to the mesh's resolver, on the grounds that *"a laptop whose tunnel is
+down could resolve nothing at all."*
+
+**That is true only of a `resolv.conf` naming the mesh's resolver alone.** The C library asks the
+servers it lists in order and moves to the next when one does not answer within its timeout. A public
+resolver listed second is asked exactly when the mesh's is unreachable — the anchor down, the tunnel
+down, a laptop behind a captive portal that has not let the tunnel up — and never otherwise. An answer
+from the first, including "no such name", is final, so `.internal` is never asked of a public resolver
+while the mesh's answers.
+
+**And the container runtime copies a machine's resolvers into its containers when they are not
+loopback addresses.** With the mesh's resolver and a public one listed, every container gets both, as
+they are, with nothing configured for the runtime.
+
+## Considered Options
+
+**1. Keep ADR 0194's stub.** Public names never touch the mesh, and a node with the anchor down
+resolves public names at full speed. It costs a module and a running service on every node, a second
+configuration for containers, and the one asymmetry ADR 0194 had to state — containers' public names
+through the mesh, nodes' not.
+
+**2. Every node asks the mesh's resolver for everything, with a public resolver as the silent
+fallback.** One server answers every node and every container; nothing on a node routes, holds names,
+or runs. Chosen.
+
+## Decision
+
+**A node's `/etc/resolv.conf` names the mesh's resolver first and a public resolver second, with a
+short timeout and a single attempt.** It is written by the module holding `node-resolver-config` — the
+existing `resolv-conf` — which now names `mesh-resolver`'s address instead of the machine's own. The
+mesh's resolver answers the mesh's names from what it holds and forwards every other name, giving the
+public answer ([ADR 0191](0191-the-meshs-resolver-holds-only-the-meshs-own-names.md) is unchanged: no
+public name gets a private answer).
+
+**Containers take the same two resolvers from their machine.** The container runtime's own `dns`
+setting is not written; the runtime copies the machine's non-loopback resolvers into every container.
+
+**This replaces, from ADR 0194:** the asking side as a stub (*"So the asking side is a
+`systemd-resolved` module"*), the container runtime's `dns` naming `mesh-resolver`, and step 2 of the
+migration as written. There is no `systemd-resolved` module. Everything else in ADR 0194 stands — one
+`mesh-resolver`, on the node every tunnel converges on, holding each node's internal domain, the
+retirement of `node-dns-resolver` and every per-node copy, and a LAN's resolver not being the mesh's.
+
+**The migration, as it now reads:**
+
+1. `mesh-resolver` is assigned and answers on the private network.
+2. Each node's `resolv-conf` names `mesh-resolver` first and a public resolver second.
+3. A LAN whose router points at a node's resolver is pointed at its router.
+4. `node-dns-resolver` is unassigned from every node, and the hosts region is withdrawn.
+
+## Consequences
+
+- **Every name a node or container asks goes through the anchor while it is up.** A public lookup
+  takes a few milliseconds longer than asking a public resolver directly, and the mesh's resolver sees
+  every name its nodes look up. It is the operator's own server.
+- **With the anchor unreachable, each lookup waits out one timeout, then resolves publicly.**
+  `.internal` names fail then — as `.internal` traffic does, every tunnel going through the anchor.
+- **A LAN is unaffected by this choice.** Devices that are not members never read a node's
+  `resolv.conf`; they get their resolver from their router, which step 3 points at itself.
+- **Nothing new runs on a node.** No stub, no module, no per-node configuration for containers.
+- **The runtime's `dns` key goes with `node-dns-resolver`.** The dnsmasq module wrote it into the
+  runtime's configuration; unassigning that module in step 4 withdraws it, and the runtime reads the
+  change only when it next starts — with `live-restore` on, that restart keeps every container
+  running.
+
+**How each is checked:**
+
+- **Order:** each node's `/etc/resolv.conf` lists `mesh-resolver`'s private address first and a public
+  resolver second, and nothing else.
+- **Fallback:** with `mesh-resolver` unreachable from a node, a public name still resolves there, after
+  the timeout.
+- **Containers:** a container started on a node lists the same two resolvers.
+- **A LAN:** the router's DHCP DNS option names the router, not a node.
+
+## References
+
+- [ADR 0194](0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md) — the one
+  resolver; this record replaces how nodes and containers ask it.
+- [ADR 0191](0191-the-meshs-resolver-holds-only-the-meshs-own-names.md) — what the resolver holds.
+- [Connectivity §2](../03-DESIGN/01-to-be/08-connectivity.md), amended alongside this record.
@@ -224,6 +224,7 @@ python3 00-META/checks/index.py            fail if stale
 - **0151** — [A route's internal name is composed under the node that serves it](0151-a-routes-internal-name-is-composed-under-the-node-that-serves-it.md)
 - **0191** — [The mesh's resolver holds only the mesh's own names; a public name resolves publicly](0191-the-meshs-resolver-holds-only-the-meshs-own-names.md)
 - **0194** — [The mesh has one resolver, and every node asks it for the mesh's names](0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md)
+- **0196** — [A node asks the mesh's resolver first, and a public one only when it is silent](0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md)

 ### What runs on them, and how it gets there

@@ -10,6 +10,7 @@ code:
  - mesh-host internal/apply (the service that reflects a rule set)
 updated: 2026-10-03
 decisions:
+  - 02-DECISIONS/0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md
  - 02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md
  - 02-DECISIONS/0191-the-meshs-resolver-holds-only-the-meshs-own-names.md
  - 02-DECISIONS/0180-the-found-front-end-is-uninstalled-once-a-machine-is-converged.md
@@ -292,8 +293,9 @@ expensively enough to be worth restating:
  that name for everything else that needs it.

 **What the host receives:** what to ask, not what to answer. The mesh has **one resolver**, holding
-every node's internal domain; a node routes the mesh's suffix to it and every other name to public
-resolvers ([ADR 0194](../../02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md)).
+every node's internal domain; a node asks it first and a public resolver only when it is silent
+([ADR 0194](../../02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md),
+[ADR 0196](../../02-DECISIONS/0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md)).

 **What goes away:** the `/etc/hosts` floor. It exists because a node had to reach the mesh
 database before its own DNS existed; with [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md)
@@ -355,21 +357,25 @@ machine — declared or not — is what a nameserver in `resolv.conf` would be f

 *2026-10-03.* **The mesh's names live in one place: the module holding `mesh-resolver`**, a mesh-scoped
 seat of capacity one, placed on the node every tunnel converges on. It holds one wildcard per node —
-`<node>.internal` and everything under it — and listens on the private network only. Every node's
-`node-resolver-config` routes the mesh's suffix to it and leaves every other name with public
-resolvers; plain `resolv.conf` cannot route by domain, so the asking side is a stub that can — a
-`systemd-resolved` module claiming `node-resolver-config` in place of `resolv-conf`, routing the
-suffix to `mesh-resolver`. The container runtime cannot use a loopback stub, so its `dns` names
-`mesh-resolver`, which forwards public names for containers — the one place a public name passes
-through the mesh
-([ADR 0194](../../02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md)).
+`<node>.internal` and everything under it — and listens on the private network only. It answers the
+mesh's names from what it holds and forwards every other name, giving the public answer.
+
+**Every node asks it for everything, and a public resolver only when it is silent.** The module
+holding `node-resolver-config` writes `/etc/resolv.conf` naming `mesh-resolver` first and a public
+resolver second, with a short timeout and one attempt: the C library moves to the second only when the
+first does not answer — the anchor or the tunnel down, a captive portal holding the tunnel back — so
+public names keep resolving then, and `.internal` is never asked of a public resolver while the mesh's
+answers. Containers take the same two from their machine, the runtime copying non-loopback resolvers
+into every container, so the runtime is given no `dns` of its own
+([ADR 0196](../../02-DECISIONS/0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md),
+replacing ADR 0194's per-node `systemd-resolved` stub).

 **No node holds a copy.** The per-node resolver, its zones file and the mesh's region of `/etc/hosts`
 go: every resolution fault found on 2026-10-03 was a copy disagreeing with the truth — a hosts file
 read once at start, an operator's old line beside the mesh's, a node's resolver lent to a LAN. No
-member's resolver answers a LAN; a router pointing at one is moved first. *Checked by `resolvectl` on
-each node (the tunnel's link, `mesh-resolver`, the suffix as routing domain), by no node but the
-holder answering DNS on a private or LAN address, and by the router's DHCP DNS option naming no node.*
+member's resolver answers a LAN; a router pointing at one is moved first. *Checked by each node's
+`/etc/resolv.conf` naming `mesh-resolver` then a public resolver, by no node but the holder answering
+DNS on any address, and by the router's DHCP DNS option naming the router.*

 *What follows describes the per-node resolver this replaces — how it was built and why the roles were
 split. The split stands; the serving role's scope is what moved.*