diff --git a/00-META/glossary.md b/00-META/glossary.md index 94f35c9b..abd85179 100644 --- a/00-META/glossary.md +++ b/00-META/glossary.md @@ -366,7 +366,7 @@ becomes a message). **Decided by** ADR 0227, 0231, 0240. **Purpose.** To make every node and module reachable by name where it should be, and unreachable where it should not. **Recorded by** the controller. **Upstream of** Provisioning and Health and repair. -**Decided by** ADR 0007, 0117, 0138, 0223, 0226. +**Decided by** ADR 0007, 0117, 0138, 0223, 0226, 0247. **Uses:** node, machine, assignment and endpoint. - **private network** — the mesh's own encrypted network between its nodes, on which every node has an @@ -376,9 +376,13 @@ it should not. **Recorded by** the controller. **Upstream of** Provisioning and - **anchor** and **hub** — the roles a node plays for the private network: the anchor is reachable from outside and every node reaches it; a hub relays for nodes that cannot reach each other directly. - **resolver** — a seat holder that answers the mesh's names; a node lists only the mesh's resolvers - (ADR 0223). **uplink** — a node's connection to the outside network, a seat - ([ADR 0117](../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md)). **hostname** — a node's own name, + (ADR 0223), or, where it holds one, its own resolver, which asks them. **uplink** — a node's + connection to the outside network, a seat ([ADR 0117](../02-DECISIONS/0117-a-machines-uplink-is-a-seat.md)). **hostname** — a node's own name, a seat. +- **split DNS** — resolving names by domain on one machine: a VPN's domains through the VPN's servers + over its link, every other name through the mesh's resolvers. The provision `split-dns`, provided by + the holder of the node seat `node-resolver`, the machine's **own resolver**, which exists only where + something requires it ([ADR 0247](../02-DECISIONS/0247-a-machine-with-a-vpn-client-routes-names-by-domain-through-a-resolver-of-its-own.md)). - **proxy** and **public name** — the module that answers a public name and forwards it to an endpoint on the private network. - **packet filter** — what the mesh enforces on a node about which packets pass, the diff --git a/01-RESEARCH/033-split-dns-with-a-vpn-client/00-overview.md b/01-RESEARCH/033-split-dns-with-a-vpn-client/00-overview.md index e7ecf4ab..9aec126e 100644 --- a/01-RESEARCH/033-split-dns-with-a-vpn-client/00-overview.md +++ b/01-RESEARCH/033-split-dns-with-a-vpn-client/00-overview.md @@ -1,6 +1,9 @@ --- -status: active +status: graduated initiated: 2026-10-07 +became: + - 02-DECISIONS/0247-a-machine-with-a-vpn-client-routes-names-by-domain-through-a-resolver-of-its-own.md + - 03-DESIGN/01-to-be/50-split-dns-on-a-machine-with-a-vpn-client.md touches: - 02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md - 02-DECISIONS/0117-a-machines-uplink-is-a-seat.md @@ -67,3 +70,10 @@ Evidence gathered read-only on the laptop and options weighed, 2026-10-07: holder runs systemd-resolved as a local router on the machine's private address, and hands it the VPN's servers and domains the moment the VPN client writes them; every other machine is unchanged. A proposed decision text with how each rule is checked. + +**Graduated 2026-10-07** into [ADR 0247](../../02-DECISIONS/0247-a-machine-with-a-vpn-client-routes-names-by-domain-through-a-resolver-of-its-own.md) +and [to-be 50](../../03-DESIGN/01-to-be/50-split-dns-on-a-machine-with-a-vpn-client.md). The mechanism is +the recommendation's: systemd-resolved routing the VPN's domains over its link. Who runs it differs, by +the operator's decision. The resolver is a module of its own on a node seat, `node-resolver`, rather than +part of the uplink's holder, which has two forms. The VPN client's module carries the adapter, so no +setting declares a VPN client. The interim *defer* setting is not adopted. The record says why. diff --git a/02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md b/02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md index 75c6606f..a227d9ea 100644 --- a/02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md +++ b/02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md @@ -109,6 +109,13 @@ goes with them. One owner for the file, and it is the program that would otherwi > `node-resolver-config` and ADR 0220's dependency retire — stands. How parts 2 and 3 are checked is > in [connectivity §2](../03-DESIGN/01-to-be/08-connectivity.md). +> **The mechanism changed — 2026-10-07, by ADR 0247.** On a machine where a module holds the node seat +> `node-resolver`, which is only where something requires `split-dns` (a VPN client that writes this +> file itself), that module writes `/etc/resolv.conf` naming the machine's own resolver, and the uplink's +> holder steps back from the file ([ADR 0247](0247-a-machine-with-a-vpn-client-routes-names-by-domain-through-a-resolver-of-its-own.md)). That resolver asks the +> mesh's two resolvers for every name but the VPN's own domains. Everywhere else the uplink's holder writes +> the file as decided here. + **3. Next, decided and not yet built: a machine's names are one seat's.** `/etc/hosts` and `/etc/hostname` belong to one seat for the machine's identity. The `hosts` module, holding `node-hosts-file` ([ADR 0199](0199-a-module-that-answers-names-declares-its-zone-and-a-nodes-hosts-file-is-one-modules.md)), diff --git a/02-DECISIONS/0241-a-machine-says-how-its-network-is-and-an-outside-writer-of-a-mesh-file-is-a-finding.md b/02-DECISIONS/0241-a-machine-says-how-its-network-is-and-an-outside-writer-of-a-mesh-file-is-a-finding.md index 9cb5971b..94b8ffd7 100644 --- a/02-DECISIONS/0241-a-machine-says-how-its-network-is-and-an-outside-writer-of-a-mesh-file-is-a-finding.md +++ b/02-DECISIONS/0241-a-machine-says-how-its-network-is-and-an-outside-writer-of-a-mesh-file-is-a-finding.md @@ -135,6 +135,14 @@ link, or ask for a reconcile. The reconcile holds the file as it always has. How its names with a VPN client is [research 033](../01-RESEARCH/033-split-dns-with-a-vpn-client/00-overview.md)'s question, and is not decided here. +> **The mechanism changed — 2026-10-07, by ADR 0247.** Research 033's question is decided in +> [ADR 0247](0247-a-machine-with-a-vpn-client-routes-names-by-domain-through-a-resolver-of-its-own.md): a machine with such a VPN client runs a resolver of its +> own, whose module writes the resolver file there. The file this rule 1 judges on that machine is +> therefore that module's, and a rewrite is raised as `machine..systemd-resolved.rewritten`. Nothing in +> the judge changed. The module's guard puts its file back sooner (rule 7 binds the engine, not the +> module): at once for a write the VPN client's module took, after 90 s for one nobody took, which is long +> enough to be raised first. + **8. The uplink seat answers what the machine resolves through.** `node-uplink` serves two read-only verbs, the same from every holder whatever manages the network. `resolvers` gives the resolver file as it is: its resolvers, search domains and options, whether it is the mesh's, and who wrote it as far as the machine diff --git a/02-DECISIONS/0247-a-machine-with-a-vpn-client-routes-names-by-domain-through-a-resolver-of-its-own.md b/02-DECISIONS/0247-a-machine-with-a-vpn-client-routes-names-by-domain-through-a-resolver-of-its-own.md new file mode 100644 index 00000000..a80dbcae --- /dev/null +++ b/02-DECISIONS/0247-a-machine-with-a-vpn-client-routes-names-by-domain-through-a-resolver-of-its-own.md @@ -0,0 +1,244 @@ +--- +topic: the tiers +status: accepted +date: 2026-10-07 +deciders: jochen +reconstructed: false +extends: 02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md +--- + +# 247. A machine with a VPN client routes names by domain, through a resolver of its own + +## Context + +**Every machine lists the mesh's two resolvers in `/etc/resolv.conf`, and nothing else** +([ADR 0223](0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md)). The module holding the +machine's uplink writes that file ([ADR 0117](0117-a-machines-uplink-is-a-seat.md)). The *uplink* is the +program that manages the machine's own network connection, NetworkManager or systemd-networkd. A +*resolver* is a server that answers questions about names. + +**On the laptop, a VPN client writes the same file.** The operator's employer supplies the VPN client, +FortiClient's Linux client. When it connects, it moves the mesh's file aside and writes its own: two +servers reached through its tunnel, and eight search domains of the company's network. It never tells +systemd-resolved or NetworkManager which servers belong to which link, and it never writes the file again +during a session. [Research 033](../01-RESEARCH/033-split-dns-with-a-vpn-client/00-overview.md) measured +what followed, from the client's log and the node-engine's journal: + +- In **nine of nine** sessions the node-engine wrote the mesh's file back at its next reconcile, 1 s to + 3.5 min after the connect. From then on **no company name resolved** for the rest of the session, and + sessions lasted up to fourteen hours. +- In the minutes before that, **no mesh name resolved**. On 2026-10-07 a delivery to the laptop failed on + 37 resources, because the artifact store's name was asked of the VPN's servers, which answered "no such + host". +- **One file cannot list both sets of servers.** musl, the C library of every Alpine container, asks every + listed server at once and takes the first reply. glibc takes the first server's "no such name" as final + ([issue 262](../04-ISSUES/262-an-alpine-container-could-not-find-a-machine-by-its-mesh-name/00-report.md)). + Whatever the order, one kind of name fails. + +[ADR 0241](0241-a-machine-says-how-its-network-is-and-an-outside-writer-of-a-mesh-file-is-a-finding.md), +merged today, makes this visible: the node-engine raises `machine...rewritten`, naming +FortiClient from the file's header. It said how the laptop should share names with the VPN was research +033's question. This record answers it. + +**Research 033 recommended** routing by domain with systemd-resolved, run by the uplink's holder on a +machine that declares a VPN client, with a path watch in the same holder taking the VPN's file. **The +operator approved a different split of the same mechanism on 2026-10-07**: the resolver is its own module +on a seat of its own, the VPN client's module carries the adapter, and nothing declares a VPN client. +The options below say why. + +**Checked against GENESIS.** *Failure must be loud*: a write nothing handles is still raised. *The mesh is +a guest on a personal node*: the VPN client is the employer's and is not changed, and its domains never +leave the machine. *Do one thing in one place*: the resolver is written once, not once per uplink holder. +Nothing conflicts. + +## Considered Options + +**What routes names by domain.** + +1. *Add the VPN's servers to the mesh's file.* Rejected (research 033 option 2): it breaks the one-answer + rule above. +2. *Forward the company's domains from the mesh's resolvers.* Rejected (option 3): they cannot reach + servers that sit behind one laptop's tunnel, and the company's zones would become every machine's. +3. *Leave the VPN's file in place for the session.* Rejected (option 4): the laptop loses the mesh, the bus + included, for whole working days. Research 033 also offered this as an interim setting until routing was + built. It is not adopted: the routing is built with this record. +4. **systemd-resolved on the machine**, which sends each name to the servers of the link whose domain + matches it, and every other name to the mesh's resolvers. Chosen. dnsmasq on the machine would also work + (option 5). It costs a configuration rewrite and a reload on every connect, where resolved takes a + link's servers live and forgets them when the link goes. resolved ships with systemd, so it needs no + new package. + +**Who runs it.** + +1. *The uplink's holder*, as research 033 recommended. Rejected: there are two uplink holders, + NetworkManager's and systemd-networkd's, so the resolver would be written twice, and kept the same by + a test. And each would have to know which VPN clients exist. +2. *The node-engine.* Rejected for ADR 0223 part 3's reason: the node-engine applies every module's + resources and owns no file's content. +3. **A module of its own, holding a new node seat, `node-resolver`.** Chosen. A *node seat* is a role that + one module holds per machine. Another module could hold it on a machine without systemd. + +**Who knows about the VPN client.** + +1. *The resolver, given a setting that names the client* (research 033 rule 1). Rejected: it would grow a + branch for every VPN client. +2. **The module that wraps the client.** Chosen. The mesh's default pattern is that the module wrapping a + program owns that program's quirks. The resolver offers a generic verb: route these domains to these + servers over this link. A VPN that tells systemd-resolved its link's DNS itself needs no adapter. + NetworkManager's VPN plugins, WireGuard under systemd-networkd and Tailscale all do this. + +**How the adapter reaches the resolver.** + +1. *Through the bus, as `/node-resolver.route`.* Rejected for this caller: the VPN's servers and + domains would cross the broker on another machine. +2. **On the machine, over a socket only root can open, with the same verbs.** Chosen. The socket's path is + the seat's, so a caller does not need to know which module holds it. The same verbs are also served on + the bus, for the operator to read and correct routes. + +**What happens to the VPN client's write.** + +1. *Put the resolver's file back at once, whatever wrote it.* Rejected: a write nothing handles would end + before the node-engine looks twice, and so would never be said. +2. *Leave it to the node-engine's reconcile*, as today. Rejected: a handled write would then still cut + off the mesh's names for minutes. +3. **Keep it, put the module's file back at once when a module took it, and otherwise after 90 s.** + Chosen. 90 s is longer than two of the node-engine's 30 s looks. + +## Decision + +**1. A machine's own resolver is a node seat, `node-resolver`, held only where something requires it.** +Its first holder is the catalogue module `systemd-resolved`, which provides the provision `split-dns` at +the machine's reach. A *provision* is something one module offers another. *The machine's reach* means a +requirement is answered only by a provider on the same machine, and the provider is never pulled in by +the requirement ([ADR 0208](0208-the-graphical-session-is-one-module-per-piece-on-the-meshs-seats.md) §3). +A machine where nothing requires `split-dns` is unchanged: its uplink's holder writes the file listing the +mesh's two resolvers. + +**2. Where it is held, the resolver writes `/etc/resolv.conf`, and the uplink's holder steps back from that +file.** The rule is the controller's. On a machine where a module holds `node-resolver` and renders the +resolver file, the uplink holder's rendering of the same path is not composed. Everything else the uplink +holder declares stays. Only the uplink holder steps back: any other module writing the file beside the +resolver is still two owners of one path, and is refused as before. + +**3. The file names the machine's own private address alone, with ADR 0223's options.** resolved listens +there and on loopback. ADR 0223 rejected a local forwarder (its option 2) for three reasons, and each is +answered here: + +- *A daemon on every machine*: only on a machine that requires `split-dns`. +- *A container cannot use a loopback resolver*: the file names the private address, which a container + can reach. The packet filter admits the machine's own guests and nobody else, so no other machine can + ask this resolver. +- *A per-node copy disagreeing with the truth*: resolved keeps no cache and reads no hosts file, and every + name not routed elsewhere goes to the mesh's two resolvers. They are its default route, with no public + fallback. + +**4. The resolver's verbs route by link, and know nothing of a VPN.** + +| verb | what it does | +|---|---| +| `routes` | what is routed where, and the resolver file's outside writes | +| `route {link, domains, servers}` | send those domains, and every name under them, to those servers over that link, and only them | +| `unroute {link}` | send that link's domains back to the mesh's resolvers | + +`route` refuses the mesh's own domain and the root. A link with servers of its own is never a default +route for names, whether this verb set it or a network manager did. resolved forgets a link's route when +the link goes. The seat is new and nothing holds it, so its verbs are required from the start +(ADR 0246). + +**5. The resolver keeps its file, and keeps an outside write for whoever handles it.** Its guard runs as +root and compares the file with the module's copy twice a second. When another program has written the +file, the guard does three things: + +- It **keeps** what was written, readable by root alone and gone at the next boot, and names the writer + from the file's header. +- If a module on the machine **takes** the write (it routed what it needed and says so), the guard puts + the module's file back at once. +- Otherwise it puts the module's file back **after 90 s**. + +Of an outside write, the resolver says nothing beyond the machine except when it happened, the writer's +name and what became of it. It never says a server or a domain from the write. + +**6. The VPN client's module carries its own adapter.** The `forticlient` module requires `split-dns` and +runs an adapter, as root, that does four things: + +- It reads the client's servers and search domains from the kept write. +- It waits up to 15 s for the client's tunnel link, then routes those domains to those servers over it + and takes the write in the same call. +- It takes the route away when the tunnel goes. +- It gives the route again if the resolver restarted and forgot it. + +The client's search domains become routing domains. A short name is not completed with them. + +**7. ADR 0241's network check agrees without a change to it.** The node-engine already judges the file +that a module declares whole at that path, so on a machine with the resolver it judges the resolver's +file. A write the adapter took is undone within seconds, before the node-engine's second look, so it is +no finding. A write nothing took stands for 90 s, so the node-engine sees it twice and raises +`machine..systemd-resolved.rewritten`, naming the writer. This covers an undeclared writer, the +adapter not running, no tunnel within its wait, and a route refused. The guard then puts the file back, +and the finding clears. + +**8. The VPN's domains stay on the machine.** They are never in the mesh's store, the controller's +renders, the mesh's resolvers, an event or another machine's files. The adapter hands them over on the +machine, never over the bus. They live in resolved's per-link state for the life of the tunnel, and in the +kept write until the next boot. They cross the bus only as the answer to `routes` when the operator asks +it, and nothing keeps that answer. + +## Consequences + +- **The laptop resolves both kinds of name while the VPN is up**, and public names through the mesh's + resolvers, as before. The fight between the two writers ends: the VPN client's servers are used, for + the VPN's domains only. +- **A machine with the resolver has one more thing that can stop its names**: resolved, or its guard, + down. Both declare `unit` health, so the node-engine judges them (ADR 0240). The node-engine's names + check asks the address the file lists, which is resolved's, so a resolved that does not answer is said + there too. +- **The client's search domains do not expand short names**, because the machine's file is the mesh's + and lists no search domain. A full company name resolves. +- **Every machine that runs the `forticlient` module needs the resolver.** The module requires + `split-dns` wherever it runs, and the workstation runs it too. Either `systemd-resolved` is assigned + there as well, or the client is unassigned from the workstation, before the module's change is + delivered. Otherwise the controller refuses the workstation's composition, naming who could provide + `split-dns`. +- **The rollout has an order.** First the controller that knows the seat, which must be running, since + the catalogue's check reads manifests with the controller the mesh runs. Then the resolver module, + assigned to the machines that need it. Then the `forticlient` change. +- **Containers keep the resolvers they started with** (ADR 0223's consequence, unchanged). On the laptop a + container started before the resolver is assigned keeps the mesh's two resolvers until it restarts. That + is correct for mesh names, but it means the company's names do not reach that container. +- **To confirm live:** resolved takes no servers from `/etc/resolv.conf` once its own are configured + (`DNS=` in its drop-in). So neither a VPN client's write, nor the file's own address, should become one + of its servers. The live test reads `routes` while a write stands to confirm it. +- **Unassigning the resolver** brings the uplink holder's rendering back. The node-engine hands a whole + file from one owner to the next at the same path, so the file is not removed in between. + +## How it is checked + +| Rule | Checked by | +|---|---| +| 1 The seat, its verbs required, its holder providing `split-dns` at the machine's reach | mesh-controller `TestTheMachinesOwnResolverIsANodeSeatWithItsVerbs`, `TestTheCataloguesResolverHoldersWriteTheFileAndProvideSplitDNS`, `TestTheSeatsAreAClosedSetAndEachNamesItsDecision`; mesh-catalog systemd-resolved `TestTheManifestSaysWhatTheGuardKeeps` | +| 1 Nothing changes where it is not held | `TestWithoutTheResolverTheUplinkWritesTheFileAsBefore` | +| 2 The uplink steps back, and only the uplink | `TestWhereTheResolverIsHeldItWritesTheFileAndTheUplinkStepsBack`, `TestOnlyTheUplinkStepsBackForTheResolver`, `TestTheCataloguesUplinksWriteTheResolverFileAndNothingElseDoes` | +| 3 One address, the mesh's resolvers as default route, no fallback, no cache | `TestTheCataloguesResolverComposesOnAMachine` (the catalogue's module rendered by the controller); `TestTheManifestSaysWhatTheGuardKeeps` | +| 4 Routes by link; the mesh's domain and the root refused; no default route for a link's servers | mesh-catalog systemd-resolved `TestARouteSendsOnlyItsDomainsOverItsLink`, `TestARouteThatWouldTakeTheMeshsNamesIsRefused`, `TestOnlyTheMeshsResolversAnswerEveryName`, `TestUnrouteRevertsTheLink`, `TestRoutesReadWhatResolvedSendsWhere` | +| 5 Kept, taken, held, said without servers or domains | `TestATakenWriteIsPutBackAtOnce`, `TestAWriteNobodyTakesStandsUntilItIsSaidThenGoes`, `TestWritesUndoneAndWrittenOverAreSaid`, `TestTheVerbsOnTheMachine` (the socket root's alone) | +| 6 The adapter | mesh-catalog forticlient `TestTheClientsDomainsAreRoutedOverItsTunnelAndGoWithIt`, `TestAWriteWithNoTunnelOrNotTheClientsIsLeftToTheResolver`, `TestARouteIsKeptAcrossARestartOfEither`, `TestTheClientsFileIsReadForItsServersAndDomains`, `TestItDeclaresTheServiceAndNothingOfTheConfiguration` | +| 7 Agrees with ADR 0241 | by construction: the node-engine's `declaredResolvConf` judges the file declared whole at the path, which rule 2 makes the resolver's; the 90 s hold is held above two looks by `TestAWriteNobodyTakesStandsUntilItIsSaidThenGoes`. Live: below | +| 8 Never leaves the machine | the socket's mode in `TestTheVerbsOnTheMachine`; no server or domain in the history (`TestATakenWriteIsPutBackAtOnce`) or the adapter's journal (`TestTheClientsDomainsAreRoutedOverItsTunnelAndGoWithIt`) | +| live | on the laptop, connected: a company name and a mesh name both resolve, on the machine and in an Alpine container on a bridge; `node-resolver.routes` shows the tunnel with its domains and the write taken by `forticlient`; no `machine..….rewritten` stays raised. Disconnected: the route is gone and the file is the resolver's | + +## References + +- [Research 033](../01-RESEARCH/033-split-dns-with-a-vpn-client/00-overview.md): the evidence and the + options, and the recommendation this record changes. +- [ADR 0223](0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md), which this extends: its + rule that a machine lists only the mesh's resolvers holds everywhere but on a machine with its own + resolver, which lists itself and asks only them. +- [ADR 0241](0241-a-machine-says-how-its-network-is-and-an-outside-writer-of-a-mesh-file-is-a-finding.md): + the finding, unchanged. +- [ADR 0117](0117-a-machines-uplink-is-a-seat.md), [ADR 0207](0207-a-module-depends-on-the-node-seats-that-apply-its-resources.md) + (resolved's package is the service manager's holder's), [ADR 0240](0240-a-module-says-how-it-is-healthy-and-the-node-engine-judges-it.md), + ADR 0246. +- [To-be 50](../03-DESIGN/01-to-be/50-split-dns-on-a-machine-with-a-vpn-client.md), the design; to-be 48 + §10 and connectivity §2, amended alongside. +- mesh-controller `internal/catalogue/node_resolver.go`, `seats.go`, `declaration.go`, `resolve.go`; + mesh-catalog `modules/systemd-resolved`, `modules/forticlient`. diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index ccdca027..6cab312b 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -248,6 +248,7 @@ python3 00-META/checks/index.py fail if stale - **0199** — [A module that answers names declares its zone, and a node's hosts file is one module's](0199-a-module-that-answers-names-declares-its-zone-and-a-nodes-hosts-file-is-one-modules.md) - **0223** — [The mesh has two resolvers, and a machine lists only them](0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md) - **0226** — [The private network is assigned by its own name, and the proxy names its public issuer](0226-the-private-network-is-assigned-by-its-own-name-and-the-proxy-names-its-public-issuer.md) +- **0247** — [A machine with a VPN client routes names by domain, through a resolver of its own](0247-a-machine-with-a-vpn-client-routes-names-by-domain-through-a-resolver-of-its-own.md) ### What runs on them, and how it gets there diff --git a/03-DESIGN/01-to-be/08-connectivity.md b/03-DESIGN/01-to-be/08-connectivity.md index c15c1022..54e66aaf 100644 --- a/03-DESIGN/01-to-be/08-connectivity.md +++ b/03-DESIGN/01-to-be/08-connectivity.md @@ -16,10 +16,13 @@ code: - mesh-catalog modules/route-proxy (the public issuer named by the proxy, ADR 0226) - mesh-controller internal/overlay/generator.go (the private network's module, assigned by its own name, ADR 0226) - mesh-catalog modules/hostname (a node's /etc/hostname and /etc/hosts) + - mesh-controller internal/catalogue/node_resolver.go (a machine's own resolver, ADR 0247) + - mesh-catalog modules/systemd-resolved (a machine's own resolver, ADR 0247) - mesh-host internal/identity/serving.go - mesh-host internal/apply (the service that reflects a rule set; a whole file handed to its new owner) -updated: 2026-10-06 +updated: 2026-10-07 decisions: + - 02-DECISIONS/0247-a-machine-with-a-vpn-client-routes-names-by-domain-through-a-resolver-of-its-own.md - 02-DECISIONS/0226-the-private-network-is-assigned-by-its-own-name-and-the-proxy-names-its-public-issuer.md - 02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md - 02-DECISIONS/0220-what-a-machine-asks-needs-its-uplink-held-and-the-retired-resolver-pieces-go.md @@ -319,6 +322,14 @@ holding every node's internal domain; a node lists both and nothing else ([ADR 0194](../../02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md), [ADR 0223](../../02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md)). +**A machine with a VPN client that writes the resolver file itself runs a resolver of its own** +([ADR 0247](../../02-DECISIONS/0247-a-machine-with-a-vpn-client-routes-names-by-domain-through-a-resolver-of-its-own.md), +[to-be 50](50-split-dns-on-a-machine-with-a-vpn-client.md)). Its file names that resolver, on the +machine's private address so its containers reach it, and the resolver asks the mesh's two resolvers for +every name except the VPN's own domains, which it sends to the VPN's servers over the VPN's link. One +server is listed, so there is still one answer per name. It is installed only where something requires +`split-dns`; everywhere else the file is as above. + **What goes away:** the `/etc/hosts` floor. It exists because a node had to reach the mesh database before its own DNS existed; with [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md) nothing needs a name before the link, and a fallback nothing needs is a path nothing tests. diff --git a/03-DESIGN/01-to-be/26-the-seats.md b/03-DESIGN/01-to-be/26-the-seats.md index 5445ae10..626aba48 100644 --- a/03-DESIGN/01-to-be/26-the-seats.md +++ b/03-DESIGN/01-to-be/26-the-seats.md @@ -148,6 +148,7 @@ convention, which later seats departed from. | `mesh-resolver` | — | mesh | — | the mesh's resolvers, each holding every node's internal domain ([ADR 0194](../../02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md)); replicated, held on the anchor and the home server ([ADR 0223](../../02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md)) | | ~~`mesh-dns-port`~~ | `the-dns-port` | node | — | retired by [ADR 0194](../../02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md): the local resolver became the mesh's one; deleted from the set, and from the store's table, once nothing claimed it ([ADR 0220](../../02-DECISIONS/0220-what-a-machine-asks-needs-its-uplink-held-and-the-retired-resolver-pieces-go.md)) | | `node-hostname` | `node-hosts-file` (renamed by [ADR 0223](../../02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md); the former name an alias) | node | — | owns a machine's names: `/etc/hostname`, written from its holder's `hostname` setting with no default and taking effect at the next boot, and in `/etc/hosts` the machine's own lines and the operator's kept region, changed through its verbs `entries`, `add`, `remove` ([ADR 0199](../../02-DECISIONS/0199-a-module-that-answers-names-declares-its-zone-and-a-nodes-hosts-file-is-one-modules.md)). Held by the module `hostname`, formerly `hosts` | +| `node-resolver` | — | node | — | a machine's own resolver, held only where something requires `split-dns`: it writes `/etc/resolv.conf` there (the uplink's holder steps back from it) and routes a VPN's domains to the VPN's servers over its link through its verbs `routes`, `route`, `unroute` ([ADR 0247](../../02-DECISIONS/0247-a-machine-with-a-vpn-client-routes-names-by-domain-through-a-resolver-of-its-own.md)). Held by the module `systemd-resolved` | | `mesh-intrusion-prevention` | `the-intrusion-prevention` | node | — | an intrusion-prevention service | | `mesh-packet-filter` | `the-packet-filter` | node | — | the packet filter | | `mesh-private-network` | `the-private-network` | node | — | the private network the mesh runs over | diff --git a/03-DESIGN/01-to-be/48-a-module-says-how-it-is-healthy.md b/03-DESIGN/01-to-be/48-a-module-says-how-it-is-healthy.md index 5d70d045..7458ce7f 100644 --- a/03-DESIGN/01-to-be/48-a-module-says-how-it-is-healthy.md +++ b/03-DESIGN/01-to-be/48-a-module-says-how-it-is-healthy.md @@ -6,6 +6,7 @@ updated: 2026-10-07 decisions: - 02-DECISIONS/0240-a-module-says-how-it-is-healthy-and-the-node-engine-judges-it.md - 02-DECISIONS/0241-a-machine-says-how-its-network-is-and-an-outside-writer-of-a-mesh-file-is-a-finding.md + - 02-DECISIONS/0247-a-machine-with-a-vpn-client-routes-names-by-domain-through-a-resolver-of-its-own.md - 02-DECISIONS/0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md - 02-DECISIONS/0227-the-core-holds-nine-rules-each-checked-and-is-built-to-them-in-six-phases.md --- @@ -259,6 +260,12 @@ the machine itself, beside its liveness looks, and says it in the same statement *wait*. The machine's own network fault holds the machine as a whole ([issue 281](../../04-ISSUES/281-a-tier-sent-one-module-at-a-time-blamed-a-module-for-its-machine/00-report.md)). - **It reads and never acts** (§7). The reconcile writes the file back as it always has. +- **On a machine with its own resolver** ([ADR 0247](../../02-DECISIONS/0247-a-machine-with-a-vpn-client-routes-names-by-domain-through-a-resolver-of-its-own.md), + [to-be 50](50-split-dns-on-a-machine-with-a-vpn-client.md)) the file judged is the resolver module's, because + that is the module declaring it whole there; the uplink's holder steps back from it. A VPN client's write + that the client's module took is undone within seconds, before the second look, and is no finding. One + nothing took stands for 90 s, so it is raised as `machine..systemd-resolved.rewritten`, naming its + writer, and cleared when the resolver puts its file back. Nothing in the judge changes for this. - **Asked further through the uplink seat.** `node-uplink` serves `resolvers` (the file, whether it is the mesh's, its writer) and `links` (each link, its default route, the resolvers its manager knows). They are the same from every holder, and optional until both holders serve them: promised, so a holder may serve them, diff --git a/03-DESIGN/01-to-be/50-split-dns-on-a-machine-with-a-vpn-client.md b/03-DESIGN/01-to-be/50-split-dns-on-a-machine-with-a-vpn-client.md new file mode 100644 index 00000000..9c4f1426 --- /dev/null +++ b/03-DESIGN/01-to-be/50-split-dns-on-a-machine-with-a-vpn-client.md @@ -0,0 +1,151 @@ +--- +layer: to-be +status: in-progress +code: + - mesh-controller internal/catalogue/node_resolver.go (the seat, its verbs, the uplink stepping back) + - mesh-controller internal/catalogue/seats.go, declaration.go, resolve.go + - mesh-catalog modules/systemd-resolved (the resolver, its guard and its verbs) + - mesh-catalog modules/forticlient (the VPN client's adapter) +updated: 2026-10-07 +decisions: + - 02-DECISIONS/0247-a-machine-with-a-vpn-client-routes-names-by-domain-through-a-resolver-of-its-own.md + - 02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md + - 02-DECISIONS/0241-a-machine-says-how-its-network-is-and-an-outside-writer-of-a-mesh-file-is-a-finding.md +--- + +# 50 — Split DNS on a machine with a VPN client + +**A machine whose VPN client pushes servers of its own runs a resolver of its own. That resolver sends the +VPN's domains to the VPN's servers over the VPN's link, and every other name to the mesh's two resolvers. +It is a module on a node seat, installed only where something requires it, and it knows nothing of any +VPN. The VPN client's module reads what the client pushed and hands it over.** +([ADR 0247](../../02-DECISIONS/0247-a-machine-with-a-vpn-client-routes-names-by-domain-through-a-resolver-of-its-own.md), +from [research 033](../../01-RESEARCH/033-split-dns-with-a-vpn-client/00-overview.md).) + +*Split DNS* is resolving names by domain: one set of domains through one set of servers, the rest through +another. A *routing domain* is a domain whose names a resolver sends to one link's servers only. + +## The parts + +``` + a machine where something requires split-dns every other machine: unchanged + + /etc/resolv.conf ── nameserver ──┐ /etc/resolv.conf, the uplink's + written by the resolver's module; │ holder's: both mesh resolvers + the uplink's holder steps back from it ▼ + containers ─── reach the private address ──► systemd-resolved + ├─ VPN's domains ──► VPN's servers, over the tunnel link + └─ every other name ─► the mesh's two resolvers (~.) + ▲ + │ route {link, domains, servers, takes} (a socket on the machine, root only) + │ + the VPN client's module: its adapter ◄── the kept write ── the resolver's guard ◄── the client writes + (puts its own file back once taken, + or after 90 s) +``` + +| part | where | does | +|---|---|---| +| the seat `node-resolver` | the controller's set | a node seat, its verbs `routes`, `route`, `unroute`, required of every holder | +| the stepping back | the controller's composition | where a module holds `node-resolver` and renders `/etc/resolv.conf`, the uplink holder's rendering of it is left out | +| `systemd-resolved` | the catalogue | holds the seat, provides `split-dns` at the machine's reach, writes the file and resolved's drop-in, runs the guard | +| the guard | `systemd-resolved`'s process, as root | keeps the file the module's, keeps an outside write, serves the verbs on the machine's socket | +| the adapter | `forticlient`'s process, as root | reads the client's write, routes its domains over its tunnel, takes the write, takes the route away when the tunnel goes | + +## 1. Where it is installed + +Only where something requires `split-dns`. The provision has the machine's reach: a requirement is +answered by a provider on the same machine or not at all, and the provider is never pulled in. A module +that wraps a VPN client which writes the resolver file itself requires it. Today that is `forticlient`, +which runs on the laptop and the workstation, so the resolver is assigned wherever the client is: the +laptop, and the workstation unless the client is unassigned there. A machine with no such +module keeps ADR 0223's file, written by its uplink's holder. + +The resolver claims port 53 on its machine, fixed, which the mesh's own resolver also claims. The two do +not share a machine. + +## 2. The file and the resolver + +- **The file** names the machine's private address alone, with ADR 0223's options, and begins as every + mesh resolver file does, so the uplink's `resolvers` verb and the node-engine read it as the mesh's. +- **The uplink's holder steps back** from that path on that machine, by the controller's rule. It does not + write the file and lose a race; it is never given the file to write. +- **resolved** is given every mesh resolver as its global servers, with `~.` so they answer every name no + link's own domains route elsewhere. It has no public fallback, no cache, no hosts file, no LLMNR or + multicast DNS, and it listens on loopback and on the private address. +- **A container** reaches the private address, as a container on a mesh resolver's machine already does. + The packet filter admits the machine's own guests and nobody else. +- **resolved's package** is systemd's, and belongs to the service manager's holder (ADR 0207). + +## 3. The verbs, on the mesh and on the machine + +`routes`, `route {link, domains, servers}` and `unroute {link}`. They are served twice from one +implementation. On the mesh, through the node's runtime as the operator account, they let the operator +read and correct routes. On the machine, over a socket only root can open at the seat's own path, they +serve the machine's modules, so what a VPN pushed never crosses the bus. On the machine, `displaced` +answers the write standing now, and `route` can also `take` it. + +The routes are resolved's own per-link state, which resolved forgets when the link goes. The module keeps +no table beside it. A link with servers of its own is never a default route for names: the guard says so +to resolved every 5 s, whoever gave the link its servers. Its routing domains stay its own. + +## 4. An outside write + +``` + write seen ──► kept (root only), writer named from its header + │ + ├─ a module takes it (route … takes) ──► module's file back within a second no finding + │ + └─ nobody takes it ──► stands 90 s ──► node-engine sees it twice: rewritten, raised + machine..systemd-resolved.rewritten + ──► module's file back ──► next look clears it +``` + +The history the `routes` verb answers says when, the writer's name and what became of each write. It +never says a server or a domain. + +## 5. The adapter: the client's quirk is the client's module's + +FortiClient's Linux client writes the resolver file on connect, never re-asserts it, and never tells +resolved a link's DNS. Its module's adapter looks once a second: + +1. a write stands, and its header says the client wrote it: it reads the servers, and the `search` and + `domain` lines; +2. it waits up to 15 s for the client's tunnel link to be up; +3. it routes the domains to the servers over that link, taking the write in the same call; +4. when the tunnel goes, it takes the route away; every 10 s it checks the route is still there and gives + it again if the resolver restarted. + +A write that is not the client's, no tunnel in time, nothing to route, or a route refused, is left to the +resolver's hold, and is therefore raised. A VPN that tells resolved its link's DNS itself needs no adapter. + +## 6. What it costs and what it does not do + +- The client's search domains route full names. They do not complete short ones. +- resolved down is no names on that machine. Its `unit` health, and the node-engine's names check against + the address the file lists, say so. +- resolved should take no servers from the resolver file, because its own are configured. The live test + confirms this while a write stands. +- A container started before the resolver keeps the mesh's two resolvers until it restarts. + +## Phases + +| Phase | Repository | Delivers | Done when | +|---|---|---|---| +| A — the seat | mesh-controller | `node-resolver` and its verbs; the uplink stepping back | its tests pass; the controller is running on the mesh | +| B — the resolver | mesh-catalog `systemd-resolved` | the module, its guard and verbs | its tests pass; assigned to every machine that runs the VPN client's module, each one's file names its own address, `routes` answers | +| C — the adapter | mesh-catalog `forticlient` | `split-dns` required; the adapter | its tests pass; on the laptop, connected, a company name and a mesh name both resolve and no rewrite stays raised | + +## As built + +One pull request in mesh-controller (phase A), and two in mesh-catalog (B, then C on top of it). None is +merged. What the build chose where ADR 0247 left it open: + +- **The kept copy.** The mesh renders the module's file twice, at `/etc/resolv.conf` and beside resolved's + configuration. The guard compares the live file with the copy and puts the copy back, so it needs nothing + but what the mesh declared. The copy's fact sorts first, so the mesh writes it before the live file. +- **The look.** The guard reads the file twice a second rather than watching it. The file is small, and a + rename onto it (the client moves the old file aside) needs no special case. +- **The mesh's own domain** reaches the module as a rendered file, so `route` refuses it whatever the mesh + calls itself. +- **The socket protocol** is one JSON line each way: `{"verb", "args"}`, then `{"result"}` or `{"error"}`. diff --git a/03-DESIGN/01-to-be/README.md b/03-DESIGN/01-to-be/README.md index d4276d9a..e7f746f4 100644 --- a/03-DESIGN/01-to-be/README.md +++ b/03-DESIGN/01-to-be/README.md @@ -49,6 +49,7 @@ document is written and this one's status becomes `implemented`. | [`46-the-conversation-with-the-operator.md`](46-the-conversation-with-the-operator.md) | **Designed.** The mesh tells and asks its operator over channels that are holders of two kinded benches, `channel` and `intake`, declaring capabilities from a fixed vocabulary; the router orders them by work context and never lowers the bar; an answer that performs an action is checked and performed by the controller, on a TOTP code or a verified Telegram sender, never on a desk click alone; operator messages addressed or answered by the mesh's responder, untrusted input kept as data, references for what words may not carry, and a recoverable factor; Telegram first, in six phases | [ADR 0234](../../02-DECISIONS/0234-the-mesh-holds-a-conversation-with-its-operator.md), [ADR 0227](../../02-DECISIONS/0227-the-core-holds-nine-rules-each-checked-and-is-built-to-them-in-six-phases.md) | | [`47-delivery-from-commit-to-delivered.md`](47-delivery-from-commit-to-delivered.md) | **Designed.** A delivery is one commit in one repository, from its pull request's head to every machine; a delivery group is deliveries sharing a branch name, ordered and checked as one future state; both owned by the `mesh-delivery` module with one state table, its state on the bus, every transition said, noted on the commit and shown on the pull request; the controller keeps the planner, the gate, sending and the walk, and the core's own updates never wait for the module | [ADR 0239](../../02-DECISIONS/0239-a-delivery-is-owned-by-the-mesh-delivery-module-and-runs-from-commit-to-delivered.md), [ADR 0238](../../02-DECISIONS/0238-a-commit-is-the-build-at-hand-one-commit-one-change-plan-checked-off-the-trunk-and-published-only-on-it.md), [ADR 0236](../../02-DECISIONS/0236-a-build-is-judged-on-its-first-machine-and-put-back-by-something-other-than-itself-and-so-it-rolls-out-unattended.md) | | [`49-the-mesh-in-domains.md`](49-the-mesh-in-domains.md) | **In progress.** Every concept belongs to one of ten domains, which owns its one word; the glossary is organised by them and is the authority, a retired word is named on its replacement's line with its scope, and two checks hold this repository's documents and the catalogue's tool descriptions to it | [ADR 0244](../../02-DECISIONS/0244-the-mesh-is-described-in-domains-and-one-word-names-one-thing.md), [ADR 0006](../../02-DECISIONS/0006-the-substrate-and-the-control-plane.md), [ADR 0008](../../02-DECISIONS/0008-a-context-owns-its-store.md) | +| [`50-split-dns-on-a-machine-with-a-vpn-client.md`](50-split-dns-on-a-machine-with-a-vpn-client.md) | **In progress.** A machine whose VPN client pushes servers of its own runs a resolver of its own, on a node seat, that routes the VPN's domains over its link and every other name to the mesh's resolvers; the VPN client's module carries the adapter | [ADR 0247](../../02-DECISIONS/0247-a-machine-with-a-vpn-client-routes-names-by-domain-through-a-resolver-of-its-own.md), [ADR 0223](../../02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md) | ## Not yet written