systemd-resolved: a machine's own resolver, routing a VPN's domains by link (hq ADR 0247)
mesh/merge-gate fail: builds new: modules/systemd-resolved, sent nowhere; no bus step; a manifest the change touches fails the module check: modules/system…
mesh/repo-check fail: its merge-check.sh failed: long-running resources without health: 70
mesh/delivery superseded: a newer head of the same pull request

Holds node-resolver and provides split-dns at the machine's reach, for a
machine whose VPN client pushes resolvers of its own. It writes the
resolver file naming the machine's private address, gives resolved the
mesh's resolvers as the default route, and serves routes, route and unroute
on the mesh and, over a root-only socket, on the machine. Its guard keeps an
outside write of the file for the module that handles it and puts the
module's file back: at once when taken, after 90 s otherwise, so a write
nothing declared to handle is still raised by the node-engine.
This commit is contained in:
jochen
2026-10-07 21:28:11 +02:00
parent 68aeee2c4e
commit d53571213c
10 changed files with 1643 additions and 0 deletions
+77
View File
@@ -0,0 +1,77 @@
# systemd-resolved
A machine's own resolver (novox/hq ADR 0247): systemd-resolved, holding the node seat `node-resolver` and
providing `split-dns` at the machine's reach. It is assigned only where something on the machine requires
`split-dns` — today a VPN client whose company domains must go to the VPN's own servers while every other
name still goes to the mesh's two resolvers. Every other machine has none, and lists the mesh's resolvers
in its resolver file as ADR 0223 says.
**It knows nothing about any VPN.** It routes a set of domains to a set of servers over one link, lists
what is routed, and takes a route away. A module wrapping a VPN client that writes `/etc/resolv.conf`
itself carries its own adapter, which calls these verbs. A VPN that tells systemd-resolved its link's DNS
itself (NetworkManager's VPNs, WireGuard set up by networkd, Tailscale) needs none.
## What it writes
| file | what |
|---|---|
| `/etc/resolv.conf` | this machine's own private address alone, with ADR 0223's options. The uplink's holder steps back from this file where this module is assigned (the controller's rule, ADR 0247), so it has one writer |
| `/etc/node-resolver/resolv.conf` | the same file, kept beside it for the guard to compare with and put back |
| `/etc/node-resolver/suffix` | the mesh's own domain, which a route may never take |
| `/etc/systemd/resolved.conf.d/50-mesh.conf` | every mesh resolver as the default route for names (`DNS=`, `Domains=~.`), no public fallback, the stub on loopback and on the private address, no cache, no LLMNR or mDNS |
**On the private address, so containers reach it.** A container copies its machine's resolver file and
cannot reach the machine's loopback, so the file names the address the machine has on the private
network, where resolved also listens (`DNSStubListenerExtra`). The packet filter admits a machine's own
guests and nobody else, so no other machine can ask it. The port is declared `from: machine` and fixed:
a fixed port is a claim on the machine, given to one module, and the mesh's own resolver (`dnsmasq`, on
the anchor and the home server) claims the same one. The two are not meant to share a machine.
**No cache.** Nothing on the machine keeps a copy of a mesh name or of a "no such name". Each question is
asked again, as it was before this resolver.
**Its package is systemd's.** resolved ships with systemd, whose package belongs to the `systemd` module
holding `node-service-manager` (ADR 0207). This module declares the service, running and enabled,
restarted when its drop-in changes, and nothing to install.
## The guard
One long-running process, as root: `resolver-tools guard`.
- **It keeps the resolver file the module's own.** Twice a second it compares `/etc/resolv.conf` with the
kept copy. Another program's write is *displaced*: the guard keeps what it wrote in
`/run/node-resolver/displaced/`, readable by root alone and gone at the next boot, and names its writer
from the file's own header.
- **A write a module takes is put back at once.** A module on the machine that reads the write and routes
what it needed says it *took* it, and the module's own file stands again within a second.
- **A write nobody takes stands for 90 seconds**, then is put back. 90 seconds is longer than the
node-engine needs to see the rewrite twice, so ADR 0241's `machine.<m>.systemd-resolved.rewritten` is
still raised, naming the writer, for a write nothing on the machine declared to handle.
- **Only the mesh's resolvers answer every name.** Every five seconds, a link that a network manager gave
servers of its own is told it is not a default route for names. Its own domains stay its own.
- **What it says elsewhere** — the `routes` verb's `outside_writes`, from `/run/node-resolver/history.json`
— is when, the writer's name and what became of each write. It never includes a server or a domain.
## Its verbs
On the mesh, through the node's runtime as the operator account (writes escalate with `sudo -n`):
| verb | does |
|---|---|
| `<node>/node-resolver.routes` (r) | the mesh's resolvers, each link given servers of its own with its routed domains and whether it is a default route, and the resolver file's outside writes |
| `<node>/node-resolver.route` (a) | `{link, domains, servers}`: these domains, and every name under them, to these servers over this link, and only them. The mesh's own domain and the root are refused |
| `<node>/node-resolver.unroute` (a) | `{link}`: that link's domains go to the mesh's resolvers again |
**On the machine, to its own root processes**, over `/run/node-resolver/verbs.sock` (mode 0600). The path
is the seat's, so a caller does not need to know which holder answers. The protocol is one JSON line
`{"verb": …, "args": {…}}` and one JSON line back, `{"result": …}` or `{"error": "…"}`. The verbs are the
same three, plus:
- `displaced`: the write standing now, with what it held, or null;
- `route` with `takes` (the write's `id`) and `by` (the module's name): route, then take that write, so
the module's file is put back.
A VPN's servers and domains go over this socket and never over the bus, so they never leave the machine.
resolved holds the routes itself, per link, and forgets a link's route when the link goes. Nothing here
keeps a table of its own that could disagree with it.
@@ -0,0 +1,432 @@
// The guard: the module's one long-running process, as root (novox/hq ADR 0247). It keeps the machine's
// resolver file the module's own, and serves the seat's verbs on the machine itself.
//
// **An outside write is kept, then put back.** Another program writing /etc/resolv.conf — a VPN client
// does it on every connect — is the module's file displaced. The guard keeps what that program wrote, for
// whoever handles it on the machine to read, and puts the module's own file back:
//
// - at once, when a module on the machine took the write: it read what it needed and routed it (`route`
// with `takes`);
// - otherwise after Hold, which is longer than the node-engine needs to see the rewrite twice — so a
// write nobody declared to handle is still said, as ADR 0241's rewrite, naming its writer, and then
// ends within a bound instead of at the next reconcile.
//
// What was written stays on this machine, in a directory only root reads, and is gone at the next boot.
// What the guard says of it anywhere else — the `routes` verb's history — is when, the writer the file's
// own header names, and what became of it: never a server or a domain.
//
// **It knows nothing of any VPN**: a writer is whatever the file's header says, and a taker whichever
// module says it took it.
package main
import (
"bufio"
"context"
"encoding/json"
"errors"
"fmt"
"net"
"os"
"path/filepath"
"regexp"
"sort"
"strings"
"sync"
"time"
)
// Where the guard keeps its things. The socket's path is the seat's protocol on the machine: any holder of
// node-resolver serves its verbs there, so a module calling them does not know which holder answers.
const (
ResolvConf = "/etc/resolv.conf"
// KeptPath is the module's own resolver file, rendered by the mesh beside the live one (the fact
// `kept`), so the guard compares and puts back exactly what the mesh declared.
KeptPath = "/etc/node-resolver/resolv.conf"
RunDir = "/run/node-resolver"
Socket = RunDir + "/verbs.sock"
History = RunDir + "/history.json"
Displaced = RunDir + "/displaced"
)
// Timing.
const (
// Look is how often the guard reads the file.
Look = 500 * time.Millisecond
// Hold is how long a write nobody took stands: two of the node-engine's looks (30 s each, ADR 0241)
// with room, so it is said before it is put back.
Hold = 90 * time.Second
// Kept is how many displaced writes are kept on the machine, and in the history.
Kept = 10
// DefaultRouteEvery is how often a link's servers are kept from being a default route.
DefaultRouteEvery = 5 * time.Second
)
// Displacement is one outside write of the resolver file, as the guard says it.
type Displacement struct {
ID string `json:"id"`
At time.Time `json:"at"`
// Writer is who the file's own header names, or empty.
Writer string `json:"writer,omitempty"`
// TakenBy is the module that took it, and when.
TakenBy string `json:"taken_by,omitempty"`
TakenAt *time.Time `json:"taken_at,omitempty"`
// Ended is when the module's own file stood again, and How.
Ended *time.Time `json:"ended,omitempty"`
How string `json:"how,omitempty"`
content string
}
// Guard is the module's file kept, and its verbs served on the machine.
type Guard struct {
Path, KeptPath, Dir string
Hold time.Duration
Now func() time.Time
Resolver *Resolver
// Log says what the guard did, on its own journal: never a server or a domain.
Log func(format string, args ...any)
mu sync.Mutex
pending *Displacement
history []Displacement
wake chan struct{}
}
// NewGuard is the machine's.
func NewGuard() *Guard {
return &Guard{Path: ResolvConf, KeptPath: KeptPath, Dir: RunDir, Hold: Hold, Now: time.Now,
Resolver: ThisResolver(), Log: func(f string, a ...any) { fmt.Fprintf(os.Stderr, f+"\n", a...) },
wake: make(chan struct{}, 1)}
}
// signs are the words a writer leaves in its file's comments, and its name. The same list the node-engine
// names a writer from (ADR 0241 rule 3): what a file says of itself.
var signs = []struct{ word, name string }{
{"forti", "FortiClient"}, {"openfortivpn", "openfortivpn"}, {"networkmanager", "NetworkManager"},
{"systemd-resolved", "systemd-resolved"}, {"resolvconf", "resolvconf"}, {"dhcpcd", "dhcpcd"},
{"dhclient", "dhclient"}, {"netconfig", "netconfig"}, {"openvpn", "OpenVPN"},
{"openconnect", "OpenConnect"}, {"vpnc", "vpnc"}, {"tailscale", "Tailscale"}, {"connman", "ConnMan"},
}
// WriterOf is the writer a file's comments name, or empty.
func WriterOf(content string) string {
for _, line := range strings.Split(content, "\n") {
line = strings.TrimSpace(line)
if !strings.HasPrefix(line, "#") && !strings.HasPrefix(line, ";") {
continue
}
lower := strings.ToLower(line)
for _, s := range signs {
if strings.Contains(lower, s.word) {
return s.name
}
}
}
return ""
}
// same is whether two resolver files say the same, apart from surrounding whitespace — the node-engine's
// own comparison (ADR 0241 rule 1).
func same(a, b string) bool { return strings.TrimSpace(a) == strings.TrimSpace(b) }
// read is the file as a reader of it sees it: its content, or what a link in its place points at.
func read(path string) (content string, isLink bool, err error) {
fi, err := os.Lstat(path)
if err != nil {
return "", false, err
}
if fi.Mode()&os.ModeSymlink != 0 {
target, _ := os.Readlink(path)
raw, _ := os.ReadFile(path)
return "# a link to " + target + "\n" + string(raw), true, nil
}
raw, err := os.ReadFile(path)
return string(raw), false, err
}
// Tick is one look: notice a write, put the module's file back when it was taken or held long enough, and
// close a displacement once the file is the module's again. It answers what it did, for the log and tests.
func (g *Guard) Tick() string {
kept, err := os.ReadFile(g.KeptPath)
if err != nil {
return "" // not yet rendered: nothing declared to keep
}
current, isLink, err := read(g.Path)
if err != nil && !errors.Is(err, os.ErrNotExist) {
return ""
}
now := g.Now()
g.mu.Lock()
defer g.mu.Unlock()
if err == nil && !isLink && same(current, string(kept)) {
if g.pending != nil {
how := "the module's file was written back by another"
if g.pending.How != "" {
how = g.pending.How
}
g.end(now, how)
return "ended"
}
return ""
}
if g.pending == nil || g.pending.content != current {
if g.pending != nil {
g.end(now, "written over again before it was put back")
}
d := &Displacement{ID: now.UTC().Format("20060102T150405.000Z"), At: now, Writer: WriterOf(current), content: current}
g.pending = d
g.keep(d)
g.Log("the resolver file was written by %s; kept as %s", orAnother(d.Writer), d.ID)
}
d := g.pending
switch {
case d.TakenBy != "":
if err := g.putBack(kept); err != nil {
g.Log("putting the resolver file back failed: %v", err)
return "failed"
}
d.How = "taken by " + d.TakenBy + ", and the module's file put back"
g.end(g.Now(), d.How)
return "put back, taken"
case now.Sub(d.At) >= g.Hold:
if err := g.putBack(kept); err != nil {
g.Log("putting the resolver file back failed: %v", err)
return "failed"
}
d.How = fmt.Sprintf("nobody took it; the module's file put back after %s", g.Hold)
g.end(g.Now(), d.How)
return "put back, held"
}
return "holding"
}
func orAnother(w string) string {
if w == "" {
return "another program"
}
return w
}
// end closes the pending displacement into the history.
func (g *Guard) end(at time.Time, how string) {
d := *g.pending
d.Ended, d.How = &at, how
g.pending = nil
g.history = append([]Displacement{d}, g.history...)
if len(g.history) > Kept {
g.history = g.history[:Kept]
}
g.writeHistory()
g.Log("displacement %s ended: %s", d.ID, how)
}
// keep writes what was written where only root reads it, and the oldest beyond Kept goes.
func (g *Guard) keep(d *Displacement) {
dir := filepath.Join(g.Dir, "displaced")
if err := os.MkdirAll(dir, 0o700); err != nil {
return
}
_ = os.WriteFile(filepath.Join(dir, d.ID+".conf"), []byte(d.content), 0o600)
entries, _ := os.ReadDir(dir)
var names []string
for _, e := range entries {
names = append(names, e.Name())
}
sort.Strings(names)
for len(names) > Kept {
_ = os.Remove(filepath.Join(dir, names[0]))
names = names[1:]
}
g.writeHistory()
}
// writeHistory says, readable by the operator's account, what became of each write: never what it held.
func (g *Guard) writeHistory() {
list := []Displacement{}
if g.pending != nil {
list = append(list, *g.pending)
}
list = append(list, g.history...)
raw, _ := json.MarshalIndent(list, "", " ")
tmp := filepath.Join(g.Dir, ".history.json")
if os.WriteFile(tmp, raw, 0o644) == nil {
_ = os.Rename(tmp, filepath.Join(g.Dir, "history.json"))
}
}
// putBack writes the module's file in place, whole, by a rename in the same directory: a reader sees the
// old file or the new one, never half, and a link in its place is replaced by the file.
func (g *Guard) putBack(kept []byte) error {
tmp := filepath.Join(filepath.Dir(g.Path), ".resolv.conf.node-resolver")
if err := os.WriteFile(tmp, kept, 0o644); err != nil {
return err
}
if err := os.Chmod(tmp, 0o644); err != nil {
return err
}
return os.Rename(tmp, g.Path)
}
// Pending is the write standing now, with what it held — for a module on this machine, over the socket.
type Pending struct {
ID string `json:"id"`
At time.Time `json:"at"`
Writer string `json:"writer,omitempty"`
Content string `json:"content"`
}
// Displaced is the write standing now, or nil.
func (g *Guard) Displaced() *Pending {
g.mu.Lock()
defer g.mu.Unlock()
if g.pending == nil {
return nil
}
return &Pending{ID: g.pending.ID, At: g.pending.At, Writer: g.pending.Writer, Content: g.pending.content}
}
// Take marks the write standing now as taken by a module, and has it put back at the next look.
func (g *Guard) Take(id, by string) error {
g.mu.Lock()
defer g.mu.Unlock()
if g.pending == nil || g.pending.ID != id {
return fmt.Errorf("no write %q stands now", id)
}
now := g.Now()
g.pending.TakenBy, g.pending.TakenAt = by, &now
g.writeHistory()
select {
case g.wake <- struct{}{}:
default:
}
return nil
}
// ReadHistory is what became of the last writes, as the guard said it; empty where no guard runs.
func ReadHistory(path string) []Displacement {
raw, err := os.ReadFile(path)
if err != nil {
return []Displacement{}
}
var list []Displacement
if json.Unmarshal(raw, &list) != nil {
return []Displacement{}
}
return list
}
// Run looks until the context ends, and keeps every link's own servers from being a default route.
func (g *Guard) Run(ctx context.Context) {
look := time.NewTicker(Look)
defer look.Stop()
routes := time.NewTicker(DefaultRouteEvery)
defer routes.Stop()
for {
select {
case <-ctx.Done():
return
case <-look.C:
g.Tick()
case <-g.wake:
g.Tick()
case <-routes.C:
if changed, err := g.Resolver.OnlyTheMeshIsADefaultRoute(ctx); err == nil && len(changed) > 0 {
g.Log("%s had servers answering every name; now only their own domains", strings.Join(changed, ", "))
}
}
}
}
// Request is one call over the socket: a verb and its arguments, one JSON line.
type Request struct {
Verb string `json:"verb"`
Args map[string]any `json:"args"`
}
// Reply is its answer, one JSON line.
type Reply struct {
Result any `json:"result,omitempty"`
Error string `json:"error,omitempty"`
}
var takerName = regexp.MustCompile(`^[a-z0-9][a-z0-9-]{0,62}$`)
// Answer is one verb, as the machine's modules call it. `route` with `takes` and `by` also takes the write
// standing now, once its route is in place.
func (g *Guard) Answer(ctx context.Context, req Request) Reply {
str := func(k string) string { s, _ := req.Args[k].(string); return strings.TrimSpace(s) }
var result any
var err error
switch req.Verb {
case "routes":
var routes *Routes
if routes, err = g.Resolver.Routes(ctx); err == nil {
result = map[string]any{"mesh": routes.Mesh, "links": routes.Links}
}
case "route":
var routed *Routed
routed, err = g.Resolver.Route(ctx, str("link"), Split(req.Args["domains"]), Split(req.Args["servers"]))
if err == nil && str("takes") != "" {
if !takerName.MatchString(str("by")) {
err = errors.New("a write is taken by a module, named in `by`")
} else {
err = g.Take(str("takes"), str("by"))
}
}
result = routed
case "unroute":
result, err = g.Resolver.Unroute(ctx, str("link"))
case "displaced":
result = g.Displaced()
default:
err = fmt.Errorf("%q is not a verb of node-resolver", req.Verb)
}
if err != nil {
return Reply{Error: err.Error()}
}
return Reply{Result: result}
}
// Serve answers the socket until the context ends. Only root can reach it: what a module hands over
// here — a VPN's servers and domains — never leaves the machine.
func (g *Guard) Serve(ctx context.Context, path string) error {
_ = os.Remove(path)
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return err
}
l, err := net.Listen("unix", path)
if err != nil {
return err
}
if err := os.Chmod(path, 0o600); err != nil {
l.Close()
return err
}
go func() { <-ctx.Done(); l.Close() }()
for {
conn, err := l.Accept()
if err != nil {
if ctx.Err() != nil {
return nil
}
return err
}
go func(c net.Conn) {
defer c.Close()
_ = c.SetDeadline(time.Now().Add(30 * time.Second))
line, err := bufio.NewReader(c).ReadBytes('\n')
var reply Reply
var req Request
if err != nil && len(line) == 0 {
return
}
if jerr := json.Unmarshal(line, &req); jerr != nil {
reply = Reply{Error: "one JSON object per line: {\"verb\": …, \"args\": {…}}"}
} else {
reply = g.Answer(ctx, req)
}
raw, _ := json.Marshal(reply)
_, _ = c.Write(append(raw, '\n'))
}(conn)
}
}
@@ -0,0 +1,251 @@
package main
import (
"bufio"
"context"
"encoding/json"
"net"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
const meshFile = "# Managed by the mesh\nnameserver 10.42.0.2\noptions timeout:1 attempts:2 edns0\n"
// vpnFile is a VPN client's file as one writes it; the addresses and domains are documentation's.
const vpnFile = "# Dynamic resolv.conf(5) file for glibc resolver(3) generated by forticlient\n" +
"nameserver 192.0.2.53\nnameserver 192.0.2.54\nsearch corp.example cloud.example\n"
// aGuardedMachine is a guard over a temporary /etc and /run, with a clock the test moves.
func aGuardedMachine(t *testing.T) (*Guard, *time.Time, *fakeResolved) {
t.Helper()
dir := t.TempDir()
for _, d := range []string{"etc/node-resolver", "run"} {
if err := os.MkdirAll(filepath.Join(dir, d), 0o755); err != nil {
t.Fatal(err)
}
}
write(t, filepath.Join(dir, "etc/node-resolver/resolv.conf"), meshFile)
write(t, filepath.Join(dir, "etc/resolv.conf"), meshFile)
now := time.Date(2026, 10, 7, 12, 0, 0, 0, time.UTC)
f := &fakeResolved{}
g := &Guard{Path: filepath.Join(dir, "etc/resolv.conf"), KeptPath: filepath.Join(dir, "etc/node-resolver/resolv.conf"),
Dir: filepath.Join(dir, "run"), Hold: Hold, Now: func() time.Time { return now },
Resolver: aMachine(t, f, "tun0"), Log: t.Logf, wake: make(chan struct{}, 1)}
return g, &now, f
}
func write(t *testing.T, path, content string) {
t.Helper()
if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
t.Fatal(err)
}
}
func contentOf(t *testing.T, path string) string {
t.Helper()
raw, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
return string(raw)
}
// The module's own file is left alone, and nothing is said.
func TestTheModulesOwnFileIsLeftAlone(t *testing.T) {
g, _, _ := aGuardedMachine(t)
if did := g.Tick(); did != "" || g.Displaced() != nil {
t.Errorf("the module's own file was taken for an outside write: %q", did)
}
}
// A write a module takes: kept for it to read, with its writer named from its header; once taken, the
// module's own file is back at the next look; and the history says when, who and what became of it —
// never a server or a domain.
func TestATakenWriteIsPutBackAtOnce(t *testing.T) {
g, now, _ := aGuardedMachine(t)
write(t, g.Path, vpnFile)
if did := g.Tick(); did != "holding" {
t.Fatalf("the write was %q, not held for a taker", did)
}
d := g.Displaced()
if d == nil || d.Writer != "FortiClient" || d.Content != vpnFile {
t.Fatalf("the write is not kept as written, naming its writer: %+v", d)
}
kept := filepath.Join(g.Dir, "displaced", d.ID+".conf")
if fi, err := os.Stat(kept); err != nil || fi.Mode().Perm() != 0o600 || contentOf(t, kept) != vpnFile {
t.Errorf("the write is not kept where only root reads it: %v", err)
}
*now = now.Add(2 * time.Second)
if err := g.Take(d.ID, "forticlient"); err != nil {
t.Fatal(err)
}
if did := g.Tick(); did != "put back, taken" {
t.Fatalf("a taken write was %q", did)
}
if contentOf(t, g.Path) != meshFile {
t.Errorf("the module's file is not back:\n%s", contentOf(t, g.Path))
}
if fi, _ := os.Stat(g.Path); fi.Mode().Perm() != 0o644 {
t.Errorf("the file is %v, not readable by everyone", fi.Mode().Perm())
}
history := contentOf(t, filepath.Join(g.Dir, "history.json"))
for _, never := range []string{"192.0.2", "corp.example", "nameserver"} {
if strings.Contains(history, never) {
t.Errorf("the history says %q, which stays on the machine:\n%s", never, history)
}
}
list := ReadHistory(filepath.Join(g.Dir, "history.json"))
if len(list) != 1 || list[0].TakenBy != "forticlient" || list[0].Ended == nil || !strings.Contains(list[0].How, "taken") {
t.Errorf("the history is %+v", list)
}
if g.Tick() != "" {
t.Error("the module's own file, back, was taken for another write")
}
}
// A write nobody takes stands for Hold — long enough for the node-engine to see it twice and say it —
// and is then put back.
func TestAWriteNobodyTakesStandsUntilItIsSaidThenGoes(t *testing.T) {
g, now, _ := aGuardedMachine(t)
write(t, g.Path, "# written by another program\nnameserver 198.51.100.1\n")
g.Tick()
*now = now.Add(61 * time.Second)
if did := g.Tick(); did != "holding" || contentOf(t, g.Path) == meshFile {
t.Fatalf("an untaken write was put back after a minute, before the node-engine's second look: %q", did)
}
if Hold < 75*time.Second {
t.Errorf("Hold is %s; two of the node-engine's 30 s looks need more", Hold)
}
*now = now.Add(Hold)
if did := g.Tick(); did != "put back, held" || contentOf(t, g.Path) != meshFile {
t.Fatalf("an untaken write was not put back after Hold: %q", did)
}
if h := ReadHistory(filepath.Join(g.Dir, "history.json")); len(h) != 1 || h[0].TakenBy != "" || h[0].Writer != "" {
t.Errorf("the history is %+v", h)
}
}
// A write the reconcile or the writer itself undoes is closed as written back by another; one written
// over before it was put back is a new one; a link in the file's place is a write too.
func TestWritesUndoneAndWrittenOverAreSaid(t *testing.T) {
g, now, _ := aGuardedMachine(t)
write(t, g.Path, vpnFile)
g.Tick()
write(t, g.Path, meshFile)
*now = now.Add(time.Second)
if did := g.Tick(); did != "ended" {
t.Errorf("a write undone by another was %q", did)
}
write(t, g.Path, vpnFile)
g.Tick()
first := g.Displaced().ID
*now = now.Add(time.Second)
write(t, g.Path, vpnFile+"search more.example\n")
g.Tick()
if g.Displaced().ID == first {
t.Error("a second write was taken for the first")
}
if err := g.Take(first, "forticlient"); err == nil {
t.Error("a write that no longer stands was taken")
}
_ = os.Remove(g.Path)
if err := os.Symlink(g.KeptPath, g.Path); err != nil {
t.Fatal(err)
}
*now = now.Add(time.Second)
g.Tick()
if d := g.Displaced(); d == nil || !strings.Contains(d.Content, "a link to") {
t.Errorf("a link in the file's place was not a write: %+v", d)
}
*now = now.Add(Hold)
g.Tick()
if fi, err := os.Lstat(g.Path); err != nil || fi.Mode()&os.ModeSymlink != 0 {
t.Errorf("the link was not replaced by the module's file: %v", err)
}
}
// Nothing is kept before the mesh rendered the module's file: there is nothing to keep it to.
func TestNothingIsGuardedBeforeTheFileIsRendered(t *testing.T) {
g, _, _ := aGuardedMachine(t)
_ = os.Remove(g.KeptPath)
write(t, g.Path, vpnFile)
if g.Tick() != "" || contentOf(t, g.Path) != vpnFile {
t.Error("the guard acted with no file of its own to keep")
}
}
// The socket: a module routes and takes the write standing now in one call, and the module's file is
// back; a bad request and an unknown verb are answered, not dropped; the socket is root's alone.
func TestTheVerbsOnTheMachine(t *testing.T) {
g, _, f := aGuardedMachine(t)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
sock := filepath.Join(g.Dir, "verbs.sock")
go func() { _ = g.Serve(ctx, sock) }()
for i := 0; i < 100; i++ {
if _, err := os.Stat(sock); err == nil {
break
}
time.Sleep(10 * time.Millisecond)
}
if fi, err := os.Stat(sock); err != nil || fi.Mode().Perm() != 0o600 {
t.Fatalf("the socket is not root's alone: %v", err)
}
call := func(line string) Reply {
t.Helper()
c, err := net.Dial("unix", sock)
if err != nil {
t.Fatal(err)
}
defer c.Close()
if _, err := c.Write([]byte(line + "\n")); err != nil {
t.Fatal(err)
}
raw, err := bufio.NewReader(c).ReadBytes('\n')
if err != nil {
t.Fatal(err)
}
var r Reply
if err := json.Unmarshal(raw, &r); err != nil {
t.Fatal(err)
}
return r
}
write(t, g.Path, vpnFile)
g.Tick()
shown := call(`{"verb":"displaced"}`)
pending, _ := shown.Result.(map[string]any)
if pending == nil || pending["content"] != vpnFile {
t.Fatalf("the write standing now is not shown on the machine: %+v", shown)
}
req, _ := json.Marshal(Request{Verb: "route", Args: map[string]any{"link": "tun0",
"domains": []any{"corp.example", "cloud.example"}, "servers": "192.0.2.53 192.0.2.54",
"takes": pending["id"], "by": "forticlient"}})
if r := call(string(req)); r.Error != "" {
t.Fatalf("route and take: %s", r.Error)
}
if len(f.changed) != 3 {
t.Errorf("resolved was asked %v", f.changed)
}
select {
case <-g.wake:
g.Tick()
case <-time.After(time.Second):
t.Fatal("taking the write did not wake the guard")
}
if contentOf(t, g.Path) != meshFile {
t.Error("the module's file is not back once its write was taken")
}
if r := call(`not json`); !strings.Contains(r.Error, "JSON") {
t.Errorf("a bad request: %+v", r)
}
if r := call(`{"verb":"flush"}`); !strings.Contains(r.Error, "not a verb") {
t.Errorf("an unknown verb: %+v", r)
}
if r := call(`{"verb":"route","args":{"link":"tun0","domains":"internal","servers":"192.0.2.53"}}`); !strings.Contains(r.Error, "mesh's own") {
t.Errorf("the mesh's domain over the socket: %+v", r)
}
}
@@ -0,0 +1,97 @@
// systemd-resolved's tools bundle (novox/hq ADR 0247): the node-resolver seat's verbs, and the module's
// guard.
//
// Started with no arguments it is served by the node's runtime as the operator account, over MCP on stdio
// through the Go SDK (ADR 0188, ADR 0193): `routes`, `route` and `unroute` on the mesh, for the operator
// to read and correct. Started as `resolver-tools guard` it is the module's long-running process, as root:
// it keeps the machine's resolver file the module's own and serves the same verbs on the machine, to the
// modules there (guard.go). stdout is the MCP channel; everything else is said on stderr.
package main
import (
"context"
"fmt"
"os"
"os/signal"
"syscall"
stdio "git.novox.be/novox/mesh-sdk/go"
)
// Seat is the role this module holds.
const Seat = "node-resolver"
func main() {
if len(os.Args) > 1 {
if os.Args[1] != "guard" || len(os.Args) != 2 {
fmt.Fprintln(os.Stderr, "usage: resolver-tools [guard]")
os.Exit(2)
}
if err := guard(); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
return
}
if err := stdio.Serve("", tools(ThisResolver(), History)); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
func guard() error {
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGTERM, syscall.SIGINT)
defer stop()
g := NewGuard()
if err := os.MkdirAll(g.Dir, 0o755); err != nil {
return err
}
g.writeHistory()
go g.Run(ctx)
return g.Serve(ctx, Socket)
}
func str(description string) map[string]any {
return map[string]any{"type": "string", "description": description}
}
func arg(a map[string]any, k string) string {
v, _ := a[k].(string)
return v
}
// verb is one of the seat's verbs: listed as `<seat>.<verb>`, so the runtime serves it on the seat's
// subject, as <node>/node-resolver.<verb>.
func verb(name, description string, input map[string]any, run func(a map[string]any) (any, error)) stdio.Tool {
return stdio.Tool{Name: Seat + "." + name, Description: description, Input: input, Run: run}
}
func tools(r *Resolver, history string) []stdio.Tool {
ctx := context.Background()
return []stdio.Tool{
verb("routes", "What this machine's own resolver sends where: the mesh's resolvers, which answer every name "+
"not routed elsewhere, and each link given servers of its own with the domains routed to them. Also the "+
"resolver file's outside writes, newest first: when, who wrote it as far as the file says, whether a "+
"module took it, and when the module's own file stood again. (r)",
nil, func(map[string]any) (any, error) {
routes, err := r.Routes(ctx)
if err != nil {
return nil, err
}
return map[string]any{"mesh": routes.Mesh, "links": routes.Links, "outside_writes": ReadHistory(history)}, nil
}),
verb("route", "Send these domains, and every name under them, to these servers over this link, and only "+
"them: the link never answers other names, and the mesh's own domain is refused. Replaces whatever the "+
"link was given before; a link that goes away takes its route with it. (a)",
map[string]any{"link": str("the network link the servers are reached over, by name"),
"domains": str("the domains to route there, separated by spaces or commas"),
"servers": str("the servers' addresses, separated by spaces or commas")},
func(a map[string]any) (any, error) {
return r.Route(ctx, arg(a, "link"), Split(a["domains"]), Split(a["servers"]))
}),
verb("unroute", "Take one link's route away: its domains go to the mesh's resolvers again. Nothing changes "+
"when the link has none. (a)",
map[string]any{"link": str("the network link, by name")},
func(a map[string]any) (any, error) { return r.Unroute(ctx, arg(a, "link")) }),
}
}
@@ -0,0 +1,119 @@
package main
import (
"encoding/json"
"os"
"strings"
"testing"
)
// The manifest and this code say one thing: the paths the guard keeps are the files the mesh renders, the
// process runs this binary as the guard, and the resolver file and its kept copy are one template.
type manifest struct {
Claims []struct {
Name string `json:"name"`
Serves []string `json:"serves"`
} `json:"claims"`
Provides []struct {
Name string `json:"name"`
Reach string `json:"reach"`
} `json:"provides"`
Upgrade struct {
Policy string `json:"policy"`
Why string `json:"why"`
} `json:"upgrade"`
Facts map[string]struct {
Path string `json:"path"`
Template string `json:"template"`
} `json:"facts"`
Resources []map[string]any `json:"resources"`
Build struct {
Artifacts []struct {
Binary string `json:"binary"`
} `json:"artifacts"`
} `json:"build"`
}
func theManifest(t *testing.T) manifest {
t.Helper()
raw, err := os.ReadFile("../../module.json")
if err != nil {
t.Fatal(err)
}
var m manifest
if err := json.Unmarshal(raw, &m); err != nil {
t.Fatal(err)
}
return m
}
func TestTheManifestSaysWhatTheGuardKeeps(t *testing.T) {
m := theManifest(t)
if m.Facts["resolvers"].Path != ResolvConf || m.Facts["kept"].Path != KeptPath || m.Facts["suffix"].Path != SuffixPath {
t.Errorf("the rendered paths are not the ones the guard reads: %+v", m.Facts)
}
if m.Facts["resolvers"].Template != m.Facts["kept"].Template {
t.Error("the resolver file and the copy the guard puts back are not one template")
}
// Sorted before the live file, so the mesh writes the copy first and the guard never puts back the
// file it is about to be given.
if !("kept" < "resolvers") {
t.Error("the kept copy is not rendered before the file")
}
if !strings.HasPrefix(m.Facts["resolvers"].Template, "# Managed by the mesh") {
t.Error("the resolver file does not begin as the mesh's own does, which is how the uplink's verb reads it")
}
var lines []string
for _, l := range strings.Split(m.Facts["resolvers"].Template, "\n") {
if strings.Contains(l, "nameserver") && !strings.HasPrefix(l, "#") {
lines = append(lines, l)
}
}
if len(lines) != 1 || strings.Contains(m.Facts["resolvers"].Template, "search ") {
t.Errorf("the resolver file lists more than this machine's own resolver: %v", lines)
}
conf := m.Facts["resolved"].Template
for _, want := range []string{`DNS={{range index .Holders "mesh-dns-resolver"}}`, "\nDomains=~.\n", "\nFallbackDNS=\n",
"DNSStubListenerExtra={{$own}}\n", "\nCache=no\n", "\nLLMNR=no\n", "\nMulticastDNS=no\n"} {
if !strings.Contains(conf, want) {
t.Errorf("resolved's drop-in lacks %q", want)
}
}
guard, service := false, false
for _, r := range m.Resources {
run, _ := r["run"].([]any)
if r["type"] == "process" && len(run) == 2 && run[0] == "./"+m.Build.Artifacts[0].Binary && run[1] == "guard" {
guard = r["user"] == nil && r["run-once"] == nil && r["health"] != nil
}
if r["type"] == "service" && r["unit"] == "systemd-resolved.service" {
on, _ := r["restart-on"].([]any)
service = r["state"] == "running" && len(on) == 1 && on[0] == "systemd-resolved.fact-resolved" && r["health"] != nil
}
}
if !guard || !service {
t.Errorf("the guard (root, long-running, its health said) %v; resolved (running, restarted on its drop-in) %v", guard, service)
}
if len(m.Claims) != 1 || m.Claims[0].Name != Seat || strings.Join(m.Claims[0].Serves, " ") != "routes route unroute" {
t.Errorf("the claim is %+v", m.Claims)
}
if len(m.Provides) != 1 || m.Provides[0].Name != "split-dns" || m.Provides[0].Reach != "machine" {
t.Errorf("split-dns is not provided at the machine's reach: %+v", m.Provides)
}
if m.Upgrade.Policy != "record" || m.Upgrade.Why == "" {
t.Error("a build of the machine's names rolls out on its own")
}
}
// Every verb the claim serves is a tool of the bundle, by the seat's name.
func TestTheBundleServesTheClaimedVerbs(t *testing.T) {
have := map[string]bool{}
for _, tool := range tools(&Resolver{}, "") {
have[tool.Name] = true
}
for _, v := range theManifest(t).Claims[0].Serves {
if !have[Seat+"."+v] {
t.Errorf("%s.%s is claimed and not served", Seat, v)
}
}
}
@@ -0,0 +1,378 @@
// The node-resolver seat's verbs, done by systemd-resolved (novox/hq ADR 0247): what is routed where,
// route a set of domains to a set of servers over one link, and take a link's route away.
//
// **resolved holds the routes, not this code.** A link's servers and routing domains are resolved's own
// per-link state, set through resolvectl and forgotten by resolved when the link goes. So nothing here
// keeps a table that could disagree with what resolved does: `routes` reads resolved, and the two
// transports that serve these verbs — the mesh, through the node's runtime as the operator account, and
// the machine, through the guard's socket as root — run the same code against the same daemon.
//
// **It knows nothing of any VPN.** A link, domains and servers. What a VPN client pushed is its own
// module's to read and hand over.
package main
import (
"bytes"
"context"
"errors"
"fmt"
"net/netip"
"os"
"os/exec"
"path/filepath"
"regexp"
"sort"
"strings"
"time"
)
// Runner runs one command — as root when it changes something — and answers what it printed.
type Runner func(ctx context.Context, name string, args ...string) (string, error)
// escalated is the command as it is run: as given when this process is root (the guard), else through
// sudo without a prompt (the runtime's account), as the hosts file's and the packet filter's verbs do.
func escalated(uid int, name string, args []string) (string, []string) {
if uid == 0 {
return name, args
}
return "sudo", append([]string{"-n", name}, args...)
}
// execRunner runs a command that changes resolved's state, escalated.
func execRunner(ctx context.Context, name string, args ...string) (string, error) {
return run(ctx, true, name, args...)
}
// readRunner runs a command that only reads, as whoever this process is.
func readRunner(ctx context.Context, name string, args ...string) (string, error) {
return run(ctx, false, name, args...)
}
func run(ctx context.Context, escalate bool, name string, args ...string) (string, error) {
ctx, cancel := context.WithTimeout(ctx, 15*time.Second)
defer cancel()
program, argv := name, args
if escalate {
program, argv = escalated(os.Getuid(), name, args)
}
var stdout, stderr bytes.Buffer
cmd := exec.CommandContext(ctx, program, argv...)
cmd.Stdout, cmd.Stderr = &stdout, &stderr
err := cmd.Run()
if err == nil {
return stdout.String(), nil
}
said := strings.TrimSpace(stdout.String() + stderr.String())
if program == "sudo" {
if errors.Is(err, exec.ErrNotFound) {
return "", fmt.Errorf("%s needs root, and sudo is not installed here for the runtime's account to escalate with", name)
}
if regexp.MustCompile(`(?m)^sudo:`).MatchString(said) {
return "", fmt.Errorf("%s needs root and the runtime's account may not run it without a prompt: %s", name, said)
}
}
if said != "" {
return "", fmt.Errorf("%s: %s", name, said)
}
return "", fmt.Errorf("%s failed: %v", name, err)
}
// Resolver is systemd-resolved on this machine, as the seat's verbs see it.
type Resolver struct {
// Change runs what changes resolved (escalated); Read what only reads it.
Change, Read Runner
// NetDir is where the machine's links are listed (/sys/class/net).
NetDir string
// SuffixFile holds the mesh's own domain, which is never routed elsewhere.
SuffixFile string
}
// ThisResolver is the machine's.
func ThisResolver() *Resolver {
return &Resolver{Change: execRunner, Read: readRunner, NetDir: "/sys/class/net", SuffixFile: SuffixPath}
}
// SuffixPath is the file the mesh renders the mesh's own domain into (the manifest's fact `suffix`).
const SuffixPath = "/etc/node-resolver/suffix"
// Scope is one place resolved sends names: the machine's global servers (the mesh's resolvers), or a link.
type Scope struct {
Link string `json:"link,omitempty"`
Servers []string `json:"servers"`
// Domains are the routing domains, without resolved's `~`: every name under one goes to these servers.
Domains []string `json:"domains"`
// DefaultRoute is whether names no domain routes may also go here. Only the mesh's resolvers are.
DefaultRoute *bool `json:"default_route,omitempty"`
}
// Routes is what resolved sends where.
type Routes struct {
// Mesh is the global scope: the mesh's resolvers, which answer every name nothing routes elsewhere.
Mesh Scope `json:"mesh"`
// Links is every link given servers of its own.
Links []Scope `json:"links"`
}
var linkLine = regexp.MustCompile(`^Link\s+\d+\s+\(([^)]+)\):\s*(.*)$`)
// perScope reads one resolvectl listing (`dns`, `domain`, `default-route`) into the global line and one
// line per link.
func perScope(out string) (global []string, links map[string][]string) {
links = map[string][]string{}
for _, line := range strings.Split(out, "\n") {
line = strings.TrimSpace(line)
if rest, ok := strings.CutPrefix(line, "Global:"); ok {
global = strings.Fields(rest)
continue
}
if m := linkLine.FindStringSubmatch(line); m != nil {
links[m[1]] = strings.Fields(m[2])
}
}
return global, links
}
func unrouted(domains []string) []string {
out := make([]string, 0, len(domains))
for _, d := range domains {
out = append(out, strings.TrimPrefix(d, "~"))
}
return out
}
// Routes reads what resolved sends where. It changes nothing and needs no root.
func (r *Resolver) Routes(ctx context.Context) (*Routes, error) {
dns, err := r.Read(ctx, "resolvectl", "dns")
if err != nil {
return nil, fmt.Errorf("systemd-resolved does not answer: %w", err)
}
domain, err := r.Read(ctx, "resolvectl", "domain")
if err != nil {
return nil, fmt.Errorf("systemd-resolved does not answer: %w", err)
}
defaults, _ := r.Read(ctx, "resolvectl", "default-route")
gServers, lServers := perScope(dns)
gDomains, lDomains := perScope(domain)
_, lDefault := perScope(defaults)
out := &Routes{Mesh: Scope{Servers: orEmpty(gServers), Domains: orEmpty(unrouted(gDomains))}, Links: []Scope{}}
names := make([]string, 0, len(lServers))
for name, servers := range lServers {
if len(servers) > 0 {
names = append(names, name)
}
}
sort.Strings(names)
for _, name := range names {
s := Scope{Link: name, Servers: lServers[name], Domains: orEmpty(unrouted(lDomains[name]))}
if d, ok := lDefault[name]; ok && len(d) > 0 {
yes := d[0] == "yes"
s.DefaultRoute = &yes
}
out.Links = append(out.Links, s)
}
return out, nil
}
func orEmpty(s []string) []string {
if s == nil {
return []string{}
}
return s
}
// Routed is what a route did.
type Routed struct {
Link string `json:"link"`
Servers []string `json:"servers"`
Domains []string `json:"domains"`
Said string `json:"said"`
}
var (
linkName = regexp.MustCompile(`^[A-Za-z0-9_.:@-]{1,15}$`)
domainName = regexp.MustCompile(`^([a-z0-9_]([a-z0-9_-]{0,61}[a-z0-9_])?\.)*[a-z0-9_]([a-z0-9_-]{0,61}[a-z0-9_])?$`)
separators = regexp.MustCompile(`[\s,]+`)
)
// Split reads a list given as one string, separated by spaces or commas, or as a list.
func Split(v any) []string {
var raw []string
switch v := v.(type) {
case string:
raw = separators.Split(v, -1)
case []any:
for _, x := range v {
if s, ok := x.(string); ok {
raw = append(raw, separators.Split(s, -1)...)
}
}
case []string:
for _, s := range v {
raw = append(raw, separators.Split(s, -1)...)
}
}
out := []string{}
for _, s := range raw {
if s = strings.TrimSpace(s); s != "" {
out = append(out, s)
}
}
return out
}
// suffix is the mesh's own domain, as the mesh rendered it; "internal" when it has not been yet.
func (r *Resolver) suffix() string {
raw, err := os.ReadFile(r.SuffixFile)
if s := strings.Trim(strings.TrimSpace(string(raw)), "."); err == nil && s != "" {
return strings.ToLower(s)
}
return "internal"
}
// checkLink refuses a link that is not one, loopback, and one that is not on this machine now.
func (r *Resolver) checkLink(link string) error {
if !linkName.MatchString(link) {
return fmt.Errorf("%q is not a link's name", link)
}
if link == "lo" {
return errors.New("loopback carries no servers of its own")
}
if _, err := os.Stat(filepath.Join(r.NetDir, link)); err != nil {
return fmt.Errorf("there is no link %q on this machine now", link)
}
return nil
}
// Domains reads the domains to route: lower-cased, without resolved's `~` or a final dot, each once. The
// root and the mesh's own domain are refused: routing either away would send the mesh's names, or every
// name, to servers that are not the mesh's (ADR 0223, ADR 0247).
func (r *Resolver) Domains(given []string) ([]string, error) {
suffix := r.suffix()
seen := map[string]bool{}
out := []string{}
for _, d := range given {
d = strings.ToLower(strings.TrimSuffix(strings.TrimPrefix(d, "~"), "."))
if d == "" {
return nil, errors.New("the root domain is every name: only the mesh's resolvers answer every name")
}
if !domainName.MatchString(d) || len(d) > 253 {
return nil, fmt.Errorf("%q is not a domain", d)
}
if d == suffix || strings.HasSuffix(d, "."+suffix) {
return nil, fmt.Errorf("%q is the mesh's own domain: the mesh's names are answered by the mesh's resolvers alone", d)
}
if !seen[d] {
seen[d] = true
out = append(out, d)
}
}
if len(out) == 0 {
return nil, errors.New("no domain given: a link's servers answer only the domains routed to them")
}
if len(out) > 64 {
return nil, fmt.Errorf("%d domains; at most 64", len(out))
}
return out, nil
}
// Servers reads the servers: addresses, each once, at most eight.
func Servers(given []string) ([]string, error) {
seen := map[string]bool{}
out := []string{}
for _, s := range given {
a, err := netip.ParseAddr(s)
if err != nil {
return nil, fmt.Errorf("%q is not an address", s)
}
if a.IsUnspecified() || a.IsMulticast() {
return nil, fmt.Errorf("%s cannot answer names", s)
}
if !seen[a.String()] {
seen[a.String()] = true
out = append(out, a.String())
}
}
if len(out) == 0 {
return nil, errors.New("no server given")
}
if len(out) > 8 {
return nil, fmt.Errorf("%d servers; at most 8", len(out))
}
return out, nil
}
// Route sends these domains, and every name under them, to these servers over this link — and only them.
// Whatever the link was given before is replaced. resolved forgets it when the link goes.
func (r *Resolver) Route(ctx context.Context, link string, domains, servers []string) (*Routed, error) {
if err := r.checkLink(link); err != nil {
return nil, err
}
ds, err := r.Domains(domains)
if err != nil {
return nil, err
}
ss, err := Servers(servers)
if err != nil {
return nil, err
}
routing := make([]string, len(ds))
for i, d := range ds {
routing[i] = "~" + d
}
// The link is never a default route: names no domain routes go to the mesh's resolvers, so set
// first, before the servers, that no question but these domains' ever reaches it.
steps := [][]string{
{"default-route", link, "false"},
append([]string{"domain", link}, routing...),
append([]string{"dns", link}, ss...),
}
for _, s := range steps {
if _, err := r.Change(ctx, "resolvectl", s...); err != nil {
return nil, err
}
}
return &Routed{Link: link, Servers: ss, Domains: ds,
Said: fmt.Sprintf("%d domains go to %d servers over %s; every other name to the mesh's resolvers", len(ds), len(ss), link)}, nil
}
// Unrouted is what taking a route away did.
type Unrouted struct {
Link string `json:"link"`
Said string `json:"said"`
}
// Unroute takes one link's route away. A link that has gone has nothing to take away.
func (r *Resolver) Unroute(ctx context.Context, link string) (*Unrouted, error) {
if !linkName.MatchString(link) || link == "lo" {
return nil, fmt.Errorf("%q is not a link's name", link)
}
if _, err := os.Stat(filepath.Join(r.NetDir, link)); err != nil {
return &Unrouted{Link: link, Said: link + " is not on this machine; resolved forgot its route with it"}, nil
}
if _, err := r.Change(ctx, "resolvectl", "revert", link); err != nil {
return nil, err
}
return &Unrouted{Link: link, Said: link + "'s domains go to the mesh's resolvers again"}, nil
}
// OnlyTheMeshIsADefaultRoute keeps every link that has servers of its own from answering names nothing
// routes to it: a network manager telling resolved a network's servers makes them a default route, and
// then resolved asks them every name beside the mesh's resolvers. Their routing domains are kept — a
// link's own domains still go to it. Answers the links it changed.
func (r *Resolver) OnlyTheMeshIsADefaultRoute(ctx context.Context) ([]string, error) {
routes, err := r.Routes(ctx)
if err != nil {
return nil, err
}
var changed []string
for _, l := range routes.Links {
if l.DefaultRoute == nil || !*l.DefaultRoute {
continue
}
if _, err := r.Change(ctx, "resolvectl", "default-route", l.Link, "false"); err != nil {
return changed, err
}
changed = append(changed, l.Link)
}
return changed, nil
}
@@ -0,0 +1,172 @@
package main
import (
"context"
"os"
"path/filepath"
"strings"
"testing"
)
// fakeResolved answers resolvectl as systemd-resolved would, and records what was asked of it.
type fakeResolved struct {
dns, domain, defaults string
changed [][]string
}
func (f *fakeResolved) read(_ context.Context, name string, args ...string) (string, error) {
switch args[0] {
case "dns":
return f.dns, nil
case "domain":
return f.domain, nil
case "default-route":
return f.defaults, nil
}
return "", nil
}
func (f *fakeResolved) change(_ context.Context, name string, args ...string) (string, error) {
f.changed = append(f.changed, append([]string{name}, args...))
return "", nil
}
// aMachine is a resolver over a fake resolved and a /sys/class/net holding the links named.
func aMachine(t *testing.T, f *fakeResolved, links ...string) *Resolver {
t.Helper()
dir := t.TempDir()
for _, l := range links {
if err := os.MkdirAll(filepath.Join(dir, "net", l), 0o755); err != nil {
t.Fatal(err)
}
}
suffix := filepath.Join(dir, "suffix")
if err := os.WriteFile(suffix, []byte("internal\n"), 0o644); err != nil {
t.Fatal(err)
}
return &Resolver{Change: f.change, Read: f.read, NetDir: filepath.Join(dir, "net"), SuffixFile: suffix}
}
// A route sends the domains to the servers over the link, and only them: the link is first told it is
// no default route, then given its domains as routing domains, then its servers.
func TestARouteSendsOnlyItsDomainsOverItsLink(t *testing.T) {
f := &fakeResolved{}
r := aMachine(t, f, "tun0")
got, err := r.Route(context.Background(), "tun0", Split("Corp.Example. ~cloud.example, corp.example"), Split("10.9.0.1 10.9.0.2"))
if err != nil {
t.Fatal(err)
}
want := []string{
"resolvectl default-route tun0 false",
"resolvectl domain tun0 ~corp.example ~cloud.example",
"resolvectl dns tun0 10.9.0.1 10.9.0.2",
}
if len(f.changed) != len(want) {
t.Fatalf("resolved was asked %v", f.changed)
}
for i, w := range want {
if strings.Join(f.changed[i], " ") != w {
t.Errorf("step %d was %v, not %s", i, f.changed[i], w)
}
}
if len(got.Domains) != 2 || len(got.Servers) != 2 {
t.Errorf("the route says %+v", got)
}
}
// The mesh's own domain, the root, a link that is not here and loopback are refused, and resolved is
// asked nothing.
func TestARouteThatWouldTakeTheMeshsNamesIsRefused(t *testing.T) {
for name, c := range map[string]struct{ link, domains, servers, says string }{
"the mesh's domain": {"tun0", "corp.example internal", "10.9.0.1", "mesh's own domain"},
"under it": {"tun0", "anchor.internal", "10.9.0.1", "mesh's own domain"},
"the root": {"tun0", "~.", "10.9.0.1", "every name"},
"no domain": {"tun0", "", "10.9.0.1", "no domain"},
"not a domain": {"tun0", "a b/c", "10.9.0.1", "not a domain"},
"no server": {"tun0", "corp.example", "", "no server"},
"not an address": {"tun0", "corp.example", "dns.corp.example", "not an address"},
"a link not here": {"tun9", "corp.example", "10.9.0.1", "no link"},
"loopback": {"lo", "corp.example", "10.9.0.1", "loopback"},
"not a link's name": {"../etc", "corp.example", "10.9.0.1", "not a link"},
"an unspecified one": {"tun0", "corp.example", "0.0.0.0", "cannot answer"},
"a multicast address": {"tun0", "corp.example", "224.0.0.251", "cannot answer"},
} {
f := &fakeResolved{}
r := aMachine(t, f, "tun0", "lo")
_, err := r.Route(context.Background(), c.link, Split(c.domains), Split(c.servers))
if err == nil || !strings.Contains(err.Error(), c.says) {
t.Errorf("%s: %v", name, err)
}
if len(f.changed) != 0 {
t.Errorf("%s: resolved was changed anyway: %v", name, f.changed)
}
}
}
// Routes reads resolved: the mesh's resolvers as the global scope, and each link with servers of its
// own, its routing domains without the `~` and whether it is a default route.
func TestRoutesReadWhatResolvedSendsWhere(t *testing.T) {
f := &fakeResolved{
dns: "Global: 10.42.0.1 10.42.0.3\nLink 2 (wlan0): 192.168.1.1\nLink 3 (mesh0):\nLink 7 (tun0): 10.9.0.1 10.9.0.2\n",
domain: "Global: ~.\nLink 2 (wlan0):\nLink 3 (mesh0):\nLink 7 (tun0): ~corp.example ~cloud.example\n",
defaults: "Link 2 (wlan0): yes\nLink 3 (mesh0): yes\nLink 7 (tun0): no\n",
}
got, err := aMachine(t, f).Routes(context.Background())
if err != nil {
t.Fatal(err)
}
if strings.Join(got.Mesh.Servers, " ") != "10.42.0.1 10.42.0.3" || strings.Join(got.Mesh.Domains, " ") != "." {
t.Errorf("the mesh's scope is %+v", got.Mesh)
}
if len(got.Links) != 2 || got.Links[0].Link != "tun0" && got.Links[1].Link != "tun0" {
t.Fatalf("the links with servers are %+v", got.Links)
}
for _, l := range got.Links {
if l.Link == "tun0" && (strings.Join(l.Domains, " ") != "corp.example cloud.example" || *l.DefaultRoute) {
t.Errorf("tun0 is %+v", l)
}
}
}
// A link a network manager gave servers is kept from answering every name: its default route goes, its
// own domains stay; a link without servers, and one already routed, are left alone.
func TestOnlyTheMeshsResolversAnswerEveryName(t *testing.T) {
f := &fakeResolved{
dns: "Global: 10.42.0.1\nLink 2 (wlan0): 192.168.1.1\nLink 3 (mesh0):\nLink 7 (tun0): 10.9.0.1\n",
domain: "Global: ~.\nLink 2 (wlan0): lan\nLink 7 (tun0): ~corp.example\n",
defaults: "Link 2 (wlan0): yes\nLink 3 (mesh0): yes\nLink 7 (tun0): no\n",
}
changed, err := aMachine(t, f).OnlyTheMeshIsADefaultRoute(context.Background())
if err != nil {
t.Fatal(err)
}
if strings.Join(changed, " ") != "wlan0" || len(f.changed) != 1 ||
strings.Join(f.changed[0], " ") != "resolvectl default-route wlan0 false" {
t.Errorf("changed %v by %v", changed, f.changed)
}
}
// Taking a route away reverts the link; a link that went has nothing to take away.
func TestUnrouteRevertsTheLink(t *testing.T) {
f := &fakeResolved{}
r := aMachine(t, f, "tun0")
if _, err := r.Unroute(context.Background(), "tun0"); err != nil {
t.Fatal(err)
}
if _, err := r.Unroute(context.Background(), "tun1"); err != nil {
t.Fatal(err)
}
if len(f.changed) != 1 || strings.Join(f.changed[0], " ") != "resolvectl revert tun0" {
t.Errorf("resolved was asked %v", f.changed)
}
}
// Escalation: root runs the command as given; the runtime's account through sudo without a prompt.
func TestChangesAreEscalatedOnlyWhenNotRoot(t *testing.T) {
if p, a := escalated(0, "resolvectl", []string{"revert", "tun0"}); p != "resolvectl" || len(a) != 2 {
t.Errorf("as root: %s %v", p, a)
}
if p, a := escalated(1000, "resolvectl", []string{"revert", "tun0"}); p != "sudo" || strings.Join(a, " ") != "-n resolvectl revert tun0" {
t.Errorf("as the account: %s %v", p, a)
}
}
+5
View File
@@ -0,0 +1,5 @@
module systemd-resolved
go 1.25.0
require git.novox.be/novox/mesh-sdk/go v0.1.7
+2
View File
@@ -0,0 +1,2 @@
git.novox.be/novox/mesh-sdk/go v0.1.7 h1:C0sTQmtTiyYH7bnqZb7PusXnqA37gKuT7Nqjn9gG47w=
git.novox.be/novox/mesh-sdk/go v0.1.7/go.mod h1:GFuZUElBZ9A++mxgIKo97aXXo+kV0uJ/UkbhQPPIbrY=
+110
View File
@@ -0,0 +1,110 @@
{
"module": "systemd-resolved",
"version": "1",
"upgrade": {
"policy": "record",
"why": "the machine's names: a build that breaks this resolver stops every name resolving on a machine a person works on, the bus's included, and then neither the gate's rollback nor a push reaches it (hq ADR 0236, ADR 0247)"
},
"provides": [
{
"name": "split-dns",
"reach": "machine"
}
],
"requires": [
"wildcard-resolution"
],
"capabilities": [
"service-manager"
],
"claims": [
{
"name": "node-resolver",
"scope": "node",
"serves": [
"routes",
"route",
"unroute"
]
}
],
"listens": [
{
"name": "dns-udp",
"port": 53,
"protocol": "udp",
"from": "machine",
"fixed": true,
"why": "this machine's own resolver (ADR 0247), on loopback and on its private address for its own containers; never another machine's, so a VPN's domains routed here are asked by nothing beyond this machine"
},
{
"name": "dns-tcp",
"port": 53,
"protocol": "tcp",
"from": "machine",
"fixed": true,
"why": "the same names over tcp, which a resolver answers on for an answer too large for a datagram"
}
],
"facts": {
"kept": {
"path": "/etc/node-resolver/resolv.conf",
"template": "# Managed by the mesh, and written by the module holding this machine's own resolver\n# (module systemd-resolved, novox/hq ADR 0247). On every other machine the module holding\n# the uplink writes this file, listing the mesh's resolvers (ADR 0223); here something\n# requires names routed by domain - a VPN client's domains to its own servers - so the file\n# names this machine's own resolver alone, which sends those domains over the VPN's link and\n# every other name to the mesh's resolvers. One server listed, so one answer per name.\n#\n# Its address on the private network, not loopback: a container copies this file, and\n# this address is one it can reach. Another program writing this file is put back by the\n# module's guard, which keeps what it wrote for whoever handles it on this machine.\n# Replaced on every push; edit nothing here.\n{{$own := \"\"}}{{range .Machines}}{{if eq .Name $.Node}}{{$own = .Address}}{{end}}{{end}}nameserver {{if $own}}{{$own}}{{else}}127.0.0.53{{end}}\noptions timeout:1 attempts:2 edns0\n"
},
"resolved": {
"path": "/etc/systemd/resolved.conf.d/50-mesh.conf",
"template": "# Managed by the mesh (module systemd-resolved, novox/hq ADR 0247). Replaced on every\n# push; a drop-in of the operator's that sorts after this one overrides it, and is theirs.\n#\n# The mesh's resolvers, every one of them (ADR 0223): they answer every name no link's own\n# domains route elsewhere. ~. makes them the default route for names, and a link's servers\n# answer only the domains routed to them (the module's verb `route`).\n[Resolve]\nDNS={{range index .Holders \"mesh-dns-resolver\"}}{{.Address}} {{end}}\nDomains=~.\n# No compiled-in public fallback: a public resolver beside the mesh's is what ADR 0223\n# removed, because one of them said \"no such name\" for a mesh name and was believed.\nFallbackDNS=\n# The stub on loopback for this machine, and on its private address for its containers,\n# which cannot reach loopback. The packet filter admits this machine's own guests and\n# nobody else: another machine never asks this resolver (ADR 0247).\nDNSStubListener=yes\n{{$own := \"\"}}{{range .Machines}}{{if eq .Name $.Node}}{{$own = .Address}}{{end}}{{end}}DNSStubListenerExtra={{$own}}\n# No cache: nothing on this machine keeps a copy of a mesh name or of a \"no such name\",\n# as before this resolver - each question is asked again (ADR 0223's objection to a\n# local forwarder was a copy disagreeing with the truth).\nCache=no\n# What this machine's own programs read from /etc/hosts they read themselves; containers\n# asking here get what the mesh's resolvers say, as before.\nReadEtcHosts=no\n# Names are the mesh's resolvers' and a routed link's to answer, never the local network's\n# guesses; validation stays the upstreams' (the mesh's resolvers pass the bit through).\nLLMNR=no\nMulticastDNS=no\nDNSSEC=no\nDNSOverTLS=no\n"
},
"resolvers": {
"path": "/etc/resolv.conf",
"template": "# Managed by the mesh, and written by the module holding this machine's own resolver\n# (module systemd-resolved, novox/hq ADR 0247). On every other machine the module holding\n# the uplink writes this file, listing the mesh's resolvers (ADR 0223); here something\n# requires names routed by domain - a VPN client's domains to its own servers - so the file\n# names this machine's own resolver alone, which sends those domains over the VPN's link and\n# every other name to the mesh's resolvers. One server listed, so one answer per name.\n#\n# Its address on the private network, not loopback: a container copies this file, and\n# this address is one it can reach. Another program writing this file is put back by the\n# module's guard, which keeps what it wrote for whoever handles it on this machine.\n# Replaced on every push; edit nothing here.\n{{$own := \"\"}}{{range .Machines}}{{if eq .Name $.Node}}{{$own = .Address}}{{end}}{{end}}nameserver {{if $own}}{{$own}}{{else}}127.0.0.53{{end}}\noptions timeout:1 attempts:2 edns0\n"
},
"suffix": {
"path": "/etc/node-resolver/suffix",
"template": "{{.Suffix}}\n"
}
},
"resources": [
{
"id": "service",
"type": "service",
"unit": "systemd-resolved.service",
"state": "running",
"boot": "enabled",
"restart-on": [
"systemd-resolved.fact-resolved"
],
"health": {
"kind": "unit"
}
},
{
"id": "guard",
"type": "process",
"name": "systemd-resolved-guard",
"artifact": "tools",
"run": [
"./resolver-tools",
"guard"
],
"health": {
"kind": "unit"
}
}
],
"build": {
"artifacts": [
{
"name": "tools",
"kind": "bundle",
"language": "go",
"system": "arch",
"from": "cmd/resolver-tools",
"binary": "resolver-tools",
"loads": [
"resolver-tools"
]
}
]
}
}