dnsmasq admits a query by the interface it arrives on when told interface=, and by the address it is sent to when told listen-address=. A container's query is sent to the machine's private address but arrives on docker0, so interface=mesh0 dropped it silently on every machine (novox/hq issue 110). Name the address, not the interface.
97 lines
9.4 KiB
JSON
97 lines
9.4 KiB
JSON
{
|
|
"module": "dnsmasq",
|
|
"version": "1",
|
|
"provides": [
|
|
"wildcard-resolution"
|
|
],
|
|
"requires": [
|
|
"mesh-addressing"
|
|
],
|
|
"emits": [
|
|
"name.added",
|
|
"name.removed"
|
|
],
|
|
"own-secrets": {
|
|
"broker": "/var/lib/mesh/dnsmasq/broker"
|
|
},
|
|
"claims": [
|
|
{
|
|
"name": "node-dns-resolver",
|
|
"scope": "node"
|
|
}
|
|
],
|
|
"listens": [
|
|
{
|
|
"name": "dns-udp",
|
|
"port": 53,
|
|
"protocol": "udp",
|
|
"from": "mesh",
|
|
"why": "every name for this machine and what it runs — the mesh's own answered here, the rest forwarded",
|
|
"fixed": true
|
|
},
|
|
{
|
|
"name": "dns-tcp",
|
|
"port": 53,
|
|
"protocol": "tcp",
|
|
"from": "mesh",
|
|
"why": "the same names over tcp, which a resolver answers on as well and is asked for whenever an answer will not fit in a datagram. Declared because the daemon serves it: a declaration that covers one of the two protocols its own service listens on leaves the other closed while everything reports success",
|
|
"fixed": true
|
|
}
|
|
],
|
|
"resources": [
|
|
{
|
|
"id": "mesh-state",
|
|
"type": "directory",
|
|
"path": "/var/lib/mesh/dnsmasq",
|
|
"mode": "0700"
|
|
},
|
|
{
|
|
"id": "package",
|
|
"type": "package",
|
|
"package": "dnsmasq"
|
|
},
|
|
{
|
|
"id": "config",
|
|
"type": "file",
|
|
"path": "/etc/dnsmasq.conf",
|
|
"mode": "0644",
|
|
"content": "# Managed by the mesh. dnsmasq's own defaults are replaced whole rather than\n# patched, because this module owns the file and a patch would leave whatever\n# was there before to be discovered later.\n\n# What the mesh computed: one wildcard per machine — its name and everything\n# under it — and the mesh's own suffix as a local domain, so a name under it is\n# answered here or not at all and is never asked upstream. Rewritten whenever a\n# machine joins or leaves, which is why the service below restarts on it: a\n# reload makes dnsmasq re-read hosts files, not its configuration, and a\n# wildcard is configuration.\nconf-file=/etc/mesh-resolver/nodes.conf\n\n# Where it answers. Both are addresses the mesh chose, so this file needs to\n# know nothing about this particular machine beyond what the mesh fills in:\n#\n# the machine's private address\n# so anything on the private network can ask — and so this\n# machine's containers can. This module writes the runtime's\n# `dns` key into its own configuration file, beside whatever the\n# machine had there (novox/hq ADR 0102), naming this address: a\n# container cannot reach the machine's loopback, and a runtime\n# whose host resolves at loopback falls back to a public resolver\n# and never sees a mesh name. The runtime reads that key when it\n# starts and not on a reload, and a restart stops every container\n# on the machine. So the mesh RELOADS the runtime when this file\n# changes (novox/hq ADR 0102: reload, don't restart) and never\n# restarts it. A reload does not make `dns` take effect — the\n# runtime reads that key only when it starts — but it does turn on\n# `live-restore`, which this file also sets, and with that on a\n# restart keeps every container running. So the one restart the\n# `dns` key needs is the operator's to make, once, and harmless\n# from the second time on. Two machines sat for weeks with this\n# file written and a runtime that predated it, handing every\n# container a public resolver while everything read as fine\n# (novox/hq 04-ISSUES/110).\n#\n# Named as an ADDRESS and not as the interface that carries it,\n# on purpose. A container's query is addressed to this address\n# but arrives on the runtime's bridge, and dnsmasq checks every\n# query against what it was told to answer on: an `interface=`\n# line admits queries by the interface they arrive on, a\n# `listen-address=` line by the address they are sent to. With\n# `interface=mesh0` alone, a query from a container was received\n# — the socket was bound, the filter admitted it — and dropped\n# without a word, on every machine, and the container waited\n# until it gave up (novox/hq 04-ISSUES/110). The address admits\n# it whatever bridge it comes in on, which is the point: the\n# bridges are the runtime's, named by it, and this file should\n# not have to know them.\n# 127.0.0.1 this machine's own use. The predecessor's resolver answered\n# here, and the resolv.conf it wrote on every machine says so;\n# that file stays in force on an adopted machine until the mesh's\n# module for it is taken, so the resolver has to answer where the\n# machine already asks or the machine loses DNS the moment this\n# module is taken. Not .53 or .54: systemd-resolved holds BOTH —\n# .53 is its stub and .54 its proxy stub — and neither is .1, so\n# the two coexist on a machine that runs it. This module used to\n# answer on 127.0.0.55 instead: a convention of its own, beside\n# the one every machine already followed. One address, this one,\n# and the modules that point a machine at the mesh name the same.\n#\n# Whatever address it listens on, it takes the machine's DNS port.\n# That is why this module claims `node-dns-resolver`.\n#\n# bind-dynamic rather than bind-interfaces: the private address does not exist\n# until the machine is on the private network, and binding an address that is\n# not there yet fails to start rather than waiting for it.\nbind-dynamic\nlisten-address=${machine:address}\nlisten-address=127.0.0.1\n\n# **It must never read resolv.conf to find out where to forward.** Whatever\n# points this machine at the mesh writes this resolver's own address there — so\n# a resolver that read it for upstreams would find itself, and every query it\n# could not answer locally would loop until its receive queue filled. That is\n# not theoretical: it filled with 15KB of queries and every lookup on the\n# machine hung. no-resolv is what makes that loop impossible: the upstreams are\n# the two lines below, and nothing on the machine can redirect them.\n#\n# It forwards, because it is now asked for everything. The module that points\n# this machine at the mesh names this resolver alone — as the predecessor's\n# did — so the host and every container resolve the world through it. The\n# upstreams are the ones the predecessor's module shipped as its defaults. The\n# mesh's own names never reach them: the local= line in the file above stops\n# them here, answered or refused.\nno-resolv\nserver=1.1.1.1\nserver=8.8.8.8\n\n# A name without a dot is never forwarded — a bare hostname is answered from\n# /etc/hosts or not at all — and reverse lookups of private ranges are answered\n# here rather than asking the world who 10.x is.\ndomain-needed\nbogus-priv\n\n# The operator's own names have a home the mesh never rewrites (novox/hq issue\n# 122: a workstation's job includes names — Mediahuis's 13, say — that are\n# neither a mesh machine nor a routed name). Two homes, because both shapes\n# exist in the wild and neither is the mesh's to own:\n#\n# /etc/dnsmasq.d/*.conf drop-in dnsmasq directives — an address=, a second\n# upstream for one domain, a cname. HAL's dnsmasq-app\n# carried exactly this line, so it is a proven shape\n# and the files a migrating workstation already has\n# land here untouched.\n# /etc/hosts.local plain `<ip> <name>` lines, the /etc/hosts a person\n# kept — read as additional hosts, so the generated\n# /etc/hosts (which the mesh owns and rewrites) never\n# has to carry an operator entry to keep it resolving.\n#\n# Both are the operator's: the mesh creates neither and rewrites neither, and a\n# machine with no such file loses nothing. This is what lets mesh-wireguard take\n# /etc/hosts without taking the names a workstation needs down with it — they\n# were moved here first.\nconf-dir=/etc/dnsmasq.d/,*.conf\naddn-hosts=/etc/hosts.local\n"
|
|
},
|
|
{
|
|
"id": "runtime-dns",
|
|
"type": "file",
|
|
"path": "/etc/docker/daemon.json",
|
|
"mode": "0644",
|
|
"merge": "json",
|
|
"into": "json",
|
|
"content": "{\"dns\": [\"${machine:address}\"], \"live-restore\": true}\n"
|
|
},
|
|
{
|
|
"id": "runtime",
|
|
"type": "service",
|
|
"unit": "docker.service",
|
|
"state": "running",
|
|
"reload-on": [
|
|
"runtime-dns"
|
|
]
|
|
},
|
|
{
|
|
"id": "service",
|
|
"type": "service",
|
|
"unit": "dnsmasq.service",
|
|
"state": "running",
|
|
"boot": "enabled",
|
|
"restart-on": [
|
|
"config",
|
|
"dnsmasq.fact-node-zones"
|
|
]
|
|
}
|
|
],
|
|
"facts": {
|
|
"node-zones": {
|
|
"path": "/etc/mesh-resolver/nodes.conf",
|
|
"template": "# Generated by the mesh. Do not edit — this file is replaced whenever a machine\n# joins or leaves, and an edit would survive until then and vanish.\n\nlocal=/{{.Suffix}}/\n{{range .Machines}}address=/{{.FQDN}}/{{.Address}}\n{{end}}"
|
|
}
|
|
}
|
|
}
|