diff --git a/00-META/glossary.md b/00-META/glossary.md index 70e498a..93686a3 100644 --- a/00-META/glossary.md +++ b/00-META/glossary.md @@ -80,7 +80,8 @@ another — and a mesh you cannot name precisely is a mesh two people describe d The set, with who holds each seat, is the overview of what a mesh has ([26 — The seats](../03-DESIGN/01-to-be/26-the-seats.md)). A seat has a **capacity**: a capacity-1 seat is exclusive (one holder); a higher-capacity seat is a **bench** (several holders - coexist). + coexist). The first bench is `mesh-dns-resolver`, a *replicated* mesh seat: one holder per machine, + each on record and each answering the same names ([ADR 0223](../02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md)). - **claim** — a module taking a spot on a seat. `claims: [{name, scope}]` in a manifest. A mesh-scoped exclusive claim is how the mesh says "there is one of me". A foundation seat is named after the server it guards: the `mesh-controller`, `postgres` and `lavinmq` modules claim diff --git a/02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md b/02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md index f128413..77614de 100644 --- a/02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md +++ b/02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md @@ -11,6 +11,11 @@ extends: 0191-the-meshs-resolver-holds-only-the-meshs-own-names.md # 194. The mesh has one resolver, and every node asks it for the mesh's names +> **The mechanism changed — 2026-10-05, by [ADR 0223](0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md).** `mesh-resolver` (named `mesh-dns-resolver` in the +> set) is no longer of capacity one: it is a replicated seat, held on the anchor and on the home +> server, each holding every node's internal domain from the same roster. One resolving module, the +> mesh's names held only by its holders, and the retirement of every per-node copy stand. + > **Narrowed, not replaced — 2026-10-03.** How a node asks is decided again by > [ADR 0196](0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md): every > node and container asks `mesh-resolver` first and a public resolver only when it is silent. There is diff --git a/02-DECISIONS/0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md b/02-DECISIONS/0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md index e07bc6a..0d19261 100644 --- a/02-DECISIONS/0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md +++ b/02-DECISIONS/0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md @@ -10,6 +10,13 @@ supersedes-in-part: # 196. A node asks the mesh's resolver first, and a public one only when it is silent +> **Narrowed, not replaced — 2026-10-05, by [ADR 0223](0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md).** The public resolver listed second goes: musl asks +> every listed server at once and takes the first reply, so a public "no such name" for a mesh name +> won it, and every Alpine build on the home server failed. Every machine now lists the mesh's +> resolvers — two holders of `mesh-dns-resolver`, its own first on a holder — and nothing else. Every +> node and container asking the mesh's resolver for every name, with no stub and no runtime `dns`, +> stands. The "fallback" consequence and check below describe what this record decided, not what runs. + ## Context **[ADR 0194](0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md) gave the diff --git a/02-DECISIONS/0199-a-module-that-answers-names-declares-its-zone-and-a-nodes-hosts-file-is-one-modules.md b/02-DECISIONS/0199-a-module-that-answers-names-declares-its-zone-and-a-nodes-hosts-file-is-one-modules.md index e4679ff..2ef3f69 100644 --- a/02-DECISIONS/0199-a-module-that-answers-names-declares-its-zone-and-a-nodes-hosts-file-is-one-modules.md +++ b/02-DECISIONS/0199-a-module-that-answers-names-declares-its-zone-and-a-nodes-hosts-file-is-one-modules.md @@ -9,6 +9,10 @@ extends: 0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-nam # 199. A module that answers names declares its zone, and a node's hosts file is one module's +> **Decided to change — 2026-10-05, by [ADR 0223](0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md), not yet built.** The `hosts` module is renamed `hostname` +> and its seat `node-hostname`, and it owns `/etc/hostname` as well as `/etc/hosts`; the operator's +> lines are kept as this record decides. Until that is built, everything below stands as decided. + ## Context **[ADR 0194](0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md) and diff --git a/02-DECISIONS/0220-what-a-machine-asks-needs-its-uplink-held-and-the-retired-resolver-pieces-go.md b/02-DECISIONS/0220-what-a-machine-asks-needs-its-uplink-held-and-the-retired-resolver-pieces-go.md index 018788b..47826fa 100644 --- a/02-DECISIONS/0220-what-a-machine-asks-needs-its-uplink-held-and-the-retired-resolver-pieces-go.md +++ b/02-DECISIONS/0220-what-a-machine-asks-needs-its-uplink-held-and-the-retired-resolver-pieces-go.md @@ -9,6 +9,10 @@ extends: 02-DECISIONS/0207-a-module-depends-on-the-node-seats-that-apply-its-res # 220. What a machine asks needs its uplink held, and the retired resolver pieces go +> **Decided to go — 2026-10-05, by [ADR 0223](0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md), not yet built.** `/etc/resolv.conf` becomes the `node-uplink` +> holder's file, and `resolv-conf`, `node-resolver-config` and this record's dependency of it on +> `node-uplink` retire with it. Until that is built, everything below stands as decided. + ## Context **Three things about a machine's resolver were left half done when the mesh moved to one resolver.** diff --git a/02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md b/02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md new file mode 100644 index 0000000..842f160 --- /dev/null +++ b/02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md @@ -0,0 +1,193 @@ +--- +topic: the tiers +status: accepted +date: 2026-10-05 +deciders: jochen +reconstructed: false +supersedes-in-part: + - 0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md +extends: 02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md +--- + +# 223. The mesh has two resolvers, and a machine lists only them + +## Context + +**[ADR 0196](0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md) wrote +every machine's `/etc/resolv.conf` as the mesh's resolver first and a public resolver second.** Its +reasoning is the C library's: servers are asked in order, the next only when one does not answer, and +an answer from the first — "no such name" included — is final. That holds for glibc. It does not hold +for musl, the C library of every Alpine image: musl sends the query to every listed server at once +and takes the first reply. + +**On the production mesh on 2026-10-05 that made the anchor's own name unresolvable from the home +server's builds.** The build agent on the home server runs its steps in Alpine containers on the host +network, so they read the machine's `resolv.conf` as it is. Asked for `.internal`, the public +resolver — which has no such name and is the nearer of the two — answered NXDOMAIN first, and musl +took it. Reproduced six times out of six inside the build agent's own container; every build on that +machine failed fetching from the anchor. The same lookup from glibc on the same machine answered every +time. [Issue 262](../04-ISSUES/262-an-alpine-container-could-not-find-a-machine-by-its-mesh-name/00-report.md) +was the same library failing on a different answer (NXDOMAIN for a missing IPv6 record); fixing that +did not touch this one, because here the wrong answer comes from a server that should never have been +asked. + +**The public line was there for one case: the mesh's resolver unreachable.** ADR 0196 kept public +names resolving with the anchor down, the tunnel down, or a laptop behind a captive portal. That case +is real and rare; the musl case is every lookup of a mesh name from every Alpine container on any +machine that is not the anchor. + +**A seat's work can already be shared by several holders.** [ADR 0190](0190-a-seats-work-is-shared-by-its-holders-and-building-is-the-first-such-role.md) +made the build role node-scoped with every holder pulling from one queue. A mesh-scoped seat has +always had exactly one holder: by derivation, or on record since +[ADR 0131](0131-everything-on-the-mesh-speaks-to-the-broker-seat.md), where a handover replaces the +holder in one write and every other eligible assignment stands beside it, silent. + +## Considered Options + +1. **The mesh's resolver alone.** Every mesh name and every public name answered consistently, by one + server. Rejected alone: with the anchor down, no machine resolves anything — the reason ADR 0194 + gave against it, and still true. +2. **A local forwarder on every machine** — a small resolver on loopback that asks the mesh's resolver + for the mesh's suffix and public resolvers for the rest, with `resolv.conf` naming only it. Rejected: + it is ADR 0194's per-node stub again, which ADR 0196 removed — a daemon and a module on every node, + and a container cannot use a loopback resolver, so containers would need a second configuration. + Every resolution fault found on 2026-10-03 was a per-node copy disagreeing with the truth. +3. **Split by kind of machine** — servers list the mesh's resolver alone, laptops keep a public + fallback. Rejected: a laptop runs Alpine containers too, and a rule that differs by machine is a + rule nobody can state about the mesh. +4. **Two mesh resolvers, and nothing else listed.** The same module, the same roster and the same zones + on two machines, and every machine lists both. Whichever answers first gives the same answer, so + musl's race is harmless; a public name still resolves through either. Chosen. + +## Decision + +**1. Now: the mesh has two resolvers.** The seat `mesh-dns-resolver` may be held on more than one +machine — the anchor and the home server — each holder answering the same mesh names: the same +module, the same machine list, the same zones, rendered by the controller into each. + +- **A seat may be replicated.** It is an attribute of the seat in the mesh's definition, compiled with + the set and never stored, as what a seat needs ([ADR 0220](0220-what-a-machine-asks-needs-its-uplink-held-and-the-retired-resolver-pieces-go.md)) + and what it receives are. `mesh-dns-resolver` is the only replicated seat. Making another one is a + decision, recorded. In the glossary's words it is the mesh's first **bench** — a seat whose + holders coexist — and *replicated* says what kind: every holder answers the same thing. +- **Every holder is on record, and each is added by an act**: `seat mesh-dns-resolver --add + /` records one more holder beside those on record. Assigning the module is not enough: + an assignment not on record stands beside the holders, eligible and silent, exactly as ADR 0131 says, + and two claimants with nothing on record are refused as for any mesh seat. `--to` still hands the + seat over, leaving exactly one holder. `--add` on a seat held once is refused, naming `--to`. +- **One per machine still**: two modules on one machine claiming it are refused. And a seat held once + stays held once: a second claimant on another machine is refused while nothing is on record, and a + store recording two holders of such a seat is refused, naming the seat. +- **Every machine's `/etc/resolv.conf` lists every holder's private address, the holder on the machine + itself first if it is one, then the rest in name order, and no public resolver.** The controller + gives a module's roster template the holders of each replicated seat, ordered so; the module holding + `node-resolver-config` writes the file from it. On a holder, its own private address is first: the + resolver listens there and on loopback, and the private address is the one a container on that + machine can reach. +- **The requirement stays.** What writes the file still requires `wildcard-resolution`, so a machine is + refused when nothing in the mesh resolves, rather than given a file listing nothing. A holder answers + its own requirement; any other machine is bound to the first holder by name. Nothing reads that + binding's address any more — the file lists every holder — and the binding is kept for the refusal + and the order of delivery. + +**2. Next, decided and not yet built: `/etc/resolv.conf` belongs to the uplink's holder.** The program +that manages the machine's network already has to be told to keep off the file +([ADR 0117](0117-a-machines-uplink-is-a-seat.md)); instead, the `node-uplink` holder writes it, given +the resolvers by the mesh: NetworkManager through its global DNS configuration, dhcpcd through static +nameservers, and systemd-networkd's module declaring the file itself. `resolv-conf` and the seat +`node-resolver-config` then retire, and ADR 0220's dependency of `node-resolver-config` on `node-uplink` +goes with them. One owner for the file, and it is the program that would otherwise rewrite it. + +**3. Next, decided and not yet built: a machine's names are one seat's.** `/etc/hosts` and +`/etc/hostname` belong to one seat for the machine's identity. The `hosts` module, holding +`node-hosts-file` ([ADR 0199](0199-a-module-that-answers-names-declares-its-zone-and-a-nodes-hosts-file-is-one-modules.md)), +is renamed **`hostname`** and holds the seat renamed **`node-hostname`**; it writes `/etc/hostname` +and the machine's own `127.0.1.1` line, and keeps every operator's line as ADR 0199 does today. + +- **Why one seat for both files**: the mesh's only content in `/etc/hosts` is the machine's own name, + and nothing owns `/etc/hostname` today. Two files saying one fact belong to one owner. +- **Why `node-hostname`**: a system seat is named `node-` for its scope and then for its role + ([ADR 0121](0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md)), and the role + is the machine's name. `node-identity` was considered and rejected: a machine's identity in the mesh + already means its key and certificate. `node-host` was rejected for the reason the module name + `host` was: "the host" is what the [node-engine](../00-META/glossary.md) was called until now, and + its repository and binary still carry `mesh-host`. +- **Why a module and not part of the node-engine**: the node-engine applies every module's resources + and owns no file's content; every file it writes belongs to the module that declared it. A file's + content belongs to a seat's holder, and a seat's holder stays replaceable — another module can hold + `node-hostname` on a machine that names itself another way, and the node-engine does not change. +- The rename goes through the seat set's rename (an alias keeps `node-hosts-file` resolving, + [ADR 0122](0122-a-seat-is-data-a-rename-is-a-database-update.md)), so nothing claiming the old name + breaks in between. + +## Consequences + +- **musl and glibc agree.** Every listed server answers a mesh name the same way, so the race musl + runs has one possible outcome; a public name resolves through either holder's upstreams. +- **Accepted cost: with no mesh resolver reachable, a machine has no DNS at all** until it can reach + one. The anchor is the hub of the private network, so with it down the private network is down too, + and the second resolver is reachable only on its own machine and from machines on its own LAN. The + mesh assumes the anchor is up about 99.9% of the time. The same holds for a laptop behind a captive + portal that keeps the tunnel down: it resolves nothing, the portal's own name included, until the + tunnel is up. +- **The resolver's options change from one attempt to two**, one second each. With no public resolver + to fall back to, a single dropped datagram would otherwise fail a lookup on a machine that reaches + only one resolver. +- **Holdings are keyed by seat and assignment.** The store's holding table takes a numbered migration; + every existing row is one per seat and satisfies the new key. A handover replaces every holder in one + transaction. +- **Unassigning a holder takes its own row only**: the other resolver keeps holding. Removing the + second resolver is unassigning it, then `push --behind`. +- **The rollout order matters.** The controller that knows replicated seats and renders the holders + rolls out first; the catalogue's `resolv-conf`, which reads the holders, second — a controller without + them cannot render it; and only then is the second holder added, because an older `resolv-conf` + reading its one binding would name the first holder by name order, which may be the new one, and + still list the public resolver. +- **A container keeps the resolvers it started with.** A container on the default bridge copies its + machine's `resolv.conf` when it starts; one on a user-defined network is answered by the runtime's + embedded resolver, which forwards to the servers it read at start. Either keeps the public resolver + until restarted. +- **Parts 2 and 3 change who writes two files on every machine**, each a handover between modules on + the same path; they wait for their own build handoff. + +## How it is checked + +| Rule | Checked by | +|---|---| +| `mesh-dns-resolver` is replicated, and no other seat is; it survives loading the set from the store | mesh-controller unit test over the compiled set and over a store row without the attribute | +| Two holders on record compose on both, with no refusal, and each lists itself first | mesh-controller resolution and composition test with the catalogue's `dnsmasq` and `resolv-conf` | +| A machine holding no resolver lists both holders, by name, and no public resolver | the same test on a third machine, asserting no public address appears | +| A holder answers its own requirement although another holder sorts first | a resolution test on the second holder (issue 258's case kept) | +| A seat held once still refuses a second claimant, and refuses two holders on record | resolution tests on `mesh-store` | +| Two claimants of the replicated seat with nothing on record are refused, naming the handover | a resolution test | +| Every consumer is bound to the same holder whatever order the mesh was resolved in | a unit test on the holder among providers | +| The store keeps several holders of one seat, once each; a handover leaves one; unassigning takes only its own row | a store test against a live database, through the migration | +| `seat --add` is refused for a seat held once | the controller's `seat` command | +| The resolver's machine list has one host record per machine (issue 262's missing check) | a mesh-controller composition test counting host records | +| `resolv-conf` lists only the seat's holders, with two short attempts | a mesh-controller test reading the catalogue's `resolv-conf` | +| Live: every machine's `/etc/resolv.conf` lists both holders' private addresses, its own first on a holder, and nothing else; an Alpine container on the host network of the home server resolves `.internal` every time | after rollout, read through each machine's tools, and `getent hosts` in an Alpine container repeated ten times | +| Parts 2 and 3 | not yet built; their checks are written with their handoff | + +## References + +- [ADR 0194](0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md) — the + mesh's resolver; now held on two machines, each holding every node's internal domain. +- [ADR 0196](0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md) — + replaced in part: its public resolver second goes. Every node and container asking the mesh's + resolvers for every name, with no stub and no runtime `dns`, stands. +- [ADR 0190](0190-a-seats-work-is-shared-by-its-holders-and-building-is-the-first-such-role.md) — + several holders sharing a role, for a node seat. +- [ADR 0131](0131-everything-on-the-mesh-speaks-to-the-broker-seat.md) — holders on record, handover, + eligible and silent. +- [ADR 0220](0220-what-a-machine-asks-needs-its-uplink-held-and-the-retired-resolver-pieces-go.md) — the + uplink's dependency, which part 2 retires. +- [ADR 0117](0117-a-machines-uplink-is-a-seat.md), [ADR 0199](0199-a-module-that-answers-names-declares-its-zone-and-a-nodes-hosts-file-is-one-modules.md), + [ADR 0121](0121-a-system-seat-is-named-for-its-scope-and-modules-define-their-own.md), + [ADR 0122](0122-a-seat-is-data-a-rename-is-a-database-update.md). +- [Issue 258](../04-ISSUES/258-every-machine-bound-the-resolver-to-itself/00-report.md), + [issue 262](../04-ISSUES/262-an-alpine-container-could-not-find-a-machine-by-its-mesh-name/00-report.md). +- [Connectivity §2](../03-DESIGN/01-to-be/08-connectivity.md) and [the seats](../03-DESIGN/01-to-be/26-the-seats.md), + amended alongside. +- mesh-controller `internal/catalogue/seats.go`, `resolve.go`, `roster.go`, + `cmd/mesh-controller/seats.go`, `holdings.go`, and the store migration keying a holding by seat and + assignment; mesh-catalog `modules/resolv-conf`, `modules/dnsmasq`. diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index 6dc4472..99a269d 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -235,6 +235,7 @@ python3 00-META/checks/index.py fail if stale - **0194** — [The mesh has one resolver, and every node asks it for the mesh's names](0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md) - **0196** — [A node asks the mesh's resolver first, and a public one only when it is silent](0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md) - **0199** — [A module that answers names declares its zone, and a node's hosts file is one module's](0199-a-module-that-answers-names-declares-its-zone-and-a-nodes-hosts-file-is-one-modules.md) +- **0223** — [The mesh has two resolvers, and a machine lists only them](0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md) ### What runs on them, and how it gets there diff --git a/03-DESIGN/01-to-be/08-connectivity.md b/03-DESIGN/01-to-be/08-connectivity.md index 1fc4039..161a3e3 100644 --- a/03-DESIGN/01-to-be/08-connectivity.md +++ b/03-DESIGN/01-to-be/08-connectivity.md @@ -9,13 +9,16 @@ code: - mesh-controller internal/catalogue/zones.go (the zones a module answers, ADR 0199) - mesh-controller internal/catalogue/seats.go (mesh-dns-resolver, node-hosts-file, node-resolver-config needing node-uplink) - mesh-controller internal/catalogue/seat_dependencies.go (a seat's need checked at assignment, ADR 0220) - - mesh-catalog modules/dnsmasq (the mesh's one resolver) + - mesh-controller cmd/mesh-controller/holdings.go (the holders of a replicated seat, ADR 0223) + - mesh-controller internal/catalogue/roster.go (each replicated seat's holders, for a template) + - mesh-catalog modules/dnsmasq (the mesh's resolvers) - mesh-catalog modules/resolv-conf (what a node asks) - mesh-catalog modules/hosts (a node's /etc/hosts) - mesh-host internal/identity/serving.go - mesh-host internal/apply (the service that reflects a rule set) updated: 2026-10-05 decisions: + - 02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md - 02-DECISIONS/0220-what-a-machine-asks-needs-its-uplink-held-and-the-retired-resolver-pieces-go.md - 02-DECISIONS/0199-a-module-that-answers-names-declares-its-zone-and-a-nodes-hosts-file-is-one-modules.md - 02-DECISIONS/0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md @@ -300,10 +303,10 @@ expensively enough to be worth restating: - **A node must not pin its own public name locally.** The duplicate record breaks resolution of that name for everything else that needs it. -**What the host receives:** what to ask, not what to answer. The mesh has **one resolver**, holding -every node's internal domain; a node asks it first and a public resolver only when it is silent +**What the host receives:** what to ask, not what to answer. The mesh has **two resolvers**, each +holding every node's internal domain; a node lists both and nothing else ([ADR 0194](../../02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md), -[ADR 0196](../../02-DECISIONS/0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md)). +[ADR 0223](../../02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md)). **What goes away:** the `/etc/hosts` floor. It exists because a node had to reach the mesh database before its own DNS existed; with [ADR 0004](../../02-DECISIONS/0004-a-node-and-how-it-joins.md) @@ -361,24 +364,47 @@ not a list of containers. somebody starts by hand is not the mesh's to configure, and reaching into every container on a machine — declared or not — is what a nameserver in `resolv.conf` would be for. -### One resolver for the mesh +### The mesh's resolvers -*2026-10-03.* **The mesh's names live in one place: the module holding `mesh-resolver`**, a mesh-scoped -seat of capacity one, placed on the node every tunnel converges on. It holds one wildcard per node — -`.internal` and everything under it — and listens on the private network only. It answers the -mesh's names from what it holds and forwards every other name, giving the public answer. +*2026-10-03, revised 2026-10-05.* **The mesh's names live in one module, held on two machines: the +holders of `mesh-dns-resolver`**, a mesh-scoped seat that is *replicated* — held on the anchor and on +the home server, each running the same module with the same machine list and the same zones, rendered +by the controller into each. Each holds one wildcard per node — `.internal` and everything +under it — and one host record per node, listens on its private address and loopback only, answers +the mesh's names from what it holds and forwards every other name, giving the public answer +([ADR 0194](../../02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md), +[ADR 0223](../../02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md)). +Each holder is on record, added by `seat mesh-dns-resolver --add /`; an assignment of +the module not on record stands beside them, eligible and silent, and a seat held once stays held +once. *Checked by the controller's resolution tests with two holders on record, with two claimants +and nothing on record, and on a seat held once.* -**Every node asks it for everything, and a public resolver only when it is silent.** The module -holding `node-resolver-config` writes `/etc/resolv.conf` naming `mesh-resolver` first and a public -resolver second, with a short timeout and one attempt: the C library moves to the second only when the -first does not answer — the anchor or the tunnel down, a captive portal holding the tunnel back — so -public names keep resolving then, and `.internal` is never asked of a public resolver while the mesh's -answers. Containers take the same two from their machine, the runtime copying non-loopback resolvers -into every container, so the runtime is given no `dns` of its own +**Every node asks them for everything, and nothing else.** The module holding +`node-resolver-config` writes `/etc/resolv.conf` listing every holder's private address — the holder +on the machine itself first if it is one, then the rest by name — with a short timeout and two +attempts, and **no public resolver**. ADR 0196 listed a public resolver second, for the anchor being +unreachable; a C library that asks every listed server at once and takes the first reply — musl, so +every Alpine container — took the public resolver's "no such name" for a mesh name, and every build on +the home server failed. With only the mesh's resolvers listed, whichever answers first gives the one +answer. The cost is stated: a machine that reaches no mesh resolver has no names until it does, and +with the anchor down the second resolver is reachable only on its own machine and its own LAN. +Containers take the same lines from their machine, the runtime copying non-loopback resolvers into +every container, so the runtime is given no `dns` of its own ([ADR 0196](../../02-DECISIONS/0196-a-node-asks-the-meshs-resolver-first-and-a-public-one-only-when-it-is-silent.md), replacing ADR 0194's per-node `systemd-resolved` stub). `resolv-conf` is the one module the catalogue offers for it; the systemd-resolved split-DNS module that once claimed the same seat is gone ([ADR 0220](../../02-DECISIONS/0220-what-a-machine-asks-needs-its-uplink-held-and-the-retired-resolver-pieces-go.md)). +*Checked by the controller's composition tests on both holders and on a third machine, and live by +each machine's `/etc/resolv.conf` and an Alpine container on the home server resolving the anchor's +name every time.* + +**Next, decided and not yet built** ([ADR 0223](../../02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md)): +`/etc/resolv.conf` becomes the file of the `node-uplink` holder, given the resolvers by the mesh — +NetworkManager through its global DNS configuration, dhcpcd through static nameservers, +systemd-networkd's module declaring the file — and `resolv-conf`, `node-resolver-config` and its need +for the uplink below retire. And a machine's own names become one seat's: the `hosts` module is +renamed `hostname` and holds `node-hostname`, writing `/etc/hostname` and the machine's `127.0.1.1` +line and keeping the operator's lines. **That file stays the mesh's only beside the uplink.** A network manager rewrites `/etc/resolv.conf` on every connectivity change unless it is told not to, and the module holding `node-uplink` is what @@ -388,17 +414,18 @@ machine with no uplink holder is refused, naming the managers' modules that coul the last uplink holder from beneath it is refused too ([ADR 0220](../../02-DECISIONS/0220-what-a-machine-asks-needs-its-uplink-held-and-the-retired-resolver-pieces-go.md)). *Checked by the controller's seat-dependency tests and by `assign` live.* -**No node holds a copy.** The per-node resolver, its zones file and the mesh's region of `/etc/hosts` +**No node holds a copy of its own.** The two holders hold the same rendering of one roster, never a +list anyone edits. The per-node resolver, its zones file and the mesh's region of `/etc/hosts` go, and the per-node resolver's seat with them, deleted from the set once nothing claimed it ([ADR 0220](../../02-DECISIONS/0220-what-a-machine-asks-needs-its-uplink-held-and-the-retired-resolver-pieces-go.md)): every resolution fault found on 2026-10-03 was a copy disagreeing with the truth — a hosts file read once at start, an operator's old line beside the mesh's, a node's resolver lent to a LAN. No member's resolver answers a LAN; a router pointing at one is moved first. *Checked by each node's -`/etc/resolv.conf` naming `mesh-resolver` then a public resolver, by no node but the holder answering +`/etc/resolv.conf` listing the resolver's holders and nothing else, by no node but a holder answering DNS on any address, and by the router's DHCP DNS option naming the router.* **Names that are neither a node nor a route.** A module that answers names declares a zone (a setting) and the listen that answers it; the controller hands the `mesh-dns-resolver` holder every -zone with its module's node address and published port, and the holder forwards that zone there and +zone with its module's node address and published port, and each holder forwards that zone there and answers nothing in it itself — the lab answers `.incus` for its running scenarios this way. An operator's own names, unrelated to the mesh, live in `/etc/hosts`'s kept region, held per node by the `node-hosts-file` seat's holder and changed through its tools; the controller holds none of them diff --git a/03-DESIGN/01-to-be/26-the-seats.md b/03-DESIGN/01-to-be/26-the-seats.md index 0c7d3aa..5d6235e 100644 --- a/03-DESIGN/01-to-be/26-the-seats.md +++ b/03-DESIGN/01-to-be/26-the-seats.md @@ -7,12 +7,15 @@ code: - mesh-controller internal/catalogue/seat_dependencies.go - mesh-controller internal/inventory/seats.go - mesh-controller internal/inventory/migrations/0039-a-seat-is-held-by-one-assignment-on-record.sql + - mesh-controller internal/inventory/migrations/0062-a-replicated-seat-has-several-holders-on-record.sql + - mesh-controller cmd/mesh-controller/holdings.go - mesh-controller cmd/mesh-controller/seats.go - mesh-controller cmd/mesh-controller/source.go - mesh-controller internal/inventory/migrations/0032-a-source-may-live-on-a-seat.sql - mesh-catalog modules/gitea/module.json updated: 2026-10-05 decisions: + - 02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md - 02-DECISIONS/0220-what-a-machine-asks-needs-its-uplink-held-and-the-retired-resolver-pieces-go.md - 02-DECISIONS/0199-a-module-that-answers-names-declares-its-zone-and-a-nodes-hosts-file-is-one-modules.md - 02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md @@ -38,7 +41,7 @@ A seat has four properties, fixed by the mesh rather than by any module: | property | is | |---|---| | name | what a definition names and an assignment holds, and what a person reads in the list | -| scope | node, site or mesh: where its capacity applies. Every seat in the set has a capacity of one, so one holder per scope. A bench, a seat with several holders, is a word the glossary keeps and no seat uses yet | +| scope | node, site or mesh: where its capacity applies. Every seat in the set has a capacity of one, so one holder per scope — except a *replicated* mesh seat, which may be held on several machines at once, one holder per machine, each on record ([ADR 0223](../../02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md)). `mesh-dns-resolver` is the only one. Replicated is part of the seat's definition, compiled with the set and never stored | | delivers | the provision its holder answers for, or nothing | | decision | the record that made it a seat | @@ -69,6 +72,17 @@ needs ([28](28-building-the-bus.md), task 5.3). **And a holding is the assignmen holder takes the row with it, so a seat never points at something that is not running anywhere, and the seat falls back to derivation rather than to nothing. +**A replicated seat has several holders on record** (revision, 2026-10-05, +[ADR 0223](../../02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md)). +`seat --add /` records one more holder beside those on record, judged exactly +as a handover is; `--to` still leaves exactly one. Each holder is added by that act and never by being +assigned: an assignment not on record is eligible and silent, and two claimants with nothing on record +are refused, as for any mesh seat. `--add` on a seat held once is refused, naming `--to`, and a store +recording two holders of a seat held once is refused at resolution, naming the seat. A requirement the +seat delivers is answered on a holder by itself, and elsewhere by the first holder in name order. What +the holders are is given to a module's roster template, its own machine first — how every machine's +resolver file lists both of the mesh's resolvers. + The handover refuses what would make the new holder wrong before anything is written: the seat must exist, the assignment must exist, and the module must be able to hold the seat — claim it at its scope and provide what it delivers, judged against the store's row and not against anything compiled into a @@ -130,13 +144,13 @@ convention, which later seats departed from. | `mesh-npm-package-registry` | `npm-package-registry` | mesh | `npm-package-registry` | the forge | | `mesh-git` | `git` | mesh | `git` | the forge | | `mesh-build-machine` | `the-build-machine` | node | — | a builder | -| `mesh-resolver` | — | mesh | — | the mesh's one resolver, holding every node's internal domain ([ADR 0194](../../02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md)) | +| `mesh-resolver` | — | mesh | — | the mesh's resolvers, each holding every node's internal domain ([ADR 0194](../../02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md)); replicated, held on the anchor and the home server ([ADR 0223](../../02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md)) | | ~~`mesh-dns-port`~~ | `the-dns-port` | node | — | retired by [ADR 0194](../../02-DECISIONS/0194-the-mesh-has-one-resolver-and-every-node-asks-it-for-the-meshs-names.md): the local resolver became the mesh's one; deleted from the set, and from the store's table, once nothing claimed it ([ADR 0220](../../02-DECISIONS/0220-what-a-machine-asks-needs-its-uplink-held-and-the-retired-resolver-pieces-go.md)) | -| `node-hosts-file` | — | node | — | owns `/etc/hosts`: the machine's own lines and the operator's kept region, changed through its verbs `entries`, `add`, `remove` ([ADR 0199](../../02-DECISIONS/0199-a-module-that-answers-names-declares-its-zone-and-a-nodes-hosts-file-is-one-modules.md)) | +| `node-hosts-file` | — | node | — | owns `/etc/hosts`: the machine's own lines and the operator's kept region, changed through its verbs `entries`, `add`, `remove` ([ADR 0199](../../02-DECISIONS/0199-a-module-that-answers-names-declares-its-zone-and-a-nodes-hosts-file-is-one-modules.md)). Next, decided: renamed `node-hostname`, held by the module renamed `hostname`, owning `/etc/hostname` as well ([ADR 0223](../../02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md)) | | `mesh-intrusion-prevention` | `the-intrusion-prevention` | node | — | an intrusion-prevention service | | `mesh-packet-filter` | `the-packet-filter` | node | — | the packet filter | | `mesh-private-network` | `the-private-network` | node | — | the private network the mesh runs over | -| `mesh-resolver-configuration` | `the-resolver-configuration` | node | — | the module writing `/etc/resolv.conf`; its holder needs the uplink's held on the same node ([ADR 0220](../../02-DECISIONS/0220-what-a-machine-asks-needs-its-uplink-held-and-the-retired-resolver-pieces-go.md)) | +| `mesh-resolver-configuration` | `the-resolver-configuration` | node | — | the module writing `/etc/resolv.conf`; its holder needs the uplink's held on the same node ([ADR 0220](../../02-DECISIONS/0220-what-a-machine-asks-needs-its-uplink-held-and-the-retired-resolver-pieces-go.md)). Next, decided: retired, the file becoming the uplink holder's ([ADR 0223](../../02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md)) | | `mesh-showcase` | `the-showcase` | node | — | the showcase module | The controller holds **the mesh's own** entries in code, and a test asserts their size and that @@ -280,6 +294,7 @@ checked as their tables say: | A singular fact about machines is a placement of capacity one, refused by name | 0161: the overlay command's test for a second hub; the store's unique index. | | A holder of `node-uplink` is the dialect the machine runs | 0161: the host reports `uplink-` in its profile with every report; a resolution test refuses the other holder naming the capability. | | A seat's holder has the seats it needs beside it: the resolver configuration is refused at `assign` without the uplink held on its node, and the uplink's last holder cannot be taken from beneath it | 0220: seat-dependency tests on the definition, on a synthetic claimant, at assign, at unassign and at composition, and one reading the catalogue for the uplink's possible holders. | +| A replicated seat has every holder on record and composes on each; a seat held once still refuses a second holder, on record or not; `--add` is refused for it | 0223: resolution and composition tests with two holders on record and a third machine, with two claimants and nothing on record, and on `mesh-store`; a unit test that only `mesh-dns-resolver` is replicated, surviving the store's rows; store tests keeping several holders once each, a handover leaving one, and unassigning one taking only its row. | | A retired seat leaves the set once nothing claims it, from the compiled set and the store's table both | 0220: the closed-set test's count, and the store migration that deletes the row. | | Holdings are derived, and the overview lists every seat | 0118: the `seats` command test, including an unheld seat. | | A build source on the seat records no address; an unheld seat refuses only self-hosted builds | 0111's tests. | diff --git a/04-ISSUES/262-an-alpine-container-could-not-find-a-machine-by-its-mesh-name/00-report.md b/04-ISSUES/262-an-alpine-container-could-not-find-a-machine-by-its-mesh-name/00-report.md index d07f5ee..3a9ab06 100644 --- a/04-ISSUES/262-an-alpine-container-could-not-find-a-machine-by-its-mesh-name/00-report.md +++ b/04-ISSUES/262-an-alpine-container-could-not-find-a-machine-by-its-mesh-name/00-report.md @@ -44,3 +44,7 @@ answered only by the wildcard, still answers NXDOMAIN for IPv6, exactly as it di Ask the mesh's resolver for the IPv6 address of a machine's name. The answer must be NOERROR with no records. Today that check is done by hand. A controller test that renders the machine list and requires one host record per machine should be added. + +*2026-10-05:* that test is added with [ADR 0223](../../02-DECISIONS/0223-the-mesh-has-two-resolvers-and-a-machine-lists-only-them.md) +— mesh-controller's composition test renders the resolver's machine list and requires exactly one +host record per machine. The live check stays by hand, on each resolver.