Files
hq/04-ISSUES/128-the-hosts-file-is-written-whole/00-report.md
T
jschoubben 14be8576f8 Grooming: five issues were fixed and never closed, and one is not
088, 089, 120, 128 and 130 each name a commit that is on main and cites them —
the forge's address following a moved port, a route naming its endpoint, a
provisioner asking the backend what is there, the hosts file written into a
marked block, and undeclaring giving a unit back the state it was found in.
Each says it was closed by reading commits rather than by a run, so nobody
reads a green that was never measured.

129 stays located on purpose: ca-trust is merged and no machine holds it, so
the symptom it opened on is still true everywhere.
2026-09-29 22:25:25 +02:00

4.7 KiB

status, fixed-by, opened, located-in
status fixed-by opened located-in
resolved mesh-host 1cb8953 and fdc768c — the mesh writes into a marked block of a text file instead of over it 2026-09-26
mesh-controller internal/catalogue/facts.go
mesh-host internal/apply

128 — the machine's hosts file is written whole, and on a workstation it is shared

What was observed

The private network asks for the node-names fact, and the mesh delivers it as /etc/hosts. nodeNames composes a complete file — its own header, localhost, the machine's own name, and every name in the mesh — and the host writes it over whatever is there.

On an adopted workstation the file the mesh holds contains, besides the predecessor's block of mesh names:

  • the distribution's own lines (localhost, the machine's .localdomain name);
  • two marked blocks (# BEGIN … # END …) maintained by a local-development tool, pointing a dozen development hostnames at 127.0.0.1 — rewritten by that tool whenever its project list changes;
  • hand-added entries of the operator's.

Today the file is only held (ADR 0100): the private network was assigned, not yet taken, so nothing was lost. Taking it — or converging the node, which takes everything — replaces the file. The development tool's entries disappear, its projects stop resolving, and every later write it makes is overwritten at the next change to the mesh's names (a machine joins, a route is contributed), silently and without a failure anywhere: the development tool thinks it wrote its block, and the mesh thinks it owns the file.

This is ADR 0102's failure exactly — a file the mesh shares with software it did not install, written over — in a file 0102 did not name, because its merge verb is structured (into: json) and a hosts file is not JSON.

A second, smaller finding from the same reading: the fact's contents depend on which machines hold the private network. A machine that is enrolled but not yet assigned the private network is in neither node-names nor node-zones; its name resolves on the others only for as long as a predecessor's hosts block survives. Taking the hosts file before every machine is on the private network loses that name too.

What would have prevented it

  • A marked-region merge in the host's vocabulary: into: "block" (or similar) — the host owns only the lines between its own begin and end markers, keeps everything outside them byte for byte, records what the region held before, and on undeclare removes the region and nothing else. The shape local tools already use for this very file.
  • The node-names fact written as that region — no header of its own, no localhost, no machine name — so the distribution's lines and every other tool's stay where they are.
  • A converge preview that names a held file the take would replace whole, with its line count before and after, so a person sees "hosts: 31 lines → 12" before the flip.

The fix, as built (in review)

  • Host: a file resource may say "into": "block". The host owns only the lines between # BEGIN mesh <id> and # END mesh <id> and keeps everything outside them byte for byte. A new region goes at the end by default, or at the start ("at": "start") for files where a line's meaning depends on what stands above it; a region already present is never moved. Undeclared, what the region held before is put back, or the region is removed and nothing else. Replacing nothing, it is written on an adopted node without being held — so a machine gets the mesh's names before its private network is taken.
  • Controller: node-names is a fact written into a shared file, emitted as that region: the mesh's names only, no header, no localhost, no 127.0.1.1 line.
  • Order: a host older than the block mode refuses the whole declaration on an unknown into, so hosts are upgraded before the controller that emits it.

Evidence to carry into diagnosis

  • internal/catalogue/facts.go, nodeNames: the complete file is built here.
  • The host's file resource supports into: "json" only; anything else is a whole write.
  • node show <node> on the adopted workstation: holds file /etc/hosts mesh-wireguard.fact-node-names, original kept.

Closed

2026-09-29, in a grooming pass rather than by whoever fixed it. A shared hosts file keeps every line that is not the mesh's. Found by reading what the code repositories' commits cite: the fix names this issue and is on main. It was not re-verified on a machine, and this record says so rather than implying a run that did not happen.