Merge pull request 'Issue 135: a container's mesh names are not compared' (#165) from issue/135-a-containers-mesh-names into main

2026-09-28 15:19:37 +00:00
parent e00862e317 4789624857
commit d6b62387f2
1 changed files with 70 additions and 0 deletions
@@ -0,0 +1,70 @@
 ---
 status: resolved
 opened: 2026-09-28
 located-in: [mesh-host internal/apply]
 fixed-by: mesh-host — a container's mesh names are part of the spec digest the host compares, sorted so the digest does not move for a reordering. A container whose names moved is now recreated like a container whose image moved, and the test fails against the previous behaviour.
 amended-design:
 ---
 # 135 — A container's mesh names are not compared, so a moved address is never noticed
 ## What was observed
 One container on this mesh had been restarting every thirty seconds for five days — 2286 times — and
 the mesh reported the machine as doing what it was told.
 Its logs said its database connected and then a query timed out. The database was reachable: the same
 query from the same network, with the same credential, answered in milliseconds. What differed was the
 name. Inside that container, `novox.internal` resolved to `10.42.0.1`; in every other container on the
 machine it resolved to `10.10.0.1`. The mesh's overlay range had moved, and this container still held
 the old one:
 ```
 umami          created 2026-09-23   novox.internal:10.42.0.1
 mesh-catalog   created today        novox.internal:10.10.0.1
 ```
 A container resolves other machines and public names through the entries the mesh gives it when it is
 created, and nothing re-reads them afterwards. The host compares a container against what was declared
 by a digest of its spec — image, name, environment, ports, volumes, arguments, resolver, address, and
 what it reads — and **the mesh's names were not in it**. So this container matched what was declared,
 was left alone, and kept an address that had not existed for five days.
 Forty-eight other containers had current names. Not because anything corrected them: each had been
 recreated for some other reason — a new image, a changed file — and picked up the current roster on the
 way. This one's image is an upstream release that had not moved, and nothing else about it changed, so
 nothing ever recreated it.
 ## Why it matters beyond this instance
 **It is the exact fault [issue 045](../045-a-container-keeps-the-values-it-started-with/00-report.md)
 named, in the one field that was left out.** That issue is why the digest carries what a container
 reads: "a container whose configuration had since been rewritten compared equal and was left alone —
 running values the machine no longer holds, while every check reported success." The same sentence
 describes this, with *names* in place of *files*.
 **The failure is invisible in exactly the way that matters.** The container runs, so the machine
 reports it applied. It restarts, but a restarting container is a normal sight during an upgrade. The
 only account of the fault is inside the container's own log, in the words of the application rather
 than of the mesh — and what it says is that a query timed out, which points at the database.
 **And it is most likely to bite what changes least.** Every container that is rebuilt often repairs
 itself by accident. The victim is the module whose image is stable — which is to say, the module that
 was working fine.
 ## What was done
 The mesh's names are part of the digest, sorted so the digest does not move for a reordering nobody
 made. A container whose names moved is now recreated exactly as one whose image moved.
 The first apply after this recreates every container that carries mesh names — one restart each,
 already the price the mesh pays for any image update — because their recorded digests predate the
 field.
 ## What is still true
 The mesh gives a container its names at creation and has no way to change them in place. That is the
 container runtime's shape, not a choice; the answer is to recreate, which is what this does. A module
 that would rather re-read a roster from a file can already ask for one as a fact
 ([ADR 0120](../../02-DECISIONS/0120-a-roster-fact-carries-its-format-as-a-template.md)) and restart on
 it.